The headline is for research nerds – reflecting on the fact that, in the absence of any national research measurement scheme since ERA was last completed in 2018, citations in paywalled journals such as Nature remain a popular, but deeply flawed proxy for research impact.
We apologise, because while hilarious, the title is also deeply misleading, as we wished to draw your attention instead to bibliometrics in the discipline of law and in particular work by QUT’s Kieran Tranter and the University of the Sunshine Coast’s Timothy D. Peters, who have produced a paper analysing benchmarks for assessing the citations of Australian law researchers.
Why read on? Perhaps because the lack of any national strategy in managing and strengthening research performance for well over a decade has spawned a range of initiatives such as this one, providing an opportunity to see what Dr Google can tell us in relation to which Australian legal academics are cited most frequently (which must not be confused with a metric of those who do the most work/ highest quality work / have the biggest impact / teach best / earn the most research grants / make the biggest contribution to improving equity in the academy or any other similar metric).
The article notes that increasing accountability is an essential part of the contemporary university.
“The progressive neoliberalisation of the university from the 1980s onwards has been accompanied by an increasing emphasis on the need for accountability assessments of the quality and impact of research produced at institutional and national levels,” The authors state.
“Universities internalise these drivers by seeking to performance manage academics towards the production of ‘high quality’ and ‘impactful’ research, so as to his improve university performance in government research excellence assessments.”
“To be clear, in undertaking this research, we are not endorsing H-indexes, Google Scholar, or bibliometrics as wonderful additions to the rich fulfilment of life as a scholar.
“In fact, our tendencies are to the opposite, upon which we reflect and theorise in the final section. However, the fact that Google Scholar data and H-indexes are being used daily within the Australian legal academy, often by junior members trying to build narratives about their research, compelled us to undertake this study.
“We regularly engage with university managers and colleagues from other disciplines who seemingly treat H-index scores as authoritative indications of a researcher’s impact and excellence.”
The authors looked at the H-index and total citation counts of Australian-based law researchers via Google Scholar.
They found a number of interesting issues in the process of conducting their study:
- Law academics were not always easy to identify via their institutional websites;
- 37% of the 1,441 Australian law researchers reviewed through the study did not have a Google Scholar profile – a critical step in building visibility for young researchers
- H-indexes for Professors were much higher (mean of 15.99) than for Associate Professors (mean of 9.4)
- There is little known about the algorithm that underpins Google Scholar, which is influential, because many researchers don’t look beyond the first page of results
The thoughtful paper concludes with concerns about the use of data to measure performance, and how this may be in conflict with the goals of the academy. “Engaging with metrics and the quantification of research involves a complicity with ways of thinking and valuing academic work that often run counter to the nobler ideas of the academy, the university, and the office of the scholar,” the authors state.
