One of the first things students are told is to aim for quality not quantity.
But no one tells them what the word quality means.
I would argue that no single person can ever define what quality is.
Quality is an emergent property that comes from societal assessments, and it is guided by people with a keen interest who have developed specific expertise in each field.
Quality is, by definition, different to quantity. Everyone understands quantities. When discussing things we can communicate about quantities, but we can not always agree on quality.
But sometimes we need to agree, because much of what humanity does depends on reaching agreement to facilitate collective action, an almost uniquely human strength.
Interestingly, we get agreement by using complex mechanisms to convert quality into quantity.
Thus restaurants are measured by hats, films by stars, universities by league tables.
Ouch.
People hate these things. Because all metrics are flawed.
But remember many metrics are useful.
Many people hate ATARs, Impact Factors, H indices, and Student Experience Survey scores. All of these measures are imperfect, but all of these measures can also be useful if interpreted carefully.
The fear is that they won’t be used carefully.
The fear is that they will be tyrannically prejudicial. That they will be interpreted by computer. It is as if a league table that puts French Cuisine on the top, might influence the rating of every restaurant and prevent a Thai outlet from ever gaining recognition for its work.
People fear that crude metrics will pre-determine and prejudice outcomes and thus argue that these metrics are dangerous and should not be used.
Here's the thing. If metrics are not used then the status quo and existing power structures and hierarchies of prestige, status, and power prevail.
Translations of quality into quantity, although imperfect, can provide a lifeboat for new comers.
The example that has been with me all my life relates to my parents’ stories. Both did well on the 11 plus exam, a test that students took at the age of 11 in England. Consequently, they got the opportunity to go to good high schools and to university.
The 11 plus exam was subsequently discredited and discontinued because it was shown to be an imperfect measure.
In Australia some metrics are banned. The NHMRC bans the use of Impact Factors. This is very odd. Every scientist I know is keenly aware of and fashions their publishing strategy with impact factors in mind, but when applying for NHMRC grants one has to pretend they don’t exist.
In Australia the Excellence in Research Australia (ERA) process has been discontinued. That process was devised during a golden age at the Australian Research Council when there was excellent ministerial and executive leadership and management. The rendering of research quality into quantities allowed Australian universities to celebrate their unique strengths. And to invest in those strengths.
With the ERA gone, the crude one-dimensional university league table metric has more prominence that ever, and it seems every university has a ChatGPT-penned strategy, that bluntly could be rendered as – let’s be as similar to Harvard as our funding allows. It would be preferable if each university invested in its strengths.
Australia has a problem with its higher education system – oops I said system. One of my mentors once explained to me it is not a system, it is a sector. But we have a chance now, with stable government and the new Australian Tertiary Education Commission, to create a system.
I believe our sector has one root cause problem – the sophistication costs associated with research. Our microscopes are bigger and their costs increase faster than inflation. To maintain the health of our nation and national security we need world class microscopes. At present we pay for them with international student fees – but that is not working.
Re-instating the Excellent Research in Australia system using a back office metric driven but transparent system will help us agree on where to put the microscopes (and other research funds). This type of thinking underpinned part of the Universities Accord paper but how many of the research recommendations were implemented – none – or rather none yet.
An unprecedented opportunity awaits Australia. We have a chance to build a higher education system and to use data to invest in strength wherever it exists, rather than continuing with a system where some institutions are thriving and others struggling as they try in vain to do everything in research.
In a world of turmoil Australia has more talent and experience in government and in the cross benches than I have ever seen. It is encouraging to hear what the PM had to say at his press club address This is a time when government has to step up, to invest in education and skills and research and innovation.
I for one would argue that metrics can be used to gain agreement about where investments should go. But we need an independent process because who knows which government will eventually follow this one – not every country has stability just now.
Professor Merlin Crossley is Deputy Vice-Chancellor (Academic Quality) at UNSW.
