‘Evidence-based practices.’
It’s a phrase we all know well.
You can likely find it in the overarching guiding plans or strategy documents of your university, and it often rears its head in government policy announcements, or the agendas of collective bodies. You may have even come across it in your own career success plan.
Yet despite being the current gold standard of how we inform our decision-making, this term is rarely unpacked further. And so, I ask, what exactly are evidence-based practices? Evidence-based to whom?
Measures such as the Quality Indicators for Learning and Teaching (QILT) data, or the Graduate Outcomes Survey (GoS) data, are taken as objective truths. And if we improve or decline in these numbers, it’s assumed that staff have either done better or worse at their jobs.
Similarly, researchers are defined by their metrics (e.g., publications, grant funding), as recently discussed in other posts by Stephen Matchett and Merlin Crossley. No exceptions are made in teaching practices either, where despite well-known flaws in the unit evaluation surveys, numbers continue to play a dominant role at most universities. Teachers can be left befuddled, trying to analyse the difference in a rating of 4.6 to 4.4 in student satisfaction.
This was the central thread of my recent keynote at this year’s Higher Education Research and Development Society of Australasia (HERDSA) conference, where I argued that the higher education sector increasingly takes a narrow view on evidence, often defined through the lens of quantitative data.
There is another, equally fruitful, way to explore value and quality in our sector.
Qualitative data.
This side of the abyss is often discounted in the encroaching managerialism of our sector. Its data written off as messy, complicated, and subjective. Its dismissal an artefact from wars fought long ago, and now supposedly over – the paradigm wars.
Yet far from being over, the paradigm wars are ever-present in our sector. Only now, our epistemic preferences and biases play out not only in the scholarly discourse, but in our meeting rooms, Zoom calls, and MS Teams messages where we debate what evidence will inform the design of an optimal student experience.
The path forward goes beyond a practical, mixed methods approach. It is time for our sector to meaningfully reflect on how we gather evidence, and how these data represent the voices and experiences of those who shared it. This is absolutely vital in the context of the myriad of wicked issues that universities are facing, from learner engagement to generative AI, to academic integrity, where the nuanced solutions we seek won’t be found on a Likert-scale question.
Significant progress towards this endeavour is already well underway, with universities such as Deakin University, University of Queensland, Western Sydney University, and many more, hosting university-wide students-as-partners programs. These programs emphasise what scholars and practitioners have long-known as the key to student success – relationships.
But the progress must also continue. Alongside the creation of such programs, there remains a need for universities to consider how to enact a cultural change around the types of evidence we value and how diverse knowledges in our ecosystems can be equally respected.
This is not about being everything at once, but rather displaying greater modesty in what we know, and what we don’t, because our evidence-based practices have real consequences.
Dr Mollie Dollinger, Senior Lecturer of Learning Futures, Deakin University.
