How research funding is allocated needs further research – commitments to “excellence” can mean whatever grant agencies want.
Adrian Barnett from QUT is exploring multiple aspects of the issues involved, notably in the survey he is conducting with Paul Glasziou (Bond U) to learn what researchers want to know about the way research funding systems work (Future Campus April 12) and in a new paper he co-authors on the costs of different funding systems.
The authors suggest seven issues to investigate.
- Data on who gets how much and for what: including analysis of success rates and funding by gender. They also propose an intriguing variable, whether technical terms in applications work better than promises of “innovative”, “ground-breaking”, or novelty
- Reliability and predictive validity of funding decisions: bibliometrics can rate peer reviewer and funding agency judgements but not impact on, for example, “societal challenges.” Question is what measures to use, “we know far more about the limitations, caveats, and gaming potential of traditional bibliometrics than those of alternative indicators, which may be as gameable or even more gameable”
- Democratisation of funding evaluations with customised bibliometrics: “rather than relying on a small and often opaque group with unclear selection criteria, researchers would be assessed by the broader scientific community through their citation counts”
- Alternative funding models: thematic grants, funding to address a specific issue, lotteries for pre-approved applicants. While the authors do not know what would work best, “a systematic review suggests that greater funding dispersal is likely to be beneficial”
- Consideration of the economic cost of submitting grants: by discipline and for basic and applied research
- Evaluation of epistemological costs: at organisation, and national level. “Are countries with a higher share of competitive funding less good at simultaneously supporting risky research, normal science, variety, and stimulating the promising parts of this variety?”
- Consideration of social and ethical costs: “go beyond” surveys to create a “more supportive and ethical research environment”
“It is crucial to be specific about the various aims of science funding and the criteria that follow, and it is as crucial to be specific on how to measure these criteria so as to avoid noise and bias,” they write
