Why the party is over for Britain’s Research Excellence Framework

Many UK universities pop the champagne when they get the results of a national research-performance review. But burnt-out academics see no cause for celebration, say Richard Watermeyer and Gemma Derrick.

The past few weeks have generated a flurry of excitement for universities in the United Kingdom with the release of the latest assessment by the Research Excellence Framework, the country’s performance-based research-funding system.

Many institutions have been busy popping champagne corks. But behind innumerable public statements celebrating the assessment results lurk dark inequities and widespread discontent.

The irony is that universities are racing to their megaphones to crow about success in the assessment while imposing precarity, pension cuts and poor working conditions on the very people who built that success.

For international colleagues who might not be familiar with it, the Research Excellence Framework (REF) is a mechanism by which the UK government distributes around £1.6 billion (US$1.9 billion) in annual research funding to the country’s universities. Assessments occur roughly every seven years. The most recent, REF 2021, took place last year — but its results were reported only in May, because of delays caused by the COVID-19 pandemic.

The REF uses peer-review panels to evaluate academic research across 34 units of assessment (UOAs) on the basis of three criteria: outputs (such as publications and books); impact (how research has influenced society, beyond academia); and environment (the extent to which a UOA provides a supportive environment for research). But assessments are arduous and costly. The 2014 exercise, for example, came with a price tag of almost £250 million. They are also commonly associated with negative impacts, particularly relating to the well-being of academic staff.

These effects might have been unintended, but they are now a widely acknowledged truth — and, for the institutions themselves, an inconvenient one. For UK universities, the REF has become an obsession, not only for securing research funding but also as a means of gaining bragging rights about research superiority. And for academics, assessment rankings are a feature of their self-presentation as ‘REFable’ researchers who produce the highest (4-star) grade of outputs.

The outcomes of REF 2021 show some slips and gains, with limited overall change from previous assessments. The 24 research-intensive universities that make up the Russell Group remain the reputed research leaders jostling for a podium finish. But despite this confirmation of the status quo, a casual scan of institutional websites reveals a cornucopia of triumphant declarations stating how universities have surpassed themselves. Whereas Russell Group universities talk up their status as epicentres of ‘world-leading’ or ‘internationally recognized’ research (REF terms that are used as evaluative criteria), smaller research institutions advertise pockets of excellence. Universities that have performed less well still find ways to exaggerate their research achievements, overlooking evidence that dents their claims.

But, for most academics, the announcement of these results might be something or nothing. Their personal involvement — if it can even be called that — occurs when institutions start preparing their REF submissions, during which the quality and competitiveness of individual research contributions are internally assessed against each other.

An academic’s ‘REFability’ (their capacity to provide a return on institutional REF investment) is thus decided long before publication of the REF results. But those reports are decoupled from individuals — an irony of sorts, given that individual research performance is key to the preparation of REF submissions.

Behind closed doors

Preparations for the REF in many institutions are almost clandestine. Involvement is highly confidential and limited to small, dedicated teams of senior academics and administrators, closed off from their wider academic communities. Because many scholars are shut out of the process and have little of understanding of it, their REF literacy is poor.

That means celebrations over REF 2021 results might be relevant only to dedicated REF personnel and to academics called in to act as quasi-managers and discipline leads for their departments. They will be breathing a sigh of relief, at least where their efforts have yielded a favourable return. Most staff, however, are indifferent to what is widely seen as a top-down, managerialist exercise. Some regard the REF as a power game used by university administrators to manipulate academics so that they overlook the precarity of research careers. Others see it as infantilizing — telling researchers what counts as research — and express embarrassment at the hullabaloo surrounding a UK-centric, parochial exercise. Claims of greatness determined through a national assessment seem feeble in the global context.

Crisis point

REF celebrations also overlook the fact that these results come at a time of attempted recovery from the COVID-19 pandemic. Its impacts are hitting hard, exacerbating the concerns of a deeply unhappy UK academic workforce that has repeatedly staged industrial action over pension cuts. Reasons for subsequent strikes include cuts to pay, increasing casualization of the workforce, staff burnout, a mental-health crisis in academia, rising inequality and major staff attrition — including large numbers of academics moving to universities in other countries.

Despite institutional claims to the contrary, the UK higher-education system is not as ‘world leading’ as it purports to be — and the REF is no remedy. A forthcoming evaluation of the REF 2021 exercise by the four UK higher-education funding bodies (called the Future of Research Assessment Programme) promises to review how such audits “can form the foundation for a healthy, inclusive and dynamic research system”.

But instead of tinkering, what is required is a wholesale repurposing of the REF. The needs of UK researchers must be placed front and centre. The REF should not exist at the margins of research lives. Instead, there should be greater emphasis from the outset on active, equal participation of researchers, bringing research practice and management closer together. This kind of bottom-up, less-autocratic approach would improve REF literacy and help to ensure that principles of equity, diversity and inclusion are given more than lip service. Teams responsible for REF preparation should be as diverse as the submissions they oversee; gender, ethnicity or career stage should not inhibit or prevent researchers’ participation.

Until then, the megaphone must be packed away and the party saved for another day.

This news feature first appeared here, on Nature.com.