Skip to content

Reflecting on the Research Excellence Framework 2014

January 24, 2015

The results of the 2014 REF are out, and by and large, the results were positive for the sector as a whole and our unit of assessment (UoA 26) in particular.  I am no fan of the REF, but it is something we are stuck with in UK universities, in the same way that we are largely stuck with insanely difficult-to-remember passwords that seem to need changing every five minutes.  As a process, it is still a long way from perfect: we are still not told which papers receive which ratings: is that too much to ask HEFCE? We had to do that ourselves about 4 times in the run-up to submission. Couldn’t you just, y’know, not systematically destroy the specific output feedback next time? Especially when each panel member (usually professorial-level researchers) had to review and rate >500 outputs. Each. But as usual I digress. The opacity of the exercise is not my point this time.

The sports-related studies UoA returned results that were considerably improved compared to last time, even considering the general grade inflation seen.  It is no exaggeration to say the results in the equivalent panel were pretty poor last time, in part for political reasons (or so I’m told). One reason for the improvement is that the study of sport, leisure and tourism is maturing – as are those working in the area!  Another, I think, is that our discipline is extremely good and demonstrating impact. London 2012 helped that, but we are also making significant in-roads into areas of disease management and public health, previously the preserve of clinical medicine, for example.

Having spoken to various people, including heads of schools and panel members, the general perception is that the REF was broadly fair in the sense that whatever strategy you adopted you were rewarded for it. My institution favoured staff inclusivity rather than maximising 4* submissions, and we have done well on that score. Post-1992 universities were more selective, justifiably so because they emphasise non-research work and income rather more than the Russell Group. Many of these did well on “Grade Point Average” but less well on “Research Intensity”. (As an aside, I think all of these metrics are bullshit, as are the league tables they feed into, but again we are stuck with them for the short term at least. I reserve the right to swear about them when they do pop up though. The twats.)

One idea that is currently doing the rounds in our institution has actually surprised and delighted me. It is that if you want to maximise 4* output – World-leading research – perhaps most people should slow their research down. That is, rather than try and publish multiple papers every year, make sure you spend at least a year on each paper. Do all the additional controls that usually make up your limitations section. Do that analysis that you could have done but didn’t. In short, chew glass, repeatedly, before submitting the paper. This idea delighted me because it is the way science used to be done before research assessment. It also delighted me because it’s the way I work, and I’ve always felt a little guilty for not publishing more papers than I do. Perhaps I shouldn’t feel that anymore.

If the REF has helped senior people see the benefits of the slow and careful method (which is also “the scientific method”) then that’s a good thing. It is the first sign that the tide might turn on the “Publish or Perish” culture that has developed over the last 20 years. It hasn’t turned yet, of course, as the tragic case of Stefan Grimm demonstrates.

There are still plenty of problems with the REF that we should not be afraid of voicing. In addition to feedback, the disproportionate effort that goes into internal auditing and drafting compared to the actual cash we get from it – which is small in comparison to tuition fees – needs to be addressed.  If we have to continue with a peer-review process (no quantitative metric really seems to do the job without encouraging game-playing), then I’d like to see a single paper per submitted staff member. That would encourage excellence, as well as inclusivity in teaching-intensive institutions. It would also dramatically reduce the reviewing panel’s workload. But I’m probably dreaming.

The conclusion to this brain dump is that the REF is still a nightmare of epic proportions, but it might yield some useful practical outcomes. There is a debate to be had about whether such outcomes are worth it, and whether they will be encouraged in reality. Time will tell.

No comments yet

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: