Friday 30 January 2015

On REF 2014: Why nobody wins unless everybody wins.

As a slightly obsessive[1] Bruce Springsteen fan, I’m familiar with the phrase “nobody wins unless everybody wins”. It was used by Mr. Springsteen at the conclusion of many of his concerts in the 70’s and 80’s, and from time to time he still uses it as a valediction (there is almost certainly a web site that cross-references all instances of his use of the phrase in concert with performances of ‘Prove it all night’ with the 78 intro the phrase, and if not there should be). What it means is that we really are all in this together, and that winning (financially, professionally) is only truly meaningful if it also brings about good for others: if a company makes lots of money, it should pay its workers better, rather than giving the CEO a bonus, sort of thing. For a good analysis of its use in a business context, see Bill Taylor’s article in the Harvard Business Review on Dec 7th 2010.

So what has this to do with the Research Excellence Framework (REF)? First of all, I don’t believe that research is about ‘winning’. Setting aside the comparative impossibility of that concept - the idea that researching the origins of cancer can be compared to researching the end of the Roman Empire - it’s incredibly difficult to adequately compare research within the same area of study, let alone the same discipline. Yet this is what the REF has set out to do. Much has been written about the sheer scale of the task of the REF research panels (and I unconditionally applaud these panels, who have worked tirelessly to peer-review submitted work in their panels, for a small stipend that works out at about 2p an hour). It must be extremely difficult to assess 4938 outputs (the total for the sub-panel which my work went into this time) and grade them on a scale of 1-4. It’s harder when also being asked to assess impact and environment: without adequate background information, how can this be really scored?

There were problems with the process: reports of manipulation of the rules, including Premiership-level moves of key researchers in the final REF transfer window of September 2014, and even more controversially the exclusion of researchers from the whole process.  The criteria meant that smaller units or departments were vulnerable without developing strategic alignments (for an excellent overview of the whole REF process, see the LSE Impact blog piece by Tony Murphy and Daniel Sage)

These issues made the REF a complicated process, but are essentially a distraction from the main problem, which is that the UK’s Higher Education sector is a functioning and effective ecosystem of a huge range of skills and strengths, and that a ranking system of assessment like the REF fails to acknowledge the fact that research is part of a wider educational system with mutual dependencies. Researchers also teach, supervise students, engage with the public, and create new knowledge in partnerships with with other public or private sector organisations: a successful ‘unit of assessment’ does all these things in a balance that works for their host organization and the communities they serve, whether undergraduates, postgraduates, colleagues in their own discipline, or researchers and students in other disciplines, the creative industries, the heritage sector... To single out research publications as the primary means of evaluating worth is to fail to understand the nature of scholarship and higher education generally. Some departments (or centres) are simply more effective at teaching, or interdisciplinary research, or experimental research that is hard to assess. Other departments benefit from this, whether in the form of receiving students that have received an excellent undergraduate education as postgraduates, or integrating outputs of research into new work, or in in benefiting from policy developed as a result of service on committees and related activities. Similarly, many academics liaise with museums, libraries and archives in two-way collaborations that create new knowledge around cultural heritage: these outputs, and their tangible and intangible benefits are hard to define, let alone evaluate. And these dependencies are more necessary now that we all have to do more with less: in higher education, we really are all in it together.

I should have this said at the start that of course, I offer warmest congratulations to those who did well in the REF. To have one’s research recognized in this way is gratifying, and I hope that the departments and universities who achieved success in the REF will be rewarded for their achievements. But that there is a finite pot of resources, and rewarding the highest rated outputs will be to the detriment the many.

There is already concern that the strong showing of the sciences will be at the expense of the humanities. Again, the ecosystem is important: the study of the humanities is strong in the UK, and expertise in the humanities brings benefits to business, medicine, and other disciplines. Science and medicine also benefit from collaborative work with humanists. Rewarding the sciences at the expense of the humanities will disrupt this balance.

Another concern is the widening of the gap in research outputs between South East and the rest of England. The balance of funding going to Those Who Have Done Well will exacerbate this divide. I’m not going to have a political rant, but this does reflect a general condition of the post-credit crunch UK economy.

The UK has a fantastic Higher Education sector, which routinely punches above its weight internationally, despite a lack of investment compared to its global competitors, especially the US Ivy League.  The rankings that really matter are where we compete internationally: seeing UK Universities in the world top 10 is far more meaningful than saying that History at University X ranks higher than history at University Y. 

Concerns are now being voiced that REF ‘winners’ will be rewarded at the expense of the losers. To me, this reflects a wider malaise where all the gains in society seem to be going to a smaller segment of the population, with the rest of us struggling to hold ground. We need to defend Higher Education as a sector, and to help it flourish by recognising that . Let’s hope that faith will be rewarded.





[1] Fanatical and deranged.

Tuesday 20 January 2015

Thoughts on public libraries

Just before Christmas, my Twitter stream was dominated by two themes.  The first was the 2014 Research Excellence Framework (REF), which had finally concluded a process of evaluating the research outputs of the UK’s Higher Education institutions, and announced its rankings of the winners and losers in this curious 6-yearly academic beauty contest. The euphoria, disappointment and schadenfreude were palpable through the ether. Fortunately, the passive aggressive swiping reached its nadir just at the point where everyone decided it was time to shut up shop for the holidays. It’s almost as though they planned it that way.

The second was the announcement of cuts to local libraries, as the tail end of the credit crunch began to really kick public services in England and Wales. Cardiff announced a series of significant cuts to Library services, including budget cuts for the wonderful Central Library, and the loss of seven branch libraries; Birmingham announced a consultation on cuts to public services that included worrying developments for the City’s libraries, including the magnificent Library of Birmingham (which the City recently invested £188 million in developing ). For a wonderful eulogy to Birmingham’s Libraries, see this passionate lament by Robert McNichol.

The stories are, of course, linked. How many REF superstars got their start in their local library? How many people who went on the research the arts, sciences, or medicine passed their O-level exams because they were able to work in their local library after school or on weekends? I’m no academic star, but the local library was an absolutely central part of my education. I was the first in my family to go to University, and the library was where I went when I needed to do to work on homework or projects. When I was at primary school, I was enormously fortunate to have a decent public library near our house in Essex – I could even go there myself on the bus. I also used the magnificent Mitchell Library on holidays in Glasgow, and the Hamburg central library for High School research. This experience enabled me to develop what we now call information literacy: the understanding that no matter how daunting the task at hand, there was a starting point. This could be in the form of a book with references to other books; a cataloguing system that enabled you to find them; or when all else failed (ok, let’s be honest, quite often the starting point), asking a librarian for help. There was also peace and quiet, a space to think and work, and the idea that there was a physical and intellectual place for work, and study, and thought. There was also the idea that working, reading, and studying was a good thing to do: Taking the bus to the Hamburg Central Library and spending a rainy Saturday researching Acid Rain (yes, I am that old) for a school project, was, well, kind of cool. And so I discovered what Betty Smith called 'the magic of learning things.' 

Those of us who owe a lot to public libraries need to defend them. The best way to do this is to use them – take your kids to the local library. Borrow books rather than buying them. And while you are at it, meet your neighbors and find out what’s going on locally. We need to defend libraries, but the best way to do this is to develop a better understanding of how they serve users: they provide services that you may not even know about, like electronic resources (subscription-only genealogy services, and electronic resources, are available in public libraries in Wales, for example) and journals. They are still training people in information literacy, but now they are also many people’s best opportunity for developing digital literacy and navigating online sources. This matters when information is increasingly digital by default. 

But we need also to develop better evidence about the deeper impact of public libraries. Again, think of all the world-class research internationally carried out by scholars who got started in their local library. Multiply this over time. One of my PhD students, Calista Williams (@Ca7ista), is working on an AHRC-funded collaborative doctoral award with the Open University and the National Library of Wales. Her project includes analysis of the early readers of the National Library of Wales, and their subsequent publications and research. Historical Network Analysis of these library patrons, their publications, and their connections and influence, is showing some interesting patterns. Adopting a similar approach to assess the global impact of the research by those who started out as readers of public libraries would start to create an insight into incredible value of these institutions, and their services. This would also make a fascinating research project, and provide concrete evidence of the risk we will face as a society if we lose our public libraries.