The Research Excellence Framework reviews and benchmarks the quality of research within UK higher education

The most recent set of results, REF2021, has just been released. They happen every 7 years and a big part of their importance is that they feed into the allocation of research funding. The research outputs (that is, the discoveries and insights that come from the research in whatever form) are split into various categories, so we’ve been looking at the proportion of web content in REF submissions.

It’s great to see in the category splits that 44% of outputs of the type ‘digital artefacts’, which includes web content, achieved the highest 4* quality level (world-leading), and a further 30% classed as 3* (internationally excellent).

But it’s quite surprising to find in this digital age that the digital category is one of the least popular – 152,442 journal articles and 28,726 books (or parts of) were submitted, as opposed to only 542 in the digital artefacts category.

And whereas it’s not always precise to compare to previous REFs because of changes in criteria, 2021 has even less digital artefacts submitted than in 2014 (542 as opposed to 761).

So why are there so few digital submissions to REF?

One explanation for this is the REF submission criteria. It’s important that every piece of research gets treated equally and fairly, so if a website gets the chance to be updated after publication whereas a research paper doesn’t, it’s got an unfair advantage.

So some website material would be put in the ‘physical output’ category by copying it onto a USB stick or DVD, for example – therefore some of the 400 physical artefacts in 2021 or 757 physical artefacts in 2014 may in fact be copies of websites.

The other way of submitting a website to REF is to create a completely separate copy of the site exactly as it was on the day it was published, and record the content on that date to prove that it hasn’t been updated since.

It’s a shame that the advantages of a website – that it can be updated easily, and that the fresh and timely content can drive impact and engagement – are not being maximised when it comes to submitting them to REF. It will be interesting to see if ways to encourage digital submissions can be developed whilst keeping a level playing field.

It seems like digital material is being underused as a means of submitting to REF, and it could be a way of making your research stand out from the crowd.

How to use digital content to drive research impact

Another explanation for the lack of digital content submissions is a concern that their impact might be a lot more subjective than that of a paper published in a high-profile journal.

However, when it comes to websites, it’s possible to set up analytics that detail the website traffic and provide proof of the level of engagement. These can be drilled down to specific pages or even particular actions on the website (button clicks, link clicks, video plays and watch time, file downloads). ‘Heatmaps’ can also be set up which visually track users’ interaction with a site, showing areas which get the most interaction – useful if you want to demonstrate the impact of an infographic.

In one paper, Project SCENe identified one of the benefits of their website was that one participant used it to talk to friends about the project. A website can also provide the possibility for interactions that are difficult or impossible with a research paper. Quizzes, animations and chatbots can bring your research to life, particularly with a non-technical audience. Once the tools available are properly set up to record and quantify user interactions, they need a lot less work than might be the case at a face-to-face event, particularly at scale.

Remember, the digital artefacts category doesn’t just cover websites: animation, audio-visual presentations, and databases are all possibilities. If you want to present complex data and concepts to a general audience, they need to be presented in an engaging and inclusive way.

We specialise in websites for academic research projects, so get in touch if you want to explore how to tap into the potential of digital impact.