Research Excellence Framework 2028 consultation response

The proposal to determine volume through Unit of Assessment (UoA) values for staff in the Dept/Faculty/School HESA staff return will require a consistent mapping between the two, and that this does not change over time. Presumably institutions would still need to select UoA for each individual submitted to HESA? There is the potential for disproportionate impact to disadvantage smaller units through, for example, a large group from a small unit being bought out outside of the census period. There appears to be an inconsistency with the recommendations allowing non-academics/PhD students to be included. Would this be HESA as a minimum plus others?

Clear definition is essential of how the assignment of expected UoA to groups/depts/individuals within HESA will work in practice. We recommend consideration to allow a mechanism to revise the volume calculation determined in the census period if it is not an accurate picture over the full REF period (due to significant departure or arrival of groups, for example).

None expected, though consistency between UoA and HESA is important, many staff may not know which they are returned in except when it comes to REF.

Whilst the inclusion of all staff in the submission is a positive development, we have concerns over removing the minimum requirement for 1 output for research staff. There can be adverse career consequences for staff who have no outputs returned to the REF, it risks papers being concentrated from a small number of high-performing individuals, possibly at the expense of those who for good reason may not be as productive in the period (e.g., transitioning to, or at the beginning of, their independent research careers, with caring responsibilities and such like). A good compromise would be to keep the minimum requirement for staff with significant responsibilities for research and not expecting the same for other categories of staff.

The intention of removing the link to individuals is good and could be considered helpful in encouraging ‘practitioners’ as well as researchers on the staff. A genuinely blind assessment process would be difficult to implement but at least the link between the contributing author and the output should not be displayed for the assessors to see. To drive submissions that are representative of the staffing base, we recommend consideration be given to a submission from each UoA, that is not disclosed to panels, describing outputs and attribution across the staff cohort (e.g., a table of staff members and number of papers with their name on).

There is already very rigorous institutional checking and selection of outputs and it takes considerable time and effort in choosing between similarly excellent papers in sometimes widely different fields. This will be exacerbated by the removal of a fixed number per individual, checking of staff eligibility (contracts, dates), and the need to monitor leavers’ outputs – there will be considerably more work required in selecting papers under the proposed changes.

Inclusion of disciplinary-level evidence statements adds additional burden to institutions. There is potential for this to have a negative impact on submissions with a small staff cohort.

A 0.2 FTE for six months contract is a very low bar and does not support the development of a progressive research environment/culture over a sustained period, with clear potential for “buy in” game playing leading to submissions being non-representative of the submitting unit/institution. We recognise it goes against the intention of breaking the link to individuals but we recommend consideration be given to implementing a limit to the number of outputs submitted for obviously ‘seconded’ staff members – perhaps proportional to time/FTE for individuals below (say) 0.4 FTE and 1year?

This introduces a whole new level of game playing. It completely negates the idea of using HESA data to include all ‘research staff’. Staff employed on teaching-only contracts could be encouraged/pressured to do research despite heavy teaching loads.

It will likely impact or see an impact from those institutes who have created new job families that are not Teaching & Research or Research only. Those staff in the Research Engineer job family that exists in some institutes are excluded from outputs but do contribute to impact in a major manner. Currently there is an advantage to having such a job family focused on impact. However, including them will change the nature of their roles to include publications, reducing their time focussing on impact, whilst simultaneously increasing the number of impact case studies required. On the Teaching side, where much of the changes might be seen, we may see more pedagogic research evident in engineering – virtually none in 2021 (if any). Some staff do change job family as they near retirement, we might see inclusion of more of these types of staff, and also very early career graduate teaching assistants, who are not yet at lecturer level, but developing as academics.

We do not support the inclusion of sole-authored outputs, including PhD theses, by postgraduate research students. PhDs are already included in REF separately and carry significant weight. PhD theses are in most cases not peer-reviewed. Institutional and REF panel output assessors cannot be expected to review theses, which are often 150+ pages. In engineering, most PhD projects are collaborations and PhD theses arising from the fostering of good future researchers generally result in peer-reviewed papers which include supervisors as co-authors. These peer-reviewed papers can already be included as outputs. The additional complication of submitting both a thesis and research papers based on the same work risks double counting.

In reality we do not think this will be a significant issue. We would be surprised if a paper were able to score highly in more than two, perhaps three disciplines. Some consideration should perhaps be given to limiting outputs to one discipline per institution only, to avoid double counting at institutional level. We are strongly supportive of maintaining the 100 words statement on impact of each paper after publication – this has worked well previously.

We would like to see a requirement for outputs to be derived from substantive research undertaken in the UK. Too many papers in REF2021 had all the work done overseas and little or no apparent work by the UK author, or the ‘UK author’ was originally at that overseas institute and moved to UK. We should be measuring the vitality of the UK research environment.

This might see different decisions at institutes on the number of UoAs they submit to and increase the number to better reflect the granularity of their staff base – so small units do not get absorbed into larger submissions.

The proportional reduction in impact case study requirements for larger volume submissions has potential for further exaggeration of existing advantage for larger research-intensive institutions. There is a case to look at this very carefully, given a lot of ‘place based’ investments that have changed the playing field around impact type resources (and also environment), e.g. city deals channelling funding through local institutes, devolved authorities placing strategic investment (Scotland, Wales and N Ireland have very different opportunities to England) and we have catapults or parts of catapults that vary in terms of their ownership/legal status. A more levelling up approach that takes account of imbalances in resource better would be welcome.

We do not understand why removal of the requirement of 2* or better underpinning research leading to impact is being proposed. Eligible impact arising from 2* or better research has worked well in the previous two REFs and seemed well justified.

We are very concerned about this as it risks setting up rankings of submissions before any assessment occurs – artificially giving more value to some impacts. It also introduces more thresholds for universities to have to try to be the optimum side of. We are not convinced a change is needed, it worked quite well in engineering during REF2021. The ICS should stand on its own and have equal merit/value irrespective of size of submission.

More broadly, there is some concern regarding the weightings change and the increase on environment, which was the most challenging part of REF to assess. Making it very prescriptive and questionnaire-like is not helpful and the validity of a staff survey approach is questionable. There are resource challenges affecting many institutes who do not enjoy the legacy of resource over many decades. Additional weighting to impact would demonstrate more clearly value for tax payer’s money, and more meaningfully tell the story of the research and its purpose.

We are seeing much larger societal challenges being addressed by funding bodies and some of the major journal channels. So, attention to this growing multidisciplinary research is worthwhile. For example, we will see, and have already seen, changes to various institutes to create cross-academy entities that comprise staff from many different HESA/REF UoAs – how do we best organise the UoA structure to assess the significance of such cross-UoA outputs, environment and impact of these on a level playing field, i.e. where different approaches have been taken by different institutes?

The impacts are quite diverse across the academic disciplines and, even within engineering, this will be a challenge to get right.

We are not sure there are any warranted so long after pandemic, much of this period will have fallen into a new REF period. Brexit is a more important impact as it has affected volume of research and talent pool coming to the UK.