Regulating student outcomes: implications for engineering

Introduction

February 2022

The ‘Regulating student outcomes’ consultation outlines a new approach to setting ‘minimum requirements’ for positive HE outcomes in England. The continuing narrative is the threat of ‘low-performing providers’ to the public purse and public confidence. The consultation is part of the wider regulatory risk-based approach we’ve seen which is couched as reducing regulatory burden for some (low-risk) providers.
The proposals include ‘non-determinative’ minimum outcomes thresholds. That means that falling short of these will trigger other processes prior to a limiting judgement (ie. a penalty). A host of metric estimates will identify potential underlying performance issues which, depending on OfS’s focus, would lead to a closer look at the providers’ context submission and compliance history. This could lead to an improvement notice.
While minimum requirements are proposed for a dizzying number of indicators, no subject level thresholds are identified; although it is proposed to unpick provider performance at subject level (‘engineering’ not course, or engineering discipline) to ‘enable us to identify pockets of provision where a provider is delivering outcomes below our numerical thresholds even though its overall performance is above our numerical thresholds.’
OfS is looking to introduce the changes as soon as July 2022, the publication of all indicators in September and the identification of providers for assessment by October 2022.

Summary of proposals

The EPC has, summarised the proposals. Read our summary here.

Further information

You can read the full OfS consultation and watch introductory video presentations here. You can watch the recorded OfS consultation / Q&A event here.

Complete our survey

The EPC is conducting a full member consultation. Please submit your views here. Note, there is a second EPC member consultation in relation to the TEF (here).

 

Access and participation

OfS’s proposals are based on the assumption that ‘a provider has a considerable influence over the outcomes students achieve, and that factors beyond its control are not so extensive as to make it impossible to establish a minimum expected level of performance.’ They also hold that ‘meaningful’ improvements in access and participation must be built on a minimum level of quality and standards including ‘taking steps to meet the needs of students from underrepresented groups where those needs are different from other students’ needs.’

There is a vast literature on the structural barriers experienced by disadvantaged learners throughout their education. By explicitly rendering universities accountable for addressing the failings of education pre-higher education, OfS will encourage HE institutions to avoid selecting students with lower prior attainment or any other circumstances that might hamper their employment outcomes (such as socioeconomic disadvantage, ethnicity, gender, disability, etc.). The effect will obviously be the opposite, undermining OfS’s legal duty to promote fair access.

Ultimately, the approach as it stands is likely to encourage universities to be risk-averse at admissions, effectively raising the bar for any students that don’t look like those who have been successful in the past. This will likely close down opportunities and make higher education ever more the preserve of those whose pre-existing privilege is the most ingrained while marginalising those for whom the transformation potential of higher education is the greatest. Furthermore, although we do not want to endorse a post-pandemic deficit model, it is unrealistic to ignore pre-HE ‘gaps’. Widely evidenced amplification of disadvantage will distort the creaming effect further.

Universities will be incentivised by these proposals not only to recruit those who are less likely to drop out and more likely to ’succeed’ in the graduate labour market, but they will also face potentially severe punishments for doing anything other than that. With entry tariff cited as irrelevant context – notwithstanding the messaging that KS5 results are not an equitable proxy for potential – creaming is inevitable. Meanwhile, providers who currently make the greatest contribution to the access and participation agenda risk being penalised for their efforts.

Diversity

There will be pressures against diversity of students for the same reasons as above, and this will be felt in engineering where SEND teaching approaches are commonly applied even when there is no diagnosis. Engineering’s efforts to better balance gender (women earn less than men) would be disincentivised. It also seems likely that these proposals would trigger the re-introduction of the traditional maths and physics A level prerequisite, undoing years of work to improve access and diversity, particularly gender diversity in engineering which is essential to address the well documented skills gaps.

With HE planners able to pore over a host of split indicators, the volume of data required to hunt the outliers to improve progression rates – together with the complexity of internal analysis of split indicators by other split indicators (i.e. subject by sex and domicile) – could reasonably be such that the resource to understand the student factors affecting underperformance is greater than the benefit. Why bother when the known ‘problem areas’ (i.e. courses in the arts and any students that don’t look like those who have been successful in the past) can simply be cut off at source? Even without such a sweeping approach, without the ability of adequate targeting, providers will have little choice but to use a sledgehammer to crack a walnut; a disaster for diversity.

Teaching and learning

This approach threatens multidisciplinarity and innovation (if-it-ain’t-broke-don’t-you-dare-try-to-fix-it and copy-whatever-works-for-someone-else-or-always-used-to-work). Institutional autonomy allows and encourages innovative approaches, driving quality and adaptation to an ever-changing world (which is particularly important in Engineering). It facilitates innovation and diversity to serve diverse needs and a diverse body of students, serving diverse societal, economic and labour market needs and is often cited as a contributing factor to the UK’s high quality HE sector. OfS should use its position and powers to encourage innovation rather than incentivise homogenous approaches designed to deliver metric-satisfying outcomes.

Outcomes

The sole definition of a ‘positive outcome’ being defined in terms of ‘the proportion of students progressing to managerial or professional employment, or further study’ fails to address the concerns raised by EPC and others that the purpose of higher education is broader than employment. As previous attempts to define “graduateness” have proved.

By recognising only the ‘exchange value of HE’ – that is the development of skills and knowledge in order to pass the required assessments and gain a ‘qualification’ that will allow the individual to access career choices that are not possible without this qualification – the OfS reject the other aims of HE including:

  • use value: the development of skills and knowledge for use in employment and everyday life;
  • self- actualisation: the benefits for mental and spiritual wellbeing given to an individual when they are able to reach their full potential;
  • education of the next generation: by educating one generation, we are providing the generations to come with a support-network better able to nurture their educational needs. The critical role played by the family in supporting education has never been more clear as during the pandemic where parents and guardians have been expected to home-school.

This leads to a distortion of education where “teaching to the test” becomes the norm; this also leads students to view their educational experience as transactional rather than the rich transcendent experience that it should be. This generates an employee pool which does not reflect the requirements of business; life-long learners driven to deliver their very best every day.

Context

Employment outcomes are only indirectly related to factors within the universities’ control such as the quality of teaching and support. The outcomes baselines do not adequately consider that these measures are highly influenced by region, by industrial sector and by imbalances in recruitment practices (gender, socio-economic background, etc.). Not to mention wider economic conditions. Outcomes will always be the product of the health of the economy in the context of regional, national and global market forces that are far beyond the control of the sector. We note that factors outside a provider’s control inform the historical performance aspect of the context submission, but with recession, pandemic and recovery accounting for most of the past decade and a half and accepting that employment measures for the next few years may be wildly unrepresentative of actual standards or performance over time, this surely undermines the whole process anyway?

Regional development and levelling up

The value of higher education in improving social mobility, developing local economies, and “levelling up” – especially post-pandemic – is in addressing the precise inequalities that these proposals ignore. Under these proposals, institutions where graduates supply workplaces in disadvantaged parts of the country will be in the bottom third of the table for employment and be at risk of being fined. It’s been pointed out that the best thing a university like Sunderland could do is relocate to London.  

The approach would benefit from a regional appraisal and nuance to prevent local brain drains and enforced geographic mobility. Local retention / employment is not only a positive outcome for graduates, but also a really positive choice for the Government’s goal of levelling up regions by creating high-skilled employment in disadvantaged areas. Given regional variations in the labour market, the goal of regional development through educating and upskilling the local workforce is inconsistent with the desirable outcomes as set out in the strategy as it stands. It should not fall to universities to resource consultants to outline regional priorities to a government regulator.

Standards

Universities will be disincentivised to let anyone fail, leave or even repeat a year. Instead, everyone will pass (grade inflation) and students who have lost enthusiasm for their course will be conveyor belted through rather than allowed to drop out.

Engineering is heavily regulated; in any accredited engineering course there is already a very clearly defined set of standards, governed by the Engineering Council and assured by Professional Engineering Institutions. Indeed, PSRBs already set standards in terms of learning outcomes in engineering (and many other professional higher education courses). Accreditation standards are high; but if meeting accreditation standards threatens performance metrics, universities may opt not to be accredited rather than risk letting students fail or drop out. This is a threat to standards.

We would suggest prioritising subjects where there are no PSRBs.

High-cost courses

While expensive, universities are likely to see that engineering has a positive effect on their indicators overall, which could be good news for engineering. However, these proposals are clearly a threat to the arts. How will universities be able to afford to run the more costly engineering courses without cross subsidisation?

Non-standard provision

These proposals may deter future partnerships, limit student choice, and disproportionately impact upon small, specialist providers. For example, data for student cohorts of fewer than 23 will be suppressed. Typically, the proposed statistical uncertainty approach in this context relates to the size of the student population, where smaller numbers can lead to greater uncertainty.