Teaching Excellence Framework (TEF): implications for engineering

Introduction

February 2022

As promised, the OfS is consulting on the future of the TEF based on the narrative that putting a reputational spotlight on provider quality informs student choice and drives improvement. The TEF builds on the proposed continuous minimum requirements for positive student outcomes under Condition B3 shifting the focus periodically to excellence in teaching, learning and student outcomes. 
The consultation follows a statutory independent review chaired by Shirley Pearce, which argued for a rebalancing of metrics and qualitative information articulating an institutional view, evidence of excellence, and institutional education gain measures. The Pearce review also recommended that students be supported to make their own independent submission alongside their provider.
 Under the new proposals, universities and colleges in England would be assessed on undergraduate courses for a TEF award – of gold, silver or bronze – every four years, plus a new designation of ‘requiring improvement’. 
It is proposed to open the submission window between September and November 2022 and announce outcomes in spring 2023. The last TEF exercise was carried out in 2019.

Summary of proposals

The EPC has, summarised the proposals. Read our summary here.

Further information

You can read the full OfS consultation and watch introductory video presentations here. You can watch the recorded OfS consultation / Q&A event here.

Complete our survey

The EPC is conducting a full member consultation. Please submit your views here. Note, there is a second EPC member consultation in relation to student outcomes (here).

 

Indicators

NSS Students’ self-reported satisfaction is emphatically not equivalent to teaching quality, but rather a reflection of the gap between the student’s expectation and what they perceive to have been delivered. This can be seen in the long continual variance in self-reported satisfaction rates in these categories in NSS data between students studying STEM subjects, which tend to be higher than among students studying arts subjects. Most students lack an objective point of reference for what good teaching at higher education level looks like and, while student satisfaction is an important indicator to monitor, it is not appropriate to use it as a proxy for teaching quality. 

There are also other problems with NSS data, including: (a) the likelihood that satisfaction is less dependent on teaching quality than on demographic patterns (gender, age, socioeconomic background, ethnicity, etc) and on students’ circumstances (commuter students, part-time study, part-time work, etc); (b) the possibility of gaming; and (c) non-comparability owing to factors such as student boycotts.

NSS is subject to a continuing review.

Educational gain

This is a seriously underdeveloped concept intended to recognise the diversity of mission, not to be confused with learning gain, HEFCE work on which all but disappeared under OfS. Under the proposals providers will report on the education gain of their students (and, importantly, how this may vary across subjects) with little guidance. See Wonkhe for an interesting blog on this important facet.

Requires improvement

The introduction of a new award category of “requires improvement” (RI). So, what is TEF if not designed to be used heuristically?  That is to say it is used over-simplistically as a shortcut to ‘good’, ‘okay’ and ‘bad.’ Heuristic information discourages students from further consideration of their choice and encourages sub-optimal choices. A greater range of outcomes compared with the previous iterations of the TEF will just shortcut some of those institutions previously at the lower end of the Bronze level to ‘really bad’. Requires improvement can only be extremely damaging unless it is applied very leniently in terms of context.

Given the availability and presentation of the proposed student outcomes data, in Engineering the dashboards will, in almost all cases, provide the confidence students need, without the need for TEF. Research shows that students chose subjects, not universities, first.

Qualitative statements

In the context of RI, these will be essential to pick out institutions adding a great deal on the basis of prior attainment, specialist (and arts) institutions, those delivering local education and regional levelling up. However, this additional weight to providers’ case for their excellence in their submission, will create inequity; smaller universities and specialist providers (those most likely to be RI) will not have the resources for specialist consultants to write these.

Accreditation

In Engineering, accreditation is the baseline, regardless of B3 proposals. There is an established system of accreditation of engineering degrees, and degrees accredited by Professional Engineering Institutions licensed by the Engineering Council are recognised internationally through a number of international accords. The accreditation process focuses on assuring that degrees will deliver to at least a threshold standard of learning outcomes specified by the engineering profession. These learning outcomes are developed and maintained in consultation with employers and other stakeholders. There may be synergies between subject-level TEF and accreditation, although these have distinct and different purposes. At the moment, the two exercises – although they are designed for fundamentally different purposes and should never be conflated – create obfuscation rather than clarity. For example, some Bronze courses may be accredited under the Engineering Council while some courses deemed Gold may not be.  Of the two, accreditation serves its purpose more effectively.

The new four-year TEF cycle means a provider will hold an award for longer and coupled with the accreditation timescales in Engineering, this is likely to mean that anyone using the rating may be looking at information that is constantly partial, out-of-date and potentially conflicting.

Timing

The plan to open the TEF submission window in September 2022 will make for a high-pressure start to the new academic year, which could put added strain on already-stretched staff who are already set to be dealing with the third academic year under the uncertainty of Covid.

The exclusion of TNE (i.e. the Aggregate Offshore Record) and HE modules or credit only courses is not commensurate with the current policy thinking around microcredentials and the Lifelong Loan Entitlement (LLE). We fear that the overall approach adopted accross OfS does not adequately accommodate courses which allow students to do anything other than join a course, stay the duration, and graduate. Given the Government’s intention to expand flexibility throughout the education sector to encourage lifelong learning, modularity and hop-on-hop-off courses, the applicability of the TEF in the light of hop-on-hop-off and emergent HTQs is remiss. This is a game changer for the sector.