View the consultation here or here.
View the EPC’s response here.
Summary
The Department for Education sought views on the proposed design of the subject-level Teaching Excellence and Student Outcomes Framework (TEF). The TEF aims to recognise and reward excellence in teaching, learning and student outcomes in higher education providers in the UK. Following an announcement in May 2016, the Government seeks comments on the best and most proportionate way to undertake TEF assessments at a disciplinary (subject) level. This consultation ran from March to May 2018.
The EPC fully supports the key rationale behind subject-level TEF; to provide potential students with timely and relevant information to help inform their decisions. However, the research available clearly shows that this type of information tends to be used over-simplistically as an indicator of quality or a shortcut to ‘good’, ‘okay’ and ‘bad. and does not directly support meaningful choice. The EPC strongly believes that the subject-level TEF must consciously avoid the trap of heuristic choice. The less granular the subject-level TEF data is, the less meaningful it will be to the point of, in practice, misleading potential students.
Members of the EPC Executive have worked closely with the Royal Academy of Engineering, the Engineering Council, and other members of E4E to draft a joint and detailed response to this important consultation. The key points of our draft joint response are summarised below. Thank you to those of you who fed back to our call for evidence, which we ensured was mapped into the response.
The subject level TEF proposes to define subjects using thirty-five subjects which groups Engineering with Computing and Technology
The EPC responded through E4E that:
- Any system of higher education (HE) subject classification will involve a degree of arbitrariness and thus limitations.
- It is important any parameters used for subjects do no more than is strictly necessary to aggregate different subjects together for administrative purposes.
- The system used for the purpose of the subject-level TEF should be one already in use (the proposed CAH2 will suffice).
- We believe the underpinning principle of any system must be for the greatest degree of granularity that is reasonably possible without losing comparative impact.
A longer duration and re-application period is proposed in subject level TEF
The EPC responded though E4E that:
- There is a balance between the data being relevant (timeliness is a key part of this) and the practicalities and burden of data collection.
- The proposed extension to five or six years between reassessments is simply too great for the awarded TEF rating to be meaningful as it may reflect an assessment made seven years previously (taking into account the time between the rating being awarded and publication).
- The HE sector is moving away from annual data collection towards in-year collections as recommended by the Higher Education Data & Information Improvement Programme (HEDIIP). The data collection that will be necessary under subject-level TEF should not act contrary to this development.
The subject level TEF could potentially retain the existing key elements of the provider level framework (including the 10 TEF criteria, the same suite of metrics, benchmarking, submissions, an independent panel assessment process and the rating system)
The EPC responded though E4E that:
- Subject-level TEF must aim to encourage prospective students to explore the context of performance rather than stripping away that context.
- We strongly advise against use of the ‘gold’, ‘silver’, ‘bronze’ grading system from the provider-level TEF as it is inappropriate.In order to avoid misrepresentation and encouragement of a heuristic misuse of the data in student choice, either the terminology of ‘TEF’ needs to be changed or the intended metrics.
- We would not wish to see the broadly acknowledged issues with many of the key elements of the provider level TEF framework amplified by carrying them across to the subject-level TEF including that:
- Student satisfaction is not a proxy for teaching excellence.
- Continuation is misrepresented as indicative of teaching quality.
- DLHE is related to outcomes and is not related to teaching.
- Grade inflation is an issue to be addressed but TEF, particularly at subject-level, is not the appropriate vehicle to do this.
- Whilst LEO data does represent progress as an employment outcome metric over DLHE, measuring income is inherently crude, valuing industrialist engineers over academics and (early stage) entrepreneurs.
- Qualified to teach and student engagement metrics may be better measures of teaching excellence.
- It is critical that the panel assessments are done by engineers with an understanding of both the discipline and HE pedagogies.
- The subject-level TEF does not purport to, and is not capable of, comparing subjects in the latter sense, but it is highly unlikely that potential students and the wider public will realise this.
- It is common for students to move between BEng and MEng programmes depending upon their performance and personal circumstances. We are concerned that continuation metrics at subject level may misrepresent the general picture for students on engineering courses at each institution. Furthermore, we are generally unclear on how integrated masters degrees fit into this scheme (over half of integrated masters students are engineers or scientists), a particular issue for engineering where a postgraduate qualification is still often demanded for working engineers. It is also not clear to us where higher apprenticeships (that range from level 4 to level 7) and degree apprenticeships (available at levels 6 and 7, and even 8) sit in this framework.
PSRB accreditation or recognition could be made a mandatory declaration
The EPC responded though E4E that:
- We cannot think why an Engineering provider would not wish to declare their course was PSRB accredited, but we would object to this being used as a metric.
- Course accreditation is completed on a five-yearly cycle and so there is a time lag factor to be considered, as well as that many good engineering degrees will not have current accreditation.
New measures are proposed
The EPC responded though E4E that:
- We agree all of forms of contact and learning should be captured.
- We are opposed to a measure of teaching intensity.
- The subject-level TEF should encourage diversity and innovation to ensure continued progress in effective teaching practice.
- There are many models of student learning not captured in the current framework and further sophistications that attempted to capture all these different ways of learning would serve only to complicate the metric and would still exclude other learning approaches.
