Authors: Dr. Natalie Wint (UCL); Dr. William Bennett (Swansea University).

Keywords: Assessment; Accreditation, AHEP, Competencies; Curriculum design; Pedagogy.

Who is this article for?: This article should be read by educators at all levels in higher education who wish to integrate ethics into the engineering and design curriculum or module design.

Related content:

• Assessing ethics: Case study example: Water wars
• Methods for assessing and evaluating ethics learning in engineering education

Guidance

Premise:

As engineering educators, it is uncommon that we were taught or assessed on ethical thinking within our own degree programmes. Although we may be able to think of plenty of ethical scenarios from our own experience, we may not necessarily be able to identify the best way to assess the ability of a student to engage in ethical thinking in a systematic and robust manner, something which is critical for both the evaluation of learning and teaching (as explained further here).

Furthermore, the complex, ill-structured nature of ethical dilemmas, which often involve a variety of diverse stakeholders, perspectives and cultural norms, necessitates an ability to navigate tensions and compromise. This results in situations in which multiple possible courses of action can be identified, meaning that there is not one single ‘good’ or ‘correct’ answer to ethical questions posed.

It is also necessary to evidence that students are able to meet the criteria outlined by accreditation bodies. Within the UK context, it is the Engineering Council (EC) that is responsible for providing the principal framework which guides engineering course content and sets accreditation threshold standards of competence through AHEP, the Accreditation of Higher Education Programs, as part of The UK Standard for Professional Engineering Competence (UKSPEC).

The knowledge, skills and attributes expected of engineering graduates constantly shifts, and since the advent of AHEP in 2004 there has been increased focus on strengthening design, and consideration for economic, ethical, environmental, legal, and social factors.

In-keeping with a need to assess engineering ethics in a robust manner, this article provides step-by-step considerations for designing assessment and is primarily intended to be used in conjunction with an existing ethics case study, such as those available through the EPC’s Engineering Ethics Toolkit (we later make use of the existing ‘Water Wars’ case study to exemplify the points made).

The guidance and accompanying rubric have been designed in a way that encourages students to grapple with the numerous tensions involved in ethical decision making, and the focus is thus on assessment of the decision-making process as opposed to the ‘answer’ given, the decision made or the outcome of the scenario.

Assessment purpose:

The first consideration is the year group you are assessing, and the competencies they have already acquired (for example in the case of Level 5 and Level 6 students). You may want to consider the (partial) learning outcome (LO) as defined by AHEP4 LO8 (Table 1). Whilst this shouldn’t act to limit what you choose to assess, it is a good place to start in terms of the level of ability your students should be demonstrating.

Note that the Engineering Council (EC) claim “This fourth edition of AHEP has reduced the total number of learning outcomes in order to focus attention on core areas, eliminate duplication and demonstrate progression between academic levels of study”. They are thus interested in the differences between level. You are recommended to make this explicit in module specification and associated assessment description. Key differentiations are shown in Table 1. For example, at Level 5 you may be more interested in students’ abilities to identify an ethical situation, whereas at Level 6 you may want them to be able to reason through options or make a judgement.

Table 1: AHEP4 Learning Outcomes

The final row in Table 1 provides our interpretation of the LO, making use of language similar to that within the EPC’s Ethics Learning Landscape. We believe this is more accessible and more easily operationalised.

The following steps outline the process involved in designing your assessment. Throughout we make reference to an existing EPC case study (Water Wars) to exemplify the points made.

1.) The first consideration is how much time you have and how much of the case study you want to use. Many of the case studies have multiple stages and could be spread over several sessions depending on time constraints.

2.) Linked to this is deciding whether you want to assess any other LOs within the assessment. For example, many of the case studies have technical elements. Furthermore, when using reports, presentations, or debates as methods of assessment you may also want to assess communication skills. Whatever you decide you should be careful to design the assessment in such a way that assesses LO8 in a robust manner, whereby the student could not pass the element without demonstrating they have met the individual LO to the required level (this is a key requirement to meet AHEP4). For example, in an assessment piece where ethics is worth 50% of the grade, a student could still pass the element as a whole (with 40%) by achieving high scores in the other grading criteria without the need to demonstrate their ability to meet LO8.

3.) Once you are aware how much of a case study you have time for and have decided which LOs (other than LO8) you are assessing, you should start to determine which questions are aligned with the level of study you are considering and/or the ability of the students (for example you may query whether students at Level 5 have already developed the skills and competencies suggested for Level 4). At each level you can make use of the accompanying rubric to help you consider how the relevant attributes might be demonstrated by students. As an example, please refer to the accompanying document where we provide our thoughts about how we would assess Water Wars at Levels 4-6.

4.) Once you have selected questions you could look to add any complementary activities or tasks (that do not necessarily have to be assessed) to help the students broaden their understanding of the problem and ability to think through their response. For example, in the Water Wars case study, there are multiple activities (for example Part 1, Q3 and Part 2, Q3, Q4, Q6, Q7) aimed at helping students understand different perspectives which may help them to answer further ethical questions. There are also technical questions (for example Part 1, Q5) which help students understand the integrated nature of technical and social aspects and contextualise scenarios.

5.) Once you have selected your questions you will need to make a marking rubric which includes details of the weightings given for each component of the assessment. (This is where you will need to be careful in selecting whether other LOs are assessed e.g., communication, and whether a student can pass the assessment/module without hitting LO8). You can then make use of the guidance provided in terms of expectations at a threshold and advanced level, to write criteria for what is expected at each grade demarcation.

Although we have focused on ‘Water Wars’ here, the suggested assessment questions within the accompanying rubric have been designed in such a way that they can be used in conjunction with the case studies available within the toolkit, or with another case study that has been created (by yourself or elsewhere) to outline an ethical dilemma.

Other considerations:

As acknowledged elsewhere within the toolkit (see here), there are “practical limits on assessment” (Davis and Feinerman, 2012) of ethics, including demands on time, pressure from other instructors or administrators, and difficulty in connecting assessment of ethics with assessment of technical content. These are some other considerations you may wish to make when planning assessment.

• Number of students and/or marking burden: With large student numbers you may be more inclined to choose a group assessment method (which may also be beneficial in allowing students to share perspectives and engage in debate), or a format which is relatively quick to mark/allows automated marking (e.g. a quiz). In the case of group work it is important to find a way in which to ensure that all students within each group meet the LO in a robust manner. Whilst assessment formats such as quizzes may be useful for assessing basic knowledge, they are limited in their ability to ensure that students have developed the higher-level competencies needed to meet the LO at output level.

• Academic integrity: As with any LO there is a need to ensure academic integrity. This may be particularly difficult for large cohorts and group work. You may wish to have a range of case studies or ensure assessment takes place in a controlled environment (e.g. an essay/report under exam conditions). This is particularly important at output level where you may wish to provide individual assessment under exam conditions (although competencies may be developed in groups in class).

• Logistics/resourcing: Many of the competencies associated with ethics are heavily linked to communication and argumentation, and answers tend to be highly individual in nature. Role play, debates, and presentations may therefore be considered the most suitable method of assessment. However, their use is often limited by staffing, room, and time constraints. Many of these methods could, instead, be used within class time to help students develop competencies prior to formal assessment. You may also choose to assess ethics in another assessment which is more heavily resourced (for example design projects or third year projects).

• Staged assessment: The ethical reasoning process benefits from different perspectives. It may therefore be desirable to stage assessment in such a way that individuals form their own answer (e.g. a moral problem statement), before sharing within a group. In this way a group problem statement, which benefits from multiple perspectives and considerations, can be formed. Similarly, individuals may take the role of an individual stakeholder in an ethical dilemma before coming together as a group.

• Use of exams: Whilst we see an increasing movement away from exams, we feel that a (closed book) exam is a suitable method of assessment of ethics based LOs in the situation that:

o There is a need to ensure academic integrity, and that each student meets the LO at output level.

o The exam is assessing competencies (e.g. ethical argumentation) as opposed to knowledge.

o All the relevant information needed is provided and there is limited content for students to learn in advance (aside from argumentation, justification, decision making skills etc developed in class).

Their use may therefore be limited to Level 6.

Rubric

This document provides the partial AHEPLO8 at each level. The competences involved in meeting this LO have then been identified, along with what students would need to demonstrate to evidence meeting a threshold level, or advanced level. Example questions are given to show how students may demonstrate their competence at each level. For each question there is an explanation of how the question supports achievement of LO at that level. The rubrics should be used alongside the accompanying guidance document which offers practical suggestions and advice.

Year 1: This year focuses on developing awareness of issues, obligations, and responsibilities, and sensitising students to ethical issues.

Year 2: This year focuses on developing the ability to identify ethical issues and to examine opposing arguments, all of which is needed to examine, analyse, and evaluate ethical dilemmas in Year 3.

Year 3: This year focuses on ensuring that students can satisfy LO8 at an output level in a robust manner.

References:

Davis, M. and A. Feinerman. (2012). ‘Assessing graduate student progress in engineering ethics’, Science and Engineering Ethics, 18(2), pp. 351-367.

Downloads:

Assessing ethics: Guidance

Assessing ethics: Rubric

Assessing ethics: Case study assessment example: Water Wars

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Any views, thoughts, and opinions expressed herein are solely that of the author(s) and do not necessarily reflect the views, opinions, policies, or position of the Engineering Professors’ Council or the Toolkit sponsors and supporters.