Informed consent Archives - Engineering Professors Council

Home
Toolkits
Informed consent

Authors: Ahmet Omurtag (Nottingham Trent University); Andrei Dragomir (National University of Singapore / University of Houston).

Topic: Data security of smart technologies.

Engineering disciplines: Electronics; Data; Biomedical engineering.

Ethical issues: Autonomy; Dignity; Privacy; Confidentiality.

Professional situations: Communication; Honesty; Transparency; Informed consent; Misuse of data.

Educational level: Advanced.

Educational aim: Practising Ethical Analysis: engaging in a process by which ethical issues are defined, affected parties and consequences are identified, so that relevant moral principles can be applied to a situation in order to determine possible courses of action.

Learning and teaching notes:

This case involves Aziza, a biomedical engineer working for Neuraltrix, a hypothetical company that develops Brain-computer interfaces (BCI) for specialised applications. Aziza has always been curious about the brain and enthusiastic about using cutting-edge technologies to help people in their daily lives. Her team has designed a BCI that can measure brain activity non-invasively and, by applying machine learning algorithms, assess the job-related proficiency and expertise level of a person. She is leading the deployment of the new system in hospitals and medical schools, to be used in evaluating candidates being considered for consultant positions. In doing so, and to respond to requests to extend and use the BCI-based system in unforeseen ways, she finds herself compelled to weigh various ethical, legal and professional responsibilities.

This case study addresses two of AHEP 4’s themes: The Engineer and Society (acknowledging that engineering activity can have a significant societal impact) and Engineering Practice (the practical application of engineering concepts, tools and professional skills). To map this case study to AHEP outcomes specific to a programme under these themes, access AHEP 4 here and navigate to pages 30-31 and 35-37.

The dilemma in this case is presented in three parts. If desired, a teacher can use the Summary and Part one in isolation, but Parts two and three develop and complicate the concepts presented in the Summary and Part one to provide for additional learning. The case allows teachers the option to stop at multiple points for questions and/or activities as desired.

Learners have the opportunity to:

analyse the ethical dimensions of an engineering situation;
identify professional responsibilities of engineers in an ethical dilemma;
determine and defend a course of action in response to an ethical dilemma;
practise professional communication;
debate viable solutions to an ethical dilemma.

Teachers have the opportunity to:

highlight professional codes of ethics and their relevance to engineering situations;
address approaches to resolve interpersonal and/or professional conflict;
integrate technical content on software and/or cybersecurity;
informally evaluate students’ critical thinking and communication skills.

Learning and teaching resources:

Legal regulations:

European Union’s General Data Protection Regulation (GDPR)

Professional organisations:

Philanthropic organisations:

Brain-computer interfaces and neuroethics

Journal articles:

Educational institutions:

Summary:

Brain-computer interfaces (BCIs) detect brain activity and utilise advanced signal analysis to identify features in the data that may be relevant to specific applications. These features might provide information about people’s thoughts and intentions or about their psychological traits or potential disorders, and may be interpreted for various purposes such as for medical diagnosis, for providing real-time feedback, or for interacting with external devices such as a computer. Some current non-invasive BCIs employ unobtrusive electroencephalography headsets or even optical (near-infrared) sensors to detect brain function and can be safe and convenient to use.

Evidence shows that the brains of people with specialised expertise have identifiable functional characteristics. Biomedical technology may translate this knowledge soon into BCIs that can be used for objectively assessing professional skills. Researchers already know that neural signals support features linked to levels of expertise, which may enable the assessment of job applicants or candidates for promotion or certification.

BCI technology would potentially benefit people by improving the match between people and their jobs, and allowing better and more nuanced career support. However, the BCI has access to additional information that may be sensitive or even troubling. For example, it could reveal a person’s health status (such as epilepsy or stroke), or it may suggest psychological traits ranging from unconscious racial bias to psychopathy. Someone sensitive about their privacy may be reluctant to consent to wearing a BCI.

In everyday life, we show what is on our minds through language and behaviour, which are normally under our control, and provide a buffer of privacy. BCIs with direct access to the brain and increasing capability to decode its activity may breach this buffer. Information collected by BCIs could be of interest not only to employers who will decide whether to hire and invest in a new employee, but also to health insurers, advertising agencies, or governments.

Optional STOP for questions and activities:

1. Activity: Risks of brain activity decoding – Identify the physical, ethical, and social difficulties that could result from the use of devices that have the ability to directly access the brain and decipher some of its psychological content such as thoughts, beliefs, and emotions.

2. Activity: Regulatory oversight – Investigate which organisations and regulatory bodies currently monitor and are responsible for the safe and ethical use of BCIs.

3. Activity: Technical integration – Investigate how BCIs work to translate brain activity into interpretable data.

Dilemma – Part one:

After the company, Neuraltrix, deployed their BCI and it had been in use for a year in several hospitals, its lead developer Aziza became part of the customer support team. While remaining proud and supportive of the technology, she had misgivings about some of its unexpected ramifications. She received the following requests from people and institutions for system modifications or for data sharing:

1. A hospital asked Neuraltrix for a technical modification that would allow the HR department to send data to their clinical neurophysiologists for “further analysis,” claiming that this might benefit people by potentially revealing a medical abnormality that might otherwise be missed.

2. An Artificial Intelligence research group partnering with Neuraltrix requested access to the data to improve their signal analysis algorithms.

3. A private health insurance company requested Neuraltrix provide access to the scan of someone who had applied for insurance coverage; they stated that they have a right to examine the scan just as life insurance agencies are allowed to perform health checks on potential customers.

4. An advertising agency asked Neuraltrix for access to their data to use them to fine-tune their customer behavioural prediction algorithms.

5. A government agency demanded access to the data to investigate a suspected case of “radicalisation”.

6. A prosecutor asked for access to the scan of a specific person because she had recently been the defendant in an assault case, where the prosecutor is gathering evidence of potential aggressive tendencies.

7. A defence attorney requested data because they were gathering potentially exonerating evidence, to prove that the defendant’s autonomy had been compromised by their brain states, following a line of argument known as “My brain made me do it.”

Optional STOP for questions and activities:

1. Activity: Identify legal issues – Students could research what laws or regulations apply to each case and consider various ways in which Neuraltrix could lawfully meet some of the above requests while rejecting others, and how their responses should be communicated within the company and to the requestor.

2. Activity: Identify ethical issues – Students could reflect on what might be the immediate ethical concerns related to sharing the data as requested.

3. Activity: Discussion or Reflection – Possible prompts:

- Do you, as a biomedical engineer, have any duty to the people who have been scanned? Do you have more or less of a responsibility to these people or to Neuraltrix?
- If you find that a fellow employee has already shared the data without telling others, how should you act? Should you worry that revealing this employee’s actions might cause distress or create distrust in the integrity of the entire system? Is there anyone else you should inform? Are there any risks you may be able to mitigate immediately?
- Do you think the reasons and justifications given for the data requests listed above are legitimate?
- Who owns the data collected by the BCI? Should it be protected? How, and for how long? Who should maintain it?

Dilemma – Part two:

The Neuraltrix BCI has an interface which allows users to provide informed consent before being scanned. The biomedical engineer developing the system was informed about a customer complaint which stated that the user had felt pressured to provide consent as the scan was part of a job interview. The complaint also stated that the user had not been aware of the extent of information gleaned from their brains, and that they would not have provided consent had been made aware of it.

Optional STOP for questions and activities:

1. Activity: Technical analysis – Students might try to determine if it is possible to design the BCI consent system and/or consent process to eliminate the difficulties cited in the complaint. Could the device be designed to automatically detect sensitive psychological content or allow the subject to stop the scan or retroactively erase the recording?

2. Activity: Determine the broader societal impact and the wider ethical context – Students should consider what issues are raised by the widespread availability of brain scans. This could be done in small groups or a larger classroom discussion.

Possible prompts:

- On the one hand, human assessors can be subject to bias and inconsistencies and, from this point of view, algorithmic assessment leaving human assessors out of the loop may be viewed as progress. On the other hand, some “black-box” algorithms used by the BCI have been criticised for opacity, hidden biases, and the difficulty of scrutinising their decisions. If a user is dissatisfied with the BCI-enhanced assessment, should they be able to opt out of it?
- If use of the Neuraltrix BCI became widespread, do you believe that humans could eventually irreversibly lose their assessment skills? Compare this with the potential loss of map-reading skills due to the easy access to Satellite Navigation systems.
- Can we dispense with human opinion and make assessment processes entirely “objective”?
- “Goodhart’s law,” named after the economist Charles Goodhart, states that when a measure is used as a tool, it becomes vulnerable to manipulation. Would Neuraltrix BCI create new opportunities for candidates to “game” the BCIs, and how would they do it?

Dilemma – Part three:

Neuraltrix BCI is about to launch its updated version, which features all data processing and storage moved to the cloud to facilitate interactive and mobile applications. This upgrade attracted investors and a major deal is about to be signed. The board is requesting a fast deployment from the management team and Aziza faces pressure from her managers to run final security checks and go live with the cloud version. During these checks, Aziza discovers a critical security issue which can be exploited once the BCI runs in the cloud, risking breaches in the database and algorithm. Managers believe this can be fixed after launch and request the engineer to start deployment and identify subsequent solutions to fix the security issue.

Optional STOP for questions and activities:

1. Activity: Students should consider if it is advisable for Aziza to follow requests from managers and the Neuraltrix BCI board and discuss possible consequences, or halt the new version deployment which may put at risk the new investment deal and possibly the future of the company.

2. Activity: Apply an analysis based on “Duty-Ethics” and “Rights Ethics.” This could be done in small groups (who would argue for management position and engineer position, respectively) or a larger classroom discussion. A tabulation approach with detailed pros and cons is recommended.

3. Activity: Apply a similar analysis as above based on the principles of “Act-Utilitarianism” and “Rule-Utilitarianism.”

Possible prompts:

- Should you, as a biomedical engineer, follow company rules and go ahead with manager’s requests or risk the future of the company (and possibly your job) and put deployment on hold until the security issue is fixed?
- Act utilitarianism principle, as advocated by John Stuart Mill, focuses on individual actions rather than on rule, therefore, actions should be judged based on whether they resulted in the most good outcome in a certain situation. Should the Neuraltrix BCI management be guided by this principle or rather by a cost-benefit approach?

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Any views, thoughts, and opinions expressed herein are solely that of the author(s) and do not necessarily reflect the views, opinions, policies, or position of the Engineering Professors’ Council or the Toolkit sponsors and supporters.

Authors: Cortney Holles (Colorado School of Mines); Ekaterina Rzyankina (University of Cape Town).

Topic: Critical digital literacy.

Engineering disciplines: Computer Science; Information Systems; Biomedical engineering.

Ethical issues: Cultural context; Social responsibility; Privacy.

Professional situations: Public health and safety; Working in area of competence; Informed consent.

Educational level: Intermediate.

Educational aim: Engaging in ethical judgement: reaching moral decisions and providing the rationale for those decisions.

Learning and teaching notes:

The case involves an engineering student whose personal choices may affect her future professional experience. It highlights both micro- and macro-ethical issues, dealing with the ways that individual actions and decisions can scale to create systemic challenges.

An ethical and responsible engineer should know how to work with and use digital information responsibly. Not all materials available online are free to use or disperse. To be digitally literate, a person must know how to access, evaluate, utilise, manage, analyse, create, and interact using digital resources (Martin, 2008). It is important to guide engineering students in understanding the media landscape and the influence of misleading information on our learning, our political choices, and our careers. A large part of critical digital literacy is evaluating information found on the web. For students working on a research project or an experiment, accessing accurate information is imperative. This case study offers several approaches to engaging students in the critique and improvement of their critical digital literacy skills. The foundations of this lesson can be applied in multiple settings and can be expanded to cover several class periods or simplified to be inserted into a single class.

The dilemma in this case is presented in two parts. If desired, a teacher can use the Summary and Part one in isolation, but Part two develops and complicates the concepts presented in the Summary and Part one to provide for additional learning. The case allows teachers the option to stop at multiple points for questions and / or activities as desired.

Learners have the opportunity to:

explore how personal privacy has become increasingly more vulnerable to surveillance and commodification;
discover ways that uploading personal information to social networks means that we will increasingly allow others access to our data;
understand how vulnerable personal information has become to information-aggregation and reselling activities;
evaluate messaging on social media and the news to critique the information they take in and the entities that post it.

Teachers have the opportunity to:

address the definition and importance of expertise;
provide students with practice in evaluating sources of information;
critique students’ approaches to searching, sharing, and comprehending information;
evaluate how students respond to and engage with messaging about science and engineering.

Learning and teaching resources:

News articles:

Educational institutions:

Legal regulations:

GDPR Policy

Non-profit organisations:

Debunking Viral Claims Archives – FactCheck.org

Business:

Meta: Working to Stop Misinformation and False News on Facebook

Summary:

Katherine is a biomedical engineering student in her 3rd year in 2022, and will have a placement in a community hospital during her last term at university. She plans to pursue a career in public health after seeing what her country went through during the Covid-19 pandemic. She wants to contribute to the systems that can prevent and track public health risks from growing too large to manage, as happened with Covid-19. She is motivated by improving systems of research and treatment for emerging diseases and knows that communication between a variety of stakeholders is of the utmost importance.

Optional STOP for questions and activities:

1. Discussion: What can you determine about Katherine’s values and motivation for her studies and her choice of career?

2. Discussion: How do you connect with her mission to improve diagnostic and treatment systems for public health threats?

3. Discussion: Who should be responsible for the messaging and processes for public health decisions? How are engineers connected to this system?

4. Activity: Research the Covid-19 vaccine rollout in the United Kingdom versus other countries – how did power, privilege, and politics influence the response?

5. Activity: Research current public health concerns and how they are being communicated to the public. In what ways might engineers affect how and what is communicated?

Dilemma – Part one:

As Katherine approaches the winter holiday season, she makes plans to visit her grandmother across the country. She hasn’t seen her since before the Covid-19 pandemic and is excited to be around her extended family for the holidays once again. However, she receives an email from her cousin informing everyone that he and his family are not vaccinated against Covid-19 because the whole vaccination operation was forced upon citizens and they refused to participate. Katherine is immediately worried for her grandmother – at 85 years old, she is at a higher risk than most – and for her brother, who suffers from Addison’s disease, an autoimmune disorder. Additionally, if Katherine comes into contact with Covid-19 while celebrating the holidays with her family, she could suffer repercussions at both her university and the hospital where she will work for her placement.

Optional STOP for questions and activities:

1. Discussion: How can Katherine communicate with her cousin about her concerns for her brother and grandmother? How might she use her expertise as a biomedical engineer in this conversation?

2. Discussion: What kind of information will be most convincing to support her decision? What sources would provide the evidence she is looking for, and which ones would provide counter arguments?

3. Discussion: What impacts might the decision have on Katherine’s position as a student or in the hospital?

4. Discussion: Do engineers, scientists, and medical professionals have more of an obligation to promote and adhere to public health guidance? Why or why not?

5. Activity: Talk to people in your life about their experience of navigating the Covid-19 vaccine. Did they choose to get it as soon as it was available? Did they avoid getting the vaccine for particular reasons? Were there impacts on their personal relationships or work because of their choices about the vaccine?

6. Activity: Research some of the impacts on individuals with health concerns and comorbidities in regard to Covid-19 and other viruses or public health concerns. How do these experiences match with or differ from your own?

7. Activity: Investigate the different ways that engineers were involved in vaccination development and response.

Dilemma – Part two:

Katherine went back to university after a lengthy break for the holidays and immediately registered for an account on Facebook as a brand-new user. She was in such a hurry to have her profile up that she did not take the time to configure any privacy settings. She stayed up late reading an article about Covid-19 that had been posted on the website of one of the online newspapers. Before she posted this report on her own Facebook page, she did not verify the accuracy of the information or the source of the information.

Optional STOP for questions and activities:

1. Discussion: What kind of impact might this social media activity have on Katherine’s position as a student or in the company/organisation/hospital she is working for as an intern? What should Katherine be worried or concerned about after posting information?

2. Discussion: Do social media companies collect or ask for any other non-essential information from you? Why does the website claim that they are collecting or asking for your information? Does the website share/sell/trade the information that they collect from you? With whom does the website share your collected information? How long does the website keep your collected information? Does the website delete your information, or simply de-personalise it?

3. Discussion: Regarding question 2, how are engineers involved with products, processes, or services that enable those choices and actions?

4. Discussion: What is real and fake news? How do you know? What do you look for to know if it is real or fake news (share guidelines)? Do you expect it to be easy to spot fake news? Why should we care if people distribute and believe fake news?

Students are particularly susceptible to being duped by propaganda, misleading information, and fake news due to the significant role that information and communication technology which is problematic to verify plays in their everyday life. Students devote a significant portion of their time to participating in various forms of online activity, including watching television, playing online games, chatting, blogging, listening to music, posting photos of themselves on social networking sites, and searching for other individuals with whom they can engage in online conversation. Students owe a significant portion of what they know about the world and how they perceive reality to the content that they read online. While many people share reliable and positive information online, others may engage in negative impact information sharing:

Mis-information – false information shared with no intention of causing harm
Dis-information – false information shared intentionally to cause harm
Mal-information – true information shared intentionally to cause harm.

5. Discussion: What are some other examples of how engineering might fall prey to negative impact information sharing?

6. Discussion: How might engineers help address the problem of fake news and negative impact information sharing?

References:

Martin, A. (2008). ‘Digital Literacy and the “Digital Society”’, in Lankshear C. and Knobel M. (eds.), Digital Literacies: Concepts, Policies, and Practices. New York: Peter Lang, (pp. 151-176).

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Authors: Dr Yujia Zhai (University of Hertfordshire); Associate Professor Scarlett Xiao (University of Hertfordshire).

Topic: Data security of industrial robots.

Disciplines: Robotics; Data; Internet of Things.

Ethical issues: Safety; Health; Privacy; Transparency.

Professional situations: Rigour; Informed consent; Misuse of data.

Educational level: Intermediate.

Educational aim: Gaining ethical knowledge. Knowing the sets of rules, theories, concepts, frameworks, and statements of duty, rights, or obligations that inform ethical attitudes, behaviours, and practices.

Learning and teaching notes:

This case study involves an engineer hired to develop and install an Industrial Internet of Things (IIoT) online machine monitoring system for a manufacturing company. The developments include designing the infrastructure of hardware and software, writing the operation manuals and setting policies. The project incorporates a variety of ethical components including law and policy, stakeholders, and risk analysis.

This case study addresses three of the themes from the Accreditation of Higher Education Programmes fourth edition (AHEP4): Design and Innovation (significant technical and intellectual challenges commensurate the level of study), the Engineer and Society (acknowledging that engineering activity can have a significant societal impact) and Engineering Practice (the practical application of engineering concepts, tools, and professional skills). To map this case study to AHEP outcomes specific to a programme under these themes, access AHEP 4 here and navigate to pages 30-31 and 35-37.

The dilemma in this case is presented in three parts. If desired, a teacher can use Part one in isolation, but Part two and Part three develop and complicate the concepts presented in Part one to provide for additional learning. The case study allows teachers the option to stop at multiple points for questions and/or activities as desired.

Learners have the opportunity to:

apply their ethical judgement relating to privacy and consent on the use of machine data;
determine the societal impact of a technical solution to a complex problem; 
analyse risks associated with ethical concerns and justify their ethical decisions;
communicate these risks and judgements to both technical and non-technical audiences.

Teachers have the opportunity to: 

highlight a range of ethical considerations within the scope of a complex engineering project;
introduce methods for risk analysis and ethical decision-making;
link Engineering Council statements of ethical principles with real world situations;
raise students’ awareness and demonstrate the importance of the ethics learning landscape.

Learning and teaching resources:

Professional organisations:

RAEng/Engineering Council Statement of Ethical Principles

Legal regulations:

UN agency:

Towards global code of ethics for AI research

Educational resource:

Government sites:

Educational institutions:

Ethical Risk Assessment

Summary: 

IIoT is a new technology that can provide accurate condition monitoring and predict component wear rates to optimise machine performance, thereby improving the machining precision of the workpiece and reducing the production cost.

Oxconn is a company that produces auto parts. The robotic manipulators and other automation machines on the production line have been developed at considerable cost and investment, and regular production line maintenance is essential to ensure its effective operation. The current maintenance scheme is based on routine check tests which are not reliable and efficient. Therefore Oxconn has decided to install an IIoT-based machine condition monitoring system. To achieve fast responses to any machine operation issues, the machine condition data collected in real time will be transferred to a cloud server for analysis, decision making, and predictive maintenance in the future.

Dilemma – Part one – Data protection on customers’ machines:

You are a leading engineer who has been hired by Oxconn to take charge of the project on the IIoT-based machine monitoring system, including designing the infrastructure of hardware and software, writing the operation manuals, setting policies, and getting the system up and running. With your background in robotic engineering and automation, you are expected to act as a technical advisor to Oxconn and liaise with the Facilities, Security, Operation, and Maintenance departments to ensure a smooth deployment. This is the first time you have worked on a project that involves real time data collection. So as part of your preparation for the project, you need to do some preliminary research as to what best practices, guidance, and regulations apply.

Optional STOP for questions and activities:

1. Discussion: What are the legal issues relating to machine condition monitoring? Machines’ real-time data allows for the identification of production status in a factory and is therefore considered as commercial data under GDPR and the Data Protection Act (2018). Are there rules specifically for IIoT, or are they the same no matter what technology is being used? Should IIoT regulations differ in any way? Why?

2. Discussion: Sharing data is a legally and ethically complex field. Are there any stakeholders with which the data could be shared? For instance, is it acceptable to share the data with an artificial intelligence research group or with the public? Why, or why not?

3. Discussion: Under GDPR, individuals must normally consent to their personal data being processed. For machine condition data, how should consent be handled in this case?

4. Discussion: What ethical codes relate to data security and privacy in an IIoT scenario?

5. Activity: Undertake a technical activity that relates to how IIoT-based machine monitoring systems are engineered.

6. Discussion: Based on your understanding of how IIoT-based machine monitoring systems are engineered, consider what additional risks, and what kind of risks (such as financial or operational), Oxconn might incur if depending on an entirely cloud-based system. How might these risks be mitigated from a technical and non-technical perspective?

Dilemma – Part two – Computer networks security issue brought by online monitoring systems:

The project has kicked off and a senior manager requests that a user interface (UI) be established specifically for the senior management team (SMT). Through this UI, the SMT members can have access to all the real-time data via their computers or mobiles and obtain the analysis result provided by artificial intelligence technology. You realise this has implications on the risk of accessing internal operating systems via the external information interface and networks. So as part of your preparation for the project, you need to investigate what platforms can be used and what risk analysis must be taken in implementation.

Optional STOP for questions and activities:

The following activities focus on macro-ethics. They address the wider ethical contexts of projects like the industrial data acquisition system.

1. Activity: Explore different manufacturers and their approaches to safety for both machines and operators.

2. Activity: Technical integration – Undertake a technical activity related to automation engineering and information engineering.

3. Activity: Research what happens with the data collected by IIoT. Who can access this data and how can the data analysis module manipulate the data?

4. Activity: Develop a risk management register, taking considerations of the findings from Activity 3 as well as the aspect of putting in place data security protocols and relevant training for SMT.

5. Discussion/activity: Use information in the Ethical Risk Assessment guide to help students consider how ethical issues are related to the risks they have just identified.

6. Discussion: In addition to cost-benefit analysis, how can the ethical factors be considered in designing the data analysis module?

7. Activity: Debate the appropriateness of installing and using the system for the SMT.

8. Discussion: What responsibilities do engineers have in developing these technologies?

Dilemma – Part three – Security breach and legal responsibility:

At the beginning of operation, the IIoT system with AI algorithms improved the efficiency of production lines by updating the parameters in robot operation and product recipes automatically. Recently, however, the efficiency degradation was observed, and after investigation, there were suspicions that the rules/data in AI algorithms have been subtly changed. Developers, contractors, operators, technicians and managers were all brought in to find out what’s going on.

Optional STOP for questions and activities:

1. Discussion: If there has been an illegal hack of the system, what might be the motive of cyber criminals?

2. Discussion: What are the impacts on company business? How could the impact of cyber-attacks on businesses be minimised?

3. Discussion: How could threats that come from internal employees, vendors, contractors or partners be prevented?

4. Discussion: When a security breach happens, what are the legal responsibilities for developers, contractors, operators, technicians and managers?

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Case enhancement: Facial recognition for access and monitoring

Activity: Prompts to facilitate discussion activities.

Author: Sarah Jayne Hitt, Ph.D. SFHEA (NMITE, Edinburgh Napier University).

Overview:

There are several points in this case during which an educator can facilitate a class discussion about relevant issues. Below are prompts for discussion questions and activities that can be used. These correspond with the stopping points outlined in the case. Each prompt could take up as little or as much time as the educator wishes, depending on where they want the focus of the discussion to be. The discussion prompts for Dilemma Part three are already well developed in the case study, so this enhancement focuses on expanding the prompts in Parts one and two.

Dilemma Part one – Discussion prompts:

1. Legal Issues. Give students ten minutes to individually or in groups do some online research on GDPR and the Data Protection Act (2018). In either small groups or as a large class, discuss the following prompts. You can explain that even if a person is not an expert in the law, it is important to try to understand the legal context. Indeed, an engineer is likely to have to interpret law and policy in their work. These questions invite critical thinking and informed consideration, but they do not necessarily have “right” answers and are suggestions that can help get a conversation started.

a. Are legal policies clear about how images of living persons should be managed when they are collected by technology of this kind?

b. What aspects of these laws might an engineer designing or deploying this system need to be aware of?

c. Do you think these laws are relevant when almost everyone walking around has a digital camera connected to the internet?

d. How could engineers help address legal or policy gaps through design choices?

2. Sharing Data. Before entering into a verbal discussion, either pass out the suggested questions listed in the case study on a worksheet or project on a screen. Have students spend five or ten minutes jotting down their personal responses. To understand the complexity of the issue, students could even create a quick mind map to show how different entities (police, security company, university, research group, etc.) interact on this issue. After the students spend some time in this personal reflection, educators could ask them to pair/share—turn to the person next to them and share what they wrote down. After about five minutes of this, each pair could amalgamate with another pair, with the educator giving them the prompt to report back to the full class on where they agree or disagree about the issues and why.

3. GDPR Consent. Before discussing this case particularly, ask students to describe a situation in which they had to give GDPR consent. Did they understand what they were doing, what the implications of consent are, and why? How did they feel about the process? Do they think it’s an appropriate system? This could be done as a large group, small group, or through individual reflection. Then turn the attention to this case and describe the change of perspective required here. Now instead of being the person who is asked for consent, you are the person requiring consent. Engineers are not lawyers, but engineers often are responsible for delivering legally compliant systems. If you were the engineer in charge in this case, what steps might you take to ensure consent is handled appropriately? This question could be answered in small groups, and then each group could report back to the larger class and a discussion could follow the report-backs.

4. Institutional Complexity. The questions listed in the case study relate to the fact that the building in which the facial recognition system will be used accommodates many different stakeholders. To help students with these questions, educators could divide the class into small groups, with each group representing one of the institutions or stakeholder groups (college, hospital, MTU, students, patients, public, etc.). Have each group investigate whether regulations related to captured images are different for their stakeholders, and debate if they should be different. What considerations will the engineer in the case have to account for related to that group? The findings can then be discussed as a large class.

Dilemma Part two – Discussion prompts:

The following questions relate to macroethical concerns, which means that the focus is on wider ethical contexts such as fairness, equality, responsibility, and implications.

1. Benefits and Burdens. To prepare to discuss the questions listed in the case study, students could make a chart of potential harms and potential benefits of the facial recognition system. They could do this individually, in pairs or small groups, or as a large class. Educators should encourage them to think deeply and broadly on this topic, and not just focus on the immediate, short-term implications. Once this chart is made, the questions listed in the case study could be discussed as a group, and students asked to weigh up these burdens and benefits. How did they make the choices as to when a burden should outweigh a benefit or vice versa?

2. Equality and Utility. To address the questions listed in the case study, students could do some preliminary individual or small group research on the accuracy of facial recognition systems for various population groups. The questions could then be discussed in pairs, small groups, or as a large class.

3. Engineer Responsibility. Engineers are experts that have much more specific technical knowledge and understanding than the general public. Indeed, the vast majority of people have no idea how a facial recognition system works and what the legal requirements are related to it, even if they are asked to give their consent. Does an engineer therefore have more of a responsibility to make people aware and reassure them? Or is an engineer just fulfilling their duty by doing what their boss says and making the system work? What could be problematic about taking either of those approaches?

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Authors: Matthew Studley (UWE Bristol); Sarah Jayne Hitt, Ph.D. SFHEA (NMITE, Edinburgh Napier University).

Keywords: Pedagogy; Personal ethics; Risk.

Who is this article for? This article should be read by educators at all levels in higher education who wish to integrate ethics into the engineering and design curriculum, or into module design and learning activities. It describes techniques that can help to provide students with opportunities to practise the communication and critical thinking skills that employers are looking for.

Premise:

Discussing ethical issues can be a daunting prospect, whether one-to-one or with an entire classroom. Ethics often addresses topics and decisions related to moral choices and delicate situations about which people may have firm and long-held beliefs. Additionally, these issues are often rooted in underlying values which may differ between people, cultures, or even time periods. For instance, something that was considered immoral or unethical in a rural community in 18th-century Ireland may have been viewed very differently at the same time in urban India. Because students come from different backgrounds and experiences, it is essential to be sensitive to this context (Kirk and Flammia, 2016). However, ethics also requires that we address tough topics in order to make decisions about what we should do in difficult situations, such as those encountered by engineers in their personal, professional, and civic lives.

Why we need to be sensitive in discussions about ethics:

Discussions about tough topics can be ‘triggering’. Psychologists define a psychological ‘trigger’ as a stimulus that causes a painful memory to resurface. A trigger can be any reminder of the traumatic event: a sound, sight, smell, physical sensation, words, or images. When a person is triggered, they’re being provoked by a stimulus that awakens or worsens the symptoms of a traumatic event or mental health condition (Gerdes, 2019). A person’s strong reaction to being triggered may come as a surprise to others because the response seems out of proportion to the stimulus, because the triggered individual is mentally reliving the original trauma. Some neurodivergencies can adapt these responses. For example, people with autism spectrum disorder (ASD) may experience stronger emotional reactions and may present this in ways which are unfamiliar or surprising to those who have not experienced the same challenges (Fuld, 2018).

Apart from triggering memories, the topics of right and wrong may be emotive. Young people are often passionate in their beliefs and may be moved to strong responses. There is nothing wrong with that, unless one person’s strong response makes another’s participation and expression less likely.

Ethics is only salient if the topics are tough:

Ethics concerns questions of moral value, of right and wrong, and relates to our deep-held beliefs and emotions. If any experience in an engineer’s education is likely to cause unpleasant memories to surface, or to stimulate strong discussion, it’s likely to be Ethics, and some of our students may have an emotional response to the topics of discussion and their impacts. This might be enough to make many educators shy away from integrating ethics.

However, research has shown that most engineers are moved by their personal sense of moral value, rather than by abstract external standards, and this can create very powerful and impactful learning experiences (Génova and González, 2016). To teach Ethics, we need to be willing to engage emotionally. Students also appreciate when educators can be vulnerable in the same way that we ask them to be, which means being willing to be honest about our own reactions to tough topics.

Approaches to tackling tough topics:

a. Prepare by reviewing resources

Several resources exist to guide educators who are engaging with tough topics in the classroom. Teaching and learning specialists recognise the challenges inherent in engaging with this kind of activity, yet also want to support educators who see the value in creating a space for students to wrestle with the difficult questions that they will encounter in the future. Many centres of teaching and learning at universities provide strategies and guidance through websites or pamphlets that are easily found by searching online. We include a list of some of our preferred resources below.

b. Prepare by finding local support

Even though we will avoid obvious triggers, there’s always the possibility that our students may become upset. We should be prepared by promoting the contact details for local support services within the institution. It can never be a bad thing for our students to know about these.

c. Give warnings and ask for consent

You might want to warn your students that discussing ethical matters is not without emotional consequence. At your discretion, seek their explicit consent to continue. There has been some criticism of this approach in the media, as some authors suggest that this infantilises the audience. Indeed, the pros and cons of trigger warnings might make an interesting topic for discussion: life can be cruel, is there value in developing a thick skin? What do we lose in this process? Being honest about your own hesitations and internal conflicts might encourage students to open up about how they wrestle with their own dilemmas. To be fully supportive, consider an advanced warning with the option to opt-out so that people aren’t stampeded into something they might prefer to avoid.

d. Recognise discomfort, and respond

Be aware of the possibility that individuals in your group could become upset. Be prepared to quietly offer time out or to change the activity in response to where the students want to take the discussion. Again, being transparent with the students that some people may be uncomfortable or upset by topics can reveal another relevant ethical topic – how to be respectful of others whose response differs from your own. And being willing to change the activity demonstrates the flexibility and adaptability required of 21st century engineers!

e. Avoid unnecessary risk

Some topics are best avoided due to the strength of emotion which they might trigger in students whose life story may be unknown to us. These topics include sexual abuse, self-harm, violence, eating disorders, homophobia, transphobia, racism, child abuse and paedophilia, and rape.

Be kind, and be brave:

Above all, let your students know that you care for their well-being. If we are to teach Ethics, let us be ethical. You might need to overcome some awkward moments with your students, but you will all learn and grow in the process!

References:

Fuld S. (2018) ‘Autism spectrum disorder: The Impact of stressful and traumatic life events and implications for clinical practice.’ Clinical Social Work Journal 46(3), pp. 210-219.

Génova, G., and González, M.R. (2016) ‘Teaching ethics to engineers: A Socratic experience,’ Science and Engineering Ethics 22, pp. 567–580.

Gerdes, K. (2019) ‘Trauma, trigger warnings, and the rhetoric of sensitivity,’ Rhetoric Society Quarterly, 49(1), pp. 3-24.

Kirk S. A. and Flammia, M. (2016) ‘Teaching the ethics of intercultural communication,’ in Teaching and Training for Global Engineering: Perspectives on Culture and Professional Communication Practices, pp.91-124.

Additional resources:

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Case enhancement: Developing a school chatbot for student support services

Activity: Stakeholder mapping to elicit value assumptions and motivations.

Author: Karin Rudolph (Collective Intelligence).

Overview:

This enhancement is for an activity found in point 5 of the Summary section of the case study.

What is stakeholder mapping?

Stakeholder mapping is the visual process of laying out the people, also called stakeholders, who are involved with or affected by an issue and can influence a project, product or idea.

What is a stakeholder?

Stakeholders are people, groups or individuals who have the power to affect or be affected by a process, project or product.

Mapping out stakeholders will help you to:

Identify the stakeholders you need to collaborate with to ensure the success of the project.
Understand the different perspectives and points of view people have and how these experiences can have an impact on your project or product.
Map out a wide range of people, groups or individuals that can affect and be affected by the project.

Stakeholder mapping:

The stakeholder mapping activity is a group exercise that provides students with the opportunity to discuss ethical and societal issues related to the School Chatbot case study. We recommend doing this activity in small groups of 6-8 students per table.

Resources:

Materials:

To carry out this activity, you will need the following resources:

1. Sticky notes (or digital notes if online).

2. A big piece of paper or digital board (Jamboard, Miro if online) divided into four categories:

- Manage closely.
- Keep satisfied.
- Keep informed.
- Monitor.

3. Markers and pencils.

The activity:

Begin the activity by asking the students to write a list of stakeholders involved in or affected by the school chatbot.
Once the list of stakeholders is ready, ask the students to allocate each stakeholder according to one of the four categories.

Board One

List of stakeholders:

Below is a list of the stakeholders involved in the Chatbot project. Put each stakeholder on a sticky note and add them to the stakeholders map, according to their level of influence and interest in the projects.

Top tip: use a different colour for each set of stakeholders.

School Chatbot – List of Stakeholders:

- CEO
- Developer
- Student placement
- Academy principal
- Academy board
- Academy students
- Academy teachers
- Teaching assistants
- Parents
- Funding body (Local Authority)
- School administrators
- School counsellors
- Career advisors
- Pastoral leads/wellbeing officers
- Special needs (SEND) workers
- GPs local practice
- Media/local press
- External staff contractors
- Competitors (companies offering similar services).

Placement:

Place the sticky notes with the stakeholders you choose from the list and add them to the stakeholders’ map.
Map out your stakeholders on the grid to classify them by both their influence and interest in the project.
The position of the stakeholder on the grid shows you the actions that you could take with their engagement.

Guidance:

Each quadrant represents the following:

Category: Manage closely. These stakeholders have high power and are highly interested. Their level of interest is an opportunity to maximise the benefit of the project.
Category: Keep satisfied. These stakeholders have high power, but they are not very involved in the project.
Category: Keep informed. These stakeholders have low power but high interest. They are supporters of the project and can be helpful. Keep them involved.
Category: Monitor. These stakeholders have low power and low interest in the project. They only require monitoring.

Board One

Motivations, assumptions, ethical and societal risks:

Materials:

1. A big piece of paper or digital board (Jamboard, Miro if online) divided into four categories:

- Stakeholders.
- Motivations.
- Assumptions.
- Ethical/societal risks.

2. Sticky notes (or digital notes if online).

3. Markers and pencils.

The activity:

Using the sticky notes from the previous stakeholders activity, ask students to discuss some of the scenarios and situations that can arise as a result of using the School Chatbot.
Ask students to write these scenarios down and add them to Board 2, according to each category: stakeholders, motivations, assumptions, and ethical and societal risks.
Discuss some issues this project can bring and ask students to write them on sticky notes according to each category.

Board Two

The Board Two activity can be done in two different ways:

Option 1:

You can use some guiding questions to direct the discussion. For example:

What role does stakeholder X play in this project?
What is their main motivation?
Can you think of any obstacle or conflict this stakeholder could face?
What is their assumption about the use of the chatbot? (Positive or negative.)
What are the potential ethical/ societal risks of using a chatbot in this context?

Option 2:

We have already written some assumptions, motivations and ethical/societal risks and you can add these as notes on a table and ask students to place according to each category: stakeholders, motivations, assumptions, and ethical and societal risks.

Motivations:

To develop a new revenue stream.
To help in developing the product.
The chatbot could present potential risks to students.
To obtain commercial gain.
To help students get access to online support 24/7.

Assumptions:

The product will help the company and the students. Any criticism shouldn’t be considered.
We can use data from students without any constraints.
The chatbot will improve the performance of students.
The institution will take precautions to prevent harm to students.
Criticism won’t be well received and it can result in losing my job.

Potential ethical and societal risks:

Lack of due diligence. Not enough understanding of the steps required to process data in an ethical manner.
Privacy risks. Data subjects (students) didn’t consent to their data being used in this context.
Risk of manipulation. Lack of engagement with the potential users can result in a lack of understanding of their needs and expectations.
Risk of surveillance. Students could be under constant monitoring, resulting in lack of freedom.
Risk of profiling. Use of sensitive historical and current data can reinforce biases and existing inequalities.
Conflict of interest.
Lack of psychological safety. Employees don’t feel empowered to express their views.
Lack of transparency/explainability. Potential lack of processes in place to explain how the algorithm works and makes decisions.
Using a human name can give the false impression of friendship, which can be used to manipulate users.
Over reliance on digital tools. This can undermine support provided by health care professionals and career advisors.
Using a gendered name (Alice) can reinforce negative stereotypes (for example, females as assistants).
Change of expectations. Constant access to information and resources can change the expectations teachers have about the student’s behaviours.
Lack of human agency/control. Students could modify their behaviours due to constant monitoring, resulting in a lack of agency and control over their environment.
Lack of involvement in the consultation that might result in poor advice/content.
Job instability and potential job losses.
Risk of disempowerment. Parents can be left out without access to information about their children’s wellbeing and performance.
Lack of diversity/consideration of a variety of students – for example students with special needs, neurodivergent students – that can result in standardised or harmful advice.

Move and match:

In the board below, you will find sticky notes with a list of stakeholders, motivations, assumptions, and ethical and societal risks.
Move and organise the sticky notes to match each category.
Discuss your options with the rest of the team and add new ethical and societal risks as you go.

Reflection:

Ask students to choose 2- 4 sticky notes and explain why they think these are important ethical/societal risks.

Potential future activity:

A more advanced activity could involve a group discussion where students are asked to think about some mitigation strategies to minimise these risks.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Authors: Professor Sarah Hitt SFHEA (NMITE); Professor Raffaella Ocone OBE FREng FRSE (Heriot Watt University); Johnny Rich (Engineering Professors’ Council); Dr Matthew Studley (University of the West of England, Bristol); Dr Nik Whitehead (University of Wales Trinity Saint David); Dr Darian Meacham (Maastricht University); Professor Mike Bramhall (TEDI-London); Isobel Grimley (Engineering Professors’ Council).

Topic: Data security of smart technologies.

Engineering disciplines: Electronics, Data, Mechatronics.

Ethical issues: Autonomy, Dignity, Privacy, Confidentiality.

Professional situations: Communication, Honesty, Transparency, Informed consent.

Educational level: Intermediate.

Educational aim: Practise ethical analysis. Ethical analysis is a process whereby ethical issues are defined and affected parties and consequences are identified so that relevant moral principles can be applied to a situation in order to determine possible courses of action.

Learning and teaching notes:

This case involves a software engineer who has discovered a potential data breach in a smart home community. The engineer must decide whether or not to report the breach, and then whether to alert and advise the residents. In doing so, considerations of the relevant legal, ethical, and professional responsibilities need to be weighed. The case also addresses communication in cases of uncertainty as well as macro-ethical concerns related to ubiquitous and interconnected digital technology.

The dilemma in this case is presented in two parts. If desired, a teacher can use Part one in isolation, but Part two develops and complicates the concepts presented in Part one to provide for additional learning. The case allows teachers the option to stop at multiple points for questions and/or activities as desired

Learners will have the opportunity to:

analyse the ethical dimensions of an engineering situation;
identify professional responsibilities of engineers in an ethical dilemma;
determine and defend a course of action in response to an ethical dilemma;
practise professional communication;
debate possible solutions to an ethical dilemma.

Teachers will have the opportunity to:

highlight professional codes of ethics and their relevance to engineering situations;
address approaches to resolve interpersonal and/or professional conflict;
integrate technical content on software and/or cybersecurity;
informally evaluate students’ critical thinking and communication skills.

Learning and teaching resources:

Summary:

Smart homes have been called “the road to independent living”. They have the potential to increase the autonomy and safety of older people and people with disabilities. In a smart home, the internet of things (IoT) is coupled with advanced sensors, chatbots and digital assistants. This combination enables residents to be connected with both family members and health and local services, so that if there there are problems, there can be a quick response.

Ferndale is a community of smart homes. It has been developed at considerable cost and investment as a pilot project to demonstrate the potential for better and more affordable care of older people and people with disabilities. The residents have a range of capabilities and all are over the age of 70. Most live alone in their home. Some residents are supported to live independently through: reminders to take their medication; prompts to complete health and fitness exercises; help completing online shopping orders and by detecting falls and trips throughout the house. The continuous assessment of habits, diet and routines allows the technology to build models that may help to predict any future negative health outcomes. These include detecting the onset of dementia or issues related to dietary deficiencies. The functionality of many smart home features depends on a reliable and secure internet connection.

Dilemma – Part one:

You are the software engineer responsible for the integrity of Ferndale’s system. During a routine inspection you discover several indicators suggesting a data breach may have occurred via some of the smart appliances, many of which have cameras and are voice-activated. Through the IoT, these appliances are also connected to Amazon Ring home security products – these ultimately link to Amazon, including supplying financial information and details about purchases.

Optional STOP for questions and activities:

1. Activity: Technical analysis – Before the ethical questions can be considered, the students might consider a number of immediate technical questions that will help inform the discussion on ethical issues. A sample data set or similar technical problem could be used for this analysis. For example:

- Is it possible to ascertain whether a breach has actually happened and data has been accessed?
- What data may have been compromised?
- Is a breach of this kind preventable, and could it be better prevented in the future?
- Has the security been subject to a hack or is the data not secure?
- Has the problem now been rectified, and all data secured?

2. Activity: Identify legal and ethical issues. The students should reflect on what might be the immediate ethical concerns of this situation. This could be done in small groups or a larger classroom discussion.

Possible prompts:

- Is there a risk that the breach comprised the residents’ personal details, financial information or even allowed remote and secret control of cameras? What else could have been compromised and what are the risks of these compromises? Are certain types of data more risky when breached than others? Why?
- What are the legal implications if there has been a breach? Do you, as a software engineer, have any duty to the residents at this point?
- At the stage where the breach and its potential implications are unknown, should you tell the community and, if so, what should you say? Some residents aren’t always able to understand the technology or how it works, so they may be unlikely to recognise the implications of situations like this. Should you worry that it might cause them distress or create distrust in the integrity of the whole system if the possible data breach is revealed?
- At the stage where the breach and its potential implications are unknown, is there anyone else you should inform? What should you tell them? Are there any risks you may be able to mitigate immediately? How?
- Who owns the data collected on a person living in a smart home? What should happen to it after that person dies?

3. Activity: Determine the wider ethical context. Students should consider what wider moral issues are raised by this situation. This could be done in small groups or a larger classroom discussion.

Possible prompts:

- When engineered products or systems go wrong, what is our responsibility to tell the people affected?
- What is our right to privacy? Can, or should, it be traded away or sacrificed for another good? Who gets to decide?
- Are smart homes a good thing if their technology is always going to present privacy risks? Should the technology be limited in some way?
- The homes in this case are inhabited by senior citizens with disabilities. Do we owe a different level of care to these people than others? Why? Should engineers working on software for these homes employ a duty of care in a different way than they would in software for homes for young able-bodied professionals? Why? Should a duty of care be delivered by people who have the capacity to care in the emotional sense?
- Should individuals have the ability to determine their own level of risk and choose what functionality to accept based on this risk? Should technology enable these kinds of choices?
- Should engineers be held responsible for unsafe systems? If not, who is responsible?

Dilemma – Part two:

You send an email to Ferndale’s manager about the potential breach, emphasising that the implications are possibly quite serious. She replies immediately, asking that you do not reveal anything to anyone until you are absolutely certain about what has happened. You email back that it may take some time to determine if the software security has been compromised and if so, what the extent of the breach has been. She replies explaining that she doesn’t want to cause a panic if there is nothing to actually worry about and says “What you don’t know won’t hurt you.” How do you respond?

Optional STOP for questions and activities:

1. Discussion: Professional values – What guidance is given by codes of ethics such as the Royal Academy of Engineering/Engineering Council’s Statement of Ethical Principles or the Association for Computing Machinery Code of Ethics?

2. Activity: Map possible courses of action. The students should think about the possible actions they might take. They can be prompted to articulate different approaches that could be adopted, such as the following, but also develop their own alternative responses.

- Do nothing. Tell no one. Try to improve the security to avoid future breaches.
- Shut down the smart home technology until any, and all, risks can be mitigated.
- Explain the situation fully to the residents, detailing subsequent risks for the future and steps they should take to mitigate the risks themselves.
- Offer a partial explanation of the situation, the solutions proposed (or carried out) and reassure them that everything is in order.

3. Activity: Hold a debate on which is the best approach and why. The students should interrogate the pros and cons of each possible course of action including the ethical, technical, and financial implications. They should decide on their own preferred course of action and explain why the balance of pros and cons is preferable to other options.

4. Activity: Role-play a conversation between the engineer and the manager, or a conversation between the engineer and a resident.

5. Discussion: consider the following questions:

- What is the role of robotics and artificial intelligence in caring for people in the future?
- Is there a limit to what data should be shared and is it justified to use other people’s data for profit?
- Could people like Ferndale’s residents be exploited through access to their data? How?
- What more could be achieved through the use of data and connectivity to care for older or ill people, in their homes or hospitals, and what additional safeguards should be put in place?

6. Activity: Change perspectives. Imagine that you are the child of one of Ferndale’s residents and that you get word of the potential data security breach. What would you hope the managers and engineers would do?

7. Activity: Write a proposal on how the system might be improved to stop this happening in the future or to mitigate unavoidable risks. To inform the proposal, the students should also explore the guidance of what might be best practice in this area. For example, in this instance, they may decide on a series of steps.

- Use human care providers to inform and explain to residents (or their families) about digital security.
- Deploy a more rigorous security protocol as well as a programme of regular testing and updates to minimise the risk of the situation occurring again.
- Shut down systems where the risks outweigh the potential benefits.
- Instigate a reporting procedure and a chain of command for decision-making in the future.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Authors: Dr Nicola Whitehead (University of Wales Trinity Saint David); Professor Sarah Hitt (NMITE); Emma Crichton (Engineers Without Borders UK); Dr Sarah Junaid (Aston University); Professor Mike Sutcliffe (TEDI-London), Isobel Grimley (Engineering Professors’ Council).

Topic: Development and use of a facial recognition system.

Engineering disciplines: Data, Electronics, Computer science, AI.

Ethical issues: Diversity, Bias, Privacy, Transparency.

Professional situations: Rigour, Informed consent, Misuse of data, Conflicts with leadership / management.

Educational level: Advanced.

Educational aim: To encourage ethical motivation. Ethical motivation occurs when a person is moved by a moral judgement, or when a moral judgement is a spur to a course of action.

Learning and teaching notes:

This case involves an engineer hired to manage the development and installation of a facial recognition project at a building used by university students, businesses and the public. It incorporates a variety of components including law and policy, stakeholder and risk analysis, and both macro- and micro-ethical elements. This example is UK-based: however, the instructor can adapt the content to better fit the laws and regulations surrounding facial recognition technology in other countries, if this would be beneficial.

This case study addresses two of AHEP 4’s themes: The Engineer and Society (acknowledging that engineering activity can have a significant societal impact) and Engineering Practice (the practical application of engineering concepts, tools and professional skills). To map this study to AHEP outcomes specific to a programme under these themes, access AHEP4 here and navigate to pages 30-31 and 35-37.

This case is presented in three parts. If desired, a teacher can use Part one in isolation, but Part two (focusing on the wider ethical context of the case) and Part three (focusing on the potential actions the engineer could take)develop and complicate the concepts presented in Part one to provide for additional learning. The case study allows teachers the option to stop at multiple points for questions and / or activities as desired.

Learners have the opportunity to:

apply their ethical judgement to a case study relating to privacy and consent;
judge the societal impact of a technical solution to a complex problem;
make and justify an ethical decision;
analyse risks associated with micro-ethical and macro-ethical concerns;
communicate these risks and judgements to both technical and non-technical audiences.

Teachers have the opportunity to:

highlight a range of ethical considerations within the scope of a complex engineering project;
introduce methods for risk analysis and ethical decision-making;
evaluate critical thinking, argumentation, and communication skills;
provide an opportunity for reflection.

Learning and teaching resources:

Summary:

Metropolitan Technical University (MTU), based in the UK, has an urban campus and many of its buildings are located in the city centre. A new student housing development in this area will be shared by MTU, a local college, and medical residents doing short rotations at the local hospital. The building has a public café on the ground floor and a couple of classrooms used by the university.

The housing development sits alongside a common route for parades and protests. In the wake of demonstrations by Extinction Rebellion and Black Lives Matter, students have raised concerns to the property manager about safety. Despite an existing system of CCTV cameras and swipe cards, the university decides to install an enhanced security system, built around facial recognition technology that would enable access to the building and cross-reference with crime databases. To comply with GDPR, building residents will be required to give explicit consent before the system is implemented. Visitors without a student ID (such as café customers) will be buzzed in, but their image will be captured and cross-referenced before entry. A side benefit of the system is that MTU’s department of Artificial Intelligence Research will help with the installation and maintenance, as well as studying how it works, in order to make improvements.

Dilemma – Part one:

You are an engineer who has been hired by MTU to take charge of the facial recognition system installation project, including setting policies and getting the system operational. With your background in AI engineering, you are expected to act as a technical advisor to MTU and liaise with the Facilities, Security and Computing departments to ensure a smooth deployment. This is the first time you have worked on a project that involves image capture. So as part of your preparation for the project, you need to do some preliminary research as to what best practices, guidance, and regulations apply.

Optional STOP for questions and activities:

1. Discussion: What are the legal issues relating to image capture? Images allow for the identification of living persons and are therefore considered as personal data under GDPR and the Data Protection Act (2018).

2. Discussion: Sharing data is a legally and ethically complex field. Is it appropriate to share images captured with the police? If not the police, then whose crime database will you use? Is it acceptable to share the data with the Artificial Intelligence Research group? Why, or why not?

3. Discussion: Under GDPR, individuals must normally consent to their personal data being processed. How should consent be handled in this case?

4. Discussion: Does the fact that the building will accommodate students from three different institutions (MTU, the local college, and the hospital) complicate these issues? Are regulations related to students’ captured images different than those related to public image capture?

5. Activity: Undertake a technical activity that relates to how facial recognition systems are engineered.

Dilemma – Part two:

The project has kicked off, and one of its deliverables is to establish the policies and safeguards that will govern the system. You convened a meeting of project stakeholders to determine what rules need to be built into the system’s software and presented a list of questions to help you make technical decisions. The questions you asked were:

Should students be able to opt in or out of image capture?
Should visitors be told that their image will be captured?
What happens if a student living in the housing development decides that they no longer wish to take part in the image recognition project?

What you had thought would be a quick meeting to agree basic principles turned out to be very lengthy and complex. You were surprised at the variety of perspectives and how heated the discussions became. The discussions raised some questions in your own mind as to the risks of the facial recognition system.

Optional STOP for questions and activities:

The following activities focus on macro-ethics. This seeks to understand the wider ethical contexts of projects like the facial recognition system.

1. Activity: Stakeholder mapping – Who are all the stakeholders and what might their positions and perspectives be? Is there a difference between the priorities of the different stakeholders?

2. Activity: There are many different values competing for priority here. Identify these values, discuss and debate how they should be weighed in the context of the project.

3. Activity: Risks can be understood as objective and / or subjective. Research the difference between these two types of risk, and identify which type(s) of risks exist related to the project.

4. Discussion: Which groups or individuals are potentially harmed by the technology and which potentially benefit? How should we go about setting priorities when there are competing harms and benefits?

5. Discussion: Does the technology used treat everyone from your stakeholders’ list equally? Should the needs of society as a whole outweigh the needs of the individual?

6. Activity: Make and defend an argument as to the appropriateness of installing and using the system.

7. Discussion: What responsibilities do engineers have in developing these technologies?

Dilemma – Part three:

A few days later, you were forwarded a screenshot of a social media post that heavily criticised the proposed facial recognition system. It was unclear where the post had originated, but it had clearly been shared and promoted among both students and the public raising concerns about privacy and transparency. Your boss believes this outcry endangers the project and has requested that you make a public statement on behalf of MTU, reaffirming its commitment to installing the system.

You share the concerns, but have been employed to complete the project. You understand that suggesting it should be abandoned, would most likely risk your job. What will you tell your boss? How will you prepare your public statement?

Optional STOP for questions and activities:

Micro-ethics concerns individuals and their responses to specific situations. The following steps are intended to help students develop their ability to practise moral analysis by considering the problem in a structured way and work towards possible solutions that they can analyse critically.

1. Discussion: What are the problems here?

- You are an employee of MTU and have a responsibility to be a representative for its interests. However, you can see that the university’s actions create significant problems relating to privacy and consent and may be ethically or legally questionable.

2. Discussion: What are the possible courses of action you can take as an employee?

Students can be prompted to consider what different approaches they might adopt, such as the following, but can also develop their own possible responses.

- You could take the university line and refuse to consider any compromise. After all, you have a duty of care towards the students.
- You could act as a whistleblower and contact the Information Commissioner’s Office,or the press, with the university’s plans.
- You could look for changes in the hardware setup for the system. Can the cameras be placed so that they only capture people coming into the building without recording anyone else?
- You could look for changes in the software setup for the system. What level of accuracy is needed to declare a match between the image and the reference image before the doors will open?
- You could look to make changes in the data management processes. How long will the data be stored? Which database(s) will images be checked against? What are the data security implications of implementing this system?
- Are there other alternatives available to you?

3. Discussion: Which is the best approach and why? – Interrogate the pros and cons of each possible course of action including the ethical, practical, cost, local relationship and the reputational damage implications. Students should decide on their own preferred course of action and explain why the balance of pros and cons is preferable to other options. The students may wish to consider this from other perspectives, such as:

- What would the best outcome be if cost was no object?
- What course of action would be taken if different perspectives were chosen as the priority. For example, if the personal privacy perspective was the main priority, what action would be taken, compared with action taken if the cost to the university were the main priority?
- What are the wider implications of the use of image recognition in public spaces and how can these be mitigated?
- Are there any other technologies that would solve the security problem without the ethical implications?
- What are the possible solutions open to you?
- Are there any short-term solutions versus longer-term solutions?

4. Activity: Public Communication – Students can practise writing a press release, giving an interview, or making a public statement about the case and the decision that they make.

5. Activity: Reflection – Students can reflect on how this case study has enabled them to see the situation from different angles. Has it motivated them to understand the ethical concerns and to come to an acceptable conclusion.

Enhancements:

An enhancement for this case study can be found here.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Authors: Professor Sarah Hitt SFHEA (NMITE); Professor Raffaella Ocone OBE FREng FRSE (Heriot Watt University); Professor Thomas Lennerfors (Uppsala University); Claire Donovan (Royal Academy of Engineering); Isobel Grimley (Engineering Professors’ Council).

Topic: Developing customised algorithms for student support.

Engineering disciplines: Computing, AI, Data.

Ethical issues: Bias, Social responsibility, Risk, Privacy.

Professional situations: Informed consent, Public health and safety, Conflicts with leadership / management, Legal implications.

Educational level: Beginner.

Educational aim: Develop ethical sensitivity. Ethical sensitivity is the broad cognisance of ethical issues and the ability to see how these might affect others.

Learning and teaching notes:

This case study involves the employees of a small software start-up that is creating a customised student support chatbot for a Sixth Form college. The employees come from different backgrounds and have different perspectives on the motivations behind their work, which leads to some interpersonal conflict. The team must also identify the ethical issues and competing values that arise in the course of developing their algorithm.

The dilemma in this case is presented in two parts which build in complexity and navigate between personal, professional, and societal contexts. If desired, a teacher can use Part one in isolation, but Part two develops and complicates the concepts presented in Part one to provide for additional learning. Pre-reading ‘Ethics of Care and Justice’ is recommended, though not required, for engaging with Part two. The case allows teachers the option to stop at multiple points for questions and / or activities as desired.

Learners have the opportunity to:

identify ethical and legal issues related to emerging technologies;
apply codes of ethics to an engineering ethics dilemma;
consider different perspectives on an ethical issue and what values inform those perspectives;
practise professional communication related to ethical dilemmas.

Teachers have the opportunity to:

introduce ethics of care and ethics of justice;
integrate technical content for developing software and algorithms;
highlight strategies to deal with conflicts between management, clients, and employees;
explore wider contexts and implications of engineering technologies;
informally evaluate students’ critical thinking and communication skills.

Learning and teaching resources:

Summary:

Exaba is a small, three-person software startup. Like all small businesses, it has been struggling with finances during the pandemic. The company began selling its services across a variety of industry sectors but is now trying to expand by developing software solutions for the growing education technology sector.

Ivan, Exaba’s founder and CEO, was thrilled to be contracted by a growing local Sixth Form College in North West England, NorthStar Academy, to create a chatbot that will optimise student support services. These services include ensuring student safety and wellbeing, study skills advice, careers guidance, counselling, and the identification for the need and implementation of extra learning support. It is such a large project that Ivan has been able to bring in Yusuf, a university student on placement from a computer systems programme, to help Nadja, Exaba’s only full-time software engineer. Ivan views the chatbot contract as not only a financial windfall that can help get the company back on track, but as the first project in a new product-development revenue stream.

Nadja and Yusuf have been working closely with the NorthStar Academy’s Principal, Nicola, to create ‘Alice’: the custom student-support chatbot to ensure that she is designed appropriately and is fit for purpose. Nicola has seen growing evidence that chatbots can identify when students are struggling with a range of issues from attendance to anxiety. She has also seen that they can be useful in helping administrators understand what students need, how to help them more quickly, and where to invest more resources to make support most effective.

Optional STOP for questions and activities:

1. Discussion: What moral or ethical issues might be at stake or arise in the course of this project?

2. Discussion: What professional or legal standards might apply to the development of Alice?

3. Discussion: What design choices might Nadja and Yusuf have to consider as they build the chatbot software in order for it to conform to those standards?

4. Discussion: is there anything risky about giving cognitive chatbots human names in general, or a female name specifically?

5. Activity: Undertake stakeholder mapping to elicit value assumptions and motivations.

6. Activity: Research any codes of ethics that might apply to AI in education, or policies / laws that apply to controlling and processing student data.

7. Activity: View the following TED talk and have a discussion on gender in digital assistants: Siri and Alexa are AI Built for the Past by Emily Liu.

Dilemma – Part one:

After undertaking work to ensure GDPR compliance through transparency, consent, and anonymisation of the data harvested by interactions with Alice, Nadja and Yusuf are now working on building the initial data set that the chatbot will call upon to provide student support. The chatbot’s information to students can only be as good as the existing data it has available to draw from. To enable this, Nicola has agreed to provide Exaba with NorthStar Academy’s existing student databases that span many years and cover both past and present students. While this data – including demographics, academic performances, and interactions with support services – is anonymised, Yusuf has begun to feel uncomfortable. One day, when the entire team was together discussing technical challenges, Yusuf said “I wonder what previous students would think if they found out that we were using all this information about them, without their permission?”

Ivan pointed out, “Nicola told us it was okay to use. They’re the data controllers, so it’s their responsibility to resolve that concern, not ours. We can’t tell them what to do with their own data. All we need to be worried about is making sure the data processing is done appropriately.”

Nadja added, “Plus, if we don’t use an existing data set, Alice will have to learn from scratch, meaning she won’t be as effective at the start. Wouldn’t it be better for our chatbot to be as intelligent and helpful as possible right away? Otherwise, she could put existing students at a disadvantage.”

Yusuf fell silent, figuring that he didn’t know as much as Ivan and Nadja. Since he was just on a placement, he felt that it wasn’t his place to push the issue any further with full-time staff.

Optional STOP for questions and activities:

1. Discussion: Expand upon Yusuf’s feelings of discomfort. What values or principles is this emotion drawing on?

2. Discussion: Do you agree with Yusuf’s perspective, or with Ivan’s and Nadja’s? Why?

3. Discussion: Does / should Yusuf have the right to voice any concerns or objections to his employer?

4. Discussion: Do / should previous NorthStar students have the right to control what the academy does with their data? To what extent, and for how long?

5. Discussion: Is there / should there be a difference between how data about children is used and that of adults? Why?

6. Discussion: Should a business, like Exaba, ever challenge its client, like NorthStar Academy, about taking potentially unethical actions?

7. Technical activity: Undertake a technical activity such as creating a process flow diagram, pieces of code and UI / UX design that either obscure or reinforce consent.

8. Activity: Undertake argument mapping to diagram and expand on the reasoning and evidence used by Yusuf, Nadja, and Ivan in their arguments.

9. Activity: Apply ethical theories to those arguments.

10. Discussion: What ethical principles are at stake? Are there potentially any conflicts or contradictions arising from those principles?

Dilemma – Part two:

Nicola, too, was under pressure. The academy’s Board had hired her as Principal to improve NorthStar’s rankings in the school performance table, to get the college’s finances back on track, and support the government efforts at ‘levelling up’ This is why one of Nicola’s main specifications for Alice is that she be able to flag students at risk of not completing their qualifications. Exaba will have to develop an algorithm that can determine what those risk factors are.

In a brainstorming session Nadja began listing some ideas on the whiteboard. “Ethnic background, family income, low marks, students who fit that profile from the past and ultimately dropped out, students who engaged with support services a lot, students with health conditions . . .”

“Wait, wait, wait,” Yusuf said. “This feels a little bit like profiling to me. You know, like we think kids from certain neighbourhoods are unlikely to succeed so we’re building this thing to almost reinforce that they don’t.”

“The opposite is true!” Ivan exclaimed. “This algorithm will HELP exactly those students.”

“I can see how that’s the intention,” Yusuf acknowledged. “But I’ve had so many friends and neighbours experience well-intentioned but not appropriate advice from mentors and counsellors who think the only solution is for everyone to complete qualifications and go to university. This is not the best path for everybody!”

Nadja had been listening carefully. “There is something to what Yusuf is saying: Is it right to nudge students to stay in a programme that’s actually not a best fit for them? Could Alice potentially give guidance that is contrary to what a personal tutor, who knows the student personally, might advise? I don’t know if that’s the sort of algorithm we should develop.”

At this point Ivan got really frustrated with his employees: “This is the proprietary algorithm that’s going to save this company!” he shouted. “Never mind the rights and wrongs of it. Think of the business potential, not to mention all the schools and students this is going to help. The last thing I need is a mutiny from my team. We have the client’s needs to think about, and that’s it.”

Optional STOP for questions and activities:

1. Activity: compare an approach to this case through the ethics of care versus the ethics of justice. What different factors come into play? How should these be weighed? Might one approach lead to a better course of action than another? Why?

2. Discussion: what technical solutions, if any, could help mitigate Yusuf and Nadja’s concerns?

3. Activity: imagine that Ivan agrees that this is a serious enough concern that they need to address it with Nicola. Role play a conversation between Ivan and Nicola.

4. Activity: undertake a classroom debate on whether or not Alice has the potential to reinforce negative stereotypes. Variations include alley debate, stand where you stand, adopt and support opposite instinct.

Enhancements:

An enhancement for this case study can be found here.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Let us know what you think of our website