This paper argues against the view that trolley cases are of little or no relevance to the ethics of automated vehicles. Four arguments for this view are outlined and rejected: the Not Going to Happen Argument, the Moral Difference Argument, the Impossible Deliberation Argument and the Wrong Question Argument. In making clear where these arguments go wrong, a positive account is developed of how trolley cases can inform the ethics of automated vehicles.

Following societal and policy pressures for responsible innovation, innovators are more and more expected to consider the broader socio-ethical context of their work, and more importantly, to integrate such considerations into their daily practices. This may require the involvement of ‘outsiders’ in innovation trajectories, including e.g. societal and governmental actors. However, methods on how to functionally organize such integration in light of responsible innovation have only recently started to emerge. We present an approach to do just that, in which we (...) first develop value profiles of the involved actors, and second, design a workshop setting that allows innovators to develop design requirements in collaboration with representatives of parties that are not usually involved in such innovation design practices. Using a case study in automated vehicle development, we positively demonstrate the possibility and utility of our approach. We stress that in this study we wish to demonstrate the functionality of our developed method, and did not search for scientifically valid outcomes regarding this technical field. (shrink)

This article considers two related and fundamental issues about morality in a virtual world. The first is whether the anonymity that is a feature of virtual worlds can shed light upon whether people are moral when they can act with impunity. The second issue is whether there are any moral obligations in a virtual world and if so what they might be. -/- Our reasons for being good are fundamental to understanding what it is that makes us moral or indeed (...) whether any of us truly are moral. Plato grapples with this problem in book two of The Republic where Socrates is challenged by his brothers Adeimantus and Glaucon. They argue that people are moral only because of the costs to them of being immoral; the external constraints of morality. -/- Glaucon asks us to imagine a magical ring that enables its wearers to become invisible and capable of acting anonymously. The ring is in some respects analogous to the possibilities created by online virtual worlds such as Second Life, so the dialogue is our entry point into considering morality within these worlds. These worlds are three dimensional user created environments where people control avatars and live virtual lives. As well as being an important social phenomenon, virtual worlds and what people chose to do in them can shed light on what people will do when they can act without fear of normal sanction. -/- This paper begins by explaining the traditional challenge to morality posed by Plato, relating this to conduct in virtual worlds. Then the paper will consider the following skeptical objection. A precondition of all moral requirements is the ability to act. There are no moral requirements in virtual worlds because they are virtual and it is impossible to act in a virtual world. Because avatars do not have real bodies and the persons controlling avatars are not truly embodied, it is impossible for people to truly act in a virtual world. We will show that it is possible to perform some actions and suggest a number of moral requirements that might plausibly be thought to result. Because avatars cannot feel physical pain or pleasure these moral requirements are interestingly different from those of real life. Hume’s arguments for why we should be moral apply to virtual worlds and we conclude by considering how this explains why morality exists in these environments. (shrink)

The recent progress in the development of autonomous cars has seen ethical questions come to the forefront. In particular, life and death decisions regarding the behavior of self-driving cars in trolley dilemma situations are attracting widespread interest in the recent debate. In this essay we want to ask whether we should implement a mandatory ethics setting for the whole of society or, whether every driver should have the choice to select his own personal ethics setting. While the consensus view seems (...) to be that people would not be willing to use an automated car that might sacrifice themselves in a dilemma situation, we will defend the somewhat contra-intuitive claim that this would be nevertheless in their best interest. The reason is, simply put, that a PES regime would most likely result in a prisoner’s dilemma. (shrink)

Consider the multi-user virtual worlds of online games such as EVE and World of Warcraft, or the multi-user virtual world of Second Life. Suppose a player performs an action in one of these worlds, via his or her virtual character, which would be wrong, if the virtual world were real. What is the moral status of this virtual action? In this paper I consider arguments for and against the Asymmetry Thesis: the thesis that such virtual actions are never wrong. I (...) also explain how the truth of the Asymmetry Thesis is closely aligned with the possibility of what Edward Castronova has called closed synthetic worlds. With some qualifications, the ultimate conclusion is that the Asymmetry Thesis is false and that these closed worlds are impossible. (shrink)

One of the reasons why most of us feel puzzled about the problem of abortion is that we want, and do not want, to allow to the unborn child the rights that belong to adults and children. When we think of a baby about to be born it seems absurd to think that the next few minutes or even hours could make so radical a difference to its status; yet as we go back in the life of the fetus we (...) are more and more reluctant to say that this is a human being and must be treated as such. No doubt this is the deepest source of our dilemma, but it is not the only one. For we are also confused about the general question of what we may and may not do where the interests of human beings conflict. We have strong intuitions about certain cases; saying, for instance, that it is all right to raise the level of education in our country, though statistics allow us to predict that a rise in the suicide rate will follow, while it is not all right to kill the feeble-minded to aid cancer research. It is not easy, however, to see the principles involved, and one way of throwing light on the abortion issue will be by setting up parallels involving adults or children once born. So we will be able to isolate the “equal rights” issue and should be able to make some advance... (shrink)

Judith Jarvis Thomson; Killing, Letting Die, and The Trolley Problem, The Monist, Volume 59, Issue 2, 1 April 1976, Pages 204–217, https://doi.org/10.5840/monis.

Self-driving vehicles offer great potential to improve efficiency on roads, reduce traffic accidents, increase productivity, and minimise our environmental impact in the process. However, they have also seen resistance from different groups claiming that they are unsafe, pose a risk of being hacked, will threaten jobs, and increase environmental pollution from increased driving as a result of their convenience. In order to reap the benefits of SDVs, while avoiding some of the many pitfalls, it is important to effectively determine what (...) challenges we will face in the future and what steps need to be taken now to avoid them. The approach taken in this paper is the construction of a likely future, through the process of a policy scenario methodology, if we continue certain trajectories over the coming years. The purpose of this is to articulate issues we currently face and the construction of a foresight analysis of how these may develop in the next 6 years. It will highlight many of the key facilitators and inhibitors behind this change and the societal impacts caused as a result. This paper will synthesise the wide range of ethical, legal, social and economic impacts that may result from SDV use and implementation by 2025, such as issues of autonomy, privacy, liability, security, data protection, and safety. It will conclude with providing steps that we need to take to avoid these pitfalls, while ensuring we reap the benefits that SDVs bring. (shrink)

Ethical thought experiments such as the trolley dilemma have been investigated extensively in the past, showing that humans act in utilitarian ways, trying to cause as little overall damage as possible. These trolley dilemmas have gained renewed attention over the past few years, especially due to the necessity of implementing moral decisions in autonomous driving vehicles. We conducted a set of experiments in which participants experienced modified trolley dilemmas as drivers in virtual reality environments. Participants had to make decisions between (...) driving in one of two lanes where different obstacles came into view. Eventually, the participants had to decide which of the objects they would crash into. Obstacles included a variety of human-like avatars of different ages and group sizes. Furthermore, the influence of sidewalks as potential safe harbors and a condition implicating self-sacrifice were tested. Results showed that participants, in general, decided in a utilitarian manner, sparing the highest number of avatars possible with a limited influence by the other variables. Derived from these findings, which are in line with the utilitarian approach in moral decision making, it will be argued for an obligatory ethics setting implemented in ADVs. (shrink)

In the Trolley Case (Figure 1), as devised by Philippa Foot and modified by Judith Jarvis Thomson, a runaway trolley (i.e. tram) is headed down a main track and will hit and kill five unless you divert it onto a side track, where it will hit and kill one.

Who should decide how a machine will decide what to do when it is driving a car, performing a medical procedure, or, more generally, when it is facing any kind of morally laden decision? More and more, machines are making complex decisions with a considerable level of autonomy. We should be much more preoccupied by this problem than we currently are. After a series of preliminary remarks, this paper will go over four possible answers to the question raised above. First, (...) we may claim that it is the maker of a machine that gets to decide how it will behave in morally laden scenarios. Second, we may claim that the users of a machine should decide. Third, that decision may have to be made collectively or, fourth, by other machines built for this special purpose. The paper argues that each of these approaches suffers from its own shortcomings, and it concludes by showing, among other things, which approaches should be emphasized for different types of machines, situations, and/or morally laden decisions. (shrink)

A number of companies including Google and BMW are currently working on the development of autonomous cars. But if fully autonomous cars are going to drive on our roads, it must be decided who is to be held responsible in case of accidents. This involves not only legal questions, but also moral ones. The first question discussed is whether we should try to design the tort liability for car manufacturers in a way that will help along the development and improvement (...) of autonomous vehicles. In particular, Patrick Lin’s concern that any security gain derived from the introduction of autonomous cars would constitute a trade-off in human lives will be addressed. The second question is whether it would be morally permissible to impose liability on the user based on a duty to pay attention to the road and traffic and to intervene when necessary to avoid accidents. Doubts about the moral legitimacy of such a scheme are based on the notion that it is a form of defamation if a person is held to blame for causing the death of another by his inattention if he never had a real chance to intervene. Therefore, the legitimacy of such an approach would depend on the user having an actual chance to do so. The last option discussed in this paper is a system in which a person using an autonomous vehicle has no duty of interfering, but is still held responsible for possible accidents. Two ways of doing so are discussed, but only one is judged morally feasible. (shrink)

This paper considers possible future legislation that requires the exclusive use of autonomous cars. The author develops and defends a ‘Liberal Argument Against Mandated Autonomous Cars’, which argues that such a law would be incompatible with classical liberalism, provided that the following condition holds: In the event where the car must ‘choose’ between running over a young person or an old person, or both, autonomous cars are programmed to respond by running over old people in order to save young people. (...) Such a law requiring the use of these cars, provided that these assumptions hold, would violate the important value in classical liberalism that all individuals ought to be treated equally before the law. The paper concludes by arguing that alternative ways of dealing with this problem come with their own set of unpalatable problems. (shrink)

Questions of what a self-driving car ought to do if it encounters a situation analogous to the ‘trolley problem’ have dominated recent discussion of the ethics of self-driving cars. This paper argues that this interest is misplaced. If a trolley-style dilemma situation actually occurs, given the limits on what information will be available to the car, the dynamics of braking and tyre traction determine that, irrespective of outcome, it is always least risky for the car to brake in a straight (...) line rather than swerve. (shrink)

The literature on self-driving cars and ethics continues to grow. Yet much of it focuses on ethical complexities emerging from an individual vehicle. That is an important but insufficient step towards determining how the technology will impact human lives and society more generally. What must complement ongoing discussions is a broader, system level of analysis that engages with the interactions and effects that these cars will have on one another and on the socio-technical systems in which they are embedded. To (...) bring the conversation of self-driving cars to the system level, we make use of two traffic scenarios which highlight some of the complexities that designers, policymakers, and others should consider related to the technology. We then describe three approaches that could be used to address such complexities and their associated shortcomings. We conclude by bringing attention to the “Moral Responsibility for Computing Artifacts: The Rules”, a framework that can provide insight into how to approach ethical issues related to self-driving cars. (shrink)

In this paper, I will argue that automated vehicles should not swerve to avoid a person or vehicle in its path, unless they can do so without imposing risks onto others. I will argue that this is the conclusion that we should reach even if we start by assuming that we should divert the trolley in the standard trolley case. In defence of this claim, I appeal to the distribution of moral and legal responsibilities, highlighting the importance of safe spaces, (...) and arguing in favour of constraints on what can be done to minimise casualties. My arguments draw on the methodology associated with the trolley problem. As such, this paper also defends this methodology, highlighting a number of ways in which authors misunderstand and misrepresent the trolley problem. For example, the ‘trolley problem’ is not the ‘name given by philosophers to classic examples of unavoidable crash scenarios, historically involving runaway trolleys’, as Millar suggests, and trolley cases should not be compared with ‘model building in the sciences’, as Gogoll and Müller suggest. Trolley cases have more in common with lab experiments than model building, and the problem referred to in the trolley problem is not the problem of deciding what to do in any one case. Rather, it refers to the problem of explaining what appear to be conflicting intuitions when we consider two cases together. The problem, for example, could be: how do we justify the claim that automated vehicles should not swerve even if we accept the claim that we should divert the trolley in an apparently similar trolley case? (shrink)

In this paper, we argue that, under a specific set of circumstances, designing and employing certain kinds of virtual reality (VR) experiences can be unethical. After a general discussion of simulations and their ethical context, we begin our argu-ment by distinguishing between the experiences generated by different media (text, film, computer game simulation, and VR simulation), and argue that VR experiences offer an unprecedented degree of what we call “perspectival fidelity” that prior modes of simulation lack. Additionally, we argue that (...) when VR experiences couple this perspectival fidelity with what we call “context realism,” VR experiences have the ability to produce “virtually real experiences.” We claim that virtually real experiences generate ethical issues for VR technologies that are unique to the medium. Because subjects of these experiences treat them as if they were real, a higher degree of ethical scrutiny should be applied to any VR scenario with the potential to generate virtually real experiences. To mitigate this unique moral hazard, we propose and defend what we call “The Equivalence Principle.” This principle states that “if it would be wrong to allow subjects to have a certain experience in reality, then it would be wrong to allow subjects to have that experience in a virtually real setting.” We argue that such a principle, although limited in scope, should be part of the risk analysis conducted by any Institutional Review Boards, psychologists, empirically oriented philosophers, or game designers who are using VR technology in their work. (shrink)

Autonomous vehicles —and accidents they are involved in—attest to the urgent need to consider the ethics of artificial intelligence. The question dominating the discussion so far has been whether we want AVs to behave in a ‘selfish’ or utilitarian manner. Rather than considering modeling self-driving cars on a single moral system like utilitarianism, one possible way to approach programming for AI would be to reflect recent work in neuroethics. The agent–deed–consequence model :3–20, 2014a, Behav Brain Sci 37:487–488, 2014b) provides a (...) promising descriptive and normative account while also lending itself well to implementation in AI. The ADC model explains moral judgments by breaking them down into positive or negative intuitive evaluations of the agent, deed, and consequence in any given situation. These intuitive evaluations combine to produce a positive or negative judgment of moral acceptability. For example, the overall judgment of moral acceptability in a situation in which someone committed a deed that is judged as negative would be mitigated if the agent had good intentions and the action had a good consequence. This explains the considerable flexibility and stability of human moral judgment that has yet to be replicated in AI. This paper examines the advantages and disadvantages of implementing the ADC model and how the model could inform future work on ethics of AI in general. (shrink)

Ethics settings allow for morally significant decisions made by humans to be programmed into autonomous machines, such as autonomous vehicles or autonomous weapons. Customizable ethics settings are a type of ethics setting in which the users of autonomous machines make such decisions. Here two arguments are provided in defence of customizable ethics settings. Firstly, by approaching ethics settings in the context of failure management, it is argued that customizable ethics settings are instrumentally and inherently valuable for building resilience into the (...) larger socio-technical systems in which autonomous machines operate. Secondly, after defining the preliminary condition of responsibility attribution and demonstrating how ethics settings enable humans to exert control over the outcomes of morally significant incidents, it is shown that ethics settings narrow the responsibility gap. (shrink)