The present paper aims at contributing to the ethical debate on the impacts of artificial intelligence (AI) systems on human autonomy. More specifically, it intends to offer a clearer understanding of the design challenges to the effort of aligning driving automation technologies to this ethical value. After introducing the discussion on the ambiguous impacts that AI systems exert on human autonomy, the analysis zooms in on how the problem has been discussed in the literature on connected and automated vehicles (CAVs). (...) On this basis, it is claimed that the issue has been mainly tackled on a fairly general level, and mostly with reference to the controversial issue of crash-optimization algorithms, so that only limited design insights have been drawn. However, integrating ethical analysis and design practices is critical to pursue the implementation of such an important ethical value into CAV technologies. To this aim, it is argued, a more applied approach targeted at examining the impacts on human autonomy of current CAV functions should also be explored. As an example of the intricacy of this task, the case of automated route planning is discussed in some detail. (shrink)

While autonomous vehicles (AVs) are not designed to harm people, harming people is an inevitable by-product of their operation. How are AVs to deal ethically with situations where harming people is inevitable? Rather than focus on the much-discussed question of what choices AVs should make, we can also ask the much less discussed question of who gets to decide what AVs should do in such cases. Here there are two key options: AVs with a personal ethics setting (PES) or an (...) “ethical knob” that end users can control or AVs with a mandatory ethics setting (MES) that end users cannot control. Which option, a PES or an MES, is best and why? This chapter argues, by drawing on the choice architecture literature, in favor of a hybrid view that requires mandated default choice settings while allowing for limited end user control. (shrink)

The potential development of self-driving cars (also known as autonomous vehicles or AVs – particularly Level 5 AVs) has called the attention of different interested parties. Yet, there are still only a few relevant international regulations on them, no emergency patterns accepted by communities and Original Equipment Manufacturers (OEMs), and no publicly accepted solutions to some of their pending ethical problems. Thus, this paper aims to provide some possible answers to these moral and practical dilemmas. In particular, we focus on (...) what AVs should do in no-win scenarios and on who should be held responsible for these types of decisions. A naturalistic perspective on ethics informs our proposal, which, we argue, could represent a pragmatic and realistic solution to the regulation of AVs. We discuss the proposals already set out in the current literature regarding both policy-making strategies and theoretical accounts. In fact, we consider and reject descriptive approaches to the problem as well as the option of using either a strict deontological view or a solely utilitarian one to set AVs’ ethical choices. Instead, to provide concrete answers to AVs’ ethical problems, we examine three hierarchical levels of decision-making processes: country-wide regulations, OEM policies, and buyers’ moral attitudes. By appropriately distributing ethical decisions and considering their practical implications, we maintain that our proposal based on ethical naturalism recognizes the importance of all stakeholders and allows the most able of them to take actions (the OEMs and buyers) to reflect on the moral leeway and weight of their options. (shrink)

One of the moral questions concerning autonomous vehicles (henceforth AVs) is the choice between types that differ in their built-in algorithms for dealing with rare situations of unavoidable lethal collision. It does not appear to be possible to avoid questions about how these algorithms should be designed. We present the results of our study of moral preferences (N = 2769) with respect to three types of AVs: (1) selfish, which protects the lives of passenger(s) over any number of bystanders; (2) (...) altruistic, which minimizes the number of casualties, even if this leads to death of passenger(s); and (3) conservative, which abstains from interfering in such situations. We differentiate between scenarios in which participants are to make their decisions privately or publicly, and for themselves or for their offspring. We aim to answer two research questions: (1) whether the public visibility of the choice of an AV type choice make this choice more altruistic and (2) which type of situation makes it more difficult to choose altruistically: when choosing for society as a whole, when choosing only for oneself, or when choosing only for one’s offspring. Our results show that respondents exhibit a preference for an altruistic strategy for AVs and that it is reinforced when signaled to others. The altruistic preference is strongest when applies to everybody else, weaker when it reflects a solely personal choice, and weakest when choosing for one’s own child. We conclude that a public choice is considerably more likely to pressure consumers into accepting a more socially beneficial solution. (shrink)

In this paper, we explore and describe what is needed to allow connected and automated vehicles to break traffic rules in order to minimise road safety risk and to operate with appropriate transparency. Reviewing current traffic rules with particular reference to two driving situations, we illustrate why current traffic rules are not suitable for CAVs and why making new traffic rules specifically for CAVs would be inappropriate. In defining an alternative approach to achieving safe CAV driving behaviours, we describe the (...) use of ethical goal functions as part of hybrid AI systems, suggesting that functions should be defined by governmental bodies with input from citizens and stakeholders. Ethical goal functions for CAVs would enable developers to optimise driving behaviours for safety under conditions of uncertainty whilst allowing for differentiation of products according to brand values. Such functions can differ between regions according to preferences for safety behaviours within that region and can be updated over time, responding to continual socio-technological feedback loops. We conclude that defining ethical goal functions is an urgent and necessary step from governmental bodies to enable the safe and transparent operation of CAVs and accelerate the reduction in road casualties they promise to achieve. (shrink)

The international debate on the ethics and legality of autonomous weapon systems (AWS) as well as the call for a ban are primarily focused on the nebulous concept of fully autonomous AWS. More specifically, on AWS that are capable of target selection and engagement without human supervision or control. This thesis argues that such a conception of autonomy is divorced both from military planning and decision-making operations as well as the design requirements that govern AWS engineering and subsequently the tracking (...) and tracing of moral responsibility. To do this, this thesis marries two different levels of meaningful human control (MHC), termed levels of abstraction, to couple military operations with design ethics. In doing so, this thesis argues that the contentious notion of ‘full’ autonomy is not problematic under this two-tiered understanding of MHC. It proceeds to propose the value sensitive design (VSD) approach as a means for designing for MHC. (shrink)

In 2017, the German ethics commission for automated and connected driving released 20 ethical guidelines for autonomous vehicles. It is now up to the research and industrial sectors to enhance the development of autonomous vehicles based on such guidelines. In the current state of the art, we find studies on how ethical theories can be integrated. To the best of the authors’ knowledge, no framework for motion planning has yet been published which allows for the true implementation of any practical (...) ethical policies. This paper makes four contributions: Firstly, we briefly present the state of the art based on recent works concerning unavoidable accidents of autonomous vehicles (AVs) and identify further need for research. While most of the research focuses on decision strategies in moral dilemmas or crash optimization, we aim to develop an ethical trajectory planning for all situations on public roads. Secondly, we discuss several ethical theories and argue for the adoption of the theory “ethics of risk.” Thirdly, we propose a new framework for trajectory planning, with uncertainties and an assessment of risks. In this framework, we transform ethical specifications into mathematical equations and thus create the basis for the programming of an ethical trajectory. We present a risk cost function for trajectory planning that considers minimization of the overall risk, priority for the worst-off and equal treatment of people. Finally, we build a connection between the widely discussed trolley problem and our proposed framework. (shrink)

The moral importance of liability to harm has so far been ignored in the lively debate about what self-driving vehicles should be programmed to do when an accident is inevitable. But liability matters a great deal to just distribution of risk of harm. While morality sometimes requires simply minimizing relevant harms, this is not so when one party is liable to harm in virtue of voluntarily engaging in activity that foreseeably creates a risky situation, while having reasonable alternatives. On plausible (...) assumptions, merely choosing to use a self-driving vehicle typically gives rise to a degree of liability, so that such vehicles should be programmed to shift the risk from bystanders to users, other things being equal. Insofar vehicles cannot be programmed to take all the factors affecting liability into account, there is a pro tanto moral reason not to introduce them, or restrict their use. (shrink)

The goal of this article is to deepen the concept of emerging urban mobility technology. Drawing on philosophical everyday and urban aesthetics, as well as the postphenomenological strand in the philosophy of technology, we explicate the relation between everyday aesthetic experience and urban mobility commoning. Thus, we shed light on the central role of aesthetics for providing depth to the important experiential and value-driven meaning of contemporary urban mobility. We use the example of self-driving vehicle (SDV), as potentially mundane, public, (...) dynamic, and social urban robots, for expanding the range of perspectives relevant for our relations to urban mobility technology. We present the range of existing SDV conceptualizations and contrast them with experiential and aesthetic understanding of urban mobility. In conclusion, we reflect on the potential undesired consequences from the depolitization of technological development, and potential new pathways for speculative thinking concerning urban mobility futures in responsible innovation processes. (shrink)

One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents (IAs). This paper explores how decision matrix algorithms, via the belief-desire-intention model for (...) autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing for the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles. (shrink)

Safe-by-Design (SBD) frameworks for the development of emerging technologies have become an ever more popular means by which scholars argue that transformative emerging technologies can safely incorporate human values. One such popular SBD methodology is called Value Sensitive Design (VSD). A central tenet of this design methodology is to investigate stakeholder values and design those values into technologies during early stage research and development (R&D). To accomplish this, the VSD framework mandates that designers consult the philosophical and ethical literature to (...) best determine how to weigh moral trade-offs. However, the VSD framework also concedes the universalism of moral values, particularly the values of freedom, autonomy, equality trust and privacy justice. This paper argues that the VSD methodology, particularly applied to nano-bio-info-cogno (NBIC) technologies, has an insufficient grounding for the determination of moral values. As such, an exploration of the value-investigations of VSD are deconstructed to illustrate both its strengths and weaknesses. This paper also provides possible modalities for the strengthening of the VSD methodology, particularly through the application of moral imagination and how moral imagination exceed the boundaries of moral intuitions in the development of novel technologies. (shrink)

Safe-by-Design (SBD) frameworks for the development of emerging technologies have become an ever more popular means by which scholars argue that transformative emerging technologies can safely incorporate human values. One such popular SBD methodology is called Value Sensitive Design (VSD). A central tenet of this design methodology is to investigate stakeholder values and design those values into technologies during early stage research and development (R&D). To accomplish this, the VSD framework mandates that designers consult the philosophical and ethical literature to (...) best determine how to weigh moral trade-offs. However, the VSD framework also concedes the universalism of moral values, particularly the values of freedom, autonomy, equality trust and privacy justice. This paper argues that the VSD methodology, particularly applied to nano-bio-info-cogno (NBIC) technologies, has an insufficient grounding for the determination of moral values. As such, an exploration of the value-investigations of VSD are deconstructed to illustrate both its strengths and weaknesses. This paper also provides possible modalities for the strengthening of the VSD methodology, particularly through the application of moral imagination and how moral imagination exceed the boundaries of moral intuitions in the development of novel technologies. (shrink)

Artificial intelligence (AI) has long been recognised as a challenge to responsibility. Much of this discourse has been framed around robots, such as autonomous weapons or self-driving cars, where we arguably lack control over a machine’s behaviour and therefore struggle to identify an agent that can be held accountable. However, most of today’s AI is based on machine-learning technology that does not act on its own, but rather serves as a decision-support tool, automatically analysing data to help human agents make (...) better decisions. I argue that decision-support tools pose a challenge to responsibility that goes beyond the familiar problem of finding someone to blame or punish for the behaviour of agent-like systems. Namely, they pose a problem for what we might call “decision ownership”: they make it difficult to identify human agents to whom we can attribute value-judgements that are reflected in decisions. Drawing on recent philosophical literature on responsibility and its various facets, I argue that this is primarily a problem of attributability rather than of accountability. This particular responsibility problem comes in different forms and degrees, most obviously when an AI provides direct recommendations for actions, but also, less obviously, when it provides mere descriptive information on the basis of which a decision is made. (shrink)

Human beings are internally inconsistent in various ways. One way to develop this thought involves using the language of value alignment: the values we hold are not always aligned with our behavior, and are not always aligned with each other. Because of this self-misalignment, there is room for potential projects of human enhancement that involve achieving a greater degree of value alignment than we presently have. Relatedly, discussions of AI ethics sometimes focus on what is known as the value alignment (...) problem, the challenge of how to build AI that acts in accordance with our human values. We argue that there is an especially close connection between solving the value alignment problem in AI ethics and using AI to pursue certain forms of human enhancement. But in addition, we also argue that there are important limits to what kinds of human enhancement can be pursued in this way, because some forms of human enhancement—namely moral revolutions—involve a kind of value misalignment rather than alignment. (shrink)

Almost all participants in the debate about the ethics of accidents with self-driving cars have so far assumed moral universalism. However, universalism may be philosophically more controversial than is commonly thought, and may lead to undesirable results in terms of non-moral consequences and feasibility. There thus seems to be a need to also start considering what I refer to as the “relativistic car” — a car that is programmed under the assumption that what is morally right, wrong, good, bad, etc. (...) is determined by the moral beliefs of one’s society or culture. My investigation of this idea involves six steps. First, I explain why and how the moral universalism/relativism debate is relevant to the issue of self-driving cars. Second, I argue that there are good reasons to consider accident algorithms that assume relativism. Third, I outline how a relativistic car would be programmed to behave. Fourth, I address what advantages such a car would have, both in terms of its non-moral consequences and feasibility. Fifth, I address the relativistic car’s disadvantages. Finally, I qualify and conclude my considerations. (shrink)

Self-driving vehicles offer great potential to improve efficiency on roads, reduce traffic accidents, increase productivity, and minimise our environmental impact in the process. However, they have also seen resistance from different groups claiming that they are unsafe, pose a risk of being hacked, will threaten jobs, and increase environmental pollution from increased driving as a result of their convenience. In order to reap the benefits of SDVs, while avoiding some of the many pitfalls, it is important to effectively determine what (...) challenges we will face in the future and what steps need to be taken now to avoid them. The approach taken in this paper is the construction of a likely future, through the process of a policy scenario methodology, if we continue certain trajectories over the coming years. The purpose of this is to articulate issues we currently face and the construction of a foresight analysis of how these may develop in the next 6 years. It will highlight many of the key facilitators and inhibitors behind this change and the societal impacts caused as a result. This paper will synthesise the wide range of ethical, legal, social and economic impacts that may result from SDV use and implementation by 2025, such as issues of autonomy, privacy, liability, security, data protection, and safety. It will conclude with providing steps that we need to take to avoid these pitfalls, while ensuring we reap the benefits that SDVs bring. (shrink)

Much of the debate on the ethics of self-driving cars has revolved around trolley scenarios. This paper instead takes up the political or institutional question of who should decide how a self-driving car drives. Specifically, this paper is on the question of whether and why passengers should be able to control how their car drives. The paper reviews existing arguments—those for passenger ethics settings and for mandatory ethics settings respectively—and argues that they fail. Although the arguments are not successful, they (...) serve as the basis to formulate desiderata that any approach to regulating the driving behavior of self-driving cars ought to fulfill. The paper then proposes one way of designing passenger ethics settings that meets these desiderata. (shrink)

At present, although artificial intelligence (AI) has made great progress in the aspects of control policy and operation, there is still no unified ethical decision mode, laws and regulations in ethical dilemma, which seriously affect application and development of artificial intelligence. Therefore, this paper studies and analyzes various ethical decision mode of AI system, then the current research status and advantages, deficiencies of these decision modes are analyzed and summarized. According to the characteristics of the current ethical decision modes and (...) existing problems, the future implementation mode for the future ethical decision of artificial intelligence is explored and discussed. A hierarchical hybrid ethical decision mode will be proposed and designed, which has the advantages of the existing ethical decision modes. In addition, it will provide a few references for research, design and formulation of laws, regulations of AI. (shrink)

The ethics of autonomous vehicles has received a great amount of attention in recent years, specifically in regard to their decisional policies in accident situations in which human harm is a likely consequence. Starting from the assumption that human harm is unavoidable, many authors have developed differing accounts of what morality requires in these situations. In this article, a strategy for AV decision-making is proposed, the Ethical Valence Theory, which paints AV decision-making as a type of claim mitigation: different road (...) users hold different moral claims on the vehicle’s behavior, and the vehicle must mitigate these claims as it makes decisions about its environment. Using the context of autonomous vehicles, the harm produced by an action and the uncertainties connected to it are quantified and accounted for through deliberation, resulting in an ethical implementation coherent with reality. The goal of this approach is not to define how moral theory requires vehicles to behave, but rather to provide a computational approach that is flexible enough to accommodate a number of ‘moral positions’ concerning what morality demands and what road users may expect, offering an evaluation tool for the social acceptability of an autonomous vehicle’s ethical decision making. (shrink)

This short paper offers a preliminary inquiry into the impacts of driving automation on personal autonomy. Personal autonomy is a key ethical value in western culture, and one that buttresses fundamental components of the moral life such as the exercise of responsible behaviour and the full enjoyment of human dignity. Driving automation simultaneously enhances and constrains it in significant ways. Hence, its moral profile with reference to the value of personal autonomy is uncertain. Ethical analysis shows that such uncertainty is (...) due not just to the complexity of the technology, but also to the multifaceted normative profile of personal autonomy, which offers reasons to support both conditional and full driving automation. The paper sheds light on this duplicity, underlines the challenges this poses to the ethics of driving automation, and advocates for further research aimed at providing practitioners with more fine-grained guidelines on such a delicate issue. (shrink)

Self‐driving cars hold out the promise of being much safer than regular cars. Yet they cannot be 100% safe. Accordingly, they need to be programmed for how to deal with crash scenarios. Should cars be programmed to always prioritize their owners, to minimize harm, or to respond to crashes on the basis of some other type of principle? The article first discusses whether everyone should have the same “ethics settings.” Next, the oft‐made analogy with the trolley problem is examined. Then (...) follows an assessment of recent empirical work on lay‐people's attitudes about crash algorithms relevant to the ethical issue of crash optimization. Finally, the article discusses what traditional ethical theories such as utilitarianism, Kantianism, virtue ethics, and contractualism imply about how cars should handle crash scenarios. The aim of the article is to provide an overview of the existing literature on these topics and to assess how far the discussion has gotten so far. (shrink)

In recent years, tech evangelists have made headlines predicting that in the future manual driving will be outlawed. This essay will investigate the question whether a ban of human driven cars can be defended on moral grounds in a future scenario in which autonomous cars are going to be significantly safer than manually driven cars. This article will argue that in such a future scenario manually driven cars, for moral reasons, indeed should be banned from participating in regular traffic. Since (...) the moral argument for outlawing manually driven cars will likely be met by resistance by car-aficionados, in the final part of the paper, we are devising a proposal for reconciling the strong moral case for a ban of manually driven cars with the widespread fondness of manual driving. (shrink)

Wie sollen sich autonome Fahrzeuge verhalten, wenn ein Unfall nicht mehr abwendbar ist? Die Komplexität spezifischer moralischer Dilemmata, die in diesem Kontext auftreten können, lässt bewährte ethische Denktraditionen an ihre Grenzen stoßen. Dieser Aufsatz versteht sich als Versuch, neue Lösungsperspektiven mithilfe einer risikoethischen Sichtweise auf die Problematik zu eröffnen und auf diese Weise deren Relevanz für die Programmierung von ethischen Unfallalgorithmen aufzuzeigen. Im Zentrum steht dabei die Frage, welche Implikationen sich aus einer Auffassung von Dilemma-Situationen als risikoethische Verteilungsprobleme im Hinblick (...) auf die Zulässigkeit von entsprechenden Risikoübertragungen ergeben. Dabei wird zunächst eine risikoethische Interpretation des zugrundeliegenden Entscheidungsproblems skizziert, welches durch seine dilemmatische Struktur eine besondere Risikokonstellation begründet. Ausgehend von den Positionen von Sven Ove Hansson und Julian Nida-Rümelin wird für einen deontologisch-risikoethischen Ansatz argumentiert, der auf Individualrechten einerseits und einer interpersonell gerechten Verteilung der entstehenden Schadensrisiken anderseits basiert. Diese beiden Kriterien werden für den Anwendungskontext des autonomen Fahrens konkretisiert. Zum einen wird in Bezug auf das erste Kriterium argumentiert, dass individuelle Rechte genau dann als angemessen gewahrt gelten können, wenn die resultierenden Risikoübertragungen auf die Einzelnen in ihrer absoluten Höhe jeweils zumutbar sind. Zum anderen werden Schwierigkeiten skizziert, die sich hinsichtlich der konkreten Umsetzung des zweiten Kriteriums der Verteilungsgerechtigkeit ergeben. In diesem Zusammenhang werden beispielsweise ethische Herausforderungen in Bezug auf einen möglichen Vorteilsausgleich, das Prinzip der Schadensminimierung sowie individuell unterschiedliche Ausgangsbedingungen der persönlichen Schadensreduktion kritisch in den Blick genommen. (shrink)

The ethics of emerging forms of artificial intelligence has become a prolific subject in both academic and public spheres. A great deal of these concerns flow from the need to ensure that these technologies do not cause harm—physical, emotional or otherwise—to the human agents with which they will interact. In the literature, this challenge has been met with the creation of artificial moral agents: embodied or virtual forms of artificial intelligence whose decision procedures are constrained by explicit normative principles, requiring (...) the implementation of what is commonly called artificial morality into these agents. To date, the types of reasoning structures and principles which inform artificial morality have been of two kinds: first, an ethically maximal vision of artificial morality which relies on the strict implementation of traditional moral theories such as Kantian deontology or Utilitarianism, and second, a more minimalist vision which applies stochastic AI techniques to large data sets of human moral preferences so as to illicit or intuit general principles and preferences for the design of artificial morality. Taken individually, each approach is unable to fully answer the challenge of producing inoffensive behavior in artificial moral agents, most especially since both forms are unable to strike a balance between the ideal set of constraints which morality imposes on one hand, and the types of constraints public acceptability imposes, on the other. We provide an alternative approach to the design of artificial morality, the Ethical Valence Theory, whose purpose is to accommodate this balance, and apply this approach to the case of autonomous vehicles. (shrink)