We are living through an era of increased robotisation. Some authors have already begun to explore the impact of this robotisation on legal rules and practice. In doing so, many highlight potential liability gaps that might arise through robot misbehaviour. Although these gaps are interesting and socially significant, they do not exhaust the possible gaps that might be created by increased robotisation. In this article, I make the case for one of those alternative gaps: the retribution gap. This gap arises (...) from a mismatch between the human desire for retribution and the absence of appropriate subjects of retributive blame. I argue for the potential existence of this gap in an era of increased robotisation; suggest that it is much harder to plug this gap than it is to plug those thus far explored in the literature; and then highlight three important social implications of this gap. (shrink)

The United States Army’s Future Combat Systems Project, which aims to manufacture a “robot army” to be ready for deployment by 2012, is only the latest and most dramatic example of military interest in the use of artificially intelligent systems in modern warfare. This paper considers the ethics of a decision to send artificially intelligent robots into war, by asking who we should hold responsible when an autonomous weapon system is involved in an atrocity of the sort that would normally (...) be described as a war crime. A number of possible loci of responsibility for robot war crimes are canvassed; the persons who designed or programmed the system, the commanding officer who ordered its use, the machine itself. I argue that in fact none of these are ultimately satisfactory. Yet it is a necessary condition for fighting a just war, under the principle of jus in bellum, that someone can be justly held responsible for deaths that occur in the course of the war. As this condition cannot be met in relation to deaths caused by an autonomous weapon system it would therefore be unethical to deploy such systems in warfare. (shrink)

Many ethicists writing about automated systems attribute agency to these systems. Not only that; they seemingly attribute an autonomous or independent form of agency to these machines. This leads some ethicists to worry about responsibility-gaps and retribution-gaps in cases where automated systems harm or kill human beings. In this paper, I consider what sorts of agency it makes sense to attribute to most current forms of automated systems, in particular automated cars and military robots. I argue that whereas it indeed (...) makes sense to attribute different forms of fairly sophisticated agency to these machines, we ought not to regard them as acting on their own, independently of any human beings. Rather, the right way to understand the agency exercised by these machines is in terms of human–robot collaborations, where the humans involved initiate, supervise, and manage the agency of their robotic collaborators. This means, I argue, that there is much less room for justified worries about responsibility-gaps and retribution-gaps than many ethicists think. (shrink)

This is a paperback edition of a major contribution to the field, first published in hard covers in 1977. The book outlines a general theory of rational behaviour consisting of individual decision theory, ethics, and game theory as its main branches. Decision theory deals with a rational pursuit of individual utility; ethics with a rational pursuit of the common interests of society; and game theory with an interaction of two or more rational individuals, each pursuing his own interests in a (...) rational manner. (shrink)

The recent progress in the development of autonomous cars has seen ethical questions come to the forefront. In particular, life and death decisions regarding the behavior of self-driving cars in trolley dilemma situations are attracting widespread interest in the recent debate. In this essay we want to ask whether we should implement a mandatory ethics setting for the whole of society or, whether every driver should have the choice to select his own personal ethics setting. While the consensus view seems (...) to be that people would not be willing to use an automated car that might sacrifice themselves in a dilemma situation, we will defend the somewhat contra-intuitive claim that this would be nevertheless in their best interest. The reason is, simply put, that a PES regime would most likely result in a prisoner’s dilemma. (shrink)

A number of companies including Google and BMW are currently working on the development of autonomous cars. But if fully autonomous cars are going to drive on our roads, it must be decided who is to be held responsible in case of accidents. This involves not only legal questions, but also moral ones. The first question discussed is whether we should try to design the tort liability for car manufacturers in a way that will help along the development and improvement (...) of autonomous vehicles. In particular, Patrick Lin’s concern that any security gain derived from the introduction of autonomous cars would constitute a trade-off in human lives will be addressed. The second question is whether it would be morally permissible to impose liability on the user based on a duty to pay attention to the road and traffic and to intervene when necessary to avoid accidents. Doubts about the moral legitimacy of such a scheme are based on the notion that it is a form of defamation if a person is held to blame for causing the death of another by his inattention if he never had a real chance to intervene. Therefore, the legitimacy of such an approach would depend on the user having an actual chance to do so. The last option discussed in this paper is a system in which a person using an autonomous vehicle has no duty of interfering, but is still held responsible for possible accidents. Two ways of doing so are discussed, but only one is judged morally feasible. (shrink)

Our brains were designed for tribal life, for getting along with a select group of others and for fighting off everyone else. But modern times have forced the world’s tribes into a shared space, resulting in epic clashes of values along with unprecedented opportunities. As the world shrinks, the moral lines that divide us become more salient and more puzzling. We fight over everything from tax codes to gay marriage to global warming, and we wonder where, if at all, we (...) can find our common ground. (shrink)

We propose that the prevalent moral aversion to AWS is supported by a pair of compelling objections. First, we argue that even a sophisticated robot is not the kind of thing that is capable of replicating human moral judgment. This conclusion follows if human moral judgment is not codifiable, i.e., it cannot be captured by a list of rules. Moral judgment requires either the ability to engage in wide reflective equilibrium, the ability to perceive certain facts as moral considerations, moral (...) imagination, or the ability to have moral experiences with a particular phenomenological character. Robots cannot in principle possess these abilities, so robots cannot in principle replicate human moral judgment. If robots cannot in principle replicate human moral judgment then it is morally problematic to deploy AWS with that aim in mind. Second, we then argue that even if it is possible for a sufficiently sophisticated robot to make ‘moral decisions’ that are extensionally indistinguishable from (or better than) human moral decisions, these ‘decisions’ could not be made for the right reasons. This means that the ‘moral decisions’ made by AWS are bound to be morally deficient in at least one respect even if they are extensionally indistinguishable from human ones. Our objections to AWS support the prevalent aversion to the employment of AWS in war. They also enjoy several significant advantages over the most common objections to AWS in the literature. (shrink)

Automated technologies and robots make decisions that cannot always be fully controlled or predicted. In addition to that, they cannot respond to punishment and blame in the ways humans do. Therefore, when automated cars harm or kill people, for example, this gives rise to concerns about responsibility-gaps and retribution-gaps. According to Sven Nyholm, however, automated cars do not pose a challenge on human responsibility, as long as humans can control them and update them. He argues that the agency exercised in (...) automated cars should be understood in terms of human–robot collaborations. This brief note focuses on the problem that arises when there are multiple people involved, but there is no obvious shared collaboration among them. Building on John Danaher’s discussion of command responsibility, it is argued that, although Nyholm might be right that autonomous cars cannot be regarded as acting on their own, independently of any human beings, worries about responsibility-gaps and retribution-gaps are still justified, because it often remains unclear how to allocate or distribute responsibility satisfactorily among the key humans involved after they have been successfully identified. (shrink)

The underlying thesis of the research in ethical autonomy for lethal autonomous unmanned systems is that they will potentially be capable of performing more ethically on the battlefield than are human soldiers. In this article this hypothesis is supported by ongoing and foreseen technological advances and perhaps equally important by an assessment of the fundamental ability of human warfighters in today's battlespace. If this goal of better-than-human performance is achieved, even if still imperfect, it can result in a reduction in (...) noncombatant casualties and property damage consistent with adherence to the Laws of War as prescribed in international treaties and conventions, and is thus worth pursuing vigorously. (shrink)

In this paper, we discuss the ethics of automated driving. More specifically, we discuss responsible human-robot coordination within mixed traffic: i.e. traffic involving both automated cars and conventional human-driven cars. We do three main things. First, we explain key differences in robotic and human agency and expectation-forming mechanisms that are likely to give rise to compatibility-problems in mixed traffic, which may lead to crashes and accidents. Second, we identify three possible solution-strategies for achieving better human-robot coordination within mixed traffic. Third, (...) we identify important ethical challenges raised by each of these three possible strategies for achieving optimized human-robot cordination in this domain. Among other things, we argue that we should not just explore ways of making robotic driving more like human driving. Rather, we ought also to take seriously potential ways of making human driving more like robotic driving. Nor should we assume that complete automation is always the ideal to aim for; in some traffic-situations, the best results may be achieved through human-robot collaboration. Ultimately, our main aim in this paper is to argue that the new field of the ethics of automated driving needs take seriously the ethics of mixed traffic and responsible human-robot coordination. (shrink)

The discourse around self-driving cars has been dominated by an emphasis on their potential to reduce the number of accidents. At the same time, proponents acknowledge that self-driving cars would inevitably be involved in fatal accidents where moral algorithms would decide the fate of those involved. This is a necessary trade-off, proponents suggest, in order to reap the benefits of this new technology. In this article, I engage this argument, demonstrating how an undue optimism and enthusiasm about this technology is (...) obscuring our ability to see what is at stake and explaining how moving beyond the dominant utilitarian framings around this technology opens up a space for both ethical inquiry and innovative design. I suggest that a genuine caring concern for the many lives lost in car accidents now and in the future—a concern that transcends false binary trade-offs and that recognizes the systemic biases and power structures that make certain groups more vulnerable than others—could serve as a starting point to rethink mobility, as it connects to the design of our cities, the well-being of our communities, and the future of our planet. (shrink)

Novel varieties of interplay between humans, robots and software agents are on the rise. Computer-based artefacts are no longer mere tools but have become interaction partners. Distributed problem solving and social agency may be modelled by social computing systems based on multi-agent systems. MAS and agent-based modelling approaches focus on the simulation of complex interactions and relationships of human and/or non-human agents. MAS may be deployed both in virtual environments and cyber-physical systems. With regard to their impact on the physical (...) environment, such systems possess a virtual actuality in a testbed. They have a real actuality when they are employed in real time in order to govern processes in the natural world. Constellations of inter-agency and distributed agency materialize. This paper presents a multidimensional, gradual classification framework for individual and joint agency. Activity level, adaptivity, interaction and personification of others are essential dimensions. The framework allows constellations of distributed and collective agency in socio-technical systems to be analysed in detail. Scenarios where solely humans act can be compared to testbed simulations based on this classification scheme. (shrink)

With their prospect for causing both novel and known forms of damage, harm and injury, the issue of responsibility has been a recurring theme in the debate concerning autonomous vehicles. Yet, the discussion of responsibility has obscured the finer details both between the underlying concepts of responsibility, and their application to the interaction between human beings and artificial decision-making entities. By developing meaningful distinctions and examining their ramifications, this article contributes to this debate by refining the underlying concepts that together (...) inform the idea of responsibility. Two different approaches are offered to the question of responsibility and autonomous vehicles: targeting and risk distribution. The article then introduces a thought experiment which situates autonomous vehicles within the context of crash optimisation impulses and coordinated or networked decision-making. It argues that guiding ethical frameworks overlook compound or aggregated effects which may arise, and which can lead to subtle forms of structural discrimination. Insofar as such effects remain unrecognised by the legal systems relied upon to remedy them, the potential for societal inequalities is increased and entrenched, situations of injustice and impunity may be unwittingly maintained. This second set of concerns may represent a hitherto overlooked type of responsibility gap arising from inadequate accountability processes capable of challenging systemic risk displacement. (shrink)

Drink driving causes great suffering and material destruction. The alcohol interlock promises to eradicate this problem by technological design. Traditional counter-measures to drink driving such as policing and punishment and information campaigns have proven insufficient. Extensive policing is expensive and intrusive. Severe punishment is disproportionate to the risks created in most single cases. If the interlock becomes inexpensive and convenient enough, and if there are no convincing moral objections to the device, it may prove the only feasible as well as (...) the only justifiable solution to the problem of drink driving. A policy of universal alcohol interlocks, in all cars, has been proposed by several political parties in Sweden and is supported by the National Road Administration and the 2006 Alcohol Interlock Commission. This article assesses two possible moral objections to a policy of universal interlocks: (i) that it displaces the responsibility of individual drivers and (ii) that it constitutes a paternalistic interference with drivers. The first objection is found unconvincing, while the second has only limited bite and may be neutralized if paternalism is accepted for the sake of greater net liberty. Given the expected technological development, the proposed policy seems a commendable health promotion measure for the near future. (shrink)

Suppose a driverless car encounters a scenario where harm to at least one person is unavoidable and a choice about how to distribute harms between different persons is required. How should the driverless car be programmed to behave in this situation? I call this the moral design problem. Santoni de Sio defends a legal-philosophical approach to this problem, which aims to bring us to a consensus on the moral design problem despite our disagreements about which moral principles provide the correct (...) account of justified harm. He then articulates an answer to the moral design problem based on the legal doctrine of necessity. In this paper, I argue that Santoni de Sio’s answer to the moral design problem does not achieve the aim of the legal-philosophical approach. This is because his answer relies on moral principles which, at least, utilitarians have reason to reject. I then articulate an alternative reading of the doctrine of necessity, and construct a partial answer to the moral design problem based on this. I argue that utilitarians, contractualists and deontologists can agree on this partial answer, even if they disagree about which moral principles offer the correct account of justified harm. (shrink)

The paper concerns the problem of the legal responsibility of autonomous machines. In our opinion it boils down to the question of whether such machines can be seen as real agents through the prism of folk-psychology. We argue that autonomous machines cannot be granted the status of legal agents. Although this is quite possible from purely technical point of view, since the law is a conventional tool of regulating social interactions and as such can accommodate various legislative constructs, including legal (...) responsibility of autonomous artificial agents, we believe that it would remain a mere ‘law in books’, never materializing as ‘law in action’. It is not impossible to imagine that the evolution of our conceptual apparatus will reach a stage, when autonomous robots become full-blooded moral and legal agents. However, today at least, we seem to be far from this point. (shrink)

Speeding is a major problem in road safety. Intelligent Speed Adaptation is a potential solution, but the moral acceptability of ISA has been called into question both in the popular media and in academic discussions. In this article, a moral case is made for making warning and limiting versions of ISA obligatory in all cars. The practice of car driving involves frequent speeding, which imposes unacceptable risks of harm on other road users. In this article, I argue that ISA can (...) therefore be justified on the basis of the harm it prevents, as is the current criminalisation of speeding. I defend obligatory ISA against three objections. First, ISA is likely to introduce some additional risk for drivers. However, drivers should accept these risks to reduce the risks from driving for other parties to an acceptable level. Second, although limiting ISA reduces drivers' options for moral agency and exercising self-restraint to some extent, this consequence is defensible. Third, accepting ISA does not commit us to accepting an entire range of other behaviour-regulating technologies. (shrink)

This article argues that it is possible to hold autonomous agents themselves, and not only their makers, users or owners, responsible for the acts of these agents. In this connection autonomous systems are computer programs that interact with the outside world without human interference. They include such systems as ‘intelligent’ weapons and self-driving cars. The argument is based on an analogy between human beings and autonomous agents and its main element asserts that if humans can be held responsible, so can, (...) in principle, autonomous agents, as humans are more like autonomous agents than is often assumed. (shrink)

Besides offering opportunities in both clinical and non-clinical domains, the application of novel neuroimaging technologies raises pressing dilemmas. ‘Responsible Research and Innovation’ (RRI) aims to stimulate research and innovation activities that take ethical and social considerations into account from the outset. We previously identified that Dutch neuroscientists interpret “responsible innovation” as educating the public on neuroimaging technologies via the popular press. Their aim is to mitigate (neuro)hype, an aim shared with the wider emerging RRI community. Here, we present results of (...) a media-analysis undertaken to establish whether the body of articles in the Dutch popular press presents balanced conversations on neuroimaging research to the public. We found that reporting was mostly positive and framed in terms of (healthcare) progress. There was rarely a balance between technology opportunities and limitations, and even fewer articles addressed societal or ethical aspects of neuroimaging research. Furthermore, neuroimaging metaphors seem to favour oversimplification. Current reporting is therefore more likely to enable hype than to mitigate it. How can neuroscientists, given their self-ascribed social responsibility, address this conundrum? We make a case for a collective and shared responsibility among neuroscientists, journalists and other stakeholders, including funders, committed to responsible reporting on neuroimaging research. (shrink)

This paper explores how the phenomenology of using self-driving cars influences conditions for exercising and ascribing responsibility. First, a working account of responsibility is presented, which identifies two classic Aristotelian conditions for responsibility and adds a relational one, and which makes a distinction between responsibility for (what one does) and responsibility to (others). Then, this account is applied to a phenomenological analysis of what happens when we use a self-driving car and participate in traffic. It is argued that self-driving cars (...) threaten the excercise and ascription of responsibility in several ways. These include the replacement of human agency by machine agency, but also the user’s changing epistemic relation to the environment and others, which can be described in terms of (dis)engagement. It is concluded that the discussion about the ethics of self-driving cars and related problems of responsibility should be restricted neither to general responsibilities related to the use of self-driving cars and its objective risks, nor to questions regarding the behavior, intelligence, autonomy, and ethical “thinking” of the car in response to the objective features of the traffic situations (e.g. various scenarios). Rather, it should also reflect on the shifting experience of the user: how the new technology reshapes the subjectivity of the user and on the morel consequences this has. (shrink)