When a computer system causes harm, who is responsible? This question has renewed significance given the proliferation of autonomous systems enabled by modern artificial intelligence techniques. At the root of this problem is a philosophical difficulty known in the literature as the responsibility gap. That is to say, because of the causal distance between the designers of autonomous systems and the eventual outcomes of those systems, the dilution of agency within the large and complex teams that design autonomous systems, (...) and the impossibility of fully predicting how autonomous systems will behave once deployed, determining who is morally responsible for harms caused by autonomous systems is unclear at a conceptual level. I review past work on this topic, criticizing prior works for suggesting workarounds rather than philosophical answers to the conceptual problem presented by the responsibility gap. The view I develop, drawing on my earlier work on vicarious moral responsibility, explains why computing professionals are ethically required to take responsibility for the systems they design, despite not being blameworthy for the harms these systems may cause. (shrink)

This paper discusses the epistemic status of biology from the standpoint of the systemic approach to living systems based on the notion of biological autonomy. This approach aims to provide an understanding of the distinctive character of biological systems and this paper analyses its theoretical and epistemological dimensions. The paper argues that, considered from this perspective, biological systems are examples of emergent phenomena, that the biological domain exhibits special features with respect to other domains, and that biology as a discipline (...) employs some core concepts, such as teleology, function, regulation among others, that are irreducible to those employed in physics and chemistry. It addresses the claim made by Jacques Monod that biology as a science is marginal. It argues that biology is general insofar as it constitutes a paradigmatic example of complexity science, both in terms of how it defines the theoretical object of study and of the epistemology and heuristics employed. As such, biology may provide lessons that can be applied more widely to develop an epistemology of complex systems. (shrink)

Abstract: The rapid advancement of artificial intelligence (AI) has catalyzed significant developments in autonomous systems, which are increasingly shaping diverse sectors including transportation, robotics, and industrial automation. This paper explores the evolution of AI technologies that underpin these autonomous systems, focusing on their capabilities, applications, and the challenges they present. Key areas of discussion include the technological innovations driving autonomy, such as machine learning algorithms and sensor integration, and the practical implementations observed in autonomous vehicles, drones, and robotic systems. Additionally, (...) the paper addresses critical challenges including safety, ethical concerns, and regulatory issues that influence the deployment and acceptance of autonomous technologies. By examining current trends and future prospects, this research aims to provide a comprehensive overview of how AI is transforming the landscape of autonomous systems and to identify key areas for future research and development. (shrink)

There are unusual challenges in ethics for RAS. Perhaps the issue can best be summarised as needing to consider “technically informed ethics”. The technology of RAS raises issues that have an ethical dimension, and perhaps uniquely so due to the possibility of moving human decision-making which is implicitly ethically informed to computer systems. Further, if seeking solutions to these problems – ethically aligned design, to use the IEEE’s terminology – then the solutions must be technically meaningful, capable of realisation, capable (...) of assurance, and suitable as a basis for regulation. Thus, ethics for RAS is a rich, complex multi-disciplinary concern, and perhaps more complex than many other ethical issues facing society today. It is also fast-moving. This paper has endeavoured to give an accessible introduction to some of the key issues, noting that many of them are quite subtle, and it is not possible to do them full justice in such a short document. However, we have sought to counterbalance this by giving an extensive list of initiatives, standards, etc. that focus on ethics of RAS and AI, see Annex A. (shrink)

Technological developments involving robotics and artificial intelligence devices are being employed evermore in elderly care and the healthcare sector more generally, raising ethical issues and practical questions warranting closer considerations of what we mean by “care” and, subsequently, how to design such software coherently with the chosen definition. This paper starts by critically examining the existing approaches to the ethical design of care robots provided by Aimee van Wynsberghe, who relies on the work on the ethics of care by Joan (...) Tronto. In doing so, it suggests an alternative to their non-principled approach, an alternative suited to tackling some of the issues raised by Tronto and van Wynsberghe, while allowing for the inclusion of two orientative principles. Our proposal centres on the principles of autonomy and vulnerability, whose joint adoption we deem able to constitute an original revision of a bottom-up approach in care ethics. Conclusively, the ethical framework introduced here integrates more traditional approaches in care ethics in view of enhancing the debate regarding the ethical design of care robots under a new lens. (shrink)

In October 2011, the “2nd European Network for Cognitive Systems, Robotics and Interaction”, EUCogII, held its meeting in Groningen on “Autonomous activity in real-world environments”, organized by Tjeerd Andringa and myself. This is a brief personal report on why we thought autonomy in real-world environments is central for cognitive systems research and what I think I learned about it. --- The theses that crystallized are that a) autonomy is a relative property and a matter of degree, b) increasing autonomy of (...) an artificial system from its makers and users is a necessary feature of increasingly intelligent systems that can deal with the real-world and c) more such autonomy means less control but at the same time improved interaction with the system. (shrink)

To many, the idea of autonomous weapons systems (AWS) killing human beings is grotesque. Yet critics have had difficulty explaining why it should make a significant moral difference if a human combatant is killed by an AWS as opposed to being killed by a human combatant. The purpose of this paper is to explore the roots of various deontological concerns with AWS and to consider whether these concerns are distinct from any concerns that also apply to long-distance, human-guided weaponry. We (...) suggest that at least one major driver of the intuitive moral aversion to lethal AWS is that their use disrespects their human targets by violating the martial contract between human combatants. On our understanding of this doctrine, service personnel cede a right not to be directly targeted with lethal violence to other human agents alone. Artificial agents, of which AWS are one example, cannot understand the value of human life. A human combatant cannot transfer his privileges of targeting enemy combatants to a robot. Therefore, the human duty-holder who deploys AWS breaches the martial contract between human combatants and disrespects the targeted combatants. We consider whether this novel deontological objection to AWS forms the foundation of several other popular yet imperfect deontological objections to AWS. (shrink)

Autonomous weapons systems (AWS), sometimes referred to as “killer robots”, are receiving evermore attention, both in public discourse as well as by scholars and policymakers. Much of this interest is connected with emerging ethical and legal problems linked to increasing autonomy in weapons systems, but there is a general underappreciation for the ways in which existing law might impact on these new technologies. In this paper, we argue that as AWS become more sophisticated and increasingly more capable than flesh-and-blood soldiers, (...) it will increasingly be the case that such soldiers are “in the power” of those AWS which fight against them. This implies that such soldiers ought to be considered hors de combat, and not targeted. In arguing for this point, we draw out a broader conclusion regarding hors de combat status, namely that it must be viewed contextually, with close reference to the capabilities of combatants on both sides of any discreet engagement. Given this point, and the fact that AWS may come in many shapes and sizes, and can be made for many different missions, we argue that each particular AWS will likely need its own standard for when enemy soldiers are deemed hors de combat. We conclude by examining how these nuanced views of hors de combat status might impact on meaningful human control of AWS. (shrink)

Unlike human soldiers, autonomous weapons systems are unaffected by psychological factors that would cause them to act outside the chain of command. This is a compelling moral justification for their development and eventual deployment in war. To achieve this level of sophistication, the software that runs AWS will have to first solve two problems: the frame problem and the representation problem. Solutions to these problems will inevitably involve complex software. Complex software will create security risks and will make AWS critically (...) vulnerable to hacking. I claim that the political and tactical consequences of hacked AWS far outweigh the purported advantages of AWS not being affected by psychological factors and always following orders. Therefore, one of the moral justifications for the deployment of AWS is undermined. (shrink)

Predictions about autonomous weapon systems are typically thought to channel fears that drove all the myths about intelligence embodied in matter. One of these is the idea that the technology can get out of control and ultimately lead to horrifi c consequences, as is the case in Mary Shelley’s classic Frankenstein. Given this, predictions about AWS are sometimes dismissed as science-fiction fear-mongering. This paper considers several analogies between AWS and other weapon systems and ultimately offers an argument that nuclear weapons (...) and their effect on the development of modern asymmetrical warfare are the best analogy to the introduction of AWS. The fi nal section focuses on this analogy and offers speculations about the likely consequences of AWS being hacked. These speculations tacitly draw on myths and tropes about technology and AI from popular fi ction, such as Frankenstein, to project a convincing model of the risks and benefi ts of AWS deployment. (shrink)

Would it be ethical to deploy autonomous weapon systems (AWS) if they were unable to reliably recognize when enemy forces had surrendered? I suggest that an inability to reliably recognize surrender would not prohibit the ethical deployment of AWS where there was a limited window of opportunity for targets to surrender between the launch of the AWS and its impact. However, the operations of AWS with a high degree of autonomy and/or long periods of time between release and impact are (...) likely to remain controversial until they have the capacity to reliably recognize surrender. (shrink)

Please contact me at [email protected] if you are interested in reading a particular chapter or being sent the entire manuscript for private use. -/- The thesis offers a comprehensive argument in favor of a regulationist approach to autonomous weapon systems (AWS). AWS, defined as all military robots capable of selecting or engaging targets without direct human involvement, are an emerging and potentially deeply transformative military technology subject to very substantial ethical controversy. AWS have both their enthusiasts and their detractors, prominently (...) advocating for a global preemptive ban on AWS development and use. Rejecting both positions, the author outlines a middle-of-the-road regulationist approach that is neither overly restrictive nor overly permissive. The disqualifying flaws of the rival prohibitionist approach are demonstrated in the process. After defining the core term of autonomy in weapon systems, the practical difficulties involved in applying an arms control regime to AWS are analyzed. The analysis shows that AWS are an extremely regulation-resistant technology. This feature when combined with their assumed high military utility makes a ban framework extremely costly to impose and enforce. As such it is ultimately very likely to fail at the benefit of the most unscrupulous international actors and at a very substantial risk to those abiding with international law. Consequently, to be ethically viable, a prohibitionist framework would need to offer substantial moral benefits impossible to attain through the rival regulationist approach. The remainder of the thesis undertakes to demonstrate that this is not the case. Comparing the considerations of military and strategic necessity to humanitarian concerns most commonly voiced by prohibitionists requires finding a common denominator for all values being referred to. Consequently, the thesis proceeds to show that both kinds of concerns are ultimately reducible to respect for basic human rights of all stakeholders, and so that the prohibitionist and regulationist approach may ultimately be compared in terms of consequences their adoption would generate for basic human rights realization. The author then evaluates both the potential humanitarian benefits, and the potential humanitarian hazards of AWS introduction. The benefits of leaving frontline combat to machines are outlined, with the unique kinds of suffering that would be abolished by such a development being described in detail. The arguments against AWS adoption are then divided into three classes: arguments related to alleged impossibility of compliance with The Laws of Armed Conflict, non-consequentialist and broad consequentialist arguments. This analysis, which comprises the greater part of the entire thesis, shows that the concerns behind compliance arguments are indeed substantial and have to be accommodated via a complex framework of best practices, regulations and localized restrictions on some kinds of AWS or AWS use in particular environments. They do not, however, justify a universal ban on using all the diverse forms of AWS in all environments. Non-consequentialist objections are found either reducible to other classes of arguments or thoroughly unconvincing, sometimes to the point of being actually vacuous. Broad consequentialist concerns are likewise found to be accommodable by regulation, empirically unfounded or causally disconnected from the actions of legitimate actors acquiring AWS, and therefore irrelevant to the moral permissibility of such actions. The author concludes that the proponents of prohibitionism are unable to point to moral benefits substantial enough to justify the costs and risks inherent in the approach. A global ban is, in fact, likely to have a worse humanitarian impact that well-regulated AWS adoption even if the strategic risks are disregarded. On the other hand, the analysis shows that there indeed exists an urgent need to regulate AWS through a variety of technological, procedural and legal solutions. These include, but are not limited to, a temporary moratorium on anti-personnel AWS use, development of internationally verified compliance software and eventual legal requirement of its employment, a temporary moratorium on AWS proliferation to state actors and a ban on their proliferation to non-state agents. (shrink)

We propose a multi-step evaluation schema designed to help procurement agencies and others to examine the ethical dimensions of autonomous systems to be applied in the security sector, including autonomous weapons systems.

This paper argues against prominent views of the impermissibility of autonomous weapons systems (AWS). It does so by assuming each theory is true and arguing towards contradiction. To arrive at a contradiction two assumptions are necessary. First, the theory of impermissibility in question is assumed. Second, a thought experiment called the ideal warfare scenario is assumed. The paper aims to demonstrate that in theory AWS could be deployed such that they bring about the best of possible warfare. However, even if (...) AWS were deployed in the fairest and safest way, these accounts would still find them to be impermissible. This is deemed to be a failure of these accounts. Finally, this paper discusses the future of artificial intelligence (AI). It explores how AI may be used in the future of warfare and the challenges advanced AI would pose to the ethics of war and the relationship between human soldiers and AWS. (shrink)

Autonomous Weapon Systems (AWS) are the next logical advancement for military technology. There is a significant concern though that by allowing such systems on the battlefield, we are collectively abdicating our moral responsibility. In this thesis, I will examine two arguments that advocate for a total ban on the use of AWS. I call these arguments the “Responsibility” and the “Agency” arguments. After presenting these arguments, I provide my own objections and demonstrate why these arguments fail to convince. I then (...) provide an argument as to why the use of AWS is a rational choice in the evolution of warfare. I conclude my thesis by providing a framework upon which future international regulations regarding AWS could be built. (shrink)

Autonomous Weapons Systems (AWS) have not gained a good reputation in the past. This attitude is odd if we look at the discussion of other-usually highly anticipated-AI-technologies, like autonomous vehicles (AVs); whereby even though these machines evoke very similar ethical issues, philosophers' attitudes towards them are constructive. In this article, I try to prove that there is an unjust bias against AWS because almost every argument against them is effective against AVs too. I start with the definition of "AWS." Then, (...) I arrange my arguments by the Just War Theory (JWT), covering jus ad bellum, jus in bello and jus post bellum problems. Meanwhile, I draw attention to similar problems against other AI-technologies outside the JWT framework. Finally, I address an exception, as addressed by Duncan Purves, Ryan Jenkins and Bradley Strawser, who realized the unjustified double standard, and deliberately tried to construct a special argument which rules out only AWS. (shrink)

International regulation of autonomous weapon systems (AWS) is increasingly conceived as an exercise in risk management. This requires a shared approach for assessing the risks of AWS. This paper presents a structured approach to risk assessment and regulation for AWS, adapting a qualitative framework inspired by the Intergovernmental Panel on Climate Change (IPCC). It examines the interactions among key risk factors—determinants, drivers, and types—to evaluate the risk magnitude of AWS and establish risk tolerance thresholds through a risk matrix informed by (...) background knowledge of event likelihood and severity. Further, it proposes a methodology to assess community risk appetite, emphasizing that such assessments and resulting tolerance levels should be determined through deliberation in a multistakeholder forum. The paper highlights the complexities of applying risk-based regulations to AWS internationally, particularly the challenge of defining a global community for risk assessment and regulatory legitimization. (shrink)

Several recent lines of inquiry have pointed to the amygdala as a potential lesion site in autism. Because one function of the amygdala may be to produce autonomic arousal at the sight of a significant face, we compared the responses of autistic children to their mothers’ face and to a plain paper cup. Unlike normals, the autistic children as a whole did not show a larger response to the person than to the cup. We also monitored sympathetic activity in (...) autistic children as they engaged in a wide range of everyday behaviours. The children tended to use self-stimulation activities in order to calm hyper-responsive activity of the sympathetic (`fight or flight’) branch of the autonomic nervous system. A small percentage of our autistic subjects had hyporesponsive sympathetic activity, with essentially no electrodermal responses except to self-injurious behaviour. We sketch a hypothesis about autism according to which autistic children use overt behaviour in order to control a malfunctioning autonomic nervous system and suggest that they have learned to avoid using certain processing areas in the temporal lobes. (shrink)

The international debate on the ethics and legality of autonomous weapon systems (AWS) as well as the call for a ban are primarily focused on the nebulous concept of fully autonomous AWS. More specifically, on AWS that are capable of target selection and engagement without human supervision or control. This thesis argues that such a conception of autonomy is divorced both from military planning and decision-making operations as well as the design requirements that govern AWS engineering and subsequently the tracking (...) and tracing of moral responsibility. To do this, this thesis marries two different levels of meaningful human control (MHC), termed levels of abstraction, to couple military operations with design ethics. In doing so, this thesis argues that the contentious notion of ‘full’ autonomy is not problematic under this two-tiered understanding of MHC. It proceeds to propose the value sensitive design (VSD) approach as a means for designing for MHC. (shrink)

Will future lethal autonomous weapon systems (LAWS), or ‘killer robots’, be a threat to humanity? The European Parliament has called for a moratorium or ban of LAWS; the ‘Contracting Parties to the Geneva Convention at the United Nations’ are presently discussing such a ban, which is supported by the great majority of writers and campaigners on the issue. However, the main arguments in favour of a ban are unsound. LAWS do not support extrajudicial killings, they do not take responsibility away (...) from humans; in fact they increase the abil-ity to hold humans accountable for war crimes. Using LAWS in war would probably reduce human suffering overall. Finally, the availability of LAWS would probably not increase the probability of war or other lethal conflict—especially as compared to extant remote-controlled weapons. The widespread fear of killer robots is unfounded: They are probably good news. (shrink)

Autonomous e-coaching systems offer their users suggestions for action, thereby affecting the user's decision-making process. More specifically, the suggestions that these systems make influence the options for action that people actually consider. Surprisingly though, options and the corresponding process of option generation --- a decision-making stage preceding intention formation and action selection --- has received very little attention in the various disciplines studying decision making. We argue that this neglect is unjustified and that it is important, particularly for designers of (...) autonomous e-coaching systems, to understand how human option generation works. The aims of this paper are threefold. The first aim is to generate awareness with designers of autonomous e-coaching systems that these systems do in fact influence their users' options. The second is to show that understanding the interplay between a person's options and the e-coaching system's suggestions is important for improving the effectiveness of the system. The third is that the very same interplay is also crucial for designing e-coaching systems that respect people's autonomy. (shrink)

While Autonomous Weapons Systems have obvious military advantages, there are prima facie moral objections to using them. By way of general reply to these objections, I point out similarities between the structure of law and morality on the one hand and of automata on the other. I argue that these, plus the fact that automata can be designed to lack the biases and other failings of humans, require us to automate the formulation, administration, and enforcement of law as much as (...) possible, including the elements of law and morality that are operated by combatants in war. I suggest that, ethically speaking, deploying a legally competent robot in some legally regulated realm is not much different from deploying a more or less well-armed, vulnerable, obedient, or morally discerning soldier or general into battle, a police officer onto patrol, or a lawyer or judge into a trial. All feature automaticity in the sense of deputation to an agent we do not then directly control. Such relations are well understood and well-regulated in morality and law; so there is not much challenging philosophically in having robots be some of these agents — excepting the implications of the limits of robot technology at a given time for responsible deputation. I then consider this proposal in light of the differences between two conceptions of law. These are distinguished by whether each conception sees law as unambiguous rules inherently uncontroversial in each application; and I consider the prospects for robotizing law on each. Likewise for the prospects of robotizing moral theorizing and moral decision-making. Finally I identify certain elements of law and morality, noted by the philosopher Immanuel Kant, which robots can participate in only upon being able to set ends and emotionally invest in their attainment. One conclusion is that while affectless autonomous devices might be fit to rule us, they would not be fit to vote with us. For voting is a process for summing felt preferences, and affectless devices would have none to weigh into the sum. Since they don't care which outcomes obtain, they don't get to vote on which ones to bring about. (shrink)

The prospective introduction of autonomous cars into public traffic raises the question of how such systems should behave when an accident is inevitable. Due to concerns with self-interest and liberal legitimacy that have become paramount in the emerging debate, a contractarian framework seems to provide a particularly attractive means of approaching this problem. We examine one such attempt, which derives a harm minimisation rule from the assumptions of rational self-interest and ignorance of one’s position in a future accident. We contend, (...) however, that both contractarian approaches and harm minimisation standards are flawed, due to a failure to account for the fundamental difference between those ‘involved’ and ‘uninvolved’ in an impending crash. Drawing from classical works on the trolley problem, we show how this notion can be substantiated by reference to either the distinction between negative and positive rights, or to differences in people’s claims. By supplementing harm minimisation with corresponding constraints, we can develop crash algorithms for autonomous cars which are both ethically adequate and promise to overcome certain significant practical barriers to implementation. (shrink)

Techno-ethics is the area in the philosophy of technology which deals with emerging robotic and digital AI technologies. In the last decade, a new techno-ethical challenge has emerged: Autonomous Weapon Systems (AWS), defensive and offensive (the article deals only with the latter). Such AI-operated lethal machines of various forms (aerial, marine, continental) raise substantial ethical concerns. Interestingly, the topic of AWS was almost not treated in Jewish law and its research. This article thus proposes an introductory ethical-halakhic perspective on AWS, (...) in the Israeli context. The article has seven sections. Section 1 defines AWS and the main ethical concerns it evokes, while providing elementary definitions and distinctions. §2 locates AWS within the realm of Jewish laws of war (hilkhot-ẓava), which recognize the right for self-defense, as well as the status of universally accepted moral norms. §3 unfolds pragmatic ethical premises of a humane techno-ethics, which are required for the identifying AWS as a moral question: I. Relationality; II. Technology is not (completely) neutral; III. The fallaciousness of transhumanism. It is argued that these premises are compatible with halakhic tradition. §4 investigates the question of the morality of AWS, within the field of military AI ethics. It is clarified why the standard categories of war-ethics (ad bello, in bellum) do not capture the singular ethical problem of AWS, which pertains to the operation of military means, rather than their human targets. It is argued that reductive perception of the human mind is misleading about the feasibility of ‘ethical robots’, capable of independent moral discretion. To provide a thick examination of human agency from the perspective of Jewish tradition, §5 explores two stories from the biblical book of Samuel (the murder of Nob’s priests and of Uriah). The lessons about the significance of moral agency within the pubic-political military sphere are made explicit, as well as the possible costs resulting from the loss of human agency in the case of AWS. Given that realpolitik considerations are basic in halakhah, §6 considers some possible contemporary socio-political implications of the AWS, that may risk the sustainability of the democratic project. §7 concludes by pointing out humane contributions of Jewish law to contemporary techno-ethics. (shrink)

Pranayamic breathing, defined as a manipulation of breath movement, has been shown to contribute to a physiologic response characterized by the presence of decreased oxygen consumption, decreased heart rate, and decreased blood pressure, as well as increased theta wave amplitude in EEG recordings, increased parasympathetic activity accompanied by the experience of alertness and reinvigoration. The mechanism of how pranayamic breathing interacts with the nervous system affecting metabolism and autonomic functions remains to be clearly understood. It is our hypothesis (...) that voluntary slow deep breathing functionally resets the autonomic nervous system through stretchinduced inhibitory signals and hyperpolarization currents propagated through both neural and non-neural tissue which synchronizes neural elements in the heart, lungs, limbic system and cortex. During inspiration, stretching of lung tissue produces inhibitory signals by action of slowly adapting stretch receptors (SARs) and hyperpolarization current by action of fibroblasts. Both inhibitory impulses and hyperpolarization current are known to synchronize neural elements leading to the modulation of the nervous system and decreased metabolic activity indicative of the parasympathetic state. In this paper we propose pranayama’s physiologic mechanism through a cellular and systems level perspective,involving both neural and non-neural elements. This theoretical description describes a common physiological mechanism underlying pranayama and elucidate the role of the respiratory and cardiovascular system on modulating the autonomic nervous system. Along with facilitating the design of clinical breathing techniques for the treatment of autonomic nervous system and other disorders, this model will also validate pranayama as a topic requiring more research. (shrink)

Our reflections on LAWS issues are the result of the work of our research group on AI and ethics at the Informatics Center in partnership with the Information Science Department, both from the Federal University of Pernambuco, Brazil. In particular, our propositions and provocations are tied to Bianca Ximenes’s ongoing doctoral thesis, advised by Prof. Geber Ramalho, from the area of computer science, and co-advised by Prof. Diego Salcedo, from the humanities. Our research group is interested in answering two tricky (...) questions: What would an ethical AI be? And how can one guarantee that a given intelligent system will follow intercultural human ethical principles? (shrink)

Lethal Autonomous Weapons (LAWs) are robotic weapons systems, primarily of value to the military, that could engage in offensive or defensive actions without human intervention. This paper assesses and engages the current arguments for and against the use of LAWs through the lens of achieving more ethical warfare. Specific interest is given particularly to ethical LAWs, which are artificially intelligent weapons systems that make decisions within the bounds of their ethics-based code. To ensure that a wide, but not exhaustive, survey (...) of the implications of employing such ethical devices to replace humans in warfare is taken into account, this paper will engage on matters related to current scholarship on the rejection or acceptance of LAWs—including contemporary technological shortcomings of LAWs to differentiate between targets and the behavioral and psychological volatility of humans—and current and proposed regulatory infrastructures for developing and using such devices. After careful consideration of these factors, this paper will conclude that only ethical LAWs should be used to replace human involvement in war, and, by extension of their consistent abilities, should remove humans from war until a more formidable discovery is made in conducting ethical warfare. (shrink)

One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents (IAs). This paper explores how decision matrix algorithms, via the belief-desire-intention model for (...) autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing for the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles. (shrink)

The application of modern technologies with artificial intelligence (AI) in all spheres of human life is growing exponentially alongside concern for its controllability. The lack of public, state, and international control over AI technologies creates large-scale risks of using such software and hardware that (un)intentionally harm humanity. The events of recent month and years, specifically regarding the Russian Federation’s war against its democratic neighbour Ukraine and other international conflicts of note, support the thesis that the uncontrolled use of AI, especially (...) in the military sphere, may lead to deliberate disregard for the moral standards of controlled AI or the spontaneous emergence of aggressive autonomous AI. The development of legal regulation for the use of technologies with AI is prolonged concerning the rapid development of these artefacts, which simultaneously cover all areas of public relations. Therefore, control over the creation and use of AI should be carried out not only by purely technical regulation (e.g., technical standards and conformance assessments, corporate and developer regulations, requirements enforced through industry-wide ethical codes); but also by comprehensive legislation and intergovernmental oversight bodies that codify and enforce specific changes in the rights and duties of legal persons. This article shall present the “Morality Problem” and “Intentionality Problem” of AI, and reflect upon various lacunae that arise when implementing AI for military purposes. (shrink)

In “Accepting Moral Responsibility for the Actions of Autonomous Weapons Systems—a Moral Gambit” (2022), Mariarosaria Taddeo and Alexander Blanchard answer one of the most vexing issues in current ethics of technology: how to close the so-called “responsibility gap”? Their solution is to require that autonomous weapons systems (AWSs) may only be used if there is some human being who accepts the ex ante responsibility for those actions of the AWS that could not have been predicted or intended (in such cases, (...) the human being takes what the authors call the “moral gambit”). The authors then propose several institutional safeguards to implement in order to ensure that the moral gambit is taken in a fair and just way. This paper explores this suggestion in the context of the institutional settings within which AWSs are most likely to be deployed. It raises some concerns as to the feasibility of Taddeo and Blanchard’s proposal, in light of the recent empirical work on the incentive structures likely to exist within militaries. It then presents a potential problem that may arise in case the accountability mechanisms are successfully implemented. (shrink)

Autonomous and automatic weapons would be fire and forget: you activate them, and they decide who, when and how to kill; or they kill at a later time a target you’ve selected earlier. Some argue that this sort of killing is always wrong. If killing is to be done, it should be done only under direct human control. (E.g., Mary Ellen O’Connell, Peter Asaro, Christof Heyns.) I argue that there are surprisingly many kinds of situation where this is false and (...) where the use of Automated Weapons Systems would in fact be morally required. These include cases where a) once one has activated a weapon expected then to behave lethally, it would be appropriate to let it continue because this is part of a plan whose goodness one was best positioned to evaluate before activating the weapon; b) one expects better long-term consequences from allowing it to continue; c) allowing it to continue would express a decision you made to be resolute, a decision that could not have advantaged you had it not been true that you would carry through with it; d) the weapon is mechanically not recallable, so that, to not allow it to carry through, you would have had to refrain from activating it in the first place, something you expected would have disastrous consequences; e) you must deputize necessary killings to autonomous machines in order to protect yourself from guilt you shouldn’t have to bear; f) it would be morally better for the burden of responsibility for the killing to be shared among several agents, and the agents deputizing killing to machines can do this, especially where it’s not predictable which machine will be successful; g) a killing would be morally better done with elements of randomness and lack of deliberation, and a (relatively stupid) machine could do this where a person could not; h) the machine would be acting as a Doomsday Device, so that it could not have had its hoped for deterrent effect had you not ensured that you would be unable to recall it if enemy action activated it; i) letting it carry through is a necessary part of its own learning process, and you expect that this learning will have salutary effects later on; j) human intervention in the machine’s operation would disastrously impair its precision, or its speed and efficiency; k) using non-automated methods would require human resources you just don’t have in a task that nevertheless must be done (e.g., using land-mines to protect remote installations); l) the weapon has such horrible and indiscriminate power that it is doubtful whether it could be actually used in ways compatible with International Humanitarian Law and the Laws of War, which require that weapons be used only in ways respecting distinctness, necessity and proportionality, but its threat of use could respect these principles in affording deterrence provided human error cannot lead to their accidental deployment, this requiring that they be controlled by carefully designed autonomous and automatic systems. I then consider objections based on conceptions of human dignity and find that very often dignity too is best served by autonomous machine killing. Examples include saving your village by activating a robot to kill invading enemies who would inflict great indignity on your village, using a suicide robot to save yourself from a less dignified death at enemy hands, using a robotic drone to kill someone otherwise not accessible in order to restore dignity to someone this person killed and to his family, and using a robot to kill someone who needs killing, but the killing of whom by a human executioner would soil the executioner’s dignity. I conclude that what matters in rightful killing isn’t necessarily that it be under the direct control of a human, but that it be under the control of morality; and that could sometimes require use of an autonomous or automated device. (This paper was formerly called "Fire and Forget: A Defense of the Use of Autonomous Weapons in War" on Philpapers; the current title is the title of the published version.). (shrink)

The international debate on the ethics and legality of autonomous weapon systems (AWS), along with the call for a ban, primarily focus on the nebulous concept of fully autonomous AWS. These are AWS capable of target selection and engagement absent human supervision or control. This paper argues that such a conception of autonomy is divorced from both military planning and decision-making operations; it also ignores the design requirements that govern AWS engineering and the subsequent tracking and tracing of moral responsibility. (...) To show how military operations can be coupled with design ethics, this paper marries two different kinds of meaningful human control (MHC) termed levels of abstraction. Under this two-tiered understanding of MHC, the contentious notion of ‘full’ autonomy becomes unproblematic. (shrink)

In this paper I provide an exposition and critique of Johnson and Noorman’s (2014) three conceptualizations of the agential roles artificial systems can play. I argue that two of these conceptions are unproblematic: that of causally efficacious agency and “acting for” or surrogate agency. Their third conception, that of “autonomous agency,” however, is one I have reservations about. The authors point out that there are two ways in which the term “autonomy” can be used: there is, firstly, the engineering sense (...) of the term, which simply refers to the ability of some system to act independently of substantive human control. Secondly, there is the moral sense of the term, which has traditionally grounded many notions of human mor-al responsibility. I argue that the continued usage of “autonomy” in discussions of artificial agency complicates matters unnecessarily. This occurs in two ways: firstly, the condition of autonomy, even in its engineering sense, fails to accurately describe the way “autonomous” systems are developed in practice. Secondly, the continued usage of autonomy in the moral sense introduces unnecessary meta-physical baggage form the free will debate into discussions about moral agency. In order to understand the debate surrounding autonomy, we would therefore first need to settle many seemingly intractable metaphysical questions regarding the existence of free will in human beings. (shrink)

Abstract: Many believe that, in addition to cognitive capacities, autonomous robots need something similar to affect. As in humans, affect, including specific emotions, would filter robot experience based on a set of goals, values, and interests. This narrows behavioral options and avoids combinatorial explosion or regress problems that challenge purely cognitive assessments in a continuously changing experiential field. Adding human-like affect to robots is not straightforward, however. Affect in organisms is an aspect of evolved biological systems, from the taxes of (...) single-cell organisms to the instincts, drives, feelings, moods, and emotions that focus human behavior through the mediation of hormones, pheromones, neurotransmitters, the autonomic nervous system, and key brain structures. We argue that human intelligence is intimately linked to biological affective systems and to the unique repertoire of potential behaviors, sometimes conflicting, they facilitate. Artificial affect is affect in name only and without genes and biological bodies, autonomous robots will lack the goals, interests, and value systems associated with human intelligence. We will take advantage of their general intelligence and expertise, but robots will not enter our intellectual world or apply for legal status in the community. -/- . (shrink)

Robert Sparrow (among others) claims that if an autonomous weapon were to commit a war crime, it would cause harm for which no one could reasonably be blamed. Since no one would bear responsibility for the soldier’s share of killing in such cases, he argues that they would necessarily violate the requirements of jus in bello, and should be prohibited by international law. I argue this view is mistaken and that our moral understanding of war is sufficient to determine blame (...) for any wrongful killing done by autonomous weapons. Analyzing moral responsibility for autonomous weapons starts by recognizing that although they are capable of causing moral consequences, they are neither praiseworthy nor blameworthy in the moral sense. As such, their military role is that of a tool, albeit a rather sophisticated one, and responsibility for their use is roughly analogous to that of existing “smart” weapons. There will likely be some difficulty in managing these systems as they become more intelligent and more prone to unpredicted behavior, but the moral notion of shared responsibility and the legal notion of command responsibility are sufficient to locate responsibility for their use. (shrink)

Robert Sparrow (among others) claims that if an autonomous weapon were to commit a war crime, it would cause harm for which no one could reasonably be blamed. Since no one would bear responsibility for the soldier’s share of killing in such cases, he argues that they would necessarily violate the requirements of jus in bello, and should be prohibited by international law. I argue this view is mistaken and that our moral understanding of war is sufficient to determine blame (...) for any wrongful killing done by autonomous weapons. Analyzing moral responsibility for autonomous weapons starts by recognizing that although they are capable of causing moral consequences, they are neither praiseworthy nor blameworthy in the moral sense. As such, their military role is that of a tool, albeit a rather sophisticated one, and responsibility for their use is roughly analogous to that of existing “smart” weapons. There will likely be some difficulty in managing these systems as they become more intelligent and more prone to unpredicted behavior, but the moral notion of shared responsibility and the legal notion of command responsibility are sufficient to locate responsibility for their use. (shrink)

Through modern driver assistant systems, algorithmic decisions already have a significant impact on the behavior of vehicles in everyday traffic. This will become even more prominent in the near future considering the development of autonomous driving functionality. The need to consider ethical principles in the design of such systems is generally acknowledged. However, scope, principles and strategies for their implementations are not yet clear. Most of the current discussions concentrate on situations of unavoidable crashes in which the life of human (...) beings is existentially affected. In this paper, we argue that ethical considerations should be mandatory for any algorithmic decision of autonomous vehicles, instead of a limitation to hazard situations. Such an ethically aligned behavior is relevant because autonomous vehicles, like any other traffic participants, operate in a shared public space, where every behavioral decision impacts the operational possibilities of others. These possibilities concern the fulfillment of a road-user’s safety, utility and comfort needs. We propose that, to operate ethically in such space, an autonomous vehicle will have to take its behavior decisions according to a just distribution of operational possibilities among all traffic participants. Using an application on a partially-autonomous prototype vehicle, we describe how to apply and implement concepts of distributive justice to the driving environment and demonstrate the impact on its behavior in comparison to an advanced but egoistic decision maker. (shrink)

AI systems, like self-driving cars, healthcare robots, or Autonomous Weapon Systems, already play an increasingly important role in our lives and will do so to an even greater extent in the near future. This raises a fundamental philosophical question: who is morally responsible when such systems cause unjustified harm? In the paper, we argue for the admittedly surprising claim that some of these systems can themselves be morally responsible for their conduct in an important and everyday sense of the term—the (...) attributability sense. More specifically, relying on work by Nomy Arpaly and Timothy Schroeder (In Praise of Desire, OUP 2014), we propose that the behavior of these systems can manifest their ‘quality of will’ and thus be regarded as something they can be blameworthy for. We develop this position in detail, justify some of its crucial presuppositions, and defend it against potential objections. (shrink)

With recent advancements in systems engineering and artificial intelligence, autonomous agents are increasingly being called upon to execute tasks that have normative relevance. These are tasks that directly—and potentially adversely—affect human well-being and demand of the agent a degree of normative-sensitivity and -compliance. Such norms and normative principles are typically of a social, legal, ethical, empathetic, or cultural nature. Whereas norms of this type are often framed in the abstract, or as high-level principles, addressing normative concerns in concrete applications of (...) autonomous agents requires the refinement of normative principles into explicitly formulated practical rules. This paper develops a process for deriving specification rules from a set of high-level norms, thereby bridging the gap between normative principles and operational practice. This enables autonomous agents to select and execute the most normatively favourable action in the intended context premised on a range of underlying relevant normative principles. In the translation and reduction of normative principles to SLEEC rules, we present an iterative process that uncovers normative principles, addresses SLEEC concerns, identifies and resolves SLEEC conflicts, and generates both preliminary and complex normatively-relevant rules, thereby guiding the development of autonomous agents and better positioning them as normatively SLEEC-sensitive or SLEEC-compliant. (shrink)

The debate on the ethical permissibility of autonomous weapon systems (AWS) is deadlocked. It could therefore benefit from a differentiated assignment of the burden of proof. This is because the discussion is not purely philosophical in nature, but has a legal and security policy component and aims to avoid the most harmful outcomes of an otherwise unchecked development. Opponents of a universal AWS ban must clearly demonstrate that AWS comply with the Law of Armed Conflict (LOAC). This requires extensive testing (...) of specific models to verify compliance in realistic combat situations; working on appropriate testing regimes is also one of the most important future tasks for AWS ethics. In this context, however, it is easy to refute the claim put forward by prohibitionists that AWS violate LOAC a priori. A state’ s right to ensure security and the protection of human rights through military capacity has a high ethical value. Other arguments against the use of AWS that are not based on LOAC conformity must therefore meet a substantial burden of proof, as they call on governments to abandon a military technology which is at least potentially transformative. None of those objections meet this standard. They include not only culturally or even individually divergent norms and ideas regarding the use of weapons, but also consequentialist (security policy) concerns on the one hand and so-called deontological arguments on the other. While the former at least point to the need for essential regulation, the latter are based on rather dubious premises. For the purposes of solutions beneficial to humanity, prohibitionist should formulate narrower and more specific points of criticism. (shrink)

Recently criticisms against autonomous weapons were presented in a video in which an AI-powered drone kills a person. However, some said that this video is a distraction from the real risk of AI—the risk of unlimitedly self-improving AI systems. In this article, we analyze arguments from both sides and turn them into conditions. The following conditions are identified as leading to autonomous weapons becoming a global catastrophic risk: 1) Artificial General Intelligence (AGI) development is delayed relative to progress in narrow (...) AI and manufacturing. 2) The potential for very cheap manufacture of drones, with prices below 1 USD each. 3) Anti-drone defense capabilities lagging offensive development. 4) Special global military posture encouraging development of drone swarms as a strategic offensive weapon, able to kill civilians. We conclude that while it is unlikely that drone swarms alone will become existential risk, lethal autonomous weapons could contribute to civilizational collapse in case of new world war. (shrink)

A. Newell and H. A. Simon were two of the most influential scientists in the emerging field of artificial intelligence (AI) in the late 1950s through to the early 1990s. This paper reviews their crucial contribution to this field, namely to symbolic AI. This contribution was constituted mostly by their quest for the implementation of general intelligence and (commonsense) knowledge in artificial thinking or reasoning artifacts, a project they shared with many other scientists but that in their case was theoretically (...) based on the idiosyncratic notions of symbol systems and the representational abilities they give rise to, in particular with respect to knowledge. While focusing on the period 1956-1982, this review cites both earlier and later literature and it attempts to make visible their potential relevance to today's greatest unifying AI challenge, to wit, the design of wholly autonomous artificial agents (a.k.a. robots) that are not only rational and ethical, but also self-conscious. (shrink)

A body schema is an agent's model of its own body that enables it to act on affordances in the environment. This paper presents a body schema system for the Learning Intelligent Decision Agent (LIDA) cognitive architecture. LIDA is a conceptual and computational implementation of Global Workspace Theory, also integrating other theories from neuroscience and psychology. This paper contends that the ‘body schema' should be split into three separate functions based on the functional role of consciousness in Global Workspace (...) Theory. There is (1) an online model of the agent's effectors and effector variables (Current Body Schema), (2) a long-term, recognitional storage of embodied capacities for action and affordances (Habitual Body Schema), and (3) "dorsal" stream information feeding directly from early perception to sensorimotor processes (Online Body Schema). This paper then discusses how the LIDA model of the body schema explains several experiments in psychology and ethology. (shrink)

English - language Wikipedia is constantly being plagued by vandalistic contributions on a massive scale. In order to fight them its volunteer contributors deploy an array of software tools and autonomous bots. After an analysis of their functioning and the ‘ coactivity ’ in use between humans and bots, this research ‘ discloses ’ the moral issues that emerge from the combined patrolling by humans and bots. Administrators provide the stronger tools only to trusted users, thereby creating a new hierarchical (...) layer. Further, surveillance exhibits several troubling features : questionable profiling practices, the use of the controversial measure of reputation, ‘ oversurveillance ’ where quantity trumps quality, and a prospective loss of the required moral skills whenever bots take over from humans. The most troubling aspect, though, is that Wikipedia has become a Janus - faced institution. One face is the basic platform of MediaWiki software, transparent to all. Its other face is the anti - vandalism system, which, in contrast, is opaque to the average user, in particular as a result of the algorithms and neural networks in use. Finally it is argued that this secrecy impedes a much needed discussion to unfold ; a discussion that should focus on a ‘ rebalancing ’ of the anti - vandalism system and the development of more ethical information practices towards contributors. (shrink)

Autonomy is often considered a core value of Western society that is deeply entrenched in moral, legal, and political practices. The development and deployment of artificial intelligence (AI) systems to perform a wide variety of tasks has raised new questions about how AI may affect human autonomy. Numerous guidelines on the responsible development of AI now emphasise the need for human autonomy to be protected. In some cases, this need is linked to the emergence of increasingly ‘autonomous’ AI systems that (...) can perform tasks without human control or supervision. Do such ‘autonomous’ systems pose a risk to our own human autonomy? In this article, I address the question of a trade-off between human autonomy and system ‘autonomy’. (shrink)

This article provides an answer to the question: What is the function of cognition? By answering this question it becomes possible to investigate what are the simplest cognitive systems. It addresses the question by treating cognition as a solution to a design problem. It defines a nested sequence of design problems: (1) How can a system persist? (2) How can a system affect its environment to improve its persistence? (3) How can a system utilize better information from (...) the environment to select better actions? And, (4) How can a system reduce its inherent informational limitations to achieve more successful behavior? This provides a corresponding nested sequence of system classes: (1) autonomous systems, (2) (re)active autonomous systems, (3) informationally controlled autonomous systems (autonomous agents), and (4) cognitive systems. -/- This article provides the following characterization of cognition: The cognitive system is the set of mechanisms of an autonomous agent that (1) allow increase of the correlation and integration between the environment and the information system of the agent, so that (2) the agent can improve the selection of actions and thereby produce more successful behavior. -/- Finally, it shows that common cognitive capacities satisfy the characterization: learning, memory, representation, decision making, reasoning, attention, and communication. (shrink)

In this article, we propose some fundamental requirements for the appearance of adaptivity. We argue that a basic metabolic organization, taken in its minimal sense, may provide the conceptual framework for naturalizing the origin of teleology and normative functionality as it appears in living systems. However, adaptivity also requires the emergence of a regulatory subsystem, which implies a certain form of dynamic decoupling within a globally integrated, autonomous system. Thus, we analyze several forms of minimal adaptivity, including the special (...) case of motility. We go on to explain how an open-ended complexity growth of motility-based adaptive agency, namely, behavior, requires the appearance of the nervous system. Finally, we discuss some implications of these ideas for embodied robotics. (shrink)

We propose a domain-level ontology of plays for the facilitation of play-based collaborative autonomy among unmanned and manned-unmanned aircraft teams in the Army’s Unmanned Aircraft System (UAS) mission domain. We define a play as a type of plan that prescribes some pattern of intentional acts that are intended to reliably result in some goal in some competitive context, and which specifies one or more roles that are realized by those prescribed intentional acts. The ontology is well suited to be (...) extended to other types of military and nonmilitary unmanned vehicle operations. (shrink)

Interaction between drivers and pedestrians is often facilitated by informal communicative cues, like hand gestures, facial expressions, and eye contact. In the near future, however, when semi- and fully autonomous vehicles are introduced into the traffic system, drivers will gradually assume the role of mere passengers, who are casually engaged in non-driving-related activities and, therefore, unavailable to participate in traffic interaction. In this novel traffic environment, advanced communication interfaces will need to be developed that inform pedestrians of the current (...) state and future behavior of an autonomous vehicle, in order to maximize safety and efficiency for all road users. The aim of the present review is to provide a comprehensive account of empirical work in the field of external human–machine interfaces for autonomous vehicle-to- pedestrian communication. In the great majority of covered studies, participants clearly benefited from the presence of a communication interface when interacting with an autonomous vehicle. Nevertheless, standardized interface evaluation procedures and optimal interface specifications are still lacking. (shrink)