Autonomous Weapons

International regulation of autonomous weapon systems (AWS) is increasingly conceived as an exercise in risk management. This requires a shared approach for assessing the risks of AWS. This paper presents a structured approach to risk assessment and regulation for AWS, adapting a qualitative framework inspired by the Intergovernmental Panel on Climate Change (IPCC). It examines the interactions among key risk factors—determinants, drivers, and types—to evaluate the risk magnitude of AWS and establish risk tolerance thresholds through a risk matrix informed by (...) background knowledge of event likelihood and severity. Further, it proposes a methodology to assess community risk appetite, emphasizing that such assessments and resulting tolerance levels should be determined through deliberation in a multistakeholder forum. The paper highlights the complexities of applying risk-based regulations to AWS internationally, particularly the challenge of defining a global community for risk assessment and regulatory legitimization. (shrink)

Danaher (2016) has argued that increasing robotization can lead to retribution gaps: Situation in which the normative fact that nobody can be justly held responsible for a harmful outcome stands in conflict with our retributivist moral dispositions. In this paper, we report a cross-cultural empirical study based on Sparrow’s (2007) famous example of an autonomous weapon system committing a war crime, which was conducted with participants from the US, Japan and Germany. We find that (i) people manifest a considerable willingness (...) to hold autonomous systems morally responsible, (ii) partially exculpate human agents when interacting with such systems, and that more generally (iii) the possibility of normative responsibility gaps is indeed at odds with people’s pronounced retributivist inclinations. We discuss what these results mean for potential implications of the retribution gap and other positions in the responsibility gap literature. (shrink)

As artificial intelligence becomes more sophisticated and robots approach autonomous decision-making, debates about how to assign moral responsibility have gained importance, urgency, and sophistication. Answering Stenseke’s (2022a) call for scaffolds that can help us classify views and commitments, we think the current debate space can be represented hierarchically, as answers to key questions. We use the resulting taxonomy of five stances to differentiate—and defend—what is known as the “blank check” proposal. According to this proposal, a person activating a robot could (...) willingly make themselves answerable for whatever events ensue, even if those events stem from the robot’s autonomous decision(s). This blank check solution was originally proposed in the context of automated warfare (Champagne & Tonkens, 2015), but we extend it to cover all robots. We argue that, because moral answerability in the blank check is accepted voluntarily and before bad outcomes are known, it proves superior to alternative ways of assigning blame. We end by highlighting how, in addition to being just, this self-initiated and prospective moral answerability for robot harm provides deterrence that the four other stances cannot match. (shrink)

In “Accepting Moral Responsibility for the Actions of Autonomous Weapons Systems—a Moral Gambit” (2022), Mariarosaria Taddeo and Alexander Blanchard answer one of the most vexing issues in current ethics of technology: how to close the so-called “responsibility gap”? Their solution is to require that autonomous weapons systems (AWSs) may only be used if there is some human being who accepts the ex ante responsibility for those actions of the AWS that could not have been predicted or intended (in such cases, (...) the human being takes what the authors call the “moral gambit”). The authors then propose several institutional safeguards to implement in order to ensure that the moral gambit is taken in a fair and just way. This paper explores this suggestion in the context of the institutional settings within which AWSs are most likely to be deployed. It raises some concerns as to the feasibility of Taddeo and Blanchard’s proposal, in light of the recent empirical work on the incentive structures likely to exist within militaries. It then presents a potential problem that may arise in case the accountability mechanisms are successfully implemented. (shrink)

I here consider the reasonableness of punishing future autonomous military robots. I argue that it is an engineering desideratum that these devices be responsive to moral considerations as well as human criticism and blame. Additionally, I argue that someday it will be possible to build such machines. I use these claims to respond to the no subject of punishment objection to deploying autonomous military robots, the worry being that an “accountability gap” could result if the robot committed a war crime. (...) (Sparrow 2007). But these robots will be blameworthy and morally responsible for their wrongful conduct. I then show that we have reasons to punish them based both on good effects and the impact on the robot, itself. (shrink)

Anticiper les épidémies est le souhait le plus vif de toutes les institutions qui veillent sur la santé publique. Un raisonnement naïf permet de penser qu’il suffit de faire évoluer en laboratoire un organisme potentiellement pathogène pour savoir comment sa descendance pourra devenir plus virulente pour l’homme. Il est alors facile de faire croître cet organisme sur des cellules humaines, qu’il infecte mal pour commencer, puis de retenir ses descendants au fur et à mesure qu’ils deviennent plus infectieux. La vision (...) optimiste affirme alors que les caractériser permettra de construire des défenses appropriées. Bien des variations sur ce thème sont possibles, y compris avec l’aide de la synthèse artificielle de variants dont on saurait imaginer la pathogénicité pour l’homme. Depuis plus de deux décennies, des chercheurs développent des expériences de ce type, dite de « gain de fonction » (GdF). Leur interprétation repose sur l’illusion que nous savons prévoir l’évolution des êtres vivants. En réalité, ceux que nous observons aujourd’hui ont été sélectionnés au cours du temps précisément parce qu’ils conduisent toujours à des comportements imprévisibles. Une règle clé de l’évolution est que la vie, pour durer dans des circonstances qui ne peuvent être prévues, doit sans cesse faire émerger des formes innovantes. À ce caractère propre à la vie s’ajoute un imprévisible lié à l’impéritie humaine, cause de l’omniprésence des accidents de laboratoire. Or, en virologie spécialement, l’accident est la règle et non l’exception. Il faut ajouter que le caractère public de ces expériences les rend utilisables par toutes les structures malveillantes qui sont hélas répandues dans le monde. Il n’est pas difficile alors d’imaginer les conséquences catastrophiques des expériences de GdF. Nous proposons donc qu’elles soient purement et simplement interdites et que toute transgression soit sévèrement punie, en frappant d’abord au porte-monnaie, ce qui semble être la meilleure façon de faire dans nos sociétés devenues universellement vénales. (shrink)

Dans un monde dominé par les conflits, il arrive qu’il faille se défendre. L’invention des armes a évolué en parallèle avec le savoir technique, puis scientifique, et il est même arrivé que la recherche d’applications militaires ait joué un rôle moteur dans la découverte scientifique. Lorsque les armes ne servent pas à attaquer d’autres nations, leur fabrication et leur commerce se justifient morale- ment. Il faut cependant qu’elles ne puissent échapper à ceux qui les construisent et doivent en contrôler la (...) diffusion. Facilement mise en œuvre, la guerre utilisant des organismes vivants ou ce qu’ils pro- duisent comme armes est très ancienne. Or l’originalité des êtres vivants est qu’ils se multiplient, qu’un seul individu suffit à recouvrir la Terre de ses descendants. La biologie ne se prête jamais à un usage militaire moralement défendable parce que le confinement au champ de bataille est impossible. De ce fait, les applications de la biologie ont été vite écartées du domaine de la Défense permise par les traités internationaux. La Convention de 1972 sur l’Interdiction des Armes Biologiques et à Toxines (CIABT), entrée en vigueur en 1975, est aujourd’hui signée par 183 pays (ratifiée par 175) sur un total de 193, sans compter Taïwan qui l’a ratifiée mais n’est pas reconnu par l’ONU. Elle interdit la mise au point, la fabrication, l’acquisition, le transfert, la conservation, le stockage et l’utilisation des armes bio- logiques et à toxines. Elle constitue un élément essentiel de la lutte contre la prolifération des armes de destruction massive. (shrink)

Technological progress has brought about the emergence of machines that have the capacity to take human lives without human control. These represent an unprecedented threat to humankind. This paper starts from the example of chemical weapons, now banned worldwide by the Geneva protocol, to illustrate how technological development initially aimed at the benefit of humankind has, ultimately, produced what is now called the “Weaponization of Artificial Intelligence (AI)”. Autonomous Weapon Systems (AWS) fail the so-called discrimination principle, yet, the wider public (...) is largely unaware of this problem. Given that ongoing scientific research on AWS, performed in the military sector, is generally not made available to the public domain, many of the viewpoints on this subject, expressed across different media, invoke common sense rather than scientific evidence. Yet, the implications of a potential weaponization of our work as scientists, especially in the field of AI, are reaching further than some may think. The potential consequences of a deployment of AWS for citizen stakeholders are incommensurable, and it is time to raise awareness in the public domain of the kind of potential threats identified, and to encourage legal policies ensuring that these threats will not materialize. (shrink)

The ethical Principle of Proportionality requires combatants not to cause collateral harm excessive in comparison to the anticipated military advantage of an attack. This principle is considered a major (and perhaps insurmountable) obstacle to ethical use of autonomous weapon systems (AWS). This article reviews three possible solutions to the problem of achieving Proportionality compliance in AWS. In doing so, I describe and discuss the three components Proportionality judgments, namely collateral damage estimation, assessment of anticipated military advantage, and judgment of “excessiveness”. (...) Some possible approaches to Proportionality compliance are then presented, such as restricting AWS operations to environments lacking civilian presence, using AWS in targeted strikes in which proportionality judgments are pre-made by human commanders, and a ‘price tag’ approach of pre-assigning acceptable collateral damage values to military hardware in conventional attritional warfare. The article argues that application of these three compliance methods would result in AWS’ achieving acceptable Proportionality compliance levels in many combat environments and scenarios, allowing AWS to perform most key tasks in conventional warfare. (shrink)

This paper presents a new trilemma with respect to resolving the control and alignment problems in machine ethics. Section 1 outlines three possible types of artificial moral agents (AMAs): (1) 'Inhuman AMAs' programmed to learn or execute moral rules or principles without understanding them in anything like the way that we do; (2) 'Better-Human AMAs' programmed to learn, execute, and understand moral rules or principles somewhat like we do, but correcting for various sources of human moral error; and (3) 'Human-Like (...) AMAs' programmed to understand and apply moral values in broadly the same way that we do, with a human-like moral psychology. Sections 2–4 then argue that each type of AMA generates unique control and alignment problems that have not been fully appreciated. Section 2 argues that Inhuman AMAs are likely to behave in inhumane ways that pose serious existential risks. Section 3 then contends that Better-Human AMAs run a serious risk of magnifying some sources of human moral error by reducing or eliminating others. Section 4 then argues that Human-Like AMAs would not only likely reproduce human moral failures, but also plausibly be highly intelligent, conscious beings with interests and wills of their own who should therefore be entitled to similar moral rights and freedoms as us. This generates what I call the New Control Problem: ensuring that humans and Human-Like AMAs exert a morally appropriate amount of control over each other. Finally, Section 5 argues that resolving the New Control Problem would, at a minimum, plausibly require ensuring what Hume and Rawls term ‘circumstances of justice’ between humans and Human-Like AMAs. But, I argue, there are grounds for thinking this will be profoundly difficult to achieve. I thus conclude on a skeptical note. Different approaches to developing ‘safe, ethical AI’ generate subtly different control and alignment problems that we do not currently know how to adequately resolve, and which may or may not be ultimately surmountable. (shrink)

The application of modern technologies with artificial intelligence (AI) in all spheres of human life is growing exponentially alongside concern for its controllability. The lack of public, state, and international control over AI technologies creates large-scale risks of using such software and hardware that (un)intentionally harm humanity. The events of recent month and years, specifically regarding the Russian Federation’s war against its democratic neighbour Ukraine and other international conflicts of note, support the thesis that the uncontrolled use of AI, especially (...) in the military sphere, may lead to deliberate disregard for the moral standards of controlled AI or the spontaneous emergence of aggressive autonomous AI. The development of legal regulation for the use of technologies with AI is prolonged concerning the rapid development of these artefacts, which simultaneously cover all areas of public relations. Therefore, control over the creation and use of AI should be carried out not only by purely technical regulation (e.g., technical standards and conformance assessments, corporate and developer regulations, requirements enforced through industry-wide ethical codes); but also by comprehensive legislation and intergovernmental oversight bodies that codify and enforce specific changes in the rights and duties of legal persons. This article shall present the “Morality Problem” and “Intentionality Problem” of AI, and reflect upon various lacunae that arise when implementing AI for military purposes. (shrink)

Please contact me at [email protected] if you are interested in reading a particular chapter or being sent the entire manuscript for private use. -/- The thesis offers a comprehensive argument in favor of a regulationist approach to autonomous weapon systems (AWS). AWS, defined as all military robots capable of selecting or engaging targets without direct human involvement, are an emerging and potentially deeply transformative military technology subject to very substantial ethical controversy. AWS have both their enthusiasts and their detractors, prominently (...) advocating for a global preemptive ban on AWS development and use. Rejecting both positions, the author outlines a middle-of-the-road regulationist approach that is neither overly restrictive nor overly permissive. The disqualifying flaws of the rival prohibitionist approach are demonstrated in the process. After defining the core term of autonomy in weapon systems, the practical difficulties involved in applying an arms control regime to AWS are analyzed. The analysis shows that AWS are an extremely regulation-resistant technology. This feature when combined with their assumed high military utility makes a ban framework extremely costly to impose and enforce. As such it is ultimately very likely to fail at the benefit of the most unscrupulous international actors and at a very substantial risk to those abiding with international law. Consequently, to be ethically viable, a prohibitionist framework would need to offer substantial moral benefits impossible to attain through the rival regulationist approach. The remainder of the thesis undertakes to demonstrate that this is not the case. Comparing the considerations of military and strategic necessity to humanitarian concerns most commonly voiced by prohibitionists requires finding a common denominator for all values being referred to. Consequently, the thesis proceeds to show that both kinds of concerns are ultimately reducible to respect for basic human rights of all stakeholders, and so that the prohibitionist and regulationist approach may ultimately be compared in terms of consequences their adoption would generate for basic human rights realization. The author then evaluates both the potential humanitarian benefits, and the potential humanitarian hazards of AWS introduction. The benefits of leaving frontline combat to machines are outlined, with the unique kinds of suffering that would be abolished by such a development being described in detail. The arguments against AWS adoption are then divided into three classes: arguments related to alleged impossibility of compliance with The Laws of Armed Conflict, non-consequentialist and broad consequentialist arguments. This analysis, which comprises the greater part of the entire thesis, shows that the concerns behind compliance arguments are indeed substantial and have to be accommodated via a complex framework of best practices, regulations and localized restrictions on some kinds of AWS or AWS use in particular environments. They do not, however, justify a universal ban on using all the diverse forms of AWS in all environments. Non-consequentialist objections are found either reducible to other classes of arguments or thoroughly unconvincing, sometimes to the point of being actually vacuous. Broad consequentialist concerns are likewise found to be accommodable by regulation, empirically unfounded or causally disconnected from the actions of legitimate actors acquiring AWS, and therefore irrelevant to the moral permissibility of such actions. The author concludes that the proponents of prohibitionism are unable to point to moral benefits substantial enough to justify the costs and risks inherent in the approach. A global ban is, in fact, likely to have a worse humanitarian impact that well-regulated AWS adoption even if the strategic risks are disregarded. On the other hand, the analysis shows that there indeed exists an urgent need to regulate AWS through a variety of technological, procedural and legal solutions. These include, but are not limited to, a temporary moratorium on anti-personnel AWS use, development of internationally verified compliance software and eventual legal requirement of its employment, a temporary moratorium on AWS proliferation to state actors and a ban on their proliferation to non-state agents. (shrink)

The 2016-17 Iraqi offensive that recaptured the city of Mosul from the Islamic State have demonstrated the inability of contemporary armed forces to retake urban areas from a determined and ruthless enemy without either suffering debilitating casualties or causing thousands of civilian deaths and virtually destroying the city itself. The enemy’s willingness to refuse civilian evacuation via a humanitarian corridor and effectively take the inhabitants hostage is all it takes to impose this tragic dilemma on an attacking force. The civilian (...) death toll of the battle of Mosul alone is estimated to have reached between seven and forty thousand, with almost one million people internally displaced and large portions of the city levelled. Avoiding another such tragedy should be a priority for all practitioners and theorists of urban warfare. Indeed, confronting and defeating this challenge may be the most important task for ethics, theory and practice of urban combat in the coming decade. I argue that the only viable way of escaping the hostage siege dilemma consists of incorporating large numbers of unmanned and autonomous weapon systems into the blue force structure and using these to substitute for artillery and aerial fires presently being used to overpower defenders. While other measures may be used to marginally improve the outcome of hostage city assaults, only the automatization of the assault can dramatically change the calculus and in result either discourage rogue actors from creating hostage city dilemmas or defeat them decisively while suffering unequivocally proportional casualties. I proceed as follows: first, I provide an account of the 2016-17 Coalition assault on Mosul, giving special attention to the events and features relevant to the hostage city dilemma. I then use this case study to introduce the theoretical concept of the hostile city and the ethical dilemma it creates, discussing possible objections and making the case that the circumstances of the Battle of Mosul are bound to repeatedly transpire in the future. This generates a duty to find a way out of the hostage city dilemma. I argue that widespread use of unmanned and autonomous weapon platforms constitutes a viable way of assaulting hostage cities in a manner that assures respect for the ius in bello principles, proposing several ways in which such weapons may be designed and used to drastically limit both military and civilian casualties. Concluding that it may be not only permissible but morally required to employ autonomous weapons in the hostage city scenario, I briefly discuss the implications of this conclusion for a wider debate concerning the moral feasibility of autonomous weapons. (shrink)

Autonomous weapons systems, often referred to as ‘killer robots’, have been a hallmark of popular imagination for decades. However, with the inexorable advance of artificial intelligence systems (AI) and robotics, killer robots are quickly becoming a reality. These lethal technologies can learn, adapt, and potentially make life and death decisions on the battlefield with little-to-no human involvement. This naturally leads to not only legal but ethical concerns as to whether we can meaningful control such machines, and if so, then how. (...) Such concerns are made even more poignant by the ever-present fear that something may go wrong, and the machine may carry out some action(s) violating the ethics or laws of war. -/- Researchers, policymakers, and designers are caught in the quagmire of how to approach these highly controversial systems and to figure out what exactly it means to have meaningful human control over them, if at all. -/- In Designed for Death, Dr Steven Umbrello aims to not only produce a realistic but also an optimistic guide for how, with human values in mind, we can begin to design killer robots. Drawing on the value sensitive design (VSD) approach to technology innovation, Umbrello argues that context is king and that a middle path for designing killer robots is possible if we consider both ethics and design as fundamentally linked. Umbrello moves beyond the binary debates of whether or not to prohibit killer robots and instead offers a more nuanced perspective of which types of killer robots may be both legally and ethically acceptable, when they would be acceptable, and how to design for them. (shrink)

Efforts to ban Autonomous Weapon Systems were both unsuccessful and controversial. Simultaneously the need to address the detrimental aspects of AWS development and proliferation continues to grow in scope and urgency. The article presents several regulatory solutions capable of addressing the issue while simultaneously respecting the requirements of military necessity and so attracting a broad consensus. Two much stricter solutions – regional AWS bans and adoption of a no first use policy – are also presented as fallback strategies in case (...) achieving AWS’ compliance with the Laws of Armed Conflict proved elusive. Together, the solutions presented form an outline of a flexible regulatory strategy able to adjust to different technological outcomes and providing a sensible compromise to solve the current deadlock on the AWS issue. (shrink)

This chapter considers how the adoption of autonomous weapons systems (AWS) may affect jus ad bellum principles of warfare. In particular, it focuses on the use of AWS in non-international armed conflicts (NIAC). Given the proliferation of NIAC, the development and use of AWS will most likely be attuned to this specific theater of war. As warfare waged by modernized liberal democracies (those most likely to develop and employ AWS at present) increasingly moves toward a model of individualized warfare, how, (...) if at all, will the principles by which we measure the justness of the commencement of such hostilities be affected by the introduction of AWS, and how will such hostilities stack up to current legal agreements surrounding their more traditional engagement? This chapter claims that such considerations give us reason to question the moral and legal necessity of ad bellum proper authority. (shrink)

Our reflections on LAWS issues are the result of the work of our research group on AI and ethics at the Informatics Center in partnership with the Information Science Department, both from the Federal University of Pernambuco, Brazil. In particular, our propositions and provocations are tied to Bianca Ximenes’s ongoing doctoral thesis, advised by Prof. Geber Ramalho, from the area of computer science, and co-advised by Prof. Diego Salcedo, from the humanities. Our research group is interested in answering two tricky (...) questions: What would an ethical AI be? And how can one guarantee that a given intelligent system will follow intercultural human ethical principles? (shrink)

Arguments from human dignity feature prominently in the Lethal Autonomous Weapons moral feasibility debate, even though their exists considerable controversy over their role and soundness and the notion of dignity remains under-defined. Drawing on the work of Dieter Birnbacher, I fix the sub-discourse as referring to the essential value of human persons in general, and to postulated moral rights of combatants not covered within the existing paradigm of the International Humanitarian Law in particular. I then review and critique dignity-based arguments (...) against LAWS: argument from faulty targeting process, argument from objectification, argument from underappreciation of the value of human life and the argument from the absence of mercy. I conclude that the argument from the absence of mercy is the only dignity-based argument that is both valid and irreducible to another class of arguments within the debate, and that it offers insufficient justification for a global ban on LAWS. (shrink)

A review and commentary on Killing By Remote Control: the Ethics of an Unmanned Military, edited by Bradley Jay Strawser (forward by Jeff McMahan), (Oxford University Press). -/- .