This paper discusses the epistemic status of biology from the standpoint of the systemic approach to living systems based on the notion of biological autonomy. This approach aims to provide an understanding of the distinctive character of biological systems and this paper analyses its theoretical and epistemological dimensions. The paper argues that, considered from this perspective, biological systems are examples of emergent phenomena, that the biological domain exhibits special features with respect to other domains, and that biology (...) as a discipline employs some core concepts, such as teleology, function, regulation among others, that are irreducible to those employed in physics and chemistry. It addresses the claim made by Jacques Monod that biology as a science is marginal. It argues that biology is general insofar as it constitutes a paradigmatic example of complexity science, both in terms of how it defines the theoretical object of study and of the epistemology and heuristics employed. As such, biology may provide lessons that can be applied more widely to develop an epistemology of complex systems. (shrink)

When a computer system causes harm, who is responsible? This question has renewed significance given the proliferation of autonomous systems enabled by modern artificial intelligence techniques. At the root of this problem is a philosophical difficulty known in the literature as the responsibility gap. That is to say, because of the causal distance between the designers of autonomous systems and the eventual outcomes of those systems, the dilution of agency within the large and complex teams (...) that design autonomous systems, and the impossibility of fully predicting how autonomous systems will behave once deployed, determining who is morally responsible for harms caused by autonomous systems is unclear at a conceptual level. I review past work on this topic, criticizing prior works for suggesting workarounds rather than philosophical answers to the conceptual problem presented by the responsibility gap. The view I develop, drawing on my earlier work on vicarious moral responsibility, explains why computing professionals are ethically required to take responsibility for the systems they design, despite not being blameworthy for the harms these systems may cause. (shrink)

Technological developments involving robotics and artificial intelligence devices are being employed evermore in elderly care and the healthcare sector more generally, raising ethical issues and practical questions warranting closer considerations of what we mean by “care” and, subsequently, how to design such software coherently with the chosen definition. This paper starts by critically examining the existing approaches to the ethical design of care robots provided by Aimee van Wynsberghe, who relies on the work on the ethics of care by Joan (...) Tronto. In doing so, it suggests an alternative to their non-principled approach, an alternative suited to tackling some of the issues raised by Tronto and van Wynsberghe, while allowing for the inclusion of two orientative principles. Our proposal centres on the principles of autonomy and vulnerability, whose joint adoption we deem able to constitute an original revision of a bottom-up approach in care ethics. Conclusively, the ethical framework introduced here integrates more traditional approaches in care ethics in view of enhancing the debate regarding the ethical design of care robots under a new lens. (shrink)

There are unusual challenges in ethics for RAS. Perhaps the issue can best be summarised as needing to consider “technically informed ethics”. The technology of RAS raises issues that have an ethical dimension, and perhaps uniquely so due to the possibility of moving human decision-making which is implicitly ethically informed to computer systems. Further, if seeking solutions to these problems – ethically aligned design, to use the IEEE’s terminology – then the solutions must be technically meaningful, capable of realisation, (...) capable of assurance, and suitable as a basis for regulation. Thus, ethics for RAS is a rich, complex multi-disciplinary concern, and perhaps more complex than many other ethical issues facing society today. It is also fast-moving. This paper has endeavoured to give an accessible introduction to some of the key issues, noting that many of them are quite subtle, and it is not possible to do them full justice in such a short document. However, we have sought to counterbalance this by giving an extensive list of initiatives, standards, etc. that focus on ethics of RAS and AI, see Annex A. (shrink)

To many, the idea of autonomous weapons systems (AWS) killing human beings is grotesque. Yet critics have had difficulty explaining why it should make a significant moral difference if a human combatant is killed by an AWS as opposed to being killed by a human combatant. The purpose of this paper is to explore the roots of various deontological concerns with AWS and to consider whether these concerns are distinct from any concerns that also apply to long- distance, (...) human-guided weaponry. We suggest that at least one major driver of the intuitive moral aversion to lethal AWS is that their use disrespects their human targets by violating the martial contract between human combatants. On our understanding of this doctrine, service personnel cede a right not to be directly targeted with lethal violence to other human agents alone. Artificial agents, of which AWS are one example, cannot understand the value of human life. A human combatant cannot transfer his privileges of targeting enemy combatants to a robot. Therefore, the human duty-holder who deploys AWS breaches the martial contract between human combatants and disrespects the targeted combatants. We consider whether this novel deontological objection to AWS forms the foundation of several other popular yet imperfect deontological objections to AWS. (shrink)

Autonomous weapons systems (AWS), sometimes referred to as “killer robots”, are receiving evermore attention, both in public discourse as well as by scholars and policymakers. Much of this interest is connected with emerging ethical and legal problems linked to increasing autonomy in weapons systems, but there is a general underappreciation for the ways in which existing law might impact on these new technologies. In this paper, we argue that as AWS become more sophisticated and increasingly more capable (...) than flesh-and-blood soldiers, it will increasingly be the case that such soldiers are “in the power” of those AWS which fight against them. This implies that such soldiers ought to be considered hors de combat, and not targeted. In arguing for this point, we draw out a broader conclusion regarding hors de combat status, namely that it must be viewed contextually, with close reference to the capabilities of combatants on both sides of any discreet engagement. Given this point, and the fact that AWS may come in many shapes and sizes, and can be made for many different missions, we argue that each particular AWS will likely need its own standard for when enemy soldiers are deemed hors de combat. We conclude by examining how these nuanced views of hors de combat status might impact on meaningful human control of AWS. (shrink)

In October 2011, the “2nd European Network for Cognitive Systems, Robotics and Interaction”, EUCogII, held its meeting in Groningen on “Autonomous activity in real-world environments”, organized by Tjeerd Andringa and myself. This is a brief personal report on why we thought autonomy in real-world environments is central for cognitive systems research and what I think I learned about it. --- The theses that crystallized are that a) autonomy is a relative property and a matter of degree, b) (...) increasing autonomy of an artificial system from its makers and users is a necessary feature of increasingly intelligent systems that can deal with the real-world and c) more such autonomy means less control but at the same time improved interaction with the system. (shrink)

Unlike human soldiers, autonomous weapons systems are unaffected by psychological factors that would cause them to act outside the chain of command. This is a compelling moral justification for their development and eventual deployment in war. To achieve this level of sophistication, the software that runs AWS will have to first solve two problems: the frame problem and the representation problem. Solutions to these problems will inevitably involve complex software. Complex software will create security risks and will make (...) AWS critically vulnerable to hacking. I claim that the political and tactical consequences of hacked AWS far outweigh the purported advantages of AWS not being affected by psychological factors and always following orders. Therefore, one of the moral justifications for the deployment of AWS is undermined. (shrink)

Predictions about autonomous weapon systems are typically thought to channel fears that drove all the myths about intelligence embodied in matter. One of these is the idea that the technology can get out of control and ultimately lead to horrifi c consequences, as is the case in Mary Shelley’s classic Frankenstein. Given this, predictions about AWS are sometimes dismissed as science-fiction fear-mongering. This paper considers several analogies between AWS and other weapon systems and ultimately offers an argument (...) that nuclear weapons and their effect on the development of modern asymmetrical warfare are the best analogy to the introduction of AWS. The fi nal section focuses on this analogy and offers speculations about the likely consequences of AWS being hacked. These speculations tacitly draw on myths and tropes about technology and AI from popular fi ction, such as Frankenstein, to project a convincing model of the risks and benefi ts of AWS deployment. (shrink)

Please contact me at [email protected] if you are interested in reading a particular chapter or being sent the entire manuscript for private use. -/- The thesis offers a comprehensive argument in favor of a regulationist approach to autonomous weapon systems (AWS). AWS, defined as all military robots capable of selecting or engaging targets without direct human involvement, are an emerging and potentially deeply transformative military technology subject to very substantial ethical controversy. AWS have both their enthusiasts and their (...) detractors, prominently advocating for a global preemptive ban on AWS development and use. Rejecting both positions, the author outlines a middle-of-the-road regulationist approach that is neither overly restrictive nor overly permissive. The disqualifying flaws of the rival prohibitionist approach are demonstrated in the process. After defining the core term of autonomy in weapon systems, the practical difficulties involved in applying an arms control regime to AWS are analyzed. The analysis shows that AWS are an extremely regulation-resistant technology. This feature when combined with their assumed high military utility makes a ban framework extremely costly to impose and enforce. As such it is ultimately very likely to fail at the benefit of the most unscrupulous international actors and at a very substantial risk to those abiding with international law. Consequently, to be ethically viable, a prohibitionist framework would need to offer substantial moral benefits impossible to attain through the rival regulationist approach. The remainder of the thesis undertakes to demonstrate that this is not the case. Comparing the considerations of military and strategic necessity to humanitarian concerns most commonly voiced by prohibitionists requires finding a common denominator for all values being referred to. Consequently, the thesis proceeds to show that both kinds of concerns are ultimately reducible to respect for basic human rights of all stakeholders, and so that the prohibitionist and regulationist approach may ultimately be compared in terms of consequences their adoption would generate for basic human rights realization. The author then evaluates both the potential humanitarian benefits, and the potential humanitarian hazards of AWS introduction. The benefits of leaving frontline combat to machines are outlined, with the unique kinds of suffering that would be abolished by such a development being described in detail. The arguments against AWS adoption are then divided into three classes: arguments related to alleged impossibility of compliance with The Laws of Armed Conflict, non-consequentialist and broad consequentialist arguments. This analysis, which comprises the greater part of the entire thesis, shows that the concerns behind compliance arguments are indeed substantial and have to be accommodated via a complex framework of best practices, regulations and localized restrictions on some kinds of AWS or AWS use in particular environments. They do not, however, justify a universal ban on using all the diverse forms of AWS in all environments. Non-consequentialist objections are found either reducible to other classes of arguments or thoroughly unconvincing, sometimes to the point of being actually vacuous. Broad consequentialist concerns are likewise found to be accommodable by regulation, empirically unfounded or causally disconnected from the actions of legitimate actors acquiring AWS, and therefore irrelevant to the moral permissibility of such actions. The author concludes that the proponents of prohibitionism are unable to point to moral benefits substantial enough to justify the costs and risks inherent in the approach. A global ban is, in fact, likely to have a worse humanitarian impact that well-regulated AWS adoption even if the strategic risks are disregarded. On the other hand, the analysis shows that there indeed exists an urgent need to regulate AWS through a variety of technological, procedural and legal solutions. These include, but are not limited to, a temporary moratorium on anti-personnel AWS use, development of internationally verified compliance software and eventual legal requirement of its employment, a temporary moratorium on AWS proliferation to state actors and a ban on their proliferation to non-state agents. (shrink)

Would it be ethical to deploy autonomous weapon systems (AWS) if they were unable to reliably recognize when enemy forces had surrendered? I suggest that an inability to reliably recognize surrender would not prohibit the ethical deployment of AWS where there was a limited window of opportunity for targets to surrender between the launch of the AWS and its impact. However, the operations of AWS with a high degree of autonomy and/or long periods of time between release and (...) impact are likely to remain controversial until they have the capacity to reliably recognize surrender. (shrink)

Autonomous Weapon Systems (AWS) are the next logical advancement for military technology. There is a significant concern though that by allowing such systems on the battlefield, we are collectively abdicating our moral responsibility. In this thesis, I will examine two arguments that advocate for a total ban on the use of AWS. I call these arguments the “Responsibility” and the “Agency” arguments. After presenting these arguments, I provide my own objections and demonstrate why these arguments fail to (...) convince. I then provide an argument as to why the use of AWS is a rational choice in the evolution of warfare. I conclude my thesis by providing a framework upon which future international regulations regarding AWS could be built. (shrink)

Autonomous Weapons Systems (AWS) have not gained a good reputation in the past. This attitude is odd if we look at the discussion of other-usually highly anticipated-AI-technologies, like autonomous vehicles (AVs); whereby even though these machines evoke very similar ethical issues, philosophers' attitudes towards them are constructive. In this article, I try to prove that there is an unjust bias against AWS because almost every argument against them is effective against AVs too. I start with the definition (...) of "AWS." Then, I arrange my arguments by the Just War Theory (JWT), covering jus ad bellum, jus in bello and jus post bellum problems. Meanwhile, I draw attention to similar problems against other AI-technologies outside the JWT framework. Finally, I address an exception, as addressed by Duncan Purves, Ryan Jenkins and Bradley Strawser, who realized the unjustified double standard, and deliberately tried to construct a special argument which rules out only AWS. (shrink)

We propose a multi-step evaluation schema designed to help procurement agencies and others to examine the ethical dimensions of autonomous systems to be applied in the security sector, including autonomous weapons systems.

Several recent lines of inquiry have pointed to the amygdala as a potential lesion site in autism. Because one function of the amygdala may be to produce autonomic arousal at the sight of a significant face, we compared the responses of autistic children to their mothers’ face and to a plain paper cup. Unlike normals, the autistic children as a whole did not show a larger response to the person than to the cup. We also monitored sympathetic activity in autistic (...) children as they engaged in a wide range of everyday behaviours. The children tended to use self-stimulation activities in order to calm hyper-responsive activity of the sympathetic (`fight or flight’) branch of the autonomic nervous system. A small percentage of our autistic subjects had hyporesponsive sympathetic activity, with essentially no electrodermal responses except to self-injurious behaviour. We sketch a hypothesis about autism according to which autistic children use overt behaviour in order to control a malfunctioning autonomic nervous system and suggest that they have learned to avoid using certain processing areas in the temporal lobes. (shrink)

The international debate on the ethics and legality of autonomous weapon systems (AWS) as well as the call for a ban are primarily focused on the nebulous concept of fully autonomous AWS. More specifically, on AWS that are capable of target selection and engagement without human supervision or control. This thesis argues that such a conception of autonomy is divorced both from military planning and decision-making operations as well as the design requirements that govern AWS engineering and (...) subsequently the tracking and tracing of moral responsibility. To do this, this thesis marries two different levels of meaningful human control (MHC), termed levels of abstraction, to couple military operations with design ethics. In doing so, this thesis argues that the contentious notion of ‘full’ autonomy is not problematic under this two-tiered understanding of MHC. It proceeds to propose the value sensitive design (VSD) approach as a means for designing for MHC. (shrink)

Autonomous e-coaching systems offer their users suggestions for action, thereby affecting the user's decision-making process. More specifically, the suggestions that these systems make influence the options for action that people actually consider. Surprisingly though, options and the corresponding process of option generation --- a decision-making stage preceding intention formation and action selection --- has received very little attention in the various disciplines studying decision making. We argue that this neglect is unjustified and that it is important, particularly (...) for designers of autonomous e-coaching systems, to understand how human option generation works. The aims of this paper are threefold. The first aim is to generate awareness with designers of autonomous e-coaching systems that these systems do in fact influence their users' options. The second is to show that understanding the interplay between a person's options and the e-coaching system's suggestions is important for improving the effectiveness of the system. The third is that the very same interplay is also crucial for designing e-coaching systems that respect people's autonomy. (shrink)

While Autonomous Weapons Systems have obvious military advantages, there are prima facie moral objections to using them. By way of general reply to these objections, I point out similarities between the structure of law and morality on the one hand and of automata on the other. I argue that these, plus the fact that automata can be designed to lack the biases and other failings of humans, require us to automate the formulation, administration, and enforcement of law as (...) much as possible, including the elements of law and morality that are operated by combatants in war. I suggest that, ethically speaking, deploying a legally competent robot in some legally regulated realm is not much different from deploying a more or less well-armed, vulnerable, obedient, or morally discerning soldier or general into battle, a police officer onto patrol, or a lawyer or judge into a trial. All feature automaticity in the sense of deputation to an agent we do not then directly control. Such relations are well understood and well-regulated in morality and law; so there is not much challenging philosophically in having robots be some of these agents — excepting the implications of the limits of robot technology at a given time for responsible deputation. I then consider this proposal in light of the differences between two conceptions of law. These are distinguished by whether each conception sees law as unambiguous rules inherently uncontroversial in each application; and I consider the prospects for robotizing law on each. Likewise for the prospects of robotizing moral theorizing and moral decision-making. Finally I identify certain elements of law and morality, noted by the philosopher Immanuel Kant, which robots can participate in only upon being able to set ends and emotionally invest in their attainment. One conclusion is that while affectless autonomous devices might be fit to rule us, they would not be fit to vote with us. For voting is a process for summing felt preferences, and affectless devices would have none to weigh into the sum. Since they don't care which outcomes obtain, they don't get to vote on which ones to bring about. (shrink)

Will future lethal autonomous weapon systems (LAWS), or ‘killer robots’, be a threat to humanity? The European Parliament has called for a moratorium or ban of LAWS; the ‘Contracting Parties to the Geneva Convention at the United Nations’ are presently discussing such a ban, which is supported by the great majority of writers and campaigners on the issue. However, the main arguments in favour of a ban are unsound. LAWS do not support extrajudicial killings, they do not take (...) responsibility away from humans; in fact they increase the abil-ity to hold humans accountable for war crimes. Using LAWS in war would probably reduce human suffering overall. Finally, the availability of LAWS would probably not increase the probability of war or other lethal conflict—especially as compared to extant remote-controlled weapons. The widespread fear of killer robots is unfounded: They are probably good news. (shrink)

Techno-ethics is the area in the philosophy of technology which deals with emerging robotic and digital AI technologies. In the last decade, a new techno-ethical challenge has emerged: Autonomous Weapon Systems (AWS), defensive and offensive (the article deals only with the latter). Such AI-operated lethal machines of various forms (aerial, marine, continental) raise substantial ethical concerns. Interestingly, the topic of AWS was almost not treated in Jewish law and its research. This article thus proposes an introductory ethical-halakhic perspective (...) on AWS, in the Israeli context. The article has seven sections. Section 1 defines AWS and the main ethical concerns it evokes, while providing elementary definitions and distinctions. §2 locates AWS within the realm of Jewish laws of war (hilkhot-ẓava), which recognize the right for self-defense, as well as the status of universally accepted moral norms. §3 unfolds pragmatic ethical premises of a humane techno-ethics, which are required for the identifying AWS as a moral question: I. Relationality; II. Technology is not (completely) neutral; III. The fallaciousness of transhumanism. It is argued that these premises are compatible with halakhic tradition. §4 investigates the question of the morality of AWS, within the field of military AI ethics. It is clarified why the standard categories of war-ethics (ad bello, in bellum) do not capture the singular ethical problem of AWS, which pertains to the operation of military means, rather than their human targets. It is argued that reductive perception of the human mind is misleading about the feasibility of ‘ethical robots’, capable of independent moral discretion. To provide a thick examination of human agency from the perspective of Jewish tradition, §5 explores two stories from the biblical book of Samuel (the murder of Nob’s priests and of Uriah). The lessons about the significance of moral agency within the pubic-political military sphere are made explicit, as well as the possible costs resulting from the loss of human agency in the case of AWS. Given that realpolitik considerations are basic in halakhah, §6 considers some possible contemporary socio-political implications of the AWS, that may risk the sustainability of the democratic project. §7 concludes by pointing out humane contributions of Jewish law to contemporary techno-ethics. (shrink)

Pranayamic breathing, defined as a manipulation of breath movement, has been shown to contribute to a physiologic response characterized by the presence of decreased oxygen consumption, decreased heart rate, and decreased blood pressure, as well as increased theta wave amplitude in EEG recordings, increased parasympathetic activity accompanied by the experience of alertness and reinvigoration. The mechanism of how pranayamic breathing interacts with the nervous system affecting metabolism and autonomic functions remains to be clearly understood. It is our hypothesis that voluntary (...) slow deep breathing functionally resets the autonomic nervous system through stretchinduced inhibitory signals and hyperpolarization currents propagated through both neural and non-neural tissue which synchronizes neural elements in the heart, lungs, limbic system and cortex. During inspiration, stretching of lung tissue produces inhibitory signals by action of slowly adapting stretch receptors (SARs) and hyperpolarization current by action of fibroblasts. Both inhibitory impulses and hyperpolarization current are known to synchronize neural elements leading to the modulation of the nervous system and decreased metabolic activity indicative of the parasympathetic state. In this paper we propose pranayama’s physiologic mechanism through a cellular and systems level perspective,involving both neural and non-neural elements. This theoretical description describes a common physiological mechanism underlying pranayama and elucidate the role of the respiratory and cardiovascular system on modulating the autonomic nervous system. Along with facilitating the design of clinical breathing techniques for the treatment of autonomic nervous system and other disorders, this model will also validate pranayama as a topic requiring more research. (shrink)

The prospective introduction of autonomous cars into public traffic raises the question of how such systems should behave when an accident is inevitable. Due to concerns with self-interest and liberal legitimacy that have become paramount in the emerging debate, a contractarian framework seems to provide a particularly attractive means of approaching this problem. We examine one such attempt, which derives a harm minimisation rule from the assumptions of rational self-interest and ignorance of one’s position in a future accident. (...) We contend, however, that both contractarian approaches and harm minimisation standards are flawed, due to a failure to account for the fundamental difference between those ‘involved’ and ‘uninvolved’ in an impending crash. Drawing from classical works on the trolley problem, we show how this notion can be substantiated by reference to either the distinction between negative and positive rights, or to differences in people’s claims. By supplementing harm minimisation with corresponding constraints, we can develop crash algorithms for autonomous cars which are both ethically adequate and promise to overcome certain significant practical barriers to implementation. (shrink)

The application of modern technologies with artificial intelligence (AI) in all spheres of human life is growing exponentially alongside concern for its controllability. The lack of public, state, and international control over AI technologies creates large-scale risks of using such software and hardware that (un)intentionally harm humanity. The events of recent month and years, specifically regarding the Russian Federation’s war against its democratic neighbour Ukraine and other international conflicts of note, support the thesis that the uncontrolled use of AI, especially (...) in the military sphere, may lead to deliberate disregard for the moral standards of controlled AI or the spontaneous emergence of aggressive autonomous AI. The development of legal regulation for the use of technologies with AI is prolonged concerning the rapid development of these artefacts, which simultaneously cover all areas of public relations. Therefore, control over the creation and use of AI should be carried out not only by purely technical regulation (e.g., technical standards and conformance assessments, corporate and developer regulations, requirements enforced through industry-wide ethical codes); but also by comprehensive legislation and intergovernmental oversight bodies that codify and enforce specific changes in the rights and duties of legal persons. This article shall present the “Morality Problem” and “Intentionality Problem” of AI, and reflect upon various lacunae that arise when implementing AI for military purposes. (shrink)

Lethal Autonomous Weapons (LAWs) are robotic weapons systems, primarily of value to the military, that could engage in offensive or defensive actions without human intervention. This paper assesses and engages the current arguments for and against the use of LAWs through the lens of achieving more ethical warfare. Specific interest is given particularly to ethical LAWs, which are artificially intelligent weapons systems that make decisions within the bounds of their ethics-based code. To ensure that a wide, but (...) not exhaustive, survey of the implications of employing such ethical devices to replace humans in warfare is taken into account, this paper will engage on matters related to current scholarship on the rejection or acceptance of LAWs—including contemporary technological shortcomings of LAWs to differentiate between targets and the behavioral and psychological volatility of humans—and current and proposed regulatory infrastructures for developing and using such devices. After careful consideration of these factors, this paper will conclude that only ethical LAWs should be used to replace human involvement in war, and, by extension of their consistent abilities, should remove humans from war until a more formidable discovery is made in conducting ethical warfare. (shrink)

One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents (IAs). This paper explores how decision matrix algorithms, via the belief-desire-intention (...) model for autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing for the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles. (shrink)

Autonomous and automatic weapons would be fire and forget: you activate them, and they decide who, when and how to kill; or they kill at a later time a target you’ve selected earlier. Some argue that this sort of killing is always wrong. If killing is to be done, it should be done only under direct human control. (E.g., Mary Ellen O’Connell, Peter Asaro, Christof Heyns.) I argue that there are surprisingly many kinds of situation where this is false (...) and where the use of Automated Weapons Systems would in fact be morally required. These include cases where a) once one has activated a weapon expected then to behave lethally, it would be appropriate to let it continue because this is part of a plan whose goodness one was best positioned to evaluate before activating the weapon; b) one expects better long-term consequences from allowing it to continue; c) allowing it to continue would express a decision you made to be resolute, a decision that could not have advantaged you had it not been true that you would carry through with it; d) the weapon is mechanically not recallable, so that, to not allow it to carry through, you would have had to refrain from activating it in the first place, something you expected would have disastrous consequences; e) you must deputize necessary killings to autonomous machines in order to protect yourself from guilt you shouldn’t have to bear; f) it would be morally better for the burden of responsibility for the killing to be shared among several agents, and the agents deputizing killing to machines can do this, especially where it’s not predictable which machine will be successful; g) a killing would be morally better done with elements of randomness and lack of deliberation, and a (relatively stupid) machine could do this where a person could not; h) the machine would be acting as a Doomsday Device, so that it could not have had its hoped for deterrent effect had you not ensured that you would be unable to recall it if enemy action activated it; i) letting it carry through is a necessary part of its own learning process, and you expect that this learning will have salutary effects later on; j) human intervention in the machine’s operation would disastrously impair its precision, or its speed and efficiency; k) using non-automated methods would require human resources you just don’t have in a task that nevertheless must be done (e.g., using land-mines to protect remote installations); l) the weapon has such horrible and indiscriminate power that it is doubtful whether it could be actually used in ways compatible with International Humanitarian Law and the Laws of War, which require that weapons be used only in ways respecting distinctness, necessity and proportionality, but its threat of use could respect these principles in affording deterrence provided human error cannot lead to their accidental deployment, this requiring that they be controlled by carefully designed autonomous and automatic systems. I then consider objections based on conceptions of human dignity and find that very often dignity too is best served by autonomous machine killing. Examples include saving your village by activating a robot to kill invading enemies who would inflict great indignity on your village, using a suicide robot to save yourself from a less dignified death at enemy hands, using a robotic drone to kill someone otherwise not accessible in order to restore dignity to someone this person killed and to his family, and using a robot to kill someone who needs killing, but the killing of whom by a human executioner would soil the executioner’s dignity. I conclude that what matters in rightful killing isn’t necessarily that it be under the direct control of a human, but that it be under the control of morality; and that could sometimes require use of an autonomous or automated device. (This paper was formerly called "Fire and Forget: A Defense of the Use of Autonomous Weapons in War" on Philpapers; the current title is the title of the published version.). (shrink)

A body schema is an agent's model of its own body that enables it to act on affordances in the environment. This paper presents a body schema system for the Learning Intelligent Decision Agent (LIDA) cognitive architecture. LIDA is a conceptual and computational implementation of Global Workspace Theory, also integrating other theories from neuroscience and psychology. This paper contends that the ‘body schema' should be split into three separate functions based on the functional role of consciousness in Global Workspace Theory. (...) There is (1) an online model of the agent's effectors and effector variables (Current Body Schema), (2) a long-term, recognitional storage of embodied capacities for action and affordances (Habitual Body Schema), and (3) "dorsal" stream information feeding directly from early perception to sensorimotor processes (Online Body Schema). This paper then discusses how the LIDA model of the body schema explains several experiments in psychology and ethology. (shrink)

Robert Sparrow (among others) claims that if an autonomous weapon were to commit a war crime, it would cause harm for which no one could reasonably be blamed. Since no one would bear responsibility for the soldier’s share of killing in such cases, he argues that they would necessarily violate the requirements of jus in bello, and should be prohibited by international law. I argue this view is mistaken and that our moral understanding of war is sufficient to determine (...) blame for any wrongful killing done by autonomous weapons. Analyzing moral responsibility for autonomous weapons starts by recognizing that although they are capable of causing moral consequences, they are neither praiseworthy nor blameworthy in the moral sense. As such, their military role is that of a tool, albeit a rather sophisticated one, and responsibility for their use is roughly analogous to that of existing “smart” weapons. There will likely be some difficulty in managing these systems as they become more intelligent and more prone to unpredicted behavior, but the moral notion of shared responsibility and the legal notion of command responsibility are sufficient to locate responsibility for their use. (shrink)

The international debate on the ethics and legality of autonomous weapon systems (AWS), along with the call for a ban, primarily focus on the nebulous concept of fully autonomous AWS. These are AWS capable of target selection and engagement absent human supervision or control. This paper argues that such a conception of autonomy is divorced from both military planning and decision-making operations; it also ignores the design requirements that govern AWS engineering and the subsequent tracking and tracing (...) of moral responsibility. To show how military operations can be coupled with design ethics, this paper marries two different kinds of meaningful human control (MHC) termed levels of abstraction. Under this two-tiered understanding of MHC, the contentious notion of ‘full’ autonomy becomes unproblematic. (shrink)

Interaction between drivers and pedestrians is often facilitated by informal communicative cues, like hand gestures, facial expressions, and eye contact. In the near future, however, when semi- and fully autonomous vehicles are introduced into the traffic system, drivers will gradually assume the role of mere passengers, who are casually engaged in non-driving-related activities and, therefore, unavailable to participate in traffic interaction. In this novel traffic environment, advanced communication interfaces will need to be developed that inform pedestrians of the current (...) state and future behavior of an autonomous vehicle, in order to maximize safety and efficiency for all road users. The aim of the present review is to provide a comprehensive account of empirical work in the field of external human–machine interfaces for autonomous vehicle-to- pedestrian communication. In the great majority of covered studies, participants clearly benefited from the presence of a communication interface when interacting with an autonomous vehicle. Nevertheless, standardized interface evaluation procedures and optimal interface specifications are still lacking. (shrink)

Through modern driver assistant systems, algorithmic decisions already have a significant impact on the behavior of vehicles in everyday traffic. This will become even more prominent in the near future considering the development of autonomous driving functionality. The need to consider ethical principles in the design of such systems is generally acknowledged. However, scope, principles and strategies for their implementations are not yet clear. Most of the current discussions concentrate on situations of unavoidable crashes in which the (...) life of human beings is existentially affected. In this paper, we argue that ethical considerations should be mandatory for any algorithmic decision of autonomous vehicles, instead of a limitation to hazard situations. Such an ethically aligned behavior is relevant because autonomous vehicles, like any other traffic participants, operate in a shared public space, where every behavioral decision impacts the operational possibilities of others. These possibilities concern the fulfillment of a road-user’s safety, utility and comfort needs. We propose that, to operate ethically in such space, an autonomous vehicle will have to take its behavior decisions according to a just distribution of operational possibilities among all traffic participants. Using an application on a partially-autonomous prototype vehicle, we describe how to apply and implement concepts of distributive justice to the driving environment and demonstrate the impact on its behavior in comparison to an advanced but egoistic decision maker. (shrink)

We propose a domain-level ontology of plays for the facilitation of play-based collaborative autonomy among unmanned and manned-unmanned aircraft teams in the Army’s Unmanned Aircraft System (UAS) mission domain. We define a play as a type of plan that prescribes some pattern of intentional acts that are intended to reliably result in some goal in some competitive context, and which specifies one or more roles that are realized by those prescribed intentional acts. The ontology is well suited to be extended (...) to other types of military and nonmilitary unmanned vehicle operations. (shrink)

Abstract: Many believe that, in addition to cognitive capacities, autonomous robots need something similar to affect. As in humans, affect, including specific emotions, would filter robot experience based on a set of goals, values, and interests. This narrows behavioral options and avoids combinatorial explosion or regress problems that challenge purely cognitive assessments in a continuously changing experiential field. Adding human-like affect to robots is not straightforward, however. Affect in organisms is an aspect of evolved biological systems, from the (...) taxes of single-cell organisms to the instincts, drives, feelings, moods, and emotions that focus human behavior through the mediation of hormones, pheromones, neurotransmitters, the autonomic nervous system, and key brain structures. We argue that human intelligence is intimately linked to biological affective systems and to the unique repertoire of potential behaviors, sometimes conflicting, they facilitate. Artificial affect is affect in name only and without genes and biological bodies, autonomous robots will lack the goals, interests, and value systems associated with human intelligence. We will take advantage of their general intelligence and expertise, but robots will not enter our intellectual world or apply for legal status in the community. -/- . (shrink)

Recently criticisms against autonomous weapons were presented in a video in which an AI-powered drone kills a person. However, some said that this video is a distraction from the real risk of AI—the risk of unlimitedly self-improving AI systems. In this article, we analyze arguments from both sides and turn them into conditions. The following conditions are identified as leading to autonomous weapons becoming a global catastrophic risk: 1) Artificial General Intelligence (AGI) development is delayed relative to (...) progress in narrow AI and manufacturing. 2) The potential for very cheap manufacture of drones, with prices below 1 USD each. 3) Anti-drone defense capabilities lagging offensive development. 4) Special global military posture encouraging development of drone swarms as a strategic offensive weapon, able to kill civilians. We conclude that while it is unlikely that drone swarms alone will become existential risk, lethal autonomous weapons could contribute to civilizational collapse in case of new world war. (shrink)

A. Newell and H. A. Simon were two of the most influential scientists in the emerging field of artificial intelligence (AI) in the late 1950s through to the early 1990s. This paper reviews their crucial contribution to this field, namely to symbolic AI. This contribution was constituted mostly by their quest for the implementation of general intelligence and (commonsense) knowledge in artificial thinking or reasoning artifacts, a project they shared with many other scientists but that in their case was theoretically (...) based on the idiosyncratic notions of symbol systems and the representational abilities they give rise to, in particular with respect to knowledge. While focusing on the period 1956-1982, this review cites both earlier and later literature and it attempts to make visible their potential relevance to today's greatest unifying AI challenge, to wit, the design of wholly autonomous artificial agents (a.k.a. robots) that are not only rational and ethical, but also self-conscious. (shrink)

Living agency is subject to a normative dimension (good-bad, adaptive-maladaptive) that is absent from other types of interaction. We review current and historical attempts to naturalize normativity from an organism-centered perspective, identifying two central problems and their solution: (1) How to define the topology of the viability space so as to include a sense of gradation that permits reversible failure, and (2) how to relate both the processes that establish norms and those that result in norm-following behavior. We present a (...) minimal metabolic system that is coupled to a gradient-climbing chemotactic mechanism. Studying the relationship between metabolic dynamics and environmental resource conditions, we identify an emergent viable region and a precarious region where the system tends to die unless environmental conditions change. We introduce the concept of normative field as the change of environmental conditions required to bring the system back to its viable region. Norm-following, or normative action, is defined as the course of behavior whose effect is positively correlated with the normative field. We close with a discussion of the limitations and extensions of our model and some final reflections on the nature of norms and teleology in agency. (shrink)

Robert Sparrow (among others) claims that if an autonomous weapon were to commit a war crime, it would cause harm for which no one could reasonably be blamed. Since no one would bear responsibility for the soldier’s share of killing in such cases, he argues that they would necessarily violate the requirements of jus in bello, and should be prohibited by international law. I argue this view is mistaken and that our moral understanding of war is sufficient to determine (...) blame for any wrongful killing done by autonomous weapons. Analyzing moral responsibility for autonomous weapons starts by recognizing that although they are capable of causing moral consequences, they are neither praiseworthy nor blameworthy in the moral sense. As such, their military role is that of a tool, albeit a rather sophisticated one, and responsibility for their use is roughly analogous to that of existing “smart” weapons. There will likely be some difficulty in managing these systems as they become more intelligent and more prone to unpredicted behavior, but the moral notion of shared responsibility and the legal notion of command responsibility are sufficient to locate responsibility for their use. (shrink)

English - language Wikipedia is constantly being plagued by vandalistic contributions on a massive scale. In order to fight them its volunteer contributors deploy an array of software tools and autonomous bots. After an analysis of their functioning and the ‘ coactivity ’ in use between humans and bots, this research ‘ discloses ’ the moral issues that emerge from the combined patrolling by humans and bots. Administrators provide the stronger tools only to trusted users, thereby creating a new (...) hierarchical layer. Further, surveillance exhibits several troubling features : questionable profiling practices, the use of the controversial measure of reputation, ‘ oversurveillance ’ where quantity trumps quality, and a prospective loss of the required moral skills whenever bots take over from humans. The most troubling aspect, though, is that Wikipedia has become a Janus - faced institution. One face is the basic platform of MediaWiki software, transparent to all. Its other face is the anti - vandalism system, which, in contrast, is opaque to the average user, in particular as a result of the algorithms and neural networks in use. Finally it is argued that this secrecy impedes a much needed discussion to unfold ; a discussion that should focus on a ‘ rebalancing ’ of the anti - vandalism system and the development of more ethical information practices towards contributors. (shrink)

In this paper, we model a relational notion of subjectivity by means of two experiments in subjective computing. The goal is to determine to what extent a cognitive and social robot can be regarded to act subjectively. The system was implemented as a reinforcement learning agent with a coaching function. To analyze the robotic agent we used the method of levels of abstraction in order to analyze the agent at four levels of abstraction. At one level the agent is described (...) in mentalistic or subjective language respectively. By mapping this mentalistic to an algorithmic, functional, and relational level, we can show to what extent the agent behaves subjectively as we make use of a relational concept of subjectivity that draws upon the relations that hold between the agent and its environment. According to a relational notion of subjectivity, an agent is supposed to be subjective if it exhibits autonomous relations to itself and others, i.e. the agent is not fully determined by a given input but is able to operate on its input and decide what to do with it. This theoretical notion is confirmed by the technical implementation of self-referentiality and social interaction in that the agent shows improved behavior compared to agents without the ability of subjective computing. On the one hand, a relational concept of subjectivity is confirmed, whereas on the other hand, the technical framework of subjective computing is being theoretically founded. (shrink)

In this paper I provide an exposition and critique of Johnson and Noorman’s (2014) three conceptualizations of the agential roles artificial systems can play. I argue that two of these conceptions are unproblematic: that of causally efficacious agency and “acting for” or surrogate agency. Their third conception, that of “autonomous agency,” however, is one I have reservations about. The authors point out that there are two ways in which the term “autonomy” can be used: there is, firstly, the (...) engineering sense of the term, which simply refers to the ability of some system to act independently of substantive human control. Secondly, there is the moral sense of the term, which has traditionally grounded many notions of human mor-al responsibility. I argue that the continued usage of “autonomy” in discussions of artificial agency complicates matters unnecessarily. This occurs in two ways: firstly, the condition of autonomy, even in its engineering sense, fails to accurately describe the way “autonomous” systems are developed in practice. Secondly, the continued usage of autonomy in the moral sense introduces unnecessary meta-physical baggage form the free will debate into discussions about moral agency. In order to understand the debate surrounding autonomy, we would therefore first need to settle many seemingly intractable metaphysical questions regarding the existence of free will in human beings. (shrink)

Colour is, according to prevailing orthodoxy in perceptual psychology, a kind of autonomous and unitary attribute. It is regarded as unitary or homogeneous by assuming that its core properties do not depend on the type of ‘perceptual object’ to which it pertains and that‘colour per se’ constitutes a natural attribute in the functional architecture of the perceptual system. It is regarded as autonomous by assuming that it can be studied in isolation of other perceptual attributes. These assumptions also (...) provide the pillars for the technical field of colorimetry, and have proved very fruitful for neurophysiological investigations into peripheral colour coding. They also have become, in a technology-driven cultural process of abstraction, part of our common-sense conception of colour. With respect to perception theory, however, both assumptions are grossly inadequate, on both empirical and theoretical grounds. All the same, the idea of an internal homogeneous and autonomous attribute of ‘colour per se’, mostly taken not as an empirical hypothesis but as a kind of truism, became a guiding idea in perceptual psychology. Here, it has impeded the identification of relevant theoretical issues and consequently has become detrimental for the development of explanatory frameworks for the role of ‘colour’ within the structure of our perceptual system. The chapter argues that enquiries into colour perception cannot be divorced from general enquiries into the structure of the conceptual forms underlying perception. (shrink)

Forgiveness as a positive response to wrongdoing is a widespread phenomenon that plays a role in the moral lives of most persons. Surprisingly, Kant has very little to say on the matter. Although Kant dedicates considerable space to discussing punishment, wrongdoing and grace, he addresses the issues of human forgiveness directly only in some short passages in the Lectures on Ethics and in one passage of the Metaphysics of Morals. As noted by Sussman, the TL passage, however, betrays some ambivalence. (...) Kant establishes a duty of virtue to be forgiving (TL, 6:460), yet he immediately warns against its excess: meek toleration of recurrent wrongs could manifest a lack of self-respect and a violation of a duty to oneself (TL, 6:461). Sussman claims that this ambivalence ultimately arises from the fact that forgiveness sits uncomfortably in Kant’s moral thought. First, forgiveness has an ‘ineluctably elective aspect’ that makes it, to a certain extent, arbitrary and dependent on particular features of the forgiver’s psychology and, as such, in tension with Kant’s central claims that human beings are autonomous agents capable of determining their own moral status. Second, according to Sussman, Kant’s moral retributivism, i.e. ‘the particular moral position that every moral wrong against another deserves punishment of the wrongdoer’ seems to be in tension with the possibility of a ‘truly redemptive forgiveness’. Moreover, forgiveness also seems to be in tension with a passage of the Religion in which Kant argues that the moral guilt from our original evil disposition cannot be understood as a debt or liability that can be compensated, erased, transferred or otherwise wiped out by others (Rel, 6:72). Thus, to the extent that forgiveness might be thought to involve the forgoing of moral guilt, it seems incompatible with Kant’s views on culpability and punishment. This chapter seeks to clarify Kant’s views on forgiveness in order to show that, although not often appreciated, personal forgiveness plays an important role in the lives of ordinary human agents as understood by Kant. In particular, I aim to show there is a conception of forgiveness available to Kant that is not incompatible with Kant’s views of punishment and culpability. In Section 1, I argue that, for Kant, far from being merely ‘elective,’ forgiveness is, under certain conditions, morally required. I provide a brief summary of an interpretation of Kant’s theory of forgiveness that I have defended in recently published work , in order to argue that Kant’s duty to be forgiving should be understood as an imperfect duty of virtue which is conditional on repentance. Kant is not ambivalent about this duty because he maintains that when the relevant conditions are not met, we have a perfect duty to ourselves not to forgive unrepentant wrongdoers. The TL passage thus identifies two different duties. In Section 2, I show that forgiveness, as conceptualised by Kant, does not require the forgoing of punishment or the overcoming of moral guilt and that this could, in fact, be seen as an attractive feature of Kant’s position. I end by offering a very brief assessment of Kant’s views. (shrink)

This article provides an answer to the question: What is the function of cognition? By answering this question it becomes possible to investigate what are the simplest cognitive systems. It addresses the question by treating cognition as a solution to a design problem. It defines a nested sequence of design problems: (1) How can a system persist? (2) How can a system affect its environment to improve its persistence? (3) How can a system utilize better information from the environment (...) to select better actions? And, (4) How can a system reduce its inherent informational limitations to achieve more successful behavior? This provides a corresponding nested sequence of system classes: (1) autonomous systems, (2) (re)active autonomous systems, (3) informationally controlled autonomous systems (autonomous agents), and (4) cognitive systems. -/- This article provides the following characterization of cognition: The cognitive system is the set of mechanisms of an autonomous agent that (1) allow increase of the correlation and integration between the environment and the information system of the agent, so that (2) the agent can improve the selection of actions and thereby produce more successful behavior. -/- Finally, it shows that common cognitive capacities satisfy the characterization: learning, memory, representation, decision making, reasoning, attention, and communication. (shrink)

The aim of this exploratory paper is to review an under-appreciated parallel between group agency and artificial intelligence. As both phenomena involve non-human goal-directed agents that can make a difference to the social world, they raise some similar moral and regulatory challenges, which require us to rethink some of our anthropocentric moral assumptions. Are humans always responsible for those entities’ actions, or could the entities bear responsibility themselves? Could the entities engage in normative reasoning? Could they even have rights and (...) a moral status? I will tentatively defend the (increasingly widely held) view that, under certain conditions, artificial intelligent systems, like corporate entities, might qualify as responsible moral agents and as holders of limited rights and legal personhood. I will further suggest that regulators should permit the use of autonomous artificial systems in high-stakes settings only if they are engineered to function as moral (not just intentional) agents and/or there is some liability-transfer arrangement in place. I will finally raise the possibility that if artificial systems ever became phenomenally conscious, there might be a case for extending a stronger moral status to them, but argue that, as of now, this remains very hypothetical. (shrink)

Danaher (2016) has argued that increasing robotization can lead to retribution gaps: Situation in which the normative fact that nobody can be justly held responsible for a harmful outcome stands in conflict with our retributivist moral dispositions. In this paper, we report a cross-cultural empirical study based on Sparrow’s (2007) famous example of an autonomous weapon system committing a war crime, which was conducted with participants from the US, Japan and Germany. We find that (i) people manifest a considerable (...) willingness to hold autonomous systems morally responsible, (ii) partially exculpate human agents when interacting with such systems, and that more generally (iii) the possibility of normative responsibility gaps is indeed at odds with people’s pronounced retributivist inclinations. We discuss what these results mean for potential implications of the retribution gap and other positions in the responsibility gap literature. (shrink)

Efforts to ban Autonomous Weapon Systems were both unsuccessful and controversial. Simultaneously the need to address the detrimental aspects of AWS development and proliferation continues to grow in scope and urgency. The article presents several regulatory solutions capable of addressing the issue while simultaneously respecting the requirements of military necessity and so attracting a broad consensus. Two much stricter solutions – regional AWS bans and adoption of a no first use policy – are also presented as fallback strategies (...) in case achieving AWS’ compliance with the Laws of Armed Conflict proved elusive. Together, the solutions presented form an outline of a flexible regulatory strategy able to adjust to different technological outcomes and providing a sensible compromise to solve the current deadlock on the AWS issue. (shrink)

Understanding the autonomic nervous system and homeostatic changes associated with emotions remains a major challenge for neuroscientists and a fundamental prerequisite to treat anxiety, stress, and emotional disorders. Based on recent publications, the inter-relationship between respiration and emotions and the influence of respiration on autonomic changes, and subsequent widespread membrane potential changes resulting from changes in homeostasis are discussed. We hypothesize that reversing homeostatic alterations with meditation and breathing techniques rather than targeting neurotransmitters with medication may be a superior method (...) to address the whole body changes that occur in stress, anxiety, and depression. Detrimental effects of stress, negative emotions, and sympathetic dominance of the autonomic nervous system have been shown to be counteracted by different forms of meditation, relaxation, and breathing techniques. We propose that these breathing techniques could be used as firstline and supplemental treatments for stress, anxiety, depression, and some emotional disorders. (shrink)

Increasingly complex and autonomous systems require machine ethics to maximize the benefits and minimize the risks to society arising from the new technology. It is challenging to decide which type of ethical theory to employ and how to implement it effectively. This survey provides a threefold contribution. First, it introduces a trimorphic taxonomy to analyze machine ethics implementations with respect to their object (ethical theories), as well as their nontechnical and technical aspects. Second, an exhaustive selection and description (...) of relevant works is presented. Third, applying the new taxonomy to the selected works, dominant research patterns, and lessons for the field are identified, and future directions for research are suggested. (shrink)

Warfare is becoming increasingly automated, from automatic missile defense systems to micro-UAVs (WASPs) that can maneuver through urban environments with ease, and each advance brings with it ethical questions in need of resolving. Proponents of lethal autonomous weapons systems (LAWS) provide varied arguments in their favor; robots are capable of better identifying combatants and civilians, thus reducing "collateral damage"; robots need not protect themselves and so can incur more risks to protect innocents or gather more information before (...) using deadly force; robots can assess situations more quickly and do so without emotion, reducing the likelihood of fatal mistakes due to human error; and sending robots to war protects our own soldiers from harm. However, these arguments only point in favor of autonomous weapons systems, failing to demonstrate why such systems need be made *lethal*. In this paper I argue that if one grants all of the proponents' points in favor of LAWS, then, contrary to what might be expected, this leads to the conclusion that it would be both immoral and illegal to deploy *lethal* autonomous weapons, because the many features that speak in favor of them also undermine the need for them to be programmed to take lives. In particular, I argue that such systems, if lethal, would violate the moral and legal principle of necessity, which forbids the use of weapons that impose superfluous injury or unnecessary harm. I conclude by highlighting that the argument is not against autonomous weapons per se, but only against *lethal* autonomous weapons. (shrink)