The phenomenon of distributed knowledge is well-known in epistemic logic. In this paper, a similar phenomenon in ethics, somewhat neglected so far, is investigated, namely distributed morality. The article explains the nature of distributed morality, as a feature of moral agency, and explores the implications of its occurrence in advanced information societies. In the course of the analysis, the concept of infraethics is introduced, in order to refer to the ensemble of moral enablers, which, although morally neutral per se, can (...) significantly facilitate or hinder both positive and negative moral behaviours. (shrink)

The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. Other animals have stronger muscles or sharper claws, but we have cleverer brains. If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of (...) our species then would come to depend on the actions of the machine superintelligence. But we have one advantage: we get to make the first move. Will it be possible to construct a seed AI or otherwise to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation? To get closer to an answer to this question, we must make our way through a fascinating landscape of topics and considerations. Read the book and learn about oracles, genies, singletons; about boxing methods, tripwires, and mind crime; about humanity's cosmic endowment and differential technological development; indirect normativity, instrumental convergence, whole brain emulation and technology couplings; Malthusian economics and dystopian evolution; artificial intelligence, and biological cognitive enhancement, and collective intelligence. (shrink)

The implementation of moral decision making abilities in artificial intelligence (AI) is a natural and necessary extension to the social mechanisms of autonomous software agents and robots. Engineers exploring design strategies for systems sensitive to moral considerations in their choices and actions will need to determine what role ethical theory should play in defining control architectures for such systems. The architectures for morally intelligent agents fall within two broad approaches: the top-down imposition of ethical theories, and the bottom-up building of (...) systems that aim at goals or standards which may or may not be specified in explicitly theoretical terms. In this paper we wish to provide some direction for continued research by outlining the value and limitations inherent in each of these approaches. (shrink)

In this classic text, Kant sets out to articulate and defend the Categorical Imperative - the fundamental principle that underlies moral reasoning - and to lay the foundation for a comprehensive account of justice and human virtues. This new edition and translation of Kant's work is designed especially for students. An extensive and comprehensive introduction explains the central concepts of Groundwork and looks at Kant's main lines of argument. Detailed notes aim to clarify Kant's thoughts and to correct some common (...) misunderstandings of his doctrines. (shrink)

John Dupre warns that our understanding of human nature is being distorted by two faulty and harmful forms of pseudo-scientific thinking. Not just in the academic world but in everyday life, we find one set of experts who seek to explain the ends at which humans aim in terms of evolutionary theory, while the other set uses economic models to give rules of how we act to achieve those ends. Dupre demonstrates that these theorists' explanations do not work and that, (...) if taken seriously, their theories tend to have dangerous social and political consequences. For these reasons, it is important to resist scientism: an exaggerated conception of what science can be expected to do for us. Dupre restores sanity to the study of human nature by pointing the way to a proper understanding of humans in the societies that are our natural and necessary environments. Anyone interested in science and human life will enjoy this book--unless they are its targets. (shrink)

Immanuel Kant's Groundwork of the Metaphysics of Morals ranks alongside Plato's Republic and Aristotle's Nicomachean Ethics as one of the most profound and influential works in moral philosophy ever written. In Kant's own words its aim is to search for and establish the supreme principle of morality, the categorical imperative. Kant argues that every human being is an end in himself or herself, never to be used as a means by others, and that moral obligation is an expression of the (...) human capacity for autonomy or self-government. This edition presents the acclaimed translation of the text by Mary Gregor, together with an introduction by Christine M. Korsgaard that examines and explains Kant's argument. (shrink)

Artificial intelligence research has foundered on the issue of representation. When intelligence is approached in an incremental manner, with strict reliance on interfacing to the real world through perception and action, reliance on representation disappears. In this paper we outline our approach to incrementally building complete intelligent Creatures. The fundamental decomposition of the intelligent system is not into independent information processing units which must interface with each other via representations. Instead, the intelligent system is decomposed into independent and parallel activity (...) producers which all interface directly to the world through perception and action, rather than interface to each other particularly much. The notions of central and peripheral systems evaporateeverything is both central and peripheral. Based on these principles we have built a very successful series of mobile robots which operate without supervision as Creatures in standard office environments. (shrink)

We continuously talk about autonomous technologies. But how can words qualifying technologies be the very same words chosen by Kant to define what is essentially human, i.e. being autonomous? The article focuses on a possible answer by reflecting upon both etymological and philosophical issues, as well as upon the case of autonomous vehicles. Most interestingly, on the one hand, we have the notion of “autonomy”, meaning that there is a “law” that is “self-given”, and, on the other hand, we have (...) the notion of “automation”, meaning that there is something “offhand” that is “self-given”. Yet, we are experiencing a kind of twofold shift: on the one hand, the shift from defining technologies in terms of automation to defining technologies in terms of autonomy and, on the other hand, the shift from defining humans in terms of autonomy to defining humans in terms of automation. From a philosophical perspective, the shift may mean that we are trying to escape precisely from what autonomy founds, i.e. individual responsibility of humans that, in the Western culture, have been defined for millennia as rational and moral decision-makers, even when their decisions have been the toughest. More precisely, the shift may mean that we are using technologies, and in particular emerging algorithmic technologies, as scapegoats that bear responsibility for us by making decisions for us. Moreover, if we consider the kind of emerging algorithmic technologies that increasingly surround us, starting from autonomous vehicles, then we may argue that we also seem to create a kind of technological divine that, by being always with us through its immanent omnipresence, omniscience, omnipotence and inscrutability, can always be our technological scapegoat freeing us from the most unbearable burden of individual responsibility resulting from individual autonomy. (shrink)

A prima facie analysis suggests that there are essentially two, mutually exclusive, ways in which risk arising from engineering design can be managed: by imposing external constraints on engineers or by engendering their feelings of responsibility and respect their autonomy. The author discusses the advantages and disadvantages of both approaches. However, he then shows that this opposition is a false one and that there is no simple relation between regulation and autonomy. Furthermore, the author argues that the most pressing need (...) is not more or less regulation but the further development of moral imagination. The enhancement of moral imagination can help engineers to discern the moral relevance of design problems, to create new design options, and to envisage the possible outcomes of their designs. The author suggests a dual program of developing regulatory frameworks that support engineers’ autonomy and responsibility simultaneously with efforts to promote their moral imagination. He describes how some existing institutional changes have started off in this direction and proposes empirical research to take this further. (shrink)

In the fields of artificial intelligence and robotics, the term “autonomy” is generally used to mean the capacity of an artificial agent to operate independently of human guidance. It is thereby assumed that the agent has a fixed goal or “utility function” with respect to which the appropriateness of its actions will be evaluated. From a philosophical perspective, this notion of autonomy seems oddly weak. For, in philosophy, the term is generally used to refer to a stronger capacity, namely the (...) capacity to “give oneself the law,” to decide by oneself what one’s goal or principle of action will be. The predominant view in the literature on the long-term prospects and risks of artificial intelligence is that an artificial agent cannot exhibit such autonomy because it cannot rationally change its own final goal, since changing the final goal is counterproductive with respect to that goal and hence undesirable. The aim of this paper is to challenge this view by showing that it is based on questionable assumptions about the nature of goals and values. I argue that a general AI may very well come to modify its final goal in the course of developing its understanding of the world. This has important implications for how we are to assess the long-term prospects and risks of artificial intelligence. (shrink)

This volume explores the ethical questions that arise in the development, creation and use of robots that are capable of semiautonomous or autonomous decision making and human-like action. It examines how ethical and moral theories can and must be applied to address the complex and critical issues of the application of these intelligent robots in society. Coverage first presents fundamental concepts and provides a general overview of ethics, artificial intelligence and robotics. Next, the book studies all principal ethical applications of (...) robots, namely medical, assistive, socialized and war roboethics. It looks at such issues as robotic surgery, children-robot and elderly-robot therapeutical/social interactions and the use of robots, especially autonomous lethal ones, in warfare. In addition, a chapter also considers Japanese roboethics as well as key intercultural and robot legislation issues. Overall, readers are provided with a thorough investigation into the moral responsibility (if any) of autonomous robots when doing harm. This volume will serve as an ideal educational source in engineering and robotics courses as well as an introductory reference for researchers in the field. (shrink)

This paper is reprinted in the book Machine Ethics, eds. M. Anderson and S. Anderson, Cambridge University Press, 2011.

A principal goal of the discipline of artificial morality is to design artificial agents to act as if they are moral agents. Intermediate goals of artificial morality are directed at building into AI systems sensitivity to the values, ethics, and legality of activities. The development of an effective foundation for the field of artificial morality involves exploring the technological and philosophical issues involved in making computers into explicit moral reasoners. The goal of this paper is to discuss strategies for implementing (...) artificial morality and the differing criteria for success that are appropriate to different strategies. (shrink)

A growing body of research can be found in which roboticists are designing for reciprocity as a key construct for successful human–robot interaction (HRI). Given the centrality of reciprocity as a component for our moral lives (for moral development and maintaining the just society), this paper confronts the possibility of what things would look like if the benchmark to achieve perceived reciprocity were accomplished. Through an analysis of the value of reciprocity from the care ethics tradition the richness of reciprocity (...) as an inherent value is revealed: on the micro-level, as mutual care for immediate care givers, and on the macro-level, as foundational for a just society. Taking this understanding of reciprocity into consideration, it becomes clear that HRI cannot achieve this bidirectional value of reciprocity; a robot must deceive users into believing it is capable of reciprocating to humans or is deserving of reciprocation from humans. Moreover, on the macro-level, designing social robots for reciprocity threatens the ability and willingness to reciprocate to human care workers across society. Because of these concerns, I suggest re-thinking the goals of reciprocity in social robotics. Designing for reciprocity in social robotics should be dedicated to the design of robots to enhance the ability to mutually care for those that provide us with care, as opposed to designing for reciprocity between human and robot. (shrink)

A prima facie analysis suggests that there are essentially two, mutually exclusive, ways in which risk arising from engineering design can be managed: by imposing external constraints on engineers or by engendering their feelings of responsibility and respect their autonomy. The author discusses the advantages and disadvantages of both approaches. However, he then shows that this opposition is a false one and that there is no simple relation between regulation and autonomy. Furthermore, the author argues that the most pressing need (...) is not more or less regulation but the further development of moral imagination. The enhancement of moral imagination can help engineers to discern the moral relevance of design problems, to create new design options, and to envisage the possible outcomes of their designs. The author suggests a dual program of developing regulatory frameworks that support engineers' autonomy and responsibility simultaneously with efforts to promote their moral imagination. He describes how some existing institutional changes have started off in this direction and proposes empirical research to take this further. (shrink)

The challenge of designing computer systems and robots with the ability to make moral judgments is stepping out of science fiction and moving into the laboratory. Engineers and scholars, anticipating practical necessities, are writing articles, participating in conference workshops, and initiating a few experiments directed at substantiating rudimentary moral reasoning in hardware and software. The subject has been designated by several names, including machine ethics, machine morality, artificial morality, or computational morality. Most references to the challenge elucidate one facet or (...) another of what is a very rich topic. This paper will offer a brief overview of the many dimensions of this new field of inquiry. (shrink)

Recently, there has been a resurgence of interest in general, comprehensive models of human cognition. Such models aim to explain higher-order cognitive faculties, such as deliberation and planning. Given a computational representation, the validity of these models can be tested in computer simulations such as software agents or embodied robots. The push to implement computational models of this kind has created the field of artificial general intelligence (AGI). Moral decision making is arguably one of the most challenging tasks for computational (...) approaches to higher-order cognition. The need for increasingly autonomous artificial agents to factor moral considerations into their choices and actions has given rise to another new field of inquiry variously known as Machine Morality, Machine Ethics, Roboethics, or Friendly AI. In this study, we discuss how LIDA, an AGI model of human cognition, can be adapted to model both affective and rational features of moral decision making. Using the LIDA model, we will demonstrate how moral decisions can be made in many domains using the same mechanisms that enable general decision making. Comprehensive models of human cognition typically aim for compatibility with recent research in the cognitive and neural sciences. Global workspace theory, proposed by the neuropsychologist Bernard Baars (1988), is a highly regarded model of human cognition that is currently being computationally instantiated in several software implementations. LIDA (Franklin, Baars, Ramamurthy, & Ventura, 2005) is one such computational implementation. LIDA is both a set of computational tools and an underlying model of human cognition, which provides mechanisms that are capable of explaining how an agent’s selection of its next action arises from bottom-up collection of sensory data and top-down processes for making sense of its current situation. We will describe how the LIDA model helps integrate emotions into the human decision-making process, and we will elucidate a process whereby an agent can work through an ethical problem to reach a solution that takes account of ethically relevant factors. (shrink)

A rule-based ethical theory is a good candidate for the practical reasoning of machine ethics because it generates duties or rules for action, and rules are computationally tractable. Among principle- or rule-based theories, the first formulation of Kant's categorical imperative offers a formalizable procedure. We explore a version of machine ethics along the lines of Kantian formalist ethics, both to suggest what computational structures such a view would require and to see what challenges remain for its successful implementation. In reformulating (...) Kant for the purposes of machine ethics, we consider three views of how the categorical imperative works: mere consistency, commonsense practical reasoning, and coherency. The first view envisions straightforward deductions of actions from facts. The second view incorporates recent work in nonmonotonic logic and commonsense reasoning. The last view takes ethical deliberation to follow a logic similar to that of belief revision. (shrink)

The possibility of autonomous artificially intelligent systems has awaken a well-known worry in the scientific community as well as in popular imaginary: the possibility that beings which have gained autonomous intelligence either turn against their creators or at least make the moral and ethical superiority of creators with respect to the created questionable. The present paper argues that such worries are wrong-headed. Specifically, if AAIs raise a worry about human ways of life or human value it is a worry for (...) a certain human way of thinking about what it is to be human. What is threatened is a way of thinking about what it is to be human, not human ways of life or human value. (shrink)