Computers are already approving financial transactions, controlling electrical supplies, and driving trains. Soon, service robots will be taking care of the elderly in their homes, and military robots will have their own targeting and firing protocols. Colin Allen and Wendell Wallach argue that as robots take on more and more responsibility, they must be programmed with moral decision-making abilities, for our own safety. Taking a fast paced tour through the latest thinking about philosophical ethics and artificial intelligence, the authors argue (...) that even if full moral agency for machines is a long way off, it is already necessary to start building a kind of functional morality, in which artificial moral agents have some basic ethical sensitivity. But the standard ethical theories don't seem adequate, and more socially engaged and engaging robots will be needed. As the authors show, the quest to build machines that are capable of telling right from wrong has begun. -/- Moral Machines is the first book to examine the challenge of building artificial moral agents, probing deeply into the nature of human decision making and ethics. (shrink)

This article described three heuristics that are employed in making judgements under uncertainty: representativeness, which is usually employed when people are asked to judge the probability that an object or event A belongs to class or process B; availability of instances or scenarios, which is often employed when people are asked to assess the frequency of a class or the plausibility of a particular development; and adjustment from an anchor, which is usually employed in numerical prediction when a relevant value (...) is available. These heuristics are highly economical and usually effective, but they lead to systematic and predictable errors. A better understanding of these heuristics and of the biases to which they lead could improve judgements and decisions in situations of uncertainty. (shrink)

The essays in this volume represent the first steps by philosophers and artificial intelligence researchers toward explaining why it is necessary to add an ...

This book explores the role of artificial intelligence in the development of a claim that morality is person-made and rational. Professor Danielson builds moral robots that do better than amoral competitors in a tournament of games like the Prisoners Dilemma and Chicken. The book thus engages in current controversies over the adequacy of the received theory of rational choice. It sides with Gauthier and McClennan, who extend the devices of rational choice to include moral constraint. Artificial Morality goes further, by (...) promoting communication, testing and copying of principles and by stressing empirical tests. (shrink)

In what ways should we include future humanoid robots, and other kinds of artificial agents, in our moral universe? We consider the Organic view, which maintains that artificial humanoid agents, based on current computational technologies, could not count as full-blooded moral agents, nor as appropriate targets of intrinsic moral concern. On this view, artificial humanoids lack certain key properties of biological organisms, which preclude them from having full moral status. Computationally controlled systems, however advanced in their cognitive or informational capacities, (...) are, it is proposed, unlikely to possess sentience and hence will fail to be able to exercise the kind of empathic rationality that is a prerequisite for being a moral agent. The organic view also argues that sentience and teleology require biologically based forms of self-organization and autonomous self-maintenance. The organic view may not be correct, but at least it needs to be taken seriously in the future development of the field of Machine Ethics. (shrink)

The 'Evolution of Cooperation' addresses a simple yet age-old question; If living things evolve through competition, how can cooperation ever emerge? Despite the abundant evidence of cooperation all around us, there existed no purely naturalistic answer to this question until 1979, when Robert Axelrod famously ran a computer tournament featuring a standard game-theory exercise called The Prisoner's Dilemma. To everyone's surprise, the program that won the tournament, named Tit for Tat, was not only the simplest but the most "cooperative" entrant. (...) This unexpected victory proved that cooperation - one might even say altruism - is mathematically possible and therefore needs no hidden hand or divine agent to create and sustain it. A great roadblock to the understanding of all sorts of behavior was at last removed. The updated edition includes an extensive new chapter on cooperation in cancer cells and among terrorist organizations. (shrink)

Research on moral judgment has been dominated by rationalist models, in which moral judgment is thought to be caused by moral reasoning. The author gives 4 reasons for considering the hypothesis that moral reasoning does not cause moral judgment; rather, moral reasoning is usually a post hoc construction, generated after a judgment has been reached. The social intuitionist model is presented as an alternative to rationalist models. The model is a social model in that it deemphasizes the private reasoning done (...) by individuals and emphasizes instead the importance of social and cultural influences. The model is an intuitionist model in that it states that moral judgment is generally the result of quick, automatic evaluations (intuitions). The model is more consistent than rationalist models with recent findings in social, cultural, evolutionary, and biological psychology, as well as in anthropology and primatology. (shrink)

As arti® cial intelligence moves ever closer to the goal of producing fully autonomous agents, the question of how to design and implement an arti® cial moral agent (AMA) becomes increasingly pressing. Robots possessing autonomous capacities to do things that are useful to humans will also have the capacity to do things that are harmful to humans and other sentient beings. Theoretical challenges to developing arti® cial moral agents result both from controversies among ethicists about moral theory itself, and from (...) computational limits to the implementation of such theories. In this paper the ethical disputes are surveyed, the possibility of a `moral Turing Test ’ is considered and the computational di culties accompanying the diŒerent types of approach are assessed. Human-like performance, which is prone to include immoral actions, may not be acceptable in machines, but moral perfection may be computationally unattainable. The risks posed by autonomous machines ignorantly or deliberately harming people and other sentient beings are great. The development of machines with enough intelligence to assess the eŒects of their actions on sentient beings and act accordingly may ultimately be the most important task faced by the designers of arti® cially intelligent automata. (shrink)

The implementation of moral decision making abilities in artificial intelligence (AI) is a natural and necessary extension to the social mechanisms of autonomous software agents and robots. Engineers exploring design strategies for systems sensitive to moral considerations in their choices and actions will need to determine what role ethical theory should play in defining control architectures for such systems. The architectures for morally intelligent agents fall within two broad approaches: the top-down imposition of ethical theories, and the bottom-up building of (...) systems that aim at goals or standards which may or may not be specified in explicitly theoretical terms. In this paper we wish to provide some direction for continued research by outlining the value and limitations inherent in each of these approaches. (shrink)

What is the nature of moral behavior? According to the study of bounded rationality, it results not from character traits or rational deliberation alone, but from the interplay between mind and environment. In this view, moral behavior is based on pragmatic social heuristics rather than moral rules or maximization principles. These social heuristics are not good or bad per se, but solely in relation to the environments in which they are used. This has methodological implications for the study of morality: (...) Behavior needs to be studied in social groups as well as in isolation, in natural environments as well as in labs. It also has implications for moral policy: Only by accepting the fact that behavior is a function of both mind and environmental structures can realistic prescriptive means of achieving moral goals be developed. (shrink)

Recently, there has been a resurgence of interest in general, comprehensive models of human cognition. Such models aim to explain higher-order cognitive faculties, such as deliberation and planning. Given a computational representation, the validity of these models can be tested in computer simulations such as software agents or embodied robots. The push to implement computational models of this kind has created the field of artificial general intelligence (AGI). Moral decision making is arguably one of the most challenging tasks for computational (...) approaches to higher-order cognition. The need for increasingly autonomous artificial agents to factor moral considerations into their choices and actions has given rise to another new field of inquiry variously known as Machine Morality, Machine Ethics, Roboethics, or Friendly AI. In this study, we discuss how LIDA, an AGI model of human cognition, can be adapted to model both affective and rational features of moral decision making. Using the LIDA model, we will demonstrate how moral decisions can be made in many domains using the same mechanisms that enable general decision making. Comprehensive models of human cognition typically aim for compatibility with recent research in the cognitive and neural sciences. Global workspace theory, proposed by the neuropsychologist Bernard Baars (1988), is a highly regarded model of human cognition that is currently being computationally instantiated in several software implementations. LIDA (Franklin, Baars, Ramamurthy, & Ventura, 2005) is one such computational implementation. LIDA is both a set of computational tools and an underlying model of human cognition, which provides mechanisms that are capable of explaining how an agent’s selection of its next action arises from bottom-up collection of sensory data and top-down processes for making sense of its current situation. We will describe how the LIDA model helps integrate emotions into the human decision-making process, and we will elucidate a process whereby an agent can work through an ethical problem to reach a solution that takes account of ethically relevant factors. (shrink)

Since its original publication in 1975, this groundbreaking work has awakened millions of concerned men and women to the shocking abuse of animals everywhere--inspiring a worldwide movement to eliminate much of the cruel and unnecessary laboratory animal experimentation of years past. In this newly revised and expanded edition, author Peter Singer exposes the chilling realities of today's "factory farms" and product-testing procedures--offering sound, humane solutions to what has become a profound environmental and social as well as moral issue. An important (...) and persuasive appeal to conscience, fairness, decency and justice, Animal Liberation is essential reading for the supporter and the skeptic alike.--From publisher description. (shrink)

Bentham's best-known book stands as a classic of both philosophy and jurisprudence. The 1789 work articulates an important statement of the foundations of utilitarian philosophy — it also represents a pioneering study of crime and punishment. Bentham's reasoning remains central to contemporary debates in moral and political philosophy, economics, and legal theory.

The foundation for a system of morals, this 1749 work is a landmark of moral and political thought. Its highly original theories of conscience, moral judgment, and virtue offer a reconstruction of the Enlightenment concept of social science, embracing both political economy and theories of law and government.

The thirty-five chapters in this book describe various judgmental heuristics and the biases they produce, not only in laboratory experiments but in important...

What psychological and philosophical significance should we attach to recent efforts at computer simulations of human cognitive capacities? In answering this question, I find it useful to distinguish what I will call "strong" AI from "weak" or "cautious" AI. According to weak AI, the principal value of the computer in the study of the mind is that it gives us a very powerful tool. For example, it enables us to formulate and test hypotheses in a more rigorous and precise fashion. (...) But according to strong AI, the computer is not merely a tool in the study of the mind; rather, the appropriately programmed computer really is a mind, in the sense that computers given the right programs can be literally said to. (shrink)

welcomed by a new generation of students and scholars in all branches of learning.

This book explores the role of artificial intelligence in the development of a claim that morality is person-made and rational. Professor Danielson builds moral robots that do better than amoral competitors in a tournament of games like the Prisoners Dilemma and Chicken. The book thus engages in current controversies over the adequacy of the received theory of rational choice. It sides with Gauthier and McClennan, who extend the devices of rational choice to include moral constraint. _Artificial Morality_ goes further, by (...) promoting communication, testing and copying of principles and by stressing empirical tests. (shrink)

The aim of the dissertation is to formulate a research program in moral cognition modeled on aspects of Universal Grammar and organized around three classic problems in moral epistemology: What constitutes moral knowledge? How is moral knowledge acquired? How is moral knowledge put to use? Drawing on the work of Rawls and Chomsky, a framework for investigating -- is proposed. The framework is defended against a range of philosophical objections and contrasted with the approach of developmentalists like Piaget and Kohlberg. (...) ;One chapter consists of an interpretation of the analogy Rawls draws in A Theory of Justice between moral theory and generative linguistics. A second chapter clarifies the empirical significance of Rawls' linguistic analogy by formulating a solution to the problem of descriptive adequacy with respect to a class of commonsense moral intuitions, including those discussed in the trolley problem literature originating in the work of Foot and Thomson. Three remaining chapters defend Rawls' linguistic analogy against some of its critics. In response to Hare's objection that Rawls' conception of moral theory is too empirical and insufficiently normative, it is argued that Hare fails to acknowledge both the centrality of the problem of empirical adequacy in the history of moral philosophy and the complexity of Rawls' approach to the problem of normative adequacy. In response to Nagel's claim that the analogy between moral theory and linguistics is false because whatever native speakers agree on is English, but whatever ordinary men agree in condemning is not necessarily wrong, it is argued that the criticism ignores both Rawls' use of the competence-performance distinction and the theory-dependence of the corresponding distinction in linguistics. In response to Dworkin's claim that Rawls' conception of moral theory is incompatible with naturalism and presupposes constructivism, it is argued that Dworkin's distinction between naturalism and constructivism represents a false antithesis; neither is an accurate interpretation of the model of moral theory Rawls describes in A Theory of Justice. The thesis concludes by situating Rawls' linguistic analogy within the context of broader debates in metaethics, democratic theory, natural law theory, and the theory of moral development. (shrink)

To what degree has biology influenced and shaped the development of moral systems? One way to determine the extent to which human moral systems might be the product of natural selection is to explore behaviour in other species that is analogous and perhaps homologous to our own. Many non-human primates, for example, have similar methods to humans for resolving, managing, and preventing conflicts of interests within their groups. Such methods, which include reciprocity and food sharing, reconciliation, consolation, conflict intervention, and (...) mediation, are the very building blocks of moral systems in that they are based on and facilitate cohesion among individuals and reflect a concerted effort by community members to find shared solutions to social conflict. Furthermore, these methods of resource distribution and conflict resolution often require or make use of capacities for empathy, sympathy, and sometimes even community concern. Non-human primates in societies in which such mechanisms are present may not be exactly moral beings, but they do show signs of a sense of social regularity that -- just like the norms and rules underlying human moral conduct -- promotes a mutually satisfactory modus vivendi. (shrink)

Marc Hauser puts forth the theory that humans have evolved a universal moral instinct, unconsciously propelling us to deliver judgments of right and wrong independent of gender, education, and religion. Combining his cutting-edge research with the latest findings in cognitive psychology, linguistics, neuroscience, evolutionary biology, economics, and anthropology, Hauser explores the startling implications of his provocative theory vis-à-vis contemporary bioethics, religion, the law, and our everyday lives.