Artificial agents (AAs), particularly but not only those in Cyberspace, extend the class of entities that can be involved in moral situations. For they can be conceived of as moral patients (as entities that can be acted upon for good or evil) and also as moral agents (as entities that can perform actions, again for good or evil). In this paper, we clarify the concept of agent and go on to separate the concerns of morality and responsibility of agents (most (...) interestingly for us, of AAs). We conclude that there is substantial and important scope, particularly in Computer Ethics, for the concept of moral agent not necessarily exhibiting free will, mental states or responsibility. This complements the more traditional approach, common at least since Montaigne and Descartes, which considers whether or not (artificial) agents have mental states, feelings, emotions and so on. By focussing directly on mind-less morality we are able to avoid that question and also many of the concerns of Artificial Intelligence. A vital component in our approach is the Method of Abstraction for analysing the level of abstraction (LoA) at which an agent is considered to act. The LoA is determined by the way in which one chooses to describe, analyse and discuss a system and its context. The Method of Abstraction is explained in terms of an interface or set of features or observables at a given LoA. Agenthood, and in particular moral agenthood, depends on a LoA. Our guidelines for agenthood are: interactivity (response to stimulus by change of state), autonomy (ability to change state without stimulus) and adaptability (ability to change the transition rules by which state is changed) at a given LoA. Morality may be thought of as a threshold defined on the observables in the interface determining the LoA under consideration. An agent is morally good if its actions all respect that threshold; and it is morally evil if some action violates it. That view is particularly informative when the agent constitutes a software or digital system, and the observables are numerical. Finally we review the consequences for Computer Ethics of our approach. In conclusion, this approach facilitates the discussion of the morality of agents not only in Cyberspace but also in the biosphere, where animals can be considered moral agents without their having to display free will, emotions or mental states, and in social contexts, where systems like organizations can play the role of moral agents. The primary cost of this facility is the extension of the class of agents and moral agents to embrace AAs. (shrink)

How should we understand the emotional rationality? This first part will explore two models of cognition and analogy strategies, test their intuition about the emotional desire. I distinguish between subjective and objective desire, then presents with a feeling from the "paradigm of drama" export semantics, here our emotional repertoire is acquired all the learned, and our emotions in the form of an object is fixed. It is pretty well in line with the general principles of rationality, especially the lowest reasonable (...) principles. Turned to the second part of this side of reasonable. I will inquire how emotional beliefs, desires, and behaviors contribute to the rationality. I will present a very general biological hypothesis: emotions by controlling highlights the characteristics of perception and reasoning, so that we remove the difficulties due in particular to lead to paralysis; they are being simulated by a simplified perception of information, thus limiting our practice and cognitive choice. How are we to understand emotional or axiological rationality? I pursue analogies with both the cognitive and the strategic models, testing them against intuitions about emotional desires. We distinguish two different classes of desires, the subjective and the objective, and propose that emotions have a semantics that derives from "paradigmatic scenarios", in terms of which our emotional repertoire is learned and the formal objects of our emotions fixed. This fits in well with emerging facts about how our emotional capacities develop, and it can also be squared with the general principles of rationality, particularly minimal rationality. In the second part, I return to the perspective of rationality. I ask how emotions contribute to the rationality of beliefs, desires, and behavior. I proffer a very general biological hypothesis: Emotions spare us the paralysis potentially induced by a particular predicament by controlling the salience of features of perception and reasoning; they temporarily mimic the informational encapsulation of perception and so circumscribe our practical and cognitive options. (shrink)

Machines will attain human levels of intelligence by the year 2040, predicts robotics expert Hans Moravec. And by 2050, they will have far surpassed us. In this mind-bending new book, Hans Moravec takes the reader on a roller coaster ride packed with such startling predictions. He tells us, for instance, that in the not-too-distant future, an army of robots will displace workers, causing massive, unprecedented unemployment. But then, says Moravec, a period of very comfortable existence will follow, as humans benefit (...) from a fully automated economy. And eventually, as machines evolve far beyond humanity, robots will supplant us. But if Moravec predicts the end of the domination by human beings, his is not a bleak vision. Far from railing against a future in which machines rule the world, Moravec embraces it, taking the startling view that intelligent robots will actually be our evolutionary heirs. "Intelligent machines, which will grow from us, learn our skills, and share our goals and values, can be viewed as children of our minds." And since they are our children, we will want them to outdistance us. In fact, in a bid for immortality, many of our descendants will choose to transform into "ex humans," as they upload themselves into advanced computers. We will become our children and live forever. In his provocative new book, the highly anticipated follow-up to his bestselling volume Mind Children, Moravec charts the trajectory of robotics in breathtaking detail. A must read for artificial intelligence, technology, and computer enthusiasts, Moravec's freewheeling but informed speculations present a future far different than we ever dared imagine. (shrink)

As arti® cial intelligence moves ever closer to the goal of producing fully autonomous agents, the question of how to design and implement an arti® cial moral agent (AMA) becomes increasingly pressing. Robots possessing autonomous capacities to do things that are useful to humans will also have the capacity to do things that are harmful to humans and other sentient beings. Theoretical challenges to developing arti® cial moral agents result both from controversies among ethicists about moral theory itself, and from (...) computational limits to the implementation of such theories. In this paper the ethical disputes are surveyed, the possibility of a `moral Turing Test ’ is considered and the computational di culties accompanying the diŒerent types of approach are assessed. Human-like performance, which is prone to include immoral actions, may not be acceptable in machines, but moral perfection may be computationally unattainable. The risks posed by autonomous machines ignorantly or deliberately harming people and other sentient beings are great. The development of machines with enough intelligence to assess the eŒects of their actions on sentient beings and act accordingly may ultimately be the most important task faced by the designers of arti® cially intelligent automata. (shrink)

A controversial scientific vision predicts a time in which humans and machines will merge and create a new form of non-biological intelligence, explaining how the occurrence will solve such issues as pollution, hunger, and aging.

How should we understand the emotional rationality? This first part will explore two models of cognition and analogy strategies, test their intuition about the emotional desire. I distinguish between subjective and objective desire, then presents with a feeling from the "paradigm of drama" export semantics, here our emotional repertoire is acquired all the learned, and our emotions in the form of an object is fixed. It is pretty well in line with the general principles of rationality, especially the lowest reasonable (...) principles. Turned to the second part of this side of reasonable. I will inquire how emotional beliefs, desires, and behaviors contribute to the rationality. I will present a very general biological hypothesis: emotions by controlling highlights the characteristics of perception and reasoning, so that we remove the difficulties due in particular to lead to paralysis; they are being simulated by a simplified perception of information, thus limiting our practice and cognitive choice. How are we to understand emotional or axiological rationality? I pursue analogies with both the cognitive and the strategic models, testing them against intuitions about emotional desires. We distinguish two different classes of desires, the subjective and the objective, and propose that emotions have a semantics that derives from "paradigmatic scenarios", in terms of which our emotional repertoire is learned and the formal objects of our emotions fixed. This fits in well with emerging facts about how our emotional capacities develop, and it can also be squared with the general principles of rationality, particularly minimal rationality. In the second part, I return to the perspective of rationality. I ask how emotions contribute to the rationality of beliefs, desires, and behavior. I proffer a very general biological hypothesis: Emotions spare us the paralysis potentially induced by a particular predicament by controlling the salience of features of perception and reasoning; they temporarily mimic the informational encapsulation of perception and so circumscribe our practical and cognitive options. (shrink)

This book explores the role of artificial intelligence in the development of a claim that morality is person-made and rational. Professor Danielson builds moral robots that do better than amoral competitors in a tournament of games like the Prisoners Dilemma and Chicken. The book thus engages in current controversies over the adequacy of the received theory of rational choice. It sides with Gauthier and McClennan, who extend the devices of rational choice to include moral constraint. Artificial Morality goes further, by (...) promoting communication, testing and copying of principles and by stressing empirical tests. (shrink)

This essay warns of eroding accountability in computerized societies. It argues that assumptions about computing and features of situations in which computers are produced create barriers to accountability. Drawing on philosophical analyses of moral blame and responsibility, four barriers are identified: 1) the problem of many hands, 2) the problem of bugs, 3) blaming the computer, and 4) software ownership without liability. The paper concludes with ideas on how to reverse this trend.

Building artificial moral agents (AMAs) underscores the fragmentary character of presently available models of human ethical behavior. It is a distinctly different enterprise from either the attempt by moral philosophers to illuminate the “ought” of ethics or the research by cognitive scientists directed at revealing the mechanisms that influence moral psychology, and yet it draws on both. Philosophers and cognitive scientists have tended to stress the importance of particular cognitive mechanisms, e.g., reasoning, moral sentiments, heuristics, intuitions, or a moral grammar, (...) in the making of moral decisions. However, assembling a system from the bottom-up which is capable of accommodating moral considerations draws attention to the importance of a much wider array of mechanisms in honing moral intelligence. Moral machines need not emulate human cognitive faculties in order to function satisfactorily in responding to morally significant situations. But working through methods for building AMAs will have a profound effect in deepening an appreciation for the many mechanisms that contribute to a moral acumen, and the manner in which these mechanisms work together. Building AMAs highlights the need for a comprehensive model of how humans arrive at satisfactory moral judgments. (shrink)

A classic collection of interlocking tales chronicles the near-future development of the robot and features models that have the ability to read minds, experience human emotions, and take over the world--and, perhaps, render humankind itself obsolete. Reprint. 25,000 first printing.

This collection of essays focuses on questions that arise when morality is considered from the perspective of recent work on rational choice and evolution. The contributors focus especially on modelling games like "The Prisoner's Dilemma". Included are noted philosophers like David Gauthier, Paul Churchland, Brian Skyrms, Ronald de Sousa, and Elliott Sober. This is the seventh volume in the Vancouver Studies in Cognitive Science series.

A principal goal of the discipline of artificial morality is to design artificial agents to act as if they are moral agents. Intermediate goals of artificial morality are directed at building into AI systems sensitivity to the values, ethics, and legality of activities. The development of an effective foundation for the field of artificial morality involves exploring the technological and philosophical issues involved in making computers into explicit moral reasoners. The goal of this paper is to discuss strategies for implementing (...) artificial morality and the differing criteria for success that are appropriate to different strategies. (shrink)

In his 1923 play R.U.R.: Rossum s Universal Robots, Karel Capek coined In 1993, we began a humanoid robotics project aimed at constructing a robot for use in exploring theories of human intelligence. In this article, we describe three aspects of our research methodology that distinguish our work from other humanoid projects. First, our humanoid robots are designed to act autonomously and safely in natural workspaces with people. Second, our robots are designed to interact socially with people by exploiting natural (...) human social cues. Third, we believe that robotics offers a unique tool for testing models of human intelligence drawn from developmental psychology and cognitive science. (shrink)