Building artificial moral agents (AMAs) underscores the fragmentary character of presently available models of human ethical behavior. It is a distinctly different enterprise from either the attempt by moral philosophers to illuminate the “ought” of ethics or the research by cognitive scientists directed at revealing the mechanisms that influence moral psychology, and yet it draws on both. Philosophers and cognitive scientists have tended to stress the importance of particular cognitive mechanisms, e.g., reasoning, moral sentiments, heuristics, intuitions, or a moral grammar, (...) in the making of moral decisions. However, assembling a system from the bottom-up which is capable of accommodating moral considerations draws attention to the importance of a much wider array of mechanisms in honing moral intelligence. Moral machines need not emulate human cognitive faculties in order to function satisfactorily in responding to morally significant situations. But working through methods for building AMAs will have a profound effect in deepening an appreciation for the many mechanisms that contribute to a moral acumen, and the manner in which these mechanisms work together. Building AMAs highlights the need for a comprehensive model of how humans arrive at satisfactory moral judgments. (shrink)

Imitation, deliberation, and mindreading are characteristically human sociocognitive skills. Research on imitation and its role in social cognition is flourishing across various disciplines. Imitation is surveyed in this target article under headings of behavior, subpersonal mechanisms, and functions of imitation. A model is then advanced within which many of the developments surveyed can be located and explained. The shared circuits model (SCM) explains how imitation, deliberation, and mindreading can be enabled by subpersonal mechanisms of control, mirroring, and simulation. It is (...) cast at a middle, functional level of description, that is, between the level of neural implementation and the level of conscious perceptions and intentional actions. The SCM connects shared informational dynamics for perception and action with shared informational dynamics for self and other, while also showing how the action/perception, self/other, and actual/possible distinctions can be overlaid on these shared informational dynamics. It avoids the common conception of perception and action as separate and peripheral to central cognition. Rather, it contributes to the situated cognition movement by showing how mechanisms for perceiving action can be built on those for active perception.;>;>The SCM is developed heuristically, in five layers that can be combined in various ways to frame specific ontogenetic or phylogenetic hypotheses. The starting point is dynamic online motor control, whereby an organism is closely attuned to its embedding environment through sensorimotor feedback. Onto this are layered functions of prediction and simulation of feedback, mirroring, simulation of mirroring, monitored inhibition of motor output, and monitored simulation of input. Finally, monitored simulation of input specifying possible actions plus inhibited mirroring of such possible actions can generate information about the possible as opposed to actual instrumental actions of others, and the possible causes and effects of such possible actions, thereby enabling strategic social deliberation. Multiple instances of such shared circuits structures could be linked into a network permitting decomposition and recombination of elements, enabling flexible control, imitative learning, understanding of other agents, and instrumental and strategic deliberation. While more advanced forms of social cognition, which require tracking multiple others and their multiple possible actions, may depend on interpretative theorizing or language, the SCM shows how layered mechanisms of control, mirroring, and simulation can enable distinctively human cognitive capacities for imitation, deliberation, and mindreading. (shrink)

The so-called "Prisoner''s Dilemma" is often referred to in business ethics, but probably not well understood. This article has three parts: (1) I claim that models derived from game theory are significant in the field for discussions of prudential ethics and the practical decisions managers make; (2) I discuss using them as a practical pedagogical exercise and some of the lessons generated; (3) more speculatively, I suggest that they are useful in discussions of corporate personhood.

Artificial agents (AAs), particularly but not only those in Cyberspace, extend the class of entities that can be involved in moral situations. For they can be conceived of as moral patients (as entities that can be acted upon for good or evil) and also as moral agents (as entities that can perform actions, again for good or evil). In this paper, we clarify the concept of agent and go on to separate the concerns of morality and responsibility of agents (most (...) interestingly for us, of AAs). We conclude that there is substantial and important scope, particularly in Computer Ethics, for the concept of moral agent not necessarily exhibiting free will, mental states or responsibility. This complements the more traditional approach, common at least since Montaigne and Descartes, which considers whether or not (artificial) agents have mental states, feelings, emotions and so on. By focussing directly on mind-less morality we are able to avoid that question and also many of the concerns of Artificial Intelligence. A vital component in our approach is the Method of Abstraction for analysing the level of abstraction (LoA) at which an agent is considered to act. The LoA is determined by the way in which one chooses to describe, analyse and discuss a system and its context. The Method of Abstraction is explained in terms of an interface or set of features or observables at a given LoA. Agenthood, and in particular moral agenthood, depends on a LoA. Our guidelines for agenthood are: interactivity (response to stimulus by change of state), autonomy (ability to change state without stimulus) and adaptability (ability to change the transition rules by which state is changed) at a given LoA. Morality may be thought of as a threshold defined on the observables in the interface determining the LoA under consideration. An agent is morally good if its actions all respect that threshold; and it is morally evil if some action violates it. That view is particularly informative when the agent constitutes a software or digital system, and the observables are numerical. Finally we review the consequences for Computer Ethics of our approach. In conclusion, this approach facilitates the discussion of the morality of agents not only in Cyberspace but also in the biosphere, where animals can be considered moral agents without their having to display free will, emotions or mental states, and in social contexts, where systems like organizations can play the role of moral agents. The primary cost of this facility is the extension of the class of agents and moral agents to embrace AAs. (shrink)

Although virtue ethics has repeatedly been proposed as a suitable framework for the development of artificial moral agents, it has been proven difficult to approach from a computational perspective. In this work, we present the first technical implementation of artificial virtuous agents in moral simulations. First, we review previous conceptual and technical work in artificial virtue ethics and describe a functionalistic path to AVAs based on dispositional virtues, bottom-up learning, and top-down eudaimonic reward. We then provide the details of a (...) technical implementation in a moral simulation based on a tragedy of the commons scenario. The experimental results show how the AVAs learn to tackle cooperation problems while exhibiting core features of their theoretical counterpart, including moral character, dispositional virtues, learning from experience, and the pursuit of eudaimonia. Ultimately, we argue that virtue ethics provides a compelling path toward morally excellent machines and that our work provides an important starting point for such endeavors. (shrink)

Virtue ethics has many times been suggested as a promising recipe for the construction of artificial moral agents due to its emphasis on moral character and learning. However, given the complex nature of the theory, hardly any work has de facto attempted to implement the core tenets of virtue ethics in moral machines. The main goal of this paper is to demonstrate how virtue ethics can be taken all the way from theory to machine implementation. To achieve this goal, we (...) critically explore the possibilities and challenges for virtue ethics from a computational perspective. Drawing on previous conceptual and technical work, we outline a version of artificial virtue based on moral functionalism, connectionist bottom–up learning, and eudaimonic reward. We then describe how core features of the outlined theory can be interpreted in terms of functionality, which in turn informs the design of components necessary for virtuous cognition. Finally, we present a comprehensive framework for the technical development of artificial virtuous agents and discuss how they can be implemented in moral environments. (shrink)

The field of AI ethics during the current and previous decade is receiving an increasing amount of attention from all involved stakeholders: the public, science, philosophy, religious organizations, enterprises, governments, and various organizations. However, this field currently lacks consensus on scope, ethico-philosophical foundations, or common methodology. This thesis aims to contribute towards filling this gap by providing an answer to the two main research questions: first, what theory can explain moral scenarios in which AI entities are participants?; and second, what (...) theory can explain the process of moral reasoning, decision and action, for AI entities in virtual, simulated and real-life moral scenarios? This thesis answers these two research questions with its two main contributions to the field of AI ethics, a substantial and a methodological contribution. The substantial contribution is a coherent and novel theory named Ethics of Systems Framework, as well as a possible inception of a new field of study: ethics of systems. The methodological contribution is the creation of its main methodological tool, the Ethics of Systems Interface. The second part of the research effort was focused on testing and demonstrating the capacities of the Ethics of Systems Framework and Interface in modeling and managing moral scenarios in which AI and other entities participate. Further work can focus on building on top of the foundations of the Framework provided here, increasing the scope of moral theories and simulated scenarios, improving the level of detail and parameters to reflect real-life situations, and field-testing the Framework on actual AI systems. (shrink)

Technological artefacts have, in recent years, invited increasingly intimate ways of interaction. But surprisingly little attention has been devoted to how such interactions, like with wearable devices or household robots, shape our minds, cognitive capacities, and moral character. In this thesis, I develop an embodied, enactive account of mind--technology interaction that takes the reciprocal influence of artefacts on minds seriously. First, I examine how recent developments in philosophy of technology can inform the phenomenology of mind--technology interaction as seen through an (...) enactivist lens. Second, I show how an enactive account of remembering can improve operationalizations of the memory palace mnemonic through virtual reality devices. Third, I draw on virtue ethics to argue that an enactivist approach allows us to better grasp the morally shaping aspects of artefacts by looking at social robots. Fourth, I fend off an underlying metaphysical concern about enactivism by arguing that an embodied, enactive account is compatible with the multiple realization of cognitive processes. This principle is often seen as a crucial test favouring accounts such as extended functionalism over enactivism and I argue that some forms of enactivism pass this test as well. Finally, I conclude by considering what the future relationship between enactivism and functionalism may have in store for the study of mind--technology interaction. (shrink)

We propose that virtue ethics can be used to address ethical issues central to discussions about sex robots. In particular, we argue virtue ethics is well equipped to focus on the implications of sex robots for human moral character. Our evaluation develops in four steps. First, we present virtue ethics as a suitable framework for the evaluation of human–robot relationships. Second, we show the advantages of our virtue ethical account of sex robots by comparing it to current instrumentalist approaches, showing (...) how the former better captures the reciprocal interaction between robots and their users. Third, we examine how a virtue ethical analysis of intimate human–robot relationships could inspire the design of robots that support the cultivation of virtues. We suggest that a sex robot which is equipped with a consent-module could support the cultivation of compassion when used in supervised, therapeutic scenarios. Fourth, we discuss the ethical implications of our analysis for user autonomy and responsibility. (shrink)

Among ethicists and engineers within robotics there is an ongoing discussion as to whether ethical robots are possible or even desirable. We answer both of these questions in the positive, based on an extensive literature study of existing arguments. Our contribution consists in bringing together and reinterpreting pieces of information from a variety of sources. One of the conclusions drawn is that artifactual morality must come in degrees and depend on the level of agency, autonomy and intelligence of the machine. (...) Moral concerns for agents such as intelligent search machines are relatively simple, while highly intelligent and autonomous artifacts with significant impact and complex modes of agency must be equipped with more advanced ethical capabilities. Systems like cognitive robots are being developed that are expected to become part of our everyday lives in future decades. Thus, it is necessary to ensure that their behaviour is adequate. In an analogy with artificial intelligence, which is the ability of a machine to perform activities that would require intelligence in humans, artificial morality is considered to be the ability of a machine to perform activities that would require morality in humans. The capacity for artificial (artifactual) morality, such as artifactual agency, artifactual responsibility, artificial intentions, artificial (synthetic) emotions, etc., come in varying degrees and depend on the type of agent. As an illustration, we address the assurance of safety in modern High Reliability Organizations through responsibility distribution. In the same way that the concept of agency is generalized in the case of artificial agents, the concept of moral agency, including responsibility, is generalized too. We propose to look at artificial moral agents as having functional responsibilities within a network of distributed responsibilities in a socio-technological system. This does not take away the responsibilities of the other stakeholders in the system, but facilitates an understanding and regulation of such networks. It should be pointed out that the process of development must assume an evolutionary form with a number of iterations because the emergent properties of artifacts must be tested in real world situations with agents of increasing intelligence and moral competence. We see this paper as a contribution to the macro-level Requirement Engineering through discussion and analysis of general requirements for design of ethical robots. (shrink)

Can computer systems ever be considered moral agents? This paper considers two factors that are explored in the recent philosophical literature. First, there are the important domains in which computers are allowed to act, made possible by their greater functional capacities. Second, there is the claim that these functional capacities appear to embody relevant human abilities, such as autonomy and responsibility. I argue that neither the first (Domain-Function) factor nor the second (Simulacrum) factor gets at the central issue in the (...) case for computer moral agency: whether they can have the kinds of intentional states that cause their decisions and actions. I give an account that builds on traditional action theory and allows us to conceive of computers as genuine moral agents in virtue of their own causally efficacious intentional states. These states can cause harm or benefit to moral patients, but do not depend on computer consciousness or intelligence. (shrink)

Recently, there has been a resurgence of interest in general, comprehensive models of human cognition. Such models aim to explain higher-order cognitive faculties, such as deliberation and planning. Given a computational representation, the validity of these models can be tested in computer simulations such as software agents or embodied robots. The push to implement computational models of this kind has created the field of artificial general intelligence (AGI). Moral decision making is arguably one of the most challenging tasks for computational (...) approaches to higher-order cognition. The need for increasingly autonomous artificial agents to factor moral considerations into their choices and actions has given rise to another new field of inquiry variously known as Machine Morality, Machine Ethics, Roboethics, or Friendly AI. In this study, we discuss how LIDA, an AGI model of human cognition, can be adapted to model both affective and rational features of moral decision making. Using the LIDA model, we will demonstrate how moral decisions can be made in many domains using the same mechanisms that enable general decision making. Comprehensive models of human cognition typically aim for compatibility with recent research in the cognitive and neural sciences. Global workspace theory, proposed by the neuropsychologist Bernard Baars (1988), is a highly regarded model of human cognition that is currently being computationally instantiated in several software implementations. LIDA (Franklin, Baars, Ramamurthy, & Ventura, 2005) is one such computational implementation. LIDA is both a set of computational tools and an underlying model of human cognition, which provides mechanisms that are capable of explaining how an agent’s selection of its next action arises from bottom-up collection of sensory data and top-down processes for making sense of its current situation. We will describe how the LIDA model helps integrate emotions into the human decision-making process, and we will elucidate a process whereby an agent can work through an ethical problem to reach a solution that takes account of ethically relevant factors. (shrink)

We can learn about human ethics from machines. We discuss the design of a working machine for making ethical decisions, the N-Reasons platform, applied to the ethics of robots. This N-Reasons platform builds on web based surveys and experiments, to enable participants to make better ethical decisions. Their decisions are better than our existing surveys in three ways. First, they are social decisions supported by reasons. Second, these results are based on weaker premises, as no exogenous expertise (aside from that (...) provided by the participants) is needed to seed the survey. Third, N-Reasons is designed to support experiments so we can learn how to improve the platform. We sketch experimental results that show the platform is a success as well as pointing to ways it can be improved. (shrink)

A principal goal of the discipline of artificial morality is to design artificial agents to act as if they are moral agents. Intermediate goals of artificial morality are directed at building into AI systems sensitivity to the values, ethics, and legality of activities. The development of an effective foundation for the field of artificial morality involves exploring the technological and philosophical issues involved in making computers into explicit moral reasoners. The goal of this paper is to discuss strategies for implementing (...) artificial morality and the differing criteria for success that are appropriate to different strategies. (shrink)

Duncan MacIntosh has argued that David Gauthier's notion of a constrained maximization disposition faces a dilemma. For if such a disposition is revocable, it is no longer rational come the time to act on it, and so acting on it is not (as Gauthier argues) rational; but if it is not revocable, acting on it is not voluntary. This paper is a response to MacIntosh's dilemma. I introduce an account of rational intention of a type which has become increasingly and (...) independently prominent in the literature, and argue that, on this account, rational and voluntary constraint is possible. (shrink)

Is game theory good for us? This may seem an odd question. In the strict sense, game theory—the axiomatic account of interaction between rational agents—is as morally neutral as arithmetic. But the popularization of game theory as a way of thinking about social interaction is far from neutral. Consider the contrast between characterizing bargaining over distribution as a “zero-sum society” and focussing on “win-win” cooperative solutions. These reflections bring us to the book under review, Prisoner's Dilemma, a popular introduction to (...) game theory and its relation to ethics by the respected science writer William Poundstone. The book begins with a moral dilemma and ends by discussing the evolution of co-operation. Poundstone emphasizes—correctly, to my mind—the ethical potential of game theory. He concludes his first chapter with this striking claim: “Today's practitioners of game theory are attempting to forge a kind of ethical progress. Is there any way to promote the common good in a prisoner's dilemma? The attempt to answer this question is one of the great intellectual adventures of our time”. (shrink)

Artificial Intelligence (AI) is a technology widely used to support human decision-making. Current areas of application include financial services, engineering, and management. A number of attempts to introduce AI decision support systems into areas which more obviously include moral judgement have been made. These include systems that give advice on patient care, on social benefit entitlement, and even ethical advice for medical professionals. Responding to these developments raises a complex set of moral questions. This paper proposes a clearer replacement question (...) to them. The replacement question asks under what circumstances, if any, people would accept a moral judgement made by some sort of machine. Since, it is argued, the answer to this replacement question is positive, urgent practical moral problems are raised. (shrink)

Abstract Newcomb's problem is regularly described as a problem arising from equally defensible yet contradictory models of rationality. Braess? paradox is regularly described as nothing more than the existence of non?intuitive (but ultimately non?contradictory) equilibrium points within physical networks of various kinds. Yet it can be shown that Newcomb's problem is structurally identical to Braess? paradox. Both are instances of a well?known result in game theory, namely that equilibria of non?cooperative games are generally Pareto?inefficient. Newcomb's problem is simply a limiting (...) case in which the number of players equals one. Braess? paradox is another limiting case in which the ?players? need not be assumed to be discrete individuals. The result is that Newcomb's problem is no more difficult to solve than (the easy to solve) Braess? paradox. (shrink)

Research into the ethics of artificial intelligence is often categorized into two subareas—robot ethics and machine ethics. Many of the definitions and classifications of the subject matter of these subfields, as found in the literature, are conflated, which I seek to rectify. In this essay, I infer that using the term ‘machine ethics’ is too broad and glosses over issues that the term computational ethics best describes. I show that the subject of inquiry of computational ethics is of great value (...) and indeed is an important frontier in developing ethical artificial intelligence systems. I also show that computational is a distinct, often neglected field in the ethics of AI. In contrast to much of the literature, I argue that the appellation ‘machine ethics’ does not sufficiently capture the entire project of embedding ethics into AI/S, and hence the need for computational ethics. This essay is unique for two reasons; first, it offers a philosophical analysis of the subject of computational ethics that is not found in the literature. Second, it offers a finely grained analysis that shows the thematic distinction among robot ethics, machine ethics and computational ethics. (shrink)

Kin selection, reciprocity and group selection are widely regarded as evolutionary mechanisms capable of sustaining altruism among humans andother cooperative species. Our research indicates, however, that these mechanisms are only particular examples of a broader set of evolutionary possibilities.In this paper we present the results of a series of simple replicator simulations, run on variations of the 2–player prisoner's dilemma, designed to illustrate the wide range of scenarios under which altruism proves to be robust under evolutionary pressures. The set of (...) mechanisms we explore is divided into four categories :correlation, group selection, imitation, and punishment. We argue that correlation is the core phenomenon at work in all four categories. (shrink)

The article investigates the interplay of moral rules in computer simulation. The investigation is based on two situations which are well-known to game theory: the prisoner''s dilemma and the game of Chicken. The prisoner''s dilemma can be taken to represent contractual situations, the game of Chicken represents a competitive situation on the one hand and the provision for a common good on the other. Unlike the rules usually used in game theory, each player knows the other''s strategy. In that way, (...) ever higher levels of reflection are reached reciprocally. Such strategies can be interpreted as moral rules.Artificial morality is related to the discipline of Artificial Life. As in artificial life, the use of genetic algorithms suggests itself. Rules of behaviour split and reunite as chromosome strings do. (shrink)

Michael Bratman''s restricted two-tier approach to rationalizing the stability of intentions contrasts with an alternative view of planning, for which all of the following claims are made: (a) it shares with Bratman''s restricted two-tier approach the virtue of reducing the magnitude of Smart''s problem; (2) it, rather than the unrestricted two-tier approach, is what is argued for in McClennen (1990); (3) there does not appear to be anything in the central analysis that Bratman has provided of plans and intentions (both (...) in his book,Intention, Plans, and Practical Reason, and in the article Planning and the Stability of Intention) that precludes his adopting this alternative approach; and (4) it is an approach that neither requires nor encourages any distinction between the standards appropriate to artificial and to human intelligence. (shrink)

Morality is serious yet it needs to be reconciled with the free play of alternatives that characterizes rational and ethical agency. Beginning with a sketch of the seriousness of morality modeled as a constraint, this paper introduces a technical conception of play as degrees of freedom. We consider two ways to apply game theory to ethics, rationalist and evolutionary game theory, contrasting the way they model moral constraint. Freedom in the rationalist account is problematic, subverting willful commitment. In the evolutionary (...) account, freedom is also problematic, leading to an infinity of possible social norms with dubious normative force. While these two approaches complement each other, we argue that the evolutionary approach is superior on both theoretical and practical grounds. (shrink)

The challenge of designing computer systems and robots with the ability to make moral judgments is stepping out of science fiction and moving into the laboratory. Engineers and scholars, anticipating practical necessities, are writing articles, participating in conference workshops, and initiating a few experiments directed at substantiating rudimentary moral reasoning in hardware and software. The subject has been designated by several names, including machine ethics, machine morality, artificial morality, or computational morality. Most references to the challenge elucidate one facet or (...) another of what is a very rich topic. This paper will offer a brief overview of the many dimensions of this new field of inquiry. (shrink)

Philosophers of information, according to Luciano Floridi (The philosophy of information. Oxford University Press, Oxford, 2010, p 32), study how information should be “adequately created, processed, managed, and used.” A small number of epistemologists have employed the concept of information as a cornerstone of their theoretical framework. How this concept can be used to make sense of seemingly intractable epistemological problems, however, has not been widely explored. This paper examines Fred Dretske’s information-based epistemology, in particular his response to radical epistemological (...) skepticism. We discuss the relationship between information, evidence and knowledge in relation to the problem of skepticism and the options available to an information-based epistemology for dealing with it. (shrink)

In this paper we present an executable approach to model interactions between agents that involve sensitive, privacy-related information. The approach is formal and based on deontic, epistemic and action logic. It is conceptually related to the Belief-Desire-Intention model of Bratman. Our approach uses the concept of sphere as developed by Waltzer to capture the notion that information is provided mostly with restrictions regarding its application. We use software agent technology to create an executable approach. Our agents hold beliefs about the (...) world, have goals and commitment to the goals. They have the capacity to reason about different courses of action, and communicate with one another. The main new ingredient of our approach is the idea to model information itself as an intentional agent whose main goal it is to preserve the integrity of the information and regulate its dissemination. We demonstrate our approach by applying it to an important process in the insurance industry: applying for a life insurance. In this paper we will: (1) describe the challenge organizational complexity poses in moral reasoning about informational relationships; (2) propose an executable approach, using software agents with reasoning capacities grounded in modal logic, in which moral constraints on informational relatio nships can be modeled and investigated; (3) describe the details of our approach, in which information itself is modeled as an intentional agent in its own right; (4) test and validate it by applying it to a concrete ‘hard case’ from the insurance industry; and (5) conclude that our approach upholds and offers potential for both research and practical application. (shrink)

By his definition of them, David Gauthier's co-operative constrained maximizers are not necessarily unsharing and disposed to exclude straight maximizers from benefits of their co-operation. Here is Gauthier's full and exact account, his official account, of constrained maximization.