The psychological and neurobiological processes underlying moral judgement have been the focus of many recent empirical studies1–11. Of central interest is whether emotions play a causal role in moral judgement, and, in parallel, how emotion-related areas of the brain contribute to moral judgement. Here we show that six patients with focal bilateral damage to the ventromedial prefrontal cortex (VMPC), a brain region necessary for the normal generation of emotions and, in particular, social emotions12–14, produce an abnor- mally ‘utilitarian’ pattern of (...) judgements on moral dilemmas that pit compelling considerations of aggregate welfare against highly emotionally aversive behaviours (for example, having to sacrifice one person’s life to save a number of other lives)7,8. In contrast, the VMPC patients’ judgements were normal in other classes of moral dilemmas. These findings indicate that, for a selective set of moral dilemmas, the VMPC is critical for normal judgements of right and wrong. The findings support a necessary role for emotion in the generation of those judgements. (shrink)

Robot ethics encompasses ethical questions about how humans should design, deploy, and treat robots; machine morality encompasses questions about what moral capacities a robot should have and how these capacities could be computationally implemented. Publications on both of these topics have doubled twice in the past 10 years but have often remained separate from one another. In an attempt to better integrate the two, I offer a framework for what a morally competent robot would look like and discuss a number (...) of ethical questions about the design, use, and treatment of such moral robots in society. Instead of searching for a fixed set of criteria of a robot’s moral competence I identify the multiple elements that make up human moral competence and probe the possibility of designing robots that have one or more of these human elements, which include: moral vocabulary; a system of norms; moral cognition and affect; moral decision making and action; moral communication. Juxtaposing empirical research, philosophical debates, and computational challenges, this article adopts an optimistic perspective: if robotic design truly commits to building morally competent robots, then those robots could be trustworthy and productive partners, caretakers, educators, and members of the human community. Moral competence does not resolve all ethical concerns over robots in society, but it may be a prerequisite to resolve at least some of them. (shrink)

In recent years, a number of prominent scientists have argued that free will is an illusion, appealing to evidence demonstrating that information about brain activity can be used to predict behavior before people are aware of having made a decision. These scientists claim that the possibility of perfect prediction based on neural information challenges the ordinary understanding of free will. In this paper we provide evidence suggesting that most people do not view the possibility of neuro-prediction as a threat to (...) free will unless it also raises concerns about manipulation of the agent’s behavior. In Experiment 1 two scenarios described future brain imaging technology that allows perfect prediction of decisions and actions based on earlier neural activity, and this possibility did not undermine most people’s attributions of free will or responsibility, except in the scenario that also allowed manipulation. In Experiment 2 the scenarios increased the salience of the physicalist implications of neuro-prediction, while in Experiment 3 the scenarios suggested dualism, with perfect prediction by mindreaders. The patterns of results for these two experiments were similar to the results in Experiment 1, suggesting that participants do not understand free will to require specific metaphysical conditions regarding the mind–body relation. Most people seem to understand free will in a way that is not threatened by perfect prediction based on neural information, suggesting that they believe that just because “my brain made me do it,” that does not mean that I didn’t do it of my own free will. (shrink)

While there are many issues to be raised in using lethal autonomous robotic weapons (beyond those of remotely operated drones), we argue that the most important question is: should the decision to take a human life be relinquished to a machine? This question is often overlooked in favor of technical questions of sensor capability, operational questions of chain of command, or legal questions of sovereign borders. We further argue that the answer must be ?no? and offer several reasons for banning (...) autonomous robots. (1) Such a robot treats a human as an object, instead of as a person with inherent dignity. (2) A machine can only mimic moral actions, it cannot be moral. (3) A machine run by a program has no human emotions, no feelings about the seriousness of killing a human. (4) Using such a robot would be a violation of military honor. We therefore conclude that the use of an autonomous robot in lethal operations should be banned. (shrink)

Mind perception entails ascribing mental capacities to other entities, whereas moral judgment entails labeling entities as good or bad or actions as right or wrong. We suggest that mind perception is the essence of moral judgment. In particular, we suggest that moral judgment is rooted in a cognitive template of two perceived minds—a moral dyad of an intentional agent and a suffering moral patient. Diverse lines of research support dyadic morality. First, perceptions of mind are linked to moral judgments: dimensions (...) of mind perception (agency and experience) map onto moral types (agents and patients), and deficits of mind perception correspond to difficulties with moral judgment. Second, not only are moral judgments sensitive to perceived agency and experience, but all moral transgressions are fundamentally understood as agency plus experienced suffering—that is, interpersonal harm—even ostensibly harmless acts such as purity violations. Third, dyadic morality uniquely accounts for the phenomena of dyadic completion (seeing agents in response to patients, and vice versa), and moral typecasting (characterizing others as either moral agents or moral patients). Discussion also explores how mind perception can unify morality across explanatory levels, how a dyadic template of morality may be developmentally acquired, and future directions. (shrink)

This article discusses mechanisms and principles for assignment of moral responsibility to intelligent robots, with special focus on military robots. We introduce the concept autonomous power as a new concept, and use it to identify the type of robots that call for moral considerations. It is furthermore argued that autonomous power, and in particular the ability to learn, is decisive for assignment of moral responsibility to robots. As technological development will lead to robots with increasing autonomous power, we should be (...) prepared for a future when people blame robots for their actions. It is important to, already today, investigate the mechanisms that control human behavior in this respect. The results may be used when designing future military robots, to control unwanted tendencies to assign responsibility to the robots. Independent of the responsibility issue, the moral quality of robots’ behavior should be seen as one of many performance measures by which we evaluate robots. How to design ethics based control systems should be carefully investigated already now. From a consequentialist view, it would indeed be highly immoral to develop robots capable of performing acts involving life and death, without including some kind of moral framework. (shrink)

Research on moral judgment has been dominated by rationalist models, in which moral judgment is thought to be caused by moral reasoning. The author gives 4 reasons for considering the hypothesis that moral reasoning does not cause moral judgment; rather, moral reasoning is usually a post hoc construction, generated after a judgment has been reached. The social intuitionist model is presented as an alternative to rationalist models. The model is a social model in that it deemphasizes the private reasoning done (...) by individuals and emphasizes instead the importance of social and cultural influences. The model is an intuitionist model in that it states that moral judgment is generally the result of quick, automatic evaluations (intuitions). The model is more consistent than rationalist models with recent findings in social, cultural, evolutionary, and biological psychology, as well as in anthropology and primatology. (shrink)

The prospective robots in healthcare intended to be included within the conclave of the nurse-patient relationship—what I refer to as care robots—require rigorous ethical reflection to ensure their design and introduction do not impede the promotion of values and the dignity of patients at such a vulnerable and sensitive time in their lives. The ethical evaluation of care robots requires insight into the values at stake in the healthcare tradition. What’s more, given the stage of their development and lack of (...) standards provided by the International Organization for Standardization to guide their development, ethics ought to be included into the design process of such robots. The manner in which this may be accomplished, as presented here, uses the blueprint of the Value-sensitive design approach as a means for creating a framework tailored to care contexts. Using care values as the foundational values to be integrated into a technology and using the elements in care, from the care ethics perspective, as the normative criteria, the resulting approach may be referred to as care centered value-sensitive design. The framework proposed here allows for the ethical evaluation of care robots both retrospectively and prospectively. By evaluating care robots in this way, we may ultimately ask what kind of care we, as a society, want to provide in the future. (shrink)

Recently, there has been a resurgence of interest in general, comprehensive models of human cognition. Such models aim to explain higher-order cognitive faculties, such as deliberation and planning. Given a computational representation, the validity of these models can be tested in computer simulations such as software agents or embodied robots. The push to implement computational models of this kind has created the field of artificial general intelligence (AGI). Moral decision making is arguably one of the most challenging tasks for computational (...) approaches to higher-order cognition. The need for increasingly autonomous artificial agents to factor moral considerations into their choices and actions has given rise to another new field of inquiry variously known as Machine Morality, Machine Ethics, Roboethics, or Friendly AI. In this study, we discuss how LIDA, an AGI model of human cognition, can be adapted to model both affective and rational features of moral decision making. Using the LIDA model, we will demonstrate how moral decisions can be made in many domains using the same mechanisms that enable general decision making. Comprehensive models of human cognition typically aim for compatibility with recent research in the cognitive and neural sciences. Global workspace theory, proposed by the neuropsychologist Bernard Baars (1988), is a highly regarded model of human cognition that is currently being computationally instantiated in several software implementations. LIDA (Franklin, Baars, Ramamurthy, & Ventura, 2005) is one such computational implementation. LIDA is both a set of computational tools and an underlying model of human cognition, which provides mechanisms that are capable of explaining how an agent’s selection of its next action arises from bottom-up collection of sensory data and top-down processes for making sense of its current situation. We will describe how the LIDA model helps integrate emotions into the human decision-making process, and we will elucidate a process whereby an agent can work through an ethical problem to reach a solution that takes account of ethically relevant factors. (shrink)

Recent research in moral psychology has attempted to characterize patterns of moral judgments of actions in terms of the causal and intentional properties of those actions. The present study directly compares the roles of consequence, causation, belief and desire in determining moral judgments. Judgments of the wrongness or permissibility of action were found to rely principally on the mental states of an agent, while judgments of blame and punishment are found to rely jointly on mental states and the causal connection (...) of an agent to a harmful consequence. Also, selectively for judgments of punishment and blame, people who attempt but fail to cause harm more are judged more leniently if the harm occurs by independent means than if the harm does not occur at all. An account of these phenomena is proposed that distinguishes two processes of moral judgment: one which begins with harmful consequences and seeks a causally responsible agent, and the other which begins with an action and analyzes the mental states responsible for that action. Ó 2008 Elsevier B.V. All rights reserved. (shrink)

In this essay, I describe and explain the standard accounts of agency, natural agency, artificial agency, and moral agency, as well as articulate what are widely taken to be the criteria for moral agency, supporting the contention that this is the standard account with citations from such widely used and respected professional resources as the Stanford Encyclopedia of Philosophy, Routledge Encyclopedia of Philosophy, and the Internet Encyclopedia of Philosophy. I then flesh out the implications of some of these well-settled theories (...) with respect to the prerequisites that an ICT must satisfy in order to count as a moral agent accountable for its behavior. I argue that each of the various elements of the necessary conditions for moral agency presupposes consciousness, i.e., the capacity for inner subjective experience like that of pain or, as Nagel puts it, the possession of an internal something-of-which-it is-is-to-be-like. I ultimately conclude that the issue of whether artificial moral agency is possible depends on the issue of whether it is possible for ICTs to be conscious. (shrink)

Artificial agents (AAs), particularly but not only those in Cyberspace, extend the class of entities that can be involved in moral situations. For they can be conceived of as moral patients (as entities that can be acted upon for good or evil) and also as moral agents (as entities that can perform actions, again for good or evil). In this paper, we clarify the concept of agent and go on to separate the concerns of morality and responsibility of agents (most (...) interestingly for us, of AAs). We conclude that there is substantial and important scope, particularly in Computer Ethics, for the concept of moral agent not necessarily exhibiting free will, mental states or responsibility. This complements the more traditional approach, common at least since Montaigne and Descartes, which considers whether or not (artificial) agents have mental states, feelings, emotions and so on. By focussing directly on mind-less morality we are able to avoid that question and also many of the concerns of Artificial Intelligence. A vital component in our approach is the Method of Abstraction for analysing the level of abstraction (LoA) at which an agent is considered to act. The LoA is determined by the way in which one chooses to describe, analyse and discuss a system and its context. The Method of Abstraction is explained in terms of an interface or set of features or observables at a given LoA. Agenthood, and in particular moral agenthood, depends on a LoA. Our guidelines for agenthood are: interactivity (response to stimulus by change of state), autonomy (ability to change state without stimulus) and adaptability (ability to change the transition rules by which state is changed) at a given LoA. Morality may be thought of as a threshold defined on the observables in the interface determining the LoA under consideration. An agent is morally good if its actions all respect that threshold; and it is morally evil if some action violates it. That view is particularly informative when the agent constitutes a software or digital system, and the observables are numerical. Finally we review the consequences for Computer Ethics of our approach. In conclusion, this approach facilitates the discussion of the morality of agents not only in Cyberspace but also in the biosphere, where animals can be considered moral agents without their having to display free will, emotions or mental states, and in social contexts, where systems like organizations can play the role of moral agents. The primary cost of this facility is the extension of the class of agents and moral agents to embrace AAs. (shrink)

Participants compared the mental capacities of various human and nonhuman characters via online surveys. Factor analysis revealed two dimensions of mind perception, Experience and Agency. The dimensions predicted different moral judgments but were both related to valuing of mind.

According to previous research, threatening people’s belief in free will may undermine moral judgments and behavior. Four studies tested this claim. Study 1 used a Velten technique to threaten people’s belief in free will and found no effects on moral behavior, judgments of blame, and punishment decisions. Study 2 used six different threats to free will and failed to find effects on judgments of blame and wrongness. Study 3 found no effects on moral judgment when manipulating general free will beliefs (...) but found strong effects when manipulating the perceived choice capacity of the judged agent. Study 4 used pretested narratives that varied agents’ apparent free will and found that perceived choice capacity mediated the relationship between free will and blame. These results suggest that people’s general beliefs about whether free will exists have no impact on moral judgments but specific judgments about the agent’s choice capacity do. (shrink)

The uncanny valley—the unnerving nature of humanlike robots—is an intriguing idea, but both its existence and its underlying cause are debated. We propose that humanlike robots are not only unnerving, but are so because their appearance prompts attributions of mind. In particular, we suggest that machines become unnerving when people ascribe to them experience, rather than agency. Experiment 1 examined whether a machine’s humanlike appearance prompts both ascriptions of experience and feelings of unease. Experiment 2 tested whether a machine capable (...) of experience remains unnerving, even without a humanlike appearance. Experiment 3 investigated whether the perceived lack of experience can also help explain the creepiness of unfeeling humans and philosophical zombies. These experiments demonstrate that feelings of uncanniness are tied to perceptions of experience, and also suggest that experience—but not agency—is seen as fundamental to humans, and fundamentally lacking in machines. (shrink)

Empathy has become a common point of debate in moral psychology. Recent developments in psychiatry, neurosciences and social psychology have led to the revival of sentimentalism, and the ‘empathy thesis’ has suggested that affective empathy, in particular, is a necessary criterion of moral agency. The case of psychopaths – individuals incapable of affective empathy and moral agency, yet capable of rationality – has been utilised in support of this case. Critics, however, have been vocal. They have asserted that the case (...) of autism proves the empathy thesis wrong; that psychopathy centres on rational rather than empathic limitations; that empathy is not relevant to many common normative behaviours; and that rationality is required when empathy fails. The present paper analyses these four criticisms. It will be claimed that they each face severe difficulties, and that moral agency ought to be approached via a multi-tier model, with affective empathy as a baseline. (shrink)

Are disgusting or disrespectful actions judged to be moral violations, even when they are harmless? Stories about victimless yet offensive actions (such as cleaning one's toilet with a flag) were presented to Brazilian and U.S. adults and children of high and low socioeconomic status (N = 360). Results show that college students at elite universities judged these stories to be matters of social convention or of personal preference. Most other Ss, especially in Brazil, took a moralizing stance toward these actions. (...) For these latter Ss, moral judgments were better predicted by affective reactions than by appraisals of harmfulness. Results support the claims of cultural psychology (R.A. Shweder, 1991a) and suggest that cultural norms and culturally shaped emotions have a substantial impact on the domain of morality and the process of moral judgment. Suggestions are made for building cross-culturally valid models of moral judgment. (shrink)

Philosophers traditionally have been concerned both to explain intentional behavior and to evaluate it from a moral point of view. Some have maintained that whether actions (and their consequences) properly count as intended sometimes hinges on moral considerations - specifically, considerations of moral responsibility. The same claim has been made about an action's properly counting as having been done intentionally. These contentions will be made more precise in subsequent sections, where influential proponents are identified. Our aim in this paper is (...) to show that familiar defenses of these more precise claims are unpersuasive and that the claims do not merit acceptance. Our concern, more broadly, is to illuminate the place occupied by intention and intentional action in a conceptual scheme suited both to explanatory needs in the philosophy of mind to evaluative needs in moral philosophy. (shrink)

Can we build ‘moral robots’? If morality depends on emotions, the answer seems negative. Current robots do not meet standard necessary conditions for having emotions: they lack consciousness, mental states, and feelings. Moreover, it is not even clear how we might ever establish whether robots satisfy these conditions. Thus, at most, robots could be programmed to follow rules, but it would seem that such ‘psychopathic’ robots would be dangerous since they would lack full moral agency. However, I will argue that (...) in the future we might nevertheless be able to build quasi-moral robots that can learn to create the appearance of emotions and the appearance of being fully moral. I will also argue that this way of drawing robots into our social-moral world is less problematic than it might first seem, since human morality also relies on such appearances. (shrink)

One familiar affirmative answer to this question holds that these facts suffice to entail that Descartes' picture of the human mind must be mistaken. On Descartes' view, our mind or soul (the only essential part of ourselves) has no spatial location. Yet it directly interacts with but one physical object, the brain of that body with which it is, 'as it were, intermingled,' so as to 'form one unit.' The radical disparity posited between a nonspatial mind, whose intentional and conscious (...) properties are had by no physical object, and a spatial body, all of whose properties are had by no mind, has prompted some to conclude that, pace Descartes, causal interaction between the two is impossible. Jaegwon Kim has recently given a new twist to this old line of thought.(1) In the present essay, I will use Kim's argument as a springboard for motivating my own favored picture of the metaphysics of mind and body and then discussing how an often vilified account of freedom of the will may be realized within it. (shrink)

Abstract Semi-autonomous robotic weapons are already carving out a role for themselves in modern warfare. Recently, Ronald Arkin has argued that autonomous lethal robotic systems could be more ethical than humans on the battlefield, and that this marks a significant reason in favour of their development and use. Here I offer a critical response to the position advanced by Arkin. Although I am sympathetic to the spirit of the motivation behind Arkin's project and agree that if we decide to develop (...) and use these machines they ought to be programmed to behave ethically, there are several major problems with his view as it stands. At present, it is not clear whether such machines would be capable of behaving more ethically than humans. More importantly, to the extent that humans would remain in the context of war, human moral transgressions will continue, especially in the face of complicated ethical challenges accompanying automated warfare. Moreover, even if machines could be more ethical than humans in certain ways and in certain situations, this says nothing about whether warfare that contains these machines would itself be overall more ethical than warfare that does not include them as participants, or whether the inclusion of lethal robots is the best way to guard against human moral transgressions in war. (shrink)

In this paper, I present a general profile of individuals with psychopathy, autism, and acquired sociopathy as well as look specifically at the abilities of these individuals with respect to the moral domain. These individuals are individually and collectively interesting because of their significant affective and social impairments. I argue that none of these individuals should be considered full moral agents based on a proposed account of moral agency consisting of the following two necessary conditions: the capacity for moral judgment (...) and the capacity for moral motivation. Since each of these groups of individuals fails to satisfy the criteria in a different way, these arguments help to distinguish three primary ways in which emotion contributes to moral motivation. I conclude with a suggested general implication of this discussion for related debates in moral theory. (shrink)

Sophisticated technology is increasingly replacing human minds to perform complicated tasks in domains ranging from medicine to education to transportation. We investigated an important theoretical determinant of people's willingness to trust such technology to perform competently—the extent to which a nonhuman agent is anthropomorphized with a humanlike mind—in a domain of practical importance, autonomous driving. Participants using a driving simulator drove either a normal car, an autonomous vehicle able to control steering and speed, or a comparable autonomous vehicle augmented with (...) additional anthropomorphic features—name, gender, and voice. Behavioral, physiological, and self-report measures revealed that participants trusted that the vehicle would perform more competently as it acquired more anthropomorphic features. Technology appears better able to perform its intended design when it seems to have a humanlike mind. These results suggest meaningful consequences of humanizing technology, and also offer insights into the inverse process of objectifying humans. (shrink)

The prospective robots in healthcare intended to be included within the conclave of the nurse-patient relationship—what I refer to as care robots—require rigorous ethical reflection to ensure their design and introduction do not impede the promotion of values and the dignity of patients at such a vulnerable and sensitive time in their lives. The ethical evaluation of care robots requires insight into the values at stake in the healthcare tradition. What’s more, given the stage of their development and lack of (...) standards provided by the International Organization for Standardization to guide their development, ethics ought to be included into the design process of such robots. The manner in which this may be accomplished, as presented here, uses the blueprint of the Value-sensitive design approach as a means for creating a framework tailored to care contexts. Using care values as the foundational values to be integrated into a technology and using the elements in care, from the care ethics perspective, as the normative criteria, the resulting approach may be referred to as care centered value-sensitive design. The framework proposed here allows for the ethical evaluation of care robots both retrospectively and prospectively. By evaluating care robots in this way, we may ultimately ask what kind of care we, as a society, want to provide in the future. (shrink)

The growing field of machine morality has becoming increasingly concerned with how to develop artificial moral agents. However, there is little consensus on what constitutes an ideal moral agent let alone an artificial one. Leveraging a recent account of heroism in humans, the aim of this paper is to provide a prospective framework for conceptualizing, and in turn designing ideal artificial moral agents, namely those that would be considered heroic robots. First, an overview of what it means to be an (...) artificial moral agent is provided. Then, an overview of a recent account of heroism that seeks to define the construct as the dynamic and interactive integration of character strengths and situational constraints that afford the opportunity for moral behavior . With this as a foundation, a discussion is provided for what it might mean for a robot to be an ideal moral agent by proposing a dynamic and interactive connectionist model of robotic heroism. Given the limited accounts of robots engaging in moral behavior, a case for extending robotic moral capacities beyond just being a moral agent to the level of heroism is supported by drawing from exemplar situations where robots demonstrate heroism in popular film and fiction. (shrink)

It has long been known that psychopathology can influence social perception, but a 2D framework of mind perception provides the opportunity for an integrative understanding of some disorders. We examined the covariation of mind perception with three subclinical syndromes—autism-spectrum disorder, schizotypy, and psychopathy—and found that each presents a unique mind-perception profile. Autism-spectrum disorder involves reduced perception of agency in adult humans. Schizotypy involves increased perception of both agency and experience in entities generally thought to lack minds. Psychopathy involves reduced perception (...) of experience in adult humans, children, and animals. Disorders are differentially linked with the over- or underperception of agency and experience in a way that helps explain their real-world consequences. (shrink)