The question of whether AI systems such as robots can or should be afforded moral agency or patiency is not one amenable either to discovery or simple reasoning, because we as societies constantly reconstruct our artefacts, including our ethical systems. Consequently, the place of AI systems in society is a matter of normative, not descriptive ethics. Here I start from a functionalist assumption, that ethics is the set of behaviour that maintains a society. This assumption allows me to exploit the (...) theoretical biology of sociality and autonomy to explain our moral intuitions. From this grounding I extend to consider possible ethics for maintaining either human- or of artefact-centred societies. I conclude that while constructing AI systems as either moral agents or patients is possible, neither is desirable. In particular, I argue that we are unlikely to construct a coherent ethics in which it it is ethical to afford AI moral subjectivity. We are therefore obliged not to build AI we are obliged to. (shrink)

When we evaluate moral agents, we consider many factors, including whether the agent acted freely, or under duress or coercion. In turn, moral evaluations have been shown to influence our (non-moral) evaluations of these same factors. For example, when we judge an agent to have acted immorally, we are subsequently more likely to judge the agent to have acted freely, not under force. Here, we investigate the cognitive signatures of this effect in interpersonal situations, in which one agent (“forcer”) forces (...) another agent (“forcee”) to act either immorally or morally. The structure of this relationship allowed us to ask questions about both the “forcer” and the “forcee.” Paradoxically, participants judged that the “forcer” forced the “forcee” to act immorally (i.e. X forced Y), but that the “forcee” was not forced to act immorally (i.e. Y was not forced by X). This pattern obtained only for human agents who acted intentionally. Directly changing participants’ focus from one agent to another (forcer vs. forcee) also changed the target of moral evaluation and therefore force attributions. The full pattern of judgments may provide a window into motivated moral reasoning and focusing bias more generally; participants may have been motivated to attribute greater force to the immoral forcer and greater freedom to the immoral forcee. (shrink)

According to the discontinuity view, people recognize a deep discontinuity between phenomenal and intentional states, such that they refrain from attributing feelings and experiences to entities that do not have the right kind of body, though they may attribute thoughts to entities that lack a biological body, like corporations, robots, and disembodied souls. We examine some of the research that has been used to motivate the discontinuity view. Specifically, we focus on experiments that examine people's aptness judgments for various mental (...) state ascriptions to groups. These studies seem to reveal that people are more inclined to think of groups as having intentionality than as having phenomenology. Combined with the fact that groups obviously lack a single biological body, this has been taken as evidence that people operate according to the relevant discontinuity. However, these studies support discontinuity only on the assumption that the experimental participants are interpreting the relevant group mental state ascriptions in a specific way. We present evidence that people are not interpreting these ascriptions in a way that supports discontinuity. Instead, we argue that people generally interpret group mental state ascriptions distributively, as attributions of mental states to various group members. (shrink)

In this article, we present evidence of a bidirectional coupling between moral concern and the attribution of properties and states that are associated with experience (e.g., conscious awareness, feelings). This coupling is also shown to be stronger with experience than for the attribution of properties and states more closely associated with agency (e.g., free will, thoughts). We report the results of four studies. In the first two studies, we vary the description of the mental capacities of a creature, and assess (...) the effects of these manipulations on moral concern. The third and fourth studies examine the effects of variations in moral concern on attributions of mindedness. Results from the first two studies indicate that moral concern depends primarily on the attribution of experience, rather than the attribution of agency. The results of the latter two studies demonstrate that moral concern increases attributions of mindedness, and that this effect is stronger for attributions of experience than for attributions of agency. (shrink)

I here consider the reasonableness of punishing future autonomous military robots. I argue that it is an engineering desideratum that these devices be responsive to moral considerations as well as human criticism and blame. Additionally, I argue that someday it will be possible to build such machines. I use these claims to respond to the no subject of punishment objection to deploying autonomous military robots, the worry being that an “accountability gap” could result if the robot committed a war crime. (...) (Sparrow 2007). But these robots will be blameworthy and morally responsible for their wrongful conduct. I then show that we have reasons to punish them based both on good effects and the impact on the robot, itself. (shrink)

Anger, disgust, elevation, sympathy, relief. If the subjective experience of each of these emotions is the same whether elicited by moral or nonmoral events, then what makes moral emotions unique? We suggest that the configuration of moral emotions is special—a configuration given by the underlying structure of morality. Research suggests that people divide the moral world along the two dimensions of valence (help/harm) and moral type (agent/patient). The intersection of these two dimensions gives four moral exemplars—heroes, villains, victims and beneficiaries—each (...) of which elicits unique emotions. For example, victims (harm/patient) elicit sympathy and sadness. Dividing moral emotions into these four quadrants provides predictions about which emotions reinforce, oppose and complement each other. (shrink)

Moral judgment has typically been characterized as a conflict between emotion and reason. In recent years, a central concern has been determining which process is the chief contributor to moral behavior. While classic moral theorists claimed that moral evaluations stem from consciously controlled cognitive processes, recent research indicates that affective processes may be driving moral behavior. Here, we propose a new way of thinking about emotion within the context of moral judgment, one in which affect is generated and transformed by (...) both automatic and controlled processes, and moral evaluations are shifted accordingly. We begin with a review of how existing theories in psychology and neuroscience address the interaction between emotion and cognition, and how these theories may inform the study of moral judgment. We then describe how brain regions involved in both affective processing and moral judgment overlap and may make distinct contributions to the moral evaluation process. Finally, we discuss how this way of thinking about emotion can be reconciled with current theories in moral psychology before mapping out future directions in the study of moral behavior. (shrink)

While robots were traditionally built to achieve economic efficiency and financial profits, their roles are likely to change in the future with the aim to provide social support and companionship. In this research, we examined whether the robot’s proposed function (social vs. economic) impacts judgments of mind and moral treatment. Studies 1a and 1b demonstrated that robots with social function were perceived to possess greater ability for emotional experience, but not cognition, compared to those with economic function and whose function (...) was not mentioned explicitly. Study 2 replicated this finding and further showed that low economic value reduced ascriptions of cognitive capacity, whereas high social value resulted in increased emotion perception. In Study 3, robots with high social value were more likely to be afforded protection from harm, and such effect was related to levels of ascribed emotional experience. Together, the findings demonstrate a dissociation between function type (social vs. economic) and ascribed mind (emotion vs. cognition). In addition, the two types of functions exert asymmetric influences on the moral treatment of robots. Theoretical and practical implications for the field of social psychology and human-computer interaction are discussed. (shrink)

The current research aims to answer the following question: “who will be held responsible for harm involving an artificial intelligence system?” Drawing upon the literature on moral judgments, we assert that when people perceive an AI system’s action as causing harm to others, they will assign blame to different entity groups involved in an AI’s life cycle, including the company, the developer team, and even the AI system itself, especially when such harm is perceived to be intentional. Drawing upon the (...) theory of mind perception, we hypothesized that two dimensions of mind: perceived agency—attributing intention, reasoning, pursuing goals, and communicating to AI, and perceived experience—attributing emotional states, such as feeling pain and pleasure, personality, and consciousness to AI—mediated the relationship between perceived intentional harm and blame judgments toward AI. We also predicted that people are likely to attribute higher mind characteristics to AI when harm is perceived to be directed to humans than when it is perceived to be directed to non-humans. We tested our research model in three experiments. In all experiments, we found that perceived intentional harm led to blame judgments toward AI. In two experiments, we found perceived experience, not agency, mediated the relationship between perceived intentional harm and blame judgments. We also found that companies and developers were held responsible for moral violations involving AI, with developers received the most blame among the entities involved. Our third experiment reconciles the findings by showing that perceived intentional harm directed to a non-human entity did not lead to increased attributions of mind to AI. These findings have implications for theory and practice concerning unethical outcomes and behavior associated with AI use. (shrink)

In rebellion against low‐dimensional (e.g., two‐factor) theories in psychology, the authors make the case for high‐dimensional theories. This change in perspective requires a shift towards a focus on computation and quantitative reasoning.

The issues of climate change and environmental degradation elicit diverse responses. This paper explores how an understanding of human moral psychology might be used to motivate conservation efforts. Moral concerns for the environment can relate to issues of harm or impurity . Aversions to harm are linked to concern for current or future generations, non-human animals, and anthropomorphized aspects of the environment. Concerns for purity are linked to viewing the environment as imbued with sacred value and therefore worthy of being (...) protected at all costs. While both harm-based and purity-based framings of environmental issues can sometimes backfire, we argue that making these moral concerns salient can nevertheless bring about moral responses to environmental issues. In sum, scientists' emerging knowledge about the moral mind can be used to facilitate the sustainable conservation of the planet. (shrink)

Perceiving others? minds is a crucial component of social life. People do not, however, always ascribe minds to other people, and sometimes ascribe minds to non-people. This article reviews when mind perception occurs, when it does not, and why mind perception is important. Causes of mind perception stem both from the perceiver and perceived, and include the need for social connection and a similarity to oneself. Mind perception also has profound consequences for both the perceiver and perceived. Ascribing mind confers (...) an entity moral rights and also makes its actions meaningful. Understanding the causes and consequences of mind perception can explain when this most social of cognitive skills will be used, and why it matters. (shrink)

Moral patients deserve moral consideration and concern – they have moral standing. What factors drive attributions of moral standing? Understanding these factors is important because it indicates how broadly individuals conceptualize the moral world, and suggests how they will treat various entities, both human and non-human. This understanding has recently been advanced by a series of studies conducted by both psychologists and philosophers, which have revealed three main drivers of moral standing: the capacity to suffer, intelligence or autonomy, and the (...) nature of an entity's disposition. These studies have also revealed causal links between moral standing and other variables of interest, namely, mental state attributions and moral behavior. In this review, I consider this recent research, aiming to clarify what the balance of evidence indicates about how moral standing is judged and about its links to mind perception and behavior. I conclude by suggesting open questions for future research on this exciting topic. (shrink)

Ethical leadership research has primarily relied on social learning and social exchange theories. Although these theories have been generative, additional theoretical perspectives hold the potential to broaden scholars’ understanding of ethical leadership’s effects. In this paper, we examine moral typecasting theory and its unique implications for followers’ leader-directed citizenship behavior. Across two studies employing both survey-based and experimental methods, we offer support for three key predictions consistent with this theory. First, the effect of ethical leadership on leader-directed citizenship behavior is (...) curvilinear, with followers helping highly ethical and highly unethical leaders the least. Second, this effect only emerges in morally intense contexts. Third, this effect is mediated by the follower’s belief in the potential for prosocial impact. Our findings suggest that a follower’s belief that his or her leader is ethical has meaningful, often counterintuitive effects that are not predicted by dominant theories of ethical leadership. These results highlight the potential importance of moral typecasting theory to better understand the dynamics of ethical leadership. (shrink)

Both individuals and organizations can engage in unethical behaviors. Across six experiments, we examine how people’s ethical judgments are affected by whether the agent engaging in unethical action is a person or an organization. People believe organizations are more unethical than individuals, even when both agents engage in identical behaviors. Using both mediation and moderation analytical approaches, we find that this effect is explained by people’s beliefs that organizations produce more harm when behaving unethically, even when they do not, as (...) well as people’s perceptions that organizations are relatively more blameworthy agents. We then explore how these judgments manifest across different kinds of organizations as well as how they produce discrepant punishments following ethically questionable business activities. Although society and the law often treat individuals and organizations as equivalent, people believe for-profit organizations’ behaviors are less ethical than identical individual behaviors. We discuss the ethical implications of this discrepancy, as well as additional implications concerning reputation management, punishment, and signaling in organizational contexts. (shrink)

In their recent paper, Kurt Gray and Daniel Wegner offer a model of moral cognition, the “Moral Typecasting” thesis, in which they claim that perceptions of moral agency are inversely related to perceptions of moral patiency. Once we see someone as a moral agent, they claim, we cannot see them as a moral patient (and vice versa). In this paper, I want both to challenge the conception of morality on which the typecasting thesis is fundamentally based and to raise some (...) concerns with the data offered in favor of moral typecasting. I first argue that the dyadic definition of morality is far too narrow to fully capture either all of morality or all of moral psychology. Further, even setting aside the problems with the dyadic notion of morality, I argue that the experimental data Gray and Wegner appeal to fail to demonstrate the sort of mutual exclusivity of and causal interaction between moral perceptions that the moral typecasting thesis proposes. Rather, I suggest, the perceptual differences that do show up in the cited studies arguably arise not from a psychological tendency towards moral typecasting, but from confounding features of the characters in the stimuli. (shrink)

Ethicists have tended to treat the psychology of attributing mental states to animals as an entirely separate issue from the moral importance of animals’ mental states. In this paper I bring these two issues together. I argue for two theses, one descriptive and one normative. The descriptive thesis holds that ordinary human agents use what are generally called phenomenal mental states to assign moral considerability to animals. I examine recent empirical research on the attribution of phenomenal states and agential states (...) to argue that phenomenal mental states are the primary factor, psychologically, for judging an animal to be morally considerable. I further argue that, given the role of phenomenal states in assigning moral considerability, certain theories in animal ethics will meet significant psychological resistance. The normative thesis holds that ethicists must take the psychology of attributing mental states into account when constructing moral ideals concerning animals. I draw from the literature in political philosophy on ideal and non-ideal theory to argue that non-ideal theories for animals must account for human psychology because—like current social and political conditions—human psychology limits the achievement of moral ideals. (shrink)

Mind attribution may be divided into the subcategories of attribution of agency, associated with moral agency, and attribution of experience and emotion, associated with moral concern and moral patiency (Gray et al. Science 315(5812):619, 2007; Gray et al. Proceedings of the National Academy of Sciences of the United States of America 108(2):477–479, 2011b; Robbins and Jack Philosophical Studies 127(1):59–85, 2006). In this paper I attend to social context and the different psychological needs influencing the different types of mind attribution. A (...) need for social connection drives the attribution of experiences and emotions. The individual’s capacity to regulate emotional reactions is crucial for empathic concern. I further relate differences of mind attribution, attention, and emotional processing, to two different modes of functioning: In an interactive state of mind (comparable to the “second-person perspective” described by Schilbach et al. Behavioral and Brain Sciences, in press), social and emotional cues are attended to; In a detached task-oriented state of mind it may be beneficial to predict the behaviors of others but emotional information may not be attended to. The complexity and plasticity of mind attribution, including the possibility to train attentional mechanisms and regulation of emotions, is promising for the field of moral enhancement. (shrink)

Can business humanize its stakeholders? And if so, how does this relate to moral consideration for stakeholders? In this paper we compare two business orientations that are relevant for current business theory and practice: a stakeholder orientation and a profit orientation. We empirically investigate the causal relationships between business orientation, humanization, and moral consideration. We report the results of six experiments, making use of different operationalizations of a stakeholder and profit orientation, different stakeholders, and different participant samples. Our findings support (...) the prediction that individual stakeholders observing a stakeholder-oriented firm see the firm’s other stakeholders as more human than individual stakeholders observing a profit-oriented firm. This humanization, in turn, increases individual stakeholders’ moral consideration for the firm’s other stakeholders. Our findings underscore the importance of humanization for stakeholders’ moral consideration for each other. This paper contributes to a deeper understanding of the firm as a moral community of stakeholders. Specifically, we move away from a focus on managers, and how they can make business more moral. Instead we direct attention to stakeholders, and how business can make these stakeholders more moral. (shrink)

It has long been known that psychopathology can influence social perception, but a 2D framework of mind perception provides the opportunity for an integrative understanding of some disorders. We examined the covariation of mind perception with three subclinical syndromes—autism-spectrum disorder, schizotypy, and psychopathy—and found that each presents a unique mind-perception profile. Autism-spectrum disorder involves reduced perception of agency in adult humans. Schizotypy involves increased perception of both agency and experience in entities generally thought to lack minds. Psychopathy involves reduced perception (...) of experience in adult humans, children, and animals. Disorders are differentially linked with the over- or underperception of agency and experience in a way that helps explain their real-world consequences. (shrink)

Companies often benefit from others’ attributions of moral conviction for prosocial behavior, for example, attributions that a company has a sincere moral desire to improve the environment when behaving sustainably. Across four studies, we explored how organizations’ changing resource positions influenced people’s attributions for the motivations underlying prosocial organizational behaviors. Observers attributed less moral conviction following prosocial behavior when they believed an organization was losing economic resources. This effect was primarily a “penalty” assessed against organizations that were losing resources, as (...) opposed to a “reward” given to organizations gaining resources. Finally, we found that this effect occurred because people perceive organizations that are losing resources as more situationally constrained, leading them to attribute less dispositional moral conviction. We discuss theoretical and practical implications stemming from how changes in resource access can lead people to be more skeptical of organizations’ motivations following prosocial behavior. (shrink)

Are judgments of praise for moral behavior modulated by knowledge of an agent's past suffering at the hands of others, and if so, in what direction? Drawing on multiple lines of research in experimental social psychology, we identify three hypotheses about the psychology of praise — typecasting, handicapping, and non-historicism — each of which supports a different answer to the question above. Typecasting predicts that information about past suffering will augment perceived patiency and thereby diminish perceived agency, making altruistic actions (...) seem less praiseworthy; handicapping predicts that this information will make altruistic actions seem more effortful, and hence more praiseworthy; and non-historicism predicts that judgments of praise will be insensitive to information about an agent's experiential history. We report the results of two studies suggesting that altruistic behavior tends to attract more praise when the experiential history of the agent involves coping with adversity in childhood rather than enjoying prosperity (Study 1, N = 348, p = .03, d = 0.45; Study 2, N = 400, p = .02, d = 0.39), as well as the results of a third study suggesting that altruistic behavior tends to be evaluated more favorably when the experiential history of the agent includes coping with adversity than in the absence of information about the agent's past experience (N = 226, p = .002). This pattern of results, we argue, is more consistent with handicapping than typecasting or non-historicism. (shrink)

Recent research in moral psychology has highlighted how the current internal states of observers can influence their moral judgments of others’ actions. In this article, we argue that an important internal state that serves such a function is the sense of control one has over one’s own actions. Across four studies, we show that an individual’s own current sense of control is positively associated with the intensity of moral judgments of the actions of others. We also show that this effect (...) extends not only to judgments of rightness and wrongness (Study 1), but also to assignments of reward and punishment (Study 2). Finally, we demonstrate that this effect is based on the current experience of control by showing a moderation of the effect via motivational states (promotion; prevention) that either lead one to incorporate or disregard internal states when making judgments (Study 3) and by subtly manipulating participants’ sense of control (Study 4). (shrink)

The notion of agency has been explored within research in moral psychology and, quite separately, within research in linguistics. Moral psychologists have suggested that agency attributions play a role in moral judgments, while linguists have argued that agency attributions play a role in syntactic intuitions. -/- To explore the connection between these two lines of research, we report the results of an experiment in which we manipulate syntactic cues for agency and show a corresponding impact on moral judgments. This result (...) suggests that the two effects observed previously — in morality and in syntax — might each be a reflection of a more general capacity to understand event structure. (shrink)

Why are we still studying well-being? After more than two thousand years of Western philosophy, why do we lack a settled account of the good life for humans? Philosophical problems in general are perennial, and the nature of human well-being is one such problem. However, we seem to stand in an epistemic relationship to this topic that is not shared by other ones. We have a vested interest in understanding the good life, and the relevant data seem to be accessible (...) to us all. The challenge is to explain why well-being is one lasting philosophical topic among others in spite of our special epistemic relationship to it. I argue that human nature renders us well-being blind. On one side this is due to the heterogeneous nature of our interests. Some are directly mediated by conscious thought, others are not. Some are individualistically realized, others relationally. On the other side we suffer from cognitive biases that lead us to under-value, indeed to miss entirely, the important aspects of human life that do not depend on conscious attention. Consequently, there is reason to think that we will never be satisfied with a theory of well-being. (shrink)

An actor's mental states—whether she acted knowingly and with bad intentions—typically play an important role in evaluating the extent to which an action is wrong and in determining appropriate levels of punishment. In four experiments, we find that this role for knowledge and intent is significantly weaker when evaluating transgressions of conventional rules as opposed to moral rules. We also find that this attenuated role for knowledge and intent is partly due to the fact that conventional rules are judged to (...) be more arbitrary than moral rules; whereas moral transgressions are associated with actions that are intrinsically wrong, conventional transgressions are associated with actions that are only contingently wrong. Finally, we find that it is the perpetrator's belief about the arbitrary or non-arbitrary basis of the rule—not the reality—that drives this differential effect of knowledge and intent across types of transgressions. (shrink)

A central claim in stakeholder theory is that, if we see stakeholders as human beings, we will attribute higher moral standing or show more moral consideration to stakeholders. But would the same hold for firms? In this paper, I apply the concepts of humanization and moral standing to firms, and I predict that individuals attribute higher moral standing to stakeholder-oriented than to profit-oriented firms, because individuals attribute more experience to stakeholder-oriented than to profit-oriented firms. Five experiments support these predictions across (...) different operationalizations of stakeholder and profit orientations. The analyses show that moral standing attributions are not fully explained by attributions of agency to firms, or by attributions of experience or agency to human stakeholders. By unearthing the importance of experience attributions for moral standing attributions to firms, this work provides novel insights in ongoing legal, philosophical and public debates related to firms’ moral standing. The findings also bring the debate about firms’ moral standing to the heart of stakeholder theory, and lead to new normative and descriptive research questions about the interests of firms and their stakeholders. (shrink)

Mind perception is the essence of moral judgment. Broadly, moral standing is linked to perceptions of mind, with moral responsibility tied to perceived agency, and moral rights tied to perceived experience. More specifically, moral judgments are based on a fundamental template of two perceived minds—an intentional agent and a suffering patient. This dyadic template grows out of the universal power of harm, and serves as a cognitive working model through which even atypical moral events are understood. Thus, all instances of (...) immorality are perceived to involve both blameworthy agents (i.e., acts) and suffering victims (i.e., consequences). Because moral cognition simultaneously concerns acts and consequences, theories which focus primarily on acts (i.e., deontology) or consequences (i.e., utilitarianism) do not accurately describe moral cognition. Indeed, the phenomenon of dyadic completion suggests that deontological and utilitarian concerns are not only simultaneously active, but also typically compatible and reinforcing: wrong acts have harmful consequences, and harmful consequences stem from wrong acts. The cognitive fusion of acts with consequences suggests that normative conflicts between deontology and utilitarianism are not reflected in everyday moral judgment. This in turn suggests that empirical conclusions drawn from moral dilemmas that pit utilitarianism against deontology—i.e., trolley problems—give an skewed account of moral cognition. (shrink)

How do people behave in interpersonal interactions? And why do they behave the way they do? In this thesis, I present a novel social-cognitive account of role-taking explaining how fixed positions affect our emotions, thoughts, and behavior in interactions. The basic notion of Role-Taking Theory is that actors represent roles in form of mental schemata, and that role-taking activates the role-specific mental schema. In its application, Role-Taking Theory was used to explain third-party reactions to conflicts. As a result, this thesis (...) entails three manuscripts currently under review for publications: one theoretical, and two empirical. In the theoretical article, we explain the core premises of RTT, discuss RTT’s contributions over and above existing theory, apply RTT to interpersonal conflicts, and outline future directions. In the two empirical articles, we report first empirical tests of hypotheses deriving from RTT’s assumptions. In sum, the manuscripts provide answers to how the number of constituting roles can be identified in any interpersonal interaction, how features of identified roles can be derived, how features of roles affect actors due to role-taking, why some actors take over certain roles in conflicts more frequently than others, why some actors benefit from taking over certain roles in conflicts more than others, why actors sometimes stick to roles even when this comes with costs, and – ultimately – how all these mechanisms tied to role-taking explain when and why interactions stabilize or destabilize. In a final chapter at the end of this thesis, the incremental contributions of these manuscripts over and above existing literature are discussed both in its novel theoretical rationale as well as in empirical insights. In sum, I argue that RTT’s universality and precision are apt for sustained empirical and theoretical work in the future. However, RTT is still in an early developmental stage, and thus, I prepare the next steps by analyzing the present state, outlining the most critical challenges in theory development, and identify the most promising directions for future research. (shrink)

In this review, I offer an overview of of the questions that caught my attention while reading Neuroexistentialism. I aim to make it clear why the issues that are raised in this volume are worth exploring in more detail. I also hope to clarify the limitations that are imposed by neural and social constraints, and to recommend ways of anchoring a third wave of existentialism in our understanding of neuroscience, our expanding sense of cultural variation, and our emerging recognition of (...) the contingency of even our most deeply sedimented ways of perceiving and acting in the world. (shrink)