Building artificial moral agents (AMAs) underscores the fragmentary character of presently available models of human ethical behavior. It is a distinctly different enterprise from either the attempt by moral philosophers to illuminate the “ought” of ethics or the research by cognitive scientists directed at revealing the mechanisms that influence moral psychology, and yet it draws on both. Philosophers and cognitive scientists have tended to stress the importance of particular cognitive mechanisms, e.g., reasoning, moral sentiments, heuristics, intuitions, or a moral grammar, (...) in the making of moral decisions. However, assembling a system from the bottom-up which is capable of accommodating moral considerations draws attention to the importance of a much wider array of mechanisms in honing moral intelligence. Moral machines need not emulate human cognitive faculties in order to function satisfactorily in responding to morally significant situations. But working through methods for building AMAs will have a profound effect in deepening an appreciation for the many mechanisms that contribute to a moral acumen, and the manner in which these mechanisms work together. Building AMAs highlights the need for a comprehensive model of how humans arrive at satisfactory moral judgments. (shrink)

Building artificial moral agents (AMAs) underscores the fragmentary character of presently available models of human ethical behavior. It is a distinctly different enterprise from either the attempt by moral philosophers to illuminate the “ought” of ethics or the research by cognitive scientists directed at revealing the mechanisms that influence moral psychology, and yet it draws on both. Philosophers and cognitive scientists have tended to stress the importance of particular cognitive mechanisms, e.g., reasoning, moral sentiments, heuristics, intuitions, or a moral grammar, (...) in the making of moral decisions. However, assembling a system from the bottom-up which is capable of accommodating moral considerations draws attention to the importance of a much wider array of mechanisms in honing moral intelligence. Moral machines need not emulate human cognitive faculties in order to function satisfactorily in responding to morally significant situations. But working through methods for building AMAs will have a profound effect in deepening an appreciation for the many mechanisms that contribute to a moral acumen, and the manner in which these mechanisms work together. Building AMAs highlights the need for a comprehensive model of how humans arrive at satisfactory moral judgments. (shrink)

Many industry leaders and academics from the field of machine ethics would have us believe that the inevitability of robots coming to have a larger role in our lives demands that robots be endowed with moral reasoning capabilities. Robots endowed in this way may be referred to as artificial moral agents. Reasons often given for developing AMAs are: the prevention of harm, the necessity for public trust, the prevention of immoral use, such machines are better moral reasoners than humans, and (...) building these machines would lead to a better understanding of human morality. Although some scholars have challenged the very initiative to develop AMAs, what is currently missing from the debate is a closer examination of the reasons offered by machine ethicists to justify the development of AMAs. This closer examination is especially needed because of the amount of funding currently being allocated to the development of AMAs coupled with the amount of attention researchers and industry leaders receive in the media for their efforts in this direction. The stakes in this debate are high because moral robots would make demands on society; answers to a host of pending questions about what counts as an AMA and whether they are morally responsible for their behavior or not. This paper shifts the burden of proof back to the machine ethicists demanding that they give good reasons to build AMAs. The paper argues that until this is done, the development of commercially available AMAs should not proceed further. (shrink)

Many industry leaders and academics from the field of machine ethics would have us believe that the inevitability of robots coming to have a larger role in our lives demands that robots be endowed with moral reasoning capabilities. Robots endowed in this way may be referred to as artificial moral agents. Reasons often given for developing AMAs are: the prevention of harm, the necessity for public trust, the prevention of immoral use, such machines are better moral reasoners than humans, and (...) building these machines would lead to a better understanding of human morality. Although some scholars have challenged the very initiative to develop AMAs, what is currently missing from the debate is a closer examination of the reasons offered by machine ethicists to justify the development of AMAs. This closer examination is especially needed because of the amount of funding currently being allocated to the development of AMAs coupled with the amount of attention researchers and industry leaders receive in the media for their efforts in this direction. The stakes in this debate are high because moral robots would make demands on society; answers to a host of pending questions about what counts as an AMA and whether they are morally responsible for their behavior or not. This paper shifts the burden of proof back to the machine ethicists demanding that they give good reasons to build AMAs. The paper argues that until this is done, the development of commercially available AMAs should not proceed further. (shrink)

In what ways should we include future humanoid robots, and other kinds of artificial agents, in our moral universe? We consider the Organic view, which maintains that artificial humanoid agents, based on current computational technologies, could not count as full-blooded moral agents, nor as appropriate targets of intrinsic moral concern. On this view, artificial humanoids lack certain key properties of biological organisms, which preclude them from having full moral status. Computationally controlled systems, however advanced in their cognitive or informational capacities, (...) are, it is proposed, unlikely to possess sentience and hence will fail to be able to exercise the kind of empathic rationality that is a prerequisite for being a moral agent. The organic view also argues that sentience and teleology require biologically based forms of self-organization and autonomous self-maintenance. The organic view may not be correct, but at least it needs to be taken seriously in the future development of the field of Machine Ethics. (shrink)

According to cognitive psychologists, moral decision-making is a dual-process phenomenon involving two types of cognitive processes: explicit reasoning and implicit intuition. Moral development involves training and integrating both types of cognitive processes through a mix of instruction, practice, and reflection. Serious games are an ideal platform for this kind of moral training, as they provide safe spaces for exploring difficult moral problems and practicing the skills necessary to resolve them. In this article, we present Morality Play, a model for the (...) design of serious games for ethical expertise development based on the Integrative Ethical Education framework from moral psychology and the Lens of the Toy model for serious game design. (shrink)

While philosophers are often concerned with the conditions for moral knowledge or justification, in practice something arguably less demanding is just as, if not more, important – reliably making correct moral judgments. Judges and juries should hand down fair sentences, government officials should decide on just laws, members of ethics committees should make sound recommendations, and so on. We want such agents, more often than not and as often as possible, to make the right decisions. The purpose of this paper (...) is to propose a method of enhancing the moral reliability of such agents. In particular, we advocate for a procedural approach; certain internal processes generally contribute to people’s moral reliability. Building on the early work of Rawls, we identify several particular factors related to moral reasoning that are specific enough to be the target of practical intervention: logical competence, conceptual understanding, empirical competence, openness, empathy and bias. Improving on these processes can in turn make people more morally reliable in a variety of contexts and has implications for recent debates over moral enhancement. (shrink)

Moral enhancement is an ostensibly laudable project. Who wouldn’t want people to become more moral? Still, the project’s approach is crucial. We can distinguish between two approaches for moral enhancement: direct and indirect. Direct moral enhancements aim at bringing about particular ideas, motives or behaviors. Indirect moral enhancements, by contrast, aim at making people more reliably produce the morally correct ideas, motives or behaviors without committing to the content of those ideas, motives and/or actions. I will argue, on Millian grounds, (...) that the value of disagreement puts serious pressure on proposals for relatively widespread direct moral enhancement. A more acceptable path would be to focus instead on indirect moral enhancements while staying neutral, for the most part, on a wide range of substantive moral claims. I will outline what such indirect moral enhancement might look like, and why we should expect it to lead to general moral improvement. (shrink)

In recent years, ‘nudge’ theory has gained increasing attention for the design of population-wide health interventions. The concept of nudge puts a label on efficacious influences that preserve freedom of choice without engaging the influencees’ deliberative capacities. Given disagreements over what it takes genuinely to preserve freedom of choice, the question is whether health influences relying on automatic cognitive processes may preserve freedom of choice in a sufficiently robust sense to be serviceable for the moral evaluation of actions and policies. (...) In this article, I offer an argument to this effect, explicating preservation of freedom of choice in terms of choice-set preservation and noncontrol. I also briefly explore the healthcare contexts in which nudges may have priority over more controlling influences. (shrink)

ABSTRACTAutonomous weapons systems pose many challenges in complex battlefield environments. Previous discussions of them have largely focused on technological or policy issues. In contrast, we focus here on the challenge of trust in an AWS. One type of human trust depends only on judgments about the predictability or reliability of the trustee, and so are suitable for all manner of artifacts. However, AWSs that are worthy of the descriptor “autonomous” will not exhibit the required strong predictability in the complex, changing (...) contexts of war. Instead, warfighters need to develop deeper, interpersonal trust that is grounded in understanding the values, beliefs, and dispositions of the AWS. Current acquisition, training, and deployment processes preclude the development of such trust, and so there are currently no routes for a warfighter to develop trust in an AWS. We thus survey three possible changes to current practices in order to facilitate the type of deep trust that is required for appropri... (shrink)

Recently there has been some discussion concerning a particular type of enhancement, namely ‘moral enhancement’. However, there is no consensus on what precisely constitutes moral enhancement, and as a result the concept is used and defined in a wide variety of ways. In this article, we develop a clarificatory taxonomy of these definitions and we identify the criteria that are used to delineate the concept. We think that the current definitions can be distinguished from each other by the criteria used (...) for determining whether an intervention is indeed moral enhancement. For example, some definitions are broad and include moral enhancement by any means, while other definitions focus only on moral enhancement by means of specific types of intervention. Moreover, for some definitions it suffices for an intervention to be aimed or intended to morally enhance a person, while other definitions only refer to ‘moral enhancement’ in relation to interventions that are actually effective. For all these differences in definitions we discuss some of their implications. This shows that definitions are significantly less descriptive and more normative than they are regularly portrayed to be. We therefore hope that the taxonomy developed in this paper and the comments on the implications for the normative debate of the variety of definitions will provide conceptual clarity in a complex and highly interesting debate. (shrink)

In this paper, we argue that, under a specific set of circumstances, designing and employing certain kinds of virtual reality (VR) experiences can be unethical. After a general discussion of simulations and their ethical context, we begin our argu-ment by distinguishing between the experiences generated by different media (text, film, computer game simulation, and VR simulation), and argue that VR experiences offer an unprecedented degree of what we call “perspectival fidelity” that prior modes of simulation lack. Additionally, we argue that (...) when VR experiences couple this perspectival fidelity with what we call “context realism,” VR experiences have the ability to produce “virtually real experiences.” We claim that virtually real experiences generate ethical issues for VR technologies that are unique to the medium. Because subjects of these experiences treat them as if they were real, a higher degree of ethical scrutiny should be applied to any VR scenario with the potential to generate virtually real experiences. To mitigate this unique moral hazard, we propose and defend what we call “The Equivalence Principle.” This principle states that “if it would be wrong to allow subjects to have a certain experience in reality, then it would be wrong to allow subjects to have that experience in a virtually real setting.” We argue that such a principle, although limited in scope, should be part of the risk analysis conducted by any Institutional Review Boards, psychologists, empirically oriented philosophers, or game designers who are using VR technology in their work. (shrink)

Empathy can be characterized as a vicarious emotion that one person experiences when reflecting on the emotion of another. So characterized, empathy is sometimes regarded as a precondition on moral judgment. This seems to have been Hume's view. I review various ways in which empathy might be regarded as a precondition and argue against each of them: empathy is not a component, a necessary cause, a reliable epistemic guide, a foundation for justification, or the motivating force behind our moral judgments. (...) In fact, empathy is prone to biases that render it potentially harmful. Another construct—concern—fares somewhat better, but it is also of limited use. I argue that, instead of empathy, moral judgments involve emotions such as anger, disgust, guilt, and admiration. These, not empathy, provide the sentimental foundation for morality. (shrink)

abstract As history shows, some human beings are capable of acting very immorally. 1 Technological advance and consequent exponential growth in cognitive power means that even rare evil individuals can act with catastrophic effect. The advance of science makes biological, nuclear and other weapons of mass destruction easier and easier to fabricate and, thus, increases the probability that they will come into the hands of small terrorist groups and deranged individuals. Cognitive enhancement by means of drugs, implants and biological (including (...) genetic) interventions could thus accelerate the advance of science, or its application, and so increase the risk of the development or misuse of weapons of mass destruction. We argue that this is a reason which speaks against the desirability of cognitive enhancement, and the consequent speedier growth of knowledge, if it is not accompanied by an extensive moral enhancement of humankind. We review the possibilities for moral enhancement by biomedical and genetic means and conclude that, though it should be possible in principle, it is in practice probably distant. There is thus a reason not to support cognitive enhancement in the foreseeable future. However, we grant that there are also reasons in its favour, but we do not attempt to settle the balance between these reasons for and against. Rather, we conclude that if research into cognitive enhancement continues, as it is likely to, it must be accompanied by research into moral enhancement. (shrink)

We respond to a number of objections raised by John Harris in this journal to our argument that we should pursue genetic and other biological means of morally enhancing human beings (moral bioenhancement). We claim that human beings now have at their disposal means of wiping out life on Earth and that traditional methods of moral education are probably insufficient to achieve the moral enhancement required to ensure that this will not happen. Hence, we argue, moral bioenhancement should be sought (...) and applied. We argue that cognitive enhancement and technological progress raise acute problems because it is easier to harm than to benefit. We address objections to this argument. We also respond to objections that moral bioenhancement: (1) interferes with freedom; (2) cannot be made to target immoral dispositions precisely; (3) is redundant, since cognitive enhancement by itself suffices. (shrink)

The idea of using biomedical means to make people more likely to behave morally may have a certain appeal. However, it is very hard to find two persons – let alone two moral philosophers – who agree on what it means to be moral or to act morally. After discussing some of the proposals for moral enhancements that all ethicists could agree on, I engage more closely with the recent idea of “procedural moral enhancement” that aims at improving deliberative processes (...) instead of particular moral views, motivations, or dispositions. I argue that it is better understood as a contribution to moral epistemology and should thus be labeled “moral-epistemic enhancement”. I then defend perspective-taking as a moral epistemic capacity which can be enhanced by both traditional and non-traditional biomedical means; a capacity which almost always contributes to the epistemic value of moral decision-making. Perspective-taking seems to be an uncontroversial non-trivial capacity for moral decision-making reasonably widely shared by proponents of ethical beliefs within the academic community. The enhancement of this capacity is thus a good candidate for an uncontroversial non-trivial moral enhancement. (shrink)

This paper critically assesses the possibility of moral enhancement with ambient intelligence technologies and artificial intelligence presented in Savulescu and Maslen (2015). The main problem with their proposal is that it is not robust enough to play a normative role in users’ behavior. A more promising approach, and the one presented in the paper, relies on an artifi-cial moral reasoning engine, which is designed to present its users with moral arguments grounded in first-order normative theories, such as Kantianism or utilitarianism, (...) that reason-responsive people can be persuaded by. This proposal can play a normative role and it is also a more promising avenue towards moral enhancement. It is more promising because such a system can be designed to take advantage of the sometimes undue trust that people put in automated technologies. We could therefore expect a well-designed moral reasoner system to be able to persuade people that may not be persuaded by similar arguments from other people. So, all things considered, there is hope in artificial intelli-gence for moral enhancement, but not in artificial intelligence that relies solely on ambient intelligence technologies. (shrink)

In this essay, I muse upon aporia’s value as a pedagogical technique in the philosophy classroom using as a guide examples of aporia that are found in Plato’s Socratic dialogues. The word aporia, translated as “without passage” or “without a way,” is used metaphorically to describe the unsettling state of confusion many find themselves in after engaging in philosophical discourse. Following a brief introduction in which I situate aporia as a pedagogy amicable to experiential learning, I examine various ways in (...) which aporia appears in certain Platonic dialogues, which enables us to draw out some paradigmatic features of aporia. I then discuss how I apply aporia as a pedagogical technique in the contemporary philosophy classroom, taking up three specific concerns in detail: aporetic discomfort, right use, and potential misuse. (shrink)

In this paper, I will argue, contra Prinz, that empathy is a crucial component of our moral lives. In particular, I argue that empathy is sometimes epistemologically necessary for identifying the right action; that empathy is sometimes psychologically necessary for motivating the agent to perform the right action; and that empathy is sometimes necessary for the agent to be most morally praiseworthy for an action. I begin by explaining what I take empathy to be. I then discuss some alleged problems (...) for empathy and explain why some argue that empathy is unnecessary and sometimes even problematic in the moral domain. Next, I criticize a prominent alternative to an empathy-based morality. Finally, I argue that that empathy is sometimes epistemologically and psychologically necessary for doing the right thing and is sometimes necessary for moral worth. I conclude with a discussion of the important role of empathy in our everyday lives. (shrink)

Governments must determine the legal procedures by which their residents are registered, or can register, as organ donors. Provided that governments recognize that people have a right to determine what happens to their organs after they die, there are four feasible options to choose from: opt-in, opt-out, mandated active choice, and voluntary active choice. We investigate the ethics of these policies' use of nudges to affect organ donor registration rates. We argue that the use of nudges in this context is (...) morally problematic. It is disrespectful of people's autonomy to take advantage of their cognitive biases since doing so involves bypassing, not engaging, their rational capacities. We conclude that while mandated active choice policies are not problem free—they are coercive, after all—voluntary active choice, opt-in, and opt-out policies are potentially less respectful of people's autonomy since their use of nudges could significantly affect people's decision making. (shrink)

The moral enhancement of human beings is a constant theme in the history of humanity. Today, faced with the threats of a new, globalised world, concern over this matter is more pressing. For this reason, the use of biotechnology to make human beings more moral has been considered. However, this approach is dangerous and very controversial. The purpose of this article is to argue that the use of another new technology, AI, would be preferable to achieve this goal. Whilst several (...) proposals have been made on how to use AI for moral enhancement, we present an alternative that we argue to be superior to other proposals that have been developed. (shrink)

This paper offers a theoretical framework that can be used to derive viable engineering strategies for the design and development of robots that can nudge people towards moral improvement. The framework relies on research in developmental psychology and insights from Stoic ethics. Stoicism recommends contemplative practices that over time help one develop dispositions to behave in ways that improve the functioning of mechanisms that are constitutive of moral cognition. Robots can nudge individuals towards these practices and can therefore help develop (...) the dispositions to, for example, extend concern to others, avoid parochialism, etc. (shrink)

In this essay, I describe and explain the standard accounts of agency, natural agency, artificial agency, and moral agency, as well as articulate what are widely taken to be the criteria for moral agency, supporting the contention that this is the standard account with citations from such widely used and respected professional resources as the Stanford Encyclopedia of Philosophy, Routledge Encyclopedia of Philosophy, and the Internet Encyclopedia of Philosophy. I then flesh out the implications of some of these well-settled theories (...) with respect to the prerequisites that an ICT must satisfy in order to count as a moral agent accountable for its behavior. I argue that each of the various elements of the necessary conditions for moral agency presupposes consciousness, i.e., the capacity for inner subjective experience like that of pain or, as Nagel puts it, the possession of an internal something-of-which-it is-is-to-be-like. I ultimately conclude that the issue of whether artificial moral agency is possible depends on the issue of whether it is possible for ICTs to be conscious. (shrink)

In his ‘Moral Enhancement, Freedom, and What We Value in Moral Behaviour’,1 David DeGrazia sets out to defend moral bioenhancement from a number of critics, me prominently among them. Here he sets out his stall: "Many scholars doubt what I assert: that there is nothing inherently wrong with MB. Some doubt this on the basis of a conviction that there is something inherently wrong with biomedical enhancement technologies in general. Chief among their objections are the charges that biomedical enhancement is (...) unnatural, use of biomedical enhancements evinces an insufficient appreciation for human “giftedness”, and biomedical enhancements pose a threat to personal identity. Elsewhere I have attempted to neutralize these objections. Here I will address a set of concerns that are directed at MB in particular and appeal to the nature and value of human freedom."Let me make clear at once that I do not believe there is anything inherently wrong with MB. I have been an advocate for human enhancement for over 30 years writing four books defending such enhancements.2–⇓4 The most recent of these published in 2007 covers much the same ground as Allen Buchanan's 2011 book cited by DeGrazia,5 but, unlike Buchanan, I do not define enhancements in terms of the intention or the motivation of those who produce them but rather in terms of their effect. I must also make clear that, like DeGrazia, I have also, for a very long time, attempted to neutralise objections 1–3 listed in the above passage.2–⇓4DeGrazia introduces his critique of my approach like this: "I will construe Harris’ argument and similar arguments as directed entirely at motivation-based MB—though I will hereafter omit the qualifier, “motivation-based.” (Certainly, these arguments do not apply to embryo selection, which … ". (shrink)

This paper identifies human enhancement as one of the most significant areas of bioethical interest in the last twenty years. It discusses in more detail one area, namely moral enhancement, which is generating significant contemporary interest. The author argues that so far from being susceptible to new forms of high tech manipulation, either genetic, chemical, surgical or neurological, the only reliable methods of moral enhancement, either now or for the foreseeable future, are either those that have been in human and (...) animal use for millennia, namely socialization, education and parental supervision or those high tech methods that are general in their application. By that is meant those forms of cognitive enhancement that operate across a wide range of cognitive abilities and do not target specifically ‘ethical’ capacities. The paper analyses the work of some of the leading contemporary advocates of moral enhancement and finds that in so far as they identify moral qualities or moral emotions for enhancement they have little prospect of success. (shrink)

This essay responds to the machine question in the affirmative, arguing that artifacts, like robots, AI, and other autonomous systems, can no longer be legitimately excluded from moral consideration. The demonstration of this thesis proceeds in four parts or movements. The first and second parts approach the subject by investigating the two constitutive components of the ethical relationship—moral agency and patiency. In the process, they each demonstrate failure. This occurs not because the machine is somehow unable to achieve what is (...) considered necessary and sufficient to be a moral agent or patient but because the characterization of agency and patiency already fail to accommodate others. The third and fourth parts respond to this problem by considering two recent alternatives—the all-encompassing ontocentric approach of Luciano Floridi’s information ethics and Emmanuel Levinas’s eccentric ethics of otherness. Both alternatives, despite considerable promise to reconfigure the scope of moral thinking by addressing previously excluded others, like the machine, also fail but for other reasons. Consequently, the essay concludes not by accommodating the alterity of the machine to the requirements of moral philosophy but by questioning the systemic limitations of moral reasoning, requiring not just an extension of rights to machines, but a thorough examination of the way moral standing has been configured in the first place. (shrink)

We describe a form of moral artificial intelligence that could be used to improve human moral decision-making. We call it the “artificial moral advisor”. The AMA would implement a quasi-relativistic version of the “ideal observer” famously described by Roderick Firth. We describe similarities and differences between the AMA and Firth’s ideal observer. Like Firth’s ideal observer, the AMA is disinterested, dispassionate, and consistent in its judgments. Unlike Firth’s observer, the AMA is non-absolutist, because it would take into account the human (...) agent’s own principles and values. We argue that the AMA would respect and indeed enhance individuals’ moral autonomy, help individuals achieve wide and a narrow reflective equilibrium, make up for the limitations of human moral psychology in a way that takes conservatives’ objections to human bioenhancement seriously, and implement the positive functions of intuitions and emotions in human morality without their downsides, such as biases and prejudices. (shrink)

It is my view that one essential difference between persons and other creatures is to be found in the structure of a person's will. Besides wanting and choosing and being moved to do this or that, men may also want to have certain desires and motives. They are capable of wanting to be different, in their preferences and purposes, from what they are. Many animals appear to have the capacity for what I shall call "first-order desires" or "desires of the (...) first order," which are simply desires to do or not to do one thing or another. No animal other than man, however, appears to have the capacity for reflective self-evaluation that is manifested in the formation of second-order desires. (shrink)

One of the reasons why moral enhancement may be controversial, is because the advantages of moral enhancement may fall upon society rather than on those who are enhanced. If directed at individuals with certain counter-moral traits it may have direct societal benefits by lowering immoral behavior and increasing public safety, but it is not directly clear if this also benefits the individual in question. In this paper, we will discuss what we consider to be moral enhancement, how different means may (...) be used to achieve it and whether the means we employ to reach moral enhancement matter morally. Are certain means to achieve moral enhancement wrong in themselves? Are certain means to achieve moral enhancement better than others, and if so, why? More specifically, we will investigate whether the difference between direct and indirect moral enhancement matters morally. Is it the case that indirect means are morally preferable to direct means of moral enhancement and can we indeed pinpoint relevant intrinsic, moral differences between both? We argue that the distinction between direct and indirect means is indeed morally relevant, but only insofar as it tracks an underlying distinction between active and passive interventions. Although passive interventions can be ethical provided specific safeguards are put in place, these interventions exhibit a greater potential to compromise autonomy and disrupt identity. (shrink)

Artificial agents (AAs), particularly but not only those in Cyberspace, extend the class of entities that can be involved in moral situations. For they can be conceived of as moral patients (as entities that can be acted upon for good or evil) and also as moral agents (as entities that can perform actions, again for good or evil). In this paper, we clarify the concept of agent and go on to separate the concerns of morality and responsibility of agents (most (...) interestingly for us, of AAs). We conclude that there is substantial and important scope, particularly in Computer Ethics, for the concept of moral agent not necessarily exhibiting free will, mental states or responsibility. This complements the more traditional approach, common at least since Montaigne and Descartes, which considers whether or not (artificial) agents have mental states, feelings, emotions and so on. By focussing directly on mind-less morality we are able to avoid that question and also many of the concerns of Artificial Intelligence. A vital component in our approach is the Method of Abstraction for analysing the level of abstraction (LoA) at which an agent is considered to act. The LoA is determined by the way in which one chooses to describe, analyse and discuss a system and its context. The Method of Abstraction is explained in terms of an interface or set of features or observables at a given LoA. Agenthood, and in particular moral agenthood, depends on a LoA. Our guidelines for agenthood are: interactivity (response to stimulus by change of state), autonomy (ability to change state without stimulus) and adaptability (ability to change the transition rules by which state is changed) at a given LoA. Morality may be thought of as a threshold defined on the observables in the interface determining the LoA under consideration. An agent is morally good if its actions all respect that threshold; and it is morally evil if some action violates it. That view is particularly informative when the agent constitutes a software or digital system, and the observables are numerical. Finally we review the consequences for Computer Ethics of our approach. In conclusion, this approach facilitates the discussion of the morality of agents not only in Cyberspace but also in the biosphere, where animals can be considered moral agents without their having to display free will, emotions or mental states, and in social contexts, where systems like organizations can play the role of moral agents. The primary cost of this facility is the extension of the class of agents and moral agents to embrace AAs. (shrink)

By using preimplantation haplotype diagnosis, prospective parents are able to select embryos to implant through in vitro fertilization. If we knew that the naturally-occurring (but theoretical) MoralKinder (MK+) haplotype would predispose individuals to a higher level of morality than average, is it permissible or obligatory to select for the MK+ haplotype? I.e., is it moral to select for morality? This paper explores the various potential issues that could arise from genetic moral enhancement.

Pharmaceuticals or other emerging technologies could be used to enhance (or diminish) feelings of lust, attraction, and attachment in adult romantic partnerships. While such interventions could conceivably be used to promote individual (and couple) well-being, their widespread development and/or adoption might lead to “medicalization” of human love and heartache—for some, a source of serious concern. In this essay, we argue that the “medicalization of love” need not necessarily be problematic, on balance, but could plausibly be expected to have either good (...) or bad consequences depending upon how it unfolds. By anticipating some of the specific ways in which these technologies could yield unwanted outcomes, bioethicists and others can help direct the course of love’s “medicalization”—should it happen to occur—more toward the “good” side than the “bad.”. (shrink)

In both theoretical and applied contexts the concept of autonomy has assumed increasing importance in recent normative philosophical discussion. Given various problems to be clarified or resolved the author characterizes the concept by first setting out conditions of adequacy. The author then links the notion of autonomy to the identification and critical reflection of an agent upon his first-order motivations. It is only when a person identifies with the influences that motivate him, assimilates them to himself, that he is autonomous. (...) In addition this process of identification must itself meet certain procedural constraints. (shrink)

In both theoretical and applied contexts the concept of autonomy has assumed increasing importance in recent normative philosophical discussion. Given various problems to be clarified or resolved the author characterizes the concept by first setting out conditions of adequacy. The author then links the notion of autonomy to the identification and critical reflection of an agent upon his first-order motivations. It is only when a person identifies with the influences that motivate him, assimilates them to himself, that he is autonomous. (...) In addition this process of identification must itself meet certain procedural constraints. (shrink)

I take as my starting point the “one very simple principle” proclaimed by Mill in On Liberty … “That principle is, that the sole end for which mankind are warranted, individually or collectively, in interfering with the liberty of action of any of their number, is self-protection. That the only purpose for which power can be rightfully exercised over any member of a civilized community, against his will, is to prevent harm to others. He cannot rightfully be compelled to do (...) or forbear because it will be better for him to do so, because it will make him happier, because, in the opinion of others, to do so would be wise, or even right.”. (shrink)

Opponents of biomedical enhancement often claim that, even if such enhancement would benefit the enhanced, it would harm others. But this objection looks unpersuasive when the enhancement in question is a moral enhancement — an enhancement that will expectably leave the enhanced person with morally better motives than she had previously. In this article I (1) describe one type of psychological alteration that would plausibly qualify as a moral enhancement, (2) argue that we will, in the medium-term future, probably be (...) able to induce such alterations via biomedical intervention, and (3) defend future engagement in such moral enhancements against possible objections. My aim is to present this kind of moral enhancement as a counter-example to the view that biomedical enhancement is always morally impermissible. (shrink)

Some argue that humans should enhance their moral capacities by adopting institutions that facilitate morally good motives and behaviour. I have defended a parallel claim: that we could permissibly use biomedical technologies to enhance our moral capacities, for example by attenuating certain counter-moral emotions. John Harris has recently responded to my argument by raising three concerns about the direct modulation of emotions as a means to moral enhancement. He argues that such means will be relatively ineffective in bringing about moral (...) improvements, that direct modulation of emotions would invariably come at an unacceptable cost to our freedom, and that we might end up modulating emotions in ways that actually lead to moral decline. In this article I outline some counter-intuitive potential implications of Harris' claims. I then respond individually to his three concerns, arguing that they license only the very weak conclusion that moral enhancement via direct emotion modulation is sometimes impermissible. However I acknowledge that his third concern might, with further argument, be developed into a more troubling objection to such enhancements. (shrink)

The enhancement of human traits has received academic attention for decades, but only recently has moral enhancement using biomedical means – moral bioenhancement (MB) – entered the discussion. After explaining why we ought to take the possibility of MB seriously, the paper considers the shape and content of moral improvement, addressing at some length a challenge presented by reasonable moral pluralism. The discussion then proceeds to this question: Assuming MB were safe, effective, and universally available, would it be morally desirable? (...) In particular, would it pose an unacceptable threat to human freedom? After defending a negative answer to the latter question – which requires an investigation into the nature and value of human freedom – and arguing that there is nothing inherently wrong with MB, the paper closes with reflections on what we should value in moral behaviour. (shrink)

In what follows, I wish to discuss empathy and sympathy’s relevance to ethics, taking recent findings into account. In particular, I want to consider sympathy’s relation to the idea of a person’s good or well-being. It is obvious and uncontroversial that sympathetic concern for a person involves some concern for her good and some desire to promote it. What I want to suggest is that the concept of a person’s good or well-being is one we have because we are capable (...) of care and sympathetic concern. Well-being is normative for care in the sense that it is intrinsic to the very idea of a person’s good that threats to it are what it makes sense to be concerned about for that person for her sake. (shrink)

Can advances in neuroscience be harnessed to enhance human moral capacities? And if so, should they? De Grazia explores these questions in ‘Moral Enhancement, Freedom, and What We Value in Moral Behaviour’.1 Here, I offer a neuroscientist's perspective on the state of the art of moral bioenhancement, and highlight some of the practical challenges facing the development of moral bioenhancement technologies.The science of moral bioenhancement is in its infancy. Laboratory studies of human morality usually employ highly simplified models aimed at (...) measuring just one facet of a cognitive process that is relevant for morality. These studies have certainly deepened our understanding of the nature of moral behaviour, but it is important to avoid overstating the conclusions of any single study. De Grazia cites several purported examples of ‘non-traditional means of moral enhancement’, including one of my own studies. According to De Grazia, we showed that ‘selective serotonin reuptake inhibitors as a means to being less inclined to assault people’. In fact, our findings are a bit more subtle and nuanced than implied in the target article, as is often the case in neuroscientific studies of complex human behaviour. In our study, we tested the effects of the selective serotonin reuptake …. (shrink)

Moral enhancement is a topic that has sparked much current interest in the world of bioethics. The possibility of making people ‘better,’ not just in the conventional enhancement sense of improving health and other desirable qualities and capacities, but by making them somehow more moral, more decent, altogether better people, has attracted attention from both advocates 1 2 and sceptics 3 alike. The concept of moral enhancement, however, is fraught with difficult questions, theoretical and practical. What does it actually mean (...) to be ‘more moral’? How would moral enhancement be defined and would it necessarily, as some have claimed, make the world a better or safer place? How would or could such enhancement be achieved safely and without undue constraint on personal liberty and autonomy? On this subject, a recent paper by Crockett et al 4 investigating the effects of the neurotransmitter serotonin on moral decision-making provides an intriguing scientific basis for examining what might or might not constitute moral enhancement. The study involved treatment with citalopram, a drug that increases the action of serotonin in the brain, and subsequent analysis of participants' decision-making behaviour in two different situations involving moral dilemmas: the well-known ‘Trolley Problem’ 5 and the ‘Ultimatum Game’. 6 The researchers found that citalopram promoted what they call ‘prosocial behaviour’, increasing the participants' aversion to causing direct harm to others: in the first scenario, they were less likely to select the option that required killing one in order to save five, and in the second, less likely to reject unfair offers at the expense of others. The Crockett study is fascinating both for its insight into human behaviour and because it appears to demonstrate that, at least on some accounts of moral behaviour, serotonin may in fact be a …. (shrink)

We critically review themushrooming literature addressing the neuralmechanisms of moral cognition (NMMC), reachingthe following broad conclusions: (1) researchmainly focuses on three inter-relatedcategories: the moral emotions, moral socialcognition, and abstract moral reasoning. (2)Research varies in terms of whether it deploysecologically valid or experimentallysimplified conceptions of moral cognition. Themore ecologically valid the experimentalregime, the broader the brain areas involved.(3) Much of the research depends on simplifyingassumptions about the domain of moral reasoningthat are motivated by the need to makeexperimental progress. This is a (...) valuablebeginning, but as more is understood about theneural mechanisms of decision-making, morerealistic conceptions will need to replace thesimplified conceptions. (4) The neuralcorrelates of real-life moral cognition areunlikely to consist in anything remotely like a``moral module'' or a ``morality center.'' Moralrepresentations, deliberations and decisionsare probably highly distributed and notconfined to any particular brainsub-system. Discovering the basic neuralprinciples governing planning, judgment anddecision-making will require vastly more basicresearch in neuroscience, but correlatingactivity in certain brain regions withwell-defined psychological conditions helpsguide neural level research. Progress on socialphenomena will also require theoreticalinnovation in understanding the brain'sdistinctly biological form of computationthat is anchored by emotions, needs, drives,and the instinct for survival. (shrink)

In a series of recent works, Julian Savulescu and Ingmar Persson insist that, given the ease by which irreversible destruction is achievable by a morally wicked minority, (i) strictly cognitive bio-enhancement is currently too risky, while (ii) moral bio-enhancement is plausibly morally mandatory (and urgently so). This article aims to show that the proposal Savulescu and Persson advance relies on several problematic assumptions about the separability of cognitive and moral enhancement as distinct aims. Specifically, we propose that the underpinnings of (...) Savulescu's and Persson's normative argument unravel once it is suitably clear how aiming to cognitively enhance an individual will in part require that one aim to bring about certain moral goods we show to be essential to cognitive flourishing; conversely, aiming to bring about moral enhancement in an individual must involve aiming to improve certain cognitive capacities we show to be essential to moral flourishing. After developing these points in some detail, and their implication for Savulescu's & Persson's proposal, we conclude by outlining some positive suggestions. (shrink)