Computers are already approving financial transactions, controlling electrical supplies, and driving trains. Soon, service robots will be taking care of the elderly in their homes, and military robots will have their own targeting and firing protocols. Colin Allen and Wendell Wallach argue that as robots take on more and more responsibility, they must be programmed with moral decision-making abilities, for our own safety. Taking a fast paced tour through the latest thinking about philosophical ethics and artificial intelligence, the authors argue (...) that even if full moral agency for machines is a long way off, it is already necessary to start building a kind of functional morality, in which artificial moral agents have some basic ethical sensitivity. But the standard ethical theories don't seem adequate, and more socially engaged and engaging robots will be needed. As the authors show, the quest to build machines that are capable of telling right from wrong has begun. -/- Moral Machines is the first book to examine the challenge of building artificial moral agents, probing deeply into the nature of human decision making and ethics. (shrink)

Better Never to Have Been argues for a number of related, highly provocative, views: (1) Coming into existence is always a serious harm. (2) It is always wrong to have children. (3) It is wrong not to abort fetuses at the earlier stages of gestation. (4) It would be better if, as a result of there being no new people, humanity became extinct. These views may sound unbelievable--but anyone who reads Benatar will be obliged to take them seriously.

The literature on the theory of social choice has grown considerably beyond the few items in existence at the time the first edition of this book appeared in 1951. Some of the new literature has dealt with the technical, mathematical aspects, more with the interpretive. My own thinking has also evolved somewhat, although I remain far from satisfied with present formulations. The exhaustion of the first edition provides a convenient time for a selective and personal stocktaking in the form of (...) an appended commentary entitled, 'Notes on the Theory of Social Choice, 1963, ' containing reflections on the text and its omissions and on some of the more recent literature. This form has seemed more appropriate than a revision of the original text, which has to some extent acquired a life of its own. (shrink)

As arti® cial intelligence moves ever closer to the goal of producing fully autonomous agents, the question of how to design and implement an arti® cial moral agent (AMA) becomes increasingly pressing. Robots possessing autonomous capacities to do things that are useful to humans will also have the capacity to do things that are harmful to humans and other sentient beings. Theoretical challenges to developing arti® cial moral agents result both from controversies among ethicists about moral theory itself, and from (...) computational limits to the implementation of such theories. In this paper the ethical disputes are surveyed, the possibility of a `moral Turing Test ’ is considered and the computational di culties accompanying the diŒerent types of approach are assessed. Human-like performance, which is prone to include immoral actions, may not be acceptable in machines, but moral perfection may be computationally unattainable. The risks posed by autonomous machines ignorantly or deliberately harming people and other sentient beings are great. The development of machines with enough intelligence to assess the eŒects of their actions on sentient beings and act accordingly may ultimately be the most important task faced by the designers of arti® cially intelligent automata. (shrink)

The term “posthuman” has been used in very different senses by different authors.2 I am sympathetic to the view that the word often causes more confusion than clarity, and that we might be better off replacing it with some alternative vocabulary.

Suppose that A and B are two kinds of goods such that more of each is better than less. A is strongly superior to B if any amount of A is better than any amount of B. It is weakly superior to B if some amount of A is better than any amount of B. There are many examples of these relations in the literature, sometimes under the labels “higher goods” and “discontinuity.” The chapter gives a precise and generalized statement (...) of Strong and Weak Superiority and discusses different ways in which these relations can be relevant to the aggregation of welfare. It also proves a number of general results. One of the results gives rise to a dilemma: It can be used as an argument against the existence of value superiority or, alternatively, as an argument against the view that superiority entails a radical difference in value. (shrink)

Can it be better or worse for a person to exist than not to exist at all? This old and challenging existential question has been raised anew in contemporary moral philosophy, mainly for two reasons. First, traditional “impersonal” ethical theories, such as utilitarianism, have counterintuitive implications in population ethics, for example, the repugnant conclusion. Second, it has seemed evident to many that an outcome can be better than another only if it is better for someone, and that only moral theories (...) that are in this sense “person affecting” can be correct. The implications of this Person-Affecting Restriction will differ radically, however, depending on which answer one gives to the existential question. The negative answer, which we argue against, would make the restriction quite untenable. Hence, many of the problems regarding our moral duties to future generations turn around the issue at hand. (shrink)

Extreme human enhancement could result in “posthuman” modes of being. After offering some definitions and conceptual clarification, I argue for two theses. First, some posthuman modes of being would be very worthwhile. Second, it could be very good for human beings to become posthuman.

Robots are becoming an increasingly pervasive feature of our personal lives. As a result, there is growing importance placed on examining what constitutes appropriate behavior when they interact with human beings. In this paper, we discuss whether companion robots should be permitted to “nudge” their human users in the direction of being “more ethical”. More specifically, we use Rawlsian principles of justice to illustrate how robots might nurture “socially just” tendencies in their human counterparts. Designing technological artifacts in such a (...) way to influence human behavior is already well-established but merely because the practice is commonplace does not necessarily resolve the ethical issues associated with its implementation. (shrink)

Population axiology concerns how to evaluate populations in regard to their goodness, that is, how to order populations by the relations \is better than" and \is as good as". This eld has been riddled with para- doxes and impossibility results which seem to show that our considered beliefs are inconsistent in cases where the number of people and their welfare varies. All of these results have one thing in common, however. They all involve an adequacy condition that rules out Derek (...) Par t's Repugnant Conclusion. Moreover, some theorists have argued that we should accept the Repugnant Conclusion and hence that avoidance of this conclusion is not a convincing adequacy condition for a population axiology. As I shall show in this chapter, however, one can replace avoid- ance of the Repugnant Conclusion with a logically weaker and intuitively more convincing condition. The resulting theorem involves, to the best of my knowledge, logically weaker and intuitively more compelling con- ditions than the other theorems presented in the literature. As such, it challenges the very existence of a satisfactory population ethics. (shrink)

In this essay we develop and argue for the adoption of a more comprehensive model of research ethics than is included within current conceptions of responsible conduct of research (RCR). We argue that our model, which we label the ethical dimensions of scientific research (EDSR), is a more comprehensive approach to encouraging ethically responsible scientific research compared to the currently typically adopted approach in RCR training. This essay focuses on developing a pedagogical approach that enables scientists to better understand and (...) appreciate one important component of this model, what we call intrinsic ethics . Intrinsic ethical issues arise when values and ethical assumptions are embedded within scientific findings and analytical methods. Through a close examination of a case study and its application in teaching, namely, evaluation of climate change integrated assessment models, this paper develops a method and case for including intrinsic ethics within research ethics training to provide scientists with a comprehensive understanding and appreciation of the critical role of values and ethical choices in the production of research outcomes. (shrink)

We report a series of experiments carried out with Palestinian and Israeli participants showing that violent opposition to compromise over issues considered sacred is increased by offering material incentives to compromise but decreased when the adversary makes symbolic compromises over their own sacred values. These results demonstrate some of the unique properties of reasoning and decision-making over sacred values. We show that the use of material incentives to promote the peaceful resolution of political and cultural conflicts may backfire when adversaries (...) treat contested issues as sacred values. (shrink)

Population axiology concerns how to evaluate populations in regard to their goodness, that is, how to order populations by the relations “is better than ” and “is as good as”. This field has been riddled with paradoxes and impossibility results which seem to show that our considered beliefs are inconsistent in cases where the number of people and their welfare varies. All of these results have one thing in common, however. They all involve an adequacy condition that rules out Derek (...) Parfit’s Repugnant Conclusion. Moreover, some theorists have argued that we should accept the Repugnant Conclusion and hence that avoidance of this conclusion is not a convincing adequacy condition for a population axiology. As I shall show in this chapter, however, one can replace avoidance of the Repugnant Conclusion with a logically weaker and intuitively more convincing condition. The resulting theorem involves, to the best of my knowledge, logically weaker and intuitively more compelling conditions than the other theorems presented in the literature. As such, it. (shrink)

Who should decide how a machine will decide what to do when it is driving a car, performing a medical procedure, or, more generally, when it is facing any kind of morally laden decision? More and more, machines are making complex decisions with a considerable level of autonomy. We should be much more preoccupied by this problem than we currently are. After a series of preliminary remarks, this paper will go over four possible answers to the question raised above. First, (...) we may claim that it is the maker of a machine that gets to decide how it will behave in morally laden scenarios. Second, we may claim that the users of a machine should decide. Third, that decision may have to be made collectively or, fourth, by other machines built for this special purpose. The paper argues that each of these approaches suffers from its own shortcomings, and it concludes by showing, among other things, which approaches should be emphasized for different types of machines, situations, and/or morally laden decisions. (shrink)

The National Science Foundation's Second Merit Criterion, or Broader Impacts Criterion , was introduced in 1997 as the result of an earlier Congressional movement to enhance the accountability and responsibility as well as the effectiveness of federally funded projects. We demonstrate that a robust understanding and appreciation of NSF BIC argues for a broader conception of research ethics in the sciences than is currently offered in Responsible Conduct of Research training. This essay advocates augmenting RCR education with training regarding broader (...) impacts. We demonstrate that enhancing research ethics training in this way provides a more comprehensive understanding of the ethics relevant to scientific research and prepares scientists to think not only in terms of responsibly conducted science, but also of the role of science in responding to identified social needs and in adhering to principles of social justice. As universities respond to the mandate from America COMPETES to “provide training and oversight in the responsible and ethical conduct of research”, we urge institutions to embrace a more adequate conception of research ethics, what we call the Ethical Dimensions of Scientific Research, that addresses the full range of ethical issues relevant to scientific inquiry, including ethical issues related to the broader impacts of scientific research and practice. (shrink)

Utilitarianism seems to be going out of fashion, amidst increasing concerns for issues of freedom, equality, and justice. At least, anti-utilitarian and non-utilitarian moral philosophers have been very active. This paper is a very modest attempt to defend utilitarianism in particular and welfarism in general. Section I provides an axiomatic defence of welfarism and utilitarianism. Section II discusses the divergences between individual preferences and individual welfares and argues in favour of welfare utilitarianism. Section III criticizes some non-utilitarian principles, including knowledge (...) as intrinsically good, rights-based ethics, and Rawls's second principle. Section IV argues that most objections to welfarism are probably based on the confusion of non-ultimate considerations with basic values. This is discussed with reference to some recent philosophical writings which abound with such confusion. Section V argues that the acceptance of utilitarianism may be facilitated by the distinction between ideal morality and self-interest which also resolves the dilemma of average versus total utility maximization in optimal population theory. (shrink)

Two theses are proposed; regarding the future evolution of the value systems of advanced AGI systems. The Value Learning Thesis is a semi-formalized version of the idea that; if an AGI system is taught human values in an interactive and experiential way as its intelligence increases toward human level; it will likely adopt these human values in a genuine way. The Value Evolution Thesis is a semi-formalized version of the idea that if an AGI system begins with human-like values; and (...) then iteratively modifies itself; it will end up in roughly the same future states as a population of human beings engaged with progressively increasing their own intelligence. Taken together; these theses suggest a worldview in which raising young AGIs to have human-like values is a sensible thing to do; and likely to produce a future that is generally desirable in a human sense. While these two theses are far from definitively proven; I argue that they are more solid and more relevant to the actual future of AGI than Bostrom’s “Instrumental Convergence Thesis” and “Orthogonality Thesis” which are core to the basis of his argument for fearing ongoing AGI development and placing AGI R&D under strict governmental control. In the context of fleshing out this argument; previous publications and discussions by Richard Loosemore and Kaj Sotala are discussed in some detail. (shrink)

A recent exchange on Benatar’s book Better never to have been between Doyal and Benatar discusses Benatar’s bold claim that people should not be brought into existence. Here, I expand the discussion of original position that the exchange focused on. I also discuss the asymmetries, between benefit and harm and between existence and non-existence, upon which Benatar’s bold claim rests. In both discussions, I show how Benatar’s bold claim can be rejected.

There are two principles often found in environmental ethics – self-realisation and environmental preservation. I argue that these are two logically independent principles. An analysis of its essential features shows that the preservation principle should be based on actual consequentialism, for it is only the actual effects of our actions and policies that are important to the main issues of environmental preservation. Aldo Leopold's land ethic is found to be an example of a consequentialistic theory of environmental preservation.

A principal goal of the discipline of artificial morality is to design artificial agents to act as if they are moral agents. Intermediate goals of artificial morality are directed at building into AI systems sensitivity to the values, ethics, and legality of activities. The development of an effective foundation for the field of artificial morality involves exploring the technological and philosophical issues involved in making computers into explicit moral reasoners. The goal of this paper is to discuss strategies for implementing (...) artificial morality and the differing criteria for success that are appropriate to different strategies. (shrink)

The implementation of moral decision making abilities in artificial intelligence (AI) is a natural and necessary extension to the social mechanisms of autonomous software agents and robots. Engineers exploring design strategies for systems sensitive to moral considerations in their choices and actions will need to determine what role ethical theory should play in defining control architectures for such systems. The architectures for morally intelligent agents fall within two broad approaches: the top-down imposition of ethical theories, and the bottom-up building of (...) systems that aim at goals or standards which may or may not be specified in explicitly theoretical terms. In this paper we wish to provide some direction for continued research by outlining the value and limitations inherent in each of these approaches. (shrink)