I analyze recent discussions about making moral decisions under normative uncertainty. I discuss whether this kind of uncertainty should have practical consequences for decisions and whether there are reliable methods of reasoning that deal with the possibility that we are wrong about some moral issues. I defend a limited use of the decision theory model of reasoning in cases of normative uncertainty.

Most ethical work is done at a low level of formality. This makes practical moral questions inaccessible to formal and natural sciences and can lead to misunderstandings in ethical discussion. In this paper, we use Bayesian inference to introduce a formalization of preference utilitarianism in physical world models, specifically cellular automata. Even though our formalization is not immediately applicable, it is a first step in providing ethics and ultimately the question of how to “make the world better” with a formal (...) basis. (shrink)

This article gives two arguments for believing that our society is unknowingly guilty of serious, large-scale wrongdoing. First is an inductive argument: most other societies, in history and in the world today, have been unknowingly guilty of serious wrongdoing, so ours probably is too. Second is a disjunctive argument: there are a large number of distinct ways in which our practices could turn out to be horribly wrong, so even if no particular hypothesized moral mistake strikes us as very likely, (...) the disjunction of all such mistakes should receive significant credence. The article then discusses what our society should do in light of the likelihood that we are doing something seriously wrong: we should regard intellectual progress, of the sort that will allow us to find and correct our moral mistakes as soon as possible, as an urgent moral priority rather than as a mere luxury; and we should also consider it important to save resources and cultivate flexibility, so that when the time comes to change our policies we will be able to do so quickly and smoothly. (shrink)

In “Rejecting Ethical Deflationism,” Jacob Ross argues that a rational decision maker is permitted, for the purposes of practical reasoning, to assume that nihilism is false. I argue that Ross’s argument fails because the principle he relies on conflicts with more plausible principles of rationality and leads to preference cycles. I then show how the infectiousness of nihilism, and of incomparability more generally, poses a serious problem for the larger project of attempting to incorporate moral uncertainty into expected value maximization (...) style reasoning. (shrink)

This paper discusses the relation between intelligence and motivation in artificial agents, developing and briefly arguing for two theses. The first, the orthogonality thesis, holds (with some caveats) that intelligence and final goals (purposes) are orthogonal axes along which possible artificial intellects can freely vary—more or less any level of intelligence could be combined with more or less any final goal. The second, the instrumental convergence thesis, holds that as long as they possess a sufficient level of intelligence, agents having (...) any of a wide range of final goals will pursue similar intermediary goals because they have instrumental reasons to do so. In combination, the two theses help us understand the possible range of behavior of superintelligent agents, and they point to some potential dangers in building such an agent. (shrink)

The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. Other animals have stronger muscles or sharper claws, but we have cleverer brains. If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of (...) our species then would come to depend on the actions of the machine superintelligence. But we have one advantage: we get to make the first move. Will it be possible to construct a seed AI or otherwise to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation? To get closer to an answer to this question, we must make our way through a fascinating landscape of topics and considerations. Read the book and learn about oracles, genies, singletons; about boxing methods, tripwires, and mind crime; about humanity's cosmic endowment and differential technological development; indirect normativity, instrumental convergence, whole brain emulation and technology couplings; Malthusian economics and dystopian evolution; artificial intelligence, and biological cognitive enhancement, and collective intelligence. (shrink)

We are often uncertain how to behave morally in complex situations. In this controversial study, Ted Lockhart contends that moral philosophy has failed to address how we make such moral decisions. Adapting decision theory to the task of decision-making under moral uncertainly, he proposes that we should not always act how we feel we ought to act, and that sometimes we should act against what we feel to be morally right. Lockhart also discusses abortion extensively and proposes new ways to (...) deal with the ethical and moral issues which surround it. (shrink)

A principal goal of the discipline of artificial morality is to design artificial agents to act as if they are moral agents. Intermediate goals of artificial morality are directed at building into AI systems sensitivity to the values, ethics, and legality of activities. The development of an effective foundation for the field of artificial morality involves exploring the technological and philosophical issues involved in making computers into explicit moral reasoners. The goal of this paper is to discuss strategies for implementing (...) artificial morality and the differing criteria for success that are appropriate to different strategies. (shrink)

Several recent articles have weighed in on the question of whether moral philosophers can be counted as moral experts. One argument denying this has been rejected by both sides of the debate. According to this argument, the extent of disagreement in modern moral philosophy prevents moral philosophers from being classified as moral experts. Call this the Argument From Disagreement. In this article, I defend a version of AD. Insofar as practical issues in moral philosophy are characterized by disagreement between moral (...) philosophers who are more or less equally well credentialed on the issue, non-philosophers have no good reasons to defer to their views. (shrink)

We examined the effects of order of presentation on the moral judgments of professional philosophers and two comparison groups. All groups showed similar-sized order effects on their judgments about hypothetical moral scenarios targeting the doctrine of the double effect, the action-omission distinction, and the principle of moral luck. Philosophers' endorsements of related general moral principles were also substantially influenced by the order in which the hypothetical scenarios had previously been presented. Thus, philosophical expertise does not appear to enhance the stability (...) of moral judgments against this presumably unwanted source of bias, even given familiar types of cases and principles. (shrink)

Many philosophers appeal to intuitions to support some philosophical views. However, there is reason to be concerned about this practice as scientific evidence has documented systematic bias in philosophically relevant intuitions as a function of seemingly irrelevant features (e.g., personality). One popular defense used to insulate philosophers from these concerns holds that philosophical expertise eliminates the influence of these extraneous factors. Here, we test this assumption. We present data suggesting that verifiable philosophical expertise in the free will debate-as measured by (...) a reliable and validated test of expert knowledge-does not eliminate the influence of one important extraneous feature (i.e., the heritable personality trait extraversion) on judgments concerning freedom and moral responsibility. These results suggest that, in at least some important cases, the expertise defense fails. Implications for the practice of philosophy, experimental philosophy, and applied ethics are discussed. (shrink)

Some philosophers have recently argued that decision-makers ought to take normative uncertainty into account in their decisionmaking. These philosophers argue that, just as it is plausible that we should maximize expected value under empirical uncertainty, it is plausible that we should maximize expected choice-worthiness under normative uncertainty. However, such an approach faces two serious problems: how to deal with merely ordinal theories, which do not give sense to the idea of magnitudes of choice-worthiness; and how, even when theories do give (...) sense to magnitudes of choice-worthiness, to compare magnitudes of choice-worthiness across different theories. Some critics have suggested that these problems are fatal to the project of developing a normative account of decision-making under normative uncertainty. The primary purpose of this article is to show that this is not the case. To this end, I develop an analogy between decision-making under normative uncertainty and the problem of social choice, and then argue that the Borda Rule provides the best way of making decisions in the face of merely ordinal theories and intertheoretic incomparability. (shrink)

It has been argued by several philosophers that a morally motivated rational agent who has to make decisions under conditions of moral uncertainty ought to maximize expected moral value in his choices, where the expectation is calculated relative to the agent's moral uncertainty. I present a counter-example to this thesis and to a larger family of decision rules for choice under conditions of moral uncertainty. Based on this counter-example, I argue against the thesis and suggest a reason for its failure (...) – that it is based on the false assumption that inter-theoretical comparisons of moral value are meaningful. (shrink)

Professional philosophers are members of bioethical committees and regulatory bodies in areas of interest to bioethicists. This suggests they possess moral expertise even if they do not exercise it directly and without constraint. Moral expertise is defined, and four arguments given in support of scepticism about their possession of such expertise are considered and rejected: the existence of extreme disagreement between moral philosophers about moral matters; the lack of a means clearly to identify moral experts; that expertise cannot be claimed (...) in that which lacks objectivity; and that ordinary people do not follow the advice of moral experts.I offer a better reason for scepticism grounded in the relation between moral philosophy and common-sense morality: namely that modern moral philosophy views even a developed moral theory as ultimately anchored in common-sense morality, that set of basic moral precepts which ordinary individuals have command of and use to regulate their own lives.Even if moral philosophers do nevertheless have a limited moral expertise, in that they alone can fully develop a set of moral judgments, I sketch reasons – grounded in the values of autonomy and of democracy – why moral philosophers should not wish non-philosophers to defer to their putative expertise. (shrink)

This paper surveys recent work on moral expertise. Much of that work defends an asymmetry thesis according to which the cognitive deference to expertise that characterizes other areas of inquiry is out of place in morality. There are two reasons why you might think asymmetry holds. The problem might lie in the existence of expertise or in deferring to it. We argue that both types of arguments for asymmetry fail. They appear to be stronger than they are because of their (...) focus on moral expertise regarding all-in judgments about rightness. We reject this emphasis on all-in judgment in favor of an account of moral expertise as typically multi-stranded and domain limited. This account of moral expertise is better able to address the problem of how to identify those who have expertise. It also offers a more nuanced picture of the contrast between accepting a moral claim on one's own and accepting it on testimony. (shrink)

The growing field of machine morality has becoming increasingly concerned with how to develop artificial moral agents. However, there is little consensus on what constitutes an ideal moral agent let alone an artificial one. Leveraging a recent account of heroism in humans, the aim of this paper is to provide a prospective framework for conceptualizing, and in turn designing ideal artificial moral agents, namely those that would be considered heroic robots. First, an overview of what it means to be an (...) artificial moral agent is provided. Then, an overview of a recent account of heroism that seeks to define the construct as the dynamic and interactive integration of character strengths and situational constraints that afford the opportunity for moral behavior . With this as a foundation, a discussion is provided for what it might mean for a robot to be an ideal moral agent by proposing a dynamic and interactive connectionist model of robotic heroism. Given the limited accounts of robots engaging in moral behavior, a case for extending robotic moral capacities beyond just being a moral agent to the level of heroism is supported by drawing from exemplar situations where robots demonstrate heroism in popular film and fiction. (shrink)

This essay defends moral expertise against the skeptical considerations raised by Gilbert Ryle and others. The core of the essay articulates an account of moral expertise that draws on work on expertise in empirical moral psychology, and develops an analogy between moral expertise and linguistic expertise. The account holds that expertise is contrastive, so that a person is an expert relative to a particular contrast. Further, expertise is domain specific and characterized by “automatic” behavior and judgment. Some disagreements in the (...) literature regarding moral expertise are diagnosed as being due to failures to adequately distinguish different ways in which someone can be a moral expert. For example, expertise in action does not imply expertise in judgment or analysis. (shrink)

Herein we make a plea to machine ethicists for the inclusion of constraints on their theories consistent with empirical data on human moral cognition. As philosophers, we clearly lack widely accepted solutions to issues regarding the existence of free will, the nature of persons and firm conditions on moral agency/patienthood; all of which are indispensable concepts to be deployed by any machine able to make moral judgments. No agreement seems forthcoming on these matters, and we don’t hold out hope for (...) machines that can both always do the right thing (on some general ethic) and produce explanations for its behavior that would be understandable to a human confederate. Our tentative solution involves understanding the folk concepts associated with our moral intuitions regarding these matters, and how they might be dependent upon the nature of human cognitive architecture. It is in this spirit that we begin to explore the complexities inherent in human moral judgment via computational theories of the human cognitive architecture, rather than under the extreme constraints imposed by rational-actor models assumed throughout much of the literature on philosophical ethics. After discussing the various advantages and challenges of taking this particular perspective on the development of artificial moral agents, we computationally explore a case study of human intuitions about the self and causal responsibility. We hypothesize that a significant portion of the variance in reported intuitions for this case might be explained by appeal to an interplay between the human ability to mindread and to the way that knowledge is organized conceptually in the cognitive system. In the present paper, we build on a pre-existing computational model of mindreading (Bello et al. 2007) by adding constraints related to psychological distance (Trope and Liberman 2010), a well-established psychological theory of conceptual organization. Our initial results suggest that studies of folk concepts involved in moral intuitions lead us to an enriched understanding of cognitive architecture and a more systematic method for interpreting the data generated by such studies. (shrink)