This paper presents a unifying diagnosis of a number of important problems facing existing models of rational choice under moral uncertainty and proposes a remedy. I argue that the problems of (i) severely limited scope, (ii) intertheoretic comparisons, and (iii) 'swamping’ all stem from the way in which values are assigned to options in decision rules such as Maximisation of Expected Choiceworthiness. By assigning values to options under a given moral theory by asking something like ‘how much do I desire (...) this option, supposing this theory is true?’ rather than ‘how much value does this theory assign to this option?’ these problems can be avoided, while the appealing features of these accounts can be preserved. This amendment provides a role for the preferences, desires, or goals of rational agents that is curiously absent from the existing discussion of what individuals rationally ought to do when they are uncertain about what they morally ought to do. (shrink)

Nick Bostrom and others have suggested treating decision-making under moral uncertainty as analogous to parliamentary decision-making. The core suggestion of this “parliamentary approach” is that the competing moral theories function like delegates to the parliament, and that these delegates then make decisions by some combination of bargaining and voting. There seems some reason to hope that such an approach might avoid standard objections to existing approaches (for example, the “maximise expected choiceworthiness” (MEC) and “my favourite theory” approaches). However, the parliamentary (...) approach is so far extremely underspecified, making it largely indeterminate how such a model will in fact behave in the respects that those concerned with moral uncertainty care about. This paper explores one way of making it precise. We treat predicaments of moral uncertainty as analogous to bargaining situations alone (setting aside voting), and apply a version of the Nash solution that is standard in bargaining theory. The resulting model does indeed perform in many of the hoped-for ways. However, so also does a version of MEC that employs a structural approach to intertheoretic comparisons. It seems to us an open question which, regarding this version of MEC and the bargaining-theoretic approach, is superior to the other. We identify the key points on which the two differ. (shrink)

We begin by showing that every theory of the value of uncertain prospects must have one of three unpalatable properties. _Reckless_ theories recommend giving up a sure thing, no matter how good, for an arbitrarily tiny chance of enormous gain; _timid_ theories permit passing up an arbitrarily large potential gain to prevent a tiny increase in risk; _non-transitive_ theories deny the principle that, if A is better than B and B is better than C, then A must be better than (...) C. Having set up this trilemma, we study its horns. Non-transitivity has been much discussed; we focus on drawing out the costs and benefits of recklessness and timidity when it comes to axiology, decision theory, and normative uncertainty. (shrink)

A straightforward way to handle moral uncertainty is simply to follow the moral theory in which you have most credence. This approach is known as My Favourite Theory. In this paper, I argue that, in some cases, My Favourite Theory prescribes choices that are, sequentially, worse in expected moral value than the opposite choices according to each moral theory you have any credence in. In addition this, problem generalizes to other approaches that avoid intertheoretic comparisons of value, such as My (...) Favourite Option, the Borda Rule, Variance Normalization, and the Principle of Maximizing Expected Normalized Moral Value. (shrink)

This paper explores the idea that it is instrumentally valuable to learn normative truths. We consider an argument for "normative hedging" based on this principle, and examine the structure of decision-making under moral uncertainty that arises from it. But it also turns out that the value of normative information is inconsistent with the principle that learning *empirical* truths is instrumentally valuable. We conclude with a brief comment on "metanormative regress.".

Automated decision making for sentencing is the use of a software algorithm to analyse a convicted offender’s case and deliver a sentence. This chapter reviews the moral arguments for and against employing automated decision making for sentencing and finds that its use is in principle morally permissible. Specifically, it argues that well-designed automated decision making for sentencing will better approximate the just sentence than human sentencers. Moreover, it dismisses common concerns about transparency, privacy and bias as unpersuasive or inapplicable. The (...) chapter also notes that moral disagreement about theories of just sentencing are plausibly resolved by applying the principle of maximising expected moral choiceworthiness, and that automated decision making is better suited to the resulting ensemble model. Finally, the chapter considers the challenge posed by penal populism. The dispiriting conclusion is that although it is in theory morally desirable to use automated decision-making for criminal sentencing, it may well be the case that we ought not to try. (shrink)

While ordinary decision theory focuses on empirical uncertainty, real decision-makers also face normative uncertainty: uncertainty about value itself. From a purely formal perspective, normative uncertainty is comparable to (Harsanyian or Rawlsian) identity uncertainty in the 'original position', where one's future values are unknown. A comprehensive decision theory must address twofold uncertainty -- normative and empirical. We present a simple model of twofold uncertainty, and show that the most popular decision principle -- maximising expected value (`Expectationalism') -- has different formulations, namely (...) Ex-Ante Expectationalism, Ex-Post Expectationalism, and hybrid theories. These alternative theories recommend different decisions, reasoning modes, and attitudes to risk. But they converge under an interesting (necessary and sufficient) condition. (shrink)

How should you decide what to do when you're uncertain about basic normative principles (e.g., Kantianism vs. utilitarianism)? A natural suggestion is to follow some "second-order" norm: e.g., "comply with the first-order norm you regard as most probable" or "maximize expected choiceworthiness". But what if you're uncertain about second-order norms too -- must you then invoke some third-order norm? If so, it seems that any norm-guided response to normative uncertainty is doomed to a vicious regress. In this paper, I aim (...) to rescue second-order norms from this threat of regress. I first elaborate and defend the suggestion some philosophers have entertained that the regress problem forces us to accept normative externalism, the view that at least one norm is incumbent on agents regardless of their beliefs or evidence concerning that norm. But, I then argue, we need not accept externalism about first-order (e.g., moral) norms, thus closing off any question of what an agent should do in light of her normative beliefs. Rather, it is more plausible to ascribe external force to a single, second-order rational norm: the enkratic principle, correctly formulated. This modest form of externalism, I argue, is both intrinsically well-motivated and sufficient to head off the threat of regress. (shrink)

How should you decide what to do when you’re uncertain about basic normative principles? A natural suggestion is to follow some "second-order:" norm: e.g., obey the most probable norm or maximize expected choiceworthiness. But what if you’re uncertain about second-order norms too—must you then invoke some third-order norm? If so, any norm-guided response to normative uncertainty appears doomed to a vicious regress. This paper aims to rescue second-order norms from the threat of regress. I first elaborate and defend the claim (...) some philosophers have made that the regress problem forces us to accept normative externalism, the view that at least one norm is incumbent on all agents regardless of their normative beliefs. But, I then argue, we need not accept externalism about first-order norms, thus closing off any question of how agents should respond to normative uncertainty. Rather, we can head off the threat of regress by ascribing external force to a single second-order norm: the enkratic principle. (shrink)

This paper presents some impossibility results for certain views about what you should do when you are uncertain about which moral theory is true. I show that under reasonable and extremely minimal ways of defining what a moral theory is, it follows that the concept of expected moral choiceworthiness is undefined, and more generally that any theory of decision-making under moral uncertainty must generate pathological results.

Rohana faces a choice where she can produce either a better outcome by lying or a worse outcome by telling the truth. She justifiably, but falsely, believes in.

ABSTRACT Is uncertainty about preferences rationally possible? And if so, does it matter for rational decision? It is argued that uncertainty about preferences is possible and should play the same role in rational decision-making as uncertainty about worldly facts. The paper develops this hypothesis and defends it against various objections.