In this paper, I argue for a new normative theory of rational choice under risk, namely expected comparative utility (ECU) theory. I first show that for any choice option, a, and for any state of the world, G, the measure of the choiceworthiness of a in G is the comparative utility (CU) of a in G—that is, the difference in utility, in G, between a and whichever alternative to a carries the greatest utility in G. On the basis of this (...) principle, I then argue that for any agent, S, faced with any decision under risk, S should rank his or her decision options (in terms of how choiceworthy they are) according to their comparative expected comparative utility (CECU) and should choose whichever option carries the greatest CECU. For any option, a, a's CECU is the difference between its ECU and that of whichever alternative to a carries the greatest ECU, where a's ECU is a probability-weighted sum of a's CUs across the various possible states of the world. I lastly demonstrate that in some ordinary decisions under risk, ECU theory delivers different verdicts from those of standard decision theory. (shrink)

New AI technologies have the potential to cause unintended harms in diverse domains including warfare, judicial sentencing, biomedicine and governance. One strategy for realising the benefits of AI whilst avoiding its potential dangers is to ensure that new AIs are properly ‘aligned’ with some form of ‘alignment target.’ One danger of this strategy is that – dependent on the alignment target chosen – our AIs might optimise for objectives that reflect the values only of a certain subset of society, and (...) that do not take into account alternative views about what constitutes desirable and safe behaviour for AI agents. In response to this problem, several AI ethicists have suggested alignment targets that are designed to be sensitive to widespread normative disagreement amongst the relevant stakeholders. Authors inspired by voting theory have suggested that AIs should be aligned with the verdicts of actual or simulated ‘moral parliaments’ whose members represent the normative views of the relevant stakeholders. Other authors inspired by decision theory and the philosophical literature on moral uncertainty have suggested that AIs should maximise socially expected choiceworthiness. In this paper, I argue that both of these proposals face several important problems. In particular, they fail to select attractive ‘compromise options’ in cases where such options are available. I go on to propose and defend an alternative, bargaining-theoretic alignment target, which avoids the problems associated with the voting- and decision-theoretic approaches. (shrink)

How should you decide what to do when you’re uncertain about basic normative principles? A natural suggestion is to follow some "second-order:" norm: e.g., obey the most probable norm or maximize expected choiceworthiness. But what if you’re uncertain about second-order norms too—must you then invoke some third-order norm? If so, any norm-guided response to normative uncertainty appears doomed to a vicious regress. This paper aims to rescue second-order norms from the threat of regress. I first elaborate and defend the claim (...) some philosophers have made that the regress problem forces us to accept normative externalism, the view that at least one norm is incumbent on all agents regardless of their normative beliefs. But, I then argue, we need not accept externalism about first-order norms, thus closing off any question of how agents should respond to normative uncertainty. Rather, we can head off the threat of regress by ascribing external force to a single second-order norm: the enkratic principle. (shrink)

This paper aims to address the question of how one ought to choose when one is uncertain about what outcomes will result from one’s choices, but when one can nevertheless assign probabilities to the different possible outcomes. These choices are commonly referred to as choices (or decisions) under risk. I assume in this paper that one ought to make instrumentally rational choices—more precisely, one ought to adopt suitable means to one’s morally permissible ends. Expected utility (EU) theory is generally accepted (...) as a normative theory of rational choice under risk, or, more specifically, as a theory of instrumental rationality. According to EU theory, when faced with a decision under risk, one ought to rank one’s options (from least to most choiceworthy) according to their EU and one ought to choose whichever option carries the greatest EU (or one of them in the event that several alternatives are tied). The EU of an option is a probability-weighted sum of each of its possible utilities. In this paper, I argue that EU theory is not the correct theory of instrumental rationality. In its place, I argue for a new theory of instrumental rationality, namely expected comparative utility (ECU) theory. I first show that for any choice option, a, and for any state of the world, G, the measure of the choiceworthiness of a in G is the comparative utility of a in G—that is, the difference in utility, in G, between a and whichever alternative to a carries the greatest utility in G. On the basis of this principle, I then argue, roughly speaking, that for any agent, S, faced with any decision under risk, S ought to rank her options (in terms of how choiceworthy they are) according to their ECU and S ought to choose whichever option carries the greatest ECU (or one of them in the event that several alternatives are tied). For any option, a, a’s ECU is a probability‐weighted sum of a’s comparative utilities across the various possible states of the world. In this paper, I show that in some commonplace decisions under risk, ECU theory delivers different verdicts from those of EU theory. (shrink)

In this paper, we present and defend a natural yet novel analysis of normative reasons. According to what we call support-explanationism, for a fact to be a normative reason to φ is for it to explain why there's normative support for φ-ing. We critically consider the two main rival forms of explanationism—ought-explanationism, on which reasons explain facts about ought, and good-explanationism, on which reasons explain facts about goodness—as well as the popular Reasons-First view, which takes the notion of a normative (...) reason to be normatively fundamental. Support-explanationism, we argue, enjoys many of the virtues of these views while avoiding their drawbacks. We conclude by exploring several further important implications: among other things, we argue that the influential metaphor of ‘weighing’ reasons is inapt, and propose a better one; that, contrary to what Berker (2019) suggests, there's no reason for non-naturalists about normativity to accept the Reasons-First view; and that, contrary to what Wodak (2020b) suggests, explanationist views can successfully accommodate what he calls ‘redundant reasons’. (shrink)

We defend three controversial claims about preference, credence, and choice. First, all agents (not just rational ones) have complete preferences. Second, all agents (again, not just rational ones) have real-valued credences in every proposition in which they are confident to any degree. Third, there is almost always some unique thing we ought to do, want, or believe.

How should you decide what to do when you're uncertain about basic normative principles (e.g., Kantianism vs. utilitarianism)? A natural suggestion is to follow some "second-order" norm: e.g., "comply with the first-order norm you regard as most probable" or "maximize expected choiceworthiness". But what if you're uncertain about second-order norms too -- must you then invoke some third-order norm? If so, it seems that any norm-guided response to normative uncertainty is doomed to a vicious regress. In this paper, I aim (...) to rescue second-order norms from this threat of regress. I first elaborate and defend the suggestion some philosophers have entertained that the regress problem forces us to accept normative externalism, the view that at least one norm is incumbent on agents regardless of their beliefs or evidence concerning that norm. But, I then argue, we need not accept externalism about first-order (e.g., moral) norms, thus closing off any question of what an agent should do in light of her normative beliefs. Rather, it is more plausible to ascribe external force to a single, second-order rational norm: the enkratic principle, correctly formulated. This modest form of externalism, I argue, is both intrinsically well-motivated and sufficient to head off the threat of regress. (shrink)

This paper looks at whether it is possible to unify the requirements of rationality with the demands of normative reasons. It might seem impossible to do because one depends upon the agent’s perspective and the other upon features of the situation. Enter Reasons Perspectivism. Reasons perspectivists think they can show that rationality does consist in responding correctly to reasons by placing epistemic constraints on these reasons. They think that if normative reasons are subject to the right epistemic constraints, rational requirements (...) will correspond to the demands generated by normative reasons. While this proposal is prima facie plausible, it cannot ultimately unify reasons and rationality. There is no epistemic constraint that can do what reasons perspectivists would need it to do. Some constraints are too strict. The rest are too slack. This points to a general problem with the reasons-first program. Once we recognize that the agent’s epistemic position helps determine what she should do, we have to reject the idea that the features of the agent’s situation can help determine what we should do. Either rationality crowds out reasons and their demands or the reasons will make unreasonable demands. (shrink)

Determining the precise nature of the connection between preference, choice, and welfare has arguably been the central project in the field of welfare economics, which aims to offer a proper guide for economists in the making of policy decisions that affect people’s welfare. The two leading approaches here historically – the revealed preference and latent preference approaches – seem equally incapable of so guiding economists. I argue that the deadlock here is due to welfare economists’ failure to recognize a crucial (...) distinction between two senses of “preference.” I analyze and defend these senses of “preference,” and argue that each shares a close connection with just one of choice and welfare. This analysis reveals how economists should conceive of both the connections between “preference,” choice, and welfare, and the proper roles of these concepts in welfare economics. I conclude by showing this analysis to best explain the plausibility of two leading alternative approaches from Hausman and Sugden. (shrink)

Normal agents in the actual world are limited: they cannot think about all the options that are available to them—or even about all options that are available to them according to their evidence. Moreover, agents cannot choose an option unless they have thought about that option. Such agents can be irrational in two ways: either by making their choice too quickly, without canvassing enough options, or by wasting time canvassing ever more options when they have already thought of enough options. (...) This chapter proposes an account of this kind of rationality for limited agents. To be rational, such agents need a kind of background sensitivity to whether or not continuing to search for more options has greater expected value than deciding right away. According to this proposal, it is the unavoidable predicament of choice that all rational decision-making depends on the unconscious operation of mental dispositions. (shrink)