Decision-making under normative uncertainty requires an agent to aggregate the assessments of options given by rival normative theories into a single assessment that tells her what to do in light of her uncertainty. But what if the assessments of rival theories differ not just in their content but in their structure -- e.g., some are merely ordinal while others are cardinal? This paper describes and evaluates three general approaches to this "problem of structural diversity": structural enrichment, structural depletion, and multi-stage (...) aggregation. All three approaches have notable drawbacks, but I tentatively defend multi-stage aggregation as least bad of the three. (shrink)

The topic of this thesis is axiological uncertainty – the question of how you should evaluate your options if you are uncertain about which axiology is true. As an answer, I defend Expected Value Maximisation (EVM), the view that one option is better than another if and only if it has the greater expected value across axiologies. More precisely, I explore the axiomatic foundations of this view. I employ results from state-dependent utility theory, extend them in various ways and interpret (...) them accordingly, and thus provide axiomatisations of EVM as a theory of axiological uncertainty. (shrink)

Rohana faces a choice where she can produce either a better outcome by lying or a worse outcome by telling the truth. She justifiably, but falsely, believes in.

Volume 98, Issue 4, December 2020, Page 747-762.

How should you decide what to do when you're uncertain about basic normative principles (e.g., Kantianism vs. utilitarianism)? A natural suggestion is to follow some "second-order" norm: e.g., "comply with the first-order norm you regard as most probable" or "maximize expected choiceworthiness". But what if you're uncertain about second-order norms too -- must you then invoke some third-order norm? If so, it seems that any norm-guided response to normative uncertainty is doomed to a vicious regress. In this paper, I aim (...) to rescue second-order norms from this threat of regress. I first elaborate and defend the suggestion some philosophers have entertained that the regress problem forces us to accept normative externalism, the view that at least one norm is incumbent on agents regardless of their beliefs or evidence concerning that norm. But, I then argue, we need not accept externalism about first-order (e.g., moral) norms, thus closing off any question of what an agent should do in light of her normative beliefs. Rather, it is more plausible to ascribe external force to a single, second-order rational norm: the enkratic principle, correctly formulated. This modest form of externalism, I argue, is both intrinsically well-motivated and sufficient to head off the threat of regress. (shrink)

According to the traditional Bayesian view of credence, its structure is that of precise probability, its objects are descriptive propositions about the empirical world, and its dynamics are given by conditionalization. Each of the three essays that make up this thesis deals with a different variation on this traditional picture. The first variation replaces precise probability with sets of probabilities. The resulting imprecise Bayesianism is sometimes motivated on the grounds that our beliefs should not be more precise than the evidence (...) calls for. One known problem for this evidentially motivated imprecise view is that in certain cases, our imprecise credence in a particular proposition will remain the same no matter how much evidence we receive. In the first essay I argue that the problem is much more general than has been appreciated so far, and that it’s difficult to avoid without compromising the initial evidentialist motivation. The second variation replaces descriptive claims with moral claims as the objects of credence. I consider three standard arguments for probabilism with respect to descriptive uncertainty—representation theorem arguments, Dutch book arguments, and accuracy arguments—in order to examine whether such arguments can also be used to establish probabilism with respect to moral uncertainty. In the second essay, I argue that by and large they can, with some caveats. First, I don’t examine whether these arguments can be given sound non-cognitivist readings, and any conclusions therefore only hold conditional on cognitivism. Second, decision-theoretic representation theorems are found to be less convincing in the moral case, because there they implausibly commit us to thinking that intertheoretic comparisons of value are always possible. Third and finally, certain considerations may lead one to think that imprecise probabilism provides a more plausible model of moral epistemology. The third variation considers whether, in addition to conditionalization, agents may also change their minds by becoming aware of propositions they had not previously entertained, and therefore not previously assigned any probability. More specifically, I argue that if we wish to make room for reflective equilibrium in a probabilistic moral epistemology, we must allow for awareness growth. In the third essay, I sketch the outline of such a Bayesian account of reflective equilibrium. Given that this account gives a central place to awareness growth, and that the rationality constraints on belief change by awareness growth are much weaker than those on belief change by conditionalization, it follows that the rationality constraints on the credences of agents who are seeking reflective equilibrium are correspondingly weaker. (shrink)

Given the deep disagreement surrounding population axiology, one should remain uncertain about which theory is best. However, this uncertainty need not leave one neutral about which acts are better or worse. We show that, as the number of lives at stake grows, the Expected Moral Value approach to axiological uncertainty systematically pushes one toward choosing the option preferred by the Total View and critical-level views, even if one’s credence in those theories is low.

The principle that rational agents should maximize expected utility or choiceworthiness is intuitively plausible in many ordinary cases of decision-making under uncertainty. But it is less plausible in cases of extreme, low-probability risk (like Pascal's Mugging), and intolerably paradoxical in cases like the St. Petersburg and Pasadena games. In this paper I show that, under certain conditions, stochastic dominance reasoning can capture most of the plausible implications of expectational reasoning while avoiding most of its pitfalls. Specifically, given sufficient background uncertainty (...) about the choiceworthiness of one's options, many expectation-maximizing gambles that do not stochastically dominate their alternatives "in a vacuum" become stochastically dominant in virtue of that background uncertainty. But, even under these conditions, stochastic dominance will not require agents to accept options whose expectational superiority depends on sufficiently small probabilities of extreme payoffs. The sort of background uncertainty on which these results depend looks unavoidable for any agent who measures the choiceworthiness of her options in part by the total amount of value in the resulting world. At least for such agents, then, stochastic dominance offers a plausible general principle of choice under uncertainty that can explain more of the apparent rational constraints on such choices than has previously been recognized. (shrink)

In the growing literature on decision-making under moral uncertainty, a number of skeptics have argued that there is an insuperable barrier to rational "hedging" for the risk of moral error, namely the apparent incomparability of moral reasons given by rival theories like Kantianism and utilitarianism. Various general theories of intertheoretic value comparison have been proposed to meet this objection, but each suffers from apparently fatal flaws. In this paper, I propose a more modest approach that aims to identify classes of (...) moral theories that share common principles strong enough to establish bases for intertheoretic comparison. I show that, contra the claims of skeptics, there are often rationally perspicuous grounds for precise, quantitative value comparisons within such classes. In light of this fact, I argue, the existence of some apparent incomparabilities between widely divergent moral theories cannot serve as a general argument against hedging for one's moral uncertainties. (shrink)

Defenders of deontological constraints in normative ethics face a challenge: how should an agent decide what to do when she is uncertain whether some course of action would violate a constraint? The most common response to this challenge has been to defend a threshold principle on which it is subjectively permissible to act iff the agent's credence that her action would be constraint-violating is below some threshold t. But the threshold approach seems arbitrary and unmotivated: what would possibly determine where (...) the threshold should be set, and why should there be any precise threshold at all? Threshold views also seem to violate ought agglomeration, since a pair of actions each of which is below the threshold for acceptable moral risk can, in combination, exceed that threshold. In this paper, I argue that stochastic dominance reasoning can vindicate and lend rigor to the threshold approach: given characteristically deontological assumptions about the moral value of acts, it turns out that morally safe options will stochastically dominate morally risky alternatives when and only when the likelihood that the risky option violates a moral constraint is greater than some precisely definable threshold (in the simplest case, .5). I also show how, in combination with the observation that deontological moral evaluation is relativized to particular choice situations, this approach can overcome the agglomeration problem. This allows the deontologist to give a precise and well-motivated response to the problem of uncertainty. (shrink)

How should an agent decide what to do when she is uncertain not just about morally relevant empirical matters, like the consequences of some course of action, but about the basic principles of morality itself? This question has only recently been taken up in a systematic way by philosophers. Advocates of moral hedging claim that an agent should weigh the reasons put forward by each moral theory in which she has positive credence, considering both the likelihood that that theory is (...) true and the strength of the reasons it posits. The view that it is sometimes rational to hedge for one's moral uncertainties, however, has recently come under attack both from those who believe that an agent should always be guided by the dictates of the single moral theory she deems most probable and from those who believe that an agent's moral beliefs are simply irrelevant to what she ought to do. Among the many objections to hedging that have been pressed in the recent literature is the worry that there is no non-arbitrary way of making the intertheoretic comparisons of moral value necessary to aggregate the value assignments of rival moral theories into a single ranking of an agent's options. -/- This dissertation has two principal objectives: First, I argue that, contra these recent objections, an agent's moral beliefs and uncertainties are relevant to what she rationally ought to do, and more particularly, that agents are at least sometimes rationally required to hedge for their moral uncertainties. My principal argument for these claims appeals to the enkratic conception of rationality, according to which the requirements of practical rationality derive from an agent's beliefs about the objective, desire-independent value or choiceworthiness of her options. Second, I outline a new general theory of rational choice under moral uncertainty. Central to this theory is the idea of content-based aggregation, that the principles according to which an agent should compare and aggregate rival moral theories are grounded in the content of those theories themselves, including not only their value assignments but also the metaethical and other non-surface-level propositions that underlie, justify, or explain those value assignments. (shrink)

Given the deep disagreement surrounding population axiology, one should remain uncertain about which theory is best. However, this uncertainty need not leave one neutral about which acts are better or worse. We show that as the number of lives at stake grows, the Expected Moral Value approach to axiological uncertainty systematically pushes one towards choosing the option preferred by the Total and Critical Level views, even if one’s credence in those theories is low.

What should we do when we are not certain about what we morally should do? There is a long history of theorizing about decision-making under empirical uncertainty, but surprisingly little has been written about the moral uncertainty expressed by this question. Only very recently have philosophers started to systematically address the nature of such uncertainty and its impacts on decision-making. This paper addresses the main problems raised by moral uncertainty and critically examines some proposed solutions.

Sometimes it’s not certain which of several mutually exclusive moral views is correct. Like almost everyone, I think that there’s some sense in which what one should do depends on which of these theories is correct, plus the way the world is non-morally. But I also think there’s an important sense in which what one should do depends upon the probabilities of each of these views being correct. Call this second claim “moral uncertaintism”. In this paper, I want to address (...) an argument against moral uncertaintism offered in the pages of this journal by Brian Weatherson, and seconded elsewhere by Brian Hedden, the crucial premises of which are: that acting on moral uncertaintist norms necessarily involves motivation by reasons or rightness as such, and that such motivation is bad. I will argue that and are false, and that at any rate, the quality of an agent’s motivation is not pertinent to the truth or falsity of moral uncertaintism in the way that Weatherson’s and Hedden’s arguments require. (shrink)

New AI technologies have the potential to cause unintended harms in diverse domains including warfare, judicial sentencing, biomedicine and governance. One strategy for realising the benefits of AI whilst avoiding its potential dangers is to ensure that new AIs are properly ‘aligned’ with some form of ‘alignment target.’ One danger of this strategy is that – dependent on the alignment target chosen – our AIs might optimise for objectives that reflect the values only of a certain subset of society, and (...) that do not take into account alternative views about what constitutes desirable and safe behaviour for AI agents. In response to this problem, several AI ethicists have suggested alignment targets that are designed to be sensitive to widespread normative disagreement amongst the relevant stakeholders. Authors inspired by voting theory have suggested that AIs should be aligned with the verdicts of actual or simulated ‘moral parliaments’ whose members represent the normative views of the relevant stakeholders. Other authors inspired by decision theory and the philosophical literature on moral uncertainty have suggested that AIs should maximise socially expected choiceworthiness. In this paper, I argue that both of these proposals face several important problems. In particular, they fail to select attractive ‘compromise options’ in cases where such options are available. I go on to propose and defend an alternative, bargaining-theoretic alignment target, which avoids the problems associated with the voting- and decision-theoretic approaches. (shrink)

How should you decide what to do when you’re uncertain about basic normative principles? A natural suggestion is to follow some "second-order:" norm: e.g., obey the most probable norm or maximize expected choiceworthiness. But what if you’re uncertain about second-order norms too—must you then invoke some third-order norm? If so, any norm-guided response to normative uncertainty appears doomed to a vicious regress. This paper aims to rescue second-order norms from the threat of regress. I first elaborate and defend the claim (...) some philosophers have made that the regress problem forces us to accept normative externalism, the view that at least one norm is incumbent on all agents regardless of their normative beliefs. But, I then argue, we need not accept externalism about first-order norms, thus closing off any question of how agents should respond to normative uncertainty. Rather, we can head off the threat of regress by ascribing external force to a single second-order norm: the enkratic principle. (shrink)

Moral latitude is only ever a matter of coincidence on the most popular decision procedure in the literature on moral uncertainty. In all possible choice situations other than those in which two or more options happen to be tied for maximal expected choiceworthiness, Maximize Expected Choiceworthiness implies that only one possible option is uniquely appropriate. A better theory of appropriateness would be more sensitive to the decision maker’s credence in theories that endorse agent-centred prerogatives. In this paper, we will develop (...) and defend a modified version of this view that allows for agent-centred prerogatives and supererogation as part of its verdicts about appropriateness. (shrink)

This paper presents a unifying diagnosis of a number of important problems facing existing models of rational choice under moral uncertainty and proposes a remedy. I argue that the problems of (i) severely limited scope, (ii) intertheoretic comparisons, and (iii) 'swamping’ all stem from the way in which values are assigned to options in decision rules such as Maximisation of Expected Choiceworthiness. By assigning values to options under a given moral theory by asking something like ‘how much do I desire (...) this option, supposing this theory is true?’ rather than ‘how much value does this theory assign to this option?’ these problems can be avoided, while the appealing features of these accounts can be preserved. This amendment provides a role for the preferences, desires, or goals of rational agents that is curiously absent from the existing discussion of what individuals rationally ought to do when they are uncertain about what they morally ought to do. (shrink)

Nick Bostrom and others have suggested treating decision-making under moral uncertainty as analogous to parliamentary decision-making. The core suggestion of this “parliamentary approach” is that the competing moral theories function like delegates to the parliament, and that these delegates then make decisions by some combination of bargaining and voting. There seems some reason to hope that such an approach might avoid standard objections to existing approaches (for example, the “maximise expected choiceworthiness” (MEC) and “my favourite theory” approaches). However, the parliamentary (...) approach is so far extremely underspecified, making it largely indeterminate how such a model will in fact behave in the respects that those concerned with moral uncertainty care about. This paper explores one way of making it precise. We treat predicaments of moral uncertainty as analogous to bargaining situations alone (setting aside voting), and apply a version of the Nash solution that is standard in bargaining theory. The resulting model does indeed perform in many of the hoped-for ways. However, so also does a version of MEC that employs a structural approach to intertheoretic comparisons. It seems to us an open question which, regarding this version of MEC and the bargaining-theoretic approach, is superior to the other. We identify the key points on which the two differ. (shrink)

This thesis is about making decisions when we are uncertain about what will happen, how valuable it will be, and even how to make decisions. Even the most sure-footed amongst us are sometimes uncertain about all three, but surprisingly little attention has been given to the latter two. The three essays that constitute my thesis hope to do a small part in rectifying this problem. The first essay is about the value of finding out how to make decisions. Society spends (...) considerable resources funding people (like me) to research decision-making, so it is natural to wonder whether society is getting a good deal. This question is so shockingly underresearched that bedrock facts are readily discoverable, such as when this kind of information is valuable. My second essay concerns whether we can compare value when we are uncertain about value. Many people are in fact uncertain about value, and how we deal with this uncertainty hinges on these comparisons. I argue that value comparisons are only sometimes possible; I call this weak comparability. This essay is largely a synthesis of the literature, but I also present an argument which begins with a peculiar view of the self: it is as if each of us is a crowd of different people separated by time (but connected by continuity of experience). I’m not the first to endorse this peculiar view of the self, but I am the first to show how it supports the benign view that value is sometimes comparable. We may be uncertain of any decision rules, even those that would tell us how to act when we face uncertainty in decision rules. We may be uncertain of how to decide how to decide how to... And so on. If so, we might have to accept infinitely many decision rules just to make any mundane decision, such as whether to pick up a five-cent piece from the gutter. My third essay addresses this problem of regress. I think all of our decisions are forced: we must decide now or continue to deliberate. Surprisingly, this allows us to avoid the original problem. I call this solution “when forced, do your best”. (shrink)

As well as disagreeing about how much one should donate to charity, moral theories also disagree about where one should donate. In light of this disagreement, how should the morally uncertain philanthropist allocate her donations? In many cases, one intuitively attractive option is for the philanthropist to split her donations across all of the charities that are recommended by moral views in which she has positive credence, with each charity’s share being proportional to her credence in the moral theories that (...) recommend that particular charity. Despite the fact that something like this approach is already being used by real-world philanthropists to distribute billions of dollars of donations, it is not supported by any of the approaches to moral uncertainty that have been proposed thus-far in the philosophical literature. In this paper, we will develop a bargaining-based approach to moral uncertainty that honors the proportionality intuition in favor of splitting one’s donations. We also show how this bargaining-based approach has several further advantages over the best alternative proposals. (shrink)

We begin by showing that every theory of the value of uncertain prospects must have one of three unpalatable properties. _Reckless_ theories recommend giving up a sure thing, no matter how good, for an arbitrarily tiny chance of enormous gain; _timid_ theories permit passing up an arbitrarily large potential gain to prevent a tiny increase in risk; _non-transitive_ theories deny the principle that, if A is better than B and B is better than C, then A must be better than (...) C. Having set up this trilemma, we study its horns. Non-transitivity has been much discussed; we focus on drawing out the costs and benefits of recklessness and timidity when it comes to axiology, decision theory, and normative uncertainty. (shrink)

A straightforward way to handle moral uncertainty is simply to follow the moral theory in which you have most credence. This approach is known as My Favourite Theory. In this paper, I argue that, in some cases, My Favourite Theory prescribes choices that are, sequentially, worse in expected moral value than the opposite choices according to each moral theory you have any credence in. In addition this, problem generalizes to other approaches that avoid intertheoretic comparisons of value, such as My (...) Favourite Option, the Borda Rule, Variance Normalization, and the Principle of Maximizing Expected Normalized Moral Value. (shrink)

How ought you to evaluate your options if you're uncertain about what's fundamentally valuable? A prominent response is Expected Value Maximisation (EVM)—the view that under axiological uncertainty, an option is better than another if and only if it has the greater expected value across axiologies. But the expected value of an option depends on quantitative probability and value facts, and in particular on value comparisons across axiologies. We need to explain what it is for such facts to hold. Also, EVM (...) is by no means self-evident. We need an argument to defend that it’s true. This book introduces an axiomatic approach to answer these worries. It provides an explication of what EVM means by use of representation theorems: intertheoretic comparisons can be understood in terms of facts about which options are better than which, and mutatis mutandis for intratheoretic comparisons and axiological probabilities. And it provides a systematic argument to the effect that EVM is true: the theory can be vindicated through simple axioms. The result is a formally cogent and philosophically compelling extension of standard decision theory, and original take on the problem of axiological or normative uncertainty. (shrink)

In many situations, people are unsure in their moral judgements. In much recent philosophical literature, this kind of moral doubt has been analysed in terms of uncertainty in one’s moral beliefs. Non-cognitivists, however, argue that moral judgements express a kind of conative attitude, more akin to a desire than a belief. This paper presents a scientifically informed reconciliation of non-cognitivism and moral doubt. The central claim is that attitudinal ambivalence—the degree to which one holds conflicting attitudes towards the same object—can (...) play the role of moral doubt for non-cognitivists. I will demonstrate that ambivalence has all of the features that we would expect it to have in order to play the role of moral doubt. It is gradable, can vary through time, covaries with strength of motivation, and is suitably distinct from the other features of our moral judgements. As well as providing a defence of non-cognitivism, this insight poses a new challenge for the view: deciding how to act under moral ambivalence. (shrink)

How ought you to evaluate your options if you’re uncertain about which axiology is true? One prominent response is Expected Moral Value Maximisation, the view that under axiological uncertainty, an option is better than another if and only if it has the greater expected moral value across axiologies. EMVM raises two fundamental questions. First, there’s a question about what it should even mean. In particular, it presupposes that we can compare moral value across axiologies. So to even understand EMVM, we (...) need to explain what it is for such comparisons to hold. Second, assuming that we understand it, there’s a question about whether EMVM is true. Since there are many plausible rivals, we need an argument to defend it. In this paper, I’ll introduce a representation theorem for axiological uncertainty to answer these two questions. Roughly, the theorem shows that if all our axiologies satisfy the von Neumann–Morgenstern axioms, and if the facts about which options are better than which in light of your uncertainty also satisfy these axioms as well as a Pareto condition, then these facts have a relevantly unique expected utility representation. If I’m right, this theorem at once affords us a compelling way to understand EMVM—and specifically intertheoretic comparisons—and a systematic argument for its truth. (shrink)

How should an agent act under normative uncertainty? We might extend the orthodox theory of rational choice to the case of uncertainty between competing normative theories. But this requires that the values assigned by different normative theories be comparable. This paper defends a strategy for avoiding the need for intertheoretic value comparisons: instead of comparing competing moral theories, I argue that values can be represented in terms of a de dicto specification of value. I provide a decision theory for de (...) dicto values that generalises expected utility theory, and I compare the proposal with alternative strategies for avoiding the problem of intertheoretic comparisons. (shrink)

In slogan form, the thesis of this dissertation is that shaping the far future is overwhelmingly important. More precisely, I argue that: Main Thesis: From a global perspective, what matters most is that we do what is best for the general trajectory along which our descendants develop over the coming millions, billions, and trillions of years. The first chapter introduces some key concepts, clarifies the main thesis, and outlines what follows in later chapters. Some of the key concepts include: existential (...) risk, the world's development trajectory, proximate benefits and ripple effects, speeding up development, trajectory changes, and the distinction between broad and targeted attempts to shape the far future. The second chapter is a defense of some methodological assumptions for developing normative theories which makes my thesis more plausible. In the third chapter, I introduce and begin to defend some key empirical and normative assumptions which, if true, strongly support my main thesis. In the fourth and fifth chapters, I argue against two of the strongest objections to my arguments. These objections come from population ethics, and are based on Person-Affecting Views and views according to which additional lives have diminishing marginal value. I argue that these views face extreme difficulties and cannot plausibly be used to rebut my arguments. In the sixth and seventh chapters, I discuss a decision-theoretic paradox which is relevant to my arguments. The simplest plausible theoretical assumptions which support my main thesis imply a view I call fanaticism, according to which any non-zero probability of an infinitely good outcome, no matter how small, is better than any probability of a finitely good outcome. I argue that denying fanaticism is inconsistent with other normative principles that seem very obvious, so that we are faced with a paradox. I have no solution to the paradox; I instead argue that we should continue to use our inconsistent principles, but we should use them tastefully. We should do this because, currently, we know of no consistent set of principles which does better. (shrink)