Pascal’s Wager involves expected utilities. In this chapter, we examine the Wager in light of two main features of expected utility theory: utilities and probabilities. We discuss infinite and finite utilities, and zero, infinitesimal, extremely low, imprecise, and undefined probabilities. These have all come up in recent literature regarding Pascal’s Wager. We consider the problems each creates and suggest prospects for the Wager in light of these problems.

Orthodox decision theory is fanatical in the way it treats small probabilities of enormous value, if unbounded utility functions are allowed. Some have suggested a fix, Nicolausian discounting, according to which outcomes with small enough probabilities should be ignored when making decisions. However, there are lotteries involving only small-probability outcomes, none of which should intuitively be ignored. So the Nicolausian discounter needs a procedure for distinguishing the problematic cases of small-probability outcomes from the unproblematic ones. In this paper, I present (...) a dilemma for Nicolausian discounting: the view must be coupled either with a procedure that delivers fanatical verdicts of its own, as bad as those of orthodox decision theory, or with one that entails intransitive cyclic preferences. (shrink)

In slogan form, the thesis of this dissertation is that shaping the far future is overwhelmingly important. More precisely, I argue that: Main Thesis: From a global perspective, what matters most is that we do what is best for the general trajectory along which our descendants develop over the coming millions, billions, and trillions of years. The first chapter introduces some key concepts, clarifies the main thesis, and outlines what follows in later chapters. Some of the key concepts include: existential (...) risk, the world's development trajectory, proximate benefits and ripple effects, speeding up development, trajectory changes, and the distinction between broad and targeted attempts to shape the far future. The second chapter is a defense of some methodological assumptions for developing normative theories which makes my thesis more plausible. In the third chapter, I introduce and begin to defend some key empirical and normative assumptions which, if true, strongly support my main thesis. In the fourth and fifth chapters, I argue against two of the strongest objections to my arguments. These objections come from population ethics, and are based on Person-Affecting Views and views according to which additional lives have diminishing marginal value. I argue that these views face extreme difficulties and cannot plausibly be used to rebut my arguments. In the sixth and seventh chapters, I discuss a decision-theoretic paradox which is relevant to my arguments. The simplest plausible theoretical assumptions which support my main thesis imply a view I call fanaticism, according to which any non-zero probability of an infinitely good outcome, no matter how small, is better than any probability of a finitely good outcome. I argue that denying fanaticism is inconsistent with other normative principles that seem very obvious, so that we are faced with a paradox. I have no solution to the paradox; I instead argue that we should continue to use our inconsistent principles, but we should use them tastefully. We should do this because, currently, we know of no consistent set of principles which does better. (shrink)

1. Here is a very simple game. You come up with a number and I come up with a number. If I come up with the higher number, I win; otherwise you win. You go first. Call this ‘The Very Simple Game’. Few would play it if they had to go first and many if they are guaranteed to go second.2. Here is another one. You come up with a number n and I come up with a number m. If (...) m times 1/ n > 1, then I win; if not, then you win. You go first. Call this ‘Still The Very Simple Game’. Since I win just in case m is greater than n, this game collapses into The Very Simple Game. Few people will play it if they have to go first.3. Here is a third one. Let us call it ‘Looks Like A Tricky Game’. Suppose you have a choice between exactly two actions: keep holding on to some thing t ; hand t over to me and accept my offer to give you, with some probability p , some thing with utility m .Suppose, you are a maximizer of expected utility with …. (shrink)

Prevailing opinion—defended by Jason Brennan and others—is that voting to change the outcome is irrational, since although the payoffs of tipping an election can be quite large, the probability of doing so is extraordinarily small. This paper argues that prevailing opinion is incorrect. Voting is shown to be rational so long as two conditions are satisfied: First, the average social benefit of electing the better candidate must be at least twice as great as the individual cost of voting, and second, (...) the chance of casting the decisive vote must be at least 1/N, where N stands for the number of citizens. It is argued that both of these conditions are often true in the real world. (shrink)

In a well-known paper, Nick Bostrom presents a confrontation between a fictionalised Blaise Pascal and a mysterious mugger. The mugger persuades Pascal to hand over his wallet by exploiting Pascal's commitment to expected utility maximisation. He does so by offering Pascal an astronomically high reward such that, despite Pascal's low credence in the mugger's truthfulness, the expected utility of accepting the mugging is higher than rejecting it. In this article, I present another sort of high value, low credence mugging. This (...) time, the mugger utilises research on existential risk and the long-term potential of humanity to exploit Pascal's expected-utility-maximising descendant. This mugging is more insidious than Bostrom's original as it relies on plausible facts about the long-term future, as well as realistic credences about how our everyday actions could, albeit with infinitesimally low likelihood, affect the future of humanity. (shrink)

The principle that rational agents should maximize expected utility or choiceworthiness is intuitively plausible in many ordinary cases of decision-making under uncertainty. But it is less plausible in cases of extreme, low-probability risk (like Pascal's Mugging), and intolerably paradoxical in cases like the St. Petersburg and Pasadena games. In this paper I show that, under certain conditions, stochastic dominance reasoning can capture most of the plausible implications of expectational reasoning while avoiding most of its pitfalls. Specifically, given sufficient background uncertainty (...) about the choiceworthiness of one's options, many expectation-maximizing gambles that do not stochastically dominate their alternatives "in a vacuum" become stochastically dominant in virtue of that background uncertainty. But, even under these conditions, stochastic dominance will not require agents to accept options whose expectational superiority depends on sufficiently small probabilities of extreme payoffs. The sort of background uncertainty on which these results depend looks unavoidable for any agent who measures the choiceworthiness of her options in part by the total amount of value in the resulting world. At least for such agents, then, stochastic dominance offers a plausible general principle of choice under uncertainty that can explain more of the apparent rational constraints on such choices than has previously been recognized. (shrink)

In response to cases that involve tiny probabilities of huge payoffs, some argue that we ought to discount small probabilities down to zero. However, this paper shows that doing so violates Independence and Continuity, and as a result of these violations, those who discount small probabilities can be exploited by money pumps. Various possible ways of avoiding exploitation will be discussed. This paper concludes that the money pump for Independence undermines the plausibility of discounting small probabilities. Much of the discussion (...) on Independence generalizes to other views that also violate Independence. (shrink)

This paper considers questions that small animals and AI systems raise for utilitarianism. Specifically, if these beings have more welfare than humans and other large animals, then utilitarianism implies that we should prioritize them, all else equal. This could lead to a ‘rebugnant conclusion’, according to which we should, say, create large populations of small animals rather than small populations of large animals. It could also lead to a ‘Pascal’s bugging’, according to which we should, say, prioritize large populations of (...) small animals even if they have a low chance of being sentient. I suggest that utilitarians should accept these implications in theory but might be able to avoid some of them in practice. (shrink)

Many philosophers assume that, when making moral decisions under uncertainty, we should choose the option that has the greatest expected moral value, regardless of how risky it is. But their arguments for maximizing expected moral value do not support it over rival, risk-averse approaches. In this paper, I present a novel argument for maximizing expected value: when we think about larger series of decisions that each decision is a part of, all but the most risk-averse agents would prefer that we (...) consistently choose the option with the highest expected value. To the extent that what we choose on a given occasion should be guided by the entire series of choices we prefer, then on each occasion, we should choose the option with the highest expected moral value. (shrink)

A classification of the global catastrophic risks of AI is presented, along with a comprehensive list of previously identified risks. This classification allows the identification of several new risks. We show that at each level of AI’s intelligence power, separate types of possible catastrophes dominate. Our classification demonstrates that the field of AI risks is diverse, and includes many scenarios beyond the commonly discussed cases of a paperclip maximizer or robot-caused unemployment. Global catastrophic failure could happen at various levels of (...) AI development, namely, before it starts self-improvement, during its takeoff, when it uses various instruments to escape its initial confinement, or after it successfully takes over the world and starts to implement its goal system, which could be plainly unaligned, or feature-flawed friendliness. AI could also halt at later stages of its development either due to technical glitches or ontological problems. Overall, we identified around several dozen scenarios of AI-driven global catastrophe. The extent of this list illustrates that there is no one simple solution to the problem of AI safety, and that AI safety theory is complex and must be customized for each AI development level. (shrink)

When we are faced with a choice among acts, but are uncertain about the true state of the world, we may be uncertain about the acts’ “choiceworthiness”. Decision theories guide our choice by making normative claims about how we should respond to this uncertainty. If we are unsure which decision theory is correct, however, we may remain unsure of what we ought to do. Given this decision-theoretic uncertainty, meta-theories attempt to resolve the conflicts between our decision theories...but we may be (...) unsure which meta-theory is correct as well. This reasoning can launch a regress of ever-higher-order uncertainty, which may leave one forever uncertain about what one ought to do. There is, fortunately, a class of circumstances under which this regress is not a problem. If one holds a cardinal understanding of subjective choiceworthiness, and accepts certain other criteria, one’s hierarchy of metanormative uncertainty ultimately converges to precise definitions of “subjective choiceworthiness” for any finite set of acts. If one allows the metanormative regress to extend to the transfinite ordinals, the convergence criteria can be weakened further. Finally, the structure of these results applies straightforwardly not just to decision-theoretic uncertainty, but also to other varieties of normative uncertainty, such as moral uncertainty. (shrink)

ABSTRACT Average utilitarianism and several related axiologies, when paired with the standard expectational theory of decision-making under risk and with reasonable empirical credences, can find their practical prescriptions overwhelmingly determined by the minuscule probability that the agent assigns to solipsism—that is, to the hypothesis that there is only one welfare subject in the world, namely, herself. This either (i) constitutes a reductio of these axiologies, (ii) suggests that they require bespoke decision theories, or (iii) furnishes an unexpected argument for ethical (...) egoism. (shrink)

A de minimis risk is defined as a risk that is so small that it may be legitimately ignored when making a decision. While ignoring small risks is common in our day-to-day decision making, attempts to introduce the notion of a de minimis risk into the framework of decision theory have run up against a series of well-known difficulties. In this paper, I will develop an enriched decision theoretic framework that is capable of overcoming two major obstacles to the modelling (...) of de minimis risk. The key move is to introduce, into decision theory, a non-probabilistic conception of risk known as normic risk. (shrink)

A 1% skeptic is someone who has about a 99% credence in non-skeptical realism and about a 1% credence in the disjunction of all radically skeptical scenarios combined. The first half of this essay defends the epistemic rationality of 1% skepticism, appealing to dream skepticism, simulation skepticism, cosmological skepticism, and wildcard skepticism. The second half of the essay explores the practical behavioral consequences of 1% skepticism, arguing that 1% skepticism need not be behaviorally inert.

People with the kind of preferences that give rise to the St. Petersburg paradox are problematic---but not because there is anything wrong with infinite utilities. Rather, such people cannot assign the St. Petersburg gamble any value that any kind of outcome could possibly have. Their preferences also violate an infinitary generalization of Savage's Sure Thing Principle, which we call the *Countable Sure Thing Principle*, as well as an infinitary generalization of von Neumann and Morgenstern's Independence axiom, which we call *Countable (...) Independence*. In violating these principles, they display foibles like those of people who deviate from standard expected utility theory in more mundane cases: they choose dominated strategies, pay to avoid information, and reject expert advice. We precisely characterize the preference relations that satisfy Countable Independence in several equivalent ways: a structural constraint on preferences, a representation theorem, and the principle we began with, that every prospect has a value that some outcome could have. (shrink)

In his Ted Sider takes care to define the notion of a temporal part and his doctrine of perdurantism using only the temporally indexed notion of parthood – ‘ x is part of y at t’ – rather than the atemporal notion of classical mereology – ‘ x is a part of y’ – in order to forestall accusations of unintelligibility from his opponents. However, as he notes, endurantists do not necessarily reject the classical mereological notion as unintelligible. They allow (...) that it makes sense and applies to atemporal subject matters and to temporal subject matters when the entities under discussion are not continuants. Thus, they allow that it makes sense to say that metaphysics is a part of philosophy, or that football is a game of two halves. What endurantists deny is only that the classical mereological notion is applicable to continuants: continuants , they say, have no proper parts simpliciter , either because it is false to say that they have or because it is unintelligible.Thus perdurantists do not have to embrace Sider's excessive caution in defining their position. 1 They can safely allow themselves classical mereological notions as long as it is a consequence of their definitions that continuants are perdurers/have temporal proper parts only if they have atemporal proper parts. 2In his Josh Parsons illuminatingly takes on the task he describes as ‘get[tting] the allegedly technical concepts of temporal part, perdurance and so on by ratcheting up from mereological relations, subregion relations among times and the concept of exact temporal location ’. He continues, ‘My definitions provide a good answer to those endurantists who claim …. (shrink)

An artificial general intelligence (AGI) might have an instrumental drive to modify its utility function to improve its ability to cooperate, bargain, promise, threaten, and resist and engage in blackmail. Such an AGI would necessarily have a utility function that was at least partially observable and that was influenced by how other agents chose to interact with it. This instrumental drive would conflict with the strong orthogonality thesis since the modifications would be influenced by the AGI’s intelligence. AGIs in highly (...) competitive environments might converge to having nearly the same utility function, one optimized to favorably influencing other agents through game theory. Nothing in our analysis weakens arguments concerning the risks of AGI. (shrink)

This paper is the introduction to a collection I guest-edited called Positive Duties to Wild Animals. The collection contains single-authored contributions from Catia Faria, Josh Milburn, Eze Paez, and Jeff Sebo; and co-authored contributions from Mara-Daria Cojocaru and Alasdair Cochrane, and Oscar Horta and Dayrón Terán. It was published as a special issue of Ethics, Policy and Environment.

This paper explores the consequences of applying two natural ideas from epistemology to decision theory: (1) that knowledge should guide our actions, and (2) that we know a lot of non-trivial things. In particular, we explore the consequences of these ideas as they are applied to standard decision theoretic puzzles such as the St. Petersburg Paradox. In doing so, we develop a “knowledge-first” decision theory and we will see how it can help us avoid fanaticism with regard to the St. (...) Petersburg puzzle and related puzzles. The result will be a decision theory that gives a novel, but well-motivated, reason for discounting small probabilities when making decisions. We examine the merits and demerits of such a decision theory. (shrink)

In this article, we re-examine Pascal’s Mugging, and argue that it is a deeper problem than the St. Petersburg paradox. We offer a way out that is consistent with classical decision theory. Specifically, we propose a “many muggers” response analogous to the “many gods” objection to Pascal’s Wager. When a very tiny probability of a great reward becomes a salient outcome of a choice, such as in the offer of the mugger, it can be discounted on the condition that there (...) are other symmetric, non-salient rewards that one may receive if one chooses otherwise. (shrink)

This thesis is about the moral reasons we have to do less than best. It consists of six chapters. Part I of the thesis proposes, extends, and defends reasons to do less than best. In Chapter One (“The Conditional Obligation”) I outline and reject two recent arguments from Joe Horton and Theron Pummer for the claim that we have a conditional obligation to bring about the most good. In Chapter Two (“Agglomeration and Agent-Relative Costs”) I argue that agent-relative costs can (...) justify permissions to less than best in a far greater range of decisions contexts than we currently believe. In Chapter Three (“Risk and Aggregation”) I discuss how anti-aggregative moral theories deal with risk, whilst proposing an argument in favour of ex-post moral theories. -/- In Part II of the thesis, I discuss longtermism, presenting a series of reasons to believe we have both significant permissions and obligations to prioritise those in the short-term. In Chapter Four (“Listening to Future People”) I criticise a recent argument presented by Andreas Mogensen for the claim we cannot have obligations grounded in the wellbeing of future people. I then show that, even permitting the existence of obligations to future people, we still have significant reasons to resist longtermism on the grounds of partiality. In Chapter Five (“The Complaints of Future People”) I argue that scepticism about aggregation is inconsistent with an obligation, or indeed even a permission, to help far future people. In Chapter Six (“Leaving the Present Behind”) I explore the moral significance of “dooming”, whilst arguing that we have a pro tanto reason of fairness to prioritise aiding those currently in existence. (shrink)

Longtermists claim that what we ought to do is mainly determined by how our actions might affect the very long-run future. A natural objection to longtermism is that these effects may be nearly impossible to predict -- perhaps so close to impossible that, despite the astronomical importance of the far future, the expected value of our present actions is mainly determined by near-term considerations. This paper aims to precisify and evaluate one version of this epistemic objection to longtermism. To that (...) end, I develop two simple models for comparing "longtermist" and "neartermist" interventions, incorporating the idea that it is harder to make a predictable difference to the further future. These models yield mixed conclusions: if we simply aim to maximize expected value, and don't mind premising our choices on minuscule probabilities of astronomical payoffs, the case for longtermism looks robust. But on some prima facie plausible empirical worldviews, the expectational superiority of longtermist interventions depends heavily on these "Pascalian" probabilities. So the case for longtermism may depend either on plausible but non-obvious empirical claims or on a tolerance for Pascalian fanaticism. (shrink)

Advocates of longtermism point out that interventions which focus on improving the prospects of people in the very far future will, in expectation, bring about a significant amount of good. Indeed, in expectation, such long-term interventions bring about far more good than their short-term counterparts. As such, longtermists claim we have compelling moral reason to prefer long-term interventions. In this paper, I show that longtermism is in conflict with plausible deontic scepticism about aggregation. I do so by demonstrating that, from (...) both an ex-ante and ex-post perspective, longtermist interventions – and, in particular, those which aim to mitigate catastrophic risk – typically generate extremely weak claims of assistance from future people. (shrink)

A number of philosophers have argued that if you care about how much goodness your actions generate, or how good the state-of-affairs you actions bring about are, then your attention should be directed towards the very far future. But many don’t care about how much goodness their actions generate, nor do they care about things like “states-of-affairs”. Amongst a multitude of things, many people care about how their actions impact individuals. And they also care about the sorts of justifications they (...) are able to offer for those actions. One moral theory which captures this moral viewpoint is contractualism. In this chapter, I argue that contractualism is, in large part, incompatible with longtermism – the view that, in many contexts, we ought to do that which will most improve the prospects of future people. I then discuss what advice, if any at all, contractualism can offer to the longtermist. (shrink)

Effective altruism is a movement which aims to maximise good. Effective altruists are concerned with extreme poverty and many of them think that individuals have an obligation to donate to effective charities to alleviate extreme poverty. Their reasoning, which I will scrutinise, is as follows: -/- Premise 1. Extreme poverty is very bad. -/- Premise 2. If it is in our power to prevent something very bad from happening, without thereby sacrificing anything else morally significant, we ought, morally, to do (...) it. -/- Premise 3. Individuals ought to choose the effective option in preventing very bad things. -/- Premise 4. Donating to effective charities is one of the best ways to alleviate extreme poverty. -/- Conclusion. Individuals ought to donate to effective charities working towards extreme poverty alleviation where doing so does not require them to give up anything of moral significance. -/- I will scrutinise each of these premises in turn. -/- For Premise 1, I focus on hedonistic utilitarianism and criticise its outlook on extreme poverty. I claim that hedonistic utilitarianism might be problematic for effective altruism. -/- Premise 2 is Peter Singer's Weaker Principle of Sacrifice. I introduce several possible interpretations of it, and press several objections to it by stressing overpermissiveness, luck, and rights. I defend strengthening the Weaker Principle of Sacrifice without making it overdemanding. -/- I claim that Premise 3 can be attractive to both consequentialists and non-consequentialists. Nevertheless, by showing that effectiveness sometimes violates fairness, I propose a method which avoids always helping the greater number and always giving everyone equal chances of being helped, which is compatible with effective altruism. -/- Against Premise 4, I assess the systemic change objection, which states that effective altruism unjustifiably distracts individuals from systemic change. By considering risk and the moral standing of the future extremely poor, I claim that the systemic change objection is partially successful, but cannot undermine effective altruism. -/- After analysing all of these, I argue that individuals have an obligation to donate to effective charities to alleviate extreme poverty where doing so does not require them to give up anything of moral significance. (shrink)

Decision theory aims to provide mathematical analysis of which choice one should rationally make in a given situation. Our current decision theory norms have been very successful, however, several problems have proven vexing for standard decision theory. In this paper, I show that these problems all share a similar structure and identify a class of problems which decision theory overvalues. I demonstrate that agents who follow current standard decision theory can be exploited and have their preferences reordered if offered decision (...) problems of this class. I show that preference reordering is a serious problem, which motivates my search for a decision theory which is immune to exploitation. I find Dr. Nick Smith’s theory of Rationally Negligible Probabilities cannot be exploited in this way and discuss why agents should adopt it. (shrink)

Given the deep disagreement surrounding population axiology, one should remain uncertain about which theory is best. However, this uncertainty need not leave one neutral about which acts are better or worse. We show that as the number of lives at stake grows, the Expected Moral Value approach to axiological uncertainty systematically pushes one towards choosing the option preferred by the Total and Critical Level views, even if one’s credence in those theories is low.