“Perhaps the most common objection to consequentialism is this: it is impossible to know the future…This means that you will never be absolutely certain as to what all the consequences of your act will be…there may be long term bad effects from your act, side effects that were unforeseen and indeed unforeseeable…So how can we tell which act will lead to the best results overall – counting all the results? This seems to mean that consequentialism will be unusable as a (...) moral guide to action. All the evidence available at the time of acting may have pointed to the conclusion that a given act was the right act to perform – and yet it may still turn out that what you did had horrible results, and so in fact was morally wrong. Indeed, if will never be possible to say for sure that any given act was right or wrong, since any event can continue to have further unseen effects down through history. Yet if it is impossible to tell whether any act is morally right or wrong, how can consequentialism possibly be a correct moral theory?”. (shrink)

Many have the intuition that human persons are both extremely and equally valuable. This seeming extremity and equality of vale is puzzling: if overall value is the sum of one’s final value and instrumental value, how could it be that persons share the same extreme value? One way that we can solve the Value Puzzle is by following Andrew Bailey and Josh Rasmussen. Philosophy and Phenomenological Research, 103, 264–277 (2020) and accepting that persons have infinite final value. But there are (...) some significant downsides to their way of thinking about values, which relies on the extended real numbers. We offer a different approach: if we model values using the hyperreal numbers, we can capture many of our intuitions about the extremity and relative equality of human value without incurring the substantial theoretical costs of using the extended real numbers. We also examine other ways of modeling the infinite value of persons and evaluate the strengths and weaknesses of these other accounts compared to ours. (shrink)

A famous mathematical theorem says that the sum of an infinite series of numbers can depend on the order in which those numbers occur. Suppose we interpret the numbers in such a series as representing instances of some physical quantity, such as the weights of a collection of items. The mathematics seems to lead to the result that the weight of a collection of items can depend on the order in which those items are weighed. But that is very hard (...) to believe. A puzzle then arises: How do we interpret the metaphysical significance of this mathematical theorem? I first argue that prior solutions to the puzzle lead to implausible consequences. Then I develop my own solution, where the basic idea is that the weight of a collection of items is equal to the limit of the weights of its finite subcollections contained within ever-expanding regions of space. I show how my solution is intuitively plausible and philosophically motivated, how it reveals an underexplored line of metaphysical inquiry about quantities and locations, and how it elucidates some classic puzzles concerning supertasks. (shrink)

Maximize Expected Choiceworthiness (MEC) is a theory of decision-making under moral uncertainty. It says that we ought to handle moral uncertainty in the way that Expected Value Theory (EVT) handles descriptive uncertainty. MEC inherits from EVT the problem of fanaticism. Roughly, a decision theory is fanatical when it requires our decision-making to be dominated by low-probability, high-payoff options. Proponents of MEC have offered two main lines of response. The first is that MEC should simply import whatever are the best solutions (...) to fanaticism on offer in decision theory. The second is to propose statistical normalization as a novel solution on behalf of MEC. This paper argues that the first response is open to serious doubt and that the second response fails. As a result, MEC appears significantly less plausible when compared to competing accounts of decision-making under moral uncertainty, which are not fanatical. (shrink)

This paper presents a unifying diagnosis of a number of important problems facing existing models of rational choice under moral uncertainty and proposes a remedy. I argue that the problems of (i) severely limited scope, (ii) intertheoretic comparisons, and (iii) 'swamping’ all stem from the way in which values are assigned to options in decision rules such as Maximisation of Expected Choiceworthiness. By assigning values to options under a given moral theory by asking something like ‘how much do I desire (...) this option, supposing this theory is true?’ rather than ‘how much value does this theory assign to this option?’ these problems can be avoided, while the appealing features of these accounts can be preserved. This amendment provides a role for the preferences, desires, or goals of rational agents that is curiously absent from the existing discussion of what individuals rationally ought to do when they are uncertain about what they morally ought to do. (shrink)

How all philosophical explanations of human consciousness and the fundamental structure of the cosmos are bizarre—and why that’s a good thing Do we live inside a simulated reality or a pocket universe embedded in a larger structure about which we know virtually nothing? Is consciousness a purely physical matter, or might it require something extra, something nonphysical? According to the philosopher Eric Schwitzgebel, it’s hard to say. In The Weirdness of the World, Schwitzgebel argues that the answers to these fundamental (...) questions lie beyond our powers of comprehension. We can be certain only that the truth—whatever it is—is weird. Philosophy, he proposes, can aim to open—to reveal possibilities we had not previously appreciated—or to close, to narrow down to the one correct theory of the phenomenon in question. Schwitzgebel argues for a philosophy that opens. According to Schwitzgebel’s “Universal Bizarreness” thesis, every possible theory of the relation of mind and cosmos defies common sense. According to his complementary “Universal Dubiety” thesis, no general theory of the relationship between mind and cosmos compels rational belief. Might the United States be a conscious organism—a conscious group mind with approximately the intelligence of a rabbit? Might virtually every action we perform cause virtually every possible type of future event, echoing down through the infinite future of an infinite universe? What, if anything, is it like to be a garden snail? Schwitzgebel makes a persuasive case for the thrill of considering the most bizarre philosophical possibilities. (shrink)

As well as disagreeing about how much one should donate to charity, moral theories also disagree about where one should donate. In light of this disagreement, how should the morally uncertain philanthropist allocate her donations? In many cases, one intuitively attractive option is for the philanthropist to split her donations across all of the charities that are recommended by moral views in which she has positive credence, with each charity’s share being proportional to her credence in the moral theories that (...) recommend that particular charity. Despite the fact that something like this approach is already being used by real-world philanthropists to distribute billions of dollars of donations, it is not supported by any of the approaches to moral uncertainty that have been proposed thus-far in the philosophical literature. In this paper, we will develop a bargaining-based approach to moral uncertainty that honors the proportionality intuition in favor of splitting one’s donations. We also show how this bargaining-based approach has several further advantages over the best alternative proposals. (shrink)

Should we make significant sacrifices to ever-so-slightly lower the chance of extremely bad outcomes, or to ever-so-slightly raise the chance of extremely good outcomes? *Fanaticism* says yes: for every bad outcome, there is a tiny chance of extreme disaster that is even worse, and for every good outcome, there is a tiny chance of an enormous good that is even better. I consider two related recent arguments for Fanaticism: Beckstead and Thomas's argument from *strange dependence on space and time*, and (...) Wilkinson's *Indology* argument. While both arguments are instructive, neither is persuasive. In fact, the general principles that underwrite the arguments (a *separability* principle in the first case, and a *reflection* principle in the second) are *inconsistent* with Fanaticism. In both cases, though, it is possible to rehabilitate arguments for Fanaticism based on restricted versions of those principles. The situation is unstable: plausible general principles tell *against* Fanaticism, but restrictions of those same principles (with strengthened auxiliary assumptions) *support* Fanaticism. All of the consistent views that emerge are very strange. (shrink)

In this article I argue that we should be prudentially and morally biased toward earlier events: other things equal, we should prefer for good events to occur earlier and disprefer for bad events to occur earlier. The argument contends that we should accord at least some credence—if only a small one—to a theoretical package featuring the growing block theory of time and that this package generates a presumptive bias toward earlier events. Rival theoretical packages are considered. Under reasonable allocations of (...) credence to them, the presumptive bias escapes defeat. The argument has several corollaries: other things equal, we should be biased toward the past over the future, the further past over the nearer past, and the nearer future over the further future. (shrink)

We begin by showing that every theory of the value of uncertain prospects must have one of three unpalatable properties. _Reckless_ theories recommend giving up a sure thing, no matter how good, for an arbitrarily tiny chance of enormous gain; _timid_ theories permit passing up an arbitrarily large potential gain to prevent a tiny increase in risk; _non-transitive_ theories deny the principle that, if A is better than B and B is better than C, then A must be better than (...) C. Having set up this trilemma, we study its horns. Non-transitivity has been much discussed; we focus on drawing out the costs and benefits of recklessness and timidity when it comes to axiology, decision theory, and normative uncertainty. (shrink)

This thesis consists of a series of papers in population ethics: a subfield of normative ethics concerned with the distinctive issues that arise in cases where our actions can affect the identities or number of people of who ever exist. Each paper can be read independently of the others. In Chapter 1, I present a dilemma for Archimedean views in population axiology: roughly, those views on which adding enough good lives to a population can make that population better than any (...) other. In Chapter 2, I extend Gustaf Arrhenius’s famous impossibility theorems in population axiology into the domain of choices under risk. My risky impossibility theorems dispense with the assumption that welfare levels are finitely fine-grained, and so tell against lexical views in population axiology. In Chapter 3, I present objections to critical-level and critical-range views in population axiology. I then sketch out what I call the ‘Imprecise Exchange Rates View’ and argue that it is an attractive alternative. In Chapter 4, I address critical-level and critical-range views again. This time, I note that they are vulnerable to objections from biographical identity: identity between lives. I suggest that these objections give us reason to reject critical-level and critical-range views and embrace the Total View. In Chapter 5, I argue that objections of the same form – objections from personal identity – tell against person-affecting views in population ethics. In Chapter 6, I draw out some counterintuitive implications of two recent complaints-based theories of the procreation asymmetry. (shrink)

Various philosophers accept moral views that are impartial, additive, and risk-neutral with respect to moral betterness. But, if that risk neutrality is spelt out according to expected value theory alone, such views face a dire reductio ad absurdum. If the expected sum of value in humanity's future is undefined--if, e.g., the probability distribution over possible values of the future resembles the Pasadena game, or a Cauchy distribution--then those views say that no option is ever better than any other. And, as (...) I argue, this holds in practice: our evidence supports such a probability distribution. Indeed, it supports a probability distribution that cannot be evaluated even if we adopt one of the various extensions of expected value theory proposed in the literature. Must we therefore reject all impartial, additive, risk-neutral moral theories? It turns out that we need not. I develop an alternative solution: a new method of extending expected value theory, which allows us to deal with this distribution and to salvage those moral views. I also examine how this solution affects one of the most notable implications of those views--namely, longtermism. (shrink)

The question of the meaning of life has long been thought to be closely intertwined with that of the existence of God. I offer a new theistic, anti-naturalist argument from the meaning of life. It is argued that the desire for life is irrational on naturalism, since there would be no good reason to believe that life is worthwhile on the whole if naturalism were true. As I show, the same cannot be argued of theism. Since it is clear that (...) the desire for life is not irrational, it is concluded that we have strong reason to prefer theism over naturalism. (shrink)

Suppose that you prefer A to B, B to C, and C to A. Your preferences violate Expected Utility Theory by being cyclic. Money-pump arguments offer a way to show that such violations are irrational. Suppose that you start with A. Then you should be willing to trade A for C and then C for B. But then, once you have B, you are offered a trade back to A for a small cost. Since you prefer A to B, you (...) pay the small sum to trade from B to A. But now you have been turned into a money pump. You are back to the alternative you started with but with less money. This Element shows how each of the axioms of Expected Utility Theory can be defended by money-pump arguments of this kind. The Element also defends money-pump arguments from the standard objections to this kind of approach. This title is also available as Open Access on Cambridge Core. (shrink)

Some people are deeply dissatisfied by the universe that modern science reveals to us. They long for the world described by traditional religion. They do not believe in God, but they wish He had existed. I argue that this is a mistake. The naturalist world we inhabit is admittedly rather bleak. It is very far from being the best of all possible worlds. But an alternative governed by God is also unwelcome, and the things that might make God’s existence attractive—cosmic (...) justice or the afterlife—could also be had without God. The most desirable of all possible worlds are therefore godless. (shrink)

Lexical views in population axiology can avoid the Repugnant Conclusion without violating Transitivity or Separability. However, they imply a dilemma: either some good life is better than any number of slightly worse lives, or else the ‘at least as good as’ relation on populations is radically incomplete. In this paper, I argue that Archimedean views face an analogous dilemma. I thus conclude that the lexical dilemma gives us little reason to prefer Archimedean views. Even if we give up on lexicality, (...) problems of the same kind remain. (shrink)

The principle of Anteriority says that prospects that are identical from the perspective of every possible person’s welfare are equally good overall. The principle enjoys prima facie plausibility, and has been employed for various theoretical purposes. Here it is shown using an analogue of the St Petersburg Paradox that Anteriority is inconsistent with central principles of axiology.

This thesis consists of several independent papers in population ethics. I begin in Chapter 1 by critiquing some well-known 'impossibility theorems', which purport to show there can be no intuitively satisfactory population axiology. I identify axiological vagueness as a promising way to escape or at least mitigate the effects of these theorems. In particular, in Chapter 2, I argue that certain of the impossibility theorems have little more dialectical force than sorites arguments do. From these negative arguments I move to (...) positive ones. In Chapter 3, I justify the use of a 'veil of ignorance', starting from three more basic normative principles. This leads to positive arguments for various kinds of utilitarianism - the best such arguments I know. But in general the implications of the veil depend on how one answers what I call 'the risky existential question': what is the value to an individual of a chance of non-existence? I chart out the main options, and raise some puzzles for non-comparativism, the view that life is incomparable to non-existence. Finally, in Chapter 4, I consider the consequences for population ethics of the idea that what is normatively relevant is not personal identity, but a degreed relation of psychological connectedness. In particular, I pursue a strategy based in population ethics for understanding the controversial 'time-relative interests' account of the badness of death. (shrink)

Many philosophers think that punishment is intentionally harmful and that this makes it especially hard to morally justify. Explanations for the latter intuition often say questionable things about the moral significance of the intent to harm. I argue that there’s a better way to explain this intuition.

How much value can our decisions create? We argue that unless our current understanding of physics is wrong in fairly fundamental ways, there exists an upper limit of value relevant to our decisions. First, due to the speed of light and the definition and conception of economic growth, the limit to economic growth is a restrictive one. Additionally, a related far larger but still finite limit exists for value in a much broader sense due to the physics of information and (...) the ability of physical beings to place value on outcomes. We discuss how this argument can handle lexicographic preferences, probabilities, and the implications for infinite ethics and ethical uncertainty. (shrink)

Aggregative moral theories face a series of devastating problems when we apply them in a physically realistic setting. According to current physics, our universe is likely _infinitely large_, and will contain infinitely many morally valuable events. But standard aggregative theories are ill-equipped to compare outcomes containing infinite total value so, applied in a realistic setting, they cannot compare any outcomes a real-world agent must ever choose between. This problem has been discussed extensively, and non-standard aggregative theories proposed to overcome it. (...) This paper addresses a further problem of similar severity. Physics tells us that, in our universe, how remotely in time an event occurs is _relative_. But our most promising aggregative theories, designed to compare outcomes containing infinitely many valuable events, are sensitive to how remote in time those events are. As I show, the evaluations of those theories are then relative too. But this is absurd; evaluations of outcomes must be absolute. So we must reject such theories. Is this objection fatal for all aggregative theories, at least in a relativistic universe like ours? I demonstrate here that, by further modifying these theories to fit with the physics, we can overcome it. (shrink)

Our universe is both chaotic and (most likely) infinite in space and time. But it is within this setting that we must make moral decisions. This presents problems. The first: due to our universe's chaotic nature, our actions often have long-lasting, unpredictable effects; and this means we typically cannot say which of two actions will turn out best in the long run. The second problem: due to the universe's infinite dimensions, and infinite population therein, we cannot compare outcomes by simply (...) adding up their total moral values - those totals will typically be infinite or undefined. Each of these problems poses a threat to aggregative moral theories. But, for each, we have solutions: a proposal from Greaves let us overcome the problem of chaos, and proposals from the infinite aggregation literature let us overcome the problem of infinite value. But a further problem emerges. If our universe is both chaotic and infinite, those solutions no longer work - outcomes that are infinite and differ by chaotic effects are incomparable, even by those proposals. In this paper, I show that we can overcome this further problem. But, to do so, we must accept some peculiar implications about how aggregation works. (shrink)

For aggregative theories of moral value, it is a challenge to rank worlds that each contain infinitely many valuable events. And, although there are several existing proposals for doing so, few provide a cardinal measure of each world's value. This raises the even greater challenge of ranking lotteries over such worlds—without a cardinal value for each world, we cannot apply expected value theory. How then can we compare such lotteries? To date, we have just one method for doing so (proposed (...) separately by Arntzenius, Bostrom, and Meacham), which is to compare the prospects for value at each individual location, and to then represent and compare lotteries by their expected values at each of those locations. But, as I show here, this approach violates several key principles of decision theory and generates some implausible verdicts. I propose an alternative—one which delivers plausible rankings of lotteries, which is implied by a plausible collection of axioms, and which can be applied alongside almost any ranking of infinite worlds. (shrink)

Which is better: a guarantee of a modest amount of moral value, or a tiny probability of arbitrarily large value? To prefer the latter seems fanatical. But, as I argue, avoiding such fanaticism brings severe problems. To do so, we must decline intuitively attractive trade-offs; rank structurally identical pairs of lotteries inconsistently, or else admit absurd sensitivity to tiny probability differences; have rankings depend on remote, unaffected events ; and often neglect to rank lotteries as we already know we would (...) if we learned more. Compared to these implications, fanaticism is highly plausible. (shrink)

Is the overall value of a world just the sum of values contributed by each value-bearing entity in that world? Additively separable axiologies (like total utilitarianism, prioritarianism, and critical level views) say 'yes', but non-additive axiologies (like average utilitarianism, rank-discounted utilitarianism, and variable value views) say 'no'. This distinction is practically important: additive axiologies support 'arguments from astronomical scale' which suggest (among other things) that it is overwhelmingly important for humanity to avoid premature extinction and ensure the existence of a (...) large future population, while non-additive axiologies need not. We show, however, that when there is a large enough 'background population' unaffected by our choices, a wide range of non-additive axiologies converge in their implications with some additive axiology -- for instance, average utilitarianism converges to critical-level utilitarianism and various egalitarian theories converge to prioritiarianism. We further argue that real-world background populations may be large enough to make these limit results practically significant. This means that arguments from astronomical scale, and other arguments in practical ethics that seem to presuppose additive separability, may be truth-preserving in practice whether or not we accept additive separability as a basic axiological principle. (shrink)

Suppose that an altruistic agent who is uncertain between evidential and causal decision theory finds herself in a situation where these theories give conflicting verdicts. We argue that even if she has significantly higher credence in CDT, she should nevertheless act in accordance with EDT. First, we claim that the appropriate response to normative uncertainty is to hedge one's bets. That is, if the stakes are much higher on one theory than another, and the credences you assign to each of (...) these theories are not very different, then it is appropriate to choose the option that performs best on the high-stakes theory. Second, we show that, given the assumption of altruism, the existence of correlated decision makers will increase the stakes for EDT but leave the stakes for CDT unaffected. Together these two claims imply that whenever there are sufficiently many correlated agents, the appropriate response is to act in accordance with EDT. (shrink)

There has been a growing worry that perdurantism—and similarly ontologically abundant views—is morally untenable. For perdurantism posits that, coinciding with persons, are person-like objects, and giving them their moral due seems to require giving up prudentially driven self-sacrifice. One way to avoid this charge is to adopt consequentialism. But Mark Johnston has argued that the marriage of consequentialism and perdurantism is in moral trouble. For, depending on the nature of time, consequentialist perdurantists either are unable to do more than one (...) good act or they are morally obliged to adopt a repugnant form of ageism. I argue both that perdurantist consequentialism doesn’t have the latter implication, and that there’s at least one plausible form of consequentialism that perdurantists can adopt to avoid the former implication. (shrink)

Decision-making under normative uncertainty requires an agent to aggregate the assessments of options given by rival normative theories into a single assessment that tells her what to do in light of her uncertainty. But what if the assessments of rival theories differ not just in their content but in their structure -- e.g., some are merely ordinal while others are cardinal? This paper describes and evaluates three general approaches to this "problem of structural diversity": structural enrichment, structural depletion, and multi-stage (...) aggregation. All three approaches have notable drawbacks, but I tentatively defend multi-stage aggregation as least bad of the three. (shrink)

We give two social aggregation theorems under conditions of risk, one for constant population cases, the other an extension to variable populations. Intra and interpersonal welfare comparisons are encoded in a single ‘individual preorder’. The theorems give axioms that uniquely determine a social preorder in terms of this individual preorder. The social preorders described by these theorems have features that may be considered characteristic of Harsanyi-style utilitarianism, such as indifference to ex ante and ex post equality. However, the theorems are (...) also consistent with the rejection of all of the expected utility axioms, completeness, continuity, and independence, at both the individual and social levels. In that sense, expected utility is inessential to Harsanyi-style utilitarianism. In fact, the variable population theorem imposes only a mild constraint on the individual preorder, while the constant population theorem imposes no constraint at all. We then derive further results under the assumption of our basic axioms. First, the individual preorder satisfies the main expected utility axiom of strong independence if and only if the social preorder has a vector-valued expected total utility representation, covering Harsanyi’s utilitarian theorem as a special case. Second, stronger utilitarian-friendly assumptions, like Pareto or strong separability, are essentially equivalent to strong independence. Third, if the individual preorder satisfies a ‘local expected utility’ condition popular in non-expected utility theory, then the social preorder has a ‘local expected total utility’ representation. Fourth, a wide range of non-expected utility theories nevertheless lead to social preorders of outcomes that have been seen as canonically egalitarian, such as rank-dependent social preorders. Although our aggregation theorems are stated under conditions of risk, they are valid in more general frameworks for representing uncertainty or ambiguity. (shrink)

Formula One isn’t very important. You can't care about it too much. The refugee crisis is more important. You can care about it much more. In this paper we investigate how important something is. By ‘importance’ we mean how much it is fitting to care about a thing. We explore a view about this which we call Proportionalism. This view says that a thing’s importance depends on that thing’s share of the world’s total value. The more of what matters there (...) is, the less you can care about each thing in particular. The less of what matters there is, the more you can care about each thing in particular. We argue that, in many respects, Proportionalism is superior to its competitors. It captures some intuitions they leave out and it has a powerful motivation. So, we suggest, you should keep things in proportion. (shrink)

Theories that use expected utility maximization to evaluate acts have difficulty handling cases with infinitely many utility contributions. In this paper I present and motivate a way of modifying such theories to deal with these cases, employing what I call “Direct Difference Taking”. This proposal has a number of desirable features: it’s natural and well-motivated, it satisfies natural dominance intuitions, and it yields plausible prescriptions in a wide range of cases. I then compare my account to the most plausible alternative, (...) a proposal offered by Arntzenius :31–58, 2014). I argue that while Arntzenius’s proposal has many attractive features, it runs into a number of problems which Direct Difference Taking avoids. (shrink)

Standard decision theory has trouble handling cases involving acts without finite expected values. This paper has two aims. First, building on earlier work by Colyvan (2008), Easwaran (2014), and Lauwers and Vallentyne (2016), it develops a proposal for dealing with such cases, Difference Minimizing Theory. Difference Minimizing Theory provides satisfactory verdicts in a broader range of cases than its predecessors. And it vindicates two highly plausible principles of standard decision theory, Stochastic Equivalence and Stochastic Dominance. The second aim is to (...) assess some recent arguments against Stochastic Equivalence and Stochastic Dominance. If successful, these arguments refute Difference Minimizing Theory. This paper contends that these arguments are not successful. (shrink)

Historically, the hypothesis that our world is a computer simulation has struck many as just another improbable-but-possible “skeptical hypothesis” about the nature of reality. Recently, however, the simulation hypothesis has received significant attention from philosophers, physicists, and the popular press. This is due to the discovery of an epistemic dependency: If we believe that our civilization will one day run many simulations concerning its ancestry, then we should believe that we are probably in an ancestor simulation right now. This essay (...) examines a troubling but underexplored feature of the ancestor-simulation hypothesis: the termination risk posed by both ancestor-simulation technology and experimental probes into whether our world is an ancestor simulation. This essay evaluates the termination risk by using extrapolations from current computing practices and simulation technology. The conclusions, while provisional, have great implications for debates concerning the fundamental nature of reality and the safety of contemporary physics. (shrink)

The principle that rational agents should maximize expected utility or choiceworthiness is intuitively plausible in many ordinary cases of decision-making under uncertainty. But it is less plausible in cases of extreme, low-probability risk (like Pascal's Mugging), and intolerably paradoxical in cases like the St. Petersburg and Pasadena games. In this paper I show that, under certain conditions, stochastic dominance reasoning can capture most of the plausible implications of expectational reasoning while avoiding most of its pitfalls. Specifically, given sufficient background uncertainty (...) about the choiceworthiness of one's options, many expectation-maximizing gambles that do not stochastically dominate their alternatives "in a vacuum" become stochastically dominant in virtue of that background uncertainty. But, even under these conditions, stochastic dominance will not require agents to accept options whose expectational superiority depends on sufficiently small probabilities of extreme payoffs. The sort of background uncertainty on which these results depend looks unavoidable for any agent who measures the choiceworthiness of her options in part by the total amount of value in the resulting world. At least for such agents, then, stochastic dominance offers a plausible general principle of choice under uncertainty that can explain more of the apparent rational constraints on such choices than has previously been recognized. (shrink)

A classification of the global catastrophic risks of AI is presented, along with a comprehensive list of previously identified risks. This classification allows the identification of several new risks. We show that at each level of AI’s intelligence power, separate types of possible catastrophes dominate. Our classification demonstrates that the field of AI risks is diverse, and includes many scenarios beyond the commonly discussed cases of a paperclip maximizer or robot-caused unemployment. Global catastrophic failure could happen at various levels of (...) AI development, namely, before it starts self-improvement, during its takeoff, when it uses various instruments to escape its initial confinement, or after it successfully takes over the world and starts to implement its goal system, which could be plainly unaligned, or feature-flawed friendliness. AI could also halt at later stages of its development either due to technical glitches or ontological problems. Overall, we identified around several dozen scenarios of AI-driven global catastrophe. The extent of this list illustrates that there is no one simple solution to the problem of AI safety, and that AI safety theory is complex and must be customized for each AI development level. (shrink)

A problem of the many Fs arises in cases where intuitively there is precisely one F (in the region you are talking about), but when you look closely you find many candidates for being that F, each one apparently as well-qualified as the next. Imagine an apparently solitary cloud in an otherwise blue sky. Look closer, and you'll see lots of water vapor molecules, with no one collection of them more eligible than the others to count as the cloud. Many (...) things are like this when you look closely enough. The problem arises for mundane things like rocks, houses and coins. It also arises for entities that present special philosophical challenges, like persons and experiencers. In this essay, I present a new argument that the problem of the many experiencers is an especially hard problem of the many, and property dualism—the view that properties that there is something it is like to instantiate are irreducible—may be the best way to solve it. (shrink)

How should deontologists approach decision-making under uncertainty, for an iterated decision problem? In this paper I explore the shortcomings of a simple expected value approach, using a novel example to raise questions about attitudes to risk, the moral significance of tiny probabilities, the independent moral reasons against imposing risks, the morality of sunk costs, and the role of agent-relativity in iterated decision problems.

Values are incommensurable when they cannot be measured on a single cardinal scale. Many philosophers suggest that incommensurability can help us solve the problems of population ethics. I agree. But some philosophers claim that populations bear incommensurable values merely because they contain different numbers of people, perhaps within some range. I argue that mere differences in how many people exist, even within some range, do not suffice for incommensurability. I argue that the intuitive neutrality of creating happy people is better (...) captured by a version of average utilitarianism. But this view is problematic. So I suggest a version of total utilitarianism that avoids the repugnant conclusion by appealing to incommensurable dimensions of wellbeing. (shrink)

How should an agent decide what to do when she is uncertain not just about morally relevant empirical matters, like the consequences of some course of action, but about the basic principles of morality itself? This question has only recently been taken up in a systematic way by philosophers. Advocates of moral hedging claim that an agent should weigh the reasons put forward by each moral theory in which she has positive credence, considering both the likelihood that that theory is (...) true and the strength of the reasons it posits. The view that it is sometimes rational to hedge for one's moral uncertainties, however, has recently come under attack both from those who believe that an agent should always be guided by the dictates of the single moral theory she deems most probable and from those who believe that an agent's moral beliefs are simply irrelevant to what she ought to do. Among the many objections to hedging that have been pressed in the recent literature is the worry that there is no non-arbitrary way of making the intertheoretic comparisons of moral value necessary to aggregate the value assignments of rival moral theories into a single ranking of an agent's options. -/- This dissertation has two principal objectives: First, I argue that, contra these recent objections, an agent's moral beliefs and uncertainties are relevant to what she rationally ought to do, and more particularly, that agents are at least sometimes rationally required to hedge for their moral uncertainties. My principal argument for these claims appeals to the enkratic conception of rationality, according to which the requirements of practical rationality derive from an agent's beliefs about the objective, desire-independent value or choiceworthiness of her options. Second, I outline a new general theory of rational choice under moral uncertainty. Central to this theory is the idea of content-based aggregation, that the principles according to which an agent should compare and aggregate rival moral theories are grounded in the content of those theories themselves, including not only their value assignments but also the metaethical and other non-surface-level propositions that underlie, justify, or explain those value assignments. (shrink)

Consider the following claim: given the choice between saving a life and preventing any number of people from temporarily experiencing a mild headache, you should always save the life. Many moral theorists accept this claim. In doing so, they commit themselves to some form of ‘moral absolutism’: the view that there are some moral considerations that cannot be outweighed by any number of lesser moral considerations. In contexts of certainty, it is clear what moral absolutism requires of you. However, what (...) does it require of you when deciding under risk? What ought you to do when there is a chance that, say, you will not succeed in saving the life? In recent years, various critics have argued that moral absolutism cannot satisfactorily deal with risk and should, therefore, be abandoned. In this paper, we show that moral absolutism can answer its critics by drawing on—of all things—orthodox expected utility theory. (shrink)

A common argument for atheism runs as follows: God would not create a world worse than other worlds he could have created instead. However, if God exists, he could have created a better world than this one. Therefore, God does not exist. In this paper I challenge the second premise of this argument. I argue that if God exists, our world will continue without end, with God continuing to create value-bearers, and sustaining and perfecting the value-bearers he has already created. (...) Given this, if God exists, our world—considered on the whole—is infinitely valuable. I further contend that this theistic picture makes our world's value unsurpassable. In support of this contention, I consider proposals for how infinitely valuable worlds might be improved upon, focusing on two main ways—adding value-bearers and increasing the value in present value-bearers. I argue that neither of these can improve our world. Depending on how each method is understood, either it would not improve our world, or our world is unsurpassable with respect to it. I conclude by considering the implications of my argument for the problem of evil more generally conceived. (shrink)

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the propects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from (...) a human to a "posthuman" society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could case our extinction or destroy the potential of Earth - originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (shrink)

Recent developments in cosmology indicate that every history having a non-zero probability is realized in infinitely many distinct regions of spacetime. Thus, it appears that the universe contains infinitely many civilizations exactly like our own, as well as infinitely many civilizations that differ from our own in any way permitted by physical laws. We explore the implications of this conclusion for ethical theory and for the doomsday argument. In the infinite universe, we find that the doomsday argument applies only to (...) effects which change the average lifetime of all civilizations, and not those which affect our civilization alone. (shrink)

First, I introduce the concept of a “non-agential subject,” where a non-agential subject exists within an organism and has phenomenally conscious experiences in a morally significant way, but is not morally responsible for the organism's voluntary actions. Second, I argue that it's a live possibility that typical adult humans contain non-agential subjects. Finally, I argue that, if there are non-agential subjects, this has important and surprising implications for a variety of ethical issues. Accordingly, ethicists should pay more attention to whether (...) there are non-agential subjects and what their implications for ethics would be. (shrink)

We develop an argument for a novel version of moral intuitionism centered on the claim that moral intuitions are trustworthy. Our argument employs an epistemic principle that we call the Trustworthiness Criterion, a distinctive feature of which is its emphasis on oft-neglected social dimensions of cognitive states, including non-doxastic attitudes such as intuition. Thus our argument is not that moral intuitions are trustworthy because they are regress-stoppers, or because they are innocent until proven guilty, or because denying their epistemic contribution (...) would be self-defeating, or because they are presupposed in rational inference, or because they are analogous to perceptions, or because they are based on understanding—individualistic claims that have elsewhere been used (controversially) in defense of the thesis that moral intuitions are in good epistemic standing. Rather, our argument appeals to the idea that moral intuitions are trustworthy because they are the outputs of a cognitive practice, which has epistemically-fecund social elements, that is in good working order. (shrink)

Most ethical work is done at a low level of formality. This makes practical moral questions inaccessible to formal and natural sciences and can lead to misunderstandings in ethical discussion. In this paper, we use Bayesian inference to introduce a formalization of preference utilitarianism in physical world models, specifically cellular automata. Even though our formalization is not immediately applicable, it is a first step in providing ethics and ultimately the question of how to “make the world better” with a formal (...) basis. (shrink)

This article sets out a formal procedure for determining the probability that God would do a specified action, using our moral knowledge and understanding God as a perfect being. To motivate developing the procedure I show how natural theology – design arguments, the problems of evil and divine hiddenness, and the treatment of miracles and religious experiences as evidence for claims about God – routinely appeals to judgments involving these probabilities. To set out the procedure, I describe a decision-theoretic model (...) for practical reasoning which is deontological so as to appeal to theists, but is designed not to presuppose any substantive moral commitments, and to accommodate normative and non-normative uncertainty. Then I explain how judgments about what we probably ought to do can be transformed into judgments about what God would probably do. Then I show the usefulness of the procedure by describing how it can help structure discussions in natural theology and a-theology, and how it offers an attractive alternative to ‘skeptical theism’. (shrink)