Evidential Decision Theory is flawed, but its flaws are not fully understood. David Lewis (1981) famously charged that EDT recommends an irrational policy of managing the news and “commends the ostrich as rational”. Lewis was right, but the case he appealed to—Newcomb’s Problem—does not demonstrate his conclusion. Indeed, decision theories other than EDT, such as Committal Decision Theory and Functional Decision Theory, agree with EDT's verdicts in Newcomb’s Problem, but their flaws, whatever they may be, do not stem from any (...) ostrich-like recommendations. We offer a new case which shows that EDT mismanages the news, thus vindicating Lewis’s original charge. We argue that this case reveals a flaw in the “Why ain’cha rich?” defense of EDT. We argue further that this case is an advance on extant putative counterexamples to EDT. (shrink)

The thesis of instrumental convergence holds that a wide range of ends have common means: for instance, self preservation, desire preservation, self improvement, and resource acquisition. Bostrom contends that instrumental convergence gives us reason to think that "the default outcome of the creation of machine superintelligence is existential catastrophe". I use the tools of decision theory to investigate whether this thesis is true. I find that, even if intrinsic desires are randomly selected, instrumental rationality induces biases towards certain kinds of (...) choices. Firstly, a bias towards choices which leave less up to chance. Secondly, a bias towards desire preservation, in line with Bostrom's conjecture. And thirdly, a bias towards choices which afford more choices later on. I do not find biases towards any other of the convergent instrumental means on Bostrom's list. I conclude that the biases induced by instrumental rationality at best weakly support Bostrom's conclusion that machine superintelligence is likely to lead to existential catastrophe. (shrink)

I respond to a dilemma for Causal Decision Theory (CDT) under determinism, posed in Adam Elga's paper “Confessions of a Causal Decision Theorist”. The treatment I present highlights (i) the status of laws as predictors, and (ii) the consequences of decision dependence, which arises natively out of Jeffrey Conditioning and CDT's characteristic equation.My argument leverages decision dependence to work around a key assumption of Elga's proof: to wit, that in the two problems he presents, the CDTer must employ subjunctive‐suppositional (rather (...) than evidential) transformations of a shared prior. (shrink)

This article explores theoretical conditions necessary for “quantum immortality” as well as its possible practical implications. It is demonstrated that QI is a particular case of “multiverse immortality”, which is based on two main assumptions: the very large size of the universe ; and a copy-friendly theory of personal identity. It is shown that a popular objection about lowering of the world-share of an observer in the case of QI does not succeed, as the world-share decline could be compensated by (...) merging timelines for the simpler minds, and because some types of personal preferences are not dependent on such changes. Despite large uncertainty about the truth of MI, it has appreciable practical consequences for some important outcomes like suicide and aging. The article demonstrates that MI could be used to significantly increase the expected subjective probability of success of risky life extension technologies, such as cryonics, but that it makes euthanasia impractical because of the risk of eternal suffering. Euthanasia should be replaced with cryothanasia, i.e. cryopreservation after voluntary death. Another possible application of MI is as a last chance to survive a global catastrophe. MI could be considered a Plan D for reaching immortality, where Plan A consists of survival until the development of beneficial Artificial Intelligence capable of fighting aging, Plan B employs cryonics, and Plan C is digital immortality. (shrink)

How much value can our decisions create? We argue that unless our current understanding of physics is wrong in fairly fundamental ways, there exists an upper limit of value relevant to our decisions. First, due to the speed of light and the definition and conception of economic growth, the limit to economic growth is a restrictive one. Additionally, a related far larger but still finite limit exists for value in a much broader sense due to the physics of information and (...) the ability of physical beings to place value on outcomes. We discuss how this argument can handle lexicographic preferences, probabilities, and the implications for infinite ethics and ethical uncertainty. (shrink)

Suppose that an altruistic agent who is uncertain between evidential and causal decision theory finds herself in a situation where these theories give conflicting verdicts. We argue that even if she has significantly higher credence in CDT, she should nevertheless act in accordance with EDT. First, we claim that the appropriate response to normative uncertainty is to hedge one's bets. That is, if the stakes are much higher on one theory than another, and the credences you assign to each of (...) these theories are not very different, then it is appropriate to choose the option that performs best on the high-stakes theory. Second, we show that, given the assumption of altruism, the existence of correlated decision makers will increase the stakes for EDT but leave the stakes for CDT unaffected. Together these two claims imply that whenever there are sufficiently many correlated agents, the appropriate response is to act in accordance with EDT. (shrink)

Consequentialist theories determine rightness solely based on real or expected consequences. Although such theories are popular, they often have difficulty with generalizing intuitions, which demand concern for questions like “What if everybody did that?” Rule consequentialism attempts to incorporate these intuitions by shifting the locus of evaluation from the consequences of acts to those of rules. However, detailed rule-consequentialist theories seem ad hoc or arbitrary compared to act consequentialist ones. We claim that generalizing can be better incorporated into consequentialism by (...) keeping the locus of evaluation on acts but adjusting the decision theory behind act selection. Specifically, we should adjust which types of dependencies the theory takes to be decision-relevant. Using this strategy, we formulate a new theory, generalized act consequentialism, which we argue is more compelling than rule consequentialism both in modeling the actual reasoning of generalizers and in delivering correct verdicts. (shrink)

Decision theorists disagree about how instrumentally rational agents, i.e., agents trying to achieve some goal, should behave in so-called Newcomb-like problems, with the main contenders being causal and evidential decision theory. Since the main goal of artificial intelligence research is to create machines that make instrumentally rational decisions, the disagreement pertains to this field. In addition to the more philosophical question of what the right decision theory is, the goal of AI poses the question of how to implement any given (...) decision theory in an AI. For example, how would one go about building an AI whose behavior matches evidential decision theory’s recommendations? Conversely, we can ask which decision theories describe the behavior of any existing AI design. In this paper, we study what decision theory an approval-directed agent, i.e., an agent whose goal it is to maximize the score it receives from an overseer, implements. If we assume that the overseer rewards the agent based on the expected value of some von Neumann–Morgenstern utility function, then such an approval-directed agent is guided by two decision theories: the one used by the agent to decide which action to choose in order to maximize the reward and the one used by the overseer to compute the expected utility of a chosen action. We show which of these two decision theories describes the agent’s behavior in which situations. (shrink)

This paper targets a nearly universal assumption in the philosophical literature: that prepunishment is unproblematic for consequentialists. Prepunishment threats do not deter, as deterrence is traditionally conceived. In fact, a pure prepunishment legal system would tend to increase the criminal disposition of the grudgingly compliant. This is a serious problem since, from many perspectives, but especially from a consequentialist one, a primary purpose of punishment is deterrence. I analyze the decision theory behind pre and postpunishments, which helps clarify both what (...) deterrence is and how it operates in consequentialist justifications of punishment. I end by sketching a road map for the future of prepunishment as artificial intelligence and other technological advances generate increasing possibilities for its use. (shrink)

Newcomb’s problem has spawned a debate about which variant of expected utility maximisation should guide rational choice. In this paper, we provide a new argument against what is probably the most popular variant: causal decision theory. In particular, we provide two scenarios in which CDT voluntarily loses money. In the first, an agent faces a single choice and following CDT’s recommendation yields a loss of money in expectation. The second scenario extends the first to a diachronic Dutch book against CDT.