Newcomb's Problem

Causal Decision Theory reckons the choice-worthiness of an option to be completely independent of its evidential bearing on its non-effects. But after one has made a choice this bearing is relevant to future decisions. Therefore it is possible to construct problems of sequential choice in which Causal Decision Theory makes a guaranteed loss. So Causal Decision Theory is wrong. The source of the problem is the idea that agents have a special perspective on their own contemplated actions, from which evidential (...) connections that observers can see are either irrelevant or invisible. (shrink)

A decision problem where Causal Decision Theory (CDT) declines a free $1,000, with the foreseeable effect that the agent is $1,000 poorer, and in no other way better off, than if she had taken the offer.

We consider an extended notion of reinforcement learning in which the environment can simulate the agent and base its outputs on the agent's hypothetical behavior. Since good performance usually requires paying attention to whatever things the environment's outputs are based on, we argue that for an agent to achieve on-average good performance across many such extended environments, it is necessary for the agent to self-reflect. Thus, an agent's self-reflection ability can be numerically estimated by running the agent through a battery (...) of extended environments. We are simultaneously releasing an open-source library of extended environments to serve as proof-of-concept of this technique. As the library is first-of-kind, we have avoided the difficult problem of optimizing it. Instead we have chosen environments with interesting properties. Some seem paradoxical, some lead to interesting thought experiments, some are even suggestive of how self-reflection might have evolved in nature. We give examples and introduce a simple transformation which experimentally seems to increase self-reflection. (shrink)

I consider a familiar argument for two-boxing in Newcomb's Problem and find it defective because it involves a type of divergence from standard Baysian reasoning, which, though sometimes justified, conflicts with the stipulations of the Newcomb scenario. In an appendix, I also find fault with a different argument for two-boxing that has been presented by Graham Priest.

Newcomb’s Problem, Arif Ahmed (editor). Cambridge University Press, 2018, 233 pages.

We present a game mechanic called pseudo-visibility for games inhabited by non-player characters (NPCs) driven by reinforcement learning (RL). NPCs are incentivized to pretend they cannot see pseudo-visible players: the training environment simulates an NPC to determine how the NPC would act if the pseudo-visible player were invisible, and penalizes the NPC for acting differently. NPCs are thereby trained to selectively ignore pseudo-visible players, except when they judge that the reaction penalty is an acceptable tradeoff (e.g., a guard might accept (...) the penalty in order to protect a treasure because losing the treasure would hurt even more). We describe an RL agent transformation which allows RL agents that would not otherwise do so to perform some limited self-reflection to learn the training environments in question. (shrink)

I present a decision problem in which causal decision theory appears to violate the independence of irrelevant alternatives (IIA) and normal-form extensive-form equivalence (NEE). I show that these violations lead to exploitable behavior and long-run poverty. These consequences appear damning, but I urge caution. This decision should lead causalists to a better understanding of what it takes for a decision between some collection of options to count as a subdecision of a decision between a larger collection of options. And with (...) this better understanding of subdecisions in hand, causalists will not violate the IIA or the NEE. This decision will also teach causalists that, in sequential decisions, a rational agent may be led to make a series of choices which are causally dominated by some other sequence of choices they could have made instead. I will encourage causalists to recognise this as an intrapersonal tragedy of the commons. (shrink)

The dispute in philosophical decision theory between causalists and evidentialists remains unsettled. Many are attracted to the causal view’s endorsement of a species of dominance reasoning, and to the intuitive verdicts it gets on a range of cases with the structure of the infamous Newcomb’s Problem. But it also faces a rising wave of purported counterexamples and theoretical challenges. In this paper I will describe a novel decision theory which saves what is appealing about the causal view while avoiding its (...) most worrying objections, and which promises to generalize to solve a set of related problems in other normative domains. (shrink)

According to orthodox causal decision theory, performing an action can give you information about factors outside of your control, but you should not take this information into account when deciding what to do. Causal decision theorists caution against an irrational policy of 'managing the news'. But, by providing information about factors outside of your control, performing an act can give you two, importantly different, kinds of good news. It can tell you that the world in which you find yourself is (...) good in ways you can't control, and it can also tell you that the act itself is in a position to make the world better. While the first kind of news does not speak in favor of performing an act, I believe that the second kind of news does. I present a revision of causal decision theory which advises you to manage the news about the good you stand to promote, while ignoring news about the good the world has provided for you. (shrink)

Historically, the hypothesis that our world is a computer simulation has struck many as just another improbable-but-possible “skeptical hypothesis” about the nature of reality. Recently, however, the simulation hypothesis has received significant attention from philosophers, physicists, and the popular press. This is due to the discovery of an epistemic dependency: If we believe that our civilization will one day run many simulations concerning its ancestry, then we should believe that we are probably in an ancestor simulation right now. This essay (...) examines a troubling but underexplored feature of the ancestor-simulation hypothesis: the termination risk posed by both ancestor-simulation technology and experimental probes into whether our world is an ancestor simulation. This essay evaluates the termination risk by using extrapolations from current computing practices and simulation technology. The conclusions, while provisional, have great implications for debates concerning the fundamental nature of reality and the safety of contemporary physics. (shrink)

Consequentialist theories determine rightness solely based on real or expected consequences. Although such theories are popular, they often have difficulty with generalizing intuitions, which demand concern for questions like “What if everybody did that?” Rule consequentialism attempts to incorporate these intuitions by shifting the locus of evaluation from the consequences of acts to those of rules. However, detailed rule-consequentialist theories seem ad hoc or arbitrary compared to act consequentialist ones. We claim that generalizing can be better incorporated into consequentialism by (...) keeping the locus of evaluation on acts but adjusting the decision theory behind act selection. Specifically, we should adjust which types of dependencies the theory takes to be decision-relevant. Using this strategy, we formulate a new theory, generalized act consequentialism, which we argue is more compelling than rule consequentialism both in modeling the actual reasoning of generalizers and in delivering correct verdicts. (shrink)

Suppose that you have to take a test tomorrow but you do not want to study. Unfortunately you should study, since you care about passing and you expect to pass only if you study. Is there anything you can do to make it the case that you should not study? Is there any way for you to ‘rationalize’ slacking off? I suggest that such rationalization is impossible. Then I show that if evidential decision theory is true, rationalization is not only (...) possible but sometimes advisable. (shrink)

A review of some major topics of debate in normative decision theory from circa 2007 to 2019. Topics discussed include the ongoing debate between causal and evidential decision theory, decision instability, risk-weighted expected utility theory, decision-making with incomplete preferences, and decision-making with imprecise credences.

The 'Why ain'cha rich?' argument for one-boxing in Newcomb's problem allegedly vindicates evidential decision theory and undermines causal decision theory. But there is a good response to the argument on behalf of causal decision theory. I develop this response. Then I pose a new problem and use it to give a new 'Why ain'cha rich?' argument. Unlike the old argument, the new argument targets evidential decision theory. And unlike the old argument, the new argument is sound.

The standard formulation of Newcomb's problem compares evidential and causal conceptions of expected utility, with those maximizing evidential expected utility tending to end up far richer. Thus, in a world in which agents face Newcomb problems, the evidential decision theorist might ask the causal decision theorist: "if you're so smart, why ain’cha rich?” Ultimately, however, the expected riches of evidential decision theorists in Newcomb problems do not vindicate their theory, because their success does not generalize. Consider a theory that allows (...) the agents who employ it to end up rich in worlds containing Newcomb problems and continues to outperform in other cases. This type of theory, which I call a “success-first” decision theory, is motivated by the desire to draw a tighter connection between rationality and success, rather than to support any particular account of expected utility. The primary aim of this paper is to provide a comprehensive justification of success-first decision theories as accounts of rational decision. I locate this justification in an experimental approach to decision theory supported by the aims of methodological naturalism. (shrink)

Causalists and Evidentialists can agree about the right course of action in an (apparent) Newcomb problem, if the causal facts are not as initially they seem. If declining $1,000 causes the Predictor to have placed $1m in the opaque box, CDT agrees with EDT that one-boxing is rational. This creates a difficulty for Causalists. We explain the problem with reference to Dummett's work on backward causation and Lewis's on chance and crystal balls. We show that the possibility that the causal (...) facts might be properly judged to be non-standard in Newcomb problems leads to a dilemma for Causalism. One horn embraces a subjectivist understanding of causation, in a sense analogous to Lewis's own subjectivist conception of objective chance. In this case the analogy with chance reveals a terminological choice point, such that either (i) CDT is completely reconciled with EDT, or (ii) EDT takes precedence in the cases in which the two theories give different recommendations. The other horn of the dilemma rejects subjectivism, but now the analogy with chance suggests that it is simply mysterious why causation so construed should constrain rational action. (shrink)

Andy Egan recently drew attention to a class of decision situations that provide a certain kind of informational feedback, which he claims constitute a counterexample to causal decision theory. Arntzenius and Wallace have sought to vindicate a form of CDT by describing a dynamic process of deliberation that culminates in a “mixed” decision. I show that, for many of the cases in question, this proposal depends on an incorrect way of calculating expected utilities, and argue that it is therefore unsuccessful. (...) I then tentatively defend an alternative proposal by Joyce, which produces a similar process of dynamic deliberation but for a different reason. (shrink)

Andy Egan has presented a dilemma for decision theory. As is well known, Newcomb cases appear to undermine the case for evidential decision theory. However, Egan has come up with a new scenario which poses difficulties for causal decision theory. I offer a simple solution to this dilemma in terms of a modified EDT. I propose an epistemological test: take some feature which is relevant to your evaluation of the scenarios under consideration, evidentially correlated with the actions under consideration albeit, (...) causally independent of them. Hold this feature fixed as a hypothesis. The test shows that, in Newcomb cases, EDT would mislead the agent. Where the test shows EDT to be misleading, I propose to use fictive conditional credences in the EDT-formula under the constraint that they are set to equal values. I then discuss Huw Price’s defence of EDT as an alternative to my diagnosis. I argue that my solution also applies if one accepts the main premisses of Price’s argument. I close with applying my solution to Nozick’s original Newcomb problem. (shrink)

Standard theories of rational decision making and rational preference embrace the idea that there is something special about the present. Standard decision theory, for example, demands that agents privilege the perspective of the present (i.e., the time of decision) in evaluating what to do. When forming preferences, most philosophers believe that a similar focus on the present is justified, at least in the sense that rationality requires or permits future experiences to be given more weight than past ones. In this (...) dissertation, I examine such theories in light of the expected success of the agents who follow them. In Chapters 2 and 3, I show that this bias toward the present is a liability: it tends to make agents less successful than they might otherwise be. I also show how these problems can be avoided: In the case of rational decision making, we must privilege the beginning rather than the present (what I call “inceptive maximization”). In the case of rational preferences, we must be completely temporally neutral. -/- In chapters 4 and 5 I introduce a larger framework in which to interpret these results. My core thesis is that practical rationality is a form of conditional reliability. Practically rational decisions, preferences, intentions, or other relevant factors reliably produce whatever we take to be of value, conditional on an agent’s beliefs. This focus on value-conduciveness is thus the analog of the focus on truth-conduciveness in reliability theories of epistemic norms. Like reliabilism in epistemology, I show that practical reliabilism is supported by a methodologically naturalistic approach to normativity. In this way and others, I argue that epistemic and practical reliabilism interconnect to create an overarching theory of normativity. (shrink)

In “Bayesianism, Infinite Decisions, and Binding”, Arntzenius et al. (Mind 113:251–283, 2004 ) present cases in which agents who cannot bind themselves are driven by standard decision theory to choose sequences of actions with disastrous consequences. They defend standard decision theory by arguing that if a decision rule leads agents to disaster only when they cannot bind themselves, this should not be taken to be a mark against the decision rule. I show that this claim has surprising implications for a (...) number of other debates in decision theory. I then assess the plausibility of this claim, and suggest that it should be rejected. (shrink)

Nicholas Rescher claims that rational decision theory “may leave us in the lurch”, because there are two apparently acceptable ways of applying “the standard machinery of expected-value analysis” to his Dr. Psycho paradox which recommend contradictory actions. He detects a similar contradiction in Newcomb’s problem. We consider his claims from the point of view of both Bayesian decision theory and causal decision theory. In Dr. Psycho and in Newcomb’s Problem, Rescher has used premisses about probabilities which he assumes to be (...) independent. From the former point of view, we show that the probability premisses are not independent but inconsistent, and their inconsistency is provable within probability theory alone. From the latter point of view, we show that their consistency can be saved, but then the contradictory recommendations evaporate. Consequently, whether one subscribes to evidential or causal decision theory, rational decision theory is not in any way vitiated by Rescher’s arguments. (shrink)

Doris Olin's Paradox is a very helpful book for those who want to be introduced to the philosophical treatment of paradoxes, or for those who already have knowledge of the general area and would like to have a helpful resource book.

The primary aim of this paper is the presentation of a foundation for causal decision theory. This is worth doing because causal decision theory (CDT) is philosophically the most adequate rational decision theory now available. I will not defend that claim here by elaborate comparison of the theory with all its competitors, but by providing the foundation. This puts the theory on an equal footing with competitors for which foundations have already been given. It turns out that it will also (...) produce a reply to the most serious objections made so far against CDT and against the particular version of CDT I will defend. (shrink)

Newcomb’s problem is a decision puzzle whose difficulty and interest stem from the fact that the possible outcomes are probabilistically dependent on, yet causally independent of, the agent’s options. The problem is named for its inventor, the physicist William Newcomb, but first appeared in print in a 1969 paper by Robert Nozick [12]. Closely related to, though less well-known than, the Prisoners’ Dilemma, it has been the subject of intense debate in the philosophical literature. After three decades, the issues remain (...) unresolved. Newcomb’s problem is of genuine importance because it poses a challenge to the theoretical adequacy of orthodox Bayesian decision theory. It has led both to the development of causal decision theory and to efforts aimed at defending the adequacy of the orthodox theory. (shrink)