On at least one of its uses, ‘higher-order evidence’ refers to evidence about what opinions are rationalized by your evidence. This chapter surveys the foundational epistemological questions raised by such evidence, the methods that have proven useful for answering them, and the potential consequences and applications of such answers.

Which rules should guide our reasoning? Human reasoners often use reasoning shortcuts, called heuristics, which function well in some contexts but lack the universality of reasoning rules like deductive implication or inference to the best explanation. Does it follow that human reasoning is hopelessly irrational? I argue: no. Heuristic reasoning often represents human reasoners reaching a local rational maximum, reasoning more accurately than if they try to implement more “ideal” rules of reasoning. I argue this is a genuine rational achievement. (...) Our ideal rational advisors would advise us to reason with heuristic rules, not more complicated ideal rules. I argue we do not need a radical new account of epistemic norms to make sense of the success of heuristic reasoning. (shrink)

Do people tend to be overconfident? Many think so. They’ve run studies on whether people are calibrated: whether their average confidence in their opinions matches the proportion of those opinions that are true. Under certain conditions, people are systematically ‘over-calibrated’—for example, of the opinions they’re 80% confident in, only 60% are true. From this empirical over-calibration, it’s inferred that people are irrationally overconfident. My question: When and why is this inference warranted? Answering it requires articulating a general connection between being (...) rational and being right—something extant studies have not done. I show how to do so using the notion of deference. This provides a theoretical foundation for calibration research, but also reveals a flaw: the connection between being rational and being right is much weaker than is standardly assumed, since rational people can often be expected to be miscalibrated. Thus we can’t test whether people are overconfident by simply testing whether they are over-calibrated; instead, we must try to predict the rational deviations from calibration, and then compare those predictions to people’s performance. I show how this can be done—and that doing so complicates the interpretation of robust empirical effects. (shrink)

I argue that evidentialism should be rejected because it cannot be reconciled with empirical work on bias in cognitive and social psychology.

I argue that bounded agents face a systematic accuracy-coherence tradeoff in cognition. Agents must choose whether to structure their cognition in ways likely to promote coherence or accuracy. I illustrate the accuracy-coherence tradeoff by showing how it arises out of at least two component tradeoffs: a coherence-complexity tradeoff between coherence and cognitive complexity, and a coherence-variety tradeoff between coherence and strategic variety. These tradeoffs give rise to an accuracy-coherence tradeoff because privileging coherence over complexity or strategic variety often leads to (...) a corresponding reduction in accuracy. I conclude with a discussion of two normative consequences for the study of bounded rationality: the importance of procedural rationality and the role of coherence in theories of bounded rationality. (shrink)

The fallacy fallacy is either the misdiagnosis of fallacy or the supposition that the conclusion of a fallacy must be a falsehood. This paper explores the relevance of these and related errors of reasoning for the appraisal of arguments, especially within virtue theories of argumentation. In particular, the fallacy fallacy exemplifies the Owl of Minerva problem, whereby tools devised to understand a norm make possible new ways of violating the norm. Fallacies are such tools and so are vices. Hence a (...) similar problem arises with argumentative vices. Fortunately, both instances of the problem have a common remedy. (shrink)

Higher-order evidence is evidence about what is rational to think in light of your evidence. Many have argued that it is special – falling into its own evidential category, or leading to deviations from standard rational norms. But it is not. Given standard assumptions, almost all evidence is higher-order evidence.

I argue that bounded agents face a systematic accuracy–coherence trade-off in cognition. Agents must choose whether to structure their cognition in ways likely to promote coherence or accuracy. I illustrate the accuracy–coherence trade-off by showing how it arises out of at least two component trade-offs: a coherence–complexity trade-off between coherence and cognitive complexity, and a coherence–variety trade-off between coherence and strategic variety. These trade-offs give rise to an accuracy–coherence trade-off because privileging coherence over complexity or strategic variety often leads to (...) a corresponding reduction in accuracy. I conclude with a discussion of two normative consequences for the study of bounded rationality: the importance of procedural rationality and the role of coherence in theories of bounded rationality. (shrink)

A new formal model of belief dynamics is proposed, in which the epistemic agent has both probabilistic beliefs and full beliefs. The agent has full belief in a proposition if and only if she considers the probability that it is false to be so close to zero that she chooses to disregard that probability. She treats such a proposition as having the probability 1, but, importantly, she is still willing and able to revise that probability assignment if she receives information (...) that gives her sufficient reasons to do so. Such a proposition is undoubted, but not undoubtable. In the formal model it is assigned a probability 1 − δ, where δ is an infinitesimal number. The proposed model employs probabilistic belief states that contain several underlying probability functions representing alternative probabilistic states of the world. Furthermore, a distinction is made between update and revision, in the same way as in the literature on belief change. The formal properties of the model are investigated, including properties relevant for learning from experience. The set of propositions whose probabilities are infinitesimally close to 1 forms a belief set. Operations that change the probabilistic belief state give rise to changes in this belief set, which have much in common with traditional operations of belief change. (shrink)

Wilfrid Sellars [1962: 1] described philosophy as an attempt to ‘understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term’. But it is distinctive of philosophy that many of us are interested not only in how the world is but in ways that it could be. That is, philosophy is concerned with facts about modality. Some of the most important arguments in philosophy hinge on modal premises, and philosophers have (...) typically assumed special expertise in evaluating these modal premises. Replicating Goldvarg and Johnson-Laird’s [2000] study of modal illusions, we show that training in mathematics and not philosophy predicts success in overcoming such illusions (n = 395). This study is the first to test the modal expertise of professional philosophers directly. Our findings undermine claims to modal expertise. Philosophical training does not inoculate expert philosophers against basic mistakes in modal judgment. (shrink)

The gambler’s fallacy is the tendency to expect random processes to switch more often than they actually do—for example, to think that after a string of tails, a heads is more likely. It’s often taken to be evidence for irrationality. It isn’t. Rather, it’s to be expected from a group of Bayesians who begin with causal uncertainty, and then observe unbiased data from an (in fact) statistically independent process. Although they converge toward the truth, they do so in an asymmetric (...) way—ruling out “streaky” hypotheses more quickly than “switchy” ones. As a result, the majority (and the average) exhibit the gambler’s fallacy. If they have limited memory, this tendency persists even with arbitrarily-large amounts of data. Indeed, such Bayesians exhibit a variety of the empirical trends found in studies of the gambler’s fallacy: they expect switches after short streaks but continuations after long ones; these nonlinear expectations vary with their familiarity with the causal system; their predictions depend on the sequence they’ve just seen; they produce sequences that are too switchy; and they exhibit greater rates of gambler’s reasoning when forming binary predictions than when forming probability estimates. In short: what’s been thought to be evidence for irrationality may instead be rational responses to limited data and memory. (shrink)

Outcome bias and hindsight bias are important in philosophical debates and have wide-ranging implications outside of philosophy. Recently, Hedden has articulated a novel line of argumnt that the empirical evidence for what he labels hindsight bias is largely misguided and that empirical researchers who postulate such a bias are engaged in a fallacy fallacy. In this paper, I articulate Hedden’s core insights in terms of two principles and argue that in the relevant empirical research, these principles are often (i) recognized, (...) (ii) invoked in theoretical explanation, and (iii) guiding study designs and interpretation of data. On this basis, I argue that there is ample empirical evidence for outcome and hindsight bias and that cognitive psychologists do not generally exhibit a fallacy fallacy of postulating such biases in cases where there are none. More constructively, the present paper articulates some general principles that help us to better understand when an empirical study provides evidence of these types of bias. Thus, the debate casts light on both the specific evidence for outcome and hindsight bias but also on the general division of labor between philosophy and psychology. (shrink)

Recent findings from psychology and behavioral economics suggest that we are “predictably irrational” in the pursuit of our interests. Paternalists from both the social sciences and philosophy use these findings to defend interfering with people's consumption choices for their own good. We should tax soda, ban cigarettes, and mandate retirement savings to make people healthier and wealthier than they’d be on their own. Our thesis is that the standard arguments offered in support of restricting people’s consumption choices for their own (...) good also imply support for “epistocratic” restrictions on people’s voting choices for their own good. Indeed, the philosophical case for paternalistic restrictions on voting choices may be stronger than the case for restricting personal consumption choices. So, paternalists face a dilemma: either endorse less interference with consumption choices or more interference with voting choices. (shrink)

I review proposed skepticisms in recent literature (or skeptical invariantisms, if we understand skepticism semantically), contrast their basic commitments and highlight some of their comparative theoretical attractions and problems. To help set the scene for the discussion, I start with Unger’s (1975) modern classic of global skepticism about knowledge (and justification). I then distinguish three extant categories of skepticism in the recent literature: two non‐traditional and one more traditional. On the non‐traditional side are fallibilist science‐based skepticism (which relaxes thestringencyof the (...) standard of knowledge) and practical skepticism (which questions theintellectualcharacter of the standard of knowledge) and on the traditional side high‐standard/infallibilist. I then briefly present and discuss representatives of each category. (shrink)

The Cognitive Reflection Test is a widely used measure of the degree to which individuals override an intuitive response and engage in reflection. For both theoretical and practical reasons, it is widely taken to assess an important component of rational thought. In this paper, I will argue that while doing well on the CRT requires valuable cognitive capacities and dispositions, doing badly does not always indicate a lack of such capacities and dispositions. The CRT, I argue, offers respondents implicit (but (...) misleading) testimony: some of those who do badly do so due to a stronger disposition to defer to testimony. Since deference is not irrational, those who do badly need not be irrational, not even in part. (shrink)

ABSTRACT Wilfrid Sellars [1962: 1] described philosophy as an attempt to ‘understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term’. But it is distinctive of philosophy that many of us are interested not only in how the world is but in ways that it could be. That is, philosophy is concerned with facts about modality. Some of the most important arguments in philosophy hinge on modal premises, and philosophers (...) have typically assumed special expertise in evaluating these modal premises. Replicating Goldvarg and Johnson-Laird’s [2000] study of modal illusions, we show that training in mathematics and not philosophy predicts success in overcoming such illusions (n = 395). This study is the first to test the modal expertise of professional philosophers directly. Our findings undermine claims to modal expertise. Philosophical training does not inoculate expert philosophers against basic mistakes in modal judgment. (shrink)