This article evaluates an emerging element in popular debate and inquiry: DYOR. (Haven’t heard of the acronym? Then Do Your Own Research.) The slogan is flexible and versatile. It is used frequently on social media platforms about topics from medical science to financial investing to conspiracy theories. Using conceptual and empirical resources drawn from philosophy and psychology, we examine key questions about the slogan’s operation in human cognition and epistemic culture.

The endorsement of epistemically suspect (i.e., paranormal, conspiracy, and pseudoscientific) beliefs is widespread and has negative consequences. Therefore, it is important to understand the reasoning processes – such as lower analytic thinking and susceptibility to cognitive biases – that might lead to the adoption of such beliefs. In two studies, I constructed and tested a novel questionnaire on epistemically suspect beliefs (Study 1, N = 263), and used it to examine probabilistic reasoning biases and belief bias in syllogistic reasoning as (...) predictors of the endorsement of those beliefs, while accounting for analytic thinking and worldview variables (Study 2, N = 397). Probabilistic reasoning biases, analytic thinking, religious faith, and political liberalism consistently predicted various epistemically suspect beliefs, whereas the effect of syllogistic belief bias was largely restricted to pseudoscientific beliefs. Further research will be needed to examine the role the biased evaluation of evidence plays in the endorsement of epistemically suspect beliefs. (shrink)

A number of important decision domains, including decisions about hiring, global warming, and weather hazards, are characterized by a global–local incompatibility. These domains involve variables that cannot be observed by a single decision maker (DM) and require the integration of observations from locally available information cues. This paper presents a new bifocal lens model that describes how the structure of the environment can lead to a unique form of overconfidence when generalizing the reliability of the local environment to a global (...) scale. When the local environment does not reliably reflect the global environment, they are incompatible. While local perspectives vary across DMs, global–local incompatibility can be understood using the structure of classical test theory as the difference between (a) perceived estimates of the reliability derived from the local environment and (b) the true reliability of the local environment. I model global–local incompatibility as the difference between the true and estimated reliability when the assumptions of classical test theory are violated. Using a series of case studies and an empirical study, I demonstrate the widespread utility of this framework, and I conclude by discussing implications for cognitive–ecological theory, risk communication, and overconfidence. (shrink)

When solving problems, like making predictions or choices, people often “sample” possibilities into mind. Here, we consider whether there is structure to the kinds of thoughts people sample by default—that is, without an explicit goal. Across three experiments we found that what comes to mind by default are samples from a probability distribution that combines what people think is likely and what they think is good. Experiment 1 found that the first quantities that come to mind for everyday behaviors and (...) events are quantities that combine what is average and ideal. Experiment 2 found, in a manipulated context, that the distribution of numbers that come to mind resemble the mathematical product of the presented statistical distribution and a (softmax-transformed) prescriptive distribution. Experiment 3 replicated these findings in a visual domain. These results provide insight into the process generating people’s conscious thoughts and invite new questions about the value of thinking about things that are both likely and good. (shrink)

This article combines findings from cognitive psychology on the role of processing fluency in truth judgments with epistemological theory on justification of belief. We first review evidence that repeated exposure to a statement increases the subjective ease with which that statement is processed. This increased processing fluency, in turn, increases the probability that the statement is judged to be true. The basic question discussed here is whether the use of processing fluency as a cue to truth is epistemically justified. In (...) the present analysis, based on Bayes’ Theorem, we adopt the reliable-process account of justification presented by Goldman (1986) and show that fluency is a reliable cue to truth, under the assumption that the majority of statements one has been exposed to are true. In the final section, we broaden the scope of this analysis and discuss how processing fluency as a potentially universal cue to judged truth may contribute to cultural differences in commonsense beliefs. (shrink)

The terms nested sets, partitive frequencies, inside-outside view, and dual processes add little but confusion to our original analysis (Gigerenzer & Hoffrage 1995; 1999). The idea of nested set was introduced because of an oversight; it simply rephrases two of our equations. Representation in terms of chances, in contrast, is a novel contribution yet consistent with our computational analysis System 1.dual process theory” is: Unless the two processes are defined, this distinction can account post hoc for almost everything. In contrast, (...) an ecological view of cognition helps to explain how insight is elicited from the outside (the external representation of information) and, more generally, how cognitive strategies match with environmental structures. (shrink)

The phenomenon of base-rate neglect has elicited much debate. One arena of debate concerns how people make judgments under conditions of uncertainty. Another more controversial arena concerns human rationality. In this target article, we attempt to unpack the perspectives in the literature on both kinds of issues and evaluate their ability to explain existing data and their conceptual coherence. From this evaluation we conclude that the best account of the data should be framed in terms of a dual-process model of (...) judgment, which attributes base-rate neglect to associative judgment strategies that fail to adequately represent the set structure of the problem. Base-rate neglect is reduced when problems are presented in a format that affords accurate representation in terms of nested sets of individuals. (shrink)

A recent critique of hierarchical Bayesian models of delusion argues that, contrary to a key assumption of these models, belief formation in the healthy (i.e., neurotypical) mind is manifestly non-Bayesian. Here we provide a deeper examination of the empirical evidence underlying this critique. We argue that this evidence does not convincingly refute the assumption that belief formation in the neurotypical mind approximates Bayesian inference. Our argument rests on two key points. First, evidence that purports to reveal the most damning violation (...) of Bayesian updating in human belief formation is counterweighted by substantial evidence that indicates such violations are the rare exception—not a common occurrence. Second, the remaining evidence does not demonstrate convincing violations of Bayesian inference in human belief updating; primarily because this evidence derives from study designs that produce results that are not obviously inconsistent with Bayesian principles. (shrink)

Consider the task of predicting which soccer team will win the next World Cup. The bookmakers may judge Brazil to be the team most likely to win, but also judge it most likely that a European rather than a Latin American team will win. This is an example of a non-aligned hierarchy structure: the most probable event at the subordinate level (Brazil wins) appears to be inconsistent with the most probable event at the superordinate level (a European team wins). In (...) this paper we exploit such structures to investigate how people make predictions based on uncertain hierarchical knowledge. We distinguish between aligned and non-aligned environments, and conjecture that people assume alignment. Participants were exposed to a non-aligned training set in which the most probable superordinate category predicted one outcome, whereas the most probable subordinate category predicted a different outcome. In the test phase participants allowed their initial probability judgments about category membership to shift their final ratings of the probability of the outcome, even though all judgments were made on the basis of the same statistical data. In effect people were primed to focus on the most likely path in an inference tree, and neglect alternative paths. These results highlight the importance of the level at which statistical data are represented, and suggest that when faced with hierarchical inference problems people adopt a simplifying heuristic that assumes alignment. (shrink)

Tversky and Kahneman (1974) originally discussed three main heuristics: availability, representativeness, and anchoring-and-adjustment. Research on judgemental forecasting suggests that the type of information on which forecasts are based is the primary factor determining the type of heuristic that people use to make their predictions. Specifically, availability is used when forecasts are based on information held in memory; representativeness is important when the value of one variable is forecast from explicit information about the value of another variable; and anchoring-and-adjustment is employed (...) when the value of a variable is forecast from explicit information about previous values of that same variable. Although there has been increased emphasis on the adaptiveness of heuristics and increased interest in specifying their use in terms of computational models, this way of structuring our knowledge about judgemental forecasting continues to be a useful one. I use it to frame discussion of some recent debates in the area. (shrink)

The social environment influences what information individuals sample: people are often exposed to alternatives that are popular. This can systematically change an individual's evaluation of an alternative if she had previously been avoiding it due to a negative evaluation. The authors show that social exposure can have positive or negative effects on evaluation, depending on how popularity and prior evaluations interact. This theory was supported by a large‐scale analysis of data from a hotel chain.

Psychological science is maturing and therefore transitioning from explorative to theory-driven research. While explorative research seeks to find something “new,” theory-driven research seeks to elaborate on already known and hence predictable effects. A consequence of these differences is that the quality of explorative and theory-driven research needs to be judged by distinct criterions that optimally support their respective development. Especially, theory-driven research needs to be judged by its methodological rigor. A focus on innovativeness, which is typical for explorative research, will (...) instead incentivize bad research practices (e.g., imprecise theorizing, ignoring previous research, parallel theories). To support the advancement of psychology, we must drop the innovation requirement for theory-driven research and instead require the strongest methods, which are marked by high internal and external validity. Precise theorizing needs to substitute novelty. Theories are advanced by requiring explicit, testable assumptions, and an explicit preference for one theory over another. These explicit and potentially wrong assumptions should not be silenced within the peer-review process, but instead be scrutinized in new publications. Importantly, these changes in scientific conduct need to be supported by senior researchers, especially, in their roles as editors, reviewers, and in the hiring process. An important obstacle to further theory-driven research is to measure scientific merit using researchers’ number of publications, which favors theoretically shallow and imprecise writing. Additionally, it makes publications the central target of scientific misconduct even though they are the main source of information for the scientific community and the public. To advance the field, researchers should be judged by their contribution to the scientific community (e.g., exchange with and support of colleagues, and mentoring). Another step to advance psychology is to clearly differentiate between measurement model and theory, and not to overgeneralize based on few stimuli, incidences, or studies. We will use ideas from the theory of science to underline the changes necessary within the field of psychology to overcome this existential replication crisis. (shrink)

Decisions in the environmental and in particular the climate domain are burdened with uncertainty. Here, we focus on uncertainties faced by individuals when making decisions about environmental behavior, and we use the statistical sampling framework to develop a classification of different sources of uncertainty they encounter. We then map these sources to different public policy strategies aiming to help individuals cope with uncertainty when making environmental decisions.

The statistics of small samples are often quite different from those of large samples, and this needs to be taken into account in assessing the rationality of human behavior. Specifically, in evaluating human responses to environmental statistics, it is the effective environment that matters; that is, the environment actually experienced by the agent needs to be considered, not simply long‐run frequencies. Significant deviations from long‐run statistics may arise through experiential limitations of the agent that stem from resource constraints and/or information‐processing (...) bounds. The article draws together recent work from a number of areas in judgment and decision making ranging from randomness perception (Hahn & Warren, ), information sampling (Hertwig & Pleskac, ; Kareev et al., ), and consequences of choice for exploration or exploitation (e.g., Denrell, ) to demonstrate how proper consideration of these deviations leads to reevaluation of behaviors that are otherwise deemed irrational. (shrink)

Legislation and ethical guidelines around the globe call for effective human oversight of AI-based systems in high-risk contexts – that is oversight that reliably reduces the risks otherwise associated with the use of AI-based systems. Such risks may relate to the imperfect accuracy of systems (e.g., inaccurate classifications) or to ethical concerns (e.g., unfairness of outputs). Given the significant role that human oversight is expected to play in the operation of AI-based systems, it is crucial to better understand the conditions (...) for effective human oversight. We argue that the reliable detection of errors (as an umbrella term for inaccuracies and unfairness) is crucial for effective human oversight. We then propose that Signal Detection Theory (SDT) offers a promising framework for better understanding what affects people’s sensitivity (i.e., how well they are able to detect errors) and response bias (i.e., the tendency to report errors given a perceived evidence of an error) in detecting errors. Whereas an SDT perspective on the detection of inaccuracies is straightforward, we demonstrate its broader applicability by detailing the specifics for an SDT perspective on unfairness detection, including the need to choose a standard for (un)fairness. Additionally, we illustrate that an SDT perspective helps to better understand the conditions for effective error detection by showing examples of task-, system-, and person-related factors that may affect the sensitivity and response bias of humans tasked with detecting unfairness associated with the use of AI-based systems. Finally, we discuss future research directions for an SDT perspective on error detection. (shrink)

Changing the information format from probabilities into frequencies as well as employing appropriate visualizations such as tree diagrams or 2 × 2 tables are important tools that can facilitate people’s statistical reasoning. Previous studies have shown that despite their widespread use in statistical textbooks, both of those visualization types are only of restricted help when they are provided with probabilities, but that they can foster insight when presented with frequencies instead. In the present study, we attempt to replicate this effect (...) and also examine, by the method of eye tracking, why probabilistic 2 × 2 tables and tree diagrams do not facilitate reasoning with regard to Bayesian inferences (i.e., determining what errors occur and whether they can be explained by scan paths), and why the same visualizations are of great help to an individual when they are combined with frequencies. All ten inferences of N = 24 participants were based solely on tree diagrams or 2 × 2 tables that presented either the famous “mammography context” or an “economics context” (without additional textual wording). We first asked participants for marginal, conjoint, and (non-inverted) conditional probabilities (or frequencies), followed by related Bayesian tasks. While solution rates were higher for natural frequency questions as compared to probability versions, eye-tracking analyses indeed yielded noticeable differences regarding eye movements between correct and incorrect solutions. For instance, heat maps (aggregated scan paths) of distinct results differed remarkably, thereby making correct and faulty strategies visible in the line of theoretical classifications. Moreover, the inherent structure of 2 × 2 tables seems to help participants avoid certain Bayesian mistakes (e.g., “Fisherian” error) while tree diagrams seem to help steer them away from others (e.g., “joint occurrence”). We will discuss resulting educational consequences at the end of the paper. (shrink)

Conflict can profoundly affect individuals and their groups. Oftentimes, conflict involves a clash between one side seeking change and increased gains through victory and the other side defending the status quo and protecting against loss and defeat. However, theory and empirical research largely neglected these conflicts between attackers and defenders, and the strategic, social, and psychological consequences of attack and defense remain poorly understood. To fill this void, we model (1) the clashing of attack and defense as games of strategy (...) and reveal that (2) attack benefits from mismatching its target's level of defense, whereas defense benefits from matching the attacker's competitiveness. This suggests that (3) attack recruits neuroendocrine pathways underlying behavioral activation and overconfidence, whereas defense invokes neural networks for behavioral inhibition, vigilant scanning, and hostile attributions; and that (4) people invest less in attack than defense, and attack often fails. Finally, we propose that (5) in intergroup conflict, out-group attack needs institutional arrangements that motivate and coordinate collective action, whereas in-group defense benefits from endogenously emerging in-group identification. We discuss how games of attack and defense may have shaped human capacities for prosociality and aggression, and how third parties can regulate such conflicts and reduce their waste. (shrink)

Corrigendum to "Exploration and exploitation during information search and consequential choice".

We present evidence supporting the target article's assertion that while the presentation of base-rate information in a natural frequency format can be helpful in enhancing sensitivity to base rates, method of presentation is not a panacea. Indeed, we review studies demonstrating that when subjects directly experience base rates as natural frequencies in a trial-by-trial setting, they evince large base-rate neglect.