An experiment was conducted to test the effectiveness of brief instruction in information structuring (i.e., representing and integrating information) for improving the coherence of probability judgments and binary choices among intelligence analysts. Forty-three analysts were presented with comparable sets of Bayesian judgment problems before and immediately after instruction. After instruction, analysts’ probability judgments were more coherent (i.e., more additive and compliant with Bayes theorem). Instruction also improved the coherence of binary choices regarding category membership: after instruction, subjects were more likely (...) to invariably choose the category to which they assigned the higher probability of a target’s membership. The research provides a rare example of evidence-based validation of effectiveness in instruction to improve the statistical assessment skills of intelligence analysts. Such instruction could also be used to improve the assessment quality of other types of experts who are required to integrate statistical information or make probabilistic assessments. (shrink)

The framework of elementary probabilistic operations (EPO) explains the structure of elementary probabilistic reasoning tasks as well as people’s performance on these tasks. The framework comprises three components: (a) Three types of probabilities: joint, marginal, and conditional probabilities; (b) three elementary probabilistic operations: combination, marginalization, and conditioning, and (c) quantitative inference schemas implementing the EPO. The formal part of the EPO framework is a computational level theory that provides a problem space representation and a classification of elementary probabilistic problems based (...) on computational requirements for solving a problem. According to the EPO framework, current methods for improving probabilistic reasoning are of two kinds: First, reduction of Bayesian problems to a type of probabilistic problems requiring less conceptual and procedural competencies. Second, enhancing people’s utilization competence by fostering the application of quantitative inference schemas. The approach suggests new applications, including the teaching of probabilistic reasoning, using analogical problem solving in probabilistic reasoning, and new methods for analyzing errors in probabilistic problem solving. (shrink)

Humans have long been characterized as poor probabilistic reasoners when presented with explicit numerical information. Bayesian word problems provide a well-known example of this, where even highly educated and cognitively skilled individuals fail to adhere to mathematical norms. It is widely agreed that natural frequencies can facilitate Bayesian reasoning relative to normalized formats (e.g. probabilities, percentages), both by clarifying logical set-subset relations and by simplifying numerical calculations. Nevertheless, between-study performance on “transparent” Bayesian problems varies widely, and generally remains rather unimpressive. (...) We suggest there has been an over-focus on this representational facilitator (i.e. transparent problem structures) at the expense of the specific logical and numerical processing requirements and the corresponding individual abilities and skills necessary for providing Bayesian-like output given specific verbal and numerical input. We further suggest that understanding this task-individual pair could benefit from considerations from the literature on mathematical cognition, which emphasizes text comprehension and problem solving, along with contributions of online executive working memory, metacognitive regulation, and relevant stored knowledge and skills. We conclude by offering avenues for future research aimed at identifying the stages in problem solving at which correct versus incorrect reasoners depart, and how individual difference might influence this time point. (shrink)

Bayesian reasoning, defined here as the updating of a posterior probability following new information, has historically been problematic for humans. Classic psychology experiments have tested human Bayesian reasoning through the use of word problems and have evaluated each participant’s performance against the normatively correct answer provided by Bayes’ theorem. The standard finding is of generally poor performance. Over the past two decades, though, progress has been made on how to improve Bayesian reasoning. Most notably, research has demonstrated that the use (...) of frequencies in a natural sampling framework —as opposed to single-event probabilities—can improve participants’ Bayesian estimates. Furthermore, pictorial aids and certain individual difference factors also can play significant roles in Bayesian reasoning success. The mechanics of how to build tasks which show these improvements is not under much debate. The explanations for why naturally-sampled frequencies and pictures help Bayesian reasoning remain hotly contested, however, with many researchers falling into ingrained “camps” organized around two dominant theoretical perspectives. The present paper evaluates the merits of these theoretical perspectives, including the weight of empirical evidence, theoretical coherence, and predictive power. By these criteria, the ecological rationality approach is clearly better than the heuristics and biases view. Progress in the study of Bayesian reasoning will depend on continued research that honestly, vigorously, and consistently engages across these different theoretical accounts rather than staying “siloed” within one particular perspective. The process of science requires an understanding of competing points of view, with the ultimate goal being integration. (shrink)

In the rationality debate, Gerd Gigerenzer and his colleagues have argued that human’s apparent inability to follow probabilistic principles does not mean our irrationality, because we can do probabilistic reasoning successfully if probability information is given in frequencies, not percentages (the natural frequency hypothesis). They also offered an evolutionary argument to this hypothesis, according to which using frequencies was evolutionarily more advantageous to our hominin ancestors than using percentages, and this is why we can reason correctly about probabilities in the (...) frequency format. This paper offers a critical review of this evolutionary argument. I show that there are reasons to believe using the frequency format was not more adaptive than using the standard (percentage) format. I also argue that there is a plausible alternative explanation (the nested-sets hypothesis) for the improved test performances of experimental subjects—one of Gigerenzer’s key explananda—which undermines the need to postulate mental mechanisms for probabilistic reasoning tuned to the frequency format. The explanatory thrust of the natural frequency hypothesis is much less significant than its advocates assume. (shrink)