Topics in Cognitive Science, Volume 14, Issue 3, Page 467-491, July 2022.

This paper aims to open a dialogue between philosophers working in decision theory and operations researchers and engineers working on decision-making under deep uncertainty. Specifically, we assess the recommendation to follow a norm of robust satisficing when making decisions under deep uncertainty in the context of decision analyses that rely on the tools of Robust Decision-Making developed by Robert Lempert and colleagues at RAND. We discuss two challenges for robust satisficing: whether the norm might derive its plausibility from an implicit (...) appeal to probabilistic representations of uncertainty of the kind that deep uncertainty is supposed to preclude; and whether there is adequate justification for adopting a satisficing norm, as opposed to an optimizing norm that is sensitive to considerations of robustness. We discuss decision-theoretic and voting-theoretic motivations for robust satisficing, and use these motivations to select among candidate formulations of the robust satisficing norm. (shrink)

I argue that bounded agents face a systematic accuracy-coherence tradeoff in cognition. Agents must choose whether to structure their cognition in ways likely to promote coherence or accuracy. I illustrate the accuracy-coherence tradeoff by showing how it arises out of at least two component tradeoffs: a coherence-complexity tradeoff between coherence and cognitive complexity, and a coherence-variety tradeoff between coherence and strategic variety. These tradeoffs give rise to an accuracy-coherence tradeoff because privileging coherence over complexity or strategic variety often leads to (...) a corresponding reduction in accuracy. I conclude with a discussion of two normative consequences for the study of bounded rationality: the importance of procedural rationality and the role of coherence in theories of bounded rationality. (shrink)

Research on “improper” linear models has shown that predetermined weighting schemes for the linear model, such as equally weighting all predictors, can be surprisingly accurate on cross-validation. We review recent advances that can characterize the optimal choice of an improper linear model. We extend this research to the understanding of fast and frugal heuristics, particularly to the ecologically rational goal of understanding in which task environments given heuristics are optimal. We demonstrate how to test this model using the Recognition Heuristic (...) and Take the Best heuristic, show how the model reconciles with the ecological rationality program, and discuss how our prescriptive, computational approach could be approximated by simpler mental rules that might be more descriptive. Echoing the arguments of van Rooij et al., we stress the virtue of having a computationally tractable model of strategy selection, even if one proposes that cognizers use a simpler heuristic process to approximate it. (shrink)

Cognitive scientists have revealed systematic errors in human reasoning. There is disagreement about what these errors indicate about human rationality, but one upshot seems clear: human reasoning does not seem to fit traditional views of human rationality. This concern about rationality has made its way through various fields and has recently caught the attention of philosophers. The concern is that if philosophers are prone to systematic errors in reasoning, then the integrity of philosophy would be threatened. In this paper, I (...) present some of the more famous work in cognitive science that has marshaled this concern. Then I present reasons to think that those with training in philosophy will be less prone to certain systematic errors in reasoning. The suggestion is that if philosophers could be shown to be less prone to such errors, then the worries about the integrity of philosophy could be constrained. Then I present evidence that, according to performance on the CRT (Frederick 2005), those who have benefited from training and selection in philosophy are indeed less prone to one kind of systematic error: irrationally arbitrating between intuitive and reflective responses. Nonetheless, philosophers are not entirely immune to this systematic error, and their proclivity for this error is statistically related to their responses to a variety of philosophical questions. So, while the evidence herein puts constraints on the worries about the integrity of philosophy, it by no means eliminates these worries. The conclusion, then, is that the present evidence offers prima facie reasons to ascribe a mitigated privilege to philosophers' ability to rationally arbitrate between intuitive and reflective responses. (shrink)

Social epistemology is autonomous: When applied to the same evidential situations, the principles of social rationality and the principles of individual rationality sometimes recommend inconsistent beliefs. If we stipulate that reasoning rationally from justified beliefs to a true belief is normally sufficient for knowledge, the autonomy thesis implies that some knowledge is essentially social. When the principles of social and individual rationality are applied to justified evidence and recommend inconsistent beliefs and the belief endorsed by social rationality is true, then (...) that true belief would be an instance of social knowledge but not individual knowledge. (shrink)

Behavior that solves a problem is distinguished by the fact that it changes another part of the solver's behavior and is strengthened when it does so. Problem solving typically involves the construction of discriminative stimuli. Verbal responses produce especially useful stimuli, because they affect other people. As a culture formulates maxims, laws, grammar, and science, its members behave more effectively without direct or prolonged contact with the contingencies thus formulated. The culture solves problems for its members, and does so by (...) transmitting the verbal discriminative stimuli called rules. Induction, deduction, and the construction of models are ways of producing rules. Behavior that solves a problem may result from direct shaping by contingencies or from rules constructed either by the problem solver or by others. Because different controlling variables are involved, contingency-shaped behavior is never exactly like rule-governed behavior. The distinction must take account of a system which establishes certain contingencies of reinforcement, such as some part of the natural environment, a piece of equipment, or a verbal community; the behavior shaped and maintained by these contingencies; rules, derived from the contingencies, which specify discriminative stimuli, responses, and consequences, and the behavior occasioned by the rules. (shrink)

The prominence of Bayesian modeling of cognition has increased recently largely because of mathematical advances in specifying and deriving predictions from complex probabilistic models. Much of this research aims to demonstrate that cognitive behavior can be explained from rational principles alone, without recourse to psychological or neurological processes and representations. We note commonalities between this rational approach and other movements in psychology – namely, Behaviorism and evolutionary psychology – that set aside mechanistic explanations or make use of optimality assumptions. Through (...) these comparisons, we identify a number of challenges that limit the rational program's potential contribution to psychological theory. Specifically, rational Bayesian models are significantly unconstrained, both because they are uninformed by a wide range of process-level data and because their assumptions about the environment are generally not grounded in empirical measurement. The psychological implications of most Bayesian models are also unclear. Bayesian inference itself is conceptually trivial, but strong assumptions are often embedded in the hypothesis sets and the approximation algorithms used to derive model predictions, without a clear delineation between psychological commitments and implementational details. Comparing multiple Bayesian models of the same task is rare, as is the realization that many Bayesian models recapitulate existing (mechanistic level) theories. Despite the expressive power of current Bayesian models, we argue they must be developed in conjunction with mechanistic considerations to offer substantive explanations of cognition. We lay out several means for such an integration, which take into account the representations on which Bayesian inference operates, as well as the algorithms and heuristics that carry it out. We argue this unification will better facilitate lasting contributions to psychological theory, avoiding the pitfalls that have plagued previous theoretical movements. (shrink)

The notion of ecological rationality implies that the accuracy of a decision strategy depends on features of the information environment in which it is tested. We demonstrate that the performance of a group may be strongly affected by the decision strategies used by its individual members and specify how this effect is moderated by environmental features. Specifically, in a set of simulation studies, we systematically compared four decision strategies used by the individual group members: two linear, compensatory decision strategies and (...) two simple, noncompensatory heuristics. Individual decisions were aggregated by using a majority rule. To assess the ecological rationality of the strategies, we varied (a) the distribution of cue validities, (b) the quantity, and (c) the quality of shared information. Group performance strongly depended on the distribution of cue validities. When validities were linearly distributed, groups using a compensatory strategy achieved the highest accuracy. Conversely, when cue validities followed a J-shaped distribution, groups using a simple lexicographic heuristic performed best. While these effects were robust across different quantities of shared information, the quality of shared information exerted stronger effects on group performance. Consequences for prescriptive theories on group decision making are discussed. (shrink)

One recent priority of the U.S. government is developing autonomous robotic systems. The U.S. Army has funded research to design a metric of evil to support military commanders with ethical decision-making and, in the future, allow robotic military systems to make autonomous ethical judgments. We use this particular project as a case study for efforts that seek to frame morality in quantitative terms. We report preliminary results from this research, describing the assumptions and limitations of a program that assesses the (...) relative evil of two courses of action. We compare this program to other attempts to simulate ethical decision-making, assess possibilities for overcoming the trade-off between input simplification and output reliability, and discuss the responsibilities of users and designers in implementing such programs. We conclude by discussing the implications that this project highlights for the successes and challenges of developing automated mechanisms for ethical decision making. (shrink)

The paper shows why and how an empirical study of fast-and-frugal heuristics can provide norms of good reasoning, and thus how (and how far) rationality can be naturalized. We explain the heuristics that humans often rely on in solving problems, for example, choosing investment strategies or apartments, placing bets in sports, or making library searches. We then show that heuristics can lead to judgments that are as accurate as or even more accurate than strategies that use more information and computation, (...) including optimization methods. A standard way to defend the use of heuristics is by reference to accuracy-effort trade-offs. We take a different route, emphasizing ecological rationality (the relationship between cognitive heuristics and environment), and argue that in uncertain environments, more information and computation are not always better (the “less-can-be-more” doctrine). The resulting naturalism about rationality is thus normative because it not only describes what heuristics people use, but also in which specific environments one should rely on a heuristic in order to make better inferences. While we desist from claiming that the scope of ecological rationality is unlimited, we think it is of wide practical use. (shrink)

Structural equation modelling (SEM) is outlined and compared with two non-linear alternatives, artificial neural networks and ''fast and frugal'' models. One particular non-linear decision-making situation is discussed, that exemplified by a lexicographic semi-order. We illustrate the use of SEM on a dataset derived from 539 volunteers' responses to questions about food-related risks. Our conclusion is that SEM is a useful member of the armoury of techniques available to the student of human judgement: it subsumes several multivariate statistical techniques and permits (...) their flexible combination, and it provides robust goodness-of-fit statistics and is available in (generally) easy-to-use computer packages. Although the number of tasks for which SEM provides a persuasive psychological model is small, it is very useful in identifying the important variables and their inter-relations that contribute to task performance, and thus can constitute a valuable intermediate staging point between raw data and a fully fledged psychological theory. (shrink)

How can anyone be rational in a world where knowledge is limited, time is pressing, and deep thought is often an unattainable luxury? Traditional models of unbounded rationality and optimization in cognitive science, economics, and animal behavior have tended to view decision-makers as possessing supernatural powers of reason, limitless knowledge, and endless time. But understanding decisions in the real world requires a more psychologically plausible notion of bounded rationality. In Simple heuristics that make us smart (Gigerenzer et al. 1999), we (...) explore fast and frugal heuristics – simple rules in the mind's adaptive toolbox for making decisions with realistic mental resources. These heuristics can enable both living organisms and artificial systems to make smart choices quickly and with a minimum of information by exploiting the way that information is structured in particular environments. In this précis, we show how simple building blocks that control information search, stop search, and make decisions can be put together to form classes of heuristics, including: ignorance-based and one-reason decision making for choice, elimination models for categorization, and satisficing heuristics for sequential search. These simple heuristics perform comparably to more complex algorithms, particularly when generalizing to new data – that is, simplicity leads to robustness. We present evidence regarding when people use simple heuristics and describe the challenges to be addressed by this research program. Key Words: adaptive toolbox; bounded rationality; decision making; elimination models; environment structure; heuristics; ignorance-based reasoning; limited information search; robustness; satisficing; simplicity. (shrink)

Assuming it rational to treat other persons' probabilities as epistemically significant, how shall their judgements be weighted (Barnes [1998])? Several plausible methods exist, but theorems in classical psychometrics greatly reduce the importance of the problem. If scientists' judgements tend to be positively correlated, the difference between two randomly weighted composites shrinks as the number of judges rises. Since, for reasons such as representative coverage, minimizing bias, and avoiding elitism, we would rarely employ small numbers of judges (e.g. less than 10), (...) the difference between two weighting systems becomes negligible. Suggestions are made for quantifying verisimilitude, identifying 'types' of scientists or theories (taxometrics), inferring latent factors, and estimating reliability of pooled judgements. (shrink)

Epistemic responsibility involves at least two central ideas. (V) To be epistemically responsible is to display the virtue(s) epistemic internalists take to be central to justification (e.g., coherence, having good reasons, fitting the evidence). (C) In normal (non-skeptical)circumstances and in thelong run, epistemic responsibility is strongly positively correlated with reliability. Sections 1 and 2 review evidence showing that for a wide range of real-world problems, the most reliable, tractable reasoning strategies audaciously flout the internalist''s epistemic virtues. In Section 3, I (...) argue that these results force us to give up either (V), our current conception of what it is to be epistemically responsible, or (C) the responsibility-reliability connection. I will argue that we should relinquish (V). This is likely to reshape our epistemic practices. It will force us to alter our epistemic judgments about certain instances of reasoning, to endorse some counterintuitive epistemic prescriptions, and to rethink what it is for cognitive agents to be epistemically responsible. (shrink)

Our aim in this paper is to bring the woefully neglected literature on predictive modeling to bear on some central questions in the philosophy of science. The lesson of this literature is straightforward: For a very wide range of prediction problems, statistical prediction rules (SPRs), often rules that are very easy to implement, make predictions than are as reliable as, and typically more reliable than, human experts. We will argue that the success of SPRs forces us to reconsider our views (...) about what is involved in understanding, explanation, good reasoning, and about how we ought to do philosophy of science. (shrink)

The object of this paper is to show why recent research in the psychology of deductive and probabilistic reasoning does not have.

Individual differences in decisiveness: pre-decisional information search and decision strategy use We investigated whether individual differences in decisiveness are associated with a tendency to use different decision strategies during pre-decisional information search. To explore these potential links we administered the Need for Cognitive Closure questionnaire to 62 participants, followed by a probabilistic inference, multi-attribute choice task. Participants high in decisiveness dimension, compared to ‘low decisives’, spent less time and acquired less information prior to making decisions, especially in the first trials (...) of the choice task. ‘High decisives’ also had a greater tendency to use a simple Lexicographic heuristic than ‘low decisives’. The results support the view that high decisiveness is associated with greater tendency to simplify the decision process. (shrink)

This article provides an overview of recent results on lexicographic, linear, and Bayesian models for paired comparison from a cognitive psychology perspective. Within each class, we distinguish subclasses according to the computational complexity required for parameter setting. We identify the optimal model in each class, where optimality is defined with respect to performance when fitting known data. Although not optimal when fitting data, simple models can be astonishingly accurate when generalizing to new data. A simple heuristic belonging to the class (...) of lexicographic models is Take The Best (Gigerenzer & Goldstein (1996) Psychol. Rev. 102: 684). It is more robust than other lexicographic strategies which use complex procedures to establish a cue hierarchy. In fact, it is robust due to its simplicity, not despite it. Similarly, Take The Best looks up only a fraction of the information that linear and Bayesian models require; yet it achieves performance comparable to that of models which integrate information. Due to its simplicity, frugality, and accuracy, Take The Best is a plausible candidate for a psychological model in the tradition of bounded rationality. We review empirical evidence showing the descriptive validity of fast and frugal heuristics. (shrink)

Statistical procedures can be applied to episodes in the history of science in order to weight attributes to predict short-term survival of theories; an asymptotic method is used to show that short-term survival is a valid proxy for ultimate survival; and a theoretical argument is made that ultimate survival is a valid proxy for objective truth. While realists will appreciate this last step, instrumentalists do not need it to benefit from the actuarial procedures of cliometric metatheory. Introduction A plausible proxy (...) for Peircean consensus Assessing the validity of theory attributes as predictors of theory survival 3.1 Linear discriminant function 3.2 Factor analysis 3.3 Taxometric analysis Verisimilitude index Satisfying both instrumentalists and realists Recapitulation Implementation of cliometric metatheory * Correspondence about this article may be addressed to Leslie Yonce at pemeehle{at}umn.edu' + u + '@' + d + ''//--> This article had been completed by Paul Meehl at the time of his death on 14 February 2003. His wife, Leslie J. Yonce, is grateful to Keith Gunderson (University of Minnesota, Center for Philosophy of Science) and Niels G. Waller (Psychology Department, Vanderbilt University) for advice with some final editing details. (shrink)

In this article we describe research methods that are used for the study of individual multiattribute evaluation processes. First we explain that a multiattribute evaluation problem involves the evaluation of a set of alternatives, described by their values on a number of alternatives. We discuss a number of evaluation strategies that may be applied to arrive at a conclusion about the attractiveness or suitability of the alternatives, and next introduce two main research paradigms in this area, structural modelling and process (...) tracing. We argue that the techniques developed within these paradigms all have their advantages and disadvantages, and conclude that the most promising technique to detect the true nature of the evaluation strategy used by a judge seems to be the analysis of verbal protocols. At the same time we think it is wise not to rely on just one technique, but to use a multimethod approach to the study of multiattribute evaluation processes whenever that is possible. (shrink)

A working assumption that processes of natural and cultural evolution have tailored the mind to fit the demands and structure of its environment begs the question: how are we to characterize the structure of cognitive environments? Decision problems faced by real organisms are not like simple multiple-choice examination papers. For example, some individual problems may occur much more frequently than others, whilst some may carry much more weight than others. Such considerations are not taken into account when (i) the performance (...) of candidate cognitive mechanisms is assessed by employing a simple accuracy metric that is insensitive to the structure of the decision-maker's environment, and (ii) reason is defined as the adherence to internalist prescriptions of classical rationality. Here we explore the impact of frequency and significance structure on the performance of a range of candidate decision-making mechanisms. We show that the character of this impact is complex, since structured environments demand that decision-makers trade off general performance against performance on important subsets of test items. As a result, environment structure obviates internalist criteria of rationality. Failing to appreciate the role of environment structure in shaping cognition can lead to mischaracterising adaptive behavior as irrational. (shrink)

Charles Taber and Milton Lodge provide compelling evidence that people's minds may be closed to information that is inconsistent with their prior beliefs. This type of inconsistency has often been termed ?irrational.? However, recent research suggests that being open or closed minded is not an unchanging variable but depends on one's goals, including one's need for closure, which vary from person to person and situation to situation. In this vein, as Taber and Lodge suggest, those who have more political information (...) may, after having been open minded enough to acquire the information, become closed minded by virtue of having acquired it. At the same time, being more open minded to inaccurate information might lead one in the wrong direction; hence, open mindedness does not necessarily enhance the accuracy of one's judgments or the quality of one's decisions. We may need to rethink the concept of an enlightened, well-informed electorate, to the extent that it assumes the unqualified benefits of open mindedness. (shrink)

Heuristics are efficient cognitive processes that ignore information. In contrast to the widely held view that less processing reduces accuracy, the study of heuristics shows that less information, computation, and time can in fact improve accuracy. We review the major progress made so far: the discovery of less-is-more effects; the study of the ecological rationality of heuristics, which examines in which environments a given strategy succeeds or fails, and why; an advancement from vague labels to computational models of heuristics; the (...) development of a systematic theory of heuristics that identifies their building blocks and the evolved capacities they exploit, and views the cognitive system as relying on an “adaptive toolbox;” and the development of an empirical methodology that accounts for individual differences, conducts competitive tests, and has provided evidence for people’s adaptive use of heuristics. Homo heuristicus has a biased mind and ignores part of the available information, yet a biased mind can handle uncertainty more efficiently and robustly than an unbiased mind relying on more resource-intensive and general-purpose processing strategies. (shrink)