No scientific conclusion follows automatically from a statistically non-significant result, yet people routinely use non-significant results to guide conclusions about the status of theories (or the effectiveness of practices). To know whether a non-significant result counts against a theory, or if it just indicates data insensitivity, researchers must use one of: power, intervals (such as confidence or credibility intervals), or else an indicator of the relative evidence for one theory over another, such as a Bayes factor. I argue Bayes factors (...) allow theory to be linked to data in a way that overcomes the weaknesses of the other approaches. Specifically, Bayes factors use the data themselves to determine their sensitivity in distinguishing theories (unlike power), and they make use of those aspects of a theory’s predictions that are often easiest to specify (unlike power and intervals, which require specifying the minimal interesting value in order to address theory). Bayes factors provide a coherent approach to determining whether non-significant results support a null hypothesis over a theory, or whether the data are just insensitive. They allow accepting and rejecting the null hypothesis to be put on an equal footing. Concrete examples are provided to indicate the range of application of a simple online Bayes calculator, which reveal both the strengths and weaknesses of Bayes factors. (shrink)

Heuristics are efficient cognitive processes that ignore information. In contrast to the widely held view that less processing reduces accuracy, the study of heuristics shows that less information, computation, and time can in fact improve accuracy. We review the major progress made so far: the discovery of less-is-more effects; the study of the ecological rationality of heuristics, which examines in which environments a given strategy succeeds or fails, and why; an advancement from vague labels to computational models of heuristics; the (...) development of a systematic theory of heuristics that identifies their building blocks and the evolved capacities they exploit, and views the cognitive system as relying on an “adaptive toolbox;” and the development of an empirical methodology that accounts for individual differences, conducts competitive tests, and has provided evidence for people’s adaptive use of heuristics. Homo heuristicus has a biased mind and ignores part of the available information, yet a biased mind can handle uncertainty more efficiently and robustly than an unbiased mind relying on more resource-intensive and general-purpose processing strategies. (shrink)

Preregistration has been proposed as a useful method for making a publicly verifiable distinction between confirmatory hypothesis tests, which involve planned tests of ante hoc hypotheses, and exploratory hypothesis tests, which involve unplanned tests of post hoc hypotheses. This distinction is thought to be important because it has been proposed that confirmatory hypothesis tests provide more compelling results (less uncertain, less tentative, less open to bias) than exploratory hypothesis tests. In this article, we challenge this proposition and argue that there (...) are several advantages of exploratory hypothesis tests that can make their results more compelling than those of confirmatory hypothesis tests. We also consider some potential disadvantages of exploratory hypothesis tests and conclude that their advantages can outweigh the disadvantages. We conclude that exploratory hypothesis tests avoid researcher commitment and researcher prophecy biases, reduce the probability of data fraud, are more appropriate in the context of unplanned deviations, facilitate inference to the best explanation, and allow peer reviewers to make additional contributions at the data analysis stage. In contrast, confirmatory hypothesis tests may lead to an inappropriate level of confidence in research conclusions, less appropriate analyses in the context of unplanned deviations, and greater bias and errors in theoretical inferences. (shrink)

The predictive processing account of action, cognition, and perception is one of the most influential approaches to unifying research in cognitive science. However, its promises of grand unification will remain unfulfilled unless the account becomes theoretically robust. In this paper, we focus on empirical commitments of PP, since they are necessary both for its theoretical status to be established and for explanations of individual phenomena to be falsifiable. First, we argue that PP is a varied research tradition, which may employ (...) various kinds of scientific representations, differing in the scope of empirical commitments they entail. Two major perspectives on PP qua cognitive theory may then be distinguished: generalized vs. hierarchical. The first one fails to provide empirical detail, and the latter constrains possible physical implementations. However, we show that even hierarchical PP is insufficiently restrictive to disallow incorrect models and may be adjusted to explain any neurocognitive phenomenon–including non-existent or impossible ones–through flexible adjustments. This renders PP a universal modeling tool with an unrestricted number of degrees of freedom. Therefore, in contrast with declarations of its proponents, it should not be understood as a unifying theoretical perspective, but as a computational framework, possibly informing further theory development in cognitive science. (shrink)

Cognitive science is an interdisciplinary conglomerate of various research fields and disciplines, which increases the risk of fragmentation of cognitive theories. However, while most previous work has focused on theoretical integration, some kinds of integration may turn out to be monstrous, or result in superficially lumped and unrelated bodies of knowledge. In this paper, I distinguish theoretical integration from theoretical unification, and propose some analyses of theoretical unification dimensions. Moreover, two research strategies that are supposed to lead to unification are (...) analyzed in terms of the mechanistic account of explanation. Finally, I argue that theoretical unification is not an absolute requirement from the mechanistic perspective, and that strategies aiming at unification may be premature in fields where there are multiple conflicting explanatory models. (shrink)

We present a comprehensive empirical evaluation of the ACT‐R–based model of sentence processing developed by Lewis and Vasishth (2005) (LV05). The predictions of the model are compared with the results of a recent meta‐analysis of published reading studies on retrieval interference in reflexive‐/reciprocal‐antecedent and subject–verb dependencies (Jäger, Engelmann, & Vasishth, 2017). The comparison shows that the model has only partial success in explaining the data; and we propose that its prediction space is restricted by oversimplifying assumptions. We then implement a (...) revised model that takes into account differences between individual experimental designs in terms of the prominence of the target and the distractor in memory‐ and context‐dependent cue‐feature associations. The predictions of the original and the revised model are quantitatively compared with the results of the meta‐analysis. Our simulations show that, compared to the original LV05 model, the revised model accounts for the data better. The results suggest that effects of prominence and variable cue‐feature associations need to be considered in the interpretation of existing empirical results and in the design and planning of future experiments. With regard to retrieval interference in sentence processing and to the broader field of psycholinguistic studies, we conclude that well‐specified models in tandem with high‐powered experiments are needed in order to uncover the underlying cognitive processes. (shrink)

I consider the role of cognitive modeling in cognitive science. Modeling, and the computers that enable it, are central to the field, but the role of modeling is often misunderstood. Models are not intended to capture fully the processes they attempt to elucidate. Rather, they are explorations of ideas about the nature of cognitive processes. In these explorations, simplification is essential—through simplification, the implications of the central ideas become more transparent. This is not to say that simplification has no downsides; (...) it does, and these are discussed. I then consider several contemporary frameworks for cognitive modeling, stressing the idea that each framework is useful in its own particular ways. Increases in computer power (by a factor of about 4 million) since 1958 have enabled new modeling paradigms to emerge, but these also depend on new ways of thinking. Will new paradigms emerge again with the next 1,000‐fold increase? (shrink)

Information generally comes from less than fully reliable sources. Rationality, it seems, requires that one take source reliability into account when reasoning on the basis of such information. Recently, Bovens and Hartmann (2003) proposed an account of the conjunction fallacy based on this idea. They show that, when statements in conjunction fallacy scenarios are perceived as coming from such sources, probability theory prescribes that the “fallacy” be committed in certain situations. Here, the empirical validity of their model was assessed. The (...) model predicts that statements added to standard conjunction problems will change the incidence of the fallacy. It also predicts that statements from reliable sources should yield an increase in fallacy rates (relative to unreliable sources). Neither the former (Experiment 1) nor the latter prediction (Experiment 3) was confirmed, although Experiment 2 showed that people can derive source reliability estimates from the likelihood of statements in a manner consistent with the tested model. In line with the experimental results, model fits and sensitivity analyses also provided very little evidence in favor of the model. This suggests that Bovens and Hartmann’s present model fails to explain fully people’s judgements in standard conjunction fallacy tasks. (shrink)

In this prcis we focus on phenomena central to the reaction against similarity-based theories that arose in the 1980s and that subsequently motivated the approach to semantic knowledge. Specifically, we consider (1) how concepts differentiate in early development, (2) why some groupings of items seem to form or coherent categories while others do not, (3) why different properties seem central or important to different concepts, (4) why children and adults sometimes attest to beliefs that seem to contradict their direct experience, (...) (5) how concepts reorganize between the ages of 4 and 10, and (6) the relationship between causal knowledge and semantic knowledge. The explanations our theory offers for these phenomena are illustrated with reference to a simple feed-forward connectionist model. The relationships between this simple model, the broader theory, and more general issues in cognitive science are discussed. (shrink)

PROBEX (PROBabilities from EXemplars), a model of probabilistic inference and probability judgment based on generic knowledge is presented. Its properties are that: (a) it provides an exemplar model satisfying bounded rationality; (b) it is a “lazy” algorithm that presumes no pre‐computed abstractions; (c) it implements a hybrid‐representation, similarity‐graded probability. We investigate the ecological rationality of PROBEX and find that it compares favorably with Take‐The‐Best and multiple regression (Gigerenzer, Todd, & the ABC Research Group, 1999). PROBEX is fitted to the point (...) estimates, decisions, and probability assessments by human participants. The best fit is obtained for a version that weights frequency heavily and retrieves only two exemplars. It is proposed that PROBEX implements speed and frugality in a psychologically plausible way. (shrink)

The article first addresses the importance of cognitive modeling, in terms of its value to cognitive science (as well as other social and behavioral sciences). In particular, it emphasizes the use of cognitive architectures in this undertaking. Based on this approach, the article addresses, in detail, the idea of a multi-level approach that ranges from social to neural levels. In physical sciences, a rigorous set of theories is a hierarchy of descriptions/explanations, in which causal relationships among entities at a high (...) level can be reduced to causal relationships among simpler entities at a more detailed level. We argue that a similar hierarchy makes possible an equally productive approach toward cognitive modeling. The levels of models that we conceive in relation to cognition include, at the highest level, sociological/anthropological models of collective human behavior, behavioral models of individual performance, cognitive models involving detailed mechanisms, representations, and processes, as well as biological/physiological models of neural circuits, brain regions, and other detailed biological processes. (shrink)

Among theories of human language comprehension, cue‐based memory retrieval has proven to be a useful framework for understanding when and how processing difficulty arises in the resolution of long‐distance dependencies. Most previous work in this area has assumed that very general retrieval cues like [+subject] or [+singular] do the work of identifying (and sometimes misidentifying) a retrieval target in order to establish a dependency between words. However, recent work suggests that general, handpicked retrieval cues like these may not be enough (...) to explain illusions of plausibility (Cunnings & Sturt, 2018), which can arise in sentences like The letter next to the porcelain plate shattered. Capturing such retrieval interference effects requires lexically specific features and retrieval cues, but handpicking the features is hard to do in a principled way and greatly increases modeler degrees of freedom. To remedy this, we use well‐established word embedding methods for creating distributed lexical feature representations that encode information relevant for retrieval using distributed retrieval cue vectors. We show that the similarity between the feature and cue vectors (a measure of plausibility) predicts total reading times in Cunnings and Sturt’s eye‐tracking data. The features can easily be plugged into existing parsing models (including cue‐based retrieval and self‐organized parsing), putting very different models on more equal footing and facilitating future quantitative comparisons. (shrink)

This chapter explores a way in which visual processing may involve innate constraints and attempts to show how such processing overcomes one enduring challenge to nativism. In particular, many challenges to nativist theories in other areas of cognitive psychology have focused on the later development of such abilities, and have argued that such development is in conflict with innate origins. Innateness, in these contexts, is seen as antidevelopmental, associated instead with static processes and principles. In contrast, certain perceptual models demonstrate (...) how the very same mental processes can both be innately specified and yet develop richly in response to experience with the environment. This process is entirely unmysterious, as shown in certain formal theories of visual perception, including those that appeal to spontaneous endogenous stimulation and those based on Bayesian inference. (shrink)

This paper is aimed at identifying how a model’s explanatory power is constructed and identified, particularly in the practice of template-based modeling (Humphreys, Philos Sci 69:1–11, 2002; Extending ourselves: computational science, empiricism, and scientific method, 2004), and what kinds of explanations models constructed in this way can provide. In particular, this paper offers an account of non-causal structural explanation that forms an alternative to causal–mechanical accounts of model explanation that are currently popular in philosophy of biology and cognitive science. Clearly, (...) defences of non-causal explanation are far from new (e.g. Batterman, Br J Philos Sci 53:21–38, 2002a; The devil in the details: asymptotic reasoning in explanation, reduction, and emergence, 2002b; Pincock, Noûs 41:253–275, 2007; Mathematics and scientific representation 2012; Rice, Noûs. doi:10.1111/nous.12042, 2013; Biol Philos 27:685–703, 2012), so the targets here are focused on a particular type of robust phenomenon and how strong invariance to interventions can block a range of causal explanations. By focusing on a common form of model construction, the paper also ties functional or computational style explanations found in cognitive science and biology more firmly with explanatory practices across model-based science in general. (shrink)

Cognitive architectures have often been applied to data from individual experiments. In this paper, I develop an ACT-R reader that can model a much larger set of data, eye-tracking corpus data. It is shown that the resulting model has a good fit to the data for the considered low-level processes. Unlike previous related works, the model achieves the fit by estimating free parameters of ACT-R using Bayesian estimation and Markov-Chain Monte Carlo techniques, rather than by relying on the mix of (...) manual selection + default values. The method used in the paper is generalizable beyond this particular model and data set and could be used on other ACT-R models. (shrink)

This article explores the view that computational models of cognition may constitute valid theories of cognition, often in the full sense of the term ‘‘theory”. In this discussion, this article examines various (existent or possible) positions on this issue and argues in favor of the view above. It also connects this issue with a number of other relevant issues, such as the general relationship between theory and data, the validation of models, and the practical benefits of computational modeling. All the (...) discussions point to the position that computational cognitive models can be true theories of cognition. Ó 2008 Elsevier B.V. All rights reserved. (shrink)

The article first addresses the importance of cognitive modeling, in terms of its value to cognitive science (as well as other social and behavioral sciences). In particular, it emphasizes the use of cognitive architectures in this undertaking. Based on this approach, the article addresses, in detail, the idea of a multi-level approach that ranges from social to neural levels. In physical sciences, a rigorous set of theories is a hierarchy of descriptions/explanations, in which causal relationships among entities at a high (...) level can be reduced to causal relationships among simpler entities at a more detailed level. We argue that a similar hierarchy makes possible an equally productive approach toward cognitive modeling. The levels of models that we conceive in relation to cognition include, at the highest level, sociological/anthropological models of collective human behavior, behavioral models of individual performance, cognitive models involving detailed mechanisms, representations, and processes, as well as biological/physiological models of neural circuits, brain regions, and other detailed biological processes. (shrink)

Residual fluctuations produced in typical experimental methodologies are examined as correlated noises. The effective range of the correlations was assessed by determining whether the decay over look‐back time is better described as a power law or exponential. Both of these decay laws contain free parameters and it is argued that it is not possible to distinguish their models on the basis of simple measures of goodness‐of‐fit. Global analyses that evaluate models on the basis of how well they generalize are conducted. (...) The models are examined in terms of three constructs that all bear on generalization: cross‐validity, flexibility, and representativeness. Quantitative assessment of a large ensemble of data suggests that the correlations decay over time as a power law. The conclusion is that human residual fluctuation is a correlated fractal. (shrink)

It has been suggested that the enterprise of developing mechanistic theories of the human cognitive architecture is flawed because the theories produced are not directly falsifiable. Newell attempted to sidestep this criticism by arguing for a Lakatosian model of scientific progress in which cognitive architectures should be understood as theories that develop over time. However, Newell's own candidate cognitive architecture adhered only loosely to Lakatosian principles. This paper reconsiders the role of falsification and the potential utility of Lakatosian principles in (...) the development of cognitive architectures. It is argued that a lack of direct falsifiability need not undermine the scientific development of a cognitive architecture if broadly Lakatosian principles are adopted. Moreover, it is demonstrated that the Lakatosian concepts of positive and negative heuristics for theory development and of general heuristic power offer methods for guiding the development of an architecture and for evaluating the contribution and potential of an architecture's research program. (shrink)

Weisberg (2006) and Godfrey-Smith (2006, 2009) distinguish between two forms of theorising: data-driven ‘abstract direct representation’ and modeling. The key difference is that when using a data-driven approach, theories are intended to represent specific phenomena, so directly represent them, while models may not be intended to represent anything, so represent targets indirectly, if at all. The aim here is to compare and analyse these practices, in order to outline an account of model-based theorising that involves direct representational relationships. This is (...) based on the way that computational templates Humphreys (2002, 2004) are now used in cognitive neuroscience, and draws on the dynamic and tentative process of any kind of theory construction, and the idea of partial, purpose-relative representation. (shrink)

Knauff and Gazzo Castañeda (2022) criticise the use of the term “new paradigm” in the psychology of reasoning and raise important issues about how to advance research in the field. In this commentary I argue that for the latter it would be helpful to clarify further the concepts that reasoning theories rely on, and to strengthen the links between the theories and the empirical observations that would and would not be compatible with them.

Cognitive architectures are theories of cognition that try to capture the essential representations and mechanisms that underlie cognition. Research in cognitive architectures has gradually moved from a focus on the functional capabilities of architectures to the ability to model the details of human behavior, and, more recently, brain activity. Although there are many different architectures, they share many identical or similar mechanisms, permitting possible future convergence. In judging the quality of a particular cognitive model, it is pertinent to not just (...) judge its fit to the experimental data but also its simplicity and ability to make predictions. (shrink)

The spacing effect is among the most widely replicated empirical phenomena in the learning sciences, and its relevance to education and training is readily apparent. Yet successful applications of spacing effect research to education and training is rare. Computational modeling can provide the crucial link between a century of accumulated experimental data on the spacing effect and the emerging interest in using that research to enable adaptive instruction. In this paper, we review relevant literature and identify 10 criteria for rigorously (...) evaluating computational models of the spacing effect. Five relate to evaluating the theoretic adequacy of a model, and five relate to evaluating its application potential. We use these criteria to evaluate a novel computational model of the spacing effect called the Predictive Performance Equation. Predictive Performance Equation combines elements of earlier models of learning and memory including the General Performance Equation, Adaptive Control of Thought—Rational, and the New Theory of Disuse, giving rise to a novel computational account of the spacing effect that performs favorably across the complete sets of theoretic and applied criteria. We implemented two other previously published computational models of the spacing effect and compare them to PPE using the theoretic and applied criteria as guides. (shrink)

We consider human performance on an optimal stopping problem where people are presented with a list of numbers independently chosen from a uniform distribution. People are told how many numbers are in the list, and how they were chosen. People are then shown the numbers one at a time, and are instructed to choose the maximum, subject to the constraint that they must choose a number at the time it is presented, and any choice below the maximum is incorrect. We (...) present empirical evidence that suggests people use threshold-based models to make decisions, choosing the first currently maximal number that exceeds a fixed threshold for that position in the list. We then develop a hierarchical generative account of this model family, and use Bayesian methods to learn about the parameters of the generative process, making inferences about the threshold decision models people use. We discuss the interesting aspects of human performance on the task, including the lack of learning, and the presence of large individual differences, and consider the possibility of extending the modeling framework to account for individual differences. We also use the modeling results to discuss the merits of hierarchical, generative and Bayesian models of cognitive processes more generally. (shrink)

When people are performing a task, it is hard to know whether they are about to make a mistake. Klaproth, Halbrügge, Krol, Vernaleken, Zander, and Russwinkel address this by recording EEG signals while people are performing a flight control task, and show that by examining the EEG signal they can determine when people failed to notice particular stimuli, which could lead to better assistive tools.

Contemporary methods of computational cognitive modeling have recently been criticized by Addyman and French (2012) on the grounds that they have not kept up with developments in computer technology and human–computer interaction. They present a manifesto for change according to which, it is argued, modelers should devote more effort to making their models accessible, both to non-modelers (with an appropriate easy-to-use user interface) and modelers alike. We agree that models, like data, should be freely available according to the normal standards (...) of science, but caution against confusing implementations with specifications. Models may embody theories, but they generally also include implementation assumptions. Cognitive modeling methodology needs to be sensitive to this. We argue that specification, replication and experimentation are methodological approaches that can address this issue. (shrink)

Automatic imitation or “imitative compatibility” is thought to be mediated by the mirror neuron system and to be a laboratory model of the motor mimicry that occurs spontaneously in naturalistic social interaction. Imitative compatibility and spatial compatibility effects are known to depend on different stimulus dimensions—body movement topography and relative spatial position. However, it is not yet clear whether these two types of stimulus–response compatibility effect are mediated by the same or different cognitive processes. We present an interactive activation model (...) of imitative and spatial compatibility, based on a dual-route architecture, which substantiates the view they are mediated by processes of the same kind. The model, which is in many ways a standard application of the interactive activation approach, simulates all key results of a recent study by Catmur and Heyes (2011). Specifically, it captures the difference in the relative size of imitative and spatial compatibility effects; the lack of interaction when the imperative and irrelevant stimuli are presented simultaneously; the relative speed of responses in a quintile analysis when the imperative and irrelevant stimuli are presented simultaneously; and the different time courses of the compatibility effects when the imperative and irrelevant stimuli are presented asynchronously. (shrink)

Text classification involves deciding whether or not a document is about a given topic. It is an important problem in machine learning, because automated text classifiers have enormous potential for application in information retrieval systems. It is also an interesting problem for cognitive science, because it involves real world human decision making with complicated stimuli. This paper develops two models of human text document classification based on random walk and accumulator sequential sampling processes. The models are evaluated using data from (...) an experiment where participants classify text documents presented one word at a time under task instructions that emphasize either speed or accuracy, and rate their confidence in their decisions. Fitting the random walk and accumulator models to these data shows that the accumulator provides a better account of the decisions made, and a “balance of evidence” measure provides the best account of confidence. Both models are also evaluated in the applied information retrieval context, by comparing their performance to established machine learning techniques on the standard Reuters‐21578 corpus. It is found that they are almost as accurate as the benchmarks, and make decisions much more quickly because they only need to examine a small proportion of the words in the document. In addition, the ability of the accumulator model to produce useful confidence measures is shown to have application in prioritizing the results of classification decisions. (shrink)