Cognitive control has long been one of the most active areas of computational modeling work in cognitive science. The focus on computational models as a medium for specifying and developing theory predates the PDP books, and cognitive control was not one of the areas on which they focused. However, the framework they provided has injected work on cognitive control with new energy and new ideas. On the occasion of the books' anniversary, we review computational modeling in the study of cognitive (...) control, with a focus on the influence that the PDP approach has brought to bear in this area. Rather than providing a comprehensive review, we offer a framework for thinking about past and future modeling efforts in this domain. We define control in terms of the optimal parameterization of task processing. From this vantage point, the development of control systems in the brain can be seen as responding to the structure of naturalistic tasks, through the filter of the brain systems with which control directly interfaces. This perspective lays open a set of fascinating but difficult research questions, which together define an important frontier for future computational research. (shrink)

Human control of action in routine situations involves a flexible interplay between (a) task-dependent serial ordering constraints; (b) top-down, or intentional, control processes; and (c) bottom-up, or environmentally triggered, affordances. In addition, the interaction between these influences is modulated by learning mechanisms that, over time, appear to reduce the need for top-down control processes while still allowing those processes to intervene at any point if necessary or if desired. We present a model of the acquisition and control of goal-directed action (...) that goes beyond existing models by operationalizing an interface between two putative systems—a routine and a non-routine system—thereby demonstrating how explicitly represented goals can interact with the emergent task representations that develop through learning in the routine system. The gradual emergence of task representations offers an explanation for the transfer of control with experience from the non-routine goal-based system to the routine system. At the same time it allows action selection to be sensitive both to environmental triggers and to biasing from multiple levels within the goal system. (shrink)

Various theories of moral cognition posit that moral intuitions can be understood as the output of a computational process performed over structured mental representations of human action. We propose that action plan diagrams—“act trees”—can be a useful tool for theorists to succinctly and clearly present their hypotheses about the information contained in these representations. We then develop a methodology for using a series of linguistic probes to test the theories embodied in the act trees. In Study 1, we validate the (...) method by testing a specific hypothesis (diagrammed by act trees) about how subjects are representing two classic moral dilemmas and finding that the data support the hypothesis. In Studies 2–4, we explore possible explanations for discrete and surprising findings that our hypothesis did not predict. In Study 5, we apply the method to a less well‐studied case and show how new experiments generated by our method can be used to settle debates about how actions are mentally represented. In Study 6, we argue that our method captures the mental representation of human action better than an alternative approach. A brief conclusion suggests that act trees can be profitably used in various fields interested in complex representations of human action, including law, philosophy, psychology, linguistics, neuroscience, computer science, robotics, and artificial intelligence. (shrink)

This paper presents an empirical solution to the puzzle of weakness of will. Specifically, it presents a theory of action, grounded in contemporary cognitive neuroscientific accounts of decision making, that explains the phenomenon of weakness of will without resulting in a puzzle.

Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). In this article, (...) we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state–action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model. (shrink)

It is generally accepted that, in the cognitive and neural sciences, there are both computational and mechanistic explanations. We ask how computational explanations can integrate into the mechanistic hierarchy. The problem stems from the fact that implementation and mechanistic relations have different forms. The implementation relation, from the states of an abstract computational system to the physical, implementing states is a homomorphism mapping relation. The mechanistic relation, however, is that of part/whole; the explaining features in a mechanistic explanation are the (...) components of the explanandum phenomenon and their causal organization. Moreover, each component in one level of mechanism is constituted and explained by components of an underlying level of mechanism. Hence, it seems, computational variables and functions cannot be mechanistically explained by the medium-dependent states and properties that implement them. How then, do the computational and the implementational integrate to create the mechanistic hierarchy? After explicating the general problem, we further demonstrate it through a concrete example, of reinforcement learning, in the cognitive and neural sciences. We then examine two possible solutions. On one solution, the mechanistic hierarchy embeds at the same levels computational and implementational properties. This picture fits with the view that computational explanations are mechanistic sketches. On the other solution, there are two separate hierarchies, one computational and another implementational, which are related by the implementation relation. This picture fits with the view that computational explanations are functional and autonomous explanations. It is less clear how these solutions fit with the view that computational explanations are full-fledged mechanistic explanations. Finally, we argue that both pictures are consistent with the reinforcement learning example, but that scientific practice does not align with the view that computational models are merely mechanistic sketches. (shrink)

Curiosity and creativity are expressions of the trade-off between leveraging that with which we are familiar or seeking out novelty. Through the computational lens of reinforcement learning, we describe how formulating the value of information seeking and generation via their complementary effects on planning horizons formally captures a range of solutions to striking this balance.

This paper presents a viewpoint on natural philosophy focusing on the organization of substance, as well as its changes as invited by the Second Law of thermodynamics. Modes of change are pointed to as definitive of levels of organization; these include physical, chemical, and biological modes of change. Conceptual uses of the subsumptive hierarchy format are employed throughout this paper. Developmental change in dissipative structures is examined in some detail, generating an argument for the use of final causality in studies (...) of natural systems. Considerations of ‘internalism’ in science are presented along the way. (shrink)

Current explanatory frameworks for social norms pay little attention to why and how brains might carry out computational functions that generate norm compliance behavior. This paper expands on existing literature by laying out the beginnings of a neurocomputational framework for social norms and social cognition, which can be the basis for advancing our understanding of the nature and mechanisms of social norms. Two neurocomputational building blocks are identified that might constitute the core of the mechanism of norm compliance. They consist (...) of Bayesian and reinforcement learning systems. It is sketched why and how the concerted activity of these systems can generate norm compliance by minimization of three specific kinds of prediction-errors. (shrink)

Butz, Achimova, Bilkey, and Knott provide a topic overview and discuss whether the special issue contributions may imply that event‐predictive abilities constitute a root for conceptual human thought, because they enable complex, mutually beneficial, but also intricately competitive, social interactions and language communication.

Topics in Cognitive Science, Volume 14, Issue 3, Page 528-549, July 2022.

Human behavior unfolds primarily in built environments, where the arrangement of objects is a result of ongoing human decisions and actions, yet these organizational decisions have received limited experimental study. In two experiments, we introduce a novel paradigm designed to explore how individuals organize task-relevant objects in space. Participants completed goals by locating and accessing sequences of objects in a computer-based task, and they were free to rearrange the positions of objects at any time. We measure a variety of organization (...) changes and evaluate how these measures relate to individual differences in performance. In Experiment 1, we show that with weak structure in task demands, changes in object positions that arise through performance of the task lead to improved order, characterized predominantly by a centralization of frequently used items and a peripheralization of infrequently used objects. In Experiment 2, with increased task structure, we observe more refined organizational tendencies, with selective contraction and clustering of interrelated task-relevant objects. We further demonstrate that these more selective organization behaviors are reliably associated with individual differences in task performance. Collectively, these two studies reveal properties of space and of task demands that support and facilitate effective organization of the environment in support of ongoing behavior. (shrink)

This paper proposes how various disciplinary theories of cognition may be combined into a unifying, sub-symbolic, computational theory of cognition. The following theories are considered for integration: psychological theories, including the theory of event coding, event segmentation theory, the theory of anticipatory behavioral control, and concept development; artificial intelligence and machine learning theories, including reinforcement learning and generative artificial neural networks; and theories from theoretical and computational neuroscience, including predictive coding and free energy-based inference. In the light of such a (...) potential unification, it is discussed how abstract cognitive, conceptualized knowledge and understanding may be learned from actively gathered sensorimotor experiences. The unification rests on the free energy-based inference principle, which essentially implies that the brain builds a predictive, generative model of its environment. Neural activity-oriented inference causes the continuous adaptation of the currently active predictive encodings. Neural structure-oriented inference causes the longer term adaptation of the developing generative model as a whole. Finally, active inference strives for maintaining internal homeostasis, causing goal-directed motor behavior. To learn abstract, hierarchical encodings, however, it is proposed that free energy-based inference needs to be enhanced with structural priors, which bias cognitive development towards the formation of particular, behaviorally suitable encoding structures. As a result, it is hypothesized how abstract concepts can develop from, and thus how they are structured by and grounded in, sensorimotor experiences. Moreover, it is sketched-out how symbol-like thought can be generated by a temporarily active set of predictive encodings, which constitute a distributed neural attractor in the form of an interactive free-energy minimum. The activated, interactive network attractor essentially characterizes the semantics of a concept or a concept composition, such as an actual or imagined situation in our environment. Temporal successions of attractors then encode unfolding semantics, which may be generated by a behavioral or mental interaction with an actual or imagined situation in our environment. Implications, further predictions, possible verification and falsifications, as well as potential enhancements into a fully spelled-out unified theory of cognition are discussed at the end of the paper. (shrink)

In reinforcement learning, a decision maker searching for the most rewarding option is often faced with the question: What is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: How can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and we describe an equivalence between the Bayesian model and (...) temporal difference learning algorithms that have been proposed as models of RL in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional RL algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty. (shrink)

In line with Allen Newell's challenge to develop complete cognitive architectures, and motivated by a recent proposal for a unifying subsymbolic computational theory of cognition, we introduce the cognitive control architecture SEMLINCS. SEMLINCS models the development of an embodied cognitive agent that learns discrete production rule-like structures from its own, autonomously gathered, continuous sensorimotor experiences. Moreover, the agent uses the developing knowledge to plan and control environmental interactions in a versatile, goal-directed, and self-motivated manner. Thus, in contrast to several well-known (...) symbolic cognitive architectures, SEMLINCS is not provided with production rules and the involved symbols, but it learns them. In this paper, the actual implementation of SEMLINCS causes learning and self-motivated, autonomous behavioral control of the game figure Mario in a clone of the computer game Super Mario Bros. Our evaluations highlight the successful development of behavioral versatility as well as the learning of suitable production rules and the involved symbols from sensorimotor experiences. Moreover, knowledge- and motivation-dependent individualizations of the agents’ behavioral tendencies are shown. Finally, interaction sequences can be planned on the sensorimotor-grounded production rule level. Current limitations directly point toward the need for several further enhancements, which may be integrated into SEMLINCS in the near future. Overall, SEMLINCS may be viewed as an architecture that allows the functional and computational modeling of embodied cognitive development, whereby the current main focus lies on the development of production rules from sensorimotor experiences. (shrink)

Pursuing a precise, focused train of thought requires cognitive effort. Even more effort is necessary when more alternatives need to be considered or when the imagined situation becomes more complex. Cognitive resources available to us limit the cognitive effort we can spend. In line with previous work, an information-theoretic, Bayesian brain approach to cognitive effort is pursued: to solve tasks in our environment, our brain needs to invest information, that is, negative entropy, to impose structure, or focus, away from a (...) uniform structure or other task-incompatible, latent structures. To get a more complete formalization of cognitive effort, a resourceful event-predictive inference model is introduced, which offers computational and algorithmic explanations about the latent structure of our generative models, the active inference dynamics that unfold within, and the cognitive effort required to steer the dynamics—to, for example, purposefully process sensory signals, decide on responses, and invoke their execution. REPI suggests that we invest cognitive resources to infer preparatory priors, activate responses, and anticipate action consequences. Due to our limited resources, though, the inference dynamics are prone to task-irrelevant distractions. For example, the task-irrelevant side of the imperative stimulus causes the Simon effect and, due to similar reasons, we fail to optimally switch between tasks. An actual model implementation simulates such task interactions and offers first estimates of the involved cognitive effort. The approach may be further studied and promises to offer deeper explanations about why we get quickly exhausted from multitasking, how we are influenced by irrelevant stimulus modalities, why we exhibit magnitude interference, and, during social interactions, why we often fail to take the perspective of others into account. (shrink)