In this paper, I examine Reinforcement Learning modelling practice in psychiatry, in the context of alcohol use disorders. I argue that the epistemic roles RL currently plays in the development of psychiatric classification and search for explanations of clinically relevant phenomena are best appreciated in terms of Chang’s account of epistemic iteration, and by distinguishing mechanistic and aetiological modes of computational explanation.

Active inference offers a unified theory of perception, learning, and decision-making at computational and neural levels of description. In this article, we address the worry that active inference may be in tension with the belief–desire–intention model within folk psychology because it does not include terms for desires at the mathematical level of description. To resolve this concern, we first provide a brief review of the historical progression from predictive coding to active inference, enabling us to distinguish between active inference formulations (...) of motor control and active inference formulations of decision processes. We then show that, despite a superficial tension when viewed at the mathematical level of description, the active inference formalism contains terms that are readily identifiable as encoding both the objects of desire and the strength of desire at the psychological level of description. We demonstrate this with simple simulations of an active inference agent motivated to leave a dark room for different reasons. Despite their consistency, we further show how active inference may increase the granularity of folk-psychological descriptions by highlighting distinctions between drives to seek information versus reward—and how it may also offer more precise, quantitative folk-psychological predictions. Finally, we consider how the implicitly conative components of active inference may have partial analogues in other systems describable by the broader free energy principle to which it conforms. (shrink)

There is a debate over whether actions that resist devaluation are primarily habit- or goal-directed. The incentive habit account of compulsive actions has received support from behavioral paradigms and brain imaging. In addition, the self-reported Creature of Habit Scale has been proposed to capture inter-individual differences in habitual tendencies. It is subdivided into two dimensions: routine and automaticity. We first considered a French version of this questionnaire for validation, based on a sample of 386 undergraduates. The relationship between two dimensions (...) of habit and the risk of substance use disorder and impulsive personality traits was also investigated. COHS has good psychometric properties with both features of habits positively associated with an Obsessive-Compulsive Inventory score. Besides, the propensity to rely more on routines was associated with lower levels of alcohol abuse and nicotine use, suggesting that some degree of routine might act as a protective factor against substance use. In contrast, a high automaticity score was associated with an increased risk of harmful alcohol use. These results demonstrate that the COHS is a valid measure of habitual tendencies and represents a useful tool for capturing inter-individual variations in drug use problems in undergraduates. (shrink)

Advances in artificial intelligence, as well as its increased presence in everyday life, have brought the emergence of many new phenomena, including an intriguing appearance of what seems to be a variant of body dysmorphic disorder, coined “Snapchat dysmorphia”. Body dysmorphic disorder is a DSM-5 psychiatric disorder defined as a preoccupation with one or more perceived defects or flaws in physical appearance that are not observable or appear slight to others. Snapchat dysmorphia is fueled by automated selfie filters that reflect (...) unrealistic sociocultural standard. In this paper, we discuss how body dysmorphic disorder and related body image distortions could arise, using the conceptual resources provided by the active inference framework. We suggest that these disorders involve dysfunctional self-modelling which entails maladaptive internalization of sociocultural preferences during adolescent identity formation. Identity formation is hereby described as cycles of interpersonal active inference that arbitrate between identity exploration and commitment. We propose that impaired self-modelling is unable to reduce interpersonal uncertainty during identity exploration, which, over time, degenerates into uncontrollable epistemic habits that isolate the body image from corrective sensory evidence. In light of these insights, we subsequently explore some of the consequences of image-centered social media platforms on the identity formation process. We conclude that heightened interpersonal uncertainty in this novel context could precipitate the onset of body dysmorphic disorder and related body image distortions, particularly when selfie filters are involved. (shrink)

Advances in artificial intelligence, as well as its increased presence in everyday life, have brought the emergence of many new phenomena, including an intriguing appearance of what seems to be a variant of body dysmorphic disorder, coined “Snapchat dysmorphia”. Body dysmorphic disorder is a DSM-5 psychiatric disorder defined as a preoccupation with one or more perceived defects or flaws in physical appearance that are not observable or appear slight to others. Snapchat dysmorphia is fueled by automated selfie filters that reflect (...) unrealistic sociocultural standard. In this paper, we discuss how body dysmorphic disorder and related body image distortions could arise, using the conceptual resources provided by the active inference framework. We suggest that these disorders involve dysfunctional self-modelling which entails maladaptive internalization of sociocultural preferences during adolescent identity formation. Identity formation is hereby described as cycles of interpersonal active inference that arbitrate between identity exploration and commitment. We propose that impaired self-modelling is unable to reduce interpersonal uncertainty during identity exploration, which, over time, degenerates into uncontrollable epistemic habits that isolate the body image from corrective sensory evidence. In light of these insights, we subsequently explore some of the consequences of image-centered social media platforms on the identity formation process. We conclude that heightened interpersonal uncertainty in this novel context could precipitate the onset of body dysmorphic disorder and related body image distortions, particularly when selfie filters are involved. (shrink)

An agent exercises instrumental rationality to the degree that she adopts appropriate means to achieving her ends. Adopting appropriate means to achieving one’s ends can, in turn, involve overcoming one’s strongest desires, that is, it can involve exercising synchronic self-control. However, contra prominent approaches, I deny that synchronic self-control is possible. Specifically, I draw on computational models and empirical evidence from cognitive neuroscience to describe a naturalistic, multi-system model of the mind. On this model, synchronic self-control is impossible. Must we, (...) then, give up on a meaningful conception of instrumental rationality? No. A multi-system view still permits something like synchronic self-control: an agent can control her very strong desires. Adopting a multi-system model of the mind thus places limitations on our conceptions of instrumental rationality, without requiring that we abandon the notion altogether. (shrink)

How can we think about things in the outside world? There is still no widely accepted theory of how mental representations get their meaning. In light of pioneering research, Nicholas Shea develops a naturalistic account of the nature of mental representation with a firm focus on the subpersonal representations that pervade the cognitive sciences.

A cognitivist account of decision-making views choice behaviour as a serial process of deliberation and commitment, which is separate from perception and action. By contrast, recent work in embodied decision-making has argued that this account is incompatible with emerging neurophysiological data. We argue that this account has significant overlap with an embodied account of predictive processing, and that both can offer mutual development for the other. However, more importantly, by demonstrating this close connection we uncover an alternative perspective on the (...) nature of decision-making, and the mechanisms that underlie our choice behaviour. This alternative perspective allows us to respond to a challenge for predictive processing, which claims that the satisfaction of distal goal-states is underspecified. Answering this challenge requires the adoption of an embodied perspective. (shrink)

Cognitive scientists have revealed systematic errors in human reasoning. There is disagreement about what these errors indicate about human rationality, but one upshot seems clear: human reasoning does not seem to fit traditional views of human rationality. This concern about rationality has made its way through various fields and has recently caught the attention of philosophers. The concern is that if philosophers are prone to systematic errors in reasoning, then the integrity of philosophy would be threatened. In this paper, I (...) present some of the more famous work in cognitive science that has marshaled this concern. Then I present reasons to think that those with training in philosophy will be less prone to certain systematic errors in reasoning. The suggestion is that if philosophers could be shown to be less prone to such errors, then the worries about the integrity of philosophy could be constrained. Then I present evidence that, according to performance on the CRT (Frederick 2005), those who have benefited from training and selection in philosophy are indeed less prone to one kind of systematic error: irrationally arbitrating between intuitive and reflective responses. Nonetheless, philosophers are not entirely immune to this systematic error, and their proclivity for this error is statistically related to their responses to a variety of philosophical questions. So, while the evidence herein puts constraints on the worries about the integrity of philosophy, it by no means eliminates these worries. The conclusion, then, is that the present evidence offers prima facie reasons to ascribe a mitigated privilege to philosophers' ability to rationally arbitrate between intuitive and reflective responses. (shrink)

Background: Prejudices against minorities can be understood as habitually negative evaluations that are kept in spite of evidence to the contrary. Therefore, individuals with strong prejudices might be dominated by habitual or “automatic” reactions at the expense of more controlled reactions. Computational theories suggest individual differences in the balance between habitual/model-free and deliberative/model-based decision-making.Methods: 127 subjects performed the two Step task and completed the blatant and subtle prejudice scale.Results: By using analyses of choices and reaction times in combination with computational (...) modeling, subjects with stronger blatant prejudices showed a shift away from model-based control. There was no association between these decision-making processes and subtle prejudices.Conclusion: These results support the idea that blatant prejudices toward minorities are related to a relative dominance of habitual decision-making. This finding has important implications for developing interventions that target to change prejudices across societies. (shrink)

This paper approaches the question of how to describe the higher-level internal structures and representations of cognitive systems across various kinds of nonhuman (neuro)cognitive systems. While much research in cognitive (neuro)science and comparative cognition is dedicated to the exploration of the (neuro)cognitive mechanisms and processes with a focus on brain-behavior relations across different non-human species, not much has been done to connect (neuro)cognitive mechanisms and processes and the associated behaviors to plausible higher-level structures and representations of distinct kinds of cognitive (...) systems in non-humans. Although the study of (neuro)cognitive mechanisms and processes can certainly be revealing, (neuro)cognitive mechanisms and processes are underspecified with respect to internal structures and representations of non-human cognitive systems because multiple such mechanisms can target, or be mapped onto, the same internal structure or vice versa. This paper outlines a biosemiotic approach to this linking problem in order to bridge the gap between functions of (neuro)cognitive systems in different species and the higher-level cognitive structures and representations. It is contended that the higher-level internal structures and representations of various cognitive systems are biosemiotic constraints on the (biological) functions of (neuro)cognitive systems that serve to restrict the range of functions (neuro)cognitive systems have or are selected for. This turns out to have implications for issues on the convergent evolution of cognitive traits. (shrink)

Conventional sacrificial moral dilemmas propose directly causing some harm to prevent greater harm. Theory suggests that accepting such actions (consistent with utilitarian philosophy) involves more reflective reasoning than rejecting such actions (consistent with deontological philosophy). However, past findings do not always replicate, confound different kinds of reflection, and employ conventional sacrificial dilemmas that treat utilitarian and deontological considerations as opposite. In two studies, we examined whether past findings would replicate when employing process dissociation to assess deontological and utilitarian inclinations independently. (...) Findings suggested two categorically different impacts of reflection: measures of arithmetic reflection, such as the Cognitive Reflection Test, predicted only utilitarian, not deontological, response tendencies. However, measures of logical reflection, such as performance on logical syllogisms, positively predicted both utilitarian and deontological tendencies. These studies replicate some findings, clarify others, and reveal opportunity for additional nuance in dual process theorist’s claims about the link between reflection and dilemma judgments. (shrink)

Matthew Liao’s edited collection Moral Brains: The Neuroscience of Morality covers a wide range of issues in moral psychology. The collection should be of interest to philosophers, psychologist, and neuroscientists alike, particularly those interested in the relation between these disciplines. I give an overview of the content and major themes of the volume and draw some important lessons about the connection between moral neuroscience and normative ethics. In particular, I argue that moving beyond some of the dichotomies implicit in some (...) of the debates advanced in the book makes the neuroscience of moral judgment much more useful in advancing normative ethics. (shrink)

In business settings, decision makers facing moral issues often experience the challenges of continuous changes. This dynamic process has been less examined in previous literature on moral decision making. We borrow theories on learning strategies and computational models from decision neuroscience to explain the updating and learning mechanisms underlying moral decision processes. Specifically, we present two main learning strategies: model-free learning, wherein the values of choices are updated in a trial-and-error fashion sustaining the formation of habits and model-based learning, wherein (...) the brain updates more general cognitive maps and associations, thus sustaining flexible and state-dependent behaviors. We then summarize studies explaining the neuro-computational processes of both learning strategies—the calculation of prediction errors and valuation. We conclude by emphasizing how the incorporation of dynamic aspects in moral decision making could open new avenues for understanding moral behaviors in a changing world. (shrink)

One of the paradigms in judgment and decision-making involves decision-makers sample information before making a final consequential choice. In the sampling paradigm, certain computational models have been proposed where a set of single or distribution parameters is calibrated to the choice proportions of a group of participants. However, currently little is known on how aggregate and hierarchical models would account for choices made by individual participants in the sampling paradigm. In this paper, we test the ability of aggregate and hierarchical (...) models to explain choices made by individual participants. Several models, Ensemble, Cumulative Prospect Theory, Best Estimation and Simulation Techniques, Natural-Mean Heuristic, and Instance-Based Learning, had their parameters calibrated to individual choices in a large dataset involving the sampling paradigm. Later, these models were generalized to two large datasets in the sampling paradigm. Results revealed that the aggregate models accounted for individual choices better than hierarchical models upon generalization to problems that were like those encountered during calibration. Furthermore, the CPT model, which relies on differential valuing of gains and losses, respectively, performed better than other models during calibration and generalization on datasets with similar set of problems. The IBL model, relying on recency and frequency of sampled information, and the NMH model, relying on frequency of sampled information, performed better than other models during generalization to a challenging dataset. Sequential analyses of results from different models showed how these models accounted for transitions from the last sample to final choice in human data. We highlight the implications of using aggregate and hierarchical models in explaining individual choices from experience. (shrink)

The ability to reason about how things were made is a pervasive aspect of how humans make sense of physical objects. Such reasoning is useful for a range of everyday tasks, from assembling a piece of furniture to making a sandwich and knitting a sweater. What enables people to reason in this way even about novel objects, and how do people draw upon prior experience with an object to continually refine their understanding of how to create it? To explore these (...) questions, we developed a virtual task environment to investigate how people come up with step‐by‐step procedures for recreating block towers whose composition was not readily apparent, and analyzed how the procedures they used to build them changed across repeated attempts. Specifically, participants (N = 105) viewed 2D silhouettes of eight unique block towers in a virtual environment simulating rigid‐body physics, and aimed to reconstruct each one in less than 60 s. We found that people built each tower more accurately and quickly across repeated attempts, and that this improvement reflected both group‐level convergence upon a tiny fraction of all possible viable procedures, as well as error‐dependent updating across successive attempts by the same individual. Taken together, our study presents a scalable approach to measuring consistency and variation in how people infer solutions to physical assembly problems. (shrink)

Theories of decision-making and its neural substrates have long assumed the existence of two distinct and competing valuation systems, variously described as goal-directed vs. habitual, or, more recently and based on statistical arguments, as model-free vs. model-based reinforcement-learning. Though both have been shown to control choices, the cognitive abilities associated with these systems are under ongoing investigation. Here we examine the link to cognitive abilities, and find that individual differences in processing speed covary with a shift from model-free to model-based (...) choice control in the presence of above-average working memory function. This suggests shared cognitive and neural processes; provides a bridge between literatures on intelligence and valuation; and may guide the development of process models of different valuation components. Furthermore, it provides a rationale for individual differences in the tendency to deploy valuation systems, which may be important for understanding the manifold neuropsychiatric diseases associated with malfunctions of valuation. (shrink)

Although seemingly irrational choice abounds, the rules governing these mis‐steps that might provide hints about the factors limiting normative behavior are unclear. We consider three experimental tasks, which probe different aspects of non‐normative choice under uncertainty. We argue for systematic statistical, algorithmic, and implementational sources of irrationality, including incomplete evaluation of long‐run future utilities, Pavlovian actions, and habits, together with computational and statistical noise and uncertainty. We suggest structural and functional adaptations that minimize their maladaptive effects.

Successfully explaining and replicating the complexity and generality of human and animal learning will require the integration of a variety of learning mechanisms. Here, we introduce a computational model which integrates associative learning (AL) and reinforcement learning (RL). We contrast the integrated model with standalone AL and RL models in three simulation studies. First, a synthetic grid‐navigation task is employed to highlight performance advantages for the integrated model in an environment where the reward structure is both diverse and dynamic. The (...) second and third simulations contrast the performances of the three models in behavioral experiments, demonstrating advantages for the integrated model in accounting for behavioral data. (shrink)

The commentaries suggest many important improvements to the target article. They clearly distinguish two varieties of rationalization – the traditional “motivated reasoning” model, and the proposed representational exchange model – and show that they have distinct functions and consequences. They describe how representational exchange occurs not only by post hoc rationalization but also by ex ante rationalization and other more dynamic processes. They argue that the social benefits of representational exchange are at least as important as its direct personal benefits. (...) Finally, they construe our search for meaning, purpose, and narrative – both individually and collectively – as a variety of representational exchange. The result is a theory of rationalization as representational exchange both wider in scope and better defined in mechanism. (shrink)

What place should formal or computational methods occupy in social psychology? We consider this question in historical perspective, survey the current state of the field, introduce the several new contributions to this special issue, and reflect on the future.

Abstract:Humans often comply with social norms, but the reasons why are disputed. Here, we unify a variety of influential explanations in a common decision framework, and identify the precise cognitive variables that norms might alter to induce compliance. Specifically, we situate current theories of norm compliance within the reinforcement learning framework, which is widely used to study value-guided learning and decision-making. This framework offers an appealingly precise language to distinguish between theories, highlights the various points of convergence and divergence, and (...) suggests novel ways in which norms might penetrate our psychology. (shrink)

In recent years, scientists have increasingly taken to investigate the predictive nature of cognition. We argue that prediction relies on abstraction, and thus theories of predictive cognition need an explicit theory of abstract representation. We propose such a theory of the abstract representational capacities that allow humans to transcend the “here-and-now.” Consistent with the predictive cognition literature, we suggest that the representational substrates of the mind are built as ahierarchy, ranging from the concrete to the abstract; however, we argue that (...) there are qualitative differences between elements along this hierarchy, generating meaningful, often unacknowledged,diversity.Echoing views from philosophy, we suggest that the representational hierarchy can be parsed into:modality-specificrepresentations, instantiated on perceptual similarity;multimodalrepresentations, instantiated primarily on the discovery of spatiotemporal contiguity; andcategoricalrepresentations, instantiated primarily on social interaction. These elements serve as the building blocks ofcomplex structures discussed in cognitive psychology (e.g., episodes, scripts) and are the inputs for mental representations that behave like functions, typically discussed in linguistics (i.e.,predicators). We support our argument for representational diversity by explaining how the elements in our ontology are all required to account for humans’ predictive cognition (e.g., in subserving logic-based prediction; in optimizing the trade-off between accurate and detailed predictions) and by examining how the neuroscientific evidence coheres with our account. In doing so, we provide a testable model of the neural bases of conceptual cognition and highlight several important implications to research on self-projection, reinforcement learning, and predictive-processing models of psychopathology. (shrink)

For decades, psychostimulants have been the gold standard pharmaceutical treatment for attention-deficit/hyperactivity disorder (ADHD). In the United States, an astounding 9% of all boys and 4% of girls will be prescribed stimulant drugs at some point during their childhood. Recent meta-analyses have revealed that individuals with ADHD have reduced brain volume loss later in life (>60 y.o.) compared to the normal aging brain, which suggests that either ADHD or its treatment may be neuroprotective. Crucially, these neuroprotective effects were significant in (...) brain regions (e.g., hippocampus, amygdala) where severe volume loss is linked to cognitive impairment and Alzheimer’s disease. Historically, the ADHD diagnosis and its pharmacotherapy came about nearly simultaneously, making it difficult to evaluate their effects in isolation. Certain evidence suggests that psychostimulants may normalize structural brain changes typically observed in the ADHD brain. If ADHD itself is neuroprotective, perhaps exercising the brain, then psychostimulants may not be recommended across the lifespan. Alternatively, if stimulant drugs are neuroprotective, then this class of medications may warrant further investigation for their therapeutic effects. Here, we take a bottom-up holistic approach to review the psychopharmacology of ADHD in the context of recent models of attention. We suggest that future studies are greatly needed to better appreciate the interactions amongst an ADHD diagnosis, stimulant treatment across the lifespan, and structure-function alterations in the aging brain. (shrink)

We propose a triple test to evaluate the usefulness of behavioral economics models for public health policy. Test 1 is whether the model provides reasonably new insights. Test 2 is on whether these have been properly applied to policy settings. Test 3 is whether they are corroborated by evidence. We exemplify by considering the cases of social interactions models, self-control models and, in relation to health message framing, prospect theory. Out of these sets of models, only a correctly applied prospect (...) theory fully passes the tests at present. Specifically, in broad agreement with the evidence, a gain frame has positive implications for welfare encourages disease prevention activity, though this does not apply if the perceived probability of the bad health outcome is large enough. We see our tests as being useful to identify how much health policy weight policy makers should assign to specific behavioral economic models; they are also useful to verify what next steps would be most useful in further research. (shrink)

Current explanatory frameworks for social norms pay little attention to why and how brains might carry out computational functions that generate norm compliance behavior. This paper expands on existing literature by laying out the beginnings of a neurocomputational framework for social norms and social cognition, which can be the basis for advancing our understanding of the nature and mechanisms of social norms. Two neurocomputational building blocks are identified that might constitute the core of the mechanism of norm compliance. They consist (...) of Bayesian and reinforcement learning systems. It is sketched why and how the concerted activity of these systems can generate norm compliance by minimization of three specific kinds of prediction-errors. (shrink)