Decision making in noisy and changing environments requires a fine balance between exploiting knowledge about good courses of action and exploring the environment in order to improve upon this knowledge. We present an experiment on a restless bandit task in which participants made repeated choices between options for which the average rewards changed over time. Comparing a number of computational models of participants’ behavior in this task, we find evidence that a substantial number of them balanced exploration and exploitation by (...) considering the probability that an option offers the maximum reward out of all the available options. (shrink)

This article reviews current methods for evaluating models in the cognitive sciences, including theoretically based approaches, such as Bayes factors and minimum description length measures; simulation approaches, including model mimicry evaluations; and practical approaches, such as validation and generalization measures. This article argues that, although often useful in specific settings, most of these approaches are limited in their ability to give a general assessment of models. This article argues that hierarchical methods, generally, and hierarchical Bayesian methods, specifically, can provide a (...) more thorough evaluation of models in the cognitive sciences. This article presents two worked examples of hierarchical Bayesian analyses to demonstrate how the approach addresses key questions of descriptive adequacy, parameter interference, prediction, and generalization in principled and coherent ways. (shrink)

The Iowa Gambling Task (IGT) has become a remarkable experimental paradigm of dynamic emotion decision making. In recent years, research has emphasized the “prominent deck B (PDB) phenomenon” among normal (control group) participants, in which they favor “bad” deck B with its high-frequency gain structure—a finding that is incongruent with the original IGT hypothesis concerning foresightedness. Some studies have attributed such performance inconsistencies to cultural differences. In the present review, 86 studies featuring data on individual deck selections were drawn from (...) an initial sample of 958 IGT-related studies published from 1994 to 2017 for further investigation. The PDB phenomenon was found in 67.44% of the studies (58 of 86), and most participants were recorded as having adopted the “gain-stay loss-randomize” strategy to cope with uncertainty. Notably, participants in our sample of studies originated from 16 areas across North America, South America, Europe, Oceania, and Asia, and the findings suggest that the PDB phenomenon may be cross-cultural. (shrink)

Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). In this article, (...) we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state–action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model. (shrink)

The Iowa Gambling Task (IGT) is widely used to study decision‐making within healthy and psychiatric populations. However, the complexity of the IGT makes it difficult to attribute variation in performance to specific cognitive processes. Several cognitive models have been proposed for the IGT in an effort to address this problem, but currently no single model shows optimal performance for both short‐ and long‐term prediction accuracy and parameter recovery. Here, we propose the Outcome‐Representation Learning (ORL) model, a novel model that provides (...) the best compromise between competing models. We test the performance of the ORL model on 393 subjects' data collected across multiple research sites, and we show that the ORL reveals distinct patterns of decision‐making in substance‐using populations. Our work highlights the importance of using multiple model comparison metrics to make valid inference with cognitive models and sheds light on learning mechanisms that play a role in underweighting of rare events. (shrink)

Computational models of learning and the theories they represent are often validated by calibrating them to human data on decision outcomes. However, only a few models explain the process by which these decision outcomes are reached. We argue that models of learning should be able to reflect the process through which the decision outcomes are reached, and validating a model on the process is likely to help simultaneously explain both the process as well as the decision outcome. To demonstrate the (...) proposed validation, we use a large dataset from the Technion Prediction Tournament and an existing Instance-based Learning model. We present two ways of calibrating the model’s parameters to human data: on an outcome measure and on a process measure. In agreement with our expectations, we find that calibrating the model on the process measure helps to explain both the process and outcome measures compared to calibrating the model on the outcome measure. These results hold when the model is generalized to a different dataset. We discuss implications for explaining the process and the decision outcomes in computational models of learning. (shrink)

The Iowa Gambling Task (IGT) and the Soochow Gambling Task (SGT) are two experience-based risky decision-making tasks for examining decision-making deficits in clinical populations. Several cognitive models, including the expectancy-valence learning model (EVL) and the prospect valence learning model (PVL), have been developed to disentangle the motivational, cognitive, and response processes underlying the explicit choices in these tasks. The purpose of the current study was to develop an improved model that can fit empirical data better than the EVL and PVL (...) models and, in addition, produce more consistent parameter estimates across the IGT and SGT. Twenty-six opiate users (mean age 34.23; SD 8.79) and 27 control participants (mean age 35; SD 10.44) completed both tasks. Eighteen cognitive models varying in evaluation, updating, and choice rules were fit to individual data and their performances were compared to that of a statistical baseline model to find a best fitting model. The results showed that the model combining the prospect utility function treating gains and losses separately, the decay-reinforcement updating rule, and the trial-independent choice rule performed the best in both tasks. Furthermore, the winning model produced more consistent individual parameter estimates across the two tasks than any of the other models. (shrink)

Prior research has used an Interactive Landslide Simulator tool to investigate human decision making against landslide risks. It has been found that repeated feedback in the ILS tool about damages due to landslides causes an improvement in human decisions against landslide risks. However, little is known on how theories of learning from feedback would account for human decisions in the ILS tool. The primary goal of this paper is to account for human decisions in the ILS tool via computational models (...) based upon reinforcement learning and to explore the model mechanisms involved when people make decisions in the ILS tool. Four different reinforcement-learning models were developed and evaluated in their ability to capture human decisions in an experiment involving two conditions in the ILS tool. The parameters of an Expectancy-Valence model, two Prospect-Valence-Learning models, a combination EV-PU model, and a random model were calibrated to human decisions in the ILS tool across the two conditions. Later, different models with their calibrated parameters were generalized to data collected in an experiment involving a new condition in ILS. When generalized to this new condition, the PVL-2 model’s parameters of both damage-feedback conditions outperformed all other RL models. We highlight the implications of our results for decision making against landslide risks. (shrink)

There is a paucity of research on how psychopathy relates to decision-making. In this study, we assessed the relationship between affective decision-making and psychopathic personality. A sample of prisoners (n D 49) was characterized in terms of psychopathic traits using the Psychopathic Checklist: Screening Version (PCL:SV). Decision-making was assessed using the Iowa Gambling Task (IGT). Higher levels of psychopathy related to more advantageous choices (p D .003). Also counter-intuitively, higher levels of antisocial traits (facet 4) predicted advantageous choices during the (...) learning phase of the task (p D .004). Our findings suggest that some psychopathic facets may be more relevant to decisionmaking under risk, and highlight the importance of further investigations considering facet and trait-level relationships with decision-making. (shrink)