Every scientist chooses a preferred level of analysis and this choice shapes the research program, even determining what counts as evidence. This contribution revisits Marr's three levels of analysis and evaluates the prospect of making progress at each individual level. After reviewing limitations of theorizing within a level, two strategies for integration across levels are considered. One is top–down in that it attempts to build a bridge from the computational to algorithmic level. Limitations of this approach include insufficient theoretical constraint (...) at the computation level to provide a foundation for integration, and that people are suboptimal for reasons other than capacity limitations. Instead, an inside-out approach is forwarded in which all three levels of analysis are integrated via the algorithmic level. This approach maximally leverages mutual data constraints at all levels. For example, algorithmic models can be used to interpret brain imaging data, and brain imaging data can be used to select among competing models. Examples of this approach to integration are provided. This merging of levels raises questions about the relevance of Marr's tripartite view. (shrink)

Decision making in noisy and changing environments requires a fine balance between exploiting knowledge about good courses of action and exploring the environment in order to improve upon this knowledge. We present an experiment on a restless bandit task in which participants made repeated choices between options for which the average rewards changed over time. Comparing a number of computational models of participants’ behavior in this task, we find evidence that a substantial number of them balanced exploration and exploitation by (...) considering the probability that an option offers the maximum reward out of all the available options. (shrink)

In reinforcement learning, a decision maker searching for the most rewarding option is often faced with the question: What is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: How can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and we describe an equivalence between the Bayesian model and (...) temporal difference learning algorithms that have been proposed as models of RL in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional RL algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty. (shrink)

Search is a prerequisite for successful performance in a broad range of tasks ranging from making decisions between consumer goods to memory retrieval. How does aging impact search processes in such disparate situations? Aging is associated with structural and neuromodulatory brain changes that underlie cognitive control processes, which in turn have been proposed as a domain‐general mechanism controlling search in external environments as well as memory. We review the aging literature to evaluate the cognitive control hypothesis that suggests that age‐related (...) change in cognitive control underlies age differences in both external and internal search. We also consider the limits of the cognitive control hypothesis and propose additional mechanisms such as changes in strategy use and affect that may be necessary to understand how aging affects search. (shrink)

Numerous studies have documented individual differences in exploratory tendencies and other phenomena related to search, and these differences have been linked to fitness. Here, I discuss the origins of these differences, focusing on how experience shapes animal search and exploration. The origin of individual differences will also depend upon the alternatives to exploration that are available. Given that search and exploration frequently carry significant costs, we might expect individuals to utilize cues indicating the potential net payoffs of exploration versus the (...) exploitation of known acts. Informative cues could arise from both recent and early-life experiences, from both the social and physical environment. Open questions are the extent to which an individual's exploratory tendencies are fixed throughout life versus being flexibly adjusted according to prevailing conditions and the actions of other individuals, and the extent to which individual differences in exploration extend across domains and are independent of other processes. (shrink)

Prior research has used an Interactive Landslide Simulator tool to investigate human decision making against landslide risks. It has been found that repeated feedback in the ILS tool about damages due to landslides causes an improvement in human decisions against landslide risks. However, little is known on how theories of learning from feedback would account for human decisions in the ILS tool. The primary goal of this paper is to account for human decisions in the ILS tool via computational models (...) based upon reinforcement learning and to explore the model mechanisms involved when people make decisions in the ILS tool. Four different reinforcement-learning models were developed and evaluated in their ability to capture human decisions in an experiment involving two conditions in the ILS tool. The parameters of an Expectancy-Valence model, two Prospect-Valence-Learning models, a combination EV-PU model, and a random model were calibrated to human decisions in the ILS tool across the two conditions. Later, different models with their calibrated parameters were generalized to data collected in an experiment involving a new condition in ILS. When generalized to this new condition, the PVL-2 model’s parameters of both damage-feedback conditions outperformed all other RL models. We highlight the implications of our results for decision making against landslide risks. (shrink)

When we learn how to throw darts we adjust how we throw based on where the darts stick. Much of skill learning is computationally similar in that we learn using feedback obtained after the completion of individual actions. We can formalize such tasks as a search problem; among the set of all possible actions, find the action that leads to the highest reward. In such cases our actions have two objectives: we want to best utilize what we already know (exploitation), (...) but we also want to learn to be more successful in the future (exploration). Here we tested how participants learn movement trajectories where feedback is provided as a monetary reward that depends on the chosen trajectory. We mathematically derived the optimal search policy for our experiment using decision theory. The search behavior of participants is well predicted by an ideal searcher model that optimally combines exploration and exploitation. (shrink)

Based on instructions people can form task representations that shield relevant from seemingly irrelevant information. It has been documented that instructions can tie people to a particular way of performing a task despite that in principle a more efficient way could be learned and used. Since task shielding can lead to persistence of inefficient variants of task performance, it is relevant to test whether individuals with attention deficit/hyperactivity disorder – characterized by less task shielding – are more likely and quicker (...) to escape a suboptimal instructed variant of performing a task. The paradigm used in this online study builds on the observation that in many environments different covarying features could be used to determine the appropriate response. For instance, as they approach a traffic light, drivers and pedestrians monitor the color and/or the position of the signal. Similarly, we instructed participants to respond to the color of a stimulus without mentioning that color covaried with the position of the stimulus. In order to assess whether with practice participants would use the non-instructed feature position to an increasing extent, we compared reaction times and error rates for standard trials to trials in which color was either ambiguous or did not match the usual covariation. Results showed that the covariation learning task can be administered online to adult participants with and without ADHD. Performance differences suggested that with practice ADHD participants might increase attention to non-instructed stimulus features. Yet, they used the non-instructed covarying stimulus feature to a similar extent as other participants. Together the results suggest that participants with ADHD do not lag behind in abandoning instructed task processing in favor of a learned alternative strategy. (shrink)

Neuromarketing relies on Brain Computer Interface (BCI) technology to gain insight into how customers react to marketing stimuli. Marketers spend about$750 billion annually on traditional marketing camping. They use traditional marketing research procedures such as Personal Depth Interviews, Surveys, Focused Group Discussions, and so on, which are frequently criticized for failing to extract true consumer preferences. On the other hand, Neuromarketing promises to overcome such constraints. This work proposes a machine learning framework for predicting consumers' purchase intention (PI) and affective (...) attitude (AA) from analyzing EEG signals. In this work, EEG signals are collected from 20 healthy participants while administering three advertising stimuli settings: product, endorsement, and promotion. After preprocessing, features are extracted in three domains (time, frequency, and time-frequency). Then, after selecting features using wrapper-based methods Recursive Feature Elimination, Support Vector Machine is used for categorizing positive and negative (AA and PI). The experimental results show that proposed framework achieves an accuracy of 84 and 87.00% for PI and AA ensuring the simulation of real-life results. In addition, AA and PI signals show N200 and N400 components when people tend to take decision after visualizing static advertisement. Moreover, negative AA signals shows more dispersion than positive AA signals. Furthermore, this work paves the way for implementing such a neuromarketing framework using consumer-grade EEG devices in a real-life setting. Therefore, it is evident that BCI-based neuromarketing technology can help brands and businesses effectively predict future consumer preferences. Hence, EEG-based neuromarketing technologies can assist brands and enterprizes in accurately forecasting future consumer preferences. (shrink)

We report the results of an experiment in which human subjects were trained to perform a perceptual matching task. Subjects were asked to manipulate comparison objects until they matched target objects using the fewest manipulations possible. An unusual feature of the experimental task is that efficient performance requires an understanding of the hidden or latent causal structure governing the relationships between actions and perceptual outcomes. We use two benchmarks to evaluate the quality of subjects’ learning. One benchmark is based on (...) optimal performance as calculated by a dynamic programming procedure. The other is based on an adaptive computational agent that uses a reinforcement-learning method known as Q-learning to learn to perform the task. Our analyses suggest that subjects were successful learners. In particular, they learned to perform the perceptual matching task in a near-optimal manner (i.e., using a small number of manipulations) at the end of training. Subjects were able to achieve near-optimal performance because they learned, at least partially, the causal structure underlying the task. In addition, subjects’ performances were broadly consistent with those of model-based reinforcement-learning agents that built and used internal models of how their actions influenced the external environment. We hypothesize that people will achieve near-optimal performances on tasks requiring sequences of action—especially sensorimotor tasks with underlying latent causal structures—when they can detect the effects of their actions on the environment, and when they can represent and reason about these effects using an internal mental model. (shrink)