Topics in Cognitive Science, Volume 14, Issue 1, Page 14-30, January 2022.

We argue that cognitive models can provide a common ground between human users and deep reinforcement learning (Deep RL) algorithms for purposes of explainable artificial intelligence (AI). Casting both the human and learner as cognitive models provides common mechanisms to compare and understand their underlying decision-making processes. This common grounding allows us to identify divergences and explain the learner's behavior in human understandable terms. We present novel salience techniques that highlight the most relevant features in each model's decision-making, as well (...) as examples of this technique in common training environments such as Starcraft II and an OpenAI gridworld. (shrink)

Topics in Cognitive Science, Volume 14, Issue 1, Page 6-13, January 2022.

The last two decades have produced unprecedented successes in the fields of artificial intelligence and machine learning (ML), due almost entirely to advances in deep neural networks (DNNs). Deep hierarchical memory networks are not a novel concept in cognitive science and can be traced back more than a half century to Simon's early work on discrimination nets for simulating human expertise. The major difference between DNNs and the deep memory nets meant for explaining human cognition is that the latter are (...) symbolic networks meant to model the dynamics of human memory and learning. Cognition-inspired symbolic deep networks (SDNs) address several known issues with DNNs, including (1) learning efficiency, where a much larger number of training examples are required for DNNs than would be expected for a human; (2) catastrophic interference, where what is learned by a DNN gets unlearned when a new problem is presented; and (3) explainability, where there is no way to explain what is learned by a DNN. This paper explores whether SDNs can achieve similar classification accuracy performance to DNNs across several popular ML datasets and discusses the strengths and weaknesses of each approach. Simulations reveal that (1) SDNs provide similar accuracy to DNNs in most cases, (2) SDNs are far more efficient than DNNs, (3) SDNs are as robust as DNNs to irrelevant/noisy attributes in the data, and (4) SDNs are far more robust to catastrophic interference than DNNs. We conclude that SDNs offer a promising path toward human-level accuracy and efficiency in category learning. More generally, ML frameworks could stand to benefit from cognitively inspired approaches, borrowing more features and functionality from models meant to simulate and explain human learning. (shrink)

Several attempts have been made previously to provide a biological grounding for cognitive architectures by relating their components to the computations of specific brain circuits. Often, the architecture's action selection system is identified with the basal ganglia. However, this identification overlooks one of the most important features of the basal ganglia—the existence of a direct and an indirect pathway that compete against each other. This characteristic has important consequences in decision-making tasks, which are brought to light by Parkinson's disease as (...) well as genetic differences in dopamine receptors. This paper shows that a standard model of action selection in a cognitive architecture cannot replicate any of these findings, details an alternative solution that reconciles action selection in the architecture with the physiology of the basal ganglia, and extends the domain of application of cognitive architectures. The implication of this solution for other architectures and existing models are discussed. (shrink)

Deep Neural Networks (DNNs) are popular for classifying large noisy analogue data. However, DNNs suffer from several known issues, including explainability, efficiency, catastrophic interference, and a need for high‐end computational resources. Our simulations reveal that psychologically‐inspired symbolic deep networks (SDNs) achieve similar accuracy and robustness to noise as DNNs on common ML problem sets, while addressing these issues.

A framing bias shows risk aversion in problems framed as “gains” and risk seeking in problems framed as “losses,” even when these are objectively equivalent and probabilities and outcomes values are explicitly provided. We test this framing bias in situations where decision makers rely on their own experience, sampling the problem's options and seeing the outcomes before making a choice. In Experiment 1, we replicate the framing bias in description-based decisions and find risk indifference in gains and losses in experience-based (...) decisions. Predictions of an Instance-Based Learning model suggest that objective probabilities as well as the number of samples taken are factors that contribute to the lack of framing effect. We test these two factors in Experiment 2 and find no framing effect when a few samples are taken but when large samples are taken, the framing effect appears regardless of the objective probability values. Implications of behavioral results and cognitive modeling are discussed. (shrink)

A major challenge for research in artificial intelligence is to develop systems that can infer the goals, beliefs, and intentions of others (i.e., systems that have theory of mind, ToM). In this research, we propose a cognitive ToM framework that uses a well-known theory of decisions from experience to construct a computational representation of ToM. Instance-based learning theory (IBLT) is used to construct a cognitive model that generates ToM from the observation of other agents' behavior. The IBL model of the (...) observer distinguishes itself from previous models of ToM that make unreasonable assumptions about human cognition, are hand-crafted for particular settings, complex, or unable to explain a cognitive development of ToM compared to human's ToM. The IBL model learns from the observation of goal-directed agents' behavior in a gridworld navigation task, and it infers and predicts the behaviors of the agents in new gridworlds across different degrees of decision complexity in similar ways to the way human observers do. We provide evidence for the alignment of the IBL observer's predictions under various levels of decision complexity. We also advance the demonstration of the IBL predictions using a classic test of false beliefs (the Sally–Anne test), which is commonly used to test ToM in humans. We discuss our results and the potential of the IBL observer model to improve human–machine interactions. (shrink)

The purpose of cognitive models is to make predictive simulations of human behaviour, but this is often done at the aggregate level. Cranford, Gonzalez, Aggarwal, Cooney, Tambe, and Lebiere show that they can automatically customize a model to a particular individual on‐the‐fly, and use it to make specific predictions about their next actions, in the context of a particular cybersecurity game.

This work is an initial step toward developing a cognitive theory of cyber deception. While widely studied, the psychology of deception has largely focused on physical cues of deception. Given that present‐day communication among humans is largely electronic, we focus on the cyber domain where physical cues are unavailable and for which there is less psychological research. To improve cyber defense, researchers have used signaling theory to extended algorithms developed for the optimal allocation of limited defense resources by using deceptive (...) signals to trick the human mind. However, the algorithms are designed to protect against adversaries that make perfectly rational decisions. In behavioral experiments using an abstract cybersecurity game (i.e., Insider Attack Game), we examined human decision‐making when paired against the defense algorithm. We developed an instance‐based learning (IBL) model of an attacker using the Adaptive Control of Thought‐Rational (ACT‐R) cognitive architecture to investigate how humans make decisions under deception in cyber‐attack scenarios. Our results show that the defense algorithm is more effective at reducing the probability of attack and protecting assets when using deceptive signaling, compared to no signaling, but is less effective than predicted against a perfectly rational adversary. Also, the IBL model replicates human attack decisions accurately. The IBL model shows how human decisions arise from experience, and how memory retrieval dynamics can give rise to cognitive biases, such as confirmation bias. The implications of these findings are discussed in the perspective of informing theories of deception and designing more effective signaling schemes that consider human bounded rationality. (shrink)

How, and how well, do people switch between exploration and exploitation to search for and accumulate resources? We study the decision processes underlying such exploration/exploitation trade‐offs using a novel card selection task that captures the common situation of searching among multiple resources (e.g., jobs) that can be exploited without depleting. With experience, participants learn to switch appropriately between exploration and exploitation and approach optimal performance. We model participants' behavior on this task with random, threshold, and sampling strategies, and find that (...) a linear decreasing threshold rule best fits participants' results. Further evidence that participants use decreasing threshold‐based strategies comes from reaction time differences between exploration and exploitation; however, participants themselves report non‐decreasing thresholds. Decreasing threshold strategies that “front‐load” exploration and switch quickly to exploitation are particularly effective in resource accumulation tasks, in contrast to optimal stopping problems like the Secretary Problem requiring longer exploration. (shrink)

Using belief elicitation, the paper investigates the process of belief formation and evolution in a signaling game in which a common prior is not induced. Both prior and posterior beliefs of Receivers about Senders’ types are elicited, as well as beliefs of Senders about Receivers’ strategies. In the experiment, subjects often start with diffuse uniform beliefs and update them in view of observations. However, the speed of updating is influenced by the strength of initial beliefs. An interesting result is that (...) beliefs about the prior distribution of types are updated slower than posterior beliefs, which incorporate Senders’ strategies. In the medium run, for some specifications of game parameters, this leads to outcomes being significantly different from the outcomes of the game in which a common prior is induced. It is also shown that elicitation of beliefs does not considerably change the pattern of play in this game. (shrink)

One of the paradigms in judgment and decision-making involves decision-makers sample information before making a final consequential choice. In the sampling paradigm, certain computational models have been proposed where a set of single or distribution parameters is calibrated to the choice proportions of a group of participants. However, currently little is known on how aggregate and hierarchical models would account for choices made by individual participants in the sampling paradigm. In this paper, we test the ability of aggregate and hierarchical (...) models to explain choices made by individual participants. Several models, Ensemble, Cumulative Prospect Theory, Best Estimation and Simulation Techniques, Natural-Mean Heuristic, and Instance-Based Learning, had their parameters calibrated to individual choices in a large dataset involving the sampling paradigm. Later, these models were generalized to two large datasets in the sampling paradigm. Results revealed that the aggregate models accounted for individual choices better than hierarchical models upon generalization to problems that were like those encountered during calibration. Furthermore, the CPT model, which relies on differential valuing of gains and losses, respectively, performed better than other models during calibration and generalization on datasets with similar set of problems. The IBL model, relying on recency and frequency of sampled information, and the NMH model, relying on frequency of sampled information, performed better than other models during generalization to a challenging dataset. Sequential analyses of results from different models showed how these models accounted for transitions from the last sample to final choice in human data. We highlight the implications of using aggregate and hierarchical models in explaining individual choices from experience. (shrink)

Prior research in judgment and decision making (JDM) has investigated the effect of problem framing on human preferences. Furthermore, research in JDM documented the absence of such reversal of preferences when making decisions from experience. However, little is known about the effect of context on preferences under the combined influence of problem framing and problem format. Also, little is known about how cognitive models would account for human choices in different problem frames and types (general/specific) in the experience format. One (...) of the primary objectives of this research is to investigate the presence of preference reversals under the influence of problem framing (gain/loss), problem format (experience/description), and problem type (general/specific). Another objective of this research is to develop cognitive models to account for human choices across different problem frames and types in the experience format. A total of 320 participants from India were randomly assigned to one of eight between-subjects conditions that differed in problem frame, format, and type. Results revealed preference reversals in the description condition; however, they were absent in the experience condition. Moreover, preference reversals were less pronounced in the general problem framing compared to the specific problem framing. Furthermore, specific problems influenced risk-seeking behavior among participants. We developed cognitive and heuristics models using instance-based learning theory and natural mean heuristic. Results reveal models’ dependency on recent and frequent observations during information sampling. These experience-based cognitive models could help build artificial intelligence models with fewer preference reversals. (shrink)

Quantum probability (QP) theory provides an alternative account of empirical phenomena in decision making that classical probability (CP) theory cannot explain. Cognitive architectures combine probabilistic mechanisms with symbolic knowledge-based representations (e.g., heuristics) to address effects that motivate QP. They provide simple and natural explanations of these phenomena based on general cognitive processes such as memory retrieval, similarity-based partial matching, and associative learning.

There is a growing chorus of voices in the scientific community calling for greater openness in the sharing of raw data that lead to a publication. In this commentary, we discuss the merits of sharing, common concerns that are raised, and practical issues that arise in developing a sharing policy. We suggest that the cognitive science community discuss the topic and establish a data-sharing policy.

Prior research has used an Interactive Landslide Simulator tool to investigate human decision making against landslide risks. It has been found that repeated feedback in the ILS tool about damages due to landslides causes an improvement in human decisions against landslide risks. However, little is known on how theories of learning from feedback would account for human decisions in the ILS tool. The primary goal of this paper is to account for human decisions in the ILS tool via computational models (...) based upon reinforcement learning and to explore the model mechanisms involved when people make decisions in the ILS tool. Four different reinforcement-learning models were developed and evaluated in their ability to capture human decisions in an experiment involving two conditions in the ILS tool. The parameters of an Expectancy-Valence model, two Prospect-Valence-Learning models, a combination EV-PU model, and a random model were calibrated to human decisions in the ILS tool across the two conditions. Later, different models with their calibrated parameters were generalized to data collected in an experiment involving a new condition in ILS. When generalized to this new condition, the PVL-2 model’s parameters of both damage-feedback conditions outperformed all other RL models. We highlight the implications of our results for decision making against landslide risks. (shrink)

Computational models of learning and the theories they represent are often validated by calibrating them to human data on decision outcomes. However, only a few models explain the process by which these decision outcomes are reached. We argue that models of learning should be able to reflect the process through which the decision outcomes are reached, and validating a model on the process is likely to help simultaneously explain both the process as well as the decision outcome. To demonstrate the (...) proposed validation, we use a large dataset from the Technion Prediction Tournament and an existing Instance-based Learning model. We present two ways of calibrating the model’s parameters to human data: on an outcome measure and on a process measure. In agreement with our expectations, we find that calibrating the model on the process measure helps to explain both the process and outcome measures compared to calibrating the model on the outcome measure. These results hold when the model is generalized to a different dataset. We discuss implications for explaining the process and the decision outcomes in computational models of learning. (shrink)

Corrigendum to "Exploration and exploitation during information search and consequential choice".