Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). In this article, (...) we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state–action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model. (shrink)

Can the output of human cognition be predicted from the assumption that it is an optimal response to the information-processing demands of the environment? A methodology called rational analysis is described for deriving predictions about cognitive phenomena using optimization assumptions. The predictions flow from the statistical structure of the environment and not the assumed structure of the mind. Bayesian inference is used, assuming that people start with a weak prior model of the world which they integrate with experience to develop (...) stronger models of specific aspects of the world. Cognitive performance maximizes the difference between the expected gain and cost of mental effort. Memory performance can be predicted on the assumption that retrieval seeks a maximal trade-off between the probability of finding the relevant memories and the effort required to do so; in categorization performance there is a similar trade-off between accuracy in predicting object features and the cost of hypothesis formation; in casual inference the trade-off is between accuracy in predicting future events and the cost of hypothesis formation; and in problem solving it is between the probability of achieving goals and the cost of both external and mental problem-solving search. The implemention of these rational prescriptions in neurally plausible architecture is also discussed. (shrink)

For many years, the game of chess has provided an invaluable task environment for research on cognition, in particular on the differences between novices and experts and the learning that removes these differences, and upon the structure of human memory and its paramaters. The template theory presented by Gobet and Simon based on the EPAM theory offers precise predictions on cognitive processes during the presentation and recall of chess positions. This article describes the behavior of CHREST, a computer implementation of (...) the template theory, in a memory task when the presentation time is varied from one second to sixty, on the recall of game and random positions, and compares the model to human data. Strong players are better than weak players in both types of positions, especially with long presentation times, but even after brief presentations. CHREST predicts the data, both qualitatively and quantitatively. Strong players' superiority with random positions is explained by the large number of chunks they hold in LTM. Their excellent recall with short presentation times is explained by templates, a special class of chunks. CHREST is compared to other theories of chess skill, which either cannot account for the superiority of Masters in random positions or predict too strong a performance of Masters in such positions. (shrink)