Marr’s seminal distinction between computational, algorithmic, and implementational levels of analysis has inspired research in cognitive science for more than 30 years. According to a widely-used paradigm, the modelling of cognitive processes should mainly operate on the computational level and be targeted at the idealised competence, rather than the actual performance of cognisers in a specific domain. In this paper, we explore how this paradigm can be adopted and revised to understand mathematical problem solving. The computational-level approach applies methods from (...) computational complexity theory and focuses on optimal strategies for completing cognitive tasks. However, human cognitive capacities in mathematical problem solving are essentially characterised by processes that are computationally sub-optimal, because they initially add to the computational complexity of the solutions. Yet, these solutions can be optimal for human cognisers given the acquisition and enactment of mathematical practices. Here we present diagrams and the spatial manipulation of symbols as two examples of problem solving strategies that can be computationally sub-optimal but humanly optimal. These aspects need to be taken into account when analysing competence in mathematical problem solving. Empirically informed considerations on enculturation can help identify, explore, and model the cognitive processes involved in problem solving tasks. The enculturation account of mathematical problem solving strongly suggests that computational-level analyses need to be complemented by considerations on the algorithmic and implementational levels. The emerging research strategy can help develop algorithms that model what we call enculturated cognitive optimality in an empirically plausible and ecologically valid way. (shrink)

The appropriate methodology for psychological research depends on whether one is studying mental algorithms or their implementation. Mental algorithms are abstract specifications of the steps taken by procedures that run in the mind. Implementational issues concern the speed and reliability of these procedures. The algorithmic level can be explored only by studying across-task variation. This contrasts with psychology's dominant methodology of looking for within-task generalities, which is appropriate only for studying implementational issues.The implementation-algorithm distinction is related to a number of (...) other “levels” considered in cognitive science. Its realization in Anderson's ACT theory of cognition is discussed. Research at the algorithmic level is more promising because it is hard to make further fundamental scientific progress at the implementational level with the methodologies available. Protocol data, which are appropriate only for algorithm-level theories, provide a richer source than data at the implementational level. Research at the algorithmic level will also yield more insight into fundamental properties of human knowledge because it is the level at which significant learning transitions are defined.The best way to study the algorithmic level is to look for differential learning outcomes in pedagogical experiments that manipulate instructional experience. This provides control and prediction in realistically complex learning situations. The intelligent tutoring paradigm provides a particularly fruitful way to implement such experiments.The implications of this analysis for the issue of modularity of mind, the status of language, research on human/computer interaction, and connectionist models are also examined. (shrink)

I consider the role of cognitive modeling in cognitive science. Modeling, and the computers that enable it, are central to the field, but the role of modeling is often misunderstood. Models are not intended to capture fully the processes they attempt to elucidate. Rather, they are explorations of ideas about the nature of cognitive processes. In these explorations, simplification is essential—through simplification, the implications of the central ideas become more transparent. This is not to say that simplification has no downsides; (...) it does, and these are discussed. I then consider several contemporary frameworks for cognitive modeling, stressing the idea that each framework is useful in its own particular ways. Increases in computer power (by a factor of about 4 million) since 1958 have enabled new modeling paradigms to emerge, but these also depend on new ways of thinking. Will new paradigms emerge again with the next 1,000‐fold increase? (shrink)

First, a number of previous theories of intrinsic motivation are reviewed. Then, several studies of highly motivating computer games are described. These studies focus on what makes the games fun, not on what makes them educational. Finally, with this background, a rudimentary theory of intrinsically motivating instruction is developed, based on three categories: challenge, fantasy, and curiosity.Challenge is hypothesized to depend on goals with uncertain outcomes. Several ways of making outcomes uncertain are discussed, including variable difficulty level, multiple level goals, (...) hidden information, and randomness. Fantasy is claimed to have both cognitive and emotional advantages in designing instructional environments. A distinction is made between extrinsic fantasies that depend only weakly on the skill used in a game, and intrinsic fantasies that are intimately related to the use of the skill. Curiosity is separated into sensory and cognitive components, and it is suggested that cognitive curiosity can be aroused by making learners believe their knowledge structures are incomplete, inconsistent, or unparsimonious. (shrink)

Systematic errors In performance are an important aspect of human behavior that have not received adequate explanation. One such systematic error is termed postcompletion error; a typical example is leaving one's card In the automatic teller after withdrawing cash. This type of error seems to occur when people have an extra step to perform in a procedure after the main goal has been satisfied. The fact that people frequently make this type of error, but do not make this error every (...) time, may best be explained by considering the working memory load at the time the step is to be performed: The error is made when the load on working memory is high, but will not be made when the load is low. A model of performance In the task was constructed using Just and Carpenter's (1992) CAPS that predicted that high working memory load should be associated with postcompletion errors. Two experiments confirmed that such errors can be produced in a laboratory as well as a naturalistic setting, and that the conditions under which the CAPS model makes the error are consistent with the conditions under which the errors occur in the laboratory. (shrink)

Many of the errors that occur in children' subtraction are due to the use of incorrect strategies rather than to the incorrect recall of number facts. A production system is presented for performing written subtraction which is consistent with an earlier analysis of the nature of such a cognitive skill. Most of the incorrect strategies used by schoolchildren can be accounted for in a principled way by simple changes in the production system, such as the omission of individual rules or (...) the inclusion of rules appropriate to other arithmetical tasks. The production system model is evaluated against a corpus of over 1500 subtraction problems done by 10‐year olds and is shown to account for about two‐thirds of the (nonnumber fact) errors. It also provides an alternative, simpler interpretation of the subtraction errors analysed by Brown and Burton (1978). Some implications for teaching are discussed. (shrink)

Understanding the nature of errors in a simple, rule‐based task—programming a VCR—required analyzing the interactions among human cognition, the artifact, and the task. This analysis was guided by least‐effort principles and yielded a control structure that combined a rule hierarchy task‐to‐device with display‐based difference‐reduction. A model based on this analysis was used to trace action protocols collected from participants as they programmed a simulated VCR. Trials that ended without success (the show was not correctly programmed) were interrogated to yield insights (...) regarding problems in acquiring the control structure. For successful trials (the show was correctly programmed), steps that the model would make were categorized as matches to the model; steps that the model would not make were violations of the model. The model was able to trace the vast majority of correct keystrokes and yielded a business‐as‐usual account of the detection and correction of errors. Violations of the model fell into one of two fundamental categories. The model provided insights into certain subcategories of errors; whereas, regularities within other subcategories of error suggested limitations to the model. Although errors were rare when compared to the total number of correct actions, they were important. Errors were made on 4% of the keypresses that, if not detected, would have prevented two‐thirds of the shows from being successfully recorded. A misprogrammed show is a minor annoyance to the user. However, devices with the approximate complexity of a VCR are ubiquitous and have found their way into emergency rooms, airplane cockpits, power plants, and so on. Errors of ignorance may be reduced by training; however, errors in the routine performance of skilled users can only be reduced by design. (shrink)

We report an empirical study of elementary algebra errors, conducted in three separate schools. The errors are diagnosed using mal‐rules, as proposed by Sleeman (1984, 1,985). Our analysis uncovers the following properties of algebra mal‐rules: The frequency of mal‐rules is severely skewed, there are many infrequent mal‐rules and few frequent ones; mal‐rules are very unstable, students typically use mal‐rules very irregularly; different mal‐rules have explanatory power in different schools (many of our most powerful mal‐rules are previously unreported); mal‐rule diagnosis Is (...) more successful with more skilled students; students' confidence ratings do not partition the total set of mal‐rules, every mal‐rule we find is associated with high confidence ratings by at least one student. The Implications of our data for cognitive theories of error generation are discussed. Contrary to commonplace assumptions, we argue that It is impossible to make a clear distinction between slips and mistakes; most errors depend on properties of the knowledge base and the cognitive architecture. Errors In a procedural skill cannot be assumed to be purely syntactic In orgin. (shrink)

Students systematically and deliberately apply rule‐based but erroneous algorithms to solving unfamiliar arithmetic problems. These algorithms result in erroneous solutions termed rational errors. Computationally, students' erroneous algorithms can be represented by perturbations or bugs in otherwise correct arithmetic algorithms (Brown & VanLehn, 1980; Langley & Ohilson, 1984; VanLehn, 1983, 1986, 1990; Young S O'Sheo, 1981). Bugs are useful for describing how rational errors occur but bugs are not sufficient for explaining their origin. A possible explanation for this is that rational (...) errors are the result of incorrect induction from examples. This prediction is termed the “induction hypothesis” (VanLehn, 1986). The purpose of the present study was to: (a) expand on post formulations of the induction hypothesis, and (b) use a new methodology to test the induction hypothesis more carefully than has been done previously. The first step involved teaching participants a new number system called NewAbacus, a written modification of the abacus system. The second step consisted of dividing them into different groups, where each individual received an example of only one port of the NewAbacus addition algorithm. During the third and final step, participants were instructed to solve both familiar and unfamiliar types of addition problems in NewAbacus. The induction hypothesis was supported by using both empirical and computational investigations. (shrink)

Online educational technologies offer opportunities for providing individualized feedback and detailed profiles of students' skills. Yet many technologies for mathematics education assess students based only on the correctness of either their final answers or responses to individual steps. In contrast, examining the choices students make for how to solve the equation and the ways in which they might answer incorrectly offers the opportunity to obtain a more nuanced perspective of their algebra skills. To automatically make sense of step‐by‐step solutions, we (...) propose a Bayesian inverse planning model for equation solving that computes an assessment of a learner's skills based on her pattern of errors in individual steps and her choices about what sequence of problem‐solving steps to take. Bayesian inverse planning builds on existing machine learning tools to create a generative model relating (mis)‐understandings to equation solving choices. Two behavioral experiments demonstrate that the model can interpret people's equation solving and that its assessments are consistent with those of experienced teachers. A third experiment uses this model to tailor guidance for learners based on individual differences in misunderstandings, closing the loop between assessing understanding, and using that assessment within an educational technology. Finally, because the bottleneck in applying inverse planning to a new domain is in creating the model of possible student misunderstandings, we show how to combine inverse planning with an existing production rule model to make inferences about student misunderstandings of fraction arithmetic. (shrink)

Watching another person take actions to complete a goal and making inferences about that person's knowledge is a relatively natural task for people. This ability can be especially important in educational settings, where the inferences can be used for assessment, diagnosing misconceptions, and providing informative feedback. In this paper, we develop a general framework for automatically making such inferences based on observed actions; this framework is particularly relevant for inferring student knowledge in educational games and other interactive virtual environments. Our (...) approach relies on modeling action planning: We formalize the problem as a Markov decision process in which one must choose what actions to take to complete a goal, where choices will be dependent on one's beliefs about how actions affect the environment. We use a variation of inverse reinforcement learning to infer these beliefs. Through two lab experiments, we show that this model can recover people's beliefs in a simple environment, with accuracy comparable to that of human observers. We then demonstrate that the model can be used to provide real-time feedback and to model data from an existing educational game. (shrink)

A general model of problem‐solving processes based on misconception elimination is presented to simulate both impasses and solving processes. The model operates on goal‐related rules and a set of constraint rules in the form of “if (state or goal), do not (Action)” for the explicit constraints in the instructions and the implicit constraints that come from misconceptions of legal moves. When impasses occur, a constraint elimination mechanism is applied. Because successive eliminations of implicit constraints enlarge the problem space and have (...) an effect on planning, the model integrates “plan‐based” and “constraint‐based” approaches to problem‐solving behavior.Simulating individual protocols of Tower of Hanoi situations shows that the model, which has a proper set of constraints, predicts a single move with no alternative on about 61% of the movements and that protocols are quite successfully simulated movement by movement. Finally, it is shown that many features of previous models are embedded in the constraint elimination model. (shrink)

When, how, and why students use conceptual knowledge during math problem solving is not well understood. We propose that when solving routine problems, students are more likely to recruit conceptual knowledge if their procedural knowledge is weak than if it is strong, and that in this context, metacognitive processes, specifically feelings of doubt, mediate interactions between procedural and conceptual knowledge. To test these hypotheses, in two studies (Ns = 64 and 138), university students solved fraction and decimal arithmetic problems while (...) thinking aloud; verbal protocols and written work were coded for overt uses of conceptual knowledge and displays of doubt. Consistent with the hypotheses, use of conceptual knowledge during calculation was not significantly positively associated with accuracy, but was positively associated with displays of doubt, which were negatively associated with accuracy. In Study 1, participants also explained solutions to rational arithmetic problems; using conceptual knowledge in this context was positively correlated with calculation accuracy, but only among participants who did not use conceptual knowledge during calculation, suggesting that the correlation did not reflect “online” effects of using conceptual knowledge. In Study 2, participants also completed a nonroutine problem‐solving task; displays of doubt on this task were positively associated with accuracy, suggesting that metacognitive processes play different roles when solving routine and nonroutine problems. We discuss implications of the results regarding interactions between procedural knowledge, conceptual knowledge, and metacognitive processes in math problem solving. (shrink)