A new diagnostic modeling system for automatically synthesizing a deep‐structure model of a student's misconceptions or bugs in his basic mathematical skills provides a mechanism for explaining why a student is making a mistake as opposed to simply identifying the mistake. This report is divided into four sections: The first provides examples of the problems that must be handled by a diagnostic model. It then introduces procedural networks as a general framework for representing the knowledge underlying a skill. The challenge (...) in designing this representation is to find one that facilitates the discovery of misconceptions or bugs existing in a particular student's encoding of this knowledge. The second section discusses some of the pedagogical issues that have emerged from the use of diagnostic models within an instructional system. This discussion is framed in the context of a computer‐based tutoring/gaming system developed to teach students and student teachers how to diagnose bugs strategically as well as how to provide a better understanding of the underlying structure of arithmetic skills. The third section describes our uses of an executable network as a tool for automatically diagnosing student behavior, for automatically generating “diagnostic” tests, and for judging the diagnostic quality of a given exam. Included in this section is a discussion of the success of this system in diagnosing 1300 school students from a data base of 20.000 test items. The last section discusses future research directions. (shrink)

Many of the errors that occur in children' subtraction are due to the use of incorrect strategies rather than to the incorrect recall of number facts. A production system is presented for performing written subtraction which is consistent with an earlier analysis of the nature of such a cognitive skill. Most of the incorrect strategies used by schoolchildren can be accounted for in a principled way by simple changes in the production system, such as the omission of individual rules or (...) the inclusion of rules appropriate to other arithmetical tasks. The production system model is evaluated against a corpus of over 1500 subtraction problems done by 10‐year olds and is shown to account for about two‐thirds of the (nonnumber fact) errors. It also provides an alternative, simpler interpretation of the subtraction errors analysed by Brown and Burton (1978). Some implications for teaching are discussed. (shrink)

This paper describes a generative theory of bugs. It claims that all bugs of a procedural skill can be derived by a highly constrained form of problem solving acting on incomplete procedures. These procedures are characterized by formal deletion operations that model incomplete learning and forgetting. The problem solver and the deletion operator have been constrained to make it impossible to derive “star‐bugs”—algorithms that are so absurd that expert diagnosticians agree that the alogorithm will never be observed as a bug. (...) Hence, the theory not only generates the observed bugs, it fails to generate star‐bugs.The theory has been tested on an extensive data base of bugs for multidigit subtraction that was collected with the aid of the diagnostic systems buggy and debuggy. In addition to predicting bug occurrence, by adoption of additional hypotheses, the theory also makes predictions about the frequency and stability of bugs, as well as the occurrence of certain latencies in processing time during testing. Arguments are given that the theory can be applied to domains other than subtraction and that it can be extended to provide a theory of procedural learning that accounts for bug acquisition. Lastly, particular care has been taken to make the theory principled so that it can not be tailored to fit any possible data. (shrink)