A new diagnostic modeling system for automatically synthesizing a deep‐structure model of a student's misconceptions or bugs in his basic mathematical skills provides a mechanism for explaining why a student is making a mistake as opposed to simply identifying the mistake. This report is divided into four sections: The first provides examples of the problems that must be handled by a diagnostic model. It then introduces procedural networks as a general framework for representing the knowledge underlying a skill. The challenge (...) in designing this representation is to find one that facilitates the discovery of misconceptions or bugs existing in a particular student's encoding of this knowledge. The second section discusses some of the pedagogical issues that have emerged from the use of diagnostic models within an instructional system. This discussion is framed in the context of a computer‐based tutoring/gaming system developed to teach students and student teachers how to diagnose bugs strategically as well as how to provide a better understanding of the underlying structure of arithmetic skills. The third section describes our uses of an executable network as a tool for automatically diagnosing student behavior, for automatically generating “diagnostic” tests, and for judging the diagnostic quality of a given exam. Included in this section is a discussion of the success of this system in diagnosing 1300 school students from a data base of 20.000 test items. The last section discusses future research directions. (shrink)

This paper describes a generative theory of bugs. It claims that all bugs of a procedural skill can be derived by a highly constrained form of problem solving acting on incomplete procedures. These procedures are characterized by formal deletion operations that model incomplete learning and forgetting. The problem solver and the deletion operator have been constrained to make it impossible to derive “star‐bugs”—algorithms that are so absurd that expert diagnosticians agree that the alogorithm will never be observed as a bug. (...) Hence, the theory not only generates the observed bugs, it fails to generate star‐bugs.The theory has been tested on an extensive data base of bugs for multidigit subtraction that was collected with the aid of the diagnostic systems buggy and debuggy. In addition to predicting bug occurrence, by adoption of additional hypotheses, the theory also makes predictions about the frequency and stability of bugs, as well as the occurrence of certain latencies in processing time during testing. Arguments are given that the theory can be applied to domains other than subtraction and that it can be extended to provide a theory of procedural learning that accounts for bug acquisition. Lastly, particular care has been taken to make the theory principled so that it can not be tailored to fit any possible data. (shrink)