Despite the accumulation of substantial cognitive science research relevant to education, there remains confusion and controversy in the application of research to educational practice. In support of a more systematic approach, we describe the Knowledge-Learning-Instruction (KLI) framework. KLI promotes the emergence of instructional principles of high potential for generality, while explicitly identifying constraints of and opportunities for detailed analysis of the knowledge students may acquire in courses. Drawing on research across domains of science, math, and language learning, we illustrate the (...) analyses of knowledge, learning, and instructional events that the KLI framework affords. We present a set of three coordinated taxonomies of knowledge, learning, and instruction. For example, we identify three broad classes of learning events (LEs): (a) memory and fluency processes, (b) induction and refinement processes, and (c) understanding and sense-making processes, and we show how these can lead to different knowledge changes and constraints on optimal instructional choices. (shrink)

The article first addresses the importance of cognitive modeling, in terms of its value to cognitive science (as well as other social and behavioral sciences). In particular, it emphasizes the use of cognitive architectures in this undertaking. Based on this approach, the article addresses, in detail, the idea of a multi-level approach that ranges from social to neural levels. In physical sciences, a rigorous set of theories is a hierarchy of descriptions/explanations, in which causal relationships among entities at a high (...) level can be reduced to causal relationships among simpler entities at a more detailed level. We argue that a similar hierarchy makes possible an equally productive approach toward cognitive modeling. The levels of models that we conceive in relation to cognition include, at the highest level, sociological/anthropological models of collective human behavior, behavioral models of individual performance, cognitive models involving detailed mechanisms, representations, and processes, as well as biological/physiological models of neural circuits, brain regions, and other detailed biological processes. (shrink)

Open‐ended tasks can be decomposed into the three levels of Newell's Cognitive Band: the Unit‐Task level, the Operation level, and the Deliberate‐Act level. We analyzed the video game Co‐op Space Fortress at these levels, reporting both the match of a cognitive model to subject behavior and the use of electroencephalogram (EEG) to track subject cognition. The Unit Task level in this game involves coordinating with a partner to kill a fortress. At this highest level of the Cognitive Band, there is (...) a good match between subject behavior and the model. The EEG signals were also strong enough to track when Unit Tasks succeeded or failed. The intermediate Operation level in this task involves legs of flight to achieve a kill. The EEG signals associated with these operations are much weaker than the signals associated with the Unit Tasks. Still, it was possible to reconstruct subject play with much better than chance success. There were significant differences in the leg behavior of subjects and models. Model behavior did not provide a good basis for interpreting a subject's behavior at this level. At the lowest Deliberate‐Act level, we observed overlapping key actions, which the model did not display. Such overlapping key actions also frustrated efforts to identify EEG signals of motor actions. We conclude that the Unit‐task level is the appropriate level both for understanding open‐ended tasks and for using EEG to track the performance of open‐ended tasks. (shrink)

In his famous 1982 paper, Allen Newell [22, 23] introduced the notion of knowledge level to indicate a level of analysis, and prediction, of the rational behavior of a cognitive arti cial agent. This analysis concerns the investigation about the availability of the agent knowledge, in order to pursue its own goals, and is based on the so-called Rationality Principle (an assumption according to which "an agent will use the knowledge it has of its environment to achieve its goals" [22, (...) p. 17]. By using the Newell's own words: "To treat a system at the knowledge level is to treat it as having some knowledge, some goals, and believing it will do whatever is within its power to attain its goals, in so far as its knowledge indicates" [22, p. 13]. In the last decades, the importance of the knowledge level has been historically and system- atically downsized by the research area in cognitive architectures (CAs), whose interests have been mainly focused on the analysis and the development of mechanisms and the processes governing human and (arti cial) cognition. The knowledge level in CAs, however, represents a crucial level of analysis for the development of such arti cial general systems and therefore deserves greater research attention [17]. In the following, we will discuss areas of broad agree- ment and outline the main problematic aspects that should be faced within a Common Model of Cognition [12]. Such aspects, departing from an analysis at the knowledge level, also clearly impact both lower (e.g. representational) and higher (e.g. social) levels. (shrink)

Differing worldviews give interdisciplinary work value. However, these same differences are the primary hurdle to productive communication between disciplines. Here, we argue that philosophical issues of metaphysics and epistemology subserve many of the differences in language, methods and motivation that plague interdisciplinary fields like educational neuroscience. Researchers attempting interdisciplinary work may be unaware that issues of philosophy are intimately tied to the way research is performed and evaluated in different fields. As such, a lack of explicit discussion about these assumptions (...) leads to many conflicts in interdisciplinary work that masquerade as more superficial issues. To illustrate, we investigate how philosophical assumptions about the mind may influence researchers in educational neuroscience. The methods employed by researchers in this field are shaped by their metaphysical beliefs, and arguments around these issues can threaten accepted disciplinary ontologies. Additionally, how a researcher understands reduction in the special sciences and how they place their colleagues in this ontology constrains the scope of interdisciplinary projects. In encouraging researchers to explicitly discuss the philosophical assumptions underlying their research we hope to alleviate some of the conflict and establish realistic expectations for collaborative projects. (shrink)

Complex problem solving (CPS) emerged in the last 30 years in Europe as a new part of the psychology of thinking and problem solving. This paper introduces into the field and provides a personal view. Also, related concepts like macrocognition or operative intelligence will be explained in this context. Two examples for the assessment of CPS, Tailorshop and MicroDYN, are presented to illustrate the concept by means of their measurement devices. Also, the relation of complex cognition and emotion in the (...) CPS context is discussed. The question if CPS requires complex cognition is answered with a tentative “yes.”. (shrink)

The spacing effect is among the most widely replicated empirical phenomena in the learning sciences, and its relevance to education and training is readily apparent. Yet successful applications of spacing effect research to education and training is rare. Computational modeling can provide the crucial link between a century of accumulated experimental data on the spacing effect and the emerging interest in using that research to enable adaptive instruction. In this paper, we review relevant literature and identify 10 criteria for rigorously (...) evaluating computational models of the spacing effect. Five relate to evaluating the theoretic adequacy of a model, and five relate to evaluating its application potential. We use these criteria to evaluate a novel computational model of the spacing effect called the Predictive Performance Equation. Predictive Performance Equation combines elements of earlier models of learning and memory including the General Performance Equation, Adaptive Control of Thought—Rational, and the New Theory of Disuse, giving rise to a novel computational account of the spacing effect that performs favorably across the complete sets of theoretic and applied criteria. We implemented two other previously published computational models of the spacing effect and compare them to PPE using the theoretic and applied criteria as guides. (shrink)

This article describes rational analyses and cognitive models of Web users developed within information foraging theory. This is done by following the rational analysis methodology of (a) characterizing the problems posed by the environment, (b) developing rational analyses of behavioral solutions to those problems, and (c) developing cognitive models that approach the realization of those solutions. Navigation choice is modeled as a random utility model that uses spreading activation mechanisms that link proximal cues (information scent) that occur in Web browsers (...) to internal user goals. Web‐site leaving is modeled as an ongoing assessment by the Web user of the expected benefits of continuing at a Web site as opposed to going elsewhere. These cost–benefit assessments are also based on spreading activation models of information scent. Evaluations include a computational model of Web user behavior called Scent‐Based Navigation and Information Foraging in the ACT Architecture, and the Law of Surfing, which characterizes the empirical distribution of the length of paths of visitors at a Web site. (shrink)

An increasing number of learning goals refer to the acquisition of cognitive skills that can be described as ‘resource-based,’ as they require the availability, coordination, and integration of multiple underlying resources such as skills and knowledge facets. However, research on the support of cognitive skills rarely takes this resource-based nature explicitly into account. This is mirrored in prior research on mathematical argumentation and proof skills: Although repeatedly highlighted as resource-based, for example relying on mathematical topic knowledge, methodological knowledge, mathematical strategic (...) knowledge, and problem-solving skills, little evidence exists on how to support mathematical argumentation and proof skills based on its resources. To address this gap, a quasi-experimental intervention study with undergraduate mathematics students examined the effectiveness of different approaches to support both mathematical argumentation and proof skills and four of its resources. Based on the part-/whole-task debate from instructional design, two approaches were implemented during students’ work on proof construction tasks: (i) asequential approachfocusing and supporting each resource of mathematical argumentation and proof skills sequentially after each other and (ii) aconcurrent approachfocusing and supporting multiple resources concurrently. Empirical analyses show pronounced effects of both approaches regarding the resources underlying mathematical argumentation and proof skills. However, the effects of both approaches are mostly comparable, and only mathematical strategic knowledge benefits significantly more from the concurrent approach. Regarding mathematical argumentation and proof skills, short-term effects of both approaches are at best mixed and show differing effects based on prior attainment, possibly indicating an expertise reversal effect of the relatively short intervention. Data suggests that students with low prior attainment benefited most from the intervention, specifically from the concurrent approach. A supplementary qualitative analysis showcases how supporting multiple resources concurrently alongside mathematical argumentation and proof skills can lead to a synergistic integration of these during proof construction and can be beneficial yet demanding for students. Although results require further empirical underpinning, both approaches appear promising to support the resources underlying mathematical argumentation and proof skills and likely also show positive long-term effects on mathematical argumentation and proof skills, especially for initially weaker students. (shrink)

This work focused on the investigation of the question how the concept of relevance in Information Retrieval can be validated. The work is motivated by the consistent difficulties of defining the meaning of the concept, and by advances in the field of cognitive science. Analytical and empirical investigations are carried out with the aim of devising a principled approach to the validation of the concept. The foundation for this work was set by interpreting relevance as a phenomenon occurring within the (...) context of two systems: An IR system and the cognitive processing system of the user. In light of the cognitive interpretation of relevance, an analysis of the learnt lessons in cognitive science with regard to the validation of cognitive phenomena was conducted. It identified that construct validity constitutes the dominant approach to the validation of constructs in cognitive science. Construct validity constitutes a proposal for the conduction of validation in scenarios, where no direct observation of a phenomenon is possible. With regard to the limitations on direct observation of a construct (i.e. a postulated theoretic concept), it bases validation on the evaluation of its relations to other constructs. Based on the interpretation of relevance as a product of cognitive processing it was concluded, that the limitations with regard to direct observation apply to its investigation. The evaluation of its applicability to an IR context, focused on the exploration of the nomological network methodology. A nomological network constitutes an analytically constructed set of constructs and their relations. The construction of such a network forms the basis for establishing construct validity through investigation of the relations between constructs. An analysis focused on contemporary insights to the nomological network methodology identified two important aspects with regard to its application in IR. The first aspect is given by a choice of context and the identification of a pool of candidate constructs for the inclusion in the network. The second consists of identifying criteria for the selection of a set of constructs from the candidate pool. The identification of the pertinent constructs for the network was based on a review of the principles of cognitive exploration, and an analysis of the state of the art in text based discourse processing and reasoning. On that basis, a listing of known sub-processes contributing to the pertinent cognitive processing was presented. Based on the identification of a large number of potential candidates, the next step consisted of the inference of criteria for the selection of an initial set of constructs for the network. The investigation of these criteria focused on the consideration of pragmatic and meta-theoretical aspects. Based on a survey of experimental means in cognitive science and IR, five pragmatic criteria for the selection of constructs were presented. Consideration of meta-theoretically motivated criteria required to investigate what the specific challenges with regard to the validation of highly abstract constructs are. This question was explored based on the underlying considerations of the Information Processing paradigm and Newell’s (1994) cognitive bands. This led to the identification of a set of three meta-theoretical criteria for the selection of constructs. Based on the criteria and the demarcated candidate pool, an IR focused nomological network was defined. The network consists of the constructs of relevance and type and grade of word relatedness. A necessary prerequisite for making inferences based on a nomological network consists of the availability of validated measurement instruments for the constructs. To that cause, two validation studies targeting the measurement of the type and grade of relations between words were conducted. The clarification of the question of the validity of the measurement instruments enabled the application of the nomological network. A first step of the application consisted of testing if the constructs in the network are related to each other. Based on the alignment of measurements of relevance and the word related constructs it was concluded to be true. The relation between the constructs was characterized by varying the word related constructs over a large parameter space and observing the effect of this variation on relevance. Three hypotheses relating to different aspects of the relations between the word related constructs and relevance. It was concluded, that the conclusive confirmation of the hypotheses requires an extension of the experimental means underlying the study. Based on converging observations from the empirical investigation of the three hypotheses it was concluded, that semantic and associative relations distinctly differ with regard to their impact on relevance estimation. (shrink)

Computational modeling of the brain holds great promise as a bridge from brain to behavior. To fulfill this promise, however, it is not enough for models to be 'biologically plausible': models must be structurally accurate. Here, we analyze what this entails for so-called psychobiological models, models that address behavior as well as brain function in some detail. Structural accuracy may be supported by (1) a model's a priori plausibility, which comes from a reliance on evidence-based assumptions, (2) fitting existing data, (...) and (3) the derivation of new predictions. All three sources of support require modelers to be explicit about the ontology of the model, and require the existence of data constraining the modeling. For situations in which such data are only sparsely available, we suggest a new approach. If several models are constructed that together form a hierarchy of models, higher-level models can be constrained by lower-level models, and low-level models can be constrained by behavioral features of the higher-level models. Modeling the same substrate at different levels of representation, as proposed here, thus has benefits that exceed the merits of each model in the hierarchy on its own. (shrink)

Periods of relatively stable, rule‐like behavior alternated with short transition periods characterize cognitive development on reasoning tasks like the balance scale task. Each transition gives rise to an improvement in behavior, until a phase is reached in which performance is flawless or improvement is not worthwhile given the necessary extra effort. Several computational models have been developed to capture the developmental phenomena associated with the balance scale task. These models, which originate from different computational traditions, explain the main phenomena of (...) development. Recently, empirical phenomena have been reported that these models cannot easily accommodate. We propose a computational model that is implemented in ACT‐R and that is based on the evaluation of success of applied knowledge, combined with a mechanism to construct newknowledge by searching for differences between the left‐ and right‐hand sides of presented balance scale problems. This model accounts for the main empirical phenomena as well as for the recently reported empirical phenomena such as learning without feedback. (shrink)

The sciences use a wide range of visual devices, practices, and imaging technologies. This diversity points to an important repertoire of visual methods that scientists use to adapt representations to meet the varied demands that their work places on cognitive processes. This paper identifies key features of the use of visualization in a range of scientific domains and considers the implications of this repertoire for understanding scientists as cognitive agents.

This paper brings the intellectual tools of cognitive science to bear on resolving the “paradox of the active user” [Interfacing Thought: Cognitive Aspects of Human–Computer Interaction, Cambridge, MIT Press, MA, USA]—the persistent use of inefficient procedures in interactive tasks by experienced or even expert users when demonstrably more efficient procedures exist. The goal of this paper is to understand the roots of this paradox by finding regularities in these inefficient procedures. We examine three very different data sets. For each data (...) set, we first satisfy ourselves that the preferred procedures used by some subjects are indeed less efficient than the recommended procedures. We then amass evidence, for each set, and conclude that when a preferred procedure is used instead of a more efficient, recommended procedure, the preferred procedure tends to have two major characteristics: (1) the preferred procedure is a well‐practiced, generic procedure that is applicable either within the same task environment in different contexts or across different task environments, and (2) the preferred procedure is composed of interactive components that bring fast, incremental feedback on the external problem states. The support amassed for these characteristics leads to a new understanding of the paradox. In interactive tasks, people are biased towards the use of general procedures that start with interactive actions. These actions require much less cognitive effort as each action results in an immediate change to the external display that, in turn, cues the next action. Unfortunately for the users, the bias to use interactive unit tasks leads to a path that requires more effort in the long run. Our data suggest that interactive behavior is composed of a series of distributed choices; that is, people seldom make a once‐and‐for‐all decision on procedures. This series of biased selection of interactive unit tasks often leads to a stable suboptimal level of performance. (shrink)