Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statis- tics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement record- ing systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and neuroscience. During perceptual experience, association areas in the (...) brain capture bottom-up patterns of activation in sensory-motor areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The stor- age and reactivation of perceptual symbols operates at the level of perceptual components – not at the level of holistic perceptual expe- riences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a com- mon frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift, run) and introspec- tion (e.g., compare, memory, happy, hungry). Once established, these simulators implement a basic conceptual system that represents types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and ab- stract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinato- rially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a per- ceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal sym- bol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored. (shrink)

It is generally accepted that there is something special about reasoning by using mental images. The question of how it is special, however, has never been satisfactorily spelled out, despite more than thirty years of research in the post-behaviorist tradition. This article considers some of the general motivation for the assumption that entertaining mental images involves inspecting a picture-like object. It sets out a distinction between phenomena attributable to the nature of mind to what is called the cognitive architecture, and (...) ones that are attributable to tacit knowledge used to simulate what would happen in a visual situation. With this distinction in mind, the paper then considers in detail the widely held assumption that in some important sense images are spatially displayed or are depictive, and that examining images uses the same mechanisms that are deployed in visual perception. I argue that the assumption of the spatial or depictive nature of images is only explanatory if taken literally, as a claim about how images are physically instantiated in the brain, and that the literal view fails for a number of empirical reasons – for example, because of the cognitive penetrability of the phenomena cited in its favor. Similarly, while it is arguably the case that imagery and vision involve some of the same mechanisms, this tells us very little about the nature of mental imagery and does not support claims about the pictorial nature of mental images. Finally, I consider whether recent neuroscience evidence clarifies the debate over the nature of mental images. I claim that when such questions as whether images are depictive or spatial are formulated more clearly, the evidence does not provide support for the picture-theory over a symbol-structure theory of mental imagery. Even if all the empirical claims were true, they do not warrant the conclusion that many people have drawn from them: that mental images are depictive or are displayed in some (possibly cortical) space. Such a conclusion is incompatible with what is known about how images function in thought. We are then left with the provisional counterintuitive conclusion that the available evidence does not support rejection of what I call the “null hypothesis”; namely, that reasoning with mental images involves the same form of representation and the same processes as that of reasoning in general, except that the content or subject matter of thoughts experienced as images includes information about how things would look. (shrink)

We discuss external and internal graphical and linguistic representational systems. We argue that a cognitive theory of peoples' reasoning performance must account for (a) the logical equivalence of inferences expressed in graphical and linguistic form, and (b) the implementational differences that affect facility of inference. Our theory proposes that graphical representation limit abstraction and thereby aid “processibility”. We discuss the ideas of specificity and abstraction, and their cognitive relevance. Empirical support both comes from tasks which involve the manipulation of external (...) graphics and tasks that do not. For the former, we take Euler's (1772) circles, provide a novel computational reconstruction, show how it captures abstractions, and contrast it with earlier construals and with Johnson‐Laird's (1983) mental models representations. We demonstrate equivalence of the graphical Euler system, and the nongraphical mental models system. For tasks not involving manipulation of external graphics, we discuss text comprehension, and the mental performance of syllogisms. By positing an internal system with the same specificity as Euler's circles, we cover the mental models data, and generate new empirical predictions. Finally, we consider how the architecture of working memory explains why such specific representations are relatively easy to store. (shrink)

We present a new model of skilled performance in geometry proof problem solving called the Diagram Configuration model (DC). While previous models plan proofs in a step‐by‐step fashion, we observed that experts plan at a more abstract level: They focus on the key steps and skip the less important ones. DC models this abstract planning behavior by parsing geometry problem diagrams into perceptual chunks, called diagram configurations, which cue relevant schematic knowledge. We provide verbal protocol evidence that DC's schemas correspond (...) with the step‐skipping inferences experts make in their initial planning. We compare DC with other models of geometry expertise and then, in the final section, we discuss more general implications of our research. DC's reasoning has important similarities with Larkin's (1988) display‐based reasoning approach and Johnson‐Laird's (1983) mental model approach. DC's perceptually based schemas are a step towards a unified explanation of (1) experts' superior problem‐solving effectiveness, (2) experts' superior problem‐state memory, and (3) experts' ability, in certain domains, to solve relatively simple problems by pure forward inferencing. We also argue that the particular and efficient knowledge organization of DC challenges current theories of skill acquisition as it presents an end‐state of learning that is difficult to explain within such theories. Finally, we discuss the implications of DC for geometry instruction. (shrink)

This research aims to clarify, by constructing and testing a computer simulation, the use of multiple representations in problem solving, focusing on their role in visual reasoning. The model is motivated by extensive experimental evidence in the literature for the features it incorporates, but this article focuses on the system's structure. We illustrate the model's behavior by simulating the cognitive and perceptual processes of an economics expert as he teaches some well‐learned economics principles while drawing a graph on a blackboard. (...) Data in the experimental literature and concurrent verbal protocols were used to guide construction of a linked production system and parallel network, CaMeRa (Computation with Multiple Representations), that employs a “Mind's Eye” representation for pictorial information, consisting of a bitmap and associated node‐link structures. Propositional list structures are used to represent verbal information and reasoning. Small individual pieces from the different representations are linked on a sequential and temporary basis to form a reasoning and inferencing chain, using visually encoded information recalled to the Mind's Eye from long‐term memory and from cues recognized on an external display. CaMeRa, like the expert, uses the diagrammatic and verbal representations to complement one another, thus exploiting the unique advantages of each. (shrink)

That diagrams are analog, i.e., homomorphic, representations of some kind, and sentential representations are not, is a generally held intuition. In this paper, we develop a formal framework in which the claim can be stated and examined, and certain puzzles resolved. We start by asking how physical things can represent information in some target domain. We lay a basis for investigating possible homomorphisms by modeling both the physical medium and the target domain as sets of variables, each with a constraint (...) structure. When a homomorphism exists, the causality of the physical medium can provide a “free ride,” automatically determining the information that is needed to solve a problem. The modeling technique enables us to show how in sentential representations the structure of the 2-D space is used non-homomorphically to define tokens, and to provide a minimal homomorphism with respect to the ordering of the arguments for n-ary functions, predicates, and operators, but otherwise there is no homomorphism with the target domain. Our treatment also characterizes diagrams as a type of a more general class of physical representations in which physical causality plays a role in providing the information needed to solve a problem. (shrink)