Despite its strengths, Leech et al.'s model fails to address the important benefits that derive from self-explanation and task feedback in analogical reasoning development. These components encourage explicit, self-reflective processes that do not necessarily link to knowledge accretion. We wonder, therefore, what mechanisms can be included within a connectionist framework to model self-reflective involvement and its beneficial consequences.

The ability to combine words into novel sentences has been used to argue that humans have symbolic language production abilities. Critiques of connectionist models of language often center on the inability of these models to generalize symbolically (Fodor & Pylyshyn, 1988; Marcus, 1998). To address these issues, a connectionist model of sentence production was developed. The model had variables (role‐concept bindings) that were inspired by spatial representations (Landau & Jackendoff, 1993). In order to take advantage of these variables, a novel (...) dual‐pathway architecture with event semantics is proposed and shown to be better at symbolic generalization than several variants. This architecture has one pathway for mapping message content to words and a separate pathway that enforces sequencing constraints. Analysis of the model's hidden units demonstrated that the model learned different types of information in each pathway, and that the model's compositional behavior arose from the combination of these two pathways. The model's ability to balance symbolic and statistical behavior in syntax acquisition and to model aphasic double dissociations provided independent support for the dual‐pathway architecture. (shrink)

Many of the mathematicians and scientists who guided the development of digital computers in the late 1940s, such as Alan Turing and John von Neumann, saw these new devices not just as tools for calculation but as devices that might employ the same principles as are exhibited in rational human thought. Thus, a subfield of what came to be called computer science assumed the label artificial intelligence. The idea of building artificial systems which could exhibit intelligent behavior comparable to that (...) of humans was a heady prospect, and the claims made on behalf of AI during the 1950s and 1960s were impossibly ambitious. Despite some theoretical and applied successes within the field, serious problems soon became evident. Instead of fulfilling the goal of quickly producing artificial intelligent agents which could compete with or outperform human beings, by the 1970s and 1980s AI had settled into a pattern of slower but real progress in modeling or simulating aspects of human intelligence. (shrink)

The relation between logic and thought has long been controversial, but has recently influenced theorizing about the nature of mental processes in cognitive science. One prominent tradition argues that to explain the systematicity of thought we must posit syntactically structured representations inside the cognitive system which can be operated upon by structure sensitive rules similar to those employed in systems of natural deduction. I have argued elsewhere that the systematicity of human thought might better be explained as resulting from the (...) fact that we have learned natural languages which are themselves syntactically structured. According to this view, symbols of natural language are external to the cognitive processing system and what the cognitive system must learn to do is produce and comprehend such symbols. In this paper I pursue that idea by arguing that ability in natural deduction itself may rely on pattern recognition abilities that enable us to operate on external symbols rather than encodings of rules that might be applied to internal representations. To support this suggestion, I present a series of experiments with connectionist networks that have been trained to construct simple natural deductions in sentential logic. These networks not only succeed in reconstructing the derivations on which they have been trained, but in constructing new derivations that are only similar to the ones on which they have been trained. (shrink)

This paper reviews four significant advances on the feedforward architecture that has dominated discussions of connectionism. The first involves introducing modularity into networks by employing procedures whereby different networks learn to perform different components of a task, and a Gating Network determines which network is best equiped to respond to a given input. The second consists in the use of recurrent inputs whereby information from a previous cycle of processing is made available on later cycles. The third development involves developing (...) compressed representations of strings in which there is no longer an explicit encoding of the components but where information about the structure of the original string can be recovered and so is present functionally. The final advance entails using connectionist learning procedures not just to change weights in networks but to change the patterns used as inputs to the network. These advances significantly increase the usefulness of connectionist networks for modeling human cognitive performance by, among other things, providing tools for explaining the productivity and systematicity of some mental activities, and developing representations that are sensitive to the content they are to represent. (shrink)

Connectionist attention to variables has been too restricted in two ways. First, it has not exploited certain ways of doing without variables in the symbolic arena. One variable-avoidance method, that of logical combinators, is particularly well established there. Secondly, the attention has been largely restricted to variables in long-term rules embodied in connection weight patterns. However, short-lived bodies of information, such as sentence interpretations or inference products, may involve quantification. Therefore short-lived activation patterns may need to achieve the effect of (...) variables. The paper is mainly a theoretical analysis of some benefits and drawbacks of using logical combinators to avoid variables in short-lived connectionist encodings without loss of expressive power. The paper also includes a brief survey of some possible methods for avoiding variables other than by using combinators. (shrink)

Over the past 15 years, there have been two increasingly popular approaches to the study of meaning in cognitive science. One, based on theories of embodied cognition, treats meaning as a simulation of perceptual and motor states. An alternative approach treats meaning as a consequence of the statistical distribution of words across spoken and written language. On the surface, these appear to be opposing scientific paradigms. In this review, we aim to show how recent cross-disciplinary developments have done much to (...) reconcile these two approaches. The foundation to these developments has been the recognition that intralinguistic distributional and sensory-motor data are interdependent. We describe recent work in philosophy, psychology, cognitive neuroscience, and computational modeling that are all based on or consistent with this conclusion. We conclude by considering some possible directions for future research that arise as a consequence of these developments. (shrink)

Theories concerning the structure, or format, of mental representation should (1) be formulated in mechanistic, rather than metaphorical terms; (2) do justice to several philosophical intuitions about mental representation; and (3) explain the human capacity to predict the consequences of worldly alterations (i.e., to think before we act). The hypothesis that thinking involves the application of syntax-sensitive inference rules to syntactically structured mental representations has been said to satisfy all three conditions. An alternative hypothesis is that thinking requires the construction (...) and manipulation of the cognitive equivalent of scale models. A reading of this hypothesis is provided that satisfies condition (1) and which, even though it may not fully satisfy condition (2), turns out (in light of the frame problem) to be the only known way to satisfy condition (3). (shrink)

A central controversy in cognitive science concerns the roles of rules versus similarity. To gain some leverage on this problem, we propose that rule‐ versus similarity‐based processes can be characterized as extremes in a multidimensional space that is composed of at least two dimensions: the number of features (Pothos, 2005) and the physical presence of features. The transition of similarity‐ to rule‐based processing is conceptualized as a transition in this space. To illustrate this, we show how a neural network model (...) uses input features (and in this sense produces similarity‐based responses) when it has a low learning rate or in the early phases of training, but it switches to using self‐generated, more abstract features (and in this sense produces rule‐based responses) when it has a higher learning rate or is in the later phases of training. Relations with categorization and the psychology of learning are pointed out. (shrink)

A number of ways of taxonomizing human learning have been proposed. We examine the evidence for one such proposal, namely, that there exist independent explicit and implicit learning systems. This combines two further distinctions, (1) between learning that takes place with versus without concurrent awareness, and (2) between learning that involves the encoding of instances (or fragments) versus the induction of abstract rules or hypotheses. Implicit learning is assumed to involve unconscious rule learning. We examine the evidence for implicit learning (...) derived from subliminal learning, conditioning, artificial grammar learning, instrumental learning, and reaction times in sequence learning. We conclude that unconscious learning has not been satisfactorily established in any of these areas. The assumption that learning in some of these tasks (e.g., artificial grammar learning) is predominantly based on rule abstraction is questionable. When subjects cannot report the rules that govern stimulus selection, this is often because their knowledge consists of instances or fragments of the training stimuli rather than rules. In contrast to the distinction between conscious and unconscious learning, the distinction between instance and rule learning is a sound and meaningful way of taxonomizing human learning. We discuss various computational models of these two forms of learning. (shrink)

A number of ways of taxonomizing human learning have been proposed. We examine the evidence for one such proposal, namely, that there exist independent explicit and implicit learning systems. This combines two further distinctions, between learning that takes place with versus without concurrent awareness, and between learning that involves the encoding of instances versus the induction of abstract rules or hypotheses. Implicit learning is assumed to involve unconscious rule learning. We examine the evidence for implicit learning derived from subliminal learning, (...) conditioning, artificial grammar learning, instrumental learning, and reaction times in sequence learning. We conclude that unconscious learning has not been satisfactorily established in any of these areas. The assumption that learning in some of these tasks is predominantly based on rule abstraction is questionable. When subjects cannot report the “implicitly learned” rules that govern stimulus selection, this is often because their knowledge consists of instances or fragments of the training stimuli rather than rules. In contrast to the distinction between conscious and unconscious learning, the distinction between instance and rule learning is a sound and meaningful way of taxonomizing human learning. We discuss various computational models of these two forms of learning. (shrink)

In this prcis we focus on phenomena central to the reaction against similarity-based theories that arose in the 1980s and that subsequently motivated the approach to semantic knowledge. Specifically, we consider (1) how concepts differentiate in early development, (2) why some groupings of items seem to form or coherent categories while others do not, (3) why different properties seem central or important to different concepts, (4) why children and adults sometimes attest to beliefs that seem to contradict their direct experience, (...) (5) how concepts reorganize between the ages of 4 and 10, and (6) the relationship between causal knowledge and semantic knowledge. The explanations our theory offers for these phenomena are illustrated with reference to a simple feed-forward connectionist model. The relationships between this simple model, the broader theory, and more general issues in cognitive science are discussed. (shrink)

This paper introduces a special issue of Cognitive Science initiated on the 25th anniversary of the publication of Parallel Distributed Processing (PDP), a two-volume work that introduced the use of neural network models as vehicles for understanding cognition. The collection surveys the core commitments of the PDP framework, the key issues the framework has addressed, and the debates the framework has spawned, and presents viewpoints on the current status of these issues. The articles focus on both historical roots and contemporary (...) developments in learning, optimality theory, perception, memory, language, conceptual knowledge, cognitive control, and consciousness. Here we consider the approach more generally, reviewing the original motivations, the resulting framework, and the central tenets of the underlying theory. We then evaluate the impact of PDP both on the field at large and within specific subdomains of cognitive science and consider the current role of PDP models within the broader landscape of contemporary theoretical frameworks in cognitive science. Looking to the future, we consider the implications for cognitive science of the recent success of machine learning systems called “deep networks”—systems that build on key ideas presented in the PDP volumes. (shrink)

The commentaries reflect three core themes that pertain not just to our theory, but to the enterprise of connectionist modeling more generally. The first concerns the relationship between a cognitive theory and an implemented computer model. Specifically, how does one determine, when a model departs from the theory it exemplifies, whether the departure is a useful simplification or a critical flaw? We argue that the answer to this question depends partially upon the model's intended function, and we suggest that connectionist (...) models have important functions beyond the commonly accepted goals of fitting data and making predictions. The second theme concerns perceived in-principle limitations of the connectionist approach to cognition, and the specific concerns these perceived limitations raise for our theory. We argue that the approach is not in fact limited in the ways our critics suggest. One common misconception, that connectionist models cannot address abstract or relational structure, is corrected through new simulations showing directly that such structure can be captured. The third theme concerns the relationship between parallel distributed processing (PDP) models and structured probabilistic approaches. In this case we argue that there the difference between the approaches is not merely one of levels. Our PDP approach differs from structured statistical approaches at all of Marr's levels, including the characterization of the goals of cognitive computations, and of the representations and algorithms used. (shrink)

Question processing involves parsing, memory retrieval, question categorization, initiation of appropriate answer‐retrieval heuristics, answer formulation, and output. Computational and psychological models have traditionally treated these processes as separate, sequential, independent, and in pursuit of a single answer type at a time. Here this view is challenged and the implications of a theory in which question processes operate simultaneously on multiple question interpretations are explored. A highly interactive model is described in which an expectation‐driven parser generates multiple question candidates, including partially‐specified (...) candidates. Question candidates act as constraints for a matcher which activates memory items. An answer retrieval process examines question candidates and the active portions of memory in an attempt to generate answer candidates. Answer candidates are examined by an output process that derives the final answer. These processes run simultaneously and interact. Three experiments on human question answering are also described which provide evidence that working memory load during question reading is affected by processes related to answer retrieval. (shrink)

People tend to perceive ongoing continuous activity as series of discrete events. This partitioning of continuous activity may occur, in part, because events correspond to dynamic patterns that have recurred across different contexts. Recurring patterns may lead to reliable sequential dependencies in observers' experiences, which then can be used to guide perception. The current set of simulations investigated whether this statistical structure within events can be used 1) to develop stable internal representations that facilitate perception and 2) to learn when (...) to update such representations in a self-organizing manner. These simulations demonstrate that experience with recurring patterns enables a system to accurately predict upcoming stimuli within an event, to identify boundaries between such events based on transient increases in prediction error, and to use such boundaries to improve prediction about subsequent activities. (shrink)

Along with the increasing popularity of connectionist language models has come a number of provocative suggestions about the challenge these models present to Chomsky's arguments for nativism. The aim of this paper is to assess these claims. We begin by reconstructing Chomsky's argument from the poverty of the stimulus and arguing that it is best understood as three related arguments, with increasingly strong conclusions. Next, we provide a brief introduction to connectionism and give a quick survey of recent efforts to (...) develop networks that model various aspects of human linguistic behavior. Finally, we explore the implications of this research for Chomsky's arguments. Our claim is that the relation between connectionism and Chomsky's views on innate knowledge is more complicated than many have assumed, and that even if these models enjoy considerable success the threat they pose for linguistic nativism is small. (shrink)

Leech et al.'s analysis adds to an emerging consensus of the role of priming in analogy-making. However, their model cannot scale up to adult-level performance because not all relations can be cast as functions. One-size-fits-all accounts cannot capture the richness of analogy. Proportional analogies and transitive inferences can be made by nonstructural mechanisms. Therefore, these tasks do not generalize to tasks that require structure mapping.