This paper introduces a special issue of Cognitive Science initiated on the 25th anniversary of the publication of Parallel Distributed Processing (PDP), a two-volume work that introduced the use of neural network models as vehicles for understanding cognition. The collection surveys the core commitments of the PDP framework, the key issues the framework has addressed, and the debates the framework has spawned, and presents viewpoints on the current status of these issues. The articles focus on both historical roots and contemporary (...) developments in learning, optimality theory, perception, memory, language, conceptual knowledge, cognitive control, and consciousness. Here we consider the approach more generally, reviewing the original motivations, the resulting framework, and the central tenets of the underlying theory. We then evaluate the impact of PDP both on the field at large and within specific subdomains of cognitive science and consider the current role of PDP models within the broader landscape of contemporary theoretical frameworks in cognitive science. Looking to the future, we consider the implications for cognitive science of the recent success of machine learning systems called “deep networks”—systems that build on key ideas presented in the PDP volumes. (shrink)

In this prcis we focus on phenomena central to the reaction against similarity-based theories that arose in the 1980s and that subsequently motivated the approach to semantic knowledge. Specifically, we consider (1) how concepts differentiate in early development, (2) why some groupings of items seem to form or coherent categories while others do not, (3) why different properties seem central or important to different concepts, (4) why children and adults sometimes attest to beliefs that seem to contradict their direct experience, (...) (5) how concepts reorganize between the ages of 4 and 10, and (6) the relationship between causal knowledge and semantic knowledge. The explanations our theory offers for these phenomena are illustrated with reference to a simple feed-forward connectionist model. The relationships between this simple model, the broader theory, and more general issues in cognitive science are discussed. (shrink)

This article addresses the ability of Parallel Distributed Processing (PDP) networks to generate stagewise cognitive development in accordance with Piaget's theory of cognitive epigenesis. We carried out a replication study of the simulation experiments by McClelland (1989) and McClelland and Jenkins (1991) in which a PDP network learns to solve balance scale problems. In objective tests motivated from catastrophe theory, a mathematical theory of transitions in epigenetical systems, no evidence for stage transitions in network performance was found. It is concluded (...) that PDP networks lack the ability to recover the positive outcomes of analogous catastrophe analyses of real cognitive developmental data. In an attempt to further characterize the learning behaviour of PDP networks, we carried out a second simulation study using the discrimination‐shift paradigm. The results thus obtained indicate that PDP learning is compatible with the learning of stimulus‐response relationships, not with the acquisition of mediating rules such as conceived in (neo‐)Piagetian theory. In closing, we speculate about the feasibility of simulating stagewise development with alternative network architectures. (shrink)

A distributed neural network model called SPEC for processing sentences with recursive relative clauses is described. The model is based on separating the tasks of segmenting the input word sequence into clauses, forming the case‐role representations, and keeping track of the recursive embeddings into different modules. The system needs to be trained only with the basic sentence constructs, and it generalizes not only to new instances of familiar relative clause structures but to novel structures as well. SPEC exhibits plausible memory (...) degradation as the depth of the center embeddings increases, its memory is primed by earlier constituents, and its performance is aided by semantic constraints between the constituents. The ability to process structure is largely due to a central executive network that monitors and controls the execution of the entire system. This way, in contrast to earlier subsymbolic systems, parsing is modeled as a controlled high‐level process rather than one based on automatic reflex responses. (shrink)

Shastri & Ajjanagadde argue convincingly that both structured connectionist networks and parallel dynamic inferencing are necessary for reflexive reasoning - a kind of inferencing and reasoning that occurs rapidly, spontaneously, and without conscious effort, and which seems necessary for everyday tasks such as natural language understanding. As S&A describe, reflexive reasoning requires a solution to thedynamic binding problem, that is, how to encode systematic and abstract knowledge and instantiate it in specific situations to draw appropriate inferences. Although symbolic artificial intelligence (...) systems trivially solve the dynamic binding problem using computers' registers and pointers, it has remained a difficult problem for connectionist systems. S&A's temporal synchrony solution to the dynamic binding problem using synchronous firing of argument units and the entities that are bound to them illustrates one way in which connectionist networks can do thisusing a constrained but important class of long-term knowledge rules. Their structured connectionist solution allows dynamic inferencing to proceed in parallel and therefore has a number of advantages for reflexive reasoning over most other connectionist and symbolic systems. (shrink)

A computational model of inference during story comprehension is presented, in which story situations are represented distributively as points in a high‐dimensional “situation‐state space.” This state space organizes itself on the basis of a constructed microworld description. From the same description, causal/temporal world knowledge is extracted. The distributed representation of story situations is more flexible than Golden and Rumelhart's [Discourse Proc 16 (1993) 203] localist representation.A story taking place in the microworld corresponds to a trajectory through situation‐state space. During the (...) inference process, world knowledge is applied to the story trajectory. This results in an adjusted trajectory, reflecting the inference of propositions that are likely to be the case. Although inferences do not result from a search for coherence, they do cause story coherence to increase. The results of simulations correspond to empirical data concerning inference, reading time, and depth of processing.An extension of the model for simulating story retention shows how coherence is preserved during retention without controlling the retention process. Simulation results correspond to empirical data concerning story recall and intrusion. (shrink)

Although for many years a sharp distinction has been made in language research between rules and words—with primary interest on rules—this distinction is now blurred in many theories. If anything, the focus of attention has shifted in recent years in favor of words. Results from many different areas of language research suggest that the lexicon is representationally rich, that it is the source of much productive behavior, and that lexically specific information plays a critical and early role in the interpretation (...) of grammatical structure. But how much information can or should be placed in the lexicon? This is the question I address here. I review a set of studies whose results indicate that event knowledge plays a significant role in early stages of sentence processing and structural analysis. This poses a conundrum for traditional views of the lexicon. Either the lexicon must be expanded to include factors that do not plausibly seem to belong there; or else virtually all information about word meaning is removed, leaving the lexicon impoverished. I suggest a third alternative, which provides a way to account for lexical knowledge without a mental lexicon. (shrink)

I argue that SHRUTl's ontology is heavily committed to a representational view of mind. This is best seen when one thinks of how SHRUTI could be developed to account for psychological data on deductive reasoning.