Each of the chapters in this volume devotes considerable attention to defining and elaborating the notion of the frame problem-one of the hard problems of artificial intelligence. Not only do the chapters clarify the problems at hand, they shed light on the different approaches taken by those in artificial intelligence and by certain philosophers who have been concerned with related problems in their field. The book should therefore not be read merely as a discussion of the frame problem narrowly conceived, (...) but also as a general analysis of what could be a major challenge to the design of computer systems exhibiting general intelligence. (shrink)

In 1982, Feldman and Ballard published "Connectionist models and their properties" in Cognitive Science , helping to focus attention on a family of similarly inspired research strategies just then under way, by giving the family a name: "connectionism." Now, seven years later, the connectionist nation has swelled to include such subfamilies as "PDP" and "neural net models." Since the ideological foes of connectionism are keen to wipe it out in one fell swoop aimed at its "essence", it is worth noting (...) the diversity of not only the models but also the aspirations of the modelers. There is no good reason to suppose that they all pledge allegiance to any one principle.. (shrink)

This paper investigates connectionism's potential to solve the frame problem. The frame problem arises in the context of modelling the human ability to see the relevant consequences of events in a situation. It has been claimed to be unsolvable for classical cognitive science, but easily manageable for connectionism. We will focus on a representational approach to the frame problem which advocates the use of intrinsic representations. We argue that although connectionism's distributed representations may look promising from this perspective, doubts can (...) be raised about the potential of distributed representations to allow large amounts of complexly structured information to be adequately encoded and processed. It is questionable whether connectionist models that are claimed to effectively represent structured information can be scaled up to a realistic extent. We conclude that the frame problem provides a difficulty to connectionism that is no less serious than the obstacle it constitutes for classical cognitive science. (shrink)

In 1988 Fodor and Pylyshyn issued a challenge to the newly-popular connectionism: explain the systematicity of cognition without merely implementing a so-called classical architecture. Since that time quite a number of connectionist models have been put forward, either by their designers or by others, as in some measure demonstrating that the challenge can be met (e.g., Pollack, 1988, 1990; Smolensky, 1990; Chalmers, 1990; Niklasson and Sharkey, 1992; Brousse, 1993). Unfortu- nately, it has generally been unclear whether these models actually do (...) have this implication (see, for instance, the extensive philosophical debate in Smolensky, 1988; Fodor and McLaughlin, 1990; van Gelder, 1990, 1991; McLaughlin, 1993a, 1993b; Clark, 1993). Indeed, we know of no major supporter of classical orthodoxy who has felt compelled, by connectionist models and argu- ments, to concede in print that connectionists have in fact delivered a non-classical explanation of systematicity. (shrink)

This paper offers both a theoretical and an experimental perspective on the relationship between connectionist and Classical (symbol-processing) models. Firstly, a serious flaw in Fodor and Pylyshyn’s argument against connectionism is pointed out: if, in fact, a part of their argument is valid, then it establishes a conclusion quite different from that which they intend, a conclusion which is demonstrably false. The source of this flaw is traced to an underestimation of the differences between localist and distributed representation. It has (...) been claimed that distributed representations cannot support systematic operations, or that if they can, then they will be mere implementations of traditional ideas. This paper presents experimental evidence against this conclusion: distributed representations can be used to support direct structure-sensitive operations, in a man- ner quite unlike the Classical approach. Finally, it is argued that even if Fodor and Pylyshyn’s argument that connectionist models of compositionality must be mere implementations were correct, then this would still not be a serious argument against connectionism as a theory of mind. (shrink)

This paper explores the difference between Connectionist proposals for cognitive a r c h i t e c t u r e a n d t h e s o r t s o f m o d e l s t hat have traditionally been assum e d i n c o g n i t i v e s c i e n c e . W e c l a i m t h a t t h (...) e m a j o r d i s t i n c t i o n i s t h a t , w h i l e b o t h Connectionist and Classical architectures postulate representational mental states, the latter but not the former are committed to a symbol-level of representation, or to a ‘language of thought’: i.e., to representational states that have combinatorial syntactic and semantic structure. Several arguments for combinatorial structure in mental representations are then reviewed. These include arguments based on the ‘systematicity’ of mental representation: i.e., on the fact that cognitive capacities always exhibit certain symmetries, so that the ability to entertain a given thought implies the ability to entertain thoughts with semantically related contents. We claim that such arguments make a powerful case that mind/brain architecture is not Connectionist at the cognitive level. We then consider the possibility that Connectionism may provide an account of the neural (or ‘abstract neurological’) structures in which Classical cognitive architecture is implemented. We survey a n u m b e r o f t h e s t a n d a r d a r g u m e n t s t h a t h a v e b e e n o f f e r e d i n f a v o r o f Connectionism, and conclude that they are coherent only on this interpretation. (shrink)

Fodor's and Pylyshyn's stand on systematicity in thought and language has been debated and criticized. Van Gelder and Niklasson, among others, have argued that Fodor and Pylyshyn offer no precise definition of systematicity. However, our concern here is with a learning based formulation of that concept. In particular, Hadley has proposed that a network exhibits strong semantic systematicity when, as a result of training, it can assign appropriate meaning representations to novel sentences (both simple and embedded) which contain words in (...) syntactic positions they did not occupy during training. The experience of researchers indicates that strong systematicity in any form is difficult to achieve in connectionist systems.Herein we describe a network which displays strong semantic systematicity in response to Hebbian, connectionist training. During training, two-thirds of all nouns are presented only in a single syntactic position (either as grammatical subject or object). Yet, during testing, the network correctly interprets thousands of sentences containing those nouns in novel positions. In addition, the network generalizes to novel levels of embedding. Successful training requires a, corpus of about 1000 sentences, and network training is quite rapid. The architecture and learning algorithms are purely connectionist, but classical insights are discernible in one respect, viz, that complex semantic representations spatially contain their semantic constituents. However, in other important respects, the architecture is distinctly non-classical. (shrink)

Artificial intelligence research has foundered on the issue of representation. When intelligence is approached in an incremental manner, with strict reliance on interfacing to the real world through perception and action, reliance on representation disappears. In this paper we outline our approach to incrementally building complete intelligent Creatures. The fundamental decomposition of the intelligent system is not into independent information processing units which must interface with each other via representations. Instead, the intelligent system is decomposed into independent and parallel activity (...) producers which all interface directly to the world through perception and action, rather than interface to each other particularly much. The notions of central and peripheral systems evaporateeverything is both central and peripheral. Based on these principles we have built a very successful series of mobile robots which operate without supervision as Creatures in standard office environments. (shrink)

Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statis- tics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement record- ing systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and neuroscience. During perceptual experience, association areas in the (...) brain capture bottom-up patterns of activation in sensory-motor areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The stor- age and reactivation of perceptual symbols operates at the level of perceptual components – not at the level of holistic perceptual expe- riences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a com- mon frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift, run) and introspec- tion (e.g., compare, memory, happy, hungry). Once established, these simulators implement a basic conceptual system that represents types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and ab- stract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinato- rially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a per- ceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal sym- bol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored. (shrink)

A great deal of philosophy of mind in the modern era has been driven by an intense aversion to Cartesian dualism. In the 1950s, materialists claimed to have succeeded once and for all in exorcising the Cartesian ghost by identifying the mind with the brain. In subsequent decades, cognitive science put scientific meat on this metaphysical skeleton by explicating mental processes as digital computation implemented in the brain's hardware.

Despite the considerable attention that the systematicity argument has enjoyed, it is worthwhile examining the argument within the context of similar explanatory arguments from the history of science. This kind of analysis helps show that Connectionism, qua Connectionism, really does not have an explanation of systematicity. Second, and more surprisingly, one finds that the systematicity argument sets such a high explanatory standard that not even Classicism can explain the systematicity of thought.

A computational model of inference during story comprehension is presented, in which story situations are represented distributively as points in a high‐dimensional “situation‐state space.” This state space organizes itself on the basis of a constructed microworld description. From the same description, causal/temporal world knowledge is extracted. The distributed representation of story situations is more flexible than Golden and Rumelhart's [Discourse Proc 16 (1993) 203] localist representation.A story taking place in the microworld corresponds to a trajectory through situation‐state space. During the (...) inference process, world knowledge is applied to the story trajectory. This results in an adjusted trajectory, reflecting the inference of propositions that are likely to be the case. Although inferences do not result from a search for coherence, they do cause story coherence to increase. The results of simulations correspond to empirical data concerning inference, reading time, and depth of processing.An extension of the model for simulating story retention shows how coherence is preserved during retention without controlling the retention process. Simulation results correspond to empirical data concerning story recall and intrusion. (shrink)

Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implicitly by its effects on processing rather than explicitly (as in a spatial representation). The current report develops a proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory. In this approach, hidden unit patterns are fed back to themselves: (...) the internal representations which develop thus reflect task demands in the context of prior internal states. A set of simulations is reported which range from relatively simple problems (temporal version of XOR) to discovering syntactic/semantic features for words. The networks are able to learn interesting internal representations which incorporate task demands with memory demands: indeed, in this approach the notion of memory is inextricably bound up with task processing. These representations reveal a rich structure, which allows them to be highly context‐dependent, while also expressing generalizations across classes of items. These representations suggest a method for representing lexical categories and the type/token distinction. (shrink)

The chapters in this book have evolved from talks originally presented at The First International Workshop on Human and Machine Cognition.

Continuing his exploration of the organization of complexity and the science of design, this new edition of Herbert Simon's classic work on artificial ...

`Folk Psychology' - our everyday talk of beliefs, desires and mental events - has long been compared with the technical language of `Cognitive Science'. Does folk psychology provide a correct account of the mental causes of our behaviour, or must our everyday terms ultimately be replaced by a language developed from computational models and neurobiology? This broad-ranging book addresses these questions, which lie at the heart of psychology and philosophy. Providing a critical overview of the key literature in the field, (...) including the seminal work of Fodor and Churchland, the author explores the classic `Frame Problem' and assesses the future prospects of cognitive science. The scope of the frame problem, touching on connec. (shrink)

The Systematicity Arguments is the only book-length treatment of the systematicity and productivity arguments.

There has been much interest in the possibility of connectionist models whose representations can be endowed with compositional structure, and a variety of such models have been proposed. These models typically use distributed representations that arise from the functional composition of constituent parts. Functional composition and decomposition alone, however, yield only an implementation of classical symbolic theories. This paper explores the possibility of moving beyond implementation by exploiting holistic structure-sensitive operations on distributed representations. An experiment is performed using Pollack’s Recursive (...) Auto-Associative Memory. RAAM is used to construct distributed representations of syntactically structured sentences. A feed-forward network is then trained to operate directly on these representations, modeling syn- tactic transformations of the represented sentences. Successful training and generalization is obtained, demonstrating that the implicit structure present in these representations can be used for a kind of structure-sensitive processing unique to the connectionist domain. (shrink)

This book is not a conventional introduction to the philosophy of mind, nor is it a contribution to the physicalist/ dualist debate. Instead The Representational Theory of Mind demonstrates that we can construct physicalist theories of important aspects of our mental life. Its aim is to explain and defend a physicalist theory of intelligence in two parts: the first six chapters consist of an exposition, elaboration and defence of human sentience (the functionalist theory of mind), and the second part considers (...) rivals and objections to this theory. Kim Sterelny aims to introduce people to this area of philosophy by exemplifying it, to show that philosophical and empirical investigations can be synthesized to the benefit of both. --From publisher's description. (shrink)

This article addresses the ability of Parallel Distributed Processing (PDP) networks to generate stagewise cognitive development in accordance with Piaget's theory of cognitive epigenesis. We carried out a replication study of the simulation experiments by McClelland (1989) and McClelland and Jenkins (1991) in which a PDP network learns to solve balance scale problems. In objective tests motivated from catastrophe theory, a mathematical theory of transitions in epigenetical systems, no evidence for stage transitions in network performance was found. It is concluded (...) that PDP networks lack the ability to recover the positive outcomes of analogous catastrophe analyses of real cognitive developmental data. In an attempt to further characterize the learning behaviour of PDP networks, we carried out a second simulation study using the discrimination‐shift paradigm. The results thus obtained indicate that PDP learning is compatible with the learning of stimulus‐response relationships, not with the acquisition of mediating rules such as conceived in (neo‐)Piagetian theory. In closing, we speculate about the feasibility of simulating stagewise development with alternative network architectures. (shrink)

Evidence from numerous studies using the visual world paradigm has revealed both that spoken language can rapidly guide attention in a related visual scene and that scene information can immediately influence comprehension processes. These findings motivated the coordinated interplay account (Knoeferle & Crocker, 2006) of situated comprehension, which claims that utterance‐mediated attention crucially underlies this closely coordinated interaction of language and scene processing. We present a recurrent sigma‐pi neural network that models the rapid use of scene information, exploiting an utterance‐mediated (...) attentional mechanism that directly instantiates the CIA. The model is shown to achieve high levels of performance (both with and without scene contexts), while also exhibiting hallmark behaviors of situated comprehension, such as incremental processing, anticipation of appropriate role fillers, as well as the immediate use, and priority, of depicted event information through the coordinated use of utterance‐mediated attention to the scene. (shrink)

1 Cognitive Architectures 2 Multilayer Perceptrons 3 Relations between Variables 4 Structured Representations 5 Individuals 6 Where does the Machinery of Symbol Manipulation Come From? 7 Conclusions.

The recognition that human minds/brains are finite systems with limited resources for computation has led some researchers to advance the Tractable Cognition thesis: Human cognitive capacities are constrained by computational tractability. This thesis, if true, serves cognitive psychology by constraining the space of computational‐level theories of cognition. To utilize this constraint, a precise and workable definition of “computational tractability” is needed. Following computer science tradition, many cognitive scientists and psychologists define computational tractability as polynomial‐time computability, leading to the P‐Cognition thesis. (...) This article explains how and why the P‐Cognition thesis may be overly restrictive, risking the exclusion of veridical computational‐level theories from scientific investigation. An argument is made to replace the P‐Cognition thesis by the FPT‐Cognition thesis as an alternative formalization of the Tractable Cognition thesis (here, FPT stands for fixed‐parameter tractable). Possible objections to the Tractable Cognition thesis, and its proposed formalization, are discussed, and existing misconceptions are clarified. (shrink)

The ability to combine words into novel sentences has been used to argue that humans have symbolic language production abilities. Critiques of connectionist models of language often center on the inability of these models to generalize symbolically (Fodor & Pylyshyn, 1988; Marcus, 1998). To address these issues, a connectionist model of sentence production was developed. The model had variables (role‐concept bindings) that were inspired by spatial representations (Landau & Jackendoff, 1993). In order to take advantage of these variables, a novel (...) dual‐pathway architecture with event semantics is proposed and shown to be better at symbolic generalization than several variants. This architecture has one pathway for mapping message content to words and a separate pathway that enforces sequencing constraints. Analysis of the model's hidden units demonstrated that the model learned different types of information in each pathway, and that the model's compositional behavior arose from the combination of these two pathways. The model's ability to balance symbolic and statistical behavior in syntax acquisition and to model aphasic double dissociations provided independent support for the dual‐pathway architecture. (shrink)

Neural networks usually work adequately on small problems but can run into trouble when they are scaled up to problems involving large amounts of input data. Circuit Complexity and Neural Networks addresses the important question of how well neural networks scale - that is, how fast the computation time and number of neurons grow as the problem size increases. It surveys recent research in circuit complexity (a robust branch of theoretical computer science) and applies this work to a theoretical understanding (...) of the problem of scalability. Most research in neural networks focuses on learning, yet it is important to understand the physical limitations of the network before the resources needed to solve a certain problem can be calculated. One of the aims of this book is to compare the complexity of neural networks and the complexity of conventional computers, looking at the computational ability and resources (neurons and time) that are a necessary part of the foundations of neural network learning. Circuit Complexity and Neural Networks contains a significant amount of background material on conventional complexity theory that will enable neural network scientists to learn about how complexity theory applies to their discipline, and allow complexity theorists to see how their discipline applies to neural networks. (shrink)

Human cognition is said to be systematic: cognitive ability generalizes to structurally related behaviours. The connectionist approach to cognitive theorizing has been strongly criticized for its failure to explain systematicity. Demonstrations of generalization notwithstanding, I show that two widely used networks (feedforward and recurrent) do not support systematicity under the condition of local input/output representations. For a connectionist explanation of systematicity, these results leave two choices, either: (1) develop models capable of systematicity under local input/output representations; or (2) justify the (...) choice of similarity-based (nonlocal) component representations sufficient for systematicity. (shrink)

Something remarkable is happening in the cognitive sciences. After a quarter of a century of cognitive models that were inspired by the metaphor of the digital computer, the newest cognitive models are inspired by the properties of the brain itself. Variously referred to as connectionist, parallel distributed processing, or neutral network models, they explore the idea that complex intellectual operations can be carried out by large networks of simple, neuron-like units. The units themselves are identical, very low-level and 'stupid'. Intelligent (...) performance is derived from the pattern of connection strengths between units, and the fundamental cognitive activity is pattern recognition and completion. Connectionism and the Mind provides an introduction to this newly emerging approach to understanding the mind. The first few chapters focus on network architecture, offering accessible treatment of the equations that describe learning and the propagation of activation. Furthermore, the reader is walked step-by-step through the activities of networks engaged in pattern recognition, learning, and cognitive tasks such as memory retrieval and prototype formation. The remainder of the book addresses the implications of connectionism for theories of the mind, both philosophical and psychological. Foe example: What Role is played by pattern recognition and completion as basic as cognitive functions? Connectionist models have particular strength in learning and pattern recognition; should they be limited to those functions, or can they provide an overall account of cognitive functioning? In particular, can connectionist models provide an adequate account of the ability to employ linguistic and other symbol systems, or must an adequate system incorporate symbol processing as a basic cognitive capacity? Finally, Connectionism and the Mind examines the relation of connectionist models to philosophical accounts of propositional attitudes, and to a variety of other inquiries in cognitive psychology, linguistics, developmental psychology, artificial intelligence and neuroscience. (shrink)

This paper discusses the relation between cognitive and implementational levels of analysis. Chalmers (1990, 1993) argues that a connectionist implementation of a classical cognitive architecture possesses a compositional semantics, and therefore undercuts Fodor and Pylyshyn's (1988) argument that connectionist networks cannot possess a compositional semantics. I argue that Chalmers argument misconstrues the relation between cognitive and implementational levels of analysis. This paper clarifies the distinction, and shows that while Fodor and Pylyshyn's argument survives Chalmers' critique, it cannot be used to (...) establish the irrelevance of neurophysiological implementation to cognitive modeling; some aspects of Chater and Oaksford's (1990) response to Fodor and Pylyshyn, though not all, are therefore cogent. (shrink)

The past decade has witnessed the emergence of a novel stance on semantic representation, and its relationship to context sensitivity. Connectionist-minded philosophers, including Clark and van Gelder, have espoused the merits of viewing hidden-layer, context-sensitive representations as possessing semantic content, where this content is partially revealed via the representations'' position in vector space. In recent work, Bodén and Niklasson have incorporated a variant of this view of semantics within their conception of semantic systematicity. Moreover, Bodén and Niklasson contend that they (...) have produced experimental results which not only satisfy a kind of context-based, semantic systematicity, but which, to the degree that reality permits, effectively deals with challenges posed by Fodor and Pylyshyn (1988), and Hadley (1994a). The latter challenge involved well-defined criteria for strong semantic systematicity. This paper examines the relevant claims and experiments of Bodén and Niklasson. It is argued that their case fatally involves two fallacies of equivocation; one concerning ''semantic content'' and the other concerning ''novel test sentences''. In addition, it is argued that their ultimate construal of context sensitive semantics contains serious confusions. These confusions are also found in certain publications dealing with "latent semantic analysis". Thus, criticisms presented here have relevance beyond the work of Bodén and Niklasson. (shrink)