Connectionism provides hope for unifying work in neuroscience, computer science, and cognitive psychology. This promise has met with some resistance from Classical Computionalists, which may have inspired Connectionists to retaliate with bold, inflationary claims on behalf of Connectionist models. This paper demonstrates, by examining three intimately connected issues, that these inflationary claims made on behalf of Connectionism are wrong. This should not be construed as an attack on Connectionism, however, since the inflated claims made on its behalf have the look (...) of cures for which there are no ailments. There is nothing wrong with Connectionism for its failure to solve illusory problems. (shrink)

A fundamental claim associated with parallel distributed processing theories of cognition is that knowledge is coded in a distributed manner in mind and brain. This approach rejects the claim that knowledge is coded in a localist fashion, with words, objects, and simple concepts, that is, coded with their own dedicated representations. One of the putative advantages of this approach is that the theories are biologically plausible. Indeed, advocates of the PDP approach often highlight the close parallels between distributed representations learned (...) in connectionist models and neural coding in brain and often dismiss localist theories as biologically implausible. The author reviews a range a data that strongly challenge this claim and shows that localist models provide a better account of single-cell recording studies. The author also contrast local and alternative distributed coding schemes and argues that common rejection of grandmother cell theories in neuroscience is due to a misunderstanding about how localist models behave. The author concludes that the localist representations embedded in theories of perception and cognition are consistent with neuroscience; biology only calls into question the distributed representations often learned in PDP models. (shrink)

The focus of much recent debate between realists and eliminativists about the propositional attitudes obscures the fact that a spectrum of positions lies between these celebrated extremes. Appealing to an influential theoretical development in cognitive neurobiology, I argue that there is reason to expect such an “intermediate” outcome. The ontology that emerges is a revisionary physicalism. The argument draws lessons about revisionistic reductions from an important historical example, the reduction of equilibrium thermodynamics to statistical mechanics, and applies them to the (...) relationship developing between propositional attitude psychology and this potential neuroscientific successor. It predicts enough conceptual change to rule out a straightforward realism about the attitudes; but at the same time it also resists the eliminativist's comparison of the fate awaiting the propositional attitudes to that befalling caloric fluid, phlogiston, and the like. (shrink)

This paper is a commentary on the target article by Dana H. Ballard, “Cortical connections and parallel processing: Structure and function”, in the same issue of the journal, pp. 67–120. -/- I raise some issues about the connectionist or neural-network implementation of information and information processing. Issues include the sharing of information by different parts of a connectionist/neural network, the copying of complex information from one place to another in a network, the possibility of connection weights not being synaptic weights, (...) and the possibility of fast communication within single neurons as complex systems in their own right. (shrink)

The cerebral cortex is a rich and diverse structure that is the basis of intelligent behavior. One of the deepest mysteries of the function of cortex is that neural processing times are only about one hundred times as fast as the fastest response times for complex behavior. At the very least, this would seem to indicate that the cortex does massive amounts of parallel computation.This paper explores the hypothesis that an important part of the cortex can be modeled as a (...) connectionist computer that is especially suited for parallel problem solving. The connectionist computer uses a special representation, termed value unit encoding, that represents small subsets of parameters in a way that allows parallel access to many different parameter values. This computer can be thought of as computing hierarchies of sensorimotor invariants. The neural substrate can be interpreted as a commitment to data structures and algorithms that compute invariants fast enough to explain the behavioral response times. A detailed consideration of this model has several implications for the underlying anatomy and physiology. (shrink)

The cerebral cortex is a rich and diverse structure that is the basis of intelligent behavior. One of the deepest mysteries of the function of cortex is that neural processing times are only about one hundred times as fast as the fastest response times for complex behavior. At the very least, this would seem to indicate that the cortex does massive amounts of parallel computation.This paper explores the hypothesis that an important part of the cortex can be modeled as a (...) connectionist computer that is especially suited for parallel problem solving. The connectionist computer uses a special representation, termed value unit encoding, that represents small subsets of parameters in a way that allows parallel access to many different parameter values. This computer can be thought of as computing hierarchies of sensorimotor invariants. The neural substrate can be interpreted as a commitment to data structures and algorithms that compute invariants fast enough to explain the behavioral response times. A detailed consideration of this model has several implications for the underlying anatomy and physiology. (shrink)

At 14 months, children appear to struggle to apply their fairly well-developed speech perception abilities to learning similar sounding words (e.g., bih/dih; Stager & Werker, 1997). However, variability in nonphonetic aspects of the training stimuli seems to aid word learning at this age. Extant theories of early word learning cannot account for this benefit of variability. We offer a simple explanation for this range of effects based on associative learning. Simulations suggest that if infants encode both noncontrastive information (e.g., cues (...) to speaker voice) and meaningful linguistic cues (e.g., place of articulation or voicing), then associative learning mechanisms predict these variability effects in early word learning. Crucially, this means that despite the importance of task variables in predicting performance, this body of work shows that phonological categories are still developing at this age, and that the structure of noninformative cues has critical influences on word learning abilities. (shrink)

Critics of the computational connectionism of the last decade suggest that it shares undesirable features with earlier empiricist or associationist approaches, and with behaviourist theories of learning. To assess the accuracy of this charge the works of earlier writers are examined for the presence of such features, and brief accounts of those found are given for Herbert Spencer, William James and the learning theorists Thorndike, Pavlov and Hull. The idea that cognition depends on associative connections among large networks of neurons (...) is indeed one with precedents, although the implications of this for psychological issues have been interpreted variously — not all versions of connectionism are alike. (shrink)

The paper uses ideas from Machine Learning, Artificial Intelligence and Genetic Algorithms to provide a model of the development of a fight-or-flight response in a simulated agent. The modelled development process involves (simulated) processes of evolution, learning and representation development. The main value of the model is that it provides an illustration of how simple learning processes may lead to the formation of structures which can be given a representational interpretation. It also shows how these may form the infrastructure for (...) closely-coupled agent/environment interaction. (shrink)

This article proposes a number of models to examine through which mechanisms a population of autonomous agents could arrive at a repertoire of perceptually grounded categories that is sufficiently shared to allow successful communication. The models are inspired by the main approaches to human categorisation being discussed in the literature: nativism, empiricism, and culturalism. Colour is taken as a case study. Although we take no stance on which position is to be accepted as final truth with respect to human categorisation (...) and naming, we do point to theoretical constraints that make each position more or less likely and we make clear suggestions on what the best engineering solution would be. Specifically, we argue that the collective choice of a shared repertoire must integrate multiple constraints, including constraints coming from communication. Key Words: autonomous agents; colour categorisation; colour naming; connectionism; cultural evolution; genetic evolution; memes; origins of language; self-organisation; semiotic dynamics; symbol grounding. (shrink)

Previous neural network models of concept learning were mainly implemented with supervised learning schemes. However, studies of human conceptual memory have shown that concepts may be learned without a teacher who provides the category name to associate with exemplars. A modular neural network architecture that realizes concept acquisition through two functionally distinct operations, categorizing and naming, is proposed as an alternative. An unsupervised algorithm realizes the categorizing module by constructing representations of categories compatible with prototype theory. The naming module associates (...) category names to the output of the categorizing module in a supervised mode. In such a modular architecture, the interface between the modules can be conceived of as an “information relay” that encodes, constrains, and propagates important information. Five experiments were conducted to analyze the relationships among internal conceptual codes and simple conceptual and lexical development. The first two experiments show a prototype effect and illustrate some basic characteristics of the system. The third experiment presents a bottom‐up model of the narrowing down of children's early lexical categories that honors mutual exclusivity. The fourth experiment introduces top‐down constraints on conceptual coding. The fifth experiment exhibits how hierarchical relationships between concepts are learned by the architecture, and also demonstrates how a spectrum of conceptual expertise may gradually emerge as a consequence of experiencing more with certain categories than with others. (shrink)

Previous neural network models of concept learning were mainly implemented with supervised learning schemes. However, studies of human conceptual memory have shown that concepts may be learned without a teacher who provides the category name to associate with exemplars. A modular neural network architecture that realizes concept acquisition through two functionally distinct operations, categorizing and naming, is proposed as an alternative. An unsupervised algorithm realizes the categorizing module by constructing representations of categories compatible with prototype theory. The naming module associates (...) category names to the output of the categorizing module in a supervised mode. In such a modular architecture, the interface between the modules can be conceived of as an “information relay” that encodes, constrains, and propagates important information. Five experiments were conducted to analyze the relationships among internal conceptual codes and simple conceptual and lexical development. The first two experiments show a prototype effect and illustrate some basic characteristics of the system. The third experiment presents a bottom‐up model of the narrowing down of children's early lexical categories that honors mutual exclusivity. The fourth experiment introduces top‐down constraints on conceptual coding. The fifth experiment exhibits how hierarchical relationships between concepts are learned by the architecture, and also demonstrates how a spectrum of conceptual expertise may gradually emerge as a consequence of experiencing more with certain categories than with others. (shrink)

Page's proposal to stipulate representations in which individual units correspond to meaningful entities is too unconstrained to support effective theorizing. An approach combining general computational principles with domain-specific assumptions, in which learning is used to discover representations that are effective in solving tasks, provides more insight into why cognitive and neural systems are organized the way they are.

Symbols enable people to organize and communicate about the world. However, the ways in which symbolic knowledge is learned and then represented in the mind are poorly understood. We present a formal analysis of symbolic learning—in particular, word learning—in terms of prediction and cue competition, and we consider two possible ways in which symbols might be learned: by learning to predict a label from the features of objects and events in the world, and by learning to predict features from a (...) label. This analysis predicts significant differences in symbolic learning depending on the sequencing of objects and labels. We report a computational simulation and two human experiments that confirm these differences, revealing the existence of Feature-Label-Ordering effects in learning. Discrimination learning is facilitated when objects predict labels, but not when labels predict objects. Our results and analysis suggest that the semantic categories people use to understand and communicate about the world can only be learned if labels are predicted from objects. We discuss the implications of this for our understanding of the nature of language and symbolic thought, and in particular, for theories of reference. (shrink)

Exploratory analysis is an area of increasing interest in the computational linguistics arena. Pragmatically speaking, exploratory analysis may be paraphrased as natural language processing by means of analyzing large corpora of text. Concerning the analysis, appropriate means are statistics, on the one hand, and artificial neural networks, on the other hand. As a challenging application area for exploratory analysis of text corpora we may certainly identify text databases, be it information retrieval or information filtering systems. With this paper we present (...) recent findings of exploratory analysis based on both statistical and neural models applied to legal text corpora. Concerning the artificial neural networks, we rely on a model adhering to the unsupervised learning paradigm. This choice appears naturally when taking into account the specific properties of large text corpora where one is faced with the fact that input-output-mappings as required by supervised learning models cannot be provided beforehand to a satisfying extent. This is due to the fact of the highly changing contents of text archives. In a nutshell, artificial neural networks count for their highly robust behavior regarding the parameters for model optimization. In particular, we found statistical classification techniques much more susceptible to minor parameter variations than unsupervised artificial neural networks. In this paper we describe two different lines of research in exploratory analysis. First, we use the classification methods for concept analysis. The general goal is to uncover different meanings of one and the same natural language concept. A task that, obviously, is of specific importance during the creation of thesauri. As a convenient environment to present the results we selected the legal term of neutrality, which is a perfect representative of a concept having a number of highly divergent meanings. Second, we describe the classification methods in the setting of document classification. The ultimate goal in such an application is to uncover semantic similarities of various text documents in order to increase the efficiency of an information retrieval system. In this sense, document classification has its fixed position in information retrieval research from the very beginning. Nowadays renewed massive interest in document classification may be witnessed due to the appearance of large-scale digital libraries. (shrink)

Preparing words in speech production is normally a fast and accurate process. We generate them two or three per second in fluent conversation; and overtly naming a clear picture of an object can easily be initiated within 600 msec after picture onset. The underlying process, however, is exceedingly complex. The theory reviewed in this target article analyzes this process as staged and feedforward. After a first stage of conceptual preparation, word generation proceeds through lexical selection, morphological and phonological encoding, phonetic (...) encoding, and articulation itself. In addition, the speaker exerts some degree of output control, by monitoring of self-produced internal and overt speech. The core of the theory, ranging from lexical selection to the initiation of phonetic encoding, is captured in a computational model, called WEAVER++. Both the theory and the computational model have been developed in interaction with reaction time experiments, particularly in picture naming or related word production paradigms, with the aim of accounting for the real-time processing in normal word production. A comprehensive review of theory, model, and experiments is presented. The model can handle some of the main observations in the domain of speech errors (the major empirical domain for most other theories of lexical access), and the theory opens new ways of approaching the cerebral organization of speech production by way of high-temporal-resolution imaging. (shrink)

The Sparse, Distributed Memory (SDM) model (Kanerva, 1984) is compared to Hopfield-type, neural-network models. A mathematical framework for comparing the two models is developed, and the capacity of each model is investigated. The capacity of the SDM can be increased independent of the dimension of the stored vectors, whereas the Hopfield capacity is limited to a fraction of this dimension. The stored information is proportional to the number of connections, and it is shown that this proportionality constant is the same (...) for the SDM, the Hopfield model, and higher-order models. The models are also compared in their ability to store and recall temporal sequences of patterns. The SDM also includes time delays so that contextual information can be used to recover sequences. A generalization of the SDM allows storage of correlated patterns. (shrink)