A number of ways of taxonomizing human learning have been proposed. We examine the evidence for one such proposal, namely, that there exist independent explicit and implicit learning systems. This combines two further distinctions, between learning that takes place with versus without concurrent awareness, and between learning that involves the encoding of instances versus the induction of abstract rules or hypotheses. Implicit learning is assumed to involve unconscious rule learning. We examine the evidence for implicit learning derived from subliminal learning, (...) conditioning, artificial grammar learning, instrumental learning, and reaction times in sequence learning. We conclude that unconscious learning has not been satisfactorily established in any of these areas. The assumption that learning in some of these tasks is predominantly based on rule abstraction is questionable. When subjects cannot report the “implicitly learned” rules that govern stimulus selection, this is often because their knowledge consists of instances or fragments of the training stimuli rather than rules. In contrast to the distinction between conscious and unconscious learning, the distinction between instance and rule learning is a sound and meaningful way of taxonomizing human learning. We discuss various computational models of these two forms of learning. (shrink)

Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and “simple” homogeneous computing elements. Using just these three features one can construct surprisingly elegant and powerful models of (...) memory, perception, motor control, categorization, and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks. (shrink)

The neurophysiological evidence from the Miyashita group's experiments on monkeys as well as cognitive experience common to us all suggests that local neuronal spike rate distributions might persist in the absence of their eliciting stimulus. In Hebb's cell-assembly theory, learning dynamics stabilize such self-maintaining reverberations. Quasi-quantitive modeling of the experimental data on internal representations in association-cortex modules identifies the reverberations (delay spike activity) as the internal code (representation). This leads to cognitive and neurophysiological predictions, many following directly from the language (...) used to describe the activity in the experimental delay period, others from the details of how the model captures the properties of the internal representations. (shrink)

The cell assembly as a simple attractor cannot explain many cognitive phenomena. It must be a highly structured network that can sustain highly structured excitation patterns. Moreover, a cell assembly must be more widely distributed in space than on a square millimeter.

This commentary questions the target articles inferences from a limited set of empirical data to support this model and conceptual scheme. Especially questionable is the attribution of internal representation properties to an assembly of cells in a discrete cortical module firing at a discrete attractor frequency. Alternative inferences are drawn from cortical cooling and cell-firing data that point to the internal representation as a broad and specific cortical network defined by cortico-cortical connectivity. Active memory, it is proposed, consists in the (...) sustained activation of the component neuron populations of the network. (shrink)

Cognitive scientists must understand not just what the mind does, but how it does what it does. In this paper, I consider four aspects of cognitive architecture: how the mind develops, the extent to which it is or is not modular, the extent to which it is or is not optimal, and the extent to which it should or should not be considered a symbol‐manipulating device (as opposed to, say, an eliminative connectionist network). In each case, I argue that insights (...) from developmental and evolutionary biology can lead to substantive and important compromises in historically vexed debates. (shrink)

Advocates of dynamic systems have suggested that higher mental processes are based on continuous representations. In order to evaluate this claim, we first define the concept of representation, and rigorously distinguish between discrete representations and continuous representations. We also explore two important bases of representational content. Then, we present seven arguments that discrete representations are necessary for any system that must discriminate between two or more states. It follows that higher mental processes require discrete representations. We also argue that discrete (...) representations are more influenced by conceptual role than continuous representations. We end by arguing that the presence of discrete representations in cognitive systems entails that computationalism (i.e., the view that the mind is a computational device) is true, and that cognitive science should embrace representational pluralism. (shrink)

The context of the target article is delimited again, underlining the intended locationof the argument in the bottomup hierarchy of brain study. The central message is that collective delay activity distributions (reverberations) in cortical modules extend the role of a spike (a potentialinformation carrier across long distances) to an active memory of structured, learned information that can be carried across long time intervals. Moreover, the population code of the reverberations makes them readable down the cortical processing stream. Most of the (...) critical comments are then interpreted and addressed in relation to misreading of the proper context. The price for the limitation of the context (in cognitive, behavioral, and computational terms) is compared with the advantages of a clear, direct contact with experiment on the one hand and with a well controlled body of modeling and analysis on the other. (shrink)

Amit's efforts to provide stronger theoretical and empirical support for Hebb's cell-assembly concept is admirable, but we have serious reservations about the perspective presented in the target article. For Hebb, the cell assembly was a building block; by contrast, the framework proposed here eschews the need to fit the assembly into a broader picture of its function.

Amit argues that the local neuronal spike rate that persists (reverberating) in the absence of the eliciting stimulus represents the code of the eliciting stimulus. Based on the general argument that the inferred functional meaning of reverberation depends in part on the type of representational assumptions, reverberations may only be important for the encoding of contextual information.

Hebbian cell-assembly theory and attractor networks are good starting points for modeling cortical processing. Detailed cell models can be useful in understanding the dynamics of attractor networks. Cell assemblies are likely to be distributed, with the cortical column as the local processing unit. Synaptic memory may be dominant in all but the first couple of seconds.

The Miyashita experiments are very interesting and the results should be examined from a viewpoint of attractor dynamics. Amit's target article shows a path toward realistic modeling by artificial neural networks (ANN), but it is not necessarily the only one. I introduce another model that can explain a substantial part of the empirical observations and makes an interesting prediction. This model consists of such units that have nonmonotonic input-output characteristics with local inhibition neurons.

Amit's Hebbian model risks being overexplanatory, since it does not depend on specific physiological modelling of cortical ANNs, but concentrates on those phenomena which are modelled by a large class of ANNs. While offering a strong demonstration of the presence of Hebb's “cell assemblies,” it does not offer an equal account of Hebb's “phase sequence” concept.

This paper examines whether a classical model could be translated into a PDP network using a standard connectionist training technique called extra output learning. In Study 1, standard machine learning techniques were used to create a decision tree that could be used to classify 8124 different mushrooms as being edible or poisonous on the basis of 21 different Features (Schlimmer, 1987). In Study 2, extra output learning was used to insert this decision tree into a PDP network being trained on (...) the identical problem. An interpretation of the trained network revealed a perfect mapping from its internal structure to the decision tree, representing a precise translation of the classical theory to the connectionist model. In Study 3, a second network was trained on the mushroom problem without using extra output learning. An interpretation of this second network revealed a different algorithm for solving the mushroom problem, demonstrating that the Study 2 network was indeed a proper theory translation. (shrink)

Abstract A survey of different senses of the term ?representation? is presented. The presentation is guided by the appraisal that this key term is employed in the cognitive literature in different senses and that the distinction between these is not always explicitly stated or appreciated. Furthermore, the different senses seem to be associated with different rationales for the postulation of representation. Given that there may be a lack of convergence between the various senses of the construct in question and the (...) rationales for its postulation, a clear differentiation of these may be crucial for any evaluation of the role of representations in cognition and in cognitive theory. (shrink)

This article describes an approach to connectionist language research that relies on the development of grammar formalisms rather than computer models. From formulations of the fundamental theoretical commitments of connectionism and of generative grammar, it is argued that these two paradigms are mutually compatible. Integrating the basic assumptions of the paradigms results in formal theories of grammar that centrally incorporate a certain degree of connectionist computation. Two such grammar formalisms—Harmonic Grammar and Optimality Theory —are briefly introduced to illustrate grammar‐based approaches (...) to connectionist language research. The strengths and weaknesses of grammar‐based research and more traditional model‐based research are argued to be complementary, suggesting a significant role for both strategies in the spectrum of connectionist language research. (shrink)

In the first section of the article, we examine some recent criticisms of the connectionist enterprise: first, that connectionist models are fundamentally behaviorist in nature (and, therefore, non-cognitive), and second that connectionist models are fundamentally associationist in nature (and, therefore, cognitively weak). We argue that, for a limited class of connectionist models (feed-forward, pattern-associator models), the first criticism is unavoidable. With respect to the second criticism, we propose that connectionist modelsare fundamentally associationist but that this is appropriate for building models (...) of human cognition. However, we do accept the point that there are cognitive capacities for which any purely associative model cannot provide a satisfactory account. The implication that we draw from is this is not that associationist models and mechanisms should be scrapped, but rather that they should be enhanced.In the next section of the article, we identify a set of connectionist approaches which are characterized by “active symbols” — recurrent circuits which are the basis of knowledge representation. We claim that such approaches avoid criticisms of behaviorism and are, in principle, capable of supporting full cognition. In the final section of the article, we speculate at some length about what we believe would be the characteristics of a fully realized active symbol system. This includes both potential problems and possible solutions (for example, mechanisms needed to control activity in a complex recurrent network) as well as the promise of such systems (in particular, the emergence of knowledge structures which would constitute genuine internal models). (shrink)

Rumelhart and McClelland's chapter about learning the past tense created a degree of controversy extraordinary even in the adversarial culture of modern science. It also stimulated a vast amount of research that advanced the understanding of the past tense, inflectional morphology in English and other languages, the nature of linguistic representations, relations between language and other phenomena such as reading and object recognition, the properties of artificial neural networks, and other topics. We examine the impact of the Rumelhart and McClelland (...) model with the benefit of 25 years of hindsight. It is not clear who “won” the debate. It is clear, however, that the core ideas that the model instantiated have been assimilated into many areas in the study of language, changing the focus of research from abstract characterizations of linguistic competence to an emphasis on the role of the statistical structure of language in acquisition and processing. (shrink)