This paper reviews the fate of the central ideas behind the complementary learning systems (CLS) framework as originally articulated in McClelland, McNaughton, and O’Reilly (1995). This framework explains why the brain requires two differentially specialized learning and memory systems, and it nicely specifies their central properties (i.e., the hippocampus as a sparse, pattern-separated system for rapidly learning episodic memories, and the neocortex as a distributed, overlapping system for gradually integrating across episodes to extract latent semantic structure). We review the application (...) of the CLS framework to a range of important topics, including the following: the basic neural processes of hippocampal memory encoding and recall, conjunctive encoding, human recognition memory, consolidation of initial hippocampal learning in cortex, dynamic modulation of encoding versus recall, and the synergistic interactions between hippocampus and neocortex. Overall, the CLS framework remains a vital theoretical force in the field, with the empirical data over the past 15 years generally confirming its key principles. (shrink)

An emerging class of theories concerning the functional structure of the brain takes the reuse of neural circuitry for various cognitive purposes to be a central organizational principle. According to these theories, it is quite common for neural circuits established for one purpose to be exapted (exploited, recycled, redeployed) during evolution or normal development, and be put to different uses, often without losing their original functions. Neural reuse theories thus differ from the usual understanding of the role of neural plasticity (...) (which is, after all, a kind of reuse) in brain organization along the following lines: According to neural reuse, circuits can continue to acquire new uses after an initial or original function is established; the acquisition of new uses need not involve unusual circumstances such as injury or loss of established function; and the acquisition of a new use need not involve (much) local change to circuit structure (e.g., it might involve only the establishment of functional connections to new neural partners). Thus, neural reuse theories offer a distinct perspective on several topics of general interest, such as: the evolution and development of the brain, including (for instance) the evolutionary-developmental pathway supporting primate tool use and human language; the degree of modularity in brain organization; the degree of localization of cognitive function; and the cortical parcellation problem and the prospects (and proper methods to employ) for function to structure mapping. The idea also has some practical implications in the areas of rehabilitative medicine and machine interface design. (shrink)

It is often assumed that similar domain-specific behavioural impairments found in cases of adult brain damage and developmental disorders correspond to similar underlying causes, and can serve as convergent evidence for the modular structure of the normal adult cognitive system. We argue that this correspondence is contingent on an unsupported assumption that atypical development can produce selective deficits while the rest of the system develops normally (Residual Normality), and that this assumption tends to bias data collection in the field. Based (...) on a review of connectionist models of acquired and developmental disorders in the domains of reading and past tense, as well as on new simulations, we explore the computational viability of Residual Normality and the potential role of development in producing behavioural deficits. Simulations demonstrate that damage to a developmental model can produce very different effects depending on whether it occurs prior to or following the training process. Because developmental disorders typically involve damage prior to learning, we conclude that the developmental process is a key component of the explanation of endstate impairments in such disorders. Further simulations demonstrate that in simple connectionist learning systems, the assumption of Residual Normality is undermined by processes of compensation or alteration elsewhere in the system. We outline the precise computational conditions required for Residual Normality to hold in development, and suggest that in many cases it is an unlikely hypothesis. We conclude that in developmental disorders, inferences from behavioural deficits to underlying structure crucially depend on developmental conditions, and that the process of ontogenetic development cannot be ignored in constructing models of developmental disorders. Key Words: Acquired and developmental disorders; connectionist models; modularity; past tense; reading. (shrink)

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine, a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can encode contextual (...) information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models. We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain. (shrink)

Perceptual input changes constantly in an unpredictable fashion, often changing before our somewhat sluggish perceptual systems have adequately processed this input. This can give rise to source confusion—how do we know whether a given perceptual activation is due to the current input, or a previous input that had yet to be completely processed? We propose that activity‐dependent neural accommodation naturally limits this source confusion by suppressing items once they have been identified. We review behavioral paradigms from different literatures that measure (...) the correlates of persistence and accommodation. Of the various accommodative mechanisms, we focus on synaptic depression, deriving a rate‐coded expression that can be used to produce accommodating dynamics in any neural network with real valued activation. We implement this expression in a hierarchical model of perception termed, “a neural mechanism for responding optimally with unknown sources of evidence” (nROUSE). This model can be viewed as a more detailed version of the more abstract ROUSE model of Huber, Shiffrin, Lyle, and Ruys (2001), which produces accommodated levels of feature evidence through an optimal calculation. We apply nROUSE to three short‐term priming experiments that manipulated prime duration. (shrink)

Determining optimal units of representing morphologically complex words in the mental lexicon is a central question in psycholinguistics. Here, we utilize advances in computational sciences to study human morphological processing using statistical models of morphology, particularly the unsupervised Morfessor model that works on the principle of optimization. The aim was to see what kind of model structure corresponds best to human word recognition costs for multimorphemic Finnish nouns: a model incorporating units resembling linguistically defined morphemes, a whole-word model, or a (...) model that seeks for an optimal balance between these two extremes. Our results showed that human word recognition was predicted best by a combination of two models: a model that decomposes words at some morpheme boundaries while keeping others unsegmented and a whole-word model. The results support dual-route models that assume that both decomposed and full-form representations are utilized to optimally process complex words within the mental lexicon. (shrink)

How should biological behaviour be modelled? A relatively new approach is to investigate problems in neuroethology by building physical robot models of biological sensorimotor systems. The explication and justification of this approach are here placed within a framework for describing and comparing models in the behavioural and biological sciences. First, simulation models – the representation of a hypothesis about a target system – are distinguished from several other relationships also termed “modelling” in discussions of scientific explanation. Seven dimensions on which (...) simulation models can differ are defined and distinctions between them discussed: 1. Relevance: whether the model tests and generates hypotheses applicable to biology. 2. Level: the elemental units of the model in the hierarchy from atoms to societies. 3. Generality: the range of biological systems the model can represent. 4. Abstraction: the complexity, relative to the target, or amount of detail included in the model. 5. Structural accuracy: how well the model represents the actual mechanisms underlying the behaviour. 6. Performance match: to what extent the model behaviour matches the target behaviour. 7. Medium: the physical basis by which the model is implemented. No specific position in the space of models thus defined is the only correct one, but a good modelling methodology should be explicit about its position and the justification for that position. It is argued that in building robot models biological relevance is more effective than loose biological inspiration; multiple levels can be integrated; that generality cannot be assumed but might emerge from studying specific instances; abstraction is better done by simplification than idealisation; accuracy can be approached through iterations of complete systems; that the model should be able to match and predict target behaviour; and that a physical medium can have significant advantages. These arguments reflect the view that biological behaviour needs to be studied and modelled in context, that is, in terms of the real problems faced by real animals in real environments. Key Words: animal behaviour; levels; models; neuroethology; realism; robotics; simulation. (shrink)

Modularity is a fundamental doctrine in the cognitive sciences. It holds a preeminent position in cognitive psychology and generative linguistics, as well as a long history in neurophysiology, with roots going all the way back to the early nineteenth century. But a mature field of neuroscience is a comparatively recent phenomenon and has challenged orthodox conceptions of the modular mind. One way of accommodating modularity within the new framework suggested by these developments is to go for increasingly soft versions of (...) modularity. One such version, which I call the “system” view, is so soft that it promises to meet practically any challenge neuroscience can throw at it. In this paper, I reconsider afresh what we ought to regard as the sine qua non of modularity and offer a few arguments against the view that an insipid “system” module could be the legitimate successor of the traditional notion. (shrink)

Cognitive neuroscience is the branch of neuroscience that studies the neural mechanisms underpinning cognition and develops theories explaining them. Within cognitive neuroscience, computational neuroscience focuses on modeling behavior, using theories expressed as computer programs. Up to now, computational theories have been formulated by neuroscientists. In this paper, we present a new approach to theory development in neuroscience: the automatic generation and testing of cognitive theories using genetic programming (GP). Our approach evolves from experimental data cognitive theories that explain “the mental (...) program” that subjects use to solve a specific task. As an example, we have focused on a typical neuroscience experiment, the delayed-match-to-sample (DMTS) task. The main goal of our approach is to develop a tool that neuroscientists can use to develop better cognitive theories. (shrink)

The scientific study of emotion faces a potentially serious problem: after over a hundred years of psychological study, we lack consensus regarding the very definition of emotion. We propose that part of the problem may be the tendency to define emotion in contrast to cognition, rather than viewing both “emotion” and “cognition” as being comprised of more elemental processes. We argue that considering emotion as a type of cognition (viewed broadly as information processing) may provide an understanding of the mechanisms (...) underlying domains that are traditionally thought to be qualitatively distinct. (shrink)

Connectionist models aiming to reveal the mechanisms of atypical development must in their undamaged form constitute plausible models of normal development and follow a developmental trajectory that matches empirical data. Constructivist models that adapt their structure to the learning task satisfy this demand. They are therefore more informative in the study of atypical development than the static models employed by Thomas & Karmiloff-Smith (T&K-S).

A fundamental claim associated with parallel distributed processing theories of cognition is that knowledge is coded in a distributed manner in mind and brain. This approach rejects the claim that knowledge is coded in a localist fashion, with words, objects, and simple concepts, that is, coded with their own dedicated representations. One of the putative advantages of this approach is that the theories are biologically plausible. Indeed, advocates of the PDP approach often highlight the close parallels between distributed representations learned (...) in connectionist models and neural coding in brain and often dismiss localist theories as biologically implausible. The author reviews a range a data that strongly challenge this claim and shows that localist models provide a better account of single-cell recording studies. The author also contrast local and alternative distributed coding schemes and argues that common rejection of grandmother cell theories in neuroscience is due to a misunderstanding about how localist models behave. The author concludes that the localist representations embedded in theories of perception and cognition are consistent with neuroscience; biology only calls into question the distributed representations often learned in PDP models. (shrink)

The posterior cortex, hippocampus, and prefrontal cortex in the Leabra architecture are specialized in terms of various neural parameters, and thus are predilections for learning and processing, but domain-general in terms of cognitive functions such as face recognition. Also, these areas are not encapsulated and violate Fodorian criteria for modularity. Anderson's terminology obscures these important points, but we applaud his overall message.

To function well in an unpredictable environment using unreliable components, a system must have a high degree of robustness. Robustness is fundamental to biological systems and is an objective in the design of engineered systems such as airplane engines and buildings. Cognitive systems, like biological and engineered systems, exist within variable environments. This raises the question, how do cognitive systems achieve similarly high degrees of robustness? The aim of this study was to identify a set of mechanisms that enhance robustness (...) in cognitive systems. We identify three mechanisms that enhance robustness in biological and engineered systems: system control, redundancy, and adaptability. After surveying the psychological literature for evidence of these mechanisms, we provide simulations illustrating how each contributes to robust cognition in a different psychological domain: psychomotor vigilance, semantic memory, and strategy selection. These simulations highlight features of a mathematical approach for quantifying robustness, and they provide concrete examples of mechanisms for robust cognition. (shrink)

In the multidisciplinary field of developmental cognitive neuroscience, statistical associations between levels of description play an increasingly important role. One example of such associations is the observation of correlations between relatively common gene variants and individual differences in behavior. It is perhaps surprising that such associations can be detected despite the remoteness of these levels of description, and the fact that behavior is the outcome of an extended developmental process involving interaction of the whole organism with a variable environment. Given (...) that they have been detected, how do such associations inform cognitive-level theories? To investigate this question, we employed a multiscale computational model of development, using a sample domain drawn from the field of language acquisition. The model comprised an artificial neural network model of past-tense acquisition trained using the backpropagation learning algorithm, extended to incorporate population modeling and genetic algorithms. It included five levels of description—four internal: genetic, network, neurocomputation, behavior; and one external: environment. Since the mechanistic assumptions of the model were known and its operation was relatively transparent, we could evaluate whether cross-level associations gave an accurate picture of causal processes. We established that associations could be detected between artificial genes and behavioral variation, even under polygenic assumptions of a many-to-one relationship between genes and neurocomputational parameters, and when an experience-dependent developmental process interceded between the action of genes and the emergence of behavior. We evaluated these associations with respect to their specificity, to their developmental stability, and to their replicability, as well as considering issues of missing heritability and gene–environment interactions. We argue that gene–behavior associations can inform cognitive theory with respect to effect size, specificity, and timing. The model demonstrates a means by which researchers can undertake multiscale modeling with respect to cognition and develop highly specific and complex hypotheses across multiple levels of description. (shrink)

The prefrontal cortex is widely believed to play an important role in facilitating people's ability to switch performance between different tasks. We present a biologically‐based computational model of prefrontal cortex (PFC) that explains its role in task switching in terms of the greater flexibility conferred by activation‐based working memory representations in PFC, as compared with more slowly adapting weight‐based memory mechanisms. Specifically we show that PFC representations can be rapidly updated when a task switches via a dynamic gating mechanism based (...) on a temporal‐differences reward‐prediction learning mechanism. Unlike prior models of this type, the present model develops all of its internal representations via learning mechanisms as shaped by the demands of continuous periodic task switching. This advance opens up a new domain of research into the interactions between working memory task demands and the representations that develop to meet them. Results on a version of the Wisconsin card sorting task are presented for the full model and a number of comparison networks that test the importance of various model features. Furthermore, we show that a lesioned model produces perseverative errors like those seen in frontal patients. (shrink)