This paper introduces a special issue of Cognitive Science initiated on the 25th anniversary of the publication of Parallel Distributed Processing (PDP), a two-volume work that introduced the use of neural network models as vehicles for understanding cognition. The collection surveys the core commitments of the PDP framework, the key issues the framework has addressed, and the debates the framework has spawned, and presents viewpoints on the current status of these issues. The articles focus on both historical roots and contemporary (...) developments in learning, optimality theory, perception, memory, language, conceptual knowledge, cognitive control, and consciousness. Here we consider the approach more generally, reviewing the original motivations, the resulting framework, and the central tenets of the underlying theory. We then evaluate the impact of PDP both on the field at large and within specific subdomains of cognitive science and consider the current role of PDP models within the broader landscape of contemporary theoretical frameworks in cognitive science. Looking to the future, we consider the implications for cognitive science of the recent success of machine learning systems called “deep networks”—systems that build on key ideas presented in the PDP volumes. (shrink)

The most compelling evidence for Epstein et al.'s central thesis that adult second language acquisition is constrained by the innate cognitive structures that constrain native language acquisition would be evidence of poverty of the stimulus. Although there are studies that point to such evidence, Epstein et al.'s primary form of argumentation, targetlike performance by second-language acquiring adults, is much less convincing.

Epstein et al. attribute second language learning to the forces of transfer and language universals. They show that transfer is minimally involved in certain types of learning and therefore conclude that universals are involved. However, they forget to consider the important role of learning in second language acquisition.

The Epstein, Flynn, and Martohardjorno full access hypothesis could be enhanced by inclusion of criteria for falsification.

Much second language acquisition (SLA) research confuses the representational and the developmental problems of language acquisition, assuming that attributes of a property theory will explain the transitions between the stages of a psychogrammar, or that induction will explain the properties of the representational systems which encode language. I argue that Principles and Parameter-setting theory deals only with the representational problem, and that induction must play a role in explaining the developmental problem. The conclusion is that both Epstein et al. and (...) the papers they criticize are right; they are just not right about the same issues. (shrink)

The current conception of the relation between UG and the grammar of a language rules out the no-access hypothesis, but does not distinguish between the full-access and partial-access hypotheses. The former raises the issue of why language acquisition in child and adult should be so different. The evidence presented in Epstein et al.'s target article seems inconclusive regarding a choice between hypotheses.

Neurobiological models of language need a level of analysis that can account for the typical range of language phenomena. Because linguistically motivated models have been successful in explaining numerous language properties, it is premature to dismiss them as biologically irrelevant. Models attempting to unify neurobiology and linguistics need to be sensitive to both sources of evidence.

The study of signed languages has inspired scientific' speculation regarding foundations of human language. Relationships between the acquisition of sign language in apes and man are discounted on logical grounds. Evidence from the differential hreakdown of sign language and manual pantomime places limits on the degree of overlap between language and nonlanguage motor systems. Evidence from functional magnetic resonance imaging reveals neural areas of convergence and divergence underlying signed and spoken languages.

Müller argues that general cognitive skills and linguistic skills are not necessarily independent. However, cross-linguistic evidence from an inherited specific language disorder affecting productive rules suggests significant degrees of modularity, innateness, and universality of language. Confident claims about the overall nature of such a complex system still await more interdisciplinary research.

The apparent very close similarity between the learning of the past tense by Adam and the Plunkett and Marchman model is exaggerated by several misleading comparisons--including arbitrary, unexplained changes in how graphs were plotted. The model's development differs from Adam's in three important ways: Children show a U-shaped sequence of development which does not depend on abrupt changes in input; U-shaped development in the simulation occurs only after an abrupt change in training regimen. Children overregularize vowel-change verbs more than no-change (...) verbs; the simulation overregularizes vowel-change verbs less often than no-change verbs. Children, including Adam, overregularize more than they irregularize; the simulation overregularized less than it irregularized. Interestingly, the RM model--widely criticized as being inadequate--does somewhat better, correctly overregularizing vowel-change verbs more often than no-change verbs, and overregularizing more often than it irregularizes. Although Plunkett and Marchman's (1993) state of the art model incorporated hidden layers and back-propagation, used a more realistic phonological coding scheme, and explored a broader range of parameters than Rumelhart and McClelland's model, their results are farther from psychological reality. It is unknown whether any connectionist model can mimic a child's performance without resorting to unrealistic exogenous changes in the training or input, but it is clear that adding a hidden-layer and back-propagation does not ensure a solution. (shrink)

I consider the role of cognitive modeling in cognitive science. Modeling, and the computers that enable it, are central to the field, but the role of modeling is often misunderstood. Models are not intended to capture fully the processes they attempt to elucidate. Rather, they are explorations of ideas about the nature of cognitive processes. In these explorations, simplification is essential—through simplification, the implications of the central ideas become more transparent. This is not to say that simplification has no downsides; (...) it does, and these are discussed. I then consider several contemporary frameworks for cognitive modeling, stressing the idea that each framework is useful in its own particular ways. Increases in computer power (by a factor of about 4 million) since 1958 have enabled new modeling paradigms to emerge, but these also depend on new ways of thinking. Will new paradigms emerge again with the next 1,000‐fold increase? (shrink)

The study of human intelligence was once dominated by symbolic approaches, but over the last 30 years an alternative approach has arisen. Symbols and processes that operate on them are often seen today as approximate characterizations of the emergent consequences of sub- or nonsymbolic processes, and a wide range of constructs in cognitive science can be understood as emergents. These include representational constructs (units, structures, rules), architectural constructs (central executive, declarative memory), and developmental processes and outcomes (stages, sensitive periods, neurocognitive (...) modules, developmental disorders). The greatest achievements of human cognition may be largely emergent phenomena. It remains a challenge for the future to learn more about how these greatest achievements arise and to emulate them in artificial systems. (shrink)

Although theFull-accesshypothesis accounts for a range of empirical generalizations of L2A, it underrepresents the role of language transfer in the construction of the L2 grammar. We suggest the possibility that linguistic principles which constrain the L2 grammar are available both directly from UG and via the L1, a logical hypothesis which Epstein et al. do not consider.

Chomskyian claims of a genetically hard-wired and cognitively autonomous “universal grammar” are being promoted by generative linguistics as facts about language to the present day. The related doctrine of an evolutionary discontinuity in language emergence, however, is based on misconceptions about the notions of homology and preadaptation. The obvious lack of equivalence between symbolic communicative capacities in existing nonhuman primates and human language does not preclude common roots. Normal and disordered language development is strongly influenced by the genome, but there (...) is no evidence for the existence of specific genes underlying “universal grammar.” In the mature brain, stages of language processing can be distinguished and “first-pass” syntactic analyses appear to precede semantic decoding. However, this partial seriality – as well as behavioral and clinical dissociations between lexical and functional categories – can be best described, not in terms of serially activated and discrete modules, but in terms of classes of cell assemblies that differ in their distributional properties. Whereas cell assemblies involved in semantic interpretation (“content word” assemblies) are widely distributed and are generally less vulnerable to focal lesion, those involved in structural decoding (functor assemblies) are primarily distributed within the left perisylvian cortices and are selectively vulnerable to left perisylvian lesion. These distributional differences are explained in terms of the perceptuomotor components involved in the acquisition of relevant representations. The emerging “motivated” or “toposemantic” brain regional specificity can only be accommodated with “soft” and maturational versions of modularity. The failure to reproduce double dissociations in current connectionist models is due to overly simple neuroscientific assumptions, notably that of overall equipotentiality. Linguistic models should not be expected to be “implemented” in the brain, but need to be constrained by neuroscientific evidence on how biological brains function. (shrink)

Contrary to Müller's claims, and in support of modular theories, genetic factors play a substantial and significant role in language. The finding that some children with specific language impairment (SLI) have nonlinguistic impairments may reflect improper diagnosis of SLI or impairments that are secondary to linguistic impairments. Thus, such findings do not argue against the modularity thesis. The lexical/functional distinction appears to be innate and specifically linguistic and could be instantiated in either symbolic or connectionist systems.

Evidence is presented from a polyglot savant to suggest that double dissociations between linguistic and nonverbal abilities are more important than Müller's target article implies. It is also argued that the special nature of syntax makes its assimilation to other aspects of language or to nonhuman communication systems radically implausible.

I enlarge on the theme that the brain mechanisms required for languageand other aspects of the human mind evolved through selective changes in the regulatory genes governing growth. Extension of the period of postnatal growth increases the role of the environment in structuring the brain, and spatiotemporal programming (heterochrony) ofgrowth might explain hierarchical representation, hemispheric specialization, and perhaps sex differences.

The concepts of the innateness, universality, species-specificity, and autonomy of the human language capacity have had an extreme impact on the psycholinguistic debate for over thirty years. These concepts are evaluated from several neurobiological perspectives, with an emphasis on the emergence of language and its decay due to brain lesion and progressive brain disease.Evidence of perceptuomotor homologies and preadaptations for human language in nonhuman primates suggests a gradual emergence of language during hominid evolution. Regarding ontogeny, the innate component of language (...) capacity is likely to be polygenic and shared with other developmental domains. Dissociations between verbal and nonverbal development are probably rooted in the perceptuomotor specializations of neural substrates rather than the autonomy of a grammar module. Aphasiologicaldata often assumed to suggest modular linguistic subsystems can be accounted for in terms of a neurofunctional model incorporating perceptuomotor-based regional specializationsand distributivity of representations. Thus, dissociations between grammatical functors and content words are due to different conditions of acquisition and resulting differences in neural representation. Human brains are characterized by multifactorial interindividual variability, and strict universality of functional organization is biologically unrealistic.A theoretical alternative is proposed according to which (1) linguistic specialization of brain areas is due to epigenetic and probabilistic maturational events, not to genetic ”hard-wiring,” and (2) linguistic knowledge is neurally represented in distributed cell assemblies whose topography reflects the perceptuomotor modalities involved in the acquisition and use of a given item of knowledge. (shrink)

Comments on G. Marcus' criticisms (see record 1996-24670-001) of K. Plunkett's and V. Marcham's (see record 1994-35650-001) connectionist account of the acquisition of the English past tense (verb morphology). The original model is reviewed. Graphing, overregularization, and other criticisms are addressed (PsycINFO Database Record (c) 2000 APA, all rights reserved).

What is the interaction between storage and computation in language processing? What is the psychological status of grammatical rules? What are the relative strengths of connectionist and symbolic models of cognition? How are the components of language implemented in the brain? The English past tense has served as an arena for debates on these issues. We defend the theory that irregular past-tense forms are stored in the lexicon, a division of declarative memory, whereas regular forms can be computed by a (...) concatenation rule, which requires the procedural system. Irregulars have the psychological, linguistic and neuropsychological signatures of lexical memory, whereas regulars often have the signatures of grammatical processing. Furthermore, because regular inflection is rule-driven, speakers can apply it whenever memory fails. (shrink)

The acquisition of English noun and verb morphology is modeled using a single-system connectionist network. The network is trained to produce the plurals and past tense forms of a large corpus of monosyllabic English nouns and verbs. The developmental trajectory of network performance is analyzed in detail and is shown to mimic a number of important features of the acquisition of English noun and verb morphology in young children. These include an initial error-free period of performance on both nouns and (...) verbs followed by a period of intermittent over-regularization of irregular nouns and verbs. Errors in the model show evidence of phonological conditioning and frequency effects. Furthermore, the network demonstrates a strong tendency to regularize denominal verbs and deverbal nouns and masters the principles of voicing assimilation. Despite their incorporation into a single-system network, nouns and verbs exhibit some important differences in their profiles of acquisition. Most importantly, noun inflections are acquired earlier than verb inflections. The simulations generate several empirical predictions that can be used to evaluate further the suitability of this type of cognitive architecture in the domain of inflectional morphology. (shrink)

The target article advanced the null, unified and widely misinterpreted generative hypothesis regarding second language (L2) acquisition. Postulating that UG (Universal Grammar) constrains L2 knowledge growth does not entail identical developmental trajectories for L2 and first language (LI) acquisition; nor does it preclude a role for the L1. In embracing this hypothesis, we maintain a distinction between competence and performance. Those who conflate the two repeat fundamental and by no means unprecedented misconstruals of the generative enterprise, and more specifically, of (...) the empirical content of the null hypothesis regarding L2 linguistic knowledge growth. We hope to have identified certain common goals, the adoption of which might constitute a firm foundation for continued productive interdisciplinary development of contemporary theoretical and experimental L2 acquisition research. (shrink)

This commentary focuses on whether the full set of categories is available to beginning L2 learners. After critiquing Epstein et al.'s experimental evidence for the presence of functional categories, I outline the results of an experiment involving English speakers learning Japanese as a second language that does indeed point toward the early presence of fuuctional categories.

This target article's handling of theory and data and the range of evidence surveyed for its main contention fall short of normal BBS standards. However, the contention itself is reasonable and can be supported if one rejects the metaphor for linguistic competence and accepts that are no more than the way the brain does language.

To what extent, if any, does Universal Grammar (UG) constrain second language (L2) acquisition? This is not only an empirical question, but one which is currently investigable. In this context, L2 acquisition is emerging as an important new domain of psycholinguistic research. Three logical possibilities have been articulated regarding the role of UG in L2 acquisition: The first is the hypothesis that claims that no aspect of UG is available to the L2 learner. The second is the hypothesis that claims (...) that only LI instantiated principles and LI instantiated parameter-values of UG are available to the learner. The third, called the hypothesis, asserts that UG in its entirety constrains L2 acquisition. (shrink)

As children gradually master grammatical rules, they often go through a period of producing form‐meaning associations that were not observed in the input. For example, 2‐ to 3‐year‐old English‐learning children use the bare form of verbs in settings that require obligatory past tense meaning while already starting to produce the grammatical –ed inflection. While many studies have focused on overgeneralization errors, fewer studies have attempted to explain the root of this earlier stage of rule acquisition. In this work, we use (...) computational modeling to replicate children's production behavior prior to the generalization of past tense production in English. We illustrate how seemingly erroneous productions emerge in a model, without being licensed in the grammar and despite the model aiming at conforming to grammatical forms. Our results show that bare form productions stem from a tension between two factors: (1) trying to produce a less frequent meaning (the past tense) and (2) being unable to restrict the production of frequent forms (the bare form) as learning progresses. Like children, our model goes through a stage of bare form production and then converges on adult‐like production of the regular past tense, showing that these different stages can be accounted for through a single learning mechanism. (shrink)

Rumelhart and McClelland's chapter about learning the past tense created a degree of controversy extraordinary even in the adversarial culture of modern science. It also stimulated a vast amount of research that advanced the understanding of the past tense, inflectional morphology in English and other languages, the nature of linguistic representations, relations between language and other phenomena such as reading and object recognition, the properties of artificial neural networks, and other topics. We examine the impact of the Rumelhart and McClelland (...) model with the benefit of 25 years of hindsight. It is not clear who “won” the debate. It is clear, however, that the core ideas that the model instantiated have been assimilated into many areas in the study of language, changing the focus of research from abstract characterizations of linguistic competence to an emphasis on the role of the statistical structure of language in acquisition and processing. (shrink)

One can disagree with Müller that it is neurobiologically questionable to suppose that human language is innate, specialized, and species-specific, yet agree that the precise brain mechanisms controlling language in any individual will be influenced by epigenesis and genetic variability, and that the interplay between inherited and acquired aspects of linguistic capacity deserves to be investigated.

Müller provides a valuable synthesis of neurobiological evidence on the epigenetic development of neural structures involved in language acquisition. The pluripotentiality of developing neural tissue crucially constrains linguistic/cognitive theorizing about supposedly innate neural mechanisms and contributes significantly to our understanding of experience–dependent processes involved in language acquisition. Without this understanding, any proposed explanation of language acquisition is suspect.

Regularly inflected forms often behave differently in language production than irregular forms. These differences are often used to argue that irregular forms are listed in the lexicon but regular forms are produced by rule. Using an experimental speech production task with adults, it is shown that overtensing errors, where a tensed verb is used in place of an infinitive, predominantly involve irregular forms, but that the differences may be due to phonological confounds, not to regularity per se. Errors involve vowel‐changing (...) irregular forms more than suffixing inflected forms, with at best a small difference between regular ‐ed and irregular ‐en. Frequency effects on overtensing errors require a model in which the past‐tense and base forms of the verb are in competition and in which activation functions are nonlinear, and rule out models with specialized subnetworks for past‐tense forms. Implications for theories of language production are discussed. (shrink)

The productive use of English past tense morphology in school‐aged children (N = 74; 3 years, 8 months to 13 years, 5 months) is explored using an elicited production task. Errors represented 20% of the responses overall. Virtually all of the children demonstrated productivity with regular (e.g., good) and irregular patterns (zero‐marking, e.g., sit → sit; vowel‐change, e.g., ride → rid). Overall frequency of errors decreased with age, yet the tendency for certain types of irregularizations increased in the older groups. (...) Analyses across items indicated that all error types were predicted by combinations of item frequency, phonological characteristics of stems and past tense forms, and aspects of phonological past tense “neighborhoods”. Contrary to “hybrid” or dual‐mechanism models incorporating a phonologically‐insensitive default mechanism (e.g., Prasada & Pinker, 1993), these results suggest that children's productivity with regular and irregular patterns is consistent with a phonologically‐based constraint satisfaction system similar to that implemented in connectionist models (Daugherty & Seidenberg, 1992; Plunkett & Marchman, 1991, 1993). (shrink)

This commentary addresses two pervasive misconceptions which emerge in Epstein et al.'s target article: (1) that study of second language acquisition (SLA) began in the mid-twentieth century; (2) that SLA has only recently become able to contribute to linguistic theory. There is abundant historical counterevidence; I argue that (1) and (2) obscure the legitimacy of Epstein et al.'s “full access” hypothesis.

The belief that syntax is an innate, autonomous, species-specific module is highly questionable. Syntax demonstrates the mosaic nature of evolutionary change, in that it made use of (and led to the enhancement of) numerous preexisting neurocognitive features. It is best understood as an emergent characteristic of the explosion of semantic complexity that occurred during hominid evolution.

The synthetic position adopted by Müller is weakened by a large number of overinterpretations and misrepresentations, together with a caricatured view of innateness and modularity.

Müller argues that double dissociations do not imply underlying modularity of the cognitive system, citing neural networks as examples of fully distributed systems that can give rise to double dissociations. We challenge this claim, noting that suchdouble dissociations typically do not “scale-up,” and that even some singledissociations can be difficult to account for in a distributed system.

The implications of Epstein et al.'s critical evaluation of much of the existing literature on L2-acquisition extends far beyond the domain they discuss. I argue that similar methodological clarification is urgently needed in analyses of the role of UG in L1-acquisition, as well as in discussions in such seemingly “distant” areas as the study of language change.

We argue that the model developed in Epstein et al.'s target article does not explain differences between child first language (LI) acquisition and adult second language (L2) acquisition. We therefore sketch an alternative view, originally developed in Clahsen and Muysken (1989), in the light of new empirical findings and theoretical developments.

Müller's review of the neuroscientific evidence undermines nativist claims for autonomous syntax and the argument from the poverty of the stimulus. Generativists will appeal to data from language acquisition, but here too there is growing evidence against the nativist position. Epigenetic naturalism, the developmental alternative to nativism, can be extended to epigenetic socionaturalism, acknowledging the importance of sociocultural processes in language and cognitive development.

Both autonomy and local specificity are compatible with observed interconnectivity at the cell level when considering two different levels: cell assemblies and brain systems. Early syntactic structuring processes in particular are likely to representan autonomous module in the language/brain system.

Müller misconstrues autonomy to mean strict locality of brain function, something quite different from the functional autonomy that linguists claim. Similarly, he misperceives the interaction of learned and innate components hypothesized in current generative models. Evidence from sign languages, Creole languages, and neurological studies of rare forms of aphasia also argues against his conclusions.