The apparent very close similarity between the learning of the past tense by Adam and the Plunkett and Marchman model is exaggerated by several misleading comparisons--including arbitrary, unexplained changes in how graphs were plotted. The model's development differs from Adam's in three important ways: Children show a U-shaped sequence of development which does not depend on abrupt changes in input; U-shaped development in the simulation occurs only after an abrupt change in training regimen. Children overregularize vowel-change verbs more than no-change (...) verbs; the simulation overregularizes vowel-change verbs less often than no-change verbs. Children, including Adam, overregularize more than they irregularize; the simulation overregularized less than it irregularized. Interestingly, the RM model--widely criticized as being inadequate--does somewhat better, correctly overregularizing vowel-change verbs more often than no-change verbs, and overregularizing more often than it irregularizes. Although Plunkett and Marchman's (1993) state of the art model incorporated hidden layers and back-propagation, used a more realistic phonological coding scheme, and explored a broader range of parameters than Rumelhart and McClelland's model, their results are farther from psychological reality. It is unknown whether any connectionist model can mimic a child's performance without resorting to unrealistic exogenous changes in the training or input, but it is clear that adding a hidden-layer and back-propagation does not ensure a solution. (shrink)

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in (...) speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research. (shrink)

Two widely accepted assumptions within cognitive science are that (1) the goal is to understand the mechanisms responsible for cognitive performances and (2) computational modeling is a major tool for understanding these mechanisms. The particular approaches to computational modeling adopted in cognitive science, moreover, have significantly affected the way in which cognitive mechanisms are understood. Unable to employ some of the more common methods for conducting research on mechanisms, cognitive scientists’ guiding ideas about mechanism have developed in conjunction with their (...) styles of modeling. In particular, mental operations often are conceptualized as comparable to the processes employed in classical symbolic AI or neural network models. These models, in turn, have been interpreted by some as themselves intelligent systems since they employ the same type of operations as does the mind. For this paper, what is significant about these approaches to modeling is that they are constructed specifically to account for behavior and are evaluated by how well they do so—not by independent evidence that they describe actual operations in mental mechanisms. (shrink)

Comments on G. Marcus' criticisms (see record 1996-24670-001) of K. Plunkett's and V. Marcham's (see record 1994-35650-001) connectionist account of the acquisition of the English past tense (verb morphology). The original model is reviewed. Graphing, overregularization, and other criticisms are addressed (PsycINFO Database Record (c) 2000 APA, all rights reserved).

What is the interaction between storage and computation in language processing? What is the psychological status of grammatical rules? What are the relative strengths of connectionist and symbolic models of cognition? How are the components of language implemented in the brain? The English past tense has served as an arena for debates on these issues. We defend the theory that irregular past-tense forms are stored in the lexicon, a division of declarative memory, whereas regular forms can be computed by a (...) concatenation rule, which requires the procedural system. Irregulars have the psychological, linguistic and neuropsychological signatures of lexical memory, whereas regulars often have the signatures of grammatical processing. Furthermore, because regular inflection is rule-driven, speakers can apply it whenever memory fails. (shrink)

The acquisition of English noun and verb morphology is modeled using a single-system connectionist network. The network is trained to produce the plurals and past tense forms of a large corpus of monosyllabic English nouns and verbs. The developmental trajectory of network performance is analyzed in detail and is shown to mimic a number of important features of the acquisition of English noun and verb morphology in young children. These include an initial error-free period of performance on both nouns and (...) verbs followed by a period of intermittent over-regularization of irregular nouns and verbs. Errors in the model show evidence of phonological conditioning and frequency effects. Furthermore, the network demonstrates a strong tendency to regularize denominal verbs and deverbal nouns and masters the principles of voicing assimilation. Despite their incorporation into a single-system network, nouns and verbs exhibit some important differences in their profiles of acquisition. Most importantly, noun inflections are acquired earlier than verb inflections. The simulations generate several empirical predictions that can be used to evaluate further the suitability of this type of cognitive architecture in the domain of inflectional morphology. (shrink)

Poverty of the Stimulus Arguments have convinced many linguists and philosophers of language that a domain specific language acquisition device (LAD) is necessary to account for language learning. Here we review empirical evidence that casts doubt on the necessity of this domain specific device. We suggest that more attention needs to be paid to the early stages of language acquisition. Many seemingly innate language-related abilities have to be learned over the course of several months. Further, the language input contains rich (...) stochastic information that can be accessed by domain-general learning mechanisms. Computer simulation has shown how mechanisms that are not domain specific can exploit the information contained in language. We conclude that (i) Poverty of the Stimulus Arguments need to be conceptually clarified and (ii) more empirical research needs to be carried out before we can rule out that data driven general purpose mechanisms can account for language learning. (shrink)

Phonemes have variant pronunciations depending on context. For instance, in American English, the [t] in pat [pæt] and the [d] in pad [pæd] are both realized with a tap [ɾ] when the –ing suffix is attached, [pæɾɪŋ]. We show that despite greater distributional and acoustic support for the [t]-tap alternation, 12-month-olds successfully relate taps to stems with a perceptually-similar final [d], not the dissimilar final-[t]. Thus, distributional learning of phonological alternations is constrained by infants' preference for the alternation of perceptually-similar (...) segments. Further, the ability to relate variant surface forms emerges between 8- and 12-months. Our findings of biased learning provide further empirical support for a role for perceptual similarity in the acquisition of linguistically-relevant categories. We discuss the implications of our findings for phonological theory, language acquisition and models of the mental lexicon. (shrink)

In the multidisciplinary field of developmental cognitive neuroscience, statistical associations between levels of description play an increasingly important role. One example of such associations is the observation of correlations between relatively common gene variants and individual differences in behavior. It is perhaps surprising that such associations can be detected despite the remoteness of these levels of description, and the fact that behavior is the outcome of an extended developmental process involving interaction of the whole organism with a variable environment. Given (...) that they have been detected, how do such associations inform cognitive-level theories? To investigate this question, we employed a multiscale computational model of development, using a sample domain drawn from the field of language acquisition. The model comprised an artificial neural network model of past-tense acquisition trained using the backpropagation learning algorithm, extended to incorporate population modeling and genetic algorithms. It included five levels of description—four internal: genetic, network, neurocomputation, behavior; and one external: environment. Since the mechanistic assumptions of the model were known and its operation was relatively transparent, we could evaluate whether cross-level associations gave an accurate picture of causal processes. We established that associations could be detected between artificial genes and behavioral variation, even under polygenic assumptions of a many-to-one relationship between genes and neurocomputational parameters, and when an experience-dependent developmental process interceded between the action of genes and the emergence of behavior. We evaluated these associations with respect to their specificity, to their developmental stability, and to their replicability, as well as considering issues of missing heritability and gene–environment interactions. We argue that gene–behavior associations can inform cognitive theory with respect to effect size, specificity, and timing. The model demonstrates a means by which researchers can undertake multiscale modeling with respect to cognition and develop highly specific and complex hypotheses across multiple levels of description. (shrink)

Rumelhart and McClelland's chapter about learning the past tense created a degree of controversy extraordinary even in the adversarial culture of modern science. It also stimulated a vast amount of research that advanced the understanding of the past tense, inflectional morphology in English and other languages, the nature of linguistic representations, relations between language and other phenomena such as reading and object recognition, the properties of artificial neural networks, and other topics. We examine the impact of the Rumelhart and McClelland (...) model with the benefit of 25 years of hindsight. It is not clear who “won” the debate. It is clear, however, that the core ideas that the model instantiated have been assimilated into many areas in the study of language, changing the focus of research from abstract characterizations of linguistic competence to an emphasis on the role of the statistical structure of language in acquisition and processing. (shrink)

The child directed speech of twelve English‐speaking motherswas analyzed in terms of utterance‐level constructions. First, the mothers' utterances were categorized in terms of general constructional categories such as Wh‐questions, copulas and transitives. Second, mothers' utterances within these categories were further specified in terms of the initial words that framed the utterance, item‐based phrases such as Are you …, I'll …, It's …, Let's …, What did … The findings were: (i) overall, only about 15% of all maternal utterances had SVO (...) form (most were questions, imperatives, copulas, and fragments); (ii) 51% of all maternal utterances began with one of 52 item‐based phrases, mostly consisting of two words or morphemes (45% began with one of just 17 words); and (iii) children used many of these same item‐based phrases, in some cases at a rate that correlated highly with their own mother's frequency of use. We suggest that analyses of adult–child linguistic interaction should take into account not just general constructional categories, but also the item‐based constructions that adults and children use and the frequency with which they use them. (shrink)

The productive use of English past tense morphology in school‐aged children (N = 74; 3 years, 8 months to 13 years, 5 months) is explored using an elicited production task. Errors represented 20% of the responses overall. Virtually all of the children demonstrated productivity with regular (e.g., good) and irregular patterns (zero‐marking, e.g., sit → sit; vowel‐change, e.g., ride → rid). Overall frequency of errors decreased with age, yet the tendency for certain types of irregularizations increased in the older groups. (...) Analyses across items indicated that all error types were predicted by combinations of item frequency, phonological characteristics of stems and past tense forms, and aspects of phonological past tense “neighborhoods”. Contrary to “hybrid” or dual‐mechanism models incorporating a phonologically‐insensitive default mechanism (e.g., Prasada & Pinker, 1993), these results suggest that children's productivity with regular and irregular patterns is consistent with a phonologically‐based constraint satisfaction system similar to that implemented in connectionist models (Daugherty & Seidenberg, 1992; Plunkett & Marchman, 1991, 1993). (shrink)

The distinction between rules and similarity is central to our understanding of much of cognitive psychology. Two aspects of existing research have motivated the present work. First, in different cognitive psychology areas we typically see different conceptions of rules and similarity; for example, rules in language appear to be of a different kind compared to rules in categorization. Second, rules processes are typically modeled as separate from similarity ones; for example, in a learning experiment, rules and similarity influences would be (...) described on the basis of separate models. In the present article, I assume that the rules versus similarity distinction can be understood in the same way in learning, reasoning, categorization, and language, and that a unified model for rules and similarity is appropriate. A rules process is considered to be a similarity one where only a single or a small subset of an object's properties are involved. Hence, rules and overall similarity operations are extremes in a single continuum of similarity operations. It is argued that this viewpoint allows adequate coverage of theory and empirical findings in learning, reasoning, categorization, and language, and also a reassessment of the objectives in research on rules versus similarity. Key Words: categorization; cognitive explanation; language; learning; reasoning; rules; similarity. (shrink)

Cognitive scientists must understand not just what the mind does, but how it does what it does. In this paper, I consider four aspects of cognitive architecture: how the mind develops, the extent to which it is or is not modular, the extent to which it is or is not optimal, and the extent to which it should or should not be considered a symbol‐manipulating device (as opposed to, say, an eliminative connectionist network). In each case, I argue that insights (...) from developmental and evolutionary biology can lead to substantive and important compromises in historically vexed debates. (shrink)

In this study we present a self-organizing connectionist model of early lexical development. We call this model DevLex-II, based on the earlier DevLex model. DevLex-II can simulate a variety of empirical patterns in children's acquisition of words. These include a clear vocabulary spurt, effects of word frequency and length on age of acquisition, and individual differences as a function of phonological short-term memory and associative capacity. Further results from lesioned models indicate developmental plasticity in the network's recovery from damage, in (...) a non-monotonic fashion. We attribute the network's abilities in accounting for lexical development to interactive dynamics in the learning process. In particular, variations displayed by the model in the rate and size of early vocabulary development are modulated by (a) input characteristics, such as word frequency and word length, (b) consolidation of lexical-semantic representation, meaning-form association, and phonological short-term memory, and (c) delayed processes due to interactions among timing, severity, and recoverability of lesion. Together, DevLex and DevLex-II provide an accurate computational account of early lexical development. (shrink)

In a series of studies children show increasing mastery of irregular plural forms (such as mice) simply by producing erroneous over‐regularized versions of them (such as mouses). We explain this phenomenon in terms of successive approximation in imitation: Children over‐regularize early in acquisition because the representations of frequent, regular plural forms develop more quickly, such that at the earliest stages of production they interfere with children's attempts to imitatively reproduce irregular forms they have heard in the input. As the strength (...) of the representations that determine children's productions settle asymptotically, the early advantage for the frequent regular forms is negated, and children's attempts to imitate the irregular forms they have observed become more likely to succeed (a process that produces the classic U‐shape in children's acquisition of plural inflection). These data show that children can acquire correct linguistic behavior without feedback in a situation where, as a result of philosophical and linguistic analyses, it has often been argued that it is logically impossible for them to do so. (shrink)

While cognitive changes in aging and neurodegenerative disease have been widely studied, language changes in these populations are less well understood. Inflecting novel words in a language with complex inflectional paradigms provides a good opportunity to observe how language processes change in normal and abnormal aging. Studies of language acquisition suggest that children inflect novel words based on their phonological similarity to real words they already know. It is unclear whether speakers continue to use the same strategy when encountering novel (...) words throughout the lifespan or whether adult speakers apply symbolic rules. We administered a simple speech elicitation task involving Finnish‐conforming pseudo‐words and real Finnish words to healthy older adults, individuals with mild cognitive impairment, and individuals with Alzheimer's disease (AD) to investigate inflectional choices in these groups and how linguistic variables and disease severity predict inflection patterns. Phonological resemblance of novel words to both a regular and an irregular inflectional type, as well as bigram frequency of the novel words, significantly influenced participants' inflectional choices for novel words among the healthy elderly group and people with AD. The results support theories of inflection by phonological analogy (single‐route models) and contradict theories advocating for formal symbolic rules (dual‐route models). (shrink)

Inflectional morphology has been taken as a paradigmatic example of rule-governed grammatical knowledge (Pinker, 1999). The plausibility of this claim may be related to the fact that it is mainly based on studies of English, which has a very simple inflectional system. We examined the representation of inflectional morphology in Serbian, which encodes number, gender, and case for nouns. Linguists standardly characterize this system as a complex set of rules, with disagreements about their exact form. We present analyses of a (...) large corpus of nouns which showed that, as in English, Serbian inflectional morphology is quasiregular: It exhibits numerous partial regularities creating neighborhoods that vary in size and consistency. We then asked whether a simple connectionist network could encode this statistical information in a manner that also supported generalization. A network trained on 3,244 Serbian nouns learned to produce correctly inflected phonological forms from a specification of a word's lemma, gender, number, and case, and generalized to untrained cases. The model's performance was sensitive to variables that also influence human performance, including surface and lemma frequency. It was also influenced by inflectional neighborhood size, a novel measure of the consistency of meaning to form mapping. A word-naming experiment with native Serbian speakers showed that this measure also affects human performance. The results suggest that, as in English, generating correctly inflected forms involves satisfying a small number of simultaneous probabilistic constraints relating form and meaning. Thus, common computational mechanisms may govern the representation and use of inflectional information across typologically diverse languages. (shrink)

A number of recent studies have examined the effects of phonological variation on the perception of speech. These studies show that both the lexical representations of words and the mechanisms of lexical access are organized so that natural, systematic variation is tolerated by the perceptual system, while a general intolerance of random deviation is maintained. Lexical abstraction distinguishes between phonetic features that form the invariant core of a word and those that are susceptible to variation. Phonological inference relies on the (...) context of surface changes to retrieve the underlying phonological form. In this article we present a model of these processes in speech perception, based on connectionist learning techniques. A simple recurrent network was trained on the mapping from the variant surface form of speech to the underlying form. Once trained, the network exhibited features of both abstraction and inference in its processing of normal speech, and predicted that similar behavior will be found in the perception of nonsense words. This prediction was confirmed in subsequent research (Gaskell & Marslen-Wilson, 1994). (shrink)

Kinds of minds, Daniel Dennett. New York: Basic Books, 1996. ISBN 0–465–07350–6Darwin's dangerous idea: evolution and the meanings of life, Daniel C. Dennett. New York: Simon & Schuster, 1995. ISBN 0–684–80290–2The cognitive neurosciences, Michael S. Gazzaniga (Ed.) Cambridge, MA: MIT Press, 1995. ISBN 0–262–07157–6Lessons from an optical illusion: on nature and nurture, knowledge and values, Edward M. Hundert. Cambridge, MA: Harvard University Press, 1995. ISBN 0–674–52540‐XWittgenstein on mind and language, David G. Stern, Oxford: Oxford University Press, 1995. ISBN 0–19–508000–9Ten problems (...) of consciousness: a representational theory of the phenomenal mind, Michael Tye. Cambridge, MA: MIT Press, 1995. ISBN: 0–262–20103–8Android epistemology, K.M. Ford, C. Glymour & P.J. Hayes (Eds). Cambridge, MA: MIT Press, 1995. ISBN 0–262–06184–8Autonomous agents, Alfred R. Mele. Oxford University Press, 1995. (shrink)

How do English‐speaking children inflect nouns for plurality and verbs for the past tense? We assess theoretical answers to this question by considering errors of omission, which occur when children produce a stem in place of its inflected counterpart (e.g., saying “dress” to refer to 5 dresses). A total of 307 children (aged 3;11–9;9) participated in 3 inflection studies. In Study 1, we show that errors of omission occur until the age of 7 and are more likely with both sibilant (...) regular nouns (e.g., dress) and irregular nouns (e.g., man) than regular nouns (e.g., dog). Sibilant nouns are more likely to be inflected if they are high frequency. In Studies 2 and 3, we show that similar effects apply to the inflection of verbs and that there is an advantage for “regular‐like” irregulars whose inflected form, but not stem form, ends in d/t. The results imply that (a) stems and inflected forms compete for production and (b) children generalize both product‐oriented and source‐oriented schemas when learning about inflectional morphology. (shrink)

The productive use of English past tense morphology in school-aged children (N= 74; 3 years, 8 months to 13 years, 5 months) is explored using on elicited production task. Errors represented 20% of the responses overall. Virtually all of the children demonstrated productivity with regular (e.g., good) and irregular patterns (zero-marking, e.g., sit + sit; vowel-change, e.g., ride -+ rid). Overall frequency of errors decreased with age, yet the tendency for certain types of irregularizations increased in the older groups. Analyses (...) across items indicated that all error types were predicted by combinations of item frequency, phonological chamcteristics of stems and past tense forms, and aspects of phonological past tense “neighborhoods.” Contrary to “hybrid” or dual-mechanism models incorporating a phonologically-insensitive default mechanism (e.g., Prasada & Pinker, 1993), these results suggest that children’s productivity with regular and irregular patterns is consistent with a phonologically-based constraint satisfaction system similar to that implemented in connectionist models (Daugherty & Seidenberg, 1992; Plunkett 81 Marchman, 1991,1993). (shrink)

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in (...) speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research. (shrink)

Are morphological patterns learned in the form of rules? Some models deny this, attributing all morphology to analogical mechanisms. The dual mechanism model (Pinker, S., & Prince, A. (1998). On language and connectionism: analysis of a parallel distributed processing model of language acquisition. Cognition, 28, 73-193) posits that speakers do internalize rules, but that these rules are few and cover only regular processes; the remaining patterns are attributed to analogy. This article advocates a third approach, which uses multiple stochastic rules (...) and no analogy. We propose a model that employs inductive learning to discover multiple rules, and assigns them confidence scores based on their performance in the lexicon. Our model is supported over the two alternatives by new "wug test" data on English past tenses, which show that participant ratings of novel pasts depend on the phonological shape of the stem, both for irregulars and, surprisingly, also for regulars. The latter observation cannot be explained under the dual mechanism approach, which derives all regulars with a single rule. To evaluate the alternative hypothesis that all morphology is analogical, we implemented a purely analogical model, which evaluates novel pasts based solely on their similarity to existing verbs. Tested against experimental data, this analogical model also failed in key respects: it could not locate patterns that require abstract structural characterizations, and it favored implausible responses based on single, highly similar exemplars. We conclude that speakers extend morphological patterns based on abstract structural properties, of a kind appropriately described with rules. (shrink)