Language acquisition may be one of the most difficult tasks that children face during development. They have to segment words from fluent speech, figure out the meanings of these words, and discover the syntactic constraints for joining them together into meaningful sentences. Over the past couple of decades, computational modeling has emerged as a new paradigm for gaining insights into the mechanisms by which children may accomplish these feats. Unfortunately, many of these models assume a computational complexity and linguistic knowledge (...) likely to be beyond the abilities of developing young children. This article shows that, using simple statistical procedures, significant correlations exist between the beginnings and endings of a word and its lexical category in English, Dutch, French, and Japanese. Therefore, phonetic information can contribute to individuating higher level structural properties of these languages. This article also presents a simple 2‐layer connectionist model that, once trained with an initial small sample of words labeled for lexical category, can infer the lexical category of a large proportion of novel words using only word‐edge phonological information, namely the first and last phoneme of a word. The results suggest that simple procedures combined with phonetic information perceptually available to children provide solid scaffolding for emerging lexical categories in language development. (shrink)

Theories of sentence production that involve a convergence of activation from conceptual‐semantic and syntactic‐sequential units inspired a connectionist model that was trained to produce simple sentences. The model used a learning algorithm that resulted in a sharing of responsibility (or “division of labor”) between syntactic and semantic inputs for lexical activation according to their predictive power. Semantically rich, or “heavy”, verbs in the model came to rely on semantic cues more than on syntactic cues, whereas semantically impoverished, or “light”, verbs (...) relied more on syntactic cues. When the syntactic and semantic inputs were lesioned, the model exhibited patterns of production characteristic of agrammatic and anomic aphasic patients, respectively. Anomic models tended to lose the ability to retrieve heavy verbs, whereas agrammatic models were more impaired in retrieving light verbs. These results obtained in both sentence production and single‐word naming simulations. Moreover, simulated agrammatic lexical retrieval was more impaired overall in sentences than in single‐word tasks, in agreement with the literature. The results provide a demonstration of the division‐of‐labor principle, as well as general support for the claim that connectionist learning principles can contribute to the understanding of non‐transparent neuropsychological dissociations. (shrink)

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in (...) speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research. (shrink)

The paper makes three points about the role of double dissociation in cognitive neuropsychology. First, arguments from double dissociation to separate modules work by inference to the best, not the only possible, explanation. Second, in the development of computational cognitive neuropsychology, the contribution of connectionist cognitive science has been to broaden the range of potential explanations of double dissociation. As a result, the competition between explanations, and the characteristic features of the assessment of theories against the criteria of probability and (...) explanatory value, are more visible. Third, cognitive neuropsychology is a division of cognitive psychology but the practice of cognitive neuropsychology proceeds on assumptions that go beyond the subject matter of cognitive psychology. Given such assumptions, neuroscientific findings about lesion location may enhance the value of double dissociation in shifting the balance of support between cognitive theories. (shrink)

Abstract In the first part of this essay (Russell, 1988a) I argued that ?cognisance? (roughly: a subject's knowledge of his relation to the physical world as an experiencer of it) cannot be explained in terms of a syntactic theory of mind, due to the ?referential? and ?holistic? nature of this knowledge. The syntactic account of the higher mental functions is immediately intelligible to us due to its derivation from computer technology, so this would not appear to be a happy result (...) for scientific psychology. In this paper I outline a strategy for developing a scientific psychology of cognisance, beginning from the assumption that such a theory should have a similar status to Chomsky's competence theory of human grammar, whilst diverging from Chomsky on the question of how such a theory should be answerable to data. Most of the paper is taken up with illustrating the way in which Jean Piaget's theory of mental development in humans is at least a first approximation to such a theory. (shrink)

The noisy computation hypothesis of developmental dyslexia (DD) is particularly appealing because it can explain deficits across a variety of domains, such as temporal, auditory, phonological, visual and attentional processes. A key prediction is that noisy computations lead to more variable and less stable word representations. A way to test this hypothesis is through repetition of words, that is, when there is noise in the system, the neural signature of repeated stimuli should be more variable. The hypothesis was tested in (...) an functional magnetic resonance imaging experiment with dyslexic and typical readers by repeating words twelve times. Variability measures were computed both at the behavioral and neural levels. At the behavioral level, we compared the standard deviation of reaction time distributions of repeated words. At the neural level, in addition to standard univariate analyses and measures of intra-item variability, we also used multivariate pattern analyses (representational similarity and classification) to find out whether there was evidence for noisier representations in dyslexic readers compared to typical readers. Results showed that there were no significant differences between the two groups in any of the analyses despite robust results within each group (i.e., high representational similarity between repeated words, good classification of words vs. non-words). In summary, there was no evidence in favor of the idea that dyslexic readers would have noisier neural representations than typical readers. (shrink)

Previous research on associative learning has uncovered detailed aspects of the process, including what types of things are learned, how they are learned, and where in the brain such learning occurs. However, perceptual processes, such as stimulus recognition and identification, take time to unfold. Previous studies of learning have not addressed when, during the course of these dynamic recognition processes, learned representations are formed and updated. If learned representations are formed and updated while recognition is ongoing, the result of learning (...) may incorporate spurious, partial information. For example, during word recognition, words take time to be identified, and competing words are often active in parallel. If learning proceeds before this competition resolves, representations may be influenced by the preliminary activations present at the time of learning. In three experiments using word learning as a model domain, we provide evidence that learning reflects the ongoing dynamics of auditory and visual processing during a learning event. These results show that learning can occur before stimulus recognition processes are complete; learning does not wait for ongoing perceptual processing to complete. (shrink)

The forms of words as they appear in text and speech are central to theories and models of lexical processing. Nonetheless, current methods for simulating their learning and representation fail to approach the scale and heterogeneity of real wordform lexicons. A connectionist architecture termed thesequence encoderis used to learn nearly 75,000 wordform representations through exposure to strings of stress‐marked phonemes or letters. First, the mechanisms and efficacy of the sequence encoder are demonstrated and shown to overcome problems with traditional slot‐based (...) codes. Then, two large‐scale simulations are reported that learned to represent lexicons of either phonological or orthographic wordforms. In doing so, the models learned the statistics of their lexicons as shown by better processing of well‐formed pseudowords as opposed to ill‐formed (scrambled) pseudowords, and by accounting for variance in well‐formedness ratings. It is discussed how the sequence encoder may be integrated into broader models of lexical processing. (shrink)

We agree with Jones & Love (J&L) that much of Bayesian modeling has taken a fundamentalist approach to cognition; but we do not believe in the potential of Bayesianism to provide insights into psychological processes. We discuss the advantages of associative explanations over Bayesian approaches to causal induction, and argue that Bayesian models have added little to our understanding of human causal reasoning.

According to the age-of-acquisition hypothesis, words acquired early in life are processed faster and more accurately than words acquired later. Connectionist models have begun to explore the influence of the age/order of acquisition of items (and also their frequency of encounter). This study attempts to reconcile two different methodological and theoretical approaches (proposed by Lambon Ralph & Ehsan, 2006 and Zevin & Seidenberg, 2002) to age-limited learning effects. The current simulations extend the findings reported by Zevin and Seidenberg (2002) that (...) have shown that frequency trajectories (FTs) have limited and specific effects on word-reading tasks. Using the methodological framework proposed by Lambon Ralph and Ehsan (2006), which makes it possible to compare word-reading and picture-naming tasks in connectionist networks, we were able to show that FT has a considerable influence on age-limited learning effects in a picture naming task. The findings show that when the input–output mappings are arbitrary (simulating picture naming tasks), the links formed by the network become entrenched as a result of early experience and that subsequent variations in frequency of exposure of the items have only a minor impact. In contrast, when the mappings between input-output are quasi-systematic or systematic (simulating word-reading tasks), the training of new items was generalized and resulted in the suppression of age-limited learning effects. At a theoretical level, we suggest that FT, which simultaneously takes account of time and the level of exposure across time, represents a more precise and modulated measure compared with the order of introduction of the items and may lead to innovative hypotheses in the field of age-limited learning effects. (shrink)

In the multidisciplinary field of developmental cognitive neuroscience, statistical associations between levels of description play an increasingly important role. One example of such associations is the observation of correlations between relatively common gene variants and individual differences in behavior. It is perhaps surprising that such associations can be detected despite the remoteness of these levels of description, and the fact that behavior is the outcome of an extended developmental process involving interaction of the whole organism with a variable environment. Given (...) that they have been detected, how do such associations inform cognitive-level theories? To investigate this question, we employed a multiscale computational model of development, using a sample domain drawn from the field of language acquisition. The model comprised an artificial neural network model of past-tense acquisition trained using the backpropagation learning algorithm, extended to incorporate population modeling and genetic algorithms. It included five levels of description—four internal: genetic, network, neurocomputation, behavior; and one external: environment. Since the mechanistic assumptions of the model were known and its operation was relatively transparent, we could evaluate whether cross-level associations gave an accurate picture of causal processes. We established that associations could be detected between artificial genes and behavioral variation, even under polygenic assumptions of a many-to-one relationship between genes and neurocomputational parameters, and when an experience-dependent developmental process interceded between the action of genes and the emergence of behavior. We evaluated these associations with respect to their specificity, to their developmental stability, and to their replicability, as well as considering issues of missing heritability and gene–environment interactions. We argue that gene–behavior associations can inform cognitive theory with respect to effect size, specificity, and timing. The model demonstrates a means by which researchers can undertake multiscale modeling with respect to cognition and develop highly specific and complex hypotheses across multiple levels of description. (shrink)

The visual attention (VA) span is defined as the amount of distinct visual elements which can be processed in parallel in a multi-element array. Both recent empirical data and theoretical accounts suggest that a VA span deficit might contribute to developmental dyslexia, independently of a phonological disorder. In this study, this hypothesis was assessed in two large samples of French and British dyslexic children whose performance was compared to that of chronological-age matched control children. Results of the French study show (...) that the VA span capacities account for a substantial amount of unique variance in reading, as do phonological skills. The British study replicates this finding and further reveals that the contribution of the VA span to reading performance remains even after controlling IQ, verbal fluency, vocabulary and single letter identification skills, in addition to phoneme awareness. In both studies, most dyslexic children exhibit a selective phonological or VA span disorder. Overall, these findings support a multi-factorial view of developmental dyslexia. In many cases, developmental reading disorders do not seem to be due to phonological disorders. We propose that a VA span deficit is a likely alternative underlying cognitive deficit in dyslexia. (shrink)

This article provides an overview of a probabilistic constraints framework for thinking about language acquisition and processing. The generative approach attempts to characterize knowledge of language (i.e., competence grammar) and then asks how this knowledge is acquired and used. Our approach is performance oriented: the goal is to explain how people comprehend and produce utterances and how children acquire this skill. Use of language involves exploiting multiple probabilistic constraints over various types of linguistic and nonlinguistic information. Acquisition is the process (...) of accumulating this information, which begins in infancy. The constraint satisfaction processes that are central to language use are the same as the bootstrapping processes that provide entry to language for the child. Framing questions about acquisition in terms of models of adult performance unifies the two topics under a set of common principles and has important consequences for arguments concerning language learnability. (shrink)

In the last decade, reading research has seen a paradigmatic shift. A new wave of computational models of orthographic processing that offer various forms of noisy position or context-sensitive coding have revolutionized the field of visual word recognition. The influx of such models stems mainly from consistent findings, coming mostly from European languages, regarding an apparent insensitivity of skilled readers to letter order. Underlying the current revolution is the theoretical assumption that the insensitivity of readers to letter order reflects the (...) special way in which the human brain encodes the position of letters in printed words. The present article discusses the theoretical shortcomings and misconceptions of this approach to visual word recognition. A systematic review of data obtained from a variety of languages demonstrates that letter-order insensitivity is neither a general property of the cognitive system nor a property of the brain in encoding letters. Rather, it is a variant and idiosyncratic characteristic of some languages, mostly European, reflecting a strategy of optimizing encoding resources, given the specific structure of words. Since the main goal of reading research is to develop theories that describe the fundamental and invariant phenomena of reading across orthographies, an alternative approach to model visual word recognition is offered. The dimensions of a possible universal model of reading, which outlines the common cognitive operations involved in orthographic processing in all writing systems, are discussed. (shrink)

There is little empirical evidence showing a direct link between a capacity for statistical learning (SL) and proficiency with natural language. Moreover, discussion of the role of SL in language acquisition has seldom focused on literacy development. Our study addressed these issues by investigating the relationship between SL and reading ability in typically developing children and healthy adults. We tested SL using visually presented stimuli within a triplet learning paradigm and examined reading ability by administering the Wide Range Achievement Test (...) (WRAT-4; Wilkinson & Robertson, 2006). A total of 38 typically developing children (mean age of 9;5 years, range 6;4–12;5) and 37 healthy adults (mean age of 21 years, range 18–34) were assessed. In children, SL was significantly related to reading ability. Importantly, this relationship was independent of grade and also age. The adult data, too, revealed that SL was significantly related to reading ability. A regression analysis of the combined child and adult data revealed that SL accounted for a unique amount of variance in reading ability, after age and attention had been taken into consideration. For the first time, this study provides empirical evidence that a capacity for more effective SL is related to higher reading ability in the general population. (shrink)

The self‐teaching hypothesis describes how children progress toward skilled sight‐word reading. It proposes that children do this via phonological recoding with assistance from contextual cues, to identify the target pronunciation for a novel letter string, and in so doing create an opportunity to self‐teach new orthographic knowledge. We present a new computational implementation of self‐teaching within the dual‐route cascaded (DRC) model of reading aloud, and we explore how decoding and contextual cues can work together to enable accurate self‐teaching under a (...) variety of circumstances. The new model (ST‐DRC) uses DRC’s sublexical route and the interactivity between the lexical and sublexical routes to simulate phonological recoding. Known spoken words are activated in response to novel printed words, triggering an opportunity for orthographic learning, which is the basis for skilled sight‐word reading. ST‐DRC also includes new computational mechanisms for simulating how contextual information aids word identification, and it demonstrates how partial decoding and ambiguous context interact to achieve irregular‐word learning. Beyond modeling orthographic learning and self‐teaching, ST‐DRC’s performance suggests new avenues for empirical research on how difficult word classes such as homographs and potentiophones are learned. (shrink)

In this study we present a self-organizing connectionist model of early lexical development. We call this model DevLex-II, based on the earlier DevLex model. DevLex-II can simulate a variety of empirical patterns in children's acquisition of words. These include a clear vocabulary spurt, effects of word frequency and length on age of acquisition, and individual differences as a function of phonological short-term memory and associative capacity. Further results from lesioned models indicate developmental plasticity in the network's recovery from damage, in (...) a non-monotonic fashion. We attribute the network's abilities in accounting for lexical development to interactive dynamics in the learning process. In particular, variations displayed by the model in the rate and size of early vocabulary development are modulated by (a) input characteristics, such as word frequency and word length, (b) consolidation of lexical-semantic representation, meaning-form association, and phonological short-term memory, and (c) delayed processes due to interactions among timing, severity, and recoverability of lesion. Together, DevLex and DevLex-II provide an accurate computational account of early lexical development. (shrink)

This article provides an overview of a probabilistic constraints framework for thinking about language acquisition and processing. The generative approach attempts to characterize knowledge of language (i.e., competence grammar) and then asks how this knowledge is acquired and used. Our approach is performance oriented: the goal is to explain how people comprehend and produce utterances and how children acquire this skill. Use of language involves exploiting multiple probabilistic constraints over various types of linguistic and nonlinguistic information. Acquisition is the process (...) of accumulating this information, which begins in infancy. The constraint satisfaction processes that are central to language use are the same as the bootstrapping processes that provide entry to language for the child. Framing questions about acquisition in terms of models of adult performance unifies the two topics under a set of common principles and has important consequences for arguments concerning language learnability. (shrink)

It is often assumed that similar domain-specific behavioural impairments found in cases of adult brain damage and developmental disorders correspond to similar underlying causes, and can serve as convergent evidence for the modular structure of the normal adult cognitive system. We argue that this correspondence is contingent on an unsupported assumption that atypical development can produce selective deficits while the rest of the system develops normally (Residual Normality), and that this assumption tends to bias data collection in the field. Based (...) on a review of connectionist models of acquired and developmental disorders in the domains of reading and past tense, as well as on new simulations, we explore the computational viability of Residual Normality and the potential role of development in producing behavioural deficits. Simulations demonstrate that damage to a developmental model can produce very different effects depending on whether it occurs prior to or following the training process. Because developmental disorders typically involve damage prior to learning, we conclude that the developmental process is a key component of the explanation of endstate impairments in such disorders. Further simulations demonstrate that in simple connectionist learning systems, the assumption of Residual Normality is undermined by processes of compensation or alteration elsewhere in the system. We outline the precise computational conditions required for Residual Normality to hold in development, and suggest that in many cases it is an unlikely hypothesis. We conclude that in developmental disorders, inferences from behavioural deficits to underlying structure crucially depend on developmental conditions, and that the process of ontogenetic development cannot be ignored in constructing models of developmental disorders. Key Words: Acquired and developmental disorders; connectionist models; modularity; past tense; reading. (shrink)