This article focuses on claims about the origin and evolution of language from the point of view of the formalist–functionalist debate in linguistics. In linguistics, an account of a grammatical phenomenon is considered “formal” if it accords center stage to the structural properties of that phenomenon, and “functional” if it appeals to the language user's communicative needs or to domain‐general human capacities. The gulf between formalism and functionalism has been bridged in language evolution research, in that some leading formalists, Ray (...) Jackendoff for one, appeal to functional mechanisms such as natural selection. In Jackendoff's view, the biological evolution of language has proceeded in stages, each stage improving communicative efficiency. This article calls into question that idea, pointing to the fact that well‐understood purely historical processes suffice to explain the emergence of many grammatical properties. However, one central aspect of formalist linguistic theorizing—the idea of the autonomy of syntax—poses a challenge to the idea, central to most functionalist approaches, that the nature of grammar is a product of purely historical (as opposed to biological) evolution. The article concludes with a discussion of the origins of the autonomy of syntax, speculating that it may well have arisen over evolutionary (as opposed to historical) time. (shrink)

Memory is fleeting. New material rapidly obliterates previous material. How, then, can the brain deal successfully with the continual deluge of linguistic input? We argue that, to deal with this “Now-or-Never” bottleneck, the brain must compress and recode linguistic input as rapidly as possible. This observation has strong implications for the nature of language processing: (1) the language system must “eagerly” recode and compress linguistic input; (2) as the bottleneck recurs at each new representational level, the language system must build (...) a multilevel linguistic representation; and (3) the language system must deploy all available information predictively to ensure that local linguistic ambiguities are dealt with “Right-First-Time”; once the original input is lost, there is no way for the language system to recover. This is “Chunk-and-Pass” processing. Similarly, language learning must also occur in the here and now, which implies that language acquisition is learning to process, rather than inducing, a grammar. Moreover, this perspective provides a cognitive foundation for grammaticalization and other aspects of language change. Chunk-and-Pass processing also helps explain a variety of core properties of language, including its multilevel representational structure and duality of patterning. This approach promises to create a direct relationship between psycholinguistics and linguistic theory. More generally, we outline a framework within which to integrate often disconnected inquiries into language processing, language acquisition, and language change and evolution. (shrink)

Naturally occurring speech contains only a limited amount of complex recursive structure, and this is reflected in the empirically documented difficulties that people experience when processing such structures. We present a connectionist model of human performance in processing recursive language structures. The model is trained on simple artificial languages. We find that the qualitative performance profile of the model matches human behavior, both on the relative difficulty of center‐embedding and cross‐dependency, and between the processing of these complex recursive structures and (...) right‐branching recursive constructions. We analyze how these differences in performance are reflected in the internal representations of the model by performing discriminant analyses on these representations both before and after training. Furthermore, we show how a network trained to process recursive structures can also generate such structures in a probabilistic fashion. This work suggests a novel explanation of people's limited recursive performance, without assuming the existence of a mentally represented competence grammar allowing unbounded recursion. (shrink)

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in (...) speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research. (shrink)

Natural language is full of patterns that appear to fit with general linguistic rules but are ungrammatical. There has been much debate over how children acquire these “linguistic restrictions,” and whether innate language knowledge is needed. Recently, it has been shown that restrictions in language can be learned asymptotically via probabilistic inference using the minimum description length (MDL) principle. Here, we extend the MDL approach to give a simple and practical methodology for estimating how much linguistic data are required to (...) learn a particular linguistic restriction. Our method provides a new research tool, allowing arguments about natural language learnability to be made explicit and quantified for the first time. We apply this method to a range of classic puzzles in language acquisition. We find some linguistic rules appear easily statistically learnable from language experience only, whereas others appear to require additional learning mechanisms (e.g., additional cues or innate constraints). (shrink)

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in (...) speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research. (shrink)

An essential aspect of knowing language is knowing the words of that language. This knowledge is usually thought to reside in the mental lexicon, a kind of dictionary that contains information regarding a word’s meaning, pronunciation, syntactic characteristics, and so on. In this article, a very different view is presented. In this view, words are understood as stimuli that operate directly on mental states. The phonological, syntactic and semantic properties of a word are revealed by the effects it has on (...) those states. (shrink)

In this article, Peter and Rowland explore the role of implicit statistical learning in syntactic development. It is often accepted that the processes observed in classic implicit learning or statistical learning experiments play an important role in the acquisition of natural language syntax. As Peter and Rowland point out, however, the results from neither research strand can be used to fully explain how children's syntax becomes adult‐like. They propose to address this shortcoming by using the structural priming paradigm.

Language acquisition may be one of the most difficult tasks that children face during development. They have to segment words from fluent speech, figure out the meanings of these words, and discover the syntactic constraints for joining them together into meaningful sentences. Over the past couple of decades, computational modeling has emerged as a new paradigm for gaining insights into the mechanisms by which children may accomplish these feats. Unfortunately, many of these models assume a computational complexity and linguistic knowledge (...) likely to be beyond the abilities of developing young children. This article shows that, using simple statistical procedures, significant correlations exist between the beginnings and endings of a word and its lexical category in English, Dutch, French, and Japanese. Therefore, phonetic information can contribute to individuating higher level structural properties of these languages. This article also presents a simple 2‐layer connectionist model that, once trained with an initial small sample of words labeled for lexical category, can infer the lexical category of a large proportion of novel words using only word‐edge phonological information, namely the first and last phoneme of a word. The results suggest that simple procedures combined with phonetic information perceptually available to children provide solid scaffolding for emerging lexical categories in language development. (shrink)

As modern deep networks become more complex, and get closer to human‐like capabilities in certain domains, the question arises as to how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language processing. We first present a diagnostic test dataset to examine the degree of abstract composable structure represented. Analyzing performance on these diagnostic tests indicates a lack of systematicity in representations (...) and decision rules, and reveals a set of heuristic strategies. We then investigate the effect of training distribution on learning these heuristic strategies, and we study changes in these representations with various augmentations to the training set. Our results reveal parallels to the analogous representations in people. We find that these systems can learn abstract rules and generalize them to new contexts under certain circumstances—similar to human zero‐shot reasoning. However, we also note some shortcomings in this generalization behavior—similar to human judgment errors like belief bias. Studying these parallels suggests new ways to understand psychological phenomena in humans as well as informs best strategies for building artificial intelligence with human‐like language understanding. (shrink)

A large body of psycholinguistic research demonstrates that both language processing and language acquisition are sensitive to the distributions of linguistic constructions in usage. Here we investigate how statistical distributions at different linguistic levels – morphological and lexical (Experiments 1 and 2), and phrasal (Experiment 2) – contribute to the ease with which morphosyntax is processed and produced by second language learners. We analyze Chinese ESL learners’ knowledge of four English inflectional morphemes:-ed,-ing, and third-person-son verbs, and plural-son nouns. In Elicited (...) Imitation Tasks, participants listened to length- and difficulty-matched sentences each containing one target morpheme and typed the whole sentence as accurately as they could after a short delay. Experiment 1 investigated lexical and morphemic levels, testing the hypotheses that a morpheme is expected to be more easily processed when it is (1) highlyavailable(i.e., occurring in frequent word-forms), and (2) highlyreliable(i.e., occurring in lemma words that are consistently conjugated in the form containing this morpheme). Thirty sentences were made for each morpheme, divided into three Availability-Reliability Distribution (ARD) groups on the basis of corpus analysis in the Corpus of Contemporary American English (COCA;Davies, 2008-): 10 target words high in availability, 10 high in reliability, and 10 low in both reliability and availability. Responses were scored on whether the target morpheme was accurately reproduced given the provision of the correct lemma. A generalized linear mixed-effects logit model (GLMM) revealed fixed effects of morpheme type, availability, and reliability on the accuracy of morpheme provision. There were no effects of lemma frequency. Experiment 2 successfully replicated these results and extended the investigation to explore phrasal formulaicity by manipulating the frequency of the four-word strings in which the morpheme was embedded. GLMMs replicated the effects of word-form availability and reliability and additionally revealed independentphrase-superiority effectswhere morphemes were better reproduced in contexts of higher string-frequency. Taken together, these findings demonstrate that morpheme acquisition reflects the distributional properties of learners’ experience and the mappings therein between lexis, morphology, phraseology, and semantics. These conclusions support an emergentist view of the statistical symbolic learning of morphology where language acquisition involves the satisfaction of competing constraints across multiple grain-sizes of units. (shrink)

In this study we use a computational model of language learning called model of syntax acquisition in children (MOSAIC) to investigate the extent to which the optional infinitive (OI) phenomenon in Dutch and English can be explained in terms of a resource-limited distributional analysis of Dutch and English child-directed speech. The results show that the same version of MOSAIC is able to simulate changes in the pattern of finiteness marking in 2 children learning Dutch and 2 children learning English as (...) the average length of their utterances increases. These results suggest that it is possible to explain the key features of the OI phenomenon in both Dutch and English in terms of the interaction between an utterance-final bias in learning and the distributional characteristics of child-directed speech in the 2 languages. They also show how computational modeling techniques can be used to investigate the extent to which cross-linguistic similarities in the developmental data can be explained in terms of common processing constraints as opposed to innate knowledge of universal grammar. (shrink)

This article provides an overview of a probabilistic constraints framework for thinking about language acquisition and processing. The generative approach attempts to characterize knowledge of language (i.e., competence grammar) and then asks how this knowledge is acquired and used. Our approach is performance oriented: the goal is to explain how people comprehend and produce utterances and how children acquire this skill. Use of language involves exploiting multiple probabilistic constraints over various types of linguistic and nonlinguistic information. Acquisition is the process (...) of accumulating this information, which begins in infancy. The constraint satisfaction processes that are central to language use are the same as the bootstrapping processes that provide entry to language for the child. Framing questions about acquisition in terms of models of adult performance unifies the two topics under a set of common principles and has important consequences for arguments concerning language learnability. (shrink)

The child directed speech of twelve English‐speaking motherswas analyzed in terms of utterance‐level constructions. First, the mothers' utterances were categorized in terms of general constructional categories such as Wh‐questions, copulas and transitives. Second, mothers' utterances within these categories were further specified in terms of the initial words that framed the utterance, item‐based phrases such as Are you …, I'll …, It's …, Let's …, What did … The findings were: (i) overall, only about 15% of all maternal utterances had SVO (...) form (most were questions, imperatives, copulas, and fragments); (ii) 51% of all maternal utterances began with one of 52 item‐based phrases, mostly consisting of two words or morphemes (45% began with one of just 17 words); and (iii) children used many of these same item‐based phrases, in some cases at a rate that correlated highly with their own mother's frequency of use. We suggest that analyses of adult–child linguistic interaction should take into account not just general constructional categories, but also the item‐based constructions that adults and children use and the frequency with which they use them. (shrink)

Research in anthropology has shown that kin terminologies have a complex combinatorial structure and vary systematically across cultures. This article argues that universals and variation in kin terminology result from the interaction of (1) an innate conceptual structure of kinship, homologous with conceptual structure in other domains, and (2) principles of optimal, “grammatical” communication active in language in general. Kin terms from two languages, English and Seneca, show how terminologies that look very different on the surface may result from variation (...) in the rankings of a universal set of constraints. Constraints on kin terms form a system: some are concerned with absolute features of kin (sex), others with the position (distance and direction) of kin in “kinship space,” others with groups and group boundaries (matrilines, patrilines, generations, etc.). Also, kin terms sometimes extend indefinitely via recursion, and recursion in kin terminology has parallels with recursion in other areas of language. Thus the study of kinship sheds light on two areas of cognition, and their phylogeny. The conceptual structure of kinship seems to borrow its organization from the conceptual structure of space, while being specialized for representing genealogy. And the grammar of kinship looks like the product of an evolved grammar faculty, opportunistically active across traditional domains of semantics, syntax, and phonology. Grammar is best understood as an offshoot of a uniquely human capacity for playing coordination games. (shrink)

We examined whether variations in contextual diversity, spacing, and retrieval practice influenced how well adults learned new words from reading experience. Eye movements were recorded as adults read novel words embedded in sentences. In the learning phase, unfamiliar words were presented either in the same sentence repeated four times (same context) or in four different sentences (diverse context). Spacing was manipulated by presenting the sentences under distributed or non‐distributed practice. After learning, half of the participants were asked to retrieve the (...) new words, and half had an extra exposure to the new words. Although words experienced in diverse contexts were acquired more slowly during learning, they enjoyed a greater benefit of learning at immediate posttest. Distributed practice also slowed learning, but no benefit was observed at posttest. Although participants who had an extra exposure showed the greatest learning benefit overall, learning also benefited from retrieval opportunity, when words were experienced in diverse contexts. These findings demonstrate that variation in the content and structure of the learning environment impacts on word learning via reading. (shrink)

It is a reasonable assumption that universal properties of natural languages are not accidental. They occur either because they are underwritten by genetic code, because they assist in language processing or language learning, or due to some combination of the two. In this paper we investigate one such language universal: the suffixing preference across the world’s languages, whereby inflections tend to be added to the end of words. A corpus analysis of child‐directed speech in English found that suffixes were more (...) accurate at cuing the grammatical category of the root word than were prefixes. An artificial language experiment found that there was a learning advantage for suffixes over prefixes in terms of grammatical categorization within an artificial language. The results are consistent with an account of language universals that originate in general purpose learning mechanisms. (shrink)

Jones introduces an intricate machinery of kin classification that overcomes limitations of previous accounts. I question whether such a machinery is plausible. Because individuals never need to learn the entire spectrum of kin terminology, they could rely on data-driven learning. The complexity of Jones's machinery for kin classification casts doubt on the existence of innate structures that cover the complete linguistic domain.

According to the transitive dynamics model, people can construct causal structures by linking together configurations of force. The predictions of the model were tested in two experiments in which participants generated new causal relationships by chaining together two (Experiment 1) or three (Experiment 2) causal relations. The predictions of the transitive dynamics model were compared against those of Goldvarg and Johnson-Laird’s model theory (Goldvarg & Johnson- Laird, 2001). The transitive dynamics model consistently predicted the overall causal relationship drawn by participants (...) for both types of causal chains, and, when compared to the model theory, provided a better fit to the data. The results suggest that certain kinds of causal reasoning may depend on force dynamic—rather than on logical or purely statistical—representations. (shrink)

In agreement with some of the ideas expressed by Perruchet & Vinter (P&V), we believe that some phenomena hitherto attributed to processing may in fact reflect a fundamental distinction between direct and reflexive forms of consciousness. This dichotomy, developed by the phenomenological tradition, is substantiated by examples coming from experimental psychology and lesion neuropsychology.

��We describe a comprehensive framework for text un- derstanding, based on the representation of context. It is designed to serve as a representation of semantics for the full range of in- terpretive and inferential needs of general natural language pro- cessing. Its most distinctive feature is its uniform representation of the various simple and independent linguistic sources that play a role in determining meaning: lexical associations, syntactic re- strictions, case-role expectations, and most importantly, contextual effects. Compositional syntactic structure from a (...) shallow parsing is represented in a neural net-based associative memory, where it then interacts through a Bayesian network with semantic associa- tions and the context or “gist” of the passage carried forward from preceding sentences. Experiments with more than 2000 sentences in different languages are included. (shrink)

The term embodiment identifies a theory that meaning and semantics cannot be captured by abstract, logical systems, but are dependent on an agent’s experience derived from being situated in an environment. This theory has recently received a great deal of support in the cognitive science literature and is having significant impact in artificial intelligence. Memetics refers to the theory that knowledge and ideas can evolve more or less independently of their human-agent substrates. While humans provide the medium for this evolution, (...) memetics holds that ideas can be developed without human comprehension or deliberate interference. Both theories have profound implications for the study of language—its potential use by machines, its acquisition by children and of particular relevance to this special issue, its evolution. This article links the theory of memetics to the established literature on semantic space, then examines the extent to which these memetic mechanisms might account for language independently of embodiment. It then seeks to explain the evolution of language through uniquely human cognitive capacities which facilitate memetic evolution. (shrink)

We present a series of three analyses of young children's linguistic input to determine the distributional information it could plausibly offer to the process of grammatical category learning. Each analysis was conducted on four separate corpora from the CHILDES database (MacWhinney, 2000) of speech directed to children under 2;5. We showthat, in accord with other findings, a distributional analysis which categorizeswords based on their co‐occurrence patterns with surroundingwords successfully categorizes the majority of nouns and verbs. In Analyses 2 and 3, (...) we attempt to make our analyses more closely relevant to natural language acquisition by adopting more realistic assumptions about howyoung children represent their input. In Analysis 2, we limit the distributional context by imposing phrase structure boundaries, and find that categorization improves even beyond that obtained from less limited contexts. In Analysis 3, we reduce the representation of input elements which young children might not fully process and we find that categorization is not adversely affected: Although noun categorization is worse than in Analyses 1 and 2, it is still good; and verb categorization actually improves. Overall, successful categorization of nouns and verbs is maintained across all analyses. These results provide promising support for theories of grammatical category formation involving distributional analysis, as long as these analyses are combined with appropriate assumptions about the child learner's computational biases and capabilities. (shrink)

Some empirical evidence in the artificial language acquisition literature has been taken to suggest that statistical learning mechanisms are insufficient for extracting structural information from an artificial language. According to the more than one mechanism (MOM) hypothesis, at least two mechanisms are required in order to acquire language from speech: (a) a statistical mechanism for speech segmentation; and (b) an additional rule-following mechanism in order to induce grammatical regularities. In this article, we present a set of neural network studies demonstrating (...) that a single statistical mechanism can mimic the apparent discovery of structural regularities, beyond the segmentation of speech. We argue that our results undermine one argument for the MOM hypothesis. (shrink)

Computational models have shown that purely statistical knowledge about words’ linguistic contexts is sufficient to learn many properties of words, including syntactic and semantic category. For example, models can infer that “postman” and “mailman” are semantically similar because they have quantitatively similar patterns of association with other words. In contrast to these computational results, artificial language learning experiments suggest that distributional statistics alone do not facilitate learning of linguistic categories. However, experiments in this paradigm expose participants to entirely novel words, (...) whereas real language learners encounter input that contains some known words that are semantically organized. In three experiments, we show that the presence of familiar semantic reference points facilitates distributional learning and this effect crucially depends both on the presence of known words and the adherence of these known words to some semantic organization. (shrink)

A basic function of cognition is to detect regularities in sensory input to facilitate the prediction and recognition of future events. It has been proposed that these implicit expectations arise from an internal predictive coding model, based on knowledge acquired through processes such as statistical learning, but it is unclear how different types of statistical information affect listeners’ memory for auditory stimuli. We used a combination of behavioral and computational methods to investigate memory for non-linguistic auditory sequences. Participants repeatedly heard (...) tone sequences varying systematically in their information-theoretic properties. Expectedness ratings of tones were collected during three listening sessions, and a recognition memory test was given after each session. Information-theoretic measures of sequential predictability significantly influenced listeners’ expectedness ratings, and variations in these properties had a significant impact on memory performance. Predictable sequences yielded increasingly better memory performance with increasing exposure. Computational simulations using a probabilistic model of auditory expectation suggest that listeners dynamically formed a new, and increasingly accurate, implicit cognitive model of the information-theoretic structure of the sequences throughout the experimental session. (shrink)

This article provides an overview of a probabilistic constraints framework for thinking about language acquisition and processing. The generative approach attempts to characterize knowledge of language (i.e., competence grammar) and then asks how this knowledge is acquired and used. Our approach is performance oriented: the goal is to explain how people comprehend and produce utterances and how children acquire this skill. Use of language involves exploiting multiple probabilistic constraints over various types of linguistic and nonlinguistic information. Acquisition is the process (...) of accumulating this information, which begins in infancy. The constraint satisfaction processes that are central to language use are the same as the bootstrapping processes that provide entry to language for the child. Framing questions about acquisition in terms of models of adult performance unifies the two topics under a set of common principles and has important consequences for arguments concerning language learnability. (shrink)

This study investigates the child‐directed speech (CDS) of four Russian‐, six German, and six English‐speaking mothers to their 2‐year‐old children. Typologically Russian has considerably less restricted word order than either German or English, with German showing more word‐order variants than English. This could lead to the prediction that the lexical restrictiveness previously found in the initial strings of English CDS by Cameron‐Faulkner, Lieven, and Tomasello (2003) would not be found in Russian or German CDS. However, despite differences between the three (...) corpora that clearly derive from typological differences between the languages, the most significant finding of this study is a high degree of lexical restrictiveness at the beginnings of CDS utterances in all three languages. (shrink)

The acquisition of syntactic categories is a crucial step in the process of acquiring syntax. At this stage, before a full grammar is available, only surface cues are available to the learner. Previous computational models have demonstrated that local contexts are informative for syntactic categorization. However, local contexts are affected by sentence-level structure. In this paper, we add sentence type as an observed feature to a model of syntactic category acquisition, based on experimental evidence showing that pre-syntactic children are able (...) to distinguish sentence type using prosody and other cues. The model, a Bayesian Hidden Markov Model, allows for adding sentence type in a few different ways; we find that sentence type can aid syntactic category acquisition if it is used to characterize the differences in word order between sentence types. In these models, knowledge of sentence type permits similar gains to those found by extending the local context. (shrink)

While rules and exemplars are usually viewed as opposites, this paper argues that they form end points of the same distribution. By representing both rules and exemplars as (partial) trees, we can take into account the fluid middle ground between the two extremes. This insight is the starting point for a new theory of language learning that is based on the following idea: If a language learner does not know which phrase‐structure trees should be assigned to initial sentences, s/he allows (...) (implicitly) for all possible trees and lets linguistic experience decide which is the “best” tree for each sentence. The best tree is obtained by maximizing “structural analogy” between a sentence and previous sentences, which is formalized by the most probable shortest combination of subtrees from all trees of previous sentences. Corpus‐based experiments with this model on the Penn Treebank and the Childes database indicate that it can learn both exemplar‐based and rule‐based aspects of language, ranging from phrasal verbs to auxiliary fronting. By having learned the syntactic structures of sentences, we have also learned the grammar implicit in these structures, which can in turn be used to produce new sentences. We show that our model mimicks children’s language development from item‐based constructions to abstract constructions, and that the model can simulate some of the errors made by children in producing complex questions. (shrink)

There is substantial variation in language experience, yet there is surprising similarity in the language structure acquired. Constraints on language structure may be external modulators that result in this canalization of language structure, or else they may derive from the broader, communicative environment in which language is acquired. In this paper, the latter perspective is tested for its adequacy in explaining robustness of language learning to environmental variation. A computational model of word learning from cross-situational, multimodal information was constructed and (...) tested. Key to the model's robustness was the presence of multiple, individually unreliable information sources to support learning. This “degeneracy” in the language system has a detrimental effect on learning, compared to a noise-free environment, but has a critically important effect on acquisition of a canalized system that is resistant to environmental noise in communication. (shrink)