This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in (...) speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research. (shrink)

Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implicitly by its effects on processing rather than explicitly (as in a spatial representation). The current report develops a proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory. In this approach, hidden unit patterns are fed back to themselves: (...) the internal representations which develop thus reflect task demands in the context of prior internal states. A set of simulations is reported which range from relatively simple problems (temporal version of XOR) to discovering syntactic/semantic features for words. The networks are able to learn interesting internal representations which incorporate task demands with memory demands: indeed, in this approach the notion of memory is inextricably bound up with task processing. These representations reveal a rich structure, which allows them to be highly context‐dependent, while also expressing generalizations across classes of items. These representations suggest a method for representing lexical categories and the type/token distinction. (shrink)

Natural language is full of patterns that appear to fit with general linguistic rules but are ungrammatical. There has been much debate over how children acquire these “linguistic restrictions,” and whether innate language knowledge is needed. Recently, it has been shown that restrictions in language can be learned asymptotically via probabilistic inference using the minimum description length (MDL) principle. Here, we extend the MDL approach to give a simple and practical methodology for estimating how much linguistic data are required to (...) learn a particular linguistic restriction. Our method provides a new research tool, allowing arguments about natural language learnability to be made explicit and quantified for the first time. We apply this method to a range of classic puzzles in language acquisition. We find some linguistic rules appear easily statistically learnable from language experience only, whereas others appear to require additional learning mechanisms (e.g., additional cues or innate constraints). (shrink)

Recent research suggests that language evolution is a process of cultural change, in which linguistic structures are shaped through repeated cycles of learning and use by domain-general mechanisms. This paper draws out the implications of this viewpoint for understanding the problem of language acquisition, which is cast in a new, and much more tractable, form. In essence, the child faces a problem of induction, where the objective is to coordinate with others (C-induction), rather than to model the structure of the (...) natural world (N-induction). We argue that, of the two, C-induction is dramatically easier. More broadly, we argue that understanding the acquisition of any cultural form, whether linguistic or otherwise, during development, requires considering the corresponding question of how that cultural form arose through processes of cultural evolution. This perspective helps resolve the “logical” problem of language acquisition and has far-reaching implications for evolutionary psychology. (shrink)

Noam Chomsky's book on syntactic structures is a serious attempts on the part of a linguist to construct within the tradition of scientific theory-construction ...

The revised edition contains a new chapter which provides an elegant description of the semantics. The various classes of lambda calculus models are described in a uniform manner. Some didactical improvements have been made to this edition. An example of a simple model is given and then the general theory (of categorical models) is developed. Indications are given of those parts of the book which can be used to form a coherent course.

Many people have argued that the evolution of the human language faculty cannot be explained by Darwinian natural selection. Chomsky and Gould have suggested that language may have evolved as the by-product of selection for other abilities or as a consequence of as-yet unknown laws of growth and form. Others have argued that a biological specialization for grammar is incompatible with every tenet of Darwinian theory – that it shows no genetic variation, could not exist in any intermediate forms, confers (...) no selective advantage, and would require more evolutionary time and genomic space than is available. We examine these arguments and show that they depend on inaccurate assumptions about biology or language or both. Evolutionary theory offers clear criteria for when a trait should be attributed to natural selection: complex design for some function, and the absence of alternative processes capable of explaining such complexity. Human language meets these criteria: Grammar is a complex mechanism tailored to the transmission of propositional structures through a serial interface. Autonomous and arbitrary grammatical phenomena have been offered as counterexamples to the position that language is an adaptation, but this reasoning is unsound: Communication protocols depend on arbitrary conventions that are adaptive as long as they are shared. Consequently, language acquisition in the child should systematically differ from language evolution in the species, and attempts to analogize them are misleading. Reviewing other arguments and data, we conclude that there is every reason to believe that a specialization for grammar evolved by a conventional neo-Darwinian process. (shrink)

This systematic investigation of computation and mental phenomena by a noted psychologist and computer scientist argues that cognition is a form of computation, that the semantic contents of mental states are encoded in the same general way as computer representations are encoded. It is a rich and sustained investigation of the assumptions underlying the directions cognitive science research is taking. 1 The Explanatory Vocabulary of Cognition 2 The Explanatory Role of Representations 3 The Relevance of Computation 4 The Psychological Reality (...) of Programs: Strong Equivalence 5 Constraining Functional Architecture 6 The Bridge from Physical to Symbolic: Transduction 7 Functional Architecture and Analogue Processes 8 Mental Imagery and Functional Architecture 9 Epilogue: What is Cognitive Science the Science of? (shrink)

The book from which these sections are excerpted is concerned with the prospects for assimilating the study of human intelligence and its products to the natural sciences through the investigation of cognitive structures, understood as systems of rules and representations that can be regarded as These mental structui′es serve as the vehicles for the exercise of various capacities. They develop in the mind on the basis of an innate endowment that permits the growth of rich and highly articulated structures along (...) an intrinsically determined course under the triggering and partially shaping effect of experience, which fixes parameters in an intricate system of predetermined form. It is argued that the mind is modular in character, with a diversity of cognitive structures, each with its specific properties arid principles. Knowledge of language, of the behavior of objects, and much else crucially involves these mental structures, and is thus not characterizable in terms of capacities, dispositions, or practical abilities, nor is it necessarily grounded in experience in the standard sense of this term. (shrink)

In Rules and Representations, first published in 1980, Noam Chomsky lays out many of the concepts that have made his approach to linguistics and human cognition so instrumental to our understanding of language.Chomsky arrives at his well-known position that there is a universal grammar, structured in the human mind and common to all human languages. Based on Chomsky's 1978 Woodbridge Lectures, this edition contains revised versions of the lectures and two new essays.

The book from which these sections are excerpted is concerned with the prospects for assimilating the study of human intelligence and its products to the natural sciences through the investigation of cognitive structures, understood as systems of rules and representations that can be regarded as “mental organs.” These mental structui′es serve as the vehicles for the exercise of various capacities. They develop in the mind on the basis of an innate endowment that permits the growth of rich and highly articulated (...) structures along an intrinsically determined course under the triggering and partially shaping effect of experience, which fixes parameters in an intricate system of predetermined form. It is argued that the mind is modular in character, with a diversity of cognitive structures, each with its specific properties arid principles. Knowledge of language, of the behavior of objects, and much else crucially involves these mental structures, and is thus not characterizable in terms of capacities, dispositions, or practical abilities, nor is it necessarily grounded in experience in the standard sense of this term.Various types of knowledge and modes of knowledge acquisition are discussed in these terms. Some of the properties of the language faculty are investigated. The basic cognitive relation is “knowing a grammar”; knowledge of language is derivative and, correspondingly, raises further problems. Language as commonly understood is not a unitary phenomenon but involves a number of interacting systems: the “computational” system of grammar, which provides the representations of sound and meaning that permit a rich range of expressive potential, is distinct from a conceptual system with its own properties; knowledge of language must be distinguished from knowledge of how to use a language; and the various systems that enter into the knowledge and use of language must be further analyzed into their specific subcomponents. (shrink)

This collection of articles and associated discussion papers focuses on a problem that has attracted increasing attention from linguists and psychologists throughout the world during the past several years. Reduced to essentials, the problem is that of discovering the character of the mental capacities that make it possible for human beings to attain knowledge of their language on the basis of fragmentary and haphazard early linguistic experience. A fundamental assumption running through all of these contributions is that people possess strong (...) innate predispositions that are critical for success in this task. (shrink)

Many people have argued that the evolution of the human language faculty cannot be explained by Darwinian natural selection. Chomsky and Gould have suggested that language may have evolved as the by-product of selection for other abilities or as a consequence of as-yet unknown laws of growth and form. Others have argued that a biological specialization for grammar is incompatible with every tenet of Darwinian theory – that it shows no genetic variation, could not exist in any intermediate forms, confers (...) no selective advantage, and would require more evolutionary time and genomic space than is available. We examine these arguments and show that they depend on inaccurate assumptions about biology or language or both. Evolutionary theory offers clear criteria for when a trait should be attributed to natural selection: complex design for some function, and the absence of alternative processes capable of explaining such complexity. Human language meets these criteria: Grammar is a complex mechanism tailored to the transmission of propositional structures through a serial interface. Autonomous and arbitrary grammatical phenomena have been offered as counterexamples to the position that language is an adaptation, but this reasoning is unsound: Communication protocols depend on arbitrary conventions that are adaptive as long as they are shared. Consequently, language acquisition in the child should systematically differ from language evolution in the species, and attempts to analogize them are misleading. Reviewing other arguments and data, we conclude that there is every reason to believe that a specialization for grammar evolved by a conventional neo-Darwinian process. (shrink)

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in (...) speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research. (shrink)

Learning theory has frequently been applied to language acquisition, but discussion has largely focused on information theoretic problems—in particular on the absence of direct negative evidence. Such arguments typically neglect the probabilistic nature of cognition and learning in general. We argue first that these arguments, and analyses based on them, suffer from a major flaw: they systematically conflate the hypothesis class and the learnable concept class. As a result, they do not allow one to draw significant conclusions about the learner. (...) Second, we claim that the real problem for language learning is the computational complexity of constructing a hypothesis from input data. Studying this problem allows for a more direct approach to the object of study—the language acquisition device—rather than the learnable class of languages, which is epiphenomenal and possibly hard to characterize. The learnability results informed by complexity studies are much more insightful. They strongly suggest that target grammars need to be objective, in the sense that the primitive elements of these grammars are based on objectively definable properties of the language itself. These considerations support the view that language acquisition proceeds primarily through data-driven learning of some form. (shrink)

A U-shaped curve in a cognitive-developmental trajectory refers to a three-step process: good performance followed by bad performance followed by good performance once again. U-shaped curves have been observed in a wide variety of cognitive-developmental and learning contexts. U-shaped learning seems to contradict the idea that learning is a monotonic, cumulative process and thus constitutes a challenge for competing theories of cognitive development and learning. U-shaped behavior in language learning (in particular in learning English past tense) has become a central (...) topic in the Cognitive Science debate about learning models. Antagonist models (e.g., connectionism versus nativism) are often judged on their ability of modeling or accounting for U-shaped behavior. The prior literature is mostly occupied with explaining how U-shaped behavior occurs. Instead, we are interested in the necessity of this kind of apparently inefficient strategy. We present and discuss a body of results in the abstract mathematical setting of (extensions of) Gold-style computational learning theory addressing a mathematically precise version of the following question: Are there learning tasks that require U-shaped behavior? All notions considered are learning in the limit from positive data. We present results about the necessity of U-shaped learning in classical models of learning as well as in models with bounds on the memory of the learner. The pattern emerges that, for parameterized, cognitively relevant learning criteria, beyond very few initial parameter values, U-shapes are necessary for full learning power! We discuss the possible relevance of the above results for the Cognitive Science debate about learning models as well as directions for future research. (shrink)

Debates concerning the types of representations that aid reading acquisition have often been influenced by the relationship between measures of early phonological awareness (the ability to process speech sounds) and later reading ability. Here, a complementary approach is explored, analyzing how the functional utility of different representational units, such as whole words, bodies (letters representing the vowel and final consonants of a syllable), and graphemes (letters representing a phoneme) may change as the number of words that can be read gradually (...) increases. Utility is measured by applying a Simplicity Principle to the problem of mapping from print to sound; that is, assuming that the “best” representational units for reading are those which allow the mapping from print to sounds to be encoded as efficiently as possible. Results indicate that when only a small number of words are read whole-word representations are most useful, whereas when many words can be read graphemic representations have the highest utility. (shrink)