This article provides an overview of a probabilistic constraints framework for thinking about language acquisition and processing. The generative approach attempts to characterize knowledge of language (i.e., competence grammar) and then asks how this knowledge is acquired and used. Our approach is performance oriented: the goal is to explain how people comprehend and produce utterances and how children acquire this skill. Use of language involves exploiting multiple probabilistic constraints over various types of linguistic and nonlinguistic information. Acquisition is the process (...) of accumulating this information, which begins in infancy. The constraint satisfaction processes that are central to language use are the same as the bootstrapping processes that provide entry to language for the child. Framing questions about acquisition in terms of models of adult performance unifies the two topics under a set of common principles and has important consequences for arguments concerning language learnability. (shrink)

The acquisition of English noun and verb morphology is modeled using a single-system connectionist network. The network is trained to produce the plurals and past tense forms of a large corpus of monosyllabic English nouns and verbs. The developmental trajectory of network performance is analyzed in detail and is shown to mimic a number of important features of the acquisition of English noun and verb morphology in young children. These include an initial error-free period of performance on both nouns and (...) verbs followed by a period of intermittent over-regularization of irregular nouns and verbs. Errors in the model show evidence of phonological conditioning and frequency effects. Furthermore, the network demonstrates a strong tendency to regularize denominal verbs and deverbal nouns and masters the principles of voicing assimilation. Despite their incorporation into a single-system network, nouns and verbs exhibit some important differences in their profiles of acquisition. Most importantly, noun inflections are acquired earlier than verb inflections. The simulations generate several empirical predictions that can be used to evaluate further the suitability of this type of cognitive architecture in the domain of inflectional morphology. (shrink)

Most words in natural languages are polysemous; that is, they have related but different meanings in different contexts. This one‐to‐many mapping of form to meaning presents a challenge to understanding how word meanings are learned, represented, and processed. Previous work has focused on solutions in which multiple static semantic representations are linked to a single word form, which fails to capture important generalizations about how polysemous words are used; in particular, the graded nature of polysemous senses, and the flexibility and (...) regularity of polysemy use. We provide a novel view of how polysemous words are represented and processed, focusing on how meaning is modulated by context. Our theory is implemented within a recurrent neural network that learns distributional information through exposure to a large and representative corpus of English. Clusters of meaning emerge from how the model processes individual word forms. In keeping with distributional theories of semantics, we suggest word meanings are generalized from contexts of different word tokens, with polysemy emerging as multiple clusters of contextually modulated meanings. We validate our results against a human‐annotated corpus of polysemy focusing on the gradedness, flexibility, and regularity of polysemous sense individuation, as well as behavioral findings of offline sense relatedness ratings and online sentence processing. The results provide novel insights into how polysemy emerges from contextual processing of word meaning from both a theoretical and computational point of view. (shrink)

This article describes an approach to connectionist language research that relies on the development of grammar formalisms rather than computer models. From formulations of the fundamental theoretical commitments of connectionism and of generative grammar, it is argued that these two paradigms are mutually compatible. Integrating the basic assumptions of the paradigms results in formal theories of grammar that centrally incorporate a certain degree of connectionist computation. Two such grammar formalisms—Harmonic Grammar and Optimality Theory —are briefly introduced to illustrate grammar‐based approaches (...) to connectionist language research. The strengths and weaknesses of grammar‐based research and more traditional model‐based research are argued to be complementary, suggesting a significant role for both strategies in the spectrum of connectionist language research. (shrink)

We suggest that the theory of dynamical systems provides a revealing general framework for modeling the representations and mechanism underlying syntactic processing. We show how a particular dynamical model, the Visitation Set Gravitation model of Tabor, Juliano, and Tanenhaus (1997), develops syntactic representations and models a set of contingent frequency effects in parsing that are problematic for other models. We also present new simulations showing how the model accounts for semantic effects in parsing, and propose a new account of the (...) distinction between syntactic and semantic incongruity. The results show how symbolic structures useful in parsing arise as emergent properties of connectionist dynamical systems. (shrink)

How people learn chunks or associations between adjacent items in sequences was modelled. Two previously successful models of how people learn artificial grammars were contrasted: the CCN, a network version of the competitive chunker of Servan‐Schreiber and Anderson [J. Exp. Psychol.: Learn. Mem. Cogn. 16 (1990) 592], which produces local and compositionally‐structured chunk representations acquired incrementally; and the simple recurrent network (SRN) of Elman [Cogn. Sci. 14 (1990) 179], which acquires distributed representations through error correction. The models' susceptibility to two (...) types of interference was determined: prediction conflicts, in which a given letter can predict two other letters that appear next with an unequal frequency; and retroactive interference, in which the prediction made by a letter changes in the second half of training. The predictions of the models were determined by exploring parameter space and seeing howdensely different regions of the space of possible experimental outcomes were populated by model outcomes. For both types of interference, human data fell squarely in regions characteristic of CCN performance but not characteristic of SRN performance. (shrink)

A critical property of the perception of spoken words is the transient ambiguity of the speech signal. In localist models of speech perception this ambiguity is captured by allowing the parallel activation of multiple lexical representations. This paper examines how a distributed model of speech perception can accommodate this property. Statistical analyses of vector spaces show that coactivation of multiple distributed representations is inherently noisy, and depends on parameters such as sparseness and dimensionality. Furthermore, the characteristics of coactivation vary considerably, (...) depending on the organization of distributed representations within the mental lexicon. This view of lexical access is supported by analyses of phonological and semantic word representations, which provide an explanation of a recent set of experiments on coactivation in speech perception (Gaskell & Marslen–Wilson, 1999). (shrink)

A connectionist approach to word reading, based on the principles of distributed representation, graded learning of statistical structure, and interactivity in processing, has led to the development of explicit computational models which account for a wide range of data on normal skilled reading and on patterns of reading impairment due to brain damage. There have, however, been recent empirical challenges to these models, and the approach in general, relating to the influence of orthographic length on the naming latencies of both (...) normal and dyslexic readers. The current work presents a simulation which generates sequential phonological output in response to written input, and which can refixate the input when encountering difficulty. The normal model reads both words and nonwords accurately, and exhibits an effect of orthographic length and a frequency‐by‐consistency interaction in its naming latencies. When subject to peripheral damage, the model exhibits an increased length effect that interacts with word frequency, characteristic of letter‐by‐letter reading in pure alexia. Although the model is far from a fully adequate account of all the relevant phenomena, it suggests how connectionist models may be extended to provide deeper insight into sequential processes in reading. (shrink)

Children learn their native language by exposure to their linguistic and communicative environment, but apparently without requiring that their mistakes be corrected. Such learning from “positive evidence” has been viewed as raising “logical” problems for language acquisition. In particular, without correction, how is the child to recover from conjecturing an over-general grammar, which will be consistent with any sentence that the child hears? There have been many proposals concerning how this “logical problem” can be dissolved. In this study, we review (...) recent formal results showing that the learner has sufficient data to learn successfully from positive evidence, if it favors the simplest encoding of the linguistic input. Results include the learnability of linguistic prediction, grammaticality judgments, language production, and form-meaning mappings. The simplicity approach can also be “scaled down” to analyze the learnability of specific linguistic constructions, and it is amenable to empirical testing as a framework for describing human language acquisition. (shrink)

Human and automated tutors attempt to choose pedagogical activities that will maximize student learning, informed by their estimates of the student's current knowledge. There has been substantial research on tracking and modeling student learning, but significantly less attention on how to plan teaching actions and how the assumed student model impacts the resulting plans. We frame the problem of optimally selecting teaching actions using a decision-theoretic approach and show how to formulate teaching as a partially observable Markov decision process planning (...) problem. This framework makes it possible to explore how different assumptions about student learning and behavior should affect the selection of teaching actions. We consider how to apply this framework to concept learning problems, and we present approximate methods for finding optimal teaching actions, given the large state and action spaces that arise in teaching. Through simulations and behavioral experiments, we explore the consequences of choosing teacher actions under different assumed student models. In two concept-learning tasks, we show that this technique can accelerate learning relative to baseline performance. (shrink)

Some contemporary philosophers suggest that we know just by introspection that folk psychological states exist. However, such an "armchair refutation" of eliminativism seems too easy. I first attack two strategems, inspired by Descartes, on how such a refutation might proceed. However, I concede that the Cartesian intuition that we have direct knowledge of representational states is very powerful. The rest of this paper then offers an error theory of how that intuition might really be mistaken. The idea is that introspection (...) does not detect any folk psychological states, but rather detects neural states that dispose us to certain sorts of linguistic behavior. Briefly, rather than detecting a mental appearance of the state-of-affairs that I am sitting near the fire, introspection detects a state disposing me to assert "It appears to me that I am sitting by the fire." The nature of this linguistic-dispositional state is then given a connectionist underpinning, where inputs from introspection lead to such assertive behavior. The Cartesian certainty we feel in self-ascribing such appearances is also explained. Even though I do not endorse such an error theory outright, I suggest that its tenability is enough to undercut introspection arguments against eliminativism. (shrink)

In this article, I survey the integrated connectionist/symbolic (ICS) cognitive architecture in which higher cognition must be formally characterized on two levels of description. At the microlevel, parallel distributed processing (PDP) characterizes mental processing; this PDP system has special organization in virtue of which it can be characterized at the macrolevel as a kind of symbolic computational system. The symbolic system inherits certain properties from its PDP substrate; the symbolic functions computed constitute optimization of a well-formedness measure called Harmony. The (...) most important outgrowth of the ICS research program is optimality theory (Prince & Smolensky, 1993/2004), an optimization based grammatical theory that provides a formal theory of cross-linguistic typology. Linguistically, Harmony maximization corresponds to minimization of markedness or structural ill-formedness. Cognitive explanation in ICS requires the collaboration of symbolic and connectionist principles. ICS is developed in detail in Smolensky and Legendre (2006a); this article is a précis of and guide to those volumes. (shrink)

Some empirical evidence in the artificial language acquisition literature has been taken to suggest that statistical learning mechanisms are insufficient for extracting structural information from an artificial language. According to the more than one mechanism (MOM) hypothesis, at least two mechanisms are required in order to acquire language from speech: (a) a statistical mechanism for speech segmentation; and (b) an additional rule-following mechanism in order to induce grammatical regularities. In this article, we present a set of neural network studies demonstrating (...) that a single statistical mechanism can mimic the apparent discovery of structural regularities, beyond the segmentation of speech. We argue that our results undermine one argument for the MOM hypothesis. (shrink)

Theory-ladenness of perception and cognition is pervasive and variable. Emerging maturationally natural perception and cognition, which are on-line, fast, automatic, unconscious, and, by virtue of their selectivity, theoretical in import, if not in form, define normal development. They contrast with off-line, slow, deliberate, conscious perceptual and cognitive judgments that reflective theories, including scientific ones, inform. Although culture tunes MN systems, their emergence and operation do not rely on culturally distinctive inputs. The sciences advance radically counter-intuitive representations that depart drastically from (...) MN systems’ deliverances. Extensive experience with RCI scientific theories can result in a practiced naturalness with their perceptual and cognitive consequences; nevertheless, automatic MN verdicts persistently intrude. Fodor suggests that the uniformity of the biases MN systems introduce can serve as a theory-neutral means for adjudicating scientific disputes. Findings about vision challenge Fodor’s proposal for circumventing problems that MN theory-ladenness presents. These considerations indicate that RCI scientific ideas are difficult to learn, master, and deploy; consequently, the corrective import of science’s social and institutional arrangements plays a critical role in its epistemic stature. (shrink)

O artigo propõe um balanço atualizado da abordagem computacional da mente, minudenciando aspectos conceituais e críticos. O balanço é pautado por três afirmações ‒ α) A mente humana é um sistema computacional; β) A mente humana pode ser descrita como um sistema computacional; γ) Sistemas computacionais precisam de conteúdo representacional ‒, a partir das quais mostro que o computacionalismo clássico se articula em termos de α∧γ e que as vertentes contemporâneas são melhor caracterizadas em termos de α∧~γ ou β∧~γ. Por (...) fim, após analisar uma série de objeções, argumentamos que o computacionalismo do século XXI é um programa de pesquisa filosoficamente relevante e que os críticos da abordagem computacional da mente incorrem em anacronismo quando se limitam a criticar as vertentes clássicas. (shrink)

Scholars studying the origins and evolution of language are also interested in the general issue of the evolution of cognition. Language is not an isolated capability of the individual, but has intrinsic relationships with many other behavioral, cognitive, and social abilities. By understanding the mechanisms underlying the evolution of linguistic abilities, it is possible to understand the evolution of cognitive abilities. Cognitivism, one of the current approaches in psychology and cognitive science, proposes that symbol systems capture mental phenomena, and attributes (...) cognitive validity to them. Therefore, in the same way that language is considered the prototype of cognitive abilities, a symbol system has become the prototype for studying language and cognitive systems. Symbol systems are advantageous as they are easily studied through computer simulation (a computer program is a symbol system itself), and this is why language is often studied using computational models. (shrink)

This paper describes different types of models for the evolution of communication and language. It uses the distinction between signals, symbols, and words for the analysis of evolutionary models of language. In particular, it show how evolutionary computation techniques, such as artificial life, can be used to study the emergence of syntax and symbols from simple communication signals. Initially, a computational model that evolves repertoires of isolated signals is presented. This study has simulated the emergence of signals for naming foods (...) in a population of foragers. This type of model studies communication systems based on simple signal-object associations. Subsequently, models that study the emergence of grounded symbols are discussed in general, including a detailed description of a work on the evolution of simple syntactic rules. This model focuses on the emergence of symbol-symbol relationships in evolved languages. Finally, computational models of syntax acquisition and evolution are discussed. These different types of computational models provide an operational definition of the signal/symbol/word distinction. The simulation and analysis of these types of models will help to understand the role of symbols and symbol acquisition in the origin of language. (shrink)