This paper introduces a special issue of Cognitive Science initiated on the 25th anniversary of the publication of Parallel Distributed Processing (PDP), a two-volume work that introduced the use of neural network models as vehicles for understanding cognition. The collection surveys the core commitments of the PDP framework, the key issues the framework has addressed, and the debates the framework has spawned, and presents viewpoints on the current status of these issues. The articles focus on both historical roots and contemporary (...) developments in learning, optimality theory, perception, memory, language, conceptual knowledge, cognitive control, and consciousness. Here we consider the approach more generally, reviewing the original motivations, the resulting framework, and the central tenets of the underlying theory. We then evaluate the impact of PDP both on the field at large and within specific subdomains of cognitive science and consider the current role of PDP models within the broader landscape of contemporary theoretical frameworks in cognitive science. Looking to the future, we consider the implications for cognitive science of the recent success of machine learning systems called “deep networks”—systems that build on key ideas presented in the PDP volumes. (shrink)

Cross-situational word learning, like any statistical learning problem, involves tracking the regularities in the environment. However, the information that learners pick up from these regularities is dependent on their learning mechanism. This article investigates the role of one type of mechanism in statistical word learning: competition. Competitive mechanisms would allow learners to find the signal in noisy input and would help to explain the speed with which learners succeed in statistical learning tasks. Because cross-situational word learning provides information at multiple (...) scales—both within and across trials/situations—learners could implement competition at either or both of these scales. A series of four experiments demonstrate that cross-situational learning involves competition at both levels of scale, and that these mechanisms interact to support rapid learning. The impact of both of these mechanisms is considered from the perspective of a process-level understanding of cross-situational learning. (shrink)

In language learning, learners engage with their environment, incorporating cues from different sources. However, in lab‐based experiments, using artificial languages, many of the cues and features that are part of real‐world language learning are stripped away. In three experiments, we investigated the role of positive, negative, and mixed feedback on the gradual learning of language‐like statistical regularities within an active guessing game paradigm. In Experiment 1, participants received deterministic feedback (100%), whereas probabilistic feedback (i.e., 75% or 50%) was introduced in (...) Experiment 2. Finally, Experiment 3 explored the impact of mixed probabilistic feedback (33% positive, 33% negative, 33% no feedback). The results showed that cross‐situational learning of words was observed without feedback, but participants were able to learn structural regularities of the miniature language only when feedback was provided. Interestingly, the presence of positive feedback was particularly helpful for the learner, promoting more in‐depth learning of the artificial language. (shrink)

Infant language learners are faced with the difficult inductive problem of determining how new words map to novel or known objects in their environment. Bayesian inference models have been successful at using the sparse information available in natural child-directed speech to build candidate lexicons and infer speakers’ referential intentions. We begin by asking how a Bayesian model optimized for monolingual input generalizes to new monolingual or bilingual corpora and find that, especially in the case of the bilingual input, the model (...) shows a significant decrease in performance. In the next experiment, we propose the ME Model, a modified Bayesian model, which approximates infants’ mutual exclusivity bias to support the differential demands of monolingual and bilingual learning situations. The extended model is assessed using the same corpora of real child-directed speech, showing that its performance is more robust against varying input and less dependent than the Intentional Model on optimization of its parsimony parameter. We argue that both monolingual and bilingual demands on word learning are important considerations for a computational model, as they can yield significantly different results than when only one such context is considered. (shrink)

According to usage‐based approaches to language acquisition, linguistic knowledge is represented in the form of constructions—form‐meaning pairings—at multiple levels of abstraction and complexity. The emergence of syntactic knowledge is assumed to be a result of the gradual abstraction of lexically specific and item‐based linguistic knowledge. In this article, we explore how the gradual emergence of a network consisting of constructions at varying degrees of complexity can be modeled computationally. Linguistic knowledge is learned by observing natural language utterances in an ambiguous (...) context. To determine meanings of constructions starting from ambiguous contexts, we rely on the principle of cross‐situational learning. While this mechanism has been implemented in several computational models, these models typically focus on learning mappings between words and referents. In contrast, in our model, we show how cross‐situational learning can be applied consistently to learn correspondences between form and meaning beyond such simple correspondences. (shrink)

If human language must be squeezed through a narrow cognitive bottleneck, what are the implications for language processing, acquisition, change, and structure? In our target article, we suggested that the implications are far-reaching and form the basis of an integrated account of many apparently unconnected aspects of language and language processing, as well as suggesting revision of many existing theoretical accounts. With some exceptions, commentators were generally supportive both of the existence of the bottleneck and its potential implications. Many commentators (...) suggested additional theoretical and linguistic nuances and extensions, links with prior work, and relevant computational and neuroscientific considerations; some argued for related but distinct viewpoints; a few, though, felt traditional perspectives were being abandoned too readily. Our response attempts to build on the many suggestions raised by the commentators and to engage constructively with challenges to our approach. (shrink)

Identifying the referent of novel words is a complex process that young children do with relative ease. When given multiple objects along with a novel word, children select the most novel item, sometimes retaining the word‐referent link. Prior work is inconsistent, however, on the role of object novelty. Two experiments examine 18‐month‐old children's performance on referent selection and retention with novel and known words. The results reveal a pervasive novelty bias on referent selection with both known and novel names and, (...) across individual children, a negative correlation between attention to novelty and retention of new word‐referent links. A computational model examines possible sources of the bias, suggesting novelty supports in‐the‐moment behavior but not retention. Together, results suggest that when lexical knowledge is weak, attention to novelty drives behavior, but alone does not sustain learning. Importantly, the results demonstrate that word learning may be driven, in part, by low‐level perceptual processes. (shrink)

Previous research on associative learning has uncovered detailed aspects of the process, including what types of things are learned, how they are learned, and where in the brain such learning occurs. However, perceptual processes, such as stimulus recognition and identification, take time to unfold. Previous studies of learning have not addressed when, during the course of these dynamic recognition processes, learned representations are formed and updated. If learned representations are formed and updated while recognition is ongoing, the result of learning (...) may incorporate spurious, partial information. For example, during word recognition, words take time to be identified, and competing words are often active in parallel. If learning proceeds before this competition resolves, representations may be influenced by the preliminary activations present at the time of learning. In three experiments using word learning as a model domain, we provide evidence that learning reflects the ongoing dynamics of auditory and visual processing during a learning event. These results show that learning can occur before stimulus recognition processes are complete; learning does not wait for ongoing perceptual processing to complete. (shrink)

Evolutionary developmental systems theory stresses that selection pressures operate on entire developmental systems rather than just genes. This study extends this approach to language evolution, arguing that selection pressure may operate on two quasi-independent timescales. First, children clearly must acquire language successfully and evolution must equip them with the tools to do so. Second, while this is developing, they must also communicate with others in the moment using partially developed knowledge. These pressures may require different solutions, and their combination may (...) underlie the evolution of complex mechanisms for language development and processing. I present two case studies to illustrate how the demands of both real-time communication and language acquisition may be subtly different. The first case study examines infant-directed speech. A recent view is that IDS underwent cultural to statistical learning mechanisms that infants use to acquire the speech categories of their language. However, recent data suggest is it may not have evolved to enhance development, but rather to serve a more real-time communicative function. The second case study examines the argument for seemingly specialized mechanisms for learning word meanings. Both behavioral and computational work suggest that learning may be much slower and served by general-purpose mechanisms like associative learning. Fast-mapping, then, may be a real-time process meant to serve immediate communication, not learning, by augmenting incomplete vocabulary knowledge with constraints from the current context. Together, these studies suggest that evolutionary accounts consider selection pressure arising from both real-time communicative demands and from the need for accurate language development. (shrink)

Theories of cognitive development must address both the issue of how children bring their knowledge to bear on behavior in‐the‐moment, and how knowledge changes over time. We argue that seeking answers to these questions requires an appreciation of the dynamic nature of the developing system in its full, reciprocal complexity. We illustrate this dynamic complexity with results from two lines of research on early word learning. The first demonstrates how the child's active engagement with objects and people supports referent selection (...) via memories for what objects were previously seen in a cued location. The second set of results highlights changes in the role of novelty and attentional processes in referent selection and retention as children's knowledge of words and objects grows. Together this work suggests that understanding systems for perception, action, attention, and memory, and their complex interaction, is critical to understand word learning. We review recent literature that highlights the complex interactions between these processes in cognitive development and point to critical issues for future work. (shrink)

Yang returns to the problem of referential ambiguity, addressed in the opening paper by Gleitman and Trueswell. Using a computational approach, he argues that “big data” approaches to resolving referential ambiguity are destined to fail, because of the inevitable computational explosion needed to keep track of contextual associations present when a word is uttered. Yang tests several computational models, two of which depend on one‐trial learning, as described in Gleitman and Trueswell’s paper. He concludes that such models outperform cross‐situational learning (...) models, thereby formalizing, reinforcing, and advancing the conclusions offered by Gleitman and Trueswell. (shrink)

Many animals can be trained to perform novel tasks. People, too, can be trained, but sometime in early childhood people transition from being trainable to something qualitatively more powerful—being programmable. We argue that such programmability constitutes a leap in the way that organisms learn, interact, and transmit knowledge, and that what facilitates or enables this programmability is the learning and use of language. We then examine how language programs the mind and argue that it does so through the manipulation of (...) embodied, sensorimotor representations. The role language plays in controlling mental representations offers important insights for understanding its origin and evolution. (shrink)

An important question in early bilingual first language acquisition concerns the development of lexical-semantic associations within and across two languages. The present study investigates the earliest emergence of lexical-semantic priming at 18 and 24 months in Spanish-English bilinguals (N= 32) and its relation to vocabulary knowledge within and across languages. Results indicate a remarkably similar pattern of development between monolingual and bilingual children, such that lexical-semantic development begins at 18 months and strengthens by 24 months. Further, measures of cross-language lexical (...) knowledge are stronger predictors of children’s lexical-semantic processing skill than measures that capture single-language knowledge only. This suggests that children make use of both languages when processing semantic information. Together these findings inform the understanding of the relation between lexical-semantic breadth and organization in the context of dual language learners in early development. (shrink)

Given that there is referential uncertainty (noise) when learning words, to what extent can forgetting filter some of that noise out, and be an aid to learning? Using a Cross Situational Learning model we find a U-shaped function of errors indicative of a “Goldilocks” zone of forgetting: an optimum store-loss ratio that is neither too aggressive nor too weak, but just the right amount to produce better learning outcomes. Forgetting acts as a high-pass filter that actively deletes (part of) the (...) referential ambiguity noise, retains intended referents, and effectively amplifies the signal. The model achieves this performance without incorporating any specific cognitive biases of the type proposed in the constraints and principles account, and without any prescribed developmental changes in the underlying learning mechanism. Instead we interpret the model performance as more of a by-product of exposure to input, where the associative strengths in the lexicon grow as a function of linguistic experience in combination with memory limitations. The result adds a mechanistic explanation for the experimental evidence on spaced learning and, more generally, advocates integrating domain-general aspects of cognition, such as memory, into the language acquisition process. (shrink)

Cognitive Science, Volume 46, Issue 4, April 2022.

Intuitively, the accuracy of initial word-referent mappings should be positively correlated with the outcome of learning. Yet recent evidence suggests an inverse effect of initial accuracy in adults, whereby greater accuracy of initial mappings is associated with poorer outcomes in a cross-situational learning task. Here, we examine the impact of initial accuracy on 4-year-olds, 10-year-olds, and adults. For half of the participants most word-referent mappings were initially correct and for the other half most mappings were initially incorrect. Initial accuracy was (...) positively related to learning outcomes in 4-year-olds, had no effect on 10-year-olds' learning, and was inversely related to learning outcomes in adults. Examination of item learning patterns revealed item interdependence for adults and 4-year-olds but not 10-year-olds. These findings point to a qualitative change in language learning processes over development. (shrink)

There is substantial variation in language experience, yet there is surprising similarity in the language structure acquired. Constraints on language structure may be external modulators that result in this canalization of language structure, or else they may derive from the broader, communicative environment in which language is acquired. In this paper, the latter perspective is tested for its adequacy in explaining robustness of language learning to environmental variation. A computational model of word learning from cross-situational, multimodal information was constructed and (...) tested. Key to the model's robustness was the presence of multiple, individually unreliable information sources to support learning. This “degeneracy” in the language system has a detrimental effect on learning, compared to a noise-free environment, but has a critically important effect on acquisition of a canalized system that is resistant to environmental noise in communication. (shrink)

One of the great challenges in word learning is that words are typically uttered in a context with many potential referents. Children's tendency to associate novel words with novel referents, which is taken to reflect a mutual exclusivity (ME) bias, forms a useful disambiguation mechanism. We study semantic learning in pragmatic agents—combining the Rational Speech Act model with gradient‐based learning—and explore the conditions under which such agents show an ME bias. This approach provides a framework for investigating a pragmatic account (...) of the ME bias in humans but also for building artificial agents that display an ME bias. A series of analyses demonstrates striking parallels between our model and human word learning regarding several aspects relevant to the ME bias phenomenon: online inference, long‐term learning, and developmental effects. By testing different implementations, we find that two components, pragmatic online inference and incremental collection of evidence for one‐to‐one correspondences between words and referents, play an important role in modeling the developmental trajectory of the ME bias. Finally, we outline an extension of our model to a deep neural network architecture that can process more naturalistic visual and linguistic input. Until now, in contrast to children, deep neural networks have needed indirect access to (supposed to be novel) test inputs during training to display an ME bias. Our model is the first one to do so without using this manipulation. (shrink)

Theories of cognitive development must address both the issue of how children bring their knowledge to bear on behavior in-the-moment, and how knowledge changes over time. We argue that seeking answers to these questions requires an appreciation of the dynamic nature of the developing system in its full, reciprocal complexity. We illustrate this dynamic complexity with results from two lines of research on early word learning. The first demonstrates how the child's active engagement with objects and people supports referent selection (...) via memories for what objects were previously seen in a cued location. The second set of results highlights changes in the role of novelty and attentional processes in referent selection and retention as children's knowledge of words and objects grows. Together this work suggests that understanding systems for perception, action, attention, and memory, and their complex interaction, is critical to understand word learning. We review recent literature that highlights the complex interactions between these processes in cognitive development and point to critical issues for future work. (shrink)

Variability is prevalent in early language acquisition, but, whether it supports or hinders learning is unclear; while target variability has been shown to facilitate word learning, variability in competitor items has been shown to make the task harder. Here, we tested whether background variability could boost learning in a referent selection task. Two groups of 2-year-old children saw arrays of one novel and two known objects on a screen, and they heard a novel or known label. Stimuli were identical across (...) conditions, with the exception that in the constant color condition objects appeared on a uniform white background, and in the variable color condition backgrounds were different, uniform colors. At test, only children in the variable condition showed evidence of retaining label-object associations. These data support findings from the adult memory literature, which suggest that variability supports learning by decontextualizing representations. We argue that these data are consistent with dynamic systems accounts of learning in which low-level entropy adds sufficient noise to the developmental system to precipitate a change in behavior. (shrink)

It is well-established that toddlers can correctly select a novel referent from an ambiguous array in response to a novel label. There is also a growing consensus that robust word learning requires repeated label-object encounters. However, the effect of the context in which a novel object is encountered is less well-understood. We present two embodied neural network replications of recent empirical tasks, which demonstrated that the context in which a target object is encountered is fundamental to referent selection and word (...) learning. Our model offers an explicit account of the bottom-up associative and embodied mechanisms which could support children’s early word learning and emphasises the importance of viewing behaviour as the interaction of learning at multiple timescales. (shrink)