A growing number of children in the United States are exposed to multiple languages at home from birth. However, relatively little is known about the early process of word learning—how words are mapped to the referent in their child-centered learning experiences. The present study defined parental input operationally as the integrated and multimodal learning experiences as an infant engages with his/her parent in an interactive play session with objects. By using a head-mounted eye tracking device, we recorded visual scenes from (...) the infant’s point of view, along with the parent’s social input with respect to gaze, labeling, and actions of object handling. Fifty-one infants and toddlers from an English monolingual or a diverse bilingual household were recruited to observe the early multimodal learning experiences in an object play session. Despite that monolingual parents spoke more and labeled more frequently relative to bilingual parents, infants from both language groups benefit from a comparable amount of socially coordinated experiences where parents name the object while the object is looked at by the infant. Also, a sequential path analysis reveals multiple social coordinated pathways that facilitate infant object looking. Specifically, young children’s attention to the referent objects is directly influenced by parent’s object handling. These findings point to the new approach to early language input and how multimodal learning experiences are coordinated socially for young children growing up with monolingual and bilingual learning contexts. (shrink)

Ballard and Zhang offer a fascinating review of how computational models of human vision have evolved since David Marr proposed his Tri‐Level Hypothesis, with a focus on the refinement of algorithm descriptions over time.

Several evolutionary accounts of human social cognition posit that language has co-evolved with the sophisticated mindreading abilities of modern humans. It has also been argued that these mindreading abilities are the product of cultural, rather than biological, evolution. Taken together, these claims suggest that the evolution of language has played an important role in the cultural evolution of human social cognition. Here we present a new computational model which formalises the assumptions that underlie this hypothesis, in order to explore how (...) language and mindreading interact through cultural evolution. This model treats communicative behaviour as an interplay between the context in which communication occurs, an agent’s individual perspective on the world, and the agent’s lexicon. However, each agent’s perspective and lexicon are private mental representations, not directly observable to other agents. Learners are therefore confronted with the task of jointly inferring the lexicon and perspective of their cultural parent, based on their utterances in context. Simulation results show that given these assumptions, an informative lexicon evolves not just under a pressure to be successful at communicating, but also under a pressure for accurate perspective-inference. When such a lexicon evolves, agents become better at inferring others’ perspectives; not because their innate ability to learn about perspectives changes, but because sharing a language with others helps them to do so. (shrink)

In robotics research with language-based interaction, simplifications are made, such that a given event can be described in a unique manner, where there is a direct mapping between event representations and sentences that can describe these events. However, common experience tells us that the same physical event can be described in multiple ways, depending on the perspective of the speaker. The current research develops methods for representing events from multiple perspectives, and for choosing the perspective that will be used for (...) generating a linguistic construal, based on attentional processes in the system. The multiple perspectives are based on the principle that events can be considered in terms of the force driving the event, and the result obtained from the event, based on the theory of Gärdenfors. In addition, within these perspectives a further refinement can be made with respect to the agent, object, and recipient perspectives. We develop a system for generating appropriate construals of meaning, and demonstrate how this can be used in a realistic dialogic interaction between a behaving robot and a human interlocutor. (shrink)

This paper aims to conceptualize the phenomenology of attentional experience as ‘embodied attention.’ Current psychological research, in describing attentional experiences, tends to apply the so-called spotlight metaphor, according to which attention is characterized as the illumination of certain surrounding objects or events. In this framework, attention is not seen as involving our bodily attitudes or modifying the way we experience those objects and events. It is primarily conceived as a purely mental and volitional activity of the cognizing subject. Against this (...) view, the phenomenology of Maurice Merleau-Ponty shows that attention is a creative activity deeply linked with bodily movements. This paper clarifies and systematizes this view and brings it into dialogue with current empirical findings as well as with current theoretical research on embodied cognition. By doing this, I spell out three main claims about embodied attention: the transcendentalism of embodiment for attention, the bodily subjectivity of attention, and the creativity of embodied attention. (shrink)

Joint attention has been extensively studied in the developmental literature because of overwhelming evidence that the ability to socially coordinate visual attention to an object is essential to healthy developmental outcomes, including language learning. The goal of this study was to understand the complex system of sensory-motor behaviors that may underlie the establishment of joint attention between parents and toddlers. In an experimental task, parents and toddlers played together with multiple toys. We objectively measured joint attention—and the sensory-motor behaviors that (...) underlie it—using a dual head-mounted eye-tracking system and frame-by-frame coding of manual actions. By tracking the momentary visual fixations and hand actions of each participant, we precisely determined just how often they fixated on the same object at the same time, the visual behaviors that preceded joint attention and manual behaviors that preceded and co-occurred with joint attention. We found that multiple sequential sensory-motor patterns lead to joint attention. In addition, there are developmental changes in this multi-pathway system evidenced as variations in strength among multiple routes. We propose that coordinated visual attention between parents and toddlers is primarily a sensory-motor behavior. Skill in achieving coordinated visual attention in social settings—like skills in other sensory-motor domains—emerges from multiple pathways to the same functional end. (shrink)

Previous research shows that people can use the co-occurrence of words and objects in ambiguous situations (i.e., containing multiple words and objects) to learn word meanings during a brief passive training period (Yu & Smith, 2007). However, learners in the world are not completely passive but can affect how their environment is structured by moving their heads, eyes, and even objects. These actions can indicate attention to a language teacher, who may then be more likely to name the attended objects. (...) Using a novel active learning paradigm in which learners choose which four objects they would like to see named on each successive trial, this study asks whether active learning is superior to passive learning in a cross-situational word learning context. Finding that learners perform better in active learning, we investigate the strategies and discover that most learners use immediate repetition to disambiguate pairings. Unexpectedly, we find that learners who repeat only one pair per trial—an easy way to infer this pair—perform worse than those who repeat multiple pairs per trial. Using a working memory extension to an associative model of word learning with uncertainty and familiarity biases, we investigate individual differences that correlate with these assorted strategies. (shrink)

The aim of this article is to mount a challenge to gesture-first hypotheses about the evolution of language by identifying constraints on the emergence of symbol use. Current debates focus on a range of pre-conditions for the emergence of language, including co-operation and related mentalising capacities, imitation and tool use, episodic memory, and vocal physiology, but little specifically on the ability to learn and understand symbols. It is argued here that such a focus raises new questions about the plausibility of (...) gesture-first hypotheses, and so about the evolution of language in general. After a brief review of the methodology used in the article, it is argued that existing uses of gesture in hominid communities may have prohibited the emergence of symbol use, rather than ‘bootstrapped’ symbolic capacities as is usually assumed, and that the vocal channel offers other advantages in both learning and using language. In this case, the vocal channel offers a more promising platform for the evolution of language than is often assumed. (shrink)

When a caregiver names objects dominating a child’s view, the association between object and name is unambiguous and children are more likely to learn the object’s name. Children also learn to name things other than solid objects, including nonsolid substances like applesauce. However, it is unknown how caregivers structure linguistic and exploratory experiences with nonsolids to support learning. In this exploratory study of caregivers and children we compare caregiver-child free-play with novel solid objects and novel nonsolid substances to identify the (...) linguistic and exploratory experiences associated with children’s word learning. We found systematic differences in interactions with novel objects, such that children performed more manual actions on solids than nonsolids and caregivers named solids more than nonsolids. Additionally, there was less synchrony between caregivers’ naming and children’s manual and visual exploration of nonsolids than solids. Consistent with prior work, we found that synchronous naming was associated with accurate recognition of solid object names. However, naming synchrony was not associated with recognition of nonsolid substance names or with generalization. Together these findings, though exploratory, suggest the coordination of caregiver-child play can shape what children remember about novel word-object associations for solid objects, but not nonsolid substances. (shrink)

The words in children's language learning environments are strongly predictive of cognitive development and school achievement. But how do we measure language environments and do so at the scale of the many words that children hear day in, day out? The quantity and quality of words in a child's input are typically measured in terms of total amount of talk and the lexical diversity in that talk. There are disagreements in the literature whether amount or diversity is the more critical (...) measure of the input. Here we analyze the properties of a large corpus of speech to children and simulate learning environments that differ in amount of talk per unit time, lexical diversity, and the contexts of talk. The central conclusion is that what researchers need to theoretically understand, measure, and change is not the total amount of words, or the diversity of words, but the function that relates total words to the diversity of words, and how that function changes across different contexts of talk. (shrink)

There are clear associations between the overall quantity of input children are exposed to and their vocabulary acquisition. However, by uncovering specific features of the input that matter, we can better understand the mechanisms involved in vocabulary learning. We examine whether exposure to wh-questions, a challenging quality of the communicative input, is associated with toddlers' vocabulary and later verbal reasoning skills in a sample of low-income, African-American fathers and their 24-month-old children. Dyads were videotaped in free play sessions at home. (...) Videotapes were transcribed and reliably coded for sheer quantity of fathers' input as well as the number of wh-questions fathers produce. Children's productive vocabulary was measured at 24 months using the McArthur Bates Communicative Development Inventory MCDI, and children's verbal reasoning skills were measured 1 year later using the Bayley Scales of Infant Development. Results indicate that the overall quantity of father talk did not relate to children's vocabulary or reasoning skills. However, fathers' use of wh-questions related to both vocabulary and reasoning outcomes. Children's responses to wh-questions were more frequent and more syntactically complex, measured using the mean length of utterance, than their responses to other questions. Thus, posing wh-questions to 2-year-olds is a challenging type of input, which elicits a verbal response from the child that likely helps build vocabulary and foster verbal reasoning abilities. (shrink)

Human development takes place in a social context. Two pervasive sources of social information are faces and hands. Here, we provide the first report of the visual frequency of faces and hands in the everyday scenes available to infants. These scenes were collected by having infants wear head cameras during unconstrained everyday activities. Our corpus of 143 hours of infant-perspective scenes, collected from 34 infants aged 1 month to 2 years, was sampled for analysis at 1/5 Hz. The major finding (...) from this corpus is that the faces and hands of social partners are not equally available throughout the first two years of life. Instead, there is an earlier period of dense face input and a later period of dense hand input. At all ages, hands in these scenes were primarily in contact with objects and the spatio-temporal co-occurrence of hands and faces was greater than expected by chance. The orderliness of the shift from faces to hands suggests a principled transition in the contents of visual experiences and is discussed in terms of the role of developmental gates on the timing and statistics of visual experiences. (shrink)

Recent laboratory experiments have shown that both infant and adult learners can acquire word‐referent mappings using cross‐situational statistics. The vast majority of the work on this topic has used unfamiliar objects presented on neutral backgrounds as the visual contexts for word learning. However, these laboratory contexts are much different than the real‐world contexts in which learning occurs. Thus, the feasibility of generalizing cross‐situational learning beyond the laboratory is in question. Adapting the Human Simulation Paradigm, we conducted a series of experiments (...) examining cross‐situational learning from children's egocentric videos captured during naturalistic play. Focusing on individually ambiguous naming moments that naturally occur during toy play, we asked how statistical learning unfolds in real time through accumulating cross‐situational statistics in naturalistic contexts. We found that even when learning situations were individually ambiguous, learners’ performance gradually improved over time. This improvement was driven in part by learners’ use of partial knowledge acquired from previous learning situations, even when they had not yet discovered correct word‐object mappings. These results suggest that word learning is a continuous process by means of real‐time information integration. (shrink)

Theories of cognitive development must address both the issue of how children bring their knowledge to bear on behavior in‐the‐moment, and how knowledge changes over time. We argue that seeking answers to these questions requires an appreciation of the dynamic nature of the developing system in its full, reciprocal complexity. We illustrate this dynamic complexity with results from two lines of research on early word learning. The first demonstrates how the child's active engagement with objects and people supports referent selection (...) via memories for what objects were previously seen in a cued location. The second set of results highlights changes in the role of novelty and attentional processes in referent selection and retention as children's knowledge of words and objects grows. Together this work suggests that understanding systems for perception, action, attention, and memory, and their complex interaction, is critical to understand word learning. We review recent literature that highlights the complex interactions between these processes in cognitive development and point to critical issues for future work. (shrink)

Theories of cognitive development must address both the issue of how children bring their knowledge to bear on behavior in-the-moment, and how knowledge changes over time. We argue that seeking answers to these questions requires an appreciation of the dynamic nature of the developing system in its full, reciprocal complexity. We illustrate this dynamic complexity with results from two lines of research on early word learning. The first demonstrates how the child's active engagement with objects and people supports referent selection (...) via memories for what objects were previously seen in a cued location. The second set of results highlights changes in the role of novelty and attentional processes in referent selection and retention as children's knowledge of words and objects grows. Together this work suggests that understanding systems for perception, action, attention, and memory, and their complex interaction, is critical to understand word learning. We review recent literature that highlights the complex interactions between these processes in cognitive development and point to critical issues for future work. (shrink)

The embodiment approach has shown that motor neural networks are involved in the processing of action verbs. There is developmental evidence that embodied effects on verb processing are already present in early years. Yet, the ontogenetic origin of this motor reuse in action verbs remains unknown. This longitudinal study investigates the co-occurrence of manual verbs and actions during mother-child daily routines when children were 1 to 2 and 2 to 3 years old. Eight mother-child dyads were video-recorded in 3-month intervals (...) across 12 months, and the timing of verbs and manual actions were coded by independent observers. Results showed that the probability of matched verb-action co-occurrences were much higher than that of random co-occurrences for Group 1 and Group 2, respectively. The distributions of the verb-action temporal intervals in both groups were quite symmetrical and skewed with the peak corresponding to both 0.00 s synchronic intervals and the shortest +5 s interval. Mother-led instances occurred in both groups whereas child-led instances were restricted to Group 2. Mothers pragmatically aligned their verbal productions, since they repeatedly used those verbs they shared with their children’s repertoire. In conclusion, the early multisensory communicative and manipulative scene affords grounding of verb meanings on the ongoing actions, facilitating verb-action pairing in the realm of social interactions, providing a new dimension to the prevailing solipsistic approach to embodiment. (shrink)

Children show a remarkable degree of consistency in learning some words earlier than others. What patterns of word usage predict variations among words in age of acquisition? We use distributional analysis of a naturalistic corpus of child‐directed speech to create quantitative features representing natural variability in word contexts. We evaluate two sets of features: One set is generated from the distribution of words into frames defined by the two adjacent words. These features primarily encode syntactic aspects of word usage. The (...) other set is generated from non‐adjacent co‐occurrences between words. These features encode complementary thematic aspects of word usage. Regression models using these distributional features to predict age of acquisition of 656 early‐acquired English words indicate that both types of features improve predictions over simpler models based on frequency and appearance in salient or simple utterance contexts. Syntactic features were stronger predictors of children's production than comprehension, whereas thematic features were stronger predictors of comprehension. Overall, earlier acquisition was predicted by features representing frames that select for nouns and verbs, and by thematic content related to food and face‐to‐face play topics; later acquisition was predicted by features representing frames that select for pronouns and question words, and by content related to narratives and object play. (shrink)

We evaluate here the performance of four models of cross-situational word learning: two global models, which extract and retain multiple referential alternatives from each word occurrence; and two local models, which extract just a single referent from each occurrence. One of these local models, dubbed Pursuit, uses an associative learning mechanism to estimate word-referent probability but pursues and tests the best referent-meaning at any given time. Pursuit is found to perform as well as global models under many conditions extracted from (...) naturalistic corpora of parent-child interactions, even though the model maintains far less information than global models. Moreover, Pursuit is found to best capture human experimental findings from several relevant cross-situational word-learning experiments, including those of Yu and Smith (), the paradigm example of a finding believed to support fully global cross-situational models. Implications and limitations of these results are discussed, most notably that the model characterizes only the earliest stages of word learning, when reliance on the co-occurring referent world is at its greatest. (shrink)

Beyond the observation that both speakers and listeners rapidly inspect the visual targets of referring expressions, it has been argued that such gaze may constitute part of the communicative signal. In this study, we investigate whether a speaker may, in principle, exploit listener gaze to improve communicative success. In the context of a virtual environment where listeners follow computer-generated instructions, we provide two kinds of support for this claim. First, we show that listener gaze provides a reliable real-time index of (...) understanding even in dynamic and complex environments, and on a per-utterance basis. Second, we show that a language generation system that uses listener gaze to provide rapid feedback improves overall task performance in comparison with two systems that do not use gaze. Aside from demonstrating the utility of listener gaze in situated communication, our findings open the door to new methods for developing and evaluating multi-modal models of situated interaction. (shrink)

One cue that may facilitate children's word learning is iconicity, or the correspondence between a word's form and meaning. Some have even proposed that iconicity in the early lexicon may serve to help children learn how to learn words, supporting the acquisition of even noniconic, or arbitrary, word–referent associations. However, this proposal remains untested. Here, we investigate the iconicity of caregivers’ speech to young children during a naturalistic free‐play session with novel stimuli and ask whether the iconicity of caregivers’ speech (...) facilitates children's learning of the noniconic novel names of those stimuli. Thirty‐four 1.5‐2‐year‐olds (19 girls; half monolingual English learners and half bilingual English‐Spanish learners) participated in a naturalistic free‐play task with their caregivers followed by a test of word‐referent retention. We found that caregivers’ use of iconicity, particularly in utterances in which they named the novel stimuli, was associated with the likelihood that children learned that novel name. This result held even when controlling for other factors associated with word learning, such as the concreteness and frequency of words in caregiver speech. Together, the results demonstrate that iconicity not only can serve to help children identify the referent of novel words (as in previous research) but can also support their ability to retain even noniconic word‐referent mappings. (shrink)

Cognitive Science, Volume 46, Issue 9, September 2022.

Cognitive Science, Volume 46, Issue 8, August 2022.

When humans are addressing multiple robots with informative speech acts, their cognitive resources are shared between all the participating robot agents. For each moment, the user’s behavior is not only determined by the actions of the robot that they are directly gazing at, but also shaped by the behaviors from all the other robots in the shared environment. We define cooperative behavior as the action performed by the robots that are not capturing the user’s direct attention. In this paper, we (...) are interested in how the human participants adjust and coordinate their own behavioral cues when the robot agents are performing different cooperative gaze behaviors. A novel gaze-contingent platform was designed and implemented. The robots’ behaviors were triggered by the participant’s attentional shifts in real time. Results showed that the human participants were highly sensitive when the robot agents were performing different cooperative gazing behaviors. Keywords: human-robot interaction; multi-robot interaction; multiparty interaction; eye gaze cue; embodied conversational agent. (shrink)