Recent advances in the field of statistical learning have established that learners are able to track regularities of multimodal stimuli, yet it is unknown whether the statistical computations are performed on integrated representations or on separate, unimodal representations. In the present study, we investigated the ability of adults to integrate audio and visual input during statistical learning. We presented learners with a speech stream synchronized with a video of a speaker’s face. In the critical condition, the visual (e.g. /gi/) and (...) auditory (e.g. /mi/) signals were occasionally incongruent, which we predicted would produce the McGurk illusion, resulting in the perception of an audiovisual syllable (e.g. /ni/). In this way, we used the McGurk illusion to manipulate the underlying statistical structure of the speech streams, such that perception of these illusory syllables facilitated participants’ ability to segment the speech stream. Our results therefore demonstrate that participants can integrate audio and visual input to perceive the McGurk illusion during statistical learning. We interpret our findings as support for modality-interactive accounts of statistical learning. (shrink)

The roles of linguistic, cognitive, and social-pragmatic processes in word learning are well established. If statistical mechanisms also contribute to word learning, they must interact with these processes; however, there exists little evidence for such mechanistic synergy. Adults use co-occurrence statistics to encode speech–object pairings with detailed sensitivity in stochastic learning environments (Vouloumanos, 2008). Here, we replicate this statistical work with nonspeech sounds and compare the results with the previous speech studies to examine whether exclusion constraints contribute equally to the (...) statistical learning of speech–object and nonspeech–object associations. In environments in which performance could benefit from exclusion, we find a learning advantage for speech over nonspeech, revealing an interaction between statistical and exclusion processes in associative word learning. (shrink)

Recent work asked if visual cues facilitate word segmentation in adults and infants. While adults showed better word segmentation when presented with a regular visual cue, infants did not. This difference was attributed to infants' lack of understanding that objects have labels. Alternatively, infants’ performance could reflect their difficulty with tracking and integrating multiple multimodal cues. We contrasted these two accounts by looking at the effect of visual cues on word segmentation in adults and across childhood. We found that older (...) children benefitted from the regular visual cues, but younger children, who already knew that objects have labels, did not. Knowing that objects have labels was not enough to use visual cues as an aid for segmentation. These findings show that the ability to integrate multimodal cues develops during childhood, and it is not yet adult-like in children. (shrink)

Two experiments were conducted to examine adult learners' ability to extract multiple statistics in simultaneously presented visual and auditory input. Experiment 1 used a cross‐situational learning paradigm to test whether English speakers were able to use co‐occurrences to learn word‐to‐object mappings and concurrently form object categories based on the commonalities across training stimuli. Experiment 2 replicated the first experiment and further examined whether speakers of Mandarin, a language in which final syllables of object names are more predictive of category membership (...) than English, were able to learn words and form object categories when trained with the same type of structures. The results indicate that both groups of learners successfully extracted multiple levels of co‐occurrence and used them to learn words and object categories simultaneously. However, marked individual differences in performance were also found, suggesting possible interference and competition in processing the two concurrent streams of regularities. (shrink)

Although speech categories are defined by multiple acoustic dimensions, some are perceptually weighted more than others and there are residual effects of native-language weightings in non-native speech perception. Recent research on nonlinguistic sound category learning suggests that the distribution characteristics of experienced sounds influence perceptual cue weights: Increasing variability across a dimension leads listeners to rely upon it less in subsequent category learning (Holt & Lotto, 2006). The present experiment investigated the implications of this among native Japanese learning English /r/-/l/ (...) categories. Training was accomplished using a videogame paradigm that emphasizes associations among sound categories, visual information, and players’ responses to videogame characters rather than overt categorization or explicit feedback. Subjects who played the game for 2.5 h across 5 days exhibited improvements in /r/-/l/ perception on par with 2–4 weeks of explicit categorization training in previous research and exhibited a shift toward more native-like perceptual cue weights. (shrink)