Sound-symbolism is the nonarbitrary link between the sound and meaning of a word. Japanese-speaking children performed better in a verb generalization task when they were taught novel sound-symbolic verbs, created based on existing Japanese sound-symbolic words, than novel nonsound-symbolic verbs (Imai, Kita, Nagumo, & Okada, 2008). A question remained as to whether the Japanese children had picked up regularities in the Japanese sound-symbolic lexicon or were sensitive to universal sound-symbolism. The present study aimed to provide support for the latter. In (...) a verb generalization task, English-speaking 3-year-olds were taught novel sound-symbolic verbs, created based on Japanese sound-symbolism, or novel nonsound-symbolic verbs. English-speaking children performed better with the sound-symbolic verbs, just like Japanese-speaking children. We concluded that children are sensitive to universal sound-symbolism and can utilize it in word learning and generalization, regardless of their native language. (shrink)

Gestures and speech are clearly synchronized in many ways. However, previous studies have shown that the semantic similarity between gestures and speech breaks down as people approach transitions in understanding. Explanations for these gesture–speech mismatches, which focus on gestures and speech expressing different cognitive strategies, have been criticized for disregarding gestures’ and speech's integration and synchronization. In the current study, we applied three different perspectives to investigate gesture–speech synchronization in an easy and a difficult task: temporal alignment, semantic similarity, and (...) complexity matching. Participants engaged in a simple cognitive task and were assigned to either an easy or a difficult condition. We automatically measured pointing gestures, and we coded participant's speech, to determine the temporal alignment and semantic similarity between gestures and speech. Multifractal detrended fluctuation analysis was used to determine the extent of complexity matching between gestures and speech. We found that task difficulty indeed influenced gesture–speech synchronization in all three domains. We thereby extended the phenomenon of gesture–speech mismatches to difficult tasks in general. Furthermore, we investigated how temporal alignment, semantic similarity, and complexity matching were related in each condition, and how they predicted participants’ task performance. Our study illustrates how combining multiple perspectives, originating from different research areas (i.e., coordination dynamics, complexity science, cognitive psychology), provides novel understanding about cognitive concepts in general and about gesture–speech synchronization and task difficulty in particular. (shrink)

Sighted speakers of different languages vary systematically in how they package and order components of a motion event in speech. These differences influence how semantic elements are organized in gesture, but only when those gestures are produced with speech, not without speech. We ask whether the cross-linguistic similarity in silent gesture is driven by the visuospatial structure of the event. We compared 40 congenitally blind adult native speakers of English or Turkish to 80 sighted adult speakers as they described three-dimensional (...) motion scenes. We found an effect of language on co-speech gesture, not on silent gesture—blind speakers of both languages organized their silent gestures as sighted speakers do. Humans may have a natural semantic organization that they impose on events when conveying them in gesture without language—an organization that relies on neither visuospatial cues nor language structure. (shrink)

Expression of physical motion (e.g.,man runs by) shows systematic variability not only between language types (i.e., inter-typological) but also within a language type (i.e., intra-typological). In this study, we asked whether the patterns of variability extend to metaphorical motion events (e.g.,time runs by). Our analysis of randomly selected 450 physical motion (150/language) and 450 metaphorical motion (150/language) event descriptions from written texts originally produced by German, Polish, and Spanish authors showed strong inter-typological differences in the expression of both event types. (...) German and Polish speakers differed from Spanish speakers in how they packaged manner and path components of a motion event; they also differed in how extensively they expressed each component in their lexicalization of motion. The strong inter-typological differences were accompanied by more modest intra-typological variability: Polish and German writers differed in their packaging and lexicalization of manner and path components of metaphorical—butnotphysical—motion events. Our results provide evidence for robust inter-typological differences evident in the expression of both physical and metaphorical motion, along with less robust intra-typological differences, largely evident in the expression of metaphorical motion. Our study thus highlights event type as an important factor in determining crosslinguistic variation in motion expression. (shrink)

Tense/aspect morphology on verbs is often thought to depend on event features like telicity, but it is not known how speakers identify these features in visual scenes. To examine this question, we asked Japanese speakers to describe computer‐generated animations of simple actions with variation in visual features related to telicity. Experiments with adults and children found that they could use goal information in the animations to select appropriate past and progressive verb forms. They also produced a large number of different (...) verb forms. To explain these findings, a deep‐learning model of verb production from visual input was created that could produce a human‐like distribution of verb forms. It was able to use visual cues to select appropriate tense/aspect morphology. The model predicted that video duration would be related to verb complexity, and past tense production would increase when it received the endpoint as input. These predictions were confirmed in a third study with Japanese adults. This work suggests that verb production could be tightly linked to visual heuristics that support the understanding of events. (shrink)

Berber languages use the directionals =d and =nn to specify the deictic path of motion verbs. These clitics occur with a range of verbs from other semantic classes (e.g. change of state verbs, verbs of vision and perception), with which they can be attributed different meanings. The first goal of this paper is to provide a cross-dialectal description of these meanings. The second goal is to show the role of verbal semantics in their constructions and the overall distribution of the (...) clitics. Adopting Talmy’s framework (2000) of lexicalisation of semantic concepts, linked to the domain of space and motion, I propose that the particular meaning components lexicalised by the modified verb govern the occurrence of the clitics and their particular meaning. (shrink)

This study explores the relationship between language and thought in similarity judgments by testing how monolingual children who speak languages with partial typological differences in motion description respond to visual motion event stimuli. Participants were either Chinese- or English-speaking, 3-year-olds, 8-year-olds and adults who judged the similarity between caused motion scenes in a match-to-sample task. The results suggest, first of all, that the two younger groups of 3-year-olds are predominantly path-oriented, irrespective of language, as evidenced by their significantly longer fixation (...) on path-match videos rather than manner-match videos in a preferential looking scheme. Using categorical measurement of overt choices, older children and adults also showed a shared tendency of being more path-oriented. However, the analysis using continuous measurement of reaction time revealed significant variations in spatial cognition that can be related to linguistic differences: English speakers tended to be more manner-oriented while Chinese speakers were equally manner- and path-oriented. On the whole, our findings indicate a likelihood that children’s non-linguistic thought is similar prior to internalising the lexicalisation pattern of motion events in their native languages, but shows divergences after such habitual use, thus suggesting that the pattern of non-linguistic thought may be linked, among other things, to linguistic structure. (shrink)

Speakers of different languages (e.g., English vs. Turkish) show a binary split in how they package and order components of a motion event in speech and co‐speech gesture but not in silent gesture. In this study, we focused on Mandarin Chinese, a language that does not follow the binary split in its expression of motion in speech, and asked whether adult Chinese speakers would follow the language‐specific speech patterns in co‐speech but not silent gesture, thus showing a pattern akin to (...) Turkish and English adult speakers in their description of animated motion events. Our results provided evidence for this pattern, with Chinese—as well as English and Turkish—speakers following language‐specific patterns in speech and co‐speech gesture but not in silent gesture. Our results provide support for the “thinking‐for‐speaking” account, namely that language influences thought only during online, but not offline, production of speech. (shrink)

To what extent do people attribute meanings to “nonsense” words? How general is such attribution of meaning? We used a set of words lacking conventional meanings to elicit drawings of made‐up creatures. Separate groups of participants rated the nonsense words and the drawings on several semantic dimensions and selected what name best corresponded to each creature. Despite lacking conventional meanings, “nonsense” words elicited a high level of consistency in the produced drawings. Meaning attributions made to nonsense words corresponded with meaning (...) attributions made by separate people to drawings that were inspired by the name. Naïve participants were able to recover the name that inspired the drawing with greater‐than‐chance accuracy. These results suggest that people make liberal and consistent use of non‐arbitrary relationships between forms and meanings.Open Research Badges This article has been awarded Open Materials and Open Data badges. All materials and data are publicly accessible via the Open Science Framework at https://osf.io/8juyc/. Learn more about the Open Practices badges from the Center for Open Science: https://osf.io/tvyxz/wiki. (shrink)