When faced with an ambiguous pronoun, comprehenders use both multimodal cues and linguistic cues to identify the antecedent. While research has shown that gestures facilitate language comprehension, improve reference tracking, and influence the interpretation of ambiguous pronouns, literature on reference resolution suggests that a wide set of linguistic constraints influences the successful resolution of ambiguous pronouns and that linguistic cues are more powerful than some multimodal cues. To address the outstanding question of the importance of gesture as a cue in (...) reference resolution relative to cues in the speech signal, we have previously investigated the comprehension of contrastive gestures that indexed abstract referents – in this case expressions of personal preference – and found that such gestures did facilitate the resolution of ambiguous statements of preference. In this study, we extend this work to investigate whether the effect of gesture on resolution is diminished when the gesture indexes a statement that is less likely to be interpreted as the correct referent. Participants watched videos in which a speaker contrasted two ideas that were either neutral or moral acceptable). A gesture to the left or right side co-occurred with speech expressing each position. In gesture-disambiguating trials, an ambiguous phrase was accompanied by a gesture to one side or the other. In gesture non-disambiguating trials, no third gesture occurred with the ambiguous phrase. Participants were more likely to choose the idea accompanied by gesture as the stimulus speaker’s preference. We found no effect of scenario type. Regardless of whether the linguistic cue expressed a view that was morally charged or neutral, observers used gesture to understand the speaker’s opinion. This finding contributes to our understanding of the strength and range of cues, both linguistic and multimodal, that listeners use to resolve ambiguous references. (shrink)

Cette étude qualitative propose une modélisation multimodale des constructions subordonnées en conversation spontanée, à partir de leur action sur les cadres interprétatifs. Les subordonnées ont longtemps été décrites en linguistique comme des éléments dépendants qui complètent des éléments primaires. Cependant, la Grammaire Cognitive a remis en question ce point de vue en montrant que l'emboîtement syntaxique ne reflète souvent que le point de départ choisi par les locuteurs pour véhiculer leur message. Les subordonnées sont des pratiques interactionnelles qui offrent une (...) reconstruction interprétative du discours. Les différents types syntaxiques de subordonnées font référence à différents cadres interprétatifs dans l'expérience du locuteur. La sélection de ces différents cadres est exprimée par le nombre différent d'indices de frontière prosodiques et gestuels produits en co-occurrence avec chaque type syntaxique de subordonnée. (shrink)

Speakers perform manual gestures in the physical space nearest them, called gesture space. We used a controlled elicitation task to explore whether speakers use gesture space in a consistent way and whether they use space in a contrastive way when talking about abstract referents. Participants answered two questions designed to elicit contrastive, abstract discourse. We investigated manual gesture behavior. Gesture hand, location on the horizontal axis, and referent in corresponding speech were coded. We also coded contrast in speech. Participants’ overall (...) tendency to use the same hand = 13.12, p =.001, 95% CI [.31,.43], d = 2.53) and same location = 7.47, p =.001, 95% CI [.27,.47], d = 1.69) when referring to an entity was higher than expected frequency. When comparing pairs of gestures produced with contrastive speech to pairs of gestures produced with non-contrastive speech, we found a greater tendency to produce gestures with different hands for contrastive speech: = 4.19, p =.001, 95% CI [.27,.82], d = 1.42). We did not find associations between dominant side and positive concepts or between left, center, and right space and past, present, and future, respectively, as predicted by previous studies. Taken together, our findings suggest that speakers do produce spatially consistent and contrastive gestures for abstract as well as concrete referents. They may be using spatial resources to assist with abstract thinking, and/or to help interlocutors with reference tracking. Our findings also highlight the complexity of predicting gesture hand and location, which appears to be the outcome of many competing variables. (shrink)

Differences in language use and structures between signed and spoken languages have often been attributed to so-called language “modality.” Indeed, this is derived from the conception that spoken languages resort to both the oral-aural channel of speech and the visual-kinesic channel of visible bodily action whereas signed languages only resort to the latter. This paper addresses the use of enactment, a depictive communicative strategy whereby language users imitate referents in signed and spoken languages. Reviewing comparative research on enactment, this paper (...) highlights theoretical and methodological shortcomings in prior works. First, a broader set of causal explanations needs to be taken into account when interpreting differences between signing and speaking communities. A more comprehensive conceptual toolbox ensures that differences are not automatically attributed to modality. In particular, less-studied factors of language diversity, like sociolinguistic and cultural ecologies, and how they interact with other factors should be considered. Second, diversity in enactment across signed and spoken languages is shown to be inadequately and insufficiently documented. It is argued that by comparing enactment across more diverse signing and speaking communities and using large, directly comparable corpora, solid analyses can be carried out, enabling a better understanding of how and why different communities use enactment in similar or different ways. (shrink)

Journal Name: Cognitive Linguistics Issue: Ahead of print.