When faced with an ambiguous pronoun, comprehenders use both multimodal cues and linguistic cues to identify the antecedent. While research has shown that gestures facilitate language comprehension, improve reference tracking, and influence the interpretation of ambiguous pronouns, literature on reference resolution suggests that a wide set of linguistic constraints influences the successful resolution of ambiguous pronouns and that linguistic cues are more powerful than some multimodal cues. To address the outstanding question of the importance of gesture as a cue in (...) reference resolution relative to cues in the speech signal, we have previously investigated the comprehension of contrastive gestures that indexed abstract referents – in this case expressions of personal preference – and found that such gestures did facilitate the resolution of ambiguous statements of preference. In this study, we extend this work to investigate whether the effect of gesture on resolution is diminished when the gesture indexes a statement that is less likely to be interpreted as the correct referent. Participants watched videos in which a speaker contrasted two ideas that were either neutral or moral acceptable). A gesture to the left or right side co-occurred with speech expressing each position. In gesture-disambiguating trials, an ambiguous phrase was accompanied by a gesture to one side or the other. In gesture non-disambiguating trials, no third gesture occurred with the ambiguous phrase. Participants were more likely to choose the idea accompanied by gesture as the stimulus speaker’s preference. We found no effect of scenario type. Regardless of whether the linguistic cue expressed a view that was morally charged or neutral, observers used gesture to understand the speaker’s opinion. This finding contributes to our understanding of the strength and range of cues, both linguistic and multimodal, that listeners use to resolve ambiguous references. (shrink)

The framework of depicting put forward byClark (2016)offers a schematic vantage point from which to examine iconic language use. Confronting the framework with empirical data, we consider some of its key theoretical notions. Crucially, by reconceptualizing the typology of depictions, we identify an overlooked domain in the literature: “speech-embedded nonverbal depictions,” namely cases where meaning is communicated iconically, nonverbally, and without simultaneously co-occurring speech. In addition to contextualizing the phenomenon in relation to existing research, we demonstrate, with examples from American (...) TV talk shows, how such depictions function in real-life language use, offering a brief sketch of their complexities and arguing also for their theoretical significance. (shrink)

The present contribution integrates a recent, multilevel approach to metaphor advanced by Kövecses (2017 and later publications), with gestures as a mode of metaphorical expression. In doing so, the paper shows how different elements of conceptual structure, varying from the abstract, recurring image schema, through more complex conceptual domains and frames, to contextually embedded and variable metaphorical scenarios, participate in the metaphoricity of gestures. This application of gestures to the multilevel approach lends direct support to the idea that human conceptual (...) system recruits various semiotic modes – not limited to language – for expressing its content. Image schemas of object and source-path-goal have been shown to participate directly in the domain of transfer, which, when embedded in context of a given communicative situation, has become the conceptual frame of coming out. The frame, when realised via different gestural forms in a shared context across many speakers, turns into a highly context-sensitive, variable, and speech co-expressive gestural metaphorical scenario of coming out is showing an object. (shrink)