We investigated the coupling between a speaker's and a listener's eye movements. Some participants talked extemporaneously about a television show whose cast members they were viewing on a screen in front of them. Later, other participants listened to these monologues while viewing the same screen. Eye movements were recorded for all speakers and listeners. According to cross‐recurrence analysis, a listener's eye movements most closely matched a speaker's eye movements at a delay of 2 sec. Indeed, the more closely a listener's (...) eye movements were coupled with a speaker's, the better the listener did on a comprehension test. In a second experiment, low‐level visual cues were used to manipulate the listeners' eye movements, and these, in turn, influenced their latencies to comprehension questions. Just as eye movements reflect the mental state of an individual, the coupling between a speaker's and a listener's eye movements reflects the success of their communication. (shrink)

Traditional mechanistic accounts of language processing derive almost entirely from the study of monologue. Yet, the most natural and basic form of language use is dialogue. As a result, these accounts may only offer limited theories of the mechanisms that underlie language processing in general. We propose a mechanistic account of dialogue, the interactive alignment account, and use it to derive a number of predictions about basic language processes. The account assumes that, in dialogue, the linguistic representations employed by the (...) interlocutors become aligned at many levels, as a result of a largely automatic process. This process greatly simplifies production and comprehension in dialogue. After considering the evidence for the interactive alignment model, we concentrate on three aspects of processing that follow from it. It makes use of a simple interactive inference mechanism, enables the development of local dialogue routines that greatly simplify language processing, and explains the origins of self-monitoring in production. We consider the need for a grammatical framework that is designed to deal with language in dialogue rather than monologue, and discuss a range of implications of the account. Key Words: common ground; dialogue; dialogue routines; language comprehension; language production; monitoring; perception-behavior link. (shrink)

What is the proper unit of analysis in the psycholinguistics of dialog? While classical approaches are largely based on models of individual linguistic processing, recent advances stress the social coordinative nature of dialog. In the influential interactive alignment model, dialogue is thus approached as the progressive entrainment of interlocutors' linguistic behaviors toward the alignment of situation models. Still, the driving mechanisms are attributed to individual cognition in the form of automatic structural priming. Challenging these ideas, we outline a dynamical framework (...) for studying dialog based on the notion of interpersonal synergy. Crucial to this synergetic model is the emphasis on dialog as an emergent, self-organizing, interpersonal system capable of functional coordination. A consequence of this model is that linguistic processes cannot be reduced to the workings of individual cognitive systems but must be approached also at the interpersonal level. From the synergy model follows a number of new predictions: beyond simple synchrony, good dialog affords complementary dynamics, constrained by contextual sensitivity and functional specificity. We substantiate our arguments by reference to recent empirical studies supporting the idea of dialog as interpersonal synergy. (shrink)

A variety of theoretical frameworks predict the resemblance of behaviors between two people engaged in communication, in the form of coordination, mimicry, or alignment. However, little is known about the time course of the behavior matching, even though there is evidence that dyads synchronize oscillatory motions (e.g., postural sway). This study examined the temporal structure of nonoscillatory actions—language, facial, and gestural behaviors—produced during a route communication task. The focus was the temporal relationship between matching behaviors in the interlocutors (e.g., facial (...) behavior in one interlocutor vs. the same facial behavior in the other interlocutor). Cross-recurrence analysis revealed that within each category tested (language, facial, gestural), interlocutors synchronized matching behaviors, at temporal lags short enough to provide imitation of one interlocutor by the other, from one conversational turn to the next. Both social and cognitive variables predicted the degree of temporal organization. These findings suggest that the temporal structure of matching behaviors provides low-level and low-cost resources for human interaction. (shrink)

This study investigates interpersonal processes underlying dialog by comparing two approaches, interactive alignment and interpersonal synergy, and assesses how they predict collective performance in a joint task. While the interactive alignment approach highlights imitative patterns between interlocutors, the synergy approach points to structural organization at the level of the interaction—such as complementary patterns straddling speech turns and interlocutors. We develop a general, quantitative method to assess lexical, prosodic, and speech/pause patterns related to the two approaches and their impact on collective (...) performance in a corpus of task-oriented conversations. The results show statistical presence of patterns relevant for both approaches. However, synergetic aspects of dialog provide the best statistical predictors of collective performance and adding aspects of the alignment approach does not improve the model. This suggests that structural organization at the level of the interaction plays a crucial role in task-oriented conversations, possibly constraining and integrating processes related to alignment. (shrink)

People coordinate body postures and gaze patterns during conversation. We review literature showing that (1) action embodies cognition, (2) postural coordination emerges spontaneously when two people converse, (3) gaze patterns influence postural coordination, (4) gaze coordination is a function of common ground knowledge and visual information that conversants believe they share, and (5) gaze coordination is causally related to mutual understanding. We then consider how coordination, generally, can be understood as temporarily coupled neuromuscular components that function as a collective unit (...) known as a coordinative structure in the motor control literature. We speculate that the coordination of gaze and body sway found in conversation may be understood as a cross‐person coordinative structure that embodies the goals of the joint action system. (shrink)

Most everyday tasks involve multiple modalities, which raises the question of how the processing of these modalities is coordinated by the cognitive system. In this paper, we focus on the coordination of visual attention and linguistic processing during speaking. Previous research has shown that objects in a visual scene are fixated before they are mentioned, leading us to hypothesize that the scan pattern of a participant can be used to predict what he or she will say. We test this hypothesis (...) using a data set of cued scene descriptions of photo-realistic scenes. We demonstrate that similar scan patterns are correlated with similar sentences, within and between visual scenes; and that this correlation holds for three phases of the language production process (target identification, sentence planning, and speaking). We also present a simple algorithm that uses scan patterns to accurately predict associated sentences by utilizing similarity-based retrieval. (shrink)

Eye movements were recorded during the display of two images of a real-world scene that were inspected to determine whether they were the same or not . In the displays where the pictures were different, one object had been changed, and this object was sometimes taken from another scene and was incongruent with the gist. The experiment established that incongruous objects attract eye fixations earlier than the congruous counterparts, but that this effect is not apparent until the picture has been (...) displayed for several seconds. By controlling the visual saliency of the objects the experiment eliminates the possibility that the incongruency effect is dependent upon the conspicuity of the changed objects. A model of scene perception is suggested whereby attention is unnecessary for the partial recognition of an object that delivers sufficient information about its visual characteristics for the viewer to know that the object is improbable in that particular scene, and in which full identification requires foveal inspection. (shrink)

It makes sense that the more information people share, the better they communicate. To evaluate the effect of knowledge overlap on the effectiveness of communication, participants played a communication game where the “director” identified objects to the “addressee”. Pairs either shared information about most objects' names (high overlap), or about the minority of objects' names (low overlap). We found that high-overlap directors tended to use more names than low overlap directors. High overlap directors also used more names with objects whose (...) names only they knew, thereby confusing their addressees more often than low-overlap directors. We conclude that while sharing more knowledge can be beneficial to communication overall, it can cause communication to be locally ineffective. Sharing more information reduces communication effectiveness precisely when there is an opportunity to inform—when people communicate information only they themselves know. (shrink)

It has been suggested that iconic graphical signs evolve into symbolic graphical signs through repeated usage. This article reports a series of interactive graphical communication experiments using a ‘pictionary’ task to establish the conditions under which the evolution might occur. Experiment 1 rules out a simple repetition based account in favor of an account that requires feedback and interaction between communicators. Experiment 2 shows how the degree of interaction affects the evolution of signs according to a process of grounding. Experiment (...) 3 confirms the prediction that those not involved directly in the interaction have trouble interpreting the graphical signs produced in Experiment 1. On the basis of these results, this article argues that icons evolve into symbols as a consequence of the systematic shift in the locus of information from the sign to the users' memory of the sign's usage supported by an interactive grounding process. (shrink)

Sharing a public language facilitates particularly efficient forms of joint perception and action by giving interlocutors refined tools for directing attention and aligning conceptual models and action. We hypothesized that interlocutors who flexibly align their linguistic practices and converge on a shared language will improve their cooperative performance on joint tasks. To test this prediction, we employed a novel experimental design, in which pairs of participants cooperated linguistically to solve a perceptual task. We found that dyad members generally showed a (...) high propensity to adapt to each other’s linguistic practices. However, although general linguistic alignment did not have a positive effect on performance, the alignment of particular task-relevant vocabularies strongly correlated with collective performance. In other words, the more dyad members selectively aligned linguistic tools fit for the task, the better they performed. Our work thus uncovers the interplay between social dynamics and sensitivity to task affordances in successful cooperation. (shrink)

Language use can be viewed as a form of joint activity that requires the coordination of meaning between individuals. Because the linguistic signal is notoriously ambiguous, interlocutors need to draw upon additional sources of information to resolve ambiguity and achieve shared understanding. One way individuals can achieve coordination is by using inferences about the interlocutor’s intentions and mental states to adapt their behavior. However, such an inferential process can be demanding in terms of both time and cognitive resources. Here, we (...) suggest that interaction provides interlocutors with many cues that can support coordination of meaning, even when they are neither produced intentionally for that purpose nor interpreted as signaling speakers’ intention. In many circumstances, interlocutors can take advantage of these cues to adapt their behavior in ways that promote coordination, bypassing the need to resort to deliberative inferential processes. (shrink)