When interlocutors repeatedly describe referents to each other, they rapidly converge on referring expressions which become increasingly systematized and abstract as the interaction progresses. Previous experimental research suggests that interactive repair mechanisms in dialogue underpin convergence. However, this research has so far only focused on the role of other-initiated repair and has not examined whether self-initiated repair might also play a role. To investigate this question, we report the results from a computer-mediated maze task experiment. In this task, participants communicate (...) with each other via an experimental chat tool, which selectively transforms participants’ private turn-revisions into public self-repairs that are made visible to the other participant. For example, if a participant, A, types “On the top square,” and then before sending, A revises the turn to “On the top row,” the server automatically detects the revision and transforms the private turn-revisions into a public self-repair, for example, “On the top square umm I meant row.” Participants who received these transformed turns used more abstract and systematized referring expressions, but performed worse at the task. We argue that this is due to the artificial self-repairs causing participants to put more effort into diagnosing and resolving the referential coordination problems they face in the task, yielding better grounded spatial semantics and consequently increased use of abstract referring expressions. (shrink)

Symbols enable people to organize and communicate about the world. However, the ways in which symbolic knowledge is learned and then represented in the mind are poorly understood. We present a formal analysis of symbolic learning—in particular, word learning—in terms of prediction and cue competition, and we consider two possible ways in which symbols might be learned: by learning to predict a label from the features of objects and events in the world, and by learning to predict features from a (...) label. This analysis predicts significant differences in symbolic learning depending on the sequencing of objects and labels. We report a computational simulation and two human experiments that confirm these differences, revealing the existence of Feature-Label-Ordering effects in learning. Discrimination learning is facilitated when objects predict labels, but not when labels predict objects. Our results and analysis suggest that the semantic categories people use to understand and communicate about the world can only be learned if labels are predicted from objects. We discuss the implications of this for our understanding of the nature of language and symbolic thought, and in particular, for theories of reference. (shrink)

Recent research indicates that interpersonal communication is noisy, and that people exhibit considerable insensitivity to problems in communication. Using a dyadic referential communication task, the goal of which is accurate information transfer, this study examined the extent to which interlocutors are sensitive to problems in communication and use other‐initiated repairs (OIRs) to address them. Participants were randomly assigned to dyads (N = 88 participants, or 44 dyads) and tried to communicate a series of recurring abstract geometric shapes to a partner (...) across a text–chat interface. Participants alternated between directing (describing shapes) and matching (interpreting shape descriptions) roles across 72 trials of the task. Replicating prior research, over repeated social interactions communication success improved and the shape descriptions became increasingly efficient. In addition, confidence in having successfully communicated the different shapes increased over trials. Importantly, matchers were less confident on trials in which communication was unsuccessful, communication success was lower on trials that contained an OIR compared to those that did not contain an OIR, and OIR trials were associated with lower Director Confidence. This pattern of results demonstrates that (a) interlocutors exhibit (a degree of) sensitivity to problems in communication, (b) they appropriately use OIRs to address problems in communication, and (c) OIRs signal problems in communication. (shrink)

Children often refer to things ambiguously but learn not to from responding to clarification requests. We review and explore this learning process here. In Study 1, eighty-four 2- and 4-year-olds were tested for their ability to request stickers from either (a) a small array with one dissimilar distracter or (b) a large array containing similar distracters. When children made ambiguous requests, they received either general feedback or specific questions about which of two options they wanted. With training, children learned to (...) produce more complex object descriptions and did so faster in the specific feedback condition. They also tended to provide more information when requesting stickers from large arrays. In Study 2, we varied only distracter similarity during training and then varied array size in a generalization test. Children found it harder to learn in this case. In the generalization test, 4-year-olds were more likely to provide information (a) when it was needed because distracters were similar to the target and (b) when the array size was greater (regardless of need for information). We discuss how clear cues to potential ambiguity are needed for children to learn to tailor their referring expression to context and how several cues of heuristic value (e.g., more distracters > say more) can promote the efficiency of communication while language is developing. Finally, we consider whether it would be worthwhile drawing on the human learning process when developing algorithms for the production of referring expressions. (shrink)

Past research has sought to elucidate how speakers and addressees establish common ground in conversation, yet few studies have focused on how visual cues such as co-speech gestures contribute to this process. Likewise, the effect of cognitive constraints on multimodal grounding remains to be established. This study addresses the relationship between the verbal and gestural modalities during grounding in referential communication. We report data from a collaborative task where repeated references were elicited, and a time constraint was imposed to increase (...) cognitive load. Our results reveal no differential effects of repetition or cognitive load on the semantic-based gesture rate, suggesting that representational gestures and speech are closely coordinated during grounding. However, gestures and speech differed in their execution, especially under time pressure. We argue that speech and gesture are two complementary streams that might be planned in conjunction but that unfold independently in later stages of language production, with speakers emphasizing the form of their gestures, but not of their words, to better meet the goals of the collaborative task. (shrink)

Two eye-tracking experiments investigated how and when pointing gestures and location descriptions affect target identification. The experiments investigated the effect of gestures and referring expressions on the time course of fixations to the target, using videos of human gestures and human voice, and animated gestures and synthesized speech. Ambiguous, yet informative pointing gestures elicited attention and facilitated target identification, akin to verbal location descriptions. Moreover, target identification was superior when both pointing gestures and verbal location descriptions were used. These findings (...) suggest that gesture not only operates as a context to verbal descriptions, or that verbal descriptions operate as a context to gesture, but that they complement one another in reference resolution. (shrink)

Following a synthesis of naturalistic and experimental studies of language creation, we propose a theoretical model that describes the process through which human communication systems might arise and evolve. Three key processes are proposed that give rise to effective, efficient and shared human communication systems: motivated signs that directly resemble their meaning facilitate cognitive alignment, improving communication success; behavioral alignment onto an inventory of shared sign-to-meaning mappings bolsters cognitive alignment between interacting partners; sign refinement, through interactive feedback, enhances the efficiency (...) of the evolving communication system. By integrating the findings across a range of diverse studies, we propose a theoretical model of the process through which the earliest human communication systems might have arisen and evolved. Importantly, because our model is not bound to a single modality it can describe the creation of shared sign systems across a range of contexts, informing theories of language creation and evolution. (shrink)

Cross-linguistic studies focus on inverse correlations (trade-offs) between linguistic variables that reflect different cues to linguistic meanings. For example, if a language has no case marking, it is likely to rely on word order as a cue for identification of grammatical roles. Such inverse correlations are interpreted as manifestations of language users’ tendency to use language efficiently. The present study argues that this interpretation is problematic. Linguistic variables, such as the presence of case, or flexibility of word order, are aggregate (...) properties, which do not represent the use of linguistic cues in context directly. Still, such variables can be useful for circumscribing the potential role of communicative efficiency in language evolution, if we move from cross-linguistic trade-offs to multivariate causal networks. This idea is illustrated by a case study of linguistic variables related to four types of Subject and Object cues: case marking, rigid word order of Subject and Object, tight semantics and verb-medial order. The variables are obtained from online language corpora in thirty languages, annotated with the Universal Dependencies. The causal model suggests that the relationships between the variables can be explained predominantly by sociolinguistic factors, leaving little space for a potential impact of efficient linguistic behavior. (shrink)

To reduce ambiguity across a conversation, interlocutors reach temporary conventions or referential precedents on how to refer to an entity. Despite their central role in communication, the cognitive underpinnings of the interpretation of precedents remain unclear, specifically the role and mechanisms by which information related to the speaker is integrated. We contrast predictions of one-stage, original two-stage, and extended two-stage models for the processing of speaker information and provide evidence favoring the latter: we show that both stages are sensitive to (...) speaker-specific information. Using an experimental paradigm based on visual-world eye tracking in the context of a referential communication task, we look at the moment-by-moment interpretation of precedents and focus on the temporal profile of the influence of the speaker and linguistic information when facing ambiguity. We find two clearly identifiable moments where speaker-specific information has its effects on reference resolution. We conclude that these two stages reflect two distinct cognitive mechanisms, with different timings, and rely on different representational formats for encoding and accessing information about the speaker: a cue-driven memory retrieval process that mediates language processing and an inferential mechanism based on perspective-taking abilities. (shrink)

During dialog, references are presented, accepted, and potentially reused. Two experiments were conducted to examine reuse in a naturalistic setting. In Experiment 1, where the participants interacted face to face, self-presented references and references accepted through verbatim repetition were reused more. Such biases persisted after the end of the interaction. In Experiment 2, where the participants interacted over the phone, reference reuse mainly depended on whether the participant could see the landmarks being referred to, although this bias seemed to be (...) only transient. Consistent with the memory-based approach to dialog, these results shed light on how differences in accessibility in memory affect the unfolding of the interaction. (shrink)

In this introduction to the special issue of Discourse Studies on `Referring to Self and Others in Conversation' we briefly survey the history of conversation analytic work on reference to persons from Sacks and Schegloff's pioneering seven-page paper to the most recently published work. We then introduce the contributions to the special issue.

This paper studies the uses of proper names within a communication-theoretic setting, looking at both the conditions that govern the use of a name by a speaker and those involved in the correct interpretation of the name by her audience. The setting in which these conditions are investigated is provided by an extension of Discourse Representation Theory, MSDRT, in which mental states are represented as combinations of propositional attitudes and entity representations . The first half of the paper presents the (...) features of this framework that are needed to understand its application to the account of names that follows. N-labelled entity representations, where N is a proper name, play a pivotal part in this account: A speaker must have an N-labelled ER in order to be in a position to use a name N, and the interpreter must either have such a representation, or else construct one as part of his interpretation. The paper distinguishes different types of name uses in terms of what they presuppose about the role of N-labelled ERs on the side of the interpreter. (shrink)

The aim of this article is to investigate whether choosing the appropriate referring expression requires taking into account the hearer’s perspective, as is predicted under some versions of bidirectional Optimality Theory but is unexpected under other versions. We did this by comparing the results of 25 young and 25 elderly adults on an elicitation task based on eight different picture stories, and a comprehension task based on eight similar written stories. With respect to the elicitation task, we found that elderly (...) adults produce pronouns significantly more often than young adults when referring to the old topic in the presence of a new topic. With respect to the comprehension task, no significant differences were found between elderly and young adults. These results support the hypothesis that speakers optimize bidirectionally and take into account hearers when selecting a referring expression. If the use of a pronoun will lead to an unintended interpretation by the hearer, the speaker will use an unambiguous definite noun phrase instead. Because elderly adults are more limited in their processing capacities, as is indicated by their smaller working memory capacity, as speakers they will not always be able to reason about the hearer’s choices. As a result, they frequently produce non-recoverable pronouns. (shrink)

Inspired by early proposals in philosophy, dominant accounts of language posit a central role for mutual knowledge, either encoded directly in common ground, or approximated through other cognitive mechanisms. Using existing empirical evidence from language and memory, we challenge this tradition, arguing that mutual knowledge captures only a subset of the mental states needed to support communication. In a novel theoretical proposal, we argue for a cognitive architecture that includes separate, distinct representations of the self and other, and a cognitive (...) process that compares these representations continuously during conversation, outputting both similarities and differences in perspective. Our theory accounts for existing data, interfaces with findings from other cognitive domains, and makes novel predictions about the role of perspective in language use. We term this new account the Multiple Perspectives Theory of mental states in communication. (shrink)

Healey et al. use experiments with chat dialogues to test the hypothesis that language co‐ordination is driven by ‘running repairs’. They replace signals of understanding such as “okay” with weaker, ‘spoof’ signals like “ummm”, and replace specific requests for clarification like “on the left?” with signals that suggest a higher degree of misunderstanding like “what?”. The latter manipulation causes participants to switch rapidly to more abstract forms of referring expression.

The emergence of shared symbol systems is considered to be a pivotal moment in human evolution and human development. These changes are normally explained by reference to changes in people's internal cognitive processes. We present 2 experiments which provide evidence that changes in the external, collaborative processes that people use to communicate can also affect the structure and organization of symbol systems independently of cognitive change. We propose that mutual‐modifiability—opportunities for people to edit or manipulate each other's contributions—is a key (...) constraint on the emergence of complex symbol systems. We discuss the implications for models of language development and the origins of compositionality. (shrink)

The language we use over the course of conversation changes as we establish common ground and learn what our partner finds meaningful. Here we draw upon recent advances in natural language processing to provide a finer‐grained characterization of the dynamics of this learning process. We release an open corpus (>15,000 utterances) of extended dyadic interactions in a classic repeated reference game task where pairs of participants had to coordinate on how to refer to initially difficult‐to‐describe tangram stimuli. We find that (...) different pairs discover a wide variety of idiosyncratic but efficient and stable solutions to the problem of reference. Furthermore, these conventions are shaped by the communicative context: words that are more discriminative in the initial context (i.e., that are used for one target more than others) are more likely to persist through the final repetition. Finally, we find systematic structure in how a speaker's referring expressions become more efficient over time: Syntactic units drop out in clusters following positive feedback from the listener, eventually leaving short labels containing open‐class parts of speech. These findings provide a higher resolution look at the quantitative dynamics of ad hoc convention formation and support further development of computational models of learning in communication. (shrink)

This paper presents two cognitive models that simulate the production of referring expressions in the iMAP task—a task-oriented dialog. One general model is based on Dale and Reiter’s (1995)incremental algorithm, and the other is a simple template model that has a higher correlation with the data but is specifically geared toward the properties of the iMAP task. The property of the iMAP task environment that is modeled here is that the color feature is unreliable for identifying referents while other features (...) are reliable. The low computational cost of the incremental algorithm for generating referring expressions makes it an interesting starting point for a cognitive model. However, its explanatory power is limited, because it generates uniquely distinguishing referring expressions and because it considers features for inclusion in the referring expression in a fixed order. The first model extends the original incremental algorithm by an ability to adapt to feedback of whether a referring expression was used successfully, but it seems to overpredict the frequency with which distinguishing expressions are made and underpredict the frequency of overspecified referring expressions. The second model produces features for referring expressions purely based on its current estimate of a feature’s utility. Both models predict the observed human behavior of decreasing use of color terms and increasing use of useful feature terms. (shrink)

Human instructors often refer to objects and actions involved in a task description using both linguistic and non-linguistic means of communication. Hence, for robots to engage in natural human-robot interactions, we need to better understand the various relevant aspects of human multi-modal task descriptions. We analyse reference resolution to objects in a data collection comprising two object manipulation tasks and find that 78.76% of all referring expressions to the objects relevant in Task 1 are verbally underspecified and 88.64% of all (...) referring expressions are verbally underspecified in Task 2. The data strongly suggests that a language processing module for robots must be genuinely multi-modal, allowing for seamless integration of information transmitted in the verbal and the visual channel, whereby tracking the speaker’s eye gaze and gestures as well as object recognition are necessary preconditions. (shrink)

Intersubjectivity refers to the variety of possible relations between perspectives. It is indispensable for understanding human social behaviour. While theoretical work on intersubjectivity is relatively sophisticated, methodological approaches to studying intersubjectivity lag behind. Most methodologies assume that individuals are the unit of analysis. In order to research intersubjectivity, however, methodologies are needed that take relationships as the unit of analysis. The first aim of this article is to review existing methodologies for studying intersubjectivity. Four methodological approaches are reviewed: comparative self-report, (...) observing behaviour, analysing talk and ethnographic engagement. The second aim of the article is to introduce and contribute to the development of a dialogical method of analysis. The dialogical approach enables the study of intersubjectivity at different levels, as both implicit and explicit, and both within and between individuals and groups. The article concludes with suggestions for using the proposed method for researching intersubjectivity both within individuals and between individuals and groups. (shrink)

What role do speakers'/authors’ communicative intentions play in language interpretation? Cognitive scientists generally assume that listeners'/readers’ recognitions of speakers'/authors’ intentions is a crucial aspect of utterance interpretation. Various philosophers, literary theorists and anthropologists criticize this intentional view and assert that speakers'/authors’ intentions do not provide either the starting point for linguistic interpretation or constrain how texts should be understood. Until now, cognitive scientists have not seriously responded to the current challenges regarding intentions in communication. My purpose in this article is (...) to provide just such a response. I briefly describe some of the empirical evidence in cognitive science on the importance of intentions in communication. I then discuss some of the criticisms of the intentional view that have arisen in the humanities and social sciences. Following this, I offer a partial resolution to the intentionalist controversy that recognizes the differing views on the idea of linguistic interpretation. I specifically suggest a way of thinking about linguistic interpretation in terms of the time‐course of understanding that provides a metric for evaluating the role of intentions in communication. This time‐course perspective on linguistic understanding provides for the plurality of ideas that now exist both in cognitive science and in postmodern philosophical thought. (shrink)

Conventions are regularities in social behaviour of the past that enable us to coordinate our actions. Some conventions are lawlike: they are expected to be observed always or nearly always. However, in order to coordinate our actions, it may suffice that a precedent has occurred often enough, and sometimes even a single precedent will do. So, in general, conventions merely enable us to solve our coordination problems; lawlike conventions are a special case. Grammatical conventions are often lawlike; sense conventions are (...) typically enabling. In order to resolve the indeterminacies that sense conventions give rise to, interlocutors must rely on the common ground. In this and other ways, common ground is a prerequisite for convention‐based communication. (shrink)

It has been suggested that iconic graphical signs evolve into symbolic graphical signs through repeated usage. This article reports a series of interactive graphical communication experiments using a ‘pictionary’ task to establish the conditions under which the evolution might occur. Experiment 1 rules out a simple repetition based account in favor of an account that requires feedback and interaction between communicators. Experiment 2 shows how the degree of interaction affects the evolution of signs according to a process of grounding. Experiment (...) 3 confirms the prediction that those not involved directly in the interaction have trouble interpreting the graphical signs produced in Experiment 1. On the basis of these results, this article argues that icons evolve into symbols as a consequence of the systematic shift in the locus of information from the sign to the users' memory of the sign's usage supported by an interactive grounding process. (shrink)

A second-person perspective in neuroscience is particularly appropriate for the study of communication. We describe how the investigation of joint language tasks can contribute to our understanding of the mechanisms underlying interaction.

We examined how social cues and representational ones jointly shape people's spatial memory representations and their subsequent descriptions. In 24 pairs, Directors studied an array with a symmetrical structure while either knowing their Matcher's subsequent viewpoint or not. During the subsequent description of the array, the array's intrinsic structure was aligned with the Director, the Matcher, or neither partner. According to memory tests preceding descriptions, Directors who had studied the array while aligned with its structure were more likely to use (...) its orientation as an organizing direction. Directors who had studied the array while misaligned with its structure used its orientation more frequently as an organizing orientation when knowing that the Matcher would be aligned with it, but used their own viewpoint more frequently as an organizing direction when not knowing the Matcher's viewpoint. Directors also adapted their descriptions strategically, using more egocentric expressions when aligned with the intrinsic structure and more partner-centered expressions when their Matchers were the ones aligned with the structure, even when this information wasn't available in advance. These findings suggest that speakers are guided by converging social and representational cues to adapt flexibly the organization of their memories and the perspectives of their descriptions. (shrink)

Common‐sense intuition suggests that, when people are engaged in informational exchanges, they communicate so as to be reasonably sure that they perform the exchanges faithfully. Over the years, we have found evidence suggesting that this intuition, which is woven into several influential theories of human communication, may be misleading. We first summarize this evidence and discuss its potential limitations. Then, we present a new study that addresses the potential limitations. A confederate instructed participants to “pick up the skask” from a (...) tray containing six objects and move it to a specific location. Since skask is a non‐word invented by us, participants had to ask for clarification to perform the instruction faithfully. In contradiction with the intuition that people pursue faithfulness when engaged in informational exchanges, 29 of the 48 participants we tested performed the instruction without asking for clarification. We identified a possible cause for this behavior, which occurred more frequently when avoiding the clarification was unlikely to result in an overt consequence (an error in the execution of the instruction that could be noticed by the confederate or the experimenter). Other factors such as individual differences and the specific interpersonal dynamics of the experimental settings, if they played a role at all, did it to an extent that is unlikely to be comparable to that of the role played by overt consequences. Considered together, our various assessments of the extent to which people engage in faithful informational exchanges converge on a simple conclusion: Communicating faithfully is a substantially demanding task, and people often fail at it. We discuss the implications of this conclusion and speculate on its relevance for understanding the evolutionary past of human communication. (shrink)