How might a human communication system be bootstrapped in the absence of conventional language? We argue that motivated signs play an important role (i.e., signs that are linked to meaning by structural resemblance or by natural association). An experimental study is then reported in which participants try to communicate a range of pre-specified items to a partner using repeated non-linguistic vocalization, repeated gesture, or repeated non-linguistic vocalization plus gesture (but without using their existing language system). Gesture proved more effective (measured (...) by communication success) and more efficient (measured by the time taken to communicate) than non-linguistic vocalization across a range of item categories (emotion, object, and action). Combining gesture and vocalization did not improve performance beyond gesture alone. We experimentally demonstrate that gesture is a more effective means of bootstrapping a human communication system. We argue that gesture outperforms non-linguistic vocalization because it lends itself more naturally to the production of motivated signs. (shrink)

This paper compares two explanations of the process by which human communication systems evolve: iterated learning and social collaboration. It then reports an experiment testing the social collaboration account. Participants engaged in a graphical communication task either as a member of a community, where they interacted with seven different partners drawn from the same pool, or as a member of an isolated pair, where they interacted with the same partner across the same number of games. Participants’ horizontal, pair‐wise interactions led (...) “bottom up” to the creation of an effective and efficient shared sign system in the community condition. Furthermore, the community‐evolved sign systems were as effective and efficient as the local sign systems developed by isolated pairs. Finally, and as predicted by a social collaboration account, and not by an iterated learning account, interaction was critical to the creation of shared sign systems, with different isolated pairs establishing different local sign systems and different communities establishing different global sign systems. (shrink)

Through cultural transmission, repeated learning by new individuals transforms cultural information, which tends to become increasingly compressible . Existing diffusion chain studies include in their design two processes that could be responsible for this tendency: learning and reproducing . This paper manipulates the presence of learning in a simple iterated drawing design experiment. We find that learning seems to be the causal factor behind the increase in compressibility observed in the transmitted information, while reproducing is a source of random heritable (...) innovations. Only a theory invoking these two aspects of cultural learning will be able to explain human culture's fundamental balance between stability and innovation. (shrink)

In the last few years, researchers have begun to investigate the emergence of novel forms of human communication in the laboratory. I survey this growing line of research, which may be called experimental semiotics, from three distinct angles. First, I situate the new approach in its theoretical and historical context. Second, I review a sample of studies that exemplify experimental semiotics. Third, I present an empirical study that illustrates how the new approach can help us understand the socio‐cognitive underpinnings of (...) human communication. The main conclusion of the paper will be that, by reproducing micro samples of historical processes in the laboratory, experimental semiotics offers new powerful tools for investigating human communication as a form of joint action. (shrink)

This paper contrasts two influential theoretical accounts of language change and evolution – Iterated Learning and Social Coordination. The contrast is based on an experiment that compares drawings produced with Garrod et al’s ‘pictionary’ task with those produced in an Iterated Learning version of the same task. The main finding is that Iterated Learning does not lead to the systematic simplification and increased symbolicity of graphical signs produced in the standard interactive version of the task. A second finding is that (...) Iterated Learning leads to less conceptual and structural alignment between participants than observed for those in the interactive condition. The paper concludes with a comparison of the two accounts in relation to how each promotes signs that are effi cient, systematic and learnable. (shrink)

This paper explores the role of iconicity in spoken language and other human communication systems. First, we concentrate on graphical and gestural communication and show how semantically motivated iconic signs play an important role in creating such communication systems from scratch. We then consider how iconic signs tend to become simplified and symbolic as the communication system matures and argue that this process is driven by repeated interactive use of the signs. We then consider evidence for iconicity at the level (...) of the system in graphical communication and finally draw comparisons between iconicity in graphical and gestural communication systems and in spoken language. (shrink)

The amount of information conveyed by linguistic conventions depends on their precision, yet the codes that humans and other animals use to communicate are quite ambiguous: they may map several vague meanings to the same symbol. How does semantic precision evolve, and what are the constraints that limit it? We address this question using a multiplayer gaming app, where individuals communicate with one another in a scaled-up referential game. Here, the goal is for a sender to use black and white (...) symbols to communicate colors. We expected that the players’ mappings between symbols and colors would grow more specific over time, through a selection process whereby precise mappings are preferentially copied. We found that players become increasingly more precise in their use of symbols over the course of their interactions. This trend did not, however, result from selective copying of precise mappings. We explore the implications of this result for the study of lexical ambiguity, Zipf's Law of Meaning, and disagreements over semantic conventions. (shrink)

When people communicate, they coordinate a wide range of linguistic and non-linguistic behaviors. This process of coordination is called alignment, and it is assumed to be fundamental to successful communication. In this paper, we question this assumption and investigate whether disalignment is a more successful strategy in some cases. More specifically, we hypothesize that alignment correlates with task success only when communication is interactive. We present results from a spot-the-difference task in which dyads of interlocutors have to decide whether they (...) are viewing the same scene or not. Interactivity was manipulated in three conditions by increasing the amount of information shared between interlocutors. We use recurrence quantification analysis to measure the alignment between the scan-patterns of the interlocutors. We found that interlocutors who could not exchange feedback aligned their gaze more, and that increased gaze alignment correlated with decreased task success in this case. When feedback was possible, in contrast, interlocutors utilized it to better organize their joint search strategy by diversifying visual attention. This is evidenced by reduced overall alignment in the minimal feedback and full dialogue conditions. However, only the dyads engaged in a full dialogue increased their gaze alignment over time to achieve successful performances. These results suggest that alignment per se does not imply communicative success, as most models of dialogue assume. Rather, the effect of alignment depends on the type of alignment, on the goals of the task, and on the presence of feedback. (shrink)

We investigate the emergence of iconicity, specifically a bouba-kiki effect in miniature artificial languages under different functional constraints: when the languages are reproduced and when they are used communicatively. We ran transmission chains of participant dyads who played an interactive communicative game and individual participants who played a matched learning game. An analysis of the languages over six generations in an iterated learning experiment revealed that in the Communication condition, but not in the Reproduction condition, words for spiky shapes tend (...) to be rated by naive judges as more spiky than the words for round shapes. This suggests that iconicity may not only be the outcome of innovations introduced by individuals, but, crucially, the result of interlocutor negotiation of new communicative conventions. We interpret our results as an illustration of cultural evolution by random mutation and selection. (shrink)

Smith gives an excellent overview on research in language evolution, in which he discusses several recent models of how linguistic systems and the cognitive capacities involved in language learning may have co‐evolved. He illustrates how combined pressures on language learning and communication/use produce compositionally structured languages. Once in place, a (culturally transmitted) communication system creates new selection pressures on the capacity for acquiring these systems.

Production and comprehension have long been viewed as inseparable components of language. The study of vision, by contrast, has centered almost exclusively on comprehension. Here we investigate drawing—the most basic form of visual production. How do we convey concepts in visual form, and how does refining this skill, in turn, affect recognition? We developed an online platform for collecting large amounts of drawing and recognition data, and applied a deep convolutional neural network model of visual cortex trained only on natural (...) images to explore the hypothesis that drawing recruits the same abstract feature representations that support natural visual object recognition. Consistent with this hypothesis, higher layers of this model captured the abstract features of both drawings and natural images most important for recognition, and people learning to produce more recognizable drawings of objects exhibited enhanced recognition of those objects. These findings could explain why drawing is so effective for communicating visual concepts, they suggest novel approaches for evaluating and refining conceptual knowledge, and they highlight the potential of deep networks for understanding human learning. (shrink)

Many types of everyday and specialized reasoning depend on diagrams: we use maps to find our way, we draw graphs and sketches to communicate concepts and prove geometrical theorems, and we manipulate diagrams to explore new creative solutions to problems. The active involvement and manipulation of representational artifacts for purposes of thinking and communicating is discussed in relation to C.S. Peirce’s notion of diagrammatical reasoning. We propose to extend Peirce’s original ideas and sketch a conceptual framework that delineates different kinds (...) of diagram manipulation: Sometimes diagrams are manipulated in order to profile known information in an optimal fashion. At other times diagrams are explored in order to gain new insights, solve problems or discover hidden meaning potentials. The latter cases often entail manipulations that either generate additional information or extract information by means of abstraction. Ideas are substantiated by reference to ethnographic, experimental and historical examples. (shrink)

Recent research indicates that interpersonal communication is noisy, and that people exhibit considerable insensitivity to problems in communication. Using a dyadic referential communication task, the goal of which is accurate information transfer, this study examined the extent to which interlocutors are sensitive to problems in communication and use other‐initiated repairs (OIRs) to address them. Participants were randomly assigned to dyads (N = 88 participants, or 44 dyads) and tried to communicate a series of recurring abstract geometric shapes to a partner (...) across a text–chat interface. Participants alternated between directing (describing shapes) and matching (interpreting shape descriptions) roles across 72 trials of the task. Replicating prior research, over repeated social interactions communication success improved and the shape descriptions became increasingly efficient. In addition, confidence in having successfully communicated the different shapes increased over trials. Importantly, matchers were less confident on trials in which communication was unsuccessful, communication success was lower on trials that contained an OIR compared to those that did not contain an OIR, and OIR trials were associated with lower Director Confidence. This pattern of results demonstrates that (a) interlocutors exhibit (a degree of) sensitivity to problems in communication, (b) they appropriately use OIRs to address problems in communication, and (c) OIRs signal problems in communication. (shrink)

The language we use over the course of conversation changes as we establish common ground and learn what our partner finds meaningful. Here we draw upon recent advances in natural language processing to provide a finer‐grained characterization of the dynamics of this learning process. We release an open corpus (>15,000 utterances) of extended dyadic interactions in a classic repeated reference game task where pairs of participants had to coordinate on how to refer to initially difficult‐to‐describe tangram stimuli. We find that (...) different pairs discover a wide variety of idiosyncratic but efficient and stable solutions to the problem of reference. Furthermore, these conventions are shaped by the communicative context: words that are more discriminative in the initial context (i.e., that are used for one target more than others) are more likely to persist through the final repetition. Finally, we find systematic structure in how a speaker's referring expressions become more efficient over time: Syntactic units drop out in clusters following positive feedback from the listener, eventually leaving short labels containing open‐class parts of speech. These findings provide a higher resolution look at the quantitative dynamics of ad hoc convention formation and support further development of computational models of learning in communication. (shrink)

Experimental Semiotics (ES) is the study of novel forms of communication that communicators develop in laboratory tasks whose designs prevent them from using language. Thus, ES relates to pragmatics in a “pure,” radical sense, capturing the process of creating the relation between signs and their interpreters as biological, psychological, and social agents. Since such a creation of meaning-making from scratch is of central importance to language evolution research, ES has become the most prolific experimental approach in this field of research. (...) In our paper, we report the results of a study on the scope of recent ES and evaluate the ways in which it is relevant to the study of language origins. We coded for multiple levels across 13 dimensions related to the properties of the emergent communication systems or properties of the study designs, such as type of goal (coordination versus referential), modality of communication, absence or presence of turn-taking, or the presence of vertical vs. horizontal transmission. We discuss our findings and our classification, focusing on the advantages and limitations of those trends in ES, and in particular their ecological validity in the context of bootstrapping communication and the evolution of language. (shrink)

Human cognition and behavior are dominated by symbol use. This paper examines the social learning strategies that give rise to symbolic communication. Experiment 1 contrasts an individual-level account, based on observational learning and cognitive bias, with an inter-individual account, based on social coordinative learning. Participants played a referential communication game in which they tried to communicate a range of recurring meanings to a partner by drawing, but without using their conventional language. Individual-level learning, via observation and cognitive bias, was sufficient (...) to produce signs that became increasingly effective, efficient, and shared over games. However, breaking a referential precedent eliminated these benefits. The most effective, most efficient, and most shared signs arose when participants could directly interact with their partner, indicating that social coordinative learning is important to the creation of shared symbols. Experiment 2 investigated the contribution of two distinct aspects of social interaction: behavior alignment and concurrent partner feedback. Each played a complementary role in the creation of shared symbols: Behavior alignment primarily drove communication effectiveness, and partner feedback primarily drove the efficiency of the evolved signs. In conclusion, inter-individual social coordinative learning is important to the evolution of effective, efficient, and shared symbols. (shrink)

An ideography is a general-purpose code made of pictures that do not encode language, which can be used autonomously – not just as a mnemonic prop – to encode information on a broad range of topics. Why are viable ideographies so hard to find? I contend that self-sufficient graphic codes need to be narrowly specialized. Writing systems are only an apparent exception: At their core, they are notations of a spoken language. Even if they also encode nonlinguistic information, they are (...) useless to someone who lacks linguistic competence in the encoded language or a related one. The versatility of writing is thus vicarious: Writing borrows it from spoken language. Why is it so difficult to build a fully generalist graphic code? The most widespread answer points to a learnability problem. We possess specialized cognitive resources for learning spoken language, but lack them for graphic codes. I argue in favor of a different account: What is difficult about graphic codes is not so much learning or teaching them as getting every user to learn and teach the same code. This standardization problem does not affect spoken or signed languages as much. Those are based on cheap and transient signals, allowing for easy online repairing of miscommunication, and require face-to-face interactions where the advantages of common ground are maximized. Graphic codes lack these advantages, which makes them smaller in size and more specialized. (shrink)

Recent work on the emergence and evolution of human communication has focused on getting novel systems to evolve from scratch in the laboratory. Many of these studies have adopted an interactive construction approach, whereby pairs of participants repeatedly interact with one another to gradually develop their own communication system whilst engaged in some shared task. This paper describes four recent studies that take a different approach, showing how adaptive structure can emerge purely as a result of cultural transmission through single (...) chains of learners. By removing elements of interactive communication and focusing only on the way in which language is repeatedly acquired by learners, we hope to gain a better understanding of how useful structural properties of language could have emerged without being intentionally designed or innovated. (shrink)

Cognitive Science, Volume 46, Issue 10, October 2022.

Human communication is often thought about in terms of transmitted messages in a conventional code like a language. But communication requires a specialized interactive intelligence. Senders have to be able to perform recipient design, while receivers need to be able to do intention recognition, knowing that recipient design has taken place. To study this interactive intelligence in the lab, we developed a new task that taps directly into the underlying abilities to communicate in the absence of a conventional code. We (...) show that subjects are remarkably successful communicators under these conditions, especially when senders get feedback from receivers. Signaling is accomplished by the manner in which an instrumental action is performed, such that instrumentally dysfunctional components of an action are used to convey communicative intentions. The findings have important implications for the nature of the human communicative infrastructure, and the task opens up a line of experimentation on human communication. (shrink)

This article explains how people synchronize their thoughts through empathetic relationships and points out the elementary neuronal mechanisms orchestrating this process. The many dimensions of empathy are discussed, as is the manner by which empathy affects health and disorders. A case study of teaching children empathy, with positive results, is presented. Mirror neurons, the recently discovered mechanism underlying empathy, are characterized, followed by a theory of brain-to-brain coupling. This neuro-tuning, seen as a kind of synchronization between brains and between individuals, (...) takes various forms, including frequency aspects of language use and the understanding that develops regardless of the difference in spoken tongues. Going beyond individual-to-individual empathy and SYNC, the article explores the phenomenon of synchronization in groups and points out how synchronization increases group cooperation and performance. (shrink)

In the absence of pre‐established communicative conventions, people create novel communication systems to successfully coordinate their actions toward a joint goal. In this study, we address two types of such novel communication systems: sensorimotor communication, where the kinematics of instrumental actions are systematically modulated, versus symbolic communication. We ask which of the two systems co‐actors preferentially create when aiming to communicate about hidden object properties such as weight. The results of three experiments consistently show that actors who knew the weight (...) of an object transmitted this weight information to their uninformed co‐actors by systematically modulating their instrumental actions, grasping objects of particular weights at particular heights. This preference for sensorimotor communication was reduced in a fourth experiment where co‐actors could communicate with weight‐related symbols. Our findings demonstrate that the use of sensorimotor communication extends beyond the communication of spatial locations to non‐spatial, hidden object properties. (shrink)

Following a synthesis of naturalistic and experimental studies of language creation, we propose a theoretical model that describes the process through which human communication systems might arise and evolve. Three key processes are proposed that give rise to effective, efficient and shared human communication systems: motivated signs that directly resemble their meaning facilitate cognitive alignment, improving communication success; behavioral alignment onto an inventory of shared sign-to-meaning mappings bolsters cognitive alignment between interacting partners; sign refinement, through interactive feedback, enhances the efficiency (...) of the evolving communication system. By integrating the findings across a range of diverse studies, we propose a theoretical model of the process through which the earliest human communication systems might have arisen and evolved. Importantly, because our model is not bound to a single modality it can describe the creation of shared sign systems across a range of contexts, informing theories of language creation and evolution. (shrink)

Arbitrary communication systems can emerge from iconic beginnings through processes of conventionalisation via interaction. Here, we explore whether this process of conventionalisation occurs with continuous, auditory signals. We conducted an artificial signalling experiment. Participants either created signals for themselves, or for a partner in a communication game. We found no evidence that the speech-like signals in our experiment became less iconic or simpler through interaction. We hypothesise that the reason for our results is that when it is difficult to be (...) iconic initially because of the constraints of the modality, then iconicity needs to emerge to enable grounding before conventionalisation can occur. Further, pressures for discrimination, caused by the expanding meaning space in our study, may cause more complexity to emerge, again as a result of the restrictive signalling modality. Our findings have possible implications for the processes of conventionalisation possible in signed and spoken languages, as the spoken modality is more restrictive than the manual modality. (shrink)

Communication is a fundamentally interactive phenomenon. Evolutionary biology recognises this fact in its definition of communication, in which signals are those actions that cause reactions, and where both action and reaction are designed for that reason. Where only one or the other is designed then the behaviours are classed as either cues or coercion. Since mutually dependent behaviours are unlikely to emerge simultaneously, the symmetry inherent in these definitions gives rise to a prediction that communication will only emerge if cues (...) or coercive behaviours do so first. They will then be co-opted for communication. A range of case studies, from animal signalling, evolutionary robotics, comparative psychology, and evolutionary linguistics are used to test this prediction. The first three are found to be supportive. However in the Embodied Communication Game, a recent experimental approach to the emergence of communication between adult humans, communication emerges even when cues or coerced behaviours are not possible. This suggests that humans are exceptional in this regard. It is argued that the reason for this is the degree to which we are able and compelled to read and interpret the behaviour of others in intentional terms. (shrink)

Coordination between speakers in dialogue requires balancing repetition and change, the old and the new. Interlocutors tend to reuse established forms, relying on communicative precedents. Yet linguistic interaction also necessitates adaptation to changing contexts or dynamic tasks, which might favor abandoning existing precedents in favor of better communicative alternatives. We explored this tension using a maze game task in which individual participants and interacting pairs had to describe figures and their positions in one of two possible maze types: a regular (...) maze, in which the grid‐like structure of the maze is highlighted, and an irregular maze, in which specific parts of the maze are salient. Participants repeated this task several times. Both individuals and interacting pairs were affected by the different maze layouts, initially using more idiosyncratic description schemes for irregular mazes and more systematic schemes for regular mazes. Interacting pairs, but not individuals, abandoned their unsystematic initial descriptions in favor of a more systematic approach, which was better adapted for repeated interaction. Our results show communicative conventions are initially shaped by context, but interaction opens up the possibility for change if better alternatives are available. (shrink)

Cocreating meaning in collaboration is challenging. Success is often determined by people's abilities to coordinate their language to converge upon shared mental representations. Here we explore one set of low‐level linguistic behaviors, linguistic alignment, that both emerges from, and facilitates, outcomes of high‐level convergence. Linguistic alignment captures the ways people reuse, that is, “align to,” the lexical, syntactic, and semantic forms of others' utterances. Our focus is on the temporal change of multi‐level linguistic alignment, as well as how alignment is (...) related to communicative outcomes within a unique collaborative problem‐solving paradigm. The primary task, situated within a virtual educational video game, requires creative thinking between three people where the paths for possible solutions are highly variable. We find that over time interactions are marked by decreasing lexical and syntactic alignment, with a trade‐off of increasing semantic alignment. However, greater semantic alignment does not translate into better team performance. Overall, these findings provide greater clarity on the role of linguistic coordination within complex and dynamic collaborative problem‐solving tasks. (shrink)

This response takes advantage of the diverse and wide-ranging series of commentaries to clarify some aspects of the target article, and flesh out other aspects. My central point is a plea to take graphic codes seriously as codes, rather than as a kind of visual art or as a byproduct of spoken language; only in this way can the puzzle of ideography be identified and solved. In this perspective, I argue that graphic codes do not derive their expressive power from (...) iconicity alone (unlike visual arts), and I clarify the peculiar relationship that ties writing to spoken language. I then discuss three possible solutions to the puzzle of ideography. I argue that a learning account still cannot explain why ideographies fail to evolve, even if we emancipate the learning account from the version that Liberman put forward; I develop my preferred solution, the “standardization account,” and contrast it with a third solution suggested by some commentaries, which says that ideographies do not evolve because they would make communication too costly. I consider, by way of conclusion, the consequences of these views for the future evolution of ideography. (shrink)

Interactions between humans and bots are increasingly common online, prompting some legislators to pass laws that require bots to disclose their identity. The Turing test is a classic thought experiment testing humans’ ability to distinguish a bot impostor from a real human from exchanging text messages. In the current study, we propose a minimal Turing test that avoids natural language, thus allowing us to study the foundations of human communication. In particular, we investigate the relative roles of conventions and reciprocal (...) interaction in determining successful communication. Participants in our task could communicate only by moving an abstract shape in a 2D space. We asked participants to categorize their online social interaction as being with a human partner or a bot impostor. The main hypotheses were that access to the interaction history of a pair would make a bot impostor more deceptive and interrupt the formation of novel conventions between the human participants. Copying their previous interactions prevents humans from successfully communicating through repeating what already worked before. By comparing bots that imitate behavior from the same or a different dyad, we find that impostors are harder to detect when they copy the participants’ own partners, leading to less conventional interactions. We also show that reciprocity is beneficial for communicative success when the bot impostor prevents conventionality. We conclude that machine impostors can avoid detection and interrupt the formation of stable conventions by imitating past interactions, and that both reciprocity and conventionality are adaptive strategies under the right circumstances. Our results provide new insights into the emergence of communication and suggest that online bots mining personal information, for example, on social media, might become indistinguishable from humans more easily. (shrink)

Despite obvious advantages, no generalised ideographic codes have evolved through cultural evolution to rely on iconicity. Morin suggests that this is because of missing means of standardisation, which glottographic codes get from natural languages. Although we agree, we also point to the important role of the available media, which might support some forms of reference more effectively than others.