When people talk, they gesture. Gesture is a fundamental component of language that contributes meaningful and unique information to a spoken message and reflects the speaker’s underlying knowledge and experiences. Theoretical perspectives of speech and gesture propose that they share a common conceptual origin and have a tightly integrated relationship, overlapping in time, meaning, and function to enrich the communicative context. We review a robust literature from the field of psychology documenting the benefits of gesture for communication for both speakers (...) and listeners, as well as its important cognitive functions for organizing spoken language, and facilitating problem-solving, learning, and memory. Despite this evidence, gesture has been relatively understudied in populations with neurogenic communication disorders. While few studies have examined the rehabilitative potential of gesture in these populations, others have ignored gesture entirely or even discouraged its use. We review the literature characterizing gesture production and its role in intervention for people with aphasia, as well as describe the much sparser literature on gesture in cognitive communication disorders including right hemisphere damage, traumatic brain injury, and Alzheimer’s disease. The neuroanatomical and behavioral profiles of these patient populations provide a unique opportunity to test theories of the relationship of speech and gesture and advance our understanding of their neural correlates. This review highlights several gaps in the field of communication disorders which may serve as a bridge for applying the psychological literature of gesture to the study of language disorders. Such future work would benefit from considering theoretical perspectives of gesture and using more rigorous and quantitative empirical methods in its approaches. We discuss implications for leveraging gesture to explore its untapped potential in understanding and rehabilitating neurogenic communication disorders. (shrink)

The present study investigates the types of verb and symbolic representational strategies used by 10 deaf signing adults and 13 deaf signing children who described in Italian Sign Language 45 video clips representing nine action types generally communicated by five general verbs in spoken Italian. General verbs, in which the same sign was produced to refer to several different physical action types, were rarely used by either group of participants. Both signing children and adults usually produced specific depicting predicates by (...) incorporating, through a representational strategy, the object and/or the modality of the action into the sign. As for the different types of representational strategies, the adults used the hand-as-object strategy more frequently than the children, who, in turn, preferred to use the hand-as-hand strategy, suggesting that different degrees of cognitive complexity are involved in these two symbolic strategies. Addressing the symbolic iconic strategies underlying sign formation could provide new insight into the perceptual and cognitive processes of linguistic meaning construction. The findings reported here support two main assumptions of cognitive linguistics applied to sign languages: there is a strong continuity between gestures and language; lexical units and depicting constructions derive from the same iconic core mechanism of sign creation. (shrink)

There is plenty of evidence that speech and gesture form a tightly integrated system, as reflected in parallelisms in language production, comprehension, and development ( McNeill, 1992 ; Kendon, 2004 ). Yet, it is a common assumption that speakers use gestures to compensate for their expressive difficulties, a notion found in developmental studies of both first and second language acquisition, and in theoretical proposals concerning the gesture-speech relationship. If gestures are compensatory, they should mainly occur in disfluent stretches of speech. (...) However, the evidence is sparse and conflicting. This study extends previous studies and tests the putative compensatory role of gestures by comparing the gestural behavior in fluent vs. disfluent stretches of narratives by competent speakers in two languages (Dutch and Italian), and by language learners (children and adult L2 learners). The results reveal that (1) in all groups speakers overwhelmingly produce gestures during fluent speech and only rarely during disfluencies. However, L2 learners are significantly more likely to gesture in disfluency than the other groups; (2) in all groups gestures during disfluencies tend to be holds; (3) in all groups the rare gestures completed in disfluencies have both referential and pragmatic functions. Overall, the data strongly suggest that when speech stops, so does gesture. The findings constitute an important challenge to both gesture and language acquisition theories assuming a mainly (lexical) compensatory role for (referential) gestures. Instead, the results provide strong support for the notion that speech and gestures form an integrated system. (shrink)

ASL can express plurals by repeating a noun, in an unpunctuated fashion, in different parts of signing space. We argue that this construction may come with a rich iconic component: the geometric arrangement of the repetitions provides information about the arrangement of the denoted plurality; in addition, the number and speed of the repetitions provide information about the size of the denoted plurality. Interestingly, the shape of the repetitions may introduce a new singular discourse referent when a vertex can be (...) inferred to denote a singular object. Thus one may point towards the first or last iteration of a horizontal repetition of BOOK to denote the left- or right-edge of the corresponding row. This yields a remarkable interaction between iconic semantics and standard logical semantics. We show that our analysis extends to ‘punctuated’ repetitions, which involve clearly individuated iterations of a singular noun. While these may initially look like coordinated indefinites, they are better handled by the same iconic framework as plural, unpunctuated repetitions. Some repetition-based mass terms also give rise to iconic effects, and to different readings depending on whether the repetition is continuous, unpunctuated, or punctuated. Our analysis highlights the need for a formal semantics with iconicity to study the integration of such iconic and logical conditions. It also raises a question: can similar facts be found in spoken language when gestures are taken into account? We suggest that several effects can be replicated, especially when one considers examples involving ‘pro-speech gestures’. (shrink)

La plupart des auteur-es ayant abordé le problème de l'extension du cognitif, tel qu'il a émergé des débats autour de la thèse de l'esprit étendu, ont supposé que cette extension devait prendre la forme d'un espace régulier, qui peut être ceint par des frontières. Cependant, la littérature en question ne traite pas explicitement de cette supposition, de sorte que, malgré son influence, il n'y a pas d'évaluation de sa véracité ou de sa légitimité. Dans ce mémoire, cette hypothèse est remise (...) en question, et nous prenons sa contrepartie pour hypothèse de recherche, à savoir que, malgré l'assurance des auteur-es qui l'épousent, il est tout à fait légitime d'explorer des propositions de caractérisation du cognitif qui aboutissent à d'autres formes d'extension. Pour démontrer notre hypothèse (H1), nous posons deux autres hypothèses : (H2) les auteur-es qui épousent la supposition selon laquelle l'esprit doit s'étendre sur un espace ceint par des frontières le font sur la base de l'intuition individualiste, et (H3) l'intuition individualiste, malgré certaines apparences, ne permet pas de justifier cette supposition. Nous démontrons H2 à l'aide d'une analyse de texte faite sur le corpus des auteur-es ayant abordé la question de l'extension du cognitif dans la foulée des débats sur l'esprit étendu, et nous montrons que la littérature en sciences cognitives et en philosophie de l'esprit nous donne des raisons de croire que H3 est vraie. Ensemble, H2 et H3 justifient H1. (shrink)

Vocal theories of the origin of language rarely make a case for the precursor functions that underlay the evolution of speech. The vocal expression of emotion is unquestionably the best candidate for such a precursor, although most evolutionary models of both language and speech ignore emotion and prosody altogether. I present here a model for a joint prosodic precursor of language and music in which ritualized group-level vocalizations served as the ancestral state. This precursor combined not only affective and intonational (...) aspects of prosody, but also holistic and combinatorial mechanisms of phrase generation. From this common stage, there was a bifurcation to form language and music as separate, though homologous, specializations. This separation of language and music was accompanied by their (re)unification in songs with words. (shrink)

Film and television portrayals of posthuman cyborgs melding biology and technology, simultaneously “animal and machine” abound. Most of us immediately think of iconic characters like Arnold Schwarzenegger’s relentless cyborg assassin in the Terminator series or Peter Weller’s crime-fighting cyborg police officer in Robocop (1987). Or perhaps we recall the many cyborgs populating the Dr. Who, Star Trek, and Star Wars television series and films—including Darth Vader, surely the most famous cinematic cyborg of all time. But lesser-known explorations of cybernetic embodiment (...) have appeared in film and television for many decades. And not all portrayals involve the sort of extreme transformations exemplified by these iconic characters. This chapter considers some of different ways that film and television have explored the transformative relation between embodiment and technology. (shrink)

Building on Elliot and SilvermanÕs (2015) embodied and enactive approach to musicing, I argue for an extended approach: namely, the idea that music can function as an environmental scaffolding supporting the development of various experiences and embodied practices that would otherwise remain inaccessible. I focus especially on the materiality of music. I argue that one of the central ways we use music, as a material resource, is to manipulate social spaceÑand in so doing, manipulate our emotions. Acts of musicing, thought (...) of as processes of environmental space manipulation, are thus examples of what I term Òemotional niche construction.Ó I explore three dimensions of this process and appeal to different strands of empirical work to support this picture. (shrink)

What does it mean to be an aesthetic beholder? Is it different than simply being a perceiver? Most theories of aesthetic perception focus on 1) features of the perceived object and its presentation or 2) on psychological evaluative or emotional responses and intentions of perceiver and artist. In this chapter I propose that we need to look at the process of engaged perception itself, and further that this temporal process of be- coming a beholder must be understood in its embodied, (...) contextual and dynamic speci- ficity. Through both phenomenological and neuroscientific explorations I analyze what is characteristic about a more “aesthetic stance” and argue that there is a certain asym- metry between beholder and beheld, which has to do with a disengagement of goal- directed action, and which allows for other kinds of perceptual involvement than in a more “practical stance”. It is a multi-disciplinary project integrating a sensorimotor no- tion of aesthetic affordances, 18th century philosophy, and large-scale brain network findings. What ensues is a new dynamic framework for future empirical and theoretical research on aesthetic perception. (shrink)

For humans, the ability to communicate and use language is instantiated not only in the vocal modality but also in the visual modality. The main examples of this are sign languages and gestures. Sign languages, the natural languages of Deaf communities, use systematic and conventionalized movements of the hands, face, and body for linguistic expression. Co-speech gestures, though non-linguistic, are produced in tight semantic and temporal integration with speech and constitute an integral part of language together with speech. The articles (...) in this issue explore and document how gestures and sign languages are similar or different and how communicative expression in the visual modality can change from being gestural to grammatical in nature through processes of conventionalization. As such, this issue contributes to our understanding of how the visual modality shapes language and the emergence of linguistic structure in newly developing systems. Studying the relationship between signs and gestures provides a new window onto the human ability to recruit multiple levels of representation in the service of using or creating conventionalized communicative systems. (shrink)

A review of 'The Hand, an Organ of the Mind: What the Manual Tells the Mental', ed. Zdravko Radman.

Cognitive research on metaphoric concepts of time has focused on differences between moving Ego and moving time models, but even more basic is the contrast between Ego‐ and temporal‐reference‐point models. Dynamic models appear to be quasi‐universal cross‐culturally, as does the generalization that in Ego‐reference‐point models, FUTURE IS IN FRONT OF EGO and PAST IS IN BACK OF EGO. The Aymara language instead has a major static model of time wherein FUTURE IS BEHIND EGO and PAST IS IN FRONT OF EGO; (...) linguistic and gestural data give strong confirmation of this unusual culture‐specific cognitive pattern. Gestural data provide crucial information unavailable to purely linguistic analysis, suggesting that when investigating conceptual systems both forms of expression should be analyzed complementarily. Important issues in embodied cognition are raised: how fully shared are bodily grounded motivations for universal cognitive patterns, what makes a rare pattern emerge, and what are the cultural entailments of such patterns? (shrink)

This paper offers a biosemiotic account of the poetic aspects of gesture and speech in schizophrenia. The argument is that speech and gesture are not the mere expression of pre-verbal thoughts. Instead, meaning is enacted by the temporal and semantic coordination of speech and gesture. The bodily basis of language is highlighted by the fact that, failing to create language that is organized around topics, individuals with schizophrenia often rely on poetic associations in directing their utterances. Accordingly, the analysis of (...) schizophrenic speech and gesture based on McNeill’s Growth Point theory is enriched with reference to both Cowley’s (2007b) views on language as distributed social coordination, and Alexander’s (Biosemiotics 2(1):77–100, 2009) description of formal causes in nature. Schizophrenic language can be seen as a hypertrophied manifestation of the discourse-scaffolding function of poetic associations in speech and gesture. The analysis of schizophrenic language shows that language behavior need not be built on pre-verbal thoughts and important aspects of schizophrenic language, such as metalinguistic awareness and semiotic agency, can be clarified by applying concepts from biosemiotics. (shrink)

Recently, in cognitive science, the enactivist account of cognition has been gaining ground, due in part to studies of movement in conjunction with thought. The idea, as Noë , has put it, that “cognition is not something happening inside us or to us, but it’s something we do, something we achieve,” is increasingly supported by research on joint attention, movement coordination, and gesture. Not surprisingly, therefore, enactivists have also begun to look at “movement specialists”—dancers—for both scientific and phenomenological accounts of (...) thinking with and through movement. In this paper, I argue that a serious exploration of dance and movement does not merely bolster the enactivist view, but rather, it suggests a radical enactivism, as envisaged by, e.g., Hutto . To support this claim, I examine an account of “Thinking in Movement” provided by Maxine Sheets-Johnstone in order to highlight the ways in which intentional agency and meaning-making occur in improvisational dance. These processes, I further argue, closely mirror some of the key components of participatory sense making, as described by De Jaegher and Di Paolo :485–507, 2007). This is beneficial to my case, because it permits a discussion of “thought-full action” that does not depend upon standard cognitivist frameworks for explanation. By carefully focusing on how agency can help to separate mere “thrashing about” from meaningful movement, this paper aim to strengthen the position of radical enactivism from the unique perspective and dance and sense-making. (shrink)

From the late 1950s until 1975, cognition was understood mainly as disembodied symbol manipulation in cognitive psychology, linguistics, artificial intelligence, and the nascent field of Cognitive Science. The idea of embodied cognition entered the field of Cognitive Linguistics at its beginning in 1975. Since then, cognitive linguists, working with neuroscientists, computer scientists, and experimental psychologists, have been developing a neural theory of thought and language (NTTL). Central to NTTL are the following ideas: (a) we think with our brains, that is, (...) thought is physical and is carried out by functional neural circuitry; (b) what makes thought meaningful are the ways those neural circuits are connected to the body and characterize embodied experience; (c) so-called abstract ideas are embodied in this way as well, as is language. Experimental results in embodied cognition are seen not only as confirming NTTL but also explained via NTTL, mostly via the neural theory of conceptual metaphor. Left behind more than three decades ago is the old idea that cognition uses the abstract manipulation of disembodied symbols that are meaningless in themselves but that somehow constitute internal “representations of external reality” without serious mediation by the body and brain. This article uniquely explains the connections between embodied cognition results since that time and results from cognitive linguistics, experimental psychology, computational modeling, and neuroscience. (shrink)

The aim of this essay is to situate the thought of Tetsurō Watsuji within contemporary approaches to social cognition. I argue for Watsuji’s current relevance, suggesting that his analysis of embodiment and social space puts him in step with some of the concerns driving ongoing treatments of social cognition in philosophy of mind and cognitive science. Yet, as I will show, Watsuji can potentially offer a fruitful contribution to this discussion by lending a phenomenologically informed critical perspective. This is because (...) Watsuji challenges the internalist and cognitivist presuppositions informing the currently dominant “Theory of Mind” paradigm that is driving much social cognition research. Additionally, I show that Watsuji’s alternative model is not merely confined to the realm of phenomenological description but that it also receives robust empirical support from a number of different sources. I thus hope to open up aspects of Watsuji’s thinking that have yet to be fully appreciated. (shrink)

In the near future, our life will normally be surrounded with fairly complicated artifacts, enabled by the autonomous robot and brain–machine interface technologies. In this paper, we argue that what we call the responsibility flaw problem and the inappropriate use problem need to be overcome in order for us to benefit from complicated artifacts. In order to solve these problems, we propose an approach to endowing artifacts with an ability of socially communicating with other agents based on the artifact-as-a-half-mirror metaphor. (...) The idea is to have future artifacts behave according to the hybrid intention composed of the owner’s intention and the social rules. We outline the approach and discuss its feasibility together with preliminary work. (shrink)

Depictive expressions of thought predate written language by thousands of years. They have evolved in communities through a kind of informal user testing that has refined them. Analyzing common visual communications reveals consistencies that illuminate how people think as well as guide design; the process can be brought into the laboratory and accelerated. Like language, visual communications abstract and schematize; unlike language, they use properties of the page (e.g., proximity and place: center, horizontal/up–down, vertical/left–right) and the marks on it (e.g., (...) dots, lines, arrows, boxes, blobs, likenesses, symbols) to convey meanings. The visual expressions of these meanings (e.g., individual, category, order, relation, correspondence, continuum, hierarchy) have analogs in language, gesture, and especially in the patterns that are created when people design the world around them, arranging things into piles and rows and hierarchies and arrays, spatial-abstraction-action interconnections termed spractions. The designed world is a diagram. (shrink)

Cognitive ecology is the study of cognitive phenomena in context. In particular, it points to the web of mutual dependence among the elements of a cognitive ecosystem. At least three fields were taking a deeply ecological approach to cognition 30 years ago: Gibson’s ecological psychology, Bateson’s ecology of mind, and Soviet cultural-historical activity theory. The ideas developed in those projects have now found a place in modern views of embodied, situated, distributed cognition. As cognitive theory continues to shift from units (...) of analysis defined by inherent properties of the elements to units defined in terms of dynamic patterns of correlation across elements, the study of cognitive ecosystems will become an increasingly important part of cognitive science. (shrink)

Nell’ultimo trentennio, le scienze cognitive hanno proposto una teoria alternativa a quelle che intendevano la metafora come strumento linguistico, cioè che il processo metaforico si potesse ridurre al livello letterale, semantico o pragmatico. Secondo la teoria della metafora concettuale, la metafora è un modo di rappresentare ed organizzare il nostro mondo, piuttosto che uno strumento semplicemente decorativo del linguaggio avente un ruolo puramente comunicativo. Questo shift paradigmatico ha influenzato anche altri aspetti delle scienze cognitive. In questo contributo si vuole delineare (...) lo stato attuale della teoria esposta da Lakoff e Johnson e la maturazione del pensiero rispetto alla prima pubblicazione di Metaphors We Live By (1980/1998). Dopo avere illustrato i principi teorici, si daranno degli esempi di metafore culturali e multimodali e si puntualizzerà il ruolo analogo, ma distinto, alla metafora che la metonimia copre nell’ambito dei nostri sistemi concettuali. (shrink)

Poverty of the Stimulus Arguments have convinced many linguists and philosophers of language that a domain specific language acquisition device (LAD) is necessary to account for language learning. Here we review empirical evidence that casts doubt on the necessity of this domain specific device. We suggest that more attention needs to be paid to the early stages of language acquisition. Many seemingly innate language-related abilities have to be learned over the course of several months. Further, the language input contains rich (...) stochastic information that can be accessed by domain-general learning mechanisms. Computer simulation has shown how mechanisms that are not domain specific can exploit the information contained in language. We conclude that (i) Poverty of the Stimulus Arguments need to be conceptually clarified and (ii) more empirical research needs to be carried out before we can rule out that data driven general purpose mechanisms can account for language learning. (shrink)

While language is presumably unique to humans, there are possible pre-linguistic features that developed in the course of human evolution which predate features of language, and might have even been essential for its evolution. A number of such possible preadaptations for human language have been discussed, like the permanent lowering of the larynx, the ability to control one’s breath, or the inclination of humans to imitate. In this paper I would like to point out another candidate for a preadaptation, namely (...) the functional differentiation of the hands and the way in which they cooperate in manual actions. (shrink)

This paper offers a novel argument against the phenomenal intentionality thesis (or PIT for short). The argument, which I'll call the extended mind argument against phenomenal intentionality, is centered around two claims: the first asserts that some source intentional states extend into the environment, while the second maintains that no conscious states extend into the environment. If these two claims are correct, then PIT is false, for PIT implies that the extension of source intentionality is predicated upon the extension of (...) phenomenal consciousness. The argument is important because it undermines an increasingly prominent account of the nature of intentionality. PIT has entered the philosophical mainstream and is now a serious contender to naturalistic views of intentionality like the tracking theory and the functional role theory (Loar 1987, 2003; Searle 1990; Strawson 1994; Horgan and Tienson 2002; Pitt 2004; Farkas 2008; Kriegel 2013; Montague 2016; Bordini 2017; Forrest 2017; Mendelovici 2018). The extended mind argument against PIT challenges the popular sentiment that consciousness grounds intentionality. (shrink)

This study investigates whether there is a relation between the semantics of linguistic expressions that indicate temporal distance and the spatial properties of their co-speech gestures. To this date, research on time gestures has focused on features such as gesture axis, direction, and shape. Here we focus on a gesture property that has been overlooked so far: the distance of the gesture in relation to the body. To achieve this, we investigate two types of temporal linguistic expressions are addressed: proximal (...) (e.g., near future, near past) and distal (e.g., distant past, distant future). Data was obtained through the NewsScape library, a multimodal corpus of television news. A total of 121 co-speech gestures were collected and divided into the two categories. The gestures were later annotated in terms of gesture space and classified in three categories: (i) center, (ii) periphery, and (iii) extreme periphery. Our results suggest that gesture and language are coherent in the expression of temporal distance: when speakers locate an event far from them, they tend to gesture further from their body; similarly, when locating an event close to them, they gesture closer to their body. These results thus reveal how co-speech gestures also reflect a space-time mapping in the dimension of distance. (shrink)

Denis Noble has produced a succinct analysis of the ‘Illusions of the Modern Synthesis’. At the heart of the matter is the place of agency in organisms. This paper examines the nature of conscious agent action in organisms, and the role of affects in shaping agent choice. It examines the dual role these have in shaping evolution, and in the social worlds of scientists that shape evolutionary theory. Its central claim follows Noble, that agency is central to the structure of (...) organisms, and raises careful consideration for the role animal agency and affective evaluations in biology, and in biologists. (shrink)

The framework of depicting put forward byClark (2016)offers a schematic vantage point from which to examine iconic language use. Confronting the framework with empirical data, we consider some of its key theoretical notions. Crucially, by reconceptualizing the typology of depictions, we identify an overlooked domain in the literature: “speech-embedded nonverbal depictions,” namely cases where meaning is communicated iconically, nonverbally, and without simultaneously co-occurring speech. In addition to contextualizing the phenomenon in relation to existing research, we demonstrate, with examples from American (...) TV talk shows, how such depictions function in real-life language use, offering a brief sketch of their complexities and arguing also for their theoretical significance. (shrink)

When learners self-explain, they try to make sense of new information. Although research has shown that bodily actions and written notes are an important part of learning, previous analyses of self-explanations rarely take into account written and non-verbal data produced spontaneously. In this paper, the extent to which interpretations of self-explanations are influenced by the systematic consideration of such data is investigated. The video recordings of 33 undergraduate students, who learned with worked-out examples dealing with complex numbers, were categorized successively (...) including three different data bases: verbal data, verbal and written data, and verbal, written and non-verbal data. Results reveal that including written data and non-verbal data leads to a more accurate analysis of self-explanations than an analysis solely based on verbal data. This influence is even stronger for the categorization of self-explanations as adequate or inadequate. (shrink)

Many studies have documented a close timing relationship between speech prosody and co-speech gesture, but some studies have not, and it is unclear whether these differences in speech-gesture alignment are due to different speaking tasks, different target gesture types, different prosodic elements, different definitions of alignment, or even different languages/speakers. This study contributes to the ongoing effort to elucidate the precise nature of the gesture–speech timing relationship by examining an understudied variety of American English, i.e., academic-lecture-style speech, with a focus (...) on an understudied type of gesture: Non-Referential gestures, which make up the majority of this corpus. Results for the 1,334 Stroke-Defined Gestures in this 20-min sample suggest that the stroke phase of a Non-Referential gesture tends to align with a pitch-accented syllable, just as reported in studies of other gesture types (e.g., deictic gestures) and in other speaking styles (such as narration). Preliminary results are presented suggesting that trajectory shapes of these Non-Referential gestures are consistent across a higher-level prosodic grouping, supporting earlier proposals for kinematic constancy across spoken prosodic constituents ( Kendon, 1972, 1980, 2004 ). Analysis also raises the possibility that the category of Non-Referential gestures is not solely made up of ‘beats,’ defined as simple bi-phasic flick-like movements that beat out the rhythm of the speech, but includes gestures with multiple phases and various types of rhythmicity. Taken together, the results of this analysis suggest (1) a wide range of gesture configurations within the undifferentiated category of Non-Referential gestures or ‘beats,’ which requires further investigation, and (2) a close coordination between co-speech gestures and the prosodic structure of spoken utterances across speaking styles and gesture referentiality, which has profound implications for modeling the process of planning an utterance. (shrink)

In dynamic theories of presupposition, a trigger pp′ with presupposition p and at-issue component p′ comes with a requirement that p should be entailed by the local context of pp′. We argue that some co-speech gestures should be analyzed within a presuppositional framework, but with a twist: an expression p co-occurring with a co-speech gesture G with content g comes with the requirement that the local context of p should guarantee that p entails g; we call such assertion-dependent presuppositions ‘cosuppositions’. (...) We show that this analysis can be combined with earlier theories of local contexts to account for complex patterns of gesture projection in quantified and in attitudinal contexts, and we compare our account to two potential alternatives: one based on supervaluations, and one, due to Cornelia Ebert, that treats co-speech gestures as supplements. We argue that the latter is correct, but for ‘post-speech’ gestures, rather than for co-speech gestures. (shrink)

Recent experiments have shown that people iconically modulate their prosody corresponding with the meaning of their utterance. This article reports findings from a story reading task that expands the investigation of iconic prosody to abstract meanings in addition to concrete ones. Participants read stories that contrasted along concrete and abstract semantic dimensions of speed and size. Participants read fast stories at a faster rate than slow stories, and big stories with a lower pitch than small stories. The effect of speed (...) was distributed across the stories, including portions that were identical across stories, whereas the size effect was localized to size-related words. Overall, these findings enrich the documentation of iconicity in spoken language and bear on our understanding of the relationship between gesture and speech. (shrink)

Research into expertise is relatively common in cognitive science concerning expertise existing across many domains. However, much less research has examined how experts within the same domain assess the performance of their peer experts. We report the results of a modified think-aloud study conducted with 18 pilots . Pairs of same-ranked pilots were asked to rate the performance of a captain flying in a critical pre-recorded simulator scenario. Findings reveal considerable variance within performance categories, differences in the process used as (...) evidence in support of a performance rating, different numbers and types of facts identified, and differences in how specific performance events affect choice of performance category and gravity of performance assessment. Such variance is consistent with low inter-rater reliability. Because raters exhibited good, albeit imprecise, reasons and facts, a fuzzy mathematical model of performance rating was developed. The model provides good agreement with observed variations. (shrink)

The set of design features developed by Charles Hockett in the 1950s and 1960s remains probably the most influential means of juxtaposing animal communication with human language. However, the general theoretical perspective of Hockett is largely incompatible with that of modern language evolution research. Consequently, we argue that his classificatory system—while useful for some descriptive purposes—is of very limited use as a theoretical framework for evolutionary linguistics. We see this incompatibility as related to the ontology of language, i.e. deriving from (...) Hockett’s interest in language as a product rather than a suite of sensorimotor, cognitive and social abilities that enable the use but also acquisition of language by biological creatures. After a reconstruction of Hockett’s views on design features, we raise two criticisms: focus on the means at the expense of content and focus on the code itself rather than the cognitive abilities of its users. Finally, referring to empirical data, we illustrate some of the problems resulting from Hockett’s approach by addressing three specific points—namely arbitrariness and semanticity, cultural transmission, and displacement—and show how the change of perspective allows to overcome those difficulties. (shrink)

This paper describes our general framework for the investigation of how human gestures can be used to facilitate the interaction and communication between humans and robots. Two studies were carried out to reveal which “naturally occurring” gestures can be observed in a scenario where users had to explain to a robot how to perform a home task. Both studies followed a within-subjects design: participants had to demonstrate how to lay a table to a robot using two different methods — utilizing (...) only gestures or gestures and speech. The first study enabled the validation of the COGNIRON coding scheme for human gestures in Human–Robot Interaction. Based on the data collected in both studies, an annotated video corpus was produced and characteristics such as frequency and duration of the different gestural classes have been gathered to help capture requirements for the designers of HRI systems. The results from the first study regarding the frequencies of the gestural types suggest an interaction between the order of presentation of the two methods and the actual type of gestures produced. However, the analysis of the speech produced along with the gestures did not reveal differences due to ordering of the experimental conditions. The second study expands the issues addressed by the first study: we aimed at extending the role of the interaction partner by introducing some positive acknowledgement of the participants’ activity. The results show no significant differences in the distribution of gestures between the two explanation methods, in contrast to the previous study. Implications for HRI are discussed focusing on issues relevant for the design of the robot’s communication skills to support the interaction loop with humans in home scenarios. (shrink)

Using a cross‐modal semantic priming paradigm, both experiments of the present study investigated the link between the mental representations of iconic gestures and words. Two groups of the participants performed a primed lexical decision task where they had to discriminate between visually presented words and nonwords (e.g., flirp). Word targets (e.g., bird) were preceded by video clips depicting either semantically related (e.g., pair of hands flapping) or semantically unrelated (e.g., drawing a square with both hands) gestures. The duration of gestures (...) was on average 3,500 ms in Experiment 1 but only 1,000 ms in Experiment 2. Significant priming effects were observed in both experiments, with faster response latencies for related gesture–word pairs than unrelated pairs. These results are consistent with the idea of interactions between the gestural and lexical representational systems, such that mere exposure to iconic gestures facilitates the recognition of semantically related words. (shrink)

One of the features of John Locke’s moral philosophy is the idea that morality is based on our beliefs concerning the future good. In An Essay Concerning Human Understanding II, xxi, §70, Locke argues that we have to decide between the probability of afterlife and our present temptations. In itself, this kind of decision model is not rare in Early Modern philosophy. Blaise Pascal’s Wager is a famous example of a similar idea of balancing between available options which Marcelo Dascal (...) has discussed in his important 2005 article “The Balance of Reason”. -/- Leibniz, however, was not always satisfied with this kind of simple balancing. In his commentary to Locke’s Essay, Nouveaux essais sur l’entendement humain, II, xxi, §66, he presented an alternative model which is based on an idea of plural, mutually conflicting inclinations. This kind of model, called as vectorial theory of rational decision by Simo Knuuttila, fits well with Leibniz’s theory of the soul where volitions are formed as a kind of compromise between different inclinations to different goods. -/- I will present these two models and show how they illustrate the practical rationality of Locke and Leibniz and how their moral philosophies differ, although being similar in certain respects. The topics include Leibniz’s criticism of Lockean hedonism and the discussion of akratic behaviour in II, xxi of Essay and Nouveaux essais. (shrink)

Although for many years a sharp distinction has been made in language research between rules and words—with primary interest on rules—this distinction is now blurred in many theories. If anything, the focus of attention has shifted in recent years in favor of words. Results from many different areas of language research suggest that the lexicon is representationally rich, that it is the source of much productive behavior, and that lexically specific information plays a critical and early role in the interpretation (...) of grammatical structure. But how much information can or should be placed in the lexicon? This is the question I address here. I review a set of studies whose results indicate that event knowledge plays a significant role in early stages of sentence processing and structural analysis. This poses a conundrum for traditional views of the lexicon. Either the lexicon must be expanded to include factors that do not plausibly seem to belong there; or else virtually all information about word meaning is removed, leaving the lexicon impoverished. I suggest a third alternative, which provides a way to account for lexical knowledge without a mental lexicon. (shrink)

Using a cross-modal semantic priming paradigm, both experiments of the present study investigated the link between the mental representations of iconic gestures and words. Two groups of the participants performed a primed lexical decision task where they had to discriminate between visually presented words and nonwords (e.g., flirp). Word targets (e.g., bird) were preceded by video clips depicting either semantically related (e.g., pair of hands flapping) or semantically unrelated (e.g., drawing a square with both hands) gestures. The duration of gestures (...) was on average 3,500 ms in Experiment 1 but only 1,000 ms in Experiment 2. Significant priming effects were observed in both experiments, with faster response latencies for related gesture–word pairs than unrelated pairs. These results are consistent with the idea of interactions between the gestural and lexical representational systems, such that mere exposure to iconic gestures facilitates the recognition of semantically related words. (shrink)