This paper offers a biosemiotic account of the poetic aspects of gesture and speech in schizophrenia. The argument is that speech and gesture are not the mere expression of pre-verbal thoughts. Instead, meaning is enacted by the temporal and semantic coordination of speech and gesture. The bodily basis of language is highlighted by the fact that, failing to create language that is organized around topics, individuals with schizophrenia often rely on poetic associations in directing their utterances. Accordingly, the analysis of (...) schizophrenic speech and gesture based on McNeill’s Growth Point theory is enriched with reference to both Cowley’s (2007b) views on language as distributed social coordination, and Alexander’s (Biosemiotics 2(1):77–100, 2009) description of formal causes in nature. Schizophrenic language can be seen as a hypertrophied manifestation of the discourse-scaffolding function of poetic associations in speech and gesture. The analysis of schizophrenic language shows that language behavior need not be built on pre-verbal thoughts and important aspects of schizophrenic language, such as metalinguistic awareness and semiotic agency, can be clarified by applying concepts from biosemiotics. (shrink)

Using a cross-modal semantic priming paradigm, both experiments of the present study investigated the link between the mental representations of iconic gestures and words. Two groups of the participants performed a primed lexical decision task where they had to discriminate between visually presented words and nonwords (e.g., flirp). Word targets (e.g., bird) were preceded by video clips depicting either semantically related (e.g., pair of hands flapping) or semantically unrelated (e.g., drawing a square with both hands) gestures. The duration of gestures (...) was on average 3,500 ms in Experiment 1 but only 1,000 ms in Experiment 2. Significant priming effects were observed in both experiments, with faster response latencies for related gesture–word pairs than unrelated pairs. These results are consistent with the idea of interactions between the gestural and lexical representational systems, such that mere exposure to iconic gestures facilitates the recognition of semantically related words. (shrink)

Using a cross‐modal semantic priming paradigm, both experiments of the present study investigated the link between the mental representations of iconic gestures and words. Two groups of the participants performed a primed lexical decision task where they had to discriminate between visually presented words and nonwords (e.g., flirp). Word targets (e.g., bird) were preceded by video clips depicting either semantically related (e.g., pair of hands flapping) or semantically unrelated (e.g., drawing a square with both hands) gestures. The duration of gestures (...) was on average 3,500 ms in Experiment 1 but only 1,000 ms in Experiment 2. Significant priming effects were observed in both experiments, with faster response latencies for related gesture–word pairs than unrelated pairs. These results are consistent with the idea of interactions between the gestural and lexical representational systems, such that mere exposure to iconic gestures facilitates the recognition of semantically related words. (shrink)

The goal of this paper is to explore, from a phenomenologically informed perspective, the phenomenon of the operative spatialization of human thinking, viewed in its relationship with the embodied human organism’s spatial experience. Operative spatialization in this context refers to the cognitive role and functioning of spatial schematizations and differentiations in human thinking. My particular focus is the domain of conceptualization. By drawing on Husserl’s discussion of the (linguistic) process of a sedimentation of meaning, I aim to show that spatialization (...) functions in a structurally ambivalent manner. On the one hand, spatialization predisposes and thus narrows the scope of human conceptual thought. On the other hand, spatialization establishes an implicit cognitive scaffold indispensable for the development and practice of human higher-order thinking. The structure and functioning of this scaffold, I argue, is intrinsically related to the spatial structure and dynamics of human embodiment. Synthesizing insights from phenomenological studies concerning the experiential constitution of space, and classic and recent research findings from fields such as cognitive linguistics and psychology concerning the cognitive dimension of image-schemas and gestural behavior, I argue that the living human body functions centrally as the sense-constitutive ‘site of conversion’ (Husserl) between concrete structures of spatial experience and abstract structures of conceptual thought. (shrink)

The set of design features developed by Charles Hockett in the 1950s and 1960s remains probably the most influential means of juxtaposing animal communication with human language. However, the general theoretical perspective of Hockett is largely incompatible with that of modern language evolution research. Consequently, we argue that his classificatory system—while useful for some descriptive purposes—is of very limited use as a theoretical framework for evolutionary linguistics. We see this incompatibility as related to the ontology of language, i.e. deriving from (...) Hockett’s interest in language as a product rather than a suite of sensorimotor, cognitive and social abilities that enable the use but also acquisition of language by biological creatures. After a reconstruction of Hockett’s views on design features, we raise two criticisms: focus on the means at the expense of content and focus on the code itself rather than the cognitive abilities of its users. Finally, referring to empirical data, we illustrate some of the problems resulting from Hockett’s approach by addressing three specific points—namely arbitrariness and semanticity, cultural transmission, and displacement—and show how the change of perspective allows to overcome those difficulties. (shrink)

The present study investigates the types of verb and symbolic representational strategies used by 10 deaf signing adults and 13 deaf signing children who described in Italian Sign Language 45 video clips representing nine action types generally communicated by five general verbs in spoken Italian. General verbs, in which the same sign was produced to refer to several different physical action types, were rarely used by either group of participants. Both signing children and adults usually produced specific depicting predicates by (...) incorporating, through a representational strategy, the object and/or the modality of the action into the sign. As for the different types of representational strategies, the adults used the hand-as-object strategy more frequently than the children, who, in turn, preferred to use the hand-as-hand strategy, suggesting that different degrees of cognitive complexity are involved in these two symbolic strategies. Addressing the symbolic iconic strategies underlying sign formation could provide new insight into the perceptual and cognitive processes of linguistic meaning construction. The findings reported here support two main assumptions of cognitive linguistics applied to sign languages: there is a strong continuity between gestures and language; lexical units and depicting constructions derive from the same iconic core mechanism of sign creation. (shrink)

This study investigates whether there is a relation between the semantics of linguistic expressions that indicate temporal distance and the spatial properties of their co-speech gestures. To this date, research on time gestures has focused on features such as gesture axis, direction, and shape. Here we focus on a gesture property that has been overlooked so far: the distance of the gesture in relation to the body. To achieve this, we investigate two types of temporal linguistic expressions are addressed: proximal (...) (e.g., near future, near past) and distal (e.g., distant past, distant future). Data was obtained through the NewsScape library, a multimodal corpus of television news. A total of 121 co-speech gestures were collected and divided into the two categories. The gestures were later annotated in terms of gesture space and classified in three categories: (i) center, (ii) periphery, and (iii) extreme periphery. Our results suggest that gesture and language are coherent in the expression of temporal distance: when speakers locate an event far from them, they tend to gesture further from their body; similarly, when locating an event close to them, they gesture closer to their body. These results thus reveal how co-speech gestures also reflect a space-time mapping in the dimension of distance. (shrink)

Depictive expressions of thought predate written language by thousands of years. They have evolved in communities through a kind of informal user testing that has refined them. Analyzing common visual communications reveals consistencies that illuminate how people think as well as guide design; the process can be brought into the laboratory and accelerated. Like language, visual communications abstract and schematize; unlike language, they use properties of the page (e.g., proximity and place: center, horizontal/up–down, vertical/left–right) and the marks on it (e.g., (...) dots, lines, arrows, boxes, blobs, likenesses, symbols) to convey meanings. The visual expressions of these meanings (e.g., individual, category, order, relation, correspondence, continuum, hierarchy) have analogs in language, gesture, and especially in the patterns that are created when people design the world around them, arranging things into piles and rows and hierarchies and arrays, spatial-abstraction-action interconnections termed spractions. The designed world is a diagram. (shrink)

This paper offers a novel argument against the phenomenal intentionality thesis (or PIT for short). The argument, which I'll call the extended mind argument against phenomenal intentionality, is centered around two claims: the first asserts that some source intentional states extend into the environment, while the second maintains that no conscious states extend into the environment. If these two claims are correct, then PIT is false, for PIT implies that the extension of source intentionality is predicated upon the extension of (...) phenomenal consciousness. The argument is important because it undermines an increasingly prominent account of the nature of intentionality. PIT has entered the philosophical mainstream and is now a serious contender to naturalistic views of intentionality like the tracking theory and the functional role theory (Loar 1987, 2003; Searle 1990; Strawson 1994; Horgan and Tienson 2002; Pitt 2004; Farkas 2008; Kriegel 2013; Montague 2016; Bordini 2017; Forrest 2017; Mendelovici 2018). The extended mind argument against PIT challenges the popular sentiment that consciousness grounds intentionality. (shrink)

In the near future, our life will normally be surrounded with fairly complicated artifacts, enabled by the autonomous robot and brain–machine interface technologies. In this paper, we argue that what we call the responsibility flaw problem and the inappropriate use problem need to be overcome in order for us to benefit from complicated artifacts. In order to solve these problems, we propose an approach to endowing artifacts with an ability of socially communicating with other agents based on the artifact-as-a-half-mirror metaphor. (...) The idea is to have future artifacts behave according to the hybrid intention composed of the owner’s intention and the social rules. We outline the approach and discuss its feasibility together with preliminary work. (shrink)

Embodied cognitive science holds that cognitive processes are deeply and inescapably rooted in our bodily interactions with the world. Our finite, contingent, and mortal embodiment may be not only supportive, but in some cases even constitutive of emotions, thoughts, and experiences. My discussion here will work outward from the neuroanatomy and neurophysiology of the brain to a nervous system which extends to the boundaries of the body. It will extend to nonneural aspects of embodiment and even beyond the boundaries of (...) the body to prosthetics of various kinds, including symbioses with a broad array of cultural artifacts, our symbolic niche, and our relationships with other embodied human beings. While cognition may not always be situated, its origins are embedded in temporally and spatially limited activities. Cognitive work also can be off-loaded to the body and to the environment in service of action, tool use, group cognition, and social coordination. This can blur the boundaries between brain areas, brain and body, and body and environment, transforming our understanding of mind and personhood to provide a different grounding for faith traditions in general, and of the historically dualist Christian tradition in particular. (shrink)

I engage phenomenological and empirical perspectives on dialogical relations in infancy in a mutually enlightening and challenging relation. On the one hand, the empirical contributions provide evidence for the primacy of first-to-second person interrelatedness in human sociality, as opposed to the claim of primary syncretism heralded by Merleau-Ponty, and also in distinction from the ego-alter ego model routinely used in phenomenology. On the other hand, phenomenological considerations regarding the lived affective experience of dialogical relatedness enrich and render intelligible the psychological (...) accounts of dialogue in terms of observable behavior. Phenomenological and empirical perspectives on dialogical relatedness thus combine to offer an affectively charged and conversationally patterned notion of primary intersubjectivity in the I-you mode. (shrink)

In an earlier paper, I set out to apply to animal mimicry the definition of the sign, and, more specifically, of the iconic sign, which I originally elaborated in the study of pictures, and which was then extended by myself and others to language, gesture, and music. The present contribution, however, while summarizing some of the results of those earlier studies, is dedicated to the demonstration that animal mimicry, as well as phenomena of the human Lifeworld comparable to it, are (...) in a sense the opposite of signs. It has often been observed, not only within speech act philosophy, but also by the semiotician Luis Prieto, that as sign can only function as such once it is recognized to be a sign. Animal mimicry, camouflage, and the like, in contrast, only work as such, to the extent that they are not perceived as signs. Unlike what speech act philosophy claims, nevertheless, the “difference which makes a difference” is not the recognition of a purpose attributed to the subject producing the sign. A footprint, for example, has to be recognized as a sign in order to function as such. Nevertheless, to the extent that a purpose is attributed to the subject setting the sign, it may be considered a sign, but one that hides its nature, a fake footprint. Mimicry and camouflage, however, are similar to such “natural meanings” as footprints in entertaining a different relation to the agent initiating the act and the agent perceiving it. Classical studies of mimicry distinguish its varieties according to what is rather vaguely called function. In this paper, we will investigate whether these classifications can be recuperated from a semiotic point of view, or whether a semiotically valid classification should start from scratch. (shrink)

ASL can express plurals by repeating a noun, in an unpunctuated fashion, in different parts of signing space. We argue that this construction may come with a rich iconic component: the geometric arrangement of the repetitions provides information about the arrangement of the denoted plurality; in addition, the number and speed of the repetitions provide information about the size of the denoted plurality. Interestingly, the shape of the repetitions may introduce a new singular discourse referent when a vertex can be (...) inferred to denote a singular object. Thus one may point towards the first or last iteration of a horizontal repetition of BOOK to denote the left- or right-edge of the corresponding row. This yields a remarkable interaction between iconic semantics and standard logical semantics. We show that our analysis extends to ‘punctuated’ repetitions, which involve clearly individuated iterations of a singular noun. While these may initially look like coordinated indefinites, they are better handled by the same iconic framework as plural, unpunctuated repetitions. Some repetition-based mass terms also give rise to iconic effects, and to different readings depending on whether the repetition is continuous, unpunctuated, or punctuated. Our analysis highlights the need for a formal semantics with iconicity to study the integration of such iconic and logical conditions. It also raises a question: can similar facts be found in spoken language when gestures are taken into account? We suggest that several effects can be replicated, especially when one considers examples involving ‘pro-speech gestures’. (shrink)

In dynamic theories of presupposition, a trigger pp′ with presupposition p and at-issue component p′ comes with a requirement that p should be entailed by the local context of pp′. We argue that some co-speech gestures should be analyzed within a presuppositional framework, but with a twist: an expression p co-occurring with a co-speech gesture G with content g comes with the requirement that the local context of p should guarantee that p entails g; we call such assertion-dependent presuppositions ‘cosuppositions’. (...) We show that this analysis can be combined with earlier theories of local contexts to account for complex patterns of gesture projection in quantified and in attitudinal contexts, and we compare our account to two potential alternatives: one based on supervaluations, and one, due to Cornelia Ebert, that treats co-speech gestures as supplements. We argue that the latter is correct, but for ‘post-speech’ gestures, rather than for co-speech gestures. (shrink)

When learners self-explain, they try to make sense of new information. Although research has shown that bodily actions and written notes are an important part of learning, previous analyses of self-explanations rarely take into account written and non-verbal data produced spontaneously. In this paper, the extent to which interpretations of self-explanations are influenced by the systematic consideration of such data is investigated. The video recordings of 33 undergraduate students, who learned with worked-out examples dealing with complex numbers, were categorized successively (...) including three different data bases: verbal data, verbal and written data, and verbal, written and non-verbal data. Results reveal that including written data and non-verbal data leads to a more accurate analysis of self-explanations than an analysis solely based on verbal data. This influence is even stronger for the categorization of self-explanations as adequate or inadequate. (shrink)

Research into expertise is relatively common in cognitive science concerning expertise existing across many domains. However, much less research has examined how experts within the same domain assess the performance of their peer experts. We report the results of a modified think-aloud study conducted with 18 pilots . Pairs of same-ranked pilots were asked to rate the performance of a captain flying in a critical pre-recorded simulator scenario. Findings reveal considerable variance within performance categories, differences in the process used as (...) evidence in support of a performance rating, different numbers and types of facts identified, and differences in how specific performance events affect choice of performance category and gravity of performance assessment. Such variance is consistent with low inter-rater reliability. Because raters exhibited good, albeit imprecise, reasons and facts, a fuzzy mathematical model of performance rating was developed. The model provides good agreement with observed variations. (shrink)

While recent studies have claimed that non-referential gestures are used to mark discourse-new and/or -accessible referents and focused information in adult speech, to our knowledge, no prior investigation has studied the relationship between information structure and gesture referentiality in children’s narrative speech from a developmental perspective. A longitudinal database consisting of 332 narratives performed by 83 children at two different time points in development was coded for IS and gesture referentiality. Results revealed that at both time points, both referential and (...) non-referential gestures were produced more with information that moves discourse forward and predication rather than topical or background information. Further, at 7–9 years of age, children tended to use more non-referential gestures to mark focus and comment constituents than referential gestures. In terms of the marking of the newness of discourse referents, non-referential gestures already seem to play a key role at 5–6 years old, whereas referential gestures did not show any patterns. This relationship was even stronger at 7–9 years old. All in all, our findings offer supporting evidence that in contrast with referential gestures, non-referential gestures have been found to play a key role in marking IS, and that the development of this relationship solidifies at a period in development that coincides with a spurt in non-referential gesture production. (shrink)

Theories of embodied cognition hypothesize interdependencies between psychological well-being and physical posture. The purpose of this study was to assess the feasibility of objectively measuring posture, and to explore the relationship between posture and affect and other patient centered outcomes in breast cancer survivors with persistent postsurgical pain over a 12-week course of therapeutic Qigong mind-body training. Twenty-one BCS with PPSP attended group Qigong training. Clinical outcomes were pain, fatigue, self-esteem, anxiety, depression, stress and exercise self-efficacy. Posture outcomes were vertical (...) spine and vertical head angles in the sagittal plane, measured with a 3D motion capture system in three conditions: eyes open, eyes open relaxed and eyes closed. Assessments were made before and after the Qigong training. The association between categorical variables was measured by Cramer’s V. In the EO condition, most participants who improved in fatigue and anxiety scales also had better vertical head values. For the EOR condition, a moderate correlation was observed between changes in vertical head angle and changes in fatigue scale. In the EC condition, most of the participants who improved in measures of fatigue also improved vertical head angle. Additionally, pain severity decreased while vertical spine angle improved. These preliminary findings support that emotion and other patient centered outcomes should be considered within an embodied framework, and that Qigong may be a promising intervention for addressing biopsychosocially complex interventions such as PPSP in BCSs. (shrink)

This paper has two objectives. The first is to formulate a critique of present-day cognitive linguistics concerning the inner workings of the cognitive system during language use, and the second is to put forward an alternative account that is inspired by the phenomenology of Merleau-Ponty. Due to its third-person methodology, CL views language use essentially as a problem-solving activity, as coping with two subproblems: the problem of minimum and maximum, which consists in selecting the appropriate expression out of an unlimited (...) multitude of possibilities, and the problem of the underdetermination of signification. This approach presupposes a notion of an isolated subject and a representationalist view of perception. We defend an alternative view of man's relation to the world in which intersubjectivity is constitutive of embodied subjectivity and which exchanges the representationalist view of perception for a direct nonrepresentationalism. We describe the ensuing view of linguistic action as intra- and interpersonal “all-at-onceness.” This approach dismisses the two subproblems CL implicitly identifies as constitutive of language use. The first is countered by rethinking what it means to be a situated speaking subject and results in the concept of “style.” The second is tackled by opposing the concept of “overdetermination” to CL's notion of underdetermination. (shrink)

For humans, the ability to communicate and use language is instantiated not only in the vocal modality but also in the visual modality. The main examples of this are sign languages and gestures. Sign languages, the natural languages of Deaf communities, use systematic and conventionalized movements of the hands, face, and body for linguistic expression. Co-speech gestures, though non-linguistic, are produced in tight semantic and temporal integration with speech and constitute an integral part of language together with speech. The articles (...) in this issue explore and document how gestures and sign languages are similar or different and how communicative expression in the visual modality can change from being gestural to grammatical in nature through processes of conventionalization. As such, this issue contributes to our understanding of how the visual modality shapes language and the emergence of linguistic structure in newly developing systems. Studying the relationship between signs and gestures provides a new window onto the human ability to recruit multiple levels of representation in the service of using or creating conventionalized communicative systems. (shrink)

Recent experiments have shown that people iconically modulate their prosody corresponding with the meaning of their utterance. This article reports findings from a story reading task that expands the investigation of iconic prosody to abstract meanings in addition to concrete ones. Participants read stories that contrasted along concrete and abstract semantic dimensions of speed and size. Participants read fast stories at a faster rate than slow stories, and big stories with a lower pitch than small stories. The effect of speed (...) was distributed across the stories, including portions that were identical across stories, whereas the size effect was localized to size-related words. Overall, these findings enrich the documentation of iconicity in spoken language and bear on our understanding of the relationship between gesture and speech. (shrink)

This paper describes our general framework for the investigation of how human gestures can be used to facilitate the interaction and communication between humans and robots. Two studies were carried out to reveal which “naturally occurring” gestures can be observed in a scenario where users had to explain to a robot how to perform a home task. Both studies followed a within-subjects design: participants had to demonstrate how to lay a table to a robot using two different methods — utilizing (...) only gestures or gestures and speech. The first study enabled the validation of the COGNIRON coding scheme for human gestures in Human–Robot Interaction. Based on the data collected in both studies, an annotated video corpus was produced and characteristics such as frequency and duration of the different gestural classes have been gathered to help capture requirements for the designers of HRI systems. The results from the first study regarding the frequencies of the gestural types suggest an interaction between the order of presentation of the two methods and the actual type of gestures produced. However, the analysis of the speech produced along with the gestures did not reveal differences due to ordering of the experimental conditions. The second study expands the issues addressed by the first study: we aimed at extending the role of the interaction partner by introducing some positive acknowledgement of the participants’ activity. The results show no significant differences in the distribution of gestures between the two explanation methods, in contrast to the previous study. Implications for HRI are discussed focusing on issues relevant for the design of the robot’s communication skills to support the interaction loop with humans in home scenarios. (shrink)

The Aymara of the Andes use absolute (cardinal) frames of reference for describing the relative position of ordinary objects. However, rather than encoding them in available absolute lexemes, they do it in lexemes that are intrinsic to the body: nayra (“front”) and qhipa (“back”), denoting east and west, respectively. Why? We use different but complementary ethnographic methods to investigate the nature of this encoding: (a) linguistic expressions and speech–gesture co-production, (b) linguistic patterns in the distinct regional Spanish-based variety Castellano Andino (...) (CA), (c) metaphorical extensions of CA’s spatial patterns to temporal ones, and (d) layouts of traditional houses. Findings indicate that, following fundamental principles of Aymara cosmology, people, objects, and land—as a whole—are conceived as having an implicit canonical orientation facing east, a primary landmark determined by the sunrise. The above bodily based lexicalizations are thus linguistic manifestations of a broader macro-cultural worldview and its psycho-cognitive reality. (shrink)

Journal Name: Semiotica Issue: Ahead of print.

Recently, in cognitive science, the enactivist account of cognition has been gaining ground, due in part to studies of movement in conjunction with thought. The idea, as Noë , has put it, that “cognition is not something happening inside us or to us, but it’s something we do, something we achieve,” is increasingly supported by research on joint attention, movement coordination, and gesture. Not surprisingly, therefore, enactivists have also begun to look at “movement specialists”—dancers—for both scientific and phenomenological accounts of (...) thinking with and through movement. In this paper, I argue that a serious exploration of dance and movement does not merely bolster the enactivist view, but rather, it suggests a radical enactivism, as envisaged by, e.g., Hutto . To support this claim, I examine an account of “Thinking in Movement” provided by Maxine Sheets-Johnstone in order to highlight the ways in which intentional agency and meaning-making occur in improvisational dance. These processes, I further argue, closely mirror some of the key components of participatory sense making, as described by De Jaegher and Di Paolo :485–507, 2007). This is beneficial to my case, because it permits a discussion of “thought-full action” that does not depend upon standard cognitivist frameworks for explanation. By carefully focusing on how agency can help to separate mere “thrashing about” from meaningful movement, this paper aim to strengthen the position of radical enactivism from the unique perspective and dance and sense-making. (shrink)

Did protolanguage users use discrete words that referred to objects, actions, locations, etc., and then, at some point, combine them; or on the contrary did they have words that globally indexed whole semantic complexes, and then come to divide them? Our answer is: early humans were forming language units consisting of global and discrete dimensions of semiosis in dynamic opposition. These units of thinking-for-speaking, or ‘growth points’ (GPs) were, jointly, analog imagery (visuo-spatio-motoric) and categorically-contrastive (-emic) linguistic encodings. This discrete-global duality (...) was a new mode of embodied cognition that enabled thinking and acting in new ways: the dawn of protolanguage. Where did this mode of cognition come from? We have some suggestions based on the hypothesis that gestures gained the power to orchestrate actions, manual and vocal, with significances other than those of the actions themselves, giving rise to cognition framed in the proposed dual terms. Note, however, our proposal is not one of the ‘gesture-first’ theories of language origins. Such theories predict what did not evolve: a language of pantomime; rather than what did evolve: an integrated system of synchronized gestures and spoken forms. GP theory is an account of the cognition underlying such an integrated system. A scenario for the evolutionary selection of this cognitive mode is ‘Mead’s Loop’, a model in which one’s cognition is enriched by one’s own gestures, insofar as they are objects in social interactions. (shrink)

Art theory has consistently emphasised the importance of situational, cultural, institutional and historical factors in viewers’ experience of fine art. However, the link between this heavily context-dependent interpretation and the workings of the mind is often left unexamined. Drawing on relevance theory—a prominent, cogent and productive body of work in cognitive pragmatics—we here argue that fine art achieves its effects by prompting the use of cognitive processes that are more commonly employed in the interpretation of words and other stimuli presented (...) in a communicative context. We describe in particular how institutional factors effectively co-opt these processes for new ends, allowing viewers to achieve cognitive effects that they otherwise would not, and so provide cognitivist backing for an Institutional Theory of Art, such as that put forward by Arthur Danto. More generally, we situate and describe the Western fine art tradition as a phenomenon that is a consequence of both the cognitive processes involved in communication, and of cultural norms, practices and institutions. (shrink)

Most native speakers of Spanish are intuitively able to construct correct structures with the marker ‘se’. On the other hand, non-native speakers, even those at advanced proficiency levels, have difficulties producing most constructions with ‘se’. This is hardly surprising as the marker ‘se’, one of the most common words in Spanish, can convey highly pragmatic nuances with a variety of functions that are still much debated among linguists. This study analyses some of the most used functions of the marker in (...) the oral production of 18 Peninsular Spanish speakers from a multimodal Cognitive Grammar approach, identifying how gestures might be employed by speakers to create or clarify meaning. Our results confirm that gestures are used differently depending on the function of the marker. Middle voices where ‘se’ marks high levels of subject involvement and/or energy co-occur with related gestures, while middle voices or impersonal structures, where the involvement and/or energy is low, co-occur more often with unrelated gestures, if any. (shrink)

In previous analyses of the influence of language on cognition, speech has been the main channel examined. In studies conducted among Yucatec Mayas, efforts to determine the preferred frame of reference in use in this community have failed to reach an agreement (Bohnemeyer & Stolz, 2006; Levinson, 2003 vs. Le Guen, 2006, 2009). This paper argues for a multimodal analysis of language that encompasses gesture as well as speech, and shows that the preferred frame of reference in Yucatec Maya is (...) only detectable through the analysis of co-speech gesture and not through speech alone. A series of experiments compares knowledge of the semantics of spatial terms, performance on nonlinguistic tasks and gestures produced by men and women. The results show a striking gender difference in the knowledge of the semantics of spatial terms, but an equal preference for a geocentric frame of reference in nonverbal tasks. In a localization task, participants used a variety of strategies in their speech, but they all exhibited a systematic preference for a geocentric frame of reference in their gestures. (shrink)

From the late 1950s until 1975, cognition was understood mainly as disembodied symbol manipulation in cognitive psychology, linguistics, artificial intelligence, and the nascent field of Cognitive Science. The idea of embodied cognition entered the field of Cognitive Linguistics at its beginning in 1975. Since then, cognitive linguists, working with neuroscientists, computer scientists, and experimental psychologists, have been developing a neural theory of thought and language (NTTL). Central to NTTL are the following ideas: (a) we think with our brains, that is, (...) thought is physical and is carried out by functional neural circuitry; (b) what makes thought meaningful are the ways those neural circuits are connected to the body and characterize embodied experience; (c) so-called abstract ideas are embodied in this way as well, as is language. Experimental results in embodied cognition are seen not only as confirming NTTL but also explained via NTTL, mostly via the neural theory of conceptual metaphor. Left behind more than three decades ago is the old idea that cognition uses the abstract manipulation of disembodied symbols that are meaningless in themselves but that somehow constitute internal “representations of external reality” without serious mediation by the body and brain. This article uniquely explains the connections between embodied cognition results since that time and results from cognitive linguistics, experimental psychology, computational modeling, and neuroscience. (shrink)

The aim of this essay is to situate the thought of Tetsurō Watsuji within contemporary approaches to social cognition. I argue for Watsuji’s current relevance, suggesting that his analysis of embodiment and social space puts him in step with some of the concerns driving ongoing treatments of social cognition in philosophy of mind and cognitive science. Yet, as I will show, Watsuji can potentially offer a fruitful contribution to this discussion by lending a phenomenologically informed critical perspective. This is because (...) Watsuji challenges the internalist and cognitivist presuppositions informing the currently dominant “Theory of Mind” paradigm that is driving much social cognition research. Additionally, I show that Watsuji’s alternative model is not merely confined to the realm of phenomenological description but that it also receives robust empirical support from a number of different sources. I thus hope to open up aspects of Watsuji’s thinking that have yet to be fully appreciated. (shrink)

The framework of depicting put forward byClark (2016)offers a schematic vantage point from which to examine iconic language use. Confronting the framework with empirical data, we consider some of its key theoretical notions. Crucially, by reconceptualizing the typology of depictions, we identify an overlooked domain in the literature: “speech-embedded nonverbal depictions,” namely cases where meaning is communicated iconically, nonverbally, and without simultaneously co-occurring speech. In addition to contextualizing the phenomenon in relation to existing research, we demonstrate, with examples from American (...) TV talk shows, how such depictions function in real-life language use, offering a brief sketch of their complexities and arguing also for their theoretical significance. (shrink)

When faced with an ambiguous pronoun, comprehenders use both multimodal cues and linguistic cues to identify the antecedent. While research has shown that gestures facilitate language comprehension, improve reference tracking, and influence the interpretation of ambiguous pronouns, literature on reference resolution suggests that a wide set of linguistic constraints influences the successful resolution of ambiguous pronouns and that linguistic cues are more powerful than some multimodal cues. To address the outstanding question of the importance of gesture as a cue in (...) reference resolution relative to cues in the speech signal, we have previously investigated the comprehension of contrastive gestures that indexed abstract referents – in this case expressions of personal preference – and found that such gestures did facilitate the resolution of ambiguous statements of preference. In this study, we extend this work to investigate whether the effect of gesture on resolution is diminished when the gesture indexes a statement that is less likely to be interpreted as the correct referent. Participants watched videos in which a speaker contrasted two ideas that were either neutral or moral acceptable). A gesture to the left or right side co-occurred with speech expressing each position. In gesture-disambiguating trials, an ambiguous phrase was accompanied by a gesture to one side or the other. In gesture non-disambiguating trials, no third gesture occurred with the ambiguous phrase. Participants were more likely to choose the idea accompanied by gesture as the stimulus speaker’s preference. We found no effect of scenario type. Regardless of whether the linguistic cue expressed a view that was morally charged or neutral, observers used gesture to understand the speaker’s opinion. This finding contributes to our understanding of the strength and range of cues, both linguistic and multimodal, that listeners use to resolve ambiguous references. (shrink)

The extended mind hypothesis (Clark and Chalmers in Analysis 58(1):7–19, 1998; Clark 2008) is an influential hypothesis in philosophy of mind and cognitive science. I argue that the extended mind hypothesis is born to be wild. It has undeniable and irrepressible tendencies of flouting grounding assumptions of the traditional information-processing paradigm. I present case-studies from social cognition which not only support the extended mind proposal but also bring out its inherent wildness. In particular, I focus on cases of action-understanding and (...) discuss the role of embodied intentionality in the extended mind project. I discuss two theories of action-understanding for exploring the support for the extended mind hypothesis in embodied intersubjective interaction, namely, simulation theory and a non-simulationist perceptual account. I argue that, if the extended mind adopts a simulation theory of action-understanding, it rejects representationalism. If it adopts a non-simulationist perceptual account of action-understanding, it rejects the classical sandwich view of the mind. (shrink)