When people talk, they gesture. Gesture is a fundamental component of language that contributes meaningful and unique information to a spoken message and reflects the speaker’s underlying knowledge and experiences. Theoretical perspectives of speech and gesture propose that they share a common conceptual origin and have a tightly integrated relationship, overlapping in time, meaning, and function to enrich the communicative context. We review a robust literature from the field of psychology documenting the benefits of gesture for communication for both speakers (...) and listeners, as well as its important cognitive functions for organizing spoken language, and facilitating problem-solving, learning, and memory. Despite this evidence, gesture has been relatively understudied in populations with neurogenic communication disorders. While few studies have examined the rehabilitative potential of gesture in these populations, others have ignored gesture entirely or even discouraged its use. We review the literature characterizing gesture production and its role in intervention for people with aphasia, as well as describe the much sparser literature on gesture in cognitive communication disorders including right hemisphere damage, traumatic brain injury, and Alzheimer’s disease. The neuroanatomical and behavioral profiles of these patient populations provide a unique opportunity to test theories of the relationship of speech and gesture and advance our understanding of their neural correlates. This review highlights several gaps in the field of communication disorders which may serve as a bridge for applying the psychological literature of gesture to the study of language disorders. Such future work would benefit from considering theoretical perspectives of gesture and using more rigorous and quantitative empirical methods in its approaches. We discuss implications for leveraging gesture to explore its untapped potential in understanding and rehabilitating neurogenic communication disorders. (shrink)

In contrast to Constructivist Views, which construe perceptual cognition as an essentially reconstructive process, this article recommends the Deictic View, which grounds perception in perceptual-demonstrative reference and the use of deictic tracking strategies for acquiring and updating knowledge about individuals. The view raises the problem of how sensory-motor tracking connects to epistemic and integrated forms of tracking. To study the strategies used to solve this problem, we report a study of the ability to track distal individuals when only their directions (...) can be perceived and not their locations. We introduce a new experimental paradigm named the 'Modified Traveling Salesman Problem' (MTSP), which requires subjects to visit n invisible targets in a 2D display once each. Surprisingly, subjects are competent at this task for up to 10 targets. We consider two types of tracking strategies that subjects might use: 'location-based' strategies and 'deictic direction-based' strategies. A number of observations suggest that subjects used the latter, at least for larger numbers of targets. We hypothesize that subjects used perceptual-demonstrative reference and deictic strategies (i) to perform the sensory-motor tracking of directional segments, (ii) to bind the segments with their updated status in the task, and (iii) to perform the epistemic tracking of invisible targets by means of perception-based inferences. (shrink)

Cognitive Science, Volume 46, Issue 2, February 2022.

This paper describes our general framework for the investigation of how human gestures can be used to facilitate the interaction and communication between humans and robots. Two studies were carried out to reveal which “naturally occurring” gestures can be observed in a scenario where users had to explain to a robot how to perform a home task. Both studies followed a within-subjects design: participants had to demonstrate how to lay a table to a robot using two different methods — utilizing (...) only gestures or gestures and speech. The first study enabled the validation of the COGNIRON coding scheme for human gestures in Human–Robot Interaction. Based on the data collected in both studies, an annotated video corpus was produced and characteristics such as frequency and duration of the different gestural classes have been gathered to help capture requirements for the designers of HRI systems. The results from the first study regarding the frequencies of the gestural types suggest an interaction between the order of presentation of the two methods and the actual type of gestures produced. However, the analysis of the speech produced along with the gestures did not reveal differences due to ordering of the experimental conditions. The second study expands the issues addressed by the first study: we aimed at extending the role of the interaction partner by introducing some positive acknowledgement of the participants’ activity. The results show no significant differences in the distribution of gestures between the two explanation methods, in contrast to the previous study. Implications for HRI are discussed focusing on issues relevant for the design of the robot’s communication skills to support the interaction loop with humans in home scenarios. (shrink)

In this paper, I present a framework where possible relations between learning and mental models are explored. In particular, I’ll be concerned with non-symbolic gestures accompanying discourse and their role in inducing the construction of models and therefore deep comprehension and learning in the listener. Also, I’ll be concerned with cognitive and socio-cognitive conflicts and their roles in inducing construction of alternative models of a problem and therefore in learning to reason. Human ability to learn is of great importance for (...) individuals interested in change. Indeed, to learn both declarative and procedural knowledge means to change, and in order to be able to intervene on change in a desired way it is necessary to have a theory of the mental representations and processes involved in learning and a theory of the communication and contexts that favour learning. (shrink)

There is common agreement that preschool-level science education affects children’s curiosity, their positive approach towards science, and their desire to engage with the subject. Children’s natural curiosity drives them to engage enthusiastically in all forms of exploration. Engaging in scientific exploration necessitates self-regulation capabilities and a wide repertoire of cognitive and metacognitive strategies. The purpose of this study was to examine to what extent preschoolers (aged 5‒6 years) implement nascent inquiry skills, metacognitive awareness, and self-regulation capabilities during play-based scientific exploration (...) tasks. An additional purpose was to investigate the relationships between these capabilities, a relationship not yet investigated in the context of play-based, scientific exploration among young children. The study consisted of 215 preschoolers, from 10 preschools. For this study, we developed two scientific exploration tasks—structured and open-ended. Our motivation was to examine whether preschoolers’ capabilities will differ in the context of structured task which is aligned with the view that young children need guidance and explicit instructions compared to the context of open-ended, play-based task–allowing the children to apply and test their intuitive theories and skills. During performance participants were videotaped. Their verbal and non-verbal responses were analyzed by means of a scoring scheme. The results of a micro-analysis of about 100 hours of video showed that given the opportunity, even without setting explicit goals and instructions, children exhibit inquiry capabilities: they ask questions, plan, hypothesize, use tools, draw conclusions. Asking questions and planning were better manifested during the structured task. Children also manifested higher levels of attention, persistence, and autonomy during the structured task. However, significant higher scores of self-regulation indications were revealed in the context of the open-ended, play-based, exploration task. Moreover, results indicate significant correlations between the five measures of preschoolers’ inquiry capabilities and measures of metacognitive strategic awareness and self-regulation. The results of the present study suggest the importance of combining various learning environments and experiences in early science education that encourage children to engage in structured exploration alongside play-based, open-ended, exploration. (shrink)

Last year at VSS, Bullot, Droulez & Pylyshyn reported studies using a Modified Traveling Salesman Paradigm in which a virtual vehicle had to visit up to 10 targets once and only once, and in which the invisible targets were identified only by line segments pointing from the vehicle toward each target. We hypothesized that subjects used two distinct strategies: A “location-based strategy”, which kept track of where targets were located in screen coordinates, and a “segment-based strategy” that kept track of (...) which segments corresponded to visited targets. We report new studies that further explore these two strategies. Subjects passively observed a computer-controlled virtual vehicle that visited a number of targets. Two forms of display were used: an “Allocentric” display, in which the vehicle moved and the targets remained fixed in screen coordinates, and an “Egocentric” display, in which the vehicle's position on the screen remained fixed while the targets moved -- as if the environment were being viewed by an observer on the vehicle. At the end of each trial, the directional segments were extended to the edge of the screen and subjects were asked to perform two tasks by referring to these segments. In the “status task” observers had to indicate for each segment whether the corresponding target had been visited or not. In the “locating task” they had to locate each target along its directional segment. Performance on these two tasks measures the use of the two hypothesized strategies. Results showed that observers do well on the status task with 4 or 6 targets in both display conditions, but do poorly on the locating task, especially in the egocentric condition when there are more targets. These results are consistent with the hypothesis that in the egocentric condition the MTSP task is carried out primarily by segment-tracking, which can be viewed as a deictic strategy. (shrink)