Many animals can be trained to perform novel tasks. People, too, can be trained, but sometime in early childhood people transition from being trainable to something qualitatively more powerful—being programmable. We argue that such programmability constitutes a leap in the way that organisms learn, interact, and transmit knowledge, and that what facilitates or enables this programmability is the learning and use of language. We then examine how language programs the mind and argue that it does so through the manipulation of (...) embodied, sensorimotor representations. The role language plays in controlling mental representations offers important insights for understanding its origin and evolution. (shrink)

We demonstrate in two experiments that real and imagined body movements appropriate to metaphorical phrases facilitate people's immediate comprehension of these phrases. Participants first learned to make different body movements given specific cues. In two reading time studies, people were faster to understand a metaphorical phrase, such as push the argument, when they had previously just made an appropriate body action (e.g., a push movement) (Experiment 1), or imagined making a specific body movement (Experiment 2), than when they first made (...) a mismatching body action (e.g., a chewing movement) or no movement. These findings support the idea that appropriate body action, or even imagined action, enhances people's embodied, metaphorical construal of abstract concepts that are referred to in metaphorical phrases. (shrink)

There is mounting evidence that language comprehension involves the activation of mental imagery of the content of utterances (; ; ; ; ; ; ). This imagery can have motor or perceptual content. Three main questions about the process remain under‐explored, however. First, are lexical associations with perception or motion sufficient to yield mental simulation, or is the integration of lexical semantics into larger structures, like sentences, necessary? Second, what linguistic elements (e.g., verbs, nouns, etc.) trigger mental simulations? Third, how (...) detailed are the visual simulations that are performed? A series of behavioral experiments address these questions, using a visual object categorization task to investigate whether up‐ or down‐related language selectively interferes with visual processing in the same part of the visual field (following ). The results demonstrate that either subject nouns or main verbs can trigger visual imagery, but only when used in literal sentences about real space—metaphorical language does not yield significant effects—which implies that it is the comprehension of the sentence as a whole and not simply lexical associations that yields imagery effects. These studies also show that the evoked imagery contains detail as to the part of the visual field where the described scene would take place. (shrink)

People often answer why-questions with what we call experiential explanations: narratives or stories with temporal structure and concrete details. In contrast, on most theories of the epistemic function of explanation, explanations should be abstractive: structured by general relationships and lacking extraneous details. We suggest that abstractive and experiential explanations differ not only in level of abstraction, but also in structure, and that each form of explanation contributes to the epistemic goals of individual learners and of science. In particular, experiential explanations (...) support mental simulation and survive transitions across background theories; as a result, they support learning and help us translate between competing frameworks. Experiential explanations play an irreducible role in human cognition – and perhaps in science. (shrink)

Poetry evokes mental imagery in its readers. But how is mental imagery precisely related to poetry? This article provides a systematic treatment. It clarifies two roles of mental imagery in relation to poetry—as an effect generated by poetry and as an efficient means for understanding and appreciating poetry. The article also relates mental imagery to the discussion on the ‘heresy of paraphrase’. It argues against the orthodox view that the imagistic effects of poetry cannot be captured by prosaic paraphrase, but (...) points to features of poetry that can shape aspects of mental imagery that are liable to be lost in paraphrase. (shrink)

When participants are asked to make sensibility judgments on sentences that describe action toward the body (i.e., “Mark dealt the cards to you”) or away from the body (i.e., “You dealt the cards to Mark”), they are faster to respond when the response requires an arm movement in the same direction as the action described by the sentence. This congruence effect is known as the Action–Sentence Compatibility Effect (ACE). This study reports 4 experiments that extend our understanding of the ACE (...) by exploring how the time at which one prepares the motor response required for the sensibility judgment affects the magnitude of the ACE. Results show that the ACE arises only when participants have the opportunity to plan their motor response while they are processing the sentence. (shrink)

To explain how abstract concepts are grounded in sensory-motor experiences, several theories have been proposed. I will discuss two of these proposals, Conceptual Metaphor Theory and Situated Cognition, and argue why they do not fully explain grounding. A central idea in Conceptual Metaphor Theory is that image schemas ground abstract concepts in concrete experiences. Image schemas might themselves be abstractions, however, and therefore do not solve the grounding problem. Moreover, image schemas are too simple to explain the full richness of (...) abstract concepts. Situated cognition might provide such richness. Research in our laboratory, however, has shown that even for concrete concepts, sensory-motor grounding is task dependent. Therefore, it is questionable whether abstract concepts can be significantly grounded in sensory-motor processing. (shrink)

Previous reports have demonstrated that the comprehension of sentences describing motion in a particular direction (toward, away, up, or down) is affected by concurrently viewing a stimulus that depicts motion in the same or opposite direction. We report 3 experiments that extend our understanding of the relation between perception and language processing in 2 ways. First, whereas most previous studies of the relation between perception and language processing have focused on visual perception, our data show that sentence processing can be (...) affected by the concurrent processing of auditory stimuli. Second, it is shown that the relation between the processing of auditory stimuli and the processing of sentences depends on whether the sentences are presented in the auditory or visual modality. (shrink)

The grounding of symbols in computational models of linguistic abilities is one of the fundamental properties of psychologically plausible cognitive models. In this article, we present an embodied model for the grounding of language in action based on epigenetic robots. Epigenetic robotics is one of the new cognitive modeling approaches to modeling autonomous mental development. The robot model is based on an integrative vision of language in which linguistic abilities are strictly dependent on and grounded in other behaviors and skills. (...) It uses simulated robots that learn through imitation the names of basic actions. Robots also learn higher order action concepts through the process of grounding transfer. The simulation demonstrates how new, higher order behavioral abilities can be autonomously built on previously grounded basic action categories following linguistic interaction with human users. (shrink)

This study investigates how speed of motion is processed in language. In three eye-tracking experiments, participants were presented with visual scenes and spoken sentences describing fast or slow events (e.g., The lion ambled/dashed to the balloon). Results showed that looking time to relevant objects in the visual scene was affected by the speed of verb of the sentence, speaking rate, and configuration of a supporting visual scene. The results provide novel evidence for the mental simulation of speed in language and (...) show that internal dynamic simulations can be played out via eye movements toward a static visual scene. (shrink)

Experientialist semantics has contributed to a broader notion of linguistic meaning by emphasizing notions such as construal, perspective, metaphor, and embodiment, but has suffered from an individualist concept of meaning and has conflated experiential motivations with conventional semantics. We argue that these problems can be redressed by methods and concepts from phenomenology, on the basis of a case study of sentences of non-actual motion such as “The mountain range goes all the way from Mexico to Canada.” Through a phenomenological reanalysis (...) of proposals of Talmy, Langacker, and Matlock, we show that non-actual motion is both experientially and linguistically non-unitary. At least three different features of human consciousness—enactive perception, visual scanning, and imagination—constitute experiential motivations for non-actual motion sentences, and each of these could be related to phenomenological analyses of human intentionality. The second problem is addressed by proposing that the experiential motivations of non-actual motion sentences can be viewed as sedimented through “passive” processes of acquisition and social transmission and that this implies an interactive loop between experience and language, yielding losses in terms of original experience, but gains in terms of communal signification. Something that is underestimated by phenomenology is that what is sedimented are not only intentional objects such as states of affairs, but aspects of how they are given, i.e., the original, temporal, bodily experiences themselves. Since cognitive semantics has emphasized such aspects of meaning, we suggest that phenomenology can itself benefit from experientialist semantics, especially when it turns its focus from prepredicative to predicative, linguistic intentionality. (shrink)

An eye-tracking while listening study based on the blank screen paradigm was conducted to investigate the processing of literal and metaphorical verbs of motion. The study was based on two assumpti...

Event sequences or scripts are the conceptual representations of activities in memory. Traditional views hold that events are represented in amodal networks and are retrieved by associative strategies. The embodied cognition approach holds that knowledge is grounded in perception and retrieved by mental simulation. We used a script generation task where event sequences of activities had to be produced. Activities varied in their degree of familiarity. In a regressional design we investigated whether amodal or perceptual variables best predicted knowledge retrieval (...) to gain insight into the underlying representation. Retrieval depended on the familiarity of the activity. While novel activities mainly relied on perception‐based simulation and to a lesser extent on associative strategies, moderately familiar activities showed the opposite pattern, and events of familiar activities were retrieved by association alone. We conclude that amodal structures exist in all representations that become stronger with increasing frequency and finally prevail over perceptual structures. (shrink)

The current study assessed the extent to which the use of referential prosody varies with communicative demand. Speaker–listener dyads completed a referential communication task during which speakers attempted to indicate one of two color swatches (one bright, one dark) to listeners. Speakers' bright sentences were reliably higher pitched than dark sentences for ambiguous (e.g., bright red versus dark red) but not unambiguous (e.g., bright red versus dark purple) trials, suggesting that speakers produced meaningful acoustic cues to brightness when the accompanying (...) linguistic content was underspecified (e.g., “Can you get the red one?”). Listening partners reliably chose the correct corresponding swatch for ambiguous trials when lexical information was insufficient to identify the target, suggesting that listeners recruited prosody to resolve lexical ambiguity. Prosody can thus be conceptualized as a type of vocal gesture that can be recruited to resolve referential ambiguity when there is communicative demand to do so. (shrink)

When describing motion along both the horizontal and vertical axes, languages from different families express the elements encoding verticality before those coding for horizontality (e.g., going up right instead of right up). In light of the motor grounding of language, the present study investigated whether the prevalence of verticality in Path expression also governs the trajectory of arm biological movements. Using a 3D virtual-reality setting, we tracked the kinematics of hand pointing movements in five spatial directions, two of which implied (...) the vertical and horizontal vectors equally (i.e., up right +45° and bottom right −45°). Movement onset could be prompted by visual or auditory verbal cues, the latter being canonical in French (“en haut à droite”/up right) or not (“à droite en haut”/right up). In two experiments, analyses of the index finger kinematics revealed a significant effect of gravity, with earlier acceleration, velocity, and deceleration peaks for upward (+45°) than downward (−45°) movements, irrespective of the instructions. Remarkably, confirming the linguistic observations, we found that vertical kinematic parameters occurred earlier than horizontal ones for upward movements, both for visual and congruent verbal cues. Non-canonical verbal instructions significantly affected this temporal dynamic: for upward movements, the horizontal and vertical components temporally aligned, while they reversed for downward movements where the kinematics of the vertical axis was delayed with respect to that of the horizontal one. This temporal dynamic is so deeply anchored that non-canonical verbal instructions allowed for horizontality to precede verticality only for movements that do not fight against gravity. Altogether, our findings provide new insights into the embodiment of language by revealing that linguistic path may reflect the organization of biological movements, giving priority to the vertical axis. (shrink)

Spatial aspects of words are associated with their canonical locations in the real world. Yet little research has tested whether spatial associations denoted in language comprehension generalize to their corresponding images. We directly tested the spatial aspects of mental imagery in picture and word processing (Experiment 1). We also tested whether spatial representations of motion words produce similar perceptual-interference effects as demonstrated by object words (Experiment 2). Findings revealed that words denoting an upward spatial location produced slower responses to targets (...) appearing at the top of the display, whereas words denoting a downward spatial location produced slower responses to targets appearing at the bottom of the display. Perceptual-interference effects did not obtain for pictures or for words lacking a spatial relation. These findings provide greater empirical support for the perceptual-symbols system theory (Barsalou, 1999, 2008). (shrink)