heories of embodied cognition (e.g., Perceptual Symbol Systems Theory; Barsalou, 1999, 2009) suggest that modality specific simulations underlie the representation of concepts. Supporting evidence comes from modality switch costs: participants are slower to verify a property in one modality (e.g., auditory, BLENDER-loud) after verifying a property in a different modality (e.g., gustatory, CRANBERRIES-tart) compared to the same modality (e.g., LEAVES-rustling, Pecher et al., 2003). Similarly, modality switching costs lead to a modulation of the N400 effect in event-related potentials (ERPs; Collins (...) et al., 2011; Hald et al., 2011). This effect of modality switching has also been shown to interact with the veracity of the sentence (Hald et al., 2011). The current ERP study further explores the role of modality match/mismatch on the processing of veracity as well as negation (sentences containing “not”). Our results indicate a modulation in the ERP based on modality and veracity, plus an interaction. The evidence supports the idea that modality specific simula- tions occur during language processing, and furthermore suggest that these simulations alter the processing of negation. (shrink)

Recently, there has been a great deal of interest in the idea that natural language enhances and extends our cognitive capabilities. Supporters of embodied cognition have been particularly interested in the way in which language may provide a solution to the problem of abstract concepts. Toward this end, some have emphasized the way in which language may act as form of cognitive scaffolding and others have emphasized the potential importance of language-based distributional information. This essay defends a version of the (...) cognitive enhancement thesis that integrates and builds on both of these proposals. I argue that the embodied representations associated with language processing serve as a supplementary medium for conceptual processing. The acquisition of a natural language provides a means of extending our cognitive reach by giving us access to an internalized combinatorial symbol system that augments and supports the context-sensitive embodied representational systems that exist independently of language. (shrink)

As embodied theories of cognition are increasingly formalized and tested, care must be taken to make informed assumptions regarding the nature of concepts and representations. In this study, we outline three reasons why one cannot, in effect, represent the same concept twice. First, online perception affects offline representation: Current representational content depends on how ongoing demands direct attention to modality-specific systems. Second, language is a fundamental facilitator of offline representation: Bootstrapping and shortcuts within the computationally cheaper linguistic system continuously modify (...) representational content. Third, time itself is a source of representational change: As the content of underlying concepts shifts with the accumulation of direct and vicarious experience, so too does the content of representations that draw upon these concepts. We discuss the ramifications of these principles for research into both human and synthetic cognitive systems. (shrink)

This study investigates how speed of motion is processed in language. In three eye-tracking experiments, participants were presented with visual scenes and spoken sentences describing fast or slow events (e.g., The lion ambled/dashed to the balloon). Results showed that looking time to relevant objects in the visual scene was affected by the speed of verb of the sentence, speaking rate, and configuration of a supporting visual scene. The results provide novel evidence for the mental simulation of speed in language and (...) show that internal dynamic simulations can be played out via eye movements toward a static visual scene. (shrink)

Humans can understand thousands of abstract words, even when they do not have clearly perceivable referents. Recent views highlight an important role of social experience in grounding of abstract concepts and sub-kinds of abstract concepts, but empirical work in this area is still in its early stages. In the present study, a picture-word semantic priming paradigm was employed to investigate the contribution effect of social experience that is provided by real-life pictures to social abstract concepts and emotional abstract concepts. Using (...) a lexical decision task, we examined responses to picture-SA word pairs and picture-EA word pairs in social/emotional semantically related and unrelated conditions. All pairs shared either positive or negative valence. The results showed quicker responses to positive SA and EA words that were preceded by related vs. unrelated prime pictures. Specifically, positive SA words were facilitated by the corresponding social scene pictures, whereas positive EA words were facilitated by pictures depict the corresponding facial expressions and gestures. However, such facilitatory effect was not observed in negative picture-SA/EA word conditions. This pattern of results suggests that a facilitatory effect of social experience on abstract concepts varies with different sub-kinds of abstract concepts, that seems to be limited to positive SA concepts. Overall, our findings confirm the crucial role of social experience for abstract concepts and further suggest that not all abstract concepts can benefit from social experience, at least in the semantic priming. (shrink)

The human conceptual system comprises simulated information of sensorimotor experience and linguistic distributional information of how words are used in language. Moreover, the linguistic shortcut hypothesis predicts that people will use computationally cheaper linguistic distributional information where it is sufficient to inform a task response. In a pre‐registered category production study, we asked participants to verbally name members of concrete and abstract categories and tested whether performance could be predicted by a novel measure of sensorimotor similarity (based on an 11‐dimensional (...) representation of sensorimotor strength) and linguistic proximity (based on word co‐occurrence derived from a large corpus). As predicted, both measures predicted the order and frequency of category production but, critically, linguistic proximity had an effect above and beyond sensorimotor similarity. A follow‐up study using typicality ratings as an additional predictor found that typicality was often the strongest predictor of category production variables, but it did not subsume sensorimotor and linguistic effects. Finally, we created a novel, fully grounded computational model of conceptual activation during category production, which best approximated typical human performance when conceptual activation was allowed to spread indirectly between concepts, and when candidate category members came from both sensorimotor and linguistic distributional representations. Critically, model performance was indistinguishable from typical human performance. Results support the linguistic shortcut hypothesis in semantic processing and provide strong evidence that both linguistic and grounded representations are inherent to the functioning of the conceptual system. All materials, data, and code are available at https://osf.io/vaq56/. (shrink)

A growing body of evidence suggests that cognition is embodied and grounded. Abstract concepts, though, remain a significant theoretical chal- lenge. A number of researchers have proposed that language makes an important contribution to our capacity to acquire and employ concepts, particularly abstract ones. In this essay, I critically examine this suggestion and ultimately defend a version of it. I argue that a successful account of how language augments cognition should emphasize its symbolic properties and incorporate a view of embodiment (...) that recognizes the flexible, multi- modal and task-related nature of action, emotion and perception systems. On this view, language is an ontogenetically disruptive cognitive technology that expands our conceptual reach. (shrink)

In an evolutionary perspective Louwerse elaborates the Symbol Interdependency Hypothesis (Louwerse, 2011), arguing that language has evolved such that it maps onto the perceptual system, allowing to bootstrap meaning also when grounding is limited. The author concludes that in principle the processing of abstract and concrete words is the same and that in both cases language users tend to rely anyway on indexical relationships that words entertain with other words.

concepts typically are defined in terms of lacking physical or perceptual referents. We argue instead that they are not devoid of perceptual information because knowledge of real-world situations is an important component of learning and using many abstract concepts. Although the relationship between perceptual information and abstract concepts is less straightforward than for concrete concepts, situation-based perceptual knowledge is part of many abstract concepts. In Experiment 1, participants made lexical decisions to abstract words that were preceded by related and unrelated (...) pictures of situations. For example, share was preceded by a picture of two girls sharing a cob of corn. When pictures were presented for 500 ms, latencies did not differ. However, when pictures were presented for 1,000 ms, decision latencies were significantly shorter for abstract words preceded by related versus unrelated pictures. Because the abstract concepts corresponded to the pictured situation as a whole, rather than a single concrete object or entity, the necessary relational processing takes time. In Experiment 2, on each trial, an abstract word was presented for 250 ms, immediately followed by a picture. Participants indicated whether or not the picture showed a normal situation. Decision latencies were significantly shorter for pictures preceded by related versus unrelated abstract words. Our experiments provide evidence that knowledge of events and situations is important for learning and using at least some types of abstract concepts. That is, abstract concepts are grounded in situations, but in a more complex manner than for concrete concepts. Although people's understanding of abstract concepts certainly includes knowledge gained from language describing situations and events for which those concepts are relevant, sensory and motor information experienced during real-life events is important as well. (shrink)

Over the past 15 years, there have been two increasingly popular approaches to the study of meaning in cognitive science. One, based on theories of embodied cognition, treats meaning as a simulation of perceptual and motor states. An alternative approach treats meaning as a consequence of the statistical distribution of words across spoken and written language. On the surface, these appear to be opposing scientific paradigms. In this review, we aim to show how recent cross-disciplinary developments have done much to (...) reconcile these two approaches. The foundation to these developments has been the recognition that intralinguistic distributional and sensory-motor data are interdependent. We describe recent work in philosophy, psychology, cognitive neuroscience, and computational modeling that are all based on or consistent with this conclusion. We conclude by considering some possible directions for future research that arise as a consequence of these developments. (shrink)

Spatial mental representations can be derived from linguistic and non‐linguistic sources of information. This study tested whether these representations could be formed from statistical linguistic frequencies of city names, and to what extent participants differed in their performance when they estimated spatial locations from language or maps. In a computational linguistic study, we demonstrated that co‐occurrences of cities in Tolkien’s Lord of the Rings trilogy and The Hobbit predicted the authentic longitude and latitude of those cities in Middle Earth. In (...) a human study, we showed that human spatial estimates of the location of cities were very similar regardless of whether participants read Tolkien’s texts or memorized a map of Middle Earth. However, text‐based location estimates obtained from statistical linguistic frequencies better predicted the human text‐based estimates than the human map‐based estimates. These findings suggest that language encodes spatial structure of cities, and that human cognitive map representations can come from implicit statistical linguistic patterns, from explicit non‐linguistic perceptual information, or from both. (shrink)

Previous studies have shown that object properties are processed faster when they follow properties from the same perceptual modality than properties from different modalities. These findings suggest that language activates sensorimotor processes, which, according to those studies, can only be explained by a modal account of cognition. The current paper shows how a statistical linguistic approach of word co-occurrences can also reliably predict the category of perceptual modality a word belongs to (auditory, olfactory–gustatory, visual–haptic), even though the statistical linguistic approach (...) is less precise than the modal approach (auditory, gustatory, haptic, olfactory, visual). Moreover, the statistical linguistic approach is compared with the modal embodied approach in an experiment in which participants verify properties that share or shift modalities. Response times suggest that fast responses can best be explained by the linguistic account, whereas slower responses can best be explained by the embodied account. These results provide further evidence for the theory that conceptual processing is both linguistic and embodied, whereby less precise linguistic processes precede precise simulation processes. (shrink)

Whether computational algorithms such as latent semantic analysis (LSA) can both extract meaning from language and advance theories of human cognition has become a topic of debate in cognitive science, whereby accounts of symbolic cognition and embodied cognition are often contrasted. Albeit for different reasons, in both accounts the importance of statistical regularities in linguistic surface structure tends to be underestimated. The current article gives an overview of the symbolic and embodied cognition accounts and shows how meaning induction attributed to (...) a specific statistical process or to activation of embodied representations should be attributed to language itself. Specifically, the performance of LSA can be attributed to the linguistic surface structure, more than special characteristics of the algorithm, and embodiment findings attributed to perceptual simulations can be explained by distributional linguistic information. (shrink)

Recent investigations on how people derive meaning from language have focused on task‐dependent shifts between two cognitive systems. The symbolic (amodal) system represents meaning as the statistical relationships between words. The embodied (modal) system represents meaning through neurocognitive simulation of perceptual or sensorimotor systems associated with a word's referent. A primary finding of literature in this field is that the embodied system is only dominant when a task necessitates it, but in certain paradigms, this has only been demonstrated using nouns (...) and adjectives. The purpose of this paper is to study whether similar effects hold with verbs. Experiment 1 evaluated a novel task in which participants rated a selection of verbs on their implied vertical movement. Ratings correlated well with distributional semantic models, establishing convergent validity, though some variance was unexplained by language statistics alone. Experiment 2 replicated previous noun‐based location‐cue congruency experimental paradigms with verbs and showed that the ratings obtained in Experiment 1 predicted reaction times more strongly than language statistics. Experiment 3 modified the location‐cue paradigm by adding movement to create an animated, temporally decoupled, movement‐verb judgment task designed to examine the relative influence of symbolic and embodied processing for verbs. Results were generally consistent with linguistic shortcut hypotheses of symbolic‐embodied integrated language processing; location‐cue congruence elicited processing facilitation in some conditions, and perceptual information accounted for reaction times and accuracy better than language statistics alone. These studies demonstrate novel ways in which embodied and linguistic information can be examined while using verbs as stimuli. (shrink)