How do we understand time and other entities we can neither touch nor see? One possibility is that we tap into our concrete, experiential knowledge, including our understanding of physical space and motion, to make sense of abstract domains such as time. To examine how pervasive an aspect of cognition this is, we investigated whether thought about a nonliteral type of motion called fictive motion (FM; as in The road runs along the coast) can influence thought about time. Our results (...) suggest that FM uses the same structures evoked in understanding literal motion, and that these literal aspects of FM influence temporal reasoning. (shrink)

Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statis- tics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement record- ing systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and neuroscience. During perceptual experience, association areas in the (...) brain capture bottom-up patterns of activation in sensory-motor areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The stor- age and reactivation of perceptual symbols operates at the level of perceptual components – not at the level of holistic perceptual expe- riences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a com- mon frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift, run) and introspec- tion (e.g., compare, memory, happy, hungry). Once established, these simulators implement a basic conceptual system that represents types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and ab- stract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinato- rially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a per- ceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal sym- bol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored. (shrink)

Two studies investigated the interaction between utterance and scene processing by monitoring eye movements in agent–action–patient events, while participants listened to related utterances. The aim of Experiment 1 was to determine if and when depicted events are used for thematic role assignment and structural disambiguation of temporarily ambiguous English sentences. Shortly after the verb identified relevant depicted actions, eye movements in the event scenes revealed disambiguation. Experiment 2 investigated the relative importance of linguistic/world knowledge and scene information. When the verb (...) identified either only the stereotypical agent of a (nondepicted) action, or the (nonstereotypical) agent of a depicted action as relevant, verb‐based thematic knowledge and depicted action each rapidly influenced comprehension. In contrast, when the verb identified both of these agents as relevant, the gaze pattern suggested a preferred reliance of comprehension on depicted events over stereotypical thematic knowledge for thematic interpretation. We relate our findings to language comprehension and acquisition theories. (shrink)

There is mounting evidence that language comprehension involves the activation of mental imagery of the content of utterances (; ; ; ; ; ; ). This imagery can have motor or perceptual content. Three main questions about the process remain under‐explored, however. First, are lexical associations with perception or motion sufficient to yield mental simulation, or is the integration of lexical semantics into larger structures, like sentences, necessary? Second, what linguistic elements (e.g., verbs, nouns, etc.) trigger mental simulations? Third, how (...) detailed are the visual simulations that are performed? A series of behavioral experiments address these questions, using a visual object categorization task to investigate whether up‐ or down‐related language selectively interferes with visual processing in the same part of the visual field (following ). The results demonstrate that either subject nouns or main verbs can trigger visual imagery, but only when used in literal sentences about real space—metaphorical language does not yield significant effects—which implies that it is the comprehension of the sentence as a whole and not simply lexical associations that yields imagery effects. These studies also show that the evoked imagery contains detail as to the part of the visual field where the described scene would take place. (shrink)

Cognitive theories of metaphor understanding are typically described in terms of the mappings between different kinds of abstract, schematic, disembodied knowledge. My claim in this paper is that part of our ability to make sense of metaphorical language, both individual utterances and extended narratives, resides in the automatic construction of a simulation whereby we imagine performing the bodily actions referred to in the language. Thus, understanding metaphorical expressions like ‘grasp a concept’ or ‘get over’ an emotion involve simulating what it (...) must be like to engage in these specific activities, even though these actions are, strictly speaking, impossible to physically perform. This process of building a simulation, one that is fundamentally embodied in being constrained by past and present bodily experiences, has specific consequences for how verbal metaphors are understood, and how cognitive scientists, more generally, characterize the nature of metaphorical language and thought. (shrink)

Eye movements during mental imagery are not epiphenomenal but assist the process of image generation. Commands to the eyes for each fixation are stored along with the visual representation and are used as spatial index in a motor‐based coordinate system for the proper arrangement of parts of an image. In two experiments, subjects viewed an irregular checkerboard or color pictures of fish and were subsequently asked to form mental images of these stimuli while keeping their eyes open. During the perceptual (...) phase, a group of subjects was requested to maintain fixation onto the screen's center, whereas another group was free to inspect the stimuli. During the imagery phase, all of these subjects were free to move their eyes. A third group of subjects (in Experiment 2) was free to explore the pattern but was requested to maintain central fixation during imagery. For subjects free to explore the pattern, the percentage of time spent fixating a specific location during perception was highly correlated with the time spent on the same (empty) locations during imagery. The order of scanning of these locations during imagery was correlated to the original order during perception. The strength of relatedness of these scanpaths and the vividness of each image predicted performance accuracy. Subjects who fixed their gaze centrally during perception did the same spontaneously during imagery. Subjects free to explore during perception, but maintaining central fixation during imagery, showed decreased ability to recall the pattern. We conclude that the eye scanpaths during visual imagery reenact those of perception of the same visual scene and that they play a functional role. (shrink)

Cognitive theories of metaphor understanding are typically described in terms of the mappings between different kinds of abstract, schematic, disembodied knowledge. My claim in this paper is that part of our ability to make sense of metaphorical language, both individual utterances and extended narratives, resides in the automatic construction of a simulation whereby we imagine performing the bodily actions referred to in the language. Thus, understanding metaphorical expressions like ‘grasp a concept’ or ‘get over’ an emotion involve simulating what it (...) must be like to engage in these specific activities, even though these actions are, strictly speaking, impossible to physically perform. This process of building a simulation, one that is fundamentally embodied in being constrained by past and present bodily experiences, has specific consequences for how verbal metaphors are understood, and how cognitive scientists, more generally, characterize the nature of metaphorical language and thought. (shrink)

Cognitive theories of metaphor understanding are typically described in terms of the mappings between different kinds of abstract, schematic, disembodied knowledge. My claim in this paper is that part of our ability to make sense of metaphorical language, both individual utterances and extended narratives, resides in the automatic construction of a simulation whereby we imagine performing the bodily actions referred to in the language. Thus, understanding metaphorical expressions like ‘grasp a concept’ or ‘get over’ an emotion involve simulating what it (...) must be like to engage in these specific activities, even though these actions are, strictly speaking, impossible to physically perform. This process of building a simulation, one that is fundamentally embodied in being constrained by past and present bodily experiences, has specific consequences for how verbal metaphors are understood, and how cognitive scientists, more generally, characterize the nature of metaphorical language and thought. (shrink)

This study provides evidence that eye movements reflect the positions of objects while participants listen to a spoken description, retell a previously heard spoken description, and describe a previously seen picture. This effect is equally strong in retelling from memory, irrespective of whether the original elicitation was spoken or visual. In addition, this effect occurs both while watching a blank white board and while sitting in complete darkness. This study includes 4 experiments. The first 2 experiments measured eye movements of (...) participants looking at a blank white board. Experiment 1 monitors eye movements of participants on 2 occasions: first, when participants listened to a prerecorded spoken scene description; second, when participants were later retelling it from memory. Experiment 2 first monitored eye movements of participants as they studied a complex picture visually, and then later as they described it from memory. The second pair of experiments (Experiments 3 and 4) replicated Experiments 1 and 2 with the only difference being that they were executed in complete darkness. This method of analysis differentiated between eye movements that are categorically correct relative to the positions of the whole eye gaze pattern (global correspondence) and eye movements that are only locally correct (local correspondence). The discussion relates the findings to the current debate on mental imagery. (shrink)