To describe phenomena that occur at different time scales, computational models of the brain must incorporate different levels of abstraction. At time scales of approximately 1/3 of a second, orienting movements of the body play a crucial role in cognition and form a useful computational level embodiment level,” the constraints of the physical system determine the nature of cognitive operations. The key synergy is that at time scales of about 1/3 of a second, the natural sequentiality of body movements can (...) be matched to the natural computational economies of sequential decision systems through a system of implicit reference called deictic in which pointing movements are used to bind objects in the world to cognitive programs. This target article focuses on how deictic bindings make it possible to perform natural tasks. Deictic computation provides a mechanism for representing the essential features that link external sensory data with internal cognitive programs and motor actions. One of the central features of cognition, working memory, can be related to moment-by-moment dispositions of body features such as eye movements and hand movements. (shrink)

Much indirect evidence supports the hypothesis that transformations of mental images are at least in part guided by motor processes, even in the case of images of abstract objects rather than of body parts. For example, rotation may be guided by processes that also prime one to see results of a specific motor action. We directly test the hypothesis by means of a dual-task paradigm in which subjects perform the Cooper-Shepard mental rotation task while executing an unseen motor rotation in (...) a given direction and at a previously learned speed. Four results support the inference that mental rotation relies on motor processes. First, motor rotation that is compatible with mental rotation results in faster times and fewer errors in the imagery task than when the two rotations are incompatible. Second, the angle through which subjects rotate their mental images, and the angle through which they rotate a joystick handle are correlated, but only if the directions of the two rotations are compatible. Third, motor rotation modifies the classical inverted V-shaped mental rotation response time function, favoring the direction of the motor rotation; indeed, in some cases motor rotation even shifts the location of the minimum of this curve in the direction of the motor rotation. Fourth, the preceding effect is sensitive not only to the direction of the motor rotation, but also to the motor speed. A change in the speed of motor rotation can correspondingly slow down or speed up the mental rotation. (shrink)

In attempting to solve a wide variety of tasks, people naturally seek to modify their external environment such that the physical space in which they work is more amenable or ‘congenial’ to achieving a desired outcome. Attempts to determine the effectiveness of certain artifacts or spatial reorganizations in aiding reasoners solve problems must be relativised to the difficulty of the task and the cognitive abilities of the reasoners. These factors were examined using a simple word production task with letter tiles. (...) Two sets of tiles that differed in terms of word production difficulty were selected. Participants were asked to produce as many words as they could within a finite time period for each letter set. In one group, participants were encouraged to rearrange or touch the tiles when producing words, and in the other group, participants could not interact with or point to the tiles. Participants were further split in a low and high verbal fluency group as a function of their score on the Thurstone word fluency test taken at the end of the experiment. In the high fluency group, letter rearrangement did not improve the participants’ ability to generate words. In contrast, in the low verbal fluency group, letter rearrangement significantly enhanced the ability to produce new words from both the hard and easy letter set. For these participants, the task was more taxing and the opportunity to restructure the letter set substantially elevated their performance. The advantages of a distributed cognition analysis of this and other reasoning tasks are discussed. (shrink)

A productive way to think about imagistic mental models of physical systems is as though they were sources of quasi‐empirical evidence. People depict or imagine events at those points in time when they would experiment with the world if possible. Moreover, just as they would do when observing the world, people induce patterns of behavior from the results depicted in their imaginations. These resulting patterns of behavior can then be cast into symbolic rules to simplify thinking about future problems and (...) to reveal higher order relationships. Using simple gear problems, three experiments explored the occasions of use for, and the inductive transitions between, depictive models and number‐based rules. The first two experiments used the convergent evidence of problem‐solving latencies, hand motions, referential language and error data to document the initial use of a model, the induction of rules from the modeling results, and the fallback to a model when a rule fails. The third experiment explored the intermediate representations that facilitate the induction of rules from depictive models. The strengths and weaknesses of depictive modeling and more analytic systems of reasoning are delineated to motivate the reasons for these transitions. (shrink)

Why do people create extra representations to help them make sense of situations, diagrams, illustrations, instructions and problems? The obvious explanation— external representations save internal memory and com- putation—is only part of the story. I discuss seven ways external representations enhance cognitive power: they change the cost structure of the inferential landscape; they provide a structure that can serve as a shareable object of thought; they create persistent referents; they facilitate re- representation; they are often a more natural representation of (...) structure than mental representations; they facilitate the computation of more explicit encoding of information; they enable the construction of arbitrarily complex structure; and they lower the cost of controlling thought—they help coordinate thought. Jointly, these functions allow people to think more powerfully with external representations than without. They allow us to think the previously unthinkable. (shrink)

We present data and argument to show that in Tetris - a real-time interactive video game - certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world rather than by performing computational actions in the head alone. We have found that some translations and rotations are best understood as using the world to improve cognition. These actions are not used to implement a plan, or to implement a reaction; they are used to (...) change the world in order to simplify the problem-solving task. Thus, we distinguish pragmatic actions ñ actions performed to bring one physically closer to a goal - from epistemic actions - actions performed to uncover information that is hidden or hard to compute mentally. To illustrate the need for epistemic actions, we first develop a standard information-processing model of Tetris-cognition, and show that it cannot explain performance data from human players of the game - even when we relax the assumption of fully sequential processing. Standard models disregard many actions taken by players because they appear unmotivated or superfluous. However, we describe many such actions that are actually taken by players that are far from superfluous, and that play valuable roles in improving human performance. We argue that traditional accounts are limited because they regard action as having a single function: to change the world. By recognizing a second function of action - an epistemic function - we can explain many of the actions that a traditional model cannot. Although, our argument is supported by numerous examples specifically from Tetris, we outline how the one category of epistemic action can be incorporated into theories of action more generally. (shrink)

Constraints and dependencies among the elements of embodied cognition form patterns or microstrategies of interactive behavior. Hard constraints determine which microstrategies are possible. Soft constraints determine which of the possible microstrategies are most likely to be selected. When selection is non‐deliberate or automatic the least effort microstrategy is chosen. In calculating the effort required to execute a microstrategy each of the three types of operations, memory retrieval, perception, and action, are given equal weight; that is, perceptual‐motor activity does not have (...) a privileged status with respect to memory. Soft constraints can work contrary to the designer's intentions by making the access of perfect knowledge in‐the‐world more effortful than the access of imperfect knowledge in‐the‐head. These implications of soft constraints are tested in two experiments. In experiment 1 we varied the perceptual‐motor effort of accessing knowledge in‐the‐world as well as the effort of retrieving items from memory. In experiment 2 we replicated one of the experiment 1 conditions to collect eye movement data. The results suggest that milliseconds matter. Soft constraints lead to a reliance on knowledge in‐the‐head even when the absolute difference in perceptual‐motor versus memory retrieval effort is small, and even when relying on memory leads to a higher error rate and lower performance. We discuss the implications of soft constraints for routine interactive behavior, accounts of embodied cognition, and tool and interface design. (shrink)

A basic claim of the embodiment framework is that all psychological processes are influenced by body morphology, sensory systems, motor systems, and emotions. As such, the framework holds the promise of providing a unifying perspective for psychological research. This article begins with a sketch of several arguments, from evolution to philosophy, as to why the embodiment framework is a good bet. These arguments are followed by a review of approaches to embodiment, including those from cognitive linguistics, perceptual symbol theory, and (...) action-based theories. Finally, examples are provided for how a unifying perspective might work for cognition (including language and memory), cognitive and social development, social psychology, neuroscience, clinical psychology, and psychology applied to education. (shrink)

We are quickly passing through the historical moment when people work in front of a single computer, dominated by a small CRT and focused on tasks involving only local information. Networked computers are becoming ubiquitous and are playing increasingly significant roles in our lives and in the basic infrastructure of science, business, and social interaction. For human-computer interaction o advance in the new millennium we need to better understand the emerging dynamic of interaction in which the focus task is no (...) longer confined to the desktop but reaches into a complex networked world of information and computer-mediated interactions. We think the theory of distributed cognition has a special role to play in understanding interactions between people and technologies, for its focus has always been on whole environments: what we really do in them and how we coordinate our activity in them. Distributed cognition provides a radical reorientation of how to think about designing and supporting human-computer interaction. As a theory it is specifically tailored to understanding interactions among people and technologies. In this article we propose distributed cognition as a new foundation for human-computer interaction, sketch an integrated research framework, and use selections from our earlier work to suggest how this framework can provide new opportunities in the design of digital work materials. (shrink)

This paper examines some of the methods animals and humans have of adapting their environment. Because there are limits on how many different tasks a creature can be designed to do well in, creatures with the capacity to redesign their environments have an adaptive advantage over those who can only passively adapt to existing environmental structures. To clarify environmental redesign I rely on the formal notion of a task environment as a directed graph where the nodes are states and the (...) links are actions. One natural form of redesign is to change the topology of this graph structure so as to increase the likelihood of task success or to reduce its expected cost, measured in physical terms. This may be done by eliminating initial states hence eliminating choice points; by changing the action repertoire; by changing the consequence function; and lastly, by adding choice points. Another major method for adapting the environment is to change its cognitive congeniality. Such changes leave the state space formally intact but reduce the number and cost of mental operations needed for task success; they reliably increase the speed, accuracy or robustness of performance. The last section of the paper describes several of these epistemic or complementary actions found in human performance. (shrink)