The human sentence processor is able to make rapid predictions about upcoming linguistic input. For example, upon hearing the verb eat, anticipatory eye-movements are launched toward edible objects in a visual scene. However, the cognitive mechanisms that underlie anticipation remain to be elucidated in ecologically valid contexts. Previous research has, in fact, mainly used clip-art scenes and object arrays, raising the possibility that anticipatory eye-movements are limited to displays containing a small number of objects in a visually impoverished context. In (...) Experiment 1, we confirm that anticipation effects occur in real-world scenes and investigate the mechanisms that underlie such anticipation. In particular, we demonstrate that real-world scenes provide contextual information that anticipation can draw on: When the target object is not present in the scene, participants infer and fixate regions that are contextually appropriate. Experiment 2 investigates whether such contextual inference requires the co-presence of the scene, or whether memory representations can be utilized instead. The same real-world scenes as in Experiment 1 are presented to participants, but the scene disappears before the sentence is heard. We find that anticipation occurs even when the screen is blank, including when contextual inference is required. We conclude that anticipatory language processing is able to draw upon global scene representations to make contextual inferences. These findings are compatible with theories assuming contextual guidance, but posit a challenge for theories assuming object-based visual indices. (shrink)

Memory is fleeting. New material rapidly obliterates previous material. How, then, can the brain deal successfully with the continual deluge of linguistic input? We argue that, to deal with this “Now-or-Never” bottleneck, the brain must compress and recode linguistic input as rapidly as possible. This observation has strong implications for the nature of language processing: (1) the language system must “eagerly” recode and compress linguistic input; (2) as the bottleneck recurs at each new representational level, the language system must build (...) a multilevel linguistic representation; and (3) the language system must deploy all available information predictively to ensure that local linguistic ambiguities are dealt with “Right-First-Time”; once the original input is lost, there is no way for the language system to recover. This is “Chunk-and-Pass” processing. Similarly, language learning must also occur in the here and now, which implies that language acquisition is learning to process, rather than inducing, a grammar. Moreover, this perspective provides a cognitive foundation for grammaticalization and other aspects of language change. Chunk-and-Pass processing also helps explain a variety of core properties of language, including its multilevel representational structure and duality of patterning. This approach promises to create a direct relationship between psycholinguistics and linguistic theory. More generally, we outline a framework within which to integrate often disconnected inquiries into language processing, language acquisition, and language change and evolution. (shrink)

Previous research has shown that language comprehenders resolve reference quickly and incrementally, but not much is known about the neural processes and representations that are involved. Studies of visual short-term memory suggest that access to the representation of an item from a previously seen display is associated with a negative evoked potential at posterior electrodes contralateral to the spatial location of that item in the display. In this paper we demonstrate that resolving the reference of a noun phrase in a (...) recently seen visual display is associated with an event-related potential that is analogous to this effect. Our design was adapted from the visual world paradigm: in each trial, participants saw a display containing three simple objects, followed by a question about the objects, such as Was the pink fish next to a boat?, presented word by word. Questions differed in whether the color adjective allowed the reader to identify the referent of the noun phrase or not (i.e., whether one or more objects of the named color were present). Consistent with our hypothesis, we observed that reference resolution by the adjective was associated with a negative evoked potential at posterior electrodes contralateral to spatial location of the referent, starting approximately 333 ms after the onset of the adjective. The fact that the laterality of the effect depended upon the location of the referent within the display suggests that reference resolution in visual domains involves, at some level, a modality-specific representation. In addition, the effect gives us an estimate of the time course of processing from perception of the written word to the point at which its meaning is brought into correspondence with the referential domain. (shrink)

We identify a number of principles with respect to prediction that, we argue, underpin adult language comprehension: (a) comprehension consists in realizing a mapping between the unfolding sentence and the event representation corresponding to the real‐world event being described; (b) the realization of this mapping manifests as the ability to predict both how the language will unfold, and how the real‐world event would unfold if it were being experienced directly; (c) concurrent linguistic and nonlinguistic inputs, and the prior internal states (...) of the system, each drive the predictive process; (d) the representation of prior internal states across a representational substrate common to the linguistic and nonlinguistic domains enables the predictive process to operate over variable time frames and variable levels of representational abstraction. We review empirical data exemplifying the operation of these principles and discuss the relationship between prediction, event structure, thematic role assignment, and incrementality. (shrink)

In the last 30 years, the key issue in developmental Theory of Mind has been if and when children are capable of representing false beliefs. Moving away from this research question, the aim of this study was to investigate the role of attentional processes in false-belief tasks. We focused on the design of the test phase and investigated two factors that may be critical for 3-year-old children’s success: the form of the wh-question and the salience of the target object. The (...) results of two experiments confirmed that 3-year olds are able to explicitly choose the correct answer in a false-belief task provided that they are allowed to focus on the protagonist throughout the task. The salience of the target object, however, was a critical factor in the design of the test phase, as increasing it had a negative effect on children’s performance. These results suggest that the experimental record of the last 30 years may be skewed since standard false-belief tasks do not control for the relative salience of the wrong response, potentially hindering the performance of children under 4. We conclude that a careful investigation of performance factors in false-belief tasks has the potential to reveal deep insights into the development of Theory of Mind skills, even if not directly focused on children’s representation of beliefs. (shrink)

It has been generally assumed in the Theory of Mind literature of the past 30 years that young children fail standard false-belief tasks because they attribute their own knowledge to the protagonist. Contrary to the traditional view, we have recently proposed that the children's bias is task induced. This alternative view was supported by studies showing that 3 year olds are able to pass a false-belief task that allows them to focus on the protagonist, without drawing their attention to the (...) target object in the test phase. For a more accurate comparison of these two accounts, the present study tested the true-belief default with adults. Four experiments measuring eye movements and response inhibition revealed that adults do not have an automatic tendency to respond to the false-belief question according to their own knowledge and the true-belief response need not be inhibited in order to correctly predict the protagonist's actions. The positive results observed in the control conditions confirm the accuracy of the various measures used. I conclude that the results of this study undermine the true-belief default view and those models that posit mechanisms of response inhibition in false-belief reasoning. Alternatively, the present study with adults and recent studies with children suggest that participants' focus of attention in false-belief tasks may be key to their performance. (shrink)

The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the visual context impinge minutely on a (...) dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP. (shrink)

This thesis investigates the relationship between eye movements, mental imagery and memory retrieval in four studies based on eye-tracking experiments. The first study is an investigation of eye movements during mental imagery elicited both visually and verbally. The use of complex stimuli and the development of a novel method where eye movements are recorded concurrently with verbal data enabled the above-mentioned relationship to be studied to an extent going beyond what previous research had been able to do. Eye movements were (...) found to closely reflect content and spatial layout while participants were listening to a spoken scene description, while they were describing the same scene from memory, and while they were describing a picture they had previously seen. This effect was equally strong during recall from memory irrespective of whether the scene visualised had originally been inspected visually by the participants or whether it was constructed whole-cloth from long-term memory. It was also found that eye movements "to nothing" appeared both when the participants were visualising scenes while looking at a blank screen and when they were doing so in complete darkness. The second study explored an effect frequently observed in the first study, involving a "scaling-down" during recall of participants' gaze patterns to an area smaller than that occupied by the stimulus encoded. It was found that this scaling effect correlated with spatial-imagery ability: the gaze patterns of participants with weaker spatial-imagery ability were closer in size to the encoded scene than the gaze patterns of those stronger in spatial-imagery ability. In the third study, the role of eye movements during mental imagery was investigated in four experiments where eye movements were prohibited during either the encoding phase or the recall phase. Experiments 1 and 2 showed that maintaining central fixation during visual or auditory encoding, respectively, had no effect on how eye movements were executed during recall. Thus, oculomotor events during recall are not reproductions of those produced during encoding. In Experiments 3 and 4, central fixation was instead maintained during recall. This turned out to alter and impair scene recollection, irrespective of the modality of encoding. Finally, in the fourth study, the functional role of eye movements in relation to memory retrieval was further investigated by means of direct eye-movement manipulation in the retrieval phase of an episodic-memory task. Four conditions were used: free viewing on a blank screen, maintaining central fixation, viewing within a square congruent with the location of the objects to be recalled, and viewing within a square incongruent with the location of the objects to be recalled. The results obtained show that gaze position plays an active and facilitatory role during memory retrieval. The findings from these studies are discussed in the light of current theories regarding eye movements during mental imagery and memory retrieval. (shrink)