The human sentence processor is able to make rapid predictions about upcoming linguistic input. For example, upon hearing the verb eat, anticipatory eye-movements are launched toward edible objects in a visual scene. However, the cognitive mechanisms that underlie anticipation remain to be elucidated in ecologically valid contexts. Previous research has, in fact, mainly used clip-art scenes and object arrays, raising the possibility that anticipatory eye-movements are limited to displays containing a small number of objects in a visually impoverished context. In (...) Experiment 1, we confirm that anticipation effects occur in real-world scenes and investigate the mechanisms that underlie such anticipation. In particular, we demonstrate that real-world scenes provide contextual information that anticipation can draw on: When the target object is not present in the scene, participants infer and fixate regions that are contextually appropriate. Experiment 2 investigates whether such contextual inference requires the co-presence of the scene, or whether memory representations can be utilized instead. The same real-world scenes as in Experiment 1 are presented to participants, but the scene disappears before the sentence is heard. We find that anticipation occurs even when the screen is blank, including when contextual inference is required. We conclude that anticipatory language processing is able to draw upon global scene representations to make contextual inferences. These findings are compatible with theories assuming contextual guidance, but posit a challenge for theories assuming object-based visual indices. (shrink)

Conversation is supported by the beliefs that people have in common and the perceptual experience that they share. The visual context of a conversation has two aspects: the information that is available to each conversant, and their beliefs about what is present for each other. In our experiment, we separated these factors for the first time and examined their impact on a spontaneous conversation. We manipulated the fact that a visual scene was shared or not and the belief that a (...) visual scene was shared or not. Participants watched videos of actors talking about a controversial topic, then discussed their own views while looking at either a blank screen or the actors. Each believed (correctly or not) that their partner was either looking at a blank screen or the same images. We recorded conversants’ eye movements, quantified how they were coordinated, and analyzed their speech patterns. Gaze coordination has been shown to be causally related to the knowledge people share before a conversation, and the information they later recall. Here, we found that both the presence of the visual scene, and beliefs about its presence for another, influenced language use and gaze coordination. (shrink)

Semantic studies on diagrammatic notations (Barwise & Etchemendy, ; Shimojima, ; Stenning & Lemon, ) have revealed that the “non-deductive,” “emergent,” or “perceptual” effects of diagrams (Chandrasekaran, Kurup, Banerjee, Josephson, & Winkler, ; Kulpa, ; Larkin & Simon, ; Lindsay, ) are all rooted in the exploitation of spatial constraints on graphical structures. Thus, theoretically, this process is a key factor in inference with diagrams, explaining the frequently observed reduction of inferential load. The purpose of this study was to examine (...) the empirical basis for this theoretical suggestion, focusing on the reality of the constraint-exploitation strategy in actual practices of diagrammatic reasoning. Eye movements were recorded while participants used simple position diagrams to solve three- or four-term transitive inference problems. Our experiments revealed that the participants could exploit spatial constraints on graphical structures even when (a) they were not in the position of actually manipulating diagrams, (b) the semantic rule for the provided diagrams did not match their preferences, and (c) the constraint-exploitation strategy invited a partly adverse effect. These findings indicate that the hypothesized process is in fact robust, with the potential to broadly account for the inferential advantage of diagrams. (shrink)

It has been generally assumed in the Theory of Mind literature of the past 30 years that young children fail standard false-belief tasks because they attribute their own knowledge to the protagonist. Contrary to the traditional view, we have recently proposed that the children's bias is task induced. This alternative view was supported by studies showing that 3 year olds are able to pass a false-belief task that allows them to focus on the protagonist, without drawing their attention to the (...) target object in the test phase. For a more accurate comparison of these two accounts, the present study tested the true-belief default with adults. Four experiments measuring eye movements and response inhibition revealed that adults do not have an automatic tendency to respond to the false-belief question according to their own knowledge and the true-belief response need not be inhibited in order to correctly predict the protagonist's actions. The positive results observed in the control conditions confirm the accuracy of the various measures used. I conclude that the results of this study undermine the true-belief default view and those models that posit mechanisms of response inhibition in false-belief reasoning. Alternatively, the present study with adults and recent studies with children suggest that participants' focus of attention in false-belief tasks may be key to their performance. (shrink)

Although it is well accepted that working memory is intimately related to consciousness, little research has illuminated the liaison between the two phenomena. To investigate this under-explored nexus, we used an imagery monitoring task to investigate the subjective aspects of WM performance. Specifically, in two experiments, we examined the effects on consciousness of holding in mind information having a low versus high memory load, and holding memoranda in mind during the presentation of distractors . Higher rates of rehearsal occurred in (...) the high load and distractor conditions than in comparable control conditions. Examination of the temporal properties of the rehearsal-based imagery revealed that, across subjects, imagery events occurred evenly throughout the delay. We hope that future variants of this new imagery monitoring task will reveal additional insights about WM, consciousness, and action control. (shrink)

We identify a number of principles with respect to prediction that, we argue, underpin adult language comprehension: (a) comprehension consists in realizing a mapping between the unfolding sentence and the event representation corresponding to the real‐world event being described; (b) the realization of this mapping manifests as the ability to predict both how the language will unfold, and how the real‐world event would unfold if it were being experienced directly; (c) concurrent linguistic and nonlinguistic inputs, and the prior internal states (...) of the system, each drive the predictive process; (d) the representation of prior internal states across a representational substrate common to the linguistic and nonlinguistic domains enables the predictive process to operate over variable time frames and variable levels of representational abstraction. We review empirical data exemplifying the operation of these principles and discuss the relationship between prediction, event structure, thematic role assignment, and incrementality. (shrink)

Theories of cognitive development must address both the issue of how children bring their knowledge to bear on behavior in-the-moment, and how knowledge changes over time. We argue that seeking answers to these questions requires an appreciation of the dynamic nature of the developing system in its full, reciprocal complexity. We illustrate this dynamic complexity with results from two lines of research on early word learning. The first demonstrates how the child's active engagement with objects and people supports referent selection (...) via memories for what objects were previously seen in a cued location. The second set of results highlights changes in the role of novelty and attentional processes in referent selection and retention as children's knowledge of words and objects grows. Together this work suggests that understanding systems for perception, action, attention, and memory, and their complex interaction, is critical to understand word learning. We review recent literature that highlights the complex interactions between these processes in cognitive development and point to critical issues for future work. (shrink)

Pronoun interpretation is often described as relying on a comprehender's mental model of discourse. For example, in some psycholinguistic accounts, interpreting pronouns involves a process of retrieval, whereby a pronoun is resolved by accessing information from its linguistic antecedent. However, linguistic antecedents are neither necessary nor sufficient for interpreting a pronoun, and even when an antecedent has been introduced in earlier discourse, there is little evidence for the retrieval of linguistic form. The current study extends our understanding of pronoun interpretation (...) by examining whether the semantics of antecedent expressions are retrieved from representations of past discourse. Participants were instructed to move displayed objects in a Visual World eye-tracking task. In some cases, the semantics of the antecedent were no longer viable after an instruction was completed (e.g., “Move the house on the left to area 12,” where the result was that a different house is now the leftmost one). In this case, retrieving antecedent semantics at the point of hearing a subsequent pronoun (“Now, move it…”) should entail a processing penalty. Instead, the results showed that antecedent semantics have no direct effect on interpretation, raising additional questions about the role that retrieval might play in pronoun interpretation. (shrink)

This thesis investigates the relationship between eye movements, mental imagery and memory retrieval in four studies based on eye-tracking experiments. The first study is an investigation of eye movements during mental imagery elicited both visually and verbally. The use of complex stimuli and the development of a novel method where eye movements are recorded concurrently with verbal data enabled the above-mentioned relationship to be studied to an extent going beyond what previous research had been able to do. Eye movements were (...) found to closely reflect content and spatial layout while participants were listening to a spoken scene description, while they were describing the same scene from memory, and while they were describing a picture they had previously seen. This effect was equally strong during recall from memory irrespective of whether the scene visualised had originally been inspected visually by the participants or whether it was constructed whole-cloth from long-term memory. It was also found that eye movements "to nothing" appeared both when the participants were visualising scenes while looking at a blank screen and when they were doing so in complete darkness. The second study explored an effect frequently observed in the first study, involving a "scaling-down" during recall of participants' gaze patterns to an area smaller than that occupied by the stimulus encoded. It was found that this scaling effect correlated with spatial-imagery ability: the gaze patterns of participants with weaker spatial-imagery ability were closer in size to the encoded scene than the gaze patterns of those stronger in spatial-imagery ability. In the third study, the role of eye movements during mental imagery was investigated in four experiments where eye movements were prohibited during either the encoding phase or the recall phase. Experiments 1 and 2 showed that maintaining central fixation during visual or auditory encoding, respectively, had no effect on how eye movements were executed during recall. Thus, oculomotor events during recall are not reproductions of those produced during encoding. In Experiments 3 and 4, central fixation was instead maintained during recall. This turned out to alter and impair scene recollection, irrespective of the modality of encoding. Finally, in the fourth study, the functional role of eye movements in relation to memory retrieval was further investigated by means of direct eye-movement manipulation in the retrieval phase of an episodic-memory task. Four conditions were used: free viewing on a blank screen, maintaining central fixation, viewing within a square congruent with the location of the objects to be recalled, and viewing within a square incongruent with the location of the objects to be recalled. The results obtained show that gaze position plays an active and facilitatory role during memory retrieval. The findings from these studies are discussed in the light of current theories regarding eye movements during mental imagery and memory retrieval. (shrink)

Theories of cognitive development must address both the issue of how children bring their knowledge to bear on behavior in‐the‐moment, and how knowledge changes over time. We argue that seeking answers to these questions requires an appreciation of the dynamic nature of the developing system in its full, reciprocal complexity. We illustrate this dynamic complexity with results from two lines of research on early word learning. The first demonstrates how the child's active engagement with objects and people supports referent selection (...) via memories for what objects were previously seen in a cued location. The second set of results highlights changes in the role of novelty and attentional processes in referent selection and retention as children's knowledge of words and objects grows. Together this work suggests that understanding systems for perception, action, attention, and memory, and their complex interaction, is critical to understand word learning. We review recent literature that highlights the complex interactions between these processes in cognitive development and point to critical issues for future work. (shrink)