When producing a description of a target referent in a visual context, speakers need to choose a set of properties that distinguish it from its distractors. Computational models of language production/generation usually model this as a search process and predict that the time taken will increase both with the number of distractors in a scene and with the number of properties required to distinguish the target. These predictions are reminiscent of classic findings in visual search; however, unlike models of reference (...) production, visual search models also predict that search can become very efficient under certain conditions, something that reference production models do not consider. This paper investigates the predictions of these models empirically. In two experiments, we show that the time taken to plan a referring expression—as reflected by speech onset latencies—is influenced by distractor set size and by the number of properties required, but this crucially depends on the discriminability of the properties under consideration. We discuss the implications for current models of reference production and recent work on the role of salience in visual search. (shrink)

Drawing on the empirical premise that attention makes objects look more intense, Ned Block has argued for mental paint, a phenomenal residue that cannot be reduced to what is perceived or represented. If sound, Block's argument would undermine direct realism and representationism, two widely held views about the nature of conscious perception. We argue that Block's argument fails because the empirical premise it is based upon is false. Attending to an object alters its salience, but not its perceived intensity. We (...) also argue that salience should be equated with mental primer, a close cousin of mental paint that reintroduces difficulties for direct realism and representationism. The upshot is that direct realism and representationism are still in trouble, but not for the reason that Block thinks. (shrink)

To make behavioral choices that are in line with our goals and our moral beliefs, we need to gather and consider information about our current situation. Most information present in our environment is not relevant to the choices we need or would want to make and thus could interfere with our ability to behave in ways that reflect our underlying values. Certain sources of information could even lead us to make choices we later regret, and thus it would be beneficial (...) to be able to ignore that information. Our ability to exert successful self-governance depends on our ability to attend to sources of information that we deem important to our decision-making processes. We generally assume that, at any moment, we have the ability to choose what we pay attention to. However, recent research indicates that what we pay attention to is influenced by our prior experiences, including reward history and past successes and failures, even when we are not aware of this history. Even momentary distractions can cause us to miss or discount information that should have a greater influence on our decisions given our values. Such biases in attention thus raise questions about the degree to which the choices that we make may be poorly informed and not truly reflect our ability to otherwise exert self-governance. (shrink)

Following the pioneering studies of the receptive field (RF), the concept gained further significance for visual perception by the discovery of input effects from beyond the classical RF. These studies demonstrated that neuronal responses could be modulated by stimuli outside their RFs, consistent with the perception of induced brightness, color, orientation, and motion. Lesion scotomata are similarly modulated perceptually from the surround by RFs that have migrated from the interior to the outer edge of the scotoma and in this way (...) provide filling-in of the void. Large RFs are advantageous to this task. In higher visual areas, such as the middle temporal and inferotemporal lobe, RFs increase in size and lose most of their retinotopic organization while encoding increasingly complex features. Whereas lowerlevel RFs mediate perceptual filling-in, contour integration, and figure–ground segregation, RFs at higher levels serve the perception of grouping by common fate, biological motion, and other biologically relevant stimuli, such as faces. Studies in alert monkeys while freely viewing natural scenes showed that classical and nonclassical RFs cooperate in forming representations of the visual world. Today, our understanding of the mechanisms underlying the RF is undergoing a quantum leap. What had started out as a hierarchical feedforward concept for simple stimuli, such as spots, lines, and bars, now refers to mechanisms involving ascending, descending, and lateral signal flow. By extension of the bottom-up paradigm, RFs are nowadays understood as adaptive processors, enabling the predictive coding of complex scenes. Top-down effects guiding attention and tuned to task-relevant information complement the bottom-up analysis. (shrink)

Prominent models of attentional control assert a dichotomy between top-down and bottom-up control, with the former determined by current selection goals and the latter determined by physical salience. This theoretical dichotomy, however, fails to explain a growing number of cases in which neither current goals nor physical salience can account for strong selection biases. For example, equally salient stimuli associated with reward can capture attention, even when this contradicts current selection goals. Thus, although 'top-down' sources of bias are sometimes defined (...) as those that are not due to physical salience, this conception conflates distinct - and sometimes contradictory - sources of selection bias. We describe an alternative framework, in which past selection history is integrated with current goals and physical salience to shape an integrated priority map. (shrink)

A general definition of consciousness is: ‘consciousness is a mental aspect of a system or a process, which is a conscious experience, a conscious function, or both depending on the context’, where the term context refers to metaphysical views, constraints, specific aims, and so on. One of the aspects of visual consciousness is the visual subjective experience (SE) or the first person experience that occurs/emerges in the visual neural-network of thalamocortical system (which includes dorsal and ventral visual pathways and frontal (...) cortex) during dynamic interactions among widely distributed neuronal groups. In this article, however, consciousness and SE are interchangeably used unless noted. Consciousness can be either access (reportable) or phenomenal consciousness. For access consciousness, the interactions are between feed forward stimulus dependent signals and fronto-parietal feedback attentional signals. The necessary ingredients for access (reportable) consciousness are (i) wakefulness, (ii) reentrant interactions among neural populations, (iii) fronto-parietal and thalamic-reticular-nucleus attentional signals that modulate consciousness, (iv) working memory that retains information for consciousness, (v) stimulus at or above threshold level, and (vi) neural-network proto-experiences (PEs) that are superposed SEs embedded in a neural-network. Attention and the ability to report are not necessary for phenomenal consciousness. The neural source for the arousal system is the ascending reticular activating system in the brain stem, which brings the thalamocortical neural networks to wakeful state as a baseline for consciousness to occur. Reentrant interactions among neural populations bind stimulus attributes (such as location and features) and entail consciousness. Attention could be the results of reentry and competitive interactions, and modulates the stimulus related feed forward signal and consciousness. The ‘sources’ of attention may be thalamic reticular nucleus for bottom-up or frontal cortex for top-down direction. The ‘target’ of visual attention is ‘V4/V8/VO’ for Red-Green (R-G) channel. The neural correlates of the psychophysical entity R-G channel appear to be ‘V4/V8/VO’-neural-network (retina → LGN ↔ V1 ↔ V2 ↔ ‘V4/V8/VO’, and areas for attention, memory, and wakefulness). The psychophysical Red-Green Channel, its neurophysiological correlates V4/V8/VO-neural-network, and related experience (such as redness) are integrated. The dual-aspect-dual-mode PE-SE framework (Vimal, J Integr Neurosci 7:49–73, 2008) was used to address the explanatory gap problem of materialism. Neural-network and related SEs are the results of the co-evolution and co-development of the material aspect (mass, charge, spin, force, quanta, and space-time) and the mental aspect of fundamental particles (strings or elementary particles (fermions and bosons)). Their mental aspects are considered as the carriers of superposed multiple possible experiences (SEs/PEs) in unexpressed form. These possibilities are actualized when neural-networks are formed via neural Darwinism, and a specific SE is selected by a matching process when the necessary ingredients of consciousness/SE are satisfied. A simple experimental design is suggested to address the necessary and sufficient attributes of consciousness. (shrink)

This article sketches an idealized strategy for the identification of neural correlates of consciousness. The proposed strategy is based on a state space approach originating from the analysis of dynamical systems. The article then focuses on one constituent of consciousness, phenomenal awareness. Several rudimentary requirements for the identification of neural correlates of phenomenal awareness are suggested. These requirements are related to empirical data on selective attention, on completely intrinsic selection and on globally unconscious states. As an example, neuroscientific findings on (...) synchronized γ activity are categorized according to these requirements. (shrink)

Our visual environment is highly predictable in terms of where and in which locations objects can be found. Based on visual experience, children extract rules about visual scene configurations, allowing them to generate scene knowledge. Similarly, children extract the linguistic rules from relatively predictable linguistic contexts. It has been proposed that the capacity of extracting rules from both domains might share some underlying cognitive mechanisms. In the present study, we investigated the link between language and scene knowledge development. To do (...) so, we assessed whether preschool children with Developmental Language Disorder, who present several difficulties in the linguistic domain, are equally attracted to object-scene inconsistencies in a visual free-viewing task in comparison with age-matched children with Typical Language Development. All children explored visual scenes containing semantic, syntactic, or both inconsistencies. Since scene knowledge interacts with image properties to guide gaze allocation during visual exploration from the early stages of development, we also included the objects’ saliency rank in the analysis. The results showed that children with DLD were less attracted to semantic and syntactic inconsistencies than children with TLD. In addition, saliency modulated syntactic effect only in the group of children with TLD. Our findings indicate that children with DLD do not activate scene knowledge to guide visual attention as efficiently as children with TLD, especially at the syntactic level, suggesting a link between scene knowledge and language development. (shrink)

Observers can learn the locations where salient distractors appear frequently to reduce potential interference. The effect that observers appear to learn the spatial distribution of salient but task-irrelevant distractors in the visual environment to reduce the interference caused by such distractors was referred to as ‘statistical learning of distractor locations’. Emerging studies agreed that the observed reduction of distractor interference is largely attributable to better suppression of distractor in frequent locations, however, concerning how this spatial distractor suppression is implemented within (...) the functional architecture of search guidance, and how the learned suppression is processed in the brain to reduce the interference, remains debated and poorly understood. The line of studies of the current dissertation, therefore, explores the cognitive and neural mechanism underlying spatial distractor suppression based on statistical learning. Using psychophysical experiments, the first two studies examined a series of potentially ‘confounding’ factors that might lead to different theoretical conclusions concerning the locus of learned spatial distractor suppression in different studies. We found that irrespective of whether a salient singleton distractor is more likely to occur at multiple locations within a subregion of the display or at a specific location, observers are both likely to adopt the priority-map-based suppression strategy and dimension-based suppression strategy to reduce interference of distractor in the frequent region. Notably, a critical factor determining which strategy is adopted is whether the distractor and non-distractor items swap colors randomly across trials: without color swapping, observers are more likely to acquire a dimension-based suppression; with color swapping, they prefer to develop a priority-map-based suppression in the first beginning, and potentially shift to dimension-based suppression later with the course of learning. The third study employed functional magnetic resonance imaging techniques with the distractor-location learning paradigm with two types of distractors defined in either the same- or different- visual dimension relative to the target. We disclosed that statistical learning of distractor locations involves suppression down to the level of the early visual cortex to reduce distractor interference, potential different neural mechanisms of spatial distractor suppression between distractors defined in a different and the same dimension to the target. With different-dimension distractors, higher-level, dimension-specific filtering mechanisms plays a role to reduce the interference; with same-dimension distractors, interference reduction relies on cutting down lower-level sensory signals. (shrink)

Underwater object detection plays an important role in research and practice, as it provides condensed and informative content that represents underwater objects. However, detecting objects from underwater images is challenging because underwater environments significantly degenerate image quality and distort the contrast between the object and background. To address this problem, this paper proposes an optical prior-based underwater object detection approach that takes advantage of optical principles to identify optical collimation over underwater images, providing valuable guidance for extracting object features. Unlike (...) data-driven knowledge, the prior in our method is independent of training samples. The fundamental novelty of our approach lies in the integration of an image prior and the object detection task. This novelty is fundamental to the satisfying performance of our approach in underwater environments, which is demonstrated through comparisons with state-of-the-art object detection methods. (shrink)

Ballard and Zhang offer a fascinating review of how computational models of human vision have evolved since David Marr proposed his Tri‐Level Hypothesis, with a focus on the refinement of algorithm descriptions over time.

Evolution fuels interindividual variability in neuroplasticity, reflected in brain anatomy and functional connectivity of the expanding neocortical regions subserving reading ability. Such variability is orchestrated by an evolutionarily conserved, competitive balance between epigenetic, stress-induced, and cognitive-growth gene expression programs. An evolutionary developmental model of dyslexia, suggests that prenatal and childhood subclinical stress becomes a risk factor for dyslexia when physiological adaptations to stress promoting adaptive fitness, may attenuate neuroplasticity in the brain regions recruited for reading. Stress has the potential to (...) blunt the cognitive-growth functions of the predominantly right hemisphere Ventral and Dorsal attention networks, which are primed with high entropic levels of synaptic plasticity, and are critical for acquiring beginning reading skills. The attentional networks, in collaboration with the stress-responsive Default Mode network, modulate the entrainment and processing of the low frequency auditory oscillations and visuospatial orienting linked etiologically to dyslexia. Thus, dyslexia may result from positive, but costly adaptations to stress system dysregulation: protective measures that reset the stress/growth balance of processing to favor the Default Mode network, compromising development of the attentional networks. Such a normal-variability conceptualization of dyslexia is at odds with the frequent assumption that dyslexia results from a neurological abnormality. To put the normal-variability model in the broader perspective of the state of the field, a traditional evolutionary account of dyslexia is presented to stimulate discussion of the scientific merits of the two approaches. (shrink)

The question regarding how to characterize aesthetics has been revived with the publication of Bence Nanay’s _Aesthetics as Philosophy of Perception_. This paper takes seriously Dustin Stokes’ criticisms of Nanay’s book regarding Nanay’s inability to distinguish between ordinary expert visual tasks (e.g., sorting for sock color or ornithology) and aesthetic experience. Using empirical research on gist perception and its reliance on low-level features in visual experience, I develop a theory that distinguishes expert visual tasks and aesthetic experiences by differentiating two (...) different kinds of distributed attention over properties. I argue that expert visual tasks are instances of property attribution in a mode of conscious attention, while aesthetics is a kind of distributed attention that significantly relies on the reiteration of gist-like lowlevel features. Gist, often referred to in visual science as “preattentive” mode, gives us a model to understand the perceptual processes that are specific to aesthetics. This comports with our common-sense definition of aesthetics as both distinguishable from ordinary expert visual tasks and an experience that makes prominent sensory aspects of visual experience. (shrink)

The process of urban landmark-based navigation has proven to be difficult to study in a rigorous fashion, primarily due to confounding variables and the problem of obtaining reliable data in real-world contexts. The development of high-resolution, immersive virtual reality technologies has opened exciting new possibilities for gathering data on human wayfinding that could not otherwise be readily obtained. We developed a research platform using a virtual environment and electroencephalography to better understand the neural processes associated with landmark usage and recognition (...) during urban navigation tasks. By adjusting the architectural parameters of different buildings in this virtual environment, we isolated and tested specific design features to determine whether or not they served as a target for landmarking. EEG theta band event-related synchronization/desynchronization over posterior scalp areas was evaluated at the time when participants observed each target building along a predetermined self-paced route. A multi-level linear model was used to investigate the effects of salient architectural features on posterior scalp areas. Our results support the conclusion that highly salient architectural features—those that contrast sharply with the surrounding environment—are more likely to attract visual attention, remain in short-term memory, and activate brain regions associated with wayfinding compared with non-salient buildings. After establishing this main aggregate effect, we evaluated specific salient architectural features and neural correlates of navigation processing. The buildings that most strongly associated extended gaze time, location recall accuracy, and changes in theta-band neural patterns with landmarking in our study were those that incorporated rotational twist designs and natural elements such as trees and gardens. Other building features, such as unusual façade patterns or building heights, were to a lesser extent also associated with landmarking. (shrink)

This thesis investigates the relationship between eye movements, mental imagery and memory retrieval in four studies based on eye-tracking experiments. The first study is an investigation of eye movements during mental imagery elicited both visually and verbally. The use of complex stimuli and the development of a novel method where eye movements are recorded concurrently with verbal data enabled the above-mentioned relationship to be studied to an extent going beyond what previous research had been able to do. Eye movements were (...) found to closely reflect content and spatial layout while participants were listening to a spoken scene description, while they were describing the same scene from memory, and while they were describing a picture they had previously seen. This effect was equally strong during recall from memory irrespective of whether the scene visualised had originally been inspected visually by the participants or whether it was constructed whole-cloth from long-term memory. It was also found that eye movements "to nothing" appeared both when the participants were visualising scenes while looking at a blank screen and when they were doing so in complete darkness. The second study explored an effect frequently observed in the first study, involving a "scaling-down" during recall of participants' gaze patterns to an area smaller than that occupied by the stimulus encoded. It was found that this scaling effect correlated with spatial-imagery ability: the gaze patterns of participants with weaker spatial-imagery ability were closer in size to the encoded scene than the gaze patterns of those stronger in spatial-imagery ability. In the third study, the role of eye movements during mental imagery was investigated in four experiments where eye movements were prohibited during either the encoding phase or the recall phase. Experiments 1 and 2 showed that maintaining central fixation during visual or auditory encoding, respectively, had no effect on how eye movements were executed during recall. Thus, oculomotor events during recall are not reproductions of those produced during encoding. In Experiments 3 and 4, central fixation was instead maintained during recall. This turned out to alter and impair scene recollection, irrespective of the modality of encoding. Finally, in the fourth study, the functional role of eye movements in relation to memory retrieval was further investigated by means of direct eye-movement manipulation in the retrieval phase of an episodic-memory task. Four conditions were used: free viewing on a blank screen, maintaining central fixation, viewing within a square congruent with the location of the objects to be recalled, and viewing within a square incongruent with the location of the objects to be recalled. The results obtained show that gaze position plays an active and facilitatory role during memory retrieval. The findings from these studies are discussed in the light of current theories regarding eye movements during mental imagery and memory retrieval. (shrink)

Comics are complex multimodal documents that make for intriguing materials to analyze with computer vision and computational linguistics. This review summarizes the growing developments in computational modeling which have been progressing to analyze visual narratives across their various substructures.

Understanding how people comprehend visual narratives (including picture stories, comics, and film) requires the combination of traditionally separate theories that span the initial sensory and perceptual processing of complex visual scenes, the perception of events over time, and comprehension of narratives. Existing piecemeal approaches fail to capture the interplay between these levels of processing. Here, we propose the Scene Perception & Event Comprehension Theory (SPECT), as applied to visual narratives, which distinguishes between front-end and back-end cognitive processes. Front-end processes occur (...) during single eye fixations and are comprised of attentional selection and information extraction. Back-end processes occur across multiple fixations and support the construction of event models, which reflect understanding of what is happening now in a narrative (stored in working memory) and over the course of the entire narrative (stored in long-term episodic memory). We describe relationships between front- and back-end processes, and medium-specific differences that likely produce variation in front-end and back-end processes across media (e.g., picture stories vs. film). We describe several novel research questions derived from SPECT that we have explored. By addressing these questions, we provide greater insight into how attention, information extraction, and event model processes are dynamically coordinated to perceive and understand complex naturalistic visual events in narratives and the real world. (shrink)

Background: Neurotypical young adults show task-based modulation and stability of their eye movements across tasks. This study aimed to determine whether persons with aphasia (PWA) modulate their eye movements and show stability across tasks similarly to control participants. Methods: Forty-eight PWA and age-matched control participants completed four eye-tracking tasks: scene search, scene memorization, text-reading, and pseudo-reading. Results: Main effects of task emerged for mean fixation duration, saccade amplitude, and standard deviations of each, demonstrating task-based modulation of eye movements. Group by (...) task interactions indicated that PWA produced shorter fixations relative to controls. This effect was most pronounced for scene memorization and for individuals who recently suffered a stroke. PWA produced longer fixations, shorter saccades, and less variable eye movements in reading tasks compared to controls. Three-way interactions of group, aphasia subtype, and task also emerged. Text-reading and scene memorization were particularly effective at distinguishing aphasia subtype. Persons with anomic aphasia showed a reduction in reading saccade amplitudes relative to their respective control group and other PWA. Persons with conduction/Wernicke’s aphasia produced shorter scene memorization fixations relative to controls or PWA of other subtypes, suggesting a memorization specific effect. Positive correlations across most tasks emerged for fixation duration and did not significantly differ between controls and PWA. Conclusions: PWA generally produced shorter fixations and smaller saccades relative to controls particularly in scene memorization and text-reading respectively. The effect was most pronounced recently after a stroke. Selectively in reading tasks, PWA produced longer fixations and shorter saccades relative to controls, consistent with reading difficulty. PWA showed task-based modulation of eye movements, though the pattern of results was somewhat abnormal relative to controls. All subtypes of PWA also demonstrated task-based modulation of eye movements. However, persons with anomic aphasia showed reduced modulation of saccade amplitude and smaller reading saccades, possibly to improve reading comprehension. Controls and PWA generally produced stabile fixation durations across tasks and did not differ in their relationship across tasks. Overall, these results suggest there is potential to differentiate among PWA with varying subtypes and from controls using eye movement measures of task-based modulation, especially reading and scene memorization tasks. (shrink)

The origin of colour categories and their relationship to colour perception have been the prime example for testing the influence of language on perception and thought and more generally for investigating the biological, ecological and cultural determination of human cognition. These themes are central to a broad range of disciplines, including vision research, neuroscience, cognitive psychology, developmental science, cultural anthropology, linguistics, computer science, and philosophy. Unfortunately, though, it has been tacitly taken for granted that the conceptual assumptions and methodological practices (...) from the dawn of empirical research on colour categorisation are the gold standard for current colour category research. Here, we show that these assumptions and practices are obsolete and flawed and have led to four fundamental and widespread misconceptions about colour categorisation: 1.) that colour perception is inherently categorical; 2.) that English Basic Colour Terms correspond to universal categories that are the end point of a fixed evolutionary sequence; 3.) that the prototypes of English basic colour terms are perceptually salient and qualify as focal colours; and 4.) that colour category research essentially revolves around the universalism-realism debate. State-of-the-art research on colour categorisation provides new, more sophisticated approaches and allows for rectifying those four statements. At the same time, some of the questions underlying those statements are not convincingly answered yet and constitute major challenges to future research. The critical considerations on colour categorisation may be transferred to research on other kinds of perceptual categorisation to inspire new, more general research questions. (shrink)