In crossmodal identification, a subject token identifies an item perceived in one sensory modality with an item perceived in another sensory modality. Does crossmodal identification always occur in cognition, or does crossmodal identification sometimes take place in perception? This paper argues that crossmodal identification occurs in cognition, and not in perception. Nevertheless, multisensory perception is not unalive to crossmodal identity. Experimental evidence demonstrates that perception is differentially sensitive to the identity of individuals presented to distinct (...) senses. Such sensitivity enhances recognition and improves action. This approach relies on distinguishing crossmodal identification from perceiving crossmodal identity. Perception registers crossmodal identity, but crossmodal identification as such belongs to thought. (shrink)

What kinds of mental states can be based on epistemic reasons? The standard answer is only beliefs. I argue that perceptual states can also be based on reasons, as the result of crossmodal interactions. A perceptual state from one modality can provide a reason on which an experience in another modality is based. My argument identifies key markers of the basing relation and locates them in the crossmodal Marimba Illusion (Schutz & Kubovy 2009). The subject’s auditory experience of (...) musical tone duration is based on the reason provided by her visual representation of the length of the musician’s gesture and other stored perceptual principles. (shrink)

We can discern two opposing viewpoints regarding synesthesia. According to the first, it is an oddity, an outlier, or a disordered condition. According to the second, synesthesia is pervasive, driving creativity, metaphor, or language itself. Which is it? Ultimately, I favor the first perspective, according to which cross-sensory synesthesia is an outlying condition. But the second perspective is not wholly misguided. My discussion has three lessons. First, synesthesia is just one of a variety of effects in which one sense modality (...) causally impacts and reshapes experience associated with another. These effects are utterly common. However, due to their unfamiliarity, and to their conflict with a widespread conception of the role of the senses in perception and perceptual experience, until recently they have been surprising. Second, synesthesia nevertheless must be distinguished from other inter-modal effects that lead to misperception, such as crossmodal illusions. Third, synesthesia also may be distinguished from the potentially much broader class of synesthetic effects, which could be common across the population and within individuals. (shrink)

Bill Seeley suggests that what follows from research into crossmodal perception for expression and emotion in the arts is that there is an emotional contour (i.e., a contour constitutive of the content of an emotion and potentially realizable across a range of media). As a response of sorts, I speculate as to what this might hold for philosophical and empirical enquiry into expression and emotion across the arts as well as into the nature of the emotions themselves.

According to the decomposition thesis, perceptual experiences resolve without remainder into their different modality-specific components. Contrary to this view, I argue that certain cases of multisensory integration give rise to experiences representing features of a novel type. Through the coordinated use of bodily awareness—understood here as encompassing both proprioception and kinaesthesis—and the exteroceptive sensory modalities, one becomes perceptually responsive to spatial features whose instances couldn’t be represented by any of the contributing modalities functioning in isolation. I develop an argument for (...) this conclusion focusing on two cases: 3D shape perception in haptic touch and experiencing an object’s egocentric location in crossmodally accessible, environmental space. (shrink)

It is common to characterize pain with touch-related terms, like ‘cutting’, ‘pressing’, ‘sharp’, and ‘pulsing’, or temperature-related terms, like ‘hot’ or ‘burning’. This suggests that many pains are phenomenally multimodal because they are experienced as having some tactile-like or thermal-like character. The goal of this chapter is to investigate the structure of phenomenally multimodal pain experiences. It is argued that the usual accounts of multimodal structure proposed in investigations regarding exteroceptive experiences cannot be plausibly applied to multimodal experiences of pain. (...) Instead, an alternative framework is proposed which characterizes the structure of tactile-like and thermal-like pains by referring to the notion of crossmodal correspondences. (shrink)

In the case of ventriloquism, seeing the movement of the ventriloquist dummy’s mouth changes your experience of the auditory location of the vocals. Some have argued that cases like ventriloquism provide evidence for the view that at least some of the content of perception is fundamentally multimodal. In the ventriloquism case, this would mean your experience has constitutively audio-visual content (not just a conjunction of an audio content and visual content). In this paper, I argue that cases like ventriloquism do (...) not in fact warrant that conclusion. I then try to make sense of crossmodal cases without appealing to fundamentally multimodal content. (shrink)

The neurocognitive structure of the acting self has recently been widely studied, yet is still perplexing and remains an often confounded issue in cognitive neuroscience, psychopathology and philosophy. We provide a new systematic account of two of its main features, the sense of agency and the sense of ownership, demonstrating that although both features appear as phenomenally uniform, they each in fact are complex crossmodal phenomena of largely heterogeneous functional and representational levels. These levels can be arranged within a (...) gradually evolving, onto- and phylogenetically plausible framework which proceeds from basic non-conceptual sensorimotor processes to more complex conceptual and meta-representational processes of agency and ownership, respectively. In particular, three fundamental levels of agency and ownership processing have to be distinguished: The level of feeling, thinking and social interaction. This naturalistic account will not only allow to “ground the self in action”, but also provide an empirically testable taxonomy for cognitive neuroscience and a new tool for disentangling agency and ownership disturbances in psychopathology. (shrink)

Predictive processing (PP) accounts of perception are unique not merely in that they postulate a unity between perception and imagination. Rather, they are unique in claiming that perception should be conceptualised in terms of imagination and that the two involve an identity of neural implementation. This paper argues against this postulated unity, on both conceptual and empirical grounds. Conceptually, the manner in which PP theorists link perception and imagination belies an impoverished account of imagery as cloistered from the external world (...) in its intentionality, akin to a virtual reality, as well as endogenously generated. Yet this ignores a whole class of imagery whose intentionality is directed on the actual environment—projected mental imagery—and also ignores the fact that imagery may be triggered crossmodally in a bottom-up, stimulus-driven way. Empirically, claiming that imagery and perception share neural circuitry ignores relevant clinical results in this area. These evidence substantial perception/imagery neural dissociations, most notably in the case of aphantasia. Taken together, the arguments here suggest that PP theorists should substantially temper, if not outright abandon, their claim to a perception/imagination unity. (shrink)

The predictive coding (PC) theory of attention identifies attention with the optimization of the precision weighting of prediction error. Here we provide some challenges for this identification. On the one hand, the precision weighting of prediction error is too broad a phenomenon to be identified with attention because such weighting plays a central role in multimodal integration. Cases of crossmodal illusions such as the rubber hand illusion and the McGurk effect involve the differential precision weighting of prediction error, yet (...) attention does not shift as one would predict. On the other hand, the precision weighting of prediction error is too narrow a phenomenon to be identified with attention, because it cannot accommodate the full range of attentional phenomena. We review criticisms that PC cannot account for volitional attention and affect-biased attention, and we propose that it may not be able to account for feature-based and intellectual attention. (shrink)

Recent research has begun to investigate sensory processing in relation to nonclinical variation in traits associated with the autism spectrum disorders (ASD). We propose that existing accounts of autistic perception can be augmented by considering a role for individual differences in top-down expectations for the precision of sensory input, related to the processing of state-dependent levels of uncertainty. We therefore examined ASD-like traits in relation to the rubber-hand illusion: an experimental paradigm that typically elicits crossmodal integration of visual, tactile, (...) and proprioceptive information in an unusual illusory context. Individuals with higher ASD-like traits showed reduced effects of the rubber-hand illusion on perceived arm position and reach-to-grasp movements, compared to individuals with lower ASD-like traits. These differences occurred despite both groups reporting the typical subjective experience of the illusion concerning visuotactile integration and ownership for the rubber hand. Together these results suggest that the integration of proprioceptive information with cues for arm position derived from the illusory context differs between individuals partly in relation to traits associated with ASD. We suggest that the observed differences in sensory integration can be best explained in terms of differing expectations regarding the precision of sensory estimates in contexts that suggest uncertainty. (shrink)

Objective: Evidence suggests people experience an oral size illusion and commonly perceive oral size inaccurately; however, the nature of the illusion remains unclear. The objectives of the present study were to confirm the presence of an oral size illusion, determine the magnitude (amount) and direction (underestimation or overestimation) of the illusion, and determine whether immediately prior crossmodal perceptual experiences affected the magnitude and direction. Design: Participants (N = 27) orally assessed 9 sizes of stainless steel spheres (1/16 in to (...) 1/2 in) categorized as small, medium, or big, and matched them with digital and visual reference sets. Each participant completed 20 matching tasks in 3 assessments. For control assessments, 6 oral spheres were matched with reference sets of same-sized spheres. For primer-control assessments, similar to control, 6 matching tasks were preceded by cross-modal experiences of the same-sized sphere. For experimental assessments, 8 matching tasks were preceded by a cross-modal experience of a differently sized sphere. Results: For control assessments, small and medium spheres were consistently underestimated, and big spheres were consistently overestimated. For experimental assessments, magnitude and direction of the oral size illusion varied according to the size of the sphere used in the cross-modal experience. Conclusion: Results seemed to confirm an oral size illusion, but direction of the illusion depended on the size of the object. Immediately prior cross-modal experiences influenced magnitude and direction of the illusion, suggesting that aspects of oral perceptual experience are dependent upon factors outside of oral perceptual anatomy and the properties of the oral stimulus. (shrink)

CONTENT Introduction .................................................................................. 9 1. The unexpected: “Epistemologically Different Worlds” .......... 15 1.1 Introduction ........................................................................ 15 1.2 Definitions .......................................................................... 16 1.3 Propositions for its .............................................................. 18 1.4 Propositions for Its and being ............................................ 24 1.5 The hyperverse ................................................................... 30 2. A general view on cognitive neuroscience................................ 37 3. Optimism for localization and the “mind reading”................... 58 3.1 Bechtel’s optimism ............................................................. 58 3.2 Gallant’s laboratory work................................................... 67 3.3 Other optimistic works ....................................................... 75 4. Skepticism in cognitive neuroscience....................................... 81 4.1 Hardcastle’s skepticism (...) ...................................................... 81 4.2 Uttal’s skepticism ............................................................... 88 5. Blood-oxygen-level-dependent (BOLD) of fMRI and local field potentials (LFPs)............................................................. 107 6. The localization problem (segmentation vs. integration) ......... 126 7. The binding problem ................................................................ 138 7.1 Introduction ....................................................................... 138 7.2 The “Feature-Integration Theory” (FIT)............................. 146 7.3 The synchrony or temporal coding theory (temporal binding). 153 7.3.1 Oscillations – a general framework ....................... 153 7.3.2 More details about frequency bands, activated neural areas and cognitive functions ........................ 160 8 7.3.3 Gamma range in visual cognition ............................. 168 7.3.4 Communication among neural areas through synchronized oscillations.......................................... 177 7.3.5 The main critics for temporal coding hypothesis... 187 8. Perception and object recognition ........................................... 194 8.1 Perception and object recognition.................................... 194 8.2 A few words about other notions in cognitive neuroscience .................................................................... 210 9. Space and the mind................................................................... 218 10. Crossmodal interactions ......................................................... 244 11. Holism in cognitive neuroscience........................................... 254 11.1 The parts-whole relationship ........................................... 254 11.2 Raichle’s default network............................................... 269 11.3 Conscious and unconscious mental states...................... 278 12. Fingelkurts’ approach or the status of cognitive neuroscience........................................................................... 293 Conclusion: The status of cognitive neuroscience: “No ontology landscape”..................................... 317. (shrink)

The first part of this survey article presented a cartography of some of the more extensively studied forms of multisensory processing. In this second part, I turn to examining some of the different possible ways in which the structure of conscious perceptual experience might also be characterized as multisensory. In addition, I discuss the significance of research on multisensory processing and multisensory consciousness for philosophical debates concerning the modularity of perception, cognitive penetration, and the individuation of the senses.

We perceive in many ways. But several dubious presuppositions about the senses mask this diversity of perception. Philosophers, scientists, and engineers alike too often presuppose that the senses (vision, audition, etc.) are independent sources of information, perception being a sum of these independent contributions. We too often presuppose that we can generalize from vision to other senses. We too often presuppose that vision itself is best understood as a passive receptacle for an image thrown by a lens. In this essay (...) we illustrate how reflection on results from the cognitive sciences undermine these (and many other) traditional presuppositions. Our illustration situates the remaining essays in the volume, Perception and its Modalities, which further extend the case against various traditional presuppositions, and offer alternatives ways to understand the senses and their role in perception. While many of the essays in this volume represent competing views and draw on different fields, from conceptual analysis to engineering, most contributors believe that to understand perception properly, we need to treat as foundational the idea that the senses work together. From that agreement, new paradigms for theorizing about perception begin to emerge. (shrink)

Multisensory processing encompasses all of the various ways in which the presence of information in one sensory modality can adaptively influence the processing of information in a different modality. In Part I of this survey article, I begin by presenting a cartography of some of the more extensively investigated forms of multisensory processing, with a special focus on two distinct types of multisensory integration. I briefly discuss the conditions under which these different forms of multisensory processing occur as well as (...) their important perceptual consequences and interrelations. In Part II, I then turn to examining of some of the different possible ways in which the structure of conscious perceptual experience might also be characterized as multisensory. In addition, I discuss the significance of research on multisensory processing and multisensory consciousness for philosophical attempts to individuate the senses. (shrink)

Intermodal representationalists hold that the phenomenal characters of experiences are fully determined by their contents. In contrast, intramodal representationalists hold that the phenomenal characters of experiences are determined by their contents together with their intentional modes or manners of representation, which are nonrepresentational features corresponding roughly to the sensory modalities. This paper discusses a kind of experience that provides evidence for an intermodal representationalist view: intermodal experiences, experiences that unify experiences in different modalities. I argue that such experiences are much (...) easier to explain on the intermodal view. (shrink)

Molyneux's Question (MQ) concerns whether a newly sighted man would recognize/distinguish a sphere and a cube by vision, assuming he could previously do this by touch. We argue that (MQ) splits into questions about (a) shared representations of space in different perceptual systems, and about (b) shared ways of constructing higher dimensional spatiotemporal features from information about lower dimensional ones, most of the technical difficulty centring on (b). So understood, MQ resists any monolithic answer: everything depends on the constraints faced (...) by particular perceptual systems in extracting features of higher dimensionality from those of lower. Each individual question of this type is empirical and must be investigated separately. We present several variations on MQ based on different levels of dimensional integration—some of these are familiar, some novel adaptations of problems known elsewhere, and some completely novel. Organizing these cases in this way is useful because it unifies a set of disparate questions about intermodal transfer that have held philosophical and psychological interest, suggests a new range of questions of the same type, sheds light on similarities and differences between members of the family, and allows us to formulate a much-augmented set of principles and questions concerning the intermodal transfer of spatiotemporal organization. (shrink)

Suppose that you are at a live jazz show. The drummer begins a solo. You see the cymbal jolt and you hear the clang. But in addition seeing the cymbal jolt and hearing the clang, you are also aware that the jolt and the clang are part of the same event. Casey O’Callaghan (forthcoming) calls this awareness “intermodal feature binding awareness.” Psychologists have long assumed that multimodal perceptions such as this one are the result of a subpersonal feature binding mechanism (...) (see Vatakis and Spence, 2007, Kubovy and Schutz, 2010, Pourtois et al., 2000, and Navarra et al., 2012). I present new evidence against this. I argue that there is no automatic feature binding mechanism that couples features like the jolt and the clang together. Instead, when you experience the jolt and the clang as part of the same event, this is the result of an associative learning process. The cymbal’s jolt and the clang are best understood as a single learned perceptual unit, rather than as automatically bound. I outline the specific learning process in perception called “unitization,” whereby we come to “chunk” the world into multimodal units. Unitization has never before been applied to multimodal cases. Yet I argue that this learning process can do the same work that intermodal binding would do, and that this issue has important philosophical implications. Specifically, whether we take multimodal cases to involve a binding mechanism or an associative process will have impact on philosophical issues from Molyneux’s question to the question of how active or passive we consider perception to be. (shrink)

This review covers experimental approaches to sound-symbolism—from infants to adults, and from Sapir’s foundational studies to twenty-first century product naming. It synthesizes recent behavioral, developmental, and neuroimaging work into a systematic overview of the cross-modal correspondences that underpin iconic links between form and meaning. It also identifies open questions and opportunities, showing how the future course of experimental iconicity research can benefit from an integrated interdisciplinary perspective. Combining insights from psychology and neuroscience with evidence from natural languages provides us with (...) opportunities for the experimental investigation of the role of sound-symbolism in language learning, language processing, and communication. The review finishes by describing how hypothesis-testing and model-building will help contribute to a cumulative science of sound-symbolism in human language. (shrink)

This report highlights and explores five questions that arose from the multisensory integration workshop at the University of Toronto on May 9th and 10th, 2014: 1. What Is Multisensory Integration? 2. Do Multisensory Percepts Involve Emergent Features? 3. What Can Multisensory Processing Tell Us about Multisensory Awareness? 4. Is Language Processing a Special Kind of Multisensory Integration? 5. What Is the Purpose of Multisensory Integration?