Cognitive scientists have long known that the modalities interact during perceptual processing. Cross-modal illusions like the ventriloquism effect show that the course of processing in one modality can alter the course of processing in another. But how do the modalities interact in the specific domain of object perception? This paper distinguishes and analyzes two kinds of multisensory interaction in object perception. First, the modalities may bind features to a single object or event. Second, the modalities may cooperate when differentiating an (...) object or event from its surroundings. I critically evaluate evidence for various forms of multisensory binding. I then consider the case for multisensory differentiation. I argue that existing evidence for multisensory differentiation is inconclusive. I highlight ways that the issue might be empirically resolved. (shrink)

This is the editors' Introduction to a special issue of the journal, Multisensory Research. European philosophers of the modern period found multisensory perception to be impossible because they thought that perceptual ideas are defined by how they are experienced. Under this conception, the individual modalities are determinables of ideas—just as colour is a determinable that embraces red and blue, so also the visual is a determinable that embraces colour and (visually experienced) shape. Since no idea is experienced as, for example, (...) both visual and auditory, there can be no such thing as audiovisual perception. This conception of modality is not directly contested, but a variety of perceptual phenomena are listed that could raise interesting questions if treated as multimodal in origin. (shrink)

Introduction to the topical collection "The Structure of Perceptual Objects"—with contributions by Mohan Matthen, EJ Green, Alisa Mandrigin, Blazej Skrzypulec, and Anna Drożdżowicz.

Is the flavor of mint reducible to the minty smell, the taste, and the menthol-like coolness on the roof of one’s mouth, or does it include something over and above these—something not properly associated with any one of the contributing senses? More generally, are there features of perceptual experiences—so-called novel features—that are not associated with any of our senses taken singly? This question has received a lot of attention of late. Yet surprisingly little attention has been paid to the question (...) of what it means to say that a feature is associated with a sensory modality in the first place. Indeed, there is only one fully developed proposal in the literature, due to Casey O’Callaghan. I argue that this proposal is too permissive to inform the debate over novel features. I go on to argue that all attempts to formulate a better proposal along these lines fail. The corollary of my arguments is that the question of the existence of novel features is poorly formed. Furthermore, the problem generalizes, with the result that we should not rely on our pre-theoretical notions of the senses as the basis of theorizing about the features of perceptual experiences. (shrink)

It is commonly believed that human perceptual experiences can be, and usually are, multimodal. What is more, a stronger thesis is often proposed that some perceptual multimodal characters cannot be described simply as a conjunction of unimodal phenomenal elements. If it is the case, then a question arises: what is the additional mode of combination that is required to adequately describe the phenomenal structure of multimodal experiences? The paper investigates what types of audio–visual experiences have phenomenal character that cannot be (...) analysed as a mere conjunction of visual and auditory elements; and how can we properly characterise the required, additional mode of perceptual combination. Three main modes of combination are considered: instantiation, parthood, and grouping. It is argued that some phenomena involving intermodal relations, like spatial and temporal ventriloquism, can be analysed in terms of audio–visual, perceptual grouping. On the other hand, cases of intermodal binding need a different treatment. Experiences involving audio–visual binding should be analysed as experiences presenting objects or events which instantiate, or which have a proper part instantiating, both visually and auditorily determined properties. (shrink)

The first part of this survey article presented a cartography of some of the more extensively studied forms of multisensory processing. In this second part, I turn to examining some of the different possible ways in which the structure of conscious perceptual experience might also be characterized as multisensory. In addition, I discuss the significance of research on multisensory processing and multisensory consciousness for philosophical debates concerning the modularity of perception, cognitive penetration, and the individuation of the senses.

Based on but not limited to material from a conference at Brown University in 2011, Sensory Integration and the Unity of Consciousness is an ambitious collection that brings together two distinct but inter- twined topics.1 In what follows, I briefly explain what sensory integration and the unity of conscious- ness amount to, highlight the contents of the papers, and finally end with general observations and suggestions. I will spend more time on sensory integration, since it is relatively unfamiliar terrain in (...) philosophy. (shrink)

Object perception deploys a suite of perceptual capacities that constrains attention, guides reidentification, subserves recognition, and anchors demonstrative thought. Objects for perception—perceptual objects—are the targets of such capacities. Characterizing perceptual objects for multisensory perception faces two puzzles. First is the diversity of objects across sensory modalities. Second is the unity of multisensory perceptual objects. This paper resolves the puzzles. Objects for perception are structured mereologically complex individuals. Perceptual objects are items that bear perceptible features and have perceptible parts arranged to (...) form a unified whole. This circumscribes the targets of object perception where neutral talk concerning intentional objects and objects of perception cannot. Nonetheless, it is more permissive than identifying perceptual objects with material bodies, which excludes too much and leans too heavily on vision. The account thus is tailored to capture the role of objects in perception beyond vision: tactual, auditory, and olfactory objects are individuals with differing structures. Its flexibility also enables the account to accommodate shared objects for multisensory perception: multisensory perceptual objects are mereologically complex individuals with hybrid structure. For instance, one can bimodally perceive a common whole with parts accessible to one but not both modalities. Each sense provides a partial perspective on a complex whole that is perceptible through the coordinated use of multiple senses. Understanding perceptual objects as structured mereologically complex individuals thus provides a theoretically useful notion of an object for multisensory perception that resolves the puzzles of diversity and of unity. (shrink)

One way to characterize the special relation that one has to one's own body is to say that only one's body appears to one from the inside. Although widely accepted, the nature of this specific experiential mode of presentation of the body is rarely spelled out. Most definitions amount to little more than lists of the various body senses (including senses of posture, movement, heat, pressure, and balance). It is true that body senses provide a kind of informational access to (...) one's own body, which one has to no other bodies, by contrast to external senses like vision, which can take many bodies as their object. But a theory of bodily awareness needs to take into account recent empirical evidence that indicates that bodily awareness is infected by a plague of multisensory effects, regardless of any dichotomy between body senses and external senses. Here I will argue in favour of a multimodal conception of bodily awareness. I will show that the body senses fail to fully account for the content of bodily experiences. I will then propose that vision helps compensate for the insufficiencies of the body senses in people who can see. I will finally argue that the multimodality of bodily experiences does not prevent privileged access to one's body. (shrink)

Psychophysics and neuroscience demonstrate that different sensory systems interact and influence each other. Perceiving involves extensive cooperation and coordination among systems associated with sight, hearing, touch, smell, and taste. Nonetheless, it remains unclear in what respects conscious perceptual awareness is multisensory. This paper distinguishes six differing varieties of multisensory awareness, explicates their consequences, and thereby elucidates the multisensory nature of perception. It argues on these grounds that perceptual awareness need not be exhausted by that which is associated with each of (...) the respective sensory modalities along with whatever accrues thanks to simple co-consciousness. (shrink)

It is tempting to hold that perceptual experience amounts to a co-conscious collection of visual, auditory, tactual, gustatory, and olfactory episodes. If so, each aspect of perceptual experience on each occasion is associated with a specific modality. This paper, however, concerns a core variety of multimodal perceptual experience. It argues that there is perceptually apparent intermodal feature binding. I present the case for this claim, explain its consequences for theorizing about perceptual experience, and defend it against objections. I maintain that (...) just as one thing may perceptually appear at once to jointly bear several features associated with the same sense modality, one thing also may perceptually appear at once to jointly bear features associated with different sense modalities. For instance, just as something may visually appear at once to be both red and round, or to have a red part and a green part, something may multimodally perceptually appear at once to be both bright and loud, or to have a red part and a rough part. The main lesson, I argue, is that perceiving is not just co-consciously seeing, hearing, feeling, tasting, and smelling at the same time. And perceptual phenomenal character is not on each occasion exhausted by that which is distinctive to or associated with a given modality, along with that which accrues thanks to simple co-consciousness. Not all ways of perceiving are modality specific. I defend this account against three main objections: that singular content theorists avoid my conclusions; that apparent infusion of perceptible features is required for perceptually apparent binding but does not occur intermodally; and that the diversity of objects across modalities makes perceptually apparent intermodal binding rare. (shrink)

Restrictivists hold that visual experience only represents low-level properties such as shape, spatial location, motion, color, etc. Expansionists contend that visual experience also represents high-level properties such as being a pine tree. I outline a new approach to support expansionism called the conflict cross-modal argument. What I call the conflict cross-modal effects occur when at least two perceptual systems disagree about some property belonging to a common stimulus, and this disagreement causes a change in the representational and phenomenal content of (...) the perceptual experience associated with one, or both, modalities. The conflict cross-modal argument works by accepting that if a property figures in a conflict cross-modal effect, then prima facie that property is a strong common sensible between the two modalities, and vision and another modality disagree about a property that is high-level for vision. After outlining this argument, and showing how it overcomes two obstacles that face traditional methods employed to defend expansionism, I turn to the well known Mcgurk effect as a case where vision and audition disagree about phoneme properties in order to employ my argument for the conclusion that phoneme properties are represented in visual experience. The upshot is that since phoneme properties are high-level for vision, we have an argument supporting expansionism. (shrink)

The Bayesian model of multisensory cue integration proposed by Ernst and Banks provides an attractive model for understanding a way that our sensory systems may interact. Moreover, it has been suggested that the process of multisensory integration that it models underpins conscious experiences with multisensory representational contents merged across modalities. Should we therefore take empirical support for the Bayesian model as evidence of the multimodality of perception? Focusing on evidence of integration across vision and touch, I argue that apparent support (...) for the model does not warrant the rejection of the view that each of our conscious perceptual experiences is associated with one and only one sense modality. (shrink)

Multimodal integration is the formation of a coherent percept from different sensory inputs such as vision, audition, and somatosensation. Most research on multimodal integration in speech perception has focused on audio-visual integration. In recent years, audio-tactile integration has also been investigated, and it has been established that puffs of air applied to the skin and timed with listening tasks shift the perception of voicing by naive listeners. The current study has replicated and extended these findings by testing the effect of (...) air puffs on gradations of voice onset time along a continuum rather than the voiced and voiceless endpoints of the original work. Three continua were tested: bilabial (“pa/ba”), velar (“ka/ga”), and a vowel continuum (“head/hid”) used as a control. The presence of air puffs was found to significantly increase the likelihood of choosing voiceless responses for the two VOT continua but had no effect on choices for the vowel continuum. Analysis of response times revealed that the presence of air puffs lengthened responses for intermediate (ambiguous) stimuli and shortened them for endpoint (non-ambiguous) stimuli. The slowest response times were observed for the intermediate steps for all three continua, but for the bilabial continuum this effect interacted with the presence of air puffs: responses were slower in the presence of air puffs, and faster in their absence. This suggests that during integration auditory and aero-tactile inputs are weighted differently by the perceptual system, with the latter exerting greater influence in those cases where the auditory cues for voicing are ambiguous. (shrink)