The first part of this survey article presented a cartography of some of the more extensively studied forms of multisensory processing. In this second part, I turn to examining some of the different possible ways in which the structure of conscious perceptual experience might also be characterized as multisensory. In addition, I discuss the significance of research on multisensory processing and multisensory consciousness for philosophical debates concerning the modularity of perception, cognitive penetration, and the individuation of the senses.

The mind of man is central to the structure and functioning of the physical world. Modern physical theory indicates that the mind stands in a relationship of equals to the physical world. Both are fundamental, neither can be reduced to the other, and both require each other for their full understanding. This thesis is at odds with the view of the universe found in Newtonian mechanics as well as the generally held view among contemporary physicists of modern physical theory. Since (...) the Renaissance, man has come to understand a great deal about the physical world, and he has gained significant control over it. This increased power over the physical world has occurred hand in hand with the assumption that the structure and functioning of the physical world is essentially independent from his cognitive functioning. According to this assumption, if man’s cognitive capacity did not exist, the functioning of the physical world would not be fundamentally altered. This last statement is not in fact correct, and modern physical theory, and even fundamentals underlying Newtonian mechanics, provide evidence to attest to this. Nonetheless, contemporary physicists for the most part do not see that the relationship of human cognition to the physical world is radically altered in their own modern theory, theory that is supported by a great deal of empirical data. Instead, attempting to preserve the thesis that the structure and functioning of the physical world is independent of the mind while on a practical level relying on modern theory that contradicts this thesis, physicists have placed themselves in the position of wondering at times exactly what is the nature of the physical world at the same time they obtain experimental results concerning the physical world that can only be labeled astonishing in their precision and the scope of their implications. Modern physical theory consists of three main components: 1) the special and general theories of relativity; 2) quantum mechanics; and 3) statistical mechanics. There are very successful theories that have been developed on the basis of these three bedrock areas. An example of one is quantum electrodynamics. But these theories owe their conceptual foundation to the three components mentioned. The basic issues at the core of these three components also are expressed in these later theories. In addition, there are new unresolved issues of a fundamental nature concerning the conceptual integrity of these later theories that do not apply to quantum mechanics, relativity theory, and statistical mechanics. Quantum mechanics and relativity theory are areas I have written about for over twelve years. The nature of statistical mechanics has also been of interest to me during this time. But when I took a serious look in 1993 at Tolman’s (1938) The Principles of Statistical Mechanics, it became clear that the mind is linked to the physical world in statistical mechanics, a relationship I had found earlier in both relativity theory and quantum mechanics. It was after reading Tolman’s justification of the method of statistical mechanics in the original that I decided to write this book. When I found that the three components of modern physical theory all pointed to the same relationship between mind and the physical world, it became clear that the fundamental isolation of the mind from the physical world that has characterized our experience since the development of Newtonian mechanics is unfounded. Based on empirically supported principles of modern physical theory, I determined that the appropriate assumption for one’s experience, that the mind is linked to the physical world, could be stated with confidence. The impact of this change in assumption concerning the relationship of man to the cosmos in modern physical theory will find its way into our everyday experience. It will perhaps have no greater effect than in reducing the sense of isolation of man from the world that has characterized modern existence. (shrink)

According to the decomposition thesis, perceptual experiences resolve without remainder into their different modality-specific components. Contrary to this view, I argue that certain cases of multisensory integration give rise to experiences representing features of a novel type. Through the coordinated use of bodily awareness—understood here as encompassing both proprioception and kinaesthesis—and the exteroceptive sensory modalities, one becomes perceptually responsive to spatial features whose instances couldn’t be represented by any of the contributing modalities functioning in isolation. I develop an argument for (...) this conclusion focusing on two cases: 3D shape perception in haptic touch and experiencing an object’s egocentric location in crossmodally accessible, environmental space. (shrink)

Which is my body and how do I distinguish it from the bodies of others, or from objects in the surrounding environment? The perception of our own body and more particularly our sense of body ownership is taken for granted. Nevertheless experimental findings from body ownership illusions (BOIs), show that under specific multisensory conditions, we can experience artificial body parts or fake bodies as our own body parts or body respectively. The aim of the present paper is to discuss how (...) and why BOIs are induced. We review several experimental findings concerning the spatial, temporal and semantic principles of crossmodal stimuli that have been applied to induce BOIs. On the basis of these principles, we discuss theoretical approaches concerning the underlying mechanism of BOIs. We propose a conceptualization based on Bayesian causal inference for addressing how our nervous system could infer whether an object belongs to our own body, using multisensory, sensorimotor and semantic information, and we discuss how this can account for several experimental findings. Finally, we point to neural network models as an implementational framework within which the computational problem behind BOIs could be addressed in the future. (shrink)

Visual, somatosensory, and perspectival cues normally provide congruent information about where the self is experienced. Separating those cues by virtual reality techniques, recent studies found that self-location was systematically biased to where a visual–tactile event was seen. Here we developed a novel, repeatable and implicit measure of self-location to compare and extend previous protocols. We investigated illusory self-location and associated phenomenological aspects in a lying body position that facilitates clinically observed abnormal self-location . The results confirm that the self is (...) located to where touch is seen. This leads to either predictable lowering or elevation of self-localization, and the latter was accompanied by sensations of floating, as during out-of-body experiences. Using a novel measurement we show that the unitary and localized character of the self can be experimentally separated from both the origin of the visual perspective and the location of the seen body, which is compatible with clinical data. (shrink)