Intermodal representationalists hold that the phenomenal characters of experiences are fully determined by their contents. In contrast, intramodal representationalists hold that the phenomenal characters of experiences are determined by their contents together with their intentional modes or manners of representation, which are nonrepresentational features corresponding roughly to the sensory modalities. This paper discusses a kind of experience that provides evidence for an intermodal representationalist view: intermodal experiences, experiences that unify experiences in different modalities. I argue that such (...) experiences are much easier to explain on the intermodal view. (shrink)

Here is something puzzling. Still Lifes can be expressive. Expression involves movement. Hence, (some) Still Lifes move. This seems odd. I consider a novel explanation to this ‘static-dynamic’ puzzle from Mitchell Green (2007). Green defends an analysis of artistic expressivity that is heavily indebted to work on intermodal perception. He says visual stimuli, like colours and shapes, can elicit experienced resemblances to sounds, smells and feelings. This enables viewers to know how an emotion feels by looking at the picture. (...) The hypothesis is intriguing, but I show that his suggestion that we empathize with the pictorial content is implausible and that this exposes a flaw in the way his argument moves from experiential mappings to experiential-affective mappings. Consequently, I register some reservations about the way Green supposes we detect these cross-modal qualities. (shrink)

The cultural consumption research landscape of the 21st century is marked by an increasing cross-disciplinary fermentation. At the same time, cultural theory and analysis have been marked by successive ‘inter-’ turns, most notably with regard to the Big Four: multimodality (or intermodality), interdiscursivity, transmediality (or intermediality), and intertextuality. This book offers an outline of interdiscursivity as an integrative platform for accommodating these notions. To this end, a call for a return to Foucault is issued via a critical engagement with the (...) so-called practice-turn. This re-turn does not seek to reconstitute venerably Foucauldianism, but to theorize ‘inters-’ as vanishing points that challenge the integrity of discrete cultural orders in non-convergent manners. The propounded interdiscursivity approach is offered as a reading strategy that permeates the contemporary cultural consumption phenomena that are scrutinized in this book, against a pan-consumptivist framework. By drawing on qualitative and mixed methods research designs, facilitated by CAQDAS software, the empirical studies that are hosted here span a vivid array of topics that are directly relevant to both traditional and new media researchers, such as the consumption of ideologies in Web 2.0 social movements, the ability of micro-celebrities to act as cultural game-changers, the post-loyalty abjective consumption ethos. The theoretically novel approaches on offer are coupled with methodological innovations in areas such as user-generated content, artists’ branding, and experiential consumption. (shrink)

Molyneux's Question (MQ) concerns whether a newly sighted man would recognize/distinguish a sphere and a cube by vision, assuming he could previously do this by touch. We argue that (MQ) splits into questions about (a) shared representations of space in different perceptual systems, and about (b) shared ways of constructing higher dimensional spatiotemporal features from information about lower dimensional ones, most of the technical difficulty centring on (b). So understood, MQ resists any monolithic answer: everything depends on the constraints faced (...) by particular perceptual systems in extracting features of higher dimensionality from those of lower. Each individual question of this type is empirical and must be investigated separately. We present several variations on MQ based on different levels of dimensional integration—some of these are familiar, some novel adaptations of problems known elsewhere, and some completely novel. Organizing these cases in this way is useful because it unifies a set of disparate questions about intermodal transfer that have held philosophical and psychological interest, suggests a new range of questions of the same type, sheds light on similarities and differences between members of the family, and allows us to formulate a much-augmented set of principles and questions concerning the intermodal transfer of spatiotemporal organization. (shrink)

Suppose that you are at a live jazz show. The drummer begins a solo. You see the cymbal jolt and you hear the clang. But in addition seeing the cymbal jolt and hearing the clang, you are also aware that the jolt and the clang are part of the same event. Casey O’Callaghan (forthcoming) calls this awareness “intermodal feature binding awareness.” Psychologists have long assumed that multimodal perceptions such as this one are the result of a subpersonal feature binding (...) mechanism (see Vatakis and Spence, 2007, Kubovy and Schutz, 2010, Pourtois et al., 2000, and Navarra et al., 2012). I present new evidence against this. I argue that there is no automatic feature binding mechanism that couples features like the jolt and the clang together. Instead, when you experience the jolt and the clang as part of the same event, this is the result of an associative learning process. The cymbal’s jolt and the clang are best understood as a single learned perceptual unit, rather than as automatically bound. I outline the specific learning process in perception called “unitization,” whereby we come to “chunk” the world into multimodal units. Unitization has never before been applied to multimodal cases. Yet I argue that this learning process can do the same work that intermodal binding would do, and that this issue has important philosophical implications. Specifically, whether we take multimodal cases to involve a binding mechanism or an associative process will have impact on philosophical issues from Molyneux’s question to the question of how active or passive we consider perception to be. (shrink)

It is commonly believed that human perceptual experiences can be, and usually are, multimodal. What is more, a stronger thesis is often proposed that some perceptual multimodal characters cannot be described simply as a conjunction of unimodal phenomenal elements. If it is the case, then a question arises: what is the additional mode of combination that is required to adequately describe the phenomenal structure of multimodal experiences? The paper investigates what types of audio–visual experiences have phenomenal character that cannot be (...) analysed as a mere conjunction of visual and auditory elements; and how can we properly characterise the required, additional mode of perceptual combination. Three main modes of combination are considered: instantiation, parthood, and grouping. It is argued that some phenomena involving intermodal relations, like spatial and temporal ventriloquism, can be analysed in terms of audio–visual, perceptual grouping. On the other hand, cases of intermodal binding need a different treatment. Experiences involving audio–visual binding should be analysed as experiences presenting objects or events which instantiate, or which have a proper part instantiating, both visually and auditorily determined properties. (shrink)

Reinterpretations of Libet's results have received support from most commentaries. Libet's arguments against alternative hypotheses are contested. Latency depends on intensity. Integration of intensity and duration explains the Minimum Train Duration. Analogies of Libet's timing of intentions with control experiments indicate biases of opposite signs, according to intramodal or intermodal results. Rosenthal's view of nonconscious intentions becoming conscious after a delay is favored. Compatibilist free will is discussed.

With a Call for Essays, the special issue Multimodality sought contributions that accept not only the material but also the body-bound dependence of media perception and understanding. To this end, contributions were included that shed light on both the structural and signifying potential of artistic works through multimodal analysis. Particular attention was paid to contributions that clarify how the structural features - the modes - of the arts, their perception, and their signifying potential in terms of content are interrelated and (...) how they are to be understood in communicative and thus socioculturally relevant terms. Thus, in addition to neuroscientific contributions, those from cultural anthropology, art history, image and art studies, and literary studies were included. -/- Articles -/- 8 Martina Sauer and Christiane Wagner - Introduction -/- 11 Giada Lombardi and Giuseppe Di Cesare - From Neuroscience to Art: The Role of ‘Vitality Forms’ in the Investigation of Multimodality -/- 25 Martina Sauer - Abstract - Affective – Multimodal: Interaction between Medium and Perception of Moving Images from the Viewpoint of Cassirer’s, Langer’s and Krois’ Embodiment Theories -/- 49 Göran Sonesson - The Secrets of Plastic Language Revealed: Multimodality, Polysemiosis, and Iconicity -/- 71 Natalia Igl - Making Sense of the Material: Multisensory Reader Involvement in Contemporary Multimodal Novels -/- 87 J. Philippe Thompson - Art is All: Proust’s Life Lessons Through Perception and Art -/- 107 Hans Dieter Huber - Philippe Parreno and the Exhibition as a Multimodal Aesthetic Experience -/- 135 Jörg U. Lensing - The Performing Human Being in a Media Interaction Space: Multi- and intermodal productions by the Düsseldorf Theater der Klänge (Theatre of Sounds) . (shrink)

Do spatial features appear the same whether they are perceived through vision or touch? This question is at stake in the puzzle that William Molyneux posed to John Locke, concerning whether a man born blind whose sight was restored would be able immediately to identify the shapes of the things he saw. A recent study purports to answer the question negatively, but I argue here that the subjects of the study likely could not see well enough for the result to (...) have been meaningful. I then propose a way to improve the study, by including cues from object motion. (shrink)