What is the correct procedure for determining the contents of perception? Philosophers tackling this question increasingly rely on empirically-oriented procedures in order to reach an answer. I argue that this constitutes an improvement over the armchair methodology constitutive of phenomenal contrast cases, but that there is a crucial respect in which current empirical procedures remain limited: they are unimodal in nature, wrongly treating the senses as isolatable faculties. I thus have two aims: first, to motivate a reorientation of the (...) admissible contents debate into a multimodal framework, charting its various significances. The second is to explore whether any experimental studies of multimodal perception support a so-called Liberal (or ‘high-level’ or ‘rich’) account of perception’s admissible contents. I conclude that the McGurk effect and the ventriloquist effect are both explicable without the postulation of high-level content, but that at least one multimodal experimental paradigm may necessitate such content: the rubber hand illusion. One upshot of this argument is that Conservatives who claim that the Liberal view intolerably broadens the scope of perceptual illusions, particularly from the perspective of perceptual psychology, should pursue other arguments against that view. (shrink)

Everyday media consumption leaves no doubt that the perception of moving images from various media is characterized by experience and understanding. Corresponding research in this field has shown that the stimulus patterns flooding in on us are not only processed mentally, but also bodily. Building on this, the following study argues that incoming stimuli are processed not only visually, but multimodally, with all senses, and moreover affectively. The classical binding of a sensory organ to a medium, on whose delimitation (...) the common understanding of multimodality is based, is thus abolished. Can arguments for this assumption be found? Building on the research of cultural anthropologist Ernst Cassirer and his successors, the two American philosophers Susanne K. Langer and John M. Krois, and supported by research from the life sciences in which, in parallel, the arts also play an important role in understanding image formation, as in the work of Heinz Werner and, following on from this, in the more recent research of the child psychologist Daniel N. Stern in collaboration with mirror neuron research by the group of neuroscientists around Giacomo Rizzolatti in Parma, justifications for this will be provided. What the multimodal processing of film images is concretely based on will finally be made clear by means of an exemplary analysis of a video film by the Swiss artist Pipilotti Rist. It aims to prove, that the prerequisites for the thesis can only lie in the fact, that both the modes of perception (visual, tactile, auditory, olfactory, gustatory) as well as the medially conditioned modes of image formation are based on non-discursive, formal-abstract structures that are also processed affectively. At the same time, the analysis thus shows how the selection and composition of the vital experienced forms can provide information about the socio-culturally shaped self-image of the producer and the viewer. (shrink)

The aim of this paper is to argue that our experience of artworks is normally multimodal. It is the result of perceptual processing in more than one sense modality. In other words, multimodal experience of art is not the exception; it is the rule. I use the example of music in order to demonstrate the various ways in which the visual sense modality influences the auditory processing of music and conclude that this should make us look more closely (...) at our practices of engaging with artworks. (shrink)

This is the editors' Introduction to a special issue of the journal, Multisensory Research. European philosophers of the modern period found multisensory perception to be impossible because they thought that perceptual ideas are defined by how they are experienced. Under this conception, the individual modalities are determinables of ideas—just as colour is a determinable that embraces red and blue, so also the visual is a determinable that embraces colour and (visually experienced) shape. Since no idea is experienced as, for (...) example, both visual and auditory, there can be no such thing as audiovisual perception. This conception of modality is not directly contested, but a variety of perceptual phenomena are listed that could raise interesting questions if treated as multimodal in origin. (shrink)

Suppose that you are at a live jazz show. The drummer begins a solo. You see the cymbal jolt and you hear the clang. But in addition seeing the cymbal jolt and hearing the clang, you are also aware that the jolt and the clang are part of the same event. Casey O’Callaghan (forthcoming) calls this awareness “intermodal feature binding awareness.” Psychologists have long assumed that multimodal perceptions such as this one are the result of a subpersonal feature binding (...) mechanism (see Vatakis and Spence, 2007, Kubovy and Schutz, 2010, Pourtois et al., 2000, and Navarra et al., 2012). I present new evidence against this. I argue that there is no automatic feature binding mechanism that couples features like the jolt and the clang together. Instead, when you experience the jolt and the clang as part of the same event, this is the result of an associative learning process. The cymbal’s jolt and the clang are best understood as a single learned perceptual unit, rather than as automatically bound. I outline the specific learning process in perception called “unitization,” whereby we come to “chunk” the world into multimodal units. Unitization has never before been applied to multimodal cases. Yet I argue that this learning process can do the same work that intermodal binding would do, and that this issue has important philosophical implications. Specifically, whether we take multimodal cases to involve a binding mechanism or an associative process will have impact on philosophical issues from Molyneux’s question to the question of how active or passive we consider perception to be. (shrink)

Mental imagery is early perceptual processing that is not triggered by corresponding sensory stimulation in the relevant sense modality. Multimodal mental imagery is early perceptual processing that is triggered by sensory stimulation in a different sense modality. For example, when early visual or tactile processing is triggered by auditory sensory stimulation, this amounts to multimodal mental imagery. Pulling together philosophy, psychology and neuroscience, I will argue in this paper that multimodal mental imagery plays a crucial role in (...) our engagement with musical works. Engagement with musical works is normally a multimodal phenomenon, where we get input from a number of sense modalities. But even if we screen out any input that is not auditory, multimodal mental imagery will still play an important role that musicians and composers often actively rely on. (shrink)

With a Call for Essays, the special issue Multimodality sought contributions that accept not only the material but also the body-bound dependence of media perception and understanding. To this end, contributions were included that shed light on both the structural and signifying potential of artistic works through multimodal analysis. Particular attention was paid to contributions that clarify how the structural features - the modes - of the arts, their perception, and their signifying potential in terms of content (...) are interrelated and how they are to be understood in communicative and thus socioculturally relevant terms. Thus, in addition to neuroscientific contributions, those from cultural anthropology, art history, image and art studies, and literary studies were included. -/- Articles -/- 8 Martina Sauer and Christiane Wagner - Introduction -/- 11 Giada Lombardi and Giuseppe Di Cesare - From Neuroscience to Art: The Role of ‘Vitality Forms’ in the Investigation of Multimodality -/- 25 Martina Sauer - Abstract - Affective – Multimodal: Interaction between Medium and Perception of Moving Images from the Viewpoint of Cassirer’s, Langer’s and Krois’ Embodiment Theories -/- 49 Göran Sonesson - The Secrets of Plastic Language Revealed: Multimodality, Polysemiosis, and Iconicity -/- 71 Natalia Igl - Making Sense of the Material: Multisensory Reader Involvement in Contemporary Multimodal Novels -/- 87 J. Philippe Thompson - Art is All: Proust’s Life Lessons Through Perception and Art -/- 107 Hans Dieter Huber - Philippe Parreno and the Exhibition as a Multimodal Aesthetic Experience -/- 135 Jörg U. Lensing - The Performing Human Being in a Media Interaction Space: Multi- and intermodal productions by the Düsseldorf Theater der Klänge (Theatre of Sounds) . (shrink)

A typical claim in anti-representationalist approaches to cognition such as ecological psychology or radical embodied cognitive science is that ecological information is sufficient for guiding behavior. According to this view, affordances are immediately perceptually available to the agent (in the so-called “ambient energy array”), so sensory data does not require much further inner processing. As a consequence, mental representations are explanatorily idle: perception is immediate and direct. Here we offer one way to formalize this direct-perception claim and identify (...) some important limits to it. We argue that the claim should be read as saying that successful behavior just implies picking out affordance-related information from the ambient energy array. By relying on the Partial Information Decomposition framework, and more concretely on its development of the notion of synergy, we show that in multimodal perception, where various energy arrays carry affordance-related information, the “just pick out affordance-related information” approach is very inefficient, as it is bound to miss all synergistic components. Efficient multimodal information combination requires transmitting sensory-specific (and not affordance-specific) information to wherever it is that the various information streams are combined. The upshot is that some amount of computation is necessary for efficient affordance reconstruction. (shrink)

This is an excerpt of a report that highlights and explores five questions which arose from The Unity of Consciousness and Sensory Integration conference at Brown University in November of 2011. This portion of the report explores the question: Are some of the basic units of consciousness multimodal?

In this short paper, I outline four philosophical arguments concerning the objects we perceive. These arguments build up to the conclusion that the objects of perceptual experience are material objects. I then show that the first three of these arguments parallel important psychological positions in vision science. Thus, (1) the notion of object used in Logical Atomism resembles the concept as it is defined in the Feature Integration Theory of Treisman and Gelade (1980). But (2) Frank Jackson's (1975) Many-Property Problem (...) leads to a substrate-quality structure for visual experience that corresponds to the structure of attended experience in Feature Integration Theory. A third philosophical argument (3) concerns the perception of movement. This argument culminates in the proposition that perceptual features are seen as belonging to material objects. There is a close resemblance between these objects and those of Zenon Pylyshyn's (1980) FINSTs. Finally, (4) I argue that material objects are the only substrates of multimodal perceptual experience. Perceptual psychology has yet to investigate this structure. (shrink)

Is it true that there is an analogy between modes of creation and such of perception? Respective to the cultural anthropological research of Ernst Cassirer, Susanne K. Langer and John M. Krois and by the analysis of a tape of the Swiss video-artist Pipilotti Rist this initial thesis of Formal Aesthetics shall be supported. - I - -/- Lässt sich die Behauptung stützen, dass zwischen Gestaltungsweisen und Wahrnehmungsweisen eine Analogie besteht? Aufbauend auf den kulturanthropologischen Forschungen von Ernst Cassirer, Susanne (...) K. Langer und John M. Krois sowie der Analyse eines Tapes der Schweizer Videokünstlerin Pipilotti Rist gilt es diese ursprünglich von der formalen Ästhetik aufgeworfene These zu untermauern. (shrink)

Within contemporary philosophy of perception, it is commonly claimed that flavour experiences are paradigmatic examples of multimodal perceptual experiences. In fact, virtually any sensory system, including vision and audition, is believed to influence how we experience flavours. However, there is a strong intuition, often expressed in these works, that not all of these sensory systems make an equal contribution to the phenomenology of flavour experiences. More specifically, it seems that the activities of some sensory systems are constitutive for (...) flavour perception while others merely influence how we experience flavours. This paper aims to answer the question regarding the constitutive factors of flavour perception in a twofold way. First, a theoretical framework is developed, relying on debates regarding constitutivity in analytic metaphysics and philosophy of science, which defines the stronger and weaker senses in which the activities of sensory systems may be constitutive for flavour perception. Second, relying on empirical results in flavour science, the constitutive status of activities related to distinct sensory systems in the context of flavour perception is investigated. (shrink)

In this paper I develop a novel account of the phenomenality of language by focusing on characteristics of perceived speech. I explore the extent to which the spoken word can be said to have a horizonal structure similar to that of spatiotemporal objects: our perception of each is informed by habitual associations and expectations formed through past experiences of the object or word and other associated objects and experiences. Specifically, the horizonal structure of speech in use can fruitfully be (...) compared to that of a tool in use. The comparison suggests an account of our linguistic faculty as continuous with more foundational faculties of perception and action. I provide empirical corroboration of this account by drawing on recent neuroimaging studies of the multimodal, sensorimotor bases of speech comprehension. I then discuss how such an understanding of our linguistic ability helps advocates of embodied, non-representationalist accounts of cognition respond to a common objection. Critics grant that embodied approaches may be adequate to account for lower-level, online modes of cognition, such as perception and action, which directly engage their object. But they question whether such approaches can “scale up” to higher modes of cognition, such as imagination, memory, thought, and language, which can entertain absent, non-existent, or abstract objects. By providing a plausible account of the continuity of lower cognition and language-involving cognition, my approach responds to this objection, at least where language is concerned. (shrink)

Sense interaction is ubiquitous. All conscious experiences involve at least some interaction between the senses. One of the most debated questions in recent scholarship concerns the proper way of characterizing the phenomenology of multisensory experiences. According to Charles Spence and Tim Bayne (2015), the phenomenal character of multisensory integration is reducible to the co-conscious sum of modality-specific features. Following Casey O'Callaghan (2015), we can call this The Thesis of Minimal Multimodality. The main goal of the paper is to refute the (...) thesis and show that the effect of multisensory integration is reflected in experience in a way that is not exhausted by (the sum of) modality-specific phenomenal features. Whereas O'Callaghan's strategy in trying to prove this consists in providing phenomenological evidence that runs contrary to what Spence and Bayne hold, he does not, however, put into question what has become since T. Nagel's famous essay (1974) the standard conception of the phenomenal. By drawing conceptual resources in the work of Husserl and Merleau-Ponty, I will elaborate another approach and explain why the phenomenal character of experience cannot be reduced to its "what it's like" character. Phenomenal experience is thicker, and also includes various forms of bodily self-experiences and felt possibilities of action and behaviours. Building on these and like insights, the main objective of the paper is to outline a phenomenological account of multimodal perception and sensory interaction. I will draw three main conclusions. First, I will argue that both Husserl and Merleau-Ponty would, too, reject The Thesis of Minimal Multimodality and endorse the strong, Constitutive Thesis that O'Callaghan defends. More interestingly still, we will see, secondly, that the widening of phenomenal consciousness that they argue for allows to generalize the constitutive thesis, thereby showing that all perceptual experience constitutively depend on the interplay of two or more senses. Spelling out the details of this claim will bring me, thirdly, to specify the intrinsic relation between normativity and perceptual experience. In short, I will argue that the mechanisms responsible for multimodality make a phenomenological difference evaluable in normative terms. (shrink)

The first premise of Schellenberg’s particularity argument reads, “If a subject S perceives a particular α, then S discriminates and singles out α” (2018: 25). But this is false if seeing a ganzfeld is possible (i.e., a homogeneous field without any particulars to discriminate). In response, Schellenberg argues that seeing a ganzfeld is impossible by appealing to the ganzfeld effect (viz. hallucinatory experiences caused by ganzfeld exposure) exclusively as a ‘sense of blindness’. I present two challenges for this line of (...) reasoning. The first concerns the need to account for the long delay before blindness is reported (if it is reported at all) and the second concerns the need to be consistent with potential causes of ganzfeld hallucinations (in general, not exclusive to the sense of blindness). I show how Schellenberg’s particularity argument can meet the first challenge but struggles to meet the second. I present evidence based on considerations related to sensory deprivation and that follow from the predictive processing theory of cognition, which describes seeing a ganzfeld as the cause of ganzfeld hallucinations. If correct, this means that you can see a homogeneous field, and thus we ought to be skeptical of the soundness of Schellenberg’s particularity argument. I conclude with a possible strategy for rehabilitating the particularity argument that reconceptualizes perceptual experience as fundamentally multimodal, constituted by both exteroceptive and interoceptive sensation. (shrink)

The aim of this paper is to raise some serious worries about anti-representationalism: the recently popular view according to which there are no perceptual representations. Although anti-representationalism is more and more popular, I will argue that we have strong empirical reasons for mistrusting it. More specifically, I will argue that it is inconsistent with some important empirical findings about dorsal perception and about the multimodality of perception.

Molyneux asked whether a newly sighted person could distinguish a sphere from a cube by sight alone, given that she was antecedently able to do so by touch. This, we contend, is a question about general ideas. To answer it, we must ask (a) whether spatial locations identified by touch can be identified also by sight, and (b) whether the integration of spatial locations into an idea of shape persists through changes of modality. Posed this way, Molyneux’s Question goes substantially (...) beyond question (a), about spatial locations, alone; for a positive answer to (a) leaves open whether a perceiver might cross-identify locations, but not be able to identify the shapes that collections of locations comprise. We further emphasize that MQ targets general ideas so as to distinguish it from corresponding questions about experiences of shape and about the property of tangible (vs. visual) shape. After proposing a generalized formulation of MQ, we extend earlier work (“Many Molyneux Questions,” Australasian Journal of Philosophy 2020) by showing that MQ does not admit a single answer across the board. Some integrative data-processes transfer across modalities; others do not. Seeing where and how such transfer succeeds and fails in individual cases has much to offer to our understanding of perception and its modalities. (shrink)

In the case of ventriloquism, seeing the movement of the ventriloquist dummy’s mouth changes your experience of the auditory location of the vocals. Some have argued that cases like ventriloquism provide evidence for the view that at least some of the content of perception is fundamentally multimodal. In the ventriloquism case, this would mean your experience has constitutively audio-visual content (not just a conjunction of an audio content and visual content). In this paper, I argue that cases like (...) ventriloquism do not in fact warrant that conclusion. I then try to make sense of crossmodal cases without appealing to fundamentally multimodal content. (shrink)

Philosophical discussions of Molyneux's problem within contemporary philosophy of mind tend to characterize the problem as primarily concerned with the role innately known principles, amodal spatial concepts, and rational cognitive faculties play in our perceptual lives. Indeed, for broadly similar reasons, rationalists have generally advocated an affirmative answer, while empiricists have generally advocated a negative one, to the question Molyneux posed after presenting his famous thought experiment. This historical characterization of the dialectic, however, somewhat obscures the role Molyneux's problem has (...) played in spawning debates within the empiricist tradition. Fortunately, the differences between various empiricist accounts have been widely recognized and discussed among historians of philosophy working on the topic. The focus of the present essay is to develop an interpretation of John Locke's views on Molyneux's problem that best coheres with his other views on human understanding as well as with the predominant scientific opinion about the nature of perception during the period in which he lived. (shrink)

While it is commonly observed that visual experiences have an egocentric character, it is less clear how to properly characterize it. This manuscript presents a new argument in favor of a thesis that (a) visual experiences represent a subject-element, i.e., an element to which the perceived objects stand in egocentric relations, and (b) the subject-element is represented as a complex bodily structure. More specifically, it is argued that there are two plausible interpretations of directional perceptual qualities such as ‘being to (...) the left’: (a) representing directional qualities consists in representing objects as having intrinsic properties, or (b) representing directional qualities consists in representing objects as related to parts of the bodily structure. Relying on this observation, it is shown that only the second option can explain the role of directional qualities in action selection based on visual experiences. Consequently, the most plausible interpretation of directional qualities is that, in representing objects as being to the left or right, visual experiences represent objects as being related to parts of the bodily structure. (shrink)

Schwenkler (2012) criticizes a 2011 experiment by R. Held and colleagues purporting to answer Molyneux’s question. Schwenkler proposes two ways to re-run the original experiment: either by allowing subjects to move around the stimuli, or by simplifying the stimuli to planar objects rather than three-dimensional ones. In Schwenkler (2013) he expands on and defends the former. I argue that this way of re-running the experiment is flawed, since it relies on a questionable assumption that newly sighted subjects will be able (...) to appreciate depth cues. I then argue that the second way of re-running the experiment is successful both in avoiding the flaw of original Held experiment, and in avoiding the problem with the first way of re-running the experiment. (shrink)

I can be wrong about things I seem to perceive; the conditions might lead me to be mistaken about them. Since I can't rule out the possibility that the conditions are misleading, I can't be sure that I am perceiving this thing in my hand correctly. But suppose that I am able to examine it actively—handling it, looking closer, shining a light on it, and so on. Then, my level of uncertainty goes down; in the limit it is eliminated entirely. (...) Of course, I might be mistaken because I am a brain in a vat, or because some other sceptical scenario obtains. But sceptical scenarios apply not just to this thing in my hand, but to everything all at once. Define "empirical certainty" as the absence of the kind of doubt that spreads to all empirical states of affairs. Sensory exploration, or active examination, can in principle lead to empirical certainty. (shrink)

This chapter offers an indirect defence of the Evansian conception of egocentric space, by showing how it resolves a puzzle concerning the unity of egocentric spatial perception. The chapter outlines several common assumptions about egocentric perspectival structure and argues that a subject’s experience, both within and across her sensory modalities, may involve multiple structures of this kind. This raises the question of how perspectival unity is achieved, such that these perspectival structures form a complex whole, rather than merely disunified (...) set of individually, distinctively structured experiences. The shortcomings of variety of accounts are considered: switch accounts ; sensory accounts; transformation accounts; and ultimate accounts. These shortcomings are addressed by a further kind of account provided by the Evansian conception – an agentive account – according to which egocentrically structured experiences present the world in relation to parts of a single thing, the body as a dynamic unity. (shrink)

Jesse Prinz (2004) argues that emotional feelings (“state emotions”) can by themselves perceptually represent significant organism-environment relations. I object to this view mainly on the grounds that (1) it does not rule out the at least equally plausible view that emotional feelings are non-representational sensory registrations rather than perceptions, as Tyler Burge (2010) draws the distinction, and (2) perception of a relation requires perception of at least one of the relation’s relata, but an emotional feeling by itself perceives (...) neither the subject’s environment, nor in many cases the relevant subject itself. I then explore two ways in which emotional feelings as non-perceptual sensory registrations might still contribute to significant relation representation when associated with representations of the subject and/or its environment. After briefly discussing some difficulties presented by a multimodal, sensory-perceptual view of such representation, I argue in favor of a “cognitive recognition theory” that holds that significant relation instances are represented during emotion occurrences via applications of emotion-type concepts to "incoming" emotional feelings and their associated mental states. (shrink)

In our everyday interactions we easily and effortlessly perceive emotions in others’ facial expressions and bodily behavior. How do we do that? Philosophers and psychologists have long argued about the fundamentals of emotion perception and the debate is far from settled. While some insist on the sufficiency of the morphological information contained in facial expressions, others construe the objects of emotion perception as more complex, comprising multimodal information such as touch, tone of voice, body postures, and so (...) on. Others, in turn, have gone so far as to deny that there is a pre-given object in emotion perception in the first place, so that processes of emotion perception must be conceptualized as emergent and co-constructed phenomena in contexts of social interaction. The purpose of this paper is to argue that John Dewey’s pragmatist theory of emotions can help us make significant theoretical advancements in the understanding of emotion perception, according to which emotions are coordinated modes of behavior that we enact with others in the contexts of social interactions. In this picture, perceiving emotions is a matter of participating in a joint activity of socio- affective coordination. This theory allows us to apprehend important insights about emotion perception that appear in these contemporary debates, while at the same time avoiding their pitfalls. (shrink)

This report highlights and explores five questions that arose from the multisensory integration workshop at the University of Toronto on May 9th and 10th, 2014: 1. What Is Multisensory Integration? 2. Do Multisensory Percepts Involve Emergent Features? 3. What Can Multisensory Processing Tell Us about Multisensory Awareness? 4. Is Language Processing a Special Kind of Multisensory Integration? 5. What Is the Purpose of Multisensory Integration?

There is converging evidence that high doses of hallucinogenic drugs can produce significant alterations of self-experience, described as the dissolution of the sense of self and the loss of boundaries between self and world. This article discusses the relevance of this phenomenon, known as “drug-induced ego dissolution (DIED)”, for cognitive neuroscience, psychology and philosophy of mind. Data from self-report questionnaires suggest that three neuropharmacological classes of drugs can induce ego dissolution: classical psychedelics, dissociative anesthetics and agonists of the kappa opioid (...) receptor (KOR). While these substances act on different neurotransmitter receptors, they all produce strong subjective effects that can be compared to the symptoms of acute psychosis, including ego dissolution. It has been suggested that neuroimaging of DIED can indirectly shed light on the neural correlates of the self. While this line of inquiry is promising, its results must be interpreted with caution. First, neural correlates of ego dissolution might reveal the necessary neurophysiological conditions for the maintenance of the sense of self, but it is more doubtful that this method can reveal its minimally sufficient conditions. Second, it is necessary to define the relevant notion of self at play in the phenomenon of DIED. This article suggests that DIED consists in the disruption of subpersonal processes underlying the “minimal” or “embodied” self, i.e., the basic experience of being a self rooted in multimodal integration of self-related stimuli. This hypothesis is consistent with Bayesian models of phenomenal selfhood, according to which the subjective structure of conscious experience ultimately results from the optimization of predictions in perception and action. Finally, it is argued that DIED is also of particular interest for philosophy of mind. On the one hand, it challenges theories according to which consciousness always involves self-awareness. On the other hand, it suggests that ordinary conscious experience might involve a minimal kind of self-awareness rooted in multisensory processing, which is what appears to fade away during DIED. (shrink)

Long-standing neglect of the chemical senses in the philosophy of perception is due, mostly, to their being regarded as ‘lower’ senses. Smell, taste, and chemically irritated touch are thought to produce mere bodily sensations. However, empirically informed theories of perception can show how these senses lead to perception of objective properties, and why they cannot be treated as special cases of perception modelled on vision. The senses of taste, touch, and smell also combine to create unified (...) perceptions of flavour. The nature of these multimodal experiences and the character of our awareness of them puts pressure on the traditional idea that each episode of perception goes one or other of the five senses. Thus, the chemical senses, far from being peripheral to the concerns of the philosophy of perception, may hold important clues to the multisensory nature of perception in general. (shrink)

The usual visual experiences possess a perspectival phenomenology as they seem to present objects from a certain perspective. Nevertheless, it is not obvious how to characterise experiential content determining such phenomenology. In particular, while there are many works investigating perspectival properties of experienced objects, a question regarding how subject is represented in visual perspectival experiences attracted less attention. In order to address this problem, I consider four popular phenomenal intuitions regarding perspectival experiences and argue that the major theories of perspectival (...) experiences do not account for all of them. Relying on these observations, I show how a theory which accommodates all these intuitions can be developed by (a) recognising that visual perspectival experiences are, in fact, multimodal visuo-bodily experiences and (b) distinguishing between egocentric and structural contents. (shrink)

The predictive coding (PC) theory of attention identifies attention with the optimization of the precision weighting of prediction error. Here we provide some challenges for this identification. On the one hand, the precision weighting of prediction error is too broad a phenomenon to be identified with attention because such weighting plays a central role in multimodal integration. Cases of crossmodal illusions such as the rubber hand illusion and the McGurk effect involve the differential precision weighting of prediction error, yet (...) attention does not shift as one would predict. On the other hand, the precision weighting of prediction error is too narrow a phenomenon to be identified with attention, because it cannot accommodate the full range of attentional phenomena. We review criticisms that PC cannot account for volitional attention and affect-biased attention, and we propose that it may not be able to account for feature-based and intellectual attention. (shrink)

This report highlights and explores five questions which arose from The Unity of Consciousness and Sensory Integration conference at Brown University in November of 2011: 1. What is the relationship between the unity of consciousness and sensory integration? 2. Are some of the basic units of consciousness multimodal? 3. How should we model the unity of consciousness? 4. Is the mechanism of sensory integration spatio-temporal? 5. How Should We Study Experience, Given Unity Relations?

This report highlights and explores five questions which arose from the workshop on perceptual learning and perceptual recognition at the University of Toronto, Mississauga on May 10th and 11th, 2012: 1. How should we demarcate perceptual learning from perceptual development? 2. What are the origins of multimodal associations? 3. Does our representation of time provide an amodal framework for multi-sensory integration? 4. What counts as cognitive penetration? 5. How can philosophers and psychologists most fruitfully collaborate?

It is often assumed that neural representation, with content that is in principle detachable from the flow of natural-factive information, is necessary to perceptually experience an object. In this paper we present and discuss two cases challenging this assumption. We take them to show that it is possible to experience an object with which you are interacting through your sensory systems without those systems constructing a representation of the object. The first example is viewing nearby medium-sized groups of objects. The (...) second is hearing objects through misbound sounds. These cases bring out two different ways object representation can fail while object experience persists, suggesting that object experience requires only that the object be revealed through sensory information, not full-blown representation. Constructing object representations is one way sensory systems reveal objects, but it is not the only way. Object representation in neural sensory systems is relatively demanding, rare, and fragile; object experience is relatively easy, pervasive, and robust. We conclude that even minimal forms of neural phenomenal internalism are false. (shrink)

Traditional medical Artificial Intelligence models, approved for clinical use, restrict themselves to single-modal data e.g. images only, limiting their applicability in the complex, multimodal environment of medical diagnosis and treatment. Multimodal Transformer Models in healthcare can effectively process and interpret diverse data forms such as text, images, and structured data. They have demonstrated impressive performance on standard benchmarks like USLME question banks and continue to improve with scale. However, the adoption of these advanced AI models is not without (...) challenges. While multimodal deep learning models like Transformers offer promising advancements in healthcare, their integration requires careful consideration of the accompanying ethical and environmental challenges. (shrink)

This research article examines the potential of multimodal techniques in promoting inclusive practices in the English language classroom, delving into the multidisciplinary fields of English Language Teaching (ELT) and inclusive education. This study intends to investigate how multimodal resources, such as visual aids, technology, and creative activities, may be effectively integrated to meet the various learning needs of students using a pedagogical lens. This paper examines the theoretical underpinnings and practical implications of using multimodal approaches in ELT (...) settings, shedding light on their potential to promote inclusivity, engagement, and language acquisition among learners with diverse abilities and backgrounds. It does this through a thorough literature review and analysis of pedagogical frameworks. (shrink)

This paper presents an analytical framework for analyzing how multimodal resources of emotion expression are semiotically materialized in discursive interactions specific to political discourse. Interested in how political personae are emotionally constructed through multimodal meaning-making practices, our analysis model assumes an interdisciplinary perspective, which integrates facial expression analysis – using FaceReader™ software –, the theory of emotional arcs and bodily actions (hand gestures) analysis that express emotions, in the analytical framework of multimodality. The results show how the (...) class='Hi'>multimodal choices that political actors make during discursive interactions allow them to build their political brand and make connections with the audience on an emotional level. (shrink)

From the perspective of distributed cognition I will stress how abduction is essentially multimodal, in that both data and hypotheses can have a full range of verbal and sensory representations, involving words, sights, images, smells, etc., but also kinesthetic – related to the ability to sense the position and location and orientation and movement of the body and its parts – and motor experiences and other feelings such as pain, and thus all sensory modalities. The presence of kinesthetic and (...) motor aspects demonstrates that abductive reasoning can be manipulative. We can also see, in this regard, how implicit factors take part in the abductive procedure, which consequently acquires the character of a kind of “thinking through doing”. This paper further describes 1) the fact that hypotheses in science can be built through different cognitive mediators and so they can also model the same cognitive aspect in different ways; how they can be carriers/ producers of knowledge in a multimodal way; 2) the problem of the possible non-explanatory and instrumental nature of abductive reasoning and the analysis of the consequences for induction; 3) the role of manipulative abduction in building new evidence/ experiments and how they trigger smart inductive inferences. (shrink)

The analysis of multimodal argumentation in advertising is a crucial and problematic area of research. While its importance is growing in a time characterized by images and pictorial messages, the methods used for interpreting and reconstructing the structure of arguments expressed through verbal and visual means capture only isolated dimensions of this complex phenomenon. This paper intends to propose and illustrate a methodology for the reconstruction and analysis of “double-mode” arguments in advertisements, combining the instruments developed in social semiotics, (...) pragmatics, and argumentation theory. An advertisement is processed through a five-step path. The analysis of its context, text genre, and images leads to a first representation of the messages that it encodes both pictorially and verbally (step 1). These first semantic representations are further enriched by including their polyphonic articulations and presuppositions (step 2), their explicatures (step 3), and their dialogical functions and illocutionary forces (step 4). These pragmatic steps retrieve the commitment structure of the ad, which allows a further argument analysis conducted through argumentation schemes (step 5). (shrink)

According to Phillips, (1) genuine perception is attributable to the individual (i.e. it’s a personal state/event, as opposed to sub-personal states/events in the individual’s brain); (2) since unconscious perceptual representations are ill-suited to guide action, there’s no good reason to attribute them to the individual; (3) not being attributable to the individual, they don’t instantiate genuine perception, thereby failing to support the hypothesis that genuine perception can occur unconsciously. I argue that this reasoning is flawed and can’t (...) be easily fixed. Phillips contends that unconscious perceptual representations don’t guide action because they fail to meet certain conditions that are sufficient for action guidance. But those conditions may not be necessary for action guidance. Consequently, unconscious perceptual representations may guide action even if they don’t meet Phillips’ conditions. Furthermore, due to his commitment to the distinction between personal and sub-personal states/events, Phillips isn’t in a position to argue that his conditions are necessary for action guidance. For the distinction applies to action as well as to perception, and when genuine action is identified by personal-level criteria, Phillips’ conditions turn out unnecessary for action guidance. (shrink)

What method should we use to determine the nature of perceptual experience? My focus here is the Kantian thought that transcendental arguments can be used to determine the nature of perceptual experience. I set out a dilemma for the use of transcendental arguments in the philosophy of perception, one which turns on a comparison ofthe transcendental method with the first-personal method of early analytic philosophy, and with the empirical methods of much contemporary philosophy of mind. The transcendental method can (...) avoid this dilemma only if it commits to our possessing a capacity for imaginative reflection, one which is capable of identifying certain formal properties of experience.This result indicates some of the commitments which must be made if transcendental arguments are to be used in the philosophy of perception, and it has implications for those views that take the philosophy of perception to be autonomous of the empirical science of perception. (shrink)

This is an excerpt from a report that highlights and explores five questions which arose from the workshop on perceptual learning and perceptual recognition at the University of Toronto, Mississauga on May 10th and 11th, 2012. This excerpt explores the question: What are the origins of multimodal associations?

When we see an object, we also represent those parts of it that are not visible. The question is how we represent them: this is the problem of amodal perception. I will consider three possible accounts: (a) we see them, (b) we have non-perceptual beliefs about them and (c) we have immediate perceptual access to them, and point out that all of these views face both empirical and conceptual objections. I suggest and defend a fourth account, according to which (...) we represent the occluded parts of perceived objects by means of mental imagery. This conclusion could be thought of as a (weak) version of the Strawsonian dictum, according to which “imagination is a necessary ingredient of perception itself”. (shrink)

In this paper, I argue that perception justifies belief about the external world in virtue of its phenomenal character together with its relations to the external world. But I argue that perceptual relations to the external world impact on the justifying role of perception only by virtue of their impact on its representational content. Epistemic level-bridging principles provide a principled rationale for avoiding more radically externalist theories of perceptual justification.

I argue that dreams can contain perceptual elements in multifarious, heretofore unthought-of ways. I also explain the difference between dreams that contain perceptual elements, perceptual experiences that contain dream elements, and having a dream and a perceptual experience simultaneously. I then discuss two applications of the resulting view. First, I explain how my taxonomy of perception in dreams will allow “dream engineers”—who try to alter the content of people’s dreams—to accurately classify different dreams and explore creating new forms of (...) perception in dreams. Second, I consider the consequences of the view for the role of memory in dreaming and imagination. I argue that not every element of dreams or sensory imaginations must rely on memory. The resultant view of sensory imagination provides a counterexample to Hume’s account of sensory imagination, according to which sensory imagination must be built up from faint copies of sensory impressions stored in memory. (shrink)

ABSTRACT The difference between perception and cognition seems introspectively obvious in many cases. Perceiving and thinking have also been assigned quite different roles, in epistemology, in theories of reference and of mental content, in philosophy of psychology, and elsewhere. Yet what is the nature of the distinction? In what way, or ways, do perception and cognition differ? The paper reviews recent work on these questions. Four main respects in which perception and cognition have been held to differ (...) are discussed. First, their phenomenal character, such as the often-remarked vivacity or immediacy of perception. Second, the way in which they represent the world, e.g. the non-propositional nature of the contents, or non-discursive character of the vehicles, that have been held to characterise perceptual representation. Third, their place in cognitive architecture, i.e., roughly, in the information-flow of the mind, such as their alleged (non-)modularity. Fourth, their mind-world relations, e.g. the way in which perceptions seem to be tightly causally linked with distal or proximal stimuli. Against this background, we distinguish some main options for an account of the perception/cognition distinction, in particular concerning whether there is one, several, or no interesting and principled distinction(s) to be drawn here. (shrink)

When we perceive an object, we perceive the object from a perspective. As a consequence of the perspectival nature of perception, when we perceive, say, a circular coin from different angles, there is a respect in which the coin looks circular throughout, but also a respect in which the coin's appearance changes. More generally, perception of shape and size properties has both a constant aspect—an aspect that remains stable across changes in perspective—and a perspectival aspect—an aspect that changes (...) depending on one's perspective on the object. How should we account for the perspectival aspect of spatial perception? We present a framework within which to discuss the perspectival aspect of perception and put forward three desiderata that any account of the perspectival aspect of perception should satisfy. We discuss views on which the perspectival aspect of perception is analyzed in terms of constitutively mind-dependent appearance properties as well as views on which the perspectival aspect of perception is analyzed in terms of representations of mind-independent perspectival properties. (shrink)

One necessary condition on any adequate account of perception is clarity regarding whether unconscious perception exists. The issue is complicated, and the debate is growing in both philosophy and science. In this paper we consider the case for unconscious perception, offering three primary achievements. First, we offer a discussion of the underspecified notion of central coordinating agency, a notion that is critical for arguments that purportedly perceptual states are not attributable to the individual, and thus not genuinely (...) perceptual. We develop an explication of what it is for a representational state to be available to central coordinating agency for guidance of behavior. Second, drawing on this explication, we place a more careful understanding of the attributability of a state to the individual in the context of a range of empirical work on vision-for-action, saccades, and skilled typing. The results place pressure on the skeptic about unconscious perception. Third, reflecting upon broader philosophical themes running through debates about unconscious perception, we highlight how our discussion places pressure on the view that perception is a manifest kind, rather than a natural kind. In doing so, we resist the tempting complaint that the debate about unconscious perception is merely verbal. (shrink)

I aim to show that perception depends counterfactually on the action we want to perform. Perception is not all-purpose: what we want to do does influence what we see. After clarifying how this claim is different from the one at stake in the cognitive penetrability debate and what counterfactual dependence means in my claim, I will give a two-step argument: one’s perceptual attention depends counterfactually on one’s intention to perform an action and one’s perceptual processing depends counterfactually on (...) one’s perceptual attention. If we put these claims together, what we get is that one’s perceptual processing depends counterfactually on one’s intention to perform an action. (shrink)

It seems plausible that visual experiences of darkness have perceptual phenomenal content that clearly differentiates them from absences of visual experiences. I argue, relying on psychological results concerning auditory attention, that the analogous claim is true about auditory experiences of silence. More specifically, I propose that experiences of silence present empty spatial directions like ‘right’ or ‘left’, and so have egocentric spatial content. Furthermore, I claim that such content is genuinely auditory and phenomenal in the sense that one can, in (...) principle, recognize that one is experiencing silence. This position is far from obvious, as the majority of theories concerning silence perception do not ascribe perceptual phenomenal content to experiences of silence. (shrink)