What is the correct procedure for determining the contents of perception? Philosophers tackling this question increasingly rely on empirically-oriented procedures in order to reach an answer. I argue that this constitutes an improvement over the armchair methodology constitutive of phenomenal contrast cases, but that there is a crucial respect in which current empirical procedures remain limited: they are unimodal in nature, wrongly treating the senses as isolatable faculties. I thus have two aims: first, to motivate a reorientation of the (...) admissible contents debate into a multimodal framework, charting its various significances. The second is to explore whether any experimental studies of multimodal perception support a so-called Liberal (or ‘high-level’ or ‘rich’) account of perception’s admissible contents. I conclude that the McGurk effect and the ventriloquist effect are both explicable without the postulation of high-level content, but that at least one multimodal experimental paradigm may necessitate such content: the rubber hand illusion. One upshot of this argument is that Conservatives who claim that the Liberal view intolerably broadens the scope of perceptual illusions, particularly from the perspective of perceptual psychology, should pursue other arguments against that view. (shrink)

Everyday media consumption leaves no doubt that the perception of moving images from various media is characterized by experience and understanding. Corresponding research in this field has shown that the stimulus patterns flooding in on us are not only processed mentally, but also bodily. Building on this, the following study argues that incoming stimuli are processed not only visually, but multimodally, with all senses, and moreover affectively. The classical binding of a sensory organ to a medium, on whose delimitation (...) the common understanding of multimodality is based, is thus abolished. Can arguments for this assumption be found? Building on the research of cultural anthropologist Ernst Cassirer and his successors, the two American philosophers Susanne K. Langer and John M. Krois, and supported by research from the life sciences in which, in parallel, the arts also play an important role in understanding image formation, as in the work of Heinz Werner and, following on from this, in the more recent research of the child psychologist Daniel N. Stern in collaboration with mirror neuron research by the group of neuroscientists around Giacomo Rizzolatti in Parma, justifications for this will be provided. What the multimodal processing of film images is concretely based on will finally be made clear by means of an exemplary analysis of a video film by the Swiss artist Pipilotti Rist. It aims to prove, that the prerequisites for the thesis can only lie in the fact, that both the modes of perception (visual, tactile, auditory, olfactory, gustatory) as well as the medially conditioned modes of image formation are based on non-discursive, formal-abstract structures that are also processed affectively. At the same time, the analysis thus shows how the selection and composition of the vital experienced forms can provide information about the socio-culturally shaped self-image of the producer and the viewer. (shrink)

The aim of this paper is to argue that our experience of artworks is normally multimodal. It is the result of perceptual processing in more than one sense modality. In other words, multimodal experience of art is not the exception; it is the rule. I use the example of music in order to demonstrate the various ways in which the visual sense modality influences the auditory processing of music and conclude that this should make us look more closely (...) at our practices of engaging with artworks. (shrink)

This is the editors' Introduction to a special issue of the journal, Multisensory Research. European philosophers of the modern period found multisensory perception to be impossible because they thought that perceptual ideas are defined by how they are experienced. Under this conception, the individual modalities are determinables of ideas—just as colour is a determinable that embraces red and blue, so also the visual is a determinable that embraces colour and (visually experienced) shape. Since no idea is experienced as, for (...) example, both visual and auditory, there can be no such thing as audiovisual perception. This conception of modality is not directly contested, but a variety of perceptual phenomena are listed that could raise interesting questions if treated as multimodal in origin. (shrink)

Suppose that you are at a live jazz show. The drummer begins a solo. You see the cymbal jolt and you hear the clang. But in addition seeing the cymbal jolt and hearing the clang, you are also aware that the jolt and the clang are part of the same event. Casey O’Callaghan (forthcoming) calls this awareness “intermodal feature binding awareness.” Psychologists have long assumed that multimodal perceptions such as this one are the result of a subpersonal feature binding (...) mechanism (see Vatakis and Spence, 2007, Kubovy and Schutz, 2010, Pourtois et al., 2000, and Navarra et al., 2012). I present new evidence against this. I argue that there is no automatic feature binding mechanism that couples features like the jolt and the clang together. Instead, when you experience the jolt and the clang as part of the same event, this is the result of an associative learning process. The cymbal’s jolt and the clang are best understood as a single learned perceptual unit, rather than as automatically bound. I outline the specific learning process in perception called “unitization,” whereby we come to “chunk” the world into multimodal units. Unitization has never before been applied to multimodal cases. Yet I argue that this learning process can do the same work that intermodal binding would do, and that this issue has important philosophical implications. Specifically, whether we take multimodal cases to involve a binding mechanism or an associative process will have impact on philosophical issues from Molyneux’s question to the question of how active or passive we consider perception to be. (shrink)

A typical claim in anti-representationalist approaches to cognition such as ecological psychology or radical embodied cognitive science is that ecological information is sufficient for guiding behavior. According to this view, affordances are immediately perceptually available to the agent (in the so-called “ambient energy array”), so sensory data does not require much further inner processing. As a consequence, mental representations are explanatorily idle: perception is immediate and direct. Here we offer one way to formalize this direct-perception claim and identify (...) some important limits to it. We argue that the claim should be read as saying that successful behavior just implies picking out affordance-related information from the ambient energy array. By relying on the Partial Information Decomposition framework, and more concretely on its development of the notion of synergy, we show that in multimodal perception, where various energy arrays carry affordance-related information, the “just pick out affordance-related information” approach is very inefficient, as it is bound to miss all synergistic components. Efficient multimodal information combination requires transmitting sensory-specific (and not affordance-specific) information to wherever it is that the various information streams are combined. The upshot is that some amount of computation is necessary for efficient affordance reconstruction. (shrink)

Mental imagery is early perceptual processing that is not triggered by corresponding sensory stimulation in the relevant sense modality. Multimodal mental imagery is early perceptual processing that is triggered by sensory stimulation in a different sense modality. For example, when early visual or tactile processing is triggered by auditory sensory stimulation, this amounts to multimodal mental imagery. Pulling together philosophy, psychology and neuroscience, I will argue in this paper that multimodal mental imagery plays a crucial role in (...) our engagement with musical works. Engagement with musical works is normally a multimodal phenomenon, where we get input from a number of sense modalities. But even if we screen out any input that is not auditory, multimodal mental imagery will still play an important role that musicians and composers often actively rely on. (shrink)

With a Call for Essays, the special issue Multimodality sought contributions that accept not only the material but also the body-bound dependence of media perception and understanding. To this end, contributions were included that shed light on both the structural and signifying potential of artistic works through multimodal analysis. Particular attention was paid to contributions that clarify how the structural features - the modes - of the arts, their perception, and their signifying potential in terms of content (...) are interrelated and how they are to be understood in communicative and thus socioculturally relevant terms. Thus, in addition to neuroscientific contributions, those from cultural anthropology, art history, image and art studies, and literary studies were included. -/- Articles -/- 8 Martina Sauer and Christiane Wagner - Introduction -/- 11 Giada Lombardi and Giuseppe Di Cesare - From Neuroscience to Art: The Role of ‘Vitality Forms’ in the Investigation of Multimodality -/- 25 Martina Sauer - Abstract - Affective – Multimodal: Interaction between Medium and Perception of Moving Images from the Viewpoint of Cassirer’s, Langer’s and Krois’ Embodiment Theories -/- 49 Göran Sonesson - The Secrets of Plastic Language Revealed: Multimodality, Polysemiosis, and Iconicity -/- 71 Natalia Igl - Making Sense of the Material: Multisensory Reader Involvement in Contemporary Multimodal Novels -/- 87 J. Philippe Thompson - Art is All: Proust’s Life Lessons Through Perception and Art -/- 107 Hans Dieter Huber - Philippe Parreno and the Exhibition as a Multimodal Aesthetic Experience -/- 135 Jörg U. Lensing - The Performing Human Being in a Media Interaction Space: Multi- and intermodal productions by the Düsseldorf Theater der Klänge (Theatre of Sounds) . (shrink)

This is an excerpt of a report that highlights and explores five questions which arose from The Unity of Consciousness and Sensory Integration conference at Brown University in November of 2011. This portion of the report explores the question: Are some of the basic units of consciousness multimodal?

In this short paper, I outline four philosophical arguments concerning the objects we perceive. These arguments build up to the conclusion that the objects of perceptual experience are material objects. I then show that the first three of these arguments parallel important psychological positions in vision science. Thus, (1) the notion of object used in Logical Atomism resembles the concept as it is defined in the Feature Integration Theory of Treisman and Gelade (1980). But (2) Frank Jackson's (1975) Many-Property Problem (...) leads to a substrate-quality structure for visual experience that corresponds to the structure of attended experience in Feature Integration Theory. A third philosophical argument (3) concerns the perception of movement. This argument culminates in the proposition that perceptual features are seen as belonging to material objects. There is a close resemblance between these objects and those of Zenon Pylyshyn's (1980) FINSTs. Finally, (4) I argue that material objects are the only substrates of multimodal perceptual experience. Perceptual psychology has yet to investigate this structure. (shrink)

Is it true that there is an analogy between modes of creation and such of perception? Respective to the cultural anthropological research of Ernst Cassirer, Susanne K. Langer and John M. Krois and by the analysis of a tape of the Swiss video-artist Pipilotti Rist this initial thesis of Formal Aesthetics shall be supported. - I - -/- Lässt sich die Behauptung stützen, dass zwischen Gestaltungsweisen und Wahrnehmungsweisen eine Analogie besteht? Aufbauend auf den kulturanthropologischen Forschungen von Ernst Cassirer, Susanne (...) K. Langer und John M. Krois sowie der Analyse eines Tapes der Schweizer Videokünstlerin Pipilotti Rist gilt es diese ursprünglich von der formalen Ästhetik aufgeworfene These zu untermauern. (shrink)

Within contemporary philosophy of perception, it is commonly claimed that flavour experiences are paradigmatic examples of multimodal perceptual experiences. In fact, virtually any sensory system, including vision and audition, is believed to influence how we experience flavours. However, there is a strong intuition, often expressed in these works, that not all of these sensory systems make an equal contribution to the phenomenology of flavour experiences. More specifically, it seems that the activities of some sensory systems are constitutive for (...) flavour perception while others merely influence how we experience flavours. This paper aims to answer the question regarding the constitutive factors of flavour perception in a twofold way. First, a theoretical framework is developed, relying on debates regarding constitutivity in analytic metaphysics and philosophy of science, which defines the stronger and weaker senses in which the activities of sensory systems may be constitutive for flavour perception. Second, relying on empirical results in flavour science, the constitutive status of activities related to distinct sensory systems in the context of flavour perception is investigated. (shrink)

Sense interaction is ubiquitous. All conscious experiences involve at least some interaction between the senses. One of the most debated questions in recent scholarship concerns the proper way of characterizing the phenomenology of multisensory experiences. According to Charles Spence and Tim Bayne (2015), the phenomenal character of multisensory integration is reducible to the co-conscious sum of modality-specific features. Following Casey O'Callaghan (2015), we can call this The Thesis of Minimal Multimodality. The main goal of the paper is to refute the (...) thesis and show that the effect of multisensory integration is reflected in experience in a way that is not exhausted by (the sum of) modality-specific phenomenal features. Whereas O'Callaghan's strategy in trying to prove this consists in providing phenomenological evidence that runs contrary to what Spence and Bayne hold, he does not, however, put into question what has become since T. Nagel's famous essay (1974) the standard conception of the phenomenal. By drawing conceptual resources in the work of Husserl and Merleau-Ponty, I will elaborate another approach and explain why the phenomenal character of experience cannot be reduced to its "what it's like" character. Phenomenal experience is thicker, and also includes various forms of bodily self-experiences and felt possibilities of action and behaviours. Building on these and like insights, the main objective of the paper is to outline a phenomenological account of multimodal perception and sensory interaction. I will draw three main conclusions. First, I will argue that both Husserl and Merleau-Ponty would, too, reject The Thesis of Minimal Multimodality and endorse the strong, Constitutive Thesis that O'Callaghan defends. More interestingly still, we will see, secondly, that the widening of phenomenal consciousness that they argue for allows to generalize the constitutive thesis, thereby showing that all perceptual experience constitutively depend on the interplay of two or more senses. Spelling out the details of this claim will bring me, thirdly, to specify the intrinsic relation between normativity and perceptual experience. In short, I will argue that the mechanisms responsible for multimodality make a phenomenological difference evaluable in normative terms. (shrink)

In this paper I develop a novel account of the phenomenality of language by focusing on characteristics of perceived speech. I explore the extent to which the spoken word can be said to have a horizonal structure similar to that of spatiotemporal objects: our perception of each is informed by habitual associations and expectations formed through past experiences of the object or word and other associated objects and experiences. Specifically, the horizonal structure of speech in use can fruitfully be (...) compared to that of a tool in use. The comparison suggests an account of our linguistic faculty as continuous with more foundational faculties of perception and action. I provide empirical corroboration of this account by drawing on recent neuroimaging studies of the multimodal, sensorimotor bases of speech comprehension. I then discuss how such an understanding of our linguistic ability helps advocates of embodied, non-representationalist accounts of cognition respond to a common objection. Critics grant that embodied approaches may be adequate to account for lower-level, online modes of cognition, such as perception and action, which directly engage their object. But they question whether such approaches can “scale up” to higher modes of cognition, such as imagination, memory, thought, and language, which can entertain absent, non-existent, or abstract objects. By providing a plausible account of the continuity of lower cognition and language-involving cognition, my approach responds to this objection, at least where language is concerned. (shrink)

The aim of this paper is to raise some serious worries about anti-representationalism: the recently popular view according to which there are no perceptual representations. Although anti-representationalism is more and more popular, I will argue that we have strong empirical reasons for mistrusting it. More specifically, I will argue that it is inconsistent with some important empirical findings about dorsal perception and about the multimodality of perception.

The first premise of Schellenberg’s particularity argument reads, “If a subject S perceives a particular α, then S discriminates and singles out α” (2018: 25). But this is false if seeing a ganzfeld is possible (i.e., a homogeneous field without any particulars to discriminate). In response, Schellenberg argues that seeing a ganzfeld is impossible by appealing to the ganzfeld effect (viz. hallucinatory experiences caused by ganzfeld exposure) exclusively as a ‘sense of blindness’. I present two challenges for this line of (...) reasoning. The first concerns the need to account for the long delay before blindness is reported (if it is reported at all) and the second concerns the need to be consistent with potential causes of ganzfeld hallucinations (in general, not exclusive to the sense of blindness). I show how Schellenberg’s particularity argument can meet the first challenge but struggles to meet the second. I present evidence based on considerations related to sensory deprivation and that follow from the predictive processing theory of cognition, which describes seeing a ganzfeld as the cause of ganzfeld hallucinations. If correct, this means that you can see a homogeneous field, and thus we ought to be skeptical of the soundness of Schellenberg’s particularity argument. I conclude with a possible strategy for rehabilitating the particularity argument that reconceptualizes perceptual experience as fundamentally multimodal, constituted by both exteroceptive and interoceptive sensation. (shrink)

Molyneux asked whether a newly sighted person could distinguish a sphere from a cube by sight alone, given that she was antecedently able to do so by touch. This, we contend, is a question about general ideas. To answer it, we must ask (a) whether spatial locations identified by touch can be identified also by sight, and (b) whether the integration of spatial locations into an idea of shape persists through changes of modality. Posed this way, Molyneux’s Question goes substantially (...) beyond question (a), about spatial locations, alone; for a positive answer to (a) leaves open whether a perceiver might cross-identify locations, but not be able to identify the shapes that collections of locations comprise. We further emphasize that MQ targets general ideas so as to distinguish it from corresponding questions about experiences of shape and about the property of tangible (vs. visual) shape. After proposing a generalized formulation of MQ, we extend earlier work (“Many Molyneux Questions,” Australasian Journal of Philosophy 2020) by showing that MQ does not admit a single answer across the board. Some integrative data-processes transfer across modalities; others do not. Seeing where and how such transfer succeeds and fails in individual cases has much to offer to our understanding of perception and its modalities. (shrink)

In the case of ventriloquism, seeing the movement of the ventriloquist dummy’s mouth changes your experience of the auditory location of the vocals. Some have argued that cases like ventriloquism provide evidence for the view that at least some of the content of perception is fundamentally multimodal. In the ventriloquism case, this would mean your experience has constitutively audio-visual content (not just a conjunction of an audio content and visual content). In this paper, I argue that cases like (...) ventriloquism do not in fact warrant that conclusion. I then try to make sense of crossmodal cases without appealing to fundamentally multimodal content. (shrink)

Philosophical discussions of Molyneux's problem within contemporary philosophy of mind tend to characterize the problem as primarily concerned with the role innately known principles, amodal spatial concepts, and rational cognitive faculties play in our perceptual lives. Indeed, for broadly similar reasons, rationalists have generally advocated an affirmative answer, while empiricists have generally advocated a negative one, to the question Molyneux posed after presenting his famous thought experiment. This historical characterization of the dialectic, however, somewhat obscures the role Molyneux's problem has (...) played in spawning debates within the empiricist tradition. Fortunately, the differences between various empiricist accounts have been widely recognized and discussed among historians of philosophy working on the topic. The focus of the present essay is to develop an interpretation of John Locke's views on Molyneux's problem that best coheres with his other views on human understanding as well as with the predominant scientific opinion about the nature of perception during the period in which he lived. (shrink)

While it is commonly observed that visual experiences have an egocentric character, it is less clear how to properly characterize it. This manuscript presents a new argument in favor of a thesis that (a) visual experiences represent a subject-element, i.e., an element to which the perceived objects stand in egocentric relations, and (b) the subject-element is represented as a complex bodily structure. More specifically, it is argued that there are two plausible interpretations of directional perceptual qualities such as ‘being to (...) the left’: (a) representing directional qualities consists in representing objects as having intrinsic properties, or (b) representing directional qualities consists in representing objects as related to parts of the bodily structure. Relying on this observation, it is shown that only the second option can explain the role of directional qualities in action selection based on visual experiences. Consequently, the most plausible interpretation of directional qualities is that, in representing objects as being to the left or right, visual experiences represent objects as being related to parts of the bodily structure. (shrink)

Schwenkler (2012) criticizes a 2011 experiment by R. Held and colleagues purporting to answer Molyneux’s question. Schwenkler proposes two ways to re-run the original experiment: either by allowing subjects to move around the stimuli, or by simplifying the stimuli to planar objects rather than three-dimensional ones. In Schwenkler (2013) he expands on and defends the former. I argue that this way of re-running the experiment is flawed, since it relies on a questionable assumption that newly sighted subjects will be able (...) to appreciate depth cues. I then argue that the second way of re-running the experiment is successful both in avoiding the flaw of original Held experiment, and in avoiding the problem with the first way of re-running the experiment. (shrink)

I can be wrong about things I seem to perceive; the conditions might lead me to be mistaken about them. Since I can't rule out the possibility that the conditions are misleading, I can't be sure that I am perceiving this thing in my hand correctly. But suppose that I am able to examine it actively—handling it, looking closer, shining a light on it, and so on. Then, my level of uncertainty goes down; in the limit it is eliminated entirely. (...) Of course, I might be mistaken because I am a brain in a vat, or because some other sceptical scenario obtains. But sceptical scenarios apply not just to this thing in my hand, but to everything all at once. Define "empirical certainty" as the absence of the kind of doubt that spreads to all empirical states of affairs. Sensory exploration, or active examination, can in principle lead to empirical certainty. (shrink)

Jesse Prinz (2004) argues that emotional feelings (“state emotions”) can by themselves perceptually represent significant organism-environment relations. I object to this view mainly on the grounds that (1) it does not rule out the at least equally plausible view that emotional feelings are non-representational sensory registrations rather than perceptions, as Tyler Burge (2010) draws the distinction, and (2) perception of a relation requires perception of at least one of the relation’s relata, but an emotional feeling by itself perceives (...) neither the subject’s environment, nor in many cases the relevant subject itself. I then explore two ways in which emotional feelings as non-perceptual sensory registrations might still contribute to significant relation representation when associated with representations of the subject and/or its environment. After briefly discussing some difficulties presented by a multimodal, sensory-perceptual view of such representation, I argue in favor of a “cognitive recognition theory” that holds that significant relation instances are represented during emotion occurrences via applications of emotion-type concepts to "incoming" emotional feelings and their associated mental states. (shrink)

In our everyday interactions we easily and effortlessly perceive emotions in others’ facial expressions and bodily behavior. How do we do that? Philosophers and psychologists have long argued about the fundamentals of emotion perception and the debate is far from settled. While some insist on the sufficiency of the morphological information contained in facial expressions, others construe the objects of emotion perception as more complex, comprising multimodal information such as touch, tone of voice, body postures, and so (...) on. Others, in turn, have gone so far as to deny that there is a pre-given object in emotion perception in the first place, so that processes of emotion perception must be conceptualized as emergent and co-constructed phenomena in contexts of social interaction. The purpose of this paper is to argue that John Dewey’s pragmatist theory of emotions can help us make significant theoretical advancements in the understanding of emotion perception, according to which emotions are coordinated modes of behavior that we enact with others in the contexts of social interactions. In this picture, perceiving emotions is a matter of participating in a joint activity of socio- affective coordination. This theory allows us to apprehend important insights about emotion perception that appear in these contemporary debates, while at the same time avoiding their pitfalls. (shrink)

This report highlights and explores five questions that arose from the multisensory integration workshop at the University of Toronto on May 9th and 10th, 2014: 1. What Is Multisensory Integration? 2. Do Multisensory Percepts Involve Emergent Features? 3. What Can Multisensory Processing Tell Us about Multisensory Awareness? 4. Is Language Processing a Special Kind of Multisensory Integration? 5. What Is the Purpose of Multisensory Integration?

This chapter offers an indirect defence of the Evansian conception of egocentric space, by showing how it resolves a puzzle concerning the unity of egocentric spatial perception. The chapter outlines several common assumptions about egocentric perspectival structure and argues that a subject’s experience, both within and across her sensory modalities, may involve multiple structures of this kind. This raises the question of how perspectival unity is achieved, such that these perspectival structures form a complex whole, rather than merely disunified (...) set of individually, distinctively structured experiences. The shortcomings of variety of accounts are considered: switch accounts ; sensory accounts; transformation accounts; and ultimate accounts. These shortcomings are addressed by a further kind of account provided by the Evansian conception – an agentive account – according to which egocentrically structured experiences present the world in relation to parts of a single thing, the body as a dynamic unity. (shrink)

Long-standing neglect of the chemical senses in the philosophy of perception is due, mostly, to their being regarded as ‘lower’ senses. Smell, taste, and chemically irritated touch are thought to produce mere bodily sensations. However, empirically informed theories of perception can show how these senses lead to perception of objective properties, and why they cannot be treated as special cases of perception modelled on vision. The senses of taste, touch, and smell also combine to create unified (...) perceptions of flavour. The nature of these multimodal experiences and the character of our awareness of them puts pressure on the traditional idea that each episode of perception goes one or other of the five senses. Thus, the chemical senses, far from being peripheral to the concerns of the philosophy of perception, may hold important clues to the multisensory nature of perception in general. (shrink)

There is converging evidence that high doses of hallucinogenic drugs can produce significant alterations of self-experience, described as the dissolution of the sense of self and the loss of boundaries between self and world. This article discusses the relevance of this phenomenon, known as “drug-induced ego dissolution (DIED)”, for cognitive neuroscience, psychology and philosophy of mind. Data from self-report questionnaires suggest that three neuropharmacological classes of drugs can induce ego dissolution: classical psychedelics, dissociative anesthetics and agonists of the kappa opioid (...) receptor (KOR). While these substances act on different neurotransmitter receptors, they all produce strong subjective effects that can be compared to the symptoms of acute psychosis, including ego dissolution. It has been suggested that neuroimaging of DIED can indirectly shed light on the neural correlates of the self. While this line of inquiry is promising, its results must be interpreted with caution. First, neural correlates of ego dissolution might reveal the necessary neurophysiological conditions for the maintenance of the sense of self, but it is more doubtful that this method can reveal its minimally sufficient conditions. Second, it is necessary to define the relevant notion of self at play in the phenomenon of DIED. This article suggests that DIED consists in the disruption of subpersonal processes underlying the “minimal” or “embodied” self, i.e., the basic experience of being a self rooted in multimodal integration of self-related stimuli. This hypothesis is consistent with Bayesian models of phenomenal selfhood, according to which the subjective structure of conscious experience ultimately results from the optimization of predictions in perception and action. Finally, it is argued that DIED is also of particular interest for philosophy of mind. On the one hand, it challenges theories according to which consciousness always involves self-awareness. On the other hand, it suggests that ordinary conscious experience might involve a minimal kind of self-awareness rooted in multisensory processing, which is what appears to fade away during DIED. (shrink)

The usual visual experiences possess a perspectival phenomenology as they seem to present objects from a certain perspective. Nevertheless, it is not obvious how to characterise experiential content determining such phenomenology. In particular, while there are many works investigating perspectival properties of experienced objects, a question regarding how subject is represented in visual perspectival experiences attracted less attention. In order to address this problem, I consider four popular phenomenal intuitions regarding perspectival experiences and argue that the major theories of perspectival (...) experiences do not account for all of them. Relying on these observations, I show how a theory which accommodates all these intuitions can be developed by (a) recognising that visual perspectival experiences are, in fact, multimodal visuo-bodily experiences and (b) distinguishing between egocentric and structural contents. (shrink)

This report highlights and explores five questions which arose from The Unity of Consciousness and Sensory Integration conference at Brown University in November of 2011: 1. What is the relationship between the unity of consciousness and sensory integration? 2. Are some of the basic units of consciousness multimodal? 3. How should we model the unity of consciousness? 4. Is the mechanism of sensory integration spatio-temporal? 5. How Should We Study Experience, Given Unity Relations?

It is often assumed that neural representation, with content that is in principle detachable from the flow of natural-factive information, is necessary to perceptually experience an object. In this paper we present and discuss two cases challenging this assumption. We take them to show that it is possible to experience an object with which you are interacting through your sensory systems without those systems constructing a representation of the object. The first example is viewing nearby medium-sized groups of objects. The (...) second is hearing objects through misbound sounds. These cases bring out two different ways object representation can fail while object experience persists, suggesting that object experience requires only that the object be revealed through sensory information, not full-blown representation. Constructing object representations is one way sensory systems reveal objects, but it is not the only way. Object representation in neural sensory systems is relatively demanding, rare, and fragile; object experience is relatively easy, pervasive, and robust. We conclude that even minimal forms of neural phenomenal internalism are false. (shrink)

The predictive coding (PC) theory of attention identifies attention with the optimization of the precision weighting of prediction error. Here we provide some challenges for this identification. On the one hand, the precision weighting of prediction error is too broad a phenomenon to be identified with attention because such weighting plays a central role in multimodal integration. Cases of crossmodal illusions such as the rubber hand illusion and the McGurk effect involve the differential precision weighting of prediction error, yet (...) attention does not shift as one would predict. On the other hand, the precision weighting of prediction error is too narrow a phenomenon to be identified with attention, because it cannot accommodate the full range of attentional phenomena. We review criticisms that PC cannot account for volitional attention and affect-biased attention, and we propose that it may not be able to account for feature-based and intellectual attention. (shrink)

This report highlights and explores five questions which arose from the workshop on perceptual learning and perceptual recognition at the University of Toronto, Mississauga on May 10th and 11th, 2012: 1. How should we demarcate perceptual learning from perceptual development? 2. What are the origins of multimodal associations? 3. Does our representation of time provide an amodal framework for multi-sensory integration? 4. What counts as cognitive penetration? 5. How can philosophers and psychologists most fruitfully collaborate?

This research article examines the potential of multimodal techniques in promoting inclusive practices in the English language classroom, delving into the multidisciplinary fields of English Language Teaching (ELT) and inclusive education. This study intends to investigate how multimodal resources, such as visual aids, technology, and creative activities, may be effectively integrated to meet the various learning needs of students using a pedagogical lens. This paper examines the theoretical underpinnings and practical implications of using multimodal approaches in ELT (...) settings, shedding light on their potential to promote inclusivity, engagement, and language acquisition among learners with diverse abilities and backgrounds. It does this through a thorough literature review and analysis of pedagogical frameworks. (shrink)

Traditional medical Artificial Intelligence models, approved for clinical use, restrict themselves to single-modal data e.g. images only, limiting their applicability in the complex, multimodal environment of medical diagnosis and treatment. Multimodal Transformer Models in healthcare can effectively process and interpret diverse data forms such as text, images, and structured data. They have demonstrated impressive performance on standard benchmarks like USLME question banks and continue to improve with scale. However, the adoption of these advanced AI models is not without (...) challenges. While multimodal deep learning models like Transformers offer promising advancements in healthcare, their integration requires careful consideration of the accompanying ethical and environmental challenges. (shrink)

This paper presents an analytical framework for analyzing how multimodal resources of emotion expression are semiotically materialized in discursive interactions specific to political discourse. Interested in how political personae are emotionally constructed through multimodal meaning-making practices, our analysis model assumes an interdisciplinary perspective, which integrates facial expression analysis – using FaceReader™ software –, the theory of emotional arcs and bodily actions (hand gestures) analysis that express emotions, in the analytical framework of multimodality. The results show how the (...) class='Hi'>multimodal choices that political actors make during discursive interactions allow them to build their political brand and make connections with the audience on an emotional level. (shrink)

From the perspective of distributed cognition I will stress how abduction is essentially multimodal, in that both data and hypotheses can have a full range of verbal and sensory representations, involving words, sights, images, smells, etc., but also kinesthetic – related to the ability to sense the position and location and orientation and movement of the body and its parts – and motor experiences and other feelings such as pain, and thus all sensory modalities. The presence of kinesthetic and (...) motor aspects demonstrates that abductive reasoning can be manipulative. We can also see, in this regard, how implicit factors take part in the abductive procedure, which consequently acquires the character of a kind of “thinking through doing”. This paper further describes 1) the fact that hypotheses in science can be built through different cognitive mediators and so they can also model the same cognitive aspect in different ways; how they can be carriers/ producers of knowledge in a multimodal way; 2) the problem of the possible non-explanatory and instrumental nature of abductive reasoning and the analysis of the consequences for induction; 3) the role of manipulative abduction in building new evidence/ experiments and how they trigger smart inductive inferences. (shrink)

This is an excerpt from a report that highlights and explores five questions which arose from the workshop on perceptual learning and perceptual recognition at the University of Toronto, Mississauga on May 10th and 11th, 2012. This excerpt explores the question: What are the origins of multimodal associations?

The analysis of multimodal argumentation in advertising is a crucial and problematic area of research. While its importance is growing in a time characterized by images and pictorial messages, the methods used for interpreting and reconstructing the structure of arguments expressed through verbal and visual means capture only isolated dimensions of this complex phenomenon. This paper intends to propose and illustrate a methodology for the reconstruction and analysis of “double-mode” arguments in advertisements, combining the instruments developed in social semiotics, (...) pragmatics, and argumentation theory. An advertisement is processed through a five-step path. The analysis of its context, text genre, and images leads to a first representation of the messages that it encodes both pictorially and verbally (step 1). These first semantic representations are further enriched by including their polyphonic articulations and presuppositions (step 2), their explicatures (step 3), and their dialogical functions and illocutionary forces (step 4). These pragmatic steps retrieve the commitment structure of the ad, which allows a further argument analysis conducted through argumentation schemes (step 5). (shrink)

It has been objected recently that naïve realism is inconsistent with an empirically well-supported hypothesis that unconscious perception is possible. Because epistemological disjunctivism is plausible only in conjunction with naïve realism (for a reason I provide), the objection reaches it too. In response, I show that the unconscious perception hypothesis can be changed from a problem into an advantage of epistemological disjunctivism. I do this by suggesting that: (i) naïve realism is consistent with the hypothesis; (ii) the contrast (...) between epistemological disjunctivism and epistemic externalism explains the difference in epistemic import between conscious and unconscious perception. (shrink)

ABSTRACT The difference between perception and cognition seems introspectively obvious in many cases. Perceiving and thinking have also been assigned quite different roles, in epistemology, in theories of reference and of mental content, in philosophy of psychology, and elsewhere. Yet what is the nature of the distinction? In what way, or ways, do perception and cognition differ? The paper reviews recent work on these questions. Four main respects in which perception and cognition have been held to differ (...) are discussed. First, their phenomenal character, such as the often-remarked vivacity or immediacy of perception. Second, the way in which they represent the world, e.g. the non-propositional nature of the contents, or non-discursive character of the vehicles, that have been held to characterise perceptual representation. Third, their place in cognitive architecture, i.e., roughly, in the information-flow of the mind, such as their alleged (non-)modularity. Fourth, their mind-world relations, e.g. the way in which perceptions seem to be tightly causally linked with distal or proximal stimuli. Against this background, we distinguish some main options for an account of the perception/cognition distinction, in particular concerning whether there is one, several, or no interesting and principled distinction(s) to be drawn here. (shrink)

Recently, it has been objected that naïve realism is inconsistent with an empirically well-supported claim that mental states of the same fundamental kind as ordinary conscious seeing can occur unconsciously (SFK). The main aim of this paper is to establish the following conditional claim: if SFK turns out to be true, the naïve realist can and should accommodate it into her theory. Regarding the antecedent of this conditional, I suggest that empirical evidence renders SFK plausible but not obvious. For it (...) is possible that what is currently advocated as unconscious perception of the stimulus is in fact momentaneous perceptual awareness (or residual perceptual awareness) of the stimulus making the subject prone to judge in some way rather than another, or to act in some way rather than another. As to the apodosis, I show that neither the core of naïve realism nor any of its main motivations is undermined if SFK is assumed. On the contrary, certain incentives for endorsing naïve realism become more tempting on this assumption. Since the main motivations for naïve realism retain force under SFK, intentionalism is neither compulsory nor the best available explanation of unconscious perception. (shrink)

One necessary condition on any adequate account of perception is clarity regarding whether unconscious perception exists. The issue is complicated, and the debate is growing in both philosophy and science. In this paper we consider the case for unconscious perception, offering three primary achievements. First, we offer a discussion of the underspecified notion of central coordinating agency, a notion that is critical for arguments that purportedly perceptual states are not attributable to the individual, and thus not genuinely (...) perceptual. We develop an explication of what it is for a representational state to be available to central coordinating agency for guidance of behavior. Second, drawing on this explication, we place a more careful understanding of the attributability of a state to the individual in the context of a range of empirical work on vision-for-action, saccades, and skilled typing. The results place pressure on the skeptic about unconscious perception. Third, reflecting upon broader philosophical themes running through debates about unconscious perception, we highlight how our discussion places pressure on the view that perception is a manifest kind, rather than a natural kind. In doing so, we resist the tempting complaint that the debate about unconscious perception is merely verbal. (shrink)

It seems plausible that visual experiences of darkness have perceptual phenomenal content that clearly differentiates them from absences of visual experiences. I argue, relying on psychological results concerning auditory attention, that the analogous claim is true about auditory experiences of silence. More specifically, I propose that experiences of silence present empty spatial directions like ‘right’ or ‘left’, and so have egocentric spatial content. Furthermore, I claim that such content is genuinely auditory and phenomenal in the sense that one can, in (...) principle, recognize that one is experiencing silence. This position is far from obvious, as the majority of theories concerning silence perception do not ascribe perceptual phenomenal content to experiences of silence. (shrink)

The chapter outlines an abstract theoretical framework that is currently (re-)emerging in the course of a theoretical convergence of several disciplines. In the first section, the fundamental problem of perception theory is formulated, namely, the generation, by the perceptual system, of meaningful categories from physicogeometric energy patterns. In the second section, it deals with basic intuitions and assumptions underlying what can be regarded as the current Standard Model of Perceptual Psychology and points out why this model is profoundly inadequate (...) for dealing with the fundamental problem of perception theory. In the third section, it discusses a level of analysis that promises to be fruitful for dealing, in conformity with established procedures of the natural sciences, with the problem of perceptual “meaning” and the problem of what constitutes a “perceptual object.” In the fourth section, it outlines a theoretical perspective on basic principles of the perceptual system which centers on the notions of complex data types and conceptual forms, and draws an entirely different theoretical picture of the role of the sensory input than traditional accounts. The final section focusses on the issue of material qualities and discusses, within the general theoretical framework outlined, some observations and results on the perception of certain material properties, namely, lustrous and glassy appearances. (shrink)

When we see an object, we also represent those parts of it that are not visible. The question is how we represent them: this is the problem of amodal perception. I will consider three possible accounts: (a) we see them, (b) we have non-perceptual beliefs about them and (c) we have immediate perceptual access to them, and point out that all of these views face both empirical and conceptual objections. I suggest and defend a fourth account, according to which (...) we represent the occluded parts of perceived objects by means of mental imagery. This conclusion could be thought of as a (weak) version of the Strawsonian dictum, according to which “imagination is a necessary ingredient of perception itself”. (shrink)

The possibilities of unconscious perception and unconscious bias prompt parallel debates about unconscious mental content. This chapter argues that claims within these debates alleging the existence of unconscious content are made fraught by ambiguity and confusion with respect to the two central concepts they involve: consciousness and content. Borrowing conceptual resources from the debate about unconscious perception, the chapter distills the two conceptual puzzles concerning each of these notions and establishes philosophical strategies for their resolution. It then argues (...) that empirical evidence for unconscious bias falls victim to these same puzzles, but that progress can be made by adopting similar philosophical strategies. Throughout, the chapter highlights paths forward in both debates, illustrates how they serve as fruitful domains in which to study the relationship between philosophy and empirical science, and uses their combined study to further understanding of a general theory of unconscious content. (shrink)

This paper argues that a common form of representationalism has trouble accommodating empirical findings about visual space perception. Vision science tells us that the visual system systematically gives rise to different experiences of the same spatial property. This, combined with a naturalistic account of content, suggests that the same spatial property can have different veridical looks. I use this to argue that a common form of representationalism about spatial experience must be rejected. I conclude by considering alternatives to this (...) view. (shrink)

In this Introduction we introduce the central themes of the Evaluative Perception volume. After identifying historical and recent contemporary work on this topic, we discuss some central questions under three headings: (1) Questions about the Existence and Nature of Evaluative Perception: Are there perceptual experiences of values? If so, what is their nature? Are experiences of values sui generis? Are values necessary for certain kinds of experience? (2) Questions about the Epistemology of Evaluative Perception: Can evaluative experiences (...) ever justify evaluative judgments? Are experiences of values necessary for certain kinds of justified evaluative judgments? (3) Questions about Value Theory and Evaluative Perception: Is the existence of evaluative experience supported or undermined by particular views in value theory? Are particular views in value theory supported or undermined by the existence of value experience? (shrink)