What is the correct procedure for determining the contents of perception? Philosophers tackling this question increasingly rely on empirically-oriented procedures in order to reach an answer. I argue that this constitutes an improvement over the armchair methodology constitutive of phenomenal contrast cases, but that there is a crucial respect in which current empirical procedures remain limited: they are unimodal in nature, wrongly treating the senses as isolatable faculties. I thus have two aims: first, to motivate a reorientation of the (...) admissible contents debate into a multimodal framework, charting its various significances. The second is to explore whether any experimental studies of multimodal perception support a so-called Liberal (or ‘high-level’ or ‘rich’) account of perception’s admissible contents. I conclude that the McGurk effect and the ventriloquist effect are both explicable without the postulation of high-level content, but that at least one multimodal experimental paradigm may necessitate such content: the rubber hand illusion. One upshot of this argument is that Conservatives who claim that the Liberal view intolerably broadens the scope of perceptual illusions, particularly from the perspective of perceptual psychology, should pursue other arguments against that view. (shrink)

Higher animals need to identify and track material objects because they depend on interactions with them for nutrition, reproduction, and social interaction. This paper investigates the perception of material objects. It argues, first, that material objects are tagged, in all five external senses, as bearers of the features detected by them. This happens through a perceptual process, here entitled Generalized Completion, which creates the appearance of objects that have properties that transcend the activation of sensory receptors. The paper shows, (...) secondly, that material objects are privileged subjects for perceived motion and interaction. That is, they are perceived as subjects for these properties while their parts seem to be subjects only derivatively. Material objects are the only perceptual subjects that are both multisensory and privileged. (shrink)

Everyday media consumption leaves no doubt that the perception of moving images from various media is characterized by experience and understanding. Corresponding research in this field has shown that the stimulus patterns flooding in on us are not only processed mentally, but also bodily. Building on this, the following study argues that incoming stimuli are processed not only visually, but multimodally, with all senses, and moreover affectively. The classical binding of a sensory organ to a medium, on whose delimitation (...) the common understanding of multimodality is based, is thus abolished. Can arguments for this assumption be found? Building on the research of cultural anthropologist Ernst Cassirer and his successors, the two American philosophers Susanne K. Langer and John M. Krois, and supported by research from the life sciences in which, in parallel, the arts also play an important role in understanding image formation, as in the work of Heinz Werner and, following on from this, in the more recent research of the child psychologist Daniel N. Stern in collaboration with mirror neuron research by the group of neuroscientists around Giacomo Rizzolatti in Parma, justifications for this will be provided. What the multimodal processing of film images is concretely based on will finally be made clear by means of an exemplary analysis of a video film by the Swiss artist Pipilotti Rist. It aims to prove, that the prerequisites for the thesis can only lie in the fact, that both the modes of perception (visual, tactile, auditory, olfactory, gustatory) as well as the medially conditioned modes of image formation are based on non-discursive, formal-abstract structures that are also processed affectively. At the same time, the analysis thus shows how the selection and composition of the vital experienced forms can provide information about the socio-culturally shaped self-image of the producer and the viewer. (shrink)

The aim of this paper is to argue that our experience of artworks is normally multimodal. It is the result of perceptual processing in more than one sense modality. In other words, multimodal experience of art is not the exception; it is the rule. I use the example of music in order to demonstrate the various ways in which the visual sense modality influences the auditory processing of music and conclude that this should make us look more closely (...) at our practices of engaging with artworks. (shrink)

With a Call for Essays, the special issue Multimodality sought contributions that accept not only the material but also the body-bound dependence of media perception and understanding. To this end, contributions were included that shed light on both the structural and signifying potential of artistic works through multimodal analysis. Particular attention was paid to contributions that clarify how the structural features - the modes - of the arts, their perception, and their signifying potential in terms of content (...) are interrelated and how they are to be understood in communicative and thus socioculturally relevant terms. Thus, in addition to neuroscientific contributions, those from cultural anthropology, art history, image and art studies, and literary studies were included. -/- Articles -/- 8 Martina Sauer and Christiane Wagner - Introduction -/- 11 Giada Lombardi and Giuseppe Di Cesare - From Neuroscience to Art: The Role of ‘Vitality Forms’ in the Investigation of Multimodality -/- 25 Martina Sauer - Abstract - Affective – Multimodal: Interaction between Medium and Perception of Moving Images from the Viewpoint of Cassirer’s, Langer’s and Krois’ Embodiment Theories -/- 49 Göran Sonesson - The Secrets of Plastic Language Revealed: Multimodality, Polysemiosis, and Iconicity -/- 71 Natalia Igl - Making Sense of the Material: Multisensory Reader Involvement in Contemporary Multimodal Novels -/- 87 J. Philippe Thompson - Art is All: Proust’s Life Lessons Through Perception and Art -/- 107 Hans Dieter Huber - Philippe Parreno and the Exhibition as a Multimodal Aesthetic Experience -/- 135 Jörg U. Lensing - The Performing Human Being in a Media Interaction Space: Multi- and intermodal productions by the Düsseldorf Theater der Klänge (Theatre of Sounds) . (shrink)

This is an excerpt of a report that highlights and explores five questions which arose from The Unity of Consciousness and Sensory Integration conference at Brown University in November of 2011. This portion of the report explores the question: Are some of the basic units of consciousness multimodal?

Mental imagery is early perceptual processing that is not triggered by corresponding sensory stimulation in the relevant sense modality. Multimodal mental imagery is early perceptual processing that is triggered by sensory stimulation in a different sense modality. For example, when early visual or tactile processing is triggered by auditory sensory stimulation, this amounts to multimodal mental imagery. Pulling together philosophy, psychology and neuroscience, I will argue in this paper that multimodal mental imagery plays a crucial role in (...) our engagement with musical works. Engagement with musical works is normally a multimodal phenomenon, where we get input from a number of sense modalities. But even if we screen out any input that is not auditory, multimodal mental imagery will still play an important role that musicians and composers often actively rely on. (shrink)

An introduction to contemporary debates in philosophy of perception.

This is the editors' Introduction to a special issue of the journal, Multisensory Research. European philosophers of the modern period found multisensory perception to be impossible because they thought that perceptual ideas are defined by how they are experienced. Under this conception, the individual modalities are determinables of ideas—just as colour is a determinable that embraces red and blue, so also the visual is a determinable that embraces colour and (visually experienced) shape. Since no idea is experienced as, for (...) example, both visual and auditory, there can be no such thing as audiovisual perception. This conception of modality is not directly contested, but a variety of perceptual phenomena are listed that could raise interesting questions if treated as multimodal in origin. (shrink)

Suppose that you are at a live jazz show. The drummer begins a solo. You see the cymbal jolt and you hear the clang. But in addition seeing the cymbal jolt and hearing the clang, you are also aware that the jolt and the clang are part of the same event. Casey O’Callaghan (forthcoming) calls this awareness “intermodal feature binding awareness.” Psychologists have long assumed that multimodal perceptions such as this one are the result of a subpersonal feature binding (...) mechanism (see Vatakis and Spence, 2007, Kubovy and Schutz, 2010, Pourtois et al., 2000, and Navarra et al., 2012). I present new evidence against this. I argue that there is no automatic feature binding mechanism that couples features like the jolt and the clang together. Instead, when you experience the jolt and the clang as part of the same event, this is the result of an associative learning process. The cymbal’s jolt and the clang are best understood as a single learned perceptual unit, rather than as automatically bound. I outline the specific learning process in perception called “unitization,” whereby we come to “chunk” the world into multimodal units. Unitization has never before been applied to multimodal cases. Yet I argue that this learning process can do the same work that intermodal binding would do, and that this issue has important philosophical implications. Specifically, whether we take multimodal cases to involve a binding mechanism or an associative process will have impact on philosophical issues from Molyneux’s question to the question of how active or passive we consider perception to be. (shrink)

Is it true that there is an analogy between modes of creation and such of perception? Respective to the cultural anthropological research of Ernst Cassirer, Susanne K. Langer and John M. Krois and by the analysis of a tape of the Swiss video-artist Pipilotti Rist this initial thesis of Formal Aesthetics shall be supported. - I - -/- Lässt sich die Behauptung stützen, dass zwischen Gestaltungsweisen und Wahrnehmungsweisen eine Analogie besteht? Aufbauend auf den kulturanthropologischen Forschungen von Ernst Cassirer, Susanne (...) K. Langer und John M. Krois sowie der Analyse eines Tapes der Schweizer Videokünstlerin Pipilotti Rist gilt es diese ursprünglich von der formalen Ästhetik aufgeworfene These zu untermauern. (shrink)

The basis of perception is the processing and categorization of perceptual stimuli from the environment. Much progress has been made in the science of perceptual categorization. Yet there is still no consensus on how the brain generates sensory individuals, from sensory input and perceptual categories in memory. This chapter argues that perceptual categorization is highly variable across perceivers due to their use of different perceptual strategies for solving perceptual problems they encounter, and that the perceptual system structurally adjusts to (...) the strategies that are most successful. Despite this variability, the different sensory individuals that result when different perceivers rely on different strategies nonetheless correspond to actual external world objects/features. This view is called 'perceptual pluralism'. (shrink)

Within contemporary philosophy of perception, it is commonly claimed that flavour experiences are paradigmatic examples of multimodal perceptual experiences. In fact, virtually any sensory system, including vision and audition, is believed to influence how we experience flavours. However, there is a strong intuition, often expressed in these works, that not all of these sensory systems make an equal contribution to the phenomenology of flavour experiences. More specifically, it seems that the activities of some sensory systems are constitutive for (...) flavour perception while others merely influence how we experience flavours. This paper aims to answer the question regarding the constitutive factors of flavour perception in a twofold way. First, a theoretical framework is developed, relying on debates regarding constitutivity in analytic metaphysics and philosophy of science, which defines the stronger and weaker senses in which the activities of sensory systems may be constitutive for flavour perception. Second, relying on empirical results in flavour science, the constitutive status of activities related to distinct sensory systems in the context of flavour perception is investigated. (shrink)

This chapter examines how our sense modalities interact in the perception of persistence. The chapter concentrates on two questions. The first concerns perceptual processing—do perceptual computations of object persistence ever integrate and compute over representations from more than one modality? It argues that this question should be answered affirmatively. The second question concerns perceptual experience—do experiences of object persistence ever exhibit a constitutively multisensory phenomenal character, or is the phenomenology of object persistence always uniquely associated with just one modality? (...) The chapter argues that the available evidence underdetermines the answer to this question, but suggests ways it might be empirically resolved. (shrink)

In this paper I develop a novel account of the phenomenality of language by focusing on characteristics of perceived speech. I explore the extent to which the spoken word can be said to have a horizonal structure similar to that of spatiotemporal objects: our perception of each is informed by habitual associations and expectations formed through past experiences of the object or word and other associated objects and experiences. Specifically, the horizonal structure of speech in use can fruitfully be (...) compared to that of a tool in use. The comparison suggests an account of our linguistic faculty as continuous with more foundational faculties of perception and action. I provide empirical corroboration of this account by drawing on recent neuroimaging studies of the multimodal, sensorimotor bases of speech comprehension. I then discuss how such an understanding of our linguistic ability helps advocates of embodied, non-representationalist accounts of cognition respond to a common objection. Critics grant that embodied approaches may be adequate to account for lower-level, online modes of cognition, such as perception and action, which directly engage their object. But they question whether such approaches can “scale up” to higher modes of cognition, such as imagination, memory, thought, and language, which can entertain absent, non-existent, or abstract objects. By providing a plausible account of the continuity of lower cognition and language-involving cognition, my approach responds to this objection, at least where language is concerned. (shrink)

The aim of this paper is to raise some serious worries about anti-representationalism: the recently popular view according to which there are no perceptual representations. Although anti-representationalism is more and more popular, I will argue that we have strong empirical reasons for mistrusting it. More specifically, I will argue that it is inconsistent with some important empirical findings about dorsal perception and about the multimodality of perception.

Molyneux asked whether a newly sighted person could distinguish a sphere from a cube by sight alone, given that she was antecedently able to do so by touch. This, we contend, is a question about general ideas. To answer it, we must ask (a) whether spatial locations identified by touch can be identified also by sight, and (b) whether the integration of spatial locations into an idea of shape persists through changes of modality. Posed this way, Molyneux’s Question goes substantially (...) beyond question (a), about spatial locations, alone; for a positive answer to (a) leaves open whether a perceiver might cross-identify locations, but not be able to identify the shapes that collections of locations comprise. We further emphasize that MQ targets general ideas so as to distinguish it from corresponding questions about experiences of shape and about the property of tangible (vs. visual) shape. After proposing a generalized formulation of MQ, we extend earlier work (“Many Molyneux Questions,” Australasian Journal of Philosophy 2020) by showing that MQ does not admit a single answer across the board. Some integrative data-processes transfer across modalities; others do not. Seeing where and how such transfer succeeds and fails in individual cases has much to offer to our understanding of perception and its modalities. (shrink)

In the case of ventriloquism, seeing the movement of the ventriloquist dummy’s mouth changes your experience of the auditory location of the vocals. Some have argued that cases like ventriloquism provide evidence for the view that at least some of the content of perception is fundamentally multimodal. In the ventriloquism case, this would mean your experience has constitutively audio-visual content (not just a conjunction of an audio content and visual content). In this paper, I argue that cases like (...) ventriloquism do not in fact warrant that conclusion. I then try to make sense of crossmodal cases without appealing to fundamentally multimodal content. (shrink)

Philosophical discussions of Molyneux's problem within contemporary philosophy of mind tend to characterize the problem as primarily concerned with the role innately known principles, amodal spatial concepts, and rational cognitive faculties play in our perceptual lives. Indeed, for broadly similar reasons, rationalists have generally advocated an affirmative answer, while empiricists have generally advocated a negative one, to the question Molyneux posed after presenting his famous thought experiment. This historical characterization of the dialectic, however, somewhat obscures the role Molyneux's problem has (...) played in spawning debates within the empiricist tradition. Fortunately, the differences between various empiricist accounts have been widely recognized and discussed among historians of philosophy working on the topic. The focus of the present essay is to develop an interpretation of John Locke's views on Molyneux's problem that best coheres with his other views on human understanding as well as with the predominant scientific opinion about the nature of perception during the period in which he lived. (shrink)

This is an excerpt from a report on the Temporal Experience Workshop at the University of Toronto in May of 2013. This portion of the report explores the question: Do we have one central clock for time, or different clocks for each sense modality?

Schwenkler (2012) criticizes a 2011 experiment by R. Held and colleagues purporting to answer Molyneux’s question. Schwenkler proposes two ways to re-run the original experiment: either by allowing subjects to move around the stimuli, or by simplifying the stimuli to planar objects rather than three-dimensional ones. In Schwenkler (2013) he expands on and defends the former. I argue that this way of re-running the experiment is flawed, since it relies on a questionable assumption that newly sighted subjects will be able (...) to appreciate depth cues. I then argue that the second way of re-running the experiment is successful both in avoiding the flaw of original Held experiment, and in avoiding the problem with the first way of re-running the experiment. (shrink)

I can be wrong about things I seem to perceive; the conditions might lead me to be mistaken about them. Since I can't rule out the possibility that the conditions are misleading, I can't be sure that I am perceiving this thing in my hand correctly. But suppose that I am able to examine it actively—handling it, looking closer, shining a light on it, and so on. Then, my level of uncertainty goes down; in the limit it is eliminated entirely. (...) Of course, I might be mistaken because I am a brain in a vat, or because some other sceptical scenario obtains. But sceptical scenarios apply not just to this thing in my hand, but to everything all at once. Define "empirical certainty" as the absence of the kind of doubt that spreads to all empirical states of affairs. Sensory exploration, or active examination, can in principle lead to empirical certainty. (shrink)

This report highlights and explores five questions that arose from the multisensory integration workshop at the University of Toronto on May 9th and 10th, 2014: 1. What Is Multisensory Integration? 2. Do Multisensory Percepts Involve Emergent Features? 3. What Can Multisensory Processing Tell Us about Multisensory Awareness? 4. Is Language Processing a Special Kind of Multisensory Integration? 5. What Is the Purpose of Multisensory Integration?

This is an excerpt from a report on the workshop on multisensory integration at the University of Toronto, on May 9th and 10th, 2014, written by Kevin Connolly, Aaron Henry, Zoe Jenkin, and Andrew MacGregor, and available at: http://networksensoryresearch.utoronto.ca/Events_%26_Discussion.html This excerpt explores the question: Do multisensory percepts involve emergent features?

Jesse Prinz (2004) argues that emotional feelings (“state emotions”) can by themselves perceptually represent significant organism-environment relations. I object to this view mainly on the grounds that (1) it does not rule out the at least equally plausible view that emotional feelings are non-representational sensory registrations rather than perceptions, as Tyler Burge (2010) draws the distinction, and (2) perception of a relation requires perception of at least one of the relation’s relata, but an emotional feeling by itself perceives (...) neither the subject’s environment, nor in many cases the relevant subject itself. I then explore two ways in which emotional feelings as non-perceptual sensory registrations might still contribute to significant relation representation when associated with representations of the subject and/or its environment. After briefly discussing some difficulties presented by a multimodal, sensory-perceptual view of such representation, I argue in favor of a “cognitive recognition theory” that holds that significant relation instances are represented during emotion occurrences via applications of emotion-type concepts to "incoming" emotional feelings and their associated mental states. (shrink)

In our everyday interactions we easily and effortlessly perceive emotions in others’ facial expressions and bodily behavior. How do we do that? Philosophers and psychologists have long argued about the fundamentals of emotion perception and the debate is far from settled. While some insist on the sufficiency of the morphological information contained in facial expressions, others construe the objects of emotion perception as more complex, comprising multimodal information such as touch, tone of voice, body postures, and so (...) on. Others, in turn, have gone so far as to deny that there is a pre-given object in emotion perception in the first place, so that processes of emotion perception must be conceptualized as emergent and co-constructed phenomena in contexts of social interaction. The purpose of this paper is to argue that John Dewey’s pragmatist theory of emotions can help us make significant theoretical advancements in the understanding of emotion perception, according to which emotions are coordinated modes of behavior that we enact with others in the contexts of social interactions. In this picture, perceiving emotions is a matter of participating in a joint activity of socio- affective coordination. This theory allows us to apprehend important insights about emotion perception that appear in these contemporary debates, while at the same time avoiding their pitfalls. (shrink)

This chapter offers an indirect defence of the Evansian conception of egocentric space, by showing how it resolves a puzzle concerning the unity of egocentric spatial perception. The chapter outlines several common assumptions about egocentric perspectival structure and argues that a subject’s experience, both within and across her sensory modalities, may involve multiple structures of this kind. This raises the question of how perspectival unity is achieved, such that these perspectival structures form a complex whole, rather than merely disunified (...) set of individually, distinctively structured experiences. The shortcomings of variety of accounts are considered: switch accounts ; sensory accounts; transformation accounts; and ultimate accounts. These shortcomings are addressed by a further kind of account provided by the Evansian conception – an agentive account – according to which egocentrically structured experiences present the world in relation to parts of a single thing, the body as a dynamic unity. (shrink)

The usual visual experiences possess a perspectival phenomenology as they seem to present objects from a certain perspective. Nevertheless, it is not obvious how to characterise experiential content determining such phenomenology. In particular, while there are many works investigating perspectival properties of experienced objects, a question regarding how subject is represented in visual perspectival experiences attracted less attention. In order to address this problem, I consider four popular phenomenal intuitions regarding perspectival experiences and argue that the major theories of perspectival (...) experiences do not account for all of them. Relying on these observations, I show how a theory which accommodates all these intuitions can be developed by (a) recognising that visual perspectival experiences are, in fact, multimodal visuo-bodily experiences and (b) distinguishing between egocentric and structural contents. (shrink)

There is converging evidence that high doses of hallucinogenic drugs can produce significant alterations of self-experience, described as the dissolution of the sense of self and the loss of boundaries between self and world. This article discusses the relevance of this phenomenon, known as “drug-induced ego dissolution (DIED)”, for cognitive neuroscience, psychology and philosophy of mind. Data from self-report questionnaires suggest that three neuropharmacological classes of drugs can induce ego dissolution: classical psychedelics, dissociative anesthetics and agonists of the kappa opioid (...) receptor (KOR). While these substances act on different neurotransmitter receptors, they all produce strong subjective effects that can be compared to the symptoms of acute psychosis, including ego dissolution. It has been suggested that neuroimaging of DIED can indirectly shed light on the neural correlates of the self. While this line of inquiry is promising, its results must be interpreted with caution. First, neural correlates of ego dissolution might reveal the necessary neurophysiological conditions for the maintenance of the sense of self, but it is more doubtful that this method can reveal its minimally sufficient conditions. Second, it is necessary to define the relevant notion of self at play in the phenomenon of DIED. This article suggests that DIED consists in the disruption of subpersonal processes underlying the “minimal” or “embodied” self, i.e., the basic experience of being a self rooted in multimodal integration of self-related stimuli. This hypothesis is consistent with Bayesian models of phenomenal selfhood, according to which the subjective structure of conscious experience ultimately results from the optimization of predictions in perception and action. Finally, it is argued that DIED is also of particular interest for philosophy of mind. On the one hand, it challenges theories according to which consciousness always involves self-awareness. On the other hand, it suggests that ordinary conscious experience might involve a minimal kind of self-awareness rooted in multisensory processing, which is what appears to fade away during DIED. (shrink)

Long-standing neglect of the chemical senses in the philosophy of perception is due, mostly, to their being regarded as ‘lower’ senses. Smell, taste, and chemically irritated touch are thought to produce mere bodily sensations. However, empirically informed theories of perception can show how these senses lead to perception of objective properties, and why they cannot be treated as special cases of perception modelled on vision. The senses of taste, touch, and smell also combine to create unified (...) perceptions of flavour. The nature of these multimodal experiences and the character of our awareness of them puts pressure on the traditional idea that each episode of perception goes one or other of the five senses. Thus, the chemical senses, far from being peripheral to the concerns of the philosophy of perception, may hold important clues to the multisensory nature of perception in general. (shrink)

The predictive coding (PC) theory of attention identifies attention with the optimization of the precision weighting of prediction error. Here we provide some challenges for this identification. On the one hand, the precision weighting of prediction error is too broad a phenomenon to be identified with attention because such weighting plays a central role in multimodal integration. Cases of crossmodal illusions such as the rubber hand illusion and the McGurk effect involve the differential precision weighting of prediction error, yet (...) attention does not shift as one would predict. On the other hand, the precision weighting of prediction error is too narrow a phenomenon to be identified with attention, because it cannot accommodate the full range of attentional phenomena. We review criticisms that PC cannot account for volitional attention and affect-biased attention, and we propose that it may not be able to account for feature-based and intellectual attention. (shrink)

This is an excerpt from a report on the workshop on multisensory integration at the University of Toronto, on May 9th and 10th, 2014, written by Kevin Connolly, Aaron Henry, Zoe Jenkin, and Andrew MacGregor, and available at: http://networksensoryresearch.utoronto.ca/Events_%26_Discussion.html This excerpt explores the question: What is multisensory integration?

This is an excerpt from a report on the workshop on multisensory integration at the University of Toronto, on May 9th and 10th, 2014, written by Kevin Connolly, Aaron Henry, Zoe Jenkin, and Andrew MacGregor, and available at: http://networksensoryresearch.utoronto.ca/Events_%26_Discussion.html This excerpt explores the question: What can multisensory processing tell us about multisensory awareness?

In crossmodal identification, a subject token identifies an item perceived in one sensory modality with an item perceived in another sensory modality. Does crossmodal identification always occur in cognition, or does crossmodal identification sometimes take place in perception? This paper argues that crossmodal identification occurs in cognition, and not in perception. Nevertheless, multisensory perception is not unalive to crossmodal identity. Experimental evidence demonstrates that perception is differentially sensitive to the identity of individuals presented to distinct senses. (...) Such sensitivity enhances recognition and improves action. This approach relies on distinguishing crossmodal identification from perceiving crossmodal identity. Perception registers crossmodal identity, but crossmodal identification as such belongs to thought. (shrink)

This report highlights and explores five questions which arose from The Unity of Consciousness and Sensory Integration conference at Brown University in November of 2011: 1. What is the relationship between the unity of consciousness and sensory integration? 2. Are some of the basic units of consciousness multimodal? 3. How should we model the unity of consciousness? 4. Is the mechanism of sensory integration spatio-temporal? 5. How Should We Study Experience, Given Unity Relations?

This report highlights and explores five questions which arose from the workshop on perceptual learning and perceptual recognition at the University of Toronto, Mississauga on May 10th and 11th, 2012: 1. How should we demarcate perceptual learning from perceptual development? 2. What are the origins of multimodal associations? 3. Does our representation of time provide an amodal framework for multi-sensory integration? 4. What counts as cognitive penetration? 5. How can philosophers and psychologists most fruitfully collaborate?

The capacity for feature binding is typically explained in terms of the attribution model: a perceptual state selects an individual and attributes properties to it (Kahneman & Treisman 1984; Clark 2004; Burge 2010). Thus features are bound together in virtue of being attributed to the same individual. While the attribution model successfully explains some cases of binding in perception, not all binding need be understood as property attribution. This chapter argues that some forms of binding—those involving holistic iconic representations, (...) which bind features outside the limits of attention and object files—don’t fit the attribution model. The chapter then sketches an alternative coordination model of binding that construes icons as complex analog representations. (shrink)

The analysis of multimodal argumentation in advertising is a crucial and problematic area of research. While its importance is growing in a time characterized by images and pictorial messages, the methods used for interpreting and reconstructing the structure of arguments expressed through verbal and visual means capture only isolated dimensions of this complex phenomenon. This paper intends to propose and illustrate a methodology for the reconstruction and analysis of “double-mode” arguments in advertisements, combining the instruments developed in social semiotics, (...) pragmatics, and argumentation theory. An advertisement is processed through a five-step path. The analysis of its context, text genre, and images leads to a first representation of the messages that it encodes both pictorially and verbally (step 1). These first semantic representations are further enriched by including their polyphonic articulations and presuppositions (step 2), their explicatures (step 3), and their dialogical functions and illocutionary forces (step 4). These pragmatic steps retrieve the commitment structure of the ad, which allows a further argument analysis conducted through argumentation schemes (step 5). (shrink)

This research article examines the potential of multimodal techniques in promoting inclusive practices in the English language classroom, delving into the multidisciplinary fields of English Language Teaching (ELT) and inclusive education. This study intends to investigate how multimodal resources, such as visual aids, technology, and creative activities, may be effectively integrated to meet the various learning needs of students using a pedagogical lens. This paper examines the theoretical underpinnings and practical implications of using multimodal approaches in ELT (...) settings, shedding light on their potential to promote inclusivity, engagement, and language acquisition among learners with diverse abilities and backgrounds. It does this through a thorough literature review and analysis of pedagogical frameworks. (shrink)

It is common to characterize pain with touch-related terms, like ‘cutting’, ‘pressing’, ‘sharp’, and ‘pulsing’, or temperature-related terms, like ‘hot’ or ‘burning’. This suggests that many pains are phenomenally multimodal because they are experienced as having some tactile-like or thermal-like character. The goal of this chapter is to investigate the structure of phenomenally multimodal pain experiences. It is argued that the usual accounts of multimodal structure proposed in investigations regarding exteroceptive experiences cannot be plausibly applied to (...) class='Hi'>multimodal experiences of pain. Instead, an alternative framework is proposed which characterizes the structure of tactile-like and thermal-like pains by referring to the notion of crossmodal correspondences. (shrink)

According to a traditional view, there is no categorical difference between the phenomenology of perception and the phenomenology of imagination; the only difference is in degree (of intensity, resolution, etc.) and/or in accompanying beliefs. There is no categorical difference between what it is like to perceive a dog and what it is like to imagine a dog; the former is simply more vivid and/or is accompanied by the belief that a dog is really there. A sustained argument against this (...) traditional view is prosecuted by Sartre, who develops an alternative according to which there is a categorical difference between the two phenomenologies. This paper draws on Sartre’s work in this area to develop a similar account of perception and imagination as categorically different. (shrink)

This paper presents an analytical framework for analyzing how multimodal resources of emotion expression are semiotically materialized in discursive interactions specific to political discourse. Interested in how political personae are emotionally constructed through multimodal meaning-making practices, our analysis model assumes an interdisciplinary perspective, which integrates facial expression analysis – using FaceReader™ software –, the theory of emotional arcs and bodily actions (hand gestures) analysis that express emotions, in the analytical framework of multimodality. The results show how the (...) class='Hi'>multimodal choices that political actors make during discursive interactions allow them to build their political brand and make connections with the audience on an emotional level. (shrink)

When we perceive an object, we perceive the object from a perspective. As a consequence of the perspectival nature of perception, when we perceive, say, a circular coin from different angles, there is a respect in which the coin looks circular throughout, but also a respect in which the coin's appearance changes. More generally, perception of shape and size properties has both a constant aspect—an aspect that remains stable across changes in perspective—and a perspectival aspect—an aspect that changes (...) depending on one's perspective on the object. How should we account for the perspectival aspect of spatial perception? We present a framework within which to discuss the perspectival aspect of perception and put forward three desiderata that any account of the perspectival aspect of perception should satisfy. We discuss views on which the perspectival aspect of perception is analyzed in terms of constitutively mind-dependent appearance properties as well as views on which the perspectival aspect of perception is analyzed in terms of representations of mind-independent perspectival properties. (shrink)

The possibilities of unconscious perception and unconscious bias prompt parallel debates about unconscious mental content. This chapter argues that claims within these debates alleging the existence of unconscious content are made fraught by ambiguity and confusion with respect to the two central concepts they involve: consciousness and content. Borrowing conceptual resources from the debate about unconscious perception, the chapter distills the two conceptual puzzles concerning each of these notions and establishes philosophical strategies for their resolution. It then argues (...) that empirical evidence for unconscious bias falls victim to these same puzzles, but that progress can be made by adopting similar philosophical strategies. Throughout, the chapter highlights paths forward in both debates, illustrates how they serve as fruitful domains in which to study the relationship between philosophy and empirical science, and uses their combined study to further understanding of a general theory of unconscious content. (shrink)

The chapter outlines an abstract theoretical framework that is currently (re-)emerging in the course of a theoretical convergence of several disciplines. In the first section, the fundamental problem of perception theory is formulated, namely, the generation, by the perceptual system, of meaningful categories from physicogeometric energy patterns. In the second section, it deals with basic intuitions and assumptions underlying what can be regarded as the current Standard Model of Perceptual Psychology and points out why this model is profoundly inadequate (...) for dealing with the fundamental problem of perception theory. In the third section, it discusses a level of analysis that promises to be fruitful for dealing, in conformity with established procedures of the natural sciences, with the problem of perceptual “meaning” and the problem of what constitutes a “perceptual object.” In the fourth section, it outlines a theoretical perspective on basic principles of the perceptual system which centers on the notions of complex data types and conceptual forms, and draws an entirely different theoretical picture of the role of the sensory input than traditional accounts. The final section focusses on the issue of material qualities and discusses, within the general theoretical framework outlined, some observations and results on the perception of certain material properties, namely, lustrous and glassy appearances. (shrink)

What are the sensory individuals of audition? What are the entities our auditory system attributes properties to? We examine various proposals about the nature of the sensory individuals of audition, and show that while each can account for some aspects of auditory perception, each also faces certain difficulties. We then put forward a new conception of sensory individuals according to which auditory sensory individuals are composite individuals. A feature shared by all existing accounts of sounds and sources is that (...) they postulate sensory individuals that are non-composite. They identify the sensory individual of sound hearing or source hearing with one type of entity in the environment, be they sound waves, vibrations, or interactions. We question this assumption and argue that our perceptual systems represent two or more aspects of the environment as a single sensory individual. Finally, we show that taking auditory individuals to be composite sensory individuals allows for an account of audition that is less problematic than its existing alternatives. (shrink)

This paper argues for deific perception, the idea that some perceptual experiences represent deific properties, as an explanation for a certain type of religious experience. Using Siegel's method of phenomenal contrast, a pair of experiences are compared: an ordinary perception of a black rose, and one where the rose seems imbued with religious significance. Intuitively, these have different phenomenologies. Deific perception posits that in the religiously-significant experience, a deific property like "being a creature of God" partly constitutes (...) the visual phenomenology. Rival explanations like differences in attention or cognitive penetration fail to account for the phenomenal contrast. Deific perception coheres with realism about religious properties and provides a naturalistic explanation of how some people directly sense the divine in nature. If valid, it suggests that with proper background beliefs and recognition capacities, perception can represent high-level deific properties, just as it may represent other abstract properties like natural kinds. The paper thus contributes to debates on the contents of perception and offers a sympathetic account of a class of numinous experiences. (shrink)

From the perspective of distributed cognition I will stress how abduction is essentially multimodal, in that both data and hypotheses can have a full range of verbal and sensory representations, involving words, sights, images, smells, etc., but also kinesthetic – related to the ability to sense the position and location and orientation and movement of the body and its parts – and motor experiences and other feelings such as pain, and thus all sensory modalities. The presence of kinesthetic and (...) motor aspects demonstrates that abductive reasoning can be manipulative. We can also see, in this regard, how implicit factors take part in the abductive procedure, which consequently acquires the character of a kind of “thinking through doing”. This paper further describes 1) the fact that hypotheses in science can be built through different cognitive mediators and so they can also model the same cognitive aspect in different ways; how they can be carriers/ producers of knowledge in a multimodal way; 2) the problem of the possible non-explanatory and instrumental nature of abductive reasoning and the analysis of the consequences for induction; 3) the role of manipulative abduction in building new evidence/ experiments and how they trigger smart inductive inferences. (shrink)

Emotions often misfire. We sometimes fear innocuous things, such as spiders or mice, and we do so even if we firmly believe that they are innocuous. This is true of all of us, and not only of phobics, who can be considered to suffer from extreme manifestations of a common tendency. We also feel too little or even sometimes no fear at all with respect to very fearsome things, and we do so even if we believe that they are fearsome. (...) Indeed, instead of shunning fearsome things, we might be attracted to them. Emotions that seem more thought-involving, such as shame, guilt or jealousy, can also misfire. You can be ashamed of your big ears even though we can agree that there is nothing shameful in having big ears, and even though you judge that having big ears does not warrant shame. And of course, it is also possible to experience too little or even no shame at all with respect to something that is really shameful. Many of these cases involve a conflict between one’s emotion and one’s evaluative judgement. Emotions that are thus conflicting with judgement can be called ‘recalcitrant emotions’. The question I am interested in is whether or not recalcitrant emotions amount to emotional illusions, that is, whether or not these cases are sufficiently similar to perceptual illusions to justify the claim that they fall under the same general heading. The answer to this depends on what emotions are. For instance, the view that emotions are evaluative judgments makes it difficult to make room for the claim that emotional errors are perceptual illusions. Fearing an innocuous spider would simply amount to making the error of judging that the spider is fearsome while it is in fact innocuous. This might involve an illusion of some sort, but it certainly does not amount to anything like a perceptual illusion. In this chapter, I argue that recalcitrant emotions are a kind of perceptual illusion.. (shrink)