Suppose that you are at a live jazz show. The drummer begins a solo. You see the cymbal jolt and you hear the clang. But in addition seeing the cymbal jolt and hearing the clang, you are also aware that the jolt and the clang are part of the same event. Casey O’Callaghan (forthcoming) calls this awareness “intermodal feature binding awareness.” Psychologists have long assumed that multimodal perceptions such as this one are the result of a subpersonal feature binding mechanism (...) (see Vatakis and Spence, 2007, Kubovy and Schutz, 2010, Pourtois et al., 2000, and Navarra et al., 2012). I present new evidence against this. I argue that there is no automatic feature binding mechanism that couples features like the jolt and the clang together. Instead, when you experience the jolt and the clang as part of the same event, this is the result of an associative learning process. The cymbal’s jolt and the clang are best understood as a single learned perceptual unit, rather than as automatically bound. I outline the specific learning process in perception called “unitization,” whereby we come to “chunk” the world into multimodal units. Unitization has never before been applied to multimodal cases. Yet I argue that this learning process can do the same work that intermodal binding would do, and that this issue has important philosophical implications. Specifically, whether we take multimodal cases to involve a binding mechanism or an associative process will have impact on philosophical issues from Molyneux’s question to the question of how active or passive we consider perception to be. (shrink)

This is the editors' Introduction to a special issue of the journal, Multisensory Research. European philosophers of the modern period found multisensory perception to be impossible because they thought that perceptual ideas are defined by how they are experienced. Under this conception, the individual modalities are determinables of ideas—just as colour is a determinable that embraces red and blue, so also the visual is a determinable that embraces colour and (visually experienced) shape. Since no idea is experienced (...) as, for example, both visual and auditory, there can be no such thing as audiovisual perception. This conception of modality is not directly contested, but a variety of perceptual phenomena are listed that could raise interesting questions if treated as multimodal in origin. (shrink)

This chapter examines how our sense modalities interact in the perception of persistence. The chapter concentrates on two questions. The first concerns perceptual processing—do perceptual computations of object persistence ever integrate and compute over representations from more than one modality? It argues that this question should be answered affirmatively. The second question concerns perceptual experience—do experiences of object persistence ever exhibit a constitutively multisensory phenomenal character, or is the phenomenology of object persistence always uniquely associated with just one (...) modality? The chapter argues that the available evidence underdetermines the answer to this question, but suggests ways it might be empirically resolved. (shrink)

The idea that there are five senses dates back to Aristotle, who was one of the first philosophers to examine them systematically. Though it has become conventional wisdom, many scientists and philosophers would argue that this idea is outdated and inaccurate. Indeed, they have given many different answers to this question, ranging from just three (the number of different kinds of physical energy we can detect) to 33 or more senses. Perhaps surprisingly, the issue remains controversial, partly because it is (...) not clear exactly what should count as a ‘sense’. Should proprioception, balance and kinaesthesis, amongst others, be considered additional senses? And if so, how do we figure out how many senses we really have? -/- (Published in Turkish under the title ‘Çok Duyumlu Algi’). (shrink)

The first part of this survey article presented a cartography of some of the more extensively studied forms of multisensory processing. In this second part, I turn to examining some of the different possible ways in which the structure of conscious perceptual experience might also be characterized as multisensory. In addition, I discuss the significance of research on multisensory processing and multisensory consciousness for philosophical debates concerning the modularity of perception, cognitive penetration, and the individuation (...) of the senses. (shrink)

Sense interaction is ubiquitous. All conscious experiences involve at least some interaction between the senses. One of the most debated questions in recent scholarship concerns the proper way of characterizing the phenomenology of multisensory experiences. According to Charles Spence and Tim Bayne (2015), the phenomenal character of multisensory integration is reducible to the co-conscious sum of modality-specific features. Following Casey O'Callaghan (2015), we can call this The Thesis of Minimal Multimodality. The main goal of the paper is to (...) refute the thesis and show that the effect of multisensory integration is reflected in experience in a way that is not exhausted by (the sum of) modality-specific phenomenal features. Whereas O'Callaghan's strategy in trying to prove this consists in providing phenomenological evidence that runs contrary to what Spence and Bayne hold, he does not, however, put into question what has become since T. Nagel's famous essay (1974) the standard conception of the phenomenal. By drawing conceptual resources in the work of Husserl and Merleau-Ponty, I will elaborate another approach and explain why the phenomenal character of experience cannot be reduced to its "what it's like" character. Phenomenal experience is thicker, and also includes various forms of bodily self-experiences and felt possibilities of action and behaviours. Building on these and like insights, the main objective of the paper is to outline a phenomenological account of multimodal perception and sensory interaction. I will draw three main conclusions. First, I will argue that both Husserl and Merleau-Ponty would, too, reject The Thesis of Minimal Multimodality and endorse the strong, Constitutive Thesis that O'Callaghan defends. More interestingly still, we will see, secondly, that the widening of phenomenal consciousness that they argue for allows to generalize the constitutive thesis, thereby showing that all perceptual experience constitutively depend on the interplay of two or more senses. Spelling out the details of this claim will bring me, thirdly, to specify the intrinsic relation between normativity and perceptual experience. In short, I will argue that the mechanisms responsible for multimodality make a phenomenological difference evaluable in normative terms. (shrink)

According to the decomposition thesis, perceptual experiences resolve without remainder into their different modality-specific components. Contrary to this view, I argue that certain cases of multisensory integration give rise to experiences representing features of a novel type. Through the coordinated use of bodily awareness—understood here as encompassing both proprioception and kinaesthesis—and the exteroceptive sensory modalities, one becomes perceptually responsive to spatial features whose instances couldn’t be represented by any of the contributing modalities functioning in isolation. I develop an argument (...) for this conclusion focusing on two cases: 3D shape perception in haptic touch and experiencing an object’s egocentric location in crossmodally accessible, environmental space. (shrink)

Multisensory processing encompasses all of the various ways in which the presence of information in one sensory modality can adaptively influence the processing of information in a different modality. In Part I of this survey article, I begin by presenting a cartography of some of the more extensively investigated forms of multisensory processing, with a special focus on two distinct types of multisensory integration. I briefly discuss the conditions under which these different forms of multisensory processing (...) occur as well as their important perceptual consequences and interrelations. In Part II, I then turn to examining of some of the different possible ways in which the structure of conscious perceptual experience might also be characterized as multisensory. In addition, I discuss the significance of research on multisensory processing and multisensory consciousness for philosophical attempts to individuate the senses. (shrink)

This report highlights and explores five questions that arose from the multisensory integration workshop at the University of Toronto on May 9th and 10th, 2014: 1. What Is Multisensory Integration? 2. Do Multisensory Percepts Involve Emergent Features? 3. What Can Multisensory Processing Tell Us about Multisensory Awareness? 4. Is Language Processing a Special Kind of Multisensory Integration? 5. What Is the Purpose of Multisensory Integration?

This chapter distinguishes between two kinds of ordinary multisensory experience that go beyond mere co-consciousness of features (e.g., the experience that results from concurrently hearing a sound in the hallway and seeing the cup on the table). In one case, a sensory experience in one modality creates a perceptual demonstrative to whose referent qualities are attributed in another sensory modality. For example, when you hear someone speak, auditory experience attributes audible qualities to a seen event, a person’s speaking motions. (...) The second kind of multisensory experience attributes features experienced in several sensory modalities to one and the same object via a process of amodal perceptual integration, i.e., integration that occurs separately from processing within the individual sensory modalities. Multisensory experiences arising from holding and seeing a tomato or from seeing the Indian curry boil and smelling it are examples of the second kind of multisensory experience. At the end of the chapter we look at synesthesia, a kind of atypical multisensory experience, and argue that one version of this phenomenon may be able to shed light on the neural mechanism underlying amodally integrated multisensory experience. (shrink)

It is tempting to think that one’s perceptual evidence comprises just what issues from perceiving with each of the respective sensory modalities. However, empirical, rational, and phenomenological considerations show that one’s perceptual evidence can outstrip what one possesses due to perceiving with each separate sense. Some novel perceptual evidence stems from the coordinated use of multiple senses. This paper argues that some perceptual evidence in this respect is distinctively multisensory.

In this paper we argue that object perception may be affected by what we call “perceptual frames.” Perceptual frames are adaptations of the perceptual system that guide how perceptual objects are singled out from a sensory environment. These adaptations are caused by perceptual learning and realized through bottom-up functional processes such that sensory information is organized in a subject-dependent way leading to idiosyncratic perceptual object representations. Through domain-specific training, perceptual learning, and the acquisition of object-knowledge, it is possible to (...) modulate the adaptive perceptual system such that its ability to represent becomes bespoke. Different perceivers with different perceptual frames may, therefore, receive the same sensory information and perceive different perceptual objects due to the effects of framing. Consequently, we demonstrate the plausibility of this account by surveying empirical data concerning the functions of (1) multisensory integration, (2) amodal completion, and (3) predictive anticipation. Regarding (1), we argue that the perceptual system’s optimization processes employ perceptual frames to facilitate multisensory feature binding. Regarding (2), we argue that amodal completion can occur with or without the help of mental imagery, yet either instance of amodal completion requires perceptual frames. Regarding (3), we demonstrate that perceptually anticipating an object’s motion involves the implementation of perceptual frames. We conclude that framing effects are a matter of perceptual diversity and highlight the need to accommodate unique perspectives in the philosophy and science of perception. (shrink)

Higher animals need to identify and track material objects because they depend on interactions with them for nutrition, reproduction, and social interaction. This paper investigates the perception of material objects. It argues, first, that material objects are tagged, in all five external senses, as bearers of the features detected by them. This happens through a perceptual process, here entitled Generalized Completion, which creates the appearance of objects that have properties that transcend the activation of sensory receptors. The paper shows, (...) secondly, that material objects are privileged subjects for perceived motion and interaction. That is, they are perceived as subjects for these properties while their parts seem to be subjects only derivatively. Material objects are the only perceptual subjects that are both multisensory and privileged. (shrink)

In crossmodal identification, a subject token identifies an item perceived in one sensory modality with an item perceived in another sensory modality. Does crossmodal identification always occur in cognition, or does crossmodal identification sometimes take place in perception? This paper argues that crossmodal identification occurs in cognition, and not in perception. Nevertheless, multisensory perception is not unalive to crossmodal identity. Experimental evidence demonstrates that perception is differentially sensitive to the identity of individuals presented to distinct (...) senses. Such sensitivity enhances recognition and improves action. This approach relies on distinguishing crossmodal identification from perceiving crossmodal identity. Perception registers crossmodal identity, but crossmodal identification as such belongs to thought. (shrink)

Higher animals need to identify and track material objects because they depend on interactions with them for nutrition, reproduction, and social interaction. This paper investigates the perception of material objects. It argues, first, that material objects are tagged, in all five external senses, as bearers of the features detected by them. This happens through a perceptual process, here entitled Generalized Completion, which creates the appearance of objects that have properties that transcend the activation of sensory receptors. The paper shows, (...) secondly, that material objects are privileged subjects for perceived motion and interaction. That is, they are perceived as subjects for these properties while their parts seem to be subjects only derivatively. Material objects are the only perceptual subjects that are both multisensory and privileged. (shrink)

Our emotional states affect how we perceive the world. If I am stressed, annoyed, or irritated, I might experience the sound of children laughing and screaming as they play around the house in a negative manner — it is unpleasant, loud, piercing, and so on. Yet, if I’m in a relaxed, happy, loving mood, the very same sounds might be experienced as pleasant, playful, warm, and so on. The sounds being made by the children are the same in both cases, (...) but how they are experienced differs. The auditory experience is affected by the emotional state that I am in. I perceive the sounds differently depending on how I’m feeling. Although this might at first seem to be a trivial observation, it certainly is not. We take for granted that our perceptual experiences give rise to justified beliefs and knowledge. Thus, if my perceptual experience of an object can differ based on my emotional state, and I take my perceptual experience to justify my beliefs and even lead to knowledge, then we might have a problem. If emotions — which are not fixed and in large part uncontrolled — affect our ability to accurately perceive the world, then they may undermine the justified beliefs and knowledge gained on the basis of our perceptual experience of the world. My goal here is to explore this potential problem. -/- I consider how we might understand the effect that emotions have on the justification of our perceptual beliefs about the world, beliefs that we acquire from a variety of sensory modalities — audition, gustation olfaction, and so on. I take the problem to be associated with one of two forms of perceptual influence: penetration or multisensory integration. In any given perceptual moment there are multiple sensory modalities and mental states at play, each affecting the overall experience. Whether we understand the influence of emotion on perception as a form of non-perceptual penetration or a form of non-visual sensory perception of the inner body — interoception — the potential epistemological difficulties remain: how can we be said to acquire justified beliefs and knowledge on the basis of such influenced perceptual experience. As has been the norm, only the five exteroceptive senses of vision, audition, olfaction, taste and touch are typically discussed in the context of sensory perception. However, as I argue, there is strong reason to accept the claim that emotional experience is a form of interoception, and that interoception ought to be considered when discussing sensory perception. In this way, then, I propose that clarifying the role played by interoception in sense perception across modalities is necessary if we are to make progress on the epistemological problems at hand. (shrink)

The dual role of olfaction in both smelling and tasting, i.e. flavour perception, makes it an important test case for philosophical theories of sensory individuation. Indeed, the psychologist Paul Rozin claimed that olfaction is a “dual sense”, leading some scientists and philosophers to propose that we have not one, but two senses of smell: orthonasal and retronasal olfaction. In this paper I consider how best to understand Rozin’s claim, and upon what grounds one might judge there to be one (...) or two distinct olfactory modalities. I conclude that while Rozin may be right that humans have dual occurrences of an olfactory ‘sense’, the concept of a sense-modality, and hence the ‘sense’ of smell, is ambiguous between two different notions: a physiological sensory channel and an experiential modality, along the lines suggested by J. J. Gibson. Furthermore, recognising that these are complementary rather than competing conceptions of a sense-modality enables the formulation of a powerful ‘dual-concept’ framework for posing and addressing questions concerning the complex architecture of human multisensory experience. (shrink)

Each of our sensory modalities—vision, touch, taste, etc.—works on a slightly different timescale, with differing temporal resolutions and processing lag. This raises the question of how, or indeed whether, these sensory streams are co-ordinated or ‘bound’ into a coherent multisensory experience of the perceptual ‘now’. In this paper I evaluate one account of how temporal binding is achieved: the temporal windows hypothesis, concluding that, in its simplest form, this hypothesis is inadequate to capture a variety of multisensory phenomena. (...) Rather, the evidence suggests the existence of a more complex temporal structure in which multiple overlapping windows support distinct functional mechanisms. To aid in the precise formulation of such views, I propose a taxonomy of temporal window types and their characteristics that in turn suggests promising avenues for future empirical and philosophical research. I conclude by examining some philosophical implications of multi-window models for the metaphysics of perception and perceptual experience more generally. (shrink)

We can discern two opposing viewpoints regarding synesthesia. According to the first, it is an oddity, an outlier, or a disordered condition. According to the second, synesthesia is pervasive, driving creativity, metaphor, or language itself. Which is it? Ultimately, I favor the first perspective, according to which cross-sensory synesthesia is an outlying condition. But the second perspective is not wholly misguided. My discussion has three lessons. First, synesthesia is just one of a variety of effects in which one sense modality (...) causally impacts and reshapes experience associated with another. These effects are utterly common. However, due to their unfamiliarity, and to their conflict with a widespread conception of the role of the senses in perception and perceptual experience, until recently they have been surprising. Second, synesthesia nevertheless must be distinguished from other inter-modal effects that lead to misperception, such as crossmodal illusions. Third, synesthesia also may be distinguished from the potentially much broader class of synesthetic effects, which could be common across the population and within individuals. (shrink)

From playing basketball to ordering at a food counter, we frequently and effortlessly coordinate our attention with others towards a common focus: we look at the ball, or point at a piece of cake. This non-verbal coordination of attention plays a fundamental role in our social lives: it ensures that we refer to the same object, develop a shared language, understand each other’s mental states, and coordinate our actions. Models of joint attention generally attribute this accomplishment to gaze coordination. But (...) are visual attentional mechanisms sufficient to achieve joint attention, in all cases? Besides cases where visual information is missing, we show how combining it with other senses can be helpful, and even necessary to certain uses of joint attention. We explain the two ways in which non-visual cues contribute to joint attention: either as enhancers, when they complement gaze and pointing gestures in order to coordinate joint attention on visible objects, or as modality pointers, when joint attention needs to be shifted away from the whole object to one of its properties, say weight or texture. This multisensory approach to joint attention has important implications for social robotics, clinical diagnostics, pedagogy and theoretical debates on the construction of a shared world. (shrink)

Humans coordinate their focus of attention with others, either by gaze following or prior agreement. Though the effects of joint attention on perceptual and cognitive processing tend to be examined in purely visual environments, they should also show in multisensory settings. According to a prevalent hypothesis, joint attention enhances visual information encoding and processing, over and above individual attention. If two individuals jointly attend to the visual components of an audiovisual event, this should affect the weighing of visual information (...) during multisensory integration. We tested this prediction in this preregistered study, using the well-documented sound-induced flash illusions, where the integration of an incongruent number of visual flashes and auditory beeps results in a single flash being seen as two (fission illusion) and two flashes as one (fusion illusion). Participants were asked to count flashes either alone or together, and expected to be less prone to both fission and fusion illusions when they jointly attended to the visual targets. However, illusions were as frequent when people attended to the flashes alone or with someone else, even though they responded faster during joint attention. Our results reveal the limitations of the theory that joint attention enhances visual processing as it does not affect temporal audiovisual integration. (shrink)

This paper presents an account of the senses and what differentiates them that is compatible with richly multisensory perception and consciousness. According to this proposal, senses are ways of perceiving. Each sense is a subfaculty that comprises a collection of perceptual capacities. What each sense shares and what differentiates one sense from another is the manner in which those capacities are exercised. Each way of perceiving involves a distinct type of information gathering, individuated by the information it functions (...) to extract and the medium from which it does so. This approach distinguishes the project of characterizing and differentiating senses from that of attributing experiences to sensory modalities. Perceptual experiences are episodes in which perceptual capacities are exer- cised. Conscious perceptual episodes may be ascribed to distinct sensory modalities, according to the manners in which perceptual capacities are deployed on an occasion. According to this account, senses are not exclusive. First, their capacities may overlap. Second, perceptual episodes, including conscious experiences, may belong to multiple senses. Indeed, some episodes require the joint use of several senses. In this account, subjects have only limited first-person knowledge of the senses they employ. (shrink)

Long-standing neglect of the chemical senses in the philosophy of perception is due, mostly, to their being regarded as ‘lower’ senses. Smell, taste, and chemically irritated touch are thought to produce mere bodily sensations. However, empirically informed theories of perception can show how these senses lead to perception of objective properties, and why they cannot be treated as special cases of perception modelled on vision. The senses of taste, touch, and smell also combine to create unified (...) perceptions of flavour. The nature of these multimodal experiences and the character of our awareness of them puts pressure on the traditional idea that each episode of perception goes one or other of the five senses. Thus, the chemical senses, far from being peripheral to the concerns of the philosophy of perception, may hold important clues to the multisensory nature of perception in general. (shrink)

Mirror neuron research has come a long way since the early 1990s, and many theorists are now stressing the heterogeneity and complexity of the sensorimotor properties of fronto-parietal circuits. However, core aspects of the initial ‘ mirror mechanism ’ theory, i.e. the idea of a symmetric encapsulated mirroring function translating sensory action perceptions into motor formats, still appears to be shaping much of the debate. This article challenges the empirical plausibility of the sensorimotor segregation implicit in the original mirror metaphor. (...) It is proposed instead that the teleological organization found in the broader fronto-parietal circuits might be inherently sensorimotor. Thus the idea of an independent ‘purely perceptual’ goal understanding process is questioned. Further, it is hypothesized that the often asymmetric, heterogeneous and contextually modulated mirror and canonical neurons support a function of multisensory mapping and tracking of the perceiving agents affordance space. Such a shift in the interpretative framework offers a different theoretical handle on how sensorimotor processes might ground various aspects of intentional action choice and social cognition. Mirror neurons would under the proposed “social affordance model” be seen as dynamic parts of larger circuits, which support tracking of currently shared and competing action possibilities. These circuits support action selection processes—but also our understanding of the options and action potentials that we and perhaps others have in the affordance space. In terms of social cognition ‘ mirror ’ circuits might thus help us understand not only the intentional actions others are actually performing—but also what they could have done, did not do and might do shortly. (shrink)

This chapter offers an indirect defence of the Evansian conception of egocentric space, by showing how it resolves a puzzle concerning the unity of egocentric spatial perception. The chapter outlines several common assumptions about egocentric perspectival structure and argues that a subject’s experience, both within and across her sensory modalities, may involve multiple structures of this kind. This raises the question of how perspectival unity is achieved, such that these perspectival structures form a complex whole, rather than merely disunified (...) set of individually, distinctively structured experiences. The shortcomings of variety of accounts are considered: switch accounts ; sensory accounts; transformation accounts; and ultimate accounts. These shortcomings are addressed by a further kind of account provided by the Evansian conception – an agentive account – according to which egocentrically structured experiences present the world in relation to parts of a single thing, the body as a dynamic unity. (shrink)

This contribution examines two radically different explanations of our phenomenal intuitions, one reductive and one strongly non-reductive, and identifies two germane ideas that could benefit many other theories of consciousness. Firstly, the ability of sophisticated agent architectures with a purely physical implementation to support certain functional forms of qualia or proto-qualia appears to entail the possibility of machine consciousness with qualia, not only for reductive theories but also for the nonreductive ones that regard consciousness as ubiquitous in Nature. Secondly, analysis (...) of introspective psychological material seems to hint that, under the threshold of our ordinary waking awareness, there exist further ‘submerged’ or ‘subliminal’ layers of consciousness which constitute a hidden foundation and support and another source of our phenomenal intuitions. These ‘submerged’ layers might help explain certain puzzling phenomena concerning subliminal perception, such as the apparently ‘unconscious’ multisensory integration and learning of subliminal stimuli. (shrink)

There is converging evidence that high doses of hallucinogenic drugs can produce significant alterations of self-experience, described as the dissolution of the sense of self and the loss of boundaries between self and world. This article discusses the relevance of this phenomenon, known as “drug-induced ego dissolution (DIED)”, for cognitive neuroscience, psychology and philosophy of mind. Data from self-report questionnaires suggest that three neuropharmacological classes of drugs can induce ego dissolution: classical psychedelics, dissociative anesthetics and agonists of the kappa opioid (...) receptor (KOR). While these substances act on different neurotransmitter receptors, they all produce strong subjective effects that can be compared to the symptoms of acute psychosis, including ego dissolution. It has been suggested that neuroimaging of DIED can indirectly shed light on the neural correlates of the self. While this line of inquiry is promising, its results must be interpreted with caution. First, neural correlates of ego dissolution might reveal the necessary neurophysiological conditions for the maintenance of the sense of self, but it is more doubtful that this method can reveal its minimally sufficient conditions. Second, it is necessary to define the relevant notion of self at play in the phenomenon of DIED. This article suggests that DIED consists in the disruption of subpersonal processes underlying the “minimal” or “embodied” self, i.e., the basic experience of being a self rooted in multimodal integration of self-related stimuli. This hypothesis is consistent with Bayesian models of phenomenal selfhood, according to which the subjective structure of conscious experience ultimately results from the optimization of predictions in perception and action. Finally, it is argued that DIED is also of particular interest for philosophy of mind. On the one hand, it challenges theories according to which consciousness always involves self-awareness. On the other hand, it suggests that ordinary conscious experience might involve a minimal kind of self-awareness rooted in multisensory processing, which is what appears to fade away during DIED. (shrink)

We perceive in many ways. But several dubious presuppositions about the senses mask this diversity of perception. Philosophers, scientists, and engineers alike too often presuppose that the senses (vision, audition, etc.) are independent sources of information, perception being a sum of these independent contributions. We too often presuppose that we can generalize from vision to other senses. We too often presuppose that vision itself is best understood as a passive receptacle for an image thrown by a lens. In (...) this essay we illustrate how reflection on results from the cognitive sciences undermine these (and many other) traditional presuppositions. Our illustration situates the remaining essays in the volume, Perception and its Modalities, which further extend the case against various traditional presuppositions, and offer alternatives ways to understand the senses and their role in perception. While many of the essays in this volume represent competing views and draw on different fields, from conceptual analysis to engineering, most contributors believe that to understand perception properly, we need to treat as foundational the idea that the senses work together. From that agreement, new paradigms for theorizing about perception begin to emerge. (shrink)

With a Call for Essays, the special issue Multimodality sought contributions that accept not only the material but also the body-bound dependence of media perception and understanding. To this end, contributions were included that shed light on both the structural and signifying potential of artistic works through multimodal analysis. Particular attention was paid to contributions that clarify how the structural features - the modes - of the arts, their perception, and their signifying potential in terms of content are (...) interrelated and how they are to be understood in communicative and thus socioculturally relevant terms. Thus, in addition to neuroscientific contributions, those from cultural anthropology, art history, image and art studies, and literary studies were included. -/- Articles -/- 8 Martina Sauer and Christiane Wagner - Introduction -/- 11 Giada Lombardi and Giuseppe Di Cesare - From Neuroscience to Art: The Role of ‘Vitality Forms’ in the Investigation of Multimodality -/- 25 Martina Sauer - Abstract - Affective – Multimodal: Interaction between Medium and Perception of Moving Images from the Viewpoint of Cassirer’s, Langer’s and Krois’ Embodiment Theories -/- 49 Göran Sonesson - The Secrets of Plastic Language Revealed: Multimodality, Polysemiosis, and Iconicity -/- 71 Natalia Igl - Making Sense of the Material: Multisensory Reader Involvement in Contemporary Multimodal Novels -/- 87 J. Philippe Thompson - Art is All: Proust’s Life Lessons Through Perception and Art -/- 107 Hans Dieter Huber - Philippe Parreno and the Exhibition as a Multimodal Aesthetic Experience -/- 135 Jörg U. Lensing - The Performing Human Being in a Media Interaction Space: Multi- and intermodal productions by the Düsseldorf Theater der Klänge (Theatre of Sounds) . (shrink)

We argue that animals are not cognitively stuck in time. Evidence pertaining to multisensory temporal order perception strongly suggests that animals can represent at least some temporal relations of perceived events.

Some philosophers have argued that we do not hear sounds as located in the environment. Others have objected that this straightforwardly contradicts the phenomenology of auditory experience. And from this they draw metaphysical conclusions about the nature of sounds—that they are events or properties of vibrating surfaces rather than waves or sensations. I argue that there is a minimal, but recognizable, notion of audition to which this phenomenal objection does not apply. While this notion doesn’t correspond to our ordinary notion (...) of auditory experience, it does—in conjunction with our lack of an uncontroversial individuation of the senses and recent interest in distinctively multisensory features of perceptual experiences—raise the possibility of more expansive notions of audition, including some that do plausibly count as corresponding to our everyday notion of audition, that lack the spatial phenomenology cited in the objection. Until this possibility is ruled out, the phenomenal objection and metaphysical conclusions drawn from it remain inconclusive. (shrink)

Many of Margaret Cavendish’s criticisms of Thomas Hobbes in the Philosophical Letters (1664) relate to the disorder and damage that she holds would result if Hobbesian pressure were the cause of visual perception. In this paper, I argue that her “two men” thought experiment in Letter IV is aimed at a different goal: to show the explanatory potency of her account. First, I connect Cavendish’s view of visual perception as “patterning” to the “two men” thought experiment in Letter (...) IV. Second, I provide a potential reply on Hobbes’s behalf that appeals to physiological differences between perceivers’ sense organs, drawing upon Hobbes’s optics in De homine. Third, I argue that such a reply would misunderstand Cavendish’s objective of showing the limited explanatory resources available in understanding visual perception as pressing when compared to her view of visual perception as patterning. (shrink)

When we see an object, we also represent those parts of it that are not visible. The question is how we represent them: this is the problem of amodal perception. I will consider three possible accounts: (a) we see them, (b) we have non-perceptual beliefs about them and (c) we have immediate perceptual access to them, and point out that all of these views face both empirical and conceptual objections. I suggest and defend a fourth account, according to which (...) we represent the occluded parts of perceived objects by means of mental imagery. This conclusion could be thought of as a (weak) version of the Strawsonian dictum, according to which “imagination is a necessary ingredient of perception itself”. (shrink)

According to Phillips, (1) genuine perception is attributable to the individual (i.e. it’s a personal state/event, as opposed to sub-personal states/events in the individual’s brain); (2) since unconscious perceptual representations are ill-suited to guide action, there’s no good reason to attribute them to the individual; (3) not being attributable to the individual, they don’t instantiate genuine perception, thereby failing to support the hypothesis that genuine perception can occur unconsciously. I argue that this reasoning is flawed and can’t (...) be easily fixed. Phillips contends that unconscious perceptual representations don’t guide action because they fail to meet certain conditions that are sufficient for action guidance. But those conditions may not be necessary for action guidance. Consequently, unconscious perceptual representations may guide action even if they don’t meet Phillips’ conditions. Furthermore, due to his commitment to the distinction between personal and sub-personal states/events, Phillips isn’t in a position to argue that his conditions are necessary for action guidance. For the distinction applies to action as well as to perception, and when genuine action is identified by personal-level criteria, Phillips’ conditions turn out unnecessary for action guidance. (shrink)

When we perceive an object, we perceive the object from a perspective. As a consequence of the perspectival nature of perception, when we perceive, say, a circular coin from different angles, there is a respect in which the coin looks circular throughout, but also a respect in which the coin's appearance changes. More generally, perception of shape and size properties has both a constant aspect—an aspect that remains stable across changes in perspective—and a perspectival aspect—an aspect that changes (...) depending on one's perspective on the object. How should we account for the perspectival aspect of spatial perception? We present a framework within which to discuss the perspectival aspect of perception and put forward three desiderata that any account of the perspectival aspect of perception should satisfy. We discuss views on which the perspectival aspect of perception is analyzed in terms of constitutively mind-dependent appearance properties as well as views on which the perspectival aspect of perception is analyzed in terms of representations of mind-independent perspectival properties. (shrink)

In the contemporary discussions concerning unconscious perception it is not uncommon to postulate that content and phenomenal character are ‘orthogonal’, i.e., there is no type of content which is essentially conscious, but instead, every representational content can be either conscious or not. Furthermore, this is not merely treated as a thesis justified by theoretical investigations, but as supported by empirical considerations concerning the actual functioning of the human cognition. In this paper, I address unconscious color perception and argue (...) for a negative thesis—that the main experimental paradigms used in studying unconscious color perception do not provide support for the position that conscious and unconscious color representations have the same type of content. More specifically, I claim that there is no significant support for the claim that unconscious vision categorically represents surface colors. (shrink)

In this Introduction we introduce the central themes of the Evaluative Perception volume. After identifying historical and recent contemporary work on this topic, we discuss some central questions under three headings: (1) Questions about the Existence and Nature of Evaluative Perception: Are there perceptual experiences of values? If so, what is their nature? Are experiences of values sui generis? Are values necessary for certain kinds of experience? (2) Questions about the Epistemology of Evaluative Perception: Can evaluative experiences (...) ever justify evaluative judgments? Are experiences of values necessary for certain kinds of justified evaluative judgments? (3) Questions about Value Theory and Evaluative Perception: Is the existence of evaluative experience supported or undermined by particular views in value theory? Are particular views in value theory supported or undermined by the existence of value experience? (shrink)

The success of the Bayesian approach to perception suggests probabilistic perceptual representations. But if perceptual representation is probabilistic, why doesn't normal conscious perception reflect the full probability distributions that the probabilistic point of view endorses? For example, neurons in MT/V5 that respond to the direction of motion are broadly tuned: a patch of cortex that is tuned to vertical motion also responds to horizontal motion, but when we see vertical motion, foveally, in good conditions, it does not look (...) at all horizontal. This article argues that the best Bayesian approach to this problem does not require probabilistic representation. (shrink)

Predictive approaches to the mind claim that perception, cognition, and action can be understood in terms of a single framework: a hierarchy of Bayesian models employing the computational strategy of predictive coding. Proponents of this view disagree, however, over the extent to which perception is direct on the predictive approach. I argue that we can resolve these disagreements by identifying three distinct notions of perceptual directness: psychological, metaphysical, and epistemological. I propose that perception is plausibly construed as (...) psychologically indirect on the predictive approach, in the sense of being constructivist or inferential. It would be wrong to conclude from this, however, that perception is therefore indirect in a metaphysical or epistemological sense on the predictive approach. In the metaphysical case, claims about the inferential properties of constructivist perceptual mechanisms are consistent with both direct and indirect solutions to the metaphysical problem of perception (e.g. naïve realism, representationalism, sense datum theory). In the epistemological case, claims about the inferential properties of constructivist perceptual mechanisms are consistent with both direct and indirect approaches to the justification of perceptual belief. In this paper, I demonstrate how proponents of the predictive approach have conflated these distinct notions of perceptual directness and indirectness, and I propose alternative strategies for developing the philosophical consequences of the approach. (shrink)

One necessary condition on any adequate account of perception is clarity regarding whether unconscious perception exists. The issue is complicated, and the debate is growing in both philosophy and science. In this paper we consider the case for unconscious perception, offering three primary achievements. First, we offer a discussion of the underspecified notion of central coordinating agency, a notion that is critical for arguments that purportedly perceptual states are not attributable to the individual, and thus not genuinely (...) perceptual. We develop an explication of what it is for a representational state to be available to central coordinating agency for guidance of behavior. Second, drawing on this explication, we place a more careful understanding of the attributability of a state to the individual in the context of a range of empirical work on vision-for-action, saccades, and skilled typing. The results place pressure on the skeptic about unconscious perception. Third, reflecting upon broader philosophical themes running through debates about unconscious perception, we highlight how our discussion places pressure on the view that perception is a manifest kind, rather than a natural kind. In doing so, we resist the tempting complaint that the debate about unconscious perception is merely verbal. (shrink)

I aim to show that perception depends counterfactually on the action we want to perform. Perception is not all-purpose: what we want to do does influence what we see. After clarifying how this claim is different from the one at stake in the cognitive penetrability debate and what counterfactual dependence means in my claim, I will give a two-step argument: one’s perceptual attention depends counterfactually on one’s intention to perform an action and one’s perceptual processing depends counterfactually on (...) one’s perceptual attention. If we put these claims together, what we get is that one’s perceptual processing depends counterfactually on one’s intention to perform an action. (shrink)

According to a traditional view, there is no categorical difference between the phenomenology of perception and the phenomenology of imagination; the only difference is in degree (of intensity, resolution, etc.) and/or in accompanying beliefs. There is no categorical difference between what it is like to perceive a dog and what it is like to imagine a dog; the former is simply more vivid and/or is accompanied by the belief that a dog is really there. A sustained argument against this (...) traditional view is prosecuted by Sartre, who develops an alternative according to which there is a categorical difference between the two phenomenologies. This paper draws on Sartre’s work in this area to develop a similar account of perception and imagination as categorically different. (shrink)

Traditional accounts of the perception/cognition divide tend to draw it in terms of subpersonal psychological processes, processes into which the subject has no first-person insight. Whatever betides such accounts, there seems to also be some first-personally accessible difference between perception and thought. At least in normal circumstances, naïve subjects can typically tell apart their perceptual states from their cognitive or intellectual ones. What are such subjects picking up on when they do so? This paper is an inconclusive search (...) for an answer. At its end, I conclude, without joy, that we may have to simply accept the perception/cognition distinction as a primitive and inexplicable bright line within the sphere of conscious phenomena. (shrink)

It seems plausible that visual experiences of darkness have perceptual phenomenal content that clearly differentiates them from absences of visual experiences. I argue, relying on psychological results concerning auditory attention, that the analogous claim is true about auditory experiences of silence. More specifically, I propose that experiences of silence present empty spatial directions like ‘right’ or ‘left’, and so have egocentric spatial content. Furthermore, I claim that such content is genuinely auditory and phenomenal in the sense that one can, in (...) principle, recognize that one is experiencing silence. This position is far from obvious, as the majority of theories concerning silence perception do not ascribe perceptual phenomenal content to experiences of silence. (shrink)

ABSTRACT The difference between perception and cognition seems introspectively obvious in many cases. Perceiving and thinking have also been assigned quite different roles, in epistemology, in theories of reference and of mental content, in philosophy of psychology, and elsewhere. Yet what is the nature of the distinction? In what way, or ways, do perception and cognition differ? The paper reviews recent work on these questions. Four main respects in which perception and cognition have been held to differ (...) are discussed. First, their phenomenal character, such as the often-remarked vivacity or immediacy of perception. Second, the way in which they represent the world, e.g. the non-propositional nature of the contents, or non-discursive character of the vehicles, that have been held to characterise perceptual representation. Third, their place in cognitive architecture, i.e., roughly, in the information-flow of the mind, such as their alleged (non-)modularity. Fourth, their mind-world relations, e.g. the way in which perceptions seem to be tightly causally linked with distal or proximal stimuli. Against this background, we distinguish some main options for an account of the perception/cognition distinction, in particular concerning whether there is one, several, or no interesting and principled distinction(s) to be drawn here. (shrink)

Perceptual processes differ from cognitive, this paper argues, in functioning to be causally controlled by proximal stimuli, and being modular, at least in a modest sense that excludes their being isotropic in Jerry Fodor's sense. This claim agrees with such theorists as Jacob Beck and Ben Phillips that a function of stimulus-control is needed for perceptual status. In support of this necessity claim, I argue, inter alia, that E.J. Green's recent architectural account misclassifies processes deploying knowledge of grammar as perceptual. (...) _Pace_ Beck and Phillips, however, I argue a function of stimulus-control is insufficient for perceptual as opposed to cognitive status. One consideration in favour of such insufficiency, noted but (I argue) not convincingly rebutted by these theorists, concerns perpetually grounded demonstrative thought. Two other considerations trade on the fact that a function of stimulus-control can arise not from blind nature but intentional design or social institutions, where so-functioning processes may but need not be perceptual. I offer two cases where such processes are cognitive, viz. skilful play-by-play announcing of ongoing events, and voluntary visualizing of ongoing events under the guidance of apt play-by-play announcements, dubbed announcement-driven visualizing (ADV). The cognitive status of these three diverse phenomena cannot be explained by an absence of a perception-like representational format or content (for ADV has such) or by a presence of personal-level mental states causally mediating between stimuli and outputs (for perception has such). A bettter explanation invokes, I argue, the non-modular character of the generating process. (shrink)

In a recent paper, Berger and Nanay consider, and reject, three ways of addressing the phenomenon of unconscious perception within a naïve realist framework. Since these three approaches seem to exhaust the options open to naïve realists, and since there is said to be excellent evidence that perception of the same fundamental kind can occur, both consciously and unconsciously, this is seen to present a problem for the view. We take this opportunity to show that all three approaches (...) considered remain perfectly plausible ways of addressing unconscious perception within a naïve realist framework. So far from undermining the credibility of naïve realism, Berger and Nanay simply draw our attention to an important question to be considered by naïve realists in future work. Namely, which of the approaches considered is most likely to provide an accurate account of unconscious perception in each of its purported incarnations? (shrink)

Outside of philosophy, ‘intuition’ means something like ‘knowing without knowing how you know’. Intuition in this broad sense is an important epistemological category. I distinguish intuition from perception and perception from perceptual experience, in order to discuss the distinctive psychological and epistemological status of evaluative property attributions. Although it is doubtful that we perceptually experience many evaluative properties and also somewhat unlikely that we perceive many evaluative properties, it is highly plausible that we intuit many instances of evaluative (...) properties as such. The resulting epistemological status of evaluative property attributions is very much like it would be if we literally perceived such properties. (shrink)

While the view that we perceive aesthetic properties may seem intuitive, it has received little in the way of explicit defence. It also gives rise to a puzzle. The first strand of this puzzle is that we often cannot perceive aesthetic properties of artworks without training, yet much aesthetic training involves the acquisition of knowledge, such as when an artwork was made, and by whom. How, if at all, can this knowledge affect our perception of an artwork’s aesthetic properties? (...) The second strand of the puzzle arises when we widen the scope of aesthetic experience. The very same aesthetic properties that seem to require training for their perception in artworks do not appear to require training to perceive in objects of everyday aesthetic appreciation and natural phenomena. In this paper I argue that a prominent extant attempt to explain how training is compatible with aesthetic perception—cognitive permeation—is an inadequate solution. I also develop a positive view of aesthetic perception that provides a unified solution to both strands of the puzzle. (shrink)

We begin with a puzzle about how to intelligibly combine the active and passive elements of perception. For counsel, we turn to Augustine’s account of perception in De Trinitate. Augustine’s trinitarian account of perception offers an attractive resolution of our puzzle. Augustine’s resolution of our puzzle, however, cannot be straightforwardly adopted. It must be adapted. We end with speculation about how this might be done.