In this paper we wish to bring together two seemingly independent areas of research: synaesthesia and sensory substitution. Synaesthesia refers to a rare condition where a sensory stimulus elicits not only the sensation that stimulus evokes in its own modality, but an additional one; a synaesthete may thus hear the word “Monday”, and, in addition to hearing it, have a concurrent visual experience of a red color. Sensory substitution, in contrast, attempts to substitute a sensory modality that a person has (...) lost by transforming the information it provided so that it can be accessed through another, intact sensory modality. To make visual information accessible to a blind person, for example, data taken by a camera would be transformed into tactile or auditory information. What do synaesthesia and sensory substitution have in common? Research in both of these areas contributes to our understanding both of cross-modal cooperation and of sensory sensations or qualia, asking under what circumstances these can arise in a modality that is not stimulated. Synaesthesia reveals that this “sensory cross-activation” is possible, and sensory substitution research hopes to induce it. In this article, we will review briefly the literature on synaesthesia, and discuss the issue of qualia for this domain of research. We will then address the evidence for synaesthesia and visual qualia in the blind, and the research on sensory substitution, to finally ask whether sensory substitution may induce a ‘synthetic’ form of synaesthesia by taking advantage of the nervous system’s capacity for generating visual images in the absence of retinal input. (shrink)

Perceptual modalities have been traditionally considered the product of dedicated biological systems producing information for higher cognitive processing. Psychological and neuropsychological evidence is offered which undermines this point of view and an alternative account of modality from the enactive approach to understanding cognition is suggested. Under this view, a perceptual modality is a stable form of perception which is structured not just by the biological sensitivities of the agent, but by their goals and the set of skills or expertise which (...) they are deploying at a given time. Such a view suggests that there is no such thing as an experience that is purely visual, auditory, or otherwise modal and that our attempts to understand consciousness and the mind must be conducted within a framework that provides an account of embodied, goal-directed adaptive coping with the world. (shrink)

Block (Trends Cogn Sci 7:285–286, 2003) and Prinz (PSYCHE 12:1–19, 2006) have defended the idea that SSD perception remains in the substituting modality (auditory or tactile). Hurley and Noë (Biol Philos 18:131–168, 2003) instead argued that after substantial training with the device, the perceptual experience that the SSD user enjoys undergoes a change, switching from tactile/auditory to visual. This debate has unfolded in something like a stalemate where, I will argue, it has become difficult to determine whether the perception acquired (...) through the coupling with an SSD remains in the substituting or the substituted modality. Within this puzzling deadlock two new approaches have been recently suggested. Ward and Meijer (Conscious Cogn 19:492–500, 2010) describe SSD perception as visual-like but characterize it as a kind of artificially induced synaesthesia. Auvray et al. (Perception 36:416–430, 2007) and Auvray and Myin (Cogn Sci 33:1036–1058, 2009) suggest that SSDs let their users experience a new kind of perception. Deroy and Auvray (forthcoming) refine this position, and argue that this new kind of perception depends on pre-existing senses without entirely aligning with any of them. So, they have talked about perceptual experience in SSDs as going "beyond vision". In a similar vein, MacPherson (Oxford University Press, New York, 2011a) claims that “if the subjects (SSD users) have experiences with both vision-like and touch-like representational characteristics then perhaps they have a sense that ordinary humans do not” (MacPherson in Oxford University Press, New York, 2011a, p. 139). (shrink)

Is the brain the biological substrate of consciousness? Most naturalistic philosophers of mind have supposed that the answer must obviously be «yes » to this question. However, a growing number of philosophers working in 4e (embodied, embedded, extended, enactive) cognitive science have begun to challenge this assumption, arguing instead that consciousness supervenes on the whole embodied animal in dynamic interaction with the environment. We call views that share this claim dynamic sensorimotor theories of consciousness (DSM). Clark (2009) a founder and (...) leading proponent of the hypothesis of the extended mind, demurs, arguing that as matter of fact the biology of consciousness doesn’t allow for a brain, body and world boundary crossing architecture. We begin by looking at one of the arguments for DSM, the variable neural correlates argument. We then outline two criticisms that Clark has made of this argument and endorse his criticisms. However we finish up by using the case of sensory substitution to argue that something of this argument for DSM nevertheless survives. We suggest that Clark ought to concede sensory substitution as a case in which the conscious mind extends. (shrink)

How could neural processes be associated with phenomenal consciousness? We present a way to answer this question by taking the counterintuitive stance that the sensory feel of an experience is not a thing that happens to us, but a thing we do: a skill we exercise. By additionally noting that sensory systems possess two important, objectively measurable properties, corporality and alerting capacity, we are able to explain why sensory experience possesses a sensory feel, but thinking and other mental processes do (...) not. We are additionally able to explain why different sensory feels differ in the way they do. (shrink)

The strategies of action employed by a human subject in order to perceive simple 2-D forms on the basis of tactile sensory feedback have been modelled by an explicit computer algorithm. The modelling process has been constrained and informed by the capacity of human subjects both to consciously describe their own strategies, and to apply explicit strategies; thus, the strategies effectively employed by the human subject have been influenced by the modelling process itself. On this basis, good qualitative and semi-quantitative (...) agreement has been achieved between the trajectories produced by a human subject, and the traces produced by a computer algorithm. The advantage of this reciprocal modelling option, besides facilitating agreement between the algorithm and the empirically observed trajectories, is that the theoretical model provides an explanation, and not just a description, of the active perception of the human subject. (shrink)

Some regularities enjoy only an attenuated existence in a body of training data. These are regularities whose statistical visibility depends on some systematic recoding of the data. The space of possible recodings is, however, infinitely large type-2 problems. they are standardly solved! This presents a puzzle. How, given the statistical intractability of these type-2 cases, does nature turn the trick? One answer, which we do not pursue, is to suppose that evolution gifts us with exactly the right set of recoding (...) biases so as to reduce specific type-2 problems to (tractable) type-1 mappings. Such a heavy-duty nativism is no doubt sometimes plausible. But we believe there are other, more general mechanisms also at work. Such mechanisms provide general (not task-specific) strategies for managing problems of type-2 complexity. Several such mechanisms are investigated. At the heart of each is a fundamental ploy representational redescription language and culture – may themselves be viewed as adaptations enabling this representation/computation trade-off to be pursued on an even grander scale. (shrink)

The Embodied Mind provides a unique, sophisticated treatment of the spontaneous and reflective dimension of human experience.

Many current neurophysiological, psychophysical, and psychological approaches to vision rest on the idea that when we see, the brain produces an internal representation of the world. The activation of this internal representation is assumed to give rise to the experience of seeing. The problem with this kind of approach is that it leaves unexplained how the existence of such a detailed internal representation might produce visual consciousness. An alternative proposal is made here. We propose that seeing is a way of (...) acting. It is a particular way of exploring the environment. Activity in internal representations does not generate the experience of seeing. The out- side world serves as its own, external, representation. The experience of seeing occurs when the organism masters what we call the gov- erning laws of sensorimotor contingency. The advantage of this approach is that it provides a natural and principled way of accounting for visual consciousness, and for the differences in the perceived quality of sensory experience in the different sensory modalities. Sev- eral lines of empirical evidence are brought forward in support of the theory, in particular: evidence from experiments in sensorimotor adaptation, visual “filling in,” visual stability despite eye movements, change blindness, sensory substitution, and color perception. (shrink)

"Perception is not something that happens to us, or in us," writes Alva Noe. "It is something we do." In Action in Perception, Noe argues that perception and perceptual consciousness depend on capacities for action and thought — that ...

This book analyzes the philosophical foundations of sensorimotor theory and discusses the most recent applications of sensorimotor theory to human computer interaction, child's play, virtual reality, robotics, and linguistics. -/- Why does a circle look curved and not angular? Why doesn't red sound like a bell? Why, as I interact with the world, is there something it is like to be me? These are simple questions to pose but more difficult to answer. An analytic philosopher might respond to the first (...) by merely observing, ``if we ponder the concept `circle' we find that it is the essence of a circle to be round''; but where does this definition come from? Was it set in stone by the gods, the divine arbiters of circle-ness (red-ness and conscious-ness)? It is a measure of the importance of Kevin O'Regan and Alva Noe's 2001 paper `A sensorimotor account of vision and visual consciousness' that it reveals deep scientific and philosophical insight on these foundational issues of perception. -/- Opening with a chapter from Kevin O'Regan, this collection of new essays continues by presenting fifteen additional essays on as many developments achieved in recent years in this field. It provides readers with a critical review of the sensorimotor theory and in so doing introduces them to a radically new approach in cognitive science. (shrink)

Sensorimotor theories understand perception to be a process of active, exploratory engagement with the environment, mediated by the possession and exercise of a certain body of knowledge concerning sensorimotor dependencies. This paper aims to characterise that exercise, and to show that it places constraints upon the content of sensorimotor knowledge itself. Sensorimotor mastery is exercised when it is put to use in the service of intentional action-planning and selection, and this rules out certain standard readings of sensorimotor contingency knowledge. Rather (...) than holding between movements and sensory inputs or appearances, sensorimotor contingencies concern the suite of ways in which an object can be revealed through exploration. Sensorimotor knowledge is thus directed through experience to the world itself. (shrink)

In this report, the phenomenology of two blind users of a sensory substitution device – “The vOICe” – that converts visual images to auditory signals is described. The users both report detailed visual phenomenology that developed within months of immersive use and has continued to evolve over a period of years. This visual phenomenology, although triggered through use of The vOICe, is likely to depend not only on online visualization of the auditory signal but also on the users’ previous (albeit (...) distant) experience of veridical vision (e.g. knowledge of shapes and visual perspective). Once established, the sensory substitution mapping between the auditory and visual domains is not confined to when the device is worn and, thus, may constitute an example of acquired synaesthesia. (shrink)

This paper tackles the problem of the nature of the space of perception. Based both on philosophical arguments and on results obtained from original experimental situations, it attempts to show how space is constituted concretely, before any distinction between the “inner” and the “outer” can be made. It thus sheds light on the presuppositions of the well-known debate between internalism and externalism in the philosophy of mind; it argues in favor of the latter position, but with arguments that are foundationally (...) antecedent to this debate. We call the position we defend enactive externalism. It is based on experimental settings which, in virtue of their minimalism, make it possible both to defend a sensori-motor/enactive theory of perception; and, especially, to inquire into the origin of the space of perception, showing how it is concretely enacted before the controversy between internalism and externalism can even take place. (shrink)

and apply it to various examples of neural plasticity in which input is rerouted intermodally or intramodally to nonstandard cortical targets. In some cases but not others, cortical activity ‘defers’ to the nonstandard sources of input. We ask why, consider some possible explanations, and propose a dynamic sensorimotor hypothesis. We believe that this distinction is important and worthy of further study, both philosophical and empirical, whether or not our hypothesis turns out to be correct. In particular, the question of how (...) the distinction should be explained is linked to explanatory gap issues for consciousness. Comparative and absolute explanatory gaps should be distinguished: why does neural activity in a particular area of cortex have this qualitative expression rather than that, and why does it have any qualitative expression at all? We use the dominance/deference distinction to address the comparative gaps, both intermodal and intramodal. We do so not by inward scrutiny but rather by expanding our gaze to include relations between brain, body and environment. (shrink)

Some regularities enjoy only an attenuated existence in a body of training data. These are regularities whose statistical visibility depends on some systematic recoding of the data. The space of possible recodings is, however, infinitely large – it is the space of applicable Turing machines. As a result, mappings that pivot on such attenuated regularities cannot, in general, be found by brute-force search. The class of problems that present such mappings we call the class of “type-2 problems.” Type-1 problems, by (...) contrast, present tractable problems of search insofar as the relevant regularities can be found by sampling the input data as originally coded. Type-2 problems, we suggest, present neither rare nor pathological cases. They are rife in biologically realistic settings and in domains ranging from simple animat behaviors to language acquisition. Not only are such problems rife – they are standardly solved! This presents a puzzle. How, given the statistical intractability of these type-2 cases, does nature turn the trick? One answer, which we do not pursue, is to suppose that evolution gifts us with exactly the right set of recoding biases so as to reduce specific type-2 problems to type-1 mappings. Such a heavy-duty nativism is no doubt sometimes plausible. But we believe there are other, more general mechanisms also at work. Such mechanisms provide general strategies for managing problems of type-2 complexity. Several such mechanisms are investigated. At the heart of each is a fundamental ploy – namely, the maximal exploitation of states of representation already achieved by prior, simpler learning so as to reduce the amount of subsequent computational search. Such exploitation both characterizes and helps make unitary sense of a diverse range of mechanisms. These include simple incremental learning, modular connectionism, and the developmental hypothesis of “representational redescription”. In addition, the most distinctive features of human cognition – language and culture – may themselves be viewed as adaptations enabling this representation/computation trade-off to be pursued on an even grander scale. (shrink)

Sensory substitution devices provide through an unusual sensory modality (the substituting modality, e.g., audition) access to features of the world that are normally accessed through another sensory modality (the substituted modality, e.g., vision). In this article, we address the question of which sensory modality the acquired perception belongs to. We have recourse to the four traditional criteria that have been used to define sensory modalities: sensory organ, stimuli, properties, and qualitative experience (Grice, 1962), to which we have added the criteria (...) of behavioral equivalence (Morgan, 1977), dedication (Keeley, 2002), and sensorimotor equivalence (O’Regan & Noe¨, 2001). We discuss which of them are fulfilled by perception through sensory substitution devices and whether this favors the view that perception belongs to the substituting or to the substituted modality. Though the application of a number of criteria might be taken to point to the conclusion that perception with a sensory substitution device belongs to the substituted modality, we argue that the evidence leads to an alternative view on sensory substitution. According to this view, the experience after sensory substitution is a transformation, extension, or augmentation of our perceptual capacities, rather than being something equivalent or reducible to an already existing sensory modality. We develop this view by comparing sensory substitution devices to other ‘‘mind-enhancing tools’’ such as pen and paper, sketchpads, or calculators. An analysis of sensory substitution in terms of mind-enhancing tools unveils it as a thoroughly transforming perceptual experience and as giving rise to a novel form of perceptual interaction with the environment. (shrink)

Alva Noë’s _Action in Perception _offers a provocative and vigorous defense of the thesis that vision is enactive: visual experience depends on dispositional motor responses. On this view, vision and action are inextricably bound. In this review, I argue against enactive perception. I raise objections to seven lines of evidence that appear in Noë’s book, and I indicate some reasons for thinking that vision can operate independently of motor responses. I conclude that the relationship between vision and action is causal, (...) not constitutive. I then address three other contentious hypotheses in the book. Noë argues that visual states are not pictorial; he argues that all perception is conceptual; and he argues that the external world makes a constitutive contribution to experience. I am unpersuaded by these arguments, and I offer reasons to resist Noë’s conclusions. (shrink)

Visual information can be provided to blind users through sensory substitution devices that convert images into sound. Through extensive use to develop expertise, some blind users have reported visual experiences when using such a device. These blind expert users have also reported visual phenomenology to other sounds even when not using the device. The blind users acquired synthetic synaesthesia, with visual experience evoked by sounds only after gaining such expertise. Sensorimotor learning may facilitate and perhaps even be required to develop (...) expertise in the use of multimodal information. Furthermore, other areas where expertise is acquired in dividing attention amongst cross-modal information or integrating such information might also give rise to synthetic synaesthesia. (shrink)

The binary divide between traditional cognitivist and enactivist paradigms is tied to their respective commitments to understanding cognition as based on knowing that as opposed to knowing how. Using O’Regan’s and No¨e’s landmark sensorimotor contingency theory of perceptual experience as a foil, I demonstrate how easy it is to fall into conservative thinking. Although their account is advertised as decidedly ‘skill-based’, on close inspection it shows itself to be riddled with suppositions threatening to reduce it to a rules-and-representations approach. To (...) remain properly enactivist it must be purged of such commitments and indeed all commitment to mediating knowledge: it must embrace a more radical enactivism. (shrink)

The mind-body problem is widely seen as the great remaining challenge to science and philosophy. Why and how did matter evolve to take on the quality of mind? The author takes the reader to the edges of current knowledge and back to the beginning of time, before mind existed, and in doing so constructs a history of consciousness.