This paper provides a categorization of cross-modal experiences. There are myriad forms. Doing so allows us to think clearly about the nature of different cross-modal experiences and allows us to clearly formulate competing hypotheses about the kind of experiences involved in different cross-modal phenomena.

I argue that we should reject the sparse view that there are or could be only a small number of rather distinct senses. When one appreciates this then one can see that there is no need to choose between the standard criteria that have been proposed as ways of individuating the senses—representation, phenomenal character, proximal stimulus and sense organ—or any other criteria that one may deem important. Rather, one can use these criteria in conjunction to form a fine-grained taxonomy of (...) the senses. We can think of these criteria as defining a multidimensional space within which we can locate each of the senses that we are familiar with and which also defines the space of possible senses there could be. (shrink)

Artificial intelligence research has foundered on the issue of representation. When intelligence is approached in an incremental manner, with strict reliance on interfacing to the real world through perception and action, reliance on representation disappears. In this paper we outline our approach to incrementally building complete intelligent Creatures. The fundamental decomposition of the intelligent system is not into independent information processing units which must interface with each other via representations. Instead, the intelligent system is decomposed into independent and parallel activity (...) producers which all interface directly to the world through perception and action, rather than interface to each other particularly much. The notions of central and peripheral systems evaporateeverything is both central and peripheral. Based on these principles we have built a very successful series of mobile robots which operate without supervision as Creatures in standard office environments. (shrink)

In _Action in Perception _Alva No develops and presents a sensorimotor account of vision and of visual consciousness. According to such an account seeing (and indeed perceiving more generally) is analysed as a kind of skilful bodily activity. Such a view is consistent with the emerging emphasis, in both philosophy and cognitive science, on the critical role of embodiment in the construction of intelligent agency. I shall argue, however, that the full sensorimotor model faces three important challenges. The first is (...) to negotiate a path between two prima facie unsatisfactory readings of the central claim that conscious perceptual experience is constituted by knowledge of patterns of sensorimotor dependence. The second is to convince us that the sensorimotor contribution, in such cases, is actually constitutive of perceptual experience rather than merely causally implicated in the origination of such experience.² And the third is to respond to the important challenge raised by what I will dub 'sensorimotor summarizing' models of the relation between conscious experience and richly detailed sensorimotor routines. According to such models³ conscious perceptual experience only rather indirectly reflects the rich detail of our actual sensorimotor engagements, which are instead lightly sampled as a coarse guide, optimized for planning and reasoning, and geared and filtered according to current needs and purposes. (shrink)

A traditional view of perception and action makestwo assumptions: that the causal flow betweenperception and action is primarily linear or one-way,and that they are merely instrumentally related toeach other, so that each is a means to the other.Either or both of these assumptions can be rejected. Behaviorism rejects the instrumental but not theone-way aspect of the traditional view, thus leavingitself open to charges of verificationism. Ecologicalviews reject the one-way aspect but not theinstrumental aspect of the traditional view, so thatperception and (...) action are seen as instrumentallyinterdependent. It is argued here that a betteralternative is to reject both assumptions, resultingin a two-level interdependence view in whichperception and action co-depend on dynamicallycircular subpersonal relations and as a result may bemore than merely instrumentally interdependent. Thisis illustrated by reference to motor theories ofperception and control theories of action. (shrink)

Many current neurophysiological, psychophysical, and psychological approaches to vision rest on the idea that when we see, the brain produces an internal representation of the world. The activation of this internal representation is assumed to give rise to the experience of seeing. The problem with this kind of approach is that it leaves unexplained how the existence of such a detailed internal representation might produce visual consciousness. An alternative proposal is made here. We propose that seeing is a way of (...) acting. It is a particular way of exploring the environment. Activity in internal representations does not generate the experience of seeing. The out- side world serves as its own, external, representation. The experience of seeing occurs when the organism masters what we call the gov- erning laws of sensorimotor contingency. The advantage of this approach is that it provides a natural and principled way of accounting for visual consciousness, and for the differences in the perceived quality of sensory experience in the different sensory modalities. Sev- eral lines of empirical evidence are brought forward in support of the theory, in particular: evidence from experiments in sensorimotor adaptation, visual “filling in,” visual stability despite eye movements, change blindness, sensory substitution, and color perception. (shrink)

Brains, it has recently been argued, are essentially prediction machines. They are bundles of cells that support perception and action by constantly attempting to match incoming sensory inputs with top-down expectations or predictions. This is achieved using a hierarchical generative model that aims to minimize prediction error within a bidirectional cascade of cortical processing. Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to (...) adaptive success. This target article critically examines this approach, concluding that it offers the best clue yet to the shape of a unified science of mind and action. Sections 1 and 2 lay out the key elements and implications of the approach. Section 3 explores a variety of pitfalls and challenges, spanning the evidential, the methodological, and the more properly conceptual. The paper ends (sections 4 and 5) by asking how such approaches might impact our more general vision of mind, experience, and agency. (shrink)

We argue that brains generate predictions only within the constraints of the action repertoire. This makes the computational complexity tractable and fosters a step-by-step parallel development of sensory and motor systems. Hence, it is more of a benefit than a literal constraint and may serve as a universal normative principle to understand sensorimotor coupling and interactions with the world.

Sensory substitution devices provide through an unusual sensory modality (the substituting modality, e.g., audition) access to features of the world that are normally accessed through another sensory modality (the substituted modality, e.g., vision). In this article, we address the question of which sensory modality the acquired perception belongs to. We have recourse to the four traditional criteria that have been used to define sensory modalities: sensory organ, stimuli, properties, and qualitative experience (Grice, 1962), to which we have added the criteria (...) of behavioral equivalence (Morgan, 1977), dedication (Keeley, 2002), and sensorimotor equivalence (O’Regan & Noe¨, 2001). We discuss which of them are fulfilled by perception through sensory substitution devices and whether this favors the view that perception belongs to the substituting or to the substituted modality. Though the application of a number of criteria might be taken to point to the conclusion that perception with a sensory substitution device belongs to the substituted modality, we argue that the evidence leads to an alternative view on sensory substitution. According to this view, the experience after sensory substitution is a transformation, extension, or augmentation of our perceptual capacities, rather than being something equivalent or reducible to an already existing sensory modality. We develop this view by comparing sensory substitution devices to other ‘‘mind-enhancing tools’’ such as pen and paper, sketchpads, or calculators. An analysis of sensory substitution in terms of mind-enhancing tools unveils it as a thoroughly transforming perceptual experience and as giving rise to a novel form of perceptual interaction with the environment. (shrink)