Many current neurophysiological, psychophysical, and psychological approaches to vision rest on the idea that when we see, the brain produces an internal representation of the world. The activation of this internal representation is assumed to give rise to the experience of seeing. The problem with this kind of approach is that it leaves unexplained how the existence of such a detailed internal representation might produce visual consciousness. An alternative proposal is made here. We propose that seeing is a way of (...) acting. It is a particular way of exploring the environment. Activity in internal representations does not generate the experience of seeing. The out- side world serves as its own, external, representation. The experience of seeing occurs when the organism masters what we call the gov- erning laws of sensorimotor contingency. The advantage of this approach is that it provides a natural and principled way of accounting for visual consciousness, and for the differences in the perceived quality of sensory experience in the different sensory modalities. Sev- eral lines of empirical evidence are brought forward in support of the theory, in particular: evidence from experiments in sensorimotor adaptation, visual “filling in,” visual stability despite eye movements, change blindness, sensory substitution, and color perception. (shrink)

A new class of visuomotor neuron has been recently discovered in the monkey’s premotor cortex: mirror neurons. These neurons respond both when a particular action is performed by the recorded monkey and when the same action, performed by another individual, is observed. Mirror neurons appear to form a cortical system matching observation and execution of goal-related motor actions. Experimental evidence suggests that a similar matching system also exists in humans. What might be the functional role of this matching system? One (...) possible function is to enable an organism to detect certain mental states of observed conspecifics. This function might be part of, or a precursor to, a more general mind-reading ability. Two different accounts of mind-reading have been suggested. According to “theory theory‘, mental states are represented as inferred posits of a naive theory. According to “simulation theory‘, other people’s mental states are represented by adopting their perspective: by tracking or matching their states with resonant states of one’s own. The activity of mirror neurons, and the fact that observers undergo motor facilitation in the same muscular groups as those utilized by target agents, are findings that accord well with simulation theory but would not be predicted by theory theory. (shrink)

The aim of this paper is to study interpretation, specifically, to work toward an account of interpretation that seems descriptively and explanatorily correct. No account of interpretation can be philosophically helpful, I submit, if it is incompatible with a correct account of what people actually do when they interpret others. My question, then, is: how does the (naive) interpreter arrive at his/her judgments about the mental attitudes of others? Philosophers who have addressed this question have not, in my view, been (...) sufficiently psychological, or cognitivist, even those who are otherwise psychologically inclined. I shall defend some proposals about the activity of interpretation that are, I believe, psychologically more realistic than their chief competitors. (shrink)

The emulation theory of representation is developed and explored as a framework that can revealingly synthesize a wide variety of representational functions of the brain. The framework is based on constructs from control theory (forward models) and signal processing (Kalman filters). The idea is that in addition to simply engaging with the body and environment, the brain constructs neural circuits that act as models of the body and environment. During overt sensorimotor engagement, these models are driven by efference copies in (...) parallel with the body and environment, in order to provide expectations of the sensory feedback, and to enhance and process sensory information. These models can also be run off-line in order to produce imagery, estimate outcomes of different actions, and evaluate and develop motor plans. The framework is initially developed within the context of motor control, where it has been shown that inner models running in parallel with the body can reduce the effects of feedback delay problems. The same mechanisms can account for motor imagery as the off-line driving of the emulator via efference copies. The framework is extended to account for visual imagery as the off-line driving of an emulator of the motor-visual loop. I also show how such systems can provide for amodal spatial imagery. Perception, including visual perception, results from such models being used to form expectations of, and to interpret, sensory input. I close by briefly outlining other cognitive functions that might also be synthesized within this framework, including reasoning, theory of mind phenomena, and language. Key Words: efference copies; emulation theory of representation; forward models; Kalman filters; motor control; motor imagery; perception; visual imagery. (shrink)

Stephen Stich and Shaun Nichols advance the debate over folk psychology with their vivid depiction of the contest between the simulation theory and the theory-theory (Stich & Nichols, this issue). At least two aspects of their presentation I find highly congenial. First, they give a generally fair characterization of the simulation theory, in some respects even improving its formulation. Though I have a few minor quarrels with their formulation, it is mostly quite faithful to the version which I have found (...) attractive (Goldman, 1989).Second, I concur with Stich and Nichols in their assertion that the theory-theory and the simulation theory are the only two games in town. Where I disagree, of course, is in Stich and Nichols’s contention that their arguments so dim the prospects of the simulation theory that it should no longer be taken seriously. On the contrary, I shall maintain, the simulation theory remains a very plausible contender, and should receive careful and respectful consideration in the continuing investigation. (shrink)

Genes adjacent to species-specific loci are 6.2% older than genes adjacent to other dynamic loci (P < 10−2 by randomization; gray bars in Fig. 3); thus, species-specific genes are not randomly distributed but are found preferentially in the older regions, indicating that the incipient Escherichia and Salmonella lineages continued to participate in recombination at loci unlinked to lineage-specific genes.

and apply it to various examples of neural plasticity in which input is rerouted intermodally or intramodally to nonstandard cortical targets. In some cases but not others, cortical activity ‘defers’ to the nonstandard sources of input. We ask why, consider some possible explanations, and propose a dynamic sensorimotor hypothesis. We believe that this distinction is important and worthy of further study, both philosophical and empirical, whether or not our hypothesis turns out to be correct. In particular, the question of how (...) the distinction should be explained is linked to explanatory gap issues for consciousness. Comparative and absolute explanatory gaps should be distinguished: why does neural activity in a particular area of cortex have this qualitative expression rather than that, and why does it have any qualitative expression at all? We use the dominance/deference distinction to address the comparative gaps, both intermodal and intramodal. We do so not by inward scrutiny but rather by expanding our gaze to include relations between brain, body and environment. (shrink)

Simulation theories of social cognition abound in the literature, but it is often unclear what simulation means and how it works. The discovery of mirror neurons, responding both to action execution and observation, suggested an embodied approach to mental simulation. Over the last years this approach has been hotly debated and alternative accounts have been proposed. We discuss these accounts and argue that they fail to capture the uniqueness of embodied simulation (ES). ES theory provides a unitary account of basic (...) social cognition, demonstrating that people e their own mental states or processes represented with a bodily format in functionally attributing them to others. (shrink)

Evidence from functional neuroimaging of the human brain indicates that information about salient properties of an object¿such as what it looks like, how it moves, and how it is used¿is stored in sensory and motor systems active when that information was acquired. As a result, object concepts belonging to different categories like animals and tools are represented in partially distinct, sensory- and motor property-based neural networks. This suggests that object concepts are not explicitly represented, but rather emerge from weighted activity (...) within property-based brain regions. However, some property-based regions seem to show a categorical organization, thus providing evidence consistent with category-based, domain-specific formulations as well.Acronyms and DefinitionsBiological motion: motion of animate agents characterized by highly flexible, fully articulated motion vectors, in contrast to the rigid, unarticulated motion vectors associated with most tools.Category-specific disorder: a relatively greater impairment in retrieving information about members of one superordinate object category (e.g., animals) as compared with other categories following brain injury or diseaseIPS: intraparietal sulcusLO: lateral occipital cortexObject concept: memory representations of a class or category of objects. Necessary for numerous cognitive functions including identifying an object as a member of a specific category and drawing inferences about object propertiespMTG: posterior middle temporal gyruspSTS: posterior superior temporal sulcusRepetition suppression: decreased neural response associated with repeated presentation of an identical, or a semantically/conceptually related, stimulusSD: semantic dementiaSemantic memory: a large division of long-term memory containing knowledge about the world including facts, ideas, beliefs, and conceptsSemantic priming: a short-lasting facilitation in processing a stimulus due to the prior presentation of a semantically related stimulusTMS: transcranial magnetic stimulationVPMC: ventral premotor cortex. (shrink)

The aim of this paper is to defend the structural concept of representation, as defined by homomorphisms, against its main objections, namely: logical objections, the objection from misrepresentation, theobjection from failing necessity, and the copy theory objection. The logical objections can be met by reserving the relation.

The aim of this paper is to defend the structural concept of representation, as defined by homomorphisms, against its main objections, namely: logical objections, the objection from misrepresentation, theobjection from failing necessity, and the copy theory objection. The logical objections can be met by reserving the relation ‘to be homomorphic to’ for the explication of potential representation (or, of the representational content). Actual reference objects (‘targets’) of representations are determined by (intentional or causal) representational mechanisms. Appealing to the independence of (...) the dimensions of ‘content’ and ‘target’ also helps to see how the structural concept can cope with misrepresentation. Finally, I argue that homomorphic representations are not necessarily ‘copies’ of their representanda, and thus can convey scientific insight. (shrink)

We studied two otherwise normal, synaesthetic subjects who `saw' a speci¢c colour every time they saw a speci¢c number or letter. We conducted four experiments in order to show that this was a genuine perceptual experience rather than merely a memory association. (i)The synaesthetically induced colours could lead to perceptual grouping, even though the inducing numerals or letters did not. (ii)Synaesthetically induced colours were not experienced if the graphemes were presented peripherally. (iii)Roman numerals were ine¡ective: the actual number grapheme was (...) required. (iv)If two graphemes were alternated the induced colours were also seen in alternation. However, colours were no longer experienced if the graphemes were alternated at more than 4 Hz. We propose that grapheme colour synaesthesia arises from `cross-wiring' between the `colour centre' (area V4 or V8)and the `number area', both of which lie in the fusiform gyrus. We also suggest a similar explanation for the representation of metaphors in the brain: hence, the higher incidence of synaesthesia among artists and poets. (shrink)

To have an ontology is to interpret a world. In this paper we argue that the brain, viewed as a representational system aimed at interpreting our world, possesses an ontology too. It creates primitives and makes existence assumptions. It decomposes target space in a way that exhibits a certain invariance, which in turn is functionally significant. We will investigate which are the functional regularities guiding this decomposition process, by answering to the following questions: What are the explicit and implicit assumptions (...) about the structure of reality, which at the same time shape the causal profile of the brain's motor output and its representational deep structure, in particular of the conscious mind arising from it (its ''phenomenal output'')? How do they constrain high-level phenomena like conscious experience, the emergence of a first-person perspective, or social cognition? By reviewing a series of neuroscientific results and integrating them with a wider philosophical perspective, we will emphasize the contribution the motor system makes to this process. As it will be shown, the motor system constructs goals, actions, and intending selves as basic constituents of the world it interprets. It does so by assigning a single, unified causal role to them. Empirical evidence demonstrates that the brain models movements and action goals in terms of multimodal representations of organism-object-relations. Under a representationalist analysis, this process can be conceived of as an internal, dynamic representation of the intentionality-relation itself. We will show how such a complex form of representational content, once it is in place, can later function as a functional building block for social cognition and for a more complex, consciously experienced representation of the first-person perspective as well. (shrink)

The representational dynamics of the brain is a subsymbolic process, and it has to be conceived as an "agent-free" type of dynamical self-organization. However, in generating a coherent internal world-model, the brain decomposes target space in a certain way. In doing so, it defines an "ontology": to have an ontology is to interpret a world. In this paper we argue that the brain, viewed as a representational system aimed at interpreting the world, possesses an ontology too. It decomposes target space (...) in a way that exhibits certain invariances, which in turn are functionally significant. A challenge for empirical research is to determine which are the functional regularities guiding this decomposition process. What are the explicit and implicit assumptions about the structure of reality, which at the same time shape the causal profile of the brain's motor output and the representational deep structure of the conscious mind arising from it (its "phenomenal output")? How do they constrain high-level phenomena like conscious experience, the emergence of a first-person perspective, or social cognition? By reviewing a series of neuroscientific results, we focus on the contribution the motor system makes to this process. As it turns out, the motor system constructs goals, actions, and intending selves as basic constituents of the world it interprets. It does so by assigning a single, unified causal role to them. Empirical evidence now clearly shows how the brain actually codes movements and action goals in terms of multimodal representations of organism-object relations. Under a representationalist analysis, this process can be interpreted as an internal representation of the intentionality relation itself. We try to show how such a more complex form of representational content, once it is in place, can later function as the building block for social cognition and a for more complex, consciously experienced representation of the first-person perspective as well. The motor system may therefore play a decisive role in understanding how the functional ontology of the human brain could be gradually extended into the subjective and social domains. (shrink)

Contemporary philosophical and scienti .c discussions of mind developed from a 'proto-concept of mind ',a mythical,tradition- alistic,animistic and quasi-sensory theory about what it means to have a mind. It can be found in many di .erent cultures and has a semantic core corresponding to the folk-phenomenological notion of a 'soul '.It will be argued that this notion originates in accurate and truthful .rst-person reports about the experiential content of a special neurophenomenological state-class called 'out-of-body experiences '.They can be undergone by (...) every human being and seem to possess a culturally invariant cluster of functional and phenomenal core properties similar to the proto-concept of mind. The common causal factor in the emergence and development of the notion of the soul and the proto-concept of mind may consist in a yet to be determined set of properties realized by the human brain, underlying the cluster of phenomenal properties described in the relevant first-person reports. This hypothesis suggests that such a neurofunctional substrate ed human beings at different times, and in widely varying cultural contexts, to postulate the existence of a soul and to begin developing a theory of mind. (shrink)

This book contains a representationalist theory of self-consciousness and of the phenomenal first-person perspective. It draws on empirical data from the cognitive and neurosciences.

We examine a condition in which units of time, such as months of the year, are associated with specific locations in space. For individuals with this time-space synaesthesia, contiguous time units such as months are spatially linked forming idiosyncratically shaped patterns such as ovals, oblongs or circles. For some individuals, each time unit appears in a highly specific colour. For instance, one of the synaesthetes we studied experienced December as a red area located at arms length to the left of (...) their body. For the same individual May was a blue area located roughly at arms length to the right of their body. We studied four synaesthetes who report spatial associations for the months of the year. We found that the time-space associations experienced by these individuals were consistent across test–retest. In addition, month names directed visual attention to particular locations in space. For some synaesthetes, this directing of spatial attention was quite rapid-in accord with their reports that month names involuntarily bring to mind spatial locations. (shrink)

This article proposes a neurobiologically motivated theory of meaning as internal representation that holds on to the principle of compositionality, but negates the principle of semantic constituency. The approach builds on neurobiological findings regarding topologically structured cortical feature maps and the mechanism of object-related binding by neuronal synchronization. It incorporates the Gestalt principles of psychology and is implemented by recurrent neural networks. The semantics to be developed is structurally analogous to some variant of model-theoretical semantics. The semantics to be developed (...) is a neuro-emulative model-theoretical semantics of a first-order language. The model-theoretical semantics is merely denotational and does not imply anything about the structures of the mind or the underlying neural mechanisms that enable producing and comprehend meaningful expressions. The relation between a mental representation and its content is some form of causal-informational covariation. (shrink)

The Kiki-Bouba effect comprises a relation between two abstract figures and two non-words: the star-shaped figure is called 'Kiki' and the rounded figure 'Bouba'. The effect is explained by a sound-vision synaesthesia: certain sounds are associated with certain shapes in a non-arbitrary manner.When we asked the participants to decide which of the two figures, the star-shaped or the rounded one, to call yin and which yang, some 85% choose the star-shaped figure as yin. There are previous cases of synaesthesia where (...) personality is attributed to numbers or letters. In our results, the word Kiki is overall happy, clever, small, thin, young, unpleasant, and nervous. The starshaped figure is overall clever, tall, small, slim, nervous, unpleasant, and upper-class. That is, the correspondence above all concerns the qualifying adjectives clever, unpleasant, and nervous, as well as the physical appearance small and thin. This brings us to the fat-thin effect. Cinema, literature, comics, and children's programmes are full of contrasting figures: Don Quixote and Sancho Panza, Laurel and Hardy , Asterix and Obelix, Tintin and Captain Haddock, Bert and Ernie , or the Spanish comic about very naughty twin boys called Zipi and Zape . Our main conclusion is that first names and last names are not entirely arbitrary. There is a correspondence between names and physical characteristics and concepts . The Kiki-Bouba effect is a semantic one. (shrink)

Is synaesthesia cognitively useful? Individuals with time–space synaesthesia experience time units as idiosyncratic spatial forms, and report that these forms aid them in mentally organising their time. In the present study, we hypothesised that time–space synaesthesia would facilitate performance on a time-related cognitive task. Synaesthetes were not specifically recruited for participation; instead, likelihood of time–space synaesthesia was assessed on a continuous scale based on participants’ responses during a semi-structured interview. Participants performed a month-manipulation task, which involved naming every second month (...) or every third month in reverse-chronological order, beginning and ending with a target month. Using hierarchical multiple regression, we found that time–space synaesthesia corresponded with faster performance on both versions of the task. We propose that time–space synaesthesia may expedite the cognitive manipulation of time-based information. Our results also indicate that synaesthesia is far less unusual than widely believed. (shrink)

In this paper we wish to bring together two seemingly independent areas of research: synaesthesia and sensory substitution. Synaesthesia refers to a rare condition where a sensory stimulus elicits not only the sensation that stimulus evokes in its own modality, but an additional one; a synaesthete may thus hear the word “Monday”, and, in addition to hearing it, have a concurrent visual experience of a red color. Sensory substitution, in contrast, attempts to substitute a sensory modality that a person has (...) lost by transforming the information it provided so that it can be accessed through another, intact sensory modality. To make visual information accessible to a blind person, for example, data taken by a camera would be transformed into tactile or auditory information. What do synaesthesia and sensory substitution have in common? Research in both of these areas contributes to our understanding both of cross-modal cooperation and of sensory sensations or qualia, asking under what circumstances these can arise in a modality that is not stimulated. Synaesthesia reveals that this “sensory cross-activation” is possible, and sensory substitution research hopes to induce it. In this article, we will review briefly the literature on synaesthesia, and discuss the issue of qualia for this domain of research. We will then address the evidence for synaesthesia and visual qualia in the blind, and the research on sensory substitution, to finally ask whether sensory substitution may induce a ‘synthetic’ form of synaesthesia by taking advantage of the nervous system’s capacity for generating visual images in the absence of retinal input. (shrink)

Mirror-touch synaesthesia is a condition where observing touch to another’s body induces a subjective tactile sensation on the synaesthetes body. The present study explores which characteristics of the inducing stimulus modulate the synaesthetic touch experience. Fourteen mirror-touch synaesthetes watched videos depicting a touch event while indicating whether the video induced a tactile sensation, on which side of their body they felt this sensation and the intensity of the experienced sensation. Results indicate that the synaesthetes experience stronger tactile sensations when observing (...) touch to real bodies, whereas observing touch to dummy bodies, pictures of bodies and disconnected dummy body parts elicited weaker sensations. These results suggest that mirror-touch synaesthesia is not entirely bottom-up driven, but top-down information, such as knowledge about real and dummy body parts, also modulate the intensity of the experience. (shrink)

The Complex-First Paradox consists in a set of collectively incompatible but individually well-confirmed propositions that regard the evolution, development, and cortical realization of the meanings of concrete nouns. Although these meanings are acquired earlier than those of other word classes, they are semantically more complex and their cortical realizations more widely distributed. For a neurally implemented syntaxsemantics interface, it should thus take more effort to establish a link between a concept and its lexical expression. However, in ontogeny and phylogeny, capabilities (...) demanding more effort, ceteris paribus, develop and evolve later than those demanding less effort. The paradox points to an explanatory deficit in linguistic theory. (shrink)

The paper argues that cognitive states of biological systems are inherently temporal. Three adequacy conditions for neuronal models of representation are vindicated: the compositionality of meaning, the compositionality of content, and the co-variation with content. Classicist and connectionist approaches are discussed and rejected. Based on recent neurobiological data, oscillatory networks are introduced as a third alternative. A mathematical description in a Hilbert space framework is developed. The states of this structure can be regarded as conceptual representations satisfying the three conditions.

We investigated grapheme-colour synaesthesia and found that The induced colours led to perceptual grouping and pop-out, a number rendered invisible through 'crowding' or lateral masking can induce synaesthetic colours -- a form of blindsight -- and peripherally presented graphemes did not induce colours even when they were clearly visible. Taken collectively, these and other experiments prove conclusively that synaesthesia is a genuine perceptual phenomenon, not an effect based on memory associations from childhood or on vague metaphorical speech. We identify different (...) subtypes of number-colour synaesthesia and propose that they are caused by hyperconnectivity between colour and number areas at different stages in processing; lower synaesthetes may have cross-wiring within the fusiform gyrus whereas higher synaesthetes may have cross-activation in the angular gyrus. This hyperconnectivity might be caused by a genetic mutation that causes defective pruning of connections between brain maps. The mutation may further be expressed selectively in the fusiform or angular gyri, and this may explain the existence of different forms of synaesthesia. If expressed very diffusely, there may be extensive cross-wiring between brain regions that represent abstract concepts, which would explain the link between creativity, metaphor and synaesthesia. Also, hyperconnectivity between the sensory cortex and amygdala would explain the heightened aversion synaesthetes experience when seeing numbers printed in the 'wrong' colour. Lastly, kindling -- induced hyperconnectivity in the temporal lobes of temporal lobe epilepsy patients -- may explain the purported higher incidence of synaesthesia in these patients. We conclude with a synaesthesia-based theory of the evolution of language. Thus, our experiments on synaesthesia and our theoretical framework attempt to link several seemingly unrelated facts about the human mind. Far from being a mere curiosity, synaesthesia may provide a window into perception, thought and language. (shrink)

Frames, i.e., recursive attribute-value structures, are a general format for the decomposition of lexical concepts. Attributes assign unique values to objects and thus describe functional relations. Concepts can be classified into four groups: sortal, individual, relational and functional concepts. The classification is reflected by different grammatical roles of the corresponding nouns. The paper aims at a cognitively adequate decomposition, particularly, of sortal concepts by means of frames. Using typed feature structures, an explicit formalism for the characterization of cognitive frames is (...) developed. The frame model can be extended to account for typicality effects. Applying the paradigm of object-related neural synchronization, furthermore, a biologically motivated model for the cortical implementation of frames is developed. Cortically distributed synchronization patterns may be regarded as the fingerprints of concepts. (shrink)

In a synesthetic metaphor, a certain perceptual mode is initially specified, but the imagery is linguistically related in terms belonging to one or more differing perceptual modes. Commonplace examples of synesthetic metaphors in English include phrases such as "loud colors", "dark sounds", and "sweet smells". Tabulations of the frequency of types of synesthesia and synesthetic metaphors in English reveals that for physiological synesthesia, colored sounds are most common; in English literature, synesthetic metaphors employed for descriptions of tactile sound predominate. Of (...) the various senses, hearing is most frequently expanded and elaborated upon by both synesthetic sensory perceptions and synesthetic metaphors. Synesthetic "visual hearing", which antedates language, may have influenced language development. (shrink)