This chapter distinguishes between two kinds of ordinary multisensory experience that go beyond mere co-consciousness of features (e.g., the experience that results from concurrently hearing a sound in the hallway and seeing the cup on the table). In one case, a sensory experience in one modality creates a perceptual demonstrative to whose referent qualities are attributed in another sensory modality. For example, when you hear someone speak, auditory experience attributes audible qualities to a seen event, a person’s speaking motions. The (...) second kind of multisensory experience attributes features experienced in several sensory modalities to one and the same object via a process of amodal perceptual integration, i.e., integration that occurs separately from processing within the individual sensory modalities. Multisensory experiences arising from holding and seeing a tomato or from seeing the Indian curry boil and smelling it are examples of the second kind of multisensory experience. At the end of the chapter we look at synesthesia, a kind of atypical multisensory experience, and argue that one version of this phenomenon may be able to shed light on the neural mechanism underlying amodally integrated multisensory experience. (shrink)

We can discern two opposing viewpoints regarding synesthesia. According to the first, it is an oddity, an outlier, or a disordered condition. According to the second, synesthesia is pervasive, driving creativity, metaphor, or language itself. Which is it? Ultimately, I favor the first perspective, according to which cross-sensory synesthesia is an outlying condition. But the second perspective is not wholly misguided. My discussion has three lessons. First, synesthesia is just one of a variety of effects in which one sense modality (...) causally impacts and reshapes experience associated with another. These effects are utterly common. However, due to their unfamiliarity, and to their conflict with a widespread conception of the role of the senses in perception and perceptual experience, until recently they have been surprising. Second, synesthesia nevertheless must be distinguished from other inter-modal effects that lead to misperception, such as crossmodal illusions. Third, synesthesia also may be distinguished from the potentially much broader class of synesthetic effects, which could be common across the population and within individuals. (shrink)

The aim of this paper is to show (a) how motion has ontological priority over space and (b) how the relationship between motion and form underpins mindness. The analysis of spatial perception reveals an interplay between spatial and temporal relations that is evident in language and surfing. Indo-Pacific Journal of Phenomenology , Volume 8, Edition 1 May 2008.

Consciousness is seen as a difficult “binding” problem. Binding, a process where different sensations evoked by an item are associated in the nervous system, can be viewed as a process similar to associative learning. Several reports that consciousness is associated with some form of memory imply that different forms of memories have a common feature contributing to consciousness. Based on a proposed synaptic mechanism capable of explaining different forms of memory, we developed a framework for consciousness. It is based on (...) the formation of semblance of sensory stimulus from (1) synaptic semblances when excitatory postsynaptic potentials arrive at functionally LINKed postsynaptic membranes, and (2) network semblances when these potentials summate to elicit action potential initiating activity in a network of neurons. It is then possible to derive a framework for consciousness as a multi-dimensional semblance. According to this framework, a continuum of semblances formed from background sensory stimuli and oscillating neuronal activities serve to maintain consciousness. Feasibility of this framework to explain various physiological and pathological states of consciousness, its subjective nature and qualia is examined. (shrink)

This non-article explores the limitations of applying brain science in “higher” disciplines. Many brain scientists believe that it is only a matter of time that everything human will be accounted for by the findings of brain science. Michael Polányi in the nineteen-sixties and recently Michael Gazzaniga argued against such determinism. They say that while “lower-level” processes constrain “higher-level” ones, they cannot determine them. The human mind is an emergent process, and it cannot be predicted from brain structure anymore than traffic (...) can be predicted from the structure of a car. I claim that in many instances, the application of brain science in psychology and literary studies merely re-states in brain-language what has already been said in psychology-language or literature-language. It can, however, be fruitfully applied when it refutes prevalent erroneous assumptions or resolves certain incongruities in the domain of “higher” disciplines. (shrink)

Studies of linguistic synaesthesias in English have shown a range of fine-grained language mechanisms governing the associations between colours on the one hand, and graphemes, phonemes and words on the other. However, virtually nothing is known about how synaesthetic colouring might operate in non-alphabetic systems. The current study shows how synaesthetic speakers of Mandarin Chinese come to colour the logographic units of their language. Both native and non-native Chinese speakers experienced synaesthetic colours for characters, and for words spelled in the (...) Chinese spelling systems of Pinyin and Bopomo. We assessed the influences of lexical tone and Pinyin/Bopomo spelling and showed that synaesthetic colours are assigned to Chinese words in a non-random fashion. Our data show that Chinese-speaking synaesthetes with very different native languages can exhibit both differences and similarities in the ways in which they come to colour their Chinese words. (shrink)

The Nagel inspired “something-it-is-like” conception of conscious experience remains a dominant approach in philosophy. In this paper I criticize a prevalent philosophical construal of SIL consciousness, one that understands SIL as a property of mental states rather than entities as a whole. I argue against thinking of SIL as a property of states, showing how such a view is in fact prevalent, under-warranted, and philosophically pernicious in that it often leads to an implausible reduction of conscious experience to qualia. I (...) then develop a holistic conception of SIL for entities and argue that it has at least equal pre-empirical warrant, is more conservative philosophically in that it decides less from the a priori “armchair,” and enjoys a fruitful two-way relationship with empirical work. (shrink)

What does it feel like to be a bat? Is conscious experience of echolocation closer to that of vision or audition? Or do bats process echolocation nonconsciously, such that they do not feel anything about echolocation? This famous question of bats' experience, posed by a philosopher Thomas Nagel in 1974, clarifies the difficult nature of the mind–body problem. Why a particular sense, such as vision, has to feel like vision, but not like audition, is totally puzzling. This is especially so (...) given that any conscious experience is supported by neuronal activity. Activity of a single neuron appears fairly uniform across modalities and even similar to those for non-conscious processing. Without any explanation on why a particular sense has to feel the way it does, researchers cannot approach the question of the bats' experience. Is there any theory that gives us a hope for such explanation? Currently, probably none, except for one. Integrated information theory has potential to offer a plausible explanation. IIT essentially claims that any system that is composed of causally interacting mechanisms can have conscious experience. And precisely how the system feels is determined by the way the mechanisms influence each other in a holistic way. In this article, I will give a brief explanation of the essence of IIT. Further, I will briefly provide a potential scientific pathway to approach bats' conscious experience and its philosophical implications. If IIT, or its improved or related versions, is validated enough, the theory will gain credibility. When it matures enough, predictions from the theory, including nature of bats' experience, will have to be accepted. I argue that a seemingly impossible question about bats' consciousness will drive empirical and theoretical consciousness research to make big breakthroughs, in a similar way as an impossible question about the age of the universe has driven modern cosmology. (shrink)

Synesthesia is a phenomenon in which an additional nonstandard perceptual experience occurs consistently in response to ordinary stimulation applied to the same or another modality. Recent studies suggest an important role of semantic representations in the induction of synesthesia. In the present proposal we try to link the empirically grounded theory of sensory-motor contingency and mirror system based embodied simulation to newly discovered cases of swimming-style color synesthesia. In the latter color experiences are evoked only by showing the synesthetes a (...) picture of a swimming person or asking them to think about a given swimming style. Neural mechanisms of mirror systems seem to be involved here. It has been shown that for mirror-sensory synesthesia, such as mirror-touch or mirror-pain synesthesia, concurrent experiences are caused by the overactivity in the mirror neuron system responding to the specific observation. The comparison of different forms of synesthesia has the potential of challenging conventional thinking on this phenomenon and providing a more general, sensory-motor account of synesthesia encompassing cases driven by semantic or emulational rather than pure sensory or motor representations. (shrink)

Following on from ecological theories of perception, such as the one proposed by [Gibson, J. J. . The senses considered as perceptual systems. Boston: Houghton Mifflin] this paper reviews the literature on the multisensory interactions underlying the perception of flavor in order to determine the extent to which it is really appropriate to consider flavor perception as a distinct perceptual system. We propose that the multisensory perception of flavor may be indicative of the fact that the taxonomy currently used to (...) define our senses is simply not appropriate. According to the view outlined here, the act of eating allows the different qualities of foodstuffs to be combined into unified percepts; and flavor can be used as a term to describe the combination of tastes, smells, trigeminal, and tactile sensations as well as the visual and auditory cues, that we perceive when tasting food. (shrink)

This review covers experimental approaches to sound-symbolism—from infants to adults, and from Sapir’s foundational studies to twenty-first century product naming. It synthesizes recent behavioral, developmental, and neuroimaging work into a systematic overview of the cross-modal correspondences that underpin iconic links between form and meaning. It also identifies open questions and opportunities, showing how the future course of experimental iconicity research can benefit from an integrated interdisciplinary perspective. Combining insights from psychology and neuroscience with evidence from natural languages provides us with (...) opportunities for the experimental investigation of the role of sound-symbolism in language learning, language processing, and communication. The review finishes by describing how hypothesis-testing and model-building will help contribute to a cumulative science of sound-symbolism in human language. (shrink)

Sound-symbolism is the nonarbitrary link between the sound and meaning of a word. Japanese-speaking children performed better in a verb generalization task when they were taught novel sound-symbolic verbs, created based on existing Japanese sound-symbolic words, than novel nonsound-symbolic verbs (Imai, Kita, Nagumo, & Okada, 2008). A question remained as to whether the Japanese children had picked up regularities in the Japanese sound-symbolic lexicon or were sensitive to universal sound-symbolism. The present study aimed to provide support for the latter. In (...) a verb generalization task, English-speaking 3-year-olds were taught novel sound-symbolic verbs, created based on Japanese sound-symbolism, or novel nonsound-symbolic verbs. English-speaking children performed better with the sound-symbolic verbs, just like Japanese-speaking children. We concluded that children are sensitive to universal sound-symbolism and can utilize it in word learning and generalization, regardless of their native language. (shrink)

What kinds of mental states can be based on epistemic reasons? The standard answer is only beliefs. I argue that perceptual states can also be based on reasons, as the result of crossmodal interactions. A perceptual state from one modality can provide a reason on which an experience in another modality is based. My argument identifies key markers of the basing relation and locates them in the crossmodal Marimba Illusion (Schutz & Kubovy 2009). The subject’s auditory experience of musical tone (...) duration is based on the reason provided by her visual representation of the length of the musician’s gesture and other stored perceptual principles. (shrink)

The aim of this article is to give both a sustained interpretation of Wittgenstein’s obscure remarks on the experience of meaning of language, synthaesthesia and secondary use and to apply his insights to recent philosophical discussions about synthaesthesia. I argue that synthaesthesia and experience of meaning are conceptually related to aspect-seeing. The concept of aspect-seeing is not reducible to either seeing or imaging but involves a modified notion of experience. Likewise, synthaesthesia involves a modified notion of experience. In particular, the (...) concept of synthaesthesia involves a secondary use of ‘experience’ and hence is intrinsically dependent on the primary use of language. Recent discussions tend to overlook this distinction between the primary and secondary use of language. (shrink)

Our target article argued that a genetically specified Universal Grammar (UG), capturing arbitrary properties of languages, is not tenable on evolutionary grounds, and that the close fit between language and language learners arises because language is shaped by the brain, rather than the reverse. Few commentaries defend a genetically specified UG. Some commentators argue that we underestimate the importance of processes of cultural transmission; some propose additional cognitive and brain mechanisms that may constrain language and perhaps differentiate humans from nonhuman (...) primates; and others argue that we overstate or understate the case against co-evolution of language genes. In engaging with these issues, we suggest that a new synthesis concerning the relationship between brains, genes, and language may be emerging. (shrink)

Experiencing a stimulus in one sensory modality is often associated with an experience in another sensory modality. For instance, seeing a lemon might produce a sensation of sourness. This might indicate some kind of cross-modal correspondence between vision and gustation. The aim of the current study was to provide explore whether such cross-modal correspondences influence cross-modal integration during perceptual learning. To that end, we conducted 2 experiments. Using a speeded classification task, Experiment 1 established a cross-modal correspondence between visual lightness (...) and the frequency of an auditory tone. Using a short-term priming procedure, Experiment 2 showed that manipulation of such cross-modal correspondences led to the creation of a crossmodal unit regardless of the nature of the correspondence (i.e., congruent, Experiment 2a or incongruent, Experiment 2b). However, a comparison of priming-effects sizes suggested that cross-modal correspondences modulate cross-modal integration during learning and thus leading to new learned units that have different stability over time. We discuss the implications of our results for the relation between cross-modal correspondence and perceptual learning in the context of a Bayesian explanation of cross-modal correspondences. (shrink)

Back when researchers thought about the various forms that color vision could take, the focus was primarily on the retinal mechanisms. Since that time, research on human color vision has shifted from an interest in retinal mechanisms to cortical color processing. This has allowed color research to provide insight into questions that are not limited to early vision but extend to cognition. Direct cortical connections from higher-level areas to lower-level areas have been found throughout the brain. One of the classic (...) questions in cognitive science is whether perception is influenced, and if so to what extent, by cognition and whether a clear distinction can be drawn between perception and cognition. Since perception is seen as providing justification for our beliefs about properties in the external world, these questions also have metaphysical and epistemological significance. The aim of this paper is to highlight some of the areas where research on color perception can shed new light on questions in the cognitive sciences. A further aim of the paper is to raise some questions about color research that are in dire need of further reflection and investigation. (shrink)

Grapheme-color synesthetes experience color, not physically present, when viewing symbols. Synesthetes cannot remember learning these associations. Must synesthetic percepts be formed during a sensitive period? Can they form later and be consistent? What determines their nature? We tested grapheme-color synesthete, MC2, before, during and after she studied Hindi abroad. We investigated whether novel graphemes elicited synesthetic percepts, changed with familiarity, and/or benefited from phonemic information. MC2 reported color percepts to novel Devanagari and Hebrew graphemes. MC2 monitored these percepts over 6 (...) months in a Hindi-speaking environment. MC2 and synesthete DN, reported synesthetic percepts for Armenian graphemes, or Cyrillic graphemes + phonemes over time. Synesthetes, not controls, reported color percepts for novel graphemes that gained consistency over time. Phonemic information did not enhance consistency. Thus, synesthetes can form and consolidate percepts to novel graphemes as adults. These percepts may depend on pre-existing grapheme-color relationships but they can flexibly shift with familiarity. (shrink)

It is being increasingly recognized that the Saussurean dictum of “the arbitrariness of the linguistic sign” is in conflict with the pervasiveness of the phenomenon commonly known as “sound symbolism”. After first presenting a historical overview of the debate, however, we conclude that both positions have been exaggerated, and that an adequate explanation of sound symbolism is still lacking. How can there, for example, be (perceived) similarity between expressionsand contents across different sensory modalities? We offer an answer, based on the (...) Peircian notion of iconic ground, and G. Sonesson’s distinction betweenprimary and secondary iconicity. Furthermore, we describe an experimental study, in a paradigm first pioneered by W. Kohler, and recently popularized by V.Ramachandran, in which we varied vowels and consonants in fictive word-forms, and conclude that both types of sounds play a role in perceiving an iconic ground between the word-forms and visual figures. The combination of historical conceptual analysis, semiotic explication and psychological experimentationpresented in this article is characteristic of the emerging paradigm of cognitive semiotics. (shrink)

I argue that an evolutionary adaptation for bodily mimesis, the volitional use of the body as a representational devise, is the “small difference” that gave rise to unique and yet pre-linguistic features of humanity such as imitation, pedagogy, intentional communication and the possibility of a cumulative, representational culture. Furthermore, it is this that made the evolution of language possible. In support for the thesis that speech evolved atop bodily mimesis and a transitional multimodal protolanguage, I review evidence for the extensive (...) presence of sound-symbolism in modern languages, for its psychological reality in adults, and for its contribution to language acquisition in children. On a meta-level, the argument is that dividing human cognitive-semiotic evolution into a sequence of stages is crucial for resolving classical dichotomies concerning human nature and language, which are both natural and cultural, both continuous with and discontinuous from those of animals. (shrink)

Synesthesia is a neurological phenomenon in which a single stimulus typically associated with one sensory modality automatically and involuntarily produces sensations not typically associated with that modality. I argue that synesthesia elucidates how two naturalistic theories of representation and phenomenal experience conflict. Strong representationalism holds that what an experience is like is determined by the experience’s representational content. Informational semantics holds that representational content is determined by causal co-variation between a representation and an external object or property. I argue that (...) according to informational semantics, synesthetes and normal perceivers represent the same content in different ways. However, according to strong representationalism, two experiences with the same content must be represented in the same way. Therefore, if strong representationalists want to account for synesthesia, they cannot hold onto informational semantics as a theory of mental content. (shrink)

Synaesthesia is considered a rare perceptual capacity, and one that is not capable of cultivation. However, meditators report the experience quite commonly, and in questionnaire surveys, respondents claimed to experience synaesthesia in 35% of meditation retreatants, in 63% of a group of regular meditators, and in 86% of advanced teachers. These rates were significantly higher than in nonmeditator controls, and displayed significant correlations with measures of amount of meditation experience. A review of ancient texts found reports suggestive of synaesthesia in (...) advanced meditators from India and China. These findings suggest that synaesthesia may be cultivated by meditation, and that laboratory studies of meditators could be rewarding. (shrink)

This article supplements our earlier paper on synaesthesia published in JCS (Ramachandran & Hubbard, 2001a). We discuss the phenomenology of synaesthesia in greater detail, raise several new questions that have emerged from recent studies, and suggest some tentative answers to these questions.