What is the relationship between space and time in the human mind? Studies in adults show an asymmetric relationship between mental representations of these basic dimensions of experience: Representations of time depend on space more than representations of space depend on time. Here we investigated the relationship between space and time in the developing mind. Native Greek‐speaking children watched movies of two animals traveling along parallel paths for different distances or durations and judged the spatial and temporal aspects of these (...) events (e.g., Which animal went for a longer distance, or a longer time?). Results showed a reliable cross‐dimensional asymmetry. For the same stimuli, spatial information influenced temporal judgments more than temporal information influenced spatial judgments. This pattern was robust to variations in the age of the participants and the type of linguistic framing used to elicit responses. This finding demonstrates a continuity between space‐time representations in children and adults, and informs theories of analog magnitude representation. (shrink)

Paintings of music are a significant presence in modern art. They are cross-modal representations, aimed at representing music, say, musical works or forms, using colors, lines, and shapes in the visual modality. This article aims to provide a conceptual framework for understanding paintings of music. Using examples from modern art, the article addresses the question of what a painting of music is. Implications for the aesthetic appreciation of paintings of music are also drawn.

The last couple of years have seen a rapid growth of interest in the study of crossmodal correspondences – the tendency for our brains to preferentially associate certain features or dimensions of stimuli across the senses. By now, robust empirical evidence supports the existence of numerous crossmodal correspondences, affecting people’s performance across a wide range of psychological tasks – in everything from the redundant target effect paradigm through to studies of the Implicit Association Test, and from speeded discrimination/classification tasks through (...) to unspeeded spatial localisation and temporal order judgment tasks. However, one question that has yet to receive a satisfactory answer is whether crossmodal correspondences automatically affect people’s performance , as opposed to reflecting more of a strategic, or top-down, phenomenon. Here, we review the latest research on the topic of crossmodal correspondences to have addressed this issue. We argue that answering the question will require researchers to be more precise in terms of defining what exactly automaticity entails. Furthermore, one’s answer to the automaticity question may also hinge on the answer to a second question: Namely, whether crossmodal correspondences are all ‘of a kind’, or whether instead there may be several different kinds of crossmodal mapping . Different answers to the automaticity question may then be revealed depending on the type of correspondence under consideration. We make a number of suggestions for future research that might help to determine just how automatic crossmodal correspondences really are. (shrink)

TEST is a novel taxonomy of knowledge representations based on three distinct hierarchically organized representational features: Tropism, Embodiment, and Situatedness. Tropic representational features reflect constraints of the physical world on the agent's ability to form, reactivate, and enrich embodied (i.e., resulting from the agent's bodily constraints) conceptual representations embedded in situated contexts. The proposed hierarchy entails that representations can, in principle, have tropic features without necessarily having situated and/or embodied features. On the other hand, representations that are situated and/or embodied (...) are likely to be simultaneously tropic. Hence, although we propose tropism as the most general term, the hierarchical relationship between embodiment and situatedness is more on a par, such that the dominance of one component over the other relies on the distinction between offline storage versus online generation as well as on representation-specific properties. (shrink)

Height-pitch associations are claimed to be universal and independent of language, but this claim remains controversial. The present study sheds new light on this debate with a multimodal analysis of individual sound and melody descriptions obtained in an interactive communication paradigm with speakers of Dutch and Farsi. The findings reveal that, in contrast to Dutch speakers, Farsi speakers do not use a height-pitch metaphor consistently in speech. Both Dutch and Farsi speakers’ co-speech gestures did reveal a mapping of higher pitches (...) to higher space and lower pitches to lower space, and this gesture space-pitch mapping tended to co-occur with corresponding spatial words (high-low). However, this mapping was much weaker in Farsi speakers than Dutch speakers. This suggests that cross-linguistic differences shape the conceptualization of pitch and further calls into question the universality of height-pitch associations. (shrink)

Conceptual congruency effects are biases induced by an irrelevant conceptual dimension of a task (e.g., location in vertical space) on the processing of another, relevant dimension (e.g., judging words’ emotional evaluation). Such effects are a central empirical pillar for recent views about how the mind/brain represents concepts. In the present paper, we show how attentional cueing (both exogenous and endogenous) to each conceptual dimension succeeds in modifying both the manifestation and the symmetry of the effect. The theoretical implications of this (...) finding are discussed. (shrink)

Do people represent space, time, number, and other conceptual domains using a generalized magnitude system (GMS)? To answer this question, numerous studies have used the spatial‐numerical association of response codes (SNARC) task and its variants. Yet, for a combination of reasons, SNARC‐like effects cannot provide evidence for a GMS, even in principle. Rather, these effects support a broader theory of how people use space metaphorically to scaffold their understanding of myriad non‐spatial domains, whether or not these domains exhibit variation in (...) magnitude. (shrink)

Research on the metaphorical mapping of valenced concepts onto space indicates that positive, neutral, and negative concepts are mapped onto upward, midward, and downward locations, respectively. More recently, this type of research has been tested for the very first time in 3D physical space. The findings corroborate the mapping of valenced concepts onto the vertical space as described above but further show that positive and negative concepts are placed close to and away from the body; neutral concepts are placed midway. (...) The current study aimed at investigating whether valenced perceptual stimuli are positioned onto 3D space akin to the way valenced concepts are positioned. By using a unique device known as the cognition cube, participants placed visual, auditory, tactile and olfactory stimuli on 3D space. The results mimicked the placing of valenced concepts onto 3D space; i.e., positive percepts were placed in upward and close-to-the-body locations and negative percepts were placed in downward and away-from-the-body locations; neutral percepts were placed midway. These pattern of results was more pronounced in the case of visual stimuli, followed by auditory, tactile, and olfactory stimuli. Significance Statement Just recently, a unique device called “the cognition cube” (CC) enabled to find that positive words are mapped onto upward and close-to-the-body locations and negative words are mapped onto downward and away-from-the-body locations; neutral words are placed midway. This way of placing words in relation to the body is consistent with an approach-avoidance effect such that “good” and “bad” things are kept close to and away from one’s body. We demonstrate for the very first time that this same pattern emerges when visual, auditory, tactile, and olfactory perceptual stimuli are placed on 3D physical space. We believe these results are significant in that the CC can be used as a new tool to diagnose emotion-related disorders. (shrink)

Number-form synesthetes consciously experience numbers in spatially-defined locations. For non-synesthete individuals, a similar association of numbers and space appears in the form of an implicit mental number line as signified by the distance effect–reaction time decreases as the numerical distance between compared numbers increases. In the current experiment, three number-form synesthetes and two different non-synesthete control groups performed a number comparison task. Synesthete participants exhibited a sizeable distance effect only when presented numbers were congruent with their number-form. In contrast, the (...) controls exhibited a distance effect regardless of congruency or presentation type. The findings suggest that: number-form synesthesia impairs the ability to represent numbers in a flexible manner according to task demands; number-form synesthesia is a genuine tangible experience, triggered involuntarily; and the classic mental number line can be more pliable than previously thought and appears to be independent of cultural-lingo direction. (shrink)

In a new integration, we show that the visual-spatial structuring of time converges with auditory-spatial left–right judgments for time-related words. In Experiment 1, participants placed past and future-related words respectively to the left and right of the midpoint on a horizontal line, reproducing earlier findings. In Experiment 2, neutral and time-related words were presented over headphones. Participants were asked to indicate whether words were louder on the left or right channel. On critical experimental trials, words were presented equally loud binaurally. (...) As predicted, participants judged future words to be louder on the right channel more often than past-related words. Furthermore, there was a significant cross-modal overlap between the visual-spatial ordering and the auditory judgments , which were continuously related. These findings provide support for the assumption that space and time have certain invariant properties that share a common structure across modalities. (shrink)

According to the Spatial Quantity Association of Response Codes (SQUARC), people hold a mental association between horizontal position and quantity (lower quantities left, higher quantities right). While a large body of research has explored this effect for response speed and judgment accuracy, the affective downstream consequences of the SQUARC remain unexplored. Aiming to address this gap, the present two experiments (pre-registered, total N = 521) investigated whether stimulus arrangements that are compatible with the SQUARC for luminance are affectively preferred to (...) stimulus arrangements that are incompatible. SQUARC-compatible square arrangements (dark-left, bright-right) were preferred over SQUARC-incompatible square arrangements (dark-right, bright-left). The preference for SQUARC compatibility was not moderated by the horizontal orientation of the response scale. Our results confirm the direction of the spatial-luminance association and provide initial support that the cognitive processing of SQUARC compatibility is hedonically marked and appears sufficient to impact affective evaluations. (shrink)

We assessed the automaticity of spatial-numerical and spatial-musical associations by testing their intentionality and load sensitivity in a dual-task paradigm. In separate sessions, 16 healthy adults performed magnitude and pitch comparisons on sung numbers with variable pitch. Stimuli and response alternatives were identical, but the relevant stimulus attribute (pitch or number) differed between tasks. Concomitant tasks required retention of either color or location information. Results show that spatial associations of both magnitude and pitch are load sensitive and that the spatial (...) association for pitch is more powerful than that for magnitude. These findings argue against the automaticity of spatial mappings in either stimulus dimension. (shrink)

ABSTRACTIn verbal communication, affective information is commonly conveyed to others through spatial terms. This study used a target location discrimination task with neutral, positive and negative stimuli to test the automaticity of the emotion-space association, both in the vertical and horizontal spatial axes. The effects of stimulus type on emotion-space representations were also probed. A congruency effect was observed in the vertical axis: detection of upper targets preceded by positive stimuli was faster. This effect occurred for all stimulus types, indicating (...) that the emotion-space association is not dependent on sensory modality and on the verbal content of affective stimuli. (shrink)

In this paper, representational structures of arithmetical thinking, encoded in human minds, are described. On the basis of empirical research, it is possible to distinguish four types of mental number lines: the shortest mental number line, summation mental number lines, point-place mental number lines and mental lines of exact numbers. These structures may be treated as generative mechanisms of forming arithmetical representations underlying our numerical acts of reference towards cardinalities, ordinals and magnitudes. In the paper, the theoretical framework for a (...) formal model of mental arithmetical representations is constructed. Many competitive conceptions of the mental system responsible for our arithmetical thinking may be unified within the presented framework. The paradigm underlying our research may be interpreted philosophically as a neo-Kantian approach to modeling the mind’s representational structures. (shrink)

Recent research in numerical cognition has begun to systematically detail the ability of humans and nonhuman animals to perceive the magnitudes of nonsymbolic ratios. These relationally defined analogs to rational numbers offer new potential insights into the nature of human numerical processing. However, research into their similarities with and connections to symbolic numbers remains in its infancy. The current research aims to further explore these similarities by investigating whether the magnitudes of nonsymbolic ratios are associated with space just as symbolic (...) numbers are. In two experiments, we found that responses were faster on the left for smaller nonsymbolic ratio magnitudes and faster on the right for larger nonsymbolic ratio magnitudes. These results further elucidate the nature of nonsymbolic ratio processing, extending the literature of spatial–numerical associations to nonsymbolic relative magnitudes. We discuss potential implications of these findings for theories of human magnitude processing in general and how this general processing relates to numerical processing. (shrink)

Everyone agrees that musical works are individuated by essential elements such as tone, harmony, and rhythm. Some argue that timbre or instrumentation can individuate musical works, too. I argue here that there can be a further element of musical works: spatial location. Some works of music are partly constituted by the location and motion of their sound sources. I begin by describing works of spatial music and arguing that they exist. I then consider the implications for the ontology of music. (...) Hardcore formalists cannot allow for spatial music. I argue that two other views, which allow a close relationship between sounds and musical works, can allow for works of spatial music. However, their ability to do so turns on issues in the philosophy of sound. I appeal to work in philosophy of sound to show that music is an art not just of hearing, but of sounds. (shrink)

This study tested to what extent young listeners metaphorically conceptualize basic musical relations. Ninety children aged 11 (30 attending a music school and 30 Serbian and 30 Romani children with no musical education) were played 5 stimuli with mutually opposed musical elements and asked to respond what the first and what the second one was like. Their answers were classified into metaphors according to the tenets of the conceptual metaphor theory. The results suggest an overwhelming dominance of metaphorical replies, in (...) which most mappings were based on image schemas within a predominantly visual–spatial modality. There were some differences in conceptualizations, however, with “high and low” tones also perceived as “big and small,” or “upward” musical motion seen as “forward.” This article analyzes the possibility of finding a common denominator for seemingly disparate replies, where the study of musical conceptualization might be instrumental in the cognitive semantic quest for metaphorical universals. (shrink)

This paper is a systematic investigation of motion expressions in programmatic music description. To address issues with defining the Source MOTION and the Target MUSIC, we utilize Gestalt models (Figure-Ground and Source-Path-Goal) while also critically examining the ontological complexity of the Target MUSIC. We also investigate music motion descriptions considering the role of the describer’s perspective and communicative goals. As previous research has demonstrated, an attentional Goal-bias is common in physical motion description, yet this has been found also to lessen (...) due to audience accommodation effects. We investigate whether this also occurs in music description. Using cognitive linguistic frameworks, we conducted an analysis of 21 English speakers’ written descriptions of dynamic orchestral excerpts. All participants gave a description of one excerpt reporting their own personal experiences and the other excerpt reporting the events of the excerpt for a fictional future participant. We find that addressee accommodation shapes the choice of the ontological types of Figures used from being more subjective and creative in describing music for oneself versus being more objective in describing music for others. However, our investigation does not find sufficient evidence for a Goal-bias in music like there is in physical motion event descriptions. (shrink)

Cognitive Science, Volume 46, Issue 5, May 2022.

When describing motion along both the horizontal and vertical axes, languages from different families express the elements encoding verticality before those coding for horizontality (e.g., going up right instead of right up). In light of the motor grounding of language, the present study investigated whether the prevalence of verticality in Path expression also governs the trajectory of arm biological movements. Using a 3D virtual-reality setting, we tracked the kinematics of hand pointing movements in five spatial directions, two of which implied (...) the vertical and horizontal vectors equally (i.e., up right +45° and bottom right −45°). Movement onset could be prompted by visual or auditory verbal cues, the latter being canonical in French (“en haut à droite”/up right) or not (“à droite en haut”/right up). In two experiments, analyses of the index finger kinematics revealed a significant effect of gravity, with earlier acceleration, velocity, and deceleration peaks for upward (+45°) than downward (−45°) movements, irrespective of the instructions. Remarkably, confirming the linguistic observations, we found that vertical kinematic parameters occurred earlier than horizontal ones for upward movements, both for visual and congruent verbal cues. Non-canonical verbal instructions significantly affected this temporal dynamic: for upward movements, the horizontal and vertical components temporally aligned, while they reversed for downward movements where the kinematics of the vertical axis was delayed with respect to that of the horizontal one. This temporal dynamic is so deeply anchored that non-canonical verbal instructions allowed for horizontality to precede verticality only for movements that do not fight against gravity. Altogether, our findings provide new insights into the embodiment of language by revealing that linguistic path may reflect the organization of biological movements, giving priority to the vertical axis. (shrink)

Much research has explored developing sound representations in language, but less work addresses developing representations of other sound patterns. This study examined preschool children's musical representations using two different tasks: discrimination and sound–picture association. Melodic contour—a musically relevant property—and instrumental timbre, which is less musically relevant, were tested. In Experiment 1, children failed to associate cartoon characters to melodies with maximally different pitch contours, with no advantage for melody preexposure. Experiment 2 also used different-contour melodies and found good discrimination, whereas (...) association was at chance. Experiment 3 replicated Experiment 2, but with a large timbre change instead of a contour change. Here, discrimination and association were both excellent. Preschool-aged children may have stronger or more durable representations of timbre than contour, particularly in more difficult tasks. Reasons for weaker association of contour than timbre information are discussed, along with implications for auditory development. (shrink)

Congenital amusia in its most common form is a disorder characterized by a musical pitch processing deficit. Although pitch is involved in conveying emotion in music, the implications for pitch deficits on musical emotion judgements is still under debate. Relatedly, both limited and spared musical emotion recognition was reported in amusia in conditions where emotion cues were not determined by musical mode or dissonance. Additionally, assumed links between musical abilities and visuo-spatial attention processes need further investigation in congenital amusics. Hence, (...) we here test to what extent musical emotions can influence attentional performance. Fifteen congenital amusic adults and fifteen healthy controls matched for age and education were assessed in three attentional conditions: executive control, alerting, and orienting while music expressing either joy, tenderness, sadness, or tension was presented. Visual target detection was in the normal range for both accuracy and response times in the amusic relative to the control participants. Moreover, in both groups, music exposure produced facilitating effects on selective attention that appeared to be driven by the arousal dimension of musical emotional content, with faster correct target detection during joyful compared to sad music. These findings corroborate the idea that pitch processing deficits related to congenital amusia do not impede other cognitive domains, particularly visual attention. Furthermore, our study uncovers an intact influence of music and its emotional content on the attentional abilities of amusic individuals. The results highlight the domain-selectivity of the pitch disorder in congenital amusia, which largely spares the development of visual attention and affective systems. (shrink)

Although acoustic frequency is not a spatial property of physical objects, in common language, pitch, i.e., the psychological correlated of frequency, is often labeled spatially (i.e., “high in pitch” or “low in pitch”). Pitch-height is known to modulate (and interact with) the response of participants when they are asked to judge spatial properties of non-auditory stimuli (e.g., visual) in a variety of behavioral tasks. In the current study we investigated whether the modulatory action of pitch-height extended to the haptic estimation (...) of height of a virtual step. We implemented a HW/SW setup which is able to render virtual 3D objects (stair-steps) haptically through a PHANTOM device, and to provide real-time continuous auditory feedback depending on the user interaction with the object. The haptic exploration was associated with a sinusoidal tone whose pitch varied as a function of the interaction point’s height within (i) a narrower and (ii) a wider pitch range, or (iii) a random pitch variation acting as a control audio condition. Explorations were also performed with no sound (haptic only). Participants were instructed to explore the virtual step freely, and to communicate height estimation by opening their thumb and index finger to mimic the step riser height, or verbally by reporting the height in centimeters of the step riser. We analyzed the role of musical expertise by dividing participants into non musicians and musicians. Results showed no effects of musical pitch on high-realistic haptic feedback. Overall there is no difference between the two groups in the proposed multimodal conditions. Additionally, we observed a different haptic response distribution between musicians and non musicians when estimations of the auditory conditions are matched with estimations in the no sound condition. (shrink)

Human infants begin very early in life to take advantage of multisensory information by extracting the invariant amodal information that is conveyed redundantly by multiple senses. Here we addressed the question as to whether infants can bind multisensory moving stimuli, and whether this occurs even if the motion produced by the stimuli is only illusory. Three- to four-month-old infants were presented with two bimodal pairings: visuo-tactile and audio-visual. Visuo-tactile pairings consisted of apparently vertically moving bars (the Barber Pole illusion) moving (...) in either the same or opposite direction with a concurrent tactile stimulus consisting of strokes given on the infant’s back. Audio-visual pairings consisted of the Barber Pole illusion in its visual and auditory version, the latter giving the impression of a continuous rising or ascending pitch. We found that infants were able to discriminate congruently (same direction) vs. incongruently moving (opposite direction) pairs irrespective of modality (Experiment 1). Importantly, we also found that congruently moving visuo-tactile and audio-visual stimuli were preferred over incongruently moving bimodal stimuli (Experiment 2). Our findings suggest that very young infants are able to extract motion as amodal component and use it to match stimuli that only apparently move in the same direction. (shrink)

Linking arbitrary shapes to personal labels or reward values results in immediate processing benefits for those stimuli that happen to be associated with the self or high rewards in perceptual matching tasks. Here we further explored how social and reward associations interact with multisensory stimuli by pairing labels and objects with tones. We also investigated whether self and reward biases persist for multisensory stimuli with the label removed after an association had been made. Both high reward stimuli and those associated (...) with the self, resulted in faster responses and improved discriminability, which persisted for multisensory stimuli even when the labels were removed. However, these self- and reward-biases partly depended on the specific alignment between the physical tones and the conceptual order. Performance for reward associations improved when the endpoints of low or high rewards were paired with low or high tones; meanwhile, for personal associations, there was a benefit when the self was paired with either low or high tones, but there was no effect when the stranger was associated with either endpoint. These results indicate that, unlike reward, social personal associations are not represented along a continuum with two marked endpoints but rather with a single reference point. (shrink)