An audiovisual correspondence (AVC) refers to an observer’s seemingly arbitrary yet consistent matching of sensory features across the two modalities; for example, between an auditory pitch and visual size. Research on AVCs has frequently used a speeded classification procedure in which participants are asked to rapidly classify an image when it is either accompanied by a congruent or an incongruent sound (or vice versa). When, as is typically the case, classification is faster in the presence of a congruent stimulus, researchers (...) have inferred that the AVC is automatic and bottom-up. Such an inference is incomplete because the procedure does not show that the AVC is not subject to top-down influences. To remedy this problem, we devised a procedure that allows us to assess the degree of “bottom-up-ness” and “top-down-ness” in the processing of an AVC. We did this in studies of AVCs between pitch and five visual features: size, height, spatial frequency, brightness, and angularity. We find that all the AVCs we studied involve both bottom-up and top-down processing, thus undermining the prevalent generalization that AVCs are automatic. (shrink)

For 4E cognitive science, minds are embodied, embedded, enacted, and extended. Proponents observe that we regularly ‘offload’ our thinking onto body and world: we use gestures and calculators to augment mathematical reasoning, and smartphones and search engines as memory aids. I argue that music is a beyond-the-head resource that affords offloading. Via this offloading, music scaffolds access to new forms of thought, experience, and behaviour. I focus on music’s capacity to scaffold emotional consciousness, including the self-regulative processes constitutive of emotional (...) consciousness. In developing this idea, I consider the ‘material’ and ‘worldmaking’ character music, and I apply these considerations to two case studies: music as a tool for religious worship, and music as a weapon for torture. (shrink)

I defend a model of the musically extended mind. I consider how acts of “musicking” grant access to novel emotional experiences otherwise inaccessible. First, I discuss the idea of “musical affordances” and specify both what musical affordances are and how they invite different forms of entrainment. Next, I argue that musical affordances – via soliciting different forms of entrainment – enhance the functionality of various endogenous, emotiongranting regulative processes, drawing novel experiences out of us with an expanded complexity and phenomenal (...) character. I argue that music therefore ought to be thought of as part of the vehicle needed to realize these emotional experiences. I appeal to different sources of empirical work to develop this idea. (shrink)

The article focuses on meaning generation in music and language cognition based on complexes of image schemas. We hypothesize that schemas are involved in dynamic interactions fostering online complex conceptual construction found both in language and elsewhere in various cognitive code expressions. Expanding on recent work, we propose that an account of image-schema interaction should include the notion of scalarity, a higher-order parameter which captures the intensity with which schemas occur and/or interact with one another. In that sense, schemas are (...) proposed to have a binary valence (positive or negative) and a three-tier range of intensity levels. We focus on some examples of the interaction of force, path, link, balance, and containment image schemas in language and music to illustrate how a formal routine emerging from our proposal may help in addressing not only aspects of propositional semantics, but also affective and connotative nuances in musical and linguistic expression. (shrink)

Emotion scientists often take an ambivalent stance concerning the role of language in a science of emotion. However, it is important for emotion researchers to contemplate some of the consequences of current practices for their theory building. There is a danger of an overreliance on the English language as a transparent window into emotion categories. More consideration has to be given to cross-linguistic comparison in the future so that models of language acquisition and of the language–cognition interface fit better the (...) extant variation found in today’s peoples. (shrink)

When researchers think about the interaction between language and emotion, they typically focus on descriptive emotion words. This review demonstrates that emotion can interact with language at many levels of structure, from the sound patterns of a language to its lexicon and grammar, and beyond to how it appears in conversation and discourse. Findings are considered from diverse subfields across the language sciences, including cognitive linguistics, psycholinguistics, linguistic anthropology, and conversation analysis. Taken together, it is clear that emotional expression is (...) finely tuned to language-specific structures. Future emotion research can better exploit cross-linguistic variation to unravel possible universal principles operating between language and emotion. (shrink)

Height-pitch associations are claimed to be universal and independent of language, but this claim remains controversial. The present study sheds new light on this debate with a multimodal analysis of individual sound and melody descriptions obtained in an interactive communication paradigm with speakers of Dutch and Farsi. The findings reveal that, in contrast to Dutch speakers, Farsi speakers do not use a height-pitch metaphor consistently in speech. Both Dutch and Farsi speakers’ co-speech gestures did reveal a mapping of higher pitches (...) to higher space and lower pitches to lower space, and this gesture space-pitch mapping tended to co-occur with corresponding spatial words (high-low). However, this mapping was much weaker in Farsi speakers than Dutch speakers. This suggests that cross-linguistic differences shape the conceptualization of pitch and further calls into question the universality of height-pitch associations. (shrink)

Much research has explored developing sound representations in language, but less work addresses developing representations of other sound patterns. This study examined preschool children's musical representations using two different tasks: discrimination and sound–picture association. Melodic contour—a musically relevant property—and instrumental timbre, which is less musically relevant, were tested. In Experiment 1, children failed to associate cartoon characters to melodies with maximally different pitch contours, with no advantage for melody preexposure. Experiment 2 also used different-contour melodies and found good discrimination, whereas (...) association was at chance. Experiment 3 replicated Experiment 2, but with a large timbre change instead of a contour change. Here, discrimination and association were both excellent. Preschool-aged children may have stronger or more durable representations of timbre than contour, particularly in more difficult tasks. Reasons for weaker association of contour than timbre information are discussed, along with implications for auditory development. (shrink)

When describing motion along both the horizontal and vertical axes, languages from different families express the elements encoding verticality before those coding for horizontality (e.g., going up right instead of right up). In light of the motor grounding of language, the present study investigated whether the prevalence of verticality in Path expression also governs the trajectory of arm biological movements. Using a 3D virtual-reality setting, we tracked the kinematics of hand pointing movements in five spatial directions, two of which implied (...) the vertical and horizontal vectors equally (i.e., up right +45° and bottom right −45°). Movement onset could be prompted by visual or auditory verbal cues, the latter being canonical in French (“en haut à droite”/up right) or not (“à droite en haut”/right up). In two experiments, analyses of the index finger kinematics revealed a significant effect of gravity, with earlier acceleration, velocity, and deceleration peaks for upward (+45°) than downward (−45°) movements, irrespective of the instructions. Remarkably, confirming the linguistic observations, we found that vertical kinematic parameters occurred earlier than horizontal ones for upward movements, both for visual and congruent verbal cues. Non-canonical verbal instructions significantly affected this temporal dynamic: for upward movements, the horizontal and vertical components temporally aligned, while they reversed for downward movements where the kinematics of the vertical axis was delayed with respect to that of the horizontal one. This temporal dynamic is so deeply anchored that non-canonical verbal instructions allowed for horizontality to precede verticality only for movements that do not fight against gravity. Altogether, our findings provide new insights into the embodiment of language by revealing that linguistic path may reflect the organization of biological movements, giving priority to the vertical axis. (shrink)