Spatial aspects of words are associated with their canonical locations in the real world. Yet little research has tested whether spatial associations denoted in language comprehension generalize to their corresponding images. We directly tested the spatial aspects of mental imagery in picture and word processing (Experiment 1). We also tested whether spatial representations of motion words produce similar perceptual-interference effects as demonstrated by object words (Experiment 2). Findings revealed that words denoting an upward spatial location produced slower responses to targets (...) appearing at the top of the display, whereas words denoting a downward spatial location produced slower responses to targets appearing at the bottom of the display. Perceptual-interference effects did not obtain for pictures or for words lacking a spatial relation. These findings provide greater empirical support for the perceptual-symbols system theory (Barsalou, 1999, 2008). (shrink)

In the past, notions of embodiment have been applied to robotics mainly in the realm of very simple robots, and supporting low-level mechanisms such as dynamics and navigation. In contrast, most human-like, interactive, and socially adept robotic systems turn away from embodiment and use amodal, symbolic, and modular approaches to cognition and interaction. At the same time, recent research in Embodied Cognition (EC) is spanning an increasing number of complex cognitive processes, including language, nonverbal communication, learning, and social behavior.This article (...) suggests adopting a modern EC approach for autonomous robots interacting with humans. In particular, we present three core principles from EC that may be applicable to such robots: (a) modal perceptual representation, (b) action/perception and action/cognition integration, and (c) a simulation-based model of top-down perceptual biasing. We describe a computational framework based on these principles, and its implementation on two physical robots. This could provide a new paradigm for embodied human–robot interaction based on recent psychological and neurological findings. (shrink)

Many current programs for cognitive science sail under the banner of “embodied cognition.” These programs typically seek to distance themselves from standard cognitive science. The present proposal for a conception of embodied cognition is less radical than most, indeed, quite compatible with many versions of traditional cognitive science. Its rationale is based on two elements, each of which is theoretically plausible and empirically well-founded. The first element invokes the idea of “bodily formats,” i.e., representational codes primarily utilized in forming interoceptive (...) or directive representations of one’s bodily states and activities. The second element appeals to wideranging evidence that the brain reuses or redeploys cognitive processes having different original uses. When the redeployment theme is applied to bodily formats of representation, they jointly provide for the possibility that body-coded cognition is a very pervasive sector of cognition. (shrink)

The literature contains a disconnect between accounts of how humans learn lexical semantic representations for words. Theories generally propose that lexical semantics are learned either through perceptual experience or through exposure to regularities in language. We propose here a model to integrate these two information sources. Specifically, the model uses the global structure of memory to exploit the redundancy between language and perception in order to generate inferred perceptual representations for words with which the model has no perceptual experience. We (...) test the model on a variety of different datasets from grounded cognition experiments and demonstrate that this diverse set of results can be explained as perceptual simulation (cf. Barsalou, Simmons, Barbey, & Wilson, 2003) within a global memory model. (shrink)

Cognition is embodied when it is deeply dependent upon features of the physical body of an agent, that is, when aspects of the agent's body beyond the brain play a significant causal or physically constitutive role in cognitive processing. In general, dominant views in the philosophy of mind and cognitive science have considered the body as peripheral to understanding the nature of mind and cognition. Proponents of embodied cognitive science view this as a serious mistake. Sometimes the nature of the (...) dependence of cognition on the body is quite unexpected, and suggests new ways of conceptualizing and exploring the mechanics of cognitive processing. Embodied cognitive science encompasses a loose-knit family of research programs in the cognitive sciences that often share a commitment to critiquing and even replacing traditional approaches to cognition and cognitive processing. Empirical research on embodied cognition has exploded in the past 10 years. As the bibliography for this article attests, the various bodies of work that will be discussed represent a serious alternative to the investigation of cognitive phenomena. Relatively recent work on the embodiment of cognition provides much food for thought for empirically-informed philosophers of mind. This is in part because of the rich range of phenomena that embodied cognitive science has studied. But it is also in part because those phenomena are often thought to challenge dominant views of the mind, such as the computational and representational theories of mind, at the heart of traditional cognitive science. And they have sometimes been taken to undermine standard positions in the philosophy of mind, such as the idea that the mind is identical to, or even realized in, the brain. (shrink)

Thirty years ago, grounded cognition had roots in philosophy, perception, cognitive linguistics, psycholinguistics, cognitive psychology, and cognitive neuropsychology. During the next 20 years, grounded cognition continued developing in these areas, and it also took new forms in robotics, cognitive ecology, cognitive neuroscience, and developmental psychology. In the past 10 years, research on grounded cognition has grown rapidly, especially in cognitive neuroscience, social neuroscience, cognitive psychology, social psychology, and developmental psychology. Currently, grounded cognition appears to be achieving increased acceptance throughout cognitive (...) science, shifting from relatively minor status to increasing importance. Nevertheless, researchers wonder whether grounded mechanisms lie at the heart of the cognitive system or are peripheral to classic symbolic mechanisms. Although grounded cognition is currently dominated by demonstration experiments in the absence of well-developed theories, the area is likely to become increasingly theory driven over the next 30 years. Another likely development is the increased incorporation of grounding mechanisms into cognitive architectures and into accounts of classic cognitive phenomena. As this incorporation occurs, much functionality of these architectures and phenomena is likely to remain, along with many original mechanisms. Future theories of grounded cognition are likely to be heavily influenced by both cognitive neuroscience and social neuroscience, and also by developmental science and robotics. Aspects from the three major perspectives in cognitive science—classic symbolic architectures, statistical/dynamical systems, and grounded cognition—will probably be integrated increasingly in future theories, each capturing indispensable aspects of intelligence. (shrink)

The notion of intelligence is relevant to several fields of research, including cognitive and comparative psychology, neuroscience, artificial intelligence, and philosophy, among others. However, there is little agreement within and across these fields on how to characterise and explain intelligence. I put forward a behavioural, operational characterisation of intelligence that can play an integrative role in the sciences of intelligence, as well as preserve the distinctive explanatory value of the notion, setting it apart from the related concepts of cognition and (...) rationality. Finally, I examine a popular hypothesis about the underpinnings of intelligence: the capacity to manipulate internal representations of the environment. I argue that the hypothesis needs refinement, and that so refined, it applies only to some forms of intelligence. (shrink)

We present a neural dynamic model that perceptually grounds nested noun phrases, that is, noun phrases that contain further (possibly also nested) noun phrases as parts. The model receives input from the visual array and a representation of a noun phrase from language processing. It organizes a search for the denoted object in the visual scene. The model is a neural dynamic architecture of interacting neural populations which has clear interfaces with perceptual processes. It solves a set of theoretical challenges, (...) including the problem of keeping a nested structure in short-term memory in a way that solves the problem of 2 and massive binding problem emphasized by Jackendoff. The model organizes a search for the objects that are referenced in that structure. We motivate the model, demonstrate simulation results, and discuss how it differs from related models. (shrink)

That words express a conceptual content is uncontroversial. This does not entail that their content should break down neatly into a grammatical part, relevant for language and to be analyzed in linguistic terms, and a conceptual part, relevant for cognition and to be analyzed in psychological terms. Various types of empirical evidence are reviewed, showing that the conceptual content of words cannot be isolated from their linguistic properties, because it is affected and shaped by them. The view of words as (...) labels or containers for a non-linguistic conceptual content stems from a naive disregard of the complex and structured nature of lexical knowledge. On the contrary, knowledge of language is shown not just to organize and categorize conceptual content in a way not reducible to non-linguistic cognition, but also to affect its scope, as the range of verbalized concepts is both limited by abstract templates and expanded by productive word formation. This suggests that lexical knowledge is a distinctively linguistic dimension of conceptualization, and that words do not so much label or package concepts, as provide an inner form for conceptual knowledge. (shrink)

The notion of politicization has been often assimilated to that of partisanship, especially in political and social sciences. However, these accounts underestimate more fine-grained, and yet pivotal, aspects at stake in processes of politicization. In addition, they overlook cognitive mechanisms underlying politicizing practices. Here, we propose an integrated approach to politicization relying on recent insights from both social and political sciences, as well as cognitive science. We outline two key facets of politicization, that we call partial indetermination and contestability, and (...) we show how these can be accounted for by appealing to recent literature in cognitive science concerned with abstract conceptual knowledge. We suggest that politicizing a concept often implies making its more abstract components more salient, hence legitimating its contestable character. Finally, we provide preliminary suggestions to test our proposal, using the concept of gender as case study. (shrink)

Tenenbaum and Griffiths (2001) have proposed that their Bayesian model of generalisation unifies Shepard’s (1987) and Tversky’s (1977) similarity-based explanations of two distinct patterns of generalisation behaviours by reconciling them under a single coherent task analysis. I argue that this proposal needs refinement: instead of unifying the heterogeneous notion of psychological similarity, the Bayesian approach unifies generalisation by rendering the distinct patterns of behaviours informationally relevant. I suggest that generalisation as a Bayesian inference should be seen as a complement to, (...) instead of a replacement of, similarity-based explanations. Furthermore, I show that the unificatory powers of the Bayesian model of generalisation can contribute to the selection of one of these models of psychological similarity. (shrink)

In recent years, language has been shown to play a number of important cognitive roles over and above the communication of thoughts. One hypothesis gaining support is that language facilitates thought about abstract categories, such as democracy or prediction. To test this proposal, a novel set of semantic memory task trials, designed for assessing abstract thought non-linguistically, were normed for levels of abstractness. The trials were rated as more or less abstract to the degree that answering them required the participant (...) to abstract away from both perceptual features and common setting associations corresponding to the target image. The normed materials were then used with a population of people with aphasia to assess the relationship of abstract thought to language. While the language-impaired group with aphasia showed lower overall accuracy and longer response times than controls in general, of special note is that their response times were significantly longer as a function of a trial’s degree of abstractness. Further, the aphasia group’s response times in reporting their degree of confidence (a separate, metacognitive measure) were negatively correlated with their language production abilities, with lower language scores predicting longer metacognitive response times. These results provide some support for the hypothesis that language is an important aid to abstract thought and to metacognition about abstract thought. (shrink)

According to embodied theories of language (ETLs), word meaning relies on sensorimotor brain areas, generally dedicated to acting and perceiving in the real world. More specifically, words denoting actions are postulated to make use of neural motor areas, while words denoting visual properties draw on the resources of visual brain areas. Therefore, there is a direct correspondence between word meaning and the experience a listener has had with a word's referent on the brain level. Behavioral and neuroimaging studies have provided (...) evidence in favor of ETLs; however, recent studies have also shown that sensorimotor information is recruited in a flexible manner during language comprehension (e.g., Raposo et al. ; Van Dam et al., ), leaving open the question as to what level of language processing sensorimotor activations contribute. In this study, we investigated the time course of modality‐specific contributions (i.e., the contribution of action information) as to word processing by manipulating both (a) the linguistic and (b) the action context in which target words were presented. Our results demonstrate that processes reflecting sensorimotor information play a role early in word processing (i.e., within 200 ms of word presentation), but that they are sensitive to the linguistic context in which a word is presented. In other words, when sensorimotor information is activated, it is activated quickly; however, specific words do not reliably activate a consistent sensorimotor pattern. (shrink)

Experientialist semantics has contributed to a broader notion of linguistic meaning by emphasizing notions such as construal, perspective, metaphor, and embodiment, but has suffered from an individualist concept of meaning and has conflated experiential motivations with conventional semantics. We argue that these problems can be redressed by methods and concepts from phenomenology, on the basis of a case study of sentences of non-actual motion such as “The mountain range goes all the way from Mexico to Canada.” Through a phenomenological reanalysis (...) of proposals of Talmy, Langacker, and Matlock, we show that non-actual motion is both experientially and linguistically non-unitary. At least three different features of human consciousness—enactive perception, visual scanning, and imagination—constitute experiential motivations for non-actual motion sentences, and each of these could be related to phenomenological analyses of human intentionality. The second problem is addressed by proposing that the experiential motivations of non-actual motion sentences can be viewed as sedimented through “passive” processes of acquisition and social transmission and that this implies an interactive loop between experience and language, yielding losses in terms of original experience, but gains in terms of communal signification. Something that is underestimated by phenomenology is that what is sedimented are not only intentional objects such as states of affairs, but aspects of how they are given, i.e., the original, temporal, bodily experiences themselves. Since cognitive semantics has emphasized such aspects of meaning, we suggest that phenomenology can itself benefit from experientialist semantics, especially when it turns its focus from prepredicative to predicative, linguistic intentionality. (shrink)

The question of whether phenomenal consciousness is limited to the capacity of cognitive access remains a contentious issue in philosophy. Overflow theorists argue that the capacity of conscious experience outstrips the capacity of cognitive access. This paper demonstrates a resolution to the overflow debate is found in acknowledging a difference in phenomenological timing required by both sides. It makes clear that the “no overflow” view requires subjects to, at the bare minimum, generate an unconscious visual image of previously presented items (...) if it is to explain performance in the change detection paradigm. It then demonstrates that conscious imagery should support better task performance than unconscious imagery because of a necessary difference in representational strength. However, this contradicts empirical findings, and so a new argument for overflow is presented without requiring the premise that subjects need to obtain a specific phenomenology of presented items during change detection. (shrink)

Affordances, i.e. the opportunity of actions offered by the environment, are one of the central research topics for the theoretical perspectives that view cognition as emerging from the interaction between the environment and the body. Being at the bridge between perception and action, affordances help to question a dichotomous view of perception and action. While Gibson’s view of affordances is mainly externalist, many contemporary approaches define affordances as the product of long-term visuomotor associations in the brain. These studies have emphasized (...) the fact that affordances are activated automatically, independently from the context and the previous intention to act: for example, affordances related to objects’ size would emerge even if the task does not require focusing on size. This emphasis on the automaticity of affordances has led to overlook their flexibility and contextual-dependency. In this contribution I will outline and discuss recent perspectives and evidence that reveal the flexibility and context-dependency of affordances, clarifying how they are modulated by the physical, cultural and social context. I will focus specifically on social affordances, i.e. on how perception of affordances might be influenced by the presence of multiple actors having different goals. (shrink)

Contemporary research on mindreading or theory of mind has resulted in three major findings: There is a difference in the age of passing of the elicited-response false belief task and its spontaneous–response version; 15-month-olds pass the latter while the former is passed only by 4-year-olds. Linguistic and social factors influence the development of the ability to mindread in many ways. There are cultures with folk psychologies significantly different from the Western one, and children from such cultures tend to show different (...) timetables of mindreading development. The traditional accounts of the data are nativism, rational constructivism, and two-systems theory. In this paper, we offer criticism of these traditional cognitivist accounts and explore an alternative, action-based framework. We argue that even though they all seem to explain the above empirical data, there are other, theoretical reasons why their explanations are untenable. Specifically, we discuss the problem of foundationalism and the related problem of innateness. Finally, we explore an alternative, action-based framework that avoids these theoretical limitations and offer an interpretation of the empirical data from that perspective. (shrink)

The Humean view that conceivability entails possibility can be criticized via input from cognitive psychology. A mainstream view here has it that there are two candidate codings for mental representations (one of them being, according to some, reducible to the other): the linguistic and the pictorial, the difference between the two consisting in the degree of arbitrariness of the representation relation. If the conceivability of P at issue for Humeans involves the having of a linguistic mental representation, then it is (...) easy to show that we can conceive the impossible, for impossibilities can be represented by meaningful bits of language. If the conceivability of P amounts to the pictorial imaginability of a situation verifying P, then the question is whether the imagination at issue works purely qualitatively, that is, only by phenomenological resemblance with the imagined scenario. If so, the range of situations imaginable in this way is too limited to have a significant role in modal epistemology. If not, imagination will involve some arbitrary labeling component, which turns out to be sufficient for imagining the impossible. And if the relevant imagination is neither linguistic nor pictorial, Humeans will appear to resort to some representational magic, until they come up with a theory of a ‘third code’ for mental representations. (shrink)

The Phenomenal Concept Strategy offers the physicalist perhaps the most promising means of explaining why the connection between mental facts and physical facts appears to be contingent even though it is not. In this article, we show that the large body of evidence suggesting that our concepts are often embodied and grounded in sensorimotor systems speaks against standard forms of the PCS. We argue, nevertheless, that it is possible to formulate a novel version of the PCS that is thoroughly in (...) keeping with embodied cognition, focuses on features of physical concepts, and succeeds in explaining the appearance of contingency. (shrink)

Traditionally, the “Imagery Debate” has opposed two main camps: depictivism and descriptivism. This debate has essentially focused on the nature of the internal representations thought to be involved in imagery, without addressing at all the question of action. More recently, a third, “embodied” view is moving the debate into a new phase. The embodied approach focuses on the interdependence of perception, cognition and action, and in its more radical line this approach promotes the idea that perception is not a process (...) involving internal world-models. The anti-representationalist version of the embodied paradigm covers, among others that we shall not discuss here, two quite different positions, namely the enactive approach and sensorimotor theory. Up to now these two anti-representationalist accounts have generally been confounded. In this paper we will argue that despite some important commonalities, enactive and sensorimotor accounts come with distinctive theoretical traits with regard to their approach to imagery. These become manifest when critically examining the role they assign to sensorimotor engagements with the world. We shall argue that enactive and sensorimotor approaches are different in their understanding of the role of embodied action, and these different notions of embodiment lead to different explanatory accounts of perception and imagery. We propose that, due to existing ambiguities in enactivism, the sensorimotor theory is a better framework for a skill-based approach to imagery. (shrink)

Embodied cognition pertains to the consequences on thought and emotion of living with our particular human sensory and motor systems. The consequences are quite varied, and researchers across the cognitive sciences have made great discoveries in line with this principle. However, while we offer this principle, it is necessarily broad, and searching for a single unifying theme has not brought researchers together behind a clearly defined endeavor. Rather than attempt to do so, we embrace the variation and specificity in research (...) endeavors across the cognitive sciences to forge a practical sense in which embodied cognition can be a useful paradigm within which to think and work. This topic, The Future of Embodied Cognition, includes contributions from eight sets of authors communicating how embodied cognition has and will influence specific approaches within their disciplines. Through this format, the lessons from each contribution can be easily shared with colleagues across disciplines. As these lessons continue to be shared, a paradigm that is of practical use will emerge, and its coherence across disciplines will follow. To illustrate the practical aspect of this approach, in this introductory paper, we take one lesson from each contribution that must be shared and illustrate how each lesson can apply to a single, specific topic of study. (shrink)

Designing, building, and experimenting with physical simulation models are central problem‐solving practices in the engineering sciences. Model‐based simulation is an epistemic activity that includes exploration, generation and testing of hypotheses, explanation, and inference. This paper argues that to interpret and understand how these simulation models function in creating knowledge and technologies requires construing problem solving as accomplished by a researcher–artifact system. It draws on and further develops the framework of “distributed cognition” to interpret data collected in ethnographic and cognitive‐historical studies (...) of two biomedical engineering research laboratories, and articulates the notion of distributed model‐based cognition to answer the question posed in the title. (shrink)

TEST is a novel taxonomy of knowledge representations based on three distinct hierarchically organized representational features: Tropism, Embodiment, and Situatedness. Tropic representational features reflect constraints of the physical world on the agent's ability to form, reactivate, and enrich embodied (i.e., resulting from the agent's bodily constraints) conceptual representations embedded in situated contexts. The proposed hierarchy entails that representations can, in principle, have tropic features without necessarily having situated and/or embodied features. On the other hand, representations that are situated and/or embodied (...) are likely to be simultaneously tropic. Hence, although we propose tropism as the most general term, the hierarchical relationship between embodiment and situatedness is more on a par, such that the dominance of one component over the other relies on the distinction between offline storage versus online generation as well as on representation-specific properties. (shrink)

Psychologists are increasingly interested in embodiment based on the assumption that thoughts, feelings, and behaviors are grounded in bodily interaction with the environment. We examine how embodiment is used in social psychology, and we explore the ways in which embodied approaches enrich traditional theories. Although research in this area is burgeoning, much of it has been more descriptive than explanatory. We provide a critical discussion of the trajectory of embodiment research in social psychology. We contend that future researchers should engage (...) in a phenomenon-based approach, highlight the theoretical boundary conditions and mediators involved, explore novel action-relevant outcome measures, and address the role of individual differences broadly defined. Such research will likely provide a more explanatory account of the role of embodiment in general terms as well as how it expands the knowledge base in social psychology. (shrink)

In this paper we discuss an approach called grounded action cognition, which aims to provide a theory of the interdependencies between motor control and action-related cognitive processes, like perceiving an action or thinking about an action. The theory contrasts with traditional views in cognitive science in that it motivates an understanding of cognition as embodied, through application of Barsalou’s general idea of grounded cognition. To guide further research towards an appropriate theory of grounded action cognition we distinguish between grounding qua (...) acquisition and grounding qua constitution. On this basis, we distinguish three possible theoretical conceptions of grounded action cognition. In addition to these methodological and conceptual analyses, we draw on recent empirical evidence to motivate our inclination towards a particular theory. According to this theory certain representations are involved in action cognition and action perception that are not modality-specific as usually proposed by advocates of grounded cognition. Further, the evidence is in favor of our more specific theory stating that for some cognitive abilities, some motor abilites are constitutive. (shrink)

Although philosophy has been only a minor contributor to cognitive science to date, this paper describes two projects in naturalistic philosophy of mind and one in naturalistic philosophy of science that have been pursued during the past 30 years and that can make theoretical and methodological contributions to cognitive science. First, stances on the mind–body problem (identity theory, functionalism, and heuristic identity theory) are relevant to cognitive science as it negotiates its relation to neuroscience and cognitive neuroscience. Second, analyses of (...) mental representations address both their vehicles and their contents; new approaches to characterizing how representations have content are particularly relevant to understanding the relation of cognitive agents to their environments. Third, the recently formulated accounts of mechanistic explanation in philosophy of science both provide perspective on the explanatory project of cognitive science and may offer normative guidance to cognitive science (e.g., by providing perspective on how multiple disciplinary perspectives can be integrated in understanding a given mechanism). (shrink)

Concept normativity is a prominent subject of inquiry in the philosophical literature on the nature of concepts. Concepts are said to be normative, in that the use of concepts to categorise is associated with an evaluation of the appropriateness of such categorisation measured against some objective external standard. Two broad groups of views have emerged in accounting for the normativity of concepts: a weaker view traces such normativity to the social practice in which the agent using the concept is embedded, (...) while a stronger view traces such normativity to a first-person capacity of reflection. However, both views have drawbacks: the weaker view seems not to do justice to the basic sense of normativity associated with an individual agent using a concept, while the stronger view ties such normativity with the first-person conscious evaluation, which appears to be too strong. Here, we propose a different view of concepts using principles from the Active Inference framework. We reconceive concepts, defining them as Bayesian beliefs—that is, conditional probability distributions—that represent causes and contingencies in the world, their form grounded in the exchange between the agent and its environment. This allows us to present a different view on the source of normativity, with an emphasis on the structure of the agent itself as well as its interaction with the environment. On the Active Inference view, concepts are normative in that they are intrinsically connected to the self-evidencing nature of an agent, whose very structure implies an evaluation of the concepts it employs. (shrink)

This paper takes an embodied and extended cognition perspective to ER integration – a cognitive process through which a learner integrates external representations (ERs) in a domain, with her internal (mental) model, as she interacts with, uses, understands and transforms between those ERs. In the paper, I argue for a theoretical as well as empirical shift in future investigations of ER integration, by proposing a model of cognitive mechanisms underlying the process, based on recent advances in extended and embodied cognition. (...) I present this new model in contrast to the still dominant classical cognitivist (information processing) approaches to ER integration, and the educational technology intervention designs such approaches inspire. I then exemplify this distinction between the information processing model and the new model through a case of arithmetic problem solving. Corroborative neuroscience evidence presented in relation to this case provides empirical support for the new model by showing how bodily actions (sensorimotor mechanisms) are critical to ER integration and learning. Finally, as educational implications of the new model, I demonstrate the need for: (i) re-viewing the development of ER integration and expertise as fine-tuning of the learner's action or sensorimotor system, and (ii) a shift of focus in new-media intervention design principles based on this newer understanding of ER integration in science, technology, engineering and mathematics. (shrink)

Formal theories of mental representation have receded from the importance they had in the early days of cognitive science. I argue that such theories are crucial in any mental domain, not just for their own sake, but to guide experimental inquiry, as well as to integrate the domain into the mind as a whole. To illustrate the criteria of adequacy for theories of mental representation, I compare two theoretical approaches to language: classical generative grammar (Chomsky, 1965, 1981, 1995) and the (...) parallel architecture (Jackendoff, 1997, 2002). The grounds for comparison include (a) the internal coherence of the theory across phonology, syntax, and semantics; (b) the relation of language to other mental faculties; (c) the relationship between grammar and lexicon; (d) relevance to theories of language processing; and (e) the possibility of languages with little or no syntax. (shrink)

Rich sensorimotor interaction facilitates language learning and is presumed to ground conceptual representations. Yet empirical support for early stages of embodied word learning is currently lacking. Finding evidence that sensorimotor interaction shapes learned linguistic representations would provide crucial support for embodied language theories. We developed a gamified word learning experiment in virtual reality in which participants learned the names of six novel objects by grasping and manipulating objects with either their left or right hand. Participants then completed a word–color match (...) task in which they were tested on the same six words and objects. Participants were faster to respond to stimuli in the match task when the response hand was compatible with the hand used to interact with the named object, an effect we refer to as affordance compatibility. In two follow up experiments, we found that merely observing virtual hands interact with the objects was sufficient to acquire a smaller affordance compatibility effect, and we found that the compatibility effect was driven primarily by responses with a compatible hand and not by responses in a compatible spatial location. Our results support theoretical views of language which ground word representations in sensorimotor experiences, and they suggest promising future routes to explore the sensorimotor foundations of higher cognition through immersive virtual experiments. (shrink)

Describing actions entails that relations between objects are discovered. A pervasively neural account of this process requires that fundamental problems are solved: the neural pointer problem, the binding problem, and the problem of generating discrete processing steps from time-continuous neural processes. We present a prototypical solution to these problems in a neural dynamic model that comprises dynamic neural fields holding representations close to sensorimotor surfaces as well as dynamic neural nodes holding discrete, language-like representations. Making the connection between these two (...) types of representations enables the model to describe actions as well as to perceptually ground movement phrases—all based on real visual input. We demonstrate how the dynamic neural processes autonomously generate the processing steps required to describe or ground object-oriented actions. By solving the fundamental problems of neural pointing, binding, and emergent discrete processing, the model may be a first but critical step toward a systematic neural processing account of higher cognition. (shrink)

As embodied theories of cognition are increasingly formalized and tested, care must be taken to make informed assumptions regarding the nature of concepts and representations. In this study, we outline three reasons why one cannot, in effect, represent the same concept twice. First, online perception affects offline representation: Current representational content depends on how ongoing demands direct attention to modality-specific systems. Second, language is a fundamental facilitator of offline representation: Bootstrapping and shortcuts within the computationally cheaper linguistic system continuously modify (...) representational content. Third, time itself is a source of representational change: As the content of underlying concepts shifts with the accumulation of direct and vicarious experience, so too does the content of representations that draw upon these concepts. We discuss the ramifications of these principles for research into both human and synthetic cognitive systems. (shrink)

Recent evidence from cognitive neuroscience suggests that certain cognitive processes employ perceptual representations. Inspired by this evidence, a few researchers have proposed that cognition is inherently perceptual. They have developed an innovative theoretical approach that rests on the notion of perceptual simulation and marshaled several general arguments supporting the centrality of perceptual representations to concepts. In this article, I identify a number of weaknesses in these arguments and defend a multiple semantic code approach that posits both perceptual and non-perceptual representations.

Computational models of meaning trained on naturally occurring text successfully model human performance on tasks involving simple similarity measures, but they characterize meaning in terms of undifferentiated bags of words or topical dimensions. This has led some to question their psychological plausibility (Murphy, 2002;Schunn, 1999). We present here a fully automatic method for extracting a structured and comprehensive set of concept descriptions directly from an English part‐of‐speech‐tagged corpus. Concepts are characterized by weighted properties, enriched with concept–property types that approximate classical (...) relations such as hypernymy and function. Our model outperforms comparable algorithms in cognitive tasks pertaining not only to concept‐internal structures (discovering properties of concepts, grouping properties by property type) but also to inter‐concept relations (clustering into superordinates), suggesting the empirical validity of the property‐based approach. (shrink)

The present online volume contains the papers prepared for the 11th Budapest Visual Learning Conference – ENVISIONING AN ELECTRIFYING FUTURE – held in a physical-online blended form on Nov. 13, 2024, organized by the University of Pécs (represented by Prof. Gábor Szécsi, Dean, Faculty of Cultural Sciences, Education and Regional Development), and the Hungarian Academy of Sciences (represented by Prof. Kristóf Nyíri, Member of the Hungarian Academy of Sciences). Nyíri and Szécsi were responsible for sending out the call for abstracts (...) and inviting plenary speakers, Szécsi organized the physical surroundings. Nyíri’s task was to build up and continuosly update the conference website, as well as to edit the accepted submissions. As we indicated in the call for submissions, the conference was planned as an interdisciplinary encounter of communication and media theory, cultural sciences, sociology, psychology, philosophy, pedagogy, history, political science, picture theory, and other disciplines. We insisted on achieving new scholarly results. Especially with AI now complementing, or intruding into, the world of the internet, what image of the future can we conceive of, what new patterns of life and in particular forms of education should we strive to create? (shrink)

Theories of grounded and embodied cognition offer a range of accounts of how reasoning and body‐based processes are related to each other. To advance theories of grounded and embodied cognition, we explore the cognitive relevance of particular body states to associated math concepts. We test competing models of action‐cognition transduction to investigate the cognitive relevance of directed actions to students’ mathematical reasoning in the area of geometry. The hypotheses we test include (1) that cognitively relevant directed actions have a direct (...) effect on performance (direct cognitive relevance hypothesis), (2) that cognitively relevant directed actions lead to more frequent production of gestures during explanations, which leads to improved performance (mediated cognitive relevance hypothesis), and (3) that performance effects of directed actions are influenced by the presence or absence of gesture production during mathematical explanations (moderated cognitive relevance hypothesis). We explore these hypotheses in an experiment where high school students (N = 85) evaluated the truth of geometry conjectures after performing cognitively relevant or cognitively irrelevant directed actions while playing a movement‐based video game. Contrary to the direct and mediated cognitive relevance hypotheses, we found no overall differences in performance or gesture production between relevant and irrelevant conditions. Consistent with the moderated cognitive relevance hypothesis, cognitive relevance influenced mathematical performance, as measured by the accuracy of students’ intuitions, insights, and the validity of their proofs, provided that students produced certain kinds of gestures during mathematical explanations (i.e., with explanatory gestures as the moderator). Implications for theories of grounded and embodied cognition and the design of embodied forms of educational interventions are discussed. (shrink)

The debate about the nature of the representational format of concepts seems to have reached an impasse. The debate faces two fundamental problems. Firstly, amodalists (i.e., those who argue that concepts are represented by amodal symbols) and modalists (i.e., those who see concepts as involving crucially representations including sensorimotor information) claim that the same empirical evidence is compatible with their views. Secondly, there is no shared understanding of what a modal or amodal format amounts to. Both camps recognize that the (...) two formats play essential roles in higher cognition, leading to an increasing number of hybrid proposals. In this paper, I argue that the existence of those fundamental problems should make us suspicious about a modal/amodal dichotomy. Also, I suggest that hybrid approaches, as they currently stand, do not provide suitable solutions to the impasse. Instead, we should overcome the dichotomy and treat the modal/amodal distinction as a graded phenomenon. I illustrate this hypothesis with an example of a cognitive-computational model of concepts based on the Predictive Processing framework. (shrink)

The aim of the paper is to determine how salient and non-salient figurative discourse nouns affect readers’ default response processing and oculo-graphic (eye-movement) reactions. Whereas the theories of the Graded Salience and the Defaultness Hypotheses, developed by R. Giora (Giora, 1999, 2003; Giora, Givoni, & Fein, 2015), have stimulated further research in the area of interpretive salience (Giora et al., 2015; Giora, Jaffe, Becker & Fein, 2018), the resonating influence of syntactic salience on default interpretations has been largely neglected. In (...) this study we provide corpus-based evidence followed by eye-tracking experiment verification, supportive of the synchronized influence of syntactic and lexical salience. The results show that default figurative responses in lexically salient positions may require more cognitive effort (longer fixations) if they are syntactically less salient. Literal responses to figurative nouns may also result from either weak lexical or syntactic salience of nouns. Therefore, apart from exemplifying resonance with lexical salience (in terms of lexical frequency, familiarity, conventionality, and prototypicality), the default figurative interpretations are also syntactically dependent. (shrink)