According to Thagard and Stewart :1–33, 2011), creativity results from the combination of neural representations, and combination results from convolution, an operation on vectors defined in the holographic reduced representation framework. They use these ideas to understand creativity as it occurs in many domains, and in particular in science. We argue that, because of its algebraic properties, convolution alone is ill-suited to the role proposed by Thagard and Stewart. The semantic pointer concept allows us to see how we can apply (...) the full range of HRR operations while keeping the modal representations so central to Thagard and Stewart’s theory. By adding another combination operation and using semantic pointers as the combinatorial basis, this modified version overcomes the limitations of the original theory and perhaps helps us explain aspects of creativity not covered by the original theory. While a priori reasons cast doubts on the use of HRR operations with modal representations :5039–5054, 1987) such as semantic pointers, recent models point in the other direction, allowing us to be optimistic about the success of the revised version. (shrink)

As modern deep networks become more complex, and get closer to human‐like capabilities in certain domains, the question arises as to how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language processing. We first present a diagnostic test dataset to examine the degree of abstract composable structure represented. Analyzing performance on these diagnostic tests indicates a lack of systematicity in representations (...) and decision rules, and reveals a set of heuristic strategies. We then investigate the effect of training distribution on learning these heuristic strategies, and we study changes in these representations with various augmentations to the training set. Our results reveal parallels to the analogous representations in people. We find that these systems can learn abstract rules and generalize them to new contexts under certain circumstances—similar to human zero‐shot reasoning. However, we also note some shortcomings in this generalization behavior—similar to human judgment errors like belief bias. Studying these parallels suggests new ways to understand psychological phenomena in humans as well as informs best strategies for building artificial intelligence with human‐like language understanding. (shrink)

Despite its long period of dominance, the details of associationism as developed by the British Empiricists in the 18th and 19th centuries are often ignored or forgotten today. Perhaps as a result, modern understandings of Empiricist associationism are often oversimplified. In fact, there is no single core view that can be viewed as definitional, or even weaker, as characteristic, of the tradition. The actual views of associationists in this tradition are much more diverse than any such view would allow, even (...) on fundamental aspects of the concept of association. This paper presents some of the most striking theoretical diversity and most significant discussions within the Empiricist associationist tradition. Recognizing this diversity, Empiricist associationism is best viewed as a project with shared questions and methods rather than a shared substantive theory of mind. This discussion can inform our understanding of the historical tradition, and also inform our current use of association and associationism. The concept of association is more flexible than it is often treated, and even its basic nature and function remain up for debate. (shrink)

Several approaches to implementing symbol-like representations in neurally plausible models have been proposed. These approaches include binding through synchrony, “mesh” binding, and conjunctive binding. Recent theoretical work has suggested that most of these methods will not scale well, that is, that they cannot encode structured representations using any of the tens of thousands of terms in the adult lexicon without making implausible resource assumptions. Here, we empirically demonstrate that the biologically plausible structured representations employed in the Semantic Pointer Architecture approach (...) to modeling cognition do scale appropriately. Specifically, we construct a spiking neural network of about 2.5 million neurons that employs semantic pointers to successfully encode and decode the main lexical relations in WordNet, which has over 100,000 terms. In addition, we show that the same representations can be employed to construct recursively structured sentences consisting of arbitrary WordNet concepts, while preserving the original lexical structure. We argue that these results suggest that semantic pointers are uniquely well-suited to providing a biologically plausible account of the structured representations that underwrite human cognition. (shrink)

Our target article argued that a genetically specified Universal Grammar (UG), capturing arbitrary properties of languages, is not tenable on evolutionary grounds, and that the close fit between language and language learners arises because language is shaped by the brain, rather than the reverse. Few commentaries defend a genetically specified UG. Some commentators argue that we underestimate the importance of processes of cultural transmission; some propose additional cognitive and brain mechanisms that may constrain language and perhaps differentiate humans from nonhuman (...) primates; and others argue that we overstate or understate the case against co-evolution of language genes. In engaging with these issues, we suggest that a new synthesis concerning the relationship between brains, genes, and language may be emerging. (shrink)

ObjectiveTo compare techniques for computing clustering and switching scores in terms of agreement, correlation, and empirical value as predictors of incident cognitive impairment.MethodsWe transcribed animal and letter F fluency recordings on 640 cases of ICI and matched controls from a national epidemiological study, amending each transcription with word timings. We then calculated clustering and switching scores, as well as scores indexing speed of responses, using techniques described in the literature. We evaluated agreement among the techniques with Cohen’s κ and calculated (...) correlations among the scores. After fitting a base model with raw scores, repetitions, and intrusions, we fit a series of Bayesian logistic regression models adding either clustering and switching scores or speed scores, comparing the models in terms of several metrics. We partitioned the ICI cases into acute and progressive cases and repeated the regression analysis for each group.ResultsFor animal fluency, we found that models with speed scores derived using the slope difference algorithm achieved the best values of the Watanabe–Akaike Information Criterion, but with good net reclassification improvement only for the progressive group. For letter fluency, different models excelled for prediction of acute and progressive cases. For acute cases, NRI was best for speed scores derived from a network model, while for progressive cases, the best model used clustering and switching scores derived from the same network model. Combining variables from the best animal and letter F models led to marginal improvements in model fit and NRI only for the all-cases and acute-cases analyses.ConclusionSpeed scores improve a base model for predicting progressive cognitive impairment from animal fluency. Letter fluency scores may provide complementary information. (shrink)

This paper argues that neurolinguistics has the potential to yield insights that can feed back into corpus-based Cognitive Linguistics. It starts by discussing how far the cognitive realism of probabilistic statements derived from corpus data currently goes. Against this background, it argues that the cognitive realism of usage-based models could be further enhanced through deeper engagement with neurolinguistics, but also highlights a number of common misconceptions about what neurolinguistics can and cannot do for linguistic theorizing.

The reconciliation of theories of concepts based on prototypes, exemplars, and theory-like structures is a longstanding problem in cognitive science. In response to this problem, researchers have recently tended to adopt either hybrid theories that combine various kinds of representational structure, or eliminative theories that replace concepts with a more finely grained taxonomy of mental representations. In this paper, we describe an alternative approach involving a single class of mental representations called “semantic pointers.” Semantic pointers are symbol-like representations that result (...) from the compression and recursive binding of perceptual, lexical, and motor representations, effectively integrating traditional connectionist and symbolic approaches. We present a computational model using semantic pointers that replicates experimental data from categorization studies involving each prior paradigm. We argue that a framework involving semantic pointers can provide a unified account of conceptual phenomena, and we compare our framework to existing alternatives in accounting for the scope, content, recursive combination, and neural implementation of concepts. (shrink)

Information is a peculiar quantity. Unlike matter and energy, which are conserved by the laws of physics, the aggregation of knowledge from many sources can in fact produce more information (synergy) or less (redundancy) than the sum of its parts. This feature can endow groups with problem‐solving strategies that are superior to those possible among noninteracting individuals and, in turn, may provide a selection drive toward collective cooperation and coordination. Here we explore the formal properties of information aggregation as a (...) general principle for explaining features of social organization. We quantify information in terms of the general formalism of information theory, which also prescribes the rules of how different pieces of evidence inform the solution of a given problem. We then show how several canonical examples of collective cognition and coordination can be understood through principles of minimization of uncertainty (maximization of predictability) under information pooling over many individuals. We discuss in some detail how collective coordination in swarms, markets, natural language processing, and collaborative filtering may be guided by the optimal aggregation of information in social collectives. We also identify circumstances when these processes fail, leading, for example, to inefficient markets. The contrast to approaches to understand coordination and collaboration via decision and game theory is also briefly discussed. (shrink)

Lexical ambiguity—the phenomenon of a single word having multiple, distinguishable senses—is pervasive in language. Both the degree of ambiguity of a word (roughly, its number of senses) and the relatedness of those senses have been found to have widespread effects on language acquisition and processing. Recently, distributional approaches to semantics, in which a word's meaning is determined by its contexts, have led to successful research quantifying the degree of ambiguity, but these measures have not distinguished between the ambiguity of words (...) with multiple related senses versus multiple unrelated meanings. In this work, we present the first assessment of whether distributional meaning representations can capture the ambiguity structure of a word, including both the number and relatedness of senses. On a very large sample of English words, we find that some, but not all, distributional semantic representations that we test exhibit detectable differences between sets of monosemes (unambiguous words; N = 964), polysemes (with multiple related senses; N = 4,096), and homonyms (with multiple unrelated senses; N = 355). Our findings begin to answer open questions from earlier work regarding whether distributional semantic representations of words, which successfully capture various semantic relationships, also reflect fine‐grained aspects of meaning structure that influence human behavior. Our findings emphasize the importance of measuring whether proposed lexical representations capture such distinctions: In addition to standard benchmarks that test the similarity structure of distributional semantic models, we need to also consider whether they have cognitively plausible ambiguity structure. (shrink)

Computational models of meaning trained on naturally occurring text successfully model human performance on tasks involving simple similarity measures, but they characterize meaning in terms of undifferentiated bags of words or topical dimensions. This has led some to question their psychological plausibility (Murphy, 2002;Schunn, 1999). We present here a fully automatic method for extracting a structured and comprehensive set of concept descriptions directly from an English part‐of‐speech‐tagged corpus. Concepts are characterized by weighted properties, enriched with concept–property types that approximate classical (...) relations such as hypernymy and function. Our model outperforms comparable algorithms in cognitive tasks pertaining not only to concept‐internal structures (discovering properties of concepts, grouping properties by property type) but also to inter‐concept relations (clustering into superordinates), suggesting the empirical validity of the property‐based approach. (shrink)

Computational models of meaning trained on naturally occurring text successfully model human performance on tasks involving simple similarity measures, but they characterize meaning in terms of undifferentiated bags of words or topical dimensions. This has led some to question their psychological plausibility (Murphy, 2002;Schunn, 1999). We present here a fully automatic method for extracting a structured and comprehensive set of concept descriptions directly from an English part‐of‐speech‐tagged corpus. Concepts are characterized by weighted properties, enriched with concept–property types that approximate classical (...) relations such as hypernymy and function. Our model outperforms comparable algorithms in cognitive tasks pertaining not only to concept‐internal structures (discovering properties of concepts, grouping properties by property type) but also to inter‐concept relations (clustering into superordinates), suggesting the empirical validity of the property‐based approach. (shrink)

In this paper, we describe a model that learns semantic representations from the distributional statistics of language. This model, however, goes beyond the common bag‐of‐words paradigm, and infers semantic representations by taking into account the inherent sequential nature of linguistic data. The model we describe, which we refer to as a Hidden Markov Topics model, is a natural extension of the current state of the art in Bayesian bag‐of‐words models, that is, the Topics model of Griffiths, Steyvers, and Tenenbaum (2007), (...) preserving its strengths while extending its scope to incorporate more fine‐grained linguistic information. (shrink)

Over the past 15 years, there have been two increasingly popular approaches to the study of meaning in cognitive science. One, based on theories of embodied cognition, treats meaning as a simulation of perceptual and motor states. An alternative approach treats meaning as a consequence of the statistical distribution of words across spoken and written language. On the surface, these appear to be opposing scientific paradigms. In this review, we aim to show how recent cross-disciplinary developments have done much to (...) reconcile these two approaches. The foundation to these developments has been the recognition that intralinguistic distributional and sensory-motor data are interdependent. We describe recent work in philosophy, psychology, cognitive neuroscience, and computational modeling that are all based on or consistent with this conclusion. We conclude by considering some possible directions for future research that arise as a consequence of these developments. (shrink)

Distributional models of lexical semantics are capable of acquiring sophisticated representations of word meanings. The main theoretical insight provided by these models is that they demonstrate the systematic connection between the knowledge that people acquire and the experience that they have with the natural language environment. However, linguistic experience is inherently variable and differs radically across people due to demographic and cultural variables. Recently, distributional models have been used to examine how word meanings vary across languages and it was found (...) that there is considerable variability in the meanings of words across languages for most semantic categories. The goal of this article is to examine how variable word meanings are across individual language users within a single language. This was accomplished by assembling 500 individual user corpora attained from the online forum Reddit. Each user corpus ranged between 3.8 and 32.3 million words each, and a count‐based distributional framework was used to extract word meanings for each user. These representations were then used to estimate the semantic alignment of word meanings across individual language users. It was found that there are significant levels of relativity in word meanings across individuals, and these differences are partially explained by other psycholinguistic factors, such as concreteness, semantic diversity, and social aspects of language usage. These results point to word meanings being fundamentally relative and contextually fluid, with this relativeness being related to the individualized nature of linguistic experience. (shrink)

The semantic fluency task is commonly used as a measure of one’s ability to retrieve semantic concepts. While performance is typically scored by counting the total number of responses, the ordering of responses can be used to estimate how individuals or groups organize semantic concepts within a category. I provide an overview of this methodology, using Alzheimer’s disease as a case study for how the approach can help advance theoretical questions about the nature of semantic representation. However, many open questions (...) surrounding the validity and reliability of this approach remain unresolved. (shrink)

This article describes the use of continuous vector space models for reasoning with a formal knowledge base. The practical significance of these models is that they support fast, approximate but robust inference and hypothesis generation, which is complementary to the slow, exact, but sometimes brittle behaviour of more traditional deduction engines such as theorem provers.The article explains the way logical connectives can be used in semantic vector models, and summarizes the development of Predication-based Semantic Indexing, which involves the use of (...) Vector Symbolic Architectures to represent the concepts and relationships from a knowledge base of subject-predicate-object triples. Experiments show that the use of continuous models for formal reasoning is not only possible, but already demonstrably effective for some recognized informatics tasks, and showing promise in other traditional problem areas. Examples described in this article include: predicting new uses for existing drugs in biomedical informatics; removing unwanted meanings from search results in information retrieval and concept navigation; type inference from attributes; comparing words based on their orthography; and representing tabular data, including modelling numerical values.The algorithms and techniques described in this article are all publicly released and freely available in the Semantic Vectors open-source software package.1. (shrink)

An incoherent visualization is when aspects of different senses of a word are present in the same visualization. We describe and implement a new model of creating contextual coherence in the visual imagination called Coherencer, based on the SOILIE model of imagination. We show that Coherencer is able to generate scene descriptions that are more coherent than SOILIE's original approach as well as a parallel connectionist algorithm that is considered competitive in the literature on general coherence. We also show that (...) co-occurrence probabilities are a better association representation than holographic vectors and that better models of coherence improve the resulting output independent of the association type that is used. Theoretically, we show that Coherencer is consistent with other models of cognitive generation. In particular, Coherencer is a similar, but more cognitively plausible model than the C3 model of concept combination created by Costello and Keane. We show that Coherencer is also consistent with both the modal schematic indices of perceptual symbol systems theory and the amodal contextual constraints of Thagard's theory of coherence. Finally, we describe how Coherencer is consistent with contemporary research on the hippocampus, and we show evidence that the process of making a visualization coherent is serial. (shrink)

A large literature suggests that the organization of words in semantic memory, reflecting meaningful relations among words and the concepts to which they refer, supports many cognitive processes, including memory encoding and retrieval, word learning, and inferential reasoning. The co‐activation of related items has been proposed as a mechanism by which semantic knowledge influences cognition, and contemporary accounts of semantic knowledge propose that this co‐activation is graded—that it depends on how strongly related the items are in semantic memory. Prior research (...) with adults yielded evidence supporting this prediction; however, there is currently no evidence of graded co‐activation early in development. This study provides the first evidence that in children the co‐activation of related items depends on their relational strength in semantic memory. Participants (N = 84, age range: 3–9 years) were asked to identify a target (e.g., bone) amid distractors. Children's responses were slowed down by the presence of a related distractor (e.g., puppy) relative to unrelated distractors (e.g., flower)—suggesting that children co‐activated related items upon hearing the name of the target. Importantly, the degree of this co‐activation was predicted by the strength of the target–distractor relation, such that distractors more strongly related to the targets slowed down children to a larger extent. These findings have important implications for understanding how organized semantic knowledge affects other cognitive processes across development. (shrink)

The pervasive use of distributional semantic models or word embeddings for both cognitive modeling and practical application is because of their remarkable ability to represent the meanings of words. However, relatively little effort has been made to explore what types of information are encoded in distributional word vectors. Knowing the internal knowledge embedded in word vectors is important for cognitive modeling using distributional semantic models. Therefore, in this paper, we attempt to identify the knowledge encoded in word vectors by conducting (...) a computational experiment using Binder et al.'s (2016) featural conceptual representations based on neurobiologically motivated attributes. In an experiment, these conceptual vectors are predicted from text‐based word vectors using a neural network and linear transformation, and prediction performance is compared among various types of information. The analysis demonstrates that abstract information is generally predicted more accurately by word vectors than perceptual and spatiotemporal information, and specifically, the prediction accuracy of cognitive and social information is higher. Emotional information is also found to be successfully predicted for abstract words. These results indicate that language can be a major source of knowledge about abstract attributes, and they support the recent view that emphasizes the importance of language for abstract concepts. Furthermore, we show that word vectors can capture some types of perceptual and spatiotemporal information about concrete concepts and some relevant word categories. This suggests that language statistics can encode more perceptual knowledge than often expected. (shrink)

The organization of our knowledge about the world into an interconnected network of concepts linked by relations profoundly impacts many facets of cognition, including attention, memory retrieval, reasoning, and learning. It is therefore crucial to understand how organized semantic representations are acquired. The present experiment investigated the contributions of readily observable environmental statistical regularities to semantic organization in childhood. Specifically, we investigated whether co‐occurrence regularities with which entities or their labels more reliably occur together than with others (a) contribute to (...) relations between concepts independently and (b) contribute to relations between concepts belonging to the same taxonomic category. Using child‐directed speech corpora to estimate reliable co‐occurrences between labels for familiar items, we constructed triads consisting of a target, a related distractor, and an unrelated distractor in which targets and related distractors consistently co‐occurred (e.g., sock‐foot), belonged to the same taxonomic category (e.g., sock‐coat), or both (e.g., sock‐shoe). We used an implicit, eye‐gaze measure of relations between concepts based on the degree to which children (N = 72, age 4–7 years) looked at related versus unrelated distractors when asked to look for a target. The results indicated that co‐occurrence both independently contributes to relations between concepts and contributes to relations between concepts belonging to the same taxonomic category. These findings suggest that sensitivity to the regularity with which different entities co‐occur in children's environments shapes the organization of semantic knowledge during development. Implications for theoretical accounts and empirical investigations of semantic organization are discussed. (shrink)

The contents and structure of semantic memory have been the focus of much recent research, with major advances in the development of distributional models, which use word co‐occurrence information as a window into the semantics of language. In parallel, connectionist modeling has extended our knowledge of the processes engaged in semantic activation. However, these two lines of investigation have rarely been brought together. Here, we describe a processing model based on distributional semantics in which activation spreads throughout a semantic network, (...) as dictated by the patterns of semantic similarity between words. We show that the activation profile of the network, measured at various time points, can successfully account for response times in lexical and semantic decision tasks, as well as for subjective concreteness and imageability ratings. We also show that the dynamics of the network is predictive of performance in relational semantic tasks, such as similarity/relatedness rating. Our results indicate that bringing together distributional semantic networks and spreading of activation provides a good fit to both automatic lexical processing (as indexed by lexical and semantic decisions) as well as more deliberate processing (as indexed by ratings), above and beyond what has been reported for previous models that take into account only similarity resulting from network structure. (shrink)

Abstract Since their inception, distributional models of semantics have been criticized as inadequate cognitive theories of human semantic learning and representation. A principal challenge is that the representations derived by distributional models are purely symbolic and are not grounded in perception and action; this challenge has led many to favor feature-based models of semantic representation. We argue that the amount of perceptual and other semantic information that can be learned from purely distributional statistics has been underappreciated. We compare the representations (...) of three feature-based and nine distributional models using a semantic clustering task. Several distributional models demonstrated semantic clustering comparable with clustering-based on feature-based representations. Furthermore, when trained on child-directed speech, the same distributional models perform as well as sensorimotor-based feature representations of children’s lexical semantic knowledge. These results suggest that, to a large extent, information relevant for extracting semantic categories is redundantly coded in perceptual and linguistic experience. Detailed analyses of the semantic clusters of the feature-based and distributional models also reveal that the models make use of complementary cues to semantic organization from the two data streams. Rather than conceptualizing feature-based and distributional models as competing theories, we argue that future focus should be on understanding the cognitive mechanisms humans use to integrate the two sources. (shrink)

Similarity is one of the most important relations humans perceive, arguably subserving category learning and categorization, generalization and discrimination, judgment and decision making, and other cognitive functions. Researchers have proposed a wide range of representations and metrics that could be at play in similarity judgment, yet have not comprehensively compared the power of these representations and metrics for predicting similarity within and across different semantic categories. We performed such a comparison by pairing nine prominent vector semantic representations with seven established (...) similarity metrics that could operate on these representations, as well as supervised methods for dimensional weighting in the similarity function. This approach yields a factorial model structure with 126 distinct representation‐metric pairs, which we tested on a novel dataset of similarity judgments between pairs of cohyponymic words in eight categories. We found that cosine similarity and Pearson correlation were the overall best performing unweighted similarity functions, and that word vectors derived from free association norms often outperformed word vectors derived from text (including those specialized for similarity). Importantly, models that used human similarity judgments to learn category‐specific weights on dimensions yielded substantially better predictions than all unweighted approaches across all types of similarity functions and representations, although dimension weights did not generalize well across semantic categories, suggesting strong category context effects in similarity judgment. We discuss implications of these results for cognitive modeling and natural language processing, as well as for theories of the representations and metrics involved in similarity. (shrink)

This commentary gives a personal perspective on modeling and modeling developments in cognitive science, starting in the 1950s, but focusing on the author’s personal views of modeling since training in the late 1960s, and particularly focusing on advances since the official founding of the Cognitive Science Society. The range and variety of modeling approaches in use today are remarkable, and for many, bewildering. Yet to come to anything approaching adequate insights into the infinitely complex fields of mind, brain, and intelligent (...) systems, an extremely wide array of modeling approaches is vital and necessary. (shrink)

Computational models of memory presented in this issue reflect varied empirical data and levels of representation. From mathematical models to neural and cognitive architectures, all aim to converge on a unified theory of the mind.

Recent studies of semantic memory have investigated two theories of optimal search adopted from the animal foraging literature: Lévy flights and marginal value theorem. Each theory makes different simplifying assumptions and addresses different findings in search behaviors. In this study, an experiment is conducted to test whether clustering in semantic memory may play a role in evidence for both theories. Labeled magnets and a whiteboard were used to elicit spatial representations of semantic knowledge about animals. Category recall sequences from a (...) separate experiment were used to trace search paths over the spatial representations of animal knowledge. Results showed that spatial distances between animal names arranged on the whiteboard were correlated with inter-response intervals during category recall, and distributions of both dependent measures approximated inverse power laws associated with Lévy flights. In addition, IRIs were relatively shorter when paths first entered animal clusters, and longer when they exited clusters, which is consistent with marginal value theorem. In conclusion, area-restricted searches over clustered semantic spaces may account for two different patterns of results interpreted as supporting two different theories of optimal memory foraging. (shrink)

The words in children's language learning environments are strongly predictive of cognitive development and school achievement. But how do we measure language environments and do so at the scale of the many words that children hear day in, day out? The quantity and quality of words in a child's input are typically measured in terms of total amount of talk and the lexical diversity in that talk. There are disagreements in the literature whether amount or diversity is the more critical (...) measure of the input. Here we analyze the properties of a large corpus of speech to children and simulate learning environments that differ in amount of talk per unit time, lexical diversity, and the contexts of talk. The central conclusion is that what researchers need to theoretically understand, measure, and change is not the total amount of words, or the diversity of words, but the function that relates total words to the diversity of words, and how that function changes across different contexts of talk. (shrink)

Over the past two decades, researchers have made great advances in the area of computational methods for extracting meaning from text. This research has to a large extent been spurred by the development of latent semantic analysis (LSA), a method for extracting and representing the meaning of words using statistical computations applied to large corpora of text. Since the advent of LSA, researchers have developed and tested alternative statistical methods designed to detect and analyze meaning in text corpora. This research (...) exemplifies how statistical models of semantics play an important role in our understanding of cognition and contribute to the field of cognitive science. Importantly, these models afford large-scale representations of human knowledge and allow researchers to explore various questions regarding knowledge, discourse processing, text comprehension, and language. This topic includes the latest progress by the leading researchers in the endeavor to go beyond LSA. (shrink)

Statistical topic models provide a general data - driven framework for automated discovery of high-level knowledge from large collections of text documents. Although topic models can potentially discover a broad range of themes in a data set, the interpretability of the learned topics is not always ideal. Human-defined concepts, however, tend to be semantically richer due to careful selection of words that define the concepts, but they may not span the themes in a data set exhaustively. In this study, we (...) review a new probabilistic framework for combining a hierarchy of human-defined semantic concepts with a statistical topic model to seek the best of both worlds. Results indicate that this combination leads to systematic improvements in generalization performance as well as enabling new techniques for inferring and visualizing the content of a document. (shrink)

The claim that language is shaped by the brain is weakened by lack of clear specification of what necessary and sufficient properties the brain actually imposes. To account for human intellectual superiority, it is proposed that language did require special brain evolution (Deacon 1997), but that what evolved was a merely quantitative change rather than a radically new invention.

Topics in Cognitive Science, Volume 14, Issue 1, Page 54-77, January 2022.

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this (...) manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach. (shrink)

We argue that word meanings are not stored in a mental lexicon but are generated in the context of working memory from long-term memory traces that record our experience with words. Current statistical models of semantics, such as latent semantic analysis and the Topic model, describe what is stored in long-term memory. The CI-2 model describes how this information is used to construct sentence meanings. This model is a dual-memory model, in that it distinguishes between a gist level and an (...) explicit level. It also incorporates syntactic information about how words are used, derived from dependency grammar. The construction of meaning is conceptualized as feature sampling from the explicit memory traces, with the constraint that the sampling must be contextually relevant both semantically and syntactically. Semantic relevance is achieved by sampling topically relevant features; local syntactic constraints as expressed by dependency relations ensure syntactic relevance. (shrink)

We demonstrate that the key components of cognitive architectures (declarative and procedural memory) and their key capabilities (learning, memory retrieval, probability judgment, and utility estimation) can be implemented as algebraic operations on vectors and tensors in a high‐dimensional space using a distributional semantics model. High‐dimensional vector spaces underlie the success of modern machine learning techniques based on deep learning. However, while neural networks have an impressive ability to process data to find patterns, they do not typically model high‐level cognition, and (...) it is often unclear how they work. Symbolic cognitive architectures can capture the complexities of high‐level cognition and provide human‐readable, explainable models, but scale poorly to naturalistic, non‐symbolic, or big data. Vector‐symbolic architectures, where symbols are represented as vectors, bridge the gap between the two approaches. We posit that cognitive architectures, if implemented in a vector‐space model, represent a useful, explanatory model of the internal representations of otherwise opaque neural architectures. Our proposed model, Holographic Declarative Memory (HDM), is a vector‐space model based on distributional semantics. HDM accounts for primacy and recency effects in free recall, the fan effect in recognition, probability judgments, and human performance on an iterated decision task. HDM provides a flexible, scalable alternative to symbolic cognitive architectures at a level of description that bridges symbolic, quantum, and neural models of cognition. (shrink)