We welcome the proposal to use forward models to understand predictive processes in language processing. However, Pickering & Garrod (P&G) miss the opportunity to provide a strong framework for future work. Forward models need to be pursued in the context of learning. This naturally leads to questions about what prediction error these models aim to minimize.

Our target article proposed that language production and comprehension are interwoven, with speakers making predictions of their own utterances and comprehenders making predictions of other people's utterances at different linguistic levels. Here, we respond to comments about such issues as cognitive architecture and its neural basis, learning and development, monitoring, the nature of forward models, communicative intentions, and dialogue.

A word often expresses many different morphological functions. Which part of a word contributes to which part of the overall meaning is not always clear, which raises the question as to how such functions are learned. While linguistic studies tacitly assume the co-occurrence of cues and outcomes to suffice in learning these functions (Baer-Henney, Kügler, & van de Vijver, 2015; Baer-Henney & van de Vijver, 2012), error-driven learning suggests that contingency rather than contiguity is crucial (Nixon, 2020; Ramscar, Yarlett, Dye, (...) Denny, & Thorpe, 2010). In error-driven learning, cues gain association strength if they predict a certain outcome, and they lose strength if the outcome is absent. This reduction of association strength is called unlearning. So far, it is unclear if such unlearning has consequences for cue–outcome associations beyond the ones that get reduced. To test for such consequences of unlearning, we taught participants morphophonological patterns in an artificial language learning experiment. In one block, the cues to two morphological outcomes—plural and diminutive—co-occurred within the same word forms. In another block, a single cue to only one of these two outcomes was presented in a different set of word forms. We wanted to find out, if participants unlearn this cue's association with the outcome that is not predicted by the cue alone, and if this allows the absent cue to be associated with the absent outcome. Our results show that if unlearning was possible, participants learned that the absent cue predicts the absent outcome better than if no unlearning was possible. This effect was stronger if the unlearned cue was more salient. This shows that unlearning takes place even if no alternative cues to an absent outcome are provided, which highlights that learners take both positive and negative evidence into account—as predicted by domain general error-driven learning. (shrink)

Sequence learning is fundamental to a wide range of cognitive functions. Explaining how sequences—and the relations between the elements they comprise—are learned is a fundamental challenge to cognitive science. However, although hundreds of articles addressing this question are published each year, the actual learning mechanisms involved in the learning of sequences are rarely investigated. We present three experiments that seek to examine these mechanisms during a typing task. Experiments 1 and 2 tested learning during typing single letters on each trial. (...) Experiment 3 tested for “chunking” of these letters into “words.” The results of these experiments were used to examine the mechanisms that could best account for them, with a focus on two particular proposals: statistical transitional probability learning and discriminative error-driven learning. Experiments 1 and 2 showed that error-driven learning was a better predictor of response latencies than either n-gram frequencies or transitional probabilities. No evidence for chunking was found in Experiment 3, probably due to interspersing visual cues with the motor response. In addition, learning occurred across a greater distance in Experiment 1 than Experiment 2, suggesting that the greater predictability that comes with increased structure leads to greater learnability. These results shed new light on the mechanism responsible for sequence learning. Despite the widely held assumption that transitional probability learning is essential to this process, the present results suggest instead that the sequences are learned through a process of discriminative learning, involving prediction and feedback from prediction error. (shrink)

In 2006, Perruchet and Pacton (2006) asked whether implicit learning and statistical learning represent two approaches to the same phenomenon. This article represents an important follow‐up to their seminal review article. As in the previous paper, the focus is on the formation of elementary cognitive units. Both approaches favor different explanations on what these units consist of and how they are formed. Perruchet weighs up the evidence for different explanations and concludes with a helpful agenda for future research.

Determining optimal units of representing morphologically complex words in the mental lexicon is a central question in psycholinguistics. Here, we utilize advances in computational sciences to study human morphological processing using statistical models of morphology, particularly the unsupervised Morfessor model that works on the principle of optimization. The aim was to see what kind of model structure corresponds best to human word recognition costs for multimorphemic Finnish nouns: a model incorporating units resembling linguistically defined morphemes, a whole-word model, or a (...) model that seeks for an optimal balance between these two extremes. Our results showed that human word recognition was predicted best by a combination of two models: a model that decomposes words at some morpheme boundaries while keeping others unsegmented and a whole-word model. The results support dual-route models that assume that both decomposed and full-form representations are utilized to optimally process complex words within the mental lexicon. (shrink)

By the time they reach early adulthood, English speakers are familiar with the meaning of thousands of words. In the last decades, computational simulations known as distributional semantic models have demonstrated that it is possible to induce word meaning representations solely from word co-occurrence statistics extracted from a large amount of text. However, while these models learn in batch mode from large corpora, human word learning proceeds incrementally after minimal exposure to new words. In this study, we run a set (...) of experiments investigating whether minimal distributional evidence from very short passages suffices to trigger successful word learning in subjects, testing their linguistic and visual intuitions about the concepts associated with new words. After confirming that subjects are indeed very efficient distributional learners even from small amounts of evidence, we test a DSM on the same multimodal task, finding that it behaves in a remarkable human-like way. We conclude that DSMs provide a convincing computational account of word learning even at the early stages in which a word is first encountered, and the way they build meaning representations can offer new insights into human language acquisition. (shrink)

Corpus surveys have shown that the exact forms with which idioms are realized are subject to variation. We report a rating experiment showing that such alternative realizations have varying degrees of acceptability. Idiom variation challenges processing theories associating idioms with fixed multi-word form units, fixed configurations of words, or fixed superlemmas, as they do not explain how it can be that speakers produce variant forms that listeners can still make sense of. A computational model simulating comprehension with naive discriminative learning (...) is introduced that provides an explanation for the different degrees of acceptability of several idiom variant types. Implications for multi-word units in general are discussed. (shrink)

Over the past 10 years, Cognitive Linguistics has taken a quantitative turn. Yet, concerns have been raised that this preoccupation with quantification and modelling may not bring us any closer to understanding how language works. We show that this objection is unfounded, especially if we rely on modelling techniques based on biologically and psychologically plausible learning algorithms. These make it possible to take a quantitative approach, while generating and testing specific hypotheses that will advance our understanding of how knowledge of (...) language emerges from exposure to usage. (shrink)

As adults age, their performance on many psychometric tests changes systematically, a finding that is widely taken to reveal that cognitive information-processing capacities decline across adulthood. Contrary to this, we suggest that older adults'; changing performance reflects memory search demands, which escalate as experience grows. A series of simulations show how the performance patterns observed across adulthood emerge naturally in learning models as they acquire knowledge. The simulations correctly identify greater variation in the cognitive performance of older adults, and successfully (...) predict that older adults will show greater sensitivity to fine-grained differences in the properties of test stimuli than younger adults. Our results indicate that older adults'; performance on cognitive tests reflects the predictable consequences of learning on information-processing, and not cognitive decline. We consider the implications of this for our scientific and cultural understanding of aging. (shrink)

I have argued that orthographic processing cannot be understood and modeled without considering the manner in which orthographic structure represents phonological, semantic, and morphological information in a given writing system. A reading theory, therefore, must be a theory of the interaction of the reader with his/her linguistic environment. This outlines a novel approach to studying and modeling visual word recognition, an approach that focuses on the common cognitive principles involved in processing printed words across different writing systems. These claims were (...) challenged by several commentaries that contested the merits of my general theoretical agenda, the relevance of the evolution of writing systems, and the plausibility of finding commonalities in reading across orthographies. Other commentaries extended the scope of the debate by bringing into the discussion additional perspectives. My response addresses all these issues. By considering the constraints of neurobiology on modeling reading, developmental data, and a large scope of cross-linguistic evidence, I argue that front-end implementations of orthographic processing that do not stem from a comprehensive theory of the complex information conveyed by writing systems do not present a viable approach for understanding reading. The common principles by which writing systems have evolved to represent orthographic, phonological, and semantic information in a language reveal the critical distributional characteristics of orthographic structure that govern reading behavior. Models of reading should thus be learning models, primarily constrained by cross-linguistic developmental evidence that describes how the statistical properties of writing systems shape the characteristics of orthographic processing. When this approach is adopted, a universal model of reading is possible. (shrink)

Cognitive science depends on abstractions made from the complex reality of human behaviour. Cognitive scientists typically wish the abstractions in their theories to be universals, but seldom attend to the ontology of universals. Two sorts of universal, resulting from Galilean abstraction and materialist abstraction respectively, are available in the philosophical literature: the abstract universal—the one-over-many universal—is the universal conventionally employed by cognitive scientists; in contrast, a concrete universal is a material entity that can appear within the set of entities it (...) describes, of which it represents the essential, paradigmatic case. The potential role of concrete universals in cognitive science is discussed. (shrink)

This special issue of Cognitive Linguistics reexamines the notions of uniformity and variability within morphological systems from a cognitive linguistic standpoint. It challenges traditional perspectives that regard morphological variability as mere deviations from the norm, suggesting instead that such variability is systematic and shaped by external influences including language acquisition and processing constraints. The contributions in this issue promote a shift from isolated analysis to a holistic view of paradigms, classes, and systems, advocating for a framework where morphological structures are (...) seen as integral to communicative and functional aspects of language. By accounting for the broad adaptive dynamics of language systems, the complex interplay between uniformity and variability is revealed as an inherent aspect of language usage. (shrink)

The present study combined magnetoencephalography recordings with fast periodic visual stimulation to investigate automatic neural responses to morphemes in developing and skilled readers. Native English-speaking children and adults were presented with rapid streams of base stimuli interleaved periodically with oddballs. In a manipulation-check condition, tapping into word recognition, oddballs featured familiar words embedded in a stream of consonant strings. In the experimental conditions, the contrast between oddball and base stimuli was manipulated in order to probe selective stem and suffix identification (...) in morphologically structured pseudowords. Neural responses at the oddball frequency and harmonics were analyzed at the sensor level using non-parametric cluster-based permutation tests. As expected, results in the manipulation-check condition revealed a word-selective response reflected by a predominantly left-lateralized cluster that emerged over temporal, parietal, and occipital sensors in both children and adults. However, across the experimental conditions, results yielded a differential pattern of oddball responses in developing and skilled readers. Children displayed a significant response that emerged in a mostly central occipital cluster for the condition tracking stem identification in the presence of suffixes. In contrast, adult participants showed a significant response that emerged in a cluster located in central and left occipital sensors for the condition tracking suffix identification in the presence of stems. The present results suggest that while the morpheme identification system in Grade 5–6 children is not yet adult-like, it is sufficiently mature to automatically analyze the morphemic structure of novel letter strings. These findings are discussed in the context of theoretical accounts of morphological processing across reading development. (shrink)

The input to phonological reasoning are alternations, that is, variations in the pronunciation of related words, such as in electri[k] ‐ electri[s]‐ity. But phonologists cannot agree what counts as a relevant alternation: the issue is highly contentious despite a research record of over 50 years. We believe that the experimental setup presented may contribute to this debate based on a kind of evidence that was not brought to bear to date. Our experiment was thus designed to distinguish between alternations where (...) phonological computation plays no role, and those where it has contributed to language production. The design manipulates two factors that to date have not been considered in experimental studies of language production: linguistic complexity and alternation specificity. The former is understood as extra processing demands issued by two types of linguistic activity, morphosyntactic and phonological. Our results show that reaction time latencies are longer when participants are burdened with both morphosyntactic and phonological tasks than when they carry out just a morphosyntactic task, and they are still shorter in absence of both types of demands. These results allowed us to address alternation specificity, that is, the fact that different alternations (within a language or across languages) may be driven by different production routines (an idea that is consensual among linguists but underdeveloped in the psycholinguistic literature). Our study shows that four different alternations in French produce alternation‐specific signatures in reaction time latencies. These findings may thus redound to the advantage of psycholinguistic studies by identifying two new factors, as well as produce results that speak to the linguistic quarrels. (shrink)

One of the central debates in the cognitive science of language has revolved around the nature of human linguistic competence. Whether syntactic competence should be characterized by abstract hierarchical structures or reduced to surface linear strings has been actively debated, but the nature of morphological competence has been insufficiently appreciated despite the parallel question in the cognitive science literature. In this paper, in order to investigate whether morphological competence should be characterized by abstract hierarchical structures, we conducted the crowdsourced acceptability (...) judgment experiment on morphologically complex words and evaluated five computational models of morphological competence against human acceptability judgments: Character Markov Models (Character), Syllable Markov Models (Syllable), Morpheme Markov Models (Morpheme), Hidden Markov Models (HMM), and Probabilistic Context-Free Grammars (PCFG). Our psycholinguistic experimentation and computational modeling demonstrated that "morphous" computational models with morpheme units outperformed "amorphous" computational models without morpheme units and, importantly, PCFG with hierarchical structures most accurately explained human acceptability judgments on several evaluation metrics, especially for morphologically complex words with nested morphological structures. Those results strongly suggest that human morphological competence should be characterized by abstract hierarchical structures internally generated by the grammar, not reduced to surface linear strings externally attested in large corpora. (shrink)

Many models of word recognition assume that processing proceeds sequentially from analysis of form to analysis of meaning. In the context of morphological processing, this implies that morphemes are processed as units of form prior to any influence of their meanings. Some interpret the apparent absence of differences in recognition latencies to targets (SNEAK) in form and semantically similar (sneaky-SNEAK) and in form similar and semantically dissimilar (sneaker-SNEAK) prime contexts at an SOA of 48 ms as consistent with this claim. (...) To determine the time course over which degree of semantic similarity between morphologically structured primes and their targets influences recognition in the forward masked priming variant of the lexical decision paradigm, we compared facilitation for the same targets after semantically similar and dissimilar primes across a range of SOAs. The effect of shared semantics on recognition latency increased linearly with SOA when long SOAs were intermixed (Exp. 1) and latencies were significantly faster after semantically similar than dissimilar primes at homogeneous SOAs of 48 ms (Exp. 2) and 34 ms (Exp. 3). Results limit the scope of form-then-semantics models of recognition and demonstrate that semantics influences even the very early stages of recognition. Finally, once general behavior across trials has been accounted for, we fail to provide evidence for individual differences in morphological processing that can be linked to measures of reading proficiency. (shrink)

In the last decade, reading research has seen a paradigmatic shift. A new wave of computational models of orthographic processing that offer various forms of noisy position or context-sensitive coding have revolutionized the field of visual word recognition. The influx of such models stems mainly from consistent findings, coming mostly from European languages, regarding an apparent insensitivity of skilled readers to letter order. Underlying the current revolution is the theoretical assumption that the insensitivity of readers to letter order reflects the (...) special way in which the human brain encodes the position of letters in printed words. The present article discusses the theoretical shortcomings and misconceptions of this approach to visual word recognition. A systematic review of data obtained from a variety of languages demonstrates that letter-order insensitivity is neither a general property of the cognitive system nor a property of the brain in encoding letters. Rather, it is a variant and idiosyncratic characteristic of some languages, mostly European, reflecting a strategy of optimizing encoding resources, given the specific structure of words. Since the main goal of reading research is to develop theories that describe the fundamental and invariant phenomena of reading across orthographies, an alternative approach to model visual word recognition is offered. The dimensions of a possible universal model of reading, which outlines the common cognitive operations involved in orthographic processing in all writing systems, are discussed. (shrink)

It is often assumed that graphemes are a crucial level of orthographic representation above letters. Current connectionist models of reading, however, do not address how the mapping from letters to graphemes is learned. One major challenge for computational modeling is therefore developing a model that learns this mapping and can assign the graphemes to linguistically meaningful categories such as the onset, vowel, and coda of a syllable. Here, we present a model that learns to do this in English for strings (...) of any letter length and any number of syllables. The model is evaluated on error rates and further validated on the results of a behavioral experiment designed to examine ambiguities in the processing of graphemes. The results show that the model (a) chooses graphemes from letter strings with a high level of accuracy, even when trained on only a small portion of the English lexicon; (b) chooses a similar set of graphemes as people do in situations where different graphemes can potentially be selected; (c) predicts orthographic effects on segmentation which are found in human data; and (d) can be readily integrated into a full-blown model of multi-syllabic reading aloud such as CDP++ (Perry, Ziegler, & Zorzi, 2010). Altogether, these results suggest that the model provides a plausible hypothesis for the kind of computations that underlie the use of graphemes in skilled reading. (shrink)

In the 1990s, language acquisition researchers and theoretical linguists developed an interest in learning mechanisms, and learning theorists rediscovered the verbal learning tradition. Nonetheless, learning theory and language acquisition continued to develop largely independently, which has stymied progress in both fields. However, exciting progress is happening in applying learning theory to language, and, more recently, in using language learning data to advance domain‐general learning theory. These developments raise hopes for a bidirectional flow of information between the fields. The importance of (...) language data for learning theory and of learning theory for understanding language is briefly discussed. (shrink)

The uncertainty associated with paradigmatic families has been shown to correlate with their phonetic characteristics in speech, suggesting that representations of complex sublexical relations between words are part of speaker knowledge. To better understand this, recent studies have used two-layer neural network models to examine the way paradigmatic uncertainty emerges in learning. However, to date this work has largely ignored the way choices about the representation of inflectional and grammatical functions (IFS) in models strongly influence what they subsequently learn. To (...) explore the consequences of this, we investigate how representations of IFS in the input-output structures of learning models affect the capacity of uncertainty estimates derived from them to account for phonetic variability in speech. Specifically, we examine whether IFS are best represented as outputs to neural networks (as in previous studies) or as inputs by building models that embody both choices and examining their capacity to account for uncertainty effects in the formant trajectories of word final [ɐ], which in German discriminates around sixty different IFS. Overall, we find that formants are enhanced as the uncertainty associated with IFS decreases. This result dovetails with a growing number of studies of morphological and inflectional families that have shown that enhancement is associated with lower uncertainty in context. Importantly, we also find that in models where IFS serve as inputs—as our theoretical analysis suggests they ought to—its uncertainty measures provide better fits to the empirical variance observed in [ɐ] formants than models where IFS serve as outputs. This supports our suggestion that IFS serve as cognitive cues during speech production, and should be treated as such in modeling. It is also consistent with the idea that when IFS serve as inputs to a learning network. This maintains the distinction between those parts of the network that represent message and those that represent signal. We conclude by describing how maintaining a “signal-message-uncertainty distinction” can allow us to reconcile a range of apparently contradictory findings about the relationship between articulation and uncertainty in context. (shrink)

We examined how language supports the expression of temporality within sentence boundaries in English, which has a rich inventory of grammatical means to express temporality. Using a computational model that mimics how humans learn from exposure we explored what the use of different tense and aspect combinations reveals about the interaction between our experience of time and the cognitive demands that talking about time puts on the language user. Our model was trained on n-grams extracted from the BNC to select (...) the TA combination that fits the context best. It revealed the existence of two different sub-systems within the set of TA combinations, a “simplex” one that is supported lexically and is easy to learn, and a “complex” one that is supported contextually and is hard to learn. The finding that some TA combinations are essentially lexical in nature necessitates a rethink of tense and aspect as grammatical categories that form the axes of the temporal system. We argue that the system of temporal reference may be more fruitfully thought of as the result of learning a system that is steeped in experience and organised along a number of functional principles. (shrink)

This paper develops a construction-based dialectometry capable of identifying previously unknown constructions and measuring the degree to which a given construction is subject to regional variation. The central idea is to learn a grammar of constructions using construction grammar induction and then to use these constructions as features for dialectometry. This offers a method for measuring the aggregate similarity between regional CxGs without limiting in advance the set of constructions subject to variation. The learned CxG is evaluated on how well (...) it describes held-out test corpora while dialectometry is evaluated on how well it can model regional varieties of English. The method is tested using two distinct datasets: First, the International Corpus of English representing eight outer circle varieties; Second, a web-crawled corpus representing five inner circle varieties. Results show that the method produces a grammar with stable quality across sub-sets of a single corpus that is capable of distinguishing between regional varieties of English with a high degree of accuracy, thus supporting dialectometric methods for measuring the similarity between varieties of English and measuring the degree to which each construction is subject to regional variation. This is important for cognitive sociolinguistics because it operationalizes the idea that competition between constructions is organized at the functional level so that dialectometry needs to represent as much of the available functional space as possible. (shrink)

Corpus-based word frequencies are one of the most important predictors in language processing tasks. Frequencies based on conversational corpora are shown to better capture the variance in lexical decision tasks compared to traditional corpora. In this study, we show that frequencies computed from social media are currently the best frequency-based estimators of lexical decision reaction times. The results are robust and are still substantial when we control for corpus size.

We present a comprehensive empirical evaluation of the ACT‐R–based model of sentence processing developed by Lewis and Vasishth (2005) (LV05). The predictions of the model are compared with the results of a recent meta‐analysis of published reading studies on retrieval interference in reflexive‐/reciprocal‐antecedent and subject–verb dependencies (Jäger, Engelmann, & Vasishth, 2017). The comparison shows that the model has only partial success in explaining the data; and we propose that its prediction space is restricted by oversimplifying assumptions. We then implement a (...) revised model that takes into account differences between individual experimental designs in terms of the prominence of the target and the distractor in memory‐ and context‐dependent cue‐feature associations. The predictions of the original and the revised model are quantitatively compared with the results of the meta‐analysis. Our simulations show that, compared to the original LV05 model, the revised model accounts for the data better. The results suggest that effects of prominence and variable cue‐feature associations need to be considered in the interpretation of existing empirical results and in the design and planning of future experiments. With regard to retrieval interference in sentence processing and to the broader field of psycholinguistic studies, we conclude that well‐specified models in tandem with high‐powered experiments are needed in order to uncover the underlying cognitive processes. (shrink)