This article examines cognitive process models of human sentence comprehension based on the idea of informed search. These models are rational in the sense that they strive to find a good syntactic analysis quickly. Informed search derives a new account of garden pathing that handles traditional counterexamples. It supports a symbolic explanation for local coherence as well as an algorithmic account of entropy reduction. The models are expressed in a broad framework for theories of human sentence comprehension.

Structurally, cognitive and biological evolution are highly similar. Random variation and constant but blind selection drive evolution within biology as well as within cognition. However, evolution of cognitive programs, and in particular of grammar systems, is not a subclass of biological evolution but a domain of its own. The abstract evolutionary principles, however, are akin in cognitive and biological evolution. In other words, insights gained in the biological domain can be cautiously applied to the cognitive domain. This paper claims that (...) the cognitively encapsulated, i.e. consciously inaccessible, aspects of grammars as cognitively represented systems, that is, the procedural and structural parts of grammars, are subject to, and results of, Darwinian evolution, applying to a domain-specific cognitive program. Other, consciously accessible aspects of language do not fall under Darwinian evolutionary principles, but are mostly instances of social changes. (shrink)

A key component of research on human sentence processing is to characterize the processing difficulty associated with the comprehension of words in context. Models that explain and predict this difficulty can be broadly divided into two kinds, expectation‐based and memory‐based. In this work, we present a new model of incremental sentence processing difficulty that unifies and extends key features of both kinds of models. Our model, lossy‐context surprisal, holds that the processing difficulty at a word in context is proportional to (...) the surprisal of the word given a lossy memory representation of the context—that is, a memory representation that does not contain complete information about previous words. We show that this model provides an intuitive explanation for an outstanding puzzle involving interactions of memory and expectations: language‐dependent structural forgetting, where the effects of memory on sentence processing appear to be moderated by language statistics. Furthermore, we demonstrate that dependency locality effects, a signature prediction of memory‐based theories, can be derived from lossy‐context surprisal as a special case of a novel, more general principle called information locality. (shrink)

I present a computational-level model of semantic interference effects in online word production within a rate–distortion framework. I consider a bounded-rational agent trying to produce words. The agent's action policy is determined by maximizing accuracy in production subject to computational constraints. These computational constraints are formalized using mutual information. I show that semantic similarity-based interference among words falls out naturally from this setup, and I present a series of simulations showing that the model captures some of the key empirical patterns (...) observed in Stroop and Picture–Word Interference paradigms, including comparisons to human data from previous experiments. (shrink)

The entropy-reduction hypothesis claims that the cognitive processing difficulty on a word in sentence context is determined by the word's effect on the uncertainty about the sentence. Here, this hypothesis is tested more thoroughly than has been done before, using a recurrent neural network for estimating entropy and self-paced reading for obtaining measures of cognitive processing load. Results show a positive relation between reading time on a word and the reduction in entropy due to processing that word, supporting the entropy-reduction (...) hypothesis. Although this effect is independent from the effect of word surprisal, we find no evidence that these two measures correspond to cognitively distinct processes. (shrink)

Definitions of syncopation share two characteristics: the presence of a meter or analogous hierarchical rhythmic structure and a displacement or contradiction of that structure. These attributes are translated in terms of a Bayesian theory of syncopation, where the syncopation of a rhythm is inferred based on a hierarchical structure that is, in turn, learned from the ongoing musical stimulus. Several experiments tested its simplest possible implementation, with equally weighted priors associated with different meters and independence of auditory events, which can (...) be decomposed into two terms representing note density and deviation from a metric hierarchy. A computational simulation demonstrated that extant measures of syncopation fall into two distinct factors analogous to the terms in the simple Bayesian model. Next, a series of behavioral experiments found that perceived syncopation is significantly related to both terms, offering support for the general Bayesian construction of syncopation. However, we also found that the prior expectations associated with different metric structures are not equal across meters and that there is an interaction between density and hierarchical deviation, implying that auditory events are not independent from each other. Together, these findings provide evidence that syncopation is a manifestation of a form of temporal expectation that can be directly represented in Bayesian terms and offer a complementary, feature‐driven approach to recent Bayesian models of temporal prediction. (shrink)

An English double-embedded relative clause from which the middle verb is omitted can often be processed more easily than its grammatical counterpart, a phenomenon known as the grammaticality illusion. This effect has been found to be reversed in German, suggesting that the illusion is language specific rather than a consequence of universal working memory constraints. We present results from three self-paced reading experiments which show that Dutch native speakers also do not show the grammaticality illusion in Dutch, whereas both German (...) and Dutch native speakers do show the illusion when reading English sentences. These findings provide evidence against working memory constraints as an explanation for the observed effect in English. We propose an alternative account based on the statistical patterns of the languages involved. In support of this alternative, a single recurrent neural network model that is trained on both Dutch and English sentences is shown to predict the cross-linguistic difference in the grammaticality effect. (shrink)

This study provides evidence for implicit learning in syntactic comprehension. By reanalyzing data from a syntactic priming experiment (Thothathiri & Snedeker, 2008), we find that the error signal associated with a syntactic prime influences comprehenders' subsequent syntactic expectations. This follows directly from error‐based implicit learning accounts of syntactic priming, but it is unexpected under accounts that consider syntactic priming a consequence of temporary increases in base‐level activation. More generally, the results raise questions about the principles underlying the maintenance of implicit (...) statistical knowledge relevant to language processing, and about possible functional motivations for syntactic priming. (shrink)

Results from two self-paced reading experiments in English are reported in which subject- and object-extracted relative clauses (SRCs and ORCs, respectively) were presented in contexts that support both types of relative clauses (RCs). Object-extracted versions were read more slowly than subject-extracted versions across both experiments. These results are not consistent with a decay-based working memory account of dependency formation where the amount of decay is a function of the number of new discourse referents that intervene between the dependents (Gibson, 1998; (...) Warren & Gibson, 2002). Rather, these results support interference-based accounts and decay-based accounts where the amount of decay depends on the number of words or on the type of noun phrases that intervene between the dependents. In Experiment 2, presentation in supportive contexts was directly contrasted with presentation in null contexts. Whereas in the null context the extraction effect was only observed during the RC region, in a supportive context the extraction effect was numerically larger and persisted into the following region, thus showing that extraction effects are enhanced in supportive contexts. A sentence completion study demonstrated that the rate of SRCs versus ORCs was similar across null and supportive contexts (with most completions being subject-extractions), ruling out the possibility that an enhanced extraction effect in supportive contexts is due to ORCs being less expected in such contexts. However, the content of the RCs differed between contexts in the completions, such that the RCs produced in supportive contexts were more constrained, reflecting the lexical and semantic content of the preceding context. This effect, which we discuss in terms of expectations/lexico-syntactic priming, suggests that the enhancement of the extraction effect in supportive contexts is due to the facilitation of the subject-extracted condition. (shrink)

Linguistic dependencies between non‐adjacent words have been shown to cause comprehension difficulty, compared with local dependencies. According to one class of sentence comprehension accounts, non‐local dependencies are difficult because they require the retrieval of the first dependent from memory when the second dependent is encountered. According to these memory‐based accounts, making the first dependent accessible at the time when the second dependent is encountered should help alleviate the difficulty associated with the processing of non‐local dependencies. In a dual‐task paradigm, participants (...) read sentences that did or did not contain a non‐local dependency (i.e., object‐ and subject‐extracted cleft constructions) while simultaneously remembering a word. The memory task was aimed at making the word held in memory accessible throughout the sentence. In an object‐extracted cleft (e.g., It was Ellen whom John consulted…), the object (Ellen) must be retrieved from memory when consulted is encountered. In the critical manipulation, the memory word was identical to the verb's object (ELLEN). In these conditions, the extraction effect was reduced in the comprehension accuracy data and eliminated in the reading time data. These results add to the body of evidence supporting memory‐based accounts of syntactic complexity. (shrink)

We explore the interaction between oculomotor control and language comprehension on the sentence level using two well-tested computational accounts of parsing difficulty. Previous work (Boston, Hale, Vasishth, & Kliegl, 2011) has shown that surprisal (Hale, 2001; Levy, 2008) and cue-based memory retrieval (Lewis & Vasishth, 2005) are significant and complementary predictors of reading time in an eyetracking corpus. It remains an open question how the sentence processor interacts with oculomotor control. Using a simple linking hypothesis proposed in Reichle, Warren, and (...) McConnell (2009), we integrated both measures with the eye movement model EMMA (Salvucci, 2001) inside the cognitive architecture ACT-R (Anderson et al., 2004). We built a reading model that could initiate short “Time Out regressions” (Mitchell, Shen, Green, & Hodgson, 2008) that compensate for slow postlexical processing. This simple interaction enabled the model to predict the re-reading of words based on parsing difficulty. The model was evaluated in different configurations on the prediction of frequency effects on the Potsdam Sentence Corpus. The extension of EMMA with postlexical processing improved its predictions and reproduced re-reading rates and durations with a reasonable fit to the data. This demonstration, based on simple and independently motivated assumptions, serves as a foundational step toward a precise investigation of the interaction between high-level language processing and eye movement control. (shrink)

Probabilistic models of sentence comprehension are increasingly relevant to questions concerning human language processing. However, such models are often limited to syntactic factors. This restriction is unrealistic in light of experimental results suggesting interactions between syntax and other forms of linguistic information in human sentence processing. To address this limitation, this article introduces two sentence processing models that augment a syntactic component with information about discourse co-reference. The novel combination of probabilistic syntactic components with co-reference classifiers permits them to more (...) closely mimic human behavior than existing models. The first model uses a deep model of linguistics, based in part on probabilistic logic, allowing it to make qualitative predictions on experimental data; the second model uses shallow processing to make quantitative predictions on a broad-coverage reading-time corpus. (shrink)

The information theoretic principle of rational adaptation predicts that individuals with aphasia adapt to their language impairments by relying more heavily on comparatively unimpaired non-linguistic knowledge to communicate. This prediction was examined by assessing the extent to which adults with chronic aphasia due to left-hemisphere stroke rely more on conceptual rather than lexical information during verb retrieval, as compared to age-matched neurotypical controls. A primed verb naming task examined the degree of facilitation each participant group received from either conceptual event-related (...) or lexical collocate cues, compared to unrelated baseline cues. The results provide evidence that adults with aphasia received amplified facilitation from conceptual cues compared to controls, whereas healthy controls received greater facilitation from lexical cues. This indicates that adaptation to alternative and relatively unimpaired information may facilitate successful word retrieval in aphasia. Implications for models of rational adaptation and clinical neurorehabilitation are discussed. (shrink)

This paper develops a novel psycholinguistic parser and tests it against experimental and corpus reading data. The parser builds on the recent research into memory structures, which argues that memory retrieval is content‐addressable and cue‐based. It is shown that the theory of cue‐based memory systems can be combined with transition‐based parsing to produce a parser that, when combined with the cognitive architecture ACT‐R, can model reading and predict online behavioral measures (reading times and regressions). The parser's modeling capacities are tested (...) against self‐paced reading experimental data (Grodner & Gibson, 2005), eye‐tracking experimental data (Staub, 2011), and a self‐paced reading corpus (Futrell et al., 2018). (shrink)

We examined how language supports the expression of temporality within sentence boundaries in English, which has a rich inventory of grammatical means to express temporality. Using a computational model that mimics how humans learn from exposure we explored what the use of different tense and aspect combinations reveals about the interaction between our experience of time and the cognitive demands that talking about time puts on the language user. Our model was trained on n-grams extracted from the BNC to select (...) the TA combination that fits the context best. It revealed the existence of two different sub-systems within the set of TA combinations, a “simplex” one that is supported lexically and is easy to learn, and a “complex” one that is supported contextually and is hard to learn. The finding that some TA combinations are essentially lexical in nature necessitates a rethink of tense and aspect as grammatical categories that form the axes of the temporal system. We argue that the system of temporal reference may be more fruitfully thought of as the result of learning a system that is steeped in experience and organised along a number of functional principles. (shrink)

A number of studies report that frequency is a poor predictor of acceptability, in particular at the lower end of the frequency spectrum. Because acceptability judgments provide a substantial part of the empirical foundation of dominant linguistic traditions, understanding how acceptability relates to frequency, one of the most robust predictors of human performance, is crucial. The relation between low frequency and acceptability is investigated using corpus‐ and behavioral data on the distribution of infinitival and finite that‐complements in Polish. Polish verbs (...) exhibit substantial subordination variation and for the majority of verbs taking an infinitival complement, the that‐complement occurs with low frequency (<0.66 ipm). These low‐frequency that‐clauses, in turn, exhibit large differences in how acceptable they are to native speakers. It is argued that acceptability judgments are based on configurations of internally structured exemplars, the acceptability of which cannot reliably be assessed until sufficient evidence about the core component has accumulated. (shrink)

Usage-based linguists and psychologists have produced a large body of empirical results suggesting that linguistic structure is derived from language use. However, while researchers agree that these results characterize grammar as an emergent phenomenon, there is no consensus among usage-based scholars as to how the various results can be explained and integrated into an explicit theory or model. Building on network theory, the current paper outlines a structured network approach to the study of grammar in which the core concepts of (...) syntax are analyzed by a set of relations that specify associations between different aspects of a speaker’s linguistic knowledge. These associations are shaped by domain-general processes that can give rise to new structures and meanings in language acquisition and language change. Combining research from linguistics and psychology, the paper proposes specific network analyses for the following phenomena: argument structure, word classes, constituent structure, constructions and construction families, and grammatical categories such as voice, case and number. The article builds on data and analyses presented inDiessel but approaches the topic from a different perspective. (shrink)

In many languages, the same particles build quantifier words and serve as connectives, additive and scalar particles, question markers, existential verbs, and so on. Do the roles of each particle form a natural class with a stable semantics? Are the particles aided by additional elements, overt or covert, in fulfilling their varied roles? I propose a unified analysis, according to which the particles impose partial ordering requirements (glb and lub) on the interpretations of their hosts and the immediate larger contexts, (...) but do not embody algebraic operations themselves. (shrink)

A probabilistic Chomsky–Schützenberger hierarchy of grammars is introduced and studied, with the aim of understanding the expressive power of generative models. We offer characterizations of the distributions definable at each level of the hierarchy, including probabilistic regular, context-free, (linear) indexed, context-sensitive, and unrestricted grammars, each corresponding to familiar probabilistic machine classes. Special attention is given to distributions on (unary notations for) positive integers. Unlike in the classical case where the "semi-linear" languages all collapse into the regular languages, using analytic tools (...) adapted from the classical setting we show there is no collapse in the probabilistic hierarchy: more distributions become definable at each level. We also address related issues such as closure under probabilistic conditioning. (shrink)

Language is a rewarding field if you are in the prediction business. A reader who is fluent in English and who knows how academic papers are typically structured will readily come up with several possible guesses as to where the title of this section could have gone, had it not been cut short by the ellipsis. Indeed, in the more natural setting of spoken language, anticipatory processing is a must: performance of machine systems for speech interpretation depends critically on the (...) availability of a good predictive model of how utterances unfold in time (Baker, 1975; Jelinek, 1990; Goodman, 2001), and there is strong evidence that prospective uncertainty affects human sentence processing too (Jurafsky, 2003; Hale, 2006; Levy, 2008). The human ability to predict where the current utterance is likely to be going is just another adaptation to the general pressure to anticipate the future (Hume, 1748; Dewey, 1910; Craik, 1943), be it in perception, thinking, or action, which is exerted on all cognitive systems by evolution (Dennett, 2003). Look-ahead in language is, however, special in one key respect: language is a medium for communication, and in communication the most interesting (that is, informative) parts of the utterance that the speaker is working through are those that cannot be predicted by the listener ahead of time. (shrink)

Argumentation should encourage autonomous decision-making. Rick Santorum’s political campaign ad Welcome to Obamaville violates this requirement by deploying a flood of subliminal images. Santorum’s ad involves a fallacy by virtue of clear intent to manipulate. Arguers are responsible for the foreseeable consequences of their action. Santorum acts in bad faith even if subliminal messages are in fact ineffective and he is wrong about the consequences foreseen.

Predictive syntactic processing plays an essential role in language comprehension. In our previous study using Japanese object-verb (OV) sentences, we showed that the left inferior frontal gyrus (IFG) responses to a verb increased at 120–140 ms after the verb onset, indicating predictive effects caused by a preceding object. To further elucidate the automaticity of the predictive effects in the present magnetoencephalography study, we examined whether a subliminally presented verb (“subliminal verb”) enhanced the predictive effects on the sentence-final verb (“target verb”) (...) unconsciously, i.e., without awareness. By presenting a subliminal verb after the object, enhanced predictive effects on the target verb would be detected in the OV sentences when the transitivity of the target verb matched with that of the subliminal verb (“congruent condition”), because the subliminal verb just after the object could determine the grammaticality of the sentence. For the OV sentences under the congruent condition, we observed significantly increased left IFG responses at 140–160 ms after the target verb onset. In contrast, responses in the precuneus and midcingulate cortex (MCC) were significantly reduced for the OV sentences under the congruent condition at 110–140 and 280–300 ms, respectively. By using partial Granger causality analyses for the OV sentences under the congruent condition, we revealed a bidirectional interaction between the left IFG and MCC at 60–160 ms, as well as a significant influence from the MCC to the precuneus. These results indicate that a top-down influence from the left IFG to the MCC, and then to the precuneus, is critical in syntactic decisions, whereas the MCC shares its task-set information with the left IFG to achieve automatic and predictive processes of syntax. (shrink)