Brains, it has recently been argued, are essentially prediction machines. They are bundles of cells that support perception and action by constantly attempting to match incoming sensory inputs with top-down expectations or predictions. This is achieved using a hierarchical generative model that aims to minimize prediction error within a bidirectional cascade of cortical processing. Such accounts offer a unifying model of perception and action, illuminate the functional role of attention, and may neatly capture the special contribution of cortical processing to (...) adaptive success. This target article critically examines this approach, concluding that it offers the best clue yet to the shape of a unified science of mind and action. Sections 1 and 2 lay out the key elements and implications of the approach. Section 3 explores a variety of pitfalls and challenges, spanning the evidential, the methodological, and the more properly conceptual. The paper ends (sections 4 and 5) by asking how such approaches might impact our more general vision of mind, experience, and agency. (shrink)

Memory is fleeting. New material rapidly obliterates previous material. How, then, can the brain deal successfully with the continual deluge of linguistic input? We argue that, to deal with this “Now-or-Never” bottleneck, the brain must compress and recode linguistic input as rapidly as possible. This observation has strong implications for the nature of language processing: (1) the language system must “eagerly” recode and compress linguistic input; (2) as the bottleneck recurs at each new representational level, the language system must build (...) a multilevel linguistic representation; and (3) the language system must deploy all available information predictively to ensure that local linguistic ambiguities are dealt with “Right-First-Time”; once the original input is lost, there is no way for the language system to recover. This is “Chunk-and-Pass” processing. Similarly, language learning must also occur in the here and now, which implies that language acquisition is learning to process, rather than inducing, a grammar. Moreover, this perspective provides a cognitive foundation for grammaticalization and other aspects of language change. Chunk-and-Pass processing also helps explain a variety of core properties of language, including its multilevel representational structure and duality of patterning. This approach promises to create a direct relationship between psycholinguistics and linguistic theory. More generally, we outline a framework within which to integrate often disconnected inquiries into language processing, language acquisition, and language change and evolution. (shrink)

Why are children better language learners than adults despite being worse at a range of other cognitive tasks? Here, we explore the role of multiword sequences in explaining L1–L2 differences in learning. In particular, we propose that children and adults differ in their reliance on such multiword units in learning, and that this difference affects learning strategies and outcomes, and leads to difficulty in learning certain grammatical relations. In the first part, we review recent findings that suggest that MWUs play (...) a facilitative role in learning. We then discuss the implications of these findings for L1–L2 differences: We hypothesize that adults are both less likely to extract MWUs and less capable of benefiting from them in the process of learning. In the next section, we draw on psycholinguistic, developmental, and computational findings to support these predictions. We end with a discussion of the relation between this proposal and other accounts of L1–L2 difficulty. (shrink)

A basic function of cognition is to detect regularities in sensory input to facilitate the prediction and recognition of future events. It has been proposed that these implicit expectations arise from an internal predictive coding model, based on knowledge acquired through processes such as statistical learning, but it is unclear how different types of statistical information affect listeners’ memory for auditory stimuli. We used a combination of behavioral and computational methods to investigate memory for non-linguistic auditory sequences. Participants repeatedly heard (...) tone sequences varying systematically in their information-theoretic properties. Expectedness ratings of tones were collected during three listening sessions, and a recognition memory test was given after each session. Information-theoretic measures of sequential predictability significantly influenced listeners’ expectedness ratings, and variations in these properties had a significant impact on memory performance. Predictable sequences yielded increasingly better memory performance with increasing exposure. Computational simulations using a probabilistic model of auditory expectation suggest that listeners dynamically formed a new, and increasingly accurate, implicit cognitive model of the information-theoretic structure of the sequences throughout the experimental session. (shrink)

Two experiments used eyetracking during reading to examine the processing of the matrix verb following object and subject relative clauses. The experiments show that the processing of the matrix verb following an object relative is indeed slowed compared to the processing of the same verb following a subject relative. However, this difficulty is entirely eliminated if additional material intervenes between the object gap and the matrix verb. An explanation in terms of spillover processing is ruled out, suggesting that it is (...) the gap-matrix verb sequence that is itself responsible for the difficulty. We consider two accounts of this difficulty, one emphasizing the potential difficulty of rapidly switching between the sentential subject's thematic or syntactic role in the embedded clause and its role in the matrix clause, and one emphasizing the potential difficulty of performing two demanding memory retrievals in rapid succession. The present experiments also closely replicate the previous findings from eyetracking that the noun phrase and the verb within an object relative are both loci of processing difficulty, but that the former induces substantially greater difficulty. (shrink)

This article describes a left-corner parser implemented within a cognitively and neurologically motivated distributed model of memory. This parser's approach to syntactic ambiguity points toward a tidy account both of surprisal effects and of locality effects, such as the parsing breakdowns caused by center embedding. The model provides an algorithmic-level account of these breakdowns: The structure of the parser's memory and the nature of incremental parsing produce a smooth degradation of processing accuracy for longer center embeddings, and a steeper degradation (...) when they are nested, in line with recall observations by Miller and Isard and speed-accuracy trade-off observations by McElree et al.. Modeling results show that this effect is distinct from the effects of ambiguity and exceeds the effect of mere sentence length. (shrink)

Recent years have seen a surge in accounts motivated by information theory that consider language production to be partially driven by a preference for communicative efficiency. Evidence from discourse production (i.e., production beyond the sentence level) has been argued to suggest that speakers distribute information across discourse so as to hold the conditional per-word entropy associated with each word constant, which would facilitate efficient information transfer (Genzel & Charniak, 2002). This hypothesis implies that the conditional (contextualized) probabilities of linguistic units (...) affect speakers’ preferences during production. Here, we extend this work in two ways. First, we explore how preceding cues are integrated into contextualized probabilities, a question which so far has received little to no attention. Specifically, we investigate how a cue's maximal informativity about upcoming words (the cue's effectiveness) decays as a function of the cue's recency. Based on properties of linguistic discourses as well as properties of human memory, we analytically derive a model of cue effectiveness decay and evaluate it against cross-linguistic data from 12 languages. Second, we relate the information theoretic accounts of discourse production to well-established mechanistic (activation-based) accounts: We relate contextualized probability distributions over words to their relative activation in a lexical network given preceding discourse. (shrink)

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Specifically, we assume (...) that actors construct forward models of their actions before they execute those actions, and that perceivers of others' actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-defined levels of linguistic representation. We show how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, how they interweave production and comprehension processes, and how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscientific data on language processing and discuss some of the implications of our proposal. (shrink)

We investigated the influence of globally ungrammatical local syntactic constraints on sentence comprehension, as well as the corresponding activation of global and local representations. In Experiment 1, participants viewed visual scenes with objects like a carousel and motorbike while hearing sentences with noun phrase (NP) or verb phrase (VP) modifiers like “The girl who likes the man (from London/very much) will ride the carousel.” In both cases, “girl” and “ride” predicted carousel as the direct object; however, the locally coherent combination (...) “the man from London will ride…” in NP cases alternatively predicted motorbike. During “ride,” local constraints, although ruled out by the global constraints, influenced prediction as strongly as global constraints: While motorbike was fixated less than carousel in VP cases, it was fixated as much as carousel in NP cases. In Experiment 2, these local constraints likewise slowed reading times. We discuss implications for theories of sentence processing. (shrink)

All other things being equal the parser favors attaching an ambiguous modifier to the most recent possible site. A plausible explanation is that locality preferences such as this arise in the service of minimizing memory costs—more distant sentential material is more difficult to reactivate than more recent material. Note that processing any sentence requires linking each new lexical item with material in the current parse. This often involves the construction of long‐distance dependencies. Under a resource‐limited view of language processing, lengthy (...) integrations should induce difficulty even in unambiguous sentences. To date there has been little direct quantitative evidence in support of this perspective. This article presents 2 self‐paced reading studies, which explore the hypothesis that dependency distance is a fundamental determinant of reading complexity in unambiguous constructions in English. The evidence suggests that the difficulty associated with integrating a new input item is heavily determined by the amount of lexical material intervening between the input item and the site of its target dependents. The patterns observed here are not straightforwardly accounted for within purely experience‐based models of complexity. Instead, this work supports the role of a memory bottleneck in language comprehension. This constraint arises because hierarchical linguistic relations must be recovered from a linear input stream. (shrink)

A well‐established principle of language is that there is a preference for closely related words to be close together in the sentence. This can be expressed as a preference for dependency length minimization (DLM). In this study, we explore quantitatively the degree to which natural languages reflect DLM. We extract the dependencies from natural language text and reorder the words in such a way as to minimize dependency length. Comparing the original text with these optimal linearizations (and also with random (...) linearizations) reveals the degree to which natural language minimizes dependency length. Tests on English data show that English shows a strong effect of DLM, with dependency length much closer to optimal than to random; the optimal English grammar also has many specific features in common with English. In German, too, dependency length is significantly less than random, but the effect is much weaker than in English. We conclude by speculating about some possible reasons for this difference between English and German. (shrink)

An English double-embedded relative clause from which the middle verb is omitted can often be processed more easily than its grammatical counterpart, a phenomenon known as the grammaticality illusion. This effect has been found to be reversed in German, suggesting that the illusion is language specific rather than a consequence of universal working memory constraints. We present results from three self-paced reading experiments which show that Dutch native speakers also do not show the grammaticality illusion in Dutch, whereas both German (...) and Dutch native speakers do show the illusion when reading English sentences. These findings provide evidence against working memory constraints as an explanation for the observed effect in English. We propose an alternative account based on the statistical patterns of the languages involved. In support of this alternative, a single recurrent neural network model that is trained on both Dutch and English sentences is shown to predict the cross-linguistic difference in the grammaticality effect. (shrink)

Linguistic dependencies between non‐adjacent words have been shown to cause comprehension difficulty, compared with local dependencies. According to one class of sentence comprehension accounts, non‐local dependencies are difficult because they require the retrieval of the first dependent from memory when the second dependent is encountered. According to these memory‐based accounts, making the first dependent accessible at the time when the second dependent is encountered should help alleviate the difficulty associated with the processing of non‐local dependencies. In a dual‐task paradigm, participants (...) read sentences that did or did not contain a non‐local dependency (i.e., object‐ and subject‐extracted cleft constructions) while simultaneously remembering a word. The memory task was aimed at making the word held in memory accessible throughout the sentence. In an object‐extracted cleft (e.g., It was Ellen whom John consulted…), the object (Ellen) must be retrieved from memory when consulted is encountered. In the critical manipulation, the memory word was identical to the verb's object (ELLEN). In these conditions, the extraction effect was reduced in the comprehension accuracy data and eliminated in the reading time data. These results add to the body of evidence supporting memory‐based accounts of syntactic complexity. (shrink)

Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implicitly by its effects on processing rather than explicitly (as in a spatial representation). The current report develops a proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory. In this approach, hidden unit patterns are fed back to themselves: (...) the internal representations which develop thus reflect task demands in the context of prior internal states. A set of simulations is reported which range from relatively simple problems (temporal version of XOR) to discovering syntactic/semantic features for words. The networks are able to learn interesting internal representations which incorporate task demands with memory demands: indeed, in this approach the notion of memory is inextricably bound up with task processing. These representations reveal a rich structure, which allows them to be highly context‐dependent, while also expressing generalizations across classes of items. These representations suggest a method for representing lexical categories and the type/token distinction. (shrink)