Processes governing the creation, perception and production of spoken words are sensitive to the patterns of speech sounds in the language user’s lexicon. Generative linguistic theory suggests that listeners infer constraints on possible sound patterning from the lexicon and apply these constraints to all aspects of word use. In contrast, emergentist accounts suggest that these phonotactic constraints are a product of interactive associative mapping with items in the lexicon. To determine the degree to which phonotactic constraints are lexically mediated, we (...) observed the effects of learning new words that violate English phonotactic constraints on phonotactic perceptual repair processes in nonword consonant-consonant-vowel stimuli. Subjects who learned such words were less likely to “repair” illegal onset clusters and report them as legal ones. Effective connectivity analyses of MRI-constrained reconstructions of simultaneously collected magnetoencephalography and EEG data showed that these behavioral shifts were accompanied by changes in the strength of influences of lexical areas on acoustic-phonetic areas. These results strengthen the interpretation of previous results suggesting that phonotactic constraints on perception are produced by top-down lexical influences on speech processing. (shrink)

The core niche for language use is in verbal interaction, involving the rapid exchange of turns at talking. This paper reviews the extensive literature about this system, adding new statistical analyses of behavioral data where they have been missing, demonstrating that turn-taking has the systematic properties originally noted by Sacks et al. (1974 ; hereafter SSJ). This system poses some significant puzzles for current theories of language processing: the gaps between turns are short (of the order of 200 ms), but (...) the latencies involved in language production are much longer (over 600 ms). This seems to imply that participants in conversation must predict (or ‘project’ as SSJ have it) the end of the current speaker’s turn in order to prepare their response in advance. This in turn implies some overlap between production and comprehension despite their use of common processing resources. Collecting together what is known behaviorally and experimentally about the system, the space for systematic explanations of language processing for conversation can be significantly narrowed, and we sketch some first model of the mental processes involved for the participant preparing to speak next. (shrink)

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Specifically, we assume (...) that actors construct forward models of their actions before they execute those actions, and that perceivers of others' actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-defined levels of linguistic representation. We show how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, how they interweave production and comprehension processes, and how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscientific data on language processing and discuss some of the implications of our proposal. (shrink)

When a piece of fruit is in a bowl, and the bowl is on a table, we appreciate not only the individual objects and their features, but also the relations containment and support, which abstract away from the particular objects involved. Independent representation of roles (e.g., containers vs. supporters) and “fillers” of those roles (e.g., bowls vs. cups, tables vs. chairs) is a core principle of language and higherlevel reasoning. But does such role-filler independence also arise in automatic visual processing? (...) Here, we show that it does, by exploring a surprising error that such independence can produce. In four experiments, participants saw a stream of images containing different objects arranged in forcedynamic relations — e.g., a phone contained in a basket, a marker resting on a garbage can, or a knife sitting in a cup. Participants had to respond to a single target image (e.g., a phone in a basket) within a stream of distractors presented under time constraints. Surprisingly, even though participants completed this task quickly and accurately, they false-alarmed more often to images matching the target’s relational category than to those that did not — even when those images involved completely different objects. In other words, participants searching for a phone in a basket were more likely to mistakenly respond to a knife in a cup than to a marker on a garbage can. Follow-up experiments ruled out strategic responses and also controlled for various confounding image features. We suggest that visual processing represents relations abstractly, in ways that separate roles from fillers. (shrink)

In order to memorize sentences we use both processes of language comprehension during encoding and processes of language production during maintenance. While the former processes are easily testable via controlled presentation of the input, the latter are more difficult to assess directly as language production is typically initiated and controlled internally. In the present event-related potential (ERP) study we track subvocal rehearsal of sentences, with the goal of studying the concomitant planning processes with the help of a silent cued-production task. (...) Native German participants read different types of sentences word-by-word, then were prompted by a visual cue to silently repeat each individual word, in a rehearsal phase. In order to assess both local and global effects of sentence planning, we presented correct sentences, syntactically or semantically violated sentences, or random word order sequences. Semantic violations during reading elicited an N400 effect at the noun violating the selectional restrictions of the preceding verb. Syntactic violations, induced by a gender incongruency between determiner and noun, led to a P600 effect at the same position. Different ERP patterns occurred during the silent production phase. Here, semantically violated sentences elicited an early fronto-central negativity at the verb, while syntactically violated sentences elicited a late right-frontal positivity at the determiner. Random word order was accompanied by long-lasting slow waves during the production phase. The findings are consistent with models of hierarchical sentence planning and further indicate that the ongoing working memory processes are qualitatively distinct from comprehension mechanisms and neurophysiologically specific for syntactic and lexical-semantic level planning. In conclusion, active working memory maintenance of sentences is likely to comprise specific stages of sentence production that are indicated by ERP correlates of syntactic and semantic planning at the phrasal and clausal level respectively. (shrink)

Inner speech has been shown to vary in form along several dimensions. Along condensation, condensed inner speech forms have been described, that are supposed to be deprived of acoustic, phonological and even syntactic qualities. Expanded forms, on the other extreme, display articulatory and auditory properties. Along dialogality, inner speech can be monologal, when we engage in internal soliloquy, or dialogal, when we recall past conversations or imagine future dialogs involving our own voice as well as that of others addressing us. (...) Along intentionality, it can be intentional (when we deliberately rehearse material in short-term memory) or it can arise unintentionally (during mind wandering). We introduce the ConDialInt model, a neurocognitive predictive control model of inner speech that accounts for its varieties along these three dimensions. ConDialInt spells out the condensation dimension by including inhibitory control at the conceptualization, formulation or articulatory planning stage. It accounts for dialogality, by assuming internal model adaptations and by speculating on neural processes underlying perspective switching. It explains the differences between intentional and spontaneous varieties in terms of monitoring. We present an fMRI study in which we probed varieties of inner speech along dialogality and intentionality, to examine the validity of the neuroanatomical correlates posited in ConDialInt. Condensation was also informally tackled. Our data support the hypothesis that expanded inner speech recruits speech production processes down to articulatory planning, resulting in a predicted signal, the inner voice, with auditory qualities. Along dialogality, covertly using an avatar’s voice resulted in the activation of right hemisphere homologs of the regions involved in internal own-voice soliloquy and in reduced cerebellar activation, consistent with internal model adaptation. Switching from first-person to third-person perspective resulted in activations in precuneus and parietal lobules. Along intentionality, compared with intentional inner speech, mind wandering with inner speech episodes was associated with greater bilateral inferior frontal activation and decreased activation in left temporal regions. This is consistent with the reported subjective evanescence and presumably reflects condensation processes. Our results provide neuroanatomical evidence compatible with predictive control and in favor of the assumptions made in the ConDialInt model. (shrink)

Comprehenders generate expectations about upcoming lexical items in language processing using various types of contextual information. However, a number of studies have shown that argument roles do not impact neural and behavioral prediction measures. Despite these robust findings, some prior studies have suggested that lexical prediction might be sensitive to argument roles in production tasks such as the cloze task or in comprehension tasks when additional time is available for prediction. This study demonstrates that both the task and additional time (...) for prediction independently influence lexical prediction using argument roles, via evidence from closely matched electroencephalogram (EEG) and speeded cloze experiments. In order to investigate the timing effect, our EEG experiment used maximally simple Japanese stimuli such as Bee-nom/acc sting, and it manipulated the time for prediction by changing the temporal interval between the context noun and the target verb without adding any further linguistic content. In order to investigate the task effect, we conducted a speeded cloze study that was matched with our EEG study both in terms of stimuli and the time available for prediction. We found that both the EEG study with additional time for prediction and the speeded cloze study with matched timing showed clear sensitivity to argument roles, while the EEG conditions with less time for prediction replicated the standard pattern of argument role insensitivity. Based on these findings, we propose that lexical prediction is initially insensitive to argument roles but a monitoring mechanism serially inhibits role-inappropriate candidates. This monitoring process operates quickly in production tasks, where it is important to quickly select a single candidate to produce, whereas it may operate more slowly in comprehension tasks, where multiple candidates can be maintained until a continuation is perceived. Computational simulations demonstrate that this mechanism can successfully explain the task and timing effects observed in our experiments. (shrink)

While the idea that thinking is a form of silent self-talk goes back at least to Plato, it is not immediately clear how to state this thesis precisely. The aim of the paper is to spell out the notion that we think in language by recourse to recent work on inner speech. To that end, inner speech and overt speech are briefly compared. I then propose that inner speaking be defined as a mental episode that substantially engages the speech production (...) system; the underlying model of speech production is sketched. Next, the cognitive role of inner speaking is explored, especially its role in thinking and reasoning. The question of whether it is a way of making thoughts accessible (to whomever) or whether it is a means of thinking itself is raised. I argue that there are two reasons for assuming that, occasionally, we think in language. More specifically, I will claim that some instances of thinking are instances of inner speaking as they exploit certain properties of natural language, and that some instances of inner speaking are instances of thinking as they play a decisive role in paradigmatic cases of thinking that result from internalizing and re-using certain social-linguistic practices. Finally, the Language-of-Thought hypothesis as an alternative account is critically discussed. (shrink)

In this event-related potential study a masking technique that prevents conscious perception of words and non-words through attentional distraction was used to reveal the temporal dynamics of word processing under non-conscious and conscious conditions. In the non-conscious condition, ERP responses differed between masked words and non-words from 112 to 160 ms after stimulus-onset over posterior brain areas. The early onset of the word–non-word differences was compatible with previous studies that reported non-conscious access to orthographic information within this time period. Moreover, (...) source localisations provided evidence for automatic activation of prelexical phonological information, whereas no evidence for non-conscious semantic processing was found. When subjects were informed about the masking technique, lexical differences occurred at later time intervals, suggesting conscious access to additional word related information. These results indicate that early visual word processing does not depend entirely on attentional resources, but that non-conscious processing probably is restricted to rather lower-level linguistic information. (shrink)

Developmental stuttering is a widely discussed speech fluency disorder. Research on its mechanism has focused on an atypical interface between the planning (PLAN) and execution (EX) processes, known collectively as the EXPLAN model. However, it remains unclear how this atypical interface influences people who stutter. A straightforward assumption is that stuttering speakers adopt a smaller scope of speech planning, whereas a defect in word retrieval can be confounding. To shed light on this issue, we took the semantic blocking effect as (...) an index to examine lexical planning in word and phrase production. In Experiment 1, for word production, pictures from the same semantic category were combined to form homogeneous blocks, and pictures from different categories were combined to form heterogeneous blocks. A typical effect of semantic blocking showing longer naming latencies for homogeneous blocks than heterogeneous ones was observed for both stuttering and fluent speakers. However, this effect was smaller for stuttering speakers, when it was subject to lexical defects in stuttering. In Experiment 2, for a conjoined noun phrase production task, the pictures referring to the first noun were manipulated into homogeneous and heterogeneous conditions. The semantic blocking effect was also much smaller for stuttering speakers, indicating a smaller scope of lexical planning. Therefore, the results provided more evidence in support of the EXPLAN model and indicated that a smaller scope of lexical planning rather than lexical defects causes the atypical interface for stuttering. Moreover, a comparison between these two tasks showed that the study findings have implications for syntactic defects in stuttering. (shrink)

In this paper we suggest the existence of a generalized task-related cortical network that is up-regulated whenever the task to be performed requires the allocation of generalized non-specific cognitive resources, independent of the specifics of the task to be performed. We have labeled this general purpose network, the extrinsic mode network (EMN) as complementary to the default mode network (DMN), such that the EMN is down-regulated during periods of task-absence, when the DMN is up-regulated, and vice versa. We conceptualize the (...) EMN as a cortical network for extrinsic neuronal activity, similar to the DMN as being a cortical network for intrinsic neuronal activity. The EMN has essentially a fronto-temporo-parietal spatial distribution, including the inferior and middle frontal gyri, inferior parietal lobule, supplementary motor area, inferior temporal gyrus. We hypothesize that this network is always active regardless of the cognitive task being performed. We further suggest that failure of network up- and down-regulation dynamics may provide neuronal underpinnings for cognitive impairments seen in many mental disorders, such as, e.g., schizophrenia. We start by describing a common observation in functional imaging, the close overlap in fronto-parietal activations in healthy individuals to tasks that denote very different cognitive processes. We now suggest that this is because the brain utilizes the EMN network as a generalized response to tasks that exceeds a cognitive demand threshold and/or requires the processing of novel information. We further discuss how the EMN is related to the DMN, and how a network for extrinsic activity is related to a network for intrinsic activity. Finally, we discuss whether the EMN and DMN networks interact in a common single brain system, rather than being two separate and independent brain systems. (shrink)

The transitions between turns at talk in conversation tend to occur quickly, with only a slight gap of approximately 100 to 300 ms between them. This estimate of central tendency, however, hides a wealth of complex variation, as a number of factors, such as the type of turns involved, have been shown to influence the timing of turn transitions. This article considers one specific type of turn that does not conform to the statistical trend, namely turns that deal with troubles (...) of speaking, hearing, and understanding, known as other-initiations of repair. The results of a quantitative analysis of 169 other-initiations of repair in face-to-face conversation reveal that the most frequent cases occur after gaps of approximately 700 ms. Furthermore, other-initiations of repair that locate a source of trouble in a prior turn specifically tend to occur after shorter gaps than those that do not, and those that correct errors in a prior turn, while rare, tend to occur without delay. An analysis of the transitions before other-initiations of repair, using methods of conversation analysis, suggests that speakers use the extra time (i) to search for a late recognition of the problematic turn, (ii) to provide an opportunity for the speaker of the problematic turn to resolve the trouble independently, (iii) and to produce visual signals, such as facial gestures. In light of these results, it is argued that other-initiations of repair take priority over other turns at talk in conversation and therefore are not subject to the same rules and constraints that motivate fast turn transitions in general. (shrink)

Our target article proposed that language production and comprehension are interwoven, with speakers making predictions of their own utterances and comprehenders making predictions of other people's utterances at different linguistic levels. Here, we respond to comments about such issues as cognitive architecture and its neural basis, learning and development, monitoring, the nature of forward models, communicative intentions, and dialogue.

In the midst of globalization, English is regarded as an international language, or Lingua Franca, but learning it as a second language (L2) remains still difficult to speakers of other languages. This is true especially for the speakers of languages distantly related to English such as Japanese. In this sense, exploring neural basis for translation between the first language (L1) and L2 is of great interest. There have been relatively many previous researches revealing brain activation patterns during translations between L1 (...) and English as L2. These studies, which focused on language translation with close or moderate linguistic distance (LD), have suggested that the Broca area (BA 44/45) and the dorsolateral prefrontal cortex (DLPFC; BA 46) may play an important role on translation. However, the neural mechanism of language translation between Japanese and English, having large LD, has not been clarified. Thus, we used functional near infrared spectroscopy (fNIRS) to investigate the brain activation patterns during word translation between Japanese and English. We also assessed the effects of translation directions and word familiarity. All participants’ first language was Japanese and they were learning English. Their English proficiency was advanced or elementary. We selected English and Japanese words as stimuli based on the familiarity for Japanese people. Our results showed that the brain activation patterns during word translation largely differed depending on their English proficiency. The advanced group elicited greater activation on the left prefrontal cortex around the Broca’s area while translating words with low familiarity, but no activation was observed while translating words with high familiarity. On the other hand, the elementary group evoked greater activation on the left temporal area including the superior temporal gyrus (STG) irrespective of the word familiarity. These results suggested that different cognitive process could be involved in word translation corresponding to English proficiency in Japanese learners of English. These difference on the brain activation patterns between the advanced and elementary group may reflect the difference on the cognitive loads depending on the levels of automatization in one’s language processing. (shrink)

Everyday conversation requires listeners to quickly recognize verbal actions, so-called speech acts, from the underspecified linguistic code and prepare a relevant response within the tight time constraints of turn-taking. The goal of this study was to determine the time-course of speech act recognition by investigating oscillatory EEG activity during comprehension of spoken dialog. Participants listened to short, spoken dialogs with target utterances that delivered three distinct speech acts (Answers, Declinations, Pre-offers). The targets were identical across conditions at lexico-syntactic and phonetic/prosodic (...) levels but differed in the pragmatic interpretation of the speech act performed. Speech act comprehension was associated with reduced power in the alpha/beta bands just prior to Declination speech acts, relative to Answers and Pre-offers. In addition, we observed reduced power in the theta band during the beginning of Declinations, relative to Answers. Based on the role of alpha and beta desynchronization in anticipatory processes, the results are taken to indicate that anticipation plays a role in speech act recognition. Anticipation of speech acts could be critical for efficient turn-taking, allowing interactants to quickly recognize speech acts and respond within the tight time frame characteristic of conversation. The results show that anticipatory processes can be triggered by the characteristics of the interaction, including the speech act type. (shrink)