Research on language production has focused on the process of utterance planning and involved studying the synchronization between visual gaze and the production of sentences that refer to objects in the immediate visual environment. However, it remains unclear how the visual grouping of these objects might influence this process. To shed light on this issue, the present research examined the effects of the visual grouping of objects in a visual display on utterance planning in two experiments. Participants produced utterances of (...) the form “The snail and the necklace are above/below/on the left/right side of the toothbrush” for objects containing these referents (e.g., a snail, a necklace and a toothbrush). These objects were grouped using classic Gestalt principles of color similarity (Experiment 1) and common region (Experiment 2) so that the induced perceptual grouping was congruent or incongruent with the required phrasal organization. The results showed that speech onset latencies were shorter in congruent than incongruent conditions. The findings therefore reveal that the congruency between the visual grouping of referents and the required phrasal organization can influence speech production. Such findings suggest that, when language is produced in a visual context, speakers make use of both visual and linguistic cues to plan utterances. (shrink)

Developmental stuttering is a widely discussed speech fluency disorder. Research on its mechanism has focused on an atypical interface between the planning (PLAN) and execution (EX) processes, known collectively as the EXPLAN model. However, it remains unclear how this atypical interface influences people who stutter. A straightforward assumption is that stuttering speakers adopt a smaller scope of speech planning, whereas a defect in word retrieval can be confounding. To shed light on this issue, we took the semantic blocking effect as (...) an index to examine lexical planning in word and phrase production. In Experiment 1, for word production, pictures from the same semantic category were combined to form homogeneous blocks, and pictures from different categories were combined to form heterogeneous blocks. A typical effect of semantic blocking showing longer naming latencies for homogeneous blocks than heterogeneous ones was observed for both stuttering and fluent speakers. However, this effect was smaller for stuttering speakers, when it was subject to lexical defects in stuttering. In Experiment 2, for a conjoined noun phrase production task, the pictures referring to the first noun were manipulated into homogeneous and heterogeneous conditions. The semantic blocking effect was also much smaller for stuttering speakers, indicating a smaller scope of lexical planning. Therefore, the results provided more evidence in support of the EXPLAN model and indicated that a smaller scope of lexical planning rather than lexical defects causes the atypical interface for stuttering. Moreover, a comparison between these two tasks showed that the study findings have implications for syntactic defects in stuttering. (shrink)

Although previous research has shown that there exist individual and cross-linguistic differences in planning strategies during language production, little is known about how such individual differences might vary depending on which language a speaker is planning. The present series of studies examines individual differences in planning strategies exhibited by speakers of American English, French, and German. Participants were asked to describe images on a computer monitor while their eye movements were monitored. In addition, we measured participants' working memory capacity and (...) speed of processing. The results indicate that in the present study, English and German were planned less incrementally (further in advance) prior to speech onset compared to French, which was planned more incrementally (not as far in advance). Crucially, speed of processing predicted the scope of planning for French speakers, but not for English or German speakers. These results suggest that the different planning strategies that are invoked by syntactic choices available in different languages are associated with the tendency for speakers to rely on different cognitive support systems as they plan sentences. (shrink)

In everyday interactions, humans naturally exhibit behavioral cues, such as gaze and head movements, that signal their intentions while interpreting the behavioral cues of others to predict their intentions. Such intention prediction enables each partner to adapt their behaviors to the intent of others, serving a critical role in joint action where parties work together to achieve a common goal. Among behavioral cues, eye gaze is particularly important in understanding a person's attention and intention. In this work, we seek to (...) quantify how gaze patterns may indicate a person's intention. Our investigation was contextualized in a dyadic sandwich-making scenario in which a “worker” prepared a sandwich by adding ingredients requested by a “customer.” In this context, we investigated the extent to which the customers' gaze cues serve as predictors of which ingredients they intend to request. Predictive features were derived to represent characteristics of the customers' gaze patterns. We developed a support vector machine-based (SVM-based) model that achieved 76% accuracy in predicting the customers' intended requests based solely on gaze features. Moreover, the predictor made correct predictions approximately 1.8 s before the spoken request from the customer. We further analyzed several episodes of interactions from our data to develop a deeper understanding of the scenarios where our predictor succeeded and failed in making correct predictions. These analyses revealed additional gaze patterns that may be leveraged to improve intention prediction. This work highlights gaze cues as a significant resource for understanding human intentions and informs the design of real-time recognizers of user intention for intelligent systems, such as assistive robots and ubiquitous devices, that may enable more complex capabilities and improved user experience. (shrink)

Human instructors often refer to objects and actions involved in a task description using both linguistic and non-linguistic means of communication. Hence, for robots to engage in natural human-robot interactions, we need to better understand the various relevant aspects of human multi-modal task descriptions. We analyse reference resolution to objects in a data collection comprising two object manipulation tasks and find that 78.76% of all referring expressions to the objects relevant in Task 1 are verbally underspecified and 88.64% of all (...) referring expressions are verbally underspecified in Task 2. The data strongly suggests that a language processing module for robots must be genuinely multi-modal, allowing for seamless integration of information transmitted in the verbal and the visual channel, whereby tracking the speaker’s eye gaze and gestures as well as object recognition are necessary preconditions. (shrink)