This article investigates the origin of editorial images, with a focus on the mental processes that enable cartoonists and illustrators across cultures to come up with novel ideas. It provides the most compelling evidence to date that recycling, where artists regularly recycle pictorial and compositional ideas they have developed earlier, is the origin of ideas. Recycling theory is thus compatible with a variety of ongoing research programs. Among these are Turner’s work on blending (2014), Musolff’s research on scenarios (2016), Langacker’s (...) ideas on constructional schemas (1991), Johnson’s work on image schemas (1987), and van Dijk’s proposals concerning discourse complexes and intertextuality (2014). More specifically, the present study gives cognitively plausible answers to the perennial questions: To what extent does recycling shape and constrain the human way of thinking? Is it bound to certain cultural traditions? Does it characterize the style of certain artists or artistic movements? How does it contribute to discourse? Are the products of recycling always imaginative and creative? Is it a mode of efficiency? The theoretical insights are applied to 440 editorial images of Egyptian and British origin. This investigation will aid both semiotics scholarship and cognition studies based on multimodal stimuli. (shrink)

As leadership discourses in higher education are increasingly being mediated online, texts previously reserved for staff are now being made available in the public domain. As such, these texts become accessible for study, critique and evaluation. Additionally, discourses previously confined to the written domain are now increasingly multimodal. Thus, an approach is required that is capable of relating detailed, complex multimodal discourse analyses to broader sociocultural perspectives to account for the complex meaning-making practices that operate in online leadership discourses. For (...) this purpose, a digital multimodal discourse approach is proposed and illustrated via a small-scale case study of the online leadership discourse of an Australian university. The analysis of two short video texts demonstrates how a digital multimodal discourse perspective facilitates the identification of key multimodal systems used for meaning-making in online communication, how meaning arises through combinations of semiotic choices, and how the results of multimodal discourse analysis using digital technology can reveal larger sociocultural patterns – in this case, divergent leadership styles and approaches as reflected in online discourse, at a time of immense change within the higher education sector. (shrink)

Summary This article questions certain assumptions concerning film form made by the recent psychological film research and compares them to those of precursors of film psychology like Hugo Münsterberg and Rudolf Arnheim, as well as the principles of Gestalt psychology. It is argued that principles of Gestalt psychology such as those of ‘good form’ and good continuation are still underlying the psychological research of film, becoming particularly apparent in its approach to continuity editing. Following an alternative Gestalt genealogy that links (...) Gestalt theory with more recent dynamic models of brain activity and with accounts of brain complexity and neuronal synchronisation, the article concludes that psychological research on film needs to shift the focus from form to transformation, both in conceiving the perceptual and cognitive processing of films and in approaching film aesthetics more broadly. (shrink)

Name der Zeitschrift: Semiotica Jahrgang: 2015 Heft: 207 Seiten: 583-606.

Popular movies present chunk-like events that promote episodic, serial updating of viewers’ representations of the ongoing narrative. Event-indexing theory would suggest that the beginnings of new scenes trigger these updates, which in turn require more cognitive processing. Typically, a new movie event is signaled by an establishing shot, one providing more background information and a longer look than the average shot. Our analysis of 24 films reconfirms this. More important, we show that, when returning to a previously shown location, the (...) re-establishing shot reduces both context and duration while remaining greater than the average shot. In general, location shifts dominate character and time shifts in event segmentation of movies. In addition, over the last 70 years re-establishing shots have become more like the noninitial shots of a scene. Establishing shots have also approached noninitial shot scales, but not their durations. Such results suggest that film form is evolving, perhaps to suit more rapid encoding of narrative events. (shrink)

Our aim is to point out some differences between verbal and visual arguments, promoting the rhetorical perspective of argumentation beyond the relevance of logic and pragmatics. In our view, if it is to be rational and successful, film as (visual) argumentation must be addressed to spectators who hold informed beliefs about the theme watched on the screen and the medium’s constraints and conventions. In our reflections to follow, we apply rhetorical analysis to film as a symbolic, human, and communicative act (...) that may sometimes be understood as a visually laid out argument. As a mixture of visual, auditory, and verbal stimuli, film demands active and complex interpretation and (re)construction. Our suggestion is to focus on five different but interrelated elements. The reconstruction and evaluation of the visual argument will be based on those elements, and the whole process will be one of visual argumentation. (shrink)

Narratives are an integral part of human expression. In the graphic form, they range from cave paintings to Egyptian hieroglyphics, from the Bayeux Tapestry to modern day comic books (Kunzle, 1973; McCloud, 1993). Yet not much research has addressed the structure and comprehension of narrative images, for example, how do people create meaning out of sequential images? This piece helps fill the gap by presenting a theory of Narrative Grammar. We describe the basic narrative categories and their relationship to a (...) canonical narrative arc, followed by a discussion of complex structures that extend beyond the canonical schema. This demands that the canonical arc be reconsidered as a generative schema whereby any narrative category can be expanded into a node in a tree structure. Narrative “pacing” is interpreted as a reflection of various patterns of this embedding: conjunction, left-branching trees, center-embedded constituencies, and others. Following this, diagnostic methods are proposed for testing narrative categories and constituency. Finally, we outline the applicability of this theory beyond sequential images, such as to film and verbal discourse, and compare this theory with previous approaches to narrative and discourse. (shrink)

Pictures in historical chapters of textbooks convey information about the values and assumptions of the authors’professions and the larger culture. We scrutinized 15 recent abnormal psychology textbooks for reproductions of art created before 1900. Thirteen works appeared in three or more textbooks. Overall, these pictures support a “Whiggish” account of history that celebrates the present and gives a distorted, incomplete rendering of the past. The 13 pictures tended to depict the mentally ill as an underclass who are released from their (...) literal and metaphorical shackles by men who are “ahead of their time” in their struggles against prevailing ignorance. The pictures also emphasized the difference of the mentally ill by presenting a catalogue of stereotypic visual elements attributed to insanity throughout Western history. We argue for the inclusion in textbooks of “history for the sake of the past” as a way of genuinely engaging different ideologies and thereby stimulating interest in our own implicit values and biases. (shrink)

Critical discourse analysts are increasingly required to account for multimodal phenomena constructed through language and other resources and to relate high-level critical insights on the social motivations of these texts to their realizations in low-level expressive phenomena, and vice versa. In this paper, we use interactive software resources for critical multimodal discourse analysis. Multimodal analysis and digital technology. In A. Baldry & E. Montagna, Interdisciplinary approaches to multimodality: Theory and practice. Readings in intersemiosis and multimedia. Campobasso: Palladino; O'Halloran, K. L., (...) Tan, S., Smith, B. A., & Podlasov, A.. Challenges in designing digital interfaces for the study of multimodal phenomena. Information Design Journal, 18, 2–12; Smith, B. A., Tan, S., Podlasov, A., & O'Halloran, K.L.. Analyzing multimodality in an interactive digital environment: Software as metasemiotic tool. Social Semiotics) to help in achieving these aims. The field of critical discourse studies, itself being interdisciplinary, along with its holistic approach, is well placed to take advantage of interactive software, as these resources encourage the application, exploration and correlation of various analytical perspectives at different levels of description. Drawing on the analysis of a short video advertisement, we demonstrate how the interactive digital environment is one in which the discourse analyst can effectively draw upon different traditions of analysis, including ‘mainstream’ and social semiotic traditions, as well as other traditions such as media studies, to interpret dynamic audiovisual media texts in a critically self-reflexive manner. (shrink)

This paper addresses the question how to analyze multimodal public discourse in such a way that the resulting reconstruction of the rhetor’s accountability either obliges the rhetor to acknowledge the argumentative reconstruction as valid or to refute its validity in a meta-discussion. This is a challenge for discourse theory as well as for argument theory because multimodal discourse seems far removed from the ‘standard’ propositional format of an argument. We argue that multimodal discourse should be analyzed as a coherent and (...) relevant discourse, assuming the possibility of instant interactions between all modes. We introduce a method that allows us to account for an argumentative reconstruction in a systematic way. We illustrate our method by analyzing the ABC news item titled Hu Jintao Visit: Economics and Panda Bears of January 20, 2011, holding ABC news as a rhetor accountable for several far reaching standpoints that are implied in the multimodal format. (shrink)

Filmmakers use continuity editing to engender a sense of situational continuity or discontinuity at editing boundaries. The goal of this study was to assess the impact of continuity editing on how people perceive the structure of events in a narrative film and to identify brain networks that are associated with the processing of different types of continuity editing boundaries. Participants viewed a commercially produced film and segmented it into meaningful events, while brain activity was recorded with functional magnetic resonance imaging (...) (MRI). We identified three degrees of continuity that can occur at editing locations: edits that are continuous in space, time, and action; edits that are discontinuous in space or time but continuous in action; and edits that are discontinuous in action as well as space or time. Discontinuities in action had the biggest impact on behavioral event segmentation, and discontinuities in space and time had minor effects. Edits were associated with large transient increases in early visual areas. Spatial-temporal changes and action changes produced strikingly different patterns of transient change, and they provided evidence that specialized mechanisms in higher order perceptual processing regions are engaged to maintain continuity of action in the face of spatiotemporal discontinuities. These results suggest that commercial film editing is shaped to support the comprehension of meaningful events that bridge breaks in low-level visual continuity, and even breaks in continuity of spatial and temporal location. (shrink)

Traditionally there has been a great divide between those practitioners of comparative religion who work on discrete and identifiably religious traditions (such as Confucianism, Judaism, Buddhism, Islam, Christianity, etc.) and those who work on identifying aspects of ‘religious’ life that often go unnoticed because they are less traditional and therefore less recognizable as religion. There has also long been a predisposition not to view Greek materials as religious, and thus to secularize one form of thriving polytheism about which we know (...) a great deal. Combining those two methodological insights in a study of contemporary North American religious life, Gary Laderman offers a fresh look at what I will call the ‘practical polytheism’ of modern US society, one that examines cultural arenas as diverse as sport, music, celebrity, sexuality and death in a riveting analysis of current spiritual trends in a more multicultural North America. (shrink)

Cohn's (2013) theory of “Visual Narrative Grammar” argues that sequential images take on categorical roles in a narrative structure, which organizes them into hierarchic constituents analogous to the organization of syntactic categories in sentences. This theory proposes that narrative categories, like syntactic categories, can be identified through diagnostic tests that reveal tendencies for their distribution throughout a sequence. This paper describes four experiments testing these diagnostics to provide support for the validity of these narrative categories. In Experiment 1, participants reconstructed (...) unordered panels of a comic strip into an order that makes sense. Experiment 2 measured viewing times to panels in sequences where the order of panels was reversed. In Experiment 3, participants again reconstructed strips but also deleted a panel from the sequence. Finally, in Experiment 4 participants identified where a panel had been deleted from a comic strip and rated that strip's coherence. Overall, categories had consistent distributional tendencies within experiments and complementary tendencies across experiments. These results point toward an interaction between categorical roles and a global narrative structure. (shrink)

Whereas repeated exposure to communication is a widespread phenomenon, it has so far received little attention in communication research. This article takes a step towards describing, differentiating, and explaining repeated exposure to communication. It discusses different forms of repeated exposure and then focuses on repeated exposure to narrative films. It explores possible motivations for reusing the same media content again and again, while taking processes of repeated exposure as well as situational and personal variables into account. The initially theoretical considerations (...) are then supported, expanded, and specified both by existent empirical evidence and findings from a focus group study. Finally, further questions about repeated exposure to narrative content in media are discussed. (shrink)

In the 21st century, the field of business and finance has become a media spectacle. Not only have advances in technology changed the ways in which audiences engage with business information, the pervasiveness of internet and cable television networks has led to the emergence of new hybrid forms of business news discourse, blending verbiage, images, graphics, audio, and video clips. Combining discourse analysis, social semiotic theory, and other interdisciplinary approaches, this article explores the multiple ways in which business news are (...) mediated on the internet by continuous 24-hour business news networks such as Bloomberg, CNBC, FOXBusiness, and Reuters. In particular, this article is concerned with how events are contextualized, that is, how identities and social relationships are constructed and represented within and across different modes, media, and networks. The analysis focuses on what is foregrounded or backgrounded, what is thematized or unthematized in each mode and medium, and what process types and categories are drawn upon to represent events, social actors, and social actions. In this context, special interest is being paid to the semiotic shifts or transformations that multimodal representations undergo across modes, media, and news networks. (shrink)

Following its invention, cinema was initially conceived and approached as photographed theatre. After a reasonable period of self-establishment, however, it has become commonplace that cinema essentially differs from theatre, and is thus a new and independent dramatic art form. Eventually, while the advent of performance art created the illusion of a basic affinity to theatre, on the grounds of spectators actually experiencing real bodies on a stage, there has been a broadening of the alleged gap between theatre and cinema, in (...) which the spectator’s experience is mediated by images of actors projected on a screen. I reconsider here both the initial and eventual approaches, reflecting my own intuition that, without ignoring fundamental differences, a feature film is a recording of a fictional world formulated in the medium of theatre. It has to be decoded and interpreted, therefore, as a theatrical text, since a recording does not change the nature of the recorded text. Consequently, differences between cinema and theatre are fundamentally the result of technical constraints and advantages. I support these theses by a thorough analysis and critic of Roland Barthes’ seminal article ‘Rhetoric of the image’ on still photography. (shrink)

A systematic rhetorical analysis may reveal elements of multimodal argumentative discourse that would otherwise remain hidden. In this article, we present simultaneously the basics of the method we have developed to integrate theories about different modalities in one parallel processing framework for rhetorical analysis and the results of its application to an intriguing ad.

Topical Themes in Argumentation Theory brings together twenty exploratory studies on important subjects of research in contemporary argumentation theory. The essays are based on papers that were presented at the 7th Conference of the International Society for the Study of Argumentation in Amsterdam in June 2010. They give an impression of the nature and the variety of the kind of research that has recently been carried out in the study of argumentation. The volume starts with three essays that provide stimulating (...) theoretical perspectives on argumentation. Subsequently, some views are explained on the intriguing topics of ‘dissensus’ and ‘deep disagreement’. After a discussion of three different approaches to the treatment of types of argumentation some classical themes from antique argumentation theory are revisited. The new research area of visual argumentation is explored in the next part. The volume concludes with three reports of experimental studies concerning argumentative discourse. The volume starts with three essays that provide stimulating theoretical perspectives on argumentation. Subsequently, some views are explained on the intriguing topics of ‘dissensus’ and ‘deep disagreement’. After a discussion of three different approaches to the treatment of types of argumentation some classical themes from antique argumentation theory are revisited. The new research area of visual argumentation is explored in the next part. The volume concludes with three reports of experimental studies concerning argumentative discourse. The volume starts with three essays that provide stimulating theoretical perspectives on argumentation. Subsequently, some views are explained on the intriguing topics of ‘dissensus’ and ‘deep disagreement’. After a discussion of three different approaches to the treatment of types of argumentation some classical themes from antique argumentation theory are revisited. The new research area of visual argumentation is explored in the next part. The volume concludes with three reports of experimental studies concerning argumentative discourse. (shrink)