Intersemiotic projection is one of the most common configurations in the knowledge construction process of academic comics. Although previous studies address some general features of intersemiotic projection, further research on interdependency relations of intersemiotic projection is needed in order to map out the whole system. This study, based on the social-semiotic approach to multimodal studies, proposes a systemic framework of image-text paratactic and hypotactic projection in academic comics. This framework identifies three sub-categories of paratactic projection and hypotactic projection, respectively: Emergent, (...) Adjoined, Sequential, Compound, Subsumed, and Fused. The use of these projection configurations contributes to the realization of the communicative purposes of academic comics to evidence, exemplify, emphasize, explain, elaborate, and visualize the discussed knowledge. This research attempts not only to contribute to the theoretical development of multimodality from the social-semiotic perspective by refining and enlarging the multimodal hypotactic projection framework but also to provide an effective and applicable tool for comics analysis. (shrink)

This article presents a fresh perspective on the interplay among visual, verbal, and auditory modalities, positing that these modalities, as semogenic resources, compete to express dynamic meanings. The theoretical paradigm emphasizes that whether a modality or an element within a modality gets or loses semantic status, it will elicit an additional layer of social meaning to depict a comprehensive picture of a story together with an explicit semiotic meaning. The article adopts a qualitative method to analyze the data, which are (...) drawn from The Good Wife and My Roommate is a Gumiho and annotated in ELAN 6.3. It was found that modal competition can shed light on the dynamic meaning-making processes in semiotic and societal orientations. Modal competition may distort space and time of different stories, and reconstruct a different discursive spatio-temporal dimension in the TV world. It can diversify the dynamic orientations from New to Given in visual, verbal, and auditory texts of multimodal discourses to tell stories. Modal competition provides a lens to understand the multidimensional reality and to appreciate the aesthetics of a modern TV series. (shrink)

This study investigates the influences of action research on primary school English instruction from five dimensions in the classroom, viz., types of questions, language errors, gestures, facial expressions, and interpersonal distance. Four English teachers’ 9 real classroom teaching videos before and after action research are collected and annotated by using ELAN software. The results show that primary school English teachers in Chinese rural areas prefer closed questions to open questions; They make some language errors; Deictic gestures are the most common (...) gestures used, while metaphoric gestures, beat gestures and iconic gestures are rare; Teachers have the same preference for three types of facial expressions, and teachers’ serious expressions accounts for most of the time; They seldom keep an intimate distance or personal distance from their students. Second, Action research is effective to motivate teachers in rural areas, who make great progress in all five dimensions after AR: more open questions are asked; pragmatic errors and grammatical errors are reduced; deictic gestures increase; apathetic decrease, and more intimate distance is exhibited in the interpersonal distance dimension. Third, teacher’s English teaching competence is partly transferrable to her future professional development, and this is also the long-lasting effect of AR. (shrink)

We aim to contribute to recent situational approaches to the study of interpersonal violence by elaborating the concept of trajectories. Trajectories are communicative processes in which antagonists act upon each other’s bodily and verbal actions to project a direction for the interaction to take, which is then (con) tested in the exchanges that follow. We use the notion of trajectories to gain insight in how participants turn an antagonistic situation into a violent encounter, which we contrast to interactionist and micro-sociological (...) understandings. Using ethnomethodological and conversation analytical tools, we detail the trajectories of three violent encounters, captured on phone camera recordings to answer the question how verbal and bodily exchanges project physical violence. Methodologically, our contribution shows how bodily actions can be studied in visual data. Our cases show how antagonists move the interaction toward violence by creating a metaconflict revolving around the conditions under which the interaction will become a physical confrontation; what we call the contested projection of violence. We conclude that the concept of trajectories offers a useful analytical tool to detail the shifts and turns of the interactive process—notably it’s bodily dimensions— that characterize antagonism and violence. Substantially, our analysis raises questions about conceptualizations of the emotional dynamics (notably the role of dominance) of violence, as proposed by earlier micro-sociological and interactionist work. We therefore suggest that future studies engage with these issues in more detail and in larger datasets. (shrink)

This study shows how three Chilean media construct in their discourse the social actor ‘secondary students’, in a multimodal way. It focuses on the representation of Televisión Nacional de Chile (also known as TVN), El Dínamo and El Ciudadano of this social group which is often suppressed suppressed from the media sphere. It is a qualitative study with a social semiotic approach, following the guidelines of Visual Grammar and the typology of representation of social actors by van Leeuwen (2003). The (...) corpus comprised 15 multimodal texts of 2018 compiled from internet sites, which constitute a semiotic chain of a news event about education. The results suggest that TVN creates the meaning of an ‘aggressor student’, El Dínamo as a ‘student to be reformed’; whereas El Ciudadano, represents him as a social actor who is an ‘active participant’. (shrink)

The aim of this article is to explore website interactivity as discourse. Whereas the use of writing, images and layout in web design has been explored extensively, interactivity, that is, interactions between a web user and the website system, remains an underdeveloped area of discourse studies. To analyze interactivity as discourse, the article uses data from a research project on offline and online shopping for electronics, viewing the offline-online relationship as recontextualization in the sense that webshop interactivity represents and transforms (...) in-store shopping actions. Using a methodology that combines analytical framework for interactive sites and approach to discourse analysis, the article maps cursor resources and interactive webshop features as actions and compares them to the actions that constitute in-store shopping. On this basis, the article offers reflections on how interactivity plays a defining role in the digital resemiotization of social practices. (shrink)

The aim of this article is to determine the representation of persons with disabilities in the print media advertisements of Costa Rica during the period 2014-2016. In the first place, we analyze the advertisements on disability that have appeared in the three most widely circulated newspapers in Costa Rica, following the Fairclough’s budgets about the representation of social actors, inclusion and exclusion strategies, legitimization and explanation that are made of the PCD in the ads. We determine the actions performed by (...) the PDC in the analyzed texts, the relationships with other people that appear in the announcements. Finally, from a multimodal perspective of Kress and Van Leeuwen, through the visual analysis and its relation with the linguistic one, we analyzed the congruent or incongruent images that are promoted in the advertising discourse. It is concluded that there is an invisibility of this population; since the number of ads on disability is less than 0.0015% and that the speech reproduces stigmas and stereotypes from a social- assistance perspective of the PCD, even though it is the State the main advertiser of the ads. (shrink)

Motivated by a call for risk discourse research in non-Western settings, this study examines how Vietnamese road safety video advertisements are constructed and interpreted. Following a literature review of risk and gender, the study first sets out socio-semiotic multimodal analysis of video advertisements to detail how concepts of risk and gender are constructed. Afterwards, I take an ethnographic-based approach in a series of interviews with professional Vietnamese women to gain insights into these concepts. Findings of the study reveal the expected (...) female roles as wives and mothers in Vietnamese advertising as interpreted by Vietnamese women. The study contributes to an understanding of risk in this Asian setting and opens up the potential for a critical, interpretive approach to risk discourses that are grounded in lived experiences. (shrink)

This paper addresses the question how to analyze multimodal public discourse in such a way that the resulting reconstruction of the rhetor’s accountability either obliges the rhetor to acknowledge the argumentative reconstruction as valid or to refute its validity in a meta-discussion. This is a challenge for discourse theory as well as for argument theory because multimodal discourse seems far removed from the ‘standard’ propositional format of an argument. We argue that multimodal discourse should be analyzed as a coherent and (...) relevant discourse, assuming the possibility of instant interactions between all modes. We introduce a method that allows us to account for an argumentative reconstruction in a systematic way. We illustrate our method by analyzing the ABC news item titled Hu Jintao Visit: Economics and Panda Bears of January 20, 2011, holding ABC news as a rhetor accountable for several far reaching standpoints that are implied in the multimodal format. (shrink)

Argumentation is a cognitive category. Texts cannot be said to be argumentation, nor can argumentation be said to lie in texts. This is an almost trivial semiotic point of departure, but it is quite relevant nevertheless. In this contribution, three reasons are developed to emphasize and to articulate the semiotic component of argumentation to show that it is a crucial element that cannot be disregarded. Two of these reasons are mentioned only in passing as other contributions in this volume deal (...) with them more substantially. The third reason, being that argumentation requires an exchange of discourse worlds and that consequently the mimetic construction of these discourse worlds is part of the argumentation, is discussed in some detail in this paper. It will be argued that a lack of attention for the mimetics of argumentation is regrettable, both theoretically and practically. Focusing on the mimetics raises questions concerning the dominant ‘propositional’ format of argumentation assumed to be essential for argumentative assessment. (shrink)

In this paper we explore quarantining as a more ethical method for delimiting the spread of Hate Speech via online social media platforms. Currently, companies like Facebook, Twitter, and Google generally respondreactivelyto such material: offensive messages that have already been posted are reviewed by human moderators if complaints from users are received. The offensive posts are onlysubsequentlyremoved if the complaints are upheld; therefore, they still cause the recipients psychological harm. In addition, this approach has frequently been criticised for delimiting freedom (...) of expression, since it requires the service providers to elaborate and implement censorship regimes. In the last few years, an emerging generation of automatic Hate Speech detection systems has started to offer new strategies for dealing with this particular kind of offensive online material. Anticipating the future efficacy of such systems, the present article advocates an approach to online Hate Speech detection that is analogous to the quarantining of malicious computer software. If a given post is automatically classified as being harmful in a reliable manner, then it can be temporarily quarantined, and the direct recipients can receive an alert, which protects them from the harmful content in the first instance. The quarantining framework is an example of more ethical online safety technology that can be extended to the handling of Hate Speech. Crucially, it provides flexible options for obtaining a more justifiable balance between freedom of expression and appropriate censorship. (shrink)

Name der Zeitschrift: Semiotica Jahrgang: 2018 Heft: 220 Seiten: 41-67.

There is a growing research interest in the value of participative arts-based strategies for enhancing wellbeing amongst adults living with dementia. One such intervention, centred around literature, is the group activity called Shared Reading. The purpose of this case study of weekly Shared Reading sessions of poetry in a care home in Merseyside is to investigate instances of how participants with mild to moderate dementia collaborate in processes of meaning-making that allow them shared experiences of being moved by poetry. An (...) under-thematised aspect of psychological wellbeing is the capacity for being moved and for sharing such moments. This article addresses the following question: how can the specific multimodality of the text in the Shared Reading model help to bring about such experiences? Using Stern’s concepts of Now Moments and Moments of Meeting, this case study discusses various instances of unpredictable, surprising and spontaneous intersubjective moments between participant and poem, participant and reader leader, participant and staff, participant and relative. (shrink)

In a 2018 referendum, the Irish electorate voted in favour of repealing Ireland's quasi-total legal ban on abortion. The referendum campaign saw important public discussions regarding gender roles in twenty-first century Ireland. While the constitutional ban on abortion was condemned by abortion rights advocates for marginalising women's agency, the legislation which replaced it has not escaped criticism either. Therefore, questions surrounding the conceptualisation of women's agency in the 2018 referendum are still relevant today. Adopting a multimodal critical discourse analysis approach, (...) this paper analyses signage from the weeks before the vote to examine the discursive construction of women's agency in the linguistic landscape of the referendum campaign. I argue that many mainstream campaign organisations – including those arguing for liberalisation of abortion laws – were complicit in the discursive diminishment of women's agency, suggesting that the campaign did not necessarily challenge prevailing ideologies of gender and agency. (shrink)

In this article, we investigate the use of social media in contemporary family interaction from a linguistic ethnographic perspective. Inspired by Auer’s work on code-switching in conversation, we study how family members choose and sometimes alternate between digitally mediated and face-to-face modes of communication in various family settings. Based on ethnographic observations, the participants’ metapragmatic reflections, and their interactional orientations to mode choices, we show how such choices serve social and metapragmatic functions in the interaction between family members who are (...) present in the same house or even in the same room. Accordingly, we argue in favor of situating peoples’ polymedia repertoires in a broader framework of communicative repertoires. (shrink)

The expression ‘Fake News’ inside Internet memes engenders significant online virulence, possibly heralding an iconoclastic emergence of weaponized propaganda for assaulting agencies reared on public trust. Internet memes are multimodal artifacts featuring ideological singularities designed for ‘flash’ consumption, often composed by numerous voices echoing popular, online culture. This study proposes that ‘Fake News’ Internet memes are weaponized iconoclastic multimodal propaganda discourse and attempts to delineate them as such by asking: What power relations and ideologies do Internet memes featuring the expression (...) ‘fake news’ harbor? How might those manifestations qualify as WIMP discourse? A multimodal critical discourse analysis of a small pool of ‘fake news’ Internet memes drawn from four popular social media websites revealed what agencies were often targeted and from what political canons they likely emerged. Findings indicate that many Internet memes featuring ‘fake news’ are specifically directed, revealing an underlying hazard that WIMP discourse could diminish democratic processes while influencing online trajectories of public discourse. (shrink)

Framed within the broader theoretical context of social semiotics, we attempt to show how university students communicate using a variety of unique means, in particular social contexts. We privilege Pennycook and Otsuji’s semiotic assemblages, Jimaima and Simungala’s semiotic creativity, and the notion of semiotic economy as critical ingredients that conspire to give rise to the unique and complex coinages and innovations constituting students’ repertoires. We argue that, born out of creativity, the students’ repertoires are semiotically and economically charged discourses that (...) generate extended narratives such that more is realized with less. We show that this reality undoubtedly constitutes a multi-semiotic meaning-making endeavor that enacts and sustains students’ imagined and lived experiences in real sociocultural, historical, and political spaces in the multilingual landscapes of university campuses. (shrink)

Instagram serves as a powerful instrument for youth socialization, self-expression, and self-performance in visual online spaces. Using social semiotics and multimodal discourse analysis, this study examines the potential ideological meanings and implications of selfie-shooting and sharing on Instagram on young adults’ self-concept. A corpus of 110 questionnaires, including almost 85 captioned selfies, was surveyed as multimodal utterances. In doing so, this study argues that selfies can create young adults’ split-selves while constructing their multiple personas in visual online spaces. This marks (...) the significance of viewing selfie-creators not only as authors of their selfies, but rather as viewers of a three-fold self: an ideal-self, a projected-self, and an internal-self, to negotiate social and power relationships, while (re)positioning observer-observed roles. This study claims originality in unraveling how young adults use visual and textual mediated communication to represent and perform their split-selves. It suggests that selfie-shooting-sharing has become a key self-performance tactic and behavior in online cultures. Therefore, young adults deploy selfies and captions to posit a redefinition of certain social values, such as aesthetics and freedom, while deploying their selfies and captions. Challenging certain orthodox social allegiances, they conceive wildness, messiness, and exuberance as emerging neo-aesthetics components of appeal. This study contributes to the literature on personal visual communication with insights on how Egyptian young adults perform their self-concept via the semiotic practice of selfie-shooting-sharing. (shrink)

This paper proposes a methodological synthesis in order to study multimodal media discourse and argumentation in the context of the so-called ‘refugee crisis’ in Greece. It follows the framework of Multimodal Critical Discourse Analysis, integrating this with argumentation studies, with a particular emphasis on the analysis of inference. Our data come from the Greek newspapers Kathimerini and Ta Nea. We contend that the proposed methodological synergy enables scrutiny of (a) racist conceptualizations cultivated by the representation of migrants and refugees in (...) headlines and photographs in newspaper articles along with (b) the argumentative potential that is implicitly sustained in these multimodal representations. (shrink)

Using a multimodal discursive approach, this study explores how the COVID-19 pandemic is constructed and used in press reportage to mobilize intergroup relations and national identities. We examine how press reporting about the development of COVID-19 in Sweden is cast as a matter of nationalism and national stereotyping in the Finnish press. The data consist of 183 images with accompanying headlines and captions published in two Finnish national newspapers between January 1 and August 31, 2020. We found three multimodal rhetorical (...) strategies of stereotyping: moralizing, demonizing, and nationalizing. These strategies construct discourses of arrogant, immoral, and dangerous Swedes sourcing from national stereotypes. The study contributes to current knowledge about the work on national stereotypes by illustrating how they are used in media discourse to achieve certain rhetorical ends, such as to persuade, mitigate, or justify intergroup relations. Furthermore, the study offers insight into the multimodal constructions and functions of stereotypes. (shrink)

In academic oral genres such as conference presentations, speakers resort to more than words to convey meaning. Research also suggests that persuasion, an important element of the communicative purpose of conference presentations, is frequently achieved through a combination of semiotic modes. Therefore, a skilful orchestration of these modes can be considered key to achieving effective communication in this genre. However, our understanding of persuasion has often focused on specific elements of the message considered in isolation and mainly from the linguistic (...) perspective. Relatively little attention has been paid to the overall persuasive effect achieved by the complex multimodal ensemble. This study approaches the analysis of persuasive strategies in conference presentations combining multimodal discourse analysis and ethnographic methods. It focuses on a particular attention-getting technique: enactment of characters, or acting the part of a person that is being referred to. Our analysis shows how it is achieved through the orchestration of different modes such as words, intonation, gestures, head movements, gaze and facial expression. (shrink)

Deepfakes, an algorithm that transposes the face of one person onto the face of another person in images and film, is a digital technology that may fundamentally alter our belief in visual modality and thus presents alarming consequences for an image-centric culture. Not only are these face-translations now so advanced that it is virtually impossible for people to tell that they are fake – this technology is also becoming accessible to laypersons who, with little or no computer skills, can use (...) them for all kinds of purposes, including criminal intentions like revenge porn and identity theft. It is therefore timely and crucial to explore the semiotic potential of deepfakes. This paper presents a semiotic technology perspective, i.e., the study of technology for meaning- making that is an emergent field in social semiotics, to report on findings from an ongoing study of how deepfake software is designed and used as a semiotic resource in erotic and political contexts. The paper advances the argument that the software is able to appropriate all signifiers of the face and their cultural history. Consequently, the semiotic operations of this technology prepare the ground for the problematic perspectives of synthetic facial imagery. On this basis, the paper calls for a critical awareness of taking visual representations of current events at face value and considers how deepfake technology is embedded in unsound sharing practices of visual artefacts that tamper with the rich meaning potential of the face. (shrink)

Firstly, this paper aims to analyze how verbal and visual modes contribute to build two basic components of cartoons: the referenced situation and the fictional situation. Secondly, it aims to unravel the semiotics of this discursive genre by offering a fine-grained picture of modal variations and continuities of the rhetorical resources deployed, by means of which the fictional situation is displayed. The corpus is composed of 50 multimodal cartoons chosen by cartoonists of 22 nationalities as the most representative of their (...) work in the frame of a wide questionnaire. Category analysis and non-parametric statistics were applied. We identified and fleshed out four different types of modal functioning of resources: visual preference, verbal preference, modal flexibility, and genuine multimodal functioning. Even if visual and multimodal metaphor were often in charge of transferring meaning from fiction to the referenced situation, results show that other resources such as antithesis, irony, and syllepsis can become the key resources of cartoons. Findings highlight the multilayered functioning of cartoons, among which plastic features can play a significant role, and provide further evidence in favor of the hyperdetermination of humor hypothesis proposed by Attardo. (shrink)

This paper will use the theoretical concepts of ‘intertextuality’, ‘interdiscursivity’ and ‘resemiotization’ to analyse four media texts on South African president, Jacob Zuma. The aims of the paper are, first, to analyse the role that intertextual references play in the construction of the identity of public figures. Second, the paper investigates the semiotic affordances of the visual and linguistic mode by tracing how previous discourses and texts about Jacob Zuma move across discursive spaces and modes. The findings suggest that reference (...) to previous discourses play a fundamental role in the way in which Jacob Zuma is constructed. More specifically, resemiotization from the verbal to the visual mode seems to serve as a way to bring multiple discourses into a text without necessarily articulating these discourses linguistically. These findings are discussed within the broader framework of critical discourse analysis. (shrink)

The introduction claims that there might exist an evolutionary bridge from possible genres in nature to human cultural genres. A sub-hypothesis is that basic life-conditions, partly common for animals and humans, in the long run can generate so-called life-genres. To investigate such hypotheses a framework of interrelated key communicational concepts is outlined in the second, main part. Four levels are suggested. Signs are seen as elements in utterances. Further, sufficiently similar utterances can be perceived as kinds of utterances or genres. (...) Genres are kinds of communication in a species' life-world. Utterance and genre are the framework’s key concepts. Both are seen as products and processes and as constituted by five reciprocal aspects: form, content, act, time, and space, the necessary elements to initiate and interpret communication. A model illustrates this integrative principle. Part three dwells with the idea of life-genres, and part four exemplifies and problematises how a chimpanzee’s ‘begging’ for meat, a fish’s nest-making, and kinds of birdsongs, could be seen as possible life-genres. The last, short part gives a brief summary, highlights challenges, and reflects over the framework’s relevance for biosemiotics. (shrink)

This paper explores a semiotic notion of body as starting point for bridging biosemiotic with social semiotic theory. The cornerstone of the argument is that the social semiotic criticism of the classic view of meaning as double articulation can support the criticism of language-centrism that lies at the foundation of biosemiotics. Besides the pragmatic epistemological advantages implicit in a theoretical synthesis, I argue that this brings a semiotic contribution to philosophy of mind broadly. Also, it contributes to overcoming the polemic (...) in linguistics between, loosely put, cognitive universalism and cultural relativism. This possibility is revealed by the recent convergence of various semiotic theories towards a criticism of the classic notion of meaning as double articulation. In biosemiotics, the interest to explicate meaning as multiply articulated stems from the construal of Umwelt as relying on the variety of sense perception channels and semiotic systems that a species has at its disposal. Recently, social semiotics developed an unexplored interest for embodiment by starting from the other end, namely the consideration of the modal heterogeneity of meaning. To bridge these notions, I employ the cognitive semantic notion of embodiment and Mittelberg’s cognitive semiotic notion of exbodiment. In light of these, I explore the possible intricacies between the biosemiotic notion of primary modeling system and concepts referring to preconceptual structures for knowledge organization stemming from cognitive linguistics. Further, Mittelberg’s concept of exbodiment allows for a construal of meaning articulation as mediation between the exbodying and embodying directions of mind. (shrink)

This paper reconsiders semiotic modelling in light of recent scholarship on Charles Peirce, particularly regarding his concept of proposition. Conceived in the vein of Peirce’s phenomenological categories as well as of his taxonomy of signs, semiotic modelling has mostly been thought of as ascending from simple, basic sign types to complex ones. This constitutes the backbone of most currently accepted semiotic modelling theories and entails the further acceptance of an unexamined a priori coherence between complexity of cognition and complexity of (...) signification. Following recent readings of Peirce’s post-1900 semiotic, we engage in a discussion as to what are the limits of this approach. From Stjernfelt’s conception of the dicisign in nature, we derive a perspective that affords understanding the practice of modelling as a reciprocal interplay between decomposition of complexity and recombination into further complexity. This discussion is facilitated by the recent extrapolation of the constructivist concept of scaffolding in biosemiotics research. Cognition, we argue, begins with a fundamental irritation of trying to make sense of a structure that is more complex than what can directly be derived from experience and, in so doing, urges meaning-seeking processes. The yet unknown object is decomposed into more tangible objects and is subsequently reassembled from these more manageable conceptions of the object. In support of our argument, we discuss the notions of semiotic competences and resources in light of such a naturalized account of meaning-making. (shrink)

This study develops a biosemiotic framework for a descriptive phenomenology. We incorporate the set utterance-genre-lifeworld in biosemiotic theory by paralleling it with the Peircean-Uexküllean notions of sign, habit, and Umwelt (respectively). This framework for empirical semiotic studies aims to complement the concepts of affordance and scaffold, as applied in studies on learning.The paper also contributes to bridging Bakhtinian-Hallidayian-Habermasian views on utterance, genre, and lifeworld with biosemiotics. We exploit the possibility that biosemiotics offers to bring together hermeneutic and phenomenological analysis. We (...) relate these views to integrated levels in a systemic framework for communication. Signs are seen as inter-dependent construction elements in utterances. Repeated use of utterances in shifting contexts generates shared recognizable kinds of communication, or genres. ‘Life-genre’ is used in a zoo-communication context, to avoid anthropocentrism. Life-genre serves animals’ life-functions. Genres make up a systemic network of communicational resources, along with the related concepts of event, affordance, and scaffold. Utterance, genre, and lifeworld have five aspects, constituting an integrative approach to communication: form, content, act, time, and space. Semiosis and positioning are processes that connect aspects and levels. Levels, aspects, and processes make up the framework as a system.While biosemiotics supports a phenomenological notion of life-genre, in turn, this notion also contributes to the development of the former, bridging a gap between organisms’ sign experience on a micro-level and organisms’ phenomenal lifeworld/Umwelt. Comparisons of lifeworld and Umwelt reveal that, although not identical, these are sufficiently similar to be perceived as an overall macro-level for signs and communication. Starting from a construal of utterance and genre as dynamic, dialogical, and reciprocal, genre is positioned as meso-level, mediating between signs in utterances and the lifeworld/Umwelt level. We propose genres, understood as semiotic scaffoldings built through affordances, as an analytical concept to capture meso-level phenomena. Scaffolding is determined by both ongoing events and (pre-)available genres. Events are manifestations of these scaffolding processes.Identifying animal and educational communication as two particularly relevant fields for future applications, we compare key socio-semiotic concepts to the notions of sign, habit, and Umwelt. The notions affordance and scaffolding are particularly important to think of learning as continuous with adaptation.An overall aim is to broaden as well as finetune semiotic phenomenology as a platform for empirical analyses of communication. We conclude by exemplifying and summarizing how the framework facilitates this. (shrink)

This article approaches sensory modalities as semiotically active factors and organizing principles in meaning-making. The focus will be on the special case where modalities mismatch in film – i.e., the soundtrack and visuals present contradictory meanings. The conflict can be characterized by the concept of synthesis that emerges in theories of Eisenstein, Barthes, Jakobson, Lotman, and cognitivists. The artistic functions of such synthesis will be discussed with the help of examples from selected feature films. In the first place, conflicting modalities (...) are inspected in the light of Juri Lotmanʼs theory of two incompatible, but still complementary languages that make up a mechanism for generating new information. In addition, the prospects of evaluating modality conflicts will be touched upon, dismissing synchrony and redundancy as the scale parameters, but acknowledging Lotmanʼs model of space as a primary modeling system that is capable of representing semantic conflicts. (shrink)

Journal Name: Semiotica Issue: Ahead of print.

Name der Zeitschrift: Semiotica Jahrgang: 2015 Heft: 207 Seiten: 657-673.

This paper analyzes transduction as an action of transposing information from one mode to another within the communication process and its implications in terms of meaning and coherence of a multimodal message. First, I discuss the multimodal method and its conjunction with some key concepts such as: sign, meaning, mode, transduction. Secondly, I approach transduction as an essential method of translating messages across the media variety, describing my interdisciplinary approach – that brings together semiotics and communications – and proposing a (...) framework of explanation for transduction in the field of advertising. Drawing from a previous model (Culache 2015), I illustrate the way transduction takes place and identify its meaning-making issues while introducing the concept of ‘dominant mode.’. (shrink)

Spain has become a country receiving immigrants in the last years. The majority of the items of news related to immigration that appear in the press exhibit negative characteristics. The main purpose of this article is to observe the linguistic and visual representation of Sub-Saharan immigrants in a sample of the Spanish press in order to answer the following research questions: how are Sub-Saharan immigrants portrayed linguistically and visually in the given press? What are the implications of the choices in (...) language and images? The researcher collected all the pieces of news related to Sub-Saharan immigrants in the three most popular Spanish newspapers, i.e. El País, ABC and El Mundo from 1 June 2011 to 31 March 2013. Visual grammar and critical discourse analysis will be used in order to deconstruct the visual and linguistic representation of such immigrants. In addition, van Leeuwen's [2008 Discourse and practice. New tools for critical discourse analysis. Oxford: Oxford University Press.] classification of social actors will be used in the analysis. The analysis demonstrates that the representation of Sub-Saharan immigrants displays the following characteristics: they are represented as vulnerable, lacking autonomy, as victims, etc. This representation does not contribute to the fact that the autochthonous population favours the integration of immigrants into the socio-economic structure, exercising the same rights as the Spanish population. (shrink)

ABSTRACTHow self-evident is a colonial rationality today? This paper begins by tracing a ‘text trajectory’ about nineteenth century imperialism and colonialism through several educational spaces: curricular guidelines, textbook, teachers’ reflections on history education, material discursive classroom interactions and pupils’ communication about the topic. In a first step, we observe how entrenched and common sensical a great-power discourse about imperialism and colonialism is in current educational practices. We suggest that pupils ‘hyperstate’ a discursive position on colonialism which appears shocking when stated (...) explicitly, but which is already subtly entextualised in curricular and media discourse. However, in a second step, the analysis orients to the molecular lines and foldings, the unexpected intra-connections and unplanned ‘lines of flight’. In this way, the apparent stability of the colonial rationality begins to wobble. The memory practices enacted by pupils, curricula, teachers, authors and the material affordances of textbooks repeat entrenched colonial hierarchies and racisms, ‘and also’ exceed any overly simple understanding of how colonial rationality plays out in contemporary discourse. (shrink)

This article, taking a social semiotic approach, analyses two pieces of music written, shared and exalted by two pre-1945 European fascist movements – the German NSDAP and the British Union of Fascists. These movements, both political and cultural, employed mythologies of unity, common identity and purpose in order to elide the realities of social distinction and political–economic inequalities between bourgeois and proletarian groups in capitalist societies. Visually and inter-personally, the fascist cultural project communicated a machine-like certainty about a vision for (...) a new society based on discipline, conformity and the might of the nation. In this article, we are interested in the ways that these very same discourses are also communicated through sound and music in two songs: The Horst Wessel Lied and the BUF marching song, two songs that used the same melody. We analyse the discourses communicated by the semiotic choices made in melody, arrangements, sound qualities, rhythms as well as in lyrics. The article first identifies some of the underlying semiotic resources for meaning making in sound and then shows how these are used in order to communicate specific ideas, values and attitudes. (shrink)

The rise in Study Abroad participation among college students has increased interest among educationalists wondering about the impact of SA on students, particularly when students return home without evidence of deep engagement and understanding of other cultures and people. The purpose of this case study was to locate one potential source of the meanings ascribed to the SA experience, through analysis of multimodal representations on the institutional website of a popular SA program provider. In this study, Kress’ model of multimodal (...) social semiotic communication was employed in the analysis of the ensemble of modes of communication on the website, and their role in the production and dissemination of discourses of SA. Findings indicate that discourses of tourism prevail over discourses of education, and the representations enacted on the institutional website are mirrored in the discursive practices of students. (shrink)

The semiotic world that we inhabit is fast changing in terms of the resources that are used and the practices in which many engage. Yet the institutional norms governing highly consequential academic texts – students’ texts assessed as part of their disciplinary-based activity and scholars’ papers submitted for publication – lack engagement with this array of resources and, epistemologically, continue to drive a monologic stance towards academic meaning making. The aim of this article is to argue for a reconfiguring of (...) the textual goal of academic writing, through one simple yet transformative tool, that of juxtaposition. Textual juxtaposition is illustrated in this article, and its intellectual and ideological value discussed, drawing on Bakhtin’s emphasis on dialogue, using several examples from published academic texts. The relevance to scholarly and student writing is explored. (shrink)

Multi-modal metaphor is a new perspective in metaphor research developed in modern times. Beyond the metaphor research of language, it combines text, image, sound, and other modes and provides new insights and perspectives for metaphor research. Food advertisements often combine sound, images, and other forms to promote the products and increase consumers’ desire to buy, and they often contain metaphors of multiple modes. However, under the perspective of cross-cultural research, when the same food brand is advertised in different countries, the (...) multi-modal metaphors are varied to appeal to different audiences. This article compares food advertisements in China and in the U.S. from the perspective of multi-modal metaphors, and analyzes the varied applications arising from different cultural influences. The research may act as a reference for advertising strategies on cross-cultural occasions. (shrink)

In critical discourse analysis, we have learned much about the nature of the marketized language that now dominates public institutions such as universities, playing a role in changing their identities. But less is known about the processes whereby this language enters the everyday practices of these institutions through documents that are used to manage teaching and research. What is the role of language in the shift to the way these activities are internally organized, managed, run and evaluated in terms of (...) productivity and market-based principles? In this paper we analyse a chain of documents taken from a wider corpus of management documents in Swedish universities to show how this language recontextualizes the practices of teaching and research. Our focus is on the important role played by lists, bullet points and tables and how these are central to decoupling language from work processes and so legitimizing this marketized discourse. The affordances of these multimodal structures allow complex processes and social relations to be abstracted, fragmented and treated as things. They are also important in allowing documents to form a complex self-referential information infrastructure. (shrink)

This paper considers the extent to which the earliest stages of learning about systems of inscription requires not just individual mental effort, but effort that is distributed across a wide physical and intellectual environment. It is particularly concerned with how children under the age of three learn about notational systems, including writing, and examines parallels with the evolution of written systems. It considers the position that children gain knowledge incrementally over the early months and years of life, supported by a (...) commonly accepted view that learning is a mental activity in which knowledge accumulates hierarchically in the minds of individual children, starting with smallest parts of the system. The paper presents evidence to the contrary, suggesting that the early learning of inscriptional systems is associated not just with individual minds, but with social and cultural cognition that is dispersed across minds, bodies, tools, and material environments. It presents data and evidence from a small study of the sign-making of children under the age of three that indicates that children of this age already use notations purposefully in the construction of signs that are intentional, multimodal, and unbounded, and that already have features associated with conventional systems of inscription. In their early sign-making they use certain underlying principles of symbolic reference associated with conventional systems, including the use of ‘generic’ structures derived from social, bodily, and material experience. Central to this process are networks of interactions between co-participants, tools, materials, and the physical environment. (shrink)

This paper proposes a multimodal ethnosemiotic conceptual framework for culturally themed selective Web archiving, taking as a practical example the curation of the London French Special Collection in the UK Web Archive. Its focus on a particular ‘community’ is presented as advantageous in overcoming the sheer scale of data available on the Web; yet, it is argued that these ethnographic boundaries may be flawed if they do not map onto the collective self-perception of the London French. The approach establishes several (...) theoretical meeting points between Pierre Bourdieu’s ethnography and Gunther Kress’s multimodal social semiotics, notably, the foregrounding of practice and the meaning-making potentialities of the everyday; the implications of language and categorisation; the interplay between subject and object; evolving notions of agency, authorship and audience; together with social engagement, and the archive as dynamic process and product. The curation rationale proposed stems from Bourdieu’s three-stage field analysis model, which places a strong emphasis on habitus, considered to be most accurately presented through blogs, yet necessitates its contextualisation within the broader field, through institutional websites, for example, whilst advocating a reflexive awareness of the researcher/curator’s role. This, alongside the Kressian acknowledgement of the inherent multimodality of on-line resources, lends itself convincingly to selection and valuation strategies, whilst the discussion of language, genre, authorship and audience is relevant to the potential cataloguing of Web objects. By conceptualising the culturally themed selective Web-archiving process within the ethnosemiotic framework constructed, concrete recommendations emerge regarding curation, classification and crowd-sourcing. (shrink)

Resumen Los textos televisivos interpretan y analizan la realidad circundante e influyen en la visión que la audiencia tiene del mundo. Es más, los programas cómicos basados en la burla y el sarcasmo hacia grupos étnicos minoritarios refuerzan estereotipos negativos y promueven la discriminación cultural. En este artículo se estudia el humor étnico en La paisana Jacinta. Esta serie peruana ha sido acusada de racista por diversas personas y grupos, entre ellos, la excongresista indígena Hilaria Supa, LUNDU y CHIRAPAQ. Primero, (...) se analizan las estrategias pragmático-lingüísticas empleadas en la construcción del humor étnico en una selección de diálogos y monólogos de Jacinta, el personaje principal de la serie. Para ello, se utiliza la Teoría General del Humor Verbal. Asimismo, dado que todo texto es multimodal, se tiene también en cuenta cómo imágenes, gestos, y elementos musicales y prosódicos en la serie contribuyen en la producción de la comicidad. Finalmente, a partir de estos resultados, se examina el contenido cómico del programa desde una perspectiva crítica para determinar si este es discriminatorio. (shrink)

Contemporary universities are characteristic of an evident proliferation of corporate discourse. A sole concentration on the production of new knowledge and the education of students does not ensure the prosperity or even survival of universities any longer, and equally important are the admission of elite students, the outcome-based evaluation of academic performance, the establishment of alumni network and also fundraising. This article examines how and to what extent this trend of marketization has invaded the order of discourse of Chinese universities. (...) The research methodology combines the paradigms of critical discourse analysis and the tenets of critical genre analysis [Bhatia VK Towards critical genre analysis. In: Bhatia VK, Flowerdew J and Jones RH Advances in Discourse Studies. London: Routledge, pp. 166–177], with particular attention paid to the notion of genre, interdiscursivity and conversationalization. The texts examined include ceremonial speeches, regulatory documents, insiders’ accounts and field notes. Analytical results show that the trend of marketization, in spite of a potential threat for academic integrity, does facilitate the institutional restructuring and transformation of universities in the context of Mainland China. (shrink)

This paper considers the ways that concepts such as social justice and law were used as semiotic objects-in-tension by a group of five US undergraduates considering law school to make sense of their ideas about entering the discourse communities and communities of practice associated with being a lawyer. This group was made up of undergraduate women who had completed a summer residency program sponsored by the Law School Admissions Council to increase enrollment of students from under-represented groups. Of the five (...) participants, two were US-born; the others immigrated to the US as teenagers; each was aware of her position as multiply marginalized, by gender as well as other factors, including refugee or immigrant status, religious affiliation, sexual identity, and/or association with “at risk” labeling. Data analyzed reflect a 3-year study of their changing perceptions of their relationships to law school discourse communities, using text, interviews, individual video narratives, and informal, face-to-face group meetings. A sociolinguistic approach to multimodal discourse analysis is used to examine the ways that the women, each in a unique way, articulated an increased investment in direct and embodied engagement, lived experience, and personal testimony—not as supplements to doing/being a lawyer, but as necessary and expected practices therein. Over time and through various modalities, they used their vantage point from outside the dominant discourse communities of law to stage social critique and to contest the binary logic and normative criteria that forge the boundaries of exclusion from and inclusion in these communities. Specifically, they resemiotized notions of being a lawyer from the margins in ways that demanded a more fluid and polysemous interpretation of what it means to do ethically rigorous social justice work—hence reworking the relationships between justice and the law and widening the semiotic potential of their own future work. Particularly significant are the ways that semiotic trajectories progressed from an emphasis on what Halliday identifies as textual functions of language to interpersonal and ideational functions. Such a trajectory away from entextualization suggests that voices and perspectives from the margins may be using those imaginary margins tactically as sites from which to contest the boundaries that define whose voices count within the legal system and to contest normative limits on semiotic potentialities for lawyers working toward more just social futures. (shrink)

To a greater extent than in other technical domains, research and progress in Artificial Intelligence has always been entwined with the fictional. Its language echoes strongly with other forms of cultural narratives, such as fairytales, myth and religion. In this essay we present varied examples that illustrate how these analogies have guided not only readings of the AI enterprise by commentators outside the community but also inspired AI researchers themselves. Owing to their influence, we pay particular attention to the similarities (...) between religious language and the way in which the potential advent of greater than human intelligence is presented contemporarily. We then move on to the role that fiction, science fiction most of all, has historically played and is still playing in the discussion of AI by influencing researchers and the public, shifting the weights of different scenarios in our collectively perceived probability space. We sum up by arguing that the lore surrounding AI research, ancient and modern, points to the ancestral and shared human motivations that drive researchers in their pursuit and fascinate humanity at large. These points of narrative entanglement where AI meets the wider culture should serve to amplify the call to engage ourselves with the discussion of the potential destination of this technology. (shrink)