ABSTRACTIn Critical Discourse Studies and in other linguistics oriented scholarly journals we now see more research which draws upon multimodality as part of carrying out analyses of how text...

Intersemiotic translation (IT) was defined by Roman Jakobson (The Translation Studies Reader, Routledge, London, p. 114, 2000) as “transmutation of signs”—“an interpretation of verbal signs by means of signs of nonverbal sign systems.” Despite its theoretical relevance, and in spite of the frequency in which it is practiced, the phenomenon remains virtually unexplored in terms of conceptual modeling, especially from a semiotic perspective. Our approach is based on two premises: (i) IT is fundamentally a semiotic operation process (semiosis) and (ii) (...) IT is a deeply iconic-dependent process. We exemplify our approach by means of literature to dance IT and we explore some implications for the development of a general model of IT. (shrink)

Name der Zeitschrift: Semiotica Jahrgang: 2015 Heft: 207 Seiten: 583-606.

The first lecture will present pictorial semiotics within the framework of general semiotic theory. It will construe semiotics as a particular point of view taken on everything which is human or, more generally, endowed with life, rather than simply the continuation of the mixed or separate doctrines due to Saussure and Peirce. The historical part will describe briefly the development of pictorial semiotics and the peculiarities of its different schools and traditions, following upon the somewhat premature founding gesture attributed to (...) Barthes. (shrink)

In this paper I chart the seismic shift that has occurred over the past three decades in attitudes towards the interpretation of visual images. My strategy implies the argument that the reading of visual images would appear to be an inevitability given the accelerating change of attitudes towards pictures as containers of determinate knowledge. French critical theorists (Foucault, Barthes, Derrida et. al.) dominated debate on interpretation of text and image in the 1980s, where my survey begins. Michel Foucault dismissed the (...) image (in Madness and Civilization 1959/1988) as a fascinating site for the madness of dreams but one standing outside of reasoned interpretation because of an inherent excess of meaning and deeply hidden attributes and allusions. Generally, however, when images were discussed using identifiable interpretive strategies in the 1980s the framework was a variant of semiotic analysis (Marin, Eco and Barthes – who famously, diverges from this mode in Camera Lucida). I use W. J. T. Mitchell’s Iconology: Image, Text, Ideology 1986 as a focal text from the 1980s and follow this with two of his publications on images, one from 1995 and another from 2005 to demonstrate the dramatic shift in the approach to pictures across almost three decades. Around these focal texts by Mitchell I reference other texts and trends that culminate in the more recent proliferation of texts related to visual studies and the re-emergence of aesthetics. (shrink)

Critical discourse analysts are increasingly required to account for multimodal phenomena constructed through language and other resources and to relate high-level critical insights on the social motivations of these texts to their realizations in low-level expressive phenomena, and vice versa. In this paper, we use interactive software resources for critical multimodal discourse analysis. Multimodal analysis and digital technology. In A. Baldry & E. Montagna, Interdisciplinary approaches to multimodality: Theory and practice. Readings in intersemiosis and multimedia. Campobasso: Palladino; O'Halloran, K. L., (...) Tan, S., Smith, B. A., & Podlasov, A.. Challenges in designing digital interfaces for the study of multimodal phenomena. Information Design Journal, 18, 2–12; Smith, B. A., Tan, S., Podlasov, A., & O'Halloran, K.L.. Analyzing multimodality in an interactive digital environment: Software as metasemiotic tool. Social Semiotics) to help in achieving these aims. The field of critical discourse studies, itself being interdisciplinary, along with its holistic approach, is well placed to take advantage of interactive software, as these resources encourage the application, exploration and correlation of various analytical perspectives at different levels of description. Drawing on the analysis of a short video advertisement, we demonstrate how the interactive digital environment is one in which the discourse analyst can effectively draw upon different traditions of analysis, including ‘mainstream’ and social semiotic traditions, as well as other traditions such as media studies, to interpret dynamic audiovisual media texts in a critically self-reflexive manner. (shrink)

This article, taking a social semiotic approach, analyses two pieces of music written, shared and exalted by two pre-1945 European fascist movements – the German NSDAP and the British Union of Fascists. These movements, both political and cultural, employed mythologies of unity, common identity and purpose in order to elide the realities of social distinction and political–economic inequalities between bourgeois and proletarian groups in capitalist societies. Visually and inter-personally, the fascist cultural project communicated a machine-like certainty about a vision for (...) a new society based on discipline, conformity and the might of the nation. In this article, we are interested in the ways that these very same discourses are also communicated through sound and music in two songs: The Horst Wessel Lied and the BUF marching song, two songs that used the same melody. We analyse the discourses communicated by the semiotic choices made in melody, arrangements, sound qualities, rhythms as well as in lyrics. The article first identifies some of the underlying semiotic resources for meaning making in sound and then shows how these are used in order to communicate specific ideas, values and attitudes. (shrink)

This study takes a systemic functional multimodal social semiotic approach to the analysis and discussion of image and text relations in two sets of data. First, patterns of contextualisation of images and text in the online magazines Dabiq and Rumiyah produced by the Islamic extremist organisation which refers to itself as Islamic State are examined. The second data set consists of a sample of texts from Western online news and blog sites which include recontextualisations of images found in the first (...) data set. A sample of examples of the use and re-use of images is discussed in order to identify patterns of similarity and difference when images and text are recontextualised. It is argued that the ISIS material tends to foreground the interpersonal metafunction in combination with the textual metafunction, while the other data set tends to foreground the ideational metafunction. These inferences indicate that further exploration of a larger data set is worth pursuing. Such studies would provide deeper insights helping to distinguish between online material which supports terrorism and that which opposes it, as well as facilitating the further development of multimodal social semiotic approaches to image and text relations. (shrink)

Animation is widely acknowledged for dynamically visualizing information and has been increasingly used in educational context. However, the growing presence of educational animation has not been accompanied by well-informed studies that focus on the semiotic features of animation. An emerging perspective influenced by Social Semiotics and Systemic Functional Linguistics greatly complements the current trend of animation studies in the field of science education. Studies taking that perspective model animation as stratified systems (consisting of an expression plane and a content plane) (...) of meaning-making options. Building on their work, this paper further develops the systems of animation on the two planes. Besides the concept of stratification, this paper also introduces a metafunctional perspective for the modeling of animation, focusing on building the ideational meaning system and textual meaning system for animation on the content plane based on an examination of online science animations. It is found that educational animation is not homogenous: animation can construe different human experience and can be used to fulfill different functions. The paper closes with a discussion of the advantages of adopting a stratified metafunctional model in studying animation and the implications for future studies in the field of multimodal semiotics and science education. (shrink)

Whilst the faculties of literacy and numeracy are rightly recognised as worthy of pedagogical nurturing, this article champions a more venerable articulacy – visualcy – crucial to a healthy culture...

Volume 16, Issue 5, November 2019, Page 514-521.

ABSTRACTThis study employs a multimodal social semiotic approach to the analysis of text and image relations in material produced by the violent extremist organisation known as Islamic State in Iraq and al-Sham. The study focuses on iconisation, where meanings are condensed and interpersonally charged through ‘bonding icons’ which embody the organisation's world view and values. A sample of issues in the online magazine Dabiq produced by ISIS are analysed using Systemic Functional Multimodal Discourse Analysis. The ISIS world view is shown (...) to be internally cohesive, based on a narrow, fundamentalist interpretation of selected Islamic scripture and violently opposed to any other world view. To represent, synthesise and justify its values and world view, ISIS uses bonding icons constructed from combinations of artefacts, supported by references to selected Islamic scripture. The study is a prelude to more detailed investigation of bonding icons in materials produced by ISIS, the attraction of SIS to potential mujahideen and ‘copy-cat’ jihadist groups and the recontextualisation of materials from Dabiq in different media platforms. Such studies would provide deeper insights into the workings of organisations such as ISIS, and facilitate the further development of multimodal social semiotic approaches to image and text relations. (shrink)

In this article I consider a two-page autobiographical recount which appears at the end of Nelson Mandela's book Long Walk to Freedom as a summary of his life and what he has learned from it. My aim is to illustrate the role of a detailed analysis of single texts in the field of discourse analysis, as opposed to studies of selected variables across a corpus of texts. The analysis is conducted within the general theoretical framework of systemic functional linguistics, with (...) special attention to transitivity, mood, theme, grammatical metaphor, lexical relations, conjunction, tense, phase, process type, hierarchy of periodicity, polarity, continuity, elaboration, extension and the analysis of images in multimodal text. Through these procedures I show the way in which Mandela reconciles the linear unfolding of his life history with the deepening understanding of freedom that gives meaning to his life - by means of a spiral texture which returns again and again to the meaning of freedom at different levels of abstraction. The effect, I think, is inspirational - with no tinge of bitterness or betrayal; rather a message of hope and wisdom - grace personified. The approach exemplifies a positive style of discourse analysis that focuses on hope and change, by way of complementing the deconstructive exposé associated with critical discourse analysis. (shrink)