Semantics traditionally focuses on linguistic meaning. In recent years, the Super Linguistics movement has tried to broaden the scope of inquiry in various directions, including an extension of semantics to talk about the meaning of pictures. There are close similarities between the interpretation of language and of pictures. Most fundamentally, pictures, like utterances, can be either true or false of a given state of affairs, and hence both express propositions (Zimmermann, 2016; Greenberg, 2013; Abusch, 2015). Moreover, sequences of pictures, like (...) sequences of utterances, can be used to tell stories. Wordless picture books, comics, and film are cases in point. In this paper I pick up the project of providing a dynamic semantic account of pictorial story-telling, started by Abusch (2012) and continued by Abusch & Rooth (2017); Maier & Bimpikou (2019); Fernando (2020). More specifically, I propose here a semantics of speech and thought bubbles by adding event reference to PicDRT. To get there I first review the projection-based semantics for pictures (section 1), noting the fundamental distinction between symbolic and iconic meaning that makes speech bubbles especially interesting (section 2). I then review the dynamic PicDRT framework for pictorial narratives (section 3), add events (section 4), and propose an account of speech bubbles as quotational event modification (section 5). I end with a brief look at other conventional, symbolic enrichments in comics (section 6). (shrink)

This chapter contains section titled: Notes.

Sometimes form-meaning mappings in language are not arbitrary, but iconic: they depict what they represent. Incorporating iconic elements of language into a compositional semantics faces a number of challenges in formal frameworks as evidenced by the lengthy literature in linguistics and philosophy on quotation/direct speech, which iconically portrays the words of another in the form that they were used. This paper compares the well-studied type of iconicity found with verbs of quotation with another form of iconicity common in sign languages: (...) classifier predicates. I argue that these two types of verbal iconicity can, and should, incorporate their iconic elements in the same way using event modification via the notion of a context dependent demonstration. This unified formal account of quotation and classifier predicates predicts that a language might use the same strategy for conveying both, and I argue that this is the case with role shift in American Sign Language. Role shift is used to report others’ language and thoughts as well as their actions, and recently has been argued to provide evidence in favor of Kaplanian “monstrous” indexical expressions. By reimagining role shift as involving either quotation for language demonstrations or “body classifier” predicates for action demonstrations, the proposed account eliminates one major argument for these monsters coming from sign languages. Throughout this paper, sign languages provide a fruitful perspective for studying quotation and other forms of iconicity in natural language due to their lack of a commonly used writing system which is otherwise often mistaken as primary data instead of speech, the rich existing literature on iconicity within sign language linguistics, and the ability of role shift to overtly mark the scope of a language report. In this view, written language is merely a special case of a more general phenomenon of sign and speech demonstration, which accounts more accurately for natural language data by permitting more strict or loose verbatim interpretations of demonstrations through the context dependent pragmatics. (shrink)

Visual narratives like comics involve a range of complex cognitive operations in order to be understood. The Parallel Interfacing Narrative‐Semantics (PINS) Model integrates an emerging literature showing that comprehension of wordless image sequences balances two representational levels of semantic and narrative structure. The neurocognitive mechanisms that guide these processes are argued to overlap with other domains, such as language and music.

Noam Chomsky's book on syntactic structures is a serious attempts on the part of a linguist to construct within the tradition of scientific theory-construction ...

In Rules and Representations, first published in 1980, Noam Chomsky lays out many of the concepts that have made his approach to linguistics and human cognition so instrumental to our understanding of language.Chomsky arrives at his well-known position that there is a universal grammar, structured in the human mind and common to all human languages. Based on Chomsky's 1978 Woodbridge Lectures, this edition contains revised versions of the lectures and two new essays.

The book from which these sections are excerpted is concerned with the prospects for assimilating the study of human intelligence and its products to the natural sciences through the investigation of cognitive structures, understood as systems of rules and representations that can be regarded as “mental organs.” These mental structui′es serve as the vehicles for the exercise of various capacities. They develop in the mind on the basis of an innate endowment that permits the growth of rich and highly articulated (...) structures along an intrinsically determined course under the triggering and partially shaping effect of experience, which fixes parameters in an intricate system of predetermined form. It is argued that the mind is modular in character, with a diversity of cognitive structures, each with its specific properties arid principles. Knowledge of language, of the behavior of objects, and much else crucially involves these mental structures, and is thus not characterizable in terms of capacities, dispositions, or practical abilities, nor is it necessarily grounded in experience in the standard sense of this term.Various types of knowledge and modes of knowledge acquisition are discussed in these terms. Some of the properties of the language faculty are investigated. The basic cognitive relation is “knowing a grammar”; knowledge of language is derivative and, correspondingly, raises further problems. Language as commonly understood is not a unitary phenomenon but involves a number of interacting systems: the “computational” system of grammar, which provides the representations of sound and meaning that permit a rich range of expressive potential, is distinct from a conceptual system with its own properties; knowledge of language must be distinguished from knowledge of how to use a language; and the various systems that enter into the knowledge and use of language must be further analyzed into their specific subcomponents. (shrink)

The central idea behind this paper is that presuppositions of soft triggers arise from the way our attention structures the informational content of a sentence. Some aspects of the information conveyed are such that we pay attention to them by default, even in the absence of contextual information. On the other hand, contextual cues or conversational goals can divert attention to types of information that we would not pay attention to by default. Either way, whatever we do not pay attention (...) to, be it by default, or in context, is what ends up presupposed by soft triggers. This paper attempts to predict what information in the sentence is likely to end up being the main point (i.e. what we pay attention to) and what information is independent from this, and therefore likely presupposed. It is proposed that this can be calculated by making reference to event times. The notion of aboutness used to calculate independence is based on that of Demolombe and Fariñas del Cerro (In: Holdobler S (ed) Intellectics and computational logic: papers in honor of Wolfgang Bibel, 2000). (shrink)

This book contributes to the analysis of film from a multimodal and textual perspective by extending formal semantics into the realm of multimodal discourse analysis. It accounts for both the inferential as well as intersemiotic meaning making processes in filmic discourse and therefore addresses one of the main questions that have been asked within film theory and multimodal analysis: How do we understand film and multimodal texts? The book offers an analytical answer to this question by providing a systematic tool (...) for the description of this comprehension process. It aims to advance knowledge of the various resources in filmic texts, the ways the resources work together in constructing meaning and the ways people understand this meaning construction. This new approach to film interpretation is thus able to remodel and improve the classical paradigm of film text analysis. (shrink)

We propose an extension of Discourse Respresentation Theory (DRT) for analyzing pictorial narratives. We test drive our PicDRT framework by analyzing the way authors represent characters’ mental states and perception in comics. Our investigation goes beyond Abusch and Rooth (2017) in handling not just free perception sequences, but also a form of apparent perspective blending somewhat reminiscent of free indirect discourse.

Why are diagrams sometimes so useful, while other times unhelpful and even misguiding? There are systematic reasons for this. Drawing on modern research in logic, Artificial Intelligence, cognitive psychology, and graphic design, "Semantic Properties of Diagrams and their Cognitive Potentials" shows that diagrams' cognitive functions are rooted in the characteristic ways they carry information about their targets. The analysis leads to an answer for the deeper question of What makes a diagram a diagram?, which is of crucial importance to the (...) foundation of a collective science of diagrams. ". (shrink)

Paul M. Pietroski presents an ambitious new account of human languages as generative procedures that respect substantive constraints. He argues that meanings are neither concepts nor extensions, and sentences do not have truth conditions; meanings are composable instructions for how to access and assemble concepts of a special sort.

Analyzes the meaning expressed in music, the social and psychological sources of meaning, and the methods of musical communication This is a book meant for ...

An inclusive survey of linguistic semantics, written by prominent experts in the field The Wiley Blackwell Companion to Semantics provides a thorough investigation of issues and phenomena central to the development of modern semantics and its interfaces. Presenting in-depth chapters written by leading experts in the field, this book investigates competing analyses and approaches, examines their conceptual foundations, and evaluates them as applied to various languages. This authoritative collection enables scholars and students of semantics—as well as those from associated areas (...) of linguistics such as syntacticians—to broaden and deepen their knowledge of the subject and the developments in the field. Presenting over 100 case studies, the collection of chapters within this Companion is organized in alphabetical order for ease of reference. This key reference work: Provides detailed coverage of the major developments in linguistic semantics over the past several decades Demonstrates how research can identify differences and similarities in a variety of languages Presents studies that encompass well-delimited empirical areas and play important roles in theoretical debates Identifies topics via famous example sentences The Wiley Blackwell Companion to Semantics is a valuable reference work for scholars, researchers, academics, and students in linguistics and related areas. (shrink)

One effect of information technology is the increasing need to present information visually. The trend raises intriguing questions. What is the logical status of reasoning that employs visualization? What are the cognitive advantages and pitfalls of this reasoning? What kinds of tools can be developed to aid in the use of visual representation? This newest volume on the Studies in Logic and Computation series addresses the logical aspects of the visualization of information. The authors of these specially commissioned papers explore (...) the properties of diagrams, charts, and maps, and their use in problem solving and teaching basic reasoning skills. As computers make visual representations more commonplace, it is important for professionals, researchers and students in computer science, philosophy, and logic to develop an understanding of these tools; this book can clarify the relationship between visuals and information. (shrink)

Because humans cannot know one another’s minds directly, every form of communication is a solution to the same basic problem: how can privately held information be made publicly accessible through manipulations of the physical environment? Language is by far the best studied response to this challenge. But there are a diversity of non-linguistic strategies for representation with external signs as well, from facial expressions and fog horns to chronological graphs and architectural renderings. The general thesis of this dissertation is that (...) there is an impressively wide spectrum of conventional systems of representation, corresponding to the many ways that the problem of communication can be solved, and that these systems can be described and explained using the tools of contemporary mathematical semantics. As a partial corrective to the countervailing norm, this work concentrates on the class of systems arguably most different from language— those governing the interpretation of pictorial images. Such representations dominate practical communication: witness the proliferation of maps, road signs, newspaper photographs, scientific illustrations, television shows, engineering drawings, and even the fleeting imagery of manual gesture. I argue that systems of depiction and languages embody a parallel technologies of communication. Both are based on semantics: systematic and conventional mappings from signs to representational content. But I also provide evidence that these semantics are profoundly divergent. Whereas the semantics of languages are based on arbitrary associations of signs and denotations, the semantics of systems of depiction are based on rules of geometrical transformation. Drawing on recent research in computer graphics and computational vision, I go on to develop a precise theory of pictorial semantics. This in turn facilitates a detailed comparison of iconic, image-based representation, and symbolic, language-based representation. A consequence of these conclusions is that the traditional, language-centric conception of semantics must be overhauled to allow for a more general semantic theory, one which countenances the wide variety of interpretive mechanisms actually at work in human communication. (shrink)

We argue that a formal semantics for music can be developed, although it will be based on very different principles from linguistic semantics and will yield less precise inferences. Our framework has the following tenets: Music cognition is continuous with normal auditory cognition. In both cases, the semantic content derived from an auditory percept can be identified with the set of inferences it licenses on its causal sources, analyzed in appropriately abstract ways. What is special about music semantics is that (...) it aggregates inferences based on normal auditory cognition with further inferences drawn on the basis of the behavior of voices in tonal pitch space. This makes it possible to define an inferential semantics but also a truth-conditional semantics for music. In particular, a voice undergoing a musical movement m is true of an object undergoing a series of events e just in case there is a certain structure-preserving map between m and e. Aspects of musical syntax might be derivable on semantic grounds from an event mereology, which also explains some cases in which tree structures are inadequate for music. Intentions and emotions may be attributed at several levels, and we speculate on possible explanations of the special relation between music and emotions. (shrink)

As part of a recent attempt to extend the methods of formal semantics beyond language, it has been claimed that music has an abstract truth-conditional semantics, albeit one that has more in common with iconic semantics than with standard compositional semantics. After summarizing this approach and addressing a common objection, we argue that music semantics should be enriched in three directions by incorporating insights of other areas of Super Semantics. First, it has been claimed by Abusch 2013 that visual narratives (...) make use of discourse referents akin to those we find in language. We argue that a similar conclusion extends to music, and we highlight it by investigating ways in which orchestration and dance may make cross-referential dependencies more explicit. Second, we show that by bringing music semantics closer to the semantics of visual narratives, we can give an account of the semantics of mixed visual and musical sequences. Third, it has been claimed that co-speech gestures trigger characteristic conditionalized presuppositions, called ‘cosuppositions’, and that their semantic status derives from their parasitic character relative to words. We argue that the same conclusion extends to some instances of film and cartoon music: it may trigger cosuppositions that can be revealed by embedding film excerpts or gifs in sentences so as to test presupposition projection. We further argue that under special discourse conditions, pro-speech gestures and pro-speech music alike can trigger cosuppositions as well. These results suggest that new insights can be gained not just by extending the methods of semantics to new objects, but also by drawing new connections among them. (shrink)

In several sign languages, plurals can be realized with unpunctuated or punctuated repetitions of a noun, with different semantic implications; similar repetition-based plurals have been described in some homesigns and silent gestures. Unpunctuated repetitions often get approximate ‘at least’ readings while punctuated repetitions typically correspond to ‘exactly’ readings. The prevalence of these mechanisms could be thought to be a case in which Universal Grammar does not just specify the abstract properties of grammatical elements, but also their phonological realization, at least (...) in the visual modality. We explore an alternative in which punctuated and unpunctuated repetitions arise from general properties of iconic representations. On an empirical level, we argue that in ASL, punctuated and unpunctuated repetitions are unlikely to be an exclusively grammatical mechanism, as they can be found with purely iconic representations. On a theoretical level, we argue for a modular account with three components. First, repetition-based plurals can create a simplified pictorial representation. Second, unpunctuated repetitions give rise to pictorial vagueness, resolved by way of quantification over precisifications. Third, a pragmatic process involving strategic reasoning maps these vague representations onto a set of candidate linguistic meanings, including some ‘at least’ plural readings that are best expressed by unpunctuated repetitions. (shrink)

We argue that some sign language loci (i.e. positions in signing space that realize discourse referents) are both formal variables and simplified representations of what they denote; in other words, they are simultaneously logical symbols and pictorial representations. We develop a 'formal semantics with iconicity' that accounts for their dual life; the key idea ('formal iconicity') is that some geometric properties of signs must be preserved by the interpretation function. We analyze in these terms three kinds of iconic effects in (...) American and French Sign Language (ASL and LSF): (i) structural iconicity, where relations of inclusion and complementation among loci are directly reflected in their denotations; (ii) locus-external iconicity, where the high or low position of a locus in signing space has a direct semantic reflex, akin to the semantic contribution of gender features of pronouns; and (iii) locus-internal iconicity, where different parts of a structured locus are targeted by different directional verbs, as was argued by Liddell and Kegl. The resulting semantics combines insights of two traditions that have been sharply divided by recent debates. In line with the 'formalist camp' (e.g. Lillo-Martin and Klima, Neidle, and Sandler and Lillo-Martin), our theory treats loci as variables, and develops an explicit formal analysis of their behavior. But we also incorporate insights of the 'iconic camp', which emphasized the role of iconic constraints in sign language in general and in pronominals in particular (e.g. Cuxac, Taub, Liddell). However, this synthesis is only possible if formal semantics makes provisions for iconic requirements at the very core of its interpretive procedure. (An Appendix discusses relevant data from Italian Sign Language [LIS].). (shrink)

ASL can express plurals by repeating a noun, in an unpunctuated fashion, in different parts of signing space. We argue that this construction may come with a rich iconic component: the geometric arrangement of the repetitions provides information about the arrangement of the denoted plurality; in addition, the number and speed of the repetitions provide information about the size of the denoted plurality. Interestingly, the shape of the repetitions may introduce a new singular discourse referent when a vertex can be (...) inferred to denote a singular object. Thus one may point towards the first or last iteration of a horizontal repetition of BOOK to denote the left- or right-edge of the corresponding row. This yields a remarkable interaction between iconic semantics and standard logical semantics. We show that our analysis extends to ‘punctuated’ repetitions, which involve clearly individuated iterations of a singular noun. While these may initially look like coordinated indefinites, they are better handled by the same iconic framework as plural, unpunctuated repetitions. Some repetition-based mass terms also give rise to iconic effects, and to different readings depending on whether the repetition is continuous, unpunctuated, or punctuated. Our analysis highlights the need for a formal semantics with iconicity to study the integration of such iconic and logical conditions. It also raises a question: can similar facts be found in spoken language when gestures are taken into account? We suggest that several effects can be replicated, especially when one considers examples involving ‘pro-speech gestures’. (shrink)

Sign languages are known to display the same general grammatical properties as spoken languages, but also to make greater use of iconic mechanisms. In Schlenker et al.’s ‘Iconic Variables’ :91–149, 2013), it was argued that loci can have an iconic semantics, in the sense that certain geometric relations among loci are preserved by the interpretation function. Here we ask whether plural and height specifications of loci display the formal behavior of phi-features in remaining uninterpreted in focus- and ellipsis-constructions. Data from (...) ASL and LSF show that plural and height specifications may indeed remain uninterpreted in these constructions; furthermore, there are cases in which a single high locus is construed iconically and left uninterpreted in the course of ellipsis resolution. We argue that our data are compatible with two theories. According to the Strong View, plural and height specifications of loci display exactly the behavior of spoken language features. According to the Weak View, our data just show that plural and height specifications share the behavior of features and other non-assertive elements in being separable from the referential terms they specify. Our LSF data are compatible with the Weak View; our ASL data might provide support for the Strong View. While our aim is merely to open the debate about the featural status of iconic specifications, the question is of some importance: if features are innate and primitive elements of grammar, and if some of them have an intrinsically geometric semantics, the signed modality might play a greater role than is usually thought at the very core of Universal Grammar. (shrink)

In dynamic theories of presupposition, a trigger pp′ with presupposition p and at-issue component p′ comes with a requirement that p should be entailed by the local context of pp′. We argue that some co-speech gestures should be analyzed within a presuppositional framework, but with a twist: an expression p co-occurring with a co-speech gesture G with content g comes with the requirement that the local context of p should guarantee that p entails g; we call such assertion-dependent presuppositions ‘cosuppositions’. (...) We show that this analysis can be combined with earlier theories of local contexts to account for complex patterns of gesture projection in quantified and in attitudinal contexts, and we compare our account to two potential alternatives: one based on supervaluations, and one, due to Cornelia Ebert, that treats co-speech gestures as supplements. We argue that the latter is correct, but for ‘post-speech’ gestures, rather than for co-speech gestures. (shrink)

We provide a new link between deductive and probabilistic reasoning fallacies. Illusory inferences from disjunction are a broad class of deductive fallacies traditionally explained by recourse to a matching procedure that looks for content overlap between premises. In two behavioral experiments, we show that this phenomenon is instead sensitive to real-world causal dependencies and not to exact content overlap. A group of participants rated the strength of the causal dependence between pairs of sentences. This measure is a near perfect predictor (...) of fallacious reasoning by an independent group of participants in illusory inference tasks with the same materials. In light of these results, we argue that all extant accounts of these deductive fallacies require non-trivial adjustments. Crucially, these novel _indirect_ illusory inferences from disjunction bear a structural similarity to seemingly unrelated _probabilistic reasoning_ problems, in particular the conjunction fallacy from the heuristics and biases literature. This structural connection was entirely obscure in previous work on these deductive problems, due to the theoretical and empirical focus on content overlap. We argue that this structural parallelism provides arguments against the need for rich descriptions and individuating information in the conjunction fallacy, and we outline a unified theory of deductive illusory inferences from disjunction and the conjunction fallacy, in terms of Bayesian confirmation theory. (shrink)

Language has a rich typology of inferential types. It was recently shown that subjects are able to divide the informational content of new visual stimuli among the various slots of the inferential typology: when gestures or visual animations are used in lieu of specific words in a sentence, they can trigger the very same inferential types as language alone (Tieu et al., 2019 ). How general are the relevant triggering algorithms? We show that they extend to the auditory modality and (...) to music cognition. We tested whether pro-speech musical gestures, i.e. musical excerpts that replace words in sentences, can give rise to the same inferences. We show that it is possible to replicate the same typology of inferences using pro-speech music. Minimal and complex musical excerpts can behave just like language, gestures, and visual animations with respect to the logical behavior of their content when embedded in sentences. Specifically, we found that pro-speech music can generate scalar implicatures, presuppositions, supplements, and homogeneity inferences. (shrink)

The psychological study of reasoning with quantifiers has predominantly focused on inference patterns studied by Aristotle about two millennia ago. Modern logic has shown a wealth of inference patterns involving quantifiers that are far beyond the expressive power of Aristotelian syllogisms, and whose psychology should be explored. We bring to light a novel class of fallacious inference patterns, some of which are so attractive that they are tantamount to cognitive illusions. In tandem with recent insights from linguistics that quantifiers like (...) “some” are treated as wh-questions, these illusory inferences are predicted by the erotetic theory of reasoning, which postulates that a process akin to question asking and answering is behind human inference making. (shrink)

This paper reviews evidence concerning the nature of grouping in music and language and their interactions with other linguistic and musical systems. I present brief typological surveys of the relationship between constituency and acoustic parameters in language and music, drawing from a wide variety of languages and musical genres. The two domains both involve correspondence between auditory discontinuities and group boundaries, reflecting the Gestalt principles of proximity and similarity, as well as a nested, hierarchical organization of constituents. Typically, computational-level theories (...) of musical grouping take the form of a function from acoustic properties through grouping representations to syntactic or interpretive constituents. Linguistic theories tend to be cast as functions in the opposite direction. This study argues that the difference in orientation is not grounded in principled differences in information flow between the two domains, and that reconceptualizing one or both theories allows for gains in analytical understanding. There are also obvious differences between musical and linguistic grouping. Grappling with those differences requires one to think in detail about modularity, information flow, levels of description, and the functional nature of cognitive domains. (shrink)

Preface This book is about semantics and logic. More specifically, it is about the semantics and logic of natural language; and, even more specifically than ...

The Montagovian hypothesis of direct model-theoretic interpretation of syntactic surface structures is supported by an account of the semantics of binding that makes no use of variables, syntactic indices, or assignment functions & shows that the interpretation of a large portion of so-called variable-binding phenomena can dispense with the level of logical form without incurring equivalent complexity elsewhere in the system. Variable-free semantics hypothesizes local interpretation of each surface constituent; binding is formalized as a type-shifting operation on expressions that denote (...) functions, & sentences containing a free pronoun are analyzed as a function from individuals to propositions having a meaning of type (e,t). Standard weak crossover effects & binding patterns in sentences with multiple pronouns are shown to submit to straightforward type-theoretic treatments that do not rely on indexation. The variable-free semantics smoothly implements full surface compositionality & requires less machinery than standard accounts to handle functional questions, their answers, sloppy inferences, & across-the-board binding. 73 References. J. Hitchcock. (shrink)

A semantics of pictorial representation should provide an account of how pictorial signs are associated with the contents they express. Unlike the familiar semantics of spoken languages, this problem has a distinctively spatial cast for depiction. Pictures themselves are two-dimensional artifacts, and their contents take the form of pictorial spaces, perspectival arrangements of objects and properties in three dimensions. A basic challenge is to explain how pictures are associated with the particular pictorial spaces they express. Inspiration here comes from recent (...) proposals that analyze depiction in terms of geometrical projection. In this essay, I will argue that, for a central class of pictures, the projection-based theory of depiction provides the best explanation for how pictures express pictorial spaces, while rival perceptual and resemblance theories fall short. Since the composition of pictorial space is itself the basis for all other aspects of pictorial content, the proposal provides a natural foundation for further pictorial semantics. (shrink)