I argue that emojis are essentially little pictures, rather than words, gestures, expressives, or diagrams. ???? means that the world looks like that, from some viewpoint. I flesh out a pictorial semantics in terms of geometric projection with abstraction and stylization. Since such a semantics delivers only very minimal contents I add an account of pragmatic enrichment, driven by coherence and nonliteral interpretation. The apparent semantic distinction between emojis depicting entities (like ????) and those depicting facial expressions (like ????) I (...) analyze as a difference between truth-conditional and use-conditional pictorial content: ???? depicts what the world of evaluation looks like, while ???? depicts what the utterance context looks like. (shrink)

It is commonly assumed that images, whether in the world or in the head, do not have a privileged analysis into constituent parts. They are thought to lack the sort of syntactic structure necessary for representing complex contents and entering into sophisticated patterns of inference. I reject this assumption. “Image grammars” are models in computer vision that articulate systematic principles governing the form and content of images. These models are empirically credible and can be construed as literal grammars for images. (...) Images can have rich syntactic structure, though of a markedly different form than sentences in language. (shrink)

As part of a recent attempt to extend the methods of formal semantics beyond language, it has been claimed that music has an abstract truth-conditional semantics, albeit one that has more in common with iconic semantics than with standard compositional semantics. After summarizing this approach and addressing a common objection, we argue that music semantics should be enriched in three directions by incorporating insights of other areas of Super Semantics. First, it has been claimed by Abusch 2013 that visual narratives (...) make use of discourse referents akin to those we find in language. We argue that a similar conclusion extends to music, and we highlight it by investigating ways in which orchestration and dance may make cross-referential dependencies more explicit. Second, we show that by bringing music semantics closer to the semantics of visual narratives, we can give an account of the semantics of mixed visual and musical sequences. Third, it has been claimed that co-speech gestures trigger characteristic conditionalized presuppositions, called ‘cosuppositions’, and that their semantic status derives from their parasitic character relative to words. We argue that the same conclusion extends to some instances of film and cartoon music: it may trigger cosuppositions that can be revealed by embedding film excerpts or gifs in sentences so as to test presupposition projection. We further argue that under special discourse conditions, pro-speech gestures and pro-speech music alike can trigger cosuppositions as well. These results suggest that new insights can be gained not just by extending the methods of semantics to new objects, but also by drawing new connections among them. (shrink)

In several sign languages, plurals can be realized with unpunctuated or punctuated repetitions of a noun, with different semantic implications; similar repetition-based plurals have been described in some homesigns and silent gestures. Unpunctuated repetitions often get approximate ‘at least’ readings while punctuated repetitions typically correspond to ‘exactly’ readings. The prevalence of these mechanisms could be thought to be a case in which Universal Grammar does not just specify the abstract properties of grammatical elements, but also their phonological realization, at least (...) in the visual modality. We explore an alternative in which punctuated and unpunctuated repetitions arise from general properties of iconic representations. On an empirical level, we argue that in ASL, punctuated and unpunctuated repetitions are unlikely to be an exclusively grammatical mechanism, as they can be found with purely iconic representations. On a theoretical level, we argue for a modular account with three components. First, repetition-based plurals can create a simplified pictorial representation. Second, unpunctuated repetitions give rise to pictorial vagueness, resolved by way of quantification over precisifications. Third, a pragmatic process involving strategic reasoning maps these vague representations onto a set of candidate linguistic meanings, including some ‘at least’ plural readings that are best expressed by unpunctuated repetitions. (shrink)

There is a widely held view that visual representations (images) do not depict negation, for example, as expressed by the sentence, “the train is not coming.” The present study focuses on the real-world visual representations of photographs and comic (manga) illustrations and empirically challenges the question of whether humans and machines, that is, modern deep neural networks, can recognize visual representations as expressing negation. By collecting data on the captions humans gave to images and analyzing the occurrences of negation phrases, (...) we show some evidence that humans recognize certain images as expressing negation. Furthermore, based on this finding, we examined whether or not humans and machines can classify novel images as expressing negation. The humans were able to correctly classify images to some extent, as expected from the analysis of the image captions. On the other hand, the machine learning model of image processing was only able to perform this classification at about the chance level, not at the same level of performance as the human. Based on these results, we discuss what makes humans capable of recognizing negation in visual representations, highlighting the role of the background commonsense knowledge that humans can exploit. Comparing human and machine learning performances suggests new ways to understand human cognitive abilities and to build artificial intelligence systems with more human-like abilities to understand logical concepts. (shrink)

We argue that formal linguistic theory, properly extended, can provide a unifying framework for diverse phenomena beyond traditional linguistic objects. We display applications to pictorial meanings, visual narratives, music, dance, animal communication, and, more abstractly, to logical and non-logical concepts in the ‘language of thought’ and reasoning. In many of these cases, a careful analysis reveals that classic linguistic notions are pervasive across these domains, such as for instance the constituency (or grouping) core principle of syntax, the use of logical (...) variables (for object tracking), or the variety of inference types investigated in semantics/pragmatics. The aim of this overview is to show how the application of formal linguistic concepts and methodology to non-linguistic objects yields non-trivial insights, thus opening the possibility of a general, precise theory of signs. (An appendix, found in the online supplements to this article, surveys applications of Super Linguistics to animal communication.). (shrink)

We argue that formal linguistic theory, properly extended, can provide a unifying framework for diverse phenomena beyond traditional linguistic objects. We display applications to pictorial meanings, visual narratives, music, dance, animal communication, and, more abstractly, to logical and non-logical concepts in the ‘language of thought’ and reasoning. In many of these cases, a careful analysis reveals that classic linguistic notions are pervasive across these domains, such as for instance the constituency (or grouping) core principle of syntax, the use of logical (...) variables (for object tracking), or the variety of inference types investigated in semantics/pragmatics. The aim of this overview is to show how the application of formal linguistic concepts and methodology to non-linguistic objects yields non-trivial insights, thus opening the possibility of a general, precise theory of signs. (An appendix, found in the online supplements to this article, surveys applications of Super Linguistics to animal communication.). (shrink)

This paper presents an analysis of face emoji (disc-shaped pictograms with stylized facial expressions) that accompany written text. We propose that there is a use of face emoji in which they comment on a target proposition expressed by the accompanying text, as opposed to making an independent contribution to discourse. Focusing on positively valenced and negatively valenced emoji (which we gloss as _happy_ and _unhappy_, respectively), we argue that the emoji comment on how the target proposition bears on a contextually (...) provided discourse value endorsed by the author. Discourse values embody what an author desires, aspires to, wishes for, or hopes for. Our analysis derives a range of non-trivial generalizations, including (i) ordering restrictions with regards to the placement of emoji and text, (ii) cases of apparent mixed emotions, and (iii) cases where the lexical content of the accompanying text influences the acceptability of a face emoji. (shrink)

We intuitively make a distinction between _lying_ and _misleading_. On the explanation of this phenomenon favored here—the _adverbial_ account—the distinction tracks whether the content and its truth-committing force are literally conveyed. On an alternative _commitment_ account, the difference between lying and misleading is predicated instead on the strength of assertoric commitment. One lies when one presents with full assertoric commitment what one believes to be false; one merely misleads when one presents it without full assertoric commitment, by merely hinting or (...) otherwise implying it. Now, as predicted by the well-supported assumption that we can also assert with pictures, the lying/misleading distinction appears to intuitively show up there too. Here I’ll explain how the debate confronting the two accounts plays out both in general and in that case, aiming to provide support for the adverbial account. (shrink)