The success of deep learning in natural language processing raises intriguing questions about the nature of linguistic meaning and ways in which it can be processed by natural and artificial systems. One such question has to do with subword segmentation algorithms widely employed in language modeling, machine translation, and other tasks since 2016. These algorithms often cut words into semantically opaque pieces, such as ‘period’, ‘on’, ‘t’, and ‘ist’ in ‘period|on|t|ist’. The system then represents the resulting segments in a dense (...) vector space, which is expected to model grammatical relations among them. This representation may in turn be used to map ‘period|on|t|ist’ (English) to ‘par|od|ont|iste’ (French). Thus, instead of being modeled at the lexical level, translation is reformulated more generally as the task of learning the best bilingual mapping between the sequences of subword segments of two languages; and sometimes even between pure character sequences: ‘p|e|r|i|o|d|o|n|t|i|s|t’ → ‘p|a|r|o|d|o|n|t|i|s|t|e’. Such subword segmentations and alignments are at work in highly efficient end-to-end machine translation systems, despite their allegedly opaque nature. The computational value of such processes is unquestionable. But do they have any linguistic or philosophical plausibility? I attempt to cast light on this question by reviewing the relevant details of the subword segmentation algorithms and by relating them to important philosophical and linguistic debates, in the spirit of making artificial intelligence more transparent and explainable. (shrink)

The Principle of Semantic Compositionality (sometimes called Frege''s Principle) is the principle that the meaning of a (syntactically complex) whole is a function only of the meanings of its (syntactic) parts together with the manner in which these parts were combined. This principle has been extremely influential throughout the history of formal semantics; it has had a tremendous impact upon modern linguistics ever since Montague Grammars became known; and it has more recently shown up as a guiding principle for a (...) certain direction in cognitive science.Despite the fact that The Principle is vague or underspecified at a number of points — such as what meaning is, what counts as a part, what counts as a syntactic complex, what counts as combination — this has not stopped some people from viewing The Principle as obviously true, true almost by definition. And it has not stopped other people from viewing The Principle as false, almost pernicious in its effect. And some of these latter theorists think that it is an empirically false principle while others think of it as a methodologically wrong-headed way to proceed. (shrink)

The goal of inquiry in this essay is to ascertain to what extent the Principle of Compositionality – the thesis that the meaning of a complex expression is determined by the meaning of its parts and its mode of composition – can be justifiably imposed as a constraint on semantic theories, and thereby provide information about what meanings are. Apart from the introduction and the concluding chapter the thesis is divided into five chapters addressing different questions pertaining to the overarching (...) goal of inquiry. Chapter Two is an attempt to determine whether the Principle of Compositionality is a trivial principle. It is argued that this is not the case in the context of providing the semantics of natural languages since it is an open question whether the best theories that respect available syntactic and semantic data are compositional. Chapter Three and Chapter Four ask whether there are reasons to think that the Principle of Compositionality is true. It is argued that the three most commonly cited reasons for thinking that correct semantic theories must be compositional – Linguistic Creativity, Productivity and Systematicity – in fact only give us reasons to think that the meaning of complex expressions depend on the meanings of their parts and their modes of composition, but not that the meaning of complex expressions depend ONLY on those factors. Chapter Five asks whether there are any reasons to think that the Principle of Compositionality is false. It is argued that all the phenomena that have been suggested entail non-compositionality are in fact compatible with some versions of compositionality. Even if this conclusion is reached in this chapter, the conclusion of the previous two chapters entails that we are not justified in imposing the Principle of Compositionality as an adequacy constraint on Semantic theories. Chapter Six explores the ways in which information about what meanings are can be derived by imposing constraints on semantic theories. It is argued that the distinction emphasized in Chapter Three and Chapter Four between depending on and depending ONLY on is of critical importance. It is demonstrated that the informative potential of the Principle of Compositionality goes far beyond that of the principle we are in fact justified in imposing on semantic theories. So not only is it the case that we cannot learn anything about meanings from the justified imposition of the Principle of Compositionality – since we cannot justifiably impose it as a constraint on semantic theories – what we could have learned from it had we been justified in imposing it is very different from what we can learn from the principle that we are justified in imposing. (shrink)

The standard approach to model how human beings understand natural languages is the symbolic, compositional approach according to which the meaning of a complex expression is a function of the meanings of its constituents. In other words, meaning plays a fundamental role in the model. In this work, because of the polysemous, flexible, dynamic, and contextual structure of natural languages, this approach is rejected. Instead, a connectionist model which eliminates the concept of meaning is proposed.