This paper analyzes the rapid and unexpected rise of deep learning within Artificial Intelligence and its applications. It tackles the possible reasons for this remarkable success, providing candidate paths towards a satisfactory explanation of why it works so well, at least in some domains. A historical account is given for the ups and downs, which have characterized neural networks research and its evolution from “shallow” to “deep” learning architectures. A precise account of “success” is given, in order to sieve out (...) aspects pertaining to marketing or sociology of research, and the remaining aspects seem to certify a genuine value of deep learning, calling for explanation. The alleged two main propelling factors for deep learning, namely computing hardware performance and neuroscience findings, are scrutinized, and evaluated as relevant but insufficient for a comprehensive explanation. We review various attempts that have been made to provide mathematical foundations able to justify the efficiency of deep learning, and we deem this is the most promising road to follow, even if the current achievements are too scattered and relevant for very limited classes of deep neural models. The authors’ take is that most of what can explain the very nature of why deep learning works at all and even very well across so many domains of application is still to be understood and further research, which addresses the theoretical foundation of artificial learning, is still very much needed. (shrink)

Simple idealized models seem to provide more understanding than opaque, complex, and hyper-realistic models. However, an increasing number of scientists are going in the opposite direction by utilizing opaque machine learning models to make predictions and draw inferences, suggesting that scientists are opting for models that have less potential for understanding. Are scientists trading understanding for some other epistemic or pragmatic good when they choose a machine learning model? Or are the assumptions behind why minimal models provide understanding misguided? In (...) this paper, using the case of deep neural networks, I argue that it is not the complexity or black box nature of a model that limits how much understanding the model provides. Instead, it is a lack of scientific and empirical evidence supporting the link that connects a model to the target phenomenon that primarily prohibits understanding. (shrink)

Although machine learning has been successful in recent years and is increasingly being deployed in the sciences, enterprises or administrations, it has rarely been discussed in philosophy beyond the philosophy of mathematics and machine learning. The present contribution addresses the resulting lack of conceptual tools for an epistemological discussion of machine learning by conceiving of deep learning networks as ‘judging machines’ and using the Kantian analysis of judgments for specifying the type of judgment they are capable of. At the center (...) of the argument is the fact that the functionality of deep learning networks is established by training and cannot be explained and justified by reference to a predefined rule-based procedure. Instead, the computational process of a deep learning network is barely explainable and needs further justification, as is shown in reference to the current research literature. Thus, it requires a new form of justification, that is to be specified with the help of Kant’s epistemology. (shrink)

Already hailed as a masterpiece, Foundations of Language offers a brilliant overhaul of the last thirty-five years of research in generative linguistics and related fields. "Few books really deserve the cliché 'this should be read by every researcher in the field'," writes Steven Pinker, author of The Language Instinct, "but Ray Jackendoff's Foundations of Language does." Foundations of Language offers a radically new understanding of how language, the brain, and perception intermesh. The book renews the promise of early generative linguistics: (...) that language can be a valuable entrée into understanding the human mind and brain. The approach is remarkably interdisciplinary. Behind its innovations is Jackendoff's fundamental proposal that the creativity of language derives from multiple parallel generative systems linked by interface components. This shift in basic architecture makes possible a radical reconception of mental grammar and how it is learned. As a consequence, Jackendoff is able to reintegrate linguistics with philosophy of mind, cognitive and developmental psychology, evolutionary biology, neuroscience, and computational linguistics. Among the major topics treated are language processing, the relation of language to perception, the innateness of language, and the evolution of the language capacity, as well as more standard issues in linguistic theory such as the roles of syntax and the lexicon. In addition, Jackendoff offers a sophisticated theory of semantics that incorporates insights from philosophy of language, logic and formal semantics, lexical semantics of various stripes, cognitive grammar, psycholinguistic and neurolinguistic approaches, and the author's own conceptual semantics. (shrink)

Epistemic opacity vis a vis human agents has been presented as an essential, ineliminable characteristic of computer simulation models resulting from the characteristics of the human cognitive agent. This paper argues, on the contrary, that such epistemic opacity as does occur in computer simulations is not a consequence of human limitations but of a failure on the part of model developers to adopt good software engineering practice for managing human error and ensuring the software artefact is maintainable. One consequence of (...) such failures is to create a “technical debt” which manifests itself in the so-called novel confirmation holism confronted by complex systems models. The argument from the supposed essential epistemic opacity of computational science to a non-anthropocentric epistemology runs counter to best practice in software engineering and overlooks empirical results of software engineering science. (shrink)

This article argues that the much-maligned ‘notational variants’ of a given formal linguistic theory play a role similar to alternative numerical measurement scales. Thus, they can be used to identify the invariant components of the grammar; i.e., those features that do not depend on the choice of empirically equivalent representation. Treating these elements as the ‘meaningful’ structure of language has numerous consequences for the philosophy of science and linguistics. I offer several such examples of how linguistic theorizing can profit from (...) adopting a measurement-theoretic viewpoint. The first concerns a measurement-theoretic response to a famous criticism of Quine's. Others follow from issues of simplicity in the current biolinguistics program. An unexpected similarity with behaviorist practices is also uncovered. I then argue that managable and useful steps can be taken in this area. (shrink)

Philosophers of science increasingly recognize the importance of idealization: the intentional introduction of distortion into scientific theories. Yet this recognition has not yielded consensus about the nature of idealization. e literature of the past thirty years contains disparate characterizations and justifications, but little evidence of convergence towards a common position.

In this paper, we explore the extent to which issues of simulation model validation take on novel characteristics when the models in question become particularly complex. Our central claim is that complex simulation models in general, and global models of climate in particular, face a form of confirmation holism. This holism, moreover, makes analytic understanding of complex models of climate either extremely difficult or even impossible. We argue that this supports a position we call convergence skepticism: the belief that the (...) existence of a plurality of different models making a plurality of different forecasts of future climate is likely to be a persistent feature of global climate science. (shrink)

This is the first part of a two-part article on semantic compositionality, that is, the principle that the meaning of a complex expression is determined by the meanings of its parts and the way they are put together. Here we provide a brief historical background, a formal framework for syntax and semantics, precise definitions, and a survey of variants of compositionality. Stronger and weaker forms are distinguished, as well as generalized forms that cover extra-linguistic context dependence as well as linguistic (...) context dependence. In the second article, we survey arguments for and arguments against the claim that natural languages are compositional, and consider some problem cases. It will be referred to as Part II. (shrink)

The Generative Lexicon presents a novel and exciting theory of lexical semantics that addresses the problem of the "multiplicity of word meaning" - that is, how ...

Nevertheless, any competent speaker will know what it means. What explains our ability to understand sentences we have never before encountered? One natural hypothesis is that those novel sentences are built up out of familiar parts, put together in familiar ways. This hypothesis requires the backing hypothesis that English has a compositional semantic theory.

This book examines the hypothesis of "direct compositionality", which requires that semantic interpretation proceed in tandem with syntactic combination. Although associated with the dominant view in formal semantics of the 1970s and 1980s, the feasibility of direct compositionality remained unsettled, and more recently the discussion as to whether or not this view can be maintained has receded. The syntax-semantics interaction is now often seen as a process in which the syntax builds representations which, at the abstract level of logical form, (...) are sent for interpretation to the semantics component of the language faculty. In the first extended discussion of the hypothesis of direct compositionality for twenty years, this book considers whether its abandonment might have been premature and whether in fact direct compositionality is not after all a simpler and more effective conception of the grammar than the conventional account of the syntax-semantics interface in generative grammar. It contains contributions from both sides of the debate, locates the debate in the setting of a variety of formal theories, and draws on examples from a range of languages and a range of empirical phenomena. (shrink)

Written by two of the leading figures in the field, this is a lucid and systematic introduction to semantics as applied to transformational grammars of the ...

Reasons are given to justify the claim that computer simulations and computational science constitute a distinctively new set of scientific methods and that these methods introduce new issues in the philosophy of science. These issues are both epistemological and methodological in kind.

In Groenendijk & Stokhof [1989] a system of dynamic predicate logic (DPL) was developed, as a compositional alternative for classical discourse representation theory (DRT ). DPL shares with DRT the restriction of being a first-order system. In the present paper, we are mainly concerned with overcoming this limitation. We shall define a dynamic semantics for a typed language with λ-abstraction which is compatible with the semantics DPL specifies for the language of first-order predicate logic. We shall propose to use this (...) new logical system as the semantic component of a Montague-style grammar (referred to as dynamic Montague grammar, DMG), which will enable us to extend the compositionality of DPL to the subsentential level. Furthermore, we shall extend this analysis also in this sense that we shall add new, dynamic interpretations for logical constants which in DPL were treated in a static fashion. This will substantially increase the descriptive coverage of DMG. (shrink)

The aim of this paper is twofold: (i) to introduce the framework of update semantics and to explain what kind of semantic phenomena may successfully be analysed in it: (ii) to give a detailed analysis of one such phenomenon: default reasoning.

has to do with the acquisition of encyclopedic knowledge.

Semantic Structures is a large-scale study of conceptual structure and its lexical and syntactic expression in English that builds on the theory of Conceptual...

This systematic investigation of computation and mental phenomena by a noted psychologist and computer scientist argues that cognition is a form of computation, that the semantic contents of mental states are encoded in the same general way as computer representations are encoded. It is a rich and sustained investigation of the assumptions underlying the directions cognitive science research is taking. 1 The Explanatory Vocabulary of Cognition 2 The Explanatory Role of Representations 3 The Relevance of Computation 4 The Psychological Reality (...) of Programs: Strong Equivalence 5 Constraining Functional Architecture 6 The Bridge from Physical to Symbolic: Transduction 7 Functional Architecture and Analogue Processes 8 Mental Imagery and Functional Architecture 9 Epilogue: What is Cognitive Science the Science of? (shrink)

This paper explores the difference between Connectionist proposals for cognitive a r c h i t e c t u r e a n d t h e s o r t s o f m o d e l s t hat have traditionally been assum e d i n c o g n i t i v e s c i e n c e . W e c l a i m t h a t t h (...) e m a j o r d i s t i n c t i o n i s t h a t , w h i l e b o t h Connectionist and Classical architectures postulate representational mental states, the latter but not the former are committed to a symbol-level of representation, or to a ‘language of thought’: i.e., to representational states that have combinatorial syntactic and semantic structure. Several arguments for combinatorial structure in mental representations are then reviewed. These include arguments based on the ‘systematicity’ of mental representation: i.e., on the fact that cognitive capacities always exhibit certain symmetries, so that the ability to entertain a given thought implies the ability to entertain thoughts with semantically related contents. We claim that such arguments make a powerful case that mind/brain architecture is not Connectionist at the cognitive level. We then consider the possibility that Connectionism may provide an account of the neural (or ‘abstract neurological’) structures in which Classical cognitive architecture is implemented. We survey a n u m b e r o f t h e s t a n d a r d a r g u m e n t s t h a t h a v e b e e n o f f e r e d i n f a v o r o f Connectionism, and conclude that they are coherent only on this interpretation. (shrink)

Several philosophical issues in connection with computer simulations rely on the assumption that results of simulations are trustworthy. Examples of these include the debate on the experimental role of computer simulations :483–496, 2009; Morrison in Philos Stud 143:33–57, 2009), the nature of computer data Computer simulations and the changing face of scientific experimentation, Cambridge Scholars Publishing, Barcelona, 2013; Humphreys, in: Durán, Arnold Computer simulations and the changing face of scientific experimentation, Cambridge Scholars Publishing, Barcelona, 2013), and the explanatory power of (...) computer simulations :277–292, 2008; Durán in Int Stud Philos Sci 31:27–45, 2017). The aim of this article is to show that these authors are right in assuming that results of computer simulations are to be trusted when computer simulations are reliable processes. After a short reconstruction of the problem of epistemic opacity, the article elaborates extensively on computational reliabilism, a specified form of process reliabilism with computer simulations located at the center. The article ends with a discussion of four sources for computational reliabilism, namely, verification and validation, robustness analysis for computer simulations, a history of successful implementations, and the role of expert knowledge in simulations. (shrink)

The paper identifies some background assumptions of compositionality in formal semantics and investigates how they shape formal semantics as a scientific discipline.

Words form a fundamental basis for our understanding of linguistic practice. However, the precise ontology of words has eluded many philosophers and linguists. A persistent difficulty for most accounts of words is the type-token distinction [Bromberger, S. 1989. “Types and Tokens in Linguistics.” In Reflections on Chomsky, edited by A. George, 58–90. Basil Blackwell; Kaplan, D. 1990. “Words.” Aristotelian Society Supplementary Volume LXIV: 93–119]. In this paper, I present a novel account of words which differs from the atomistic and platonistic (...) conceptions of previous accounts which I argue fall prey to this problem. Specifically, I proffer a structuralist account of linguistic items, along the lines of structuralism in the philosophy of mathematics [Shapiro, S. 1997. Philosophy of Mathematics: Structure and Ontology. Oxford University Press], in which words are defined in part as positions in larger linguistic structures. I then follow Szabò [1999. “Expressions and Their Representations.” The Philosophical Quarterly 49 (195): 145–163] and Parsons [1990. “The Structuralist View of Mathematical Objects.” Synthese 84: 303–346] in further defining words as quasi-concrete objects according to a representation relation. This view aims for general correspondence with contemporary generative linguistic approaches to the study of language. (shrink)

The ideas of contextuality and of compositionality were discussed at the beginnings of the nineteenth century in Germany, but the contextuality was the significant one. In 1880, Wundt published a work called Logik, comprising two volumes, which evolved to the German standard text on logic. Gottlob Frege, a German mathematician, logician, and philosopher, presented contextuality as his basic principle; his solution of the foundational problems is based upon it, he meant the principle literally, and would have rejected compositionality. He introduced (...) the distinction between sense and reference in his paper, “Über Sinn und Bedeutung” in 1892. He did not accept the notion the reference of an expression; instead, his approach is based upon the notion the reference of an expression in a given sentence. In the period before Tarski, logics were regarded as deductive theories. Tarski made an important step with his paper on “Truth in formalized languages,” in which he characterized the notion of a true sentence in a purely semantic way. (shrink)

Computer simulations are an exciting tool that plays important roles in many scientific disciplines. This has attracted the attention of a number of philosophers of science. The main tenor in this literature is that computer simulations not only constitute interesting and powerful new science, but that they also raise a host of new philosophical issues. The protagonists in this debate claim no less than that simulations call into question our philosophical understanding of scientific ontology, the epistemology and semantics of models (...) and theories, and the relation between experimentation and theorising, and submit that simulations demand a fundamentally new philosophy of science in many respects. The aim of this paper is to critically evaluate these claims. Our conclusion will be sober. We argue that these claims are overblown and that simulations, far from demanding a new metaphysics, epistemology, semantics and methodology, raise few if any new philosophical problems. The philosophical problems that do come up in connection with simulations are not specific to simulations and most of them are variants of problems that have been discussed in other contexts before. (shrink)

This article presents three more-or-less-traditional considerations for compositionality. The first is that the usual statement of the compositionality principle is massively ambiguous. One of the eight available readings rules out all sources of multiplicity in meaning in complex expressions besides the lexicon and the syntax. Others are more permissive—how much more is not always clear. The second claim is that traditional considerations in favour of compositionality are less powerful than is often assumed. Compositionality is best construed as an empirical hypothesis (...) on meanings expressed in natural languages. Finally, the third claim is that, even if compositionality is true, most of the debates in philosophy, linguistics, and psychology surrounding compositionality will remain open. These debates tend to be about significantly stronger theses. (shrink)