This paper attempts to describe and address a specific puzzle related to compositionality in artificial networks such as Deep Neural Networks and machine learning in general. The puzzle identified here touches on a larger debate in Artificial Intelligence related to epistemic opacity but specifically focuses on computational applications of human level linguistic abilities or properties and a special difficulty with relation to these. Thus, the resulting issue is both general and unique. A partial solution is suggested.

Semantic composition in language must be closely related to semantic composition in thought. But the way the two processes are explained differs considerably. Focusing primarily on propositional content, language theorists generally take semantic composition to be a truth‐conditional process. Focusing more on extensional content, cognitive theorists take it to be a form of concept combination. But though deep, this disconnect is not irreconcilable. Both areas of theory assume that extensional (i.e., denotational) meanings must play a role. As this article demonstrates, (...) they also have the potential to fulfill a mediative function. What is shown is that extensional meanings are themselves inherently compositional. On this basis, it becomes possible to model semantic composition without assuming the existence of any specifically linguistic/conceptual apparatus. Examples are presented to demonstrate this direct style of modeling. Abstract connections between composition in thought and language can then be made, raising the prospect of a more unified, theoretical account of semantic composition. (shrink)