In 1988 Fodor and Pylyshyn issued a challenge to the newly-popular connectionism: explain the systematicity of cognition without merely implementing a so-called classical architecture. Since that time quite a number of connectionist models have been put forward, either by their designers or by others, as in some measure demonstrating that the challenge can be met (e.g., Pollack, 1988, 1990; Smolensky, 1990; Chalmers, 1990; Niklasson and Sharkey, 1992; Brousse, 1993). Unfortu- nately, it has generally been unclear whether these models actually do (...) have this implication (see, for instance, the extensive philosophical debate in Smolensky, 1988; Fodor and McLaughlin, 1990; van Gelder, 1990, 1991; McLaughlin, 1993a, 1993b; Clark, 1993). Indeed, we know of no major supporter of classical orthodoxy who has felt compelled, by connectionist models and argu- ments, to concede in print that connectionists have in fact delivered a non-classical explanation of systematicity. (shrink)

This paper attempts to describe and address a specific puzzle related to compositionality in artificial networks such as Deep Neural Networks and machine learning in general. The puzzle identified here touches on a larger debate in Artificial Intelligence related to epistemic opacity but specifically focuses on computational applications of human level linguistic abilities or properties and a special difficulty with relation to these. Thus, the resulting issue is both general and unique. A partial solution is suggested.

An ongoing philosophical discussion concerns how various types of mental states fall within broad representational genera—for example, whether perceptual states are “iconic” or “sentential,” “analog” or “digital,” and so on. Here, I examine the grounds for making much more specific claims about how mental states are structured from constituent parts. For example, the state I am in when I perceive the shape of a mountain ridge may have as constituent parts my representations of the shapes of each peak and saddle (...) of the ridge. More specific structural claims of this sort are a guide to how mental states fall within broader representational kinds. Moreover, these claims have significant implications of their own about semantic, functional, and epistemic features of our mental lives. But what are the conditions on a mental state's having one type of constituent structure rather than another? Drawing on explanatory strategies in vision science, I argue that, other things being equal, the constituent structure of a mental state determines what I call its distributional properties—namely, how mental states of that type can, cannot, or must co‐occur with other mental states in a given system. Distributional properties depend critically on and are informative about the underlying structures of mental states, they abstract in important ways from aspects of how mental states are processed, and they can yield significant insights into the variegation of psychological capacities. (shrink)

Provided we agree about the thing, it is needless to dispute about the terms. —David Hume, A treatise of human nature, Book 1, section VIIMap-like representations are frequently invoked as an alternative type of representational vehicle to a language of thought. This view presupposes that map-systems and languages form legitimate natural kinds of cognitive representational systems. I argue that they do not, because the collections of features that might be taken as characteristic of maps or languages do not themselves provide (...) scientifically useful information above and beyond what the individual features provide. To bring this out, I sketch several allegedly distinctive features of maps, and show how they can easily be grafted onto a simple logical language, resulting in a hybrid “manguage.” The ease with which these linguistic and map-like properties can be integrated into a single representational system raises the question of what work broader categories like language and map are doing. I maintain.. (shrink)

In their provocative 1988 paper, Fodor and Pylyshyn issued a formidable challenge to connectionists, i.e. to provide a non‐classical explanation of the empirical phenomenon of systematicity in cognitive agents. Since the appearance of F&P's challenge, a number of connectionist systems have emerged which prima facie meet this challenge. However, Fodor and McLaughlin (1990) advance an argument, based upon a general principle of nomological necessity, to show that one of these systems (Smolensky's) could not satisfy the Fodor‐Pylyshyn challenge. Yet, if Fodor (...) and McLaughlin's analysis is correct, it is doubtful whether any existing connectionist system would fare better than Smolensky's. In the view of Fodor and McLaughlin, humans and classical architectures display systematicity as a matter of nomological necessity (necessity by virtue of natural law), but connectionist architectures do not. However, I argue that the Fodor‐Pylyshyn‐McLaughlin appeal to nomological necessity is untenable. There is a sense in which neither classical nor connectionist architectures possess nomological (or‘nomic’) necessity. However, the sense in which classical architectures do possess nomic necessity applies equally well to at least some connectionist architectures. Representational constituents can have causal efficacy within both classical and connectionist architectures. (shrink)

In their provocative 1988 paper, Fodor and Pylyshyn issued a formidable challenge to connectionists, i.e. to provide a non‐classical explanation of the empirical phenomenon of systematicity in cognitive agents. Since the appearance of F&P's challenge, a number of connectionist systems have emerged which prima facie meet this challenge. However, Fodor and McLaughlin (1990) advance an argument, based upon a general principle of nomological necessity, to show that one of these systems (Smolensky's) could not satisfy the Fodor‐Pylyshyn challenge. Yet, if Fodor (...) and McLaughlin's analysis is correct, it is doubtful whether any existing connectionist system would fare better than Smolensky's. In the view of Fodor and McLaughlin, humans and classical architectures display systematicity as a matter of nomological necessity (necessity by virtue of natural law), but connectionist architectures do not. However, I argue that the Fodor‐Pylyshyn‐McLaughlin appeal to nomological necessity is untenable. There is a sense in which neither classical nor connectionist architectures possess nomological (or‘nomic’) necessity. However, the sense in which classical architectures do possess nomic necessity applies equally well to at least some connectionist architectures. Representational constituents can have causal efficacy within both classical and connectionist architectures. (shrink)

At present, the prevailing Connectionist methodology forrepresenting rules is toimplicitly embody rules in neurally-wired networks. That is, the methodology adopts the stance that rules must either be hard-wired or trained into neural structures, rather than represented via explicit symbolic structures. Even recent attempts to implementproduction systems within connectionist networks have assumed that condition-action rules (or rule schema) are to be embodied in thestructure of individual networks. Such networks must be grown or trained over a significant span of time. However, arguments (...) are presented herein that humanssometimes follow rules which arevery rapidly assignedexplicit internal representations, and that humans possessgeneral mechanisms capable of interpreting and following such rules. In particular, arguments are presented that thespeed with which humans are able to follow rules ofnovel structure demonstrates the existence of general-purpose rule following mechanisms. It is further argued that the existence of general-purpose rule following mechanisms strongly indicates that explicit rule following is not anisolated phenomenon, but may well be a common and important aspect of cognition. The relationship of the foregoing conclusions to Smolensky''s view of explicit rule following is also explored. The arguments presented here are pragmatic in nature, and are contrasted with thekind of arguments developed by Fodor and Pylyshyn in their recent, influential paper. (shrink)

Critical notice of Andy Clark's "Associative Engines".

Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implicitly by its effects on processing rather than explicitly (as in a spatial representation). The current report develops a proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory. In this approach, hidden unit patterns are fed back to themselves: (...) the internal representations which develop thus reflect task demands in the context of prior internal states. A set of simulations is reported which range from relatively simple problems (temporal version of XOR) to discovering syntactic/semantic features for words. The networks are able to learn interesting internal representations which incorporate task demands with memory demands: indeed, in this approach the notion of memory is inextricably bound up with task processing. These representations reveal a rich structure, which allows them to be highly context‐dependent, while also expressing generalizations across classes of items. These representations suggest a method for representing lexical categories and the type/token distinction. (shrink)

The problem of representing the spatial structure of images, which arises in visual object processing, is commonly described using terminology borrowed from propositional theories of cognition, notably, the concept of compositionality. The classical propositional stance mandates representations composed of symbols, which stand for atomic or composite entities and enter into arbitrarily nested relationships. We argue that the main desiderata of a representational system — productivity and systematicity — can (indeed, for a number of reasons, should) be achieved without recourse to (...) the classical, proposition-like compositionality. We show how this can be done, by describing a systematic and productive model of the representation of visual structure, which relies on static rather than dynamic binding and uses coarsely coded rather than atomic shape primitives. (shrink)

The problem of representing the spatial structure of images, which arises in visual object processing, is commonly described using terminology borrowed from propositional theories of cognition, notably, the concept of compositionality. The classical propositional stance mandates representations composed of symbols, which stand for atomic or composite entities and enter into arbitrarily nested relationships. We argue that the main desiderata of a representational system—productivity and systematicity—can (indeed, for a number of reasons, should) be achieved without recourse to the classical, proposition‐like compositionality. (...) We show how this can be done, by describing a systematic and productive model of the representation of visual structure, which relies on static rather than dynamic binding and uses coarsely coded rather than atomic shape primitives. (shrink)