Naturally occurring speech contains only a limited amount of complex recursive structure, and this is reflected in the empirically documented difficulties that people experience when processing such structures. We present a connectionist model of human performance in processing recursive language structures. The model is trained on simple artificial languages. We find that the qualitative performance profile of the model matches human behavior, both on the relative difficulty of center‐embedding and cross‐dependency, and between the processing of these complex recursive structures and (...) right‐branching recursive constructions. We analyze how these differences in performance are reflected in the internal representations of the model by performing discriminant analyses on these representations both before and after training. Furthermore, we show how a network trained to process recursive structures can also generate such structures in a probabilistic fashion. This work suggests a novel explanation of people's limited recursive performance, without assuming the existence of a mentally represented competence grammar allowing unbounded recursion. (shrink)

Over the last quarter century, the dominant tendency in comparative cognitive psychology has been to emphasize the similarities between human and nonhuman minds and to downplay the differences as (Darwin 1871). In the present target article, we argue that Darwin was mistaken: the profound biological continuity between human and nonhuman animals masks an equally profound discontinuity between human and nonhuman minds. To wit, there is a significant discontinuity in the degree to which human and nonhuman animals are able to approximate (...) the higher-order, systematic, relational capabilities of a physical symbol system (PSS) (Newell 1980). We show that this symbolic-relational discontinuity pervades nearly every domain of cognition and runs much deeper than even the spectacular scaffolding provided by language or culture alone can explain. We propose a representational-level specification as to where human and nonhuman animals' abilities to approximate a PSS are similar and where they differ. We conclude by suggesting that recent symbolic-connectionist models of cognition shed new light on the mechanisms that underlie the gap between human and nonhuman minds. (shrink)

This paper investigates connectionism's potential to solve the frame problem. The frame problem arises in the context of modelling the human ability to see the relevant consequences of events in a situation. It has been claimed to be unsolvable for classical cognitive science, but easily manageable for connectionism. We will focus on a representational approach to the frame problem which advocates the use of intrinsic representations. We argue that although connectionism's distributed representations may look promising from this perspective, doubts can (...) be raised about the potential of distributed representations to allow large amounts of complexly structured information to be adequately encoded and processed. It is questionable whether connectionist models that are claimed to effectively represent structured information can be scaled up to a realistic extent. We conclude that the frame problem provides a difficulty to connectionism that is no less serious than the obstacle it constitutes for classical cognitive science. (shrink)

Fodor's and Pylyshyn's stand on systematicity in thought and language has been debated and criticized. Van Gelder and Niklasson, among others, have argued that Fodor and Pylyshyn offer no precise definition of systematicity. However, our concern here is with a learning based formulation of that concept. In particular, Hadley has proposed that a network exhibits strong semantic systematicity when, as a result of training, it can assign appropriate meaning representations to novel sentences (both simple and embedded) which contain words in (...) syntactic positions they did not occupy during training. The experience of researchers indicates that strong systematicity in any form is difficult to achieve in connectionist systems.Herein we describe a network which displays strong semantic systematicity in response to Hebbian, connectionist training. During training, two-thirds of all nouns are presented only in a single syntactic position (either as grammatical subject or object). Yet, during testing, the network correctly interprets thousands of sentences containing those nouns in novel positions. In addition, the network generalizes to novel levels of embedding. Successful training requires a, corpus of about 1000 sentences, and network training is quite rapid. The architecture and learning algorithms are purely connectionist, but classical insights are discernible in one respect, viz, that complex semantic representations spatially contain their semantic constituents. However, in other important respects, the architecture is distinctly non-classical. (shrink)

A comprehensive introduction to the Language of Though Hypothesis (LOTH) accessible to general audiences. LOTH is an empirical thesis about thought and thinking. For their explication, it postulates a physically realized system of representations that have a combinatorial syntax (and semantics) such that operations on representations are causally sensitive only to the syntactic properties of representations. According to LOTH, thought is, roughly, the tokening of a representation that has a syntactic (constituent) structure with an appropriate semantics. Thinking thus consists in (...) syntactic operations defined over representations. Most of the arguments for LOTH derive their strength from their ability to explain certain empirical phenomena like productivity, systematicity of thought and thinking. (shrink)

This paper explores early Australasian philosophy in some detail. Two approaches have dominated Western philosophy in Australia: idealism and materialism. Idealism was prevalent between the 1880s and the 1930s, but dissipated thereafter. Idealism in Australia often reflected Kantian themes, but it also reflected the revival of interest in Hegel through the work of ‘absolute idealists’ such as T. H. Green, F. H. Bradley, and Henry Jones. A number of the early New Zealand philosophers were also educated in the idealist tradition (...) and were influential in their communities, but produced relatively little. In Australia, materialism gained prominence through the work of John Anderson, who arrived in Australia in 1927, and continues to be influential. John Anderson had been a student of Henry Jones, who might therefore be said to have influenced both main strands of Australian philosophical thought. (shrink)

Philosophy and Memory Traces defends two theories of autobiographical memory. One is a bewildering historical view of memories as dynamic patterns in fleeting animal spirits, nervous fluids which rummaged through the pores of brain and body. The other is new connectionism, in which memories are 'stored' only superpositionally, and reconstructed rather than reproduced. Both models, argues John Sutton, depart from static archival metaphors by employing distributed representation, which brings interference and confusion between memory traces. Both raise urgent issues about control (...) of the personal past, and about relations between self and body. Sutton demonstrates the role of bizarre body fluids in moral physiology, as philosophers from Descartes and Locke to Coleridge struggled to control their own innards and impose cognitive discipline on 'the phantasmal chaos of association'. Going on to defend connectionism against Fodor and critics of passive mental representations, he shows how problems of the self are implicated in cognitive science. (shrink)

The systematicity argument only challenges connectionism if systematicity is a general property of cognition. I examine this thesis in terms of properties of concepts. First, I propose that Evans's Generality Constraint only applies to attributions of belief. Then I defend a variety of conceptual pluralism, arguing that concepts share two fundamental properties related to centrality and belief-attribution, and contending that there are two kinds of concepts that differ in their compositional properties. Finally, I rely on Dual Systems Theory and on (...) differences between animal and human cognition to suggest a scenario of two processing systems that work on different kinds of concepts, with only one of them supporting full systematicity. I sketch a non-classical systematicity argument that rules out classicism as the basis of one of those systems given that it would wrongly entail that both systems are fully systematic. (shrink)

The distributional principle according to which morphemes that occur in identical contexts belong, in some sense, to the same category [1] has been advanced as a means for extracting syntactic structures from corpus data. We extend this principle by applying it recursively, and by using mutual information for estimating category coherence. The resulting model learns, in an unsupervised fashion, highly structured, distributed representations of syntactic knowledge from corpora. It also exhibits promising behavior in tasks usually thought to require representations anchored (...) in a grammar, such as systematicity. (shrink)

A lengthy debate in the philosophy of the cognitive sciences has turned on whether the phenomenon known as ‘systematicity’ of language and thought shows that connectionist explanatory aspirations are misguided. We investigate the issue of just which phenomenon ‘systematicity’ is supposed to be. The much-rehearsed examples always suggest that being systematic has something to do with ways in which some parts of expressions in natural languages (and, more conjecturally, some parts of thoughts) can be substituted for others without altering well-formedness. (...) We show that under one construal this yields a grossly weak claim that is not just compatible with a narrow version of associationist psychology but essentially coincides with a formalization of its descriptive power. Under another construal we get a claim (apparently unintended) that requires natural languages to fall within the context-free class, a claim that most linguists regard as too strong. Looking more closely at this proposed reconstruction of systematicity leads us to endorse, with further illustrations, the suggestion of Johnson (2004) that systematicity as a matter of substitutability of co-categorial constituents for one another does not appearto hold of natural languages at all. The appeal of the ill-delineated notion of systematicity may lie in the fact that within certain subclasses of lexical items mutual intersubstitutability does seem to hold, and theexplanation for that lies in a limitation on human memory: we simply cannot learn separate privileges of syntactic distribution for all of the huge number of words and phrases that we know. (shrink)

One of the main challenges that Jerry Fodor and Zenon Pylyshyn (Cognition 28:3–71, 1988) posed for any connectionist theory of cognitive architecture is to explain the systematicity of thought without implementing a Language of Thought (LOT) architecture. The systematicity challenge presents a dilemma: if connectionism cannot explain the systematicity of thought, then it fails to offer an adequate theory of cognitive architecture; and if it explains the systematicity of thought by implementing a LOT architecture, then it fails to offer an (...) alternative to the LOT hypothesis. Given that thought is systematic, connectionism can offer an adequate alternative to the LOT hypothesis only if it can meet the challenge. Although some critics tried to meet the challenge, others argued that it need not be met since thought is not in fact systematic; and some claimed not to even understand the claim that thought is systematic. I do not here examine attempts to answer the challenge. Instead, I defend the challenge itself by explicating the notion of systematicity in a way that I hope makes clear that thought is indeed systematic, and so that to offer an adequate alternative to the LOT hypothesis, connectionism must meet the challenge. (shrink)

In their provocative 1988 paper, Fodor and Pylyshyn issued a formidable challenge to connectionists, i.e. to provide a non‐classical explanation of the empirical phenomenon of systematicity in cognitive agents. Since the appearance of F&P's challenge, a number of connectionist systems have emerged which prima facie meet this challenge. However, Fodor and McLaughlin (1990) advance an argument, based upon a general principle of nomological necessity, to show that one of these systems (Smolensky's) could not satisfy the Fodor‐Pylyshyn challenge. Yet, if Fodor (...) and McLaughlin's analysis is correct, it is doubtful whether any existing connectionist system would fare better than Smolensky's. In the view of Fodor and McLaughlin, humans and classical architectures display systematicity as a matter of nomological necessity (necessity by virtue of natural law), but connectionist architectures do not. However, I argue that the Fodor‐Pylyshyn‐McLaughlin appeal to nomological necessity is untenable. There is a sense in which neither classical nor connectionist architectures possess nomological (or‘nomic’) necessity. However, the sense in which classical architectures do possess nomic necessity applies equally well to at least some connectionist architectures. Representational constituents can have causal efficacy within both classical and connectionist architectures. (shrink)

Reverse Compositionality (RC) is the thesis that one understands a complex expression only if one understands its parts. I argue that this thesis is false for natural languages. I then argue that the phenomenon that motivates the thesis is more likely to be a fact about human sentence-processing than linguistic understanding per se. Finally, I argue that RC is not useful in the debates about prototype-style theories of concepts in which it figures heavily.

The emphasis in the connectionist sentence‐processing literature on distributed representation and emergence of grammar from such systems can easily obscure the often close relations between connectionist and symbolist systems. This paper argues that the Simple Recurrent Network (SRN) models proposed by Jordan (1989) and Elman (1990) are more directly related to stochastic Part‐of‐Speech (POS) Taggers than to parsers or grammars as such, while auto‐associative memory models of the kind pioneered by Longuet–Higgins, Willshaw, Pollack and others may be useful for grammar (...) induction from a network‐based conceptual structure as well as for structure‐building. These observations suggest some interesting new directions for specifically connectionist sentence processing research, including more efficient representations for finite state machines, and acquisition devices based on a distinctively connectionist basis for grounded symbolist conceptual structure. (shrink)

In the late 1980s, there were many who heralded the emergence of connectionism as a new paradigm – one which would eventually displace the classically symbolic methods then dominant in AI and Cognitive Science. At present, there remain influential connectionists who continue to defend connectionism as a more realistic paradigm for modeling cognition, at all levels of abstraction, than the classical methods of AI. Not infrequently, one encounters arguments along these lines: given what we know about neurophysiology, it is just (...) not plausible to suppose that our brains are digital computers. Thus, they could not support a classical architecture. I argue here for a middle ground between connectionism and classicism. I assume, for argument's sake, that some form(s) of connectionism can provide reasonably approximate models – at least for lower-level cognitive processes. Given this assumption, I argue on theoretical and empirical grounds that most human mental skills must reside in separate connectionist modules or sub-networks. Ultimately, it is argued that the basic tenets of connectionism, in conjunction with the fact that humans often employ novel combinations of skill modules in rule following and problem solving, lead to the plausible conclusion that, in certain domains, high level cognition requires some form of classical architecture. During the course of argument, it emerges that only an architecture with classical structure could support the novel patterns of information flow and interaction that would exist among the relevant set of modules. Such a classical architecture might very well reside in the abstract levels of a hybrid system whose lower-level modules are purely connectionist. (shrink)

The past decade has witnessed the emergence of a novel stance on semantic representation, and its relationship to context sensitivity. Connectionist-minded philosophers, including Clark and van Gelder, have espoused the merits of viewing hidden-layer, context-sensitive representations as possessing semantic content, where this content is partially revealed via the representations'' position in vector space. In recent work, Bodén and Niklasson have incorporated a variant of this view of semantics within their conception of semantic systematicity. Moreover, Bodén and Niklasson contend that they (...) have produced experimental results which not only satisfy a kind of context-based, semantic systematicity, but which, to the degree that reality permits, effectively deals with challenges posed by Fodor and Pylyshyn (1988), and Hadley (1994a). The latter challenge involved well-defined criteria for strong semantic systematicity. This paper examines the relevant claims and experiments of Bodén and Niklasson. It is argued that their case fatally involves two fallacies of equivocation; one concerning ''semantic content'' and the other concerning ''novel test sentences''. In addition, it is argued that their ultimate construal of context sensitive semantics contains serious confusions. These confusions are also found in certain publications dealing with "latent semantic analysis". Thus, criticisms presented here have relevance beyond the work of Bodén and Niklasson. (shrink)

The emphasis in the connectionist sentence‐processing literature on distributed representation and emergence of grammar from such systems can easily obscure the often close relations between connectionist and symbolist systems. This paper argues that the Simple Recurrent Network (SRN) models proposed by Jordan (1989) and Elman (1990) are more directly related to stochastic Part‐of‐Speech (POS) Taggers than to parsers or grammars as such, while auto‐associative memory models of the kind pioneered by Longuet–Higgins, Willshaw, Pollack and others may be useful for grammar (...) induction from a network‐based conceptual structure as well as for structure‐building. These observations suggest some interesting new directions for specifically connectionist sentence processing research, including more efficient representations for finite state machines, and acquisition devices based on a distinctively connectionist basis for grounded symbolist conceptual structure. (shrink)