This paper is a commentary on the target article by Lokendra Shastri & Venkat Ajjanagadde [S&A]: “From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony” in same issue of the journal, pp.417–451. -/- It puts S&A's temporal-synchrony binding method in a broader context, comments on notions of pointing and other ways of associating information - in both computers and connectionist systems - and mentions types of reasoning that are a challenge to (...) S&A's system (amongst many other connectionist systems, recognizing that S&A's is much more capable as regards reasoning than most other contemporary systems). (shrink)

The current renewal of connectionist techniques using networks of neuron-like units has started to have an influence on cognitive modelling. However, compared with classical artificial intelligence methods, the position of connectionism is still not clear. In this article artificial intelligence and connectionism are systematically compared as cognitive models so as to bring out the advantages and shortcomings of each. The problem of structured representations appears to be particularly important, suggesting likely research directions.

Preparing words in speech production is normally a fast and accurate process. We generate them two or three per second in fluent conversation; and overtly naming a clear picture of an object can easily be initiated within 600 msec after picture onset. The underlying process, however, is exceedingly complex. The theory reviewed in this target article analyzes this process as staged and feedforward. After a first stage of conceptual preparation, word generation proceeds through lexical selection, morphological and phonological encoding, phonetic (...) encoding, and articulation itself. In addition, the speaker exerts some degree of output control, by monitoring of self-produced internal and overt speech. The core of the theory, ranging from lexical selection to the initiation of phonetic encoding, is captured in a computational model, called WEAVER++. Both the theory and the computational model have been developed in interaction with reaction time experiments, particularly in picture naming or related word production paradigms, with the aim of accounting for the real-time processing in normal word production. A comprehensive review of theory, model, and experiments is presented. The model can handle some of the main observations in the domain of speech errors (the major empirical domain for most other theories of lexical access), and the theory opens new ways of approaching the cerebral organization of speech production by way of high-temporal-resolution imaging. (shrink)

Human agents draw a variety of inferences effortlessly, spontaneously, and with remarkable efficiency – as though these inferences were a reflexive response of their cognitive apparatus. Furthermore, these inferences are drawn with reference to a large body of background knowledge. This remarkable human ability seems paradoxical given the complexity of reasoning reported by researchers in artificial intelligence. It also poses a challenge for cognitive science and computational neuroscience: How can a system of simple and slow neuronlike elements represent a large (...) body of systemic knowledge and perform a range of inferences with such speed? We describe a computational model that takes a step toward addressing the cognitive science challenge and resolving the artificial intelligence paradox. We show how a connectionist network can encode millions of facts and rules involving n-ary predicates and variables and perform a class of inferences in a few hundred milliseconds. Efficient reasoning requires the rapid representation and propagation of dynamic bindings. Our model (which we refer to as SHRUTI) achieves this by representing (1) dynamic bindings as the synchronous firing of appropriate nodes, (2) rules as interconnection patterns that direct the propagation of rhythmic activity, and (3) long-term facts as temporal pattern-matching subnetworks. The model is consistent with recent neurophysiological evidence that synchronous activity occurs in the brain and may play a representational role in neural information processing. The model also makes specific psychologically significant predictions about the nature of reflexive reasoning. It identifies constraints on the form of rules that may participate in such reasoning and relates the capacity of the working memory underlying reflexive reasoning to biological parameters such as the lowest frequency at which nodes can sustain synchronous oscillations and the coarseness of synchronization. (shrink)

There has been much interest in the possibility of connectionist models whose representations can be endowed with compositional structure, and a variety of such models have been proposed. These models typically use distributed representations that arise from the functional composition of constituent parts. Functional composition and decomposition alone, however, yield only an implementation of classical symbolic theories. This paper explores the possibility of moving beyond implementation by exploiting holistic structure-sensitive operations on distributed representations. An experiment is performed using Pollack’s Recursive (...) Auto-Associative Memory. RAAM is used to construct distributed representations of syntactically structured sentences. A feed-forward network is then trained to operate directly on these representations, modeling syn- tactic transformations of the represented sentences. Successful training and generalization is obtained, demonstrating that the implicit structure present in these representations can be used for a kind of structure-sensitive processing unique to the connectionist domain. (shrink)

There is a vast literature within philosophy of mind that focuses on artificial intelligence, but hardly mentions methodological questions. There is also a growing body of work in philosophy of science about modeling methodology that hardly mentions examples from cognitive science. Here these discussions are connected. Insights developed in the philosophy of science literature about the importance of idealization provide a way of understanding the neural implausibility of connectionist networks. Insights from neurocognitive science illuminate how relevant similarities between models and (...) targets are picked out, how modeling inferences are justified, and the metaphysical status of models. (shrink)

I argue that SHRUTl's ontology is heavily committed to a representational view of mind. This is best seen when one thinks of how SHRUTI could be developed to account for psychological data on deductive reasoning.

Shastri & Ajjanagadde argue convincingly that both structured connectionist networks and parallel dynamic inferencing are necessary for reflexive reasoning - a kind of inferencing and reasoning that occurs rapidly, spontaneously, and without conscious effort, and which seems necessary for everyday tasks such as natural language understanding. As S&A describe, reflexive reasoning requires a solution to thedynamic binding problem, that is, how to encode systematic and abstract knowledge and instantiate it in specific situations to draw appropriate inferences. Although symbolic artificial intelligence (...) systems trivially solve the dynamic binding problem using computers' registers and pointers, it has remained a difficult problem for connectionist systems. S&A's temporal synchrony solution to the dynamic binding problem using synchronous firing of argument units and the entities that are bound to them illustrates one way in which connectionist networks can do thisusing a constrained but important class of long-term knowledge rules. Their structured connectionist solution allows dynamic inferencing to proceed in parallel and therefore has a number of advantages for reflexive reasoning over most other connectionist and symbolic systems. (shrink)

It is widely mooted that a plausible computational cognitive model should involve both symbolic and connectionist components. However, sound principles for combining these components within a hybrid system are currently lacking; the design of such systems is oftenad hoc. In an attempt to ameliorate this we provide a framework of types of hybrid systems and constraints therein, within which to explore the issues. In particular, we suggest the use of system independent constraints, whose source lies in general considerations about cognitive (...) systems, rather than in particular technological or task-based considerations. We illustrate this through a detailed examination of an interruptibility constraint: handling interruptions is a fundamental facet of cognition in a dynamic world. Aspects of interruptions are delineated, as are their precise expression in symbolic and connectionist systems. We illustrate the interaction of the various constraints from interruptibility in the different types of hybrid systems. The picture that emerges of the relationship between the connectionist and the symbolic within a hybrid system provides for sufficient flexibility and complexity to suggest interesting general implications for cognition, thus vindicating the utility of the framework. (shrink)

There is currently a debate over whether cognitive architecture is classical or connectionist in nature. One finds the following three comparisons between classical architecture and connectionist architecture made in the pro-connectionist literature in this debate: (1) connectionist architecture is neurally plausible and classical architecture is not; (2) connectionist architecture is far better suited to model pattern recognition capacities than is classical architecture; and (3) connectionist architecture is far better suited to model the acquisition of pattern recognition capacities by learning than (...) is classical architecture. If true, (1)–(3) would yield a compelling case against the view that cognitive architecture is classical, and would offer some reason to think that cognitive architecture may be connectionist. We first present the case for (1)–(3) in the very words of connectionist enthusiasts. We then argue that the currently available evidence fails to support any of (1)–(3). (shrink)

The performance of symbolic inference tasks has long been a challenge to connectionists. In this paper, we present an extended survey of this area. Existing connectionist inference systems are reviewed, with particular reference to how they perform variable binding and rule- based reasoning and whether they involve distributed or localist representations. The bene®ts and disadvantages of different representations and systems are outlined, and conclusions drawn regarding the capabilities of connectionist inference systems when compared with symbolic inference systems or when used (...) for cognitive modelling. q 2001 Elsevier Science Ltd. All rights reserved. (shrink)

At present, the prevailing Connectionist methodology forrepresenting rules is toimplicitly embody rules in neurally-wired networks. That is, the methodology adopts the stance that rules must either be hard-wired or trained into neural structures, rather than represented via explicit symbolic structures. Even recent attempts to implementproduction systems within connectionist networks have assumed that condition-action rules (or rule schema) are to be embodied in thestructure of individual networks. Such networks must be grown or trained over a significant span of time. However, arguments (...) are presented herein that humanssometimes follow rules which arevery rapidly assignedexplicit internal representations, and that humans possessgeneral mechanisms capable of interpreting and following such rules. In particular, arguments are presented that thespeed with which humans are able to follow rules ofnovel structure demonstrates the existence of general-purpose rule following mechanisms. It is further argued that the existence of general-purpose rule following mechanisms strongly indicates that explicit rule following is not anisolated phenomenon, but may well be a common and important aspect of cognition. The relationship of the foregoing conclusions to Smolensky''s view of explicit rule following is also explored. The arguments presented here are pragmatic in nature, and are contrasted with thekind of arguments developed by Fodor and Pylyshyn in their recent, influential paper. (shrink)

Lange & Dyer misunderstand what is meant by an “entity” and confuse a medium of representation with the content being represented. This leads them to the erroneous conclusion that SHRUTI will run out of phases and that its representation of bindings lacks semantic content. It is argued that the limit on the number of phases suffices, and SHRUTI can be interpreted as using “dynamic signatures” that offer significant advantages over fixed preexisting signatures. Bonatti refers to three levels of commitment to (...) a representational theory of the mind and states that SHRUTI is committed to RTM at levels 1 and 2. He acknowledges that SHRUTI is not committed at level 3, but argues that an extended SHRUTI would have to make such a commitment. We agree that SHRUTI is committed to RTM at level 1 and, in a sense, also at level 2. SHRUTI, however, is not committed to RTM at level 3 and, even though it has been extended in several ways, it has retained its original representational character. (shrink)

If artificial neural networks are ever to form the foundation for higher level cognitive behaviors in machines or to realize their full potential as explanatory devices for human cognition, they must show signs of autonomy, multifunction operation, and intersystem integration that are absent in most existing models. This model begins to address these issues by integrating predictive learning, sequence interleaving, and sequence creation components to simulate a spectrum of higher-order cognitive behaviors which have eluded the grasp of simpler systems. Its (...) capabilities are described based on simulations calling for increasing levels of functionality and are used to show how the model can progress from fundamental sequence learning and recall tasks to sophisticated behaviors such as an ability to solve simple mathematical expressions and a creative capacity for the formation and application of inductive rules. (shrink)