Arguably no concept is more fundamental to science than that of causality, for investigations into cases of existence, persistence, and change in the natural world are largely investigations into the causes of these phenomena. Yet the metaphysics and epistemology of causality remain unclear. For example, the ontological categories of the causal relata have been taken to be objects (Hume 1739), events (Davidson 1967), properties (Armstrong 1978), processes (Salmon 1984), variables (Hitchcock 1993), and facts (Mellor 1995). (For convenience, causes and effects (...) will usually be understood as events in what follows.) Complicating matters, causal relations may be singular (Socrates’s drinking hemlock caused Socrates’s death) or general (Drinking hemlock causes death); hence the relata might be tokens (e.g., instances of properties) or types (e.g., types of events) of the category in question. Other questions up for grabs are: Are singular causes metaphysically and/or epistemologically prior to general causes or vice versa (or neither)? What grounds the intuitive asymmetry of the causal relation? Are macro-causal relations reducible to micro-causal relations? And perhaps most importantly: Are causal facts (e.g., the holding of causal relations) reducible to non-causal facts (e.g., the holding of certain spatiotemporal relations)? (shrink)

Are similar, or even identical, mechanisms used in the computational modeling of speech segmentation, serial image processing, and music processing? We address this question by exploring how TRACX2, a recognition‐based, recursive connectionist autoencoder model of chunking and sequence segmentation, which has successfully simulated speech and serial‐image processing, might be applied to elementary melody perception. The model, a three‐layer autoencoder that recognizes “chunks” of short sequences of intervals that have been frequently encountered on input, is trained on the tone intervals of (...) melodically simple French children's songs. It dynamically incorporates the internal representations of these chunks into new input. Its internal representations cluster in a manner that is consistent with “human‐recognizable” melodic categories. TRACX2 is sensitive to both contour and proximity information in the musical chunks that it encounters in its input. It shows the “end‐of‐word” superiority effect demonstrated by Saffran et al. (1999) for short musical phrases. The overall findings suggest that the recursive autoassociative chunking mechanism, as implemented in TRACX2, may be a general segmentation and chunking mechanism, underlying not only word‐ and image‐chunking, but also elementary melody processing. (shrink)

Several approaches to implementing symbol-like representations in neurally plausible models have been proposed. These approaches include binding through synchrony, “mesh” binding, and conjunctive binding. Recent theoretical work has suggested that most of these methods will not scale well, that is, that they cannot encode structured representations using any of the tens of thousands of terms in the adult lexicon without making implausible resource assumptions. Here, we empirically demonstrate that the biologically plausible structured representations employed in the Semantic Pointer Architecture approach (...) to modeling cognition do scale appropriately. Specifically, we construct a spiking neural network of about 2.5 million neurons that employs semantic pointers to successfully encode and decode the main lexical relations in WordNet, which has over 100,000 terms. In addition, we show that the same representations can be employed to construct recursively structured sentences consisting of arbitrary WordNet concepts, while preserving the original lexical structure. We argue that these results suggest that semantic pointers are uniquely well-suited to providing a biologically plausible account of the structured representations that underwrite human cognition. (shrink)

I argue that analogue mental representations possess a canonical decomposition into privileged constituents from which they compose. I motivate this suggestion, and rebut arguments to the contrary, through reflection on the approximate number system, whose representations are widely expected to have an analogue format. I then argue that arguments for the compositionality and constituent structure of these analogue representations generalize to other analogue mental representations posited in the human mind, such as those in early vision and visual imagery.

This paper offers both a theoretical and an experimental perspective on the relationship between connectionist and Classical (symbol-processing) models. Firstly, a serious flaw in Fodor and Pylyshyn’s argument against connectionism is pointed out: if, in fact, a part of their argument is valid, then it establishes a conclusion quite different from that which they intend, a conclusion which is demonstrably false. The source of this flaw is traced to an underestimation of the differences between localist and distributed representation. It has (...) been claimed that distributed representations cannot support systematic operations, or that if they can, then they will be mere implementations of traditional ideas. This paper presents experimental evidence against this conclusion: distributed representations can be used to support direct structure-sensitive operations, in a man- ner quite unlike the Classical approach. Finally, it is argued that even if Fodor and Pylyshyn’s argument that connectionist models of compositionality must be mere implementations were correct, then this would still not be a serious argument against connectionism as a theory of mind. (shrink)

Connectionist research first emerged in the 1940s. The first phase of connectionism attracted a certain amount of media attention, but scant philosophical interest. The phase came to an abrupt halt, due to the efforts of Minsky and Papert (1969), when they argued for the intrinsic limitations of the approach. In the mid-1980s connectionism saw a resurgence. This marked the beginning of the second phase of connectionist research. This phase did attract considerable philosophical attention. It was of philosophical interest, as it (...) offered a way of counteracting the conceptual ties to the philosophical traditions of atomism, rationalism, logic, nativism, rule realism and a concern with the role symbols play in human cognitive functioning, which was prevalent as a consequence of artificial intelligence research. The surge in philosophical interest waned, possibly in part due to the efforts of some traditionalists and the so-called black box problem. Most recently, what may be thought of as a third phase of connectionist research, based on so-called deep learning methods, is beginning to show some signs of again exciting philosophical interest. (shrink)

Advocates of dynamical systems theory (DST) sometimes employ revolutionary rhetoric. In an attempt to clarify how DST models differ from others in cognitive science, I focus on two issues raised by DST: the role for representations in mental models and the conception of explanation invoked. Two features of representations are their role in standing-in for features external to the system and their format. DST advocates sometimes claim to have repudiated the need for stand-ins in DST models, but I argue that (...) they are mistaken. Nonetheless, DST does offer new ideas as to the format of representations employed in cognitive systems. With respect to explanation, I argue that some DST models are better seen as conforming to the covering-law conception of explanation than to the mechanistic conception of explanation implicit in most cognitive science research. But even here, I argue, DST models are a valuable complement to more mechanistic cognitive explanations. (shrink)

The relation between logic and thought has long been controversial, but has recently influenced theorizing about the nature of mental processes in cognitive science. One prominent tradition argues that to explain the systematicity of thought we must posit syntactically structured representations inside the cognitive system which can be operated upon by structure sensitive rules similar to those employed in systems of natural deduction. I have argued elsewhere that the systematicity of human thought might better be explained as resulting from the (...) fact that we have learned natural languages which are themselves syntactically structured. According to this view, symbols of natural language are external to the cognitive processing system and what the cognitive system must learn to do is produce and comprehend such symbols. In this paper I pursue that idea by arguing that ability in natural deduction itself may rely on pattern recognition abilities that enable us to operate on external symbols rather than encodings of rules that might be applied to internal representations. To support this suggestion, I present a series of experiments with connectionist networks that have been trained to construct simple natural deductions in sentential logic. These networks not only succeed in reconstructing the derivations on which they have been trained, but in constructing new derivations that are only similar to the ones on which they have been trained. (shrink)

This paper reviews four significant advances on the feedforward architecture that has dominated discussions of connectionism. The first involves introducing modularity into networks by employing procedures whereby different networks learn to perform different components of a task, and a Gating Network determines which network is best equiped to respond to a given input. The second consists in the use of recurrent inputs whereby information from a previous cycle of processing is made available on later cycles. The third development involves developing (...) compressed representations of strings in which there is no longer an explicit encoding of the components but where information about the structure of the original string can be recovered and so is present functionally. The final advance entails using connectionist learning procedures not just to change weights in networks but to change the patterns used as inputs to the network. These advances significantly increase the usefulness of connectionist networks for modeling human cognitive performance by, among other things, providing tools for explaining the productivity and systematicity of some mental activities, and developing representations that are sensitive to the content they are to represent. (shrink)

This paper is a commentary on the target article by Lokendra Shastri & Venkat Ajjanagadde [S&A]: “From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony” in same issue of the journal, pp.417–451. -/- It puts S&A's temporal-synchrony binding method in a broader context, comments on notions of pointing and other ways of associating information - in both computers and connectionist systems - and mentions types of reasoning that are a challenge to (...) S&A's system (amongst many other connectionist systems, recognizing that S&A's is much more capable as regards reasoning than most other contemporary systems). (shrink)

Connectionist attention to variables has been too restricted in two ways. First, it has not exploited certain ways of doing without variables in the symbolic arena. One variable-avoidance method, that of logical combinators, is particularly well established there. Secondly, the attention has been largely restricted to variables in long-term rules embodied in connection weight patterns. However, short-lived bodies of information, such as sentence interpretations or inference products, may involve quantification. Therefore short-lived activation patterns may need to achieve the effect of (...) variables. The paper is mainly a theoretical analysis of some benefits and drawbacks of using logical combinators to avoid variables in short-lived connectionist encodings without loss of expressive power. The paper also includes a brief survey of some possible methods for avoiding variables other than by using combinators. (shrink)

Prior to the twentieth century, theories of knowledge were inherently perceptual. Since then, developments in logic, statis- tics, and programming languages have inspired amodal theories that rest on principles fundamentally different from those underlying perception. In addition, perceptual approaches have become widely viewed as untenable because they are assumed to implement record- ing systems, not conceptual systems. A perceptual theory of knowledge is developed here in the context of current cognitive science and neuroscience. During perceptual experience, association areas in the (...) brain capture bottom-up patterns of activation in sensory-motor areas. Later, in a top-down manner, association areas partially reactivate sensory-motor areas to implement perceptual symbols. The stor- age and reactivation of perceptual symbols operates at the level of perceptual components – not at the level of holistic perceptual expe- riences. Through the use of selective attention, schematic representations of perceptual components are extracted from experience and stored in memory (e.g., individual memories of green, purr, hot). As memories of the same component become organized around a com- mon frame, they implement a simulator that produces limitless simulations of the component (e.g., simulations of purr). Not only do such simulators develop for aspects of sensory experience, they also develop for aspects of proprioception (e.g., lift, run) and introspec- tion (e.g., compare, memory, happy, hungry). Once established, these simulators implement a basic conceptual system that represents types, supports categorization, and produces categorical inferences. These simulators further support productivity, propositions, and ab- stract concepts, thereby implementing a fully functional conceptual system. Productivity results from integrating simulators combinato- rially and recursively to produce complex simulations. Propositions result from binding simulators to perceived individuals to represent type-token relations. Abstract concepts are grounded in complex simulations of combined physical and introspective events. Thus, a per- ceptual theory of knowledge can implement a fully functional conceptual system while avoiding problems associated with amodal sym- bol systems. Implications for cognition, neuroscience, evolution, development, and artificial intelligence are explored. (shrink)

Various defenses of amodal symbol systems are addressed, including amodal symbols in sensory-motor areas, the causal theory of concepts, supramodal concepts, latent semantic analysis, and abstracted amodal symbols. Various aspects of perceptual symbol systems are clarified and developed, including perception, features, simulators, category structure, frames, analogy, introspection, situated action, and development. Particular attention is given to abstract concepts, language, and computational mechanisms.

Fodor and Pylyshyn's critique of connectionism has posed a challenge to connectionists: Adequately explain such nomological regularities as systematicity and productivity without postulating a "language of thought" (LOT). Some connectionists like Smolensky took the challenge very seriously, and attempted to meet it by developing models that were supposed to be non-classical. At the core of these attempts lies the claim that connectionist models can provide a representational system with a combinatorial syntax and processes sensitive to syntactic structure. They are not (...) implementation models because, it is claimed, the way they obtain syntax and structure sensitivity is not "concatenative," hence "radically different" from the way classicists handle them. In this paper, I offer an analysis of what it is to physically satisfy/realize a formal system. In this context, I examine the minimal truth-conditions of LOT Hypothesis. From my analysis it will follow that concatenative realization of formal systems is irrelevant to LOTH since the very notion of LOT is indifferent to such an implementation level issue as concatenation. I will conclude that to the extent to which they can explain the law-like cognitive regularities, a certain class of connectionist models proposed as radical alternatives to the classical LOT paradigm will in fact turn out to be LOT models, even though new and potentially very exciting ones. (shrink)

Despite the considerable attention that the systematicity argument has enjoyed, it is worthwhile examining the argument within the context of similar explanatory arguments from the history of science. This kind of analysis helps show that Connectionism, qua Connectionism, really does not have an explanation of systematicity. Second, and more surprisingly, one finds that the systematicity argument sets such a high explanatory standard that not even Classicism can explain the systematicity of thought.

Despite the considerable attention that the systematicity argument has enjoyed, it is worthwhile examining the argument within the context of similar explanatory arguments from the history of science. This kind of analysis helps show that Connectionism, qua Connectionism, really does not have an explanation of systematicity. Second, and more surprisingly, one finds that the systematicity argument sets such a high explanatory standard that not even Classicism can explain the systematicity of thought.

The prominence of Bayesian modeling of cognition has increased recently largely because of mathematical advances in specifying and deriving predictions from complex probabilistic models. Much of this research aims to demonstrate that cognitive behavior can be explained from rational principles alone, without recourse to psychological or neurological processes and representations. We note commonalities between this rational approach and other movements in psychology – namely, Behaviorism and evolutionary psychology – that set aside mechanistic explanations or make use of optimality assumptions. Through (...) these comparisons, we identify a number of challenges that limit the rational program's potential contribution to psychological theory. Specifically, rational Bayesian models are significantly unconstrained, both because they are uninformed by a wide range of process-level data and because their assumptions about the environment are generally not grounded in empirical measurement. The psychological implications of most Bayesian models are also unclear. Bayesian inference itself is conceptually trivial, but strong assumptions are often embedded in the hypothesis sets and the approximation algorithms used to derive model predictions, without a clear delineation between psychological commitments and implementational details. Comparing multiple Bayesian models of the same task is rare, as is the realization that many Bayesian models recapitulate existing (mechanistic level) theories. Despite the expressive power of current Bayesian models, we argue they must be developed in conjunction with mechanistic considerations to offer substantive explanations of cognition. We lay out several means for such an integration, which take into account the representations on which Bayesian inference operates, as well as the algorithms and heuristics that carry it out. We argue this unification will better facilitate lasting contributions to psychological theory, avoiding the pitfalls that have plagued previous theoretical movements. (shrink)

Complex systems are to be seen as typically having multiple levels of organization. For instance, in the behavioural and cognitive sciences, there has been a long lasting trend, promoted by the seminal work of David Marr, putting focus on three distinct levels of analysis: the computational level, accounting for the What and Why issues, the algorithmic and the implementational levels specifying the How problem. However, the tremendous developments in neuroscience knowledge about processes at different scales of organization together with the (...) complexity of today cognitive theories suggest that there will hardly be only three levels of explanation. Instead, there will be many different degrees of commitments corresponding to the different granularities—from high-level (behavioural) models to low-level (neural and molecular) models of the cognitive research program. For instance, Bayesian approaches, that are usually advocated for formalizing Marr’s computational level and rational behaviour, have even been adopted to model synaptic plasticity and axon guidance by molecular gradients. As a result, we can consider the behavioural scientist as dealing with models at a multiplicity of levels. The purpose of this Research Topic in Frontiers in Theoretical and Philosophical Psychology is to promote an approach to the role of the levels and explanation and models which is of interest for cognitive scientists, neuroscientists, psychologists, behavioural scientists, and philosophers of science. (shrink)

A comprehensive introduction to the Language of Though Hypothesis (LOTH) accessible to general audiences. LOTH is an empirical thesis about thought and thinking. For their explication, it postulates a physically realized system of representations that have a combinatorial syntax (and semantics) such that operations on representations are causally sensitive only to the syntactic properties of representations. According to LOTH, thought is, roughly, the tokening of a representation that has a syntactic (constituent) structure with an appropriate semantics. Thinking thus consists in (...) syntactic operations defined over representations. Most of the arguments for LOTH derive their strength from their ability to explain certain empirical phenomena like productivity, systematicity of thought and thinking. (shrink)

Figure 1: A pr ototyp ical exa mple of a three-layer feed forward network, used by Plunkett and M archm an (1 991 ) to simulate learning the past-tense of En glish verbs. The inpu t units encode representations of the three phonemes of the present tense of the artificial words used in this simulation. Th e netwo rk is trained to produce a representation of the phonemes employed in the past tense form and the suffix (/d/, /ed/, or /t/) (...) used on regular verbs. To run the network, each input unit is assigned an activation value o f 0 or 1 , dep ending on whethe r the featu re is present or not. Eac h input unit is conne cted to each of the 30 hidden units by a we ighted conn ection and p rovid es an inp ut to each hidden unit equal to the product of the input unit's activation and the weight. Each hidd en unit's activation is then determined by summing ov er the va lues co ming fro m each inp ut unit to deter mine a netinput, and then applying a non-linear function (e.g., the logistic function 1/(1+e^netinput). Th is whole proced ure is. (shrink)

Cognitive architectures are unified theories of cognition that take the form of computational formalisms. They support computational models that collectively account for large numbers of empirical regularities using small numbers of computational mechanisms. Empirical coverage and parsimony are the most prominent criteria by which architectures are designed and evaluated, but they are not the only ones. This paper considers three additional criteria that have been comparatively undertheorized. (a) Successful architectures possess subjective and intersubjective meaning, making cognition comprehensible to individual cognitive (...) scientists and organizing groups of like-minded cognitive scientists into genuine communities. (b) Successful architectures provide idioms that structure the design and interpretation of computational models. (c) Successful architectures are strange: They make provocative, often disturbing, and ultimately compelling claims about human information processing that demand evaluation. (shrink)

According to the dominant computational approach in cognitive science, cognitive agents are digital computers; according to the alternative approach, they are dynamical systems. This target article attempts to articulate and support the dynamical hypothesis. The dynamical hypothesis has two major components: the nature hypothesis (cognitive agents are dynamical systems) and the knowledge hypothesis (cognitive agents can be understood dynamically). A wide range of objections to this hypothesis can be rebutted. The conclusion is that cognitive systems may well be dynamical systems, (...) and only sustained empirical research in cognitive science will determine the extent to which that is true. (shrink)

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine, a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can encode contextual (...) information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models. We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain. (shrink)

Mental representations have continuous as well as discrete, combinatorial properties. For example, while predominantly discrete, phonological representations also vary continuously; this is reflected by gradient effects in instrumental studies of speech production. Can an integrated theoretical framework address both aspects of structure? The framework we introduce here, Gradient Symbol Processing, characterizes the emergence of grammatical macrostructure from the Parallel Distributed Processing microstructure (McClelland, Rumelhart, & The PDP Research Group, 1986) of language processing. The mental representations that emerge, Distributed Symbol Systems, (...) have both combinatorial and gradient structure. They are processed through Subsymbolic Optimization–Quantization, in which an optimization process favoring representations that satisfy well-formedness constraints operates in parallel with a distributed quantization process favoring discrete symbolic structures. We apply a particular instantiation of this framework, λ-Diffusion Theory, to phonological production. Simulations of the resulting model suggest that Gradient Symbol Processing offers a way to unify accounts of grammatical competence with both discrete and continuous patterns in language performance. (shrink)

In this article, I survey the integrated connectionist/symbolic (ICS) cognitive architecture in which higher cognition must be formally characterized on two levels of description. At the microlevel, parallel distributed processing (PDP) characterizes mental processing; this PDP system has special organization in virtue of which it can be characterized at the macrolevel as a kind of symbolic computational system. The symbolic system inherits certain properties from its PDP substrate; the symbolic functions computed constitute optimization of a well-formedness measure called Harmony. The (...) most important outgrowth of the ICS research program is optimality theory (Prince & Smolensky, 1993/2004), an optimization based grammatical theory that provides a formal theory of cross-linguistic typology. Linguistically, Harmony maximization corresponds to minimization of markedness or structural ill-formedness. Cognitive explanation in ICS requires the collaboration of symbolic and connectionist principles. ICS is developed in detail in Smolensky and Legendre (2006a); this article is a précis of and guide to those volumes. (shrink)