Memory is fleeting. New material rapidly obliterates previous material. How, then, can the brain deal successfully with the continual deluge of linguistic input? We argue that, to deal with this “Now-or-Never” bottleneck, the brain must compress and recode linguistic input as rapidly as possible. This observation has strong implications for the nature of language processing: (1) the language system must “eagerly” recode and compress linguistic input; (2) as the bottleneck recurs at each new representational level, the language system must build (...) a multilevel linguistic representation; and (3) the language system must deploy all available information predictively to ensure that local linguistic ambiguities are dealt with “Right-First-Time”; once the original input is lost, there is no way for the language system to recover. This is “Chunk-and-Pass” processing. Similarly, language learning must also occur in the here and now, which implies that language acquisition is learning to process, rather than inducing, a grammar. Moreover, this perspective provides a cognitive foundation for grammaticalization and other aspects of language change. Chunk-and-Pass processing also helps explain a variety of core properties of language, including its multilevel representational structure and duality of patterning. This approach promises to create a direct relationship between psycholinguistics and linguistic theory. More generally, we outline a framework within which to integrate often disconnected inquiries into language processing, language acquisition, and language change and evolution. (shrink)

The default view in the epistemology of forgetting is that human memory would be epistemically better if we were not so susceptible to forgetting—that forgetting is in general a cognitive vice. In this paper, I argue for the opposed view: normal human forgetting—the pattern of forgetting characteristic of cognitively normal adult human beings—approximates a virtue located at the mean between the opposed cognitive vices of forgetting too much and remembering too much. I argue, first, that, for any finite cognizer, a (...) certain pattern of forgetting is necessary if her memory is to perform its function well. I argue, second, that, by eliminating clutter from her memory store, this pattern of forgetting improves the overall shape of the subject’s total doxastic state. I conclude by reviewing work in psychology which suggests that normal human forgetting approximates this virtuous pattern of forgetting. (shrink)

From its genesis in the 1960s, the focus of inquiry in neuroscience has been on the cellular and molecular processes underlying neural activity. In this pursuit neuroscience has been enormously successful. Like any successful scientific inquiry, initial successes have raised new questions that inspire ongoing research. While there is still much that is not known about the molecular processes in brains, a great deal of very important knowledge has been secured, especially in the last 50 years. It has also attracted (...) the attention of a number of philosophers, some of whom have viewed it as evidence for a ruthlessly reductionistic program that will eventually explain how mental processes are performed in the brain in purely molecular terms. As neuroscience developed, however, there emerged a smaller group of researchers who focused on systems, behavioral, and cognitive neuroscience. These investigators have also made impressive advances in the last 50 years and they have been the focus of an even larger group of philosophers, who have appealed to systems level understanding of the brain as providing the appropriate point of connection to the information processing accounts advanced in psychology. (shrink)

Figure 1: A pr ototyp ical exa mple of a three-layer feed forward network, used by Plunkett and M archm an (1 991 ) to simulate learning the past-tense of En glish verbs. The inpu t units encode representations of the three phonemes of the present tense of the artificial words used in this simulation. Th e netwo rk is trained to produce a representation of the phonemes employed in the past tense form and the suffix (/d/, /ed/, or /t/) (...) used on regular verbs. To run the network, each input unit is assigned an activation value o f 0 or 1 , dep ending on whethe r the featu re is present or not. Eac h input unit is conne cted to each of the 30 hidden units by a we ighted conn ection and p rovid es an inp ut to each hidden unit equal to the product of the input unit's activation and the weight. Each hidd en unit's activation is then determined by summing ov er the va lues co ming fro m each inp ut unit to deter mine a netinput, and then applying a non-linear function (e.g., the logistic function 1/(1+e^netinput). Th is whole proced ure is. (shrink)

Philosophy and Memory Traces defends two theories of autobiographical memory. One is a bewildering historical view of memories as dynamic patterns in fleeting animal spirits, nervous fluids which rummaged through the pores of brain and body. The other is new connectionism, in which memories are 'stored' only superpositionally, and reconstructed rather than reproduced. Both models, argues John Sutton, depart from static archival metaphors by employing distributed representation, which brings interference and confusion between memory traces. Both raise urgent issues about control (...) of the personal past, and about relations between self and body. Sutton demonstrates the role of bizarre body fluids in moral physiology, as philosophers from Descartes and Locke to Coleridge struggled to control their own innards and impose cognitive discipline on 'the phantasmal chaos of association'. Going on to defend connectionism against Fodor and critics of passive mental representations, he shows how problems of the self are implicated in cognitive science. (shrink)

Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and “simple” homogeneous computing elements. Using just these three features one can construct surprisingly elegant and powerful models of (...) memory, perception, motor control, categorization, and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks. (shrink)

We argue that atomistic learning?learning that requires training only on a novel item to be learned?is problematic for networks in which every weight is available for change in every learning situation. This is potentially significant because atomistic learning appears to be commonplace in humans and most non-human animals. We briefly review various proposed fixes, concluding that the most promising strategy to date involves training on pseudo-patterns along with novel items, a form of learning that is not strictly atomistic, but which (...) looks very much like it ?from the outside? (shrink)

Starting from Marr's ideas about levels of explanation, a theory of the data structures and access processes in human memory is demonstrated on 10 tasks. Functional characteristics of human memory are captured implementation-independently. Our theory generates a multidimensional task classification subsuming existing classifications such as the distinction between tasks that are implicit versus explicit, data driven versus conceptually driven, and simple associative (two-way bindings) versus higher order (threeway bindings), providing a broad basis for new experiments. The formal language clarifies the (...) binding problem in episodic memory, the role of input pathways in both episodic and semantic (lexical) memory, the importance of the input set in episodic memory, and the ubiquitous calculation of an intersection in theories of episodic and lexical access. (shrink)

This article argues that rapid eye movement (REM) dreaming is elaborative encoding for episodic memories. Elaborative encoding in REM can, at least partially, be understood through ancient art of memory (AAOM) principles: visualization, bizarre association, organization, narration, embodiment, and location. These principles render recent memories more distinctive through novel and meaningful association with emotionally salient, remote memories. The AAOM optimizes memory performance, suggesting that its principles may predict aspects of how episodic memory is configured in the brain. Integration and segregation (...) are fundamental organizing principles in the cerebral cortex. Episodic memory networks interconnect profusely within the cortex, creating omnidirectional “landmark” junctions. Memories may be integrated at junctions but segregated along connecting network paths that meet at junctions. Episodic junctions may be instantiated during non–rapid eye movement (NREM) sleep after hippocampal associational function during REM dreams. Hippocampal association involves relating, binding, and integrating episodic memories into a mnemonic compositional whole. This often bizarre, composite image has not been present to the senses; it is not “real” because it hyperassociates several memories. During REM sleep, on the phenomenological level, this composite image is experienced as a dream scene. A dream scene may be instantiated as omnidirectional neocortical junction and retained by the hippocampus as an index. On episodic memory retrieval, an external stimulus (or an internal representation) is matched by the hippocampus against its indices. One or more indices then reference the relevant neocortical junctions from which episodic memories can be retrieved. Episodic junctions reach a processing (rather than conscious) level during normal wake to enable retrieval. If this hypothesis is correct, the stuff of dreams is the stuff of memory. (shrink)

Rumelhart and McClelland's chapter about learning the past tense created a degree of controversy extraordinary even in the adversarial culture of modern science. It also stimulated a vast amount of research that advanced the understanding of the past tense, inflectional morphology in English and other languages, the nature of linguistic representations, relations between language and other phenomena such as reading and object recognition, the properties of artificial neural networks, and other topics. We examine the impact of the Rumelhart and McClelland (...) model with the benefit of 25 years of hindsight. It is not clear who “won” the debate. It is clear, however, that the core ideas that the model instantiated have been assimilated into many areas in the study of language, changing the focus of research from abstract characterizations of linguistic competence to an emphasis on the role of the statistical structure of language in acquisition and processing. (shrink)

We present a model of similarity‐based retrieval that attempts to capture three seemingly contradictory psychological phenomena: (a) structural commonalities are weighed more heavily than surface commonalities in similarity judgments for items in working memory; (b) in retrieval, superficial similarity is more important than structural similarity; and yet (c) purely structural (analogical) remindings e sometimes experienced. Our model, MAC/FAC, explains these phenomena in terms of a two‐stage process. The first stage uses a computationally cheap, non‐structural matcher to filter candidate long‐term memory (...) items. It uses content vectors, a redundant encoding of structured representations whose dot product estimates how well the corresponding structural representations will match. The second stage uses SME (structure‐mapping engine) to compute structural matches on the handful of items found by the first stage. We show the utility of the MAC/FAC model through a series of computational experiments: (a) We demonstrate that MAC/FAC can model patterns of access found in psychological data; (b) we argue via sensitivity analyses that these simulation results rely on the theory; and (c) we compare the performance of MAC/FAC with ARCS, an alternate model of similarity‐based retrieval, and demonstrate that MAC/FAC explains the data better than ARCS. Finally, we discuss limitations and possible extensions of the model, relationships with other recent retrieval models, and place MAC/FAC in the context of other recent work on the nature of similarity. (shrink)

Neural network models of memory are notorious for catastrophic interference: Old items are forgotten as new items are memorized (French, 1999; McCloskey & Cohen, 1989). While working memory (WM) in human adults shows severe capacity limitations, these capacity limitations do not reflect neural network style catastrophic interference. However, our ability to quickly apprehend the numerosity of small sets of objects (i.e., subitizing) does show catastrophic capacity limitations, and this subitizing capacity and WM might reflect a common capacity. Accordingly, computational investigations (...) (Knops, Piazza, Sengupta, Eger & Melcher, 2014; Sengupta, Surampudi & Melcher, 2014) suggest that mutual inhibition among neurons can explain both kinds of capacity limitations as well as why our ability to estimate the numerosity of larger sets is limited according to a Weber ratio signature. Based on simulations with a saliency map‐like network and mathematical proofs, we provide three results. First, mutual inhibition among neurons leads to catastrophic interference when items are presented simultaneously. The network can remember a limited number of items, but when more items are presented, the network forgets all of them. Second, if memory items are presented sequentially rather than simultaneously, the network remembers the most recent items rather than forgetting all of them. Hence, the tendency in WM tasks to sequentially attend even to simultaneously presented items might not only reflect attentional limitations, but also an adaptive strategy to avoid catastrophic interference. Third, the mean activation level in the network can be used to estimate the number of items in small sets, but it does not accurately reflect the number of items in larger sets. Rather, we suggest that the Weber ratio signature of large number discrimination emerges naturally from the interaction between the limited precision of a numeric estimation system and a multiplicative gain control mechanism. (shrink)