_role, especially in learning, and through devising hybrid neural network models that (in a qualitative manner) approxi-_ _mate characteristics of human consciousness. In doing so, the paper examines explicit and implicit learning in a variety_ _of psychological experiments and delineates the conscious/unconscious distinction in terms of the two types of learning_ _and their respective products. The distinctions are captured in a two-level action-based model C_larion_. Some funda-_ _mental theoretical issues are also clari?ed with the help of the model. Comparisons with (...) existing models of conscious-_. (shrink)

referred to as implicit learning (Reber, 1989). Implicit learning contrasts with explicit learning (exhibited for.

Many studies of language, whether in philosophy, linguistics, or psychology, have focused on highly developed human languages. In their highly developed forms, such as are employed in scientific discourse, languages have a unique set of properties that have been the focus of much attention. For example, descriptive sentences in a language have the property of being "true" or "false," and words of a language have senses and referents. Sentences in a language are structured in accord with complex syntactic rules. Theorists (...) focusing on language are naturally led to ask questions such as what constitutes the meanings of words and sentences and how are the principles of syntax encoded in the heads of language users. While there is an important function for inquiries into the highly developed forms of these cultural products (Abrahamsen, 1987), such a focus can be quite misleading when we want to explain how these products have arisen or the human capacity to use language. The problem is that focusing on its most developed forms makes linguistic ability seem to be a _sui generis_ phenomenon, not related to, and hence not explicable in terms of other cognitive capacities. Chomsky's (1980) postulation of a specific language module equipped with specialized resources needed to process language and possessed only by hum ans is not a surprising result. (shrink)

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine, a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can encode contextual (...) information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models. We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain. (shrink)

Consciousness remains a mystery—“a phenomenon that people do not know how to think about—yet” (Dennett, , p. 21). Here, I consider how the connectionist perspective on information processing may help us progress toward the goal of understanding the computational principles through which conscious and unconscious processing differ. I begin by delineating the conceptual challenges associated with classical approaches to cognition insofar as understanding unconscious information processing is concerned, and to highlight several contrasting computational principles that are constitutive of the connectionist (...) approach. This leads me to suggest that conscious and unconscious processing are fundamentally connected, that is, rooted in the very same computational principles. I further develop a perspective according to which the brain continuously and unconsciously learns to redescribe its own activity itself based on constant interaction with itself, with the world, and with other minds. The outcome of such interactions is the emergence of internal models that are metacognitive in nature and that function so as to make it possible for an agent to develop a (limited, implicit, practical) understanding of itself. In this light, plasticity and learning are constitutive of what makes us conscious, for it is in virtue of our own experiences with ourselves and with other people that our mental life acquires its subjective character. The connectionist framework continues to be uniquely positioned in the Cognitive Sciences to address the challenge of identifying what one could call the “computational correlates of consciousness” (Mathis & Mozer, ) because it makes it possible to focus on the mechanisms through which information processing takes place. (shrink)

Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and “simple” homogeneous computing elements. Using just these three features one can construct surprisingly elegant and powerful models of (...) memory, perception, motor control, categorization, and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks. (shrink)

In our opinion, the discontinuity between extant human and nonhuman minds is much broader and deeper than most researchers admit. We are happy to report that Evans & Levinson's (E&L's) target article strongly corroborates our unpopular hypothesis, and that the comparative evidence, in turn, bolsters E&L's provocative argument. Both a Universal Grammar and the “mental continuity” between human and nonhuman minds turn out to be modern myths.

Studies based on the Contemporary Theory of Metaphor (Lakoff & Johnson, 1980, 1999) usually identify conceptual metaphors by analysing linguistic expressions and creating a post hoc interpretation of the findings. This method has been questioned for a variety of reasons, including its circularity (Müller, 2008), lack of falsifiability (Vervaeke & Kennedy, 1996, 2004), and lack of predictive power (Ritchie, 2003). It has been argued that CTM requires additional constraints to improve its applicability for empirical research (Gibbs, 2011; Ritchie, 2003). This (...) paper sets out to propose additional methodological structure to CTM, a theory of conceptual metaphor in which much of abstract thought is generated by metaphorical mapping from embodied experience (Ruiz de Mendoza Ibáñez & Pérez Hernández, 2011). Introducing Objectification Theory defined by Szwedek (2002, 2007, 2011) ameliorates a number of methodological issues in CTM. First, the embodiment claim of CTM in its current form cannot be empirically proven incorrect (Vervaeke & Kennedy, 2004) as any mapping within it is possible (although only some actually happen). Objectification introduces pre-metaphorical structure of the kind suggested by Glucksberg (2001), constraining source and target domain selection, predicting which mappings are more likely to happen. Second, while many claim that metaphors trace back to a literal concept based on embodied physical experience (Gibbs, Costa Lima, & Francozo, 2004), it is unclear what criteria are used to define „physical”. Metaphorical domains are often described using the terms „abstract” and „concrete”, Objectification proposes objective criteria for deciding whether a concept is experientially grounded. Finally, Objectification provides grounds for introducing a hierarchical framework for metaphor typology, preventing post-hoc addition of metaphor types if and when suitable for the explanation of a phenomenon; thus increasing the consistency of the CTM framework, both internally and with other cognitive science disciplines. This thesis focuses on providing evidence for Objectification Theory and identifying its applications in metaphor and gesture research. (shrink)

The entropy-reduction hypothesis claims that the cognitive processing difficulty on a word in sentence context is determined by the word's effect on the uncertainty about the sentence. Here, this hypothesis is tested more thoroughly than has been done before, using a recurrent neural network for estimating entropy and self-paced reading for obtaining measures of cognitive processing load. Results show a positive relation between reading time on a word and the reduction in entropy due to processing that word, supporting the entropy-reduction (...) hypothesis. Although this effect is independent from the effect of word surprisal, we find no evidence that these two measures correspond to cognitively distinct processes. (shrink)

Does connectionism spell doom for folk psychology? I examine the proposal that cognitive representational states such as beliefs can play no role if connectionist models - - interpreted as radical new cognitive theories -- take hold and replace other cognitive theories. Though I accept that connectionist theories are radical theories that shed light on cognition, I reject the conclusion that neural networks do not represent. Indeed, I argue that neural networks may actually give us a better working notion of cognitive (...) representational states such as beliefs, and in so doing give us a better understanding of how these states might be instantiated in neural wetware. (shrink)

The study of human intelligence was once dominated by symbolic approaches, but over the last 30 years an alternative approach has arisen. Symbols and processes that operate on them are often seen today as approximate characterizations of the emergent consequences of sub- or nonsymbolic processes, and a wide range of constructs in cognitive science can be understood as emergents. These include representational constructs (units, structures, rules), architectural constructs (central executive, declarative memory), and developmental processes and outcomes (stages, sensitive periods, neurocognitive (...) modules, developmental disorders). The greatest achievements of human cognition may be largely emergent phenomena. It remains a challenge for the future to learn more about how these greatest achievements arise and to emulate them in artificial systems. (shrink)

Leech et al.'s model proposes representing relations as primed transformations rather than as structured representations (explicit representations of relations and their roles dynamically bound to fillers). However, this renders the model unable to explain several developmental trends (including relational integration and all changes not attributable to growth in relational knowledge). We suggest looking to an alternative computational model that learns structured representations from examples.

Connectionism and phenomenology can mutually inform and mutually constrain each other. In this manifesto I outline an approach to consciousness based on distinctions developed by connectionists. Two core identities are central to a connectionist theory of consciouness: conscious states of mind are identical to occurrent activation patterns of processing units; and the variable dispositional strengths on connections between units store latent and unconscious information. Within this broad framework, a connectionist model of consciousness succeeds according to the degree of correspondence between (...) the content of human consciousness (the world as it is experienced) and the interpreted content of the network. Constitutive self-awareness and reflective self-awareness can be captured in a model through its ability to respond to self-reflexive information, identify self-referential categories, and process information in the absence of simultaneous input. The qualitative feel of sensation appears in a model as states of activation that are not fully discriminated by later processing. Connectionism also uniquely explains several specific features of experience. The most important of these is the superposition of information in consciousness — our ability to perceive more than meets the eye, and to apprehend complex categorical and temporal information in a single highly-cognized glance. This superposition in experience matches a superposition of representational content in distributed representations. (shrink)

Comparing the relative sensitivity of direct and indirect measures of learning is proposed as the best way to provide evidence for unconscious learning when both conceptual and operative definitions of awareness are lacking. This approach was first proposed by Reingold & Merikle (1988) in the context of subliminal perception. In this paper, we apply it to a choice reaction time task in which the material is generated based on a probabilistic finite-state grammar (Cleeremans, 1993). We show (1) that participants progressively (...) learn about the statistical structure of the stimulus material over training with the choice reaction time task, and (2) that they can use some of this knowledge to predict the location of the next stimulus in a subsequent “generation” task. However, detailed partial correlational analyses of the correspondence between performance during the reaction time task and the statistical structure of the training material showed that large effects remained even when controlling for explicit knowledge as assessed by the generation task. Hence we conclude (1) that at least some of the knowledge expressed through reaction time performance can not be characterized as conscious, and (2) that even when associations are found at a global level of analysis, dissociations can still be obtained when more detailed analyses are conducted. Finally, we also show that participants are limited in the depth of the contingencies they can learn about, and that these limitations are shared by the Simple Recurrent Network model of Cleeremans & McClelland (1991). (shrink)

This paper proposes several recurrent neural network-based models for recognizing requisite and effectuation parts in Legal Texts. Firstly, we propose a modification of BiLSTM-CRF model that allows the use of external features to improve the performance of deep learning models in case large annotated corpora are not available. However, this model can only recognize RE parts which are not overlapped. Secondly, we propose two approaches for recognizing overlapping RE parts including the cascading approach which uses the sequence of BiLSTM-CRF models (...) and the unified model approach with the multilayer BiLSTM-CRF model and the multilayer BiLSTM-MLP-CRF model. Experimental results on two Japan law RRE datasets demonstrated advantages of our proposed models. For the Japanese National Pension Law dataset, our approaches obtained an \ score of 93.27% and exhibited a significant improvement compared to previous approaches. For the Japan Civil Code RRE dataset which is written in English, our approaches produced an \ score of 78.24% in recognizing RE parts that exhibited a significant improvement over strong baselines. In addition, using external features and in-domain pre-trained word embeddings also improved the performance of RRE systems. (shrink)

Our response takes advantage of the wide-ranging commentary to clarify some aspects of our original proposal and augment others. We argue against the generative critics of our coevolutionary program for the language sciences, defend the use of close-to-surface models as minimizing cross-linguistic data distortion, and stress the growing role of stochastic simulations in making generalized historical accounts testable. These methods lead the search for general principles away from idealized representations and towards selective processes. Putting cultural evolution central in understanding language (...) diversity makes learning fundamental in the cognition of language: increasingly powerful models of general learning, paired with channelled caregiver input, seem set to manage language acquisition without recourse to any innate “universal grammar.” Understanding why human language has no clear parallels in the animal world requires a cross-species perspective: crucial ingredients are vocal learning (for which there are clear non-primate parallels) and an intention-attributing cognitive infrastructure that provides a universal base for language evolution. We conclude by situating linguistic diversity within a broader trend towards understanding human cognition through the study of variation in, for example, human genetics, neurocognition, and psycholinguistic processing. (shrink)

Our target article proposed that language production and comprehension are interwoven, with speakers making predictions of their own utterances and comprehenders making predictions of other people's utterances at different linguistic levels. Here, we respond to comments about such issues as cognitive architecture and its neural basis, learning and development, monitoring, the nature of forward models, communicative intentions, and dialogue.

We must rethink our assessment of Hume’s theory of probabilistic inference. Hume scholars have traditionally dismissed his naturalistic explanation of how we make inferences under conditions of uncertainty; however, psychological experiments and computer models from cognitive science provide substantial support for Hume’s account. Hume’s theory of probabilistic inference is far from obsolete or outdated; on the contrary, it stands at the leading edge of our contemporary science of the mind.

The study addresses the cyclically temporal aspect of sequence recognition, storage and recall using the Recurrent Oscillatory Self-Organizing Map (ROSOM), first introduced by Kaipainen, Papadopoulos and Karhu (1997). The unique solution of the network is that oscillatory States are assigned to network units, corresponding to their `readiness-to-fire''. The ROSOM is a categorizer, a temporal sequence storage system and a periodicity detector designed for use in an ambiguous cyclically repetitive environment. As its external input, the model accepts a multidimensional stream of (...) environment-describing feature configurations with implicit periodicities. The output of the model is one or a few closed cycles abstracted from such a stream, mapped as trajectories on a two-dimensional sheet with an organization reminiscent of multi-dimensional scaling. The model''s capabilities are explored with a variety of workbench data. (shrink)

Figure 1: A pr ototyp ical exa mple of a three-layer feed forward network, used by Plunkett and M archm an (1 991 ) to simulate learning the past-tense of En glish verbs. The inpu t units encode representations of the three phonemes of the present tense of the artificial words used in this simulation. Th e netwo rk is trained to produce a representation of the phonemes employed in the past tense form and the suffix (/d/, /ed/, or /t/) (...) used on regular verbs. To run the network, each input unit is assigned an activation value o f 0 or 1 , dep ending on whethe r the featu re is present or not. Eac h input unit is conne cted to each of the 30 hidden units by a we ighted conn ection and p rovid es an inp ut to each hidden unit equal to the product of the input unit's activation and the weight. Each hidd en unit's activation is then determined by summing ov er the va lues co ming fro m each inp ut unit to deter mine a netinput, and then applying a non-linear function (e.g., the logistic function 1/(1+e^netinput). Th is whole proced ure is. (shrink)

A large body of psycholinguistic research demonstrates that both language processing and language acquisition are sensitive to the distributions of linguistic constructions in usage. Here we investigate how statistical distributions at different linguistic levels – morphological and lexical, and phrasal – contribute to the ease with which morphosyntax is processed and produced by second language learners. We analyze Chinese ESL learners’ knowledge of four English inflectional morphemes: -ed, -ing, and third-person -s on verbs, and plural -s on nouns. In Elicited (...) Imitation Tasks, participants listened to length- and difficulty-matched sentences each containing one target morpheme and typed the whole sentence as accurately as they could after a short delay. Experiment 1 investigated lexical and morphemic levels, testing the hypotheses that a morpheme is expected to be more easily processed when it is highly available, and highly reliable. Thirty sentences were made for each morpheme, divided into three Availability-Reliability Distribution groups on the basis of corpus analysis in the Corpus of Contemporary American English : 10 target words high in availability, 10 high in reliability, and 10 low in both reliability and availability. Responses were scored on whether the target morpheme was accurately reproduced given the provision of the correct lemma. A generalized linear mixed-effects logit model revealed fixed effects of morpheme type, availability, and reliability on the accuracy of morpheme provision. There were no effects of lemma frequency. Experiment 2 successfully replicated these results and extended the investigation to explore phrasal formulaicity by manipulating the frequency of the four-word strings in which the morpheme was embedded. GLMMs replicated the effects of word-form availability and reliability and additionally revealed independent phrase-superiority effects where morphemes were better reproduced in contexts of higher string-frequency. Taken together, these findings demonstrate that morpheme acquisition reflects the distributional properties of learners’ experience and the mappings therein between lexis, morphology, phraseology, and semantics. These conclusions support an emergentist view of the statistical symbolic learning of morphology where language acquisition involves the satisfaction of competing constraints across multiple grain-sizes of units. (shrink)

Ten years ago, researchers using event-related brain potentials to study language comprehension were puzzled by what looked like a Semantic Illusion: Semantically anomalous, but structurally well-formed sentences did not affect the N400 component—traditionally taken to reflect semantic integration—but instead produced a P600 effect, which is generally linked to syntactic processing. This finding led to a considerable amount of debate, and a number of complex processing models have been proposed as an explanation. What these models have in common is that they (...) postulate two or more separate processing streams, in order to reconcile the Semantic Illusion and other semantically induced P600 effects with the traditional interpretations of the N400 and the P600. Recently, however, these multi-stream models have been called into question, and a simpler single-stream model has been proposed. According to this alternative model, the N400 component reflects the retrieval of word meaning from semantic memory, and the P600 component indexes the integration of this meaning into the unfolding utterance interpretation. In the present paper, we provide support for this “Retrieval–Integration ” account by instantiating it as a neurocomputational model. This neurocomputational model is the first to successfully simulate the N400 and P600 amplitude in language comprehension, and simulations with this model provide a proof of concept of the single-stream RI account of semantically induced patterns of N400 and P600 modulations. (shrink)

Children learn their native language by exposure to their linguistic and communicative environment, but apparently without requiring that their mistakes be corrected. Such learning from “positive evidence” has been viewed as raising “logical” problems for language acquisition. In particular, without correction, how is the child to recover from conjecturing an over-general grammar, which will be consistent with any sentence that the child hears? There have been many proposals concerning how this “logical problem” can be dissolved. In this study, we review (...) recent formal results showing that the learner has sufficient data to learn successfully from positive evidence, if it favors the simplest encoding of the linguistic input. Results include the learnability of linguistic prediction, grammaticality judgments, language production, and form-meaning mappings. The simplicity approach can also be “scaled down” to analyze the learnability of specific linguistic constructions, and it is amenable to empirical testing as a framework for describing human language acquisition. (shrink)

Despite its strengths, Leech et al.'s model fails to address the important benefits that derive from self-explanation and task feedback in analogical reasoning development. These components encourage explicit, self-reflective processes that do not necessarily link to knowledge accretion. We wonder, therefore, what mechanisms can be included within a connectionist framework to model self-reflective involvement and its beneficial consequences.