Neural Networks and Connectionism

Despite its centrality in the philosophy of cognitive science, there has been little prior philosophical work engaging with the notion of representation in contemporary NLP practice. This paper attempts to fill that lacuna: drawing on ideas from cognitive science, I introduce a framework for evaluating the representational claims made about components of neural NLP models, proposing three criteria with which to evaluate whether a component of a model represents a property and operationalising these criteria using probing classifiers, a popular analysis (...) technique in NLP (and deep learning more broadly). The project of operationalising a philosophically-informed notion of representation should be of interest to both philosophers of science and NLP practitioners. It affords philosophers a novel testing-ground for claims about the nature of representation, and helps NLPers organise the large literature on probing experiments, suggesting novel avenues for empirical research. (shrink)

Context sensitivity is one of the distinctive marks of human intelligence. Understanding the flexible way in which humans think and act in a potentially infinite number of circumstances, even though they’re only finite and limited beings, is a central challenge for the philosophy of mind and cognitive science, particularly in the case of those using representational theories. In this work, the frame problem, that is, the challenge of explaining how human cognition efficiently acknowledges what is relevant from what is not (...) in each context, has been adopted as a guide. By using it, we’ve been able to describe a fundamental tension between context sensitivity and the mental representations used in cognition theories. The first chapter discusses the nature of the frame problem,as well as the reasons for its persistence. In the second and third chapters, the problem is used as a measure tool in order to inquiry a few representational approaches and check how well suited they are to deal with context dependencies. The problems found are then correlated with the frame problem. Throughout the discussion, we try to show that 1) none of the evaluated approaches is capable of dealing with context sensitivity in a proper manner, but 2) that’s not a reason to think that the frame problem constitutes an argument against representational approaches in general, and 3) that it constitutes a fundamental conceptual tool in contemporary research -/- A sensibilidade ao contexto é uma das marcas distintivas da inteligência humana. Compreender o modo flexível como o ser humano pensa e age em função de um número potencialmente infinito de circunstâncias, ainda que munido de recursos finitos e limitados, é um desafio central para a filosofia da mente e para a ciência cognitiva, em particular aos que fazem uso de teorias representacionalistas. Nesse trabalho, adotou-se como fio condutor o modo como isso se manifesta no "frame problem": a dificuldade em explicar como a cognição humana reconhece, de maneira eficiente, o que é ou não relevante em cada contexto. A partir dele, buscou-se caracterizar uma tensão fundamental entre a sensibilidade ao contexto e o uso de representações mentais em teorias da cognição. O primeiro capítulo discute a natureza do frame problem, bem como as razões de sua resiliência. No segundo e terceiro capítulos, faz-se uso do problema como métrica para investigar o quão adequado é o tratamento das dependências contextuais no âmbito de várias abordagens representacionais. No decorrer da discussão, realiza-se um esforço argumentativo para mostrar que 1) nenhuma das estratégias abordadas é capaz tratar adequadamente da sensibilidade ao contexto, mas que 2) apesar disso, o frame problem não constitui argumento fatal para teorias representacionalistas em geral, e que 3) ele constitui uma ferramenta conceitual fundamental para pesquisas contemporâneas. (shrink)

Two universal functional principles of Grossberg’s Adaptive Resonance Theory decipher the brain code of all biological learning and adaptive intelligence. Low-level representations of multisensory stimuli in their immediate environmental context are formed on the basis of bottom-up activation and under the control of top-down matching rules that integrate high-level, long-term traces of contextual configuration. These universal coding principles lead to the establishment of lasting brain signatures of perceptual experience in all living species, from aplysiae to primates. They are re-visited in (...) this concept paper on the basis of examples drawn from the original code and from some of the most recent related empirical findings on contextual modulation in the brain, highlighting the potential of Grossberg’s pioneering insights and groundbreaking theoretical work for intelligent solutions in the domain of developmental and cognitive robotics. (shrink)

Two universal functional principles of Grossberg’s Adaptive Resonance Theory [19] decipher the brain code of all biological learning and adaptive intelligence. Low-level representations of multisensory stimuli in their immediate environmental context are formed on the basis of bottom-up activation and under the control of top-down matching rules that integrate high-level long-term traces of contextual configuration. These universal coding principles lead to the establishment of lasting brain signatures of perceptual experience in all living species, from aplysiae to primates. They are re-visited (...) in this concept paper here on the basis of examples drawn from the original code and from some of the most recent related empirical findings on contextual modulation in the brain, highlighting the potential of Grossberg’s pioneering insights and groundbreaking theoretical work for intelligent solutions in the domain of developmental and cognitive robotics. (shrink)

[This is an edited version of a keynote talk at the conference on Neural Information Processing Systems (NeurIPS) on November 28, 2022, with some minor additions and subtractions.] -/- There has recently been widespread discussion of whether large language models might be sentient or conscious. Should we take this idea seriously? I will break down the strongest reasons for and against. Given mainstream assumptions in the science of consciousness, there are significant obstacles to consciousness in current models: for example, their (...) lack of recurrent processing, a global workspace, and unified agency. At the same time, it is quite possible that these obstacles will be overcome in the next decade or so. I conclude that while it is somewhat unlikely that current large language models are conscious, we should take seriously the possibility that successors to large language models may be conscious in the not-too-distant future. (shrink)

Detecting quality in large unstructured datasets requires capacities far beyond the limits of human perception and communicability and, as a result, there is an emerging trend towards increasingly complex analytic solutions in data science to cope with this problem. This new trend towards analytic complexity represents a severe challenge for the principle of parsimony (Occam’s razor) in science. This review article combines insight from various domains such as physics, computational science, data engineering, and cognitive science to review the specific properties (...) of big data. Problems for detecting data quality without losing the principle of parsimony are then highlighted on the basis of specific examples. Computational building block approaches for data clustering can help to deal with large unstructured datasets in minimized computation time, and meaning can be extracted rapidly from large sets of unstructured image or video data parsimoniously through relatively simple unsupervised machine learning algorithms. Why we still massively lack in expertise for exploiting big data wisely to extract relevant information for specific tasks, recognize patterns and generate new information, or simply store and further process large amounts of sensor data is then reviewed, and examples illustrating why we need subjective views and pragmatic methods to analyze big data contents are brought forward. The review concludes on how cultural differences between East and West are likely to affect the course of big data analytics, and the development of increasingly autonomous artificial intelligence (AI) aimed at coping with the big data deluge in the near future. Keywords: big data; non-dimensionality; applied data science; paradigm shift; artificial intelligence; principle of parsimony (Occam’s razor) . (shrink)

Can the machines that play board games or recognize images only in the comfort of the virtual world be intelligent? To become reliable and convenient assistants to humans, machines need to learn how to act and communicate in the physical reality, just like people do. The authors propose two novel ways of designing and building Artificial General Intelligence (AGI). The first one seeks to unify all participants at any instance of the Turing test – the judge, the machine, the human (...) subject as well as the means of observation instead of building a separating wall. The second one aims to design AGI programs in such a way so that they can move in various environments. The authors of the article thoroughly discuss four areas of interaction for robots with AGI and introduce a new idea of techno-umwelt bridging artificial intelligence with biology in a new way. (shrink)

A crucial question for artificial cognition systems is what meaning is and how it arises. In pursuit of that question, this paper extends earlier work in which we show that emergence of simple signaling in biologically inspired models using arrays of locally interactive agents. Communities of "communicators" develop in an environment of wandering food sources and predators using any of a variety of mechanisms: imitation of successful neighbors, localized genetic algorithms and partial neural net training on successful neighbors. Here we (...) focus on environmental variability, comparing results for environments with (a) constant resources, (b) random resources, and (c) cycles of "boom and bust." In both simple and complex models across all three mechanisms of strategy change, the emergence of communication is strongly favored by cycles of "boom and bust." These results are particularly intriguing given the importance of environmental variability in fields as diverse as psychology, ecology and cultural anthropology. (shrink)

in a nervous system of a given species. This chapter provides a critical perspective on the role of connectomes in neuroscientific practice and asks how the connectomic approach fits into a larger context in which network thinking permeates technology, infrastructure, social life, and the economy. In the first part of this chapter, we argue that, seen from the perspective of ongoing research, the notion of connectomes as “complete descriptions” is misguided. Our argument combines Rachel Ankeny’s analysis of neuroanatomical wiring diagrams (...) as “descriptive models” with Hans-Joerg Rheinberger’s notion of “epistemic objects,” i.e., targets of research that are still partially unknown. Combining these aspects we conclude that connectomes are constitutively epistemic objects: there just is no way to turn them into permanent and complete technical standards because the possibilities to map connection properties under different modeling assumptions are potentially inexhaustible. In the second part of the chapter, we use this understanding of connectomes as constitutively epistemic objects in order to critically assess the historical and political dimensions of current neuroscientific research. We argue that connectomics shows how the notion of the “brain as a network” has become the dominant metaphor of contemporary brain research. We further point out that this metaphor shares (potentially problematic) affinities to the form of contemporary “network societies.” We close by pointing out how the relation between connectomes and networks in society could be used in a more fruitful manner. (shrink)

In this paper, I argue that theories of perception that appeal to Helmholtz’s idea of unconscious inference (“Helmholtzian” theories) should be taken literally, i.e. that the inferences appealed to in such theories are inferences in the full sense of the term, as employed elsewhere in philosophy and in ordinary discourse. -/- In the course of the argument, I consider constraints on inference based on the idea that inference is a deliberate acton, and on the idea that inferences depend on the (...) syntactic structure of representations. I argue that inference is a personal-level but sometimes unconscious process that cannot in general be distinguished from association on the basis of the structures of the representations over which it’s defined. I also critique arguments against representationalist interpretations of Helmholtzian theories, and argue against the view that perceptual inference is encapsulated in a module. (shrink)

Artificial intelligence (AI) research enjoyed an initial period of enthusiasm in the 1970s and 80s. But this enthusiasm was tempered by a long interlude of frustration when genuinely useful AI applications failed to be forthcoming. Today, we are experiencing once again a period of enthusiasm, fired above all by the successes of the technology of deep neural networks or deep machine learning. In this paper we draw attention to what we take to be serious problems underlying current views of artificial (...) intelligence encouraged by these successes, especially in the domain of language processing. We then show an alternative approach to language-centric AI, in which we identify a role for philosophy. (shrink)

Endowing artificial systems with explanatory capacities about the reasons guiding their decisions, represents a crucial challenge and research objective in the current fields of Artificial Intelligence (AI) and Computational Cognitive Science [Langley et al., 2017]. Current mainstream AI systems, in fact, despite the enormous progresses reached in specific tasks, mostly fail to provide a transparent account of the reasons determining their behavior (both in cases of a successful or unsuccessful output). This is due to the fact that the classical problem (...) of opacity in artificial neural networks (ANNs) explodes with the adoption of current Deep Learning techniques [LeCun, Bengio, Hinton, 2015]. In this paper we argue that the explanatory deficit of such techniques represents an important problem, that limits their adoption in the cognitive modelling and computational cognitive science arena. In particular we will show how the current attempts of providing explanations of the deep nets behaviour (see e.g. [Ritter et al. 2017] are not satisfactory. As a possibile way out to this problem, we present two different research strategies. The first strategy aims at dealing with the opacity problem by providing a more abstract interpretation of neural mechanisms and representations. This approach is adopted, for example, by the biologically inspired SPAUN architecture [Eliasmith et al., 2012] and by other proposals suggesting, for example, the interpretation of neural networks in terms of the Conceptual Spaces framework [Gärdenfors 2000, Lieto, Chella and Frixione, 2017]. All such proposals presuppose that the neural level of representation can be considered somehow irrelevant for attacking the problem of explanation [Lieto, Lebiere and Oltramari, 2017]. In our opinion, pursuing this research direction can still preserve the use of deep learning techniques in artificial cognitive models provided that novel and additional results in terms of “transparency” are obtained. The second strategy is somehow at odds with respect to the previous one and tries to address the explanatory issue by avoiding to directly solve the “opacity” problem. In this case, the idea is that one of resorting to pre-compiled plausible explanatory models of the word used in combination with deep-nets (see e.g. [Augello et al. 2017]). We argue that this research agenda, even if does not directly fits the explanatory needs of Computational Cognitive Science, can still be useful to provide results in the area of applied AI aiming at shedding light on the models of interaction between low level and high level tasks (e.g. between perceptual categorization and explanantion) in artificial systems. (shrink)

In artificial intelligence, recent research has demonstrated the remarkable potential of Deep Convolutional Neural Networks (DCNNs), which seem to exceed state-of-the-art performance in new domains weekly, especially on the sorts of very difficult perceptual discrimination tasks that skeptics thought would remain beyond the reach of artificial intelligence. However, it has proven difficult to explain why DCNNs perform so well. In philosophy of mind, empiricists have long suggested that complex cognition is based on information derived from sensory experience, often appealing to (...) a faculty of abstraction. Rationalists have frequently complained, however, that empiricists never adequately explained how this faculty of abstraction actually works. In this paper, I tie these two questions together, to the mutual benefit of both disciplines. I argue that the architectural features that distinguish DCNNs from earlier neural networks allow them to implement a form of hierarchical processing that I call “transformational abstraction”. Transformational abstraction iteratively converts sensory-based representations of category exemplars into new formats that are increasingly tolerant to “nuisance variation” in input. Reflecting upon the way that DCNNs leverage a combination of linear and non-linear processing to efficiently accomplish this feat allows us to understand how the brain is capable of bi-directional travel between exemplars and abstractions, addressing longstanding problems in empiricist philosophy of mind. I end by considering the prospects for future research on DCNNs, arguing that rather than simply implementing 80s connectionism with more brute-force computation, transformational abstraction counts as a qualitatively distinct form of processing ripe with philosophical and psychological significance, because it is significantly better suited to depict the generic mechanism responsible for this important kind of psychological processing in the brain. (shrink)

Report for "The Reasoner" on the conference "Philosophy and Theory of Artificial Intelligence", 3 & 4 October 2011, Thessaloniki, Anatolia College/ACT, http://www.pt-ai.org. --- Organization: Vincent C. Müller, Professor of Philosophy at ACT & James Martin Fellow, Oxford http://www.sophia.de --- Sponsors: EUCogII, Oxford-FutureTech, AAAI, ACM-SIGART, IACAP, ECCAI.

This paper is a commentary on the target article by Dana H. Ballard, “Cortical connections and parallel processing: Structure and function”, in the same issue of the journal, pp. 67–120. -/- I raise some issues about the connectionist or neural-network implementation of information and information processing. Issues include the sharing of information by different parts of a connectionist/neural network, the copying of complex information from one place to another in a network, the possibility of connection weights not being synaptic weights, (...) and the possibility of fast communication within single neurons as complex systems in their own right. (shrink)

The paper is a commentary on the target article by Christine A. Skarda & Walter J. Freeman, “How brains make chaos in order to make sense of the world”, in the same issue of the journal, pp.161–195. -/- I confine my comments largely to some philosophical claims that Skarda & Freeman make and to the relationship of their model to connectionism. Some of the comments hinge on what symbols are and how they might sit in neural systems.

Critics of the computational connectionism of the last decade suggest that it shares undesirable features with earlier empiricist or associationist approaches, and with behaviourist theories of learning. To assess the accuracy of this charge the works of earlier writers are examined for the presence of such features, and brief accounts of those found are given for Herbert Spencer, William James and the learning theorists Thorndike, Pavlov and Hull. The idea that cognition depends on associative connections among large networks of neurons (...) is indeed one with precedents, although the implications of this for psychological issues have been interpreted variously — not all versions of connectionism are alike. (shrink)

The Problems of Mental Causation. Functionalism in the philosophy of mind identifies mental states with their dispositional connections with other mental states, perceptions and actions. Many theories of the mind have sailed under the Functionalist flag. But what I take to be essential to Functionalism is that mental states are individuated causally: the reality of mental states depends essentially on their causal efficacy.

The goal of philosophy of information is to understand what information is, how it operates, and how to put it to work. But unlike ‘information’ in the technical sense of information theory, what we are interested in is meaningful information. To understand the nature and dynamics of information in this sense we have to understand meaning. What we offer here are simple computational models that show emergence of meaning and information transfer in randomized arrays of neural nets. These we take (...) to be formal instantiations of a tradition of theories of meaning as use. What they offer, we propose, is a glimpse into the origin and dynamics of at least simple forms of meaning and information transfer as properties inherent in behavioral coordination across a community. (shrink)

In his book The Sensory Order, Hayek anticipates many of the central ideas behind what we now call the connectionist paradigm, and develops on this basis a theory of the workings of the human mind that extends the thinking of Hume and Mach. He shows that the idea of neural networks is can be applied not only in psychology and neurology but also in the sphere of economics. For the mind, from the perspective of The Sensory Order, is a dynamic, (...) relational affair that is in many respects analogous to a market process. The mind, as Hayek puts it, is a 'continuous stream of impulses, the significance of each and every contribution of which is determined by the place in the pattern of channels through which they flow' , so that the flow of representative neural impulses can be compared 'to a stock of capital being nourished by inputs and giving a continuous stream of outputs'. (shrink)

In his famous 1982 paper, Allen Newell [22, 23] introduced the notion of knowledge level to indicate a level of analysis, and prediction, of the rational behavior of a cognitive arti cial agent. This analysis concerns the investigation about the availability of the agent knowledge, in order to pursue its own goals, and is based on the so-called Rationality Principle (an assumption according to which "an agent will use the knowledge it has of its environment to achieve its goals" [22, (...) p. 17]. By using the Newell's own words: "To treat a system at the knowledge level is to treat it as having some knowledge, some goals, and believing it will do whatever is within its power to attain its goals, in so far as its knowledge indicates" [22, p. 13]. In the last decades, the importance of the knowledge level has been historically and system- atically downsized by the research area in cognitive architectures (CAs), whose interests have been mainly focused on the analysis and the development of mechanisms and the processes governing human and (arti cial) cognition. The knowledge level in CAs, however, represents a crucial level of analysis for the development of such arti cial general systems and therefore deserves greater research attention [17]. In the following, we will discuss areas of broad agree- ment and outline the main problematic aspects that should be faced within a Common Model of Cognition [12]. Such aspects, departing from an analysis at the knowledge level, also clearly impact both lower (e.g. representational) and higher (e.g. social) levels. (shrink)

We describe and try to motivate our project to build systems using both a knowledge based and a neural network approach. These two approaches are used at different stages in the solution of a problem, instead of using knowledge bases exclusively on some problems, and neural nets exclusively on others. The knowledge base (KB) is defined first in a declarative, symbolic language that is easy to use. It is then compiled into an efficient neural network (NN) representation, run, and the (...) results from run time and (eventually) from learning are decompiled to a symbolic description of the knowledge contained in the network. After inspecting this recovered knowledge, a designer would be able to modify the KB and go through the whole cycle of compiling, running, and decompiling again. The central question with which this project is concerned is, therefore, How do we go from a KB to an NN, and back again? We are investigating this question by building tools consisting of a repertoire of language/translation/network types, and trying them on problems in a variety of domains. (shrink)

The systematicity argument only challenges connectionism if systematicity is a general property of cognition. I examine this thesis in terms of properties of concepts. First, I propose that Evans's Generality Constraint only applies to attributions of belief. Then I defend a variety of conceptual pluralism, arguing that concepts share two fundamental properties related to centrality and belief-attribution, and contending that there are two kinds of concepts that differ in their compositional properties. Finally, I rely on Dual Systems Theory and on (...) differences between animal and human cognition to suggest a scenario of two processing systems that work on different kinds of concepts, with only one of them supporting full systematicity. I sketch a non-classical systematicity argument that rules out classicism as the basis of one of those systems given that it would wrongly entail that both systems are fully systematic. (shrink)

Fodor and Pylyshyn's critique of connectionism has posed a challenge to connectionists: Adequately explain such nomological regularities as systematicity and productivity without postulating a "language of thought" (LOT). Some connectionists like Smolensky took the challenge very seriously, and attempted to meet it by developing models that were supposed to be non-classical. At the core of these attempts lies the claim that connectionist models can provide a representational system with a combinatorial syntax and processes sensitive to syntactic structure. They are not (...) implementation models because, it is claimed, the way they obtain syntax and structure sensitivity is not "concatenative," hence "radically different" from the way classicists handle them. In this paper, I offer an analysis of what it is to physically satisfy/realize a formal system. In this context, I examine the minimal truth-conditions of LOT Hypothesis. From my analysis it will follow that concatenative realization of formal systems is irrelevant to LOTH since the very notion of LOT is indifferent to such an implementation level issue as concatenation. I will conclude that to the extent to which they can explain the law-like cognitive regularities, a certain class of connectionist models proposed as radical alternatives to the classical LOT paradigm will in fact turn out to be LOT models, even though new and potentially very exciting ones. (shrink)

A comprehensive introduction to the Language of Though Hypothesis (LOTH) accessible to general audiences. LOTH is an empirical thesis about thought and thinking. For their explication, it postulates a physically realized system of representations that have a combinatorial syntax (and semantics) such that operations on representations are causally sensitive only to the syntactic properties of representations. According to LOTH, thought is, roughly, the tokening of a representation that has a syntactic (constituent) structure with an appropriate semantics. Thinking thus consists in (...) syntactic operations defined over representations. Most of the arguments for LOTH derive their strength from their ability to explain certain empirical phenomena like productivity, systematicity of thought and thinking. (shrink)

The quantization error in a fixed-size Self-Organizing Map (SOM) with unsupervised winner-take-all learning has previously been used successfully to detect, in minimal computation time, highly meaningful changes across images in medical time series and in time series of satellite images. Here, the functional properties of the quantization error in SOM are explored further to show that the metric is capable of reliably discriminating between the finest differences in local contrast intensities and contrast signs. While this capability of the QE is (...) akin to functional characteristics of a specific class of retinal ganglion cells (the so-called Y-cells) in the visual systems of the primate and the cat, the sensitivity of the QE surpasses the capacity limits of human visual detection. Here, the quantization error in the SOM is found to reliably signal changes in contrast or colour when contrast information is removed from or added to the image, but not when the amount and relative weight of contrast information is constant and only the local spatial position of contrast elements in the pattern changes. While the RGB Mean reflects coarser changes in colour or contrast well enough, the SOM-QE is shown to outperform the RGB Mean in the detection of single-pixel changes in images with up to five million pixels. This could have important implications in the context of unsupervised image learning and computational building block approaches to large sets of image data (big data). (shrink)

Symmetry in biological and physical systems is a product of self-organization driven by evolutionary processes, or mechanical systems under constraints. Symmetry-based feature extraction or representation by neural networks may unravel the most informative contents in large image databases. Despite significant achievements of artificial intelligence in recognition and classification of regular patterns, the problem of uncertainty remains a major challenge in ambiguous data. In this study, we present an artificial neural network that detects symmetry uncertainty states in human observers. To this (...) end, we exploit a neural network metric in the output of a biologically inspired Self- Organizing Map Quantization Error (SOM-QE). Shape pairs with perfect geometry mirror symmetry but a non-homogenous appearance, caused by local variations in hue, saturation, or lightness within and/or across the shapes in a given pair produce, as shown here, a longer choice response time (RT) for “yes” responses relative to symmetry. These data are consistently mirrored by the variations in the SOM-QE from unsupervised neural network analysis of the same stimulus images. The neural network metric is thus capable of detecting and scaling human symmetry uncertainty in response to patterns. Such capacity is tightly linked to the metric’s proven selectivity to local contrast and color variations in large and highly complex image data. (shrink)

Many philosophers claim that the neurocomputational framework of predictive processing entails a globally inferentialist and representationalist view of cognition. Here, I contend that this is not correct. I argue that, given the theoretical commitments these philosophers endorse, no structure within predictive processing systems can be rightfully identified as a representational vehicle. To do so, I first examine some of the theoretical commitments these philosophers share, and show that these commitments provide a set of necessary conditions the satisfaction of which allows (...) us to identify representational vehicles. Having done so, I introduce a predictive processing system capable of active inference, in the form of a simple robotic “brain”. I examine it thoroughly, and show that, given the necessary conditions highlighted above, none of its components qualifies as a representational vehicle. I then consider and allay some worries my claim could raise. I consider whether the anti-representationalist verdict thus obtained could be generalized, and provide some reasons favoring a positive answer. I further consider whether my arguments here could be blocked by allowing the same representational vehicle to possess multiple contents, and whether my arguments entail some extreme form of revisionism, answering in the negative in both cases. A quick conclusion follows. (shrink)

This is an in-depth review of "The Embodied Mind: Cognitive Science and Human Experience" by Francisco Varela, Evan Thompson and Eleanor Rosch.

This paper presents a theoretical study of the binary oppositions underlying the mechanisms of natural computation understood as dynamical processes on natural information morphologies. Of special interest are the oppositions of discrete vs. continuous, structure vs. process, and differentiation vs. integration. The framework used is that of computing nature, where all natural processes at different levels of organisation are computations over informational structures. The interactions at different levels of granularity/organisation in nature, and the character of the phenomena that unfold through (...) those interactions, are modeled from the perspective of an observing agent. This brings us to the movement from binary oppositions to dynamic networks built upon mutually related binary oppositions, where each node has several properties. (shrink)

“There’s Plenty of Room at the Bottom”, said the title of Richard Feynman’s 1959 seminal conference at the California Institute of Technology. Fifty years on, nanotechnologies have led computer scientists to pay close attention to the links between physical reality and information processing. Not all the physical requirements of optimal computation are captured by traditional models—one still largely missing is reversibility. The dynamic laws of physics are reversible at microphysical level, distinct initial states of a system leading to distinct final (...) states. On the other hand, as von Neumann already conjectured, irreversible information processing is expensive: to erase a single bit of information costs ~3 × 10−21 joules at room temperature. Information entropy is a thermodynamic cost, to be paid in non-computational energy dissipation. This paper addresses the problem drawing on Edward Fredkin’s Finite Nature hypothesis: the ultimate nature of the universe is discrete and finite, satisfying the axioms of classical, atomistic mereology. The chosen model is a cellular automaton with reversible dynamics, capable of retaining memory of the information present at the beginning of the universe. Such a CA can implement the Boolean logical operations and the other building bricks of computation: it can develop and host all-purpose computers. The model is a candidate for the realization of computational systems, capable of exploiting the resources of the physical world in an efficient way, for they can host logical circuits with negligible internal energy dissipation. (shrink)

During the last decades, many cognitive architectures (CAs) have been realized adopting different assumptions about the organization and the representation of their knowledge level. Some of them (e.g. SOAR [35]) adopt a classical symbolic approach, some (e.g. LEABRA[ 48]) are based on a purely connectionist model, while others (e.g. CLARION [59]) adopt a hybrid approach combining connectionist and symbolic representational levels. Additionally, some attempts (e.g. biSOAR) trying to extend the representational capacities of CAs by integrating diagrammatical representations and reasoning are (...) also available [34]. In this paper we propose a reflection on the role that Conceptual Spaces, a framework developed by Peter G¨ardenfors [24] more than fifteen years ago, can play in the current development of the Knowledge Level in Cognitive Systems and Architectures. In particular, we claim that Conceptual Spaces offer a lingua franca that allows to unify and generalize many aspects of the symbolic, sub-symbolic and diagrammatic approaches (by overcoming some of their typical problems) and to integrate them on a common ground. In doing so we extend and detail some of the arguments explored by G¨ardenfors [23] for defending the need of a conceptual, intermediate, representation level between the symbolic and the sub-symbolic one. In particular we focus on the advantages offered by Conceptual Spaces (w.r.t. symbolic and sub-symbolic approaches) in dealing with the problem of compositionality of representations based on typicality traits. Additionally, we argue that Conceptual Spaces could offer a unifying framework for interpreting many kinds of diagrammatic and analogical representations. As a consequence, their adoption could also favor the integration of diagrammatical representation and reasoning in CAs. (shrink)

We describe and try to motivate our project to build systems using both a knowledge based and a neural network approach. These two approaches are used at different stages in the solution of a problem, instead of using knowledge bases exclusively on some problems, and neural nets exclusively on others. The knowledge base (KB) is defined first in a declarative, symbolic language that is easy to use. It is then compiled into an efficient neural network (NN) representation, run, and the (...) results from run time and (eventually) from learning are decompiled to a symbolic description of the knowledge contained in the network. After inspecting this recovered knowledge, a designer would be able to modify the KB and go through the whole cycle of compiling, running, and decompiling again. The central question with which this project is concerned is, therefore, How do we go from a KB to an NN, and back again? We are investigating this question by building tools consisting of a repertoire of language/translation/network types, and trying them on problems in a variety of domains. (shrink)

Functionalism of robot pain claims that what is definitive of robot pain is functional role, defined as the causal relations pain has to noxious stimuli, behavior and other subjective states. Here, I propose that the only way to theorize role-functionalism of robot pain is in terms of type-identity theory. I argue that what makes a state pain for a neuro-robot at a time is the functional role it has in the robot at the time, and this state is type identical (...) to a specific circuit state. Support from an experimental study shows that if the neural network that controls a robot includes a specific 'emotion circuit', physical damage to the robot will cause the disposition to avoid movement, thereby enhancing fitness, compared to robots without the circuit. Thus, pain for a robot at a time is type identical to a specific circuit state. (shrink)

Bayesian models can be related to cognitive processes in a variety of ways that can be usefully understood in terms of Marr's distinction among three levels of explanation: computational, algorithmic and implementation. In this note, we discuss how an integrated probabilistic account of the different levels of explanation in cognitive science is resulting, at least for the current research practice, in a sort of unpredicted epistemological shift with respect to Marr's original proposal.

In this paper I discuss one of the key issuesin the philosophy of neuroscience:neurosemantics. The project of neurosemanticsinvolves explaining what it means for states ofneurons and neural systems to haverepresentational contents. Neurosemantics thusinvolves issues of common concern between thephilosophy of neuroscience and philosophy ofmind. I discuss a problem that arises foraccounts of representational content that Icall ``the economy problem'': the problem ofshowing that a candidate theory of mentalrepresentation can bear the work requiredwithin in the causal economy of a mind and (...) anorganism. My approach in the current paper isto explore this and other key themes inneurosemantics through the use of computermodels of neural networks embodied and evolvedin virtual organisms. The models allow for thelaying bare of the causal economies of entireyet simple artificial organisms so that therelations between the neural bases of, forinstance, representation in perception andmemory can be regarded in the context of anentire organism. On the basis of thesesimulations, I argue for an account ofneurosemantics adequate for the solution of theeconomy problem. (shrink)

Some contemporary philosophers suggest that we know just by introspection that folk psychological states exist. However, such an "armchair refutation" of eliminativism seems too easy. I first attack two strategems, inspired by Descartes, on how such a refutation might proceed. However, I concede that the Cartesian intuition that we have direct knowledge of representational states is very powerful. The rest of this paper then offers an error theory of how that intuition might really be mistaken. The idea is that introspection (...) does not detect any folk psychological states, but rather detects neural states that dispose us to certain sorts of linguistic behavior. Briefly, rather than detecting a mental appearance of the state-of-affairs that I am sitting near the fire, introspection detects a state disposing me to assert "It appears to me that I am sitting by the fire." The nature of this linguistic-dispositional state is then given a connectionist underpinning, where inputs from introspection lead to such assertive behavior. The Cartesian certainty we feel in self-ascribing such appearances is also explained. Even though I do not endorse such an error theory outright, I suggest that its tenability is enough to undercut introspection arguments against eliminativism. (shrink)

Three decades ago, William Ramsey, Steven Stich & Joseph Garon put forward an argument in favor of the following conditional: if connectionist models that implement parallelly distributed processing represent faithfully human cognitive processing, eliminativism about propositional attitudes is true. The corollary of their argument (if it proves to be sound) is that there is no place for folk psychology in contemporary cognitive science. This understanding of connectionism as a hypothesis about cognitive architecture compatible with eliminativism is also endorsed by Paul (...) Churchland, a radical opponent of folk psychology and a prominent supporter of eliminative materialism. I aim to examine whether current connectionist models based on long-short term memory (LSTM) neural networks can back up these arguments in favor of eliminativism. Nonetheless, I will rather put my faith in the eliminativism of the limited domain. This position amounts to the claim that even though connectionist cognitive science has no need whatsoever for folk psychology qua theory, this does not entail the illegitimacy of folk psychology per se in other scientific domains, most notably in humanities, but only on condition that one regards folk psychology as mere heuristics. (shrink)

The topic of this dissertation is the nature of linguistic competence, the capacity to understand and produce sentences of natural language. I defend the empiricist account of linguistic competence embedded in the connectionist cognitive science. This strand of cognitive science has been opposed to the traditional symbolic cognitive science, coupled with transformational-generative grammar, which was committed to nativism due to the view that human cognition, including language capacity, should be construed in terms of symbolic representations and hardwired rules. Similarly, linguistic (...) competence in this framework was regarded as being innate, rule-governed, domain-specific, and fundamentally different from performance, i.e., idiosyncrasies and factors governing linguistic behavior. I analyze state-of-the-art connectionist, deep learning models of natural language processing, most notably large language models, to see what they can tell us about linguistic competence. Deep learning is a statistical technique for the classification of patterns through which artificial intelligence researchers train artificial neural networks containing multiple layers that crunch a gargantuan amount of textual and/or visual data. I argue that these models suggest that linguistic competence should be construed as stochastic, pattern-based, and stemming from domain-general mechanisms. Moreover, I distinguish syntactic from semantic competence, and I show for each the ramifications of the endorsement of a connectionist research program as opposed to the traditional symbolic cognitive science and transformational-generative grammar. I provide a unifying front, consisting of usage-based theories, a construction grammar approach, and an embodied approach to cognition to show that the more multimodal and diverse models are in terms of architectural features and training data, the stronger the case is for the connectionist linguistic competence. I also propose to discard the competence vs. performance distinction as theoretically inferior so that a novel and integrative account of linguistic competence originating in connectionism and empiricism that I propose and defend in the dissertation could be put forward in scientific and philosophical literature. (shrink)

This piece is a comment on Quilty-Dunn, Jake, Nicolas Porot, and Eric Mandelbaum. 2023. “The Best Game in Town: The Reemergence of the Language-of-Thought Hypothesis across the Cognitive Sciences.” Behavioral and Brain Sciences 46: e261. -/- The target article signal boosts important ongoing work across the cognitive sciences. However, its theoretical claims, generative value, and purported contributions are – where not simply restatements of arguments extensively explored elsewhere – imprecise, noncommittal, and underdeveloped to a degree that makes them difficult to (...) evaluate. The article's apparent force results from engaging with straw rather than steel opponents. (shrink)

Cognitive scientists used to deem reasoning either as a higher cognitive process based on the manipulation of abstract rules or as a higher cognitive process that is stochastic rather than involving abstract rules. I maintain that these different perspectives are closely intertwined with a theoretical and methodological endorsement of either cognitivism or connectionism. Cognitivism and connectionism represent two prevailing and opposed paradigms in cognitive science. I aim to extoll the virtues of connectionist models of enthymematic reasoning by following means: via (...) the phenomenon of creative enthymeme, viz. the inference where one cannot even articulate the missing premise, I introduce a connectionist mechanism of pattern recognition as underlying expertise; via Gestalt switch or Gestalt click, I demonstrate how differences in pattern recognition of an expert and a novice can be construed as qualitatively different, and not merely a matter of faster reasoning. (shrink)

This paper aims to offer a new view of the role of connectionist models in the study of human cognition through the conceptualization of the history of connectionism – from the simplest perceptrons to convolutional neural nets based on deep learning techniques, as well as through the interpretation of criticism coming from symbolic cognitive science. Namely, the connectionist approach in cognitive science was the target of sharp criticism from the symbolists, which on several occasions caused its marginalization and almost complete (...) abandonment of its assumptions in the study of cognition. Criticisms have mostly pointed to its explanatory inadequacy as a theory of cognition or to its biological implausibility as a theory of implementation, and critics often focused on specific shortcomings of some connectionist models and argued that they apply to connectionism in general. In this paper, we want to show that both types of critique are based on the assumption that the only valid explanations in cognitive science are instances of homuncular functionalism and that by removing this assumption and by adopting an alternative methodology – exploratory mechanistic strategy, we can reject most objections to connectionism as irrelevant, explain the progress of connectionist models despite their shortcomings and sketch the trajectory of their future development. By adopting mechanistic explanations and by criticizing functionalism, we will reject the objections of explanatory inadequacy, by characterizing connectionist models as generic rather than concrete mechanisms, we will reject the objections of biological implausibility, and by attributing the exploratory character to connectionist models we will show that practice of generalizing current to general failures of connectionism is unjustified. (shrink)

Dullest book by a major scientist I have ever read. I suppose if you know almost nothing about cognition or AI research you might find this book useful. For anyone else it is a horrific bore. There are hundreds of books in cog sci, robotics, AI, evolutionary psychology and philosophy offering far more info and insight on cognition than this one. Minsky is a top rate senior scientist but it barely shows here. He has alot of good references but they (...) are seldom discussed in any depth and there is lots more left out than included on the subject of AI, cognition and the mind. -/- Those wishing a comprehensive up to date framework for human behavior from the modern two systems view may consult my book ‘The Logical Structure of Philosophy, Psychology, Mind and Language in Ludwig Wittgenstein and John Searle’ 2nd ed (2019). Those interested in more of my writings may see ‘Talking Monkeys--Philosophy, Psychology, Science, Religion and Politics on a Doomed Planet--Articles and Reviews 2006-2019 3rd ed (2019), The Logical Structure of Human Behavior (2019), and Suicidal Utopian Delusions in the 21st Century 4th ed (2019). (shrink)

We describe and try to motivate our project to build systems using both a knowledge based and a neural network approach. These two approaches are used at different stages in the solution of a problem, instead of using knowledge bases exclusively on some problems, and neural nets exclusively on others. The knowledge base (KB) is defined first in a declarative, symbolic language that is easy to use. It is then compiled into an efficient neural network (NN) representation, run, and the (...) results from run time and (eventually) from learning are decompiled to a symbolic description of the knowledge contained in the network. After inspecting this recovered knowledge, a designer would be able to modify the KB and go through the whole cycle of compiling, running, and decompiling again. The central question with which this project is concerned is, therefore, How do we go from a KB to an NN, and back again? We are investigating this question by building tools consisting of a repertoire of language/translation/network types, and trying them on problems in a variety of domains. (shrink)

1. Do models formulated in programming languages use explicit rules where connectionist models do not? 2. Are rules as found in programming languages hard, precise, and exceptionless, where connectionist rules are not? 3. Do connectionist models use rules operating on distributed representations where models formulated in programming languages do not? 4. Do connectionist models fail to use structure sensitive rules of the sort found in "classical" computer architectures? In this chapter we argue that the answer to each of these questions (...) is negative. (shrink)

A comprehensive introduction to the Language of Though Hypothesis (LOTH) accessible to general audiences. LOTH is an empirical thesis about thought and thinking. For their explication, it postulates a physically realized system of representations that have a combinatorial syntax (and semantics) such that operations on representations are causally sensitive only to the syntactic properties of representations. According to LOTH, thought is, roughly, the tokening of a representation that has a syntactic (constituent) structure with an appropriate semantics. Thinking thus consists in (...) syntactic operations defined over representations. Most of the arguments for LOTH derive their strength from their ability to explain certain empirical phenomena like productivity, systematicity of thought and thinking. (shrink)

Насколько дефекты культурного окружения людей сбивают их с толку, поскольку находятся в противоречии с их биологическими, т.е. жизненно важными потребностями, я хотел бы проиллюстрировать на примере сна, приснившегося мне в ночь с 13 на 14 апреля 2023 года.

Subjectology (from Latin subject and logos) studies the internal states of living and nonliving systems capable of symbolic representation of any real content, i.e. to display sensory perceptible information and to transform it into world pictures, the elements of which are symbols whose meaning or sense is determined in the context of the symbolic representation. Субъектология (от лат. subject и logos) изучает внутренние состояния живых и неживых систем, способных к символической репрезентации какого–либо реального содержания, т.е. к отображению чувственно воспринимаемой информации (...) и её преобразованию в картины мира, элементами которых являются символы, значение или смысл которых определяется в контексте символического отображения. (shrink)

We introduce and discuss a knowledge-driven distillation approach to explaining black-box models by means of two kinds of interpretable models. The first is perceptron (or threshold) connectives, which enrich knowledge representation languages such as Description Logics with linear operators that serve as a bridge between statistical learning and logical reasoning. The second is Trepan Reloaded, an ap- proach that builds post-hoc explanations of black-box classifiers in the form of decision trees enhanced by domain knowledge. Our aim is, firstly, to target (...) a model-agnostic distillation approach exemplified with these two frameworks, secondly, to study how these two frameworks interact on a theoretical level, and, thirdly, to investigate use-cases in ML and AI in a comparative manner. Specifically, we envision that user-studies will help determine human understandability of explanations generated using these two frameworks. (shrink)