Interactions between an intelligent software agent and a human user are ubiquitous in everyday situations such as access to information, entertainment, and purchases. In such interactions, the ISA mediates the user’s access to the content, or controls some other aspect of the user experience, and is not designed to be neutral about outcomes of user choices. Like human users, ISAs are driven by goals, make autonomous decisions, and can learn from experience. Using ideas from bounded rationality, we frame these interactions (...) as instances of an ISA whose reward depends on actions performed by the user. Such agents benefit by steering the user’s behaviour towards outcomes that maximise the ISA’s utility, which may or may not be aligned with that of the user. Video games, news recommendation aggregation engines, and fitness trackers can all be instances of this general case. Our analysis facilitates distinguishing various subcases of interaction, as well as second-order effects that might include the possibility for adaptive interfaces to induce behavioural addiction, and/or change in user belief. We present these types of interaction within a conceptual framework, and review current examples of persuasive technologies and the issues that arise from their use. We argue that the nature of the feedback commonly used by learning agents to update their models and subsequent decisions could steer the behaviour of human users away from what benefits them, and in a direction that can undermine autonomy and cause further disparity between actions and goals as exemplified by addictive and compulsive behaviour. We discuss some of the ethical, social and legal implications of this technology and argue that it can sometimes exploit and reinforce weaknesses in human beings. (shrink)

Mainstream teleosemantics is the view that mental representation should be understood in terms of biological functions, which, in turn, should be understood in terms of selection processes. One of the traditional criticisms of teleosemantics is the problem of novel contents: how can teleosemantics explain our ability to represent properties that are evolutionarily novel? In response, some have argued that by generalizing the notion of a selection process to include phenomena such as operant conditioning, and the neural selection that underlies it, (...) we can resolve this problem. Here, we do four things: we develop this suggestion in a rigorous way through a simple example, we draw on recent neurobiological research to support its empirical plausibility, we defend the move from a host of objections in the literature, and we sketch how the picture can be extended to help us think about more complex “conceptual” representations and not just perceptual ones. (shrink)

This paper sets out a view about the explanatory role of representational content and advocates one approach to naturalising content – to giving a naturalistic account of what makes an entity a representation and in virtue of what it has the content it does. It argues for pluralism about the metaphysics of content and suggests that a good strategy is to ask the content question with respect to a variety of predictively successful information processing models in experimental psychology and cognitive (...) neuroscience; and hence that data from psychology and cognitive neuroscience should play a greater role in theorising about the nature of content. Finally, the contours of the view are illustrated by drawing out and defending a surprising consequence: that individuation of vehicles of content is partly externalist. (shrink)

Multiple realizability says that the same kind of mental states may be manifested by systems with very different physical constitutions. Putnam ( 1967 ) supposed it to be “overwhelmingly probable” that there exist psychological properties with different physical realizations in different creatures. But because function constrains possible physical realizers, this empirical bet is far less favorable than it might initially have seemed, especially when we take on board the richer picture of neural and brain function that neuroscience has been uncovering (...) over the past forty years, in which all sorts of brain activities play crucial roles beyond all-or-nothing electrical signaling. Because of its evolutionary history, the brain’s metabolic and informational processes are inextricably intertwined. The resulting complex integrated functions impose more constraints on possible realizers than the clean, single-purpose functions usually cited as examples in discussions of multiple realizability—with ramifications for the functionalist and computationalist foundations of cognitive science, artificial intelligence, and the philosophy of mind. (shrink)

Philosophical proponents of predictive processing cast the novelty of predictive models of perception in terms of differences in the functional role and information content of neural signals. However, they fail to provide constraints on how the crucial semantic mapping from signals to their informational contents is determined. Beyond a novel interpretative gloss on neural signals, they have little new to say about the causal structure of the system, or even what statistical information is carried by the signals. That means that (...) the predictive framework for perception can be relabeled in traditional, non-predictive terms, with no empirical consequences relevant to existing or future data. To the extent that neuroscientific research based on predictive processing is both innovative and productive, it will be due to the framework’s suggestive heuristic effects, or perhaps auxiliary empirical claims about implementation, rather than a difference in the information-processing structure that it describes. (shrink)

Although for many years a sharp distinction has been made in language research between rules and words—with primary interest on rules—this distinction is now blurred in many theories. If anything, the focus of attention has shifted in recent years in favor of words. Results from many different areas of language research suggest that the lexicon is representationally rich, that it is the source of much productive behavior, and that lexically specific information plays a critical and early role in the interpretation (...) of grammatical structure. But how much information can or should be placed in the lexicon? This is the question I address here. I review a set of studies whose results indicate that event knowledge plays a significant role in early stages of sentence processing and structural analysis. This poses a conundrum for traditional views of the lexicon. Either the lexicon must be expanded to include factors that do not plausibly seem to belong there; or else virtually all information about word meaning is removed, leaving the lexicon impoverished. I suggest a third alternative, which provides a way to account for lexical knowledge without a mental lexicon. (shrink)

1. Introduction 2. Reward-Guided Decision Making 3. Content in the Model 4. How to Deflate a Metarepresentational Reading Proust and Carruthers on metacognitive feelings 5. A Deflationary Treatment of RPEs? 5.1 Dispensing with prediction errors 5.2 What is use of the RPE focused on? 5.3 Alternative explanations—worldly correlates 5.4 Contrast cases 6. Conclusion Appendix: Temporal Difference Learning Algorithms.

Addiction is almost universally held to be characterized by a loss of control over drug-seeking and consuming behavior. But the actions of addicts, even of those who seem to want to abstain from drugs, seem to be guided by reasons. In this paper, I argue that we can explain this fact, consistent with continuing to maintain that addiction involves a loss of control, by understanding addiction as involving an oscillation between conflicting judgments. I argue that the dysfunction of the mesolimbic (...) dopamine system that typifies addictions causes the generation of a mismatch between the top-down model of the world that reflects the judgment that the addict ought to refrain from drugs, and bottom-up input caused by cues predictive of drug availability. This constitutes a powerful pressure toward revising the judgment and thereby attenuating the prediction error. But the new model is not stable, and shifts under the pressure of bottom-up inputs in different contexts; hence the oscillation of all-things-considered judgment. Evidence from social psychology is adduced, to suggest that a similar process may be involved in ordinary cases of weakness of will. (shrink)

There is surprising evidence that introspection of our phenomenal states varies greatly between individuals and within the same individual over time. This puts pressure on the notion that introspection gives reliable access to our own phenomenology: introspective unreliability would explain the variability, while assuming that the underlying phenomenology is stable. I appeal to a body of neurocomputational, Bayesian theory and neuroimaging findings to provide an alternative explanation of the evidence: though some limited testing conditions can cause introspection to be unreliable, (...) mostly it is our phenomenology itself that is variable. With this account of phenomenal variability, the occurrence of the surprising evidence can be explained while generally retaining introspective reliability. (shrink)

Cognitive control has long been one of the most active areas of computational modeling work in cognitive science. The focus on computational models as a medium for specifying and developing theory predates the PDP books, and cognitive control was not one of the areas on which they focused. However, the framework they provided has injected work on cognitive control with new energy and new ideas. On the occasion of the books' anniversary, we review computational modeling in the study of cognitive (...) control, with a focus on the influence that the PDP approach has brought to bear in this area. Rather than providing a comprehensive review, we offer a framework for thinking about past and future modeling efforts in this domain. We define control in terms of the optimal parameterization of task processing. From this vantage point, the development of control systems in the brain can be seen as responding to the structure of naturalistic tasks, through the filter of the brain systems with which control directly interfaces. This perspective lays open a set of fascinating but difficult research questions, which together define an important frontier for future computational research. (shrink)

I distinguish between two styles of research that are both called . Neurocellular economics (NE) uses the modelling techniques and mathematics of economics to model relatively encapsulated functional parts of brains. This approach rests upon the fact that brains are, like markets, massively distributed information-processing networks over which executive systems can exert only limited and imperfect governance. Harrison's (2008) deepest criticisms of neuroeconomics do not apply to NE. However, the more famous style of neuroeconomics is behavioural economics in the scanner. (...) This is often motivated by complaints about conventional economics frequently heard from behavioural economists. It attempts to use neuroimaging data to justify arguments for replacing standard aspects of microeconomic theory by facts and conjectures about human psychology. Harrison's grounds for unease about neuroeconomics apply to most BES, or at least to its explicit methodology. This methodology is naively reductionist and illegitimately assumes that economics should not do what all successful science does, namely, model abstract aspects of its target phenomena instead of would-be complete and fully ecologically situated facsimiles of them. (shrink)

Following in a psychological and musicological tradition beginning with Leonard Meyer, and continuing through David Huron, we present a functional, cognitive account of the phenomenon of expectation in music, grounded in computational, probabilistic modeling. We summarize a range of evidence for this approach, from psychology, neuroscience, musicology, linguistics, and creativity studies, and argue that simulating expectation is an important part of understanding a broad range of human faculties, in music and beyond.

This article considers how a generalized signalling game may self-assemble as the saliences of the agents evolve by reinforcement on those sources of information that in fact lead to successful action. On the present account, generalized signalling games self-assemble even as the agents co-evolve meaningful representations and successful dispositions for using those representations. We will see how reinforcement on successful information sources also provides a mechanism whereby simpler games might compose to form more complex games. Along the way, I consider (...) how an old game might be appropriated to a new context by reinforcement on successful associations between old and new saliences. (shrink)

Affective valence is increasingly thought to be the common currency underlying all forms of intuitive, non-discursive decision making, in both humans and other animals. And it is thought to constitute the good or bad (pleasant or unpleasant) aspects of all desires, emotions, and moods. This article contrasts two theories of valence. According to one, valence is an experience-directed imperative (‘more of this!’ or ‘less of this!’); according to the other, valence is a representation of adaptive value or disvalue. The latter (...) is shown to provide the better account. (shrink)

Emotion plays a crucial role, both in general human experience and in psychiatric illnesses. Despite the importance of emotion, the relative lack of objective methodologies to scientifically studying emotional phenomena limits our current understanding and thereby calls for the development of novel methodologies, such us the study of illustrative animal models. Analysis of Drosophila and other insects has unlocked new opportunities to elucidate the behavioral phenotypes of fundamentally emotional phenomena. Here we propose an integrative model of basic emotions based on (...) observations of this animal model. The basic emotions are internal states that are modulated by neuromodulators, and these internal states are externally expressed as certain stereotypical behaviors, such as instinct, which is proposed as ancient mechanisms of survival. There are four kinds of basic emotions: happiness, sadness, fear, and anger, which are differentially associated with three core affects: reward (happiness), punishment (sadness), and stress (fear and anger). These core affects are analogous to the three primary colors (red, yellow, and blue) in that they are combined in various proportions to result in more complex “higher order” emotions, such as love and aesthetic emotion. We refer to our proposed model of emotions as called the “ Three Primary Color Model of Basic Emotions.”. (shrink)

It is generally accepted that, in the cognitive and neural sciences, there are both computational and mechanistic explanations. We ask how computational explanations can integrate into the mechanistic hierarchy. The problem stems from the fact that implementation and mechanistic relations have different forms. The implementation relation, from the states of an abstract computational system to the physical, implementing states is a homomorphism mapping relation. The mechanistic relation, however, is that of part/whole; the explaining features in a mechanistic explanation are the (...) components of the explanandum phenomenon and their causal organization. Moreover, each component in one level of mechanism is constituted and explained by components of an underlying level of mechanism. Hence, it seems, computational variables and functions cannot be mechanistically explained by the medium-dependent states and properties that implement them. How then, do the computational and the implementational integrate to create the mechanistic hierarchy? After explicating the general problem, we further demonstrate it through a concrete example, of reinforcement learning, in the cognitive and neural sciences. We then examine two possible solutions. On one solution, the mechanistic hierarchy embeds at the same levels computational and implementational properties. This picture fits with the view that computational explanations are mechanistic sketches. On the other solution, there are two separate hierarchies, one computational and another implementational, which are related by the implementation relation. This picture fits with the view that computational explanations are functional and autonomous explanations. It is less clear how these solutions fit with the view that computational explanations are full-fledged mechanistic explanations. Finally, we argue that both pictures are consistent with the reinforcement learning example, but that scientific practice does not align with the view that computational models are merely mechanistic sketches. (shrink)

I introduce an extension of the Lewis-Skyrms signaling game, analysed from a dynamical perspective via simple reinforcement learning. In Lewis’ (Convention, Blackwell, Oxford, 1969) conception of a signaling game, salience is offered as an explanation for how individuals may come to agree upon a linguistic convention. Skyrms (Signals: evolution, learning & information, Oxford University Press, Oxford, 2010a) offers a dynamic explanation of how signaling conventions might arise presupposing no salience whatsoever. The extension of the atomic signaling game examined here—which I (...) will refer to as a salience game—introduces a variable parameter into the atomic signaling game which allows for degrees of salience, thus filling in the continuum between Skyrms’ and Lewis’ models. The model does not presuppose any salience at the outset, but illustrates a process by which accidentally evolved salience is amplified, to the benefit of the players. It is shown that increasing degrees of salience allow populations to avoid sub-optimal pooling equilibria and to coordinate upon conventions more quickly. (shrink)

Some mental states have valence—they are pleasant or unpleasant. According to imperativism, valence depends on imperative content, while evaluativism tells us that it depends on evaluative content. We argue that if one considers valence’s informational profile, it becomes evident that imperativism is superior to evaluativism. More precisely, we show that if one applies the best available metasemantics to the role played by (un)pleasant mental states in our cognitive economy, then these states turn out to have imperative rather than evaluative content, (...) since: (i) they are much more informative about behaviour than they are about the world; and (ii) they occupy a stage in the information-processing chain that is closer to behaviour production than it is to the uptake of sensory information. This is our metasemantic argument for imperativism. (shrink)

Natural languages are compositional in that the meaning of complex expressions depends on those of the parts and how they are put together. Here, I ask the following question: why are languages compositional? I answer this question by extending Lewis–Skyrms signaling games with a rudimentary form of compositional signaling and exploring simple reinforcement learning therein. As it turns out: in complex worlds, having compositional signaling helps simple agents learn to communicate. I am also able to show that learning the meaning (...) of a function word, once meanings of atomic words are known, presents no difficulty. (shrink)

The medial prefrontal cortex (mPFC) has been the subject of intense interest as a locus of cognitive control. Several computational models have been proposed to account for a range of effects, including error detection, conflict monitoring, error likelihood prediction, and numerous other effects observed with single-unit neurophysiology, fMRI, and lesion studies. Here, we review the state of computational models of cognitive control and offer a new theoretical synthesis of the mPFC as signaling response–outcome predictions. This new synthesis has two interacting (...) components. The first component learns to predict the various possible outcomes of a planned action, and the second component detects discrepancies between the actual and intended responses; the detected discrepancies in turn update the outcome predictions. This single construct is consistent with a wide array of performance monitoring effects in mPFC and suggests a unifying account of the cognitive role of medial PFC in performance monitoring. (shrink)

In the past two decades, reinforcement learning has become a popular framework for understanding brain function. A key component of RL models, prediction error, has been associated with neural signals throughout the brain, including subcortical nuclei, primary sensory cortices, and prefrontal cortex. Depending on the location in which activity is observed, the functional interpretation of prediction error may change: Prediction errors may reflect a discrepancy in the anticipated and actual value of reward, a signal indicating the salience or novelty of (...) a stimulus, and many other interpretations. Anterior cingulate cortex has long been recognized as a region involved in processing behavioral error, and recent computational models of the region have expanded this interpretation to include a more general role for the region in predicting likely events, broadly construed, and signaling deviations between expected and observed events. Ongoing modeling work investigating the interaction between ACC and additional regions involved in cognitive control suggests an even broader role for cingulate in computing a hierarchically structured surprise signal critical for learning models of the environment. The result is a predictive coding model of the frontal lobes, suggesting that predictive coding may be a unifying computational principle across the neocortex. (shrink)

Veit’s central claims are, first, that the function of valenced consciousness is to deal with pathological complexity, and, second, that pathological complexity is a trade-off problem associated with maximizing fitness. I argue that Veit’s hints about what pathological complexity amounts to pull in conflicting directions, and that the specific contribution of consciousness to dealing with a computational problem is under-motivated.

Tension and suspense are powerful emotional experiences that occur in a wide variety of contexts (e.g., in music, film, literature, and everyday life). The omnipresence of tension experiences suggests that they build on very basic cognitive and affective mechanisms. However, the psychological underpinnings of tension experiences remain largely unexplained, and tension and suspense are rarely discussed from a general, domain-independent perspective. In this paper, we argue that tension experiences in different contexts (e.g., musical tension or suspense in a movie) build (...) on the same underlying psychological processes. We discuss key components of tension experiences and propose a domain-independent model of tension and suspense. According to this model, tension experiences originate from states of conflict, instability, dissonance, or uncertainty that trigger predictive processes directed at future events of emotional significance. We also discuss possible neural mechanisms underlying experiences of tension. The model provides a theoretical framework that can inform future empirical research on tension phenomena. (shrink)

Recent studies indicate that the processing of an unexpected word is costly when the initial, disconfirmed prediction was strong. This penalty was suggested to stem from commitment to the strongly predicted word, requiring its inhibition when disconfirmed. Additional studies show that comprehenders rationally adapt their predictions in different situations. In the current study, we hypothesized that since the disconfirmation of strong predictions incurs costs, it would also trigger adaptation mechanisms influencing the processing of subsequent strong predictions. In two experiments, participants (...) made speeded congruency judgments on two-word phrases in which the first word was either highly constraining or not. We manipulated the proportion of disconfirmed predictions in highly constraining contexts between participants. The results provide additional evidence of the costs associated with the disconfirmation of strong predictions. Moreover, they show a reduction in these costs when participants experience a high proportion of disconfirmed strong predictions throughout the experiment, indicating that participants adjust the strength of their predictions when strong prediction is discouraged. We formulate a Bayesian adaptation model whereby prediction failure cost is weighted by the participant’s belief about the likelihood of encountering the expected word, and show that it accounts for the trial-by-trial data. (shrink)

What are the distinctive psychological features that explain why humans are free, but many other creatures, such as simple animals, are not? It is natural to think that the answer has something to do with unique human capacities for decision-making. Philosophical discussions of how decision-making works, however, are tellingly incomplete. In particular, these discussions invariably presuppose an agent who has a mentally represented set of options already fully in hand. The emphasis is largely on the selective processes that identify the (...) options most worth doing and then execute them. But where do mentally represented sets of options come from in the first place? Once we focus on this constructive aspect of decision-making, an important fact becomes apparent: While the option sets of simpler animals are sharply limited, humans have a number of striking psychological powers—including remarkable powers of prospection and creativity—that enable them to construct option sets of unparalleled size and diversity. As a result, humans can express themselves in countless ways. This latitude for self-expression is, I argue, the distinctive feature that explains why humans are free. (shrink)

Rationalist accounts of self-knowledge are motivated in important part by the claim that only by looking to our reasons to discover our beliefs and desires are we active in relation to them and only thereby do we take responsibility for them. These kinds of account seem to predict that self-knowledge generated using third-personal methods or analogues of these methods will tend to undermine the capacity to exercise self-control. In this light, the insistence by treatment programs that addicts acknowledge that they (...) are addicts seems puzzling. I argue that because addicts—and perhaps ordinary akratics, to some extent at least, too—are vulnerable to losing control of their actions via losing control over their beliefs, advising them to look to their reasons for actions is counterproductive and facilitates loss of control. In contrast, an insistence on what I call impersonal self-knowledge, knowledge of some of one's states and dispositions generated by third-personal means, may help to reestablish control. (shrink)

According to the reward-prediction error hypothesis of dopamine, the phasic activity of dopaminergic neurons in the midbrain signals a discrepancy between the predicted and currently experienced reward of a particular event. It can be claimed that this hypothesis is deep, elegant and beautiful, representing one of the largest successes of computational neuroscience. This paper examines this claim, making two contributions to existing literature. First, it draws a comprehensive historical account of the main steps that led to the formulation and subsequent (...) success of the RPEH. Second, in light of this historical account, it explains in which sense the RPEH is explanatory and under which conditions it can be justifiably deemed deeper than the incentive salience hypothesis of dopamine, which is arguably the most prominent contemporary alternative to the RPEH. (shrink)

Several attempts have been made previously to provide a biological grounding for cognitive architectures by relating their components to the computations of specific brain circuits. Often, the architecture's action selection system is identified with the basal ganglia. However, this identification overlooks one of the most important features of the basal ganglia—the existence of a direct and an indirect pathway that compete against each other. This characteristic has important consequences in decision-making tasks, which are brought to light by Parkinson's disease as (...) well as genetic differences in dopamine receptors. This paper shows that a standard model of action selection in a cognitive architecture cannot replicate any of these findings, details an alternative solution that reconciles action selection in the architecture with the physiology of the basal ganglia, and extends the domain of application of cognitive architectures. The implication of this solution for other architectures and existing models are discussed. (shrink)

The moral enhancement of humans by biological or genetic means has recently been urged as a response to the pressing concerns facing human civilization. In this paper, I argue that proponents of biological moral enhancement have misrepresented the facts of human moral psychology. As a result, the likely effectiveness of traditional methods of moral enhancement has been underestimated, relative to biological or genetic means. I review arguments in favor of biological moral enhancement and argue that the complexity of moral psychology (...) raises serious problems for such interventions. I offer a programmatic sketch of the ways in which our improved understanding of moral psychology can help facilitate more traditional methods of moral enhancement. I conclude that the best response to the dangers faced by human civilization is the continued use of traditional methods of moral enhancement and the use of our improved understanding of moral psychology to further refine and develop these methods. (shrink)

Patients with damage to the ventromedial prefrontal cortex (VMPFC) are often described as having impaired ability for planning and decision making despite retaining intact capacities for explicit reasoning. The somatic marker hypothesis is that the VMPFC associates implicitly represented affective information with explicit representations of actions or outcomes. Consequently, when the VMPFC is damaged explicit reasoning is no longer scaffolded by affective information, leading to characteristic deficits. These deficits are exemplified in performance on the Iowa Gambling Task (IGT) in which (...) subjects with VMPFC perform significantly worse than neurotypicals in a task which requires them learn from rewarding and punishing experience to make decisions. The somatic marker theory adopts a canonical theory of emotion, in which emotions function as part of a valencing system, to explain the role of affective processes. The first part of the paper argues against this canonical account. The second part provides a different account of the role of the role of the VMPFC in decision-making which does not depend on the canonical account of emotion. Together the first and second parts of the paper provide the basis for a different interpretation of results on the Iowa Gambling Task (IGT). In fact the IGT may be probing a deficit in what has been called mental time travel: the ability to access and use information from previous experience and imaginatively rehearse future experiences as part of the process of deliberation. (shrink)

We are concerned here with how structural properties of language may come to reflect features of the world in which it evolves. As a concrete example, we will consider how a simple term language might evolve to support the principle of indifference over state descriptions in that language. The point is not that one is justified in applying the principle of indifference to state descriptions in natural language. Instead, it is that one should expect a language that has evolved in (...) the context of facilitating successful action to reflect probabilistic features of the world in which it evolved. (shrink)

This paper is aimed at identifying how a model’s explanatory power is constructed and identified, particularly in the practice of template-based modeling (Humphreys, Philos Sci 69:1–11, 2002; Extending ourselves: computational science, empiricism, and scientific method, 2004), and what kinds of explanations models constructed in this way can provide. In particular, this paper offers an account of non-causal structural explanation that forms an alternative to causal–mechanical accounts of model explanation that are currently popular in philosophy of biology and cognitive science. Clearly, (...) defences of non-causal explanation are far from new (e.g. Batterman, Br J Philos Sci 53:21–38, 2002a; The devil in the details: asymptotic reasoning in explanation, reduction, and emergence, 2002b; Pincock, Noûs 41:253–275, 2007; Mathematics and scientific representation 2012; Rice, Noûs. doi:10.1111/nous.12042, 2013; Biol Philos 27:685–703, 2012), so the targets here are focused on a particular type of robust phenomenon and how strong invariance to interventions can block a range of causal explanations. By focusing on a common form of model construction, the paper also ties functional or computational style explanations found in cognitive science and biology more firmly with explanatory practices across model-based science in general. (shrink)