Despite its centrality in the philosophy of cognitive science, there has been little prior philosophical work engaging with the notion of representation in contemporary NLP practice. This paper attempts to fill that lacuna: drawing on ideas from cognitive science, I introduce a framework for evaluating the representational claims made about components of neural NLP models, proposing three criteria with which to evaluate whether a component of a model represents a property and operationalising these criteria using probing classifiers, a popular analysis (...) technique in NLP (and deep learning more broadly). The project of operationalising a philosophically-informed notion of representation should be of interest to both philosophers of science and NLP practitioners. It affords philosophers a novel testing-ground for claims about the nature of representation, and helps NLPers organise the large literature on probing experiments, suggesting novel avenues for empirical research. (shrink)

This paper argues that a range of current AI systems have learned how to deceive humans. We define deception as the systematic inducement of false beliefs in the pursuit of some outcome other than the truth. We first survey empirical examples of AI deception, discussing both special-use AI systems (including Meta's CICERO) built for specific competitive situations, and general-purpose AI systems (such as large language models). Next, we detail several risks from AI deception, such as fraud, election tampering, and losing (...) control of AI systems. Finally, we outline several potential solutions to the problems posed by AI deception: first, regulatory frameworks should subject AI systems that are capable of deception to robust risk-assessment requirements; second, policymakers should implement bot-or-not laws; and finally, policymakers should prioritize the funding of relevant research, including tools to detect AI deception and to make AI systems less deceptive. Policymakers, researchers, and the broader public should work proactively to prevent AI deception from destabilizing the shared foundations of our society. (shrink)

How should thinkers cope with uncertainty? Julia Staffel breaks new ground in the study of rationality by answering this question and many others. She also explains how it is better to be less irrational, because less irrational degrees of belief are generally more accurate and better at guiding our actions.

In everyday life we either express our beliefs in all-or-nothing terms or we resort to numerical probabilities: I believe it's going to rain or my chance of winning is one in a million. The Stability of Belief develops a theory of rational belief that allows us to reason with all-or-nothing belief and numerical belief simultaneously.

This article described three heuristics that are employed in making judgements under uncertainty: representativeness, which is usually employed when people are asked to judge the probability that an object or event A belongs to class or process B; availability of instances or scenarios, which is often employed when people are asked to assess the frequency of a class or the plausibility of a particular development; and adjustment from an anchor, which is usually employed in numerical prediction when a relevant value (...) is available. These heuristics are highly economical and usually effective, but they lead to systematic and predictable errors. A better understanding of these heuristics and of the biases to which they lead could improve judgements and decisions in situations of uncertainty. (shrink)

Richard Pettigrew offers an extended investigation into a particular way of justifying the rational principles that govern our credences. The main principles that he justifies are the central tenets of Bayesian epistemology, though many other related principles are discussed along the way. Pettigrew looks to decision theory in order to ground his argument. He treats an agent's credences as if they were a choice she makes between different options, gives an account of the purely epistemic utility enjoyed by different sets (...) of credences, and then appeals to the principles of decision theory to show that, when epistemic utility is measured in this way, the credences that violate the principles listed above are ruled out as irrational. The account of epistemic utility set out here is the veritist's: the sole fundamental source of epistemic utility for credences is their accuracy. Thus, Pettigrew conducts an investigation in the version of epistemic utility theory known as accuracy-first epistemology. (shrink)

Brian Hedden defends a radical view about the relationship between rationality, personal identity, and time. On the standard view, personal identity over time plays a central role in thinking about rationality, because there are rational norms for how a person's attitudes and actions at one time should fit with her attitudes and actions at other times. But these norms are problematic. They make what you rationally ought to believe or do depend on facts about your past that aren't part of (...) your current perspective on the world, and they make rationality depend on controversial, murky metaphysical facts about what binds different instantaneous snapshots into a single person extended in time. Hedden takes a different approach, treating the relationship between different time-slices of the same person as no different from the relationship between different people. On his account, the locus of rationality is the time-slice rather than the temporally extended agent. This impersonal, time-slice-centric approach to rationality yields a unified approach to the rationality of beliefs, preferences, and actions where what rationality demands of you is solely determined by your evidence, with no special weight given to your past beliefs or actions. (shrink)

Davidson attacks the intelligibility of conceptual relativism, i.e. of truth relative to a conceptual scheme. He defines the notion of a conceptual scheme as something ordering, organizing, and rendering intelligible empirical content, and calls the position that employs both notions scheme-content dualism. He argues that such dualism is untenable since: not only can we not parcel out empirical content sentence per sentence but also the notion of uninterpreted content to which several schemes are relative, and the related notion of a (...) theory ”fitting the evidence’, can be shown to lack intelligibility too. Davidson argues further that belief in incommensurable schemes or non-intertranslatable languages is possible only on violating a correct understanding of interpretability : if we succeed in interpreting someone else then we have shown there is no need to speak of two conceptual schemes, while if we fail ”there is no ground for speaking of two.’. (shrink)

Behaviour norms are considered for decision trees which allow both objective probabilities and uncertain states of the world with unknown probabilities. Terminal nodes have consequences in a given domain. Behaviour is required to be consistent in subtrees. Consequentialist behaviour, by definition, reveals a consequence choice function independent of the structure of the decision tree. It implies that behaviour reveals a revealed preference ordering satisfying both the independence axiom and a novel form of sure-thing principle. Continuous consequentialist behaviour must be expected (...) utility maximizing. Other plausible assumptions then imply additive utilities, subjective probabilities, and Bayes' rule. (shrink)

The pragmatic character of the Dutch book argument makes it unsuitable as an "epistemic" justification for the fundamental probabilist dogma that rational partial beliefs must conform to the axioms of probability. To secure an appropriately epistemic justification for this conclusion, one must explain what it means for a system of partial beliefs to accurately represent the state of the world, and then show that partial beliefs that violate the laws of probability are invariably less accurate than they could be otherwise. (...) The first task can be accomplished once we realize that the accuracy of systems of partial beliefs can be measured on a gradational scale that satisfies a small set of formal constraints, each of which has a sound epistemic motivation. When accuracy is measured in this way it can be shown that any system of degrees of belief that violates the axioms of probability can be replaced by an alternative system that obeys the axioms and yet is more accurate in every possible world. Since epistemically rational agents must strive to hold accurate beliefs, this establishes conformity with the axioms of probability as a norm of epistemic rationality whatever its prudential merits or defects might be. (shrink)

A critical examination of the Reflection principle in Bayesian epistemology, and of the diachronic Dutch-book-style arguments that have been invoked to support Reflection and Conditionalization.

What knowledge would suffice to yield an interpretation of an arbitrary utterance of a language when such knowledge is based on evidence plausibly available to a nonspeaker of that language? it is argued that it is enough to know a theory of truth for the language and that the theory satisfies tarski's 'convention t' and that it gives an optimal fit to data about sentences held true, Under specified conditions, By native speakers.

We consider the questions of whether or not large language models (LLMs) have beliefs, and, if they do, how we might measure them. First, we consider whether or not we should expect LLMs to have something like beliefs in the first place. We consider some recent arguments aiming to show that LLMs cannot have beliefs. We show that these arguments are misguided. We provide a more productive framing of questions surrounding the status of beliefs in LLMs, and highlight the empirical (...) nature of the problem. With this lesson in hand, we evaluate two existing approaches for measuring the beliefs of LLMs, one due to Azaria and Mitchell (The internal state of an llm knows when its lying, 2023) and the other to Burns et al. (Discovering latent knowledge in language models without supervision, 2022). Moving from the armchair to the desk chair, we provide empirical results that show that these methods fail to generalize in very basic ways. We then argue that, even if LLMs have beliefs, these methods are unlikely to be successful for conceptual reasons. Thus, there is still no lie-detector for LLMs. We conclude by suggesting some concrete paths for future work. (shrink)

Probability, Frequency and Reasonable Expectation.

This paper explores the space of (propositional) probabilistic logical languages, ranging from a purely `qualitative' comparative language to a highly `quantitative' language involving arbitrary polynomials over probability terms. While talk of qualitative vs. quantitative may be suggestive, we identify a robust and meaningful boundary in the space by distinguishing systems that encode (at most) additive reasoning from those that encode additive and multiplicative reasoning. The latter includes not only languages with explicit multiplication but also languages expressing notions of dependence and (...) conditionality. We show that the distinction tracks a divide in computational complexity: additive systems remain complete for NP, while multiplicative systems are robustly complete for ∃ℝ. We also address axiomatic questions, offering several new completeness results as well as a proof of non-finite-axiomatizability for comparative probability. Repercussions of our results for conceptual and empirical questions are addressed, and open problems are discussed. (shrink)

There is much to learn from Durán and Jongsma’s paper.1 One particularly important insight concerns the relationship between epistemology and ethics in medical artificial intelligence. In clinical environments, the task of AI systems is to provide risk estimates or diagnostic decisions, which then need to be weighed by physicians. Hence, while the implementation of AI systems might give rise to ethical issues—for example, overtreatment, defensive medicine or paternalism2—the issue that lies at the heart is an epistemic problem: how can physicians (...) know whether to trust decisions made by AI systems? In this manner, various studies examining the interaction of AI systems and physicians have shown that without being able to evaluate their trustworthiness, especially novice physicians become over-reliant on algorithmic support—and ultimately are led astray by incorrect decisions.3–5 This leads to a second insight from the paper, namely that even if some AI system happens to be opaque, it is still not built on the moon. To assess its trustworthiness, AI developers or physicians have different sorts of higher order evidence at hand. Most importantly, …. (shrink)

Classic analysis of the subject and the development of personal probability; one of the greatest controversies in modern statistcal thought. New preface and new footnotes to 1954 edition, with a supplementary 180-item annotated bibliography by author. Calculus, probability, statistics, and Boolean algebra are recommended.

This paper develops an account of the opacity problem in medical machine learning (ML). Guided by pragmatist assumptions, I argue that opacity in ML models is problematic insofar as it potentially undermines the achievement of two key purposes: ensuring generalizability and optimizing clinician–machine decision-making. Three opacity amelioration strategies are examined, with explainable artificial intelligence (XAI) as the predominant approach, challenged by two revisionary strategies in the form of reliabilism and the interpretability by design. Comparing the three strategies, I argue that (...) interpretability by design is most promising to overcome opacity in medical ML. Looking beyond the individual opacity amelioration strategies, the paper also contributes to a deeper understanding of the problem space and the solution space regarding opacity in medical ML. (shrink)