What can contemporary machine learning (ML) models do? Given the proliferation of ML models in society, answering this question matters to a variety of stakeholders, both public and private. The evaluation of models' capabilities is rapidly emerging as a key subfield of modern ML, buoyed by regulatory attention and government grants. Despite this, the notion of an ML model possessing a capability has not been interrogated: what are we saying when we say that a model is able to do something? (...) And what sorts of evidence bear upon this question? -/- In this paper, we aim to answer these questions, using the capabilities of large language models (LLMs) as a running example. Drawing on the large philosophical literature on abilities, we develop an account of ML models' capabilities which can be usefully applied to the nascent science of model evaluation. Our core proposal is a conditional analysis of model abilities (CAMA): crudely, a machine learning model has a capability to X just when it would reliably succeed at doing X if it 'tried'. The main contribution of the paper is making this proposal precise in the context of ML, resulting in an operationalisation of CAMA applicable to LLMs. We then put CAMA to work, showing that it can help make sense of various features of ML model evaluation practice, as well as suggest procedures for performing fair inter-model comparisons. (shrink)

Discussions of artificial intelligence have been shaped by two brilliant thought-experiments: Turing’s Imitation Test for thinking systems and Searle’s Chinese Room Argument. In many ways, debates about large language models (LLMs) struggle to move beyond these original, opposing thought-experiments. So, in this paper, I ask whether we can move debate forward by exploring the features Sceptics about LLM abilities take to ground meaning. Section 1 sketches the options, while Sections 2 and 3 explore the common requirement for a robust relation (...) between linguistic signs and external objects. Section 3 argues that concerns about worldly connections can be met and thus that outputs of LLMs should be viewed as genuinely meaningful. Section 4 then turns to the argument that the kind of derived meaning explored in Section 3 is insufficient for LLMs to count as meaningful systems per se, looking at the claim that they must possess original intentionality before we admit them to the space of meaning. I argue that this demand is not a prerequisite for meaning per se, but rather for some kinds of agency or conscious understanding. I suggest that the demands for original intentionality are not currently met by LLMs and that we should be cautious about whether we want them to be. (shrink)

The present paper looks at one of the most thorough articles on the intelligence of GPT, research conducted by engineers at Microsoft. Although there is a great deal of value in their work, I will argue that, for familiar philosophical reasons, their methodology, ‘Black-box Interpretability’ is wrongheaded. But there is a better way. There is an exciting and emerging discipline of ‘Inner Interpretability’ (also sometimes called ‘White-box Interpretability’) that aims to uncover the internal activations and weights of models in order (...) to understand what they represent and the algorithms they implement. In my view, a crucial mistake in Black-box Interpretability is the failure to appreciate that how processes are carried out matters when it comes to intelligence and understanding. I can’t pretend to have a full story that provides both necessary and sufficient conditions for being intelligent, but I do think that Inner Interpretability dovetails nicely with plausible philosophical views of what intelligence requires. So the conclusion is modest, but the important point in my view is seeing how to get the research on the right track. Towards the end of the paper, I will show how some of the philosophical concepts can be used to further refine how Inner Interpretability is approached, so the paper helps draw out a profitable, future two-way exchange between Philosophers and Computer Scientists. (shrink)

Contemporary AI models owe much of their success and discontents to connectionism, a framework in cognitive science that has been (and continues to be) highly influential. Herein, we analyze artificial neural networks (ANNs): a) when used as scientific instruments of study; and b) when functioning as emergent arbiters of the zeitgeist in the cognitive, computational, and neural sciences. Building on our previous work with respect to analogizing between ANNs and cognition, brains, or behaviour (Guest & Martin, 2023), we use metatheoretical (...) analysis techniques (Guest, 2024), including formal logic, to characterise two distinct tendencies within connectionism that we dub classical and modern, with divergent properties, e.g. goals, mechanisms, scientific questions. We also demonstrate how we, as a field, often fail to follow important lines of argument to their end --- this results in a paradoxical praxis. By engaging more deeply with (meta)theory surrounding ANNs, our field can obviate the cycle of AI winters and summers, which need not be inevitable. (shrink)

El propósito de este trabajo es analizar los objetos digitales que pueblan nuestras pantallas y redes: vídeos de YouTube, perfiles de Facebook, imágenes de Flickr, archivos de Dropbox, correos electrónicos, podcasts, blogs, y demás. Estos objetos están codificados en binario (0/1) y, en su forma más elemental, son bits y bytes. Para desentrañar estas entidades intangibles, que son realidades insertas en el tejido de nuestra existencia cotidiana, se examinan los supuestos ontológicos y epistemológicos que las rodean. Desde esta perspectiva, se (...) analizan diferentes marcos conceptuales y definiciones, así como los enfoques filosóficos más representativos. También se explora el papel central de los nuevos paradigmas computacionales, la infraestructura TIC y las tecnologías emergentes. Se introduce un modelo de realismo integrativo que encapsula las tesis recogidas a lo largo del documento. El texto concluye con unas breves reflexiones sobre los hallazgos. (shrink)