The ‘black box’ in Stoic axiology refers to the mysterious connection between the input of Stoic deliberation (reasons generated by the value of indifferents) and the output (appropriate actions). In this paper, I peer into the black box by drawing an analogy between Stoic and Kantian axiology. The value and disvalue of indifferents is intrinsic, but conditional. An extrinsic condition on the value of a token indifferent is that one's selection of that indifferent is sanctioned by context-relative ethical (...) principles. The value of an indifferent that does not meet this condition is normatively silenced, such that it fails to constitute a reason for action. (shrink)

Without doctors being able to explain medical decisions to patients, I argue their use of black box AIs would erode the effective and respectful care they provide patients. In addition, I argue that physicians should use AI black boxes only for patients in dire straits, or when physicians use AI as a “co-pilot” (analogous to a spellchecker) but can independently confirm its accuracy. I respond to A.J. London’s objection that physicians already prescribe some drugs without knowing why they (...) work. (shrink)

My aim in this article is to contribute to the larger project of assessing the relative merits of different theories of substance. An important preliminary step in this project is assessing the explanatory resources of one main theory of substance, the so-called bundle theory. This article works towards such an assessment. I identify and explain three distinct explanatory challenges an adequate bundle theory must meet. Each points to a putative explanatory gap, so I call them the Gap Challenges. I consider (...) three bundle-theoretic strategies for meeting these challenges. I argue that none of them goes very far. The upshot is that, absent other strategies for meeting the challenges, bundle theory involves a significant amount of stipulation. This black box makes bundle theory relatively weak with respect to its explanatory power—unless, of course, rival theories of substance are unable to do better. (shrink)

A ‘Black Box’ cannot be opened to reveal its mechanism. Rather, its operations are inferred through input from (and output to) an ‘observer’. All of us are observers, who attempt to understand the Black Boxes that are Minds. The Black Box and its observer constitute a system, differing from either component alone: a ‘greater’ Black Box to any further-external-observer. To Glanville (1982), the further-external-observer probes the greater-Black-Box by interacting directly with its core Black Box, (...) ignoring that Box’s immediate observer. In later accounts, however, Glanville’s greater-Black-Box inexplicably becomes unitary. Why the discrepancy? To resolve it, we start with von Foerster’s archetype ‘machines’, that are of two kinds: ‘Trivial’ (predictable) and ‘Non-Trivial’ (non-predictable). Early-on, Glanville treated the core Black Box and its observer as Trivial Machines, that gradually ‘whiten’ (reveal) each other though input and output, becoming ‘white boxes’. Later, however, Black Box and observer became Non-Trivial Machines, never fully ‘whitenable’. But Non-Trivial Machines can be concatenated from Trivial Machines, and are the only true Black Boxes; any greater-Black-Box (Non-Trivial Machine) may (within its core Black Box) involve white boxes (that are Trivial Machines). White boxes, therefore, could be the ultimate source of the greatest Black Box of all: the Mind. (shrink)

Fears of black-box algorithms are multiplying. Black-box algorithms are said to prevent accountability, make it harder to detect bias and so on. Some fears concern the epistemology of black-box algorithms in medicine and the ethical implications of that epistemology. Durán and Jongsma (2021) have recently sought to allay such fears. While some of their arguments are compelling, we still see reasons for fear.

Computational systems biologists create and manipulate computational models of biological systems, but they do not always have straightforward epistemic access to the content and behavioural profile of such models because of their length, coding idiosyncrasies, and formal complexity. This creates difficulties both for modellers in their research groups and for their bioscience collaborators who rely on these models. In this paper we introduce a new kind of visualization that was developed to address just this sort of epistemic opacity. The visualization (...) is unusual in that it depicts the dynamics and structure of a computer model instead of that model’s target system, and because it is generated algorithmically. Using considerations from epistemology and aesthetics, we explore how this new kind of visualization increases scientific understanding of the content and function of computer models in systems biology to reduce epistemic opacity. (shrink)

This paper concerns the Black Box. It is not the engineer’s black box that can be opened to reveal its mechanism, but rather one whose operations are inferred through input from (and output to) a companion observer. We are observers ourselves, and we attempt to understand minds through interactions with their host organisms. To this end, Ranulph Glanville followed W. Ross Ashby in elaborating the Black Box. The Black Box and its observer together form a system (...) having different properties than either component alone, making it a greater Black Box to any further-external observer. How far into this greater box can a further-external observer probe? The answer is crucial to understanding Black Boxes, and so an answer is offered here. It employs von Foerster’s machines, abstract entities having mechano-electrical bases, just like putative Black Boxes. Von Foerster follows Turing, Ashby, E. F. Moore, and G. H. Mealy in recognizing archetype machines that he calls trivial (predictable) and non-trivial (non-predictable). It is argued here that non-trivial machines are the only true Black Boxes. But non-trivial machines can be concatenated from trivial machines. Hence, the utter core of any greater Black Box (a non-trivial machine) may involve two (or more) White Boxes (trivial machines). This is how an unpredictable thing emerges from predictable parts. Interactions of White Boxes—of trivial machines—may be the ultimate source of the mind. Keywords: . (shrink)

This chapter explores the idea that causal inference is warranted if and only if the mechanism underlying the inferred causal association is identified. This mechanistic stance is discernible in the epidemiological literature, and in the strategies adopted by epidemiologists seeking to establish causal hypotheses. But the exact opposite methodology is also discernible, the black box stance, which asserts that epidemiologists can and should make causal inferences on the basis of their evidence, without worrying about the mechanisms that might underlie (...) their hypotheses. I argue that the mechanistic stance is indeed a bad methodology for causal inference. However, I detach and defend a mechanistic interpretation of causal generalisations in epidemiology as existence claims about underlying mechanisms. (shrink)

This article argues that actor-network theory, as an alternative to critical theory, has lost its critical impetus when examining commodification in healthcare. The paper claims that the reason for this, is the way in which actor-network theory’s anti-essentialist ontology seems to black box 'intentionality' and ethics of human agency as contingent interests. The purpose of this paper was to open the normative black box of commodification, and compare how Marxism, Habermas and ANT can deal with commodification and ethics (...) in healthcare. Moreover, a new account of 'intentionality' in critical thinking was elaborated. Using Strawson's analysis of 'reactive attitudes and resentment,' the ethical implications of commodification in health care were examined as an assessment of intentions. Synthesizing critical theory with the relational materialism of actor-network theory, this article advances a new approach that seeks to bridge interdisciplinary boundaries, and guide actor-network theory in a critical and humanist direction. Providing new theoretical insights on commodification and 'intentionality' in health care. (shrink)

Federated learning (FL) is a machine learning approach that allows multiple devices or institutions to collaboratively train a model without sharing their local data with a third-party. FL is considered a promising way to address patient privacy concerns in medical artificial intelligence. The ethical risks of medical FL systems themselves, however, have thus far been underexamined. This paper aims to address this gap. We argue that medical FL presents a new variety of opacity -- federation opacity -- that, in turn, (...) generates a distinctive *double* black box problem in healthcare AI. We highlight several instances in which the anticipated benefits of medical FL may be exaggerated, and conclude by highlighting key challenges that must be overcome to make FL ethically feasible in medicine. (shrink)

We introduce and discuss a knowledge-driven distillation approach to explaining black-box models by means of two kinds of interpretable models. The first is perceptron (or threshold) connectives, which enrich knowledge representation languages such as Description Logics with linear operators that serve as a bridge between statistical learning and logical reasoning. The second is Trepan Reloaded, an ap- proach that builds post-hoc explanations of black-box classifiers in the form of decision trees enhanced by domain knowledge. Our aim is, firstly, (...) to target a model-agnostic distillation approach exemplified with these two frameworks, secondly, to study how these two frameworks interact on a theoretical level, and, thirdly, to investigate use-cases in ML and AI in a comparative manner. Specifically, we envision that user-studies will help determine human understandability of explanations generated using these two frameworks. (shrink)

Machine learning algorithms may radically improve our ability to diagnose and treat disease. For moral, legal, and scientific reasons, it is essential that doctors and patients be able to understand and explain the predictions of these models. Scalable, customisable, and ethical solutions can be achieved by working together with relevant stakeholders, including patients, data scientists, and policy makers.

Im Beitrag wird die Apple-Watch vor dem Hintergrund ihrer „Ästhetik der Existenz“ als biopolitisches Artefakt und kontrollgesellschaftliches Dispositiv, vor allem aber als kybernetische Black Box aufgefasst und analysiert. Ziel des Aufsatzes ist es, aufzuzeigen, dass sich in dem feedbacklogischen Rückkopplungsapparat nicht nur grundlegende Diskurse des digitalen Zeitalters (Prävention, Gesundheit, bio- und psychopolitische Regulierungsformen etc.) verdichten, sondern dass dieser schon ob seiner inhärenten Logik qua Opazität Transparenz, qua Komplexität Simplizität (d.h. Orientierung) generiert und damit nicht zuletzt ein ganz spezifisches Menschenbild (...) forciert. In methodischer Hinsicht sollen zur Aufschlüsselung der für die digitale Gesellschaft repräsentativen Black Box „Apple-Watch“ wirksame Begriffskonstellationen der Kybernetik – d.h. Feedback, Regulierung, Selbstkontrolle und nicht zuletzt das Prinzip des „schwarzen Kastens“ selbst (Wiener/ Ashby), in dem sich schon die Funktion des Apparats als (Selbst-)Regulierungstool zwischen In- und Outputs reflektiert – in den Blick genommen und mit behavioristischen Konzepten (besonders solche der Gegenwart, d.h. nudging-Konzepte etc.) kontextualisiert werden. Auf dieser Grundlage lässt sich schließlich, so die These, nicht nur ein Subjektivierungsmodell ableiten, sondern auch eine (durchaus realpolitisch wirksame) Regierungskunst („kybernetische Gouvernementalität“) dekodieren. In diesem Zusammenhang ist schließlich die These zu plausibilisieren, dass das zeitgenössische „Sicherheitsregime“ des ‚new dark age‘ (Bridle) respektive der ‚Black-Box-Society‘ (Pasquale) sich nicht auf „die Unterdrückung von Subjektivität [bezieht], sondern vor allem auf ihre ‚(Selbst)-Produktion‘, […] auf die […] Förderung von Selbsttechnologien, die an Regierungsziele gekoppelt werden können.“ (Lemke) Entsprechend fragt der Aufsatz, welche – auch politischen – Auswirkungen die exzessive Verbreitung dieser und ähnlicher „smart devices“ zeitigt: Denn nicht zuletzt lässt sich, mediiert über derartige datengenerierende schwarze Kästen, eine Form der Regierung und des Regierbar-Machens erkennen, die weniger explizit-imperativisch auf Subjekte als vielmehr implizit-suggestiv auf Datenpunkte und -profile einwirkt. (shrink)

Even where an act appears to be responsible, and satisfies all the conditions for responsibility laid down by society, the response to it may be unjust where that appearance is false, and where those conditions are insufficient. This paper argues that those who want to place considerations of responsibility at the centre of distributive and criminal justice ought to take this concern seriously. The common strategy of relying on what Susan Hurley describes as a 'black box of responsibility' has (...) the advantage of not taking responsibility considerations to be irrelevant merely because some specific account of responsibility is mistaken. It can, furthermore, cope perfectly well with an absence of responsibility, even of the global sort implied by hard determinism and other strongly sceptical accounts. Problems for the black box view come in where responsibility is present, but in a form that is curtailed in one significant regard or another. The trick, then, is to open the box of responsibility just enough that its contents can be the basis for judgements of justice. I identify three 'moderately sceptical' forms of compatibilism that cannot ground judgements of justice, and are therefore expunged by the strongest 'grey box' view. (shrink)

Advanced AI systems are rapidly making their way into medical research and practice, and, arguably, it is only a matter of time before they will surpass human practitioners in terms of accuracy, reliability, and knowledge. If this is true, practitioners will have a prima facie epistemic and professional obligation to align their medical verdicts with those of advanced AI systems. However, in light of their complexity, these AI systems will often function as black boxes: the details of their contents, (...) calculations, and procedures cannot be meaningfully understood by human practitioners. When AI systems reach this level of complexity, we can also speak of black-box medicine. In this paper, we want to argue that black-box medicine conflicts with core ideals of patient-centered medicine. In particular, we claim, black-box medicine is not conducive for supporting informed decision-making based on shared information, shared deliberation, and shared mind between practitioner and patient. (shrink)

Recent years have witnessed intensive efforts to specify which requirements ethical artificial intelligence (AI) must meet. General guidelines for ethical AI consider a varying number of principles important. A frequent novel element in these guidelines, that we have bundled together under the term explicability, aims to reduce the black-box character of machine learning algorithms. The centrality of this element invites reflection on the conceptual relation between explicability and the four bioethical principles. This is important because the application of general (...) ethical frameworks to clinical decision-making entails conceptual questions: Is explicability a free-standing principle? Is it already covered by the well-established four bioethical principles? Or is it an independent value that needs to be recognized as such in medical practice? We discuss these questions in a conceptual-ethical analysis, which builds upon the findings of an empirical document analysis. On the example of the medical specialty of radiology, we analyze the position of radiological associations on the ethical use of medical AI. We address three questions: Are there references to explicability or a similar concept? What are the reasons for such inclusion? Which ethical concepts are referred to? (shrink)

Artificial intelligence (AI) systems have demonstrated impressive performance across a variety of clinical tasks. However, notoriously, sometimes these systems are “black boxes.” The initial response in the literature was a demand for “explainable AI.” However, recently, several authors have suggested that making AI more explainable or “interpretable” is likely to be at the cost of the accuracy of these systems and that prioritizing interpretability in medical AI may constitute a “lethal prejudice.” In this paper, we defend the value of (...) interpretability in the context of the use of AI in medicine. Clinicians may prefer interpretable systems over more accurate black boxes, which in turn is sufficient to give designers of AI reason to prefer more interpretable systems in order to ensure that AI is adopted and its benefits realized. Moreover, clinicians may be justified in this preference. Achieving the downstream benefits from AI is critically dependent on how the outputs of these systems are interpreted by physicians and patients. A preference for the use of highly accurate black box AI systems, over less accurate but more interpretable systems, may itself constitute a form of lethal prejudice that may diminish the benefits of AI to—and perhaps even harm—patients. (shrink)

A gas in a box is perhaps the most important model system studied in thermodynamics and statistical mechanics. Usually, studies focus on the gas, whereas the box merely serves as an idealized confinement. The present article focuses on the box as the central object and develops a thermodynamic theory by treating the geometric degrees of freedom of the box as the degrees of freedom of a thermodynamic system. Applying standard mathematical methods to the thermody- namics of an empty box allows (...) equations with the same structure as those of cosmology and classical and quantum mechanics to be derived. The simple model system of an empty box is shown to have interesting connections to classical mechanics, special relativity, and quantum field theory. (shrink)

Artificial intelligence (AI) systems have demonstrated impressive performance across a variety of clinical tasks. However, notoriously, sometimes these systems are 'black boxes'. The initial response in the literature was a demand for 'explainable AI'. However, recently, several authors have suggested that making AI more explainable or 'interpretable' is likely to be at the cost of the accuracy of these systems and that prioritising interpretability in medical AI may constitute a 'lethal prejudice'. In this paper, we defend the value of (...) interpretability in the context of the use of AI in medicine. Clinicians may prefer interpretable systems over more accurate black boxes, which in turn is sufficient to give designers of AI reason to prefer more interpretable systems in order to ensure that AI is adopted and its benefits realised. Moreover, clinicians may be justified in this preference. Achieving the downstream benefits from AI is critically dependent on how the outputs of these systems are interpreted by physicians and patients. A preference for the use of highly accurate black box AI systems, over less accurate but more interpretable systems , may itself constitute a form of lethal prejudice that may diminish the benefits of AI to - and perhaps even harm - patients. (shrink)

Since the early days of the Explainable AI movement, post-hoc explanations have been praised for their potential to improve user understanding, promote trust, and reduce patient safety risks in black box medical AI systems. Recently, however, critics have argued that the benefits of post-hoc explanations are greatly exaggerated since they merely approximate, rather than replicate, the actual reasoning processes that black box systems take to arrive at their outputs. In this article, we aim to defend the value of (...) post-hoc explanations against this recent critique. We argue that even if post-hoc explanations do not replicate the exact reasoning processes of black box systems, they can still improve users’ functional understanding of black box systems, increase the accuracy of clinician-AI teams, and assist clinicians in justifying their AI-informed decisions. While post-hoc explanations are not a “silver bullet” solution to the black box problem in medical AI, we conclude that they remain a useful strategy for addressing the black box problem in medical AI. (shrink)

It is known that big data analytics and AI pose a threat to privacy, and that some of this is due to some kind of “black box problem” in AI. I explain how this becomes a problem in the context of justification for judgments and actions. Furthermore, I suggest distinguishing three kinds of opacity: 1) the subjects do not know what the system does (“shallow opacity”), 2) the analysts do not know what the system does (“standard black box (...) opacity”), or 3) the analysts cannot possibly know what the system might do (“deep opacity”). If the agents, data subjects as well as analytics experts, operate under opacity, then these agents cannot provide justifications for judgments that are necessary to protect privacy, e.g., they cannot give “informed consent”, or guarantee “anonymity.” It follows from these points that agents in big data analytics and AI ofen cannot make the judgments needed to protect privacy. So I conclude that big data analytics makes the privacy problems worse and the remedies less effective. As a positive note, I provide a brief outlook on technical ways to handle this situation. (shrink)

Machine learning (ML) systems are vulnerable to performance decline over time due to dataset shift. To address this problem, experts often suggest that ML systems should be regularly updated to ensure ongoing performance stability. Some scholarly literature has begun to address the epistemic and ethical challenges associated with different updating methodologies. Thus far, however, little attention has been paid to the impact of model updating on the ML-assisted decision-making process itself, particularly in the AI ethics and AI epistemology literatures. This (...) article aims to address this gap in the literature. It argues that model updating introduces a new sub-type of opacity into ML-assisted decision-making – update opacity – that occurs when users cannot understand how or why an update has changed the reasoning or behaviour of an ML system. This type of opacity presents a variety of distinctive epistemic and safety concerns that available solutions to the black box problem in ML are largely ill-equipped to address. A variety of alternative strategies may be developed or pursued to address the problem of update opacity more directly, including bi-factual explanations, dynamic model reporting, and update compatibility. However, each of these strategies presents its own risks or carries significant limitations. Further research will be needed to address the epistemic and safety concerns associated with model updating and update opacity going forward. (shrink)

This paper investigates whether it is possible to intentionally construct a structure that is judgementally inaccessible—i.e., a form that resists or negates the conditions of the Judgemental Triad: Constructivity, Coherence, and Resonance. We examine examples from architecture, cryptography, Al black-box systems, and abstract art to assess whether such structures truly lie beyond the reach of judgement, or whether their unjudgeability is a product of strategic opacity. We argue that the creation of judgementally closed systems is structurally possible but ontologically (...) unstable, and ethically suspect. Such systems do not transcend judgement—they collapse it, and in doing so, negate their own meaning. (shrink)

Simple idealized models seem to provide more understanding than opaque, complex, and hyper-realistic models. However, an increasing number of scientists are going in the opposite direction by utilizing opaque machine learning models to make predictions and draw inferences, suggesting that scientists are opting for models that have less potential for understanding. Are scientists trading understanding for some other epistemic or pragmatic good when they choose a machine learning model? Or are the assumptions behind why minimal models provide understanding misguided? In (...) this paper, using the case of deep neural networks, I argue that it is not the complexity or black box nature of a model that limits how much understanding the model provides. Instead, it is a lack of scientific and empirical evidence supporting the link that connects a model to the target phenomenon that primarily prohibits understanding. (shrink)

Recent breakthroughs in artificial intelligence have primarily resulted from training deep neural networks (DNNs) with vast numbers of adjustable parameters on enormous datasets. Due to their complex internal structure, DNNs are frequently characterized as inscrutable ``black boxes,'' making it challenging to interpret the mechanisms underlying their impressive performance. This opacity creates difficulties for explanation, safety assurance, trustworthiness, and comparisons to human cognition, leading to divergent perspectives on these systems. This chapter examines recent developments in interpretability methods for DNNs, with (...) a focus on interventionist approaches inspired by causal explanation in philosophy of science. We argue that these methods offer a promising avenue for understanding how DNNs process information compared to merely behavioral benchmarking and correlational probing. We review key interventionist methods and illustrate their application through practical case studies. These methods allow researchers to identify and manipulate specific computational components within DNNs, providing insights into their causal structure and internal representations. We situate these approaches within the broader framework of causal abstraction, which aims to align low-level neural computations with high-level interpretable models. While acknowledging current limitations, we contend that interventionist methods offer a path towards more rigorous and theoretically grounded interpretability research, potentially informing both AI development and computational cognitive neuroscience. (shrink)

As systems based on opaque Artificial Intelligence (AI) continue to flourish in diverse real-world applications, understanding these black box models has become paramount. In response, Explainable AI (XAI) has emerged as a field of research with practical and ethical benefits across various domains. This paper not only highlights the advancements in XAI and its application in real-world scenarios but also addresses the ongoing challenges within XAI, emphasizing the need for broader perspectives and collaborative efforts. We bring together experts from (...) diverse fields to identify open problems, striving to synchronize research agendas and accelerate XAI in practical applications. By fostering collaborative discussion and interdisciplinary cooperation, we aim to propel XAI forward, contributing to its continued success. Our goal is to put forward a comprehensive proposal for advancing XAI. To achieve this goal, we present a manifesto of 27 open problems categorized into nine categories. These challenges encapsulate the complexities and nuances of XAI and offer a road map for future research. For each problem, we provide promising research directions in the hope of harnessing the collective intelligence of interested stakeholders. (shrink)

This paper analyses the phenomenology and epistemology of chatbots such as ChatGPT and Bard. The computational architecture underpinning these chatbots are large language models (LLMs), which are generative artificial intelligence (AI) systems trained on a massive dataset of text extracted from the Web. We conceptualise these LLMs as multifunctional computational cognitive artifacts, used for various cognitive tasks such as translating, summarizing, answering questions, information-seeking, and much more. Phenomenologically, LLMs can be experienced as a “quasi-other”; when that happens, users anthropomorphise them. (...) For most users, current LLMs are black boxes, i.e., for the most part, they lack data transparency and algorithmic transparency. They can, however, be phenomenologically and informationally transparent, in which case there is an interactional flow. Anthropomorphising and interactional flow can, in some users, create an attitude of (unwarranted) trust towards the output LLMs generate. We conclude this paper by drawing on the epistemology of trust and testimony to examine the epistemic implications of these dimensions. Whilst LLMs generally generate accurate responses, we observe two epistemic pitfalls. Ideally, users should be able to match the level of trust that they place in LLMs to the degree that LLMs are trustworthy. However, both their data and algorithmic opacity and their phenomenological and informational transparency can make it difficult for users to calibrate their trust correctly. The effects of these limitations are twofold: users may adopt unwarranted attitudes of trust towards the outputs of LLMs (which is particularly problematic when LLMs hallucinate), and the trustworthiness of LLMs may be undermined. (shrink)

Recently, various philosophers have argued that we can obtain knowledge via the imagination. In particular, it has been suggested that we can come to know concrete, empirical matters of everyday significance by appropriately imagining relevant scenarios. Arguments for this thesis come in two main varieties: black box reliability arguments and constraints-based arguments. We suggest that both strategies are unsuccessful. Against black-box arguments, we point to evidence from empirical psychology, question a central case-study, and raise concerns about a (claimed) (...) evolutionary rationale for the imagination's reliability. Against the constraints-based account, we argue that to the extent that it works, this does not give rise to knowledge that is distinctively from the imagination. We conclude by suggesting that the imagination's role in raising possibilities, traditionally seen as part of the context of discovery, can in fact play a role in justification, including as a bulwark against certain sorts of skepticism. (shrink)

Techniques from artificial intelligence (AI) can be used in forensic evidence evaluation and are currently applied in biometric fields. However, it is generally not possible to fully understand how and why these algorithms reach their conclusions. Whether and how we should include such ‘black box’ algorithms in this crucial part of the criminal law system is an open question that has not only scientific but also ethical, legal, and philosophical angles. Ideally, the question should be debated by people with (...) diverse backgrounds. Here, we present a view on the question from the philosophy of science angle: computational reliabilism (CR). CR posits that we are justified in believing the output of an AI system, if we have grounds for believing its reliability. Under CR, these grounds are classified into ‘reliability indicators’ of three types: technical, scientific, and societal. This framework enables debates on the suitability of AI methods for forensic evidence evaluation that take a wider view than explainability and validation. We argue that we are justified in believing the AI’s output for forensic comparison of voices and forensic comparison of faces. Technical indicators include the validation of the AI algorithm in itself, validation of its application in the forensic setting, and case-based validation. Scientific indicators include the simple notion that we know faces and voices contain identifying information along with operationalizing well-established metrics and forensic practices. Societal indicators are the emerging scientific consensus on the use of these methods, as well as their application and interpretation by well-educated and certified practitioners. We expect expert witnesses to rely more on technical indicators to be justified in believing AIsystems, and triers-of-fact to rely more on societal indicators to believe the expert witness supported by the AIsystem. (shrink)

Darwin provided us with a powerful theoretical framework to explain the evolution of living systems. Natural selection alone, however, has sometimes been seen as insufficient to explain the emergence of new levels of selection. The problem is one of “circularity” for evolutionary explanations: how to explain the origins of Darwinian properties without already invoking their presence at the level they emerge. That is, how does evolution by natural selection commence in the first place? Recent results in experimental evolution suggest a (...) way forward: Paul Rainey and his collaborators have shown that Darwinian properties could be exogenously imposed via what they call “ecological scaffolding.” This could solve the “black box” dilemma faced by Darwinian explanations of new levels of organization. Yet, despite “scaffolding” recently becoming a popular term in the study of cognition, culture, and evolution, the concept has suffered from vagueness and ambiguity. This article aims to show that scaffolding can be turned into a proper scientific concept able to do explanatory work within the context of the major evolutionary transitions. Doing so will allow us to once again extend the scope of the Darwinian model of evolution by natural selection. (shrink)

One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents (IAs). This paper explores how decision matrix algorithms, via the belief-desire-intention model (...) for autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing for the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles. (shrink)

There are two distinct approaches to Bayesian modelling in cognitive science. Black-box approaches use Bayesian theory to model the relationship between the inputs and outputs of a cognitive system without reference to the mediating causal processes; while mechanistic approaches make claims about the neural mechanisms which generate the outputs from the inputs. This paper concerns the relationship between these two approaches. We argue that the dominant trend in the philosophical literature, which characterizes the relationship between black-box and mechanistic (...) approaches to Bayesian cognitive science in terms of the dichotomy between instrumentalism and realism, is misguided. We propose that the two distinctions are orthogonal: black-box and mechanistic approaches to Bayesian modelling can each be given either an instrumentalist or a realist interpretation. We argue that the current tendency to conflate black-box approaches with instrumentalism and mechanistic approaches with realism stems from unwarranted assumptions about the nature of scientific explanation, the ontological commitments of scientific theories, and the role of abstraction and idealization in scientific models. We challenge each of these assumptions to reframe the debates over Bayesian modelling in cognitive science. (shrink)

The increasing application of artificial intelligence (AI) to healthcare raises both hope and ethical concerns. Some advanced machine learning methods provide accurate clinical predictions at the expense of a significant lack of explainability. Alex John London has defended that accuracy is a more important value than explainability in AI medicine. In this article, we locate the trade-off between accurate performance and explainable algorithms in the context of distributive justice. We acknowledge that accuracy is cardinal from outcome-oriented justice because it helps (...) to maximize patients’ benefits and optimizes limited resources. However, we claim that the opaqueness of the algorithmic black box and its absence of explainability threatens core commitments of procedural fairness such as accountability, avoidance of bias, and transparency. To illustrate this, we discuss liver transplantation as a case of critical medical resources in which the lack of explainability in AI-based allocation algorithms is procedurally unfair. Finally, we provide a number of ethical recommendations for when considering the use of unexplainable algorithms in the distribution of health-related resources. (shrink)

Explainable AI (xAI) methods are important for establishing trust in using black-box models. However, recent criticism has mounted against current xAI methods that they disagree, are necessarily false, and can be manipulated, which has started to undermine the deployment of black-box models. Rudin (2019) goes so far as to say that we should stop using black-box models altogether in high-stakes cases because xAI explanations ‘must be wrong’. However, strict fidelity to the truth is historically not a desideratum (...) in science. Idealizations – the intentional distortions introduced to scientific theories and models – are commonplace in the natural sciences and are seen as a successful scientific tool. Thus, it is not falsehood qua falsehood that is the issue. In this paper, I outline the need for xAI research to engage in idealization evaluation. Drawing on the use of idealizations in the natural sciences and philosophy of science, I introduce a novel framework for evaluating whether xAI methods engage in successful idealizations or deceptive explanations (SIDEs). SIDEs evaluates whether the limitations of xAI methods, and the distortions that they introduce, can be part of a successful idealization or are indeed deceptive distortions as critics suggest. I discuss the role that existing research can play in idealization evaluation and where innovation is necessary. Through a qualitative analysis we find that leading feature importance methods and counterfactual explanations are subject to idealization failure and suggest remedies for ameliorating idealization failure. (shrink)

The present paper looks at one of the most thorough articles on the intelligence of GPT, research conducted by engineers at Microsoft. Although there is a great deal of value in their work, I will argue that, for familiar philosophical reasons, their methodology, ‘Black-box Interpretability’ is wrongheaded. But there is a better way. There is an exciting and emerging discipline of ‘Inner Interpretability’ (also sometimes called ‘White-box Interpretability’) that aims to uncover the internal activations and weights of models in (...) order to understand what they represent and the algorithms they implement. In my view, a crucial mistake in Black-box Interpretability is the failure to appreciate that how processes are carried out matters when it comes to intelligence and understanding. I can’t pretend to have a full story that provides both necessary and sufficient conditions for being intelligent, but I do think that Inner Interpretability dovetails nicely with plausible philosophical views of what intelligence requires. So the conclusion is modest, but the important point in my view is seeing how to get the research on the right track. Towards the end of the paper, I will show how some of the philosophical concepts can be used to further refine how Inner Interpretability is approached, so the paper helps draw out a profitable, future two-way exchange between Philosophers and Computer Scientists. (shrink)

This article addresses computational procedures that are no longer constrained by human modes of representation and considers how these procedures could be philosophically understood in terms of ‘algorithmic thought’. Research in deep learning is its case study. This artificial intelligence (AI) technique operates in computational ways that are often opaque. Such a black-box character demands rethinking the abstractive operations of deep learning. The article does so by entering debates about explainability in AI and assessing how technoscience and technoculture tackle (...) the possibility to ‘re-present’ the algorithmic procedures of feature extraction and feature learning to the human mind. The article thus mobilises the notion of incommensurability (originally developed in the philosophy of science) to address explainability as a communicational and representational issue, which challenges phenomenological and existential modes of comparison between human and algorithmic ‘thinking’ operations. (shrink)

This article outlines links between cybernetics and psychology through the black box metaphor using a tripartite narrative. The first part explores first-order cybernetic approaches to opening the black box. These developments run parallel to the decline of radical behaviorism and advancements in information processing theory and neuropsychology. We then describe how cybernetics migrates towards a second-order approach (expanding and questioning features of first-order inquiry), understanding applications of rule-based tools to sociocultural phenomena and dynamic mental models, inspiring radical constructivism, (...) and also accepting social constructivism. Psychology, however, enters the cognitive revolution, adhering to the computer metaphor of first-order cyberneticians to streamline human consciousness. The article concludes by outlining how second-order cybernetic approaches emerging in the 1990s may provide cues to psychologists to adopt mixed methods, and bioecological models in the information age, uniting understandings of observable human activity, inner perceptions, and physiological processes across contexts to understand consciousness. (shrink)

Explainability and comprehensibility of AI are important requirements for intelligent systems deployed in real-world domains. Users want and frequently need to understand how decisions impacting them are made. Similarly it is important to understand how an intelligent system functions for safety and security reasons. In this paper, we describe two complementary impossibility results (Unexplainability and Incomprehensibility), essentially showing that advanced AIs would not be able to accurately explain some of their decisions and for the decisions they could explain people would (...) not understand some of those explanations. (shrink)

The Argument from Inductive Risk (AIR) is taken to show that values are inevitably involved in making judgements or forming beliefs. After reviewing this conclusion, I pose cases which are prima facie counterexamples: the unreflective application of conventions, use of black-boxed instruments, reliance on opaque algorithms, and unskilled observation reports. These cases are counterexamples to the AIR posed in ethical terms as a matter of personal values. Nevertheless, it need not be understood in those terms. The values which load (...) a theory choice may be those of institutions or past actors. This means that the challenge of responsibly handling inductive risk is not merely an ethical issue, but is also social, political, and historical. (shrink)

This article analyzes emerging artificial intelligence (AI)-enhanced lie detection systems from ethical and human resource (HR) management perspectives. I show how these AI enhancements transform lie detection, followed with analyses as to how the changes can lead to moral problems. Specifically, I examine how these applications of AI introduce human rights issues of fairness, mental privacy, and bias and outline the implications of these changes for HR management. The changes that AI is making to lie detection are altering the roles (...) of human test administrators and human subjects, adding machine learning-based AI agents to the situation and establishing invasive data collection processes as well as introducing certain biases in results. I project that the potentials for pervasive and continuous lie detection initiatives (“truth machines”) are substantial, displacing human-centered efforts to establish trust and foster integrity in organizations. I argue that if it is possible for HR managers to do so, they should cease using technologically-based lie detection systems entirely and work to foster trust and accountability on a human scale. However, if these AI-enhanced technologies are put into place by organizations by law, agency mandate, or other compulsory measures, care should be taken that the impacts of the technologies on human rights and wellbeing are considered. The article explores how AI can displace the human agent in some aspects of lie detection and credibility assessment scenarios, expanding the prospects for inscrutable, “black box” processes and novel physiological constructs (such as “biomarkers of deceit”) that may increase the potential for such human rights concerns as fairness, mental privacy, and bias. Employee interactions with autonomous lie detection systems rather with than human beings who administer specific tests can reframe organizational processes and rules concerning the assessment of personal honesty and integrity. The dystopian projection of organizational life in which analyses and judgments of the honesty of one’s utterances are made automatically and in conjunction with one’s personal profile provides unsettling prospects for the autonomy of self-representation. (shrink)

As one of the basic tasks in natural language processing (NLP), named entity recognition (NER) is an important basic tool for downstream tasks of NLP, such as information extraction, syntactic analysis, machine translation and so on. The internal operation logic of the current name entity recognition model is black-box to the user, so the user has no basis to determine which name entity makes more sense. Therefore, a user-friendly explainable recognition process would be very useful for many people. In (...) this paper, we propose a novel interpretable method, BTPK (Binary Talmudic Public Announcement Logic model), to help users understand the internal recognition logic of the name entity recognition tasks based on Talmudic Public Announcement Logic. BTPK model can also capture the semantic information in the input sentences, that is, the context dependency of the sentence. We observed the public announcement of BTPK presents the inner decision logic of Bidirectional Recurrent Neural Networks (BRNNs), and the explanations obtained from a BTPK model show us how BRNNs essentially handle NER tasks. (shrink)

“AI4Science” refers to the use of Artificial Intelligence (AI) in scientific research. As AI systems become more widely used in science, we need guidelines for when such uses are acceptable and when they are unacceptable. To that end, I propose that the distinction between the context of discovery and the context of justification, which comes from philosophy of science, may provide a preliminary but still useful guideline for acceptable uses of AI in science. Given that AI systems used in scientific (...) research are black boxes, for the most part, we should use such systems in the context of discovery but not in the context of justification. The former refers to processes of idea generation, which may be unproblematically opaque whether they occur in human brains or artificial neural networks, whereas the latter refers to scientific methods by which scientific ideas are tested, confirmed, verified, and justified, which should be transparent. (shrink)

Nudge is a popular public policy tool that harnesses well-known biases in human judgement to subtly guide people’s decisions, often to improve their choices or to achieve some socially desirable outcome. Thanks to recent developments in artificial intelligence (AI) methods new possibilities emerge of how and when our decisions can be nudged. On the one hand, algorithmically personalized nudges have the potential to vastly improve human daily lives. On the other hand, blindly outsourcing the development and implementation of nudges to (...) “black box” AI systems means that the ultimate reasons for why such nudges work, that is, the underlying human cognitive processes that they harness, will often be unknown. In this paper, we unpack this concern by considering a series of examples and case studies that demonstrate how AI systems can learn to harness biases in human judgment to reach a specified goal. Drawing on an analogy in a philosophical debate concerning the methodology of economics, we call for the need of an interdisciplinary oversight of AI systems that are tasked and deployed to nudge human behaviours. (shrink)

The use of machine learning algorithms has become ubiquitous in hiring decisions. Recent studies have shown that many of these algorithms generate unlawful discriminatory effects in every step of the process. The training phase of the machine learning models used in these decisions has been identified as the main source of bias. For a long time, discrimination cases have been analyzed under the banner of disparate treatment and disparate impact, but these concepts have been shown to be ineffective in the (...) context of AI. This paper examines the possibility of studying algorithmic discrimination from the perspective of negligence law. Negligent selection and validation of datasets and classifiers is a salient cause of improper training in machine learning. It is a breach of an employer’s duty to protect others, a failure of its exercise of due care in the manner of choosing employees. Adopting negligence to algorithmic discrimination requires a reformulation of what is foreseeable and reasonable within the context of black box algorithms, and a shift in the burden of proof to the employer, who must present evidence of its efforts to avoid bias. (shrink)

With the present paper I maintain that the group field theory (GFT) approach to quantum gravity can help us clarify and distinguish the problems of spacetime emergence from the questions about the nature of the quanta of space. I will show that the mechanism of phase transition suggests a form of indifference between scales (or phases) and that such an indifference allows us to black-box questions about the nature of the ontology of the fundamental levels of the theory. I (...) consider the GFT approach to quantum gravity which derives spacetime as an emergent property from abstract non-geometrical tetrahedra through a processes of phase transition. Because the physical interpretation of the tetrahedra is problematic in view of their (non)-spatiotemporal nature, I will apply the considerations about phase transitions to GFT and conclude that the fundamental ontology of the theory and the mechanism of spacetime emergence can and should be treated separately. (shrink)

This essay proposes a functional reinterpretation of the concepts of free will and consciousness. Rather than treating them as metaphysical properties or ontological facts, it analyzes them as perceived phenomena that serve practical cognitive and behavioral functions. Drawing on a structured input-output model using the “black box” analogy, the essay classifies different types of behavioral patterns—deterministic, unstable, unique, and adaptive—and associates each with intuitive labels such as determinism, madness, free will, and consciousness, respectively. It argues that these labels function (...) as predictive tools tied to the consistency and distinctiveness of observed behavior, rather than to any intrinsic mental essence. In doing so, the essay offers a framework that may contribute to ongoing discussions in philosophy of mind, ethics, and artificial intelligence, by treating free will and consciousness as functional descriptions of system-level behavior, without committing to any specific metaphysical assumptions. (shrink)

Can AI and humans genuinely communicate? In this article, after giving some background and motivating my proposal (§1–3), I explore a way to answer this question that I call the ‘mental-behavioral methodology’ (§4–5). This methodology follows the following three steps: First, spell out what mental capacities are sufficient for human communication (as opposed to communication more generally). Second, spell out the experimental paradigms required to test whether a behavior exhibits these capacities. Third, apply or adapt these paradigms to test whether (...) an AI displays the relevant behaviors. If the first two steps are successfully completed, and if the AI passes the tests with human-like results, this constitutes evidence that this AI and humans can genuinely communicate. This mental-behavioral methodology has the advantage that we don’t need to understand the workings of black-box algorithms, such as standard deep neural networks. This is comparable to the fact that we don’t need to understand how human brains work to know that humans can genuinely communicate. This methodology also has its disadvantages and I will discuss some of them (§6). (shrink)

Background: Microbiome biomarker discovery for patient diagnosis, prognosis, and risk evaluation is attracting broad interest. Selected groups of microbial features provide signatures that characterize host disease states such as cancer or cardio-metabolic diseases. Yet, the current predictive models stemming from machine learning still behave as black boxes and seldom generalize well. Their interpretation is challenging for physicians and biologists, which makes them difficult to trust and use routinely in the physician-patient decision-making process. Novel methods that provide interpretability and biological (...) insight are needed. Here, we introduce "predomics", an original machine learning approach inspired by microbial ecosystem interactions that is tailored for metagenomics data. It discovers accurate predictive signatures and provides unprecedented interpretability. The decision provided by the predictive model is based on a simple, yet powerful score computed by adding, subtracting, or dividing cumulative abundance of microbiome measurements. -/- Results: Tested on >100 datasets, we demonstrate that predomics models are simple and highly interpretable. Even with such simplicity, they are at least as accurate as state-of-the-art methods. The family of best models, discovered during the learning process, offers the ability to distil biological information and to decipher the predictability signatures of the studied condition. In a proof-of-concept experiment, we successfully predicted body corpulence and metabolic improvement after bariatric surgery using pre-surgery microbiome data. -/- Conclusions: Predomics is a new algorithm that helps in providing reliable and trustworthy diagnostic decisions in the microbiome field. Predomics is in accord with societal and legal requirements that plead for an explainable artificial intelligence approach in the medical field. (shrink)

In the context of the digital economy development, the priority component of the economic security of an enterprise is changing from material to digital, constituting an independent element of enterprise security. The relevance of the present research is driven by the need to solve the issue of modernizing the economic security of the enterprise taking into account the new risks and opportunities of digitalization. The purpose of the academic paper lies in identifying the features of preventing internal and external negative (...) influences (threats) in order to guarantee the effective and stable functioning and dynamic social development of the enterprise in the context of digitalization. The research methods are as follows: general scientific research methods, in particular, logical analysis, theoretical substantiation, methods of induction and deduction, formalization and generalization, statistical observation. Results. It has been proposed to introduce the concept of “digital security of the enterprise” for replacing the concept of “information component of economic security” in order to bring the terminology in line with new economic realities. The implementation of the “black box” model has made it possible to identify the latest risks and threats to the economic security of an enterprise within the conditions of the digitalization that differs from the existing ones. The assessment of enterprises’ digital security of the European Union member states has revealed that the digital security level does not depend on the size of the country, however, it is influenced by the institutional environment (in particular, digital development tools in the EU) and the size of enterprises. Also, within the research framework, an assessment of the digital security level of enterprises in the context of digitalization has been proposed. In order to characterize enterprises by the level of digital security, a calculation procedure using the coefficient method has been proposed. (shrink)