Artificial intelligence (AI) plays a rapidly increasing role in clinical care. Many of these systems, for instance, deep learning-based applications using multilayered Artificial Neural Nets, exhibit epistemic opacity in the sense that they preclude comprehensive human understanding. In consequence, voices from industry, policymakers, and research have suggested trust as an attitude for engaging with clinical AI systems. Yet, in the philosophical and ethical literature on medical AI, the notion of trust remains fiercely debated. Trust skeptics hold that talking about trust (...) in nonhuman agents constitutes a category error and worry about the concept being misused for ethics washing. Proponents of trust have responded to these worries from various angles, disentangling different concepts and aspects of trust in AI, potentially organized in layers or dimensions. Given the substantial disagreements across these accounts of trust and the important worries about ethics washing, we embrace a diverging strategy here. Instead of aiming for a positive definition of the elements and nature of trust in AI, we proceed ex negativo, that is we look at cases where trust or distrust are misplaced. Comparing these instances with trust expedited in doctor–patient relationships, we systematize these instances and propose a taxonomy of both misplaced trust and distrust. By inverting the perspective and focusing on negative examples, we develop an account that provides useful ethical constraints for decisions in clinical as well as regulatory contexts and that highlights how we should not engage with medical AI. (shrink)

Should we be worried that the concept of trust is increasingly used when we assess non-human agents and artefacts, say robots and AI systems? Whilst some authors have developed explanations of the concept of trust with a view to accounting for trust in AI systems and other non-agents, others have rejected the idea that we should extend trust in this way. The article advances this debate by bringing insights from conceptual engineering to bear on this issue. After setting up a (...) target concept of trust in terms of four functional desiderata (trust-reliance distinction, explanatory strength, tracking affective responses, and accounting for distrust), I analyze how agential vs. non-agential accounts can satisfy these. A final section investigates how ‘non-ideal’ circumstances—that is, circumstances where the manifest and operative concept use diverge amongst concept users—affect our choice about which rendering of trust is to be preferred. I suggest that some prominent arguments against extending the language of trust to non-agents are not decisive and reflect on an important oversight in the current debate, namely a failure to address how narrower, agent-centred accounts curtail our ability to distrust non-agents. (shrink)

We address an open problem in the philosophy of artificial intelligence (AI): how to justify the epistemic attitudes we have towards the trustworthiness of AI systems. The problem is important, as providing reasons to believe that AI systems are worthy of trust is key to appropriately rely on these systems in human-AI interactions. In our approach, we consider the trustworthiness of an AI as a time-relative, composite property of the system with two distinct facets. One is the actual trustworthiness of (...) the AI and the other is the perceived trustworthiness of the system as assessed by its users while interacting with it. We show that credences, namely, beliefs we hold with a degree of confidence, are the appropriate attitude for capturing the facets of the trustworthiness of an AI over time. Then, we introduce a reliabilistic account providing justification to the credences in the trustworthiness of AI, which we derive from Tang’s probabilistic theory of justified credence. Our account stipulates that a credence in the trustworthiness of an AI system is justified if and only if it is caused by an assessment process that tends to result in a high proportion of credences for which the actual and perceived trustworthiness of the AI are calibrated. This approach informs research on the ethics of AI and human-AI interactions by providing actionable recommendations on how to measure the reliability of the process through which users perceive the trustworthiness of the system, investigating its calibration to the actual levels of trustworthiness of the AI as well as users’ appropriate reliance on the system. (shrink)

Achieving trustworthy AI is increasingly considered an essential desideratum to integrate AI systems into sensitive societal fields, such as criminal justice, finance, medicine, and healthcare, among others. For this reason, it is important to spell out clearly its characteristics, merits, and shortcomings. This article is the first survey in the specialized literature that maps out the philosophical landscape surrounding trust and trustworthiness in AI. To achieve our goals, we proceed as follows. We start by discussing philosophical positions on trust and (...) trustworthiness, focusing on interpersonal accounts of trust. This allows us to explain why trust, in its most general terms, is to be understood as reliance plus some “extra factor”. We then turn to the first part of the definition provided, i.e., reliance, and analyze two opposing approaches to establishing AI systems’ reliability. On the one hand, we consider transparency and, on the other, computational reliabilism. Subsequently, we focus on debates revolving around the “extra factor”. To this end, we consider viewpoints that most actively resist the possibility and desirability of trusting AI systems before turning to the analysis of the most prominent advocates of it. Finally, we take up the main conclusions of the previous sections and briefly point at issues that remain open and need further attention. (shrink)

The burden of research ethics review falls not just on researchers, but on those who serve on research ethics committees (RECs). With the advent of automated text analysis and generative artificial intelligence, it has recently become possible to teach models to support human judgment, for example by highlighting relevant parts of a text and suggesting actionable precedents and explanations. It is time to consider how such tools might be used to support ethics review and oversight. This commentary argues that with (...) a suitable strategy of engagement, artificial intelligence can be used in a variety of ways that can genuinely support RECs and manage their workload and improve the quality of review. It would be wiser to take an active role in the development of AI tools for ethics review, rather than to adopt ad hoc tools after the fact. (shrink)

Trust and trustworthiness are essential for good healthcare, especially in mental healthcare. New technologies, such as mobile health apps, can affect trust relationships. In mental health, some apps need the trust of their users for therapeutic efficacy and explicitly ask for it, for example, through an avatar. Suppose an artificial character in an app delivers healthcare. In that case, the following questions arise: Whom does the user direct their trust to? Whether and when can an avatar be considered trustworthy? Our (...) study aims to analyze different dimensions of trustworthiness in the context of mobile health app use. We integrate O'Neill's account of autonomy, trust, and trustworthiness into a model of trustworthiness as a relational concept with four relata: B is trustworthy with respect to A regarding the performance of Z because of C. Together with O'Neill's criteria of trustworthiness (honesty, competence, and reliability), this four-sided model is used to analyze different dimensions of trustworthiness in an exemplary case of mobile health app use. Our example focuses on an app that uses an avatar and is intended to treat sleep difficulties. The conceptual analysis shows that interpreting trust and trustworthiness in health app use is multi-layered and involves a net of interwoven universal obligations. At the same time, O'Neill's approach to autonomy, trust, and trustworthiness offers a normative account to structure and analyze these complex relations of trust and trustworthiness using mobile health apps. (shrink)

Following the publication of numerous ethical principles and guidelines, the concept of 'Trustworthy AI' has become widely used. However, several AI ethicists argue against using this concept, often backing their arguments with decades of conceptual analyses made by scholars who studied the concept of trust. In this paper, I describe the historical-philosophical roots of their objection and the premise that trust entails a human quality that technologies lack. Then, I review existing criticisms about 'Trustworthy AI' and the consequence of ignoring (...) these criticisms: if the concept of 'Trustworthy AI' is kept being used, we risk attributing responsibilities to agents who cannot be held responsible, and consequently, deteriorate social structures which regard accountability and liability. Nevertheless, despite suggestions to shift the paradigm from 'Trustworthy AI' to 'Reliable AI', I argue that, realistically, this concept will be kept being used. I end by arguing that, ultimately, AI ethics is also about power, social justice, and scholarly activism. Therefore, I propose that community-driven and social justice-oriented ethicists of AI and trust scholars further focus on (a) democratic aspects of trust formation; and (b) draw attention to critical social aspects highlighted by phenomena of distrust. This way, it will be possible to further reveal shifts in power relations, challenge unfair status quos, and suggest meaningful ways to keep the interests of citizens. (shrink)

In view of the dramatic advancements in the development of artificial intelligence technology in recent years, it has become a commonplace to demand that AI systems be trustworthy. This view presupposes that it is possible to trust AI technology in the first place. The aim of this paper is to challenge this view. In order to do that, it is argued that the philosophy of trust really revolves around the problem of how to square the epistemic and the normative dimensions (...) of trust. Given this double nature of trust it is possible to extract a threefold challenge to the defenders of the possibility of AI trust without presupposing any particular trust theory. They have to show (1) how trust in AI systems is more than mere reliance; (2) how AI systems can become objects of normative expectations; and (3) how the resulting attitude gives human agents reassurance in their interactions with AI systems. In order to demonstrate how difficult this task is, the threefold challenge is then applied to two recent accounts that defend the possibility of trust in AI systems. By way of conclusion it is suggested that instead of trusting AI systems, we should strive to make them reliable. (shrink)

This paper begins with the thought that there is something out of place about offloading inquiry into one's own mind to AI. The paper's primary goal is to articulate the unease felt when considering cases of doing so. It draws a parallel between the use of algorithms in the criminal law: in both cases one feels entitled to be treated as an exception to a verdict made on the basis of a certain kind of evidence. Then it identifies an account (...) of first‐person authority that can make good on this: agentialism. Thus, the paper constitutes an argument in favor of an agentialist treatment of self‐knowledge and first‐person authority. (shrink)

Recently, a growing number of experts in artificial intelligence (AI) and medicine have be-gun to suggest that the use of AI systems, particularly machine learning (ML) systems, is likely to humanise the practice of medicine by substantially improving the quality of clinician-patient relationships. In this thesis, however, I argue that medical ML systems are more likely to negatively impact these relationships than to improve them. In particular, I argue that the use of medical ML systems is likely to comprise the (...) quality of trust, care, empathy, understanding, and communication between clinicians and patients. (shrink)

The principle of trust has been placed at the centre as an attitude for engaging with clinical machine learning systems. However, the notions of trust and distrust remain fiercely debated in the philosophical and ethical literature. In this article, we proceed on a structural level ex negativo as we aim to analyse the concept of “institutional distrustworthiness” to achieve a proper diagnosis of how we should not engage with medical machine learning. First, we begin with several examples that hint at (...) the emergence of a climate of distrust in the context of medical machine learning. Second, we introduce the concept of institutional trustworthiness based on an expansion of Hawley’s commitment account. Third, we argue that institutional opacity can undermine the trustworthiness of medical institutions and can lead to new forms of testimonial injustices. Finally, we focus on possible building blocks for repairing institutional distrustworthiness. (shrink)

Definition of the problem This article critically addresses the conceptualization of trust in the ethical discussion on artificial intelligence (AI) in the specific context of social robots in care. First, we attempt to define in which respect we can speak of ‘social’ robots and how their ‘social affordances’ affect the human propensity to trust in human–robot interaction. Against this background, we examine the use of the concept of ‘trust’ and ‘trustworthiness’ with respect to the guidelines and recommendations of the High-Level (...) Expert Group on AI of the European Union. Arguments Trust is analyzed as a multidimensional concept and phenomenon that must be primarily understood as departing from trusting as a human functioning and capability. To trust is an essential part of the human basic capability to form relations with others. We further want to discuss the concept of _responsivity _which has been established in phenomenological research as a foundational structure of the relation between the self and the other. We argue that trust and trusting as a capability is fundamentally _responsive_ and needs responsive others to be realized. An understanding of _responsivity_ is thus crucial to conceptualize trusting in the ethical framework of human flourishing. We apply a phenomenological–anthropological analysis to explore the link between certain qualities of social robots that construct responsiveness and thereby simulate responsivity and the human propensity to trust. Conclusion Against this background, we want to critically ask whether the concept of trustworthiness in social human–robot interaction could be misguided because of the limited ethical demands that the constructed responsiveness of social robots is able to answer to. (shrink)

Computational Reliabilism (CR) has emerged as a promising framework for assessing the trustworthiness of AI systems, particularly in domains where complete transparency is infeasible. However, the rise of sophisticated adversarial attacks poses a significant challenge to CR’s key reliability indicators. This paper critically examines the robustness of CR in the face of evolving adversarial threats, revealing the limitations of verification and validation methods, robustness analysis, implementation history, and expert knowledge when confronted with malicious actors. Our analysis suggests that CR, in (...) its current form, is inadequate to address the dynamic nature of adversarial attacks. We argue that while CR’s core principles remain valuable, the framework must be extended to incorporate adversarial resilience, adaptive reliability criteria, and context-specific reliability thresholds. By embracing these modifications, CR can evolve to provide a more comprehensive and resilient approach to assessing AI reliability in an increasingly adversarial landscape. (shrink)

This paper provides four insights relating to policy-making, focusing on the complex relationship between the abstract concept of trust—with its numerous empirical expressions—and the concept of opacity in AI technologies. First, we set the ground by discussing the nature of trust as it evolves from interpersonal to technological realms (§ 2), examining the plethora of measurement methods and objects that reflect the concept’s rich diversity. We then investigate the concept of opacity in AI systems (§ 3), challenging the conventional wisdom (...) that less opacity invariably leads to greater trust, and illustrate this relationship’s nuanced and context-dependent reality. The conclusion (§ 4) bridges these theoretical discussions with pragmatic policy-making considerations, crystallizing abstract ideas into actionable directives. We present four key insights for policy-makers: First, policies must mediate between the abstract concept of trust and the concrete outcomes of AI interactions. Second, they should mirror societal values. Third, policy-making must be adaptive, evolving alongside technological advancements to effectively address both technical and policy issues relating to trust and opacity. Lastly, we anticipate that academic discourse will inform future policies, suggesting the creation of a standardized trust index for AI systems, transparency guidelines based on opacity levels, and explainability protocols for high-stakes applications. These insights aim to ensure that AI-based technologies are governed in a manner that upholds societal values and fosters the common good. (shrink)