When used in psychotherapy, Sedlakova and Trachsel (2023) hypothesize that conversational artificial intelligence (CAI) ought to be considered as a new and hybrid artifact somewhere on a spectrum b...

A large part of the explainable AI literature focuses on what explanations are in general, what algorithmic explainability is more specifically, and how to code these principles of explainability into AI systems. Much less attention has been devoted to the question of why algorithmic decisions and systems should be explainable and whether there ought to be a right to explanation and why. We therefore explore the normative landscape of the need for AI to be explainable and individuals having a right (...) to such explanation. This exploration is particularly relevant to the medical domain where the (im)possibility of explainable AI is high on both the research and practitioners’ agenda. The dominant intuition overall is that explainability has and should play a key role in the health context. Notwithstanding the strong normative intuition for having a right to explanation, intuitions can be wrong. So, we need more than an appeal to intuitions when it comes to explaining the normative significance of having a right to explanation when being subject to AI-based decision-making. The aim of the paper is therefore to provide an account of what might underlie the normative intuition. We defend the ‘symmetry thesis’ according to which there is no special normative reason to have a right to explanation when ‘machines’ in the broad sense, make decisions, recommend treatment, discover tumors, and so on. Instead, we argue that we have a right to explanation in cases that involve automated processing that significantly affect our core deliberative agency and which we do not understand, because we have a general moral right to explanation when choices are made which significantly affect us but which we do not understand. (shrink)

In this paper, we analyze two key claims offered by recruitment AI companies in relation to the development and deployment of AI-powered HR tools: (1) recruitment AI can objectively assess candidates by removing gender and race from their systems, and (2) this removal of gender and race will make recruitment fairer, help customers attain their DEI goals, and lay the foundations for a truly meritocratic culture to thrive within an organization. We argue that these claims are misleading for four reasons: (...) First, attempts to “strip” gender and race from AI systems often misunderstand what gender and race are, casting them as isolatable attributes rather than broader systems of power. Second, the attempted outsourcing of “diversity work” to AI-powered hiring tools may unintentionally entrench cultures of inequality and discrimination by failing to address the systemic problems within organizations. Third, AI hiring tools’ supposedly neutral assessment of candidates’ traits belie the power relationship between the observer and the observed. Specifically, the racialized history of character analysis and its associated processes of classification and categorization play into longer histories of taxonomical sorting and reflect the current demands and desires of the job market, even when not explicitly conducted along the lines of gender and race. Fourth, recruitment AI tools help produce the “ideal candidate” that they supposedly identify through by constructing associations between words and people’s bodies. From these four conclusions outlined above, we offer three key recommendations to AI HR firms, their customers, and policy makers going forward. (shrink)

In this reply to my reviewers, I touch upon Husserl’s notion of fantasy. Whereas Kant positions fantasy outside the scope of his own work, Husserl brings it back. The importance of this notion lies in freeing imagination from the tight link to images, as for Husserl imagination is an activity that functions as a “quasi perception.” Ihde and Stiegler enrich Husserl’s analysis of imagination with various aspects of technology: Ihde shows how changes in the technologies that mediate our imagination will (...) necessarily change our imagination; Stiegler broadens Husserl’s analysis of retention. The two theories can be combined into a new understanding of subjectivity that is modeled as layers. Some layers can be performed by AI. The second part deals with the question of whether AI can be imaginative. Traditionally, imaginative creativity is associated with art. Bioart and AI art are brought as examples of a new definition of art, according to which art is the arrangement of materials in a way that produces a meaning. This definition does not refer specifically to creativity. In both forms of art, the biological/artificial and the human cooperate so that the former arranges the materials and the latter produces the meaning, albeit this division of labor is not clear-cut. The result is a co-shaping process. My conclusion is that algorithms can be considered creative by human standards, but this entails a new mode of imagination that is co-shaped and co-shared by humans and algorithms. The layer paradigm explains how such co-shaping works in practice. (shrink)

An Artificial Intelligence algorithm trained on data that reflect racial biases may yield racially biased outputs, even if the algorithm on its own is unbiased. For example, algorithms used to schedule medical appointments in the USA predict that Black patients are at a higher risk of no-show than non-Black patients, though technically accurate given existing data that prediction results in Black patients being overwhelmingly scheduled in appointment slots that cause longer wait times than non-Black patients. This perpetuates racial inequity, in (...) this case lesser access to medical care. This gives rise to one type of Accuracy-Fairness trade-off: preserve the efficiency offered by using AI to schedule appointments or discard that efficiency in order to avoid perpetuating ethno-racial disparities. Similar trade-offs arise in a range of AI applications including others in medicine, as well as in education, judicial systems, and public security, among others. This article presents a framework for addressing such trade-offs where Machine Learning and Optimization components of the algorithm are decoupled. Applied to medical appointment scheduling, our framework articulates four approaches intervening in different ways on different components of the algorithm. Each yields specific results, in one case preserving accuracy comparable to the current state-of-the-art while eliminating the disparity. (shrink)

This article uses critical discourse analysis to investigate artificial intelligence (AI) generated images of aged care nurses and considers how perspectives and perceptions impact upon the recruitment and retention of nurses. The article demonstrates a recontextualization of aged care nursing, giving rise to hidden ideologies including harmful stereotypes which allow for discrimination and exploitation. It is argued that this may imply that nurses require fewer clinical skills in aged care, diminishing the value of working in this area. AI relies on (...) existing data sets, and thus represent existing stereotypes and biases. The discourse analysis has highlighted key issues which may further impact upon nursing recruitment and retention, and advocates for stronger ethical consideration, including the use of experts in data validation, for the way that aged care services and nurses are depicted and thus valued. (shrink)

The vision of integrating artificial intelligence in education is part of an ongoing push for harnessing digital solutions to improve teaching and learning. Drawing from Jasanoff and Hasse, this paper deliberates on how sociotechnical imaginaries are interrelated to the implications of new technologies, such as AI, in education. Complicating Hasses’s call for the development of Socratic ignorance to consider our predispositions about new technologies and open new prospects of thought, this paper revisits postphenomenology and Feenberg’s critical constructivist theories. While embracing (...) the notion of Socratic ignorance, this paper stresses the importance of developing a nuanced understanding of technology that realizes its lack of neutrality and supports the creation of a deeper understanding of how knowledge is produced, deployed, and interpreted in the digital age. Thus, this paper argues that an amalgam of Hasse’s call for advancing Socratic ignorance combined with postphenomenology and critical constructivism can support students in developing a critical understanding of technology and opening new landscapes of imaginaries. (shrink)

Emotional facial expressions can inform researchers about an individual's emotional state. Recent technological advances open up new avenues to automatic Facial Expression Recognition. Based on machine learning, such technology can tremendously increase the amount of processed data. FER is now easily accessible and has been validated for the classification of standardized prototypical facial expressions. However, applicability to more naturalistic facial expressions still remains uncertain. Hence, we test and compare performance of three different FER systems with human emotion recognition for standardized (...) posed facial expressions and for non-standardized acted facial expressions. For the standardized images, all three systems classify basic emotions accurately and they are mostly on par with human raters. For the non-standardized stimuli, performance drops remarkably for all three systems, but Azure still performs similarly to humans. In addition, all systems and humans alike tend to misclassify some of the non-standardized emotional facial expressions as neutral. In sum, emotion recognition by automated facial expression recognition can be an attractive alternative to human emotion recognition for standardized and non-standardized emotional facial expressions. However, we also found limitations in accuracy for specific facial expressions; clearly there is need for thorough empirical evaluation to guide future developments in computer vision of emotional facial expressions. (shrink)