More and more, we face AI-based products and services. Using these services often requires our explicit consent, e.g., by agreeing to the services’ Terms and Conditions clause. Current advances introduce the ability of AI to evolve and change its own modus operandi over time in such a way that we cannot know, at the moment of consent, what it is in the future to which we are now agreeing. Therefore, informed consent is impossible regarding certain kinds of AI. Call this (...) the problem of radical ignorance. Interestingly, radical ignorance exists in consent contexts other than AI, where it seems that individuals can provide informed consent. The article argues that radical ignorance can undermine informed consent in some contexts but not others because, under certain institutional, autonomy-protecting conditions, consent can be valid without being (perfectly) informed. By understanding these institutional conditions, we can formulate practical solutions to foster valid, albeit imperfectly informed consent across various decision contexts and within different institutions. (shrink)

Social media have undoubtedly changed our ways of living. Their presence concerns an increasing number of users (over 4,74 billion) and pervasively expands in the most diverse areas of human life. Marketing, education, news, data, and sociality are just a few of the many areas in which social media play now a central role. Recently, some attention toward the link between social media and political participation has emerged. Works in the field of artificial intelligence have already pointed out that there (...) is a close link between the use of machine learning algorithms in social media and possible epistemic isolation, which could lead to political radicalization. The idea supporting this paper is that artificial intelligence for social media can actively put users’ deliberative capacity at risk and foster political extremism. To prove these claims, I proceed along two lines of inquiry. First, I focus on filter bubbles, namely the result of selections made by algorithms that recommend contents that meet users’ expectations and opinions. To analyze this phenomenon, I refer to the Deweyan model of experience. Second, I connect the filter bubbles problem to the Deweyan idea of deliberative and participatory democracy and Nussbaum’s concept of political compassion. The purpose of this paper is to provide a philosophical foundation that can both (1) effectively serve as a method for analyzing machine learning algorithms and their potential problems in relation to political extremism, and (2) be adopted as a standard to counter the danger of extremism associated with social media experience. (shrink)

The use of artificial intelligence in healthcare contexts is highly controversial for the (bio)ethical conundrums it creates. One of the main problems arising from its implementation is the lack of transparency of machine learning algorithms, which is thought to impede the patient’s autonomous choice regarding their medical decisions. If the patient is unable to clearly understand why and how an AI algorithm reached certain medical decision, their autonomy is being hovered. However, there are alternatives to prevent the negative impact of (...) AI opacity in shared (healthcare professional–patient) decision-making processes, and benefit from the high accuracy of such systems. Despite these ethical issues, this paper advocates for its usage and also defends the positive role that it could have in enhancing patients’ autonomy. I advocate for a system where the implementation of AI is only partial, and humans will remain in control of oversighting algorithm interpretation of medical images and patients’ data. (shrink)

This paper argues that one problem that besets black-box AI is that it lacks algorithmic justifiability. We argue that the norm of shared decision making in medical care presupposes that treatment decisions ought to be justifiable to the patient. Medical decisions are justifiable to the patient only if they are compatible with the patient’s values and preferences and the patient is able to see that this is so. Patient-directed justifiability is threatened by black-box AIs because the lack of rationale provided (...) for the decision makes it difficult for patients to ascertain whether there is adequate fit between the decision and the patient’s values. This paper argues that achieving algorithmic transparency does not help patients bridge the gap between their medical decisions and values. We introduce a hypothetical model we call Justifiable AI to illustrate this argument. Justifiable AI aims at modelling normative and evaluative considerations in an explicit way so as to provide a stepping stone for patient and physician to jointly decide on a course of treatment. If our argument succeeds, we should prefer these justifiable models over alternatives if the former are available and aim to develop said models if not. (shrink)

Responsible AI (RAI) guidelines aim to ensure that AI systems respect democratic values. While a step in the right direction, they currently fail to impact practice. Our work discusses reasons for this lack of impact and clusters them into five areas: (1) the abstract nature of RAI guidelines, (2) the problem of selecting and reconciling values, (3) the difficulty of operationalising RAI success metrics, (4) the fragmentation of the AI pipeline, and (5) the lack of internal advocacy and accountability. Afterwards, (...) we introduce a number of approaches to RAI from a range of disciplines, exploring their potential as solutions to the identified challenges. We anchor these solutions in practice through concrete examples, bridging the gap between the theoretical considerations of RAI and on-the-ground processes that currently shape how AI systems are built. Our work considers the socio-technical nature of RAI limitations and the resulting necessity of producing socio-technical solutions. (shrink)

In an important and widely cited paper, Zerilli, Knott, Maclaurin, and Gavaghan (2019) argue that opaque AI decision makers are at least as transparent as human decision makers and therefore the concern that opaque AI is not sufficiently transparent is mistaken. I argue that the concern about opaque AI should not be understood as the concern that such AI fails to be transparent in a way that humans are transparent. Rather, the concern is that the way in which opaque AI (...) is opaque is very different than the way in which humans are opaque. What matters is the degree to which the opaque processes of a class of decision makers are stable, uniform, and safe. The degree to which such processes have these features in humans is higher than the degree to which such processes have these features in opaque AI. And therefore we should require AI to be more transparent than humans. (shrink)

Odgovornost pripisujemo sa namerom da postignemo neki cilj. Jedno od opših mesta u filozofskoj literaturi je da osobi možemo pripisati moralnu odgovornost ako su zadovoljena bar dva uslova: da subjekt delanja ima kontrolu nad svojim postupcima i da je u stanju da navede razloge u prilog svog postupka. Međutim, četvrtu industrijsku revoluciju karakterišu sociotehnološke pojave koje nas potencijalno suočavaju sa tzv. problemom jaza u odgovornosti. Rasprave o odgovornosti u kontekstu veštačke inteligencije karakteriše nejasna i neodređena upotreba ovog pojma. Da bismo (...) razumeli problem jaza u odgovornosti, njegove domete, kao i eventualna rešenja, u radu razmatramo različite vrste pojma odgovornosti. Identifikujući uslove primene ovih pojmova uviđamo da problem jaza u moralnoj odgovornosti dobija na značaju utoliko što nije očigledno na koji način bi mogao biti premošćen. (shrink)

Drawing on the analytic of the “colonial matrix of power” developed by Aníbal Quijano within the Latin American modernity/coloniality research program, this article theorises how a system of coloniality underpins the structuring logic of artificial intelligence (AI) systems. We develop a framework for critiquing the regimes of global labour exploitation and knowledge extraction that are rendered invisible through discourses of the purported universality and objectivity of AI. ​​Through bringing the political economy literature on AI production into conversation with scholarly work (...) on decolonial AI and the modernity/coloniality research program, we advance three main arguments. First, the global economic and political power imbalances in AI production are inextricably linked to the continuities of historical colonialism, constituting the colonial supply chain of AI. Second, this is produced through an international division of digital labour that extracts value from majority world labour for the benefit of Western technology companies. Third, this perpetuates hegemonic knowledge production through Western values and knowledge that marginalises non-Western alternatives within AI’s production and limits the possibilities for decolonising AI. By locating the production of AI systems within the colonial matrix of power, we contribute to critical and decolonial literature on the legacies of colonialism in AI and the hierarchies of power and extraction that shape the development of AI today. (shrink)

Artificial Intelligence (AI) technologies offer new ways of conducting decision-making tasks that influence the daily lives of citizens, such as coordinating traffic, energy distributions, and crowd flows. They can sort, rank, and prioritize the distribution of fines or public funds and resources. Many of the changes that AI technologies promise to bring to such tasks pertain to decisions that are collectively binding. When these technologies become part of critical infrastructures, such as energy networks, citizens are affected by these decisions whether (...) they like it or not, and they usually do not have much say in them. The democratic challenge for those working on AI technologies with collectively binding effects is both to _develop_ and _deploy_ technologies in such a way that the democratic legitimacy of the relevant decisions is safeguarded. In this paper, we develop a conceptual framework to help policymakers, project managers, innovators, and technologists to assess and develop approaches to democratize AI. This framework embraces a broad sociotechnical perspective that highlights the interactions between technology and the complexities and contingencies of the context in which these technologies are embedded. We start from the problem-based and practice-oriented approach to democracy theory as developed by political theorist Mark Warren. We build on this approach to describe practices that can enhance or challenge democracy in political systems and extend it to integrate a sociotechnical perspective and make the role of technology explicit. We then examine how AI technologies can play a role in these practices to improve or inhibit the democratic nature of political systems. We focus in particular on AI-supported political systems in the energy domain. (shrink)

Artificial Intelligence (AI) is increasingly adopted in society, creating numerous opportunities but at the same time posing ethical challenges. Many of these are familiar, such as issues of fairness, responsibility and privacy, but are presented in a new and challenging guise due to our limited ability to steer and predict the outputs of AI systems. This chapter first introduces these ethical challenges, stressing that overviews of values are a good starting point but frequently fail to suffice due to the context (...) sensitivity of ethical challenges. Frequently, additional (ethical) values emerge for specific applications, as e.g. challenges with fraud detection are very different from those around language technologies. Second, methods to tackle these challenges are discussed. Main ethical theories (virtue ethics, consequentialism, and deontology) are shown to provide a starting point, but often lack the details needed for actionable AI Ethics. Instead, mid-level philosophical theories coupled to design-approaches such as Design for Values, together with interdisciplinary working methods, offer the best way forwards. The chapter aims to show how these approaches can lead to an ethics of AI that is actionable and that can be proactively integrated in the design of AI systems. (shrink)

In informationally mature societies, almost all organisations record, generate, process, use, share and disseminate data. In particular, the rise of AI and autonomous systems has corresponded to an improvement in computational power and in solving complex problems. However, the resulting possibilities have been coupled with an upsurge of ethical risks. To avoid the misuse, underuse, and harmful use of data and data-based systems like AI, we should use an ethical framework appropriate to the object of its reasoning. Unfortunately, in recent (...) years, the space for data-related ethics has not been precisely defined in organisations. As a consequence, there has been an overlapping of responsibilities and a void of clear accountabilities. Ethical issues have, therefore, been dealt with using inadequate levels of abstraction (e.g. legal, technical). Yet, if building an ethical infrastructure requires the collaboration of each body, addressing ethical issues related to data requires leaving room for the appropriate level of abstraction. This paper first aims to show how the space of data ethics is already latent in organisations. It then highlights how to redefine roles (chief data ethics officer, data ethics committee, etc.) and codes (code of data ethics) to create and maintain an environment where ethical reasoning about data, information, and AI systems may flourish. (shrink)

The ongoing explosion of interest in artificial intelligence is fueled in part by recently developed techniques in machine learning. Those techniques allow automated systems to process huge amounts of data, utilizing mathematical methods that depart from traditional statistical approaches, and resulting in impressive advancements in our ability to make predictions and uncover correlations across a host of interesting domains. But as is now widely discussed, the way that those systems arrive at their outputs is often opaque, even to the experts (...) who design and deploy them. Is it morally problematic to make use of opaque automated methods when making high-stakes decisions, like whether to issue a loan to an applicant, or whether to approve a parole request? Many scholars answer in the affirmative. However, there is no widely accepted explanation for why transparent systems are morally preferable to opaque systems. We argue that the use of automated decision-making systems sometimes violates duties of consideration that are owed by decision-makers to decision-subjects, duties that are both epistemic and practical in character. Violations of that kind generate a weighty consideration against the use of opaque decision systems. In the course of defending our approach, we show that it is able to address three major challenges sometimes leveled against attempts to defend the moral import of transparency in automated decision-making. (shrink)

Artificial intelligence and automation have become pervasive in news media, influencing journalism from news gathering to news distribution. As algorithms are increasingly determining editorial decisions, specific concerns have been raised with regard to the responsible and accountable use of AI-driven tools by news media, encompassing new regulatory and ethical questions. This contribution aims to analyze whether and to what extent the use of AI technology in news media and journalism is currently regulated and debated within the European Union and the (...) Council of Europe. Through a document analysis of official policy documents, combined with a data mining approach and an inductive thematic analysis, the study looks at how news media are dealt with, in particular regarding their responsibilities towards their users and society. The findings show that regulatory frameworks about AI rarely include media, but if they do, they associate them with issues such as disinformation, data, and AI literacy, as well as diversity, plurality, and social responsibility. (shrink)

Purpose Privacy has been understood as about one’s own information, information that is not one’s own is not typically considered with regards to an individual’s privacy. This paper aims to draw attention to this issue for conceptualizing privacy when one’s privacy is breached by others’ information. Design/methodology/approach To illustrate the issue that others' information can breach one's own privacy, this paper uses real-world applications of forensic genealogy and recommender systems to motivate the discussion. Findings In both forensic genealogy and recommender (...) systems, the individual’s privacy is breached by information that is not one’s own. The information that breached one’s privacy, by its nature, is beyond the scope of an individual, which is a phenomenon that has already been captured by emerging discussions about group privacy. This paper further argues that the underlying issue reflected by the examples of forensic genealogy is an extreme case even under the consideration of group privacy. This is because, unlike recommender systems that rely on large amounts of data to make inferences about an individual, forensic genealogy exposes one’s identity by using only one other individual’s information. This paper echoes existing discussions that this peculiar situation where others’ information breaches one’s own privacy reveals the problematic nature of conceptualizing privacy relying only on individualistic assumptions. Moreover, this paper suggests a relational perspective as an alternative for theorizing privacy. Originality/value This situation that others’ information breached one’s own privacy calls for an updated understanding of not only privacy but also the relationship between the person and their information. Privacy scholars need to renew their ethical language and vocabularies to properly understand the issue, which recent privacy conceptualizations are already doing (e.g. group privacy). (shrink)

We are moving towards a future where Artificial Intelligence (AI) based agents make many decisions on behalf of humans. From healthcare decision-making to social media censoring, these agents face problems, and make decisions with ethical and societal implications. Ethical behaviour is a critical characteristic that we would like in a human-centric AI. A common observation in human-centric industries, like the service industry and healthcare, is that their professionals tend to break rules, if necessary, for pro-social reasons. This behaviour among humans (...) is defined as pro-social rule breaking. To make AI agents more human-centric, we argue that there is a need for a mechanism that helps AI agents identify when to break rules set by their designers. To understand when AI agents need to break rules, we examine the conditions under which humans break rules for pro-social reasons. In this paper, we present a study that introduces a ‘vaccination strategy dilemma’ to human participants and analyzes their response. In this dilemma, one needs to decide whether they would distribute COVID-19 vaccines only to members of a high-risk group (follow the enforced rule) or, in selected cases, administer the vaccine to a few social influencers (break the rule), which might yield an overall greater benefit to society. The results of the empirical study suggest a relationship between stakeholder utilities and pro-social rule breaking (PSRB), which neither deontological nor utilitarian ethics completely explain. Finally, the paper discusses the design characteristics of an ethical agent capable of PSRB and the future research directions on PSRB in the AI realm. We hope that this will inform the design of future AI agents, and their decision-making behaviour. (shrink)

Zusammenfassung Die rasanten Entwicklungen im Bereich der Künstlichen Intelligenz und Robotik stellen nicht nur die Ethik, sondern auch das Recht vor neue Herausforderungen, gerade im Bereich der Medizin und Pflege. Grundsätzlich hat der Einsatz von KI dabei das Potenzial, sowohl die Heilbehandlungen als auch den adäquaten Umgang im Rahmen der Pflege zu erleichtern, wenn nicht sogar zu verbessern. Verwaltungsaufgaben, die Überwachung von Vitalfunktionen und deren Parameter sowie die Untersuchung von Gewebeproben etwa könnten autonom ablaufen. In Diagnostik und Therapie können Systeme (...) die behandelnde Ärztin unterstützen. Intelligente Betten ermöglichen eine Frühmobilisierung der Patient:innen bei gleichzeitig geringerem Personalaufwand. Gleichwohl birgt der Einsatz der Systeme rechtliche Herausforderungen. So besteht das Risiko einer Verletzung der beteiligten Personen. Im Gegensatz zu herkömmlichen Technologien „leidet“ KI unter der „Black-Box-Problematik“: Die von den Systemen generierten Ergebnisse sind nicht mehr vollständig vorhersehbar und nachvollziehbar. Der Einsatz birgt unbekannte und unkalkulierbare Risiken, was sich insbesondere auf die zivilrechtliche Haftung und strafrechtliche Verantwortung auswirkt. Wem die Entscheidungen der Systeme normativ zuzurechnen sind, ist eine Kernfrage des juristischen Diskurses. Die aus praktikabilitätsgründen naheliegende Wahl, dem letztentscheidenden Menschen das Verhalten eines KI-Systems zuzurechnen, überzeugt nicht in allen Fällen, sondern degradiert ihn häufig zum symbolischen Haftungsknecht und legt ihm einseitig die Risiken der Technologien auf. Weiterhin ergeben sich im Bereich der Medizin und Pflege Fragen hinsichtlich der Zulassung von KI-Systemen, da die Maschinen während der Nutzung weiterlernen und so ihren strukturellen Aufbau kontinuierlich verändern. Es ist daher angebracht, sich frühzeitig mit dem Konfliktpotential aus ethischer und rechtlicher Sicht auseinander zu setzen, um einer potenziellen gesellschaftlichen Angst vor derartigen Systemen vorzubeugen und einen praxisgerechten Handlungsrahmen zu schaffen. (shrink)

Algorithm engineering is sometimes portrayed as a new 21st century return of manipulative social engineering. Yet algorithms are necessary tools for individuals to navigate online platforms. Algorithms are like a sensory apparatus through which we perceive online platforms: this is also why individuals can be subtly but pervasively manipulated by biased algorithms. How can we better understand the nature of algorithm engineering and its proper function? In this chapter I argue that algorithm engineering can be best conceptualized as a type (...) of environmental engineering aimed at making the online environment more hospitable to human use, in particular by safeguarding the conditions that allow for trust. (shrink)

Danaher (2016) has argued that increasing robotization can lead to retribution gaps: Situation in which the normative fact that nobody can be justly held responsible for a harmful outcome stands in conflict with our retributivist moral dispositions. In this paper, we report a cross-cultural empirical study based on Sparrow’s (2007) famous example of an autonomous weapon system committing a war crime, which was conducted with participants from the US, Japan and Germany. We find that (i) people manifest a considerable willingness (...) to hold autonomous systems morally responsible, (ii) partially exculpate human agents when interacting with such systems, and that more generally (iii) the possibility of normative responsibility gaps is indeed at odds with people’s pronounced retributivist inclinations. We discuss what these results mean for potential implications of the retribution gap and other positions in the responsibility gap literature. (shrink)

This article explores the trend of increasing automation in law enforcement and criminal justice settings through three use cases: predictive policing, machine evidence and recidivism algorithms. The focus lies on artificial-intelligence-driven tools and technologies employed, whether at pre-investigation stages or within criminal proceedings, in order to decode human behaviour and facilitate decision-making as to whom to investigate, arrest, prosecute, and eventually punish. In this context, this article first underlines the existence of a persistent dilemma between the goal of increasing the (...) operational efficiency of police and judicial authorities and that of safeguarding fundamental rights of the affected individuals. Subsequently, it shifts the focus onto key principles of criminal procedure and the presumption of innocence in particular. Using Article 6 ECHR and the Directive (EU) 2016/343 as a starting point, it discusses challenges relating to the protective scope of presumption of innocence, the burden of proof rule and the _in dubio pro reo_ principle as core elements of it. Given the transformations law enforcement and criminal proceedings go through in the era of algorithms, big data and artificial intelligence, this article advocates the adoption of specific procedural safeguards that will uphold rule of law requirements, and particularly transparency, fairness and explainability. In doing so, it also takes into account EU legislative initiatives, including the reform of the EU data protection _acquis_, the E-evidence Proposal, and the Proposal for an EU AI Act. Additionally, it argues in favour of revisiting the protective scope of key fundamental rights, considering, _inter alia_, the new dimensions suspicion has acquired. (shrink)

This paper presents an ethical framework designed to support the development of critical data literacy for research methods courses and data training programmes in higher education. The framework we present draws upon our reviews of literature, course syllabi and existing frameworks on data ethics. For this research we reviewed 250 research methods syllabi from across the disciplines, as well as 80 syllabi from data science programmes to understand how or if data ethics was taught. We also reviewed 12 data ethics (...) frameworks drawn from different sectors. Finally, we reviewed an extensive and diverse body of literature about data practices, research ethics, data ethics and critical data literacy, in order to develop a transversal model that can be adopted across higher education. To promote and support ethical approaches to the collection and use of data, ethics training must go beyond securing informed consent to enable a critical understanding of the techno-centric environment and the intersecting hierarchies of power embedded in technology and data. By fostering ethics as a method, educators can enable research that protects vulnerable groups and empower communities. (shrink)

This paper examines the paradoxical transparency involved in training machine-learning models. Existing literature typically critiques the opacity of machine-learning models such as neural networks or collaborative filtering, a type of critique that parallels the black-box critique in technology studies. Accordingly, people in power may leverage the models’ opacity to justify a biased result without subjecting the technical operations to public scrutiny, in what Dan McQuillan metaphorically depicts as an “algorithmic state of exception”. This paper attempts to differentiate the black-box abstraction (...) that wraps around complex computational systems from the opacity of machine-learning models. It contends that the degree of asymmetry in knowledge is greater in the former than the latter. In the case of software systems, software codes are difficult to understand as only software experts with sufficient domain knowledge are equipped to formulate a sound critique. In contrast, the meanings of trained parameters in a machine-learning model are obscure even to the data scientists who configure and train the model. Hence, the asymmetry of knowledge lies only in how data examples are collected, the choice and configuration of machine-learning models, and the specification of features in model design. Under the trend of algorithmic decision-making proliferating with machine-learning heuristics, the paper contends that the more symmetric distribution of knowledge in machine learning could lead to a more transparent production process if proper policies are in place. (shrink)

In this paper, we build upon Bruno Latour’s political writings to address the current impasse regarding algorithms in public life. We assert that the increasing difficulties at governing algorithms—be they qualified as “machine learning,” “big data,” or “artificial intelligence”—can be related to their current ontological thinness: deriving from constricted views on theoretical practices, algorithms’ standard definition as problem-solving computerized methods provides poor grips for affective dissensions. We then emphasize on the role historical and ethnographic studies of algorithms can potentially play (...) in the politicization of algorithms. By both digging into the genealogy of algorithms’ constricted definition and by making their contemporary constitutive relationships more visible, both historical and ethnographic studies can contribute to vascularizing algorithms and making them objects of enlarged disputes. We conclude by giving a flavor of the political potential of the vascularization efforts we call for, using materials from an ethnographic study conducted in a computer science laboratory. (shrink)

Artificial Social Agents (ASAs), which are AI software driven entities programmed with rules and preferences to act autonomously and socially with humans, are increasingly playing roles in society. As their sophistication grows, humans will share greater amounts of personal information, thoughts, and feelings with ASAs, which has significant ethical implications. We conducted a study to investigate what ethical principles are of relative importance when people engage with ASAs and whether there is a relationship between people’s values and the ethical principles (...) they prioritise. The study uses scenarios, embedded with the five AI4People ethical principles (Beneficence, Non-maleficence, Autonomy, Justice, and Explicability), involving ASAs taking on roles traditionally played by humans to understand whether these roles and behaviours (including dialogues) are seen as acceptable or unacceptable. Results from 268 participants reveal the greatest sensitivity to ASA behaviours that relate to Autonomy, Justice, Explicability, and the privacy of their personal data. Models were created using Schwartz’s Refined Values as a possible indicator of how stakeholders discern and prioritise the different AI4People ethical principles when interacting with ASAs. Our findings raise issues around the ethical acceptability of ASAs for nudging and changing behaviour due to participants’ desire for autonomy and their concerns over deceptive ASA behaviours such as pretending to have memories and emotions. (shrink)

Policy discussions and corporate strategies on machine learning are increasingly championing data reuse as a key element in digital transformations. These aspirations are often coupled with a focus on responsibility, ethics and transparency, as well as emergent forms of regulation that seek to set demands for corporate conduct and the protection of civic rights. And the Protective measures include methods of traceability and assessments of ‘good’ and ‘bad’ datasets and algorithms that are considered to be traceable, stable and contained. However, (...) these ways of thinking about both technology and ethics obscure a fundamental issue, namely that machine learning systems entangle data, algorithms and more-than-human environments in ways that challenge a well-defined separation. This article investigates the fundamental fallacy of most data reuse strategies as well as their regulation and mitigation strategies that data can somehow be followed, contained and controlled in machine learning processes. Instead, the article argues that we need to understand the reuse of data as an inherently entangled phenomenon. To examine this tension between the discursive regimes and the realities of data reuse, we advance the notion of reuse entanglements as an analytical lens. The main contribution of the article is the conceptualization of reuse that places entanglements at its core and the articulation of its relevance using empirical illustrations. This is important, we argue, for our understanding of the nature of data and algorithms, for the practical uses of data and algorithms and our attitudes regarding ethics, responsibility and regulation. (shrink)

This article advocates for a hermeneutic model for children-AI interactions in which the desirable purpose of children’s interaction with artificial intelligence systems is children's growth. The article perceives AI systems with machine-learning components as having a recursive element when interacting with children. They can learn from an encounter with children and incorporate data from interaction, not only from prior programming. Given the purpose of growth and this recursive element of AI, the article argues for distinguishing the interpretation of bias within (...) the artificial intelligence ethics and responsible AI discourse. Interpreting bias as a preference and distinguishing between positive and negative bias is needed as this would serve children's healthy psychological and moral development. The human-centric AI discourse advocates for an alignment of capacities of humans and capabilities of machines by a focus both on the purpose of humans and on the purpose of machines for humans. The emphasis on mitigating negative biases through data protection, AI law, and certain value-sensitive design frameworks demonstrates that the purpose of the machine for humans is prioritized over the purpose of humans. These top–down frameworks often narrow down the purpose of machines to do-no-harm and they miss accounting for the bottom-up views and developmental needs of children. Therefore, applying a growth model for children-AI interactions that incorporates learning from negative AI-mediated biases and amplifying positive ones would positively benefit children’s development and children-centric AI innovation. Consequently, the article explores: What challenges arise from mitigating negative biases and amplifying positive biases in children-AI interactions and how can a growth model address these? To answer this, the article recommends applying a growth model in open AI co-creational spaces with and for children. In such spaces human–machine and human–human value alignment methods can be collectively applied in such a manner that children can become sensitized toward the effects of AI-mediated negative biases on themselves and others; enable children to appropriate and imbue top-down values of diversity, and non-discrimination with their meanings; enforce children’s right to identity and non-discrimination; guide children in developing an inclusive mindset; inform top-down normative AI frameworks by children’s bottom-up views; contribute to design criteria for children-centric AI. Applying such methods under a growth model in AI co-creational spaces with children could yield an inclusive co-evolution between responsible young humans in the loop and children-centric AI systems. (shrink)

Many artificial intelligence (AI) systems currently used for decision-making are opaque, i.e., the internal factors that determine their decisions are not fully known to people due to the systems’ computational complexity. In response to this problem, several researchers have argued that human decision-making is equally opaque and since simplifying, reason-giving explanations (rather than exhaustive causal accounts) of a decision are typically viewed as sufficient in the human case, the same should hold for algorithmic decision-making. Here, I contend that this argument (...) overlooks that human decision-making is sometimes significantly more transparent and trustworthy than algorithmic decision-making. This is because when people explain their decisions by giving reasons for them, this frequently prompts those giving the reasons to govern or regulate themselves so as to think and act in ways that confirm their reason reports. AI explanation systems lack this self-regulative feature. Overlooking it when comparing algorithmic and human decision-making can result in underestimations of the transparency of human decision-making and in the development of explainable AI that may mislead people by activating generally warranted beliefs about the regulative dimension of reason-giving. (shrink)

While the consequences of mathematically-based software, algorithms and strategies have become ever wider and better appreciated, ethical reflection on mathematics has remained primitive. We review the somewhat disconnected suggestions of commentators in recent decades with a view to piecing together a coherent approach to ethics in mathematics. Calls for a Hippocratic Oath for mathematicians are examined and it is concluded that while lessons can be learned from the medical profession, the relation of mathematicians to those affected by their work is (...) significantly different. There is something to be learned also from the codes of conduct of cognate but professionalised quantitative disciplines such as engineering and accountancy, as well as from legal principles bearing on professional work. We conclude with recommendations that professional societies in mathematics should sponsor an (international) code of ethics, institutional mission statements for mathematicians and syllabuses of ethics courses for incorporation into mathematics degrees. (shrink)

Embedding ethics modules within computer science courses has become a popular response to the growing recognition that CS programs need to better equip their students to navigate the ethical dimensions of computing technologies like AI, machine learning, and big data analytics. However, the popularity of this approach has outpaced the evidence of its positive outcomes. To help close that gap, this empirical study reports positive results from Northeastern’s program that embeds values analysis modules into CS courses. The resulting data suggest (...) that such modules have a positive effect on students’ moral attitudes and that students leave the modules believing they are more prepared to navigate the ethical dimensions they’ll likely face in their eventual careers. Importantly, these gains were accomplished at an institution without a philosophy doctoral program, suggesting this strategy can be effectively employed by a wider range of institutions than many have thought. (shrink)

This short commentary on Peters identifies the entrenchment of political positions as one additional concern related to algorithmic political bias, beyond those identified by Peters. First, it is observed that the political positions detected and predicted by algorithms are typically contingent and largely explained by “political tribalism”, as argued by Brennan. Second, following Hacking, the social construction of political identities is analyzed and it is concluded that algorithmic political bias can contribute to such identities. Third, following Nozick, it is argued (...) that purist political positions may stand in the way of the pursuit of all worthy values and goals to be pursued in the political realm and that to the extent that algorithmic political bias entrenches political positions, it also hinders this healthy “zigzag of politics”. (shrink)

Among the myriad of technical approaches and abstract guidelines proposed to the topic of AI bias, there has been an urgent call to translate the principle of fairness into the operational AI reality with the involvement of social sciences specialists to analyse the context of specific types of bias, since there is not a generalizable solution. This article offers an interdisciplinary contribution to the topic of AI and societal bias, in particular against the poor, providing a conceptual framework of the (...) issue and a tailor-made model from which meaningful data are obtained using Natural Language Processing word vectors in pretrained Google Word2Vec, Twitter and Wikipedia GloVe word embeddings. The results of the study offer the first set of data that evidences the existence of bias against the poor and suggest that Google Word2vec shows a higher degree of bias when the terms are related to beliefs, whereas bias is higher in Twitter GloVe when the terms express behaviour. This article contributes to the body of work on bias, both from and AI and a social sciences perspective, by providing evidence of a transversal aggravating factor for historical types of discrimination. The evidence of bias against the poor also has important consequences in terms of human development, since it often leads to discrimination, which constitutes an obstacle for the effectiveness of poverty reduction policies. (shrink)

AI systems have often been found to contain gender biases. As a result of these gender biases, AI routinely fails to adequately recognize the needs, rights, and accomplishments of women. In this article, we use Axel Honneth’s theory of recognition to argue that AI’s gender biases are not only an ethical problem because they can lead to discrimination, but also because they resemble forms of misrecognition that can hurt women’s self-development and self-worth. Furthermore, we argue that Honneth’s theory of recognition (...) offers a fruitful framework for improving our understanding of the psychological and normative implications of gender bias in modern technologies. Moreover, our Honnethian analysis of gender bias in AI shows that the goal of responsible AI requires us to address these issues not only through technical interventions, but also through a change in how we grant and deny recognition to each other. (shrink)

Biases in cognition are ubiquitous. Social psychologists suggested biases and stereotypes serve a multifarious set of cognitive goals, while at the same time stressing their potential harmfulness. Recently, biases and stereotypes became the purview of heated debates in the machine learning community too. Researchers and developers are becoming increasingly aware of the fact that some biases, like gender and race biases, are entrenched in the algorithms some AI applications rely upon. Here, taking into account several existing approaches that address the (...) problem of implicit biases and stereotypes, we propose that a strategy to cope with this phenomenon is to unmask those found in AI systems by understanding their cognitive dimension, rather than simply trying to correct algorithms. To this extent, we present a discussion bridging together findings from cognitive science and insights from machine learning that can be integrated in a state-of-the-art semantic network. Remarkably, this resource can be of assistance to scholars (e.g., cognitive and computer scientists) while at the same time contributing to refine AI regulations affecting social life. We show how only through a thorough understanding of the cognitive processes leading to biases, and through an interdisciplinary effort, we can make the best of AI technology. (shrink)

The increasing implementation of and reliance on machine-learning (ML) algorithms to perform tasks, deliver services and make decisions in health and healthcare have made the need for fairness in ML, and more specifically in healthcare ML algorithms (HMLA), a very important and urgent task. However, while the debate on fairness in the ethics of artificial intelligence (AI) and in HMLA has grown significantly over the last decade, the very concept of fairness as an ethical value has not yet been sufficiently (...) explored. Our paper aims to fill this gap and address the AI ethics principle of fairness from a conceptual standpoint, drawing insights from accounts of fairness elaborated in moral philosophy and using them to conceptualise fairness as an ethical value and to redefine fairness in HMLA accordingly. To achieve our goal, following a first section aimed at clarifying the background, methodology and structure of the paper, in the second section, we provide an overview of the discussion of the AI ethics principle of fairness in HMLA and show that the concept of fairness underlying this debate is framed in purely distributive terms and overlaps with non-discrimination, which is defined in turn as the absence of biases. After showing that this framing is inadequate, in the third section, we pursue an ethical inquiry into the concept of fairness and argue that fairness ought to be conceived of as an ethical value. Following a clarification of the relationship between fairness and non-discrimination, we show that the two do not overlap and that fairness requires much more than just non-discrimination. Moreover, we highlight that fairness not only has a distributive but also a socio-relational dimension. Finally, we pinpoint the constitutive components of fairness. In doing so, we base our arguments on a renewed reflection on the concept of respect, which goes beyond the idea of equal respect to include respect for individual persons. In the fourth section, we analyse the implications of our conceptual redefinition of fairness as an ethical value in the discussion of fairness in HMLA. Here, we claim that fairness requires more than non-discrimination and the absence of biases as well as more than just distribution; it needs to ensure that HMLA respects persons both as persons and as particular individuals. Finally, in the fifth section, we sketch some broader implications and show how our inquiry can contribute to making HMLA and, more generally, AI promote the social good and a fairer society. (shrink)

This chapter examines the possibility of using AI technologies to improve human moral reasoning and decision-making, especially in the context of purchasing and consumer decisions. We characterize such AI technologies as artificial ethics assistants (AEAs). We focus on just one part of the AI-aided moral improvement question: the case of the individual who wants to improve their morality, where what constitutes an improvement is evaluated by the individual’s own values. We distinguish three broad areas in which an individual might think (...) their own moral reasoning and decision-making could be improved: one’s actions, character, or other evaluable attributes fall short of one’s values and moral beliefs; one sometimes misjudges or is uncertain about what the right thing to do is in particular situations, given one’s values; one is uncertain about some fundamental moral questions or recognizes a possibility that some of one’s core moral beliefs and values are mistaken. We sketch why one might think that AI tools could be used to support moral improvement in those areas, and describe two types of assistance: preparatory assistance, including advice and training supplied in advance of moral deliberation; and on-the-spot assistance, including on-the-spot advice and facilitation of moral functioning over the course of moral deliberation. Then, we turn to some of the ethical issues that AEAs might raise, looking in particular at three under-appreciated problems posed by the use of AI for moral self-improvement: namely, reliance on sensitive moral data; the inescapability of outside influences on AEAs; and AEA usage prompting the user to adopt beliefs and make decisions without adequate reasons. (shrink)

Algorithmic systems are increasingly used by state agencies to inform decisions about humans. They produce scores on risks of recidivism in criminal justice, indicate the probability for a job seeker to find a job in the labor market, or calculate whether an applicant should get access to a certain university program. In this contribution, we take an interdisciplinary perspective, provide a bird’s eye view of the different key decisions that are to be taken when state actors decide to use an (...) algorithmic system, and illustrate these decisions with empirical examples from case studies. Building on these insights, we discuss the main pitfalls and promises of the use of algorithmic system by the state and focus on four levels: The most basic question whether an algorithmic system should be used at all, the regulation and governance of the system, issues of algorithm design, and, finally, questions related to the implementation of the system on the ground and the human–machine-interaction that comes with it. Based on our assessment of the advantages and challenges that arise at each of these levels, we propose a set of crucial questions to be asked when such intricate matters are addressed. (shrink)

There has been an increased focus within the AI ethics literature on questions of power, reflected in the ideal of accountability supported by many Responsible AI guidelines. While this recent debate points towards the power asymmetry between those who shape AI systems and those affected by them, the literature lacks normative grounding and misses conceptual clarity on how these power dynamics take shape. In this paper, I develop a workable conceptualization of said power dynamics according to Cristiano Castelfranchi’s conceptual framework (...) of power and argue that end-users depend on a system’s developers and users, because end-users rely on these systems to satisfy their goals, constituting a power asymmetry between developers, users and end-users. I ground my analysis in the neo-republican moral wrong of domination, drawing attention to legitimacy concerns of the power-dependence relation following from the current lack of accountability mechanisms. I illustrate my claims on the basis of a risk-prediction machine learning system, and propose institutional and project-specific solutions to mitigate domination. (shrink)

The amount of data available to healthcare practitioners is growing, and the rapid increase in available patient data is becoming a problem for healthcare practitioners, as they are often unable to fully survey and process the data relevant for the treatment or care of a patient. Consequently, there are currently several efforts to develop systems that can aid healthcare practitioners with reading and processing patient data and, in this way, provide them with a better foundation for decision-making about the treatment (...) and care of patients. There are also efforts to develop algorithms that provide suggestions for such decisions. However, the development of these systems and algorithms raises several concerns related to the privacy of patients, the patient–practitioner relationship, and the autonomy of healthcare practitioners. The aim of this article is to provide a foundation for understanding the ethical challenges related to the development of a specific form of data-processing systems, namely clinical algorithms. (shrink)

Responsible innovation in artificial intelligence calls for public deliberation: well-informed “deep democratic” debate that involves actors from the public, private, and civil society sectors in joint efforts to critically address the goals and means of AI. Adopting such an approach constitutes a challenge, however, due to the opacity of AI and strong knowledge boundaries between experts and citizens. This undermines trust in AI and undercuts key conditions for deliberation. We approach this challenge as a problem of situating the knowledge of (...) actors from the AI industry within a deliberative system. We develop a new framework of responsibilities for AI innovation as well as a deliberative governance approach for enacting these responsibilities. In elucidating this approach, we show how actors from the AI industry can most effectively engage with experts and nonexperts in different social venues to facilitate well-informed judgments on opaque AI systems and thus effectuate their democratic governance. (shrink)

Technologies that deploy data science methods are liable to result in epistemic harms involving the diminution of individuals with respect to their standing as knowers or their credibility as sources of testimony. Not all harms of this kind are unjust but when they are we ought to try to prevent or correct them. Epistemically unjust harms will typically intersect with other more familiar and well-studied kinds of harm that result from the design, development, and use of data science technologies. However, (...) we argue that epistemic injustices can be distinguished conceptually from more familiar kinds of harm. We argue that epistemic harms are morally relevant even in cases where those who suffer them are unharmed in other ways. Via a series of examples from the criminal justice system, workplace hierarchies, and educational contexts we explain the kinds of epistemic injustice that can result from common uses of data science technologies. (shrink)

This chapter will explore the dynamics of power underpinning ethical issues within smart cities via a new paradigm derived from Systems Theory. The smart city is an expression of technology as a socio-technical system. The vision of the smart city contains a deep fusion of many different technical systems into a single integrated “ambient intelligence”. ETICA Project, 2010, p. 102). Citizens of the smart city will not experience a succession of different technologies, but a single intelligent and responsive environment through (...) which they move. Analysis of such an environment requires a framework which transcends traditional ontologically-based models in order to accommodate this deep fusion. This chapter will outline a framework based on Latour’s Actor-Network Theory and Luhmann’s treatment of society as an autopoetic system. We shall use this framework to map the influence of relevant factors on ethical issues, irrespective of their composition or type. For example, under this treatment, both human praxis and technical design can be viewed as comparable tools of domination. This chapter will provide a framework for the analysis of relations between any elements of the smart city, ranging from top-level urban management processes down to individual device operations. While we will illustrate the use of this schema through examination of ethical issues arising from power dynamics within the smart city, it is intended that this example will demonstrate the wider utility of the model in general. (shrink)

Research on the ethics of algorithms has grown substantially over the past decade. Alongside the exponential development and application of machine learning algorithms, new ethical problems and solutions relating to their ubiquitous use in society have been proposed. This article builds on a review of the ethics of algorithms published in 2016, 2016). The goals are to contribute to the debate on the identification and analysis of the ethical implications of algorithms, to provide an updated analysis of epistemic and normative (...) concerns, and to offer actionable guidance for the governance of the design, development and deployment of algorithms. (shrink)

Companies increasingly deploy artificial intelligence technologies in their personnel recruiting and selection process to streamline it, making it faster and more efficient. AI applications can be found in various stages of recruiting, such as writing job ads, screening of applicant resumes, and analyzing video interviews via face recognition software. As these new technologies significantly impact people’s lives and careers but often trigger ethical concerns, the ethicality of these AI applications needs to be comprehensively understood. However, given the novelty of AI (...) applications in recruiting practice, the subject is still an emerging topic in academic literature. To inform and strengthen the foundation for future research, this paper systematically reviews the extant literature on the ethicality of AI-enabled recruiting to date. We identify 51 articles dealing with the topic, which we synthesize by mapping the ethical opportunities, risks, and ambiguities, as well as the proposed ways to mitigate ethical risks in practice. Based on this review, we identify gaps in the extant literature and point out moral questions that call for deeper exploration in future research. (shrink)

We explore aversion to the use of algorithms in moral decision-making. So far, this aversion has been explained mainly by the fear of opaque decisions that are potentially biased. Using incentivized experiments, we study which role the desire for human discretion in moral decision-making plays. This seems justified in light of evidence suggesting that people might not doubt the quality of algorithmic decisions, but still reject them. In our first study, we found that people prefer humans with decision-making discretion to (...) algorithms that rigidly apply exogenously given human-created fairness principles to specific cases. In the second study, we found that people do not prefer humans to algorithms because they appreciate flesh-and-blood decision-makers per se, but because they appreciate humans’ freedom to transcend fairness principles at will. Our results contribute to a deeper understanding of algorithm aversion. They indicate that emphasizing the transparency of algorithms that clearly follow fairness principles might not be the only element for fostering societal algorithm acceptance and suggest reconsidering certain features of the decision-making process. (shrink)

The modern abundance and prominence of data has led to the development of “data science” as a new field of enquiry, along with a body of epistemological reflections upon its foundations, methods, and consequences. This article provides a systematic analysis and critical review of significant open problems and debates in the epistemology of data science. We propose a partition of the epistemology of data science into the following five domains: (i) the constitution of data science; (ii) the kind of enquiry (...) that it identifies; (iii) the kinds of knowledge that data science generates; (iv) the nature and epistemological significance of “black box” problems; and (v) the relationship between data science and the philosophy of science more generally. (shrink)

The rapid and dynamic nature of digital transformation challenges companies that wish to develop and deploy novel digital technologies. Like other actors faced with this transformation, companies need to find robust ways to ethically guide their innovations and business decisions. Digital ethics has recently featured in a plethora of both practical corporate guidelines and compilations of high-level principles, but there remains a gap concerning the development of sound ethical guidance in specific business contexts. As a multinational science and technology company (...) faced with a broad range of digital ventures and associated ethical challenges, Merck KGaA has laid the foundations for bridging this gap by developing a Code of Digital Ethics (CoDE) tailored for this context. Following a comprehensive analysis of existing digital ethics guidelines, we used a reconstructive social research approach to identify 20 relevant principles and derive a code designed as a multi-purpose tool. Versatility was prioritised by defining non-prescriptive guidelines that are open to different perspectives and thus well-suited for operationalisation for varied business purposes. We also chose a clear nested structure that highlights the relationships between five core and fifteen subsidiary principles as well as the different levels of reference—data and algorithmic systems—to which they apply. The CoDE will serve Merck KGaA and its new Digital Ethics Advisory Panel to guide ethical reflection, evaluation and decision-making across the full spectrum of digital developments encountered and undertaken by the company whilst also offering an opportunity to increase transparency for external partners, and thus trust. (shrink)

In this paper, we examine the qualitative moral impact of machine learning-based clinical decision support systems in the process of medical diagnosis. To date, discussions about machine learning in this context have focused on problems that can be measured and assessed quantitatively, such as by estimating the extent of potential harm or calculating incurred risks. We maintain that such discussions neglect the qualitative moral impact of these technologies. Drawing on the philosophical approaches of technomoral change and technological mediation theory, which (...) explore the interplay between technologies and morality, we present an analysis of concerns related to the adoption of machine learning-aided medical diagnosis. We analyze anticipated moral issues that machine learning systems pose for different stakeholders, such as bias and opacity in the way that models are trained to produce diagnoses, changes to how health care providers, patients, and developers understand their roles and professions, and challenges to existing forms of medical legislation. Albeit preliminary in nature, the insights offered by the technomoral change and the technological mediation approaches expand and enrich the current discussion about machine learning in diagnostic practices, bringing distinct and currently underexplored areas of concern to the forefront. These insights can contribute to a more encompassing and better informed decision-making process when adapting machine learning techniques to medical diagnosis, while acknowledging the interests of multiple stakeholders and the active role that technologies play in generating, perpetuating, and modifying ethical concerns in health care. (shrink)

We describe the design and evaluation of a sub-clinical digital assessment tool that integrates digital biomarkers of depression. Based on three standard cognitive tasks on which people with depression have been known to perform differently than a control group, we iteratively designed a digital assessment tool that could be deployed outside of laboratory contexts, in uncontrolled home environments on computer systems with widely varying system characteristics. We conducted two online studies, in which participants used the assessment tool in their own (...) homes, and completed subjective questionnaires including the Patient Health Questionnaire —a standard self-report tool for assessing depression in clinical contexts. In a first study, we demonstrate that each task can be used in isolation to significantly predict PHQ-9 scores. In a second study, we replicate these results and further demonstrate that when used in combination, behavioral metrics from the three tasks significantly predicted PHQ-9 scores, even when taking into account demographic factors known to influence depression such as age and gender. A multiple regression model explained 34.4% of variance in PHQ-9 scores with behavioral metrics from each task providing unique and significant contributions to the prediction. (shrink)

Data analytics and data-driven approaches in Machine Learning are now among the most hailed computing technologies in many industrial domains. One major application is predictive analytics, which is used to predict sensitive attributes, future behavior, or cost, risk and utility functions associated with target groups or individuals based on large sets of behavioral and usage data. This paper stresses the severe ethical and data protection implications of predictive analytics if it is used to predict sensitive information about single individuals or (...) treat individuals differently based on the data many unrelated individuals provided. To tackle these concerns in an applied ethics, first, the paper introduces the concept of “predictive privacy” to formulate an ethical principle protecting individuals and groups against differential treatment based on Machine Learning and Big Data analytics. Secondly, it analyses the typical data processing cycle of predictive systems to provide a step-by-step discussion of ethical implications, locating occurrences of predictive privacy violations. Thirdly, the paper sheds light on what is qualitatively new in the way predictive analytics challenges ethical principles such as human dignity and the (liberal) notion of individual privacy. These new challenges arise when predictive systems transform statistical inferences, which provide knowledge about the cohort of training data donors, into individual predictions, thereby crossing what I call the “prediction gap”. Finally, the paper summarizes that data protection in the age of predictive analytics is a collective matter as we face situations where an individual’s (or group’s) privacy is violated using dataotherindividuals provide about themselves, possibly even anonymously. (shrink)

There is a long history of the science of intelligent machines and its potential to provide scientific insights have been debated since the dawn of AI. In particular, there is renewed interest in the role of AI in research and research policy as an enabler of new methods, processes, management and evaluation which is still relatively under-explored. This empirical paper explores interviews with leading scholars on the potential impact of AI on research practice and culture through deductive, thematic analysis to (...) show the issues affecting academics and universities today. Our interviewees identify positive and negative consequences for research and researchers with respect to _collective_ and _individual use_. AI is perceived as helpful with respect to information gathering and other narrow tasks, and in support of impact and interdisciplinarity. However, using AI as a way of ‘speeding up—to keep up’ with bureaucratic and metricised processes, may proliferate negative aspects of academic culture in that the expansion of AI in research should assist and not replace human creativity. Research into the future role of AI in the research process needs to go further to address these challenges, and ask fundamental questions about how AI might assist in providing new tools able to question the values and principles driving institutions and research processes. We argue that to do this an explicit movement of meta-research on the role of AI in research should consider the effects for research and researcher creativity. Anticipatory approaches and engagement of diverse and critical voices at policy level and across disciplines should also be considered. (shrink)