Big data techniques, data-driven science and their technological applications raise many serious ethical questions, notably about privacy protection. In this paper, we highlight an entanglement between epistemology and ethics of big data. Discussing the mobilisation of big data in the fields of biomedical research and health care, we show how an overestimation of big data epistemic power – of their objectivity or rationality understood through the lens of neutrality – can become ethically threatening. Highlighting the irreducible non-neutrality at play in (...) big data tools, we insist upon the ethical importance of a critical epistemological approach in which big data are understood as possibly valuable only when coupled with human intelligence and evaluative rationality. (shrink)

There has been an increased focus within the AI ethics literature on questions of power, reflected in the ideal of accountability supported by many Responsible AI guidelines. While this recent debate points towards the power asymmetry between those who shape AI systems and those affected by them, the literature lacks normative grounding and misses conceptual clarity on how these power dynamics take shape. In this paper, I develop a workable conceptualization of said power dynamics according to Cristiano Castelfranchi’s conceptual framework (...) of power and argue that end-users depend on a system’s developers and users, because end-users rely on these systems to satisfy their goals, constituting a power asymmetry between developers, users and end-users. I ground my analysis in the neo-republican moral wrong of domination, drawing attention to legitimacy concerns of the power-dependence relation following from the current lack of accountability mechanisms. I illustrate my claims on the basis of a risk-prediction machine learning system, and propose institutional and project-specific solutions to mitigate domination. (shrink)

The concept of the digital phenotype has been used to refer to digital data prognostic or diagnostic of disease conditions. Medical conditions may be inferred from the time pattern in an insomniac’s tweets, the Facebook posts of a depressed individual, or the web searches of a hypochondriac. This paper conceptualizes digital data as an extended phenotype of humans, that is as digital information produced by humans and affecting human behavior and culture. It argues that there are ethical obligations to persons (...) affected by generalizable knowledge of a digital phenotype, not only those who are personally identifiable or involved in data generation. This claim is illustrated by considering the health-related digital phenotypes of precision medicine and digital epidemiology. (shrink)

How should we understand the surveillance state post Snowden? This paper is concerned with the relationship between increased surveillance capacity and state power. The paper begins by analysing two metaphors used in public post Snowden discourse to describe state surveillance practices: the haystack and the panopticon. It argues that these metaphors share a flawed common entailment regarding surveillance, knowledge and power which cannot accurately capture important aspects of state anxiety generated by mass surveillance in an age of big data. The (...) paper shows that the nature of big data itself complicates the power attributed to mass surveillance states by these metaphors and those who use them. Relying heavily on Ezrahi’s distinction between information and knowledge, the paper situates this argument concerning the state and anxiety borne of information overload in the context of literature that concerns the state and information management. Drawing primarily on James Scott’s work on legibility, it argues that the big data born of mass surveillance problematises the concept of information as empowering the state. Instead, understanding mass surveillance in an age of big data requires understanding the relationship between the surveillance state and information in terms of anxiety as well as power. (shrink)

On virtue ethical grounds, Barbro Fröding and Martin Peterson recently recommended that near-future AIs should be developed as ‘Friendly AI’. AI in social interaction with humans should be programmed such that they mimic aspects of human friendship. While it is a reasonable goal to implement AI systems interacting with humans as Friendly AI, I identify four issues that need to be addressed concerning Friendly AI with Fröding’s and Peterson’s understanding of Friendly AI as a starting point. In a first step, (...) I briefly recapitulate Fröding’s and Peterson’s arguments for Friendly AI. I then highlight some issues with Fröding’s and Peterson’s approach and line of reasoning and identify four problems related to the notion of Friendly AI, which all pertain to the role and need for humans’ moral development. These are that one should consider the moral tendencies and preferences of the humans interacting with a friendly AI, it needs to be considered whether the humans interacting with a Friendly AI are still developing their virtues and character traits, the indirect effects of replacing humans with Friendly AI should be considered with respect to the possibilities for humans to develop their moral virtues and that the question whether the AI is perceived as some form of Artificial General Intelligence cannot be neglected. In conclusion, I argue that all of these four problems are related to humans moral development and that this observation strongly emphasizes the role and need for humans moral development in correlation to the accelerating development of AI-systems. (shrink)

Organizations increasingly rely on algorithm-based HR decision-making to monitor their employees. This trend is reinforced by the technology industry claiming that its decision-making tools are efficient and objective, downplaying their potential biases. In our manuscript, we identify an important challenge arising from the efficiency-driven logic of algorithm-based HR decision-making, namely that it may shift the delicate balance between employees’ personal integrity and compliance more in the direction of compliance. We suggest that critical data literacy, ethical awareness, the use of participatory (...) design methods, and private regulatory regimes within civil society can help overcome these challenges. Our paper contributes to literature on workplace monitoring, critical data studies, personal integrity, and literature at the intersection between HR management and corporate responsibility. (shrink)

Despite the rapid adoption of technology in human resource departments, there is little empirical work that examines the potential challenges of algorithmic decision-making in the recruitment process. In this paper, we take the perspective of job applicants and examine how they perceive the use of algorithms in selection and recruitment. Across four studies on Amazon Mechanical Turk, we show that people in the role of a job applicant perceive algorithm-driven recruitment processes as less fair compared to human only or algorithm-assisted (...) human processes. This effect persists regardless of whether the outcome is favorable to the applicant or not. A potential mechanism underlying algorithm resistance is the belief that algorithms will not be able to recognize their uniqueness as a candidate. Although the use of algorithms has several benefits for organizations such as improved efficiency and bias reduction, our results highlight a potential cost of using them to screen potential employees during recruitment. (shrink)

Previous research in Explainable Artificial Intelligence (XAI) suggests that a main aim of explainability approaches is to satisfy specific interests, goals, expectations, needs, and demands regarding artificial systems (we call these “stakeholders' desiderata”) in a variety of contexts. However, the literature on XAI is vast, spreads out across multiple largely disconnected disciplines, and it often remains unclear how explainability approaches are supposed to achieve the goal of satisfying stakeholders' desiderata. This paper discusses the main classes of stakeholders calling for explainability (...) of artificial systems and reviews their desiderata. We provide a model that explicitly spells out the main concepts and relations necessary to consider and investigate when evaluating, adjusting, choosing, and developing explainability approaches that aim to satisfy stakeholders' desiderata. This model can serve researchers from the variety of different disciplines involved in XAI as a common ground. It emphasizes where there is interdisciplinary potential in the evaluation and the development of explainability approaches. (shrink)

The usefulness of machine learning algorithms has led to their widespread adoption prior to the development of a conceptual framework for making sense of them. One common response to this situation is to say that machine learning suffers from a “black box problem.” That is, machine learning algorithms are “opaque” to human users, failing to be “interpretable” or “explicable” in terms that would render categorization procedures “understandable.” The purpose of this paper is to challenge the widespread agreement about the existence (...) and importance of a black box problem. The first section argues that “interpretability” and cognates lack precise meanings when applied to algorithms. This makes the concepts difficult to use when trying to solve the problems that have motivated the call for interpretability. Furthermore, since there is no adequate account of the concepts themselves, it is not possible to assess whether particular technical features supply formal definitions of those concepts. The second section argues that there are ways of being a responsible user of these algorithms that do not require interpretability. In many cases in which a black box problem is cited, interpretability is a means to a further end such as justification or non-discrimination. Since addressing these problems need not involve something that looks like an “interpretation” of an algorithm, the focus on interpretability artificially constrains the solution space by characterizing one possible solution as the problem itself. Where possible, discussion should be reformulated in terms of the ends of interpretability. (shrink)

Departing from the claim that AI needs to be trustworthy, we find that ethical advice from an AI-powered algorithm is trusted even when its users know nothing about its training data and when they learn information about it that warrants distrust. We conducted online experiments where the subjects took the role of decision-makers who received advice from an algorithm on how to deal with an ethical dilemma. We manipulated the information about the algorithm and studied its influence. Our findings suggest (...) that AI is overtrusted rather than distrusted. We suggest digital literacy as a potential remedy to ensure the responsible use of AI. (shrink)

Academic and public debates are increasingly concerned with the question whether and how algorithmic decision-making may reinforce social inequality. Most previous research on this topic originates from computer science. The social sciences, however, have huge potentials to contribute to research on social consequences of ADM. Based on a process model of ADM systems, we demonstrate how social sciences may advance the literature on the impacts of ADM on social inequality by uncovering and mitigating biases in training data, by understanding data (...) processing and analysis, as well as by studying social contexts of algorithms in practice. Furthermore, we show that fairness notions need to be evaluated with respect to specific outcomes of ADM systems and with respect to concrete social contexts. Social sciences may evaluate how individuals handle algorithmic decisions in practice and how single decisions aggregate to macro social outcomes. In this overview, we highlight how social sciences can apply their knowledge on social stratification and on substantive domains of ADM applications to advance the understanding of social impacts of ADM. (shrink)

A wave of recent scholarship has warned about the potential for discriminatory harms of algorithmic systems, spurring an interest in algorithmic accountability and regulation. Meanwhile, parallel concerns about surveillance practices have already led to multiple successful regulatory efforts of surveillance technologies—many of which have algorithmic components. Here, we examine municipal surveillance regulation as offering lessons for algorithmic oversight. Taking the 2017 Seattle Surveillance Ordinance as our primary case study and surveying efforts across five other cities, we describe the features of (...) existing surveillance regulation; including procedures for describing surveillance technologies in detail, requirements for public engagement, and processes for establishing acceptable uses. Although the Seattle Surveillance Ordinance was not intended to address algorithmic accountability, we find these considerations to be relevant to the law’s aim of surfacing disparate impacts of systems in use. We also find that in notable cases government employees did not identify regulated algorithmic surveillance technologies as reliant on algorithmic or machine learning systems, highlighting definitional gaps that could hinder future efforts toward algorithmic regulation. We argue that finer-grained distinctions between types of information systems in the language of law and policy, and risk assessment tools integrated into their implementation would strengthen future regulatory efforts by rendering underlying algorithmic components more legible and accountable to political and community stakeholders. (shrink)

A growing literature is taking an institutionalist and governance perspective on how algorithms shape society based on unprecedented capacities for managing social complexity. Algorithmic governance altogether emerges as a novel and distinctive kind of societal steering. It appears to transcend established categories and modes of governance—and thus seems to call for new ways of thinking about how social relations can be regulated and ordered. However, as this paper argues, despite its novel way of realizing outcomes of collective steering and coordination, (...) it can nevertheless be grasped with an old and fundamental figure in political philosophy: that of Thomas Hobbes’ Leviathan. Comparing algorithmic governance with this figure serves to highlight their similarities as socio-political arrangements, and specifically to clarify how algorithmic governance parallels the apolitical traits of the Leviathan—it eliminates the political as it requires compliance and forgoing contestation to best fulfill its role and to produce satisfying outcomes. (shrink)

Algorithmic systems that provide services to people by supporting or replacing human decision-making promise greater convenience in various areas. The opacity of these applications, however, means that it is not clear how much they truly serve their users. A promising way to address the issue of possible undesired biases consists in giving users control by letting them configure a system and aligning its performance with users’ own preferences. However, as the present paper argues, this form of control over an algorithmic (...) system demands an algorithmic literacy that also entails a certain way of making oneself knowable: users must interrogate their own dispositions and see how these can be formalized such that they can be translated into the algorithmic system. This may, however, extend already existing practices through which people are monitored and probed and means that exerting such control requires users to direct a computational mode of thinking at themselves. (shrink)

Fairness of Artificial Intelligence decisions has become a big challenge for governments, companies, and societies. We offer a theoretical contribution to consider AI ethics outside of high-level and top-down approaches, based on the distinction between “reality” and “world” from Luc Boltanski. To do so, we provide a new perspective on the debate on AI fairness and show that criticism of ML unfairness is “realist”, in other words, grounded in an already instituted reality based on demographic categories produced by institutions. Second, (...) we show that the limits of “realist” fairness corrections lead to the elaboration of “radical responses” to fairness, that is, responses that radically change the format of data. Third, we show that fairness correction is shifting to a “domination regime” that absorbs criticism, and we provide some theoretical and practical avenues for further development in AI ethics. Using an ad hoc critical space stabilized by reality tests alongside the algorithm, we build a shared responsibility model which is compatible with the radical response to fairness issues. Finally, this paper shows the fundamental contribution of pragmatic sociology theories, insofar as they afford a social and political perspective on AI ethics by giving an active role to material actors such as database formats on ethical debates. In a context where data are increasingly numerous, granular, and behavioral, it is essential to renew our conception of AI ethics on algorithms in order to establish new models of responsibility for companies that take into account changes in the computing paradigm. (shrink)

We explore aversion to the use of algorithms in moral decision-making. So far, this aversion has been explained mainly by the fear of opaque decisions that are potentially biased. Using incentivized experiments, we study which role the desire for human discretion in moral decision-making plays. This seems justified in light of evidence suggesting that people might not doubt the quality of algorithmic decisions, but still reject them. In our first study, we found that people prefer humans with decision-making discretion to (...) algorithms that rigidly apply exogenously given human-created fairness principles to specific cases. In the second study, we found that people do not prefer humans to algorithms because they appreciate flesh-and-blood decision-makers per se, but because they appreciate humans’ freedom to transcend fairness principles at will. Our results contribute to a deeper understanding of algorithm aversion. They indicate that emphasizing the transparency of algorithms that clearly follow fairness principles might not be the only element for fostering societal algorithm acceptance and suggest reconsidering certain features of the decision-making process. (shrink)

In this paper, we build upon Bruno Latour’s political writings to address the current impasse regarding algorithms in public life. We assert that the increasing difficulties at governing algorithms—be they qualified as “machine learning,” “big data,” or “artificial intelligence”—can be related to their current ontological thinness: deriving from constricted views on theoretical practices, algorithms’ standard definition as problem-solving computerized methods provides poor grips for affective dissensions. We then emphasize on the role historical and ethnographic studies of algorithms can potentially play (...) in the politicization of algorithms. By both digging into the genealogy of algorithms’ constricted definition and by making their contemporary constitutive relationships more visible, both historical and ethnographic studies can contribute to vascularizing algorithms and making them objects of enlarged disputes. We conclude by giving a flavor of the political potential of the vascularization efforts we call for, using materials from an ethnographic study conducted in a computer science laboratory. (shrink)

This theoretical paper considers the morality of machine learning algorithms and systems in the light of the biases that ground their correctness. It begins by presenting biases not as a priori negative entities but as contingent external referents—often gathered in benchmarked repositories called ground-truth datasets—that define what needs to be learned and allow for performance measures. I then argue that ground-truth datasets and their concomitant practices—that fundamentally involve establishing biases to enable learning procedures—can be described by their respective morality, here (...) defined as the more or less accounted experience of hesitation when faced with what pragmatist philosopher William James called “genuine options”—that is, choices to be made in the heat of the moment that engage different possible futures. I then stress three constitutive dimensions of this pragmatist morality, as far as ground-truthing practices are concerned: the definition of the problem to be solved, the identification of the data to be collected and set up, and the qualification of the targets to be learned. I finally suggest that this three-dimensional conceptual space can be used to map machine learning algorithmic projects in terms of the morality of their respective and constitutive ground-truthing practices. Such techno-moral graphs may, in turn, serve as equipment for greater governance of machine learning algorithms and systems. (shrink)

BackgroundElectronic patient record (EPR) technology is a key enabler for improvements to healthcare service and management. To ensure these improvements and the means to achieve them are socially and ethically desirable, careful consideration of the ethical implications of EPRs is indicated. The purpose of this scoping review was to map the literature related to the ethics of EPR technology. The literature review was conducted to catalogue the prevalent ethical terms, to describe the associated ethical challenges and opportunities, and to identify (...) the actors involved. By doing so, it aimed to support the future development of ethics guidance in the EPR domain.MethodsTo identify journal articles debating the ethics of EPRs, Scopus, Web of Science, and PubMed academic databases were queried and yielded 123 eligible articles. The following inclusion criteria were applied: articles need to be in the English language; present normative arguments and not solely empirical research; include an abstract for software analysis; and discuss EPR technology.ResultsThe medical specialty, type of information captured and stored in EPRs, their use and functionality varied widely across the included articles. Ethical terms extracted were categorised into clusters ‘privacy’, ‘autonomy’, ‘risk/benefit’, ‘human relationships’, and ‘responsibility’. The literature shows that EPR-related ethical concerns can have both positive and negative implications, and that a wide variety of actors with rights and/or responsibilities regarding the safe and ethical adoption of the technology are involved.ConclusionsWhile there is considerable consensus in the literature regarding EPR-related ethical principles, some of the associated challenges and opportunities remain underdiscussed. For example, much of the debate is presented in a manner more in keeping with a traditional model of healthcare and fails to take account of the multidimensional ensemble of factors at play in the EPR era and the consequent need to redefine/modify ethical norms to align with a digitally-enabled health service. Similarly, the academic discussion focuses predominantly on bioethical values. However, approaches from digital ethics may also be helpful to identify and deliberate about current and emerging EPR-related ethical concerns. (shrink)

This paper undertakes a comparative legal study to analyze the challenges of privacy and personal data protection posed by Artificial Intelligence embedded in Robots, and to offer policy suggestions. After identifying the benefits from various AI usages and the risks posed by AI-related technologies, I then analyze legal frameworks and relevant discussions in the EU, USA, Canada, and Japan, and further consider the efforts of Privacy by Design originating in Ontario, Canada. While various AI usages provide great convenience, many issues, (...) including profiling, discriminatory decisions, lack of transparency, and impeding consent, have emerged. The unpredictability arising from the AI machine learning function poses further difficulties, which have only been partially addressed by legal frameworks in the aforementioned jurisdictions. However, analyzing the relevant discussions yielded several suggestions. The first priority is adopting PbD as the most flexible, soft-legal, and preferable approach toward AI-oriented issues. Implementing PbD will protect individual privacy and personal data without specific efforts, and achieve both the development of AI and the advancement of privacy and personal data protection. Technical measures that can adapt to an individual’s dynamic choices according to the “context” should be further developed. Furthermore, alternative technical measures, including those to solve the “algorithmic black box” or achieve differential privacy, warrant thorough examination. If AI surpasses human intelligence, a terminating function, such as a “kill switch” will be the last resort to preserve individual choice. Despite numerous difficulties, we must prepare for the coming AI-prevalent society by taking a flexible approach. (shrink)

Si la democracia consiste en posibilitar que todas las personas tengan iguales posibilidades de influir en las decisiones que les afectan, las sociedades digitales tienen que interrogarse por el modo de conseguir que los nuevos entornos hagan factible esa igualdad. Las primeras dificultades son conceptuales: entender cómo se configura la interacción entre los humanos y los algoritmos, en qué consiste el aprendizaje de estos dispositivos y cuál es la naturaleza de sus sesgos. Inmediatamente después nos topamos con la cuestión ineludible (...) de qué clase de igualdad estamos tratando de asegurar, teniendo en cuenta la diversidad de concepciones de la justicia que hay en nuestras sociedades. Si articular ese pluralismo no es un asunto que pueda resolverse con una técnica agregativa, sino que requiere compromisos políticos, entonces una concepción deliberativa de la democracia parece la más apta para conseguir esa igualdad a la que aspiran las sociedades democráticas. (shrink)

The growing presence of smart devices in our lives turns all of society into something largely unknown to us. The strategy of demanding transparency stems from the desire to reduce the ignorance to which this automated society seems to condemn us. An evaluation of this strategy first requires that we distinguish the different types of non-transparency. Once we reveal the limits of the transparency needed to confront these devices, the article examines the alternative strategy of explainable artificial intelligence and concludes (...) with the idea that these types of complex realities exceed individual capacities and are only comprehensible in a collective fashion. (shrink)

Companies increasingly deploy artificial intelligence technologies in their personnel recruiting and selection process to streamline it, making it faster and more efficient. AI applications can be found in various stages of recruiting, such as writing job ads, screening of applicant resumes, and analyzing video interviews via face recognition software. As these new technologies significantly impact people’s lives and careers but often trigger ethical concerns, the ethicality of these AI applications needs to be comprehensively understood. However, given the novelty of AI (...) applications in recruiting practice, the subject is still an emerging topic in academic literature. To inform and strengthen the foundation for future research, this paper systematically reviews the extant literature on the ethicality of AI-enabled recruiting to date. We identify 51 articles dealing with the topic, which we synthesize by mapping the ethical opportunities, risks, and ambiguities, as well as the proposed ways to mitigate ethical risks in practice. Based on this review, we identify gaps in the extant literature and point out moral questions that call for deeper exploration in future research. (shrink)

Artificial intelligence systems advising healthcare professionals will be widely introduced into healthcare settings within the next 5–10 years. This paper considers how this will sit with tort/negligence based legal approaches to compensation for medical error. It argues that the introduction of AI systems will provide an additional argument pointing towards no-fault compensation as the better legal solution to compensation for medical error in modern health care systems. The paper falls into four parts. The first part rehearses the main arguments for (...) and against no-fault compensation. The second explains why it is likely that AI systems will be widely introduced. The third part analyses why it is difficult to fit AI systems into fault-based compensation systems while the final part suggests how no-fault compensation could provide a possible solution to such challenges. (shrink)

There has been considerable debate around the ethical issues raised by data-driven technologies such as artificial intelligence. Ethical principles for the field have focused on the need to ensure...

Artificial intelligence is shaping strategy, activities, interactions, and relationships in business and specifically in marketing. The drawback of the substantial opportunities AI systems and applications provide in marketing are ethical controversies. Building on the literature on AI ethics, the authors systematically scrutinize the ethical challenges of deploying AI in marketing from a multi-stakeholder perspective. By revealing interdependencies and tensions between ethical principles, the authors shed light on the applicability of a purely principled, deontological approach to AI ethics in marketing. To (...) reconcile some of these tensions and account for the AI-for-social-good perspective, the authors make suggestions of how AI in marketing can be leveraged to promote societal and environmental well-being. (shrink)

Artificial intelligence can be a game changer to address the global challenge of humanity-threatening climate change by fostering sustainable development. Since chemical research and development lay the foundation for innovative products and solutions, this study presents a novel chemical research and development process backed with artificial intelligence and guiding ethical principles to account for both process- and outcome-related sustainability. Particularly in ethically salient contexts, ethical principles have to accompany research and development powered by artificial intelligence to promote social and environmental (...) good and sustainability while preventing any harm for all stakeholders affected. (shrink)

In more and more contexts, human decision-making is replaced by algorithmic decision-making. While promising to deliver efficient and objective decisions, algorithmic decision systems have specific weaknesses, some of which are particularly dangerous if data are collected and processed by profit-oriented companies. In this paper, I focus on two problems that are at the root of the logic of algorithmic decision-making: (1) (in)tolerance for ambiguity, and (2) instantiations of Campbell’s law, i. e. of indicators that are used for “social decision-making” being (...) subject to “corruption pressures” and tending to “distort and corrupt” the underlying social processes. As a result, algorithmic decision-making can risk missing the point of the social practice in question. These problems are intertwined with problems of structural injustice; hence, if algorithms are to deliver on their promises of efficiency and objectivity, accountability and critical scrutiny are needed. (shrink)

In this paper, I examine whether the use of artificial intelligence (AI) and automated decision-making (ADM) aggravates issues of discrimination as has been argued by several authors. For this purpose, I first take up the lively philosophical debate on discrimination and present my own definition of the concept. Equipped with this account, I subsequently review some of the recent literature on the use AI/ADM and discrimination. I explain how my account of discrimination helps to understand that the general claim in (...) view of the aggravation of discrimination is unwarranted. Finally, I argue that the use of AI/ADM can, in fact, increase issues of discrimination, but in a different way than most critics assume: it is due to its epistemic opacity that AI/ADM threatens to undermine our moral deliberation which is essential for reaching a common understanding of what should count as discrimination. As a consequence, it turns out that algorithms may actually help to detect hidden forms of discrimination. (shrink)

The primary goal of this essay is to provide a comprehensive overview and analysis of the various relations between material artifacts and the embodied mind. A secondary goal of this essay is to identify some of the trends in the design and use of artifacts. First, based on their functional properties, I identify four categories of artifacts co-opted by the embodied mind, namely (1) embodied artifacts, (2) perceptual artifacts, (3) cognitive artifacts, and (4) affective artifacts. These categories can overlap and (...) so some artifacts are members of more than one category. I also identify some of the techniques (or skills) we use when interacting with artifacts. Identifying these categories of artifacts and techniques allows us to map the landscape of relations between embodied minds and the artifactual world. Second, having identified categories of artifacts and techniques, this essay then outlines some of the trends in the design and use of artifacts, focussing on neuroprosthetics, brain-computer interfaces, and personalisation algorithms nudging their users towards particular epistemic paths of information consumption. (shrink)

This article presents a conceptual investigation into the value impacts and relations of algorithms in the domain of justice and security. As a conceptual investigation, it represents one step in a value sensitive design based methodology. Here, we explicate and analyse the expression of values of accuracy, privacy, fairness and equality, property and ownership, and accountability and transparency in this context. We find that values are sensitive to disvalue if algorithms are designed, implemented or deployed inappropriately or without sufficient consideration (...) for their value impacts, potentially resulting in problems including discrimination and constrained autonomy. Furthermore, we outline a framework of conceptual relations of values indicated by our analysis, and potential value tensions in their implementation and deployment with a view towards supporting future research, and supporting the value sensitive design of algorithms in justice and security. (shrink)

Journal of Philosophy of Education, EarlyView.

This paper explores how calculative cultures shape perceptions of models and practices of model use in the financial industry. A calculative culture comprises a specific set of practices and norms concerning data and model use in an organizational setting. Drawing on interviews with model users working in algorithmic securities trading, I argue that the introduction of complex machine-learning models changes the dynamics in calculative cultures, which leads to a displacement of human judgment in quantitative finance. In this paper, I distinguish (...) between three calculative cultures: an idealistic culture of undivided trust in models, a pragmatic culture of skepticism toward model accuracy, and a pragmatic idealist culture of early stage skepticism and implementation and production-phase idealism. Based on the empirical material, the analysis engages with examples of each of the three calculative cultures. The study contributes to the social studies of finance and science and technology studies more broadly by showing how perceptions of models shape and are shaped through model work in data-intensive, computerized finance. (shrink)

Algorithmic systems are increasingly used by state agencies to inform decisions about humans. They produce scores on risks of recidivism in criminal justice, indicate the probability for a job seeker to find a job in the labor market, or calculate whether an applicant should get access to a certain university program. In this contribution, we take an interdisciplinary perspective, provide a bird’s eye view of the different key decisions that are to be taken when state actors decide to use an (...) algorithmic system, and illustrate these decisions with empirical examples from case studies. Building on these insights, we discuss the main pitfalls and promises of the use of algorithmic system by the state and focus on four levels: The most basic question whether an algorithmic system should be used at all, the regulation and governance of the system, issues of algorithm design, and, finally, questions related to the implementation of the system on the ground and the human–machine-interaction that comes with it. Based on our assessment of the advantages and challenges that arise at each of these levels, we propose a set of crucial questions to be asked when such intricate matters are addressed. (shrink)

The ongoing explosion of interest in artificial intelligence is fueled in part by recently developed techniques in machine learning. Those techniques allow automated systems to process huge amounts of data, utilizing mathematical methods that depart from traditional statistical approaches, and resulting in impressive advancements in our ability to make predictions and uncover correlations across a host of interesting domains. But as is now widely discussed, the way that those systems arrive at their outputs is often opaque, even to the experts (...) who design and deploy them. Is it morally problematic to make use of opaque automated methods when making high-stakes decisions, like whether to issue a loan to an applicant, or whether to approve a parole request? Many scholars answer in the affirmative. However, there is no widely accepted explanation for why transparent systems are morally preferable to opaque systems. We argue that the use of automated decision-making systems sometimes violates duties of consideration that are owed by decision-makers to decision-subjects, duties that are both epistemic and practical in character. Violations of that kind generate a weighty consideration against the use of opaque decision systems. In the course of defending our approach, we show that it is able to address three major challenges sometimes leveled against attempts to defend the moral import of transparency in automated decision-making. (shrink)

Governments and citizens of nearly every nation have been compelled to respond to COVID-19. Many measures have been adopted, including contact tracing and risk assessment algorithms, whereby citizen whereabouts are monitored to trace contact with other infectious individuals in order to generate a risk status via algorithmic evaluation. Based on 38 in-depth interviews, we investigate how people make sense of Health Code, the Chinese contact tracing and risk assessment algorithmic sociotechnical assemblage. We probe how people accept or resist Health Code (...) by examining their ongoing, dynamic, and relational interactions with it. Participants display a rich variety of attitudes toward privacy and surveillance, ranging from fatalism to the possibility of privacy to trade-offs for surveillance in exchange for public health, which is mediated by the perceived effectiveness of Health Code and changing views on the intentions of institutions who deploy it. We show how perceived competency varies not just on how well the technology works, but on the social and cultural enforcement of various non-technical aspects like quarantine, citizen data inputs, and cell reception. Furthermore, we illustrate how perceptions of Health Code are nested in people’s broader interpretations of disease control at the national and global level, and unexpectedly strengthen the Chinese authority’s legitimacy. None of the Chinese public, Health Code, or people’s perceptions toward Health Code are predetermined, fixed, or categorically consistent, but are co-constitutive and dynamic over time. We conclude with a theorization of a relational perception and methodological reflections to study algorithmic sociotechnical assemblages beyond COVID-19. (shrink)

The increasing implementation of and reliance on machine-learning (ML) algorithms to perform tasks, deliver services and make decisions in health and healthcare have made the need for fairness in ML, and more specifically in healthcare ML algorithms (HMLA), a very important and urgent task. However, while the debate on fairness in the ethics of artificial intelligence (AI) and in HMLA has grown significantly over the last decade, the very concept of fairness as an ethical value has not yet been sufficiently (...) explored. Our paper aims to fill this gap and address the AI ethics principle of fairness from a conceptual standpoint, drawing insights from accounts of fairness elaborated in moral philosophy and using them to conceptualise fairness as an ethical value and to redefine fairness in HMLA accordingly. To achieve our goal, following a first section aimed at clarifying the background, methodology and structure of the paper, in the second section, we provide an overview of the discussion of the AI ethics principle of fairness in HMLA and show that the concept of fairness underlying this debate is framed in purely distributive terms and overlaps with non-discrimination, which is defined in turn as the absence of biases. After showing that this framing is inadequate, in the third section, we pursue an ethical inquiry into the concept of fairness and argue that fairness ought to be conceived of as an ethical value. Following a clarification of the relationship between fairness and non-discrimination, we show that the two do not overlap and that fairness requires much more than just non-discrimination. Moreover, we highlight that fairness not only has a distributive but also a socio-relational dimension. Finally, we pinpoint the constitutive components of fairness. In doing so, we base our arguments on a renewed reflection on the concept of respect, which goes beyond the idea of equal respect to include respect for individual persons. In the fourth section, we analyse the implications of our conceptual redefinition of fairness as an ethical value in the discussion of fairness in HMLA. Here, we claim that fairness requires more than non-discrimination and the absence of biases as well as more than just distribution; it needs to ensure that HMLA respects persons both as persons and as particular individuals. Finally, in the fifth section, we sketch some broader implications and show how our inquiry can contribute to making HMLA and, more generally, AI promote the social good and a fairer society. (shrink)

According to a recent survey by the HR Research Institute, as the presence of artificial intelligence (AI) becomes increasingly common in the workplace, HR professionals are worried that the use of recruitment algorithms will lead to a “dehumanization” of the hiring process. Our main goals in this paper are threefold: i) to bring attention to this neglected issue, ii) to clarify what exactly this concern about dehumanization might amount to, and iii) to sketch an argument for why dehumanizing the hiring (...) process is ethically suspect. After distinguishing the use of the term “dehumanization” in this context (i.e., removing the human presence) from its more common meaning in the interdisciplinary field of dehumanization studies (i.e., conceiving of other humans as subhuman), we argue that the use of hiring algorithms may negatively impact the employee-employer relationship. We argue that there are good independent reasons to accept a substantive employee-employer relationship, as well as an applicant-employer relationship, both of which are consistent with a stakeholder theory of corporate obligations. We further argue that dehumanizing the hiring process may negatively impact these relationships because of the difference between the values of human recruiters and the values embedded in recruitment algorithms. Drawing on Nguyen’s (2021) critique of how Twitter “gamifies communication”, we argue that replacing human recruiters with algorithms imports artificial values into the hiring process. We close by briefly considering some ways to potentially mitigate the problems posed by recruitment algorithms, along with the possibility that some difficult trade-offs will need to be made. (shrink)

Research conducted on Brain-Computer Interfaces has grown considerably during the last decades. With the help of BCIs, users can gain a wide range of functions. Our aim in this paper is to analyze the impact of BCIs on autonomy. To this end, we introduce three abilities that most accounts of autonomy take to be essential: the ability to use information and knowledge to produce reasons; the ability to ensure that intended actions are effectively realized ; and the ability to enact (...) intentions within concrete relationships and contexts. We then consider the impact of BCI technology on each of these abilities. Although on first glance, BCIs solely enhance self-determination because they restore or improve abilities, we will show that there are other positive, but also negative impacts on user autonomy, which require further philosophical and ethical discussions. (shrink)

This short commentary on Peters identifies the entrenchment of political positions as one additional concern related to algorithmic political bias, beyond those identified by Peters. First, it is observed that the political positions detected and predicted by algorithms are typically contingent and largely explained by “political tribalism”, as argued by Brennan. Second, following Hacking, the social construction of political identities is analyzed and it is concluded that algorithmic political bias can contribute to such identities. Third, following Nozick, it is argued (...) that purist political positions may stand in the way of the pursuit of all worthy values and goals to be pursued in the political realm and that to the extent that algorithmic political bias entrenches political positions, it also hinders this healthy “zigzag of politics”. (shrink)

In October 2016, the White House, the European Parliament, and the UK House of Commons each issued a report outlining their visions on how to prepare society for the widespread use of artificial intelligence. In this article, we provide a comparative assessment of these three reports in order to facilitate the design of policies favourable to the development of a ‘good AI society’. To do so, we examine how each report addresses the following three topics: the development of a ‘good (...) AI society’; the role and responsibility of the government, the private sector, and the research community in pursuing such a development; and where the recommendations to support such a development may be in need of improvement. Our analysis concludes that the reports address adequately various ethical, social, and economic topics, but come short of providing an overarching political vision and long-term strategy for the development of a ‘good AI society’. In order to contribute to fill this gap, in the conclusion we suggest a two-pronged approach. (shrink)

In this article, we develop the concept of Transparency by Design that serves as practical guidance in helping promote the beneficial functions of transparency while mitigating its challenges in automated-decision making environments. With the rise of artificial intelligence and the ability of AI systems to make automated and self-learned decisions, a call for transparency of how such systems reach decisions has echoed within academic and policy circles. The term transparency, however, relates to multiple concepts, fulfills many functions, and holds different (...) promises that struggle to be realized in concrete applications. Indeed, the complexity of transparency for ADM shows tension between transparency as a normative ideal and its translation to practical application. To address this tension, we first conduct a review of transparency, analyzing its challenges and limitations concerning automated decision-making practices. We then look at the lessons learned from the development of Privacy by Design, as a basis for developing the Transparency by Design principles. Finally, we propose a set of nine principles to cover relevant contextual, technical, informational, and stakeholder-sensitive considerations. Transparency by Design is a model that helps organizations design transparent AI systems, by integrating these principles in a step-by-step manner and as an ex-ante value, not as an afterthought. (shrink)

Although overarching if not foundational conceptualizations of digital governance in the field of critical data studies aptly account for and explain subjection, calculated resistance is left conceptually unattended despite case studies that document instances of resistance. I ask at the outset why conceptualizations of digital governance are so bleak, and I argue that all are underscored implicitly by a Deleuzian theory of desire that overlooks agency, defined here in Foucauldian terms. I subsequently conceptualize digital governance as encompassing subjection as well (...) as resistance, and I cast the two in relational perspective by making use of the concepts “affordance” and “assemblage” in conjunction with multiple subjectivities and Foucault's view of power as productive as well as his view of resistance as an “antagonism of strategies” in his late scholarship on resistance, ethics, and subjectivity. I offer examples of salient modes of what I call “productive” resistance, and from a Foucauldian perspective I explain how each mode targets and subverts technologies of repressive power to produce new elements of the digital environment and construct new truths. I conclude by recognizing the agency embodied in resistance as an end in itself, but I also consider that modes of productive resistance can have extrinsic value as they affect the fluid interaction among elements of the digital environment, potentially disrupting the presumed structure of dominance and dependence, and opening our conceptualization of algorithmic life to hopeful possibilities for change. (shrink)

The use of black box algorithms in medicine has raised scholarly concerns due to their opaqueness and lack of trustworthiness. Concerns about potential bias, accountability and responsibility, patient autonomy and compromised trust transpire with black box algorithms. These worries connect epistemic concerns with normative issues. In this paper, we outline that black box algorithms are less problematic for epistemic reasons than many scholars seem to believe. By outlining that more transparency in algorithms is not always necessary, and by explaining that (...) computational processes are indeed methodologically opaque to humans, we argue that the reliability of algorithms provides reasons for trusting the outcomes of medical artificial intelligence. To this end, we explain how computational reliabilism, which does not require transparency and supports the reliability of algorithms, justifies the belief that results of medical AI are to be trusted. We also argue that several ethical concerns remain with black box algorithms, even when the results are trustworthy. Having justified knowledge from reliable indicators is, therefore, necessary but not sufficient for normatively justifying physicians to act. This means that deliberation about the results of reliable algorithms is required to find out what is a desirable action. Thus understood, we argue that such challenges should not dismiss the use of black box algorithms altogether but should inform the way in which these algorithms are designed and implemented. When physicians are trained to acquire the necessary skills and expertise, and collaborate with medical informatics and data scientists, black box algorithms can contribute to improving medical care. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Would transparency contribute to restoring accountability for such systems as is often maintained? Several objections to full transparency are examined: the loss of privacy when datasets become public, the perverse effects of disclosure of the very algorithms themselves, the potential loss of companies’ competitive edge, and the limited gains in answerability to be expected since sophisticated algorithms usually are (...) inherently opaque. It is concluded that, at least presently, full transparency for oversight bodies alone is the only feasible option; extending it to the public at large is normally not advisable. Moreover, it is argued that algorithmic decisions preferably should become more understandable; to that effect, the models of machine learning to be employed should either be interpreted ex post or be interpretable by design ex ante. (shrink)

Artificial Intelligence (AI) is increasingly adopted in society, creating numerous opportunities but at the same time posing ethical challenges. Many of these are familiar, such as issues of fairness, responsibility and privacy, but are presented in a new and challenging guise due to our limited ability to steer and predict the outputs of AI systems. This chapter first introduces these ethical challenges, stressing that overviews of values are a good starting point but frequently fail to suffice due to the context (...) sensitivity of ethical challenges. Frequently, additional (ethical) values emerge for specific applications, as e.g. challenges with fraud detection are very different from those around language technologies. Second, methods to tackle these challenges are discussed. Main ethical theories (virtue ethics, consequentialism, and deontology) are shown to provide a starting point, but often lack the details needed for actionable AI Ethics. Instead, mid-level philosophical theories coupled to design-approaches such as Design for Values, together with interdisciplinary working methods, offer the best way forwards. The chapter aims to show how these approaches can lead to an ethics of AI that is actionable and that can be proactively integrated in the design of AI systems. (shrink)

Many artificial intelligence (AI) systems currently used for decision-making are opaque, i.e., the internal factors that determine their decisions are not fully known to people due to the systems’ computational complexity. In response to this problem, several researchers have argued that human decision-making is equally opaque and since simplifying, reason-giving explanations (rather than exhaustive causal accounts) of a decision are typically viewed as sufficient in the human case, the same should hold for algorithmic decision-making. Here, I contend that this argument (...) overlooks that human decision-making is sometimes significantly more transparent and trustworthy than algorithmic decision-making. This is because when people explain their decisions by giving reasons for them, this frequently prompts those giving the reasons to govern or regulate themselves so as to think and act in ways that confirm their reason reports. AI explanation systems lack this self-regulative feature. Overlooking it when comparing algorithmic and human decision-making can result in underestimations of the transparency of human decision-making and in the development of explainable AI that may mislead people by activating generally warranted beliefs about the regulative dimension of reason-giving. (shrink)

The modern abundance and prominence of data has led to the development of “data science” as a new field of enquiry, along with a body of epistemological reflections upon its foundations, methods, and consequences. This article provides a systematic analysis and critical review of significant open problems and debates in the epistemology of data science. We propose a partition of the epistemology of data science into the following five domains: (i) the constitution of data science; (ii) the kind of enquiry (...) that it identifies; (iii) the kinds of knowledge that data science generates; (iv) the nature and epistemological significance of “black box” problems; and (v) the relationship between data science and the philosophy of science more generally. (shrink)