In this paper, I examine whether the use of artificial intelligence (AI) and automated decision-making (ADM) aggravates issues of discrimination as has been argued by several authors. For this purpose, I first take up the lively philosophical debate on discrimination and present my own definition of the concept. Equipped with this account, I subsequently review some of the recent literature on the use AI/ADM and discrimination. I explain how my account of discrimination helps to understand that the general claim in (...) view of the aggravation of discrimination is unwarranted. Finally, I argue that the use of AI/ADM can, in fact, increase issues of discrimination, but in a different way than most critics assume: it is due to its epistemic opacity that AI/ADM threatens to undermine our moral deliberation which is essential for reaching a common understanding of what should count as discrimination. As a consequence, it turns out that algorithms may actually help to detect hidden forms of discrimination. (shrink)

The primary goal of this essay is to provide a comprehensive overview and analysis of the various relations between material artifacts and the embodied mind. A secondary goal of this essay is to identify some of the trends in the design and use of artifacts. First, based on their functional properties, I identify four categories of artifacts co-opted by the embodied mind, namely (1) embodied artifacts, (2) perceptual artifacts, (3) cognitive artifacts, and (4) affective artifacts. These categories can overlap and (...) so some artifacts are members of more than one category. I also identify some of the techniques (or skills) we use when interacting with artifacts. Identifying these categories of artifacts and techniques allows us to map the landscape of relations between embodied minds and the artifactual world. Second, having identified categories of artifacts and techniques, this essay then outlines some of the trends in the design and use of artifacts, focussing on neuroprosthetics, brain-computer interfaces, and personalisation algorithms nudging their users towards particular epistemic paths of information consumption. (shrink)

PurposeThis paper aims to argue that traditional ethical theories used in disaster response may be inadequate and particularly strained by the emergence of new technologies and social media, particularly with regard to privacy. The paper suggests incorporation of care ethics into the disaster ethics nexus to better include the perspectives of disaster affected communities.Design/methodology/approachThis paper presents a theoretical examination of privacy and care ethics in the context of social media/digitally enhanced disaster response.FindingsThe paper proposes an ethics of care can fruitfully (...) by used by public and private agents in disaster management. Its relational ontology restores the priority of fostering good relationships between stakeholders, thus giving central importance to values such as transparency and trust and the situated knowledge of disaster-affected communities.Research limitations/implicationsThis paper presents theoretical research and is limited by the availability of empirical data. There is opportunity for future research to evaluate the impact of a conscious adoption of an ethics of care by disaster management agents.Practical implicationsAn ethos of care ethics needs to be mainstreamed into disaster management organisations and digital initiatives.Social implicationsThis paper argues that power asymmetry in disaster response renders the public vulnerable to abuse, and that the adoption of care ethics can support disaster management agents in recognising this power imbalance and wielding power responsibly.Originality/valueThis paper examines the applicability of an alternative ethical framework to novel circumstances. (shrink)

This article presents a conceptual investigation into the value impacts and relations of algorithms in the domain of justice and security. As a conceptual investigation, it represents one step in a value sensitive design based methodology. Here, we explicate and analyse the expression of values of accuracy, privacy, fairness and equality, property and ownership, and accountability and transparency in this context. We find that values are sensitive to disvalue if algorithms are designed, implemented or deployed inappropriately or without sufficient consideration (...) for their value impacts, potentially resulting in problems including discrimination and constrained autonomy. Furthermore, we outline a framework of conceptual relations of values indicated by our analysis, and potential value tensions in their implementation and deployment with a view towards supporting future research, and supporting the value sensitive design of algorithms in justice and security. (shrink)

Journal of Philosophy of Education, EarlyView.

This paper explores how calculative cultures shape perceptions of models and practices of model use in the financial industry. A calculative culture comprises a specific set of practices and norms concerning data and model use in an organizational setting. Drawing on interviews with model users working in algorithmic securities trading, I argue that the introduction of complex machine-learning models changes the dynamics in calculative cultures, which leads to a displacement of human judgment in quantitative finance. In this paper, I distinguish (...) between three calculative cultures: an idealistic culture of undivided trust in models, a pragmatic culture of skepticism toward model accuracy, and a pragmatic idealist culture of early stage skepticism and implementation and production-phase idealism. Based on the empirical material, the analysis engages with examples of each of the three calculative cultures. The study contributes to the social studies of finance and science and technology studies more broadly by showing how perceptions of models shape and are shaped through model work in data-intensive, computerized finance. (shrink)

The current “AI Summer” is marked by scientific breakthroughs and economic successes in the fields of research, development, and application of systems with artificial intelligence. But, aside from the great hopes and promises associated with artificial intelligence, there are a number of challenges, shortcomings and even limitations of the technology. For one, these challenges arise from methodological and epistemological misconceptions about the capabilities of artificial intelligence. Secondly, they result from restrictions of the social context in which the development of applications (...) of machine learning is embedded. And third, they are a consequence of current technical limitations in the development and use of artificial intelligence. The paper intends to provide an overview of current challenges which the research and development of applications in the field of artificial intelligence and machine learning have to face, whereas all three mentioned areas are to be further explored in this paper. (shrink)

Algorithmic systems are increasingly used by state agencies to inform decisions about humans. They produce scores on risks of recidivism in criminal justice, indicate the probability for a job seeker to find a job in the labor market, or calculate whether an applicant should get access to a certain university program. In this contribution, we take an interdisciplinary perspective, provide a bird’s eye view of the different key decisions that are to be taken when state actors decide to use an (...) algorithmic system, and illustrate these decisions with empirical examples from case studies. Building on these insights, we discuss the main pitfalls and promises of the use of algorithmic system by the state and focus on four levels: The most basic question whether an algorithmic system should be used at all, the regulation and governance of the system, issues of algorithm design, and, finally, questions related to the implementation of the system on the ground and the human–machine-interaction that comes with it. Based on our assessment of the advantages and challenges that arise at each of these levels, we propose a set of crucial questions to be asked when such intricate matters are addressed. (shrink)

The use of artificial intelligence in healthcare contexts is highly controversial for the (bio)ethical conundrums it creates. One of the main problems arising from its implementation is the lack of transparency of machine learning algorithms, which is thought to impede the patient’s autonomous choice regarding their medical decisions. If the patient is unable to clearly understand why and how an AI algorithm reached certain medical decision, their autonomy is being hovered. However, there are alternatives to prevent the negative impact of (...) AI opacity in shared (healthcare professional–patient) decision-making processes, and benefit from the high accuracy of such systems. Despite these ethical issues, this paper advocates for its usage and also defends the positive role that it could have in enhancing patients’ autonomy. I advocate for a system where the implementation of AI is only partial, and humans will remain in control of oversighting algorithm interpretation of medical images and patients’ data. (shrink)

The ongoing explosion of interest in artificial intelligence is fueled in part by recently developed techniques in machine learning. Those techniques allow automated systems to process huge amounts of data, utilizing mathematical methods that depart from traditional statistical approaches, and resulting in impressive advancements in our ability to make predictions and uncover correlations across a host of interesting domains. But as is now widely discussed, the way that those systems arrive at their outputs is often opaque, even to the experts (...) who design and deploy them. Is it morally problematic to make use of opaque automated methods when making high-stakes decisions, like whether to issue a loan to an applicant, or whether to approve a parole request? Many scholars answer in the affirmative. However, there is no widely accepted explanation for why transparent systems are morally preferable to opaque systems. We argue that the use of automated decision-making systems sometimes violates duties of consideration that are owed by decision-makers to decision-subjects, duties that are both epistemic and practical in character. Violations of that kind generate a weighty consideration against the use of opaque decision systems. In the course of defending our approach, we show that it is able to address three major challenges sometimes leveled against attempts to defend the moral import of transparency in automated decision-making. (shrink)

Governments and citizens of nearly every nation have been compelled to respond to COVID-19. Many measures have been adopted, including contact tracing and risk assessment algorithms, whereby citizen whereabouts are monitored to trace contact with other infectious individuals in order to generate a risk status via algorithmic evaluation. Based on 38 in-depth interviews, we investigate how people make sense of Health Code, the Chinese contact tracing and risk assessment algorithmic sociotechnical assemblage. We probe how people accept or resist Health Code (...) by examining their ongoing, dynamic, and relational interactions with it. Participants display a rich variety of attitudes toward privacy and surveillance, ranging from fatalism to the possibility of privacy to trade-offs for surveillance in exchange for public health, which is mediated by the perceived effectiveness of Health Code and changing views on the intentions of institutions who deploy it. We show how perceived competency varies not just on how well the technology works, but on the social and cultural enforcement of various non-technical aspects like quarantine, citizen data inputs, and cell reception. Furthermore, we illustrate how perceptions of Health Code are nested in people’s broader interpretations of disease control at the national and global level, and unexpectedly strengthen the Chinese authority’s legitimacy. None of the Chinese public, Health Code, or people’s perceptions toward Health Code are predetermined, fixed, or categorically consistent, but are co-constitutive and dynamic over time. We conclude with a theorization of a relational perception and methodological reflections to study algorithmic sociotechnical assemblages beyond COVID-19. (shrink)

The increasing implementation of and reliance on machine-learning (ML) algorithms to perform tasks, deliver services and make decisions in health and healthcare have made the need for fairness in ML, and more specifically in healthcare ML algorithms (HMLA), a very important and urgent task. However, while the debate on fairness in the ethics of artificial intelligence (AI) and in HMLA has grown significantly over the last decade, the very concept of fairness as an ethical value has not yet been sufficiently (...) explored. Our paper aims to fill this gap and address the AI ethics principle of fairness from a conceptual standpoint, drawing insights from accounts of fairness elaborated in moral philosophy and using them to conceptualise fairness as an ethical value and to redefine fairness in HMLA accordingly. To achieve our goal, following a first section aimed at clarifying the background, methodology and structure of the paper, in the second section, we provide an overview of the discussion of the AI ethics principle of fairness in HMLA and show that the concept of fairness underlying this debate is framed in purely distributive terms and overlaps with non-discrimination, which is defined in turn as the absence of biases. After showing that this framing is inadequate, in the third section, we pursue an ethical inquiry into the concept of fairness and argue that fairness ought to be conceived of as an ethical value. Following a clarification of the relationship between fairness and non-discrimination, we show that the two do not overlap and that fairness requires much more than just non-discrimination. Moreover, we highlight that fairness not only has a distributive but also a socio-relational dimension. Finally, we pinpoint the constitutive components of fairness. In doing so, we base our arguments on a renewed reflection on the concept of respect, which goes beyond the idea of equal respect to include respect for individual persons. In the fourth section, we analyse the implications of our conceptual redefinition of fairness as an ethical value in the discussion of fairness in HMLA. Here, we claim that fairness requires more than non-discrimination and the absence of biases as well as more than just distribution; it needs to ensure that HMLA respects persons both as persons and as particular individuals. Finally, in the fifth section, we sketch some broader implications and show how our inquiry can contribute to making HMLA and, more generally, AI promote the social good and a fairer society. (shrink)

According to a recent survey by the HR Research Institute, as the presence of artificial intelligence (AI) becomes increasingly common in the workplace, HR professionals are worried that the use of recruitment algorithms will lead to a “dehumanization” of the hiring process. Our main goals in this paper are threefold: i) to bring attention to this neglected issue, ii) to clarify what exactly this concern about dehumanization might amount to, and iii) to sketch an argument for why dehumanizing the hiring (...) process is ethically suspect. After distinguishing the use of the term “dehumanization” in this context (i.e., removing the human presence) from its more common meaning in the interdisciplinary field of dehumanization studies (i.e., conceiving of other humans as subhuman), we argue that the use of hiring algorithms may negatively impact the employee-employer relationship. We argue that there are good independent reasons to accept a substantive employee-employer relationship, as well as an applicant-employer relationship, both of which are consistent with a stakeholder theory of corporate obligations. We further argue that dehumanizing the hiring process may negatively impact these relationships because of the difference between the values of human recruiters and the values embedded in recruitment algorithms. Drawing on Nguyen’s (2021) critique of how Twitter “gamifies communication”, we argue that replacing human recruiters with algorithms imports artificial values into the hiring process. We close by briefly considering some ways to potentially mitigate the problems posed by recruitment algorithms, along with the possibility that some difficult trade-offs will need to be made. (shrink)

Research conducted on Brain-Computer Interfaces has grown considerably during the last decades. With the help of BCIs, users can gain a wide range of functions. Our aim in this paper is to analyze the impact of BCIs on autonomy. To this end, we introduce three abilities that most accounts of autonomy take to be essential: the ability to use information and knowledge to produce reasons; the ability to ensure that intended actions are effectively realized ; and the ability to enact (...) intentions within concrete relationships and contexts. We then consider the impact of BCI technology on each of these abilities. Although on first glance, BCIs solely enhance self-determination because they restore or improve abilities, we will show that there are other positive, but also negative impacts on user autonomy, which require further philosophical and ethical discussions. (shrink)

This short commentary on Peters identifies the entrenchment of political positions as one additional concern related to algorithmic political bias, beyond those identified by Peters. First, it is observed that the political positions detected and predicted by algorithms are typically contingent and largely explained by “political tribalism”, as argued by Brennan. Second, following Hacking, the social construction of political identities is analyzed and it is concluded that algorithmic political bias can contribute to such identities. Third, following Nozick, it is argued (...) that purist political positions may stand in the way of the pursuit of all worthy values and goals to be pursued in the political realm and that to the extent that algorithmic political bias entrenches political positions, it also hinders this healthy “zigzag of politics”. (shrink)

In October 2016, the White House, the European Parliament, and the UK House of Commons each issued a report outlining their visions on how to prepare society for the widespread use of artificial intelligence. In this article, we provide a comparative assessment of these three reports in order to facilitate the design of policies favourable to the development of a ‘good AI society’. To do so, we examine how each report addresses the following three topics: the development of a ‘good (...) AI society’; the role and responsibility of the government, the private sector, and the research community in pursuing such a development; and where the recommendations to support such a development may be in need of improvement. Our analysis concludes that the reports address adequately various ethical, social, and economic topics, but come short of providing an overarching political vision and long-term strategy for the development of a ‘good AI society’. In order to contribute to fill this gap, in the conclusion we suggest a two-pronged approach. (shrink)

In this article, we develop the concept of Transparency by Design that serves as practical guidance in helping promote the beneficial functions of transparency while mitigating its challenges in automated-decision making environments. With the rise of artificial intelligence and the ability of AI systems to make automated and self-learned decisions, a call for transparency of how such systems reach decisions has echoed within academic and policy circles. The term transparency, however, relates to multiple concepts, fulfills many functions, and holds different (...) promises that struggle to be realized in concrete applications. Indeed, the complexity of transparency for ADM shows tension between transparency as a normative ideal and its translation to practical application. To address this tension, we first conduct a review of transparency, analyzing its challenges and limitations concerning automated decision-making practices. We then look at the lessons learned from the development of Privacy by Design, as a basis for developing the Transparency by Design principles. Finally, we propose a set of nine principles to cover relevant contextual, technical, informational, and stakeholder-sensitive considerations. Transparency by Design is a model that helps organizations design transparent AI systems, by integrating these principles in a step-by-step manner and as an ex-ante value, not as an afterthought. (shrink)

Although overarching if not foundational conceptualizations of digital governance in the field of critical data studies aptly account for and explain subjection, calculated resistance is left conceptually unattended despite case studies that document instances of resistance. I ask at the outset why conceptualizations of digital governance are so bleak, and I argue that all are underscored implicitly by a Deleuzian theory of desire that overlooks agency, defined here in Foucauldian terms. I subsequently conceptualize digital governance as encompassing subjection as well (...) as resistance, and I cast the two in relational perspective by making use of the concepts “affordance” and “assemblage” in conjunction with multiple subjectivities and Foucault's view of power as productive as well as his view of resistance as an “antagonism of strategies” in his late scholarship on resistance, ethics, and subjectivity. I offer examples of salient modes of what I call “productive” resistance, and from a Foucauldian perspective I explain how each mode targets and subverts technologies of repressive power to produce new elements of the digital environment and construct new truths. I conclude by recognizing the agency embodied in resistance as an end in itself, but I also consider that modes of productive resistance can have extrinsic value as they affect the fluid interaction among elements of the digital environment, potentially disrupting the presumed structure of dominance and dependence, and opening our conceptualization of algorithmic life to hopeful possibilities for change. (shrink)

The use of black box algorithms in medicine has raised scholarly concerns due to their opaqueness and lack of trustworthiness. Concerns about potential bias, accountability and responsibility, patient autonomy and compromised trust transpire with black box algorithms. These worries connect epistemic concerns with normative issues. In this paper, we outline that black box algorithms are less problematic for epistemic reasons than many scholars seem to believe. By outlining that more transparency in algorithms is not always necessary, and by explaining that (...) computational processes are indeed methodologically opaque to humans, we argue that the reliability of algorithms provides reasons for trusting the outcomes of medical artificial intelligence. To this end, we explain how computational reliabilism, which does not require transparency and supports the reliability of algorithms, justifies the belief that results of medical AI are to be trusted. We also argue that several ethical concerns remain with black box algorithms, even when the results are trustworthy. Having justified knowledge from reliable indicators is, therefore, necessary but not sufficient for normatively justifying physicians to act. This means that deliberation about the results of reliable algorithms is required to find out what is a desirable action. Thus understood, we argue that such challenges should not dismiss the use of black box algorithms altogether but should inform the way in which these algorithms are designed and implemented. When physicians are trained to acquire the necessary skills and expertise, and collaborate with medical informatics and data scientists, black box algorithms can contribute to improving medical care. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Would transparency contribute to restoring accountability for such systems as is often maintained? Several objections to full transparency are examined: the loss of privacy when datasets become public, the perverse effects of disclosure of the very algorithms themselves, the potential loss of companies’ competitive edge, and the limited gains in answerability to be expected since sophisticated algorithms usually are (...) inherently opaque. It is concluded that, at least presently, full transparency for oversight bodies alone is the only feasible option; extending it to the public at large is normally not advisable. Moreover, it is argued that algorithmic decisions preferably should become more understandable; to that effect, the models of machine learning to be employed should either be interpreted ex post or be interpretable by design ex ante. (shrink)

The paper presents a framework to realise “meaningful human control” over Automated Driving Systems. The framework is based on an original synthesis of the results of the multidisciplinary research project “Meaningful Human Control over Automated Driving Systems” lead by a team of engineers, philosophers, and psychologists at Delft University of the Technology from 2017 to 2021. Meaningful human control aims at protecting safety and reducing responsibility gaps. The framework is based on the core assumption that human persons and institutions, not (...) hardware and software and their algorithms, should remain ultimately—though not necessarily directly—in control of, and thus morally responsible for, the potentially dangerous operation of driving in mixed traffic. We propose an Automated Driving System to be under meaningful human control if it behaves according to the relevant reasons of the relevant human actors (tracking), and that any potentially dangerous event can be related to a human actor (tracing). We operationalise the requirements for meaningful human control through multidisciplinary work in philosophy, behavioural psychology and traffic engineering. The tracking condition is operationalised via a proximal scale of reasons and the tracing condition via an evaluation cascade table. We review the implications and requirements for the behaviour and skills of human actors, in particular related to supervisory control and driver education. We show how the evaluation cascade table can be applied in concrete engineering use cases in combination with the definition of core components to expose deficiencies in traceability, thereby avoiding so-called responsibility gaps. Future research directions are proposed to expand the philosophical framework and use cases, supervisory control and driver education, real-world pilots and institutional embedding. (shrink)

Decision-making assisted by algorithms developed by machine learning is increasingly determining our lives. Unfortunately, full opacity about the process is the norm. Would transparency contribute to restoring accountability for such systems as is often maintained? Several objections to full transparency are examined: the loss of privacy when datasets become public, the perverse effects of disclosure of the very algorithms themselves, the potential loss of companies’ competitive edge, and the limited gains in answerability to be expected since sophisticated algorithms usually are (...) inherently opaque. It is concluded that, at least presently, full transparency for oversight bodies alone is the only feasible option; extending it to the public at large is normally not advisable. Moreover, it is argued that algorithmic decisions preferably should become more understandable; to that effect, the models of machine learning to be employed should either be interpreted ex post or be interpretable by design ex ante. (shrink)

Social relationships are essential for humans; neglecting our social needs can reduce wellbeing or even lead to the development of more severe issues such as depression or substance dependency. Although essential, some individuals face major challenges in forming and maintaining social relationships due to the experience of social anxiety. The burden of social anxiety can be reduced through accessible assessment that leads to treatment. However, socially anxious individuals who seek help face many barriers stemming from geography, fear, or disparities in (...) access to systems of care. But recent research suggested digital behavioral markers as a way to deliver cheap and easily accessible digital assessment for social anxiety: As earlier work shows, players with social anxiety show similar behaviors in virtual worlds as in the physical world, including tending to walk farther around other avatars and standing farther away from other avatars. The characteristics of the movement behavior in-game can be harnessed for the development of digital behavioral markers for the assessment of social anxiety. In this paper, we investigate whether implicit as well as explicit digital behavioral markers, proposed by prior work, for social anxiety can be used for predicting the level of social anxiety. We show that both, explicit and implicit digital behavioral markers can be harnessed for the assessment. Our findings provide further insights about how game-based digital behavioral markers can be used for the assessment of social anxiety. (shrink)

In this paper, we examine the qualitative moral impact of machine learning-based clinical decision support systems in the process of medical diagnosis. To date, discussions about machine learning in this context have focused on problems that can be measured and assessed quantitatively, such as by estimating the extent of potential harm or calculating incurred risks. We maintain that such discussions neglect the qualitative moral impact of these technologies. Drawing on the philosophical approaches of technomoral change and technological mediation theory, which (...) explore the interplay between technologies and morality, we present an analysis of concerns related to the adoption of machine learning-aided medical diagnosis. We analyze anticipated moral issues that machine learning systems pose for different stakeholders, such as bias and opacity in the way that models are trained to produce diagnoses, changes to how health care providers, patients, and developers understand their roles and professions, and challenges to existing forms of medical legislation. Albeit preliminary in nature, the insights offered by the technomoral change and the technological mediation approaches expand and enrich the current discussion about machine learning in diagnostic practices, bringing distinct and currently underexplored areas of concern to the forefront. These insights can contribute to a more encompassing and better informed decision-making process when adapting machine learning techniques to medical diagnosis, while acknowledging the interests of multiple stakeholders and the active role that technologies play in generating, perpetuating, and modifying ethical concerns in health care. (shrink)

In recent years, the ethical impact of AI has been increasingly scrutinized, with public scandals emerging over biased outcomes, lack of transparency, and the misuse of data. This has led to a growing mistrust of AI and increased calls for mandated ethical audits of algorithms. Current proposals for ethical assessment of algorithms are either too high level to be put into practice without further guidance, or they focus on very specific and technical notions of fairness or transparency that do not (...) consider multiple stakeholders or the broader social context. In this article, we present an auditing framework to guide the ethical assessment of an algorithm. The audit instrument itself is comprised of three elements: a list of possible interests of stakeholders affected by the algorithm, an assessment of metrics that describe key ethically salient features of the algorithm, and a relevancy matrix that connects the assessed metrics to stakeholder interests. The proposed audit instrument yields an ethical evaluation of an algorithm that could be used by regulators and others interested in doing due diligence, while paying careful attention to the complex societal context within which the algorithm is deployed. (shrink)

Personal AI assistants are now nearly ubiquitous. Every leading smartphone operating system comes with a personal AI assistant that promises to help you with basic cognitive tasks: searching, planning, messaging, scheduling and so on. Usage of such devices is effectively a form of algorithmic outsourcing: getting a smart algorithm to do something on your behalf. Many have expressed concerns about this algorithmic outsourcing. They claim that it is dehumanising, leads to cognitive degeneration, and robs us of our freedom and autonomy. (...) Some people have a more subtle view, arguing that it is problematic in those cases where its use may degrade important interpersonal virtues. In this article, I assess these objections to the use of AI assistants. I will argue that the ethics of their use is complex. There are no quick fixes or knockdown objections to the practice, but there are some legitimate concerns. By carefully analysing and evaluating the objections that have been lodged to date, we can begin to articulate an ethics of personal AI use that navigates those concerns. In the process, we can locate some paradoxes in our thinking about outsourcing and technological dependence, and we can think more clearly about what it means to live a good life in the age of smart machines. (shrink)

This chapter will explore the dynamics of power underpinning ethical issues within smart cities via a new paradigm derived from Systems Theory. The smart city is an expression of technology as a socio-technical system. The vision of the smart city contains a deep fusion of many different technical systems into a single integrated “ambient intelligence”. ETICA Project, 2010, p. 102). Citizens of the smart city will not experience a succession of different technologies, but a single intelligent and responsive environment through (...) which they move. Analysis of such an environment requires a framework which transcends traditional ontologically-based models in order to accommodate this deep fusion. This chapter will outline a framework based on Latour’s Actor-Network Theory and Luhmann’s treatment of society as an autopoetic system. We shall use this framework to map the influence of relevant factors on ethical issues, irrespective of their composition or type. For example, under this treatment, both human praxis and technical design can be viewed as comparable tools of domination. This chapter will provide a framework for the analysis of relations between any elements of the smart city, ranging from top-level urban management processes down to individual device operations. While we will illustrate the use of this schema through examination of ethical issues arising from power dynamics within the smart city, it is intended that this example will demonstrate the wider utility of the model in general. (shrink)

This paper clarifies why bias cannot be completely mitigated in Machine Learning (ML) and proposes an end-to-end methodology to translate the ethical principle of justice and fairness into the practice of ML development as an ongoing agreement with stakeholders. The pro-ethical iterative process presented in the paper aims to challenge asymmetric power dynamics in the fairness decision making within ML design and support ML development teams to identify, mitigate and monitor bias at each step of ML systems development. The process (...) also provides guidance on how to explain the always imperfect trade-offs in terms of bias to users. (shrink)

Autonomous mechanisms have been proposed to regulate certain aspects of society and are already being used to regulate business organisations. We take seriously recent proposals for algorithmic regulation of society, and we identify the existing technologies that can be used to implement them, most of them originally introduced in business contexts. We build on the notion of ‘social machine’ and we connect it to various ongoing trends and ideas, including crowdsourced task-work, social compiler, mechanism design, reputation management systems, and social (...) scoring. After showing how all the building blocks of algorithmic regulation are already well in place, we discuss the possible implications for human autonomy and social order. The main contribution of this paper is to identify convergent social and technical trends that are leading towards social regulation by algorithms, and to discuss the possible social, political, and ethical consequences of taking this path. (shrink)

This paper discusses the problem of responsibility attribution raised by the use of artificial intelligence technologies. It is assumed that only humans can be responsible agents; yet this alone already raises many issues, which are discussed starting from two Aristotelian conditions for responsibility. Next to the well-known problem of many hands, the issue of “many things” is identified and the temporal dimension is emphasized when it comes to the control condition. Special attention is given to the epistemic condition, which draws (...) attention to the issues of transparency and explainability. In contrast to standard discussions, however, it is then argued that this knowledge problem regarding agents of responsibility is linked to the other side of the responsibility relation: the addressees or “patients” of responsibility, who may demand reasons for actions and decisions made by using AI. Inspired by a relational approach, responsibility as answerability thus offers an important additional, if not primary, justification for explainability based, not on agency, but on patiency. (shrink)

There is a long history of the science of intelligent machines and its potential to provide scientific insights have been debated since the dawn of AI. In particular, there is renewed interest in the role of AI in research and research policy as an enabler of new methods, processes, management and evaluation which is still relatively under-explored. This empirical paper explores interviews with leading scholars on the potential impact of AI on research practice and culture through deductive, thematic analysis to (...) show the issues affecting academics and universities today. Our interviewees identify positive and negative consequences for research and researchers with respect to _collective_ and _individual use_. AI is perceived as helpful with respect to information gathering and other narrow tasks, and in support of impact and interdisciplinarity. However, using AI as a way of ‘speeding up—to keep up’ with bureaucratic and metricised processes, may proliferate negative aspects of academic culture in that the expansion of AI in research should assist and not replace human creativity. Research into the future role of AI in the research process needs to go further to address these challenges, and ask fundamental questions about how AI might assist in providing new tools able to question the values and principles driving institutions and research processes. We argue that to do this an explicit movement of meta-research on the role of AI in research should consider the effects for research and researcher creativity. Anticipatory approaches and engagement of diverse and critical voices at policy level and across disciplines should also be considered. (shrink)

The paper deals with the difference between who and what we are in order to take an ethical perspective on algorithms and their regulation. The present casting of ourselves as homo digitalis implies the possibility of projecting who we are as social beings sharing a world, into the digital medium, thereby engendering what can be called digital whoness, or a digital reification of ourselves. A main ethical challenge for the evolving digital age consists in unveiling this ethical difference, particularly when (...) dealing with algorithms and their regulation in the context of human relationships. The paper addresses by way of example some issues raised by autonomous cars. (shrink)

Traditional medical practices and relationships are changing given the widespread adoption of AI-driven technologies across the various domains of health and healthcare. In many cases, these new technologies are not specific to the field of healthcare. Still, they are existent, ubiquitous, and commercially available systems upskilled to integrate these novel care practices. Given the widespread adoption, coupled with the dramatic changes in practices, new ethical and social issues emerge due to how these systems nudge users into making decisions and changing (...) behaviours. This article discusses how these AI-driven systems pose particular ethical challenges with regards to nudging. To confront these issues, the value sensitive design approach is adopted as a principled methodology that designers can adopt to design these systems to avoid harming and contribute to the social good. The AI for Social Good factors are adopted as the norms constraining maleficence. In contrast, higher-order values specific to AI, such as those from the EU High-Level Expert Group on AI and the United Nations Sustainable Development Goals, are adopted as the values to be promoted as much as possible in design. The use case of Amazon Alexa's Healthcare Skills is used to illustrate this design approach. It provides an exemplar of how designers and engineers can begin to orientate their design programs of these technologies towards the social good. (shrink)

Two definitions of algorithm, their uses, and their implied models of computing in society, are reviewed. The first, termed the structural programming definition, aligns more with usage in computer science, and as the name suggests, the intellectual project of structured programming. The second, termed the systemic definition, is more informal and emerges from ethnographic observations of discussions of software in both professional and everyday settings. Specific examples of locating algorithms within modern codebases are shared, as well as code directly impacting (...) social and ethical concerns. The structural distinction between algorithms and social concerns is explained as mirroring the engineering construct of algorithms and data structures. It is proposed that, rather than this separation being an attempt to enforce a professional boundary and evade social responsibility, it is a crucial technical distinction within code which makes it clearer and more transparent. The power structures reinforced by the broader, cultural interpretations of algorithm are reconsidered, along with what it would mean for software to have an inclusive design culture. (shrink)

The potential use, relevance, and application of AI and other technologies in the democratic process may be obvious to some. However, technological innovation and, even, its consideration may face an intuitive push-back in the form of algorithm aversion (Dietvorst et al. J Exp Psychol 144(1):114–126, 2015). In this paper, I confront this intuition and suggest that a more ‘extreme’ form of technological change in the democratic process does not necessarily result in a worse outcome in terms of the fundamental concepts (...) of democracy and the Rule of Law. To provoke further consideration and illustrate that initial intuitions regarding democratic innovation may not always be accurate, I pose and explore four ways that AI and other forms of technology could be used to augment the representative democratic process. The augmentations range from voting online to the wholesale replacement of the legislature’s human representatives with algorithms. After first noting the intuition that less invasive forms of augmented democracy may be less objectionable than more extreme forms, I go on to critically assess whether the augmentation of existing systems satisfies or enhances ideas associated with democracy and the Rule of Law (provided by Dahl and Fuller). By imagining a (not too far-fetched) future in a (not too far-removed) democratic society, my conclusion is that, when it comes to democracy and the Rule of Law, intuitions regarding technology may lead us astray. (shrink)

Responsible innovation in artificial intelligence calls for public deliberation: well-informed “deep democratic” debate that involves actors from the public, private, and civil society sectors in joint efforts to critically address the goals and means of AI. Adopting such an approach constitutes a challenge, however, due to the opacity of AI and strong knowledge boundaries between experts and citizens. This undermines trust in AI and undercuts key conditions for deliberation. We approach this challenge as a problem of situating the knowledge of (...) actors from the AI industry within a deliberative system. We develop a new framework of responsibilities for AI innovation as well as a deliberative governance approach for enacting these responsibilities. In elucidating this approach, we show how actors from the AI industry can most effectively engage with experts and nonexperts in different social venues to facilitate well-informed judgments on opaque AI systems and thus effectuate their democratic governance. (shrink)

While organizations today make extensive use of complex algorithms, the notion of algorithmic accountability remains an elusive ideal due to the opacity and fluidity of algorithms. In this article, we develop a framework for managing algorithmic accountability that highlights three interrelated dimensions: reputational concerns, engagement strategies, and discourse principles. The framework clarifies that accountability processes for algorithms are driven by reputational concerns about the epistemic setup, opacity, and outcomes of algorithms; that the way in which organizations practically engage with emergent (...) expectations about algorithms may be manipulative, adaptive, or moral; and that when accountability relationships are heavily burdened by the opacity and fluidity of complex algorithmic systems, the emphasis of engagement should shift to a rational communication process through which a continuous and tentative assessment of the development, workings, and consequences of algorithms can be achieved over time. The degree to which such engagement is, in fact, rational can be assessed based on four discourse-ethical principles of participation, comprehension, multivocality, and responsiveness. We conclude that the framework may help organizations and their environments to jointly work toward greater accountability for complex algorithms. It may further help organizations in reputational positioning surrounding accountability issues. The discourse-ethical principles introduced in this article are meant to elevate these positioning contests to extend beyond mere adaption or compliance and help guide organizations to find moral and forward-looking solutions to accountability issues. (shrink)

Digital hyperconnectivity is a defining fact of our time. In addition to recasting social interaction, culture, economics, and politics, it has profoundly transformed the self. It has created new ways of being and constructing a self, but also new ways of being constructed as a self from the outside, new ways of being configured, represented, and governed as a self by sociotechnical systems. Rather than analyze theories of the self, I focus on practices of the self, using this expression in (...) a looser, more general sense than that used by Foucault. I begin by considering and reformulating two early lines of argument about the web as a medium for exploring and emancipating the self. Subsequent sections show how digital hyperconnectivity has engendered new ways of objectifying, quantifying, producing, and regulating the self—considered both as active, reflexive practices and as systemic, data- and algorithm-driven processes. I conclude by reflecting on the broader implications of contemporary modes of governing the self and by underscoring the ways in which hyperconnectivity has colonized the territories of the self, conscripting the self into the service of techno-social systems. (shrink)

This paper argues that data-driven medicine gives rise to a particular normative challenge. Against the backdrop of a distinction between the good and the right, harnessing personal health data towards the development and refinement of data-driven medicine is to be welcomed from the perspective of the good. Enacting solidarity drives progress in research and clinical practice. At the same time, such acts of sharing could—especially considering current developments in big data and artificial intelligence—compromise the right by leading to injustices and (...) affecting concrete modes of individual self-determination. In order to address this potential tension, two key elements for ethical reflection on data-driven medicine are proposed: the controllability of information flows, including technical infrastructures that are conducive towards controllability, and a paradigm shift towards output-orientation in governance and policy. (shrink)

Advanced AI systems are rapidly making their way into medical research and practice, and, arguably, it is only a matter of time before they will surpass human practitioners in terms of accuracy, reliability, and knowledge. If this is true, practitioners will have a prima facie epistemic and professional obligation to align their medical verdicts with those of advanced AI systems. However, in light of their complexity, these AI systems will often function as black boxes: the details of their contents, calculations, (...) and procedures cannot be meaningfully understood by human practitioners. When AI systems reach this level of complexity, we can also speak of black-box medicine. In this paper, we want to argue that black-box medicine conflicts with core ideals of patient-centered medicine. In particular, we claim, black-box medicine is not conducive for supporting informed decision-making based on shared information, shared deliberation, and shared mind between practitioner and patient. (shrink)

Since the inception of recorded music there has been a need for standards and reliability across sound formats and listening environments. The role of the audio mastering engineer is prestigious and akin to a craft expert combining scientific knowledge, musical learning, manual precision and skill, and an awareness of cultural fashions and creative labour. With the advent of algorithms, big data and machine learning, loosely termed artificial intelligence in this creative sector, there is now the possibility of automating human audio (...) mastering processes and radically disrupting mastering careers. The emergence of dedicated products and services in artificial intelligence-driven audio mastering poses profound questions for the future of the music industry, already having faced significant challenges due to the digitalization of music over the past decades. The research reports on qualitative and ethnographic inquiry with audio mastering engineers on the automation of their expertise and the potential for artificial intelligence to augment or replace aspects of their workflows. Investigating audio mastering engineers' awareness of artificial intelligence, the research probes the importance of criticality in their labour. The research identifies intuitive performance and critical listening as areas where human ingenuity and communication pose problems for simulation. Affective labour disrupts speculation of algorithmic domination by highlighting the pragmatic strategies available for humans to adapt and augment digital technologies. (shrink)

As a relational concept, responsible innovation can be made more tangible by asking innovation of what and responsibility of whom for what? Arranging the scattered field of responsible innovation comprehensively, starting from an anthropological point of view, into five fields of tension and five categories of spearheads, may be theoretically and practically helpful while offering suggestions for both research and management.

In this article, I address the issue of evil and roboethics in the context of management studies and suggest that management scholars should locate evil in the realm of the human rather than of the artificial. After discussing the possibility of addressing the reality of evil machines in ontological terms, I explore users’ reaction to robots in a social context. I conclude that the issue of evil machines in management is more precisely a case of technology anthropomorphization.

The rapid and dynamic nature of digital transformation challenges companies that wish to develop and deploy novel digital technologies. Like other actors faced with this transformation, companies need to find robust ways to ethically guide their innovations and business decisions. Digital ethics has recently featured in a plethora of both practical corporate guidelines and compilations of high-level principles, but there remains a gap concerning the development of sound ethical guidance in specific business contexts. As a multinational science and technology company (...) faced with a broad range of digital ventures and associated ethical challenges, Merck KGaA has laid the foundations for bridging this gap by developing a Code of Digital Ethics (CoDE) tailored for this context. Following a comprehensive analysis of existing digital ethics guidelines, we used a reconstructive social research approach to identify 20 relevant principles and derive a code designed as a multi-purpose tool. Versatility was prioritised by defining non-prescriptive guidelines that are open to different perspectives and thus well-suited for operationalisation for varied business purposes. We also chose a clear nested structure that highlights the relationships between five core and fifteen subsidiary principles as well as the different levels of reference—data and algorithmic systems—to which they apply. The CoDE will serve Merck KGaA and its new Digital Ethics Advisory Panel to guide ethical reflection, evaluation and decision-making across the full spectrum of digital developments encountered and undertaken by the company whilst also offering an opportunity to increase transparency for external partners, and thus trust. (shrink)

We argue that explainable artificial intelligence (XAI), specifically reason-giving XAI, often constitutes the most suitable way of ensuring that someone can properly be held responsible for decisions that are based on the outputs of artificial intelligent (AI) systems. We first show that, to close moral responsibility gaps (Matthias 2004), often a human in the loop is needed who is directly responsible for particular AI-supported decisions. Second, we appeal to the epistemic condition on moral responsibility to argue that, in order to (...) be responsible for her decision, the human in the loop has to have an explanation available of the system’s recommendation. Reason explanations are especially well-suited to this end and we examine whether – and how – it might be possible to make such explanations fit with AI systems. We support our claims by focusing on a case of disagreement between human in the loop and AI system. (shrink)

There is a deluge of AI-assisted decision-making systems, where our data serve as proxy to our actions, suggested by AI. The closer we investigate our data (raw input, or their learned representations, or the suggested actions), we begin to discover “bugs”. Outside of their test, controlled environments, AI systems may encounter situations investigated primarily by those in other disciplines, but experts in those fields are typically excluded from the design process and are only invited to attest to the ethical features (...) of the resulting system or to comment on demonstrations of intelligence and aspects of craftmanship after the fact. This communicative impasse must be overcome. Our idea is that philosophical and engineering considerations interact and can be fruitfully combined in the AI design process from the very beginning. We embody this idea in the role of a philosopher engineer. We discuss the role of philosopher engineers in the three main design stages of an AI system: deployment management (what is the system’s intended use, in what environment?); objective setting (what should the system be trained to do, and how?); and training (what model should be used, and why?). We then exemplify the need for philosopher engineers with an illustrative example, investigating how the future decisions of an AI-based hiring system can be fairer than those contained in the biased input data on which it is trained; and we briefly sketch the kind of interdisciplinary education that we envision will help to bring about better AI. (shrink)

Critical algorithm scholarship has demonstrated the difficulties of attributing accountability for the actions and effects of algorithmic systems. In this commentary, we argue that we cannot stop at denouncing the lack of accountability for algorithms and their effects but must engage the broader systems and distributed agencies that algorithmic systems exist within; including standards, regulations, technologies, and social relations. To this end, we explore accountability in “the Generated Detective,” an algorithmically generated comic. Taking up the mantle of detectives ourselves, we (...) investigate accountability in relation to this piece of experimental fiction. We problematize efforts to effect accountability through transparency by undertaking a simple operation: asking for permission to re-publish a set of the algorithmically selected and modified words and images which make the frames of the comic. Recounting this process, we demonstrate slippage between the “complication” of the algorithm and the obscurity of the legal and institutional structures in which it exists. (shrink)