The use of machine learning algorithms has become ubiquitous in hiring decisions. Recent studies have shown that many of these algorithms generate unlawful discriminatory effects in every step of the process. The training phase of the machine learning models used in these decisions has been identified as the main source of bias. For a long time, discrimination cases have been analyzed under the banner of disparate treatment and disparate impact, but these concepts have been shown to be ineffective in (...) the context of AI. This paper examines the possibility of studying algorithmic discrimination from the perspective of negligence law. Negligent selection and validation of datasets and classifiers is a salient cause of improper training in machine learning. It is a breach of an employer’s duty to protect others, a failure of its exercise of due care in the manner of choosing employees. Adopting negligence to algorithmic discrimination requires a reformulation of what is foreseeable and reasonable within the context of black box algorithms, and a shift in the burden of proof to the employer, who must present evidence of its efforts to avoid bias. (shrink)

Algorithmic discrimination has rapidly become a topic of intense public and academic interest. This article explores three issues raised by algorithmic discrimination: 1) the distinction between direct and indirect discrimination, 2) the notion of disadvantageous treatment, and 3) the moral badness of discriminatory automated decision-making. It argues that some conventional distinctions between direct and indirect discrimination appear not to apply to algorithmic discrimination, that algorithmic discrimination may often be discrimination (...) between groups, as opposed to against groups, and that it is not necessarily the case that morally bad algorithmic discrimination gives us reason to not use automated decision-making. For each of the three issues, the article explores implications for algorithmic discrimination, suggests some alternative answers, and clarifies how we may want to think of discrimination more broadly in light of lessons drawn from the context of algorithmic discrimination. (shrink)

Over the past decade, scholars, institutions, and activists have voiced strong concerns about the potential of automated decision systems to indirectly discriminate against vulnerable groups. This article analyses the ethics of algorithmic indirect discrimination, and argues that we can explain what is morally bad about such discrimination by reference to the fact that it causes harm. The article first sketches certain elements of the technical and conceptual background, including definitions of direct and indirect algorithmic differential treatment. (...) It next introduces three prominent accounts of fairness as potential explanations if the badness of algorithmic indirect discrimination, but argues that all three are vulnerable to powerful levelling-down-style objections. Instead, the article demonstrates how proper attention to the way differences in decision-scenarios affect the distribution of harms can help us account for intuitions in prominent cases. Finally, the article considers a potential objection based on the fact that certain forms of algorithmic indirect discrimination appear to distribute rather than cause harm, and notes that we can explain how such distributions cause harm by attending to differences in individual and group vulnerability. (shrink)

As algorithms have become ubiquitous in consequential domains, societal concerns about the potential for discriminatory outcomes have prompted urgent calls to address algorithmic bias. In response, a rich literature across computer science, law, and ethics is rapidly proliferating to advance approaches to designing fair algorithms. Yet computer scientists, legal scholars, and ethicists are often not speaking the same language when using the term ‘bias.’ Debates concerning whether society can or should tackle the problem of algorithmic bias are hampered (...) by conflations of various understandings of bias, ranging from neutral deviations from a standard to morally problematic instances of injustice due to prejudice, discrimination, and disparate treatment. This terminological confusion impedes efforts to address clear cases of discrimination. -/- In this paper, we examine the promises and challenges of different approaches to disambiguating bias and designing for justice. While both approaches aid in understanding and addressing clear algorithmic harms, we argue that they also risk being leveraged in ways that ultimately deflect accountability from those building and deploying these systems. Applying this analysis to recent examples of generative AI, our argument highlights unseen dangers in current methods of evaluating algorithmic bias and points to ways to redirect approaches to addressing bias in generative AI at its early stages in ways that can more robustly meet the demands of justice. (shrink)

Implementing an algorithm on part of our causally-interconnected physical environment requires three choices that are typically considered arbitrary, i.e. no single option is innately privileged without invoking an external observer perspective. First, how to delineate one set of local causal relationships from the environment. Second, within this delineation, which inputs and outputs to designate for attention. Third, what meaning to assign to particular states of the designated inputs and outputs. Having explained these types of arbitrariness, we assess their relevance for (...) algorithms from various computational theories of mind (CTM) seeking to account for phenomenal consciousness. Some CTM accounts can survive some of these arbitrariness challenges, but eventually each one examined faces a trilemma. Each is under pressure to dilute at least one of three important CTM desiderata: its causal relevance, its substrate neutrality, or its assignment discrimination (e.g. ability to avoid near-panpsychism). However, unlike previous research, we suggest the debate cannot be fully settled at the level of principle. Certain complex CTM algorithms may be salvageable, albeit not the algorithms specified to date. The paper specifies three areas of investigative work, as well as further theoretical avenues of research, which computationalists may wish to explore to escape the trilemma. (shrink)

Some artificial intelligence systems can display algorithmic bias, i.e. they may produce outputs that unfairly discriminate against people based on their social identity. Much research on this topic focuses on algorithmic bias that disadvantages people based on their gender or racial identity. The related ethical problems are significant and well known. Algorithmic bias against other aspects of people’s social identity, for instance, their political orientation, remains largely unexplored. This paper argues that algorithmic bias against people’s political (...) orientation can arise in some of the same ways in which algorithmic gender and racial biases emerge. However, it differs importantly from them because there are strong social norms against gender and racial biases. This does not hold to the same extent for political biases. Political biases can thus more powerfully influence people, which increases the chances that these biases become embedded in algorithms and makes algorithmic political biases harder to detect and eradicate than gender and racial biases even though they all can produce similar harm. Since some algorithms can now also easily identify people’s political orientations against their will, these problems are exacerbated. Algorithmic political bias thus raises substantial and distinctive risks that the AI community should be aware of and examine. (shrink)

In this position paper, we introduce a new epistemic lens for analyzing algorithmic harm. We argue that the epistemic lens we propose herein has two key contributions to help reframe and address some of the assumptions underlying inquiries into algorithmic fairness. First, we argue that using the framework of epistemic injustice helps to identify the root causes of harms currently framed as instances of representational harm. We suggest that the epistemic lens offers a theoretical foundation for expanding approaches (...) to algorithmic fairness in order to address a wider range of harms not recognized by existing technical or legal definitions. Second, we argue that the epistemic lens helps to identify the epistemic goals of inquiries into algorithmic fairness. There are two distinct contexts within which we examine algorithmic harm: at times, we seek to understand and describe the world as it is, and, at other times, we seek to build a more just future. The epistemic lens can serve to direct our attention to the epistemic frameworks that shape our interpretations of the world as it is and the ways we envision possible futures. Clarity with respect to which epistemic context is relevant in a given inquiry can further help inform choices among the different ways of measuring and addressing algorithmic harms. We introduce this framework with the goal of initiating new research directions bridging philosophical, legal, and technical approaches to understanding and mitigating algorithmic harms. (shrink)

There is growing concern that decision-making informed by machine learning (ML) algorithms may unfairly discriminate based on personal demographic attributes, such as race and gender. Scholars have responded by introducing numerous mathematical definitions of fairness to test the algorithm, many of which are in conflict with one another. However, these reductionist representations of fairness often bear little resemblance to real-life fairness considerations, which in practice are highly contextual. Moreover, fairness metrics tend to be implemented in narrow and targeted toolkits that (...) are difficult to integrate into an algorithm’s broader ethical assessment. In this paper, we derive lessons from ethical philosophy and welfare economics as they relate to the contextual factors relevant for fairness. In particular we highlight the debate around acceptability of particular inequalities and the inextricable links between fairness, welfare and autonomy. We propose Key Ethics Indicators (KEIs) as a way towards providing a more holistic understanding of whether or not an algorithm is aligned to the decision-maker’s ethical values. (shrink)

Over the years, companies have adopted hiring algorithms because they promise wider job candidate pools, lower recruitment costs and less human bias. Despite these promises, they also bring perils. Using them can inflict unintentional harms on individual human rights. These include the five human rights to work, equality and nondiscrimination, privacy, free expression and free association. Despite the human rights harms of hiring algorithms, the AI ethics literature has predominantly focused on abstract ethical principles. This is problematic for two reasons. (...) First, AI principles have been criticized for being vague and not actionable. Second, the use of vague ethical principles to discuss algorithmic risks does not provide any accountability. This lack of accountability creates an algorithmic accountability gap. Closing this gap is crucial because, without accountability, the use of hiring algorithms can lead to discrimination and unequal access to employment opportunities. This paper makes two contributions to the AI ethics literature. First, it frames the ethical risks of hiring algorithms using international human rights law as a universal standard for determining algorithmic accountability. Second, it evaluates four types of algorithmic impact assessments in terms of how effectively they address the five human rights of job applicants implicated in hiring algorithms. It determines which of the assessments can help companies audit their hiring algorithms and close the algorithmic accountability gap. (shrink)

If whatever counts as merit for some purpose is unevenly distributed, a decision procedure that accurately sorts people on that basis will “pick up” and reproduce the pre-existing pattern in ways that more random, less merit-tracking procedures would not. This dynamic is an important cause for concern about the use of predictive models to allocate goods and opportunities. In this article, I distinguish two different objections that give voice to that concern in different ways. First, decision procedures may contribute to (...) future social injustice and other social ills by sustaining or aggravating patterns that undermine equality of status and opportunity. Second, the same decision procedures may wrong particular individuals by compounding prior injustices that explain those persons’ predicted or actual characteristics. I argue for the importance of the first idea and raise doubts about the second. In normative assessments and legal regulation of algorithmic decisionmaking, as in our thinking about anti-discrimination norms more broadly, a central concern ought to be the prospect of entrenching harmful and unjust patterns—quite apart from any personal wrong done to the individuals about whom predictions are made. (shrink)

Abstract: Heart diseases are increasing daily at a rapid rate and it is alarming and vital to predict heart diseases early. The diagnosis of heart diseases is a challenging task i.e. it must be done accurately and proficiently. The aim of this study is to determine which patient is more likely to have heart disease based on a number of medical features. We organized a heart disease prediction model to identify whether the person is likely to be diagnosed with a (...) heart disease or not using the medical features of the person. We used many different algorithms of machine learning such as Gaussian Mixture, Nearest Centroid, MultinomialNB, Logistic RegressionCV, Linear SVC, Linear Discriminant Analysis, SGD Classifier, Extra Tree Classifier, Calibrated ClassifierCV, Quadratic Discriminant Analysis, GaussianNB, Random Forest Classifier, ComplementNB, MLP Classifier, BernoulliNB, Bagging Classifier, LGBM Classifier, Ada Boost Classifier, K Neighbors Classifier, Logistic Regression, Gradient Boosting Classifier, Decision Tree Classifier, and Deep Learning to predict and classify the patient with heart disease. A quite helpful approach was used to regulate how the model can be used to improve the accuracy of prediction of heart diseases in any person. The strength of the proposed model was very satisfying and was able to predict evidence of having a heart disease in a particular person by using Deep Learning and Random Forest Classifier which showed a good accuracy in comparison to the other used classifiers. The proposed heart disease prediction model will enhances medical care and reduces the cost. This study gives us significant knowledge that can help us predict the person with heart disease. The dataset was collected from Kaggle depository and the model is implemented using python. (shrink)

El uso de sistemas de machine learning en los procesos de selección laboral ha sido de gran utilidad para agilizarlos y volverlos más eficientes, pero al mismo tiempo ha generado problemas en términos de equidad, confiabilidad y transparencia. En este artículo comenzamos explicando los diferentes usos de la Inteligencia Artificial en los procesos de selección laboral en Estados Unidos. Presentamos los sesgos sexuales y raciales que han sido detectados en algunos de ellos y explicamos los obstáculos jurídicos y prácticos para (...) su detección y análisis. Un posible remedio jurídico a la discriminación algorítmica es el régimen de impacto diferencial sistémico utilizado en Estados Unidos. Usamos algunas características de este régimen para señalar los vacíos jurídicos frente a la discriminación en el acceso al empleo en el ámbito del derecho laboral colombiano, y ofrecemos algunas características generales que podría tener un régimen de esa naturaleza. (shrink)

The gig economy is a phenomenon that is rapidly expanding, redefining the nature of work and contributing to a significant change in how contemporary economies are organised. Its expansion is not unproblematic. This article provides a clear and systematic analysis of the main ethical challenges caused by the gig economy. Following a brief overview of the gig economy, its scope and scale, we map the key ethical problems that it gives rise to, as they are discussed in the relevant literature. (...) We map them onto three categories: the new organisation of work (what is done), the new nature of work (how it is done), and the new status of workers (who does it). We then evaluate a recent initiative from the EU that seeks to address the challenges of the gig economy. The 2019 report of the European High-Level Expert Group on the Impact of the Digital Transformation on EU Labour Markets is a positive step in the right direction. However, we argue that ethical concerns relating to algorithmic systems as mechanisms of control, and the discrimination, exclusion and disconnectedness faced by gig workers require further deliberation and policy response. A brief conclusion completes the analysis. The appendix presents the methodology underpinning our literature review. (shrink)

In order to fight massive vandalism the English- language Wikipedia has developed a system of surveillance which is carried out by humans and bots, supported by various tools. Central to the selection of edits for inspection is the process of using filters or profiles. Can this profiling be justified? On the basis of a careful reading of Frederick Schauer’s books about rules in general (1991) and profiling in particular (2003) I arrive at several conclusions. The effectiveness, efficiency, and risk-aversion of (...) edit selection all greatly increase as a result. The argument for increasing predictability suggests making all details of profiling manifestly public. Also, a wider distribution of the more sophisticated anti-vandalism tools seems indicated. As to the specific dimensions used in profiling, several critical remarks are developed. When patrollers use ‘assisted edit- ing’ tools, severe ‘overuse’ of several features (anonymity, warned before) is a definite possibility, undermining profile efficacy. The easy remedy suggested is to render all of them invisible on the interfaces as displayed to patrollers. Finally, concerning not only assisted editing tools but tools against vandalism generally, it is argued that the anonymity feature is a sensitive category: anons have been in dispute for a long time (while being more prone to vandalism). Targeting them as a special category violates the social contract upon which Wikipedia is based. The feature is therefore a candidate for mandatory ‘underuse’: it should be banned from all anti-vandalism filters and profiling algorithms, and no longer be visible as a special edit trait. (shrink)

To speak about revolution, either as an event or as a concept, must appear presumptuous at a moment when racial discrimination, fascist politics, and the totalitarian war against women and minorities are amplified by a market economy based on systemic division. In the digital present, the systematization of division is magnified by newly algorithmic structures of machinic capitalism. In that context, the more the intellect aims to grasp the depth of revolutionary actions, the less the latter seem to (...) make sense. In other words, both the failure and the promise of revolution have created a modern paradigm to address the political struggles of our times, one that is cyclically repeating itself to challenge the limits of humankind. And yet, one can always find reasons to interrogate the symptoms of revolutionary tragedies that proliferate against a background of intellectual numbness. (shrink)

Dwork, Hardt, Pitassi, Reingold, & Zemel [6] introduced two notions of fairness, each of which is meant to formalize the notion of similar treatment for similarly qualified individuals. The first of these notions, which we call additive metric fairness, has received much attention in subsequent work studying the fairness of a system composed of classifiers which are fair when considered in isolation [3, 4, 7, 8, 12] and in work studying the relationship between fair treatment of individuals and fair treatment (...) of groups [6, 7, 13]. Here, we extend these lines of research to the second, less-studied notion, which we call multiplicative metric fairness. In particular, we exactly characterize the fairness of conjunctions and disjunctions of multiplicative metric fair classifiers, and the extent to which a classifier which satisfies multiplicative metric fairness also treats groups fairly. This characterization reveals that whereas additive metric fairness becomes easier to satisfy when probabilities of acceptance are small, leading to unfairness under functional and group compositions, multiplicative metric fairness is better-behaved, due to its scale-invariance. (shrink)

Endre Begby’s Prejudice: A Study in Non-Ideal Epistemology engages a wide range of issues of enduring interest to epistemologists, applied ethicists, and anyone concerned with how knowledge and justice intersect. Topics include stereotypes and generics, evidence and epistemic justification, epistemic injustice, ethical-epistemic dilemmas, moral encroachment, and the relations between blame and accountability. Begby applies his views about these topics to an equally wide range of pressing social questions, such as conspiracy theories, misinformation, algorithmic bias, discrimination, and criminal justice. (...) Through it all, the book’s central thesis is that prejudices can be epistemically rational, a corrective against what Begby takes to be the received view that prejudices are always and everywhere bad. However, Begby’s arguments do not engage consistently with relevant empirical literatures, misrepresent the positions of his interlocutors, and rehearse ideas already well-established across a range of intellectual traditions. (shrink)

We present a rigorous way to evaluate the visual perception of correlation in scatterplots, based on classical psychophysical methods originally developed for simple properties such as brightness. Although scatterplots are graphically complex, the quantity they convey is relatively simple. As such, it may be possible to assess the perception of correlation in a similar way. Scatterplots were each of 5.0 extent, containing 100 points with a bivariate normal distribution. Means were 0.5 of the range of the points, and standard deviations (...) 0.2 of this range. Precision was determined via an adaptive algorithm to find the just noticeable differences (jnds) in correlation, i.e., the difference between two side-by-side scatterplots that could be discriminated 75% of the time. Accuracy was measured by direct estimation, using reference scatterplots with fixed upper and lower values, with a test scatterplot adjusted so that its correlation appeared to be halfway between these. This process was recursively applied to yield several further estimates. Results of the discrimination tests show jnd(r) = k (1/b – r), where r is the Pearson correlation, and parameters 0 < k, b < 1. Integration yields a subjective estimate of correlation g(r) = ln(1 – br) / ln(1– b). The values of b found via discrimination closely match those found via direct estimation. As such, it appears that the perception of correlation in a scatterplot is completely described by two related performance curves, specified by two easily-measured parameters. (shrink)

The notion of representation in neuroscience has largely been predicated on localizing the components of computational processes that explain cognitive function. On this view, which I call “algorithmic homuncularism,” individual, spatially and temporally distinct parts of the brain serve as vehicles for distinct contents, and the causal relationships between them implement the transformations specified by an algorithm. This view has a widespread influence in philosophy and cognitive neuroscience, and has recently been ably articulated and defended by Shea. Still, I (...) am skeptical about algorithmic homuncularism, and I argue against it by focusing on recent methods for complex data analysis in systems neuroscience. I claim that analyses such as principle components analysis and linear discriminant analysis prevent individuating vehicles as algorithmic homuncularism recommends. Rather, each individual part contributes to a global state space, trajectories of which vary with important task parameters. I argue that, while homuncularism is false, this view still supports a kind of “vehicle realism,” and I apply this view to debates about the explanatory role of representation. (shrink)

An issue about the privacy of the clustered groups designed by algorithms arises when attempts are made to access certain pieces of information about those groups that would likely be used to harm them. Therefore, limitations must be imposed regarding accessing such information about clustered groups. In the discourse on group privacy, it is argued that the right to privacy of such groups should be recognised to respect group privacy, protecting clustered groups against discrimination. According to this viewpoint, this (...) right places a duty on others, for example, private companies, institutions, and governments, to refrain from accessing such information. To defend the idea that the right to privacy should be recognised for clustered groups, at least two requirements must be satisfied. First, clustered group privacy must be conceived of as either a collective good or a participatory good. Since these forms of good are of the type from which no member of a group can be excluded from benefiting, the right to them is defined as a group right. Second, there must be group interests on which to base a group right. Group interests can be either the interests of those members that are a result of their being in the group or the interests of the group as a whole that transcend the interests of its members. However, this paper argues that clustered group privacy cannot be conceived of as either a collective or a participatory good because it is possible for some individuals to be excluded from benefiting from it. Furthermore, due to the lack of awareness among individuals that they are members of a clustered group and the nature of a clustered group itself, such groups cannot have the group interests necessary to establish a group right. Hence, the group right to privacy cannot be recognised for these groups, implying that the group right cannot be considered a means to protect clustered groups against discrimination. Instead, this paper suggests that moral principles need to be articulated within an ethics of vulnerability to identify the moral obligations of protecting vulnerable clustered groups. The duty owed to the vulnerable should involve refraining from accessing certain information about clustered groups in specific contexts. This duty is not engendered by the right to privacy of such groups; it is the duty owed to the vulnerable. The findings highlight the need to articulate moral principles regarding privacy and data protection to protect clustered groups in contexts in which accessing information about them could constitute a reason for discriminatory targeting. (shrink)

Most of the existing studies on Bourdieu and the digital regards the social and class distinctions in the use of digital technologies, thus presupposing a certain transparency of technologies themselves. Our proposal is to refer to this attitude as “Bourdieu outside the digital.” Yet in this paper, another perspective called “Bourdieu inside the digital” is developed, which moves the focus on the effects of some emerging technologies on social distinctions and discrimination. The main hypothesis is that algorithms of machine (...) learning are producers and reproducers of habitus. Although their results present a greater granularity with respect to the standard techniques of the past, these algorithms still reduce individuals to categories, general trends, classes, and behaviors. Such a reduction has flattening effects on the individuals’ self-understanding, especially in terms of identity and interaction with the social world. This is the phenomenon described in the article as the “personalization without personality,” whose consequences are both existential and social-political. This idea will be illustrated through qualitative and comparative analysis between the correspondence analysis and the multiple correspondence analysis used by Bourdieu in his works and some of those techniques that are performed today in big data analytics. (shrink)

From music recommendation to assessment of asylum applications, machine-learning algorithms play a fundamental role in our lives. Naturally, the rise of AI implementation strategies has brought to public attention the ethical risks involved. However, the dominant anti-discrimination discourse, too often preoccupied with identifying particular instances of harmful AIs, has yet to bring clearly into focus the more structural roots of AI-based injustice. This paper addresses the problem of AI-based injustice from a distinctively epistemic angle. More precisely, I argue that (...) the injustice generated by the implementation of AI machines in our societies is, in some paradigmatic cases, also a form of epistemic injustice. With a particular focus on AIs employed as gatekeepers of our epistemic resources, this paper shows how their epistemically conformist behaviour is responsible for the marginalisation and the ostracism of minoritarian perspectives. Because it clarifies key structural flaws and weaknesses of current AI design, this paper helps make headway in critical discussion of current AI technologies. And because it forges new theoretical tools to understand forms of epistemic oppression, this paper also contributes to the advancement of feminist theorisation. (shrink)

Purpose – In the last half-century, individual sensory neurons have been bestowed with characteristics of the whole human being, such as behavior and its oft-presumed precursor, consciousness. This anthropomorphization is pervasive in the literature. It is also absurd, given what we know about neurons, and it needs to be abolished. This study aims to first understand how it happened, and hence why it persists. Design/methodology/approach – The peer-reviewed sensory-neurophysiology literature extends to hundreds (perhaps thousands) of papers. Here, more than 90 (...) mainstream papers were scrutinized. Findings – Anthropomorphization arose because single neurons were cast as “observers” who “identify”, “categorize”, “recognize”, “distinguish” or “discriminate” the stimuli, using math-based algorithms that reduce (“decode”) the stimulus-evoked spike trains to the particular stimuli inferred to elicit them. Without “decoding”, there is supposedly no perception. However, “decoding” is both unnecessary and unconfirmed. The neuronal “observer” in fact consists of the laboratory staff and the greater society that supports them. In anthropomorphization, the neuron becomes the collective. Research limitations/implications – Anthropomorphization underlies the widespread application to neurons Information Theory and Signal Detection Theory, making both approaches incorrect. Practical implications – A great deal of time, money and effort has been wasted on anthropomorphic Reductionist approaches to understanding perception and consciousness. Those resources should be diverted into more-fruitful approaches. Originality/value – A long-overdue scrutiny of sensory-neuroscience literature reveals that anthropomorphization, a form of Reductionism that involves the presumption of single-neuron consciousness, has run amok in neuroscience. Consciousness is more likely to be an emergent property of the brain. (shrink)

Mark Coeckelbergh starts his book with a very powerful picture based on a real incident: On the 9th of January 2020, Robert Williams was wrongfully arrested by Detroit police officers in front of his two young daughters, wife and neighbors. For 18 hours the police would not disclose the grounds for his arrest (American Civil Liberties Union 2020; Hill 2020). The decision to arrest him was primarily based on a facial detection algorithm which matched Mr. Williams’ driving license photo with (...) the picture of a man who was suspected of watch theft two years earlier. Not only did the computer ‘get it wrong’ as one of the detectives said, when Mr. Williams made them aware that the picture of the suspect obviously wasn’t resembling him, the probably unreliable algorithm very likely contributed to racial discrimination (Hill 2020). It is well documented that many available facial detection algorithms at this time had significant problems (e.g. a comparably high false positive rate) with respect to black persons, like Mr. Williams (NIST 2019). Multiple causes may exist, such as unbalanced training datasets and insufficient optimization. Coeckelbergh compares the disturbing case of Mr. Williams with a political interpretation of Franz Kafka’s The Trial, where the protagonist, Josef K., is accused of an unspecified crime by an opaque, oppressive and absurd bureaucracy: “In the 21st-century United States, Josef K. is black and is falsely accused by an algorithm, without explanation” (p. 2). (shrink)

Algorithms can now identify patterns and correlations in the (big) datasets, and predict outcomes based on those identified patterns and correlations with the use of machine learning techniques and big data, decisions can then be made by algorithms themselves in accordance with the predicted outcomes. Yet, algorithms can inherit questionable values from the datasets and acquire biases in the course of (machine) learning, and automated algorithmic decision-making makes it more difficult for people to see algorithms as biased. While researchers (...) have taken the problem of algorithmic bias seriously, but the current discussion on algorithmic fairness tends to conceptualize ‘fairness’ in algorithmic fairness primarily as a technical issue and attempts to implement pre-existing ideas of ‘fairness’ into algorithms. In this paper, I show that such a view of algorithmic fairness as technical issue is unsatisfactory for the type of problem algorithmic fairness presents. Since decisions on fairness measure and the related techniques for algorithms essentially involve choices between competing values, ‘fairness’ in algorithmic fairness should be conceptualized first and foremost as a political issue, and it should be (re)solved by democratic communication. The aim of this paper, therefore, is to explicitly reconceptualize algorithmic fairness as a political question and suggest the current discussion of algorithmic fairness can be strengthened by adopting the accountability for reasonableness framework. (shrink)

Discrimination, understood as differential treatment of individuals on the basis of their respective group memberships, is widely considered to be morally wrong. This moral judgment is backed in many jurisdictions with the passage of equality of opportunity legislation, which aims to ensure that racial, ethnic, religious, sexual, sexual-orientation, disability and other groups are not subjected to discrimination. This chapter explores the conceptual underpinnings of discrimination and equality of opportunity using the tools of analytical moral and political philosophy.

I consider statistical criteria of algorithmic fairness from the perspective of the _ideals_ of fairness to which these criteria are committed. I distinguish and describe three theoretical roles such ideals might play. The usefulness of this program is illustrated by taking Base Rate Tracking and its ratio variant as a case study. I identify and compare the ideals of these two criteria, then consider them in each of the aforementioned three roles for ideals. This ideals program may present a (...) way forward in the normative evaluation of candidate statistical criteria of algorithmic fairness. (shrink)

It is well-established that algorithms can be instruments of injustice. It is less frequently discussed, however, how current modes of AI deployment often make the very discovery of injustice difficult, if not impossible. In this article, we focus on the effects of algorithmic profiling on epistemic agency. We show how algorithmic profiling can give rise to epistemic injustice through the depletion of epistemic resources that are needed to interpret and evaluate certain experiences. By doing so, we not only (...) demonstrate how the philosophical conceptual framework of epistemic injustice can help pinpoint potential, systematic harms from algorithmic profiling, but we also identify a novel source of hermeneutical injustice that to date has received little attention in the relevant literature: epistemic fragmentation. Epistemic fragmentation is a structural characteristic of algorithmically-mediated environments that isolate individuals, making it more difficult to develop, uptake and apply new epistemic resources. This, in turn, can impede the identification and conceptualisation of emerging harms in these environments. We trace the occurrence of hermeneutical injustice back to the fragmentation of the epistemic experiences of individuals, who are left more vulnerable by the inability to share, compare, and learn from shared experiences. (shrink)

Machine intelligence already helps medical staff with a number of tasks. Ethical decision-making, however, has not been handed over to computers. In this proof-of-concept study, we show how an algorithm based on Beauchamp and Childress’ prima-facie principles could be employed to advise on a range of moral dilemma situations that occur in medical institutions. We explain why we chose fuzzy cognitive maps to set up the advisory system and how we utilized machine learning to train it. We report on the (...) difficult task of operationalizing the principles of beneficence, non-maleficence and patient autonomy, and describe how we selected suitable input parameters that we extracted from a training dataset of clinical cases. The first performance results are promising, but an algorithmic approach to ethics also comes with several weaknesses and limitations. Should one really entrust the sensitive domain of clinical ethics to machine intelligence? (shrink)

The application of machine-learning technologies to medical practice promises to enhance the capabilities of healthcare professionals in the assessment, diagnosis, and treatment, of medical conditions. However, there is growing concern that algorithmic bias may perpetuate or exacerbate existing health inequalities. Hence, it matters that we make precise the different respects in which algorithmic bias can arise in medicine, and also make clear the normative relevance of these different kinds of algorithmic bias for broader questions about justice and (...) fairness in healthcare. In this paper, we provide the building blocks for an account of algorithmic bias and its normative relevance in medicine. (shrink)

In light of the recent emergence of predictive techniques in law enforcement to forecast crimes before they occur, this paper examines the temporal operation of power exercised by predictive policing algorithms. I argue that predictive policing exercises power through a paranoid style that constitutes a form of temporal governmentality. Temporality is especially pertinent to understanding what is ethically at stake in predictive policing as it is continuous with a historical racialized practice of organizing, managing, controlling, and stealing time. After first (...) clarifying the concept of temporal governmentality, I apply this lens to Chicago Police Department’s Strategic Subject List. This predictive algorithm operates, I argue, through a paranoid logic that aims to preempt future possibilities of crime on the basis of a criminal past codified in historical crime data. (shrink)

Algorithms wield increasing control over our lives—over the jobs we get, the loans we're granted, the information we see online. Algorithms can and often do wield their power in a biased way, and much work has been devoted to algorithmic bias. In contrast, algorithmic neutrality has been largely neglected. I investigate algorithmic neutrality, tackling three questions: What is algorithmic neutrality? Is it possible? And when we have it in mind, what can we learn about algorithmic (...) bias? (shrink)

In information societies, operations, decisions and choices previously left to humans are increasingly delegated to algorithms, which may advise, if not decide, about how data should be interpreted and what actions should be taken as a result. More and more often, algorithms mediate social processes, business transactions, governmental decisions, and how we perceive, understand, and interact among ourselves and with the environment. Gaps between the design and operation of algorithms and our understanding of their ethical implications can have severe consequences (...) affecting individuals as well as groups and whole societies. This paper makes three contributions to clarify the ethical importance of algorithmic mediation. It provides a prescriptive map to organise the debate. It reviews the current discussion of ethical aspects of algorithms. And it assesses the available literature in order to identify areas requiring further work to develop the ethics of algorithms. (shrink)

Law-enforcement agencies are increasingly able to leverage crime statistics to make risk predictions for particular individuals, employing a form of inference that some condemn as violating the right to be “treated as an individual.” I suggest that the right encodes agents’ entitlement to a fair distribution of the burdens and benefits of the rule of law. Rather than precluding statistical prediction, it requires that citizens be able to anticipate which variables will be used as predictors and act intentionally to avoid (...) them. Furthermore, it condemns reliance on various indexes of distributive injustice, or unchosen properties, as evidence of law-breaking. (shrink)

The stakes associated with an algorithmic decision are often said to play a role in determining whether the decision engenders a right to an explanation. More specifically, “high stakes” decisions are often said to engender such a right to explanation whereas “low stakes” or “non-high” stakes decisions do not. While the overall gist of these ideas is clear enough, the details are lacking. In this paper, we aim to provide these details through a detailed investigation of what we will (...) call the “Simple Stakes Thesis”. The Simple Stakes Thesis, as it will turn out, is too simple. For even if the stakes associated with a specific one-off decision are low—and hence does not engender a right to an explanation—such decisions may nevertheless form part of a high stakes pattern or aggregate of decisions. In such cases, we argue, even a low stakes decision may engender a right to explanation. Not only does this show that the right to explanation is more demanding than so far recognized. It also shows that the stakes thesis is significantly harder to apply in practice. (shrink)

In Algorithms & Autonomy, Rubel, Castro, and Pham (hereafter RCP), argue that the concept of autonomy is especially central to understanding important moral problems about algorithms. In particular, autonomy plays a role in analyzing the version of social contract theory that they endorse. I argue that although RCP are largely correct in their diagnosis of what is wrong with the algorithms they consider, those diagnoses can be appropriated by moral theories RCP see as in competition with their autonomy based theory. (...) Most notably, proponents of consequentialism and virtue ethics can appropriate RCPs insights. This is good news because RCP’s social contract theory is vulnerable to a well known class of counterexamples. The most significant contribution of RCP, if I am right, is in their identification, presentation, and evaluation of concrete cases involving algorithms and not in the more controversial claims about theoretical ethics that RCP themselves see as central to what they are doing. (shrink)

We cooperate with other people despite the risk of being exploited or hurt. If future artificial intelligence (AI) systems are benevolent and cooperative toward us, what will we do in return? Here we show that our cooperative dispositions are weaker when we interact with AI. In nine experiments, humans interacted with either another human or an AI agent in four classic social dilemma economic games and a newly designed game of Reciprocity that we introduce here. Contrary to the hypothesis that (...) people mistrust algorithms, participants trusted their AI partners to be as cooperative as humans. However, they did not return AI's benevolence as much and exploited the AI more than humans. These findings warn that future self-driving cars or co-working robots, whose success depends on humans' returning their cooperativeness, run the risk of being exploited. This vulnerability calls not just for smarter machines but also better human-centered policies. (shrink)

Algorithmic systems and predictive analytics play an increasingly important role in various aspects of modern life. Scholarship on the moral ramifications of such systems is in its early stages, and much of it focuses on bias and harm. This paper argues that in understanding the moral salience of algorithmic systems it is essential to understand the relation between algorithms, autonomy, and agency. We draw on several recent cases in criminal sentencing and K–12 teacher evaluation to outline four key (...) ways in which issues of agency, autonomy, and respect for persons can conflict with algorithmic decision-making. Three of these involve failures to treat individual agents with the respect they deserve. The fourth involves distancing oneself from a morally suspect action by attributing one’s decision to take that action to an algorithm, thereby laundering one’s agency. (shrink)

Discrimination laws usually prohibit discrimination based on some traits, like race, caste, and sex, and not on others, like sports team allegiance. Should socioeconomic class be included among the protected traits? We examine an argument for the view that it should which leads to the conclusion that both direct and indirect socioeconomic discrimination should be prohibited by the state. The argument has three premises: (1) direct paradigmatic discrimination should be prohibited by law; (2) if direct paradigmatic (...) discrimination should be prohibited by law, then so should direct socioeconomic discrimination; and (3) where direct discrimination based on a certain trait should be prohibited by law, so should indirect discrimination on the basis of the same trait. Since we assume most readers will accept (1), the paper is largely devoted to the defense of (2) and (3). The outlawing of direct socioeconomic discrimination would involve an adjustment of the political and legal status quo, but we argue that it should nevertheless be implemented. By contrast, a prohibition on indirect socioeconomic discrimination would require a radical change to the way in which our societies are structured, because many of our laws, policies, and practices systematically (re)produce inequality to the disproportionate detriment of the poor. Our main aim, therefore, is not to defend the prohibition of indirect socioeconomic discrimination, but to show that most people’s views about discrimination are in deep tension with their attitude to current law. Resolving this tension would require for most of us to revise our beliefs either about whether indirect socioeconomic discrimination should be prohibited or about (1)–(3). One might reject the view that law should prohibit socioeconomic discrimination and still accept this claim about a deep tension in most people’s views on discrimination. (shrink)

Artificial intelligence (AI) systems are increasingly adopted to make decisions in domains such as business, education, health care, and criminal justice. However, such algorithmic decision systems can have prevalent biases against marginalized social groups and undermine social justice. Explainable artificial intelligence (XAI) is a recent development aiming to make an AI system’s decision processes less opaque and to expose its problematic biases. This paper argues against technical XAI, according to which the detection and interpretation of algorithmic bias can (...) be handled more or less independently by technical experts who specialize in XAI methods. Drawing on resources from feminist epistemology, we show why technical XAI is mistaken. Specifically, we demonstrate that the proper detection of algorithmic bias requires relevant interpretive resources, which can only be made available, in practice, by actively involving a diverse group of stakeholders. Finally, we suggest how feminist theories can help shape integrated XAI: an inclusive social-epistemic process that facilitates the amelioration of algorithmic bias. (shrink)

The prospective introduction of autonomous cars into public traffic raises the question of how such systems should behave when an accident is inevitable. Due to concerns with self-interest and liberal legitimacy that have become paramount in the emerging debate, a contractarian framework seems to provide a particularly attractive means of approaching this problem. We examine one such attempt, which derives a harm minimisation rule from the assumptions of rational self-interest and ignorance of one’s position in a future accident. We contend, (...) however, that both contractarian approaches and harm minimisation standards are flawed, due to a failure to account for the fundamental difference between those ‘involved’ and ‘uninvolved’ in an impending crash. Drawing from classical works on the trolley problem, we show how this notion can be substantiated by reference to either the distinction between negative and positive rights, or to differences in people’s claims. By supplementing harm minimisation with corresponding constraints, we can develop crash algorithms for autonomous cars which are both ethically adequate and promise to overcome certain significant practical barriers to implementation. (shrink)

Inspired by recent breakthroughs in predictive modeling, practitioners in both industry and government have turned to machine learning with hopes of operationalizing predictions to drive automated decisions. Unfortunately, many social desiderata concerning consequential decisions, such as justice or fairness, have no natural formulation within a purely predictive framework. In efforts to mitigate these problems, researchers have proposed a variety of metrics for quantifying deviations from various statistical parities that we might expect to observe in a fair world and offered a (...) variety of algorithms in attempts to satisfy subsets of these parities or to trade o the degree to which they are satised against utility. In this paper, we connect this approach to fair machine learning to the literature on ideal and non-ideal methodological approaches in political philosophy. The ideal approach requires positing the principles according to which a just world would operate. In the most straightforward application of ideal theory, one supports a proposed policy by arguing that it closes a discrepancy between the real and the perfectly just world. However, by failing to account for the mechanisms by which our non-ideal world arose, the responsibilities of various decision-makers, and the impacts of proposed policies, naive applications of ideal thinking can lead to misguided interventions. In this paper, we demonstrate a connection between the fair machine learning literature and the ideal approach in political philosophy, and argue that the increasingly apparent shortcomings of proposed fair machine learning algorithms reflect broader troubles faced by the ideal approach. We conclude with a critical discussion of the harms of misguided solutions, a reinterpretation of impossibility results, and directions for future research. (shrink)

Algorithms influence every facet of modern life: criminal justice, education, housing, entertainment, elections, social media, news feeds, work… the list goes on. Delegating important decisions to machines, however, gives rise to deep moral concerns about responsibility, transparency, freedom, fairness, and democracy. Algorithms and Autonomy connects these concerns to the core human value of autonomy in the contexts of algorithmic teacher evaluation, risk assessment in criminal sentencing, predictive policing, background checks, news feeds, ride-sharing platforms, social media, and election interference. Using (...) these case studies, the authors provide a better understanding of machine fairness and algorithmic transparency. They explain why interventions in algorithmic systems are necessary to ensure that algorithms are not used to control citizens' participation in politics and undercut democracy. This title is also available as Open Access on Cambridge Core. (shrink)

Since the Enlightenment, there have been advocates for the rationalizing efficiency of enlightened sovereigns, bureaucrats, and technocrats. Today these enthusiasms are joined by calls for replacing or augmenting government with algorithms and artificial intelligence, a process already substantially under way. Bureaucracies are in effect algorithms created by technocrats that systematize governance, and their automation simply removes bureaucrats and paper. The growth of algorithmic governance can already be seen in the automation of social services, regulatory oversight, policing, the justice system, (...) and the military. However, there have also always been justified democratic and economic criticisms of autocracy, bureaucracy, technocracy, and algorithmic governance. Bureaucrats, technocrats, and algorithms embody biases that tend to serve the interests of elites, and all require transparency and democratic accountability, oversight individual citizens are ill equipped to exercise. As state apparatuses are increasingly automated, mechanisms for collective action and democratic oversight also need to be automated. Algorithms and cyborg citizenry will enable a posthuman democracy. Democratically accountable algorithmic governance, enabled by artificial intelligence and human enhancement, can automate bottom-up citizen surveillance, inform debate, aggregate decision making, and ensure the efficient working of the gradually withering state. As paid work disappears and we transition to a postcapitalist economy with a universal basic income, market mechanisms can be replaced with democratic planning. Indeed, only algorithmic governance can secure our future against accelerating threats from technological innovation. (shrink)

Does algorithmic political bias contribute to an entrenchment and polarization of political positions? Franke argues that it may do so because the bias involves classifications of people as liberals, conservatives, etc., and individuals often conform to the ways in which they are classified. I provide a novel example of this phenomenon in human–computer interactions and introduce a social psychological mechanism that has been overlooked in this context but should be experimentally explored. Furthermore, while Franke proposes that algorithmic political (...) classifications entrench political identities, I contend that they may often produce the opposite result. They can lead people to change in ways that disconfirm the classifications. Consequently and counterintuitively, algorithmic political bias can in fact decrease political entrenchment and polarization. (shrink)

In this paper, we distinguish between different sorts of assessments of algorithmic systems, describe our process of assessing such systems for ethical risk, and share some key challenges and lessons for future algorithm assessments and audits. Given the distinctive nature and function of a third-party audit, and the uncertain and shifting regulatory landscape, we suggest that second-party assessments are currently the primary mechanisms for analyzing the social impacts of systems that incorporate artificial intelligence. We then discuss two kinds of (...) as-sessments: an ethical risk assessment and a narrower, technical algo-rithmic bias assessment. We explain how the two assessments depend on each other, highlight the importance of situating the algorithm within its particular socio-technical context, and discuss a number of lessons and challenges for algorithm assessments and, potentially, for algorithm audits. The discussion builds on our team’s experience of advising and conducting ethical risk assessments for clients across dif-ferent industries in the last four years. Our main goal is to reflect on the key factors that are potentially ethically relevant in the use of algo-rithms, and draw lessons for the nascent algorithm assessment and audit industry, in the hope of helping all parties minimize the risk of harm from their use. (shrink)

This introduction to a special section on algorithmic thought provides a framework through which the articles in that collection can be contextualised and their individual contributions highlighted. Over the past decade, there has been a growing interest in artificial intelligence (AI). This special section reflects on this AI boom and its implications for studying what thinking is. Focusing on the algorithmic character of computing machines and the thinking that these machines might express, each of the special section’s essays (...) considers different dimensions of algorithmic thought, engaging with a diverse set of epistemological questions and issues. (shrink)

Predictive algorithms are playing an increasingly prominent role in society, being used to predict recidivism, loan repayment, job performance, and so on. With this increasing influence has come an increasing concern with the ways in which they might be unfair or biased against individuals in virtue of their race, gender, or, more generally, their group membership. Many purported criteria of algorithmic fairness concern statistical relationships between the algorithm’s predictions and the actual outcomes, for instance requiring that the rate of (...) false positives be equal across the relevant groups. We might seek to ensure that algorithms satisfy all of these purported fairness criteria. But a series of impossibility results shows that this is impossible, unless base rates are equal across the relevant groups. What are we to make of these pessimistic results? I argue that none of the purported criteria, except for a calibration criterion, are necessary conditions for fairness, on the grounds that they can all be simultaneously violated by a manifestly fair and uniquely optimal predictive algorithm, even when base rates are equal. I conclude with some general reflections on algorithmic fairness. (shrink)

An AI-based “patient preference predictor” (PPP) is a proposed method for guiding healthcare decisions for patients who lack decision-making capacity. The proposal is to use correlations between sociodemographic data and known healthcare preferences to construct a model that predicts the unknown preferences of a particular patient. In this paper, I highlight a distinction that has been largely overlooked so far in debates about the PPP–that between algorithmic prediction and decision-making–and argue that much of the recent philosophical disagreement stems from (...) this oversight. I show how three prominent objections to the PPP only challenge its use as the sole determinant of a choice, and actually support its use as a source of evidence about patient preferences to inform human decision-making. The upshot is that we should adopt the evidential conception of the PPP and shift our evaluation of this technology towards the ethics of algorithmic prediction, rather than decision-making. (shrink)

This article argues that, as commonly understood, indirect discrimination is not necessarily unjust: 1) indirect discrimination involves the disadvantaging in relation to a particular benefit and such disadvantages are not unjust if the overall distribution of benefits and burdens is just; 2) indirect discrimination focuses on groups and group averages and ignores the distribution of harms and benefits within groups subjected to discrimination, but distributive justice is concerned with individuals; and 3) if indirect discrimination as (...) such is unjust, strict egalitarianism has to be the correct account of distributive justice, but such egalitarianism appears vulnerable to the leveling down objection (whether decisively or not), and many theorists explicitly reject strict egalitarianism anyway. The last point threatens the position of liberals who oppose indirect discrimination but think significant inequalities can be just. (shrink)