Often machine learning programs inherit social patterns reflected in their training data without any directed effort by programmers to include such biases. Computer scientists call this algorithmic bias. This paper explores the relationship between machine bias and human cognitive bias. In it, I argue similarities between algorithmic and cognitive biases indicate a disconcerting sense in which sources of bias emerge out of seemingly innocuous patterns of information processing. The emergent nature of this bias obscures the existence of the bias itself, (...) making it difficult to identify, mitigate, or evaluate using standard resources in epistemology and ethics. I demonstrate these points in the case of mitigation techniques by presenting what I call ‘the Proxy Problem’. One reason biases resist revision is that they rely on proxy attributes, seemingly innocuous attributes that correlate with socially-sensitive attributes, serving as proxies for the socially-sensitive attributes themselves. I argue that in both human and algorithmic domains, this problem presents a common dilemma for mitigation: attempts to discourage reliance on proxy attributes risk a tradeoff with judgement accuracy. This problem, I contend, admits of no purely algorithmic solution. (shrink)

While many argue investment-based criteria for immigration are wrong or at least problematic, skill-based criteria remain relatively uncontroversial. This is normatively inconsistent. This article assesses three prominent normative objections to investment-based selection criteria for immigrants: that they wrongfully discriminate between prospective immigrants that they are unfair, and that they undermine political equality among citizens. It argues that either skill-based criteria are equally susceptible to these objections, or that investment-based criteria are equally shielded from them. Indeed, in some ways investment-based criteria (...) are less normatively problematic than skill-based criteria. Given this analysis, the resistance to investment-based migration criteria, but not to skill-based criteria, is inconsistent. (shrink)

I argue that there exists a natural kind social bias that subsumes seemingly heterogenous cases of implicit bias and other forms of social cognition. I explore the implications of this explicated notion of bias for the organization of the mind, theories of consciousness, and the system-dependence of biases.

Many legal, social, and medical theorists and practitioners, as well as lay people, seem to be concerned with the harmfulness of discriminative practices. However, the philosophical literature on the moral wrongness of discrimination, with a few exceptions, does not focus on harm. In this paper, I examine, and improve, a recent account of wrongful discrimination, which divides into a definition of group discrimination, and a characterisation of its moral wrong-making feature in terms of harm. The resulting account analyses the wrongness (...) of discrimination in terms of intrapersonal comparisons of the discriminatee’s actual, and relevantly counterfactual, well-being levels. I show that the account faces problems from counterfactuals, which can be traced back specifically to the orthodox - comparative, counterfactual, welfarist - concept of harm. I argue that non-counterfactual and non-comparative harm concepts face problems of their own, and don’t fit easily with our best understanding of discrimination; hence they are unsuitable to replace the orthodox concept here. I then propose a non-orthodox - comparative, counterfactual, hybrid - concept of harm, which relies on counterfactual comparisons of ways of being treated. I suggest how such a concept can help us handle the problems from counterfactuals, at least for my account of discrimination. I also show that there are similar proposals in other harm-related debates. An upshot of the paper is thus to corroborate the case for a non-orthodox, hybrid concept of harm, which seems better able to fulfil its functional roles in a variety of contexts. (shrink)

A frequent caveat in online dating profiles – “No fats, femmes, or Asians” – caused an LGBT activist to complain about the bias against Asians in the American gay community, which he called “racial looksism”. In response, he was asked that, if he himself would not date a fat person, why he should find others not dating Asians so upsetting. This response embodies a popular attitude that personal preferences or tastes are simply personal matters – they are not subject to (...) moral evaluation. In this paper, I argue, against this popular attitude, that a personal preference like racial looksism is indeed wrong. A preference like racial looksism is wrong because it is an overgeneralization that disrespects individuality by treating people as exchangeable tokens of one type, and such disrespect denies its objects appreciation that their dignity entitles them to. As it turns out, there is on my account a relevant moral difference between racial looksism and simple looksism. (shrink)

Discrimination is a central moral and legal concept. However, it is also a contested one. Particularly, accounts of the wrongness of discrimination often rely on controversial and particular assumptions. In this paper, I argue that a theory of discrimination that relies on premises that are very general (rather than unique to the concept of discrimination) and widely accepted provides a plausible (exhaustive) account of the concept of wrongful discrimination. According to the combined theory, wrongful discrimination consists of allocating a benefit (...) that is not supported by a morally significant fact (a valid reason), or in a way that involves distributive injustice, or both. (shrink)

The article analyses the concept of indirect discrimination, arguing first that existing conceptualisations are unsatisfactory and second that it is best understood as equal treatment that is disadvantageous to the discriminatees because of their group-membership. I explore four ways of further refining the definition, arguing that only an added condition of moral wrongness is at once plausible and helpful, but that it entails a number of new problems that may outweigh its benefits. Finally, I suggest that the moral wrongness of (...) indirect discrimination is best accounted for in terms of the harm it does to discriminatees, and sketch three ways in which it may do so. I conclude that the analysis provides both a clearer understanding of the concept of indirect discrimination as well as indirect support for a harm-based account of the wrongness of discrimination, while suggesting that our moral obligations qua non-discrimination may be more extensive than is frequently assumed. (shrink)

Algorithmic discrimination has rapidly become a topic of intense public and academic interest. This article explores three issues raised by algorithmic discrimination: 1) the distinction between direct and indirect discrimination, 2) the notion of disadvantageous treatment, and 3) the moral badness of discriminatory automated decision-making. It argues that some conventional distinctions between direct and indirect discrimination appear not to apply to algorithmic discrimination, that algorithmic discrimination may often be discrimination between groups, as opposed to against groups, and that it is (...) not necessarily the case that morally bad algorithmic discrimination gives us reason to not use automated decision-making. For each of the three issues, the article explores implications for algorithmic discrimination, suggests some alternative answers, and clarifies how we may want to think of discrimination more broadly in light of lessons drawn from the context of algorithmic discrimination. (shrink)

In this paper we make the case for the emergence of novel kind of bias with the use of algorithmic decision-making systems. We argue that the distinctive generative process of feature creation, characteristic of machine learning (ML), contorts feature parameters in ways that can lead to emerging feature spaces that encode novel algorithmic bias involving already marginalized groups. We term this bias _assembled bias._ Moreover, assembled biases are distinct from the much-discussed algorithmic bias, both in source (training data versus feature (...) creation) and in content (mimics of extant societal bias versus reconfigured categories). As such, this problem is distinct from issues arising from bias-encoding training feature sets or proxy features. Assembled bias is not epistemically transparent in source or content. Hence, when these ML models are used as a basis for decision-making in social contexts, algorithmic fairness concerns are compounded. (shrink)

The article explores one prominent account of what makes discrimination morally bad (when it is) – the disrespect-based account. The article first reviews and clarifies the account, arguing that it is most charitably understood as the claim that discrimination is morally bad when the discriminator gives lower weight to reasons grounded in the moral status of the discriminatee(s) in her decision-making. It then presents three challenges to the account, and reviews a recent argument in defense of it. The first challenge (...) is the fact that intuitions that might support the disrespect-based account can also be explained by reference to the fact that disrespect reflects poorly on the moral character of the discriminator. The second challenge is that there are cases where disrespectful discrimination is not intuitively worse than non-disrespectful discrimination. The third challenge is that the disrespect-based account has difficulties explaining our intuitions about cases of “doing right for the wrong reasons”. Finally, pace a recent argument in its defense, cases of harmless and apparently morally bad discrimination are plausibly best explained by other factors than disrespect. In the light of this analysis, the article concludes that the disrespect-based account should be abandoned. (shrink)

In this paper, I examine whether the use of artificial intelligence (AI) and automated decision-making (ADM) aggravates issues of discrimination as has been argued by several authors. For this purpose, I first take up the lively philosophical debate on discrimination and present my own definition of the concept. Equipped with this account, I subsequently review some of the recent literature on the use AI/ADM and discrimination. I explain how my account of discrimination helps to understand that the general claim in (...) view of the aggravation of discrimination is unwarranted. Finally, I argue that the use of AI/ADM can, in fact, increase issues of discrimination, but in a different way than most critics assume: it is due to its epistemic opacity that AI/ADM threatens to undermine our moral deliberation which is essential for reaching a common understanding of what should count as discrimination. As a consequence, it turns out that algorithms may actually help to detect hidden forms of discrimination. (shrink)