Recommendation systems (RSs) predict what the user likes and recommend it to them. While at the onset of RSs, the latter was designed to maximize the recommendation accuracy (i.e., accuracy was their only goal), nowadays many RSs models include diversity in recommendations (which thus is a further goal of RSs). In the computer science community, the introduction of diversity in RSs is justified mainly through economic reasons: diversity increases user satisfaction and, in niche markets, profits.I contend that, first, the economic (...) justification of diversity in RSs risks reducing it to an empirical matter of preference; second, diversity is ethically relevant as it supports two autonomy rights of the user: the right to an open present and the right to be treated as an individual. So far, diversity in RSs has been morally defended only in the case of RSs of news and scholarly content: diversity is held to have a depolarizing effect in a democratic society and the scientific community and make the users more autonomous in their news choices. I provide a justification of diversity in RSs that embraces all kinds of RSs (i.e., a holistic moral defense) and is based on a normative principle founded on the agency of the user, which I call the right to be an exception to predictions. Such a right holds that the proper treatment of a RS user qua agent forbids providing them with recommendations based only on their past or similar users’ choices. (shrink)

Firms use algorithms to make important business decisions. To date, the algorithmic accountability literature has elided a fundamentally empirical question important to business ethics and management: Under what circumstances, if any, are algorithmic decision-making systems considered legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the impact of decision importance, governance, outcomes, and data inputs on perceptions of the legitimacy of algorithmic decisions made by firms. We find that many of the procedural governance (...) mechanisms in practice today, such as notices and impact statements, do not lead to algorithmic decisions being perceived as more legitimate in general, and, consistent with legitimacy theory, that algorithmic decisions with good outcomes are perceived as more legitimate than bad outcomes. Yet, robust governance, such as offering an appeal process, can create a legitimacy dividend for decisions with bad outcomes. However, when arbitrary or morally dubious factors are used to make decisions, most legitimacy dividends are erased. In other words, companies cannot overcome the legitimacy penalty of using arbitrary or morally dubious factors, such as race or the day of the week, with a good outcome or an appeal process for individuals. These findings add new perspectives to both the literature on legitimacy and policy discussions on algorithmic decision-making in firms. (shrink)

The abundance of literature on ethical concerns regarding artificial intelligence (AI) highlights the need to systematize, integrate, and categorize existing efforts through a systematic literature review. The article aims to investigate prevalent concerns, proposed solutions, and prominent ethical approaches within the field. Considering 309 articles from the beginning of the publications in this field up until December 2021, this systematic literature review clarifies what the ethical concerns regarding AI are, and it charts them into two groups: (i) ethical concerns that (...) arise from the design of AI and (ii) ethical concerns that arise from human–AI interactions. The analysis of the obtained sample highlights the most recurrent ethical concerns. Finally, it exposes the main proposals of the literature to handle the ethical concerns according to the main ethical approaches. It interprets the findings to lay the foundations for future research on the ethics of AI. (shrink)

In this paper, we analyze two key claims offered by recruitment AI companies in relation to the development and deployment of AI-powered HR tools: (1) recruitment AI can objectively assess candidates by removing gender and race from their systems, and (2) this removal of gender and race will make recruitment fairer, help customers attain their DEI goals, and lay the foundations for a truly meritocratic culture to thrive within an organization. We argue that these claims are misleading for four reasons: (...) First, attempts to “strip” gender and race from AI systems often misunderstand what gender and race are, casting them as isolatable attributes rather than broader systems of power. Second, the attempted outsourcing of “diversity work” to AI-powered hiring tools may unintentionally entrench cultures of inequality and discrimination by failing to address the systemic problems within organizations. Third, AI hiring tools’ supposedly neutral assessment of candidates’ traits belie the power relationship between the observer and the observed. Specifically, the racialized history of character analysis and its associated processes of classification and categorization play into longer histories of taxonomical sorting and reflect the current demands and desires of the job market, even when not explicitly conducted along the lines of gender and race. Fourth, recruitment AI tools help produce the “ideal candidate” that they supposedly identify through by constructing associations between words and people’s bodies. From these four conclusions outlined above, we offer three key recommendations to AI HR firms, their customers, and policy makers going forward. (shrink)