We have seen through many examples that data science and artificial intelligence can reinforce structural inequalities like sexism and racism. Data is power, and that power is distributed unequally. This book offers a vision for a feminist data science that can challenge power and work towards justice. This book takes a stand against a world that benefits some (including the authors, two white women) at the expense of others. It seeks to provide concrete steps for data scientists seeking to learn (...) how feminism can help them work towards justice, and for feminists seeking to learn how their own work can carry over to the growing field of data science. It is addressed to professionals in all fields where data-driven decisions are being made, as well as to communities that want to better understand the data that surrounds them. It is written for everyone who seeks to better understand the charts and statistics that they encounter in their day-to-day lives, and for everyone who seeks to better communicate the significance of such charts and statistics to others. This is an example-driven book written with a broad audience of scholars, students, and practitioners in mind. It offers a way of thinking about data, both their uses and their limits, that is informed by direct experience, by a commitment to action, and by the ideas associated with intersectional feminist thought. (shrink)

This article argues that many situations in social life can be analyzed by their requirement for the justification of action. It is in particular in situations of dispute that a need arises to explicate the grounds on which responsibility for errors is distributed and on which new agreement can be reached. Since a plurality of mutually incompatible modes of justification exists, disputes can be understood as disagreements either about whether the accepted rule of justification has not been violated or about (...) which mode of justification to apply at all. The article develops a grammar of such modes of justification, called orders of worth (grandeur), and argues that the human capacity for criticism becomes visible in the daily occurrence of disputes over criteria for justification. At the same time, it is underlined that not all social situations can be interpreted with the help of such a sense of justice, which resides on a notion of equivalence. Regimes of love, of violence or of familiarity are systematically distinct from regimes of justification. (shrink)

We argue that while digital health technologies (e.g. artificial intelligence, smartphones, and virtual reality) present significant opportunities for improving the delivery of healthcare, key concepts that are used to evaluate and understand their impact can obscure significant ethical issues related to patient engagement and experience. Specifically, we focus on the concept of empowerment and ask whether it is adequate for addressing some significant ethical concerns that relate to digital health technologies for mental healthcare. We frame these concerns using five key (...) ethical principles for AI ethics (i.e. autonomy, beneficence, non-maleficence, justice, and explicability), which have their roots in the bioethical literature, in order to critically evaluate the role that digital health technologies will have in the future of digital healthcare. (shrink)

In the age of Big Data, companies and governments are increasingly using algorithms to inform hiring decisions, employee management, policing, credit scoring, insurance pricing, and many more aspects of our lives. Artificial intelligence systems can help us make evidence-driven, efficient decisions, but can also confront us with unjustified, discriminatory decisions wrongly assumed to be accurate because they are made automatically and quantitatively. It is becoming evident that these technological developments are consequential to people’s fundamental human rights. Despite increasing attention to (...) these urgent challenges in recent years, technical solutions to these complex socio-ethical problems are often developed without empirical study of societal context and the critical input of societal stakeholders who are impacted by the technology. On the other hand, calls for more ethically and socially aware AI often fail to provide answers for how to proceed beyond stressing the importance of transparency, explainability, and fairness. Bridging these socio-technical gaps and the deep divide between abstract value language and design requirements is essential to facilitate nuanced, context-dependent design choices that will support moral and social values. In this paper, we bridge this divide through the framework of Design for Values, drawing on methodologies of Value Sensitive Design and Participatory Design to present a roadmap for proactively engaging societal stakeholders to translate fundamental human rights into context-dependent design requirements through a structured, inclusive, and transparent process. (shrink)

In recent years, all major consumer technology corporations have moved into the domain of health research. This ‘Googlization of health research’ begs the question of how the common good will be served in this research. As critical data scholars contend, such phenomena must be situated within the political economy of digital capitalism in order to foreground the question of public interest and the common good. Here, trends like GHR are framed within a double, incommensurable logic, where private gain and economic (...) value are pitted against public good and societal value. While helpful for highlighting the exploitative potential of digital capitalism, this framing is limiting, insofar as it acknowledges only one conception of the common good. This article uses the analytical framework of modes of justification developed by Boltanksi and Thévenot to identify a plurality of orders of worth and conceptualizations of the common good at work in GHR. Not just the ‘civic’ and ‘market’ orders, but also an ‘industrial’, a ‘project’, and what I call a ‘vitalist’ order. Using promotional material of GHR initiatives and preliminary interviews with participants in GHR projects, I ask what moral orientations guide different actors in GHR. Engaging seriously with these different conceptions of the common good is paramount. First, in order to critically evaluate them and explicate what is at stake in the move towards GHR, and ultimately, in order to develop viable governance solutions that ensure strong ‘civic’ components. (shrink)

A variety of approaches have appeared in academic literature and in design practice representing “ethics-first” methods. These approaches typically focus on clarifying the normative dimensions of design, or outlining strategies for explicitly incorporating values into design. While this body of literature has developed considerably over the last 20 years, two themes central to the endeavour of ethics and values in design (E + VID) have yet to be systematically discussed in relation to each other: (a) designer agency, and (b) the (...) strength of normative claims informing the design process. To address this gap, we undertook a structured review of leading E + VID approaches and critiques, and classified them according to their positions on normative strength, and views regarding designer agency. We identified 18 distinct approaches and 13 critiques that met the inclusion criteria for our review. Included papers were distributed across the spectrum of views regarding normative strength, and we found that no approaches and only one critique represented a view characteristic of “low” designer agency. We suggest that the absence of “low” designer agency approaches results in the neglect of crucial influences on design as targets of intervention by designers. We conclude with suggestions for future research that might illuminate strategies to achieve ethical design in information mature societies, and argue that without attending to the tensions raised by balancing normatively “strong” visions of the future with limitations imposed on designer agency in corporate-driven design settings, “meaningful” ethical design will continue to encounter challenges in practice. (shrink)

Critical Data Studies explore the unique cultural, ethical, and critical challenges posed by Big Data. Rather than treat Big Data as only scientifically empirical and therefore largely neutral phenomena, CDS advocates the view that Big Data should be seen as always-already constituted within wider data assemblages. Assemblages is a concept that helps capture the multitude of ways that already-composed data structures inflect and interact with society, its organization and functioning, and the resulting impact on individuals’ daily lives. CDS questions the (...) many assumptions about Big Data that permeate contemporary literature on information and society by locating instances where Big Data may be naively taken to denote objective and transparent informational entities. In this introduction to the Big Data & Society CDS special theme, we briefly describe CDS work, its orientations, and principles. (shrink)

Recently, there is increased attention to the integration of moral values into the conception, design, and development of emerging IT. The most reviewed approach for this purpose in ethics and technology so far is Value-Sensitive Design (VSD). This article considers VSD as the prime candidate for implementing normative considerations into design. Its methodology is considered from a conceptual, analytical, normative perspective. The focus here is on the suitability of VSD for integrating moral values into the design of technologies in a (...) way that joins in with an analytical perspective on ethics of technology. Despite its promising character, it turns out that VSD falls short in several respects: (1) VSD does not have a clear methodology for identifying stakeholders, (2) the integration of empirical methods with conceptual research within the methodology of VSD is obscure, (3) VSD runs the risk of committing the naturalistic fallacy when using empirical knowledge for implementing values in design, (4) the concept of values, as well as their realization, is left undetermined and (5) VSD lacks a complimentary or explicit ethical theory for dealing with value trade-offs. For the normative evaluation of a technology, I claim that an explicit and justified ethical starting point or principle is required. Moreover, explicit attention should be given to the value aims and assumptions of a particular design. The criteria of adequacy for such an approach or methodology follow from the evaluation of VSD as the prime candidate for implementing moral values in design. (shrink)

Advancements in technology are shifting the ways that biomedical data are collected, managed, and used. The pervasiveness of connected devices is expanding the types of information that are defined as ‘health data.’ Additionally, cloud-based mechanisms for data collection and distribution are shifting biomedical research away from traditional infrastructure towards a more distributed and interconnected ecosystem. This shift provides an opportunity for us to reimagine the roles of scientists and participants in health research, with the potential to more meaningfully engage in (...) partnership across the research process. At the same time, these emerging practices present a potential to expose research participants to unanticipated and unintended consequences. Social norms and policy can help to mitigate these risks, but their development is often slow relative to the pace of technological advances and, as such, they can become reactive rather than prospective. As an alternative, the integrated development of data governance structures within technological advancements, supports their effective implementation, evaluation and evolution in a manner that can balance the benefits and risks of biomedical researcher in a decentralized ecosystem. (shrink)

We live in the Age of Big Data. In medicine, artificial intelligence and machine learning algorithms, fueled by big data, promise to change how physicians make diagnoses, determine prognoses, and develop new treatments. An exponential rise in articles on these topics is seen in the medical literature. Recent applications range from the use of deep learning neural networks to diagnose diabetic retinopathy and skin cancer from image databases, to the use of various machine learning algorithms for prognostication in cancer and (...) cardiovascular disease. Many factors are driving the adoption of AI in health care, from... (shrink)

Part of understanding the meaning and power of algorithms means asking what new demands they might make of ethical frameworks, and how they might be held accountable to ethical standards. I develop a definition of networked information algorithms as assemblages of institutionally situated code, practices, and norms with the power to create, sustain, and signify relationships among people and data through minimally observable, semiautonomous action. Starting from Merrill’s prompt to see ethics as the study of “what we ought to do,” (...) I examine ethical dimensions of contemporary NIAs. Specifically, in an effort to sketch an empirically grounded, pragmatic ethics of algorithms, I trace an algorithmic assemblage’s power to convene constituents, suggest actions based on perceived similarity and probability, and govern the timing and timeframes of ethical action. (shrink)