This project explores the political economy, logic, strategies, agents, values, and ethical implications of this latest iteration of modern capitalism, and it seeks to delineate what surveillance capitalism is and what its consequences are for human dignity and worth. Using technologies of which they are ignorant, surveillance capitalists interfere with our ability to become ourselves individually and collectively. Without consent, they invade privacy, impede moral autonomy, harm democracy, and muddle care. Surveillance capitalists also violate a number of foundational ethical principles, (...) failing the standards of Kant, Mill, and Aristotle. I sketch out a broad alternative to surveillance capitalism, one where our media systems are built on decentered understandings of a care-based democracy. Reinvigorating both the press and collective self-government, the caring digital citizen has less need for surveillance, requiring good faith understanding and the moral autonomy to pursue excellence. (shrink)

Many researchers from robotics, machine ethics, and adjacent fields seem to assume that norms represent good behavior that social robots should learn to benefit their users and society. We would like to complicate this view and present seven key troubles with norm-compliant robots: (1) norm biases, (2) paternalism (3) tyrannies of the majority, (4) pluralistic ignorance, (5) paths of least resistance, (6) outdated norms, and (7) technologically-induced norm change. Because discussions of why norm-compliant robots can be problematic are noticeably absent (...) from the robot and machine ethics literature, this paper fills an important research gap. We argue that it is critical for researchers to take these issues into account if they wish to make norm-compliant robots. (shrink)

In this chapter, I argue that it is impossible to trust the Big Tech companies, in an ethically important sense of trust. The argument is not that these companies are untrustworthy. Rather, I argue that the power to hold the trustee accountable is a necessary component of this sense of trust, and, because these companies are so powerful, they are immune to our attempts, as individuals or nation-states, to hold them to account. It is, therefore, literally impossible to trust Big (...) Tech. After introducing the accounts of trust and power that I deploy, I argue that Big Tech companies have four kinds of power that render them unaccountable: fiscal power, political power, data power, and cognitive power. I conclude by reflecting on recent calls to break up the Big Tech firms, suggesting a new antitrust test in the light of my arguments. (shrink)

This paper presents an ethical framework designed to support the development of critical data literacy for research methods courses and data training programmes in higher education. The framework we present draws upon our reviews of literature, course syllabi and existing frameworks on data ethics. For this research we reviewed 250 research methods syllabi from across the disciplines, as well as 80 syllabi from data science programmes to understand how or if data ethics was taught. We also reviewed 12 data ethics (...) frameworks drawn from different sectors. Finally, we reviewed an extensive and diverse body of literature about data practices, research ethics, data ethics and critical data literacy, in order to develop a transversal model that can be adopted across higher education. To promote and support ethical approaches to the collection and use of data, ethics training must go beyond securing informed consent to enable a critical understanding of the techno-centric environment and the intersecting hierarchies of power embedded in technology and data. By fostering ethics as a method, educators can enable research that protects vulnerable groups and empower communities. (shrink)

In 2020, the European Union announced the award of the contract for the biometric part of the new database for border control, the Entry Exit System, to two companies: IDEMIA and Sopra Steria. Both companies had been previously involved in the development of databases for border and migration management. While there has been a growing amount of publicly available documents that show what kind of technologies are being implemented, for how much money, and by whom, there has been limited engagement (...) with digital methods in this field. Moreover, critical border and security scholarship has largely focused on qualitative and ethnographic methods. Building on a data feminist approach, we propose a transdisciplinary methodology that goes beyond binaries of qualitative/quantitative and opacity/transparency, examines power asymmetries and makes the labour of coding visible. Empirically, we build and analyse a dataset of the contracts awarded by two European Union agencies key to its border management policies – the European Agency for Large-Scale Information Systems (eu-LISA) and the European Border and Coast Guard Agency (Frontex). We supplement the digital analysis and visualisation of networks of companies with close reading of tender documents. In so doing, we show how a transdisciplinary methodology can be a device for making datafication ‘intelligible’ at the European Union borders. (shrink)

We argue that machine learning algorithms can inflict microaggressions on members of marginalized groups and that recognizing these harms as instances of microaggressions is key to effectively addressing the problem. The concept of microaggression is also illuminated by being studied in algorithmic contexts. We contribute to the microaggression literature by expanding the category of environmental microaggressions and highlighting the unique issues of moral responsibility that arise when we focus on this category. We theorize two kinds of algorithmic microaggression, stereotyping and (...) erasure microaggressions, and argue that corporations are responsible for the microaggressions their algorithms create. As a case study, we look at the problems faced by Google’s autocomplete prediction and at the insufficiency of their solutions. The case study of autocomplete demonstrates our core claim that microaggressions constitute a distinct form of algorithmic bias and that identifying them as such allows us to avoid seeming solutions that recreate the same kinds of harms. Google has a responsibility to make information freely available, without exposing users to degradation. To fulfill its duties to marginalized groups, Google must abandon the fiction of neutral prediction and instead embrace the liberatory power of suggestion. (shrink)

Ethics plays a key role in the normative analysis of the impacts of technology. We know that computers in general and the processing of data, the use of artificial intelligence, and the combination of computers and/or artificial intelligence with robotics are all associated with ethically relevant implications for individuals, groups, and society. In this article, we argue that while all technologies are ethically relevant, there is no need to create a separate ‘ethics of X’ or ‘X ethics’ for each and (...) every subtype of technology or technological property—e.g. computer ethics, AI ethics, data ethics, information ethics, robot ethics, and machine ethics. Specific technologies might have specific impacts, but we argue that they are often sufficiently covered and understood through already established higher-level domains of ethics. Furthermore, the proliferation of tech ethics is problematic because (a) the conceptual boundaries between the subfields are not well-defined, (b) it leads to a duplication of effort and constant reinventing the wheel, and (c) there is danger that participants overlook or ignore more fundamental ethical insights and truths. The key to avoiding such outcomes lies in a taking the discipline of ethics seriously, and we consequently begin with a brief description of what ethics is, before presenting the main forms of technology related ethics. Through this process, we develop a hierarchy of technology ethics, which can be used by developers and engineers, researchers, or regulators who seek an understanding of the ethical implications of technology. We close by deducing two principles for positioning ethical analysis which will, in combination with the hierarchy, promote the leveraging of existing knowledge and help us to avoid an exaggerated proliferation of tech ethics. (shrink)

Gender, sexuality and embodiment in digital spheres have been increasingly studied from various critical perspectives: From research highlighting the articulation of intimacies, desires, and sexualities in and through digital spaces to theoretical explorations of materiality in the digital realm. With such a high level of (inter)disciplinarity, theories, methods, and analyses of gender, sexuality, and embodiment in relation to digital spheres have become highly diversified. Aiming to reflect this diversity, this special issue brings together innovative and newly developed theoretical, empirical, analytical, (...) and critical approaches in the study of gender, sexuality, and embodiment in digital spheres. By connecting intersectionality and digitality to one another, it adopts an integrated approach that reflects the intricacy and interconnectedness of social categories and markers of difference, privilege, performance, and discrimination. The contributions explore a range of differently situated digital cultural practices, including intimate and sexual experiences with(in) digital media, online self-presentation, expressions of digital resistance, and forms of backlash and online attacks. What connects all these articles, is their critical approach to intersectional inequalities and privileges in relation to digitality, plus their nuanced perspective on gender, sexuality, and embodiment interferentially. The final article is based on a roundtable discussion and aims to encourage interdisciplinary connections and suggests ways of doing research that builds bridges between academia and activism. (shrink)

This essay argues that prediction markets, as one approach for aggregating dispersed private information, may not only be praised for their epistemic accuracy. They also feature characteristics that are morally desirable from the point of view of epistemic justice. Notably, they are a promising approach when we are trying to address testimonial injustice. The impersonality of market transactions effectively tackles the issue of identity prejudice, which underlies many forms of testimonial injustice. This is not to say that prediction markets do (...) not pose challenges for epistemic justice of their own like the risk of excluding knowers based on ability to pay. Or, in light of identity prejudice’s systematicity, the risk of discouraging some knowers because of differences in education or social background that lead to a certain unease when faced with prediction markets. The essay discusses these challenges, concedes limitations but also offers countermeasures to certain worries – e.g. hosting prediction markets on a play-money basis. Throughout, the essay addresses the question of the potential scope of using prediction markets for epistemic purposes, notably by means of the practical example of a prediction market for monetary policy. (shrink)

This paper shares results from a pedagogical experiment that assigns undergraduates to “cheat” on a final class essay by requiring their use of text-generating AI software. For this assignment, students harvested content from an installation of GPT-2, then wove that content into their final essay. At the end, students offered a “revealed” version of the essay as well as their own reflections on the experiment. In this assignment, students were specifically asked to confront the oncoming availability of AI as a (...) writing tool. What are the ethics of using AI this way? What counts as plagiarism? What are the conditions, if any, we should place on AI assistance for student writing? And how might working with AI change the way we think about writing, authenticity, and creativity? While students (and sometimes GPT-2) offered thoughtful reflections on these initial questions, actually composing with GPT-2 opened their perspectives more broadly on the ethics and practice of writing with AI. In this paper, I share how students experienced those issues, connect their insights to broader conversations in the humanities about writing and communication, and explain their relevance for the ethical use and evaluation of language models. (shrink)

Machine learning algorithms pervade contemporary society. They are integral to social institutions, inform processes of governance, and animate the mundane technologies of daily life. Consistently, the outcomes of machine learning reflect, reproduce, and amplify structural inequalities. The field of fair machine learning has emerged in response, developing mathematical techniques that increase fairness based on anti-classification, classification parity, and calibration standards. In practice, these computational correctives invariably fall short, operating from an algorithmic idealism that does not, and cannot, address systemic, Intersectional (...) stratifications. Taking present fair machine learning methods as our point of departure, we suggest instead the notion and practice of algorithmic reparation. Rooted in theories of Intersectionality, reparative algorithms name, unmask, and undo allocative and representational harms as they materialize in sociotechnical form. We propose algorithmic reparation as a foundation for building, evaluating, adjusting, and when necessary, omitting and eradicating machine learning systems. (shrink)

Frequent public uproar over forms of data science that rely on information about people demonstrates the challenges of defining and demonstrating trustworthy digital data research practices. This paper reviews problems of trustworthiness in what we term pervasive data research: scholarship that relies on the rich information generated about people through digital interaction. We highlight the entwined problems of participant unawareness of such research and the relationship of pervasive data research to corporate datafication and surveillance. We suggest a way forward by (...) drawing from the history of a different methodological approach in which researchers have struggled with trustworthy practice: ethnography. To grapple with the colonial legacy of their methods, ethnographers have developed analytic lenses and researcher practices that foreground relations of awareness and power. These lenses are inspiring but also challenging for pervasive data research, given the flattening of contexts inherent in digital data collection. We propose ways that pervasive data researchers can incorporate reflection on awareness and power within their research to support the development of trustworthy data science. (shrink)

Applications of artificial intelligence/machine learning in health care are dynamic and rapidly growing. One strategy for anticipating and addressing ethical challenges related to AI/ml for health care is patient and public involvement in the design of those technologies – often referred to as ‘co-design’. Co-design has a diverse intellectual and practical history, however, and has been conceptualized in many different ways. Moreover, AI/ml introduces challenges to co-design that are often underappreciated. Informed by perspectives from critical data studies and critical digital (...) health studies, we review the research literature on involvement in health care, and involvement in design, and examine the extent to which co-design as commonly conceptualized is capable of addressing the range of normative issues raised by AI/ml for health care. We suggest that AI/ml technologies have amplified and modified existing challenges related to patient and public involvement, and created entirely new challenges. We outline three pitfalls associated with co-design for ethical AI/ml for health care and conclude with suggestions for addressing these practical and conceptual challenges. (shrink)

Though rapid collection of big data is ubiquitous across domains, from industry settings to academic contexts, the ethics of big data collection and research are contested. A nexus of data ethics issues is the concept of creep, or repurposing of data for other applications or research beyond the conditions of original collection. Data creep has proven controversial and has prompted concerns about the scope of ethical oversight. Institutional review boards offer little guidance regarding big data, and problematic research can still (...) meet ethical standards. While ethics seem concrete through institutional deployment, I frame ethics as produced. Informed by my ethnographic research at a large public university in the U.S., I explore ethics through two models: ethics as institutional procedures and ethics as acts and intentions. The university where I conducted fieldwork is the development grounds for a predictive model that uses student data to anticipate academic success. While students consent to data collection, the circumstances of consent and the degree to which they are informed are not so apparent, as many data are a product of creep. Drawing from interviews and participant observation with administrators, data scientists, developers, and students, I examine data ethics, from a larger institutional model to everyday enactments related to data creep. After demonstrating the limits of such models, I propose a remodeling of ethics that draws on recent works on data, justice, and refusal to pose generative questions for rethinking ethics in institutional contexts. (shrink)

Machine learning has become a key component of contemporary information systems. Unlike prior information systems explicitly programmed in formal languages, ML systems infer rules from data. This paper shows what this difference means for the critical analysis of socio-technical systems based on machine learning. To provide a foundation for future critical analysis of machine learning-based systems, we engage with how the term is framed and constructed in self-education resources. For this, we analyze machine learning tutorials, an important information source for (...) self-learners and a key tool for the formation of the practices of the machine learning community. Our analysis identifies canonical examples of machine learning as well as important misconceptions and problematic framings. Our results show that machine learning is presented as being universally applicable and that the application of machine learning without special expertise is actively encouraged. Explanations of machine learning algorithms are missing or strongly limited. Meanwhile, the importance of data is vastly understated. This has implications for the manifestation of social inequalities through machine learning-based systems. (shrink)

Frantic warnings about the threat the Internet poses to our privacy are now a dime-a-dozen. Though gallons of ink have been spilled on the topic, it has largely gone unanalyzed from the perspective...

The Covid-19 pandemic broke on a world whose grip on epistemic trust was already in disarray. The first months of the pandemic saw many governments publicly performing reliance on epidemiological and modelling expertise in order to signal that data would be the basis for justifying whatever population-level measures of control were judged necessary. But comprehensive data has not become available, and instead scientists, policymakers and the public find themselves in a situation where policy inputs determine the data available and vice (...) versa. This essay asks how we can live with what Amoore has termed ‘post-Cartesian doubt’ in situations of existential risk, and what kind of approach to science and data can answer the moral and human demands of a situation such as the Covid-19 pandemic. I suggest that science and policy could be able to control the pandemic better by addressing the sources of uncertainty and missing data not as gaps in the information landscape, but as individuals who are likely to be members of less-visible and less powerful groups including low-wage workers, the elderly, migrants, prisoners and others. This would shift both data use and policy toward an ethics of care, an embodied approach which asks what people need and how they behave in relation to each other, rather than how to manage population-level behaviour. This approach, I argue, is more appropriate for pandemic response than a utilitarian calculation of how many people each country should expect to lose as a result of the disease. (shrink)

In recent years, the ethical impact of AI has been increasingly scrutinized, with public scandals emerging over biased outcomes, lack of transparency, and the misuse of data. This has led to a growing mistrust of AI and increased calls for mandated ethical audits of algorithms. Current proposals for ethical assessment of algorithms are either too high level to be put into practice without further guidance, or they focus on very specific and technical notions of fairness or transparency that do not (...) consider multiple stakeholders or the broader social context. In this article, we present an auditing framework to guide the ethical assessment of an algorithm. The audit instrument itself is comprised of three elements: a list of possible interests of stakeholders affected by the algorithm, an assessment of metrics that describe key ethically salient features of the algorithm, and a relevancy matrix that connects the assessed metrics to stakeholder interests. The proposed audit instrument yields an ethical evaluation of an algorithm that could be used by regulators and others interested in doing due diligence, while paying careful attention to the complex societal context within which the algorithm is deployed. (shrink)

In analyzing oppressive systems like racism, social theorists have articulated accounts of the dynamic interaction and mutual dependence between psychological components, such as individuals’ patterns of thought and action, and social components, such as formal institutions and informal interactions. We argue for the further inclusion of physical components, such as material artifacts and spatial environments. Drawing on socially situated and ecologically embedded approaches in the cognitive sciences, we argue that physical components of racism are not only shaped by, but also (...) shape psychological and social components of racism. Indeed, while our initial focus is on racism and racist things, we contend that our framework is also applicable to other oppressive systems, including sexism, classism, and ableism. This is because racist things are part of a broader class of oppressive things, which are material artifacts and spatial environments that are in congruence with an oppressive system. (shrink)

**No longer the current version available on SEP; see revised version by Sharon Crasnow** -/- Feminists have a number of distinct interests in, and perspectives on, science. The tools of science have been a crucial resource for understanding the nature, impact, and prospects for changing gender-based forms of oppression; in this spirit, feminists actively draw on, and contribute to, the research programs of a wide range of sciences. At the same time, feminists have identified the sciences as a source as (...) well as a locus of gender inequalities: the institutions of science have a long tradition of excluding women as practitioners; feminist critics of science find that women and gender (or, more broadly, issues of concern to women and sex/gender minorities) are routinely marginalized as subjects of scientific inquiry, or are treated in ways that reproduce gender-normative stereotypes; and, closing the circle, scientific authority has frequently served to rationalize the kinds of social roles and institutions that feminists call into question. -/- Feminist perspectives on science therefore reflect a broad spectrum of epistemic attitudes toward and appraisals of science. Some urge the reform of gender inequities in the institutions of science and call for attention to neglected questions with the aim of improving the sciences in their own terms; they do not challenge the standards and practices of the sciences they engage. Others pursue jointly critical and constructive programs of research that, to varying degrees, aim at transforming the methodologies, substantive content, framework assumptions, and epistemic ideals that animate the sciences. The content of these perspectives, and the degree to which they generate transformative critique, depends not only on the types of philosophical and political commitments that inform them but also on the nature of the sciences and subject domains on which they bear. Feminist perspectives have had greatest impact on sciences that deal with inherently gendered subjects—the social and human sciences—and, secondarily, on sciences that study subjects characterized in gendered terms, metaphorically or by analogy (projectively gendered subjects), chiefly the biological and life sciences. Feminist perspectives are relevant to sciences that deal with non-gendered subject matters, but perspectives vary substantially in content and in critical import depending on the sciences and the particular research programs they engage. (shrink)

Audiovisual communication is greatly contributing to the emerging research field of affective computing. The use of audiovisual stimuli within immersive virtual reality environments is providing very intense emotional reactions, which provoke spontaneous physical and physiological changes that can be assimilated into real responses. In order to ensure high-quality recognition, the artificial intelligence system must be trained with adequate data sets, including not only those gathered by smart sensors but also the tags related to the elicited emotion. Currently, there are very (...) few techniques available for the labeling of emotions. Among them, the Self-Assessment Manikin devised by Lang is one of the most popular. This study shows experimentally that the graphic proposal for the original SAM labelling system, as devised by Lang, is not neutral to gender and contains gender biases in its design and representation. Therefore, a new graphic design has been proposed and tested according to the guidelines of expert judges. The results of the experiment show an overall improvement in the labeling of emotions in the pleasure–arousal–dominance affective space, particularly, for women. This research proves the relevance of applying the gender perspective in the validation of tools used throughout the years. (shrink)

In this paper, we analyze two key claims offered by recruitment AI companies in relation to the development and deployment of AI-powered HR tools: (1) recruitment AI can objectively assess candidates by removing gender and race from their systems, and (2) this removal of gender and race will make recruitment fairer, help customers attain their DEI goals, and lay the foundations for a truly meritocratic culture to thrive within an organization. We argue that these claims are misleading for four reasons: (...) First, attempts to “strip” gender and race from AI systems often misunderstand what gender and race are, casting them as isolatable attributes rather than broader systems of power. Second, the attempted outsourcing of “diversity work” to AI-powered hiring tools may unintentionally entrench cultures of inequality and discrimination by failing to address the systemic problems within organizations. Third, AI hiring tools’ supposedly neutral assessment of candidates’ traits belie the power relationship between the observer and the observed. Specifically, the racialized history of character analysis and its associated processes of classification and categorization play into longer histories of taxonomical sorting and reflect the current demands and desires of the job market, even when not explicitly conducted along the lines of gender and race. Fourth, recruitment AI tools help produce the “ideal candidate” that they supposedly identify through by constructing associations between words and people’s bodies. From these four conclusions outlined above, we offer three key recommendations to AI HR firms, their customers, and policy makers going forward. (shrink)

The paper presents an ethical analysis and constructive critique of the current practice of AI ethics. It identifies conceptual substantive and procedural challenges and it outlines strategies to address them. The strategies include countering the hype and understanding AI as ubiquitous infrastructure including neglected issues of ethics and justice such as structural background injustices into the scope of AI ethics and making the procedures and fora of AI ethics more inclusive and better informed with regard to philosophical ethics. These measures (...) integrate the perspective of AI justice into AI ethics, strengthening its capacity to provide comprehensive normative orientation and guidance for the development and use of AI that actually improves human lives and living together. (shrink)

Makerspaces and hacklabs are believed to encourage a positive attitude towards gaining computer skills. Within these communities for peer production, citizens can apply cutting-edge technologies in DIY projects. In recent decades, mushrooming makerspaces and hacklabs were embraced by the tech industry and governments alike. Feminist makerspaces and hacklabs, however, as they are centred around a queer feminist agenda, have raised eyebrows. In order to foster diversity in tech development, they create safer spaces for self-expression. Here, feminist laymen*, makers, designers, artists (...) and tinkerers experiment with open-source hardware and software. Art and design projects emerging from feminist hacklabs focus on issues of representation and democratic participation in digital media, as well as on ways of reclaiming one’s own body. This article tries to unpack how, after an exhibition on sexual health norms, a feminist hacklab was attacked by local right-wing and conservative politicians. The attack resulted in the defunding of the feminist hacklab. But it also started a transformation process within the collective, as members became aware of critical interferences of diffracting marginalisations. The crisis triggered a discussion on how each member was threatened to very different degrees; for example, there was more at stake for members depending on their legal status in the country. The right-wing and conservative campaign against the feminist hacklab damaged the initiative, but at the same time it pushed the collective to generate increased vehemence and resilience. (shrink)

Data about the accessibility of United States municipalities is infrastructure in the smart city. What is counted and how, reflects the sociotechnical imaginary of a time or place. In this paper we focus on features identified by people with disabilities as promoting or hindering safe pedestrian travel. We use a regionally stratified sample of 178 cities across the United States. The municipalities were scored on two factors: their open data practices, and the degree to which they cataloged the environmental features (...) that persons with disabilities deemed critical for safe movement through urban spaces. In contradiction to the dominating narrative of too much data and not enough analyses, we find that when it comes to data points that might be useful to persons with disabilities, data are lacking. This data gap has consequences both politically and materially—on one hand data could help enforce compliance with the Americans with Disabilities Act, on the other they would allow for safe route planning. We find that reading these data formats and collection patterns from the perspective of critical disability studies—particularly those whose work disrupts notions of “normal” —helps answer questions about potential benefits and harms of data practices. This lens has the potential to promote analysis that is as disruptive to injustices as it is practical. (shrink)

In this paper, we explore how the development and affordances of predictive analytics may impact how teachers and other educational actors think about and teach students and, more broadly, how society understands education. Our particular focus is on the data dashboards of learning support systems which are based on Machine Learning. While previous research has focused on how these systems produce credible knowledge, we explore here how they also produce compelling, persuasive and convincing narratives. Our main argument is that particular (...) kinds of stories are written by predictive analytics and written into their data dashboards. Based on a case study of a leading predictive analytics system, we explore how data dashboards imply causality between the ‘facts’ they are visualising. To do so, we analyse the stories they tell according to their spatial and temporal dimensions, characters and events, sequentiality as well as tellability. In the stories we identify, teachers are managers, students are at greater or lesser risk, and students’ sociality is reduced to machine-readable interactions. Overall, only data marked as individual behaviours becomes relevant to the system, rendering structural inequalities invisible. Reflecting on the implications of these systems, we suggest ways in which the uptake of these systems can interrupt such stories and reshape them in other directions. (shrink)

Design bioethics provides a promising framework for incorporating technological design into bioethics research and education. Beyond the development of digital empirical tool...

A growing number of people are relying on technologies like Google Maps not only to navigate and locate themselves in cartographic space but also to search, discover and evaluate urban places. While the spatial data that underlies such technology frequently appears as a combination of Google-created maps and locational information passively collected from mobile (GPS-enabled) devices, in this article we argue that for such systems to function as both useful tools for exploration for users and sources of revenue, users must (...) actively produce massive quantities of granular spatial data that would otherwise be significantly more difficult and expensive to collect. The production of this qualitative information about places constitutes significant unremunerated affective labour. In this article, we build on the tradition of feminist geographies, especially feminist and critical Geographic Information Systems (GIS), to examine how the labour done by gendered, raced and often classed members of a local community is alienated by Google (i.e. Alphabet, Inc.) to produce commodified spatial data/media in the Local Guides platform. We analyse how Google’s presentation of the platform hails women as care labourers—sharing their thoughts, feelings and knowledge of place for free in the name of supporting and caring for a community, however vaguely it may be constituted. At the same time, we argue that digital labour that produces the Local Guides platform draws from and reproduces specific gendering of spaces. We draw on a case study of a commercial corridor in the US city of Worcester, Massachusetts to show how the dialectic between affective digital labour and urban space has material effects on the production of raced, classed and gendered spaces and places. The article concludes with a call to maintain critical, feminist engagements with these types of platforms in order to further develop forms of digital praxis towards more just cities. (shrink)

In this article, drawing upon a feminist epistemology, we examine the critical roles that philosophical standpoint, historical usage, gender, and language play in a knowledge arena which is increasingly opaque to the general public. Focussing on the language dimension in particular, in its historical and social dimensions, we explicate how some keywords in use across artificial intelligence (AI) discourses inform and misinform non-expert understandings of this area. The insights gained could help to imagine how AI technologies could be better conceptualised, (...) explained, and governed, so that they are leveraged for social good. (shrink)

This article delves into the ongoing consequences of UK ‘Hostile Environment’ policies, notably the Windrush Scandal and the challenges of techno-solutionism in migration governance. There is an exploration of how borders have permeated the internal boundaries of the UK and pushed private citizens and institutions to become new border agents. In this article there is a reflection on the infrastructure that has become reinforced, made visible and technologically upholds Hostile Environment policies. This article investigates the Home Office’s new case working (...) system, Atlas, to illuminate the intersection of border policies, technology and ethics. Through disentangling the political promises placed into the new case working system, this article argues the technological solutions to unjust policies are doomed to repeat and reinforce historic racialised practices. This article argues that technology projects in development, like Atlas, offer an opportunity to identify new private actors responsible for maintaining internal borders within the UK, private technology consultancy groups. Tracing the privatisation of border technology crystallises the new power dynamics introduced through technological projects developed to translate the goals of the Hostile Environment into operational technology used by the Home Office. (shrink)

Efforts to promote equitable public policy with algorithms appear to be fundamentally constrained by the “impossibility of fairness” (an incompatibility between mathematical definitions of fairness). This technical limitation raises a central question about algorithmic fairness: How can computer scientists and policymakers support equitable policy reforms with algorithms? In this article, I argue that promoting justice with algorithms requires reforming the methodology of algorithmic fairness. First, I diagnose the problems of the current methodology for algorithmic fairness, which I call “formal algorithmic (...) fairness.” Because formal algorithmic fairness restricts analysis to isolated decision-making procedures, it leads to the impossibility of fairness and to models that exacerbate oppression despite appearing “fair.” Second, I draw on theories of substantive equality from law and philosophy to propose an alternative methodology, which I call “substantive algorithmic fairness.” Because substantive algorithmic fairness takes a more expansive scope of analysis, it enables an escape from the impossibility of fairness and provides a rigorous guide for alleviating injustice with algorithms. In sum, substantive algorithmic fairness presents a new direction for algorithmic fairness: away from formal mathematical models of “fair” decision-making and toward substantive evaluations of whether and how algorithms can promote justice in practice. (shrink)

This article focuses on emotions, conceptualised as emotional labour, evoked during data practices used to repurpose and enable healthcare data journeys for Finnish public healthcare. Combined approaches from critical data studies and the sociology of emotions were used to contribute to a better understanding of the mundane but often invisible work of the emotions of experts involved in data practices, such as facilitating data journeys and building data technologies. The article is based on a two-and-a-half-year ethnographic study conducted in a (...) Finnish regional public healthcare and social service organisation. The study results were derived from the analysis of 39 interviews and fieldnotes produced by observing 170 h of various meetings, events and work activities performed by experts. The results were organised into three forms of observed experts’ emotional labour related to three phases of healthcare data journeys: caring for data production and preparing data for travel, managing excitement and frustration in data processing for continually building the data management system, and reassuring users in making sense of obtained data analytics. The results contribute to a greater understanding of the emotions and emotional labour generated by healthcare data journeys and in relation to the volatile nature of healthcare data and the collaborative character of data practices. This work advocates for a better recognition of the emotional aspects of data practices and their implications on data-based knowledge and datafication processes in healthcare. (shrink)

Different people have different perceptions about artificial intelligence (AI). It is extremely important to bring together all the alternative frames of thinking—from the various communities of developers, researchers, business leaders, policymakers, and citizens—to properly start acknowledging AI. This article highlights the ‘fruitful collaboration’ that sociology and AI could develop in both social and technical terms. We discuss how biases and unfairness are among the major challenges to be addressed in such a sociotechnical perspective. First, as intelligent machines reveal their nature (...) of ‘magnifying glasses’ in the automation of existing inequalities, we show how the AI technical community is calling for transparency and explainability, accountability and contestability. Not to be considered as panaceas, they all contribute to ensuring human control in novel practices that include requirement, design and development methodologies for a fairer AI. Second, we elaborate on the mounting attention for technological narratives as technology is recognized as a social practice within a specific institutional context. Not only do narratives reflect organizing visions for society, but they also are a tangible sign of the traditional lines of social, economic, and political inequalities. We conclude with a call for a diverse approach within the AI community and a richer knowledge about narratives as they help in better addressing future technical developments, public debate, and policy. AI practice is interdisciplinary by nature and it will benefit from a socio-technical perspective. (shrink)

The paper problematises the reliability and ethics of using social media data, such as sourced from Twitter or Instagram, to carry out health-related research. As in many other domains, the opportunity to mine social media for information has been hailed as transformative for research on well-being and disease. Considerations around the fairness, responsibilities and accountabilities relating to using such data have often been set aside, on the understanding that as long as data were anonymised, no real ethical or scientific issue (...) would arise. We first counter this perception by emphasising that the use of social media data in health research can yield problematic and unethical results. We then provide a conceptualisation of methodological data fairness that can complement data management principles such as FAIR by enhancing the actionability of social media data for future research. We highlight the forms that methodological data fairness can take at different stages of the research process and identify practical steps through which researchers can ensure that their practices and outcomes are scientifically sound as well as fair to society at large. We conclude that making research data fair as well as FAIR is inextricably linked to concerns around the adequacy of data practices. The failure to act on those concerns raises serious ethical, methodological and epistemic issues with the knowledge and evidence that are being produced. (shrink)

In this article, I would like to take a somewhat closer look at the politics of hashtags surrounding wave of street actions known as Black Protest, held nation-wide in Poland on October 2016. Analysing the use of social media as the form of digital activism, I strive at both mitigating the fallacy of digital dualism and demystifying the notion of ‘Twitter revolutions’. The term was popularized by over-enthusiastic accounts of the social movements between 2009 and 2011. I propose to see (...) the employment of social media platforms as the form of weak opposition and to some extent, to explain its efficiency by the ability to reclaim and mobilize the narrative power of hashtags. ‘Weak’ here means everyday, often mundane, and hence under-recognized acts as opposed to activity considered ‘heroic’ and placed in the spotlight, with all gender-based ideological and interpretative undercurrents associated with such a juxtaposition. (shrink)

As natural language processing tools powered by big data become increasingly ubiquitous, questions of how to design, develop, and manage these tools and their impacts on diverse populations are pressing. We propose utilizing the concept of linguistic justice—the realization of equitable access to social and political life regardless of language—to provide a framework for examining natural language processing tools that learn from and use human language data. To support linguistic justice, we argue that natural language processing tools must be examined (...) not only from the perspective of a privileged, majority language user, but also from the perspectives of minoritized language users. Considering such perspectives can help to surface areas in which the data used within natural language processing tools may be working against linguistic justice by failing to provide access to information, services, or opportunities in users’ language of choice, underperforming for certain linguistic groups, or advancing harmful stereotypes that can lead to negative life outcomes for members of marginalized groups. At the same time, this framework can help to illuminate ways that these shortcomings can be addressed and allow us to use inclusive language data and approaches to leverage natural language processing technologies that advance linguistic justice. (shrink)

The United Nations confirmed that privacy remains a human right in the digital age, but our daily digital experiences and seemingly ever-increasing amounts of data suggest that privacy is a mundane, distributed and technologically mediated concept. This article explores privacy by mapping out different legal and conceptual approaches to privacy protection in the context of datafication. It provides an essential starting point to explore the entwinement of technological, ethical and regulatory dynamics. It clarifies why each of the presented approaches emphasises (...) particular aspects and analyses the tensions that arise. The resulting overview provides insight into the main strengths and limitations of the different approaches arising from specific traditions. This analytic overview therefore serves as a key resource to analyse the usefulness of the approaches in the context of the increasing datafication of both private and public spheres.Specifically, we contrast the approach focusing on data subjects whose data are being ‘protected’ with others, including Fair Information Practice Principles, the German right to ‘informational self-determination’, and the South American ‘habeas data’ doctrine. We also present and contrast emerging approaches to privacy and discuss their intersection with datafication. In conclusion, we put forth that rather than aiming for one single solution that works worldwide and across all situations, it is essential to identify synergies and stumbling blocks between the various regulatory settings and newly emerging approaches. (shrink)

This paper explores the important role of critical science, and in particular of post-colonial and decolonial theories, in understanding and shaping the ongoing advances in artificial intelligence. Artificial intelligence is viewed as amongst the technological advances that will reshape modern societies and their relations. While the design and deployment of systems that continually adapt holds the promise of far-reaching positive change, they simultaneously pose significant risks, especially to already vulnerable peoples. Values and power are central to this discussion. Decolonial theories (...) use historical hindsight to explain patterns of power that shape our intellectual, political, economic, and social world. By embedding a decolonial critical approach within its technical practice, AI communities can develop foresight and tactics that can better align research and technology development with established ethical principles, centring vulnerable peoples who continue to bear the brunt of negative impacts of innovation and scientific progress. We highlight problematic applications that are instances of coloniality, and using a decolonial lens, submit three tactics that can form a decolonial field of artificial intelligence: creating a critical technical practice of AI, seeking reverse tutelage and reverse pedagogies, and the renewal of affective and political communities. The years ahead will usher in a wave of new scientific breakthroughs and technologies driven by AI research, making it incumbent upon AI communities to strengthen the social contract through ethical foresight and the multiplicity of intellectual perspectives available to us, ultimately supporting future technologies that enable greater well-being, with the goal of beneficence and justice for all. (shrink)

Questions of trust are increasingly important in relation to data and its use. The authors focus on humanities data and its visualisation, through analysis of their own recent projects with museums, archives and libraries internationally. Their account connects the specifics of hands-on digital humanities work to larger epistemological questions. They discuss the sources of potential mistrust, and examine how different expectations and assumptions emerge depending on the use and user of the data; they offer a simple schema through which the (...) implications may be traced. It is argued that vital issues of trust can be engaged with through design, which, rather than being conceived as a cosmetic finish, is seen as contributing insights and questions that affect the whole process. The article concludes with recommendations intended to be useful in both theory and practice. (shrink)

All datasets emerge from and are enmeshed in power-laden semiotic systems. While emerging data ethics curriculum is supporting data science students in identifying data biases and their consequences, critical attention to the cultural histories and vested interests animating data semantics is needed to elucidate the assumptions and political commitments on which data rest, along with the externalities they produce. In this article, I introduce three modes of reading that can be engaged when studying datasets—a denotative reading, a connotative reading, and (...) a deconstructive reading. I then outline how I have taught students to engage these methods when analyzing three datasets in Data and Society—a course designed to cultivate student competency in politically aware data analysis and interpretation. I show how combined, the reading strategies prompt students to grapple with the double binds of perceiving contemporary problems through systems of representation that are always situated, incomplete, and inflected with diverse politics. While I introduce these methods in the context of teaching, I argue that the methods are integral to any data practice in the conclusion. (shrink)

This essay offers seven intersectional feminist principles for equitable and actionable COVID-19 data, drawing from the authors' prior work on data feminism. Our book, Data Feminism, offers seven principles which suggest possible points of entry for challenging and changing power imbalances in data science. In this essay, we offer seven sets of examples, one inspired by each of our principles, for both identifying existing power imbalances with respect to the impact of the novel coronavirus and its response, and for beginning (...) the work of change. (shrink)

Smart cities build on the idea of collecting data about the city in order for city administration to be operated more efficiently. Within a research project gathering an interdisciplinary team of researchers – engineers, designers, gender scholars and human geographers – we have been working together using participatory design approaches to explore how paying attention to the diversity of human needs may contribute to making urban spaces comfortable and safe for more people. The project team has deployed sensors collecting data (...) on air quality, sound and mobility in a smart city testbed in Norrköping, Sweden. While these sensors are meant to capture an accurate ‘map’ of the street and what is going on along it, our interdisciplinary conversations around the sensors have revealed the heterogeneity both of smart city planning and spatial formulations of the city. The discussions have given rise to questions regarding the work that goes into constructing the sensor box itself, as well as the work of deploying it, and how these influence the ‘map’ that the sensors produce. In this paper, we draw on Lefebvre to explore how the sensors themselves produce smart spaces. We analyze how the box depends on perceived space to function (e.g. requiring electricity), and simultaneously it produces conceptualizations of space that are influenced by the materiality of the box itself (e.g. sensors being affected by heat and noise). Further, we explore how the (in)visibility of sensor technology influences lived space. (shrink)

The social impacts of computer technology are often glorified in public discourse, but there is growing concern about its actual effects on society. In this article, we ask: how does “consent” as an analytical framework make visible the social dynamics and power relations in the capture, extraction, and labor of data science knowledge production? We hypothesize that a form of boundary violation in data science workplaces—gender harassment—may correlate with the ways humans’ lived experiences are extracted to produce Big Data. The (...) concept of consent offers a useful way to draw comparisons between gender relations in data science and the means by which machines are trained to learn and reason. Inspired by how Big Tech leaders describe unsupervised machine learning, and the co-optation of “revolutionary” rhetoric they use to do so, we introduce a concept we call “techniques of invisibility.” Techniques of invisibility are the ways in which an extreme imbalance between exposure and opacity, demarcated along fault lines of power, are fabricated and maintained, closing down the possibility for bidirectional transparency in the production and applications of algorithms. Further, techniques of invisibility, which we group into two categories—epistemic injustice and the Brotherhood—include acts of subjection by powerful actors in data science designed to quell resistance to exploitative relations. These techniques may be useful in making further connections between epistemic violence, sexism, and surveillance, sussing out persistent boundary violations in data science to render the social in data science visible, and open to scrutiny and debate. (shrink)

As heritage-as-the-already-occurred folds into heritage-in-the-making practices, temporal and spatial fluidity is made more complex by digital mediation and particularly by Big Data. Such liveliness evokes ontological, epistemological and methodological challenges. Drawing on more-than-human theorizing, this article reframes the notion of data-bodies to advance data activist-oriented research in heritage. Focused primarily on women, it examines how their distributed agency and voice with respect to data practices and the makings of heritage could be amplified. I describe three methodological directions, influenced by feminist (...) work in critical data studies, which could be employed by researchers: attuning to and becoming with data, making data physical and changing narratives. From data-bodies to haunted data, performative data curation and mapping data-bodies, and attuning to data streams and re-voicing narratives, this article contributes to discussions of how to engage critically and creatively with the datafication of digital heritage practices, knowings and ontologies. (shrink)

During 2018, as part of a research project funded by the Deviant Practice Grant, artist Bruno Moreschi and digital media researcher Gabriel Pereira worked with the Van Abbemuseum collection (Eindhoven, NL), reading their artworks through commercial image-recognition (computer vision) artificial intelligences from leading tech companies. The main takeaways were: somewhat as expected, AI is constructed through a capitalist and product-focused reading of the world (values that are embedded in this sociotechnical system); and that this process of using AI is an (...) innovative way for doing institutional critique, as AI offers an untrained eye that reveals the inner workings of the art system through its glitches. This paper aims to regard these glitches as potentially revealing of the art system, and even poetic at times. We also look at them as a way of revealing the inherent fallibility of the commercial use of AI and machine learning to catalogue the world: it cannot comprehend other ways of knowing about the world, outside the logic of the algorithm. But, at the same time, due to their “glitchy” capacity to level and reimagine, these faulty readings can also serve as a new way of reading art; a new way for thinking critically about the art image in a moment when visual culture has changed form to hybrids of human–machine cognition and “machine-to-machine seeing”. (shrink)

Although databases have been well-defined and thoroughly discussed in the computer science literature, the actual users of databases often have varying definitions and expectations of this essential computational infrastructure. Systems administrators and computer science textbooks may expect databases to be instantiated in a small number of technologies, but there are numerous examples of databases in non-conventional or unexpected technologies, such as spreadsheets or other assemblages of files linked through code. Consequently, we ask: How do the materialities of non-conventional databases differ (...) from or align with the materialities of conventional relational systems? What properties of the database do the creators of these artifacts invoke in their rhetoric describing these systems—or in the data models underlying these digital objects? To answer these questions, we conducted a close analysis of four non-conventional scientific databases. By examining the materialities of information representation in each case, we show how scholarly communication regimes shape database materialities—and how information organization paradigms shape scholarly communication. These cases show abandonment of certain constraints of relational database construction alongside maintenance of some key relational data organization strategies. We discuss the implications that these relational data paradigms have for data use, preservation, and sharing, and discuss the need to support a plurality of data practices and paradigms. (shrink)

Volume 20, Issue 10, October 2020, Page 1-4.

Human activities are being increasingly regulated by means of technologies. Smart borders regulating human movement are no exception. I argue that the process of digitization – including through AI, Big Data and algorithmic processing – falls short of respecting (fundamental) rights to the extent to which it ignores what I term to be the problem of indeterminacy. While adopting a data feminist approach in this paper, assuming that data is the ‘new oil’, that is power, I begin theorizing indeterminacy from (...) the imminent risks of datafication as a new instrument of oppression perpetuating injustice and widening inequality gaps. I conclude that technologies regulating human activities must stand ethical scrutiny, especially if they can and do result in (human) rights violations. Unlike the oil being extracted from the ground, data is de facto extracted from people endowed with agency, autonomy, rights and contexts – all which ought to be respected and protected. (shrink)