Abstract: Cheating behaviors have been construed as a continuing and somewhat vexing issue for academic institutions as they increasingly conduct educational processes online and impose metrics on instructional evaluation. Research, development, and implementation initiatives on cheating detection have gained new dimensions in the advent of artificial intelligence (AI) applications; they have also engendered special challenges in terms of their social, ethical, and cultural implications. An assortment of commercial cheating–detection systems have been injected into educational contexts with little input on the (...) part of relevant stakeholders. This paper expands several specific cases of how systems for the detection of cheating have recently been implemented in higher education institutions in the US and UK. It investigates how such vehicles as wearable technologies, eye scanning, and keystroke capturing are being used to collect the data used for anti-cheating initiatives, often involving systems that have not gone through rigorous testing and evaluation for their validity and potential educational impacts. The paper discusses accountability- and policy-related issues concerning the outsourcing of cheating detection in institutional settings in the light of these emerging technological practices as well as student resistance against the systems involved. The practices can place students in a disempowered position. (shrink)

Abstract: Cheating behaviors have been construed as a continuing and somewhat vexing issue for academic institutions as they increasingly conduct educational processes online and impose metrics on instructional evaluation. Research, development, and implementation initiatives on cheating detection have gained new dimensions in the advent of artificial intelligence (AI) applications; they have also engendered special challenges in terms of their social, ethical, and cultural implications. An assortment of commercial cheating–detection systems have been injected into educational contexts with little input on the (...) part of relevant stakeholders. This paper expands several specific cases of how systems for the detection of cheating have recently been implemented in higher education institutions in the US and UK. It investigates how such vehicles as wearable technologies, eye scanning, and keystroke capturing are being used to collect the data used for anti-cheating initiatives, often involving systems that have not gone through rigorous testing and evaluation for their validity and potential educational impacts. The paper discusses accountability- and policy-related issues concerning the outsourcing of cheating detection in institutional settings in the light of these emerging technological practices as well as student resistance against the systems involved. The cheating-detection practices can place students in a disempowered, asymmetrical position that is often at substantial variance with their cultural backgrounds. (shrink)

The central thesis of this paper is that human unconscious processes influence the behavior and design of artificial intelligence (AI). This thesis is discussed through the case study of a chatbot called Replika, which intends to provide psychological assistance and friendship but has been accused of inciting murder and suicide. Replika originated from a trauma and a work of mourning lived by its creator. The traces of these unconscious dynamics can be detected in the design of the app and the (...) narratives about it. Therefore, a process of de-psychologization and de-humanization of the unconscious takes place through AI. This psychosocial approach helps criticize and overcome the so-called “standard model of intelligence” shared by most AI researchers. It facilitates a new interpretation of some classic problems in AI, such as control and responsibility. (shrink)

In light of the recent emergence of predictive techniques in law enforcement to forecast crimes before they occur, this paper examines the temporal operation of power exercised by predictive policing algorithms. I argue that predictive policing exercises power through a paranoid style that constitutes a form of temporal governmentality. Temporality is especially pertinent to understanding what is ethically at stake in predictive policing as it is continuous with a historical racialized practice of organizing, managing, controlling, and stealing time. After first (...) clarifying the concept of temporal governmentality, I apply this lens to Chicago Police Department’s Strategic Subject List. This predictive algorithm operates, I argue, through a paranoid logic that aims to preempt future possibilities of crime on the basis of a criminal past codified in historical crime data. (shrink)

AI technology is capturing the African imaginations as a gateway to progress and prosperity. There is a growing interest in AI by different actors across the continent including scientists, researchers, humanitarian and aid organizations, academic institutions, tech start-ups, and media organizations. Several African states are looking to adopt AI technology to capture economic growth and development opportunities. On the other hand, African researchers highlight the gap in regulatory frameworks and policies that govern the development of AI in the continent. They (...) argue that this could lead to AI technology exacerbating problems of inequalities and injustice in the continent. However, most of the literature on AI ethics is biased toward Euro-American perspectives and lack the understanding of how AI development is apprehended in the Global South, and particularly Africa. Drawing on the case study of the first African Master’s in Machine Intelligence program, this paper argues for looking beyond the question of ethics in AI and examining AI governance issues through the analytical lens of the raciality of computing and the political economy of technoscience to understand AI development in Africa. By doing so, this paper seeks a different theorization for AI ethics from the South that is based on lived experiences of those in the margins and avoids the framings of technological futures that simplistically pathologize or celebrate Africa. (shrink)

Algorithms have developed into somewhat of a modern myth. On the one hand, they have been depicted as powerful entities that rule, sort, govern, shape, or otherwise control our lives. On the other hand, their alleged obscurity and inscrutability make it difficult to understand what exactly is at stake. What sustains their image as powerful yet inscrutable entities? And how to think about the politics and governance of something that is so difficult to grasp? This editorial essay provides a critical (...) backdrop for the special issue, treating algorithms not only as computational artifacts but also as sensitizing devices that can help us rethink some entrenched assumptions about agency, transparency, and normativity. (shrink)

A common theme in social science studies of algorithms is that they are profoundly opaque and function as “black boxes.” Scholars have developed several methodological approaches in order to address algorithmic opacity. Here I argue that we can explicitly enroll algorithms in ethnographic research, which can shed light on unexpected aspects of algorithmic systems—including their opacity. I delineate three meso-level strategies for algorithmic ethnography. The first, algorithmic refraction, examines the reconfigurations that take place when computational software, people, and institutions interact. (...) The second strategy, algorithmic comparison, relies on a similarity-and-difference approach to identify the instruments’ unique features. The third strategy, algorithmic triangulation, enrolls algorithms to help gather rich qualitative data. I conclude by discussing the implications of this toolkit for the study of algorithms and future of ethnographic fieldwork. (shrink)

Este artículo explora las consideraciones éticas que rodean el uso de la Inteligencia Artificial (IA) en la academia. Se establecen principios éticos generales para la IA en este ámbito, como la transparencia, la equidad, la responsabilidad, la privacidad y la integridad académica. En cuanto a la educación asistida por IA, se enfatiza la importancia de la accesibilidad, la no discriminación y la evaluación crítica de los resultados. Se recomienda que la IA se use para complementar y no para reemplazar la (...) enseñanza tradicional. En cuanto al ámbito de la investigación, se hace énfasis en la necesidad de garantizar la calidad y la veracidad de los datos utilizados, así como la transparencia en los métodos y resultados. Se deben considerar los posibles sesgos y riesgos de la IA en la investigación. El artículo concluye que la adopción responsable de la IA en la academia requiere una reflexión crítica sobre sus implicaciones éticas. Se recomienda la creación de marcos éticos específicos para cada contexto y la participación de diversos actores en la toma de decisiones. (shrink)

This article deals with choice-inducing algorithms––algorithms that are explicitly designed to affect people’s choices. Based on an ethnographic account of three Israeli data analytics companies, I explore how algorithms are being designed to drive people into choice-making and examine their co-constitution by an assemblage of specifically positioned human and nonhuman agents. I show that the functioning, logic, and even ethics of choice-inducing algorithms are deeply influenced by the epistemologies, meaning systems, and practices of the individuals who devise and use them (...) and that such algorithms are similarly affected by interorganizational relationships, various nonhuman agents, and changing geopolitical contexts. I conclude by discussing the flexibility of choice-inducing algorithms and by arguing that such algorithms are not programmed to induce specific choices but to more generally convert people into choosers, and thus, to algorithmically create the modern need to choose. This article contributes to the growing literature on algorithms and culture and to our understanding of choice-making in contemporary life. At the same time, it provides a new vocabulary that offers to critically engage with algorithms and their power without losing sight of the often very specific contexts from which they arise. (shrink)

Algorithms, once obscure objects of technical art, have lately been subject to considerable popular and scholarly scrutiny. What does it mean to adopt the algorithm as an object of analytic attention? What is in view, and out of view, when we focus on the algorithm? Using Niklaus Wirth's 1975 formulation that “algorithms + data structures = programs” as a launching-off point, this paper examines how an algorithmic lens shapes the way in which we might inquire into contemporary digital culture.

From ‘connecting the dots’ and finding ‘the needle in the haystack’ to predictive policing and data mining for counterinsurgency, security professionals have increasingly adopted the language and methods of computing for the purposes of prediction. Digital devices and big data appear to offer answers to a wide array of problems of (in)security by promising insights into unknown futures. This article investigates the transformation of prediction today by placing it within governmental apparatuses of discipline, biopower and big data. Unlike disciplinary and (...) biopolitical governmentality, we argue that prediction with big data is underpinned by the production of a different time/space of ‘between-ness’. The digital mode of prediction with big data reconfigures how we are governed today, which we illustrate through an analysis of how predictive policing actualizes between-ness as hotspots and near-real-time decisions. (shrink)

Scholars and practitioners across domains are increasingly concerned with algorithmic transparency and opacity, interrogating the values and assumptions embedded in automated, black-boxed systems, particularly in user-generated content platforms. I report from an ethnography of infrastructure in Wikipedia to discuss an often understudied aspect of this topic: the local, contextual, learned expertise involved in participating in a highly automated social–technical environment. Today, the organizational culture of Wikipedia is deeply intertwined with various data-driven algorithmic systems, which Wikipedians rely on to help manage (...) and govern the “anyone can edit” encyclopedia at a massive scale. These bots, scripts, tools, plugins, and dashboards make Wikipedia more efficient for those who know how to work with them, but like all organizational culture, newcomers must learn them if they want to fully participate. I illustrate how cultural and organizational expertise is enacted around algorithmic agents by discussing two autoethnographic vignettes, which relate my personal experience as a veteran in Wikipedia. I present thick descriptions of how governance and gatekeeping practices are articulated through and in alignment with these automated infrastructures. Over the past 15 years, Wikipedian veterans and administrators have made specific decisions to support administrative and editorial workflows with automation in particular ways and not others. I use these cases of Wikipedia’s bot-supported bureaucracy to discuss several issues in the fields of critical algorithms studies; critical data studies; and fairness, accountability, and transparency in machine learning—most principally arguing that scholarship and practice must go beyond trying to “open up the black box” of such systems and also examine sociocultural processes like newcomer socialization. (shrink)

Algorithms silently structure our lives. Algorithms can determine whether someone is hired, promoted, offered a loan, or provided housing as well as determine which political ads and news articles consumers see. Yet, the responsibility for algorithms in these important decisions is not clear. This article identifies whether developers have a responsibility for their algorithms later in use, what those firms are responsible for, and the normative grounding for that responsibility. I conceptualize algorithms as value-laden, rather than neutral, in that algorithms (...) create moral consequences, reinforce or undercut ethical principles, and enable or diminish stakeholder rights and dignity. In addition, algorithms are an important actor in ethical decisions and influence the delegation of roles and responsibilities within these decisions. As such, firms should be responsible not only for the value-laden-ness of an algorithm but also for designing who-does-what within the algorithmic decision. As such, firms developing algorithms are accountable for designing how large a role individual will be permitted to take in the subsequent algorithmic decision. Counter to current arguments, I find that if an algorithm is designed to preclude individuals from taking responsibility within a decision, then the designer of the algorithm should be held accountable for the ethical implications of the algorithm in use. (shrink)

The discourse around self-driving cars has been dominated by an emphasis on their potential to reduce the number of accidents. At the same time, proponents acknowledge that self-driving cars would inevitably be involved in fatal accidents where moral algorithms would decide the fate of those involved. This is a necessary trade-off, proponents suggest, in order to reap the benefits of this new technology. In this article, I engage this argument, demonstrating how an undue optimism and enthusiasm about this technology is (...) obscuring our ability to see what is at stake and explaining how moving beyond the dominant utilitarian framings around this technology opens up a space for both ethical inquiry and innovative design. I suggest that a genuine caring concern for the many lives lost in car accidents now and in the future—a concern that transcends false binary trade-offs and that recognizes the systemic biases and power structures that make certain groups more vulnerable than others—could serve as a starting point to rethink mobility, as it connects to the design of our cities, the well-being of our communities, and the future of our planet. (shrink)

Governments and citizens of nearly every nation have been compelled to respond to COVID-19. Many measures have been adopted, including contact tracing and risk assessment algorithms, whereby citizen whereabouts are monitored to trace contact with other infectious individuals in order to generate a risk status via algorithmic evaluation. Based on 38 in-depth interviews, we investigate how people make sense of Health Code, the Chinese contact tracing and risk assessment algorithmic sociotechnical assemblage. We probe how people accept or resist Health Code (...) by examining their ongoing, dynamic, and relational interactions with it. Participants display a rich variety of attitudes toward privacy and surveillance, ranging from fatalism to the possibility of privacy to trade-offs for surveillance in exchange for public health, which is mediated by the perceived effectiveness of Health Code and changing views on the intentions of institutions who deploy it. We show how perceived competency varies not just on how well the technology works, but on the social and cultural enforcement of various non-technical aspects like quarantine, citizen data inputs, and cell reception. Furthermore, we illustrate how perceptions of Health Code are nested in people’s broader interpretations of disease control at the national and global level, and unexpectedly strengthen the Chinese authority’s legitimacy. None of the Chinese public, Health Code, or people’s perceptions toward Health Code are predetermined, fixed, or categorically consistent, but are co-constitutive and dynamic over time. We conclude with a theorization of a relational perception and methodological reflections to study algorithmic sociotechnical assemblages beyond COVID-19. (shrink)

Organizations increasingly rely on algorithm-based HR decision-making to monitor their employees. This trend is reinforced by the technology industry claiming that its decision-making tools are efficient and objective, downplaying their potential biases. In our manuscript, we identify an important challenge arising from the efficiency-driven logic of algorithm-based HR decision-making, namely that it may shift the delicate balance between employees’ personal integrity and compliance more in the direction of compliance. We suggest that critical data literacy, ethical awareness, the use of participatory (...) design methods, and private regulatory regimes within civil society can help overcome these challenges. Our paper contributes to literature on workplace monitoring, critical data studies, personal integrity, and literature at the intersection between HR management and corporate responsibility. (shrink)

The power of the digital platforms and the increasing scope of their control over individuals and institutions have begun to generate societal concern. However, the ways in which digital platforms exercise power and organize immaturity—defined as the erosion of the individual’s capacity for public use of reason—have not yet been theorized sufficiently. Drawing on Bourdieu’s concepts of field, capitals, and habitus, we take a sociosymbolic perspective on platforms’ power dynamics, characterizing the digital habitus and identifying specific forms of platform power (...) and counterpower accumulation. We make two main contributions. First, we expand the concept of organized immaturity by adopting a sociological perspective, from which we develop a novel sociosymbolic view of platforms’ power dynamics. Our framework explains fundamental aspects of immaturity, such as self-infliction and emergence. Second, we contribute to the platform literature by developing a three-phase model of platform power dynamics over time. (shrink)

Two definitions of algorithm, their uses, and their implied models of computing in society, are reviewed. The first, termed the structural programming definition, aligns more with usage in computer science, and as the name suggests, the intellectual project of structured programming. The second, termed the systemic definition, is more informal and emerges from ethnographic observations of discussions of software in both professional and everyday settings. Specific examples of locating algorithms within modern codebases are shared, as well as code directly impacting (...) social and ethical concerns. The structural distinction between algorithms and social concerns is explained as mirroring the engineering construct of algorithms and data structures. It is proposed that, rather than this separation being an attempt to enforce a professional boundary and evade social responsibility, it is a crucial technical distinction within code which makes it clearer and more transparent. The power structures reinforced by the broader, cultural interpretations of algorithm are reconsidered, along with what it would mean for software to have an inclusive design culture. (shrink)

Information security governance has become an elusive goal and a murky concept. This paper problematizes both information security governance and the broader concept of governance. What does it mean to govern information security, or for that matter, anything? Why have information technologies proven difficult to govern? And what assurances can governance provide for the billions of people who rely on information technologies every day? Drawing together several distinct bodies of literature—including multiple strands of governance theory, actor–network theory, and scholarship on (...) sociotechnical regimes—this paper conceptualizes networked action on a spectrum from uncertain governance to governing uncertainty. I advance a twofold argument. First, I argue that networks can better govern uncertainty as they become more able not only to enroll actors in a collective agenda, but also to cut ties with those who seek to undermine that agenda. And second, I argue that the dominant conception of information security governance, which emphasizes governing uncertainty through risk management, in practice devolves to uncertain governance. This is largely because information technologies have evolved toward greater connectedness—and with it, greater vulnerability—creating a regime of insecurity. This evolution is illustrated using the history of the US government’s efforts to govern information security. (shrink)

Metrics have long served as tools for governing at a distance. In the food industry, major manufacturers have embraced metrics as tools to govern the sustainability of the farms producing their commodity raw materials. This metrical turn has been influenced but also complicated by agricultural datafication, that is, the increasing quantities of data generated on and about farms. Despite the sheer abundance of data that companies might use to measure and drive improvement in on-farm sustainability, they have struggled to collect (...) data suitable for such purposes. Attention to the different kinds of distance and diversity across which metrics are supposed to govern suggests reasons why they may fail to do so, even when wielded by otherwise powerful corporations. (shrink)

The article analyzes the relation of humans and technology concerning so called ‘intelligent’ or ‘autonomous’ algorithms that are applied in everyday contexts but are far removed from any form of substantial artificial intelligence. In particular, the use of algorithms in surveillance and in architecture is discussed. These examples are structured by a particular combination of continuity and difference between humans and technology. The article provides a detailed analysis of boundary practices that establish continuity and oppositions between humans and information technology, (...) referring to their exemplary depiction in movies. Both strands of boundary practices have the potential to challenge as well as sustain the position of the human as liberal, autonomous subject. Finally, it is shown how the particular combination of continuity and difference that structures the use of algorithms maintains the power of liberal, autonomous subject positions, while the shift of decisions to the algorithms seems to decenter the human. (shrink)

Choice is a sine qua non of contemporary life. From childhood until death, we are faced with an unending series of choices through which we cultivate a sense of self, govern conduct, and shape the future. Nowadays, individuals increasingly experience and enact consumer choice online through web-based platforms such as Yelp.com, TripAdvisor.com and Amazon.com. These platforms not only provide a sprawling array of goods and services to choose from, but also reviews, ratings and ranking devices and systems of classification to (...) navigate this landscape of choice. This paper suggests a radical reconsideration of platform architectures and design features to consider how they reconfigure and respecify choice, ‘choosers’, and choice-making practices. Platforms are not simply cameras that present choice and enable comparisons between different options, but are more akin to engines that govern, drive and expand choice, configuring users within particular discourses, practices and subjectivities. In making sense of the entangled trajectories of consumer choice, platform architectures and Big Data, I suggest that ‘hyper-choice’ emerges as a condition of the contemporary platform-driven web. I examine hyper-choice not only in terms of the relationship between platforms and a growing abundance of choice, but more importantly how platforms reconfigure choice in ways that go beyond and fundamentally challenge existing understandings of what choice is, who and what is involved in producing knowledge about choice, and what it means to be a ‘chooser’. (shrink)