With the increase in the costs of providing education and concerns about financial responsibility, heightened consideration of accountability and results, elevated awareness of the range of teacher skills and student learning styles and needs, more focus is being placed on the promises offered by online software and educational technology. One of the most heavily marketed, exciting and controversial applications of edtech involves the varied educational programs to which different students are exposed based on how big data applications have evaluated their (...) likely learning profiles. Characterized most often as ‘personalized learning,’ these programs raise a number of ethical concerns especially when used at the K-12 level. This paper analyzes the range of these ethical concerns arguing that characterizing them under the general rubric of ‘privacy’ oversimplifies the concerns and makes it too easy for advocates to dismiss or minimize them. Six distinct ethical concerns are identified: information privacy; anonymity; surveillance; autonomy; non-discrimination; and ownership of information. Particular attention is paid to whether personalized learning programs raise concerns similar to those raised about educational tracking in the 1950s. The paper closes with discussion of three themes that are important to consider in ethical and policy discussions. (shrink)

The use of algorithmic prediction in insurance is regarded as the beginning of a new era, because it promises to personalise insurance policies and premiums on the basis of individual behaviour and level of risk. The core idea is that the price of the policy would no longer refer to the calculated uncertainty of a pool of policyholders, with the consequence that everyone would have to pay only for her real exposure to risk. For insurance, however, uncertainty is not only (...) a problem – shared uncertainty is a resource. The availability of individual risk information could undermine the principle of risk-pooling and risk-spreading on which insurance is based. The article examines this disruptive change first by exploring the possible consequences of the use of predictive algorithms to set insurance premiums. Will it endanger the principle of mutualisation of risks, producing new forms of discrimination and exclusion from coverage? In a second step, we analyse how the relationship between the insurer and the policyholder changes when the customer knows that the company has voluminous, and continuously updated, data about her real behaviour. (shrink)

The Snowden revelations about National Security Agency surveillance, starting in 2013, along with the ambiguous complicity of internet companies and the international controversies that followed provide a perfect segue into contemporary conundrums of surveillance and Big Data. Attention has shifted from late C20th information technologies and networks to a C21st focus on data, currently crystallized in “Big Data.” Big Data intensifies certain surveillance trends associated with information technology and networks, and is thus implicated in fresh but fluid configurations. This is (...) considered in three main ways: One, the capacities of Big Data intensify surveillance by expanding interconnected datasets and analytical tools. Existing dynamics of influence, risk-management, and control increase their speed and scope through new techniques, especially predictive analytics. Two, while Big Data appears to be about size, qualitative change in surveillance practices is also perceptible, accenting consequences. Important trends persist – the control motif, faith in technology, public-private synergies, and user-involvement – but the future-orientation increasingly severs surveillance from history and memory and the quest for pattern-discovery is used to justify unprecedented access to data. Three, the ethical turn becomes more urgent as a mode of critique. Modernity's predilection for certain definitions of privacy betrays the subjects of surveillance who, so far from conforming to the abstract, disembodied image of both computing and legal practices, are engaged and embodied users-in-relation whose activities both fuel and foreclose surveillance. (shrink)

Whereas using artificial intelligence (AI) to predict natural hazards is promising, applying a predictive policing algorithm (PPA) to predict human threats to others continues to be debated. Whereas PPAs were reported to be initially successful in Germany and Japan, the killing of Black Americans by police in the US has sparked a call to dismantle AI in law enforcement. However, although PPAs may statistically associate suspects with economically disadvantaged classes and ethnic minorities, the targeted groups they aim to protect are (...) often vulnerable populations as well (e.g., victims of human trafficking, kidnapping, domestic violence, or drug abuse). Thus, determining how to enhance the benefits of PPA while reducing bias through better management is important. In this paper, we propose a policy schema to address this issue. First, after clarifying relevant concepts, we examine major criticisms of PPAs and argue that some of them should be addressed. If banning AI or making it taboo is an unrealistic solution, we must learn from our errors to improve AI. We next identify additional challenges of PPAs and offer recommendations from a policy viewpoint. We conclude that the employment of PPAs should be merged into broader governance of the social safety net and audited publicly by parliament and civic society so that the unjust social structure that breeds bias can be revised. (shrink)

Through the exponential growth in digital devices and computational capabilities, big data technologies are putting pressure upon the boundaries of what can or cannot be considered acceptable from an ethical perspective. Much of the literature on ethical issues related to big data and big data technologies focuses on separate values such as privacy, human dignity, justice or autonomy. More holistic approaches, allowing a more comprehensive view and better balancing of values, usually focus on either a design-based approach, in which it (...) is tried to implement values into the design of new technologies, or an application-based approach, in which it is tried to address the ways in which new technologies are used. Some integrated approaches do exist, but typically are more general in nature. This offers a broad scope of application, but may not always be tailored to the specific nature of big data related ethical issues. In this paper we distil a comprehensive set of ethical values from existing design-based and application-based ethical approaches for new technologies and further focus these values to the context of emerging big data technologies. A total of four value lists were selected for this. The integrated list consists of a total of ten values: human welfare, autonomy, non-maleficence, justice, accountability, trustworthiness, privacy, dignity, solidarity and environmental welfare. Together, this set of values provides a comprehensive and in-depth overview of the values that are to be taken into account for emerging big data technologies. (shrink)

Data science is not simply a method but an organising idea. Commitment to the new paradigm overrides concerns caused by collateral damage, and only a counterculture can constitute an effective critique. Understanding data science requires an appreciation of what algorithms actually do; in particular, how machine learning learns. The resulting ‘insight through opacity’ drives the observable problems of algorithmic discrimination and the evasion of due process. But attempts to stem the tide have not grasped the nature of data science as (...) both metaphysical and machinic. Data science strongly echoes the neoplatonism that informed the early science of Copernicus and Galileo. It appears to reveal a hidden mathematical order in the world that is superior to our direct experience. The new symmetry of these orderings is more compelling than the actual results. Data science does not only make possible a new way of knowing but acts directly on it; by converting predictions to pre-emptions, it becomes a machinic metaphysics. The people enrolled in this apparatus risk an abstraction of accountability and the production of ‘thoughtlessness’. Susceptibility to data science can be contested through critiques of science, especially standpoint theory, which opposes the ‘view from nowhere’ without abandoning the empirical methods. But a counterculture of data science must be material as well as discursive. Karen Barad’s idea of agential realism can reconfigure data science to produce both non-dualistic philosophy and participatory agency. An example of relevant praxis points to the real possibility of ‘machine learning for the people’. (shrink)

The objective of this work is to analyse the phenomena and the implications of hyperglobalization on current companies. To introduce this purpose, we are going to review the different processes of global economic cohesion produced throughout history. We will focus on the hyperglobalization stage and on the effect that hyperconnectivity has had for companies and their operation. We will analyze the phenomenon of big data, his new technological innovations and how they have affected the companies act and the global society. (...) Finally, we will present an ethical analysis of these new technological phenomena and their implications for society. (shrink)

The distinction between distributive and non-distributive profiles figures prominently in current evaluations of the ethical and epistemological risks that are associated with automated profiling practices. The diagnosis that non-distributive profiles may coincidentally situate an individual in the wrong category is often perceived as the central shortcoming of such profiles. According to this diagnosis, most risks can be retraced to the use of non-universal generalisations and various other statistical associations. This article develops a top-down analysis of non-distributive profiles in which this (...) fallibility of non-distributive profiles is no longer central. Instead, it focuses on how profiling creates various asymmetries between an individual data-subject and a profiler. The emergence of informational, interest, and perspectival asymmetries between data-subject and profiler explains how non-distributive profiles weaken the epistemic position of a profiled individual. This alternative analysis provides a more balanced assessment of the epistemic risks associated with non-distributive profiles. (shrink)

Within the political economy of informational capitalism, commercial surveillance practices are tools for resource extraction. That process requires an enabling legal construct, which this essay identifies and explores. Contemporary practices of personal information processing constitute a new type of public domain—a repository of raw materials that are there for the taking and that are framed as inputs to particular types of productive activity. As a legal construct, the biopolitical public domain shapes practices of appropriation and use of personal information in (...) two complementary and interrelated ways. First, it constitutes personal information as available and potentially valuable: as a pool of materials that may be freely appropriated as inputs to economic production. That framing supports the reorganization of sociotechnical activity in ways directed toward extraction and appropriation. Second, the biopolitical public domain constitutes the personal information harvested within networked information environments as raw. That framing creates the backdrop for culturally situated techniques of knowledge production and for the logic that designates those techniques as sites of legal privilege. (shrink)

How should we understand the surveillance state post Snowden? This paper is concerned with the relationship between increased surveillance capacity and state power. The paper begins by analysing two metaphors used in public post Snowden discourse to describe state surveillance practices: the haystack and the panopticon. It argues that these metaphors share a flawed common entailment regarding surveillance, knowledge and power which cannot accurately capture important aspects of state anxiety generated by mass surveillance in an age of big data. The (...) paper shows that the nature of big data itself complicates the power attributed to mass surveillance states by these metaphors and those who use them. Relying heavily on Ezrahi’s distinction between information and knowledge, the paper situates this argument concerning the state and anxiety borne of information overload in the context of literature that concerns the state and information management. Drawing primarily on James Scott’s work on legibility, it argues that the big data born of mass surveillance problematises the concept of information as empowering the state. Instead, understanding mass surveillance in an age of big data requires understanding the relationship between the surveillance state and information in terms of anxiety as well as power. (shrink)

In a time of big data, thinking about how we are seen and how that affects our lives means changing our idea about who does the seeing. Data produced by machines is most often ‘seen’ by other machines; the eye is in question is algorithmic. Algorithmic seeing does not produce a computational panopticon but a mechanism of prediction. The authority of its predictions rests on a slippage of the scientific method in to the world of data. Data science inherits some (...) of the problems of science, especially the disembodied ‘view from above’, and adds new ones of its own. As its core methods like machine learning are based on seeing correlations not understanding causation, it reproduces the prejudices of its input. Rising in to the apparatuses of governance, it reinforces the problematic sides of ‘seeing like a state’ and links to the recursive production of paranoia. It forces us to ask the question ‘what counts as rational seeing?’. Answering this from a position of feminist empiricism reveals different possibilities latent in seeing with machines. Grounded in the idea of conviviality, machine learning may reveal forgotten non-market patterns and enable free and critical learning. It is proposed that a programme to challenge the production of irrational pre-emption is also a search for the possibility of algorithmic conviviality. (shrink)

Building on conceptual insights from the history and sociology of numbers, media and surveillance studies, and theories of governance and risk, this article analyzes the forms of transparency produced by the use of numbers in social life. It examines what it is about numbers that often makes their ‘truth claims’ so powerful, investigates the role that numerical operations play in the production of retrospective, real-time and anticipatory forms of transparency in contemporary politics and economic transactions, and discusses some of the (...) implications resulting from the increasingly abstract and machine-driven use of numbers. It argues that the forms of transparency generated by machine-driven numerical operations open up for individual and collective practices in ways that are intimately linked to precautionary and pre-emptive aspirations and interventions characteristic of contemporary governance. As such, these numerical operations raise important political and ethical questions that deserve further conceptual and empirical scrutiny. (shrink)