This article analyzes US drone operations through a historical and ethnographic analysis of the remote split paradigm used by the US Air Force. Remote split refers to the globally distributed command and control of drone operations and entails a network of human operators and analysts in the Middle East, Europe, and Southeast Asia as well as in the continental United States. Though often viewed as a teleological progression of “unmanned” warfare, this paper argues that historically specific technopolitical logics establish the (...) conditions of possibility for the work of war to be divisible into discreet and computationally mediated tasks that are viewed as effective in US military engagements. To do so, the article traces how new forms of authorized evidence and expertise have shaped developments in military operations and command and control priorities from the Cold War and the “electronic battlefield” of Vietnam through the Gulf War and the conflict in the Balkans to contemporary deployments of drone operations. The article concludes by suggesting that it is by paying attention to divisions of labor and human–machine configurations that we can begin to understand the everyday and often invisible structures that sustain perpetual war as a military strategy of the United States. (shrink)

The international debate on the ethics and legality of autonomous weapon systems (AWS), along with the call for a ban, primarily focus on the nebulous concept of fully autonomous AWS. These are AWS capable of target selection and engagement absent human supervision or control. This paper argues that such a conception of autonomy is divorced from both military planning and decision-making operations; it also ignores the design requirements that govern AWS engineering and the subsequent tracking and tracing of moral responsibility. (...) To show how military operations can be coupled with design ethics, this paper marries two different kinds of meaningful human control (MHC) termed levels of abstraction. Under this two-tiered understanding of MHC, the contentious notion of ‘full’ autonomy becomes unproblematic. (shrink)

Value Sensitive Design (VSD) is an established method for integrating values into technical design. It has been applied to different technologies and, more recently, to artificial intelligence (AI). We argue that AI poses a number of challenges specific to VSD that require a somewhat modified VSD approach. Machine learning (ML), in particular, poses two challenges. First, humans may not understand how an AI system learns certain things. This requires paying attention to values such as transparency, explicability, and accountability. Second, ML (...) may lead to AI systems adapting in ways that ‘disembody’ the values embedded in them. To address this, we propose a threefold modified VSD approach: 1) integrating a known set of VSD principles (AI4SG) as design norms from which more specific design requirements can be derived; 2) distinguishing between values that are promoted and respected by the design to ensure outcomes that not only do no harm but also contribute to good; and 3) extending the VSD process to encompass the whole life cycle of an AI technology in order to monitor unintended value consequences and redesign as needed. We illustrate our VSD for AI approach with an example use case of a SARS-CoV-2 contact tracing app. (shrink)

According to the Value-Neutrality Thesis, technology is morally and politically neutral, neither good nor bad. A knife may be put to bad use to murder an innocent person or to good use to peel an apple for a starving person, but the knife itself is a mere instrument, not a proper subject for moral or political evaluation. While contemporary philosophers of technology widely reject the VNT, it remains unclear whether claims about values in technology are just a figure of speech (...) or nontrivial empirical claims with genuine factual content and real-world implications. This paper provides the missing argument. I argue that by virtue of their material properties, technological artifacts are part of the normative order rather than external to it. I illustrate how values can be empirically identified in technology. The reason why value-talk is not trivial or metaphorical is that due to the endurance and longevity of technological artifacts, values embedded in them have long-term implications that surpass their designers and builders. I further argue that taking sides in this debate has real-world implications in the form of moral constraints on the development of technology. (shrink)

Stuart Russell defines the value alignment problem as follows: How can we build autonomous systems with values that “are aligned with those of the human race”? In this article I outline some distinctions that are useful for understanding the value alignment problem and then propose a solution: I argue that the methods currently applied by computer scientists for embedding moral values in autonomous systems can be improved by representing moral principles as conceptual spaces, i.e. as Voronoi tessellations of morally similar (...) choice situations located in a multidimensional geometric space. The advantage of my preferred geometric approach is that it can be implemented without specifying any utility function ex ante. (shrink)

This article reviews the academic literature that emerged under value sensitive design. It investigates those VSD projects that employed the tripartite methodology, examining the use of VSD methodological elements, and illustrating common practices and identifying shortcomings. The article provides advice for VSD researchers on how to complete and enhance their methodological approach as the research community moves forward.

Lethal Autonomous Weapons (LAWs) are robotic weapons systems, primarily of value to the military, that could engage in offensive or defensive actions without human intervention. This paper assesses and engages the current arguments for and against the use of LAWs through the lens of achieving more ethical warfare. Specific interest is given particularly to ethical LAWs, which are artificially intelligent weapons systems that make decisions within the bounds of their ethics-based code. To ensure that a wide, but not exhaustive, survey (...) of the implications of employing such ethical devices to replace humans in warfare is taken into account, this paper will engage on matters related to current scholarship on the rejection or acceptance of LAWs—including contemporary technological shortcomings of LAWs to differentiate between targets and the behavioral and psychological volatility of humans—and current and proposed regulatory infrastructures for developing and using such devices. After careful consideration of these factors, this paper will conclude that only ethical LAWs should be used to replace human involvement in war, and, by extension of their consistent abilities, should remove humans from war until a more formidable discovery is made in conducting ethical warfare. (shrink)

This paper argues that although moral intuitions are insufficient for making judgments on new technological innovations, they maintain great utility for informing responsible innovation. To do this, this paper employs the Value Sensitive Design (VSD) methodology as an illustrative example of how stakeholder values can be better distilled to inform responsible innovation. Further, it is argued that moral intuitions are necessary for determining stakeholder values required for the design of responsible technologies. This argument is supported by the claim that the (...) moral intuitions of stakeholders allow designers to conceptualize stakeholder values and incorporate them into the early phases of design. It is concluded that design-for-values (DFV) frameworks like the VSD methodology can remain potent if developers adopt heuristic tools to diminish the influence of cognitive biases thus strengthening the reliability of moral intuitions. (shrink)

The underlying thesis of the research in ethical autonomy for lethal autonomous unmanned systems is that they will potentially be capable of performing more ethically on the battlefield than are human soldiers. In this article this hypothesis is supported by ongoing and foreseen technological advances and perhaps equally important by an assessment of the fundamental ability of human warfighters in today's battlespace. If this goal of better-than-human performance is achieved, even if still imperfect, it can result in a reduction in (...) noncombatant casualties and property damage consistent with adherence to the Laws of War as prescribed in international treaties and conventions, and is thus worth pursuing vigorously. (shrink)

A variety of ethical objections have been raised against the military employment of uninhabited aerial vehicles (UAVs, drones). Some of these objections are technological concerns over UAVs abilities’ to function on par with their inhabited counterparts. This paper sets such concerns aside and instead focuses on supposed objections to the use of UAVs in principle. I examine several such objections currently on offer and show them all to be wanting. Indeed, I argue that we have a duty to protect an (...) agent engaged in a justified act from harm to the greatest extent possible, so long as that protection does not interfere with the agent's ability to act justly. UAVs afford precisely such protection. Therefore, we are obligated to employ UAV weapon systems if it can be shown that their use does not significantly reduce a warfighter's operational capability. Of course, if a given military action is unjustified to begin with, then carrying out that act via UAVs is wrong, just as it would be with any weapon. But the point of this paper is to show that there is nothing wrong in principle with using a UAV and that, other things being equal, using such technology is, in fact, obligatory. (shrink)

The prospective robots in healthcare intended to be included within the conclave of the nurse-patient relationship—what I refer to as care robots—require rigorous ethical reflection to ensure their design and introduction do not impede the promotion of values and the dignity of patients at such a vulnerable and sensitive time in their lives. The ethical evaluation of care robots requires insight into the values at stake in the healthcare tradition. What’s more, given the stage of their development and lack of (...) standards provided by the International Organization for Standardization to guide their development, ethics ought to be included into the design process of such robots. The manner in which this may be accomplished, as presented here, uses the blueprint of the Value-sensitive design approach as a means for creating a framework tailored to care contexts. Using care values as the foundational values to be integrated into a technology and using the elements in care, from the care ethics perspective, as the normative criteria, the resulting approach may be referred to as care centered value-sensitive design. The framework proposed here allows for the ethical evaluation of care robots both retrospectively and prospectively. By evaluating care robots in this way, we may ultimately ask what kind of care we, as a society, want to provide in the future. (shrink)

When thinking about ethics, technology is often only mentioned as the source of our problems, not as a potential solution to our moral dilemmas. When thinking about technology, ethics is often only mentioned as a constraint on developments, not as a source and spring of innovation. In this paper, we argue that ethics can be the source of technological development rather than just a constraint and technological progress can create moral progress rather than just moral problems. We show this by (...) an analysis of how technology can contribute to the solution of so-called moral overload or moral dilemmas. Such dilemmas typically create a moral residue that is the basis of a second-order principle that tells us to reshape the world so that we can meet all our moral obligations. We can do so, among other things, through guided technological innovation. (shrink)

The implementation of moral decision making abilities in artificial intelligence (AI) is a natural and necessary extension to the social mechanisms of autonomous software agents and robots. Engineers exploring design strategies for systems sensitive to moral considerations in their choices and actions will need to determine what role ethical theory should play in defining control architectures for such systems. The architectures for morally intelligent agents fall within two broad approaches: the top-down imposition of ethical theories, and the bottom-up building of (...) systems that aim at goals or standards which may or may not be specified in explicitly theoretical terms. In this paper we wish to provide some direction for continued research by outlining the value and limitations inherent in each of these approaches. (shrink)

Artificial agents (AAs), particularly but not only those in Cyberspace, extend the class of entities that can be involved in moral situations. For they can be conceived of as moral patients (as entities that can be acted upon for good or evil) and also as moral agents (as entities that can perform actions, again for good or evil). In this paper, we clarify the concept of agent and go on to separate the concerns of morality and responsibility of agents (most (...) interestingly for us, of AAs). We conclude that there is substantial and important scope, particularly in Computer Ethics, for the concept of moral agent not necessarily exhibiting free will, mental states or responsibility. This complements the more traditional approach, common at least since Montaigne and Descartes, which considers whether or not (artificial) agents have mental states, feelings, emotions and so on. By focussing directly on mind-less morality we are able to avoid that question and also many of the concerns of Artificial Intelligence. A vital component in our approach is the Method of Abstraction for analysing the level of abstraction (LoA) at which an agent is considered to act. The LoA is determined by the way in which one chooses to describe, analyse and discuss a system and its context. The Method of Abstraction is explained in terms of an interface or set of features or observables at a given LoA. Agenthood, and in particular moral agenthood, depends on a LoA. Our guidelines for agenthood are: interactivity (response to stimulus by change of state), autonomy (ability to change state without stimulus) and adaptability (ability to change the transition rules by which state is changed) at a given LoA. Morality may be thought of as a threshold defined on the observables in the interface determining the LoA under consideration. An agent is morally good if its actions all respect that threshold; and it is morally evil if some action violates it. That view is particularly informative when the agent constitutes a software or digital system, and the observables are numerical. Finally we review the consequences for Computer Ethics of our approach. In conclusion, this approach facilitates the discussion of the morality of agents not only in Cyberspace but also in the biosphere, where animals can be considered moral agents without their having to display free will, emotions or mental states, and in social contexts, where systems like organizations can play the role of moral agents. The primary cost of this facility is the extension of the class of agents and moral agents to embrace AAs. (shrink)

Suppose that we develop a medically safe and affordable means of enhancing human intelligence. For concreteness, we shall assume that the technology is genetic engineering (either somatic or germ line), although the argument we will present does not depend on the technological implementation. For simplicity, we shall speak of enhancing “intelligence” or “cognitive capacity,” but we do not presuppose that intelligence is best conceived of as a unitary attribute. Our considerations could be applied to specific cognitive abilities such as verbal (...) fluency, memory, abstract reasoning, social intelligence, spatial cognition, numerical ability, or musical talent. It will emerge that the form of argument that we use can be applied much more generally to help assess other kinds of enhancement technologies as well as other kinds of reform. However, to give a detailed illustration of how the argument form works, we will focus on the prospect of cognitive enhancement. (shrink)

Social media use is pervasive and demographically diverse; thus, researchers can investigate the social media behavior of a target population to extract its values as expressed in a web-based network. This article posits that social media is a relatively untapped resource for researchers engaged in value sensitive design. It proposes methods and ethical considerations for researchers during value sensitive design conceptual, technical, and empirical investigations.

The prospective robots in healthcare intended to be included within the conclave of the nurse-patient relationship—what I refer to as care robots—require rigorous ethical reflection to ensure their design and introduction do not impede the promotion of values and the dignity of patients at such a vulnerable and sensitive time in their lives. The ethical evaluation of care robots requires insight into the values at stake in the healthcare tradition. What’s more, given the stage of their development and lack of (...) standards provided by the International Organization for Standardization to guide their development, ethics ought to be included into the design process of such robots. The manner in which this may be accomplished, as presented here, uses the blueprint of the Value-sensitive design approach as a means for creating a framework tailored to care contexts. Using care values as the foundational values to be integrated into a technology and using the elements in care, from the care ethics perspective, as the normative criteria, the resulting approach may be referred to as care centered value-sensitive design. The framework proposed here allows for the ethical evaluation of care robots both retrospectively and prospectively. By evaluating care robots in this way, we may ultimately ask what kind of care we, as a society, want to provide in the future. (shrink)

Proactively including the ethical and societal issues of new technologies could have a positive effect on their acceptance. These issues could be captured in terms of values. In the literature, the values stakeholders deem important for the development of technology have often been identified. However, the relative ranking of these values in relation to each other have not been studied often. The best worst method is proposed as a possible method to determine the weights of values, hence it is used (...) in an evaluative fashion. The applicability of the method is tested by applying it to the case of smart meters, one of the main components of the smart grid. The importance of values is examined for three dimensions of acceptance namely sociopolitical, market, and household acceptance. (shrink)

In the value sensitive design literature, there has been little attention for how values may change during the adoption and use of a sociotechnical system, and what that implies for design. A value change taxonomy is proposed, as well as a number of technical features that allow dealing with value change.

Lethal Autonomous Weapons (LAWs) have becomes the subject of continuous debate both at national and international levels. Arguments have been proposed both for the development and use of LAWs as well as their prohibition from combat landscapes. Regardless, the development of LAWs continues in numerous nation-states. This paper builds upon previous philosophical arguments for the development and use of LAWs and proposes a design framework that can be used to ethically direct their development. The conclusion is that the philosophical arguments (...) that underpin the adoption of LAWs rather than not, although prima facie insufficient, can be actualized through the proposed Value Sensitive Design (VSD) approach. Hence, what is proposed is a principled design approach that can be used to embed stakeholder values into a design, encourage stakeholder cooperation and coordination and as a result promote social acceptance of LAWs as a preferable future fact of war. (shrink)

We argue that some algorithms are value-laden, and that two or more persons who accept different value-judgments may have a rational reason to design such algorithms differently. We exemplify our claim by discussing a set of algorithms used in medical image analysis: In these algorithms it is often necessary to set certain thresholds for whether e.g. a cell should count as diseased or not, and the chosen threshold will partly depend on the software designer’s preference between avoiding false positives and (...) false negatives. This preference ultimately depends on a number of value-judgments. In the last section of the paper we discuss some general principles for dealing with ethical issues in algorithm-design. (shrink)

The extent to which normative cognition varies across cultures has implications for a number of important philosophical questions. This chapter examines several striking commonalities and differences in normative cognition across cultures. We focus on cross-cultural commonality and difference in norm typologies (especially the moral-conventional distinction); the externalization of norms; which aspects of life are normativized; and some of the concepts and principles associated with the normative domain. We argue that the distinction between moral and conventional norms is probably not universal (...) and that several concepts and principles that have been suggested as candidate universals may in fact be culturally local. We nevertheless isolate two possible universals: the phenomenon of externalizing norms and the normativization of a preferred class of subject matters. (shrink)

Currently, value sensitive design (VSD) does not commit to a particular ethical theory. Critiques contend that without such an explicit commitment, VSD lacks a methodology for distinguishing genuine moral values from mere stakeholders-preferences and runs the risk of attending to a set of values that is unprincipled or unbounded. We argue that VSD practitioners need to complement it with an ethical theory. We argue in favour of a mid-level ethical theory to fulfil this role.

A major approach to the ethics of artificial intelligence is to use social choice, in which the AI is designed to act according to the aggregate views of society. This is found in the AI ethics of “coherent extrapolated volition” and “bottom–up ethics”. This paper shows that the normative basis of AI social choice ethics is weak due to the fact that there is no one single aggregate ethical view of society. Instead, the design of social choice AI faces three (...) sets of decisions: standing, concerning whose ethics views are included; measurement, concerning how their views are identified; and aggregation, concerning how individual views are combined to a single view that will guide AI behavior. These decisions must be made up front in the initial AI design—designers cannot “let the AI figure it out”. Each set of decisions poses difficult ethical dilemmas with major consequences for AI behavior, with some decision options yielding pathological or even catastrophic results. Furthermore, non-social choice ethics face similar issues, such as whether to count future generations or the AI itself. These issues can be more important than the question of whether or not to use social choice ethics. Attention should focus on these issues, not on social choice. (shrink)

Several philosophers of technology have argued that technology mediates human actions. For example, in the branch of post-phenomenology, authors such as Don Ihde and Peter-Paul Verbeek have described the mediating aspects of technology in terms of morality of technology as well as in the sense that technology changes our perception of ourselves and the world. In this article, different existing types of mediation are presented, critiqued, and enriched. The four types are illustrated by referring to military high-tech environments with a (...) focus on visual data and imaging technologies. These technologies can mediate actions by inviting certain behavior, through amplification and reduction, through built-in norms, and through interpretation. The four types of mediation mainly focus on the technology or technological artifact itself. What these approaches fail to grasp, however, is the specific user practices in which most technologies function. In this article, it is argued that to understand the mediating aspects of technology more fully, attention should be paid to the specific user context in which the technology functions. Therefore, an enriched understanding of the four types of mediation of technology is proposed by taking the lens of normative practices and analyzing the different types of mediation through this lens. The Kunduz airstrike incident, which took place in 2009 in Afghanistan, is a case in which a visual data sharing device called Rover played a prominent role. This case is used in this article to illustrate how technology mediates human actions in military practice. (shrink)

Several philosophers of technology have argued that technology mediates human actions. For example, in the branch of post-phenomenology, authors such as Don Ihde and Peter-Paul Verbeek have described the mediating aspects of technology in terms of morality of technology (more prominent in Verbeek) as well as in the sense that technology changes our perception of ourselves and the world (more prominent in Ihde). In this article, different existing types of mediation are presented, critiqued, and enriched. The four types are illustrated (...) by referring to military high-tech environments with a focus on visual data and imaging technologies. These technologies can mediate actions (1) by inviting certain behavior, (2) through amplification and reduction, (3) through built-in norms, and (4) through interpretation. The four types of mediation mainly focus on the technology or technological artifact itself. What these approaches fail to grasp, however, is the specific user practices in which most technologies function. In this article, it is argued that to understand the mediating aspects of technology more fully, attention should be paid to the specific user context in which the technology functions. Therefore, an enriched understanding of the four types of mediation of technology is proposed by taking the lens of normative practices and analyzing the different types of mediation through this lens. The Kunduz airstrike incident, which took place in 2009 in Afghanistan, is a case in which a visual data sharing device called Rover played a prominent role. This case is used in this article to illustrate how technology mediates human actions in military practice. (shrink)