Many ethical concerns have been voiced about Clinical Decision Support Systems (CDSSs). Special attention has been paid to the effect of CDSSs on autonomy, responsibility, fairness and transparency. This journal has featured a discussion between Rosalind McDougall and Ezio Di Nucci that focused on the impact of IBM’s Watson for Oncology (Watson) on autonomy. The present article elaborates on this discussion in three ways. First, using Jonathan Pugh’s account of rational autonomy we show that how Watson presents its results might (...) impact decisional autonomy, while how Watson produces knowledge might affect practical autonomy. Second, by drawing an analogy with patient decision aids we identify an empirical way of estimating Watson’s impact on autonomy (ie, value-congruence). Lastly, McDougall introduced the notion of value-flexible design as a way to account for the diverging preferences patients hold. We will clarify its relation with the established domain of value-sensitive design. In terms of the tripartite methodology of value-sensitive design, we offer aconceptualclarification using Pugh’s account of rational autonomy, anempiricaltool to evaluate Watson’s impact on autonomy and situate a group oftechnicaloptions to incorporate autonomy in Watson’s design. (shrink)

The Accreditation Board of Engineering and Technology (ABET) has declared that to achieve accredited status, “engineering programs must demonstrate that their graduates have an understanding of professional and ethical responsibility.” Many engineering professors struggle to integrate this required ethics instruction in technical classes and projects because of the lack of a formalized ethics-in-design approach. However, one methodology developed in human-computer interaction research, the Value-Sensitive Design approach, can serve as an engineering education tool which bridges the gap between design and ethics (...) for many engineering disciplines. The three major components of Value-Sensitive Design, conceptual, technical, and empirical, exemplified through a case study which focuses on the development of a command and control supervisory interface for a military cruise missile. (shrink)

Traditional medical practices and relationships are changing given the widespread adoption of AI-driven technologies across the various domains of health and healthcare. In many cases, these new technologies are not specific to the field of healthcare. Still, they are existent, ubiquitous, and commercially available systems upskilled to integrate these novel care practices. Given the widespread adoption, coupled with the dramatic changes in practices, new ethical and social issues emerge due to how these systems nudge users into making decisions and changing (...) behaviours. This article discusses how these AI-driven systems pose particular ethical challenges with regards to nudging. To confront these issues, the value sensitive design approach is adopted as a principled methodology that designers can adopt to design these systems to avoid harming and contribute to the social good. The AI for Social Good factors are adopted as the norms constraining maleficence. In contrast, higher-order values specific to AI, such as those from the EU High-Level Expert Group on AI and the United Nations Sustainable Development Goals, are adopted as the values to be promoted as much as possible in design. The use case of Amazon Alexa's Healthcare Skills is used to illustrate this design approach. It provides an exemplar of how designers and engineers can begin to orientate their design programs of these technologies towards the social good. (shrink)

This article offers a discussion of the connection between technology and values and, specifically, I take a closer look at ethically sound design. In order to bring the discussion into a concrete context, the theory of Value Sensitive Design (VSD) will be the focus point. To illustrate my argument concerning design ethics, the discussion involves a case study of an augmented window, designed by the VSD Research Lab, which has turned out to be a potentially surveillance-enabling technology. I call attention (...) to a “positivist problem” that has to do with the connection between the design context and the use context, which VSD seems to presuppose, and I argue that it is necessary to clearly distinguish between the two, since the designers’ intentions do not always correspond with the users’ practice; in fact, the relation between design and use is very complex and principally unpredictable. Thus, a design theory must accept that foresight is limited to anticipation rather than prediction. To overcome the positivist problem, I suggest a phenomenological approach to technology inspired by Don Ihde’s concept of multistability. This argument, which is general in nature and thus applies to any theory of design ethics, is intended as a constructive criticism, which can hopefully contribute to the further development of design ethics. (shrink)

This study deals with an underexplored area of the emerging technologies debate: robotics in the healthcare setting. The author explores the role of care and develops a value-sensitive ethical framework for the eventual employment of care robots. Highlighting the range of positive and negative aspects associated with the initiative to design and use care robots, it draws out essential content as a guide to future design both reinforcing this study’s contemporary relevance, and giving weight to its prescriptions. The book speaks (...) to, and is meant to be read by, a range of disciplines from science and engineering to philosophers and ethicists. (shrink)

A robust stakeholder analysis requires extensive conceptual and empirical work. Yet it is often unclear how to effectively do so. This paper introduces a new method—the Stakeholder Tokens—for designers to elicit a more inclusive set of stakeholders and gain better understanding of stakeholder interrelationships and dynamics. Stakeholder Tokens present a playful hands-on design approach to support value sensitive design stakeholder analyses by employing a style of role play.

The prospective robots in healthcare intended to be included within the conclave of the nurse-patient relationship—what I refer to as care robots—require rigorous ethical reflection to ensure their design and introduction do not impede the promotion of values and the dignity of patients at such a vulnerable and sensitive time in their lives. The ethical evaluation of care robots requires insight into the values at stake in the healthcare tradition. What’s more, given the stage of their development and lack of (...) standards provided by the International Organization for Standardization to guide their development, ethics ought to be included into the design process of such robots. The manner in which this may be accomplished, as presented here, uses the blueprint of the Value-sensitive design approach as a means for creating a framework tailored to care contexts. Using care values as the foundational values to be integrated into a technology and using the elements in care, from the care ethics perspective, as the normative criteria, the resulting approach may be referred to as care centered value-sensitive design. The framework proposed here allows for the ethical evaluation of care robots both retrospectively and prospectively. By evaluating care robots in this way, we may ultimately ask what kind of care we, as a society, want to provide in the future. (shrink)

This article reviews the academic literature that emerged under value sensitive design. It investigates those VSD projects that employed the tripartite methodology, examining the use of VSD methodological elements, and illustrating common practices and identifying shortcomings. The article provides advice for VSD researchers on how to complete and enhance their methodological approach as the research community moves forward.

In projects concerning big data, ethical questions need to be answered during the design process. In this paper the Value Sensitive Design method is applied in the context of data-driven health services aimed at disease prevention. It shows how Value Sensitive Design, with the use of a moral dialogue and an ethical matrix, can support the identification and operationalization of moral values that are at stake in the design of such services. It also shows that using this method can support (...) meeting the requirements of the General Data Protection Regulation. (shrink)

When thinking about ethics, technology is often only mentioned as the source of our problems, not as a potential solution to our moral dilemmas. When thinking about technology, ethics is often only mentioned as a constraint on developments, not as a source and spring of innovation. In this paper, we argue that ethics can be the source of technological development rather than just a constraint and technological progress can create moral progress rather than just moral problems. We show this by (...) an analysis of how technology can contribute to the solution of so-called moral overload or moral dilemmas. Such dilemmas typically create a moral residue that is the basis of a second-order principle that tells us to reshape the world so that we can meet all our moral obligations. We can do so, among other things, through guided technological innovation. (shrink)

The prospective robots in healthcare intended to be included within the conclave of the nurse-patient relationship—what I refer to as care robots—require rigorous ethical reflection to ensure their design and introduction do not impede the promotion of values and the dignity of patients at such a vulnerable and sensitive time in their lives. The ethical evaluation of care robots requires insight into the values at stake in the healthcare tradition. What’s more, given the stage of their development and lack of (...) standards provided by the International Organization for Standardization to guide their development, ethics ought to be included into the design process of such robots. The manner in which this may be accomplished, as presented here, uses the blueprint of the Value-sensitive design approach as a means for creating a framework tailored to care contexts. Using care values as the foundational values to be integrated into a technology and using the elements in care, from the care ethics perspective, as the normative criteria, the resulting approach may be referred to as care centered value-sensitive design. The framework proposed here allows for the ethical evaluation of care robots both retrospectively and prospectively. By evaluating care robots in this way, we may ultimately ask what kind of care we, as a society, want to provide in the future. (shrink)

In the value sensitive design literature, there has been little attention for how values may change during the adoption and use of a sociotechnical system, and what that implies for design. A value change taxonomy is proposed, as well as a number of technical features that allow dealing with value change.

In the age of Big Data, companies and governments are increasingly using algorithms to inform hiring decisions, employee management, policing, credit scoring, insurance pricing, and many more aspects of our lives. Artificial intelligence systems can help us make evidence-driven, efficient decisions, but can also confront us with unjustified, discriminatory decisions wrongly assumed to be accurate because they are made automatically and quantitatively. It is becoming evident that these technological developments are consequential to people’s fundamental human rights. Despite increasing attention to (...) these urgent challenges in recent years, technical solutions to these complex socio-ethical problems are often developed without empirical study of societal context and the critical input of societal stakeholders who are impacted by the technology. On the other hand, calls for more ethically and socially aware AI often fail to provide answers for how to proceed beyond stressing the importance of transparency, explainability, and fairness. Bridging these socio-technical gaps and the deep divide between abstract value language and design requirements is essential to facilitate nuanced, context-dependent design choices that will support moral and social values. In this paper, we bridge this divide through the framework of Design for Values, drawing on methodologies of Value Sensitive Design and Participatory Design to present a roadmap for proactively engaging societal stakeholders to translate fundamental human rights into context-dependent design requirements through a structured, inclusive, and transparent process. (shrink)

Safe-by-Design (SBD) frameworks for the development of emerging technologies have become an ever more popular means by which scholars argue that transformative emerging technologies can safely incorporate human values. One such popular SBD methodology is called Value Sensitive Design (VSD). A central tenet of this design methodology is to investigate stakeholder values and design those values into technologies during early stage research and development (R&D). To accomplish this, the VSD framework mandates that designers consult the philosophical and ethical literature to (...) best determine how to weigh moral trade-offs. However, the VSD framework also concedes the universalism of moral values, particularly the values of freedom, autonomy, equality trust and privacy justice. This paper argues that the VSD methodology, particularly applied to nano-bio-info-cogno (NBIC) technologies, has an insufficient grounding for the determination of moral values. As such, an exploration of the value-investigations of VSD are deconstructed to illustrate both its strengths and weaknesses. This paper also provides possible modalities for the strengthening of the VSD methodology, particularly through the application of moral imagination and how moral imagination exceed the boundaries of moral intuitions in the development of novel technologies. (shrink)

Intelligent algorithms together with various machine learning techniques hold a dominant position among major challenges for contemporary value sensitive design. Self-learning capabilities of current AI applications blur the causal link between programmer and computer behavior. This creates a vital challenge for the design, development and implementation of digital technologies nowadays. This paper seeks to provide an account of this challenge. The main question that shapes the current analysis is the following: What conceptual tools can be developed within the value sensitive (...) design school of thought for evaluating machine learning algorithms where the causal relation between designers and the behavior of their computer systems has been eroded? The answer to this question will be provided through two levels of investigation within the value sensitive design methodology. The first level is conceptual. Within the conceptual level, we will introduce the notion of computer intentionality and will show how this term may be used for solving an issue of non-causal relation between designer and computer system. The second level of investigation is technical. At this level the emphasis will be given to machine learning algorithms. (shrink)

Recently, there is increased attention to the integration of moral values into the conception, design, and development of emerging IT. The most reviewed approach for this purpose in ethics and technology so far is Value-Sensitive Design (VSD). This article considers VSD as the prime candidate for implementing normative considerations into design. Its methodology is considered from a conceptual, analytical, normative perspective. The focus here is on the suitability of VSD for integrating moral values into the design of technologies in a (...) way that joins in with an analytical perspective on ethics of technology. Despite its promising character, it turns out that VSD falls short in several respects: (1) VSD does not have a clear methodology for identifying stakeholders, (2) the integration of empirical methods with conceptual research within the methodology of VSD is obscure, (3) VSD runs the risk of committing the naturalistic fallacy when using empirical knowledge for implementing values in design, (4) the concept of values, as well as their realization, is left undetermined and (5) VSD lacks a complimentary or explicit ethical theory for dealing with value trade-offs. For the normative evaluation of a technology, I claim that an explicit and justified ethical starting point or principle is required. Moreover, explicit attention should be given to the value aims and assumptions of a particular design. The criteria of adequacy for such an approach or methodology follow from the evaluation of VSD as the prime candidate for implementing moral values in design. (shrink)

Recent developments in theories for responsible innovation have focused on the importance of actively accounting for values in our technological designs. Leading among these theories is that of Value Sensitive Design which attempts to guide the design process on the basis of evaluative analysis. However, values often come into conflict and VSD has been criticized for not providing a proper method to resolve such inevitable conflicts. This paper examines three such methods and argues that although each has its merits, they (...) all fail to account for a common source of value conflicts known as value incommensurability. Drawing on literature from the field of axiology, this paper argues that by incorporating the evaluative relation of ‘parity’ each of these three methods, and the VSD framework in general, will be able to properly understand the relation which holds between conflicting design options stemming from the incommensurable of values and be able to guide designers in making rational decision in the face of such conflicts. (shrink)

In this article, we first offer a model of design knowledge types and their interrelationships in value sensitive design. Then we demonstrate that value sensitive design is a formative framework, which provides a shaping influence on practice, enables creative appropriation, and supports theory and method development.

This article presents a conceptual investigation into the value impacts and relations of algorithms in the domain of justice and security. As a conceptual investigation, it represents one step in a value sensitive design based methodology. Here, we explicate and analyse the expression of values of accuracy, privacy, fairness and equality, property and ownership, and accountability and transparency in this context. We find that values are sensitive to disvalue if algorithms are designed, implemented or deployed inappropriately or without sufficient consideration (...) for their value impacts, potentially resulting in problems including discrimination and constrained autonomy. Furthermore, we outline a framework of conceptual relations of values indicated by our analysis, and potential value tensions in their implementation and deployment with a view towards supporting future research, and supporting the value sensitive design of algorithms in justice and security. (shrink)

This article presents a systematic literature review documenting how technical investigations have been adapted in value sensitive design (VSD) studies from 1996 to 2023. We present a systematic review, including theoretical and applied studies that either discuss or conduct technical investigations in VSD. This systematic review contributes to the VSD community when seeking to further refine the methodological framework for carrying out technical investigations in VSD.

In this article, we report on eight grand challenges for value sensitive design, which were developed at a one-week workshop, Value Sensitive Design: Charting the Next Decade, Lorentz Center, Leiden, The Netherlands, November 14–18, 2016. A grand challenge is a substantial problem, opportunity, or question that motives sustained research and design activity. The eight grand challenges are: Accounting for Power, Evaluating Value Sensitive Design, Framing and Prioritizing Values, Professional and Industry Appropriation, Tech policy, Values and Human Emotions, Value Sensitive Design (...) and Intelligent Algorithms, and Value Tensions. Each grand challenge consists of a discussion of its importance and a set of tractable key questions. (shrink)

One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents (IAs). This paper explores how decision matrix algorithms, via the belief-desire-intention model for (...) autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing for the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles. (shrink)

Although artificial intelligence has been given an unprecedented amount of attention in both the public and academic domains in the last few years, its convergence with other transformative technologies like cloud computing, robotics, and augmented/virtual reality is predicted to exacerbate its impacts on society. The adoption and integration of these technologies within industry and manufacturing spaces is a fundamental part of what is called Industry 4.0, or the Fourth Industrial Revolution. The impacts of this paradigm shift on the human operators (...) who continue to work alongside and symbiotically with these technologies in the industry bring with it novel ethical issues. Therefore, how to design these technologies for human values becomes the critical area of intervention. This paper takes up the case study of robotic AI-based assistance systems to explore the potential value implications that emerge due to current design practices and use. The design methodology known as Value Sensitive Design (VSD) is proposed as a sufficient starting point for designing these technologies for human values to address these issues. (shrink)

Manufacturing and industry practices are undergoing an unprecedented revolution as a consequence of the convergence of emerging technologies such as artificial intelligence, robotics, cloud computing, virtual and augmented reality, among others. This fourth industrial revolution is similarly changing the practices and capabilities of operators in their industrial environments. This paper introduces and explores the notion of the Operator 4.0 as well as how this novel way of conceptualizing the human operator necessarily implicates human values in the technologies that constitute it. (...) The design approach known as value sensitive design (VSD) is used to explore how these Operator 4.0 technologies can be designed for human values. Expert elicitation surveys were used to determine the values of industry stakeholders and examples of how the VSD methodology can be adopted by engineers in order to design for these values is illustrated. The results provide preliminary adoption strategies that industrial teams can take to Operator 4.0 technology for human values. (shrink)