In this paper, we consider the meaning, roles, and uses of trust in the economic and public domain, focusing on the task of designing systems for trust in information technology. We analyze this task by means of a survey of what trust means in the economic and public domain, using the model proposed by Lewicki and Bunker, and using the emerging paradigm of value-sensitive design. We explore the difficulties developers face when designing information technology for trust and show how our (...) analysis in conjunction with existing engineering design methods provides means to address these difficulties. Our main case concerns a concrete problem in the economic domain, namely the transfer of control from customs agencies to companies. Control of individual items is increasingly untenable and is replaced by control on the level of companies aimed at determining whether companies can be trusted to be in control of their business and to be in compliance with applicable regulations. This transfer sets the task for companies to establish this trust by means of information technology systems. We argue that this trust can be achieved by taking into account philosophical analyses of trust and by including both parties in the trust relationship as clients for whom the information technology systems are to be designed. (shrink)

This study extended the scope of previous findings in human–computer interaction research within the computers are social actors paradigm by showing that online users attribute perceptions of moral qualities to Websites and, further, that differential perceptions of morality affected the extent of persuasion. In an experiment (N = 138) that manipulated four morality conditions (universalist, relativist, egotistic, control) across worldview, a measured independent variable, users were asked to evaluate a Web site designed to aid them in making ethical decisions. Web (...) sites offered four different types of ethical advice as participants contemplated cases involving ethical quandaries. Perceptions of the Web sites’ moral qualities varied depending on the type of advice given. Further, the Web sites’ perceived morality and participants’ worldview predicted credibility, persuasiveness, and attitudes toward the Web sites. (shrink)

In the age of Big Data, companies and governments are increasingly using algorithms to inform hiring decisions, employee management, policing, credit scoring, insurance pricing, and many more aspects of our lives. Artificial intelligence systems can help us make evidence-driven, efficient decisions, but can also confront us with unjustified, discriminatory decisions wrongly assumed to be accurate because they are made automatically and quantitatively. It is becoming evident that these technological developments are consequential to people’s fundamental human rights. Despite increasing attention to (...) these urgent challenges in recent years, technical solutions to these complex socio-ethical problems are often developed without empirical study of societal context and the critical input of societal stakeholders who are impacted by the technology. On the other hand, calls for more ethically and socially aware AI often fail to provide answers for how to proceed beyond stressing the importance of transparency, explainability, and fairness. Bridging these socio-technical gaps and the deep divide between abstract value language and design requirements is essential to facilitate nuanced, context-dependent design choices that will support moral and social values. In this paper, we bridge this divide through the framework of Design for Values, drawing on methodologies of Value Sensitive Design and Participatory Design to present a roadmap for proactively engaging societal stakeholders to translate fundamental human rights into context-dependent design requirements through a structured, inclusive, and transparent process. (shrink)

Traditional approaches to conflict are oriented towards establishing consensus, either in the form of a resolution of the conflict or in the form of an ‘agree-to-disagree’ standstill between the stakeholders. In this paper, we criticize these traditional approaches, each for specific reasons, and we propose and develop the agonistic approach to conflict. Based on Chantal Mouffe’s agonistic democratic theory, the agonistic approach to conflict is more welcoming of dissensus, replacing discussion stoppers with discussion starters and replacing standstills with contestation. We (...) illustrate such replacements and develop this approach, we analyse technological conflicts in a concrete R&D setting: the global hydrogen economy. From this context, we focus on the conflict between the proponents of blue hydrogen and those of green hydrogen. We conclude by highlighting the advantage of the agonistic approach but also drawing attention to its own specific risk, namely, antagonism. (shrink)

As inductive decision-making procedures, the inferences made by machine learning programs are subject to underdetermination by evidence and bear inductive risk. One strategy for overcoming these challenges is guided by a presumption in philosophy of science that inductive inferences can and should be value-free. Applied to machine learning programs, the strategy assumes that the influence of values is restricted to data and decision outcomes, thereby omitting internal value-laden design choice points. In this paper, I apply arguments from feminist philosophy of (...) science to machine learning programs to make the case that the resources required to respond to these inductive challenges render critical aspects of their design constitutively value-laden. I demonstrate these points specifically in the case of recidivism algorithms, arguing that contemporary debates concerning fairness in criminal justice risk-assessment programs are best understood as iterations of traditional arguments from inductive risk and demarcation, and thereby establish the value-laden nature of automated decision-making programs. Finally, in light of these points, I address opportunities for relocating the value-free ideal in machine learning and the limitations that accompany them. (shrink)

In this panel, we explore the future of value sensitive design (VSD). The stakes are high. Many in public and private sectors and in civil society are gradually realizing that taking our values seriously implies that we have to ensure that values effectively inform the design of technology which, in turn, shapes people’s lives. Value sensitive design offers a highly developed set of theory, tools, and methods to systematically do so.

One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents (IAs). This paper explores how decision matrix algorithms, via the belief-desire-intention model for (...) autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing for the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles. (shrink)

Safe-by-Design (SBD) frameworks for the development of emerging technologies have become an ever more popular means by which scholars argue that transformative emerging technologies can safely incorporate human values. One such popular SBD methodology is called Value Sensitive Design (VSD). A central tenet of this design methodology is to investigate stakeholder values and design those values into technologies during early stage research and development (R&D). To accomplish this, the VSD framework mandates that designers consult the philosophical and ethical literature to (...) best determine how to weigh moral trade-offs. However, the VSD framework also concedes the universalism of moral values, particularly the values of freedom, autonomy, equality trust and privacy justice. This paper argues that the VSD methodology, particularly applied to nano-bio-info-cogno (NBIC) technologies, has an insufficient grounding for the determination of moral values. As such, an exploration of the value-investigations of VSD are deconstructed to illustrate both its strengths and weaknesses. This paper also provides possible modalities for the strengthening of the VSD methodology, particularly through the application of moral imagination and how moral imagination exceed the boundaries of moral intuitions in the development of novel technologies. (shrink)

This paper argues that although moral intuitions are insufficient for making judgments on new technological innovations, they maintain great utility for informing responsible innovation. To do this, this paper employs the Value Sensitive Design (VSD) methodology as an illustrative example of how stakeholder values can be better distilled to inform responsible innovation. Further, it is argued that moral intuitions are necessary for determining stakeholder values required for the design of responsible technologies. This argument is supported by the claim that the (...) moral intuitions of stakeholders allow designers to conceptualize stakeholder values and incorporate them into the early phases of design. It is concluded that design-for-values (DFV) frameworks like the VSD methodology can remain potent if developers adopt heuristic tools to diminish the influence of cognitive biases thus strengthening the reliability of moral intuitions. (shrink)

Autonomous e-coaching systems offer their users suggestions for action, thereby affecting the user's decision-making process. More specifically, the suggestions that these systems make influence the options for action that people actually consider. Surprisingly though, options and the corresponding process of option generation --- a decision-making stage preceding intention formation and action selection --- has received very little attention in the various disciplines studying decision making. We argue that this neglect is unjustified and that it is important, particularly for designers of (...) autonomous e-coaching systems, to understand how human option generation works. The aims of this paper are threefold. The first aim is to generate awareness with designers of autonomous e-coaching systems that these systems do in fact influence their users' options. The second is to show that understanding the interplay between a person's options and the e-coaching system's suggestions is important for improving the effectiveness of the system. The third is that the very same interplay is also crucial for designing e-coaching systems that respect people's autonomy. (shrink)

Various Value-Conscious Design frameworks have recently emerged to introduce moral and ethical intelligence into business and technical design contexts, with the goal of proactively influencing the design of technologies to account for moral and ethical values during the conception and design process. Two attempts to insert ethical intelligence into technical design communities to influence the design of technologies in ethical- and value-conscious ways are described, revealing discouraging results. Learning from these failed attempts, the article identifies three key challenges of pragmatic (...) engagement with technical design communities: confronting competing values; identifying the role of the values advocate; and the justification of a value framework. Addressing these challenges must become a priority if one is to be successful in pragmatically engaging with real-world business and design contexts to bring moral and ethical intelligence to bear in the design of emerging information and communication technologies. (shrink)

Over the past decade Identity Management has become a central theme in information technology, policy, and administration in the public and private sectors. In these contexts the term ‘Identity Management’ is used primarily to refer to ways and methods of dealing with registration and authorization issues regarding persons in organizational and service-oriented domains. Especially due to the growing range of choices and options for, and the enhanced autonomy and rights of, employees, citizens, and customers, there is a growing demand for (...) systems that enable the regulation of rights, duties, responsibilities, entitlements and access of innumerable people simultaneously. ‘Identity Management’ or ‘Identity Management Systems’ have become important headings under which such systems are designed and implemented. But there is another meaning of the term ‘identity management’, which is clearly related and which has gained currency. This second construal refers to the need to manage our moral identities and our identity related information. This paper explores the relation between the management of our (moral) identities and ‘Identity Management’ as conceptualized in IT discourse. (shrink)

The Accreditation Board of Engineering and Technology (ABET) has declared that to achieve accredited status, “engineering programs must demonstrate that their graduates have an understanding of professional and ethical responsibility.” Many engineering professors struggle to integrate this required ethics instruction in technical classes and projects because of the lack of a formalized ethics-in-design approach. However, one methodology developed in human-computer interaction research, the Value-Sensitive Design approach, can serve as an engineering education tool which bridges the gap between design and ethics (...) for many engineering disciplines. The three major components of Value-Sensitive Design, conceptual, technical, and empirical, exemplified through a case study which focuses on the development of a command and control supervisory interface for a military cruise missile. (shrink)

This article offers a discussion of the connection between technology and values and, specifically, I take a closer look at ethically sound design. In order to bring the discussion into a concrete context, the theory of Value Sensitive Design (VSD) will be the focus point. To illustrate my argument concerning design ethics, the discussion involves a case study of an augmented window, designed by the VSD Research Lab, which has turned out to be a potentially surveillance-enabling technology. I call attention (...) to a “positivist problem” that has to do with the connection between the design context and the use context, which VSD seems to presuppose, and I argue that it is necessary to clearly distinguish between the two, since the designers’ intentions do not always correspond with the users’ practice; in fact, the relation between design and use is very complex and principally unpredictable. Thus, a design theory must accept that foresight is limited to anticipation rather than prediction. To overcome the positivist problem, I suggest a phenomenological approach to technology inspired by Don Ihde’s concept of multistability. This argument, which is general in nature and thus applies to any theory of design ethics, is intended as a constructive criticism, which can hopefully contribute to the further development of design ethics. (shrink)

From past research it is well known that social exclusion has detrimental consequences for mental health. To deal with these adverse effects, socially excluded individuals frequently turn to other humans for emotional support. While chatbots can elicit social and emotional responses on the part of the human interlocutor, their effectiveness in the context of social exclusion has not been investigated. In the present study, we examined whether an empathic chatbot can serve as a buffer against the adverse effects of social (...) ostracism. After experiencing exclusion on social media, participants were randomly assigned to either talk with an empathetic chatbot about it (e.g., “I'm sorry that this happened to you”) or a control condition where their responses were merely acknowledged (e.g., “Thank you for your feedback”). Replicating previous research, results revealed that experiences of social exclusion dampened the mood of participants. Interacting with an empathetic chatbot, however, appeared to have a mitigating impact. In particular, participants in the chatbot intervention condition reported higher mood than those in the control condition. Theoretical, methodological, and practical implications, as well as directions for future research are discussed. (shrink)

Recently, amid growing awareness that computer algorithms are not neutral tools but can cause harm by reproducing and amplifying bias, attempts to detect and prevent such biases have intensified. An approach that has received considerable attention in this regard is the Value Sensitive Design (VSD) methodology, which aims to contribute to both the critical analysis of (dis)values in existing technologies and the construction of novel technologies that account for specific desired values. This article provides a brief overview of the key (...) features of the Value Sensitive Design approach, examines its contributions to understanding and addressing issues around bias in computer systems, outlines the current debates on algorithmic bias and fairness in machine learning, and discusses how such debates could profit from VSD-derived insights and recommendations. Relating these debates on values in design and algorithmic bias to research on cognitive biases, we conclude by stressing our collective duty to not only detect and counter biases in software systems, but to also address and remedy their societal origins. (shrink)

Algorithms, or rather algorithmic actions, are seen as problematic because they are inscrutable, automatic, and subsumed in the flow of daily practices. Yet, they are also seen to be playing an important role in organizing opportunities, enacting certain categories, and doing what David Lyon calls “social sorting.” Thus, there is a general concern that this increasingly prevalent mode of ordering and organizing should be governed more explicitly. Some have argued for more transparency and openness, others have argued for more democratic (...) or value-centered design of such actors. In this article, we argue that governing practices—of, and through algorithmic actors—are best understood in terms of what Foucault calls governmentality. Governmentality allows us to consider the performative nature of these governing practices. They allow us to show how practice becomes problematized, how calculative practices are enacted as technologies of governance, how such calculative practices produce domains of knowledge and expertise, and finally, how such domains of knowledge become internalized in order to enact self-governing subjects. In other words, it allows us to show the mutually constitutive nature of problems, domains of knowledge, and subjectivities enacted through governing practices. In order to demonstrate this, we present attempts to govern academic writing with a specific focus on the algorithmic action of Turnitin. (shrink)

In a landmark essay, Andreas Matthias claimed that current developments in autonomous, artificially intelligent (AI) systems are creating a so-called responsibility gap, which is allegedly ever-widening and stands to undermine both the moral and legal frameworks of our society. But how severe is the threat posed by emerging technologies? In fact, a great number of authors have indicated that the fear is thoroughly instilled. The most pessimistic are calling for a drastic scaling-back or complete moratorium on AI systems, while the (...) optimists aim to show that the gap can be bridged nonetheless. Contrary to both camps, I argue against the prevailing assumption that there is a technology-based responsibility gap. I show how moral responsibility is a dynamic and flexible process, one that can effectively encompass emerging technological entities. (shrink)

In this paper we present the perceptions of those engagedin a digital research project. We explored participants’ ethical knowledge,understanding and feelings about the whole process. We also tried toimplement a plan in order to see if certain proactive actions will benefitthe understanding of ethical issues among those stakeholders.The investigation presented here explored the need for ethical awarenessamong stakeholders in technology research projects.1 The Project tried to overcome the existing accessibility barriers faced by people unfamiliarwith ICT, people with disabilities and older (...) people when interactingwith Public Digital Terminals, such as ATMs and Ticket VendingMachines. As part of the research an investigation was conducted aboutthe ethical issues that rose on the use of digital technologies and, moreimportantly, about stakeholders ethical issues awareness.In this report the results from the field research carried out on ethicalissues are presented. At the beginning of the project a need for researchinto ethical awareness among project stakeholders was identified. As partof the project´s legal and ethical awareness and compliance, a limitedresearch was carried out on ethical issues. The research carried out encompassedtesting the validity and adequacy of the information providedon a purpose-made guide that also included more ethical awareness questionsto explore.During the testing sessions carried out in the pilot the questionnairewas presented to volunteers. Another questionnaire was presented todevelopers, and to the APSIS4 all project personnel. The expresion ofethical concerns was also encouraged from both the staff working in theproject and volunteers participating in the trials.Results showed that the information provided on the documentationwas adequate and, likewise, that both volunteers and consortium personnelwere aware of volunteers’ rights and personnel obligations. (shrink)

The technical details of Internet architecture affect social debates about privacy and autonomy, intellectual property, cybersecurity, and the basic performance and reliability of Internet services. This paper explores one method for practicing anticipatory ethics in order to understand how a new infrastructure for the Internet might impact these social debates. This paper systematically examines values expressed by an Internet architecture engineering team—the Named Data Networking project—based on data gathered from publications and internal documents. Networking engineers making technical choices also weigh (...) non-technical values when working on Internet infrastructure. Analysis of the team’s documents reveals both values invoked in response to technical constraints and possibilities, such as efficiency and dynamism, as well as values, including privacy, security and anonymity, which stem from a concern for personal liberties. More peripheral communitarian values espoused by the engineers include democratization and trust. The paper considers the contextual and social origins of these values, and then uses them as a method of practicing anticipatory ethics: considering the impact such priorities may have on a future Internet. (shrink)

Information and communication technology is changing many aspects ofhuman endeavour and existence. This is beyond dispute for most. Whatare contested are the social and ethical implications of thesechanges. Possible sources of these disputes are the multiple ways inwhich one can conceptualize and interpret the informationtechnology/society interrelationship. Each of these ways ofconceptualization and interpretation enables one to see theinformation technology/society relationship differently and thereforeconstrue its social and ethical implications in a different manner. Atthe center of this technology/society interrelationship we find manycomplex (...) questions about the nature of the human, the technical,agency, autonomy, freedom and much more. This is indeed a vastintellectual landscape, which can obviously not be explored here inits fullness. This entry is about just one particular perspective onthis landscape. It is primarily concerned with the phenomenologicalapproach to interpreting information technology and its social andethical implications. It should be noted from the start that there isnot a unified phenomenological tradition or approach to informationtechnology in particular, or other phenomena more generally. Thephenomenological tradition consists of many different approaches thatshare certain characteristics but not all. We may however suggest, with Don Ihde,that they all accept that “phenomenology investigates theconditions of what makes things appear as such [as that which we takethem to be].” Differently stated, phenomenology suggests thatthere is a co-constitutive relationship between us and the phenomenawe encounter in our engagement with the world. In this sensephenomenologists would suggest that to understand thetechnology/society relationship we need to reveal how theyco-constitute each other—i.e. draw on each other for theirongoing meaning and sense. We will elaborate more precisely what thismeans in section 2 below. However, in order to understand the distinctiveness of thephenomenological approach other possible ways of interpreting thistechnology/society relationship will also be outlined brieflybelow., It can be said that information technology has become in a very realsense ubiquitous. Most everyday technologies such as elevators,automobiles, microwaves, watches, and so forth depend onmicroprocessors for their ongoing operation. Most organizations andinstitutions have become reliant on their information technologyinfrastructure to a lesser or greater degree. Indeed informationtechnology is seen by many as a cost-efficient way to solve amultitude of problems facing our complex contemporary society. One canalmost say that information technology has become construed as thedefault technology for solving a whole raft of technical and socialproblems such as health provision, security, governance, etc. Onecould also argue that it has become synonymous with society’sview of modernization and progress. For most it seems obvious thatinformation technology has made it possible for humans to continue toconstruct increasingly complex systems of coordination and socialordering—systems without which contemporary society would not beable to exist in its present form. To say the least, we, ascontemporary human beings, have our manner of being made possiblethrough a rather comprehensive entanglement with information andcommunication technology. Indeed, the economic, organizational andsocial benefit of information technology is not widely disputed. Thedispute is more often about the way information technology is changingor transforming the social domain, and in particular, the ethicaldomain. This dispute is largely centered around different ways ofconceptualizing and interpreting the nature of our entanglement withinformation technology. This debate is not merely an academic debateabout different and competing theoretical ‘models.’Rather, these different ways of conceptualizing it is central to ourunderstanding of how we go about managing our increasingly entangledrelationship with information technology. (shrink)

The US National Institute of Health’s Human Microbiome Project aims to use genomic techniques to understand the microbial communities that live on the human body. The emergent field of microbiome science brought together diverse disciplinary perspectives and technologies, thus facilitating the negotiation of differing values. Here, we describe how values are conceptualized and negotiated within microbiome research. Analyzing discussions from a series of interdisciplinary workshops conducted with microbiome researchers, we argue that negotiations of epistemic, social, and institutional values were inextricable (...) from the reflective and strategic category work that defined and organized the microbiome as an object of study and a potential future site of biomedical intervention. Negotiating the divergence or tension between emerging scientific and regulatory classifications also activated “values levers” and opened up reflective discussions of how classifications embody values and how these values might differ across domains. These data suggest that scholars at the intersections of science and technology studies, ethics, and policy could leverage such openings to identify and intervene in the ways that ethical/regulatory and scientific/technical practices are coproduced within unfolding research. (shrink)

Internet protocol development is a social process, and resulting protocols are shaped by their developers’ politics and values. This article argues that the work of protocol development poses barriers to developers’ reflection upon values and politics in protocol design. A participant observation of a team developing internet protocols revealed that difficulties defining the stakeholders in an infrastructure and tensions between local and global viewpoints both complicated values reflection. Further, Internet architects tended to equate a core value of interoperability with values (...) neutrality. The article describes how particular work practices within infrastructure development overcame these challenges by engaging developers in praxis: situated, lived experience of the social nature of technology. (shrink)

In the applied sciences and in engineering there is often a significant overlap between work at universities and in industry. For the individual scholar, this may lead to serious conflicts when working on joint university–industry projects. Differences in goals, such as the university’s aim to disseminate knowledge while industry aims to appropriate knowledge, might lead to complicated situations and conflicts of interest. The detailed cases of two electrical engineers and two architects working at two different universities of technology illustrate the (...) kinds of problems individual scholars face in university–business collaborations. These cases are based on qualitative interviews and additional data and demonstrate that, while value conflicts emerge on the organizational level, it is primarily the individual researcher who must deal with such conflicts. This analysis adds to existing studies in two ways: first, it explicitly addresses normative issues framed in terms of ethical and social values, thereby going beyond the common social-science perspective of university–business collaboration. Secondly, it provides qualitative insights, thereby identifying details and issues not apparent in quantitative studies. In particular, it is evident that university–industry collaborations are prone to value conflicts not only in research but also in education and job training. (shrink)

Although applications are being developed and have reached the market, nanopharmacy to date is generally still conceived as an emerging technology. Its concept is ill-defined. Nanopharmacy can also be construed as a converging technology, which combines features of multiple technologies, ranging from nanotechnology to medicine and ICT. It is still debated whether its features give rise to new ethical issues or that issues associated with nanopharma are merely an extension of existing issues in the underlying fields. We argue here that, (...) regardless of the alleged newness of the ethical issues involved, developments occasioned by technological advances affect the roles played by stakeholders in the field of nanopharmacy to such an extent that this calls for a different approach to responsible innovation in this field. Specific features associated with nanopharmacy itself and features introduced to the associated converging technologies- bring about a shift in the roles of stakeholders that call for a different approach to responsibility. We suggest that Value Sensitive Design is a suitable framework to involve stakeholders in addressing moral issues responsibly at an early stage of development of new nanopharmaceuticals. (shrink)

Academia-intelligence agency collaborations are on the rise for a variety of reasons. These can take many forms, one of which is in the classroom, using students to stand in for intelligence analysts. Classrooms, however, are ethically complex spaces, with students considered vulnerable populations, and become even more complex when layering multiple goals, activities, tools, and stakeholders over those traditionally present. This does not necessarily mean one must shy away from academia-intelligence agency partnerships in classrooms, but that these must be conducted (...) carefully and reflexively. This paper hopes to contribute to this conversation by describing one purposeful classroom encounter that occurred between a professor, students, and intelligence practitioners in the fall of 2015 at North Carolina State University: an experiment conducted as part of a graduate-level political science class that involved students working with a prototype analytic technology, a type of participatory sensing/self-tracking device, developed by the National Security Agency. This experiment opened up the following questions that this paper will explore: What social, ethical, and pedagogical considerations arise with the deployment of a prototype intelligence technology in the college classroom, and how can they be addressed? How can academia-intelligence agency collaboration in the classroom be conducted in ways that provide benefits to all parties, while minimizing disruptions and negative consequences? This paper will discuss the experimental findings in the context of ethical perspectives involved in values in design and participatory/self-tracking data practices, and discuss lessons learned for the ethics of future academia-intelligence agency partnerships in the classroom. (shrink)