With the increasing degrees A.I.s and machines are applied across different social contexts, the need for implementing ethics in A.I.s is pressing. In this paper, we argue for a pluralist hybrid model for the implementation of moral A.I.s. We first survey current approaches to moral A.I.s and their inherent limitations. Then we propose the pluralist hybrid approach and show how these limitations of moral A.I.s can be partly alleviated by the pluralist hybrid approach. The (...) core ethical decision-making capacity of an A.I. based on the pluralist hybrid approach consists of two systems. The first is a deterministic algorithm system that incorporates different moral rules for making explicit moral decisions. The second is a machine learning system that calculates the value of the variables required by the application of moral principles. The pluralist hybrid system is better than the existing proposals as (i) it better addresses the moral disagreement problem of the top-down approach by including different moral principles, while (ii) it reduces the opacity of ethical decision-making compared with a bottom-up system by implementing explicit moral principles for moral decision making. (shrink)

This chapter focuses on the connection between moral disagreement and moral relativism. Moral relativists, generally speaking, think both (i) that there is no unique objectively correct moral standard and (ii) that the rightness and wrongness of an action depends in some way on a moral standard accepted by some group or an individual. This chapter will first consider the metaphysical and epistemic arguments for moral relativism that begin from the premise that there is (...) considerable amount of moral disagreement both within individual societies and between them. The second half of the chapter, by contrast, focuses on the objection that moral relativism threatens to make us unable to have moral disagreements because it seems to make us speak past one another. This part of the chapter also evaluates relativist responses to this disagreement problem that rely on semantic opacity, disagreement in attitude, metalinguistic negotiations, and truth relativism. The chapter finally concludes by considering future directions of research in this area. (shrink)

Artificial intelligence (AI) and robotics are digital technologies that will have significant impact on the development of humanity in the near future. They have raised fundamental questions about what we should do with these systems, what the systems themselves should do, what risks they involve, and how we can control these. - After the Introduction to the field (§1), the main themes (§2) of this article are: Ethical issues that arise with AI systems as objects, i.e., tools made and used (...) by humans. This includes issues of privacy (§2.1) and manipulation (§2.2), opacity (§2.3) and bias (§2.4), human-robot interaction (§2.5), employment (§2.6), and the effects of autonomy (§2.7). Then AI systems as subjects, i.e., ethics for the AI systems themselves in machine ethics (§2.8) and artificial moral agency (§2.9). Finally, the problem of a possible future AI superintelligence leading to a “singularity” (§2.10). We close with a remark on the vision of AI (§3). - For each section within these themes, we provide a general explanation of the ethical issues, outline existing positions and arguments, then analyse how these play out with current technologies and finally, what policy consequences may be drawn. (shrink)

The increasing demand for transparency in AI has recently come under scrutiny. The question is often posted in terms of “epistemic double standards”, and whether the standards for transparency in AI ought to be higher than, or equivalent to, our standards for ordinary human reasoners. I agree that the push for increased transparency in AI deserves closer examination, and that comparing these standards to our standards of transparency for other opaque systems is an appropriate starting point. I suggest that a (...) more fruitful exploration of this question will involve a different comparison class. We routinely treat judgments made by highly trained experts in specialized fields as fair or well-grounded even though—by the nature of expert/layperson division of epistemic labor—an expert will not be able to provide an explanation of the reasoning behind these judgments that makes sense to most other people. Regardless, laypeople are thought to be acting reasonably—and ethically—in deferring to the judgments of experts that concern their areas of specialization. I suggest that we reframe our question regarding the appropriate standards of transparency in AI as one that asks when, why, and to what degree it would be ethical to accept opacity in AI. I argue that our epistemic relation to certain opaque AI technology may be relevantly similar to the layperson’s epistemic relation to the expert in certain respects, such that the successful expert/layperson division of epistemic labor can serve as a blueprint for the ethical use of opaque AI. (shrink)

The development of machine learning and deep learning (DL) in the field of AI (artificial intelligence) is the direct result of the advancement of nano-electronics. Machine learning is a function that provides the system with the capacity to learn from data without being programmed explicitly. It is basically a mathematical and probabilistic model. DL is part of machine learning methods based on artificial neural networks, simply called neural networks (NNs), as they are inspired by the biological NNs that constitute organic (...) brains. Despite its similarity to biological organs such as human brains, major problems arise in trying to attribute moral responsibility to autonomic systems based on hardware including nano-electronic devices, which are sought to replace humans (moral agents) in the context of AI. It is suggested that the required emotional environment which enables actions according to reasons in humans is not witnessed at AI devices. Though AI technology raises an enticing resemblance to human actions, this resemblance is to be considered with a skeptical eye. It is because actions are associated with reasons while causes are connected to operations. Human agents are capable of acting upon their reasons, while AI devices are limited only to operations as they are conditioned by their programming, which is considered as an embodiment of some causes. As moral responsibility goes hand in hand with the capacity to act (according to reasons), attributing moral responsibility to AI devices is revealed to be but a misleading metaphor. (shrink)

Just as humans can draw conclusions responsibly or irresponsibly, so too can computers. Machine learning systems that have been trained on data sets that include irresponsible judgments are likely to yield irresponsible predictions as outputs. In this paper I focus on a particular kind of inference a computer system might make: identification of the intentions with which a person acted on the basis of photographic evidence. Such inferences are liable to be morally objectionable, because of a way in which they (...) are presumptuous. After elaborating this moral concern, I explore the possibility that carefully procuring the training data for image recognition systems could ensure that the systems avoid the problem. The lesson of this paper extends beyond just the particular case of image recognition systems and the challenge of responsibly identifying a person’s intentions. Reflection on this particular case demonstrates the importance (as well as the difficulty) of evaluating machine learning systems and their training data from the standpoint of moral considerations that are not encompassed by ordinary assessments of predictive accuracy. (shrink)

Because of its non-representational nature, music has always had familiarity with computational and algorithmic methodologies for automatic composition and performance. Today, AI and computer technology are transforming systems of automatic music production from passive means within musical creative processes into ever more autonomous active collaborators of human musicians. This raises a large number of interrelated questions both about the theoretical problems of artificial musical creativity and about its ethical consequences. Considering two of the most urgent ethical problems of Musical AI (...) (music job replacement and machine musical authorship), we show in this essay the strict dependence of every form of acknowledgment of a moral and legal status to systems of automatic music production from the theoretical account of musical creativity by turns implicitly or explicitly adopted, arguing, on the basis of pragmatic reasons, for the necessity and the desirability of this acknowledgment. (shrink)

Tae Wan Kim, John Hooker, and Thomas Donaldson make an attempt, in recent articles, to solve the alignment problem. As they define the alignment problem, it is the issue of how to give AI systems moral intelligence. They contend that one might program machines with a version of Kantian ethics cast in deontic modal logic. On their view, machines can be aligned with human values if such machines obey principles of universalization and autonomy, as well as a (...) deontic utilitarian principle. Programming machines to do so might be useful, in their view, for applications such as future autonomous vehicles. Their proposal draws both on traditional logic-based and contemporary connectionist approaches, to fuse factual information with normative principles. I will argue that this approach makes demands of machines that go beyond what is currently feasible, and may extend past the limits of the possible for AI. I also argue that a deontological ethics for machines should place greater stress on the formula of humanity of the Kantian categorical imperative. On this principle, one ought never treat a person as a mere means. Recognition of what makes a person a person requires ethical insight. Similar insight is needed to tell treatment as a means from treatment as a mere means. The resources in Kim, Hooker, and Donaldson’s approach is insufficient for this reason. Hesitation regarding deployment of autonomous machines is warranted in light of these alignment concerns. (shrink)

When a computer system causes harm, who is responsible? This question has renewed significance given the proliferation of autonomous systems enabled by modern artificial intelligence techniques. At the root of this problem is a philosophical difficulty known in the literature as the responsibility gap. That is to say, because of the causal distance between the designers of autonomous systems and the eventual outcomes of those systems, the dilution of agency within the large and complex teams that design autonomous systems, (...) and the impossibility of fully predicting how autonomous systems will behave once deployed, determining who is morally responsible for harms caused by autonomous systems is unclear at a conceptual level. I review past work on this topic, criticizing prior works for suggesting workarounds rather than philosophical answers to the conceptual problem presented by the responsibility gap. The view I develop, drawing on my earlier work on vicarious moral responsibility, explains why computing professionals are ethically required to take responsibility for the systems they design, despite not being blameworthy for the harms these systems may cause. (shrink)

The application of modern technologies with artificial intelligence (AI) in all spheres of human life is growing exponentially alongside concern for its controllability. The lack of public, state, and international control over AI technologies creates large-scale risks of using such software and hardware that (un)intentionally harm humanity. The events of recent month and years, specifically regarding the Russian Federation’s war against its democratic neighbour Ukraine and other international conflicts of note, support the thesis that the uncontrolled use of AI, especially (...) in the military sphere, may lead to deliberate disregard for the moral standards of controlled AI or the spontaneous emergence of aggressive autonomous AI. The development of legal regulation for the use of technologies with AI is prolonged concerning the rapid development of these artefacts, which simultaneously cover all areas of public relations. Therefore, control over the creation and use of AI should be carried out not only by purely technical regulation (e.g., technical standards and conformance assessments, corporate and developer regulations, requirements enforced through industry-wide ethical codes); but also by comprehensive legislation and intergovernmental oversight bodies that codify and enforce specific changes in the rights and duties of legal persons. This article shall present the “Morality Problem” and “Intentionality Problem” of AI, and reflect upon various lacunae that arise when implementing AI for military purposes. (shrink)

Many artificial intelligence (AI) systems currently used for decision-making are opaque, i.e., the internal factors that determine their decisions are not fully known to people due to the systems’ computational complexity. In response to this problem, several researchers have argued that human decision-making is equally opaque and since simplifying, reason-giving explanations (rather than exhaustive causal accounts) of a decision are typically viewed as sufficient in the human case, the same should hold for algorithmic decision-making. Here, I contend that this (...) argument overlooks that human decision-making is sometimes significantly more transparent and trustworthy than algorithmic decision-making. This is because when people explain their decisions by giving reasons for them, this frequently prompts those giving the reasons to govern or regulate themselves so as to think and act in ways that confirm their reason reports. AI explanation systems lack this self-regulative feature. Overlooking it when comparing algorithmic and human decision-making can result in underestimations of the transparency of human decision-making and in the development of explainable AI that may mislead people by activating generally warranted beliefs about the regulative dimension of reason-giving. (shrink)

The existence of deep and persistent moral disagreement poses a problem for a defender of moral knowledge. It seems particularly clear that a philosopher who thinks that we know a great many moral truths should explain how human populations have failed to converge on those truths. In this paper, I do two things. First, I show that the problem is more difficult than it is often taken to be, and second, I criticize a popular (...) response, which involves claiming that many false moral beliefs are the product of nonmoral ignorance. (shrink)

Moral non-cognitivists hope to explain the nature of moral agreement and disagreement as agreement and disagreement in non-cognitive attitudes. In doing so, they take on the task of identifying the relevant attitudes, distinguishing the non-cognitive attitudes corresponding to judgements of moral wrongness, for example, from attitudes involved in aesthetic disapproval or the sports fan’s disapproval of her team’s performance. We begin this paper by showing that there is a simple recipe for generating apparent counterexamples to (...) any informative specification of the moral attitudes. This may appear to be a lethal objection to non-cognitivism, but a similar recipe challenges attempts by non-cognitivism’s competitors to specify the conditions underwriting the contrast between genuine and merely apparent moral disagreement. Because of its generality, this specification problem requires a systematic response, which, we argue, is most easily available for the non-cognitivist. Building on premisses congenial to the non-cognitivist tradition, we make the following claims: (1) In paradigmatic cases, wrongness-judgements constitute a certain complex but functionally unified state, and paradigmatic wrongness-judgements form a functional kind, preserved by homeostatic mechanisms. (2) Because of the practical function of such judgements, we should expect judges’ intuitive understanding of agreement and disagreement to be accommodating, treating states departing from the paradigm in various ways as wrongness-judgements. (3) This explains the intuitive judgements required by the counterexample-generating recipe, and more generally why various kinds of amoralists are seen as making genuine wrongness-judgements. (shrink)

The problem of moral disagreement has been presented as an objection to contextualist semantics for ‘ought’, since it is not clear that contextualism can accommodate or give a convincing gloss of such disagreement. I argue that independently of our semantics, disagreements over ‘ought’ in non-cooperative contexts are best understood as indirect metalinguistic disputes, which is easily accommodated by contextualism. If this is correct, then rather than posing a problem for contextualism, the data from moral (...) disagreements provides some reason to adopt a semantics that allows contextual variance in the meanings of ‘ought’. (shrink)

Our rapidly increasing reliance on frictionless AI interactions may increase cognitive and emotional distance, thereby letting our adaptive resilience slacken and our ethical virtues atrophy from disuse. Many trends already well underway involve the offloading of cognitive, emotional, and ethical labor to AI software in myriad social, civil, personal, and professional contexts. Gradually, we may lose the inclination and capacity to engage in critically reflective thought, making us more cognitively and emotionally vulnerable and thus more anxious and prone to manipulation (...) from false news, deceptive advertising, and political rhetoric. In this article, I consider the overarching features of this problem and provide a framework to help AI designers tackle it through system enhancements in smartphones and other products and services in the burgeoning internet of things (IoT) marketplace. The framework is informed by two ideas: psychologist Daniel Kahneman’s cognitive dual process theory and moral self-awareness theory, a four-level model of moral identity that I developed with Benjamin M. Cole. (shrink)

On the shared-­ends account of close friendship, proper care for a friend as an agent requires seeing yourself as having important reasons to accommodate and promote the friend’s valuable ends for her own sake. However, that friends share ends doesn't inoculate them against disagreements about how to pursue those ends. This paper defends the claim that, in certain circumstances of reasonable disagreement, proper care for a friend as a practical and moral agent sometimes requires allowing her judgment to (...) decide what you are to do, even when you disagree with her judgment (and even when her judgment is in fact mistaken). In these instances, your friendship can make it the case that you may not act on your own practical and even moral judgments because, at those times, you have a duty as her close friend to defer to her judgments. As a result, treating your friend properly as a responsible agent can require that you assist her in committing what may in fact be serious moral wrongs. (shrink)

Motivated by the question of responsible AI and value alignment, I seek to offer a uniquely Foucauldian reconstruction of the problem as the emergence of an ethical subject in a disciplinary setting. This reconstruction contrasts with the strictly human-oriented programme typical to current scholarship that often views technology in instrumental terms. With this in mind, I problematise the concept of a technological subjectivity through an exploration of various aspects of ChatGPT in light of Foucault’s work, arguing that current systems (...) lack the reflexivity and self-formative characteristics inherent in the notion of the subject. By drawing upon a recent dialogue between Foucault and phenomenology, I suggest four techno-philosophical desiderata that would address the gaps in this search for a technological subjectivity: embodied self-care, embodied intentionality, imagination and reflexivity. Thus I propose that advanced AI be reconceptualised as a subject capable of “technical” self-crafting and reflexive self-conduct, opening new pathways to grasp the intertwinement of the human and the artificial. This reconceptualization holds the potential to render future AI technology more transparent and responsible in the circulation of knowledge, care and power. (shrink)

The existence of fundamental moral disagreements is a central problem for moral realism and has often been contrasted with an alleged absence of disagreement in mathematics. However, mathematicians do in fact disagree on fundamental questions, for example on which set-theoretic axioms are true, and some philosophers have argued that this increases the plausibility of moral vis-à-vis mathematical realism. I argue that the analogy between mathematical and moral disagreement is not as straightforward as those (...) arguments present it. In particular, I argue that pluralist accounts of mathematics render fundamental mathematical disagreements compatible with mathematical realism in a way in which moral disagreements and moral realism are not. 11. (shrink)

In his recent work, "The Collapse of the Fact-Value Dichotomy," Hilary Putnam traces the history of the fact-value dichotomy from Hume to Stevenson and Logical Positivism. The aim of this historical reconstruction is to undermine the foundations of the dichotomy, showing that it is of a piece with the dichotomy - untenable, as we know now - of "analytic" and "synthetic" judgments. Putnam's own thesis is that facts and values are "entangled" in a way that precludes any attempt to draw (...) a sharp distinction between "value judgments" and "matters of fact." The idea of an "entanglement of facts and values" Putnam rightly attributes to John Dewey, who - we should add - made of it the main issue in his controversy with Logical Positivism. Nevertheless, a closer inspection of the problem whose history Putnam summarizes could bring into light important aspects of it that have been neglected. It is worth reading in this connection the intercourse between Dewey and Stevenson. Secondly, it is striking that Putnam's version of the history of the fact-value dichotomy hardly mentions the problem that caused this very dichotomy to arise in the first place: i. e., the problem of insoluble moral disagreements. By contrast, Dewey's attack on the dualism of fact and value can be read as an attempt to redescribe this kind of disagreements in a way that makes room for intelligent inquiry, and consequently for rational expectations of agreement. The "entanglement thesis" surely is a part of this re-description. But then it must have implications - particularly for the analysis of value-justification - which are overlooked, or by-passed, by Putnam. This leaves the question open whether Putnam and Dewey subscribe to different versions of pragmatism with regard to norms and values. (shrink)

Abstract: Artificial Morality is a new, emerging interdisciplinary field that centres around the idea of creating artificial moral agents, or AMAs, by implementing moral competence in artificial systems. AMAs are ought to be autonomous agents capable of socially correct judgements and ethically functional behaviour. This request for moral machines comes from the changes in everyday practice, where artificial systems are being frequently used in a variety of situations from home help and elderly care purposes to banking and (...) court algorithms. It is therefore important to create reliable and responsible machines based on the same ethical principles that society demands from people. New challenges in creating such agents appear. There are philosophical questions about a machine’s potential to be an agent, or mora l agent, in the first place. Then comes the problem of social acceptance of such machines, regardless of their theoretic agency status. As a result of efforts to resolve this problem, there are insinuations of needed additional psychological (emotional and cogn itive) competence in cold moral machines. What makes this endeavour of developing AMAs even harder is the complexity of the technical, engineering aspect of their creation. Implementation approaches such as top- down, bottom-up and hybrid approach aim to find the best way of developing fully moral agents, but they encounter their own problems throughout this effort. (shrink)

The existence of deep moral disagreement is used in support of views ranging from moral relativism to the impossibility of moral expertise. This is done despite the fact that it is not at all clear whether deep moral disagreements actually occur, as the usually given examples are never of real life situations, but of some generalized debates on controversial issues. The paper will try to remedy this, as any strength of arguments appealing to deep (...) class='Hi'>moral disagreement is partly depended on the fact the disagreement exists. This will be done by showing that some real life conflicts that are intractable, i.e. notoriously difficult to resolve, share some important features with deep moral disagreement. The article also deals with the objection that the mere conceptual possibility renders illustrations of actually happening deep moral disagreements unnecessary. The problem with such objection is that it depends on theoretical assumptions (i.e. denial of moral realism) that are not uncontroversial. Instead, the article claims we need not only suppose deep moral disagreements exist because they actually occur when some intractable conflicts occur. Thus, in so far as to the deep moral disagreement’s existence, the arguments appealing to it are safe. But as intractable conflicts can be resolved, by seeing deep moral disagreements as constitutive part of them, we might have to consider whether deep moral disagreements are resolvable too. A brief suggestion of how that might look like is given in the end of the paper. (shrink)

In the case of AI, automated systems are making increasingly complex decisions with significant ethical implications, raising questions about who is responsible for decisions made by AI and how to ensure that these decisions align with society's ethical and moral values, both in India and the West. Jonathan Haidt has conducted research on moral and ethical decision-making. Today, solving problems like decision-making in autonomous vehicles can draw on the literature of the trolley dilemma in that it illustrates the (...) complexity of ethical decisions faced in emergency situations, as well as the moral implications of the decisions made. However, a series of moral principles must be understood and cited to avoid insufficient mechanistic explanations. Here we present a compilation of ethical positions both in India and the West. (shrink)

We paraphrase Descartes’ famous dictum in the area of AI ethics where the “I doubt and therefore I am” is suggested as a necessary aspect of morality. Therefore AI, which cannot doubt itself, cannot possess moral agency. Of course, this is not the end of the story. We explore various aspects of the human mind that substantially differ from AI, which includes the sensory grounding of our knowing, the act of understanding, and the significance of being able to doubt (...) ourselves. The foundation of our argument is the discipline of ethics, one of the oldest and largest knowledge projects of human history, yet, we seem only to be beginning to get a grasp of it. After a couple of thousand years of studying the ethics of humans, we (humans) arrived at a point where moral psychology suggests that our moral decisions are intuitive, and all the models from ethics become relevant only when we explain ourselves. This recognition has a major impact on what and how we can do regarding AI ethics. We do not offer a solution, we explore some ideas and leave the problem open, but we hope somewhat better understood than before our study. (shrink)

Endowing artificial systems with explanatory capacities about the reasons guiding their decisions, represents a crucial challenge and research objective in the current fields of Artificial Intelligence (AI) and Computational Cognitive Science [Langley et al., 2017]. Current mainstream AI systems, in fact, despite the enormous progresses reached in specific tasks, mostly fail to provide a transparent account of the reasons determining their behavior (both in cases of a successful or unsuccessful output). This is due to the fact that the classical (...) class='Hi'>problem of opacity in artificial neural networks (ANNs) explodes with the adoption of current Deep Learning techniques [LeCun, Bengio, Hinton, 2015]. In this paper we argue that the explanatory deficit of such techniques represents an important problem, that limits their adoption in the cognitive modelling and computational cognitive science arena. In particular we will show how the current attempts of providing explanations of the deep nets behaviour (see e.g. [Ritter et al. 2017] are not satisfactory. As a possibile way out to this problem, we present two different research strategies. The first strategy aims at dealing with the opacity problem by providing a more abstract interpretation of neural mechanisms and representations. This approach is adopted, for example, by the biologically inspired SPAUN architecture [Eliasmith et al., 2012] and by other proposals suggesting, for example, the interpretation of neural networks in terms of the Conceptual Spaces framework [Gärdenfors 2000, Lieto, Chella and Frixione, 2017]. All such proposals presuppose that the neural level of representation can be considered somehow irrelevant for attacking the problem of explanation [Lieto, Lebiere and Oltramari, 2017]. In our opinion, pursuing this research direction can still preserve the use of deep learning techniques in artificial cognitive models provided that novel and additional results in terms of “transparency” are obtained. The second strategy is somehow at odds with respect to the previous one and tries to address the explanatory issue by avoiding to directly solve the “opacity” problem. In this case, the idea is that one of resorting to pre-compiled plausible explanatory models of the word used in combination with deep-nets (see e.g. [Augello et al. 2017]). We argue that this research agenda, even if does not directly fits the explanatory needs of Computational Cognitive Science, can still be useful to provide results in the area of applied AI aiming at shedding light on the models of interaction between low level and high level tasks (e.g. between perceptual categorization and explanantion) in artificial systems. (shrink)

Examining AI spirituality can illuminate problematic assumptions about human spirituality and AI cognition, suggest possible directions for AI development, reduce uncertainty about future AI, and yield a methodological lens sufficient to investigate human-AI sociotechnical interaction and morality. Incompatible philosophical assumptions about human spirituality and AI limit investigations of both and suggest a vast gulf between them. An emergentist approach can replace dualist assumptions about human spirituality and identify emergent behavior in AI computation to overcome overly reductionist assumptions about computation. Using (...) general systems theory to organize models of human experience yields insight into human morality and spirituality, upon which AI modeling can also draw. In this context, the pragmatist Josiah Royce’s semiotic philosophy of spirituality identifies unanticipated overlap between symbolic AI and spirituality and suggests criteria for a human-AI community focused on modeling morality that would result in an emergent Interpreter-Spirit sufficient to influence the ongoing development of human and AI morality and spirituality. (shrink)

This paper presents a new trilemma with respect to resolving the control and alignment problems in machine ethics. Section 1 outlines three possible types of artificial moral agents (AMAs): (1) 'Inhuman AMAs' programmed to learn or execute moral rules or principles without understanding them in anything like the way that we do; (2) 'Better-Human AMAs' programmed to learn, execute, and understand moral rules or principles somewhat like we do, but correcting for various sources of human moral (...) error; and (3) 'Human-Like AMAs' programmed to understand and apply moral values in broadly the same way that we do, with a human-like moral psychology. Sections 2–4 then argue that each type of AMA generates unique control and alignment problems that have not been fully appreciated. Section 2 argues that Inhuman AMAs are likely to behave in inhumane ways that pose serious existential risks. Section 3 then contends that Better-Human AMAs run a serious risk of magnifying some sources of human moral error by reducing or eliminating others. Section 4 then argues that Human-Like AMAs would not only likely reproduce human moral failures, but also plausibly be highly intelligent, conscious beings with interests and wills of their own who should therefore be entitled to similar moral rights and freedoms as us. This generates what I call the New Control Problem: ensuring that humans and Human-Like AMAs exert a morally appropriate amount of control over each other. Finally, Section 5 argues that resolving the New Control Problem would, at a minimum, plausibly require ensuring what Hume and Rawls term ‘circumstances of justice’ between humans and Human-Like AMAs. But, I argue, there are grounds for thinking this will be profoundly difficult to achieve. I thus conclude on a skeptical note. Different approaches to developing ‘safe, ethical AI’ generate subtly different control and alignment problems that we do not currently know how to adequately resolve, and which may or may not be ultimately surmountable. (shrink)

The aim of this thesis is to advance a philosophically justifiable account of Artificial Moral Agency (AMA). Concerns about the moral status of Artificial Intelligence (AI) traditionally turn on questions of whether these systems are deserving of moral concern (i.e. if they are moral patients) or whether they can be sources of moral action (i.e. if they are moral agents). On the Organic View of Ethical Status, being a moral patient is a necessary (...) condition for an entity to qualify as a moral agent. This view claims that because artificial agents (AAs) lack sentience, they cannot be proper subjects of moral concern and hence cannot be considered to be moral agents. I raise conceptual and epistemic issues with regards to the sense of sentience employed on this view, and I argue that the Organic View does not succeed in showing that machines cannot be moral patients. Nevertheless, irrespective of this failure, I also argue that the entire project is misdirected in that moral patiency need not be a necessary condition for moral agency. Moreover, I claim that whereas machines may conceivably be moral patients in the future, there is a strong case to be made that they are (or will very soon be) moral agents. Whereas it is often argued that machines cannot be agents simpliciter, let alone moral agents, I claim that this argument is predicated on a conception of agency that makes unwarranted metaphysical assumptions even in the case of human agents. Once I have established the shortcomings of this “standard account”, I move to elaborate on other, more plausible, conceptions of agency, on which some machines clearly qualify as agents. Nevertheless, the argument is still often made that while some machines may be agents, they cannot be moral agents, given their ostensible lack of the requisite phenomenal states. Against this thesis, I argue that the requirement of internal states for moral agency is philosophically unsound, as it runs up against the problem of other minds. In place of such intentional accounts of moral agency, I provide a functionalist alternative, which makes conceptual room for the existence of AMAs. The implications of this thesis are that at some point in the future we may be faced with situations for which no human being is morally responsible, but a machine may be. Moreover, this responsibility holds, I claim, independently of whether the agent in question is “punishable” or not. (shrink)

Why should moral philosophers, moral psychologists, and machine ethicists care about computational complexity? Debates on whether artificial intelligence (AI) can or should be used to solve problems in ethical domains have mainly been driven by what AI can or cannot do in terms of human capacities. In this paper, we tackle the problem from the other end by exploring what kind of moral machines are possible based on what computational systems can or cannot do. To do (...) so, we analyze normative ethics through the lens of computational complexity. First, we introduce computational complexity for the uninitiated reader and discuss how the complexity of ethical problems can be framed within Marr’s three levels of analysis. We then study a range of ethical problems based on consequentialism, deontology, and virtue ethics, with the aim of elucidating the complexity associated with the problems themselves (e.g., due to combinatorics, uncertainty, strategic dynamics), the computational methods employed (e.g., probability, logic, learning), and the available resources (e.g., time, knowledge, learning). The results indicate that most problems the normative frameworks pose lead to tractability issues in every category analyzed. Our investigation also provides several insights about the computational nature of normative ethics, including the differences between rule- and outcome-based moral strategies, and the implementation-variance with regard to moral resources. We then discuss the consequences complexity results have for the prospect of moral machines in virtue of the trade-off between optimality and efficiency. Finally, we elucidate how computational complexity can be used to inform both philosophical and cognitive-psychological research on human morality by advancing the moral tractability thesis. (shrink)

In this paper, I discuss a new problem for moral realism, the problem of moral aliens. In the first section, I introduce this problem. Moral aliens are people who radically disagree with us concerning moral matters. Moral aliens are neither obviously incoherent nor do they seem to lack rational support from their own perspective. On the one hand, moral realists claim that we should stick to our guns when we encounter (...) class='Hi'>moral aliens. On the other hand, moral realists, in contrast to anti-realists, seem to be committed to an epistemic symmetry between us and our moral aliens that forces us into rational suspension of our moral beliefs. Unless one disputes the very possibility of moral aliens, this poses a severe challenge to the moral realist. In the second section, I will address this problem. It will turn out that, on closer scrutiny, we cannot make any sense of the idea that moral aliens should be taken as our epistemic peers. Consequently, there is no way to argue that encountering moral aliens gives us any reason to revise our moral beliefs. If my argument is correct, the possibility of encountering moral aliens poses no real threat to moral realism. (shrink)

As ever more advanced digital systems are created, it becomes increasingly likely that some of these systems will be digital minds, i.e. digital subjects of experience. With digital minds comes the risk of digital suffering. The problem of digital suffering is that of mitigating this risk. We argue that the problem of digital suffering is a high stakes moral problem and that formidable epistemic obstacles stand in the way of solving it. We then propose a strategy (...) for solving it: Access Monitor Prevent (AMP). AMP uses a ‘dancing qualia’ argument to link the functional states of certain digital systems to their experiences—this yields epistemic access to digital minds. With that access, we can prevent digital suffering by only creating advanced digital systems that we have such access to, monitoring their functional profiles, and preventing them from entering states with functional markers of suffering. After introducing and motivating AMP, we confront limitations it faces and identify some options for overcoming them. We argue that AMP fits especially well with—and so provides a moral reason to prioritize—one approach to creating such systems: whole brain emulation. We also contend that taking other paths to digital minds would be morally risky. (shrink)

Advancements in machine learning have fuelled the popularity of using AI decision algorithms in procedures such as bail hearings, medical diagnoses and recruitment. Academic articles, policy texts, and popularizing books alike warn that such algorithms tend to be opaque: they do not provide explanations for their outcomes. Building on a causal account of transparency and opacity as well as recent work on the value of causal explanation, I formulate a moral concern for opaque algorithms that is yet to (...) receive a systematic treatment in the literature: when such algorithms are used in life-changing decisions, they can obstruct us from effectively shaping our lives according to our goals and preferences, thus undermining our autonomy. I argue that this concern deserves closer attention as it furnishes the call for transparency in algorithmic decision-making with both new tools and new challenges. (shrink)

According to moral intuitionism, at least some moral seeming states are justification-conferring. The primary defense of this view currently comes from advocates of the standard account, who take the justification-conferring power of a moral seeming to be determined by its phenomenological credentials alone. However, the standard account is vulnerable to a problem. In brief, the standard account implies that moral knowledge is seriously undermined by those commonplace moral disagreements in which both agents have equally (...) good phenomenological credentials supporting their disputed moral beliefs. However, it is implausible to think that commonplace disagreement seriously undermines moral knowledge, and thus it is implausible to think that the standard account of moral intuitionism is true. (shrink)

Intractable political disagreements threaten to fracture the common ground upon which we can build a political community. The deepening divisions in society are partly fueled by the ways social media has shaped political engagement. Social media allows us to sort ourselves into increasingly likeminded groups, consume information from different sources, and end up in polarized and insular echo chambers. To solve this, many argue for various ways of cultivating more responsible epistemic agency. This chapter argues that this epistemic lens does (...) not reveal the complete picture and therefore misses a form of moral respect required to reestablish cooperation across disagreements in a divided society. The breakdown of discourse online provides renewed reasons to draw out not an epistemic but a moral basis for political cooperation among diverse citizens—one inspired by Rawlsian political liberalism. We need ways to cultivate mutual respect for our fellow citizens in order to reestablish common moral ground for political debate. (shrink)

Unlike human soldiers, autonomous weapons systems are unaffected by psychological factors that would cause them to act outside the chain of command. This is a compelling moral justification for their development and eventual deployment in war. To achieve this level of sophistication, the software that runs AWS will have to first solve two problems: the frame problem and the representation problem. Solutions to these problems will inevitably involve complex software. Complex software will create security risks and will (...) make AWS critically vulnerable to hacking. I claim that the political and tactical consequences of hacked AWS far outweigh the purported advantages of AWS not being affected by psychological factors and always following orders. Therefore, one of the moral justifications for the deployment of AWS is undermined. (shrink)

To speak of being religious lucky certainly sounds odd. But then, so does “My faith holds value in God’s plan, while yours does not.” This book argues that these two concerns — with the concept of religious luck and with asymmetric or sharply differential ascriptions of religious value — are inextricably connected. It argues that religious luck attributions can profitably be studied from a number of directions, not just theological, but also social scientific and philosophical. There is a strong tendency (...) among adherents of different faith traditions to invoke asymmetric explanations of the religious value or salvific status of the home religion vis-à-vis all others. Attributions of good/bad religious luck and exclusivist dismissal of the significance of religious disagreement are the central phenomena that the book studies. Part I lays out a taxonomy of kinds of religious luck, a taxonomy that draws upon but extends work on moral and epistemic luck. It asks: What is going on when persons, theologies, or purported revelations ascribe various kinds of religiously-relevant traits to insiders and outsiders of a faith tradition in sharply asymmetric fashion? “I am saved but you are lost”; “My religion is holy but yours is idolatrous”; “My faith tradition is true, and valued by God, but yours is false and valueless.” Part II further develops the theory introduced in Part I, pushing forward both the descriptive/explanatory and normative sides of what the author terms his inductive risk account. Firstly, the concept of inductive risk is shown to contribute to the needed field of comparative fundamentalism by suggesting new psychological markers of fundamentalist orientation. The second side of what is termed an inductive risk account is concerned with the epistemology of religious belief, but more especially with an account of the limits of reasonable religious disagreement. Problems of inductively risky modes of belief-formation problematize claims to religion-specific knowledge. But the inductive risk account does not aim to set religion apart, or to challenge the reasonableness of religious belief tout court. Rather the burden of the argument is to challenge the reasonableness of attitudes of religious exclusivism, and to demotivate the “polemical apologetics” that exclusivists practice and hope to normalize. Lexington Books Pages: 290 978-1-4985-5017-8 • Hardback • December 2018 • $95.00 • (£65.00) 978-1-4985-5018-5 • eBook • December 2018 • $90.00 • (£60.00) ISBN 978-1-4985-5018-5 (pbk: alk. paper) (coming 2020) [Download the 30% personal use Discount Order Form I uploaded for hardcover or e-book, and please ask your library to purchase a copy for their collection.]. (shrink)

Although moral relativists often appeal to cases of apparent moral disagreement between members of different communities to motivate their view, accounting for these exchanges as evincing genuine disagreements constitutes a challenge to the coherence of moral relativism. While many moral relativists acknowledge this problem, attempts to solve it so far have been wanting. In response, moral relativists either give up the claim that there can be moral disagreement between members of different (...) communities or end up with a view on which these disagreements have no “epistemic significance” because they are always faultless. This paper introduces an alternative strategy: accounting for disagreement in terms of “metalinguistic negotiation”. It argues that this strategy constitutes a better solution to the challenge disagreement poses for moral relativists because it leads to a nuanced understanding of the epistemic significance of moral disagreement between members of different communities. The upshot is a novel account of disagreement for moral relativists that has consequences for how moral relativism should be understood. (shrink)

SPECIAL ISSUE ON DISAGREEMENTS: The fact of moral disagreement is often raised as a problem for moral realism. The idea is that disagreement amongst people or communities on moral issues is to be taken as evidence that there are no objective moral facts. While the fact of ‘folk’ moral disagreement has been of interest, the fact of expert moral disagreement, that is, widespread and longstanding disagreement amongst expert (...) class='Hi'>moral philosophers, is even more compelling. In this paper, I present three arguments against the anti-realist explanation for widespread and longstanding disagreement amongst expert moral philosophers. Each argument shows the argument from expert disagreement for moral anti-realism, that is, denial of morality’s objectivity, to be in one way or another self-undermining. I conclude that widespread and longstanding disagreement amongst expert moral philosophers is not a problem for moral realism. (shrink)

Techno-ethics is the area in the philosophy of technology which deals with emerging robotic and digital AI technologies. In the last decade, a new techno-ethical challenge has emerged: Autonomous Weapon Systems (AWS), defensive and offensive (the article deals only with the latter). Such AI-operated lethal machines of various forms (aerial, marine, continental) raise substantial ethical concerns. Interestingly, the topic of AWS was almost not treated in Jewish law and its research. This article thus proposes an introductory ethical-halakhic perspective on AWS, (...) in the Israeli context. The article has seven sections. Section 1 defines AWS and the main ethical concerns it evokes, while providing elementary definitions and distinctions. §2 locates AWS within the realm of Jewish laws of war (hilkhot-ẓava), which recognize the right for self-defense, as well as the status of universally accepted moral norms. §3 unfolds pragmatic ethical premises of a humane techno-ethics, which are required for the identifying AWS as a moral question: I. Relationality; II. Technology is not (completely) neutral; III. The fallaciousness of transhumanism. It is argued that these premises are compatible with halakhic tradition. §4 investigates the question of the morality of AWS, within the field of military AI ethics. It is clarified why the standard categories of war-ethics (ad bello, in bellum) do not capture the singular ethical problem of AWS, which pertains to the operation of military means, rather than their human targets. It is argued that reductive perception of the human mind is misleading about the feasibility of ‘ethical robots’, capable of independent moral discretion. To provide a thick examination of human agency from the perspective of Jewish tradition, §5 explores two stories from the biblical book of Samuel (the murder of Nob’s priests and of Uriah). The lessons about the significance of moral agency within the pubic-political military sphere are made explicit, as well as the possible costs resulting from the loss of human agency in the case of AWS. Given that realpolitik considerations are basic in halakhah, §6 considers some possible contemporary socio-political implications of the AWS, that may risk the sustainability of the democratic project. §7 concludes by pointing out humane contributions of Jewish law to contemporary techno-ethics. (shrink)

Where artificial agents are not liable to be ascribed true moral agency and responsibility in their own right, we can understand them as acting as proxies for human agents, as making decisions on their behalf. What I call the ‘Moral Proxy Problem’ arises because it is often not clear for whom a specific artificial agent is acting as a moral proxy. In particular, we need to decide whether artificial agents should be acting as proxies for low-level (...) agents — e.g. individual users of the artificial agents — or whether they should be moral proxies for high-level agents — e.g. designers, distributors or regulators, that is, those who can potentially control the choice behaviour of many artificial agents at once. Who we think an artificial agent is a moral proxy for determines from which agential perspective the choice problems artificial agents will be faced with should be framed: should we frame them like the individual choice scenarios previously faced by individual human agents? Or should we, rather, consider the expected aggregate effects of the many choices made by all the artificial agents of a particular type all at once? This paper looks at how artificial agents should be designed to make risky choices, and argues that the question of risky choice by artificial agents shows the moral proxy problem to be both practically relevant and difficult. (shrink)

Autonomy has been central to moral and political philosophy for millenia, and has been positioned as a critical aspect of both justice and wellbeing. Research in psychology supports this position, providing empirical evidence that autonomy is critical to motivation, personal growth and psychological wellness. Responsible AI will require an understanding of, and ability to effectively design for, human autonomy (rather than just machine autonomy) if it is to genuinely benefit humanity. Yet the effects on human autonomy of digital experiences (...) are neither straightforward nor consistent, and are complicated by commercial interests and tensions around compulsive overuse. This multi-layered reality requires an analysis that is itself multidimensional and that takes into account human experience at various levels of resolution. We borrow from HCI and psychological research to apply a model (“METUX”) that identifies six distinct spheres of technology experience. We demonstrate the value of the model for understanding human autonomy in a technology ethics context at multiple levels by applying it to the real-world case study of an AI-enhanced video recommender system. In the process we argue for the following three claims: 1) There are autonomy-related consequences to algorithms representing the interests of third parties, and they are not impartial and rational extensions of the self, as is often perceived; 2) Designing for autonomy is an ethical imperative critical to the future design of responsible AI; and 3) Autonomy-support must be analysed from at least six spheres of experience in order to approriately capture contradictory and downstream effects. (shrink)

Philosophers should consider a hybrid meta-ethical theory that includes elements of both moral expressivism and moral error theory. Proponents of such an expressivist-error theory hold that all moral utterances are either expressions of attitudes or expressions of false beliefs. Such a hybrid theory has two advantages over pure expressivism, because hybrid theorists can offer a more plausible account of the moral utterances that seem to be used to express beliefs, and hybrid theorists (...) can provide a simpler solution to the Frege-Geach problem. The hybrid theory has three advantages over pure error theory, because hybrid theorists can offer a more plausible account of the moral utterances that seem to be used to express attitudes, hybrid theorists can more easily explain moral motivation, and hybrid theorists can avoid the implausible claim that all moral discourse is radically mistaken. Accordingly, such a hybrid theory should be more attractive than pure expressivism or pure error theory to philosophers who are skeptical about moral facts and truth. (shrink)

We sometimes seek expert guidance when we don’t know what to think or do about a problem. In challenging cases concerning medical ethics, we may seek a clinical ethics consultation for guidance. The assumption is that the bioethicist, as an expert on ethical issues, has knowledge and skills that can help us better think about the problem and improve our understanding of what to do regarding the issue. The widespread practice of ethics consultations raises these questions and more: (...) -/- • What would it take to be a moral expert? • Is anyone a moral expert, and if so, how could a non-expert identify one? • Is it in any way problematic to accept and follow the advice of a moral expert as opposed to coming to moral conclusions on your own? • What should we think and do when moral experts disagree about a practical ethical issue? -/- In what follows, we address these theoretical and practical questions about moral expertise. (shrink)

There is a need recognized by the National Institute of Dental & Craniofacial Research and the National Cancer Institute to advance basic, translational and clinical saliva research. The goal of the Salivaomics Knowledge Base (SKB) is to create a data management system and web resource constructed to support human salivaomics research. To maximize the utility of the SKB for retrieval, integration and analysis of data, we have developed the Saliva Ontology and SDxMart. This article reviews the informatics advances in saliva (...) diagnostics made possible by the Saliva Ontology and SDxMart. (shrink)

In this chapter, we discuss a selection of current views of the neural correlates of consciousness (NCC). We focus on the different predictions they make, in particular with respect to the role of prefrontal cortex (PFC) during visual experiences, which is an area of critical interest and some source of contention. Our discussion of these views focuses on the level of functional anatomy, rather than at the neuronal circuitry level. We take this approach because we currently understand more about experimental (...) evidence at this coarse level and because these results are appropriate for arbitrating between current theoretical frameworks. We discuss the Two-Visual-Systems Hypothesis (Milner & Goodale 1995; 2006), Local Recurrency (Lamme 2010; Lamme 2006), Higher Order (Lau 2008; Lau & Rosenthal 2011) and Global Workspace theories (Baars 1997; Baars 2005; Dehaene & Naccache 2001; Dehaene 2014). Despite the apparent stark differences between conscious and unconscious perceptual processing, available evidence suggests that their neural substrates must be largely shared. This indicates that the difference in neural activity between conscious and unconscious perceptual processing is likely to be subtle and highly specialized. We argue that current experimental evidence about the involvement of specific activity in prefrontal cortex supports the higher order neural theory of consciousness. In consequence, imaging techniques that focus only on marked differences between conscious and unconscious level of activity are likely to be insensitive to the relevant neural activity patterns that underlie conscious experiences. Finally, it follows from the evidence we discuss that the functional advantages of conscious over unconscious perceptual processing may be more limited than commonly thought. (shrink)

As Christian List (2021) has recently argued, the increasing arrival of powerful AI systems that operate autonomously in high-stakes contexts creates a need for “future-proofing” our regulatory frameworks, i.e., for reassessing them in the face of these developments. One core part of our regulatory frameworks that dominates our everyday moral interactions is blame. Therefore, “future-proofing” our extant regulatory frameworks in the face of the increasing arrival of powerful AI systems requires, among others things, that we ask whether it makes (...) sense to extend our blaming practices to these systems. In the paper, we argue for the admittedly surprising thesis that this question should be answered in the affirmative: contrary to what one might initially think, it can make a lot of sense to blame AI systems, since, as we furthermore argue, many of the important functions that are fulfilled by blaming humans can also be served by blaming AI systems. The paper concludes that this result gives us a good pro tanto reason to extend our blame practices to AI systems. (shrink)

In The Stubborn System of Moral Responsibility (2015), Bruce Waller sets out to explain why the belief in individual moral responsibility is so strong. He begins by pointing out that there is a strange disconnect between the strength of philosophical arguments in support of moral responsibility and the strength of philosophical belief in moral responsibility. While the many arguments in favor of moral responsibility are inventive, subtle, and fascinating, Waller points out that even the most (...) ardent supporters of moral responsibility acknowledge that the arguments in its favor are far from conclusive; and some of the least confident concerning the arguments for moral responsibility—such as Van Inwagen—are most confident of the truth of moral responsibility. Thus, argues Waller, whatever the verdict on the strength of philosophical arguments for moral responsibility, it is clear that belief in moral responsibility—whether among philosophers or the folk—is based on something other than philosophical reasons. -/- He goes on to argue that there are several sources for the strong belief in moral responsibility, but the following four are particularly influential: First, moral responsibility is based in a powerful “strike back” emotion that we share with other animals. Second, there is a deep-rooted “belief in a just world”—a belief that, according to Waller, most philosophers reject when they consciously consider it, but which has a deep nonconscious influence on what we regard as just treatment and which provides subtle (but mistaken) support for belief in moral responsibility. Third, there is a pervasive moral responsibility system—extending over criminal justice as well as “common sense”—that makes the truth of moral responsibility seem obvious, and makes challenges to moral responsibility seem incoherent. Finally, there is the enormous confidence we have in the power of reason, which mistakenly leads us to believe that our conscious, rational, and critically reflective selves are constantly guiding our behavior in accordance with our deep values. -/- In these comments, I would like to discuss the many points of agreement I have with Waller, providing along the way additional fuel for his skeptical fire (i.e., his moral responsibility skepticism and his skeptical analysis of the source of our strong belief in moral responsibility). I will also discuss, however, my one main point of disagreement—i.e., his desire to preserve the conception of free will. Waller believes free will can “flourish” in the absence of moral responsibility (see Ch.8), while I maintain they that the variety of free will that is of central philosophical and practical importance is the sort required for moral responsibility in a particular but pervasive sense. This sense of moral responsibility is set apart by the notion of basic desert and is purely backward-looking and non-consequentialist (see Pereboom 2001, 2014; Caruso and Morris, forthcoming). Understood this way, the sort of free will at issue in the historical debate is a kind of power or ability an agent must possess in order to justify certain kinds of desert-based judgments, attitudes, or treatments in response to decisions or actions that the agent performed or failed to perform. (shrink)

Artificial intelligence (AI) is a digital technology that will be of major importance for the development of humanity in the near future. AI has raised fundamental questions about what we should do with such systems, what the systems themselves should do, what risks they involve and how we can control these. - After the background to the field (1), this article introduces the main debates (2), first on ethical issues that arise with AI systems as objects, i.e. tools made and (...) used by humans; here, the main sections are privacy (2.1), manipulation (2.2), opacity (2.3), bias (2.4), autonomy & responsibility (2.6) and the singularity (2.7). Then we look at AI systems as subjects, i.e. when ethics is for the AI systems themselves in machine ethics (2.8.) and artificial moral agency (2.9). Finally we look at future developments and the concept of AI (3). For each section within these themes, we provide a general explanation of the ethical issues, we outline existing positions and arguments, then we analyse how this plays out with current technologies and finally what policy conse-quences may be drawn. (shrink)

The problem of personal identity is a longstanding philosophical topic albeit without final consensus. In this article the somewhat similar problem of AI identity is discussed, which has not gained much traction yet, although this investigation is increasingly relevant for different fields, such as ownership issues, personhood of AI, AI welfare, brain–machine interfaces, the distinction between singletons and multi-agent systems as well as to potentially support finding a solution to the problem of personal identity. The AI identity (...) problem analyses the criteria for two AIs to be considered the same at different points in time. Two approaches to tackle the problem are proposed: One is based on the personal identity problem and the concept of computational irreducibility, while the other one applies multi-factor authentication to the AI identity problem. Also, a range of scenarios is examined regarding AI identity, such as replication, fission, fusion, switch off, resurrection, change of hardware, transition from non-sentient to sentient, journey to the past, offspring and identity change. (shrink)

Concerns over epistemic opacity abound in contemporary debates on Artificial Intelligence (AI). However, it is not always clear to what extent these concerns refer to the same set of problems. We can observe, first, that the terms 'transparency' and 'opacity' are used either in reference to the computational elements of an AI model or to the models to which they pertain. Second, opacity and transparency might either be understood to refer to the properties of AI systems or (...) to the epistemic situation of human agents with respect to these systems. While these diagnoses are independently discussed in the literature, juxtaposing them and exploring possible interrelations will help to get a view of the relevant distinctions between conceptions of opacity and their empirical bearing. In pursuit of this aim, two pertinent conditions affecting computer models in general and contemporary AI in particular are outlined and discussed: opacity as a problem of computational tractability and opacity as a problem of the universality of the computational method. (shrink)

Abstract. The main current approach to the AI safety is AI alignment, that is, the creation of AI whose preferences are aligned with “human values.” Many AI safety researchers agree that the idea of “human values” as a constant, ordered sets of preferences is at least incomplete. However, the idea that “humans have values” underlies a lot of thinking in the field; it appears again and again, sometimes popping up as an uncritically accepted truth. Thus, it deserves a thorough deconstruction, (...) which I will do by listing and analyzing comprehensively the hidden assumptions of the idea that “humans have values.” This deconstruction of human values will be centered around the following ideas: “Human values” are useful descriptions, but not real objects; “human values” are bad predictors of behavior; the idea of a “human value system” has flaws; “human values” are not good by default; and human values cannot be separated from human minds. The method of analysis is listing hidden assumptions on which the idea of “human values” is built. I recommend that either the idea of “human values” should be replaced with something better for the goal of AI safety, or at least be used very cautiously. The approaches to AI safety which don’t use the idea of human values at all may require more attention, like the use of full brain models, boxing, and capability limiting. (shrink)