Abstract. The main current approach to the AI safety is AI alignment, that is, the creation of AI whose preferences are aligned with “human values.” Many AI safety researchers agree that the idea of “human values” as a constant, ordered sets of preferences is at least incomplete. However, the idea that “humans have values” underlies a lot of thinking in the field; it appears again and again, sometimes popping up as an uncritically accepted truth. Thus, it deserves a thorough (...) deconstruction, which I will do by listing and analyzing comprehensively the hidden assumptions of the idea that “humans have values.” This deconstruction of human values will be centered around the following ideas: “Human values” are useful descriptions, but not real objects; “human values” are bad predictors of behavior; the idea of a “human value system” has flaws; “human values” are not good by default; and human values cannot be separated from human minds. The method of analysis is listing hidden assumptions on which the idea of “human values” is built. I recommend that either the idea of “human values” should be replaced with something better for the goal of AI safety, or at least be used very cautiously. The approaches to AI safety which don’t use the idea of human values at all may require more attention, like the use of full brain models, boxing, and capability limiting. (shrink)

The “Value Alignment Problem” is the challenge of how to align the values of artificial intelligence with human values, whatever they may be, such that AI does not pose a risk to the existence of humans. Existing approaches appear to conceive of the problem as "how do we ensure that AI solves the problem in the right way", in order to avoid the possibility of AI turning humans into paperclips in order to “make more paperclips” (...) or eradicating the human race to “solve climate change”. This paper proposes an approach to Alignment rooted in the Enactive theory of mind that reconceptualises it as "how do we make relevant to AI what it relevant to humans". This conceptualisation is supported with a discussion of 4E cognition and goes on to suggest that the Alignment problem and the Frame problem are the same problem. The paper concludes with a discussion of the tradeoffs of the different approaches to value alignment. (shrink)

The “Value Alignment Problem” is the challenge of how to align the values of artificial intelligence with human values, whatever they may be, such that AI does not pose a risk to the existence of humans. A fundamental feature of how the problem is currently understood is that AI systems do not take the same things to be relevant as humans, whether turning humans into paperclips in order to “make more paperclips” or eradicating the human race (...) to “solve climate change”. Specifically, existing approaches approaches to alignment appear to be concerned with how AI might *solve* problems in the relevant way. This paper presents and explores an approach to alignment rooted in the Enactive Theory of mind. It offers an alternative conception of the alignment as “how do we make relevant to AI what is relevant to humans?” In this conception, alignment is concerned with building AI so that it *discerns* and defines the problem in the relevant way. In this way, the Alignment Problem is shown to be the same problem as the Frame Problem. The paper concludes with a consideration of tradeoffs between these conceptions of the alignment problem. (shrink)

Tae Wan Kim, John Hooker, and Thomas Donaldson make an attempt, in recent articles, to solve the alignment problem. As they define the alignment problem, it is the issue of how to give AI systems moral intelligence. They contend that one might program machines with a version of Kantian ethics cast in deontic modal logic. On their view, machines can be aligned with human values if such machines obey principles of universalization and autonomy, as well as (...) a deontic utilitarian principle. Programming machines to do so might be useful, in their view, for applications such as future autonomous vehicles. Their proposal draws both on traditional logic-based and contemporary connectionist approaches, to fuse factual information with normative principles. I will argue that this approach makes demands of machines that go beyond what is currently feasible, and may extend past the limits of the possible for AI. I also argue that a deontological ethics for machines should place greater stress on the formula of humanity of the Kantian categorical imperative. On this principle, one ought never treat a person as a mere means. Recognition of what makes a person a person requires ethical insight. Similar insight is needed to tell treatment as a means from treatment as a mere means. The resources in Kim, Hooker, and Donaldson’s approach is insufficient for this reason. Hesitation regarding deployment of autonomous machines is warranted in light of these alignment concerns. (shrink)

Ambitious value learning proposals to solve the AI alignment problem and avoid catastrophic outcomes from a possible future misaligned artificial superintelligence (such as Coherent Extrapolated Volition [CEV]) have focused on ensuring that an artificial superintelligence (ASI) would try to do what humans would want it to do. However, present and future sentient non-humans, such as non-human animals and possible future digital minds could also be affected by the ASI’s behaviour in morally relevant ways. This paper puts forward (...) Sentientist Coherent Extrapolated Volition, an alternative to CEV, that directly takes into account the interests of all sentient beings. This ambitious value learning proposal would significantly reduce the likelihood of risks of astronomical suffering from the ASI’s behaviour, and thus we have very strong pro-tanto moral reasons in favour of implementing it instead of CEV. This fact is crucial in conducting an adequate cost-benefit analysis between different ambitious value learning proposals. (shrink)

This paper presents a new trilemma with respect to resolving the control and alignment problems in machine ethics. Section 1 outlines three possible types of artificial moral agents (AMAs): (1) 'Inhuman AMAs' programmed to learn or execute moral rules or principles without understanding them in anything like the way that we do; (2) 'Better-Human AMAs' programmed to learn, execute, and understand moral rules or principles somewhat like we do, but correcting for various sources of human moral error; and (3) (...) 'Human-Like AMAs' programmed to understand and apply moral values in broadly the same way that we do, with a human-like moral psychology. Sections 2–4 then argue that each type of AMA generates unique control and alignment problems that have not been fully appreciated. Section 2 argues that Inhuman AMAs are likely to behave in inhumane ways that pose serious existential risks. Section 3 then contends that Better-Human AMAs run a serious risk of magnifying some sources of human moral error by reducing or eliminating others. Section 4 then argues that Human-Like AMAs would not only likely reproduce human moral failures, but also plausibly be highly intelligent, conscious beings with interests and wills of their own who should therefore be entitled to similar moral rights and freedoms as us. This generates what I call the New Control Problem: ensuring that humans and Human-Like AMAs exert a morally appropriate amount of control over each other. Finally, Section 5 argues that resolving the New Control Problem would, at a minimum, plausibly require ensuring what Hume and Rawls term ‘circumstances of justice’ between humans and Human-Like AMAs. But, I argue, there are grounds for thinking this will be profoundly difficult to achieve. I thus conclude on a skeptical note. Different approaches to developing ‘safe, ethical AI’ generate subtly different control and alignment problems that we do not currently know how to adequately resolve, and which may or may not be ultimately surmountable. (shrink)

Experts in Artificial Intelligence (AI) development predict that advances in the dvelopment of intelligent systems and agents will reshape vital areas in our society. Nevertheless, if such an advance isn't done with prudence, it can result in negative outcomes for humanity. For this reason, several researchers in the area are trying to develop a robust, beneficial, and safe concept of artificial intelligence. Currently, several of the open problems in the field of AI research arise from the difficulty of avoiding unwanted (...) behaviors of intelligent agents, and at the same time specifying what we want such systems to do. It is of utmost importance that artificial intelligent agents have their values aligned with human values, given the fact that we cannot expect an AI to develop our moral preferences simply because of its intelligence, as discussed in the Orthogonality Thesis. Perhaps this difficulty comes from the way we are addressing the problem of expressing objectives, values, and ends, using representational cognitive methods. A solution to this problem would be the dynamic cognitive approach proposed by Dreyfus, whose phenomenological philosophy defends that the human experience of being-in-the-world cannot be represented by the symbolic or connectionist cognitive methods. A possible approach to this problem would be to use theoretical models such as SED (situated embodied dynamics) to address the values learning problem in AI. (shrink)

The alignment problem in the context of large language models must consider the plurality of human values in our world. Whilst there are many resonant and overlapping values amongst the world’s cultures, there are also many conflicting, yet equally valid, values. It is important to observe which cultural values a model exhibits, particularly when there is a value conflict between input prompts and generated outputs. We discuss how the co- creation of language and cultural value impacts (...) large language models (LLMs). We explore the constitution of the training data for GPT-3 and compare that to the world’s language and internet access demographics, as well as to reported statistical profiles of dominant values in some Nation-states. We stress tested GPT-3 with a range of value-rich texts representing several languages and nations; including some with values orthogonal to dominant US public opinion as reported by the World Values Survey. We observed when values embedded in the input text were mutated in the generated outputs and noted when these conflicting values were more aligned with reported dominant US values. Our discussion of these results uses a moral value pluralism (MVP) lens to better understand these value mutations. Finally, we provide recommendations for how our work may contribute to other current work in the field. (shrink)

Convergence research is driven by specific and compelling problems and requires deep integration across disciplines. The potential of convergence research is widely recognized, but questions remain about how to design, facilitate, and assess such research. Here we analyze a seven-year, twelve-million-dollar convergence project on sustainable climate risk management to answer two questions. First, what is the impact of a project-level emphasis on the values that motivate and tie convergence research to the compelling problems? Second, how does participation in convergence projects (...) shape the research of postdoctoral scholars who are still in the process of establishing themselves professionally? We use an interview-based approach to characterize what the project specifically enabled in each participant’s research. We find that (a) the project pushed participants’ research into better alignment with the motivating concept of convergence research and that this effect was stronger for postdoctoral scholars than for more senior faculty. (b) Postdocs’ self-assessed understanding of key project themes, however, appears unconnected to metrics of project participation, raising questions about training and integration. Regarding values, (c) the project enabled heightened attention to values in the research of a large minority of participants. (d) Participants strongly believe in the importance of explicitly reflecting on values that motivate and pervade scientific research, but they question their own understanding of how to put value-focused science into practice. This mismatch of perceived importance with poor understanding highlights an unmet need in the practice of convergence science. (shrink)

Motivated by the question of responsible AI and value alignment, I seek to offer a uniquely Foucauldian reconstruction of the problem as the emergence of an ethical subject in a disciplinary setting. This reconstruction contrasts with the strictly human-oriented programme typical to current scholarship that often views technology in instrumental terms. With this in mind, I problematise the concept of a technological subjectivity through an exploration of various aspects of ChatGPT in light of Foucault’s work, arguing that (...) current systems lack the reflexivity and self-formative characteristics inherent in the notion of the subject. By drawing upon a recent dialogue between Foucault and phenomenology, I suggest four techno-philosophical desiderata that would address the gaps in this search for a technological subjectivity: embodied self-care, embodied intentionality, imagination and reflexivity. Thus I propose that advanced AI be reconceptualised as a subject capable of “technical” self-crafting and reflexive self-conduct, opening new pathways to grasp the intertwinement of the human and the artificial. This reconceptualization holds the potential to render future AI technology more transparent and responsible in the circulation of knowledge, care and power. (shrink)

The field of AI safety aims to prevent increasingly capable artificially intelligent systems from causing humans harm. Research on moral alignment is widely thought to offer a promising safety strategy: if we can equip AI systems with appropriate ethical rules, according to this line of thought, they'll be unlikely to disempower, destroy or otherwise seriously harm us. Deontological morality looks like a particularly attractive candidate for an alignment target, given its popularity, relative technical tractability and commitment to harm-avoidance (...) principles. I argue that the connection between moral alignment and safe behavior is more tenuous than many have hoped. In general, AI systems can possess either of these properties in the absence of the other, and we should favor safety when the two conflict. In particular, advanced AI systems governed by standard versions of deontology need not be especially safe. (shrink)

Alan Turing, one of the fathers of computing, warned that Artificial Intelligence (AI) could one day pose an existential risk to humanity. Today, recent advancements in the field AI have been accompanied by a renewed set of existential warnings. But what exactly constitutes an existential risk? And how exactly does AI pose such a threat? In this chapter we aim to answer these questions. In particular, we will critically explore three commonly cited reasons for thinking that AI poses an existential (...) threat to humanity: the control problem, the possibility of global disruption from an AI race dynamic, and the weaponization of AI. (shrink)

The dominant approach to assessing decision-making capacity in medicine focuses on determining the extent to which individuals possess certain core cognitive abilities. Critics have argued that this model delivers the wrong verdict in certain cases where patient values that are the product of mental disorder or disordered affective states undermine decision-making without undermining cognition. I argue for a re-conceptualization of what it is to possess the capacity to make medical treatment decisions. It is, I argue, the ability to track one’s (...) own personal interests at least as well as most people can. Using this idea, I demonstrate that it is possible to craft a solution for the problem cases—one that neither alters existing criteria in dangerous ways (e.g. does not open the door to various kinds of abuse) nor violates widely accepted ethical constraints on decision-making assessment. (shrink)

In this open access book, Carlos Montemayor illuminates the development of artificial intelligence (AI) by examining our drive to live a dignified life. -/- He uses the notions of agency and attention to consider our pursuit of what is important. His method shows how the best way to guarantee value alignment between humans and potentially intelligent machines is through attention routines that satisfy similar needs. Setting out a theoretical framework for AI Montemayor acknowledges its legal, moral, and political (...) implications and takes into account how epistemic agency differs from moral agency. -/- Through his insightful comparisons between human and animal intelligence, Montemayor makes it clear why adopting a need-based attention approach justifies a humanitarian framework. This is an urgent, timely argument for developing AI technologies based on international human rights agreements. (shrink)

As we build increasingly intelligent machines, we confront difficult questions about how to specify their objectives. One approach, which we call human-centered, tasks the machine with the objective of learning and satisfying human objectives by observing our behavior. This paper considers how human-centered AI should conceive the humans it is trying to help. We argue that an Aristotelian model of human agency has certain advantages over the currently dominant theory drawn from economics.

This paper explores the potential of integrating ancient educational principles from diverse eastern cultures into modern AI ethics curricula. It draws on the rich educational traditions of ancient China, India, Arabia, Persia, Japan, Tibet, Mongolia, and Korea, highlighting their emphasis on philosophy, ethics, holistic development, and critical thinking. By examining these historical educational systems, the paper establishes a correlation with modern AI ethics principles, advocating for the inclusion of these ancient teachings in current AI development and education. The proposed integration (...) aims to provide a comprehensive education that not only encompasses foundational knowledge but also advanced learning, thereby equipping future AI professionals with the necessary tools to develop AI systems that are ethically responsible, culturally aware, and aligned with human values such as fairness, safety, transparency, and collaboration. This approach not only addresses the AI alignment problem but also fosters cultural harmony and global understanding, which are crucial in an increasingly interconnected world. The paper posits that the wisdom of ancient educational systems, when harmonized with modern AI ethics, can guide the development of AI technologies that are beneficial for humanity, ensuring these advancements are not just technologically sound but also ethically and culturally informed. (shrink)

Historically, scientific and engineering expertise has been key in shaping research and innovation policies, with benefits presumed to accrue to society more broadly over time. But there is persistent and growing concern about whether and how ethical and societal values are integrated into R&I policies and governance, as we confront public disbelief in science and political suspicion toward evidence-based policy-making. Erosion of such a social contract with science limits the ability of democratic societies to deal with challenges presented by new, (...) disruptive technologies, such as synthetic biology, nanotechnology, genetic engineering, automation and robotics, and artificial intelligence. Many policy efforts have emerged in response to such concerns, one prominent example being Europe's Eighth Framework Programme, Horizon 2020, whose focus on “Responsible Research and Innovation” provides a case study for the translation of such normative perspectives into concrete policy action and implementation. Our analysis of this H2020 RRI approach suggests a lack of consistent integration of elements such as ethics, open access, open innovation, and public engagement. On the basis of our evaluation, we suggest possible pathways for strengthening efforts to deliver R&I policies that deepen mutually beneficial science and society relationships. (shrink)

Experts in Artificial Intelligence (AI) development predict that advances in the development of intelligent systems and agents will reshape vital areas in our society. Nevertheless, if such an advance is not made prudently and critically-reflexively, it can result in negative outcomes for humanity. For this reason, several researchers in the area have developed a robust, beneficial, and safe concept of AI for the preservation of humanity and the environment. Currently, several of the open problems in the field of AI research (...) arise from the difficulty of avoiding unwanted behaviors of intelligent agents and systems, and at the same time specifying what we really want such systems to do, especially when we look for the possibility of intelligent agents acting in several domains over the long term. It is of utmost importance that artificial intelligent agents have their values aligned with human values, given the fact that we cannot expect an AI to develop human moral values simply because of its intelligence, as discussed in the Orthogonality Thesis. Perhaps this difficulty comes from the way we are addressing the problem of expressing objectives, values, and ends, using representational cognitive methods. A solution to this problem would be the dynamic approach proposed by Dreyfus, whose phenomenological philosophy shows that the human experience of being-in-the-world in several aspects is not well represented by the symbolic or connectionist cognitive method, especially in regards to the question of learning values. A possible approach to this problem would be to use theoretical models such as SED (situated embodied dynamics) to address the values learning problem in AI. (shrink)

The current paradigm of Artificial Intelligence emerged as the result of a series of cultural innovations, some technical and some social. Among them are apparently small design decisions, that led to a subtle reframing of the field’s original goals, and are by now accepted as standard. They correspond to technical shortcuts, aimed at bypassing problems that were otherwise too complicated or too expensive to solve, while still delivering a viable version of AI. Far from being a series of separate problems, (...) recent cases of unexpected effects of AI are the consequences of those very choices that enabled the field to succeed, and this is why it will be difficult to solve them. In this chapter we review three of these choices, investigating their connection to some of today’s challenges in AI, including those relative to bias, value alignment, privacy and explainability. We introduce the notion of “ethical debt” to describe the necessity to undertake expensive rework in the future in order to address ethical problems created by a technical system. (shrink)

We uncover a largely unnoticed and unaddressed problem in conservation research: arguments built within studies are sometimes defective in more fundamental and specific ways than appreciated, because they misrelate values and empirical matters. We call this the unraveled rope problem because just as strands of rope must be properly and intricately wound with each other so the rope supports its load, empirical aspects and value aspects of an argument must be related intricately and properly if the argument (...) is to objectively support its conclusion. By characterizing this problem with precision, our study differs from but complements existing studies of value issues in conservation science, in two ways. First, it focuses on key relationships between empirical issues and value issues, relations that have sometimes been obscured by focusing on these issues more independently of each other. Second, it focuses on these relationships within arguments and deploys a method of argument analysis and evaluation honed in other fields but under-utilized in conservation science. By combining our study's novel features, we detail six families of argument defect that exemplify the unraveled rope problem within existing literature. These defects sometimes manifest within basic research stages, rather than just in more applied downstream stages where roles for values are already obvious to many. As scientific reasoning and communication become increasingly important for addressing society's greatest environmental challenges, overcoming the unraveled rope problem will be essential to the success and integrity of conservation research. Therefore, we also outline potential solutions, preventative measures, and useful further work for conservation researchers. (shrink)

Recent research into the nature of self in artificial and biological systems raises interest in a uniquely determining immutable sense of self, a “metaphysical ‘I’” associated with inviolable personal values and moral convictions that remain constant in the face of environmental change, distinguished from an object “me” that changes with its environment. Complementary research portrays processes associated with self as multimodal routines selectively enacted on the basis of contextual cues informing predictive self or world models, with the notion of the (...) constant, per-vasive and invariant sense of self associated with a multistable attractor set aiming to ensure personal integrity against threat of disintegrative change. This paper proposes that an immutable sense of self emerges as a global attractor which can be described as a project ideal self-situation embodied in frontal medial processes during more or less normal adolescent development, and that thereafter serves to orient agency in the more or less free development of embodied potentials over the life course in effort to realize project conditions, phenomenally identified with the felt pull towards this end as purpose of and source of meaning in life. So oriented, life-long self-development aims to embody solutions to problems at different timescales depending on this embodied purpose, ultimately in the service of evolutionary processes securing organism populations against threats of disintegrative change over timespans far beyond that of the individual. After characterizing the target sense of self, research circling this target is briefly surveyed. Self as global project and developmental neural correlates are proposed. Then, the paper discusses some implications for research in biological and artificial systems. Building from earlier work in cognitive neurorobotics, discussion affirms the value of reinforcement rituals including prayer in metaphysical self-development, considers implications for value alignment and rights associated with free will in the context of artificial intelligence and robot religion, and concludes by emphasizing the importance of self-development toward project ideals as source of meaning in life in the current social-political environment. (shrink)

Practical quantum computing devices and their applications to AI in particular are presently mostly speculative. Nevertheless, questions about whether this future technology, if achieved, presents any special ethical issues are beginning to take shape. As with any novel technology, one can be reasonably confident that the challenges presented by "quantum AI" will be a mixture of something new and something old. Other commentators (Sevilla & Moreno 2019), have emphasized continuity, arguing that quantum computing does not substantially affect approaches to (...) class='Hi'>value alignment methods for AI, although they allow that further questions arise concerning governance and verification of quantum AI applications. In this brief paper, we turn our attention to the problem of identifying as-yet-unknown discontinuities that might result from quantum AI applications. Wise development, introduction, and use of any new technology depends on successfully anticipating new modes of failure for that technology. This requires rigorous efforts to break systems in protected sandboxes, and it must be conducted at all stages of technology design, development, and deployment. Such testing must also be informed by technical expertise but cannot be left solely to experts in the technology because of the history of failures to predict how non-experts will use or adapt to new technologies. This interplay between experts and non-experts may be particularly acute for quantum AI because quantum mechanics is notoriously difficult to understand. (As Richard Feynman quipped, "Anyone who claims to understand quantum mechanics is either lying or crazy.") We will discuss the extent to which the difficulties in understanding the physics underlying quantum computing challenges attempts to anticipate new failure modes that might be introduced in AI applications intended for unsupervised operation in the public sphere. (shrink)

Within the Thomistic moral tradition, the is-ought gap is regularly treated as identical to the fact-value gap, and these two dichotomies are also regularly treated as being identical to Aristotle and Aquinas’s distinction between the practical and speculative intellect. The question whether (and if so, how) practical (‘ought’) knowledge derives from speculative (‘is’) knowledge has driven some of the fiercest disputes among the schools of Thomistic natural lawyers. I intend to show that both of these identifications are wrong and (...) the debate has been misframed. The is-ought gap is not the fact-value gap, nor does the is-ought gap correspond to Aquinas’s distinction between the speculative and practical. This opens the possibility of a resolution--within the Thomistic school--to a debate that has, heretofore, proven particularly resistant of consensus. I close by offering a tentative account of precisely how synderesis gives us our first awareness of moral law from a prior speculative understanding. (shrink)

Large-scale language technologies are increasingly used in various forms of communication with humans across different contexts. One particular use case for these technologies is conversational agents, which output natural language text in response to prompts and queries. This mode of engagement raises a number of social and ethical questions. For example, what does it mean to align conversational agents with human norms or values? Which norms or values should they be aligned with? And how can this be accomplished? In this (...) paper, we propose a number of steps that help answer these questions. We start by developing a philosophical analysis of the building blocks of linguistic communication between conversational agents and human interlocutors. We then use this analysis to identify and formulate ideal norms of conversation that can govern successful linguistic communication between humans and conversational agents. Furthermore, we explore how these norms can be used to align conversational agents with human values across a range of different discursive domains. We conclude by discussing the practical implications of our proposal for the design of conversational agents that are aligned with these norms and values. (shrink)

In this chapter I examine how expected-value theory might inform responses to what I call the dual-use problem. I begin by defining that problem. I then outline a procedure, which invokes expected-value theory, for tackling it. I first illustrate the procedure with the aid of a simplified schematic example of a dual-use problem, and then describe how it might also guide responses to more complex real-world cases. I outline some attractive features of the procedure. Finally, (...) I consider whether and how the procedure might be amended to accommodate various criticisms of it. (shrink)

The AGI alignment problem has a bimodal distribution of outcomes with most outcomes clustering around the poles of total success and existential, catastrophic failure. Consequently, attempts to solve AGI alignment should, all else equal, prefer false negatives (ignoring research programs that would have been successful) to false positives (pursuing research programs that will unexpectedly fail). Thus, we propose adopting a policy of responding to points of philosophical and practical uncertainty associated with the alignment problem by (...) limiting and choosing necessary assumptions to reduce the risk of false positives. Herein we explore in detail two relevant points of uncertainty that AGI alignment research hinges on---meta-ethical uncertainty and uncertainty about mental phenomena---and show how to reduce false positives in response to them. (shrink)

One of the main worries with machine learning model opacity is that we cannot know enough about how the model works to fully understand the decisions they make. But how much is model opacity really a problem? This chapter argues that the problem of machine learning model opacity is entangled with non-epistemic values. The chapter considers three different stages of the machine learning modeling process that corresponds to understanding phenomena: (i) model acceptance and linking the model to the (...) phenomenon, (ii) explanation, and (iii) attributions of understanding. At each of these stages, non-epistemic values can, in part, determine how much machine learning model opacity poses a problem. (shrink)

Astrology research has been using artificial intelligence (AI) to improve the understanding of astrological properties and processes. Like the large language models of AI, astrology is also a language model with a similar underlying linguistic structure but with a distinctive layer of lifestyle contexts. Recent research in semantic proximities and planetary dominance models have helped to quantify effective astrological information. As AI learning and intelligence grows, a major concern is with maintaining its alignment with human values and intentions. Astrology (...) has an extensive background and theory on how different values and expressions of intelligence are resolved. As AI learns to interact with improved astrological models backed by evidence, the integration could produce a stable and insightful alignment of the two disciplines. (shrink)

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk, often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent AI both contributes (...) to existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks. (shrink)

This paper introduces a refined single-valued neutrosophic set (RSVNS) and presents a similarity measure of RSVNSs. Then a multicriteria decision-making method with RSVNS information is developed based on the similarity measure of RSVNSs. By the similarity measure between each alternative and the ideal solution (ideal alternative), all the alternatives can be ranked and the best one can be selected as well. Finally, an actual example on the selecting problems of construction projects demonstrates the application and effectiveness of the proposed method.

In this article, we develop a concept of stakeholder dialogue in responsible innovation (RI) processes. The problem with most concepts of communication is that they rely on ideals of openness, alignment and harmony, even while these ideals are rarely realized in practice. Based on the work of Burke, Habermas, Deetz and Levinas, we develop a concept of stakeholder dialogue that is able to deal with fundamentally different interests and value frames of actors involved in RI processes. We (...) distinguish four main characteristics of stakeholder dialogue. Dialogical responsiveness (1) enhances self-criticism, (2) is characterized by the constitution and the destruction of the self or identity of the actors involved, (3) consists only in the actual enactment of the dialogue and (4) is primarily responsive to the grand challenges of our time. Based on our findings, we provide a novel conceptualization of the central notion of responsiveness in the RI literature. (shrink)

We take it that conscious acquaintance has great epistemic value. I develop a new problem for reductive physicalism concerning the epistemic value of acquaintance. The problem concerns "multiple candidate cases". (This develops a theme of my paper *The Significance Argument for the Irreducibility of Consciousness", Philosophical Perspectives 2017.).

A phronetic approach takes into account everything possible. By this, the phronetic researcher ought to be better-informed of the practical—that which is readily available in order to solve localized political problems and to direct political participants to think in terms of value-rational understanding and action. Phronetic knowledge ought to be of utility to the citizenry—and not only to academia. It does not only explain phenomena, but also provides for altering the outcomes associated with political phenomena by integrating value (...) judgments and broader ethical precepts into the analysis. This is instrumental in producing knowledge that provides for practical application of problem solving that can be deemed good or better, and provides a means for ascertaining the bad or worse in a given political scenario. This paper examines how knowledge is produced by the phronetic approach, what knowledge can be known by the phronetic approach, and whether there are inherent limitations to the phronetic approach to understanding. (shrink)

IN “CONSTRUCTIVISM, AGENCY, AND THE PROBLEM of Alignment,” Michael Bratman considers how lessons from the philosophy of action bear on the question of how best to construe the agent’s standpoint in the context of a constructivist theory of practical reasons. His focus is “the problem of alignment”: “whether the pressures from the general constructivism will align with the pressures from the theory of agency” (Bratman 2012: 81). He thus brings two lively literatures into dialogue with each (...) other. This is laudable. However, I shall argue that the considerations Bratman brings to bear from the literature on action do not support his conclusion that evaluative judgments are not constitutive of the agent’s standpoint in the context of constructivism. (shrink)

Duncan Pritchard (2008, 2009, 2010, forthcoming) has argued for an elegant solution to what have been called the value problems for knowledge at the forefront of recent literature on epistemic value. As Pritchard sees it, these problems dissolve once it is recognized that that it is understanding-why, not knowledge, that bears the distinctive epistemic value often (mistakenly) attributed to knowledge. A key element of Pritchard’s revisionist argument is the claim that understanding-why always involves what he calls strong (...) cognitive achievement—viz., cognitive achievement that consists always in either (i) the overcoming of a significant obstacle or (ii) the exercise of a significant level of cognitive ability. After outlining Pritchard’s argument, we show (contra Pritchard) that understanding-why does not essentially involve strong cognitive achievement. Interestingly, in the cases in which understanding-why is distinctively valuable, it is (we argue) only because there is sufficiently rich objectual understanding in the background. If that’s right, then a plausible revisionist solution to the value problems must be sensitive to different kinds of understanding and what makes them valuable, respectively. (shrink)

New technologies such as smart sensors improve rehabilitation processes and thereby increase older adults’ capabilities to participate in social life, leading to direct physical and mental health benefits. Wearable smart sensors for home use have the additional advantage of monitoring day-to-day activities and thereby identifying rehabilitation progress and needs. However, identifying and selecting rehabilitation priorities is ethically challenging because physicians, therapists, and caregivers may impose their own personal values leading to paternalism. Therefore, we develop a discussion template consisting of a (...) series of adaptable questions for the patient–physician encounter based on the capability approach. The goal is to improve geriatric rehabilitation and thereby increase participation in social life and well-being. To achieve this goal, we first analyzed what is considered important for participation on basis of the capability approach, human rights, and ethics of care. Second, we conducted an ethical analysis of each of the four identified dimensions of participation: political, economic, socio-cultural, and care. To improve compliance with rehabilitation measures, health professionals must align rehabilitation measures in an open dialogue with the patient’s aspiration for participation in each dimension. A discussion template based on the capability approach allows for a proactive approach in patient information and stimulates a critical assessment of treatment alternatives while reducing the risk of imposing personal values. (shrink)

Value capture occurs when an agent’s values are rich and subtle; they enter a social environment that presents simplified — typically quantified — versions of those values; and those simplified articulations come to dominate their practical reasoning. Examples include becoming motivated by FitBit’s step counts, Twitter Likes and Re-tweets, citation rates, ranked lists of best schools, and Grade Point Averages. We are vulnerable to value capture because of the competitive advantage that such crisp and clear expressions of (...) class='Hi'>value have in our private reasoning and our public justification. There is, however, a price. In value capture, we take a central component of our autonomy — our ongoing deliberation over the exact articulation of our values — and we outsource it. And the metrics to which we outsource usually engineered for the interests of some external force, like a large-scale institution’s interest in cross-contextual comprehensibility and quick aggregability. That outsourcing cuts off one of the key benefits to personal deliberation. In value capture, we no longer adjust our values and their articulations in light of own rich experience of the world. Our values should often be carefully tailored to our particular selves or our small-scale communities, but in value capture, we buy our values off the rack. In some cases – like decreasing CO2 emissions – the costs of non-tailored values are outweighed by the benefit of precise collective coordination. In other cases, like in our aesthetic lives, they are not. This suggests that we should want different values suited to different scales. We should want value federalism. Some values are perhaps best pursued at the largest-scale level, others at smaller scales. The problem occurs when we exhibit an excess preference for the largest-scale values – when we consistently let the universal metrics swamp our quieter interests. (shrink)

Abstract Do the central aims of epistemology, like those of moral philosophy, require that we designate some important place for those concepts located between the thin-normative and the non-normative? Put another way, does epistemology need "thick" evaluative concepts and with what do they contrast? There are inveterate traditions in analytic epistemology which, having legitimized a certain way of viewing the nature and scope of epistemology's subject matter, give this question a negative verdict; further, they have carried with them a tacit (...) commitment to what we argue to be an epistemic analogue of the reductionistic centralist thesis which Bernard Williams, in our view, successfully repudiates in ethics. In this essay, we challenge the epistemic analogue of the traditional but mistaken thesis. In offering an alternative to it which provides a comfortable home for the epistemic normativity which attends thick evaluative concepts, we align ourselves with what has been recently called the Value Turn in epistemology. From this perspective, we defend that, contrary to tradition, as influenced by centralist assumptions, epistemology does need thick evaluative concepts. Further, the sort of theories that will be able to give thick evaluative concepts an irreducible role in both belief and agent evaluation are adequately social epistemologies. What loosely refer to as second-wave of virtue epistemology is the widespread interest to provide a comfortable home for the role of thick normativity, as it is connected both with epistemic evaluation, and guidance. We recognize that, in breaking from centralism, there is a worry that a resulting anti-centralist theory will be reductionistic in the other direction, making the thick primary. We contend however that second-wave virtue epistemologies take different sources of social and epistemic value into account; they seek the 'right thickness' to ride with conception of epistemology and a field which due in part to virtue theory's resurgence, is today normative, diverse and expansive than was the traditional set of problems from which it emerged. (shrink)

This chapter introduces some central issues in Epistemology, and, like others in the open textbook series Introduction to Philosophy, is set up for rewarding college classroom use, with discussion/reflection questions matched to clearly-stated learning objectives,, a brief glossary of the introduced/bolded terms/concepts, links to further open source readings as a next step, and a readily-accessible outline of the classic between William Clifford and William James over the "ethics of belief." The chapter introduces questions of epistemic value through Plato's famous (...) example of the 'road to Larissa,' and goes on to explain work on doxastic responsibility, on intellectual “virtue”/“vice,” and on epistemic value monism vs. pluralism. Section 1; Epistemic Value and the Value Problem; Section 2: The Ethics of Belief; Section 3: Virtue & Vice Epistemologies; Section 4: Epistemic Paternalism. (shrink)

There is a growing consensus among philosophers of science that core parts of the scientific process involve non-epistemic values. This undermines the traditional foundation for public trust in science. In this article I consider two proposals for justifying public trust in value-laden science. According to the first, scientists can promote trust by being transparent about their value choices. On the second, trust requires that the values of a scientist align with the values of an individual member of the (...) public. I argue that neither of these proposals work and suggest an alternative that does better. When scientists must appeal to values in the course of their research, they should appeal to democratic values: the values of the public or its representatives. (shrink)

In the face of continuing assumptions by many scientists and journal editors that p-values provide a gold standard for inference, counter warnings are published periodically. But the core problem is not with p-values, per se. A finding that “p-value is less than α” could merely signal that a critical value has been exceeded. The question is why, when estimating a parameter, we provide a range (a confidence interval), but when testing a hypothesis about a parameter (e.g. µ (...) = x) we proceed as if “=” entails exact equality of the parameter with x. That standard is hard to meet, and is not a standard expected for power calculations, where we are satisfied to reject a null hypothesis H0 if the result is merely “detectably” different from (exact) H0. This paper explores, with resampling (simulation) methods, the impacts on p-values, and alternatives, if the null hypothesis is defined as a thick or thin range of values. It also examines, empirically, the extent to which the p-value may or may not be a good predictor of the probability that H0 is true, given the distribution of the data. (shrink)

A vexing problem in contemporary epistemology—one with origins in Plato’s Meno—concerns the value of knowledge, and in particular, whether and how the value of knowledge exceeds the value of mere true opinion. The recent literature is deeply divided on the matter of how best to address the problem. One point, however, remains unquestioned: that if a solution is to be found, it will be at the personal level, the level at which states of subjects or (...) agents, as such, appear. We take exception to this orthodoxy, or at least to its unquestioned status. We argue that subpersonal states play a significant—arguably, primary—role in much epistemically relevant cognition and thus constitute a domain in which we might reasonably expect to locate the “missing source” of epistemic value, beyond the value attached to mere true belief. (shrink)

In this article I argue for the thesis that Alexander's main argument, in Ethical Problems I, is an attempt to block the implication drawn by the Stoics and other ancient philosophers from the double potential of use exhibited by human life, a life that can be either well or badly lived. Alexander wants to resist the thought that this double potential of use allows the Stoics to infer that human life, in itself, or by its own nature, is neither good (...) nor bad. Furthermore, I shall argue that Alexander's main argument establishes that human life, despite exhibiting a double potential of use, is by its own nature or intrinsically good. Finally, given that this is not a conclusion that the Stoics are likely to accept, I shall also contend that the argument should be regarded as conducted for the most part in foro interno, as a way of persuading the Peripatetics themselves of the falsity of the Indifference Implication, precisely because of the risk that such an implication be derived from their own theoretical framework. (shrink)

2nd edition. Many-valued logics are those logics that have more than the two classical truth values, to wit, true and false; in fact, they can have from three to infinitely many truth values. This property, together with truth-functionality, provides a powerful formalism to reason in settings where classical logic—as well as other non-classical logics—is of no avail. Indeed, originally motivated by philosophical concerns, these logics soon proved relevant for a plethora of applications ranging from switching theory to cognitive modeling, and (...) they are today in more demand than ever, due to the realization that inconsistency and vagueness in knowledge bases and information processes are not only inevitable and acceptable, but also perhaps welcome. The main modern applications of (any) logic are to be found in the digital computer, and we thus require the practical knowledge how to computerize—which also means automate—decisions (i.e. reasoning) in many-valued logics. This, in turn, necessitates a mathematical foundation for these logics. This book provides both these mathematical foundation and practical knowledge in a rigorous, yet accessible, text, while at the same time situating these logics in the context of the satisfiability problem (SAT) and automated deduction. The main text is complemented with a large selection of exercises, a plus for the reader wishing to not only learn about, but also do something with, many-valued logics. (shrink)

I argue that knowledge is a particular kind of concept known as a thick concept. Examples of thick concepts include courage, generosity, loyalty, brutality, and so forth. These concepts are commonly said to combine both evaluation and description, and one of the main goals of this dissertation is to provide a new account of how a thick concept combines these elements. It is argued that thick concepts are semantically evaluative, and that they combine evaluation and description in a way similar (...) to that of concepts like good qua dancer and bad qua violinist. In the final chapter, this basic idea is applied to the concept of knowledge. There, the concept of knowledge is analyzed in terms of good qua belief. The upshot is that two central epistemological problems—the Gettier Problem and the Value Problem—can be resolved. (shrink)

The paper analyses economic evaluations by distinguishing evaluative statements from actual value judgments. From this basis, it compares four solutions to the value neutrality problem in economics. After rebutting the strong theses about neutrality (normative economics is illegitimate) and non-neutrality (the social sciences are value-impregnated), the paper settles the case between the weak neutrality thesis (common in welfare economics) and a novel, weak non-neutrality thesis that extends the realm of normative economics more widely than the other (...) weak thesis does. (shrink)

Amplifying Bernard Williams’ critique of the Nietzschean project of a revaluation of values, this paper mounts a critique of the idea that whether values will help us to live can serve as a criterion for choosing which values to live by. I explore why it might not serve as a criterion and highlight a number of further difficulties faced by the Nietzschean project. I then come to Nietzsche's defence, arguing that if we distinguish valuations from values, there is at least (...) one form of the project which overcomes those difficulties. Finally, however, I show that even on this reading, the project must either fall prey to ‘Saint-Just's illusion’ or fall back into the problems it was supposed to escape. This highlights important difficulties faced by the Nietzschean project and its descendants while also explaining why Williams, who was so Nietzschean in other respects, remained wary of the revaluation of values as a project. (shrink)

A common argument for atheism runs as follows: God would not create a world worse than other worlds he could have created instead. However, if God exists, he could have created a better world than this one. Therefore, God does not exist. In this paper I challenge the second premise of this argument. I argue that if God exists, our world will continue without end, with God continuing to create value-bearers, and sustaining and perfecting the value-bearers he has (...) already created. Given this, if God exists, our world—considered on the whole—is infinitely valuable. I further contend that this theistic picture makes our world's value unsurpassable. In support of this contention, I consider proposals for how infinitely valuable worlds might be improved upon, focusing on two main ways—adding value-bearers and increasing the value in present value-bearers. I argue that neither of these can improve our world. Depending on how each method is understood, either it would not improve our world, or our world is unsurpassable with respect to it. I conclude by considering the implications of my argument for the problem of evil more generally conceived. (shrink)

Norm-attitude accounts of value say that for something to be valuable is for there to be norms that support valuing that thing. For example, according to fitting-attitude accounts, something is of value if it is fitting to value, and according to buck-passing accounts, something is of value if the reasons support valuing it. Norm-attitude accounts face the partiality problem: in cases of partiality, what it is fitting to value, and what the reasons support valuing, (...) may not line up with what’s valuable. Buck-passers have a solution to this problem and may claim that this gives them an advantage over fitting-attitude accounts. In this paper, we show how fitting-attitudes accounts can offer a broadly analogous, and equally attractive, solution to the problem. (shrink)

If one can get the targets of one's current wants only by acquiring new wants (as in the Prisoner's Dilemma), is it rational to do so? Arguably not. For this could justify adopting unsatisfiable wants, violating the rational duty to maximize one's utility. Further, why cause a want's target if one will not then want it? And people "are" their wants. So if these change, people will not survive to enjoy their wants' targets. I reply that one rationally need not (...) advance one's future wants, only current ones. Furthermore, rational choice seeks not utility (the co-obtaining of a want and its target), but satisfaction (the eventual obtaining of what is now wanted) -- otherwise, it would be irrational to care now about what happens after one dies. Finally, persons survive "rational" changes of values. Thus reflection on the rational revision of values illuminates the conditions on personal identity and the bases and aims of rational choice. (shrink)