This paper presents a unifying diagnosis of a number of important problems facing existing models of rational choice under moral uncertainty and proposes a remedy. I argue that the problems of (i) severely limited scope, (ii) intertheoretic comparisons, and (iii) 'swamping’ all stem from the way in which values are assigned to options in decision rules such as Maximisation of Expected Choiceworthiness. By assigning values to options under a given moral theory by asking something like ‘how much do I desire (...) this option, supposing this theory is true?’ rather than ‘how much value does this theory assign to this option?’ these problems can be avoided, while the appealing features of these accounts can be preserved. This amendment provides a role for the preferences, desires, or goals of rational agents that is curiously absent from the existing discussion of what individuals rationally ought to do when they are uncertain about what they morally ought to do. (shrink)

Many philosophers think that descriptive uncertainty is relevant to what we subjectively ought to do. This leads to a further question: is what we subjectively ought to do sensitive to our moral uncertainty as well? Includers say yes—what we subjectively ought to do is sensitive to both descriptive uncertainty and moral uncertainty. Excluders say no—only descriptive uncertainty matters to what we subjectively ought to do (i.e., moral uncertainty is irrelevant). Excluders argue that common motivations for the subjective ought only give (...) us reason to think that descriptive uncertainty matters. This paper focuses on one motivation: accessibility. Excluders argue that accessibility does not motivate the Includers’ view because moral truths are always accessible – unlike descriptive facts which are not always accessible. My goal is to defend the Includers’ view by arguing that moral truths are not always accessible in the relevant sense. (shrink)

This thesis is about making decisions when we are uncertain about what will happen, how valuable it will be, and even how to make decisions. Even the most sure-footed amongst us are sometimes uncertain about all three, but surprisingly little attention has been given to the latter two. The three essays that constitute my thesis hope to do a small part in rectifying this problem. The first essay is about the value of finding out how to make decisions. Society spends (...) considerable resources funding people (like me) to research decision-making, so it is natural to wonder whether society is getting a good deal. This question is so shockingly underresearched that bedrock facts are readily discoverable, such as when this kind of information is valuable. My second essay concerns whether we can compare value when we are uncertain about value. Many people are in fact uncertain about value, and how we deal with this uncertainty hinges on these comparisons. I argue that value comparisons are only sometimes possible; I call this weak comparability. This essay is largely a synthesis of the literature, but I also present an argument which begins with a peculiar view of the self: it is as if each of us is a crowd of different people separated by time (but connected by continuity of experience). I’m not the first to endorse this peculiar view of the self, but I am the first to show how it supports the benign view that value is sometimes comparable. We may be uncertain of any decision rules, even those that would tell us how to act when we face uncertainty in decision rules. We may be uncertain of how to decide how to decide how to... And so on. If so, we might have to accept infinitely many decision rules just to make any mundane decision, such as whether to pick up a five-cent piece from the gutter. My third essay addresses this problem of regress. I think all of our decisions are forced: we must decide now or continue to deliberate. Surprisingly, this allows us to avoid the original problem. I call this solution “when forced, do your best”. (shrink)

We are sometimes unsure of the moral status of our relationships with other entities. Recent case studies in this uncertainty include our relationships with artificial agents (robots, assistant AI, etc.), animals, and patients with “locked-in” syndrome. Do these entities have basic moral standing? Could they count as true friends or lovers? What should we do when we do not know the answer to these questions? An influential line of reasoning suggests that, in such cases of moral uncertainty, we need meta-moral (...) decision rules that allow us to either minimize the risks of moral wrongdoing or improve the choice-worthiness of our actions. One particular argument adopted in this literature is the “risk asymmetry argument,” which claims that the risks associated with accepting or rejecting some moral facts may be sufficiently asymmetrical as to warrant favoring a particular practical resolution of this uncertainty. Focusing on the case study of artificial beings, this article argues that this is best understood as an ethical-epistemic challenge. The article argues that taking potential risk asymmetries seriously can help resolve disputes about the status of human–AI relationships, at least in practical terms (philosophical debates will, no doubt, continue); however, the resolution depends on a proper, empirically grounded assessment of the risks involved. Being skeptical about basic moral status, but more open to the possibility of meaningful relationships with such entities, may be the most sensible approach to take. (shrink)

If non-human animals have high moral status, then we commit a grave moral error by eating them. Eating animals is thus morally risky, while many agree that it is morally permissible to not eat animals. According to some philosophers, then, non-animal ethicists should err on the side of caution and refrain from eating animals. I argue that this precautionary argument assumes a false dichotomy of dietary options: a diet that includes farm-raised animals or a diet that does not include animals (...) of any kind. There is a third dietary option, namely, a diet of plants and non-traditional animal protein, and there is evidence that such a diet results in the least amount of harm to animals. It follows therefore that moral uncertainty does not support the adoption of a vegetarian diet. (shrink)

How ought you to evaluate your options if you're uncertain about what's fundamentally valuable? A prominent response is Expected Value Maximisation (EVM)—the view that under axiological uncertainty, an option is better than another if and only if it has the greater expected value across axiologies. But the expected value of an option depends on quantitative probability and value facts, and in particular on value comparisons across axiologies. We need to explain what it is for such facts to hold. Also, EVM (...) is by no means self-evident. We need an argument to defend that it’s true. This book introduces an axiomatic approach to answer these worries. It provides an explication of what EVM means by use of representation theorems: intertheoretic comparisons can be understood in terms of facts about which options are better than which, and mutatis mutandis for intratheoretic comparisons and axiological probabilities. And it provides a systematic argument to the effect that EVM is true: the theory can be vindicated through simple axioms. The result is a formally cogent and philosophically compelling extension of standard decision theory, and original take on the problem of axiological or normative uncertainty. (shrink)

We propose four policies of ethical design of human-grade Artificial Intelligence. Two of our policies are precautionary. Given substantial uncertainty both about ethical theory and about the conditions under which AI would have conscious experiences, we should be cautious in our handling of cases where different moral theories or different theories of consciousness would produce very different ethical recommendations. Two of our policies concern respect and freedom. If we design AI that deserves moral consideration equivalent to that of human beings, (...) that AI should be designed with self-respect and with the freedom to explore values other than those we might impose. We are especially concerned about the temptation to create human-grade AI pre-installed with the desire to cheerfully sacrifice itself for its creators’ benefit. (shrink)

This paper explores the idea that it is instrumentally valuable to learn normative truths. We consider an argument for "normative hedging" based on this principle, and examine the structure of decision-making under moral uncertainty that arises from it. But it also turns out that the value of normative information is inconsistent with the principle that learning *empirical* truths is instrumentally valuable. We conclude with a brief comment on "metanormative regress.".

Almost all participants in the debate about the ethics of accidents with self-driving cars have so far assumed moral universalism. However, universalism may be philosophically more controversial than is commonly thought, and may lead to undesirable results in terms of non-moral consequences and feasibility. There thus seems to be a need to also start considering what I refer to as the “relativistic car” — a car that is programmed under the assumption that what is morally right, wrong, good, bad, etc. (...) is determined by the moral beliefs of one’s society or culture. My investigation of this idea involves six steps. First, I explain why and how the moral universalism/relativism debate is relevant to the issue of self-driving cars. Second, I argue that there are good reasons to consider accident algorithms that assume relativism. Third, I outline how a relativistic car would be programmed to behave. Fourth, I address what advantages such a car would have, both in terms of its non-moral consequences and feasibility. Fifth, I address the relativistic car’s disadvantages. Finally, I qualify and conclude my considerations. (shrink)

Suppose you want to do as much good as possible. What should you do? According to members of the effective altruism movement—which has produced much of the thinking on this issue and counts several moral philosophers as its key protagonists—we should prioritise among the world’s problems by assessing their scale, solvability, and neglectedness. Once we’ve done this, the three top priorities, not necessarily in this order, are (1) aiding the world’s poorest people by providing life-saving medical treatments or alleviating poverty (...) itself, (2) preventing global catastrophic risks, such as those posed by nuclear war or rogue artificial intelligence, and (3) ending factory farming. These claims are both plausible and striking. If correct, they should prompt a stark revision of how we approach our altruistic activities. However, the project of determining how to do the most good—as opposed to say, whether we should do the most good—has only recently, within the last ten years, become the subject of serious academic attention. Many key claims have not yet been carefully scrutinised. This is a cause for concern: are effective altruists doing good badly? In this thesis, I critique and develop some of the latest claims about how individuals can do the most good. I do this in three areas: the value of saving lives (preventing premature deaths), how best to improve lives (making people happier during their lives), and cause prioritisation methodology (frameworks for determining which problems are the highest priorities). In each case, I raise novel theoretical considerations that, when incorporated, change the analysis. Roughly speaking, my main conclusions are (1) saving lives is not as straightforwardly good we tend to suppose, may not be good at all, and is not clearly a priority; (2) happiness can be measured through self-reports and, based on the self-reported evidence, treating mental health stands out as an overlooked problem that may be an even more cost-effective way to improve lives than alleviating poverty; (3) the cause prioritisation methodology proposed by effective altruists needs to be moderately reconceptualised and, when it is, it turns out it is not as illuminating a tool as we might have thought and hoped. (shrink)

In this paper, we explore an under-investigated question concerning the class of formal models that aim at providing normative guidance. We call such models normative models. In particular, we examine the question of how normative models can successfully exert normative guidance. First, we highlight the absence of a discussion of this question – which is surprising given the extensive debate about the success conditions of descriptive models – and motivate its importance. Second, we introduce and discuss two potential accounts of (...) the success conditions of normative models. Our tentative conclusion is that the second account is more promising. (shrink)

This paper explores the role of moral uncertainty in explaining the morally disruptive character of new technologies. We argue that existing accounts of technomoral change do not fully explain its disruptiveness. This explanatory gap can be bridged by examining the epistemic dimensions of technomoral change, focusing on moral uncertainty and inquiry. To develop this account, we examine three historical cases: the introduction of the early pregnancy test, the contraception pill, and brain death. The resulting account highlights what we call “differential (...) disruption” and provides a resource for fields such as technology assessment, ethics of technology, and responsible innovation. (shrink)

Automated decision making for sentencing is the use of a software algorithm to analyse a convicted offender’s case and deliver a sentence. This chapter reviews the moral arguments for and against employing automated decision making for sentencing and finds that its use is in principle morally permissible. Specifically, it argues that well-designed automated decision making for sentencing will better approximate the just sentence than human sentencers. Moreover, it dismisses common concerns about transparency, privacy and bias as unpersuasive or inapplicable. The (...) chapter also notes that moral disagreement about theories of just sentencing are plausibly resolved by applying the principle of maximising expected moral choiceworthiness, and that automated decision making is better suited to the resulting ensemble model. Finally, the chapter considers the challenge posed by penal populism. The dispiriting conclusion is that although it is in theory morally desirable to use automated decision-making for criminal sentencing, it may well be the case that we ought not to try. (shrink)

Suppose that an altruistic agent who is uncertain between evidential and causal decision theory finds herself in a situation where these theories give conflicting verdicts. We argue that even if she has significantly higher credence in CDT, she should nevertheless act in accordance with EDT. First, we claim that the appropriate response to normative uncertainty is to hedge one's bets. That is, if the stakes are much higher on one theory than another, and the credences you assign to each of (...) these theories are not very different, then it is appropriate to choose the option that performs best on the high-stakes theory. Second, we show that, given the assumption of altruism, the existence of correlated decision makers will increase the stakes for EDT but leave the stakes for CDT unaffected. Together these two claims imply that whenever there are sufficiently many correlated agents, the appropriate response is to act in accordance with EDT. (shrink)

A critical survey of recent work in epistemology on higher-order evidence. It discusses the nature of higher-order evidence, some puzzles it raises, responses to those puzzles, and problems facing them. It concludes by indicating connections between debates concerning higher-order evidence in epistemology and parallel debates in ethics and aesthetics.

Jotta olisimme moraalisesti vastuussa teoistamme, meidän on kyettävä muodostamaan käsityksiä oikeasta ja väärästä ja toimimaan ainakin jossain määrin niiden mukaisesti. Jos olemme täysivaltaisia moraalitoimijoita, myös ymmärrämme miksi jotkin teot ovat väärin, ja kykenemme siten joustavasti mukauttamaan toimintaamme eri tilanteisiin. Esitän, ettei näköpiirissä ole tekoälyjärjestelmiä, jotka kykenisivät aidosti välittämään oikein tekemisestä tai ymmärtämään moraalin vaatimuksia, koska nämä kyvyt vaativat kokemustietoisuutta ja kokonaisvaltaista arvostelukykyä. Emme siten voi sysätä koneille vastuuta teoistaan. Meidän on sen sijaan pyrittävä rakentamaan keinotekoisia oikeintekijöitä - järjestelmiä, jotka eivät (...) ole moraalitoimijoita, mutta kuitenkin toimivat likipitäen siten kuin me itse parhaimmillamme tekisimme. (shrink)

According to the traditional Bayesian view of credence, its structure is that of precise probability, its objects are descriptive propositions about the empirical world, and its dynamics are given by conditionalization. Each of the three essays that make up this thesis deals with a different variation on this traditional picture. The first variation replaces precise probability with sets of probabilities. The resulting imprecise Bayesianism is sometimes motivated on the grounds that our beliefs should not be more precise than the evidence (...) calls for. One known problem for this evidentially motivated imprecise view is that in certain cases, our imprecise credence in a particular proposition will remain the same no matter how much evidence we receive. In the first essay I argue that the problem is much more general than has been appreciated so far, and that it’s difficult to avoid without compromising the initial evidentialist motivation. The second variation replaces descriptive claims with moral claims as the objects of credence. I consider three standard arguments for probabilism with respect to descriptive uncertainty—representation theorem arguments, Dutch book arguments, and accuracy arguments—in order to examine whether such arguments can also be used to establish probabilism with respect to moral uncertainty. In the second essay, I argue that by and large they can, with some caveats. First, I don’t examine whether these arguments can be given sound non-cognitivist readings, and any conclusions therefore only hold conditional on cognitivism. Second, decision-theoretic representation theorems are found to be less convincing in the moral case, because there they implausibly commit us to thinking that intertheoretic comparisons of value are always possible. Third and finally, certain considerations may lead one to think that imprecise probabilism provides a more plausible model of moral epistemology. The third variation considers whether, in addition to conditionalization, agents may also change their minds by becoming aware of propositions they had not previously entertained, and therefore not previously assigned any probability. More specifically, I argue that if we wish to make room for reflective equilibrium in a probabilistic moral epistemology, we must allow for awareness growth. In the third essay, I sketch the outline of such a Bayesian account of reflective equilibrium. Given that this account gives a central place to awareness growth, and that the rationality constraints on belief change by awareness growth are much weaker than those on belief change by conditionalization, it follows that the rationality constraints on the credences of agents who are seeking reflective equilibrium are correspondingly weaker. (shrink)

The principle that rational agents should maximize expected utility or choiceworthiness is intuitively plausible in many ordinary cases of decision-making under uncertainty. But it is less plausible in cases of extreme, low-probability risk (like Pascal's Mugging), and intolerably paradoxical in cases like the St. Petersburg and Pasadena games. In this paper I show that, under certain conditions, stochastic dominance reasoning can capture most of the plausible implications of expectational reasoning while avoiding most of its pitfalls. Specifically, given sufficient background uncertainty (...) about the choiceworthiness of one's options, many expectation-maximizing gambles that do not stochastically dominate their alternatives "in a vacuum" become stochastically dominant in virtue of that background uncertainty. But, even under these conditions, stochastic dominance will not require agents to accept options whose expectational superiority depends on sufficiently small probabilities of extreme payoffs. The sort of background uncertainty on which these results depend looks unavoidable for any agent who measures the choiceworthiness of her options in part by the total amount of value in the resulting world. At least for such agents, then, stochastic dominance offers a plausible general principle of choice under uncertainty that can explain more of the apparent rational constraints on such choices than has previously been recognized. (shrink)

Recently, several philosophers have argued that, when faced with moral uncertainty, we ought to choose the option with the maximal expected choiceworthiness (MEC). This view has been challenged on the grounds that it is implausibly demanding. In response, those who endorse MEC have argued that we should take into account the all-things-considered choiceworthiness of our options. I argue that this gives rise to another problem: acts that we consider to be supererogatory are rendered impermissible, and acts that we consider to (...) be suberogatory are rendered obligatory, under MEC. I suggest a way to reformulate MEC to solve this problem. (shrink)

Even if we know all relevant descriptive facts about an act, we can still be uncertain about its moral acceptability. Most literature on how to act under such normative uncertainty operates on moral realism, the metaethical view that there are objective moral facts. Lay people largely report anti-realist intuitions, which poses the question of how these intuitions affect their interpretation and handling of normative uncertainty. Results from two quasi-experimental studies (total N = 365) revealed that most people did not interpret (...) normative uncertainty as referring to objective moral facts but rather as uncertainty regarding one’s own view, uncertainty regarding the culturally accepted view or as the result of ambivalence. Especially the anti-realist majority of participants interpreted normative uncertainty different to how it is described in the literature on choice under normative uncertainty. Metaethical views were also associated with lay peoples’ choice of uncertainty reduction strategies and with assumptions about the intended aim of such strategies. The current findings suggest that empirical investigations of normative uncertainty might benefit from considering folk metaethical pluralism, as the lay public largely disagrees with the metaethical assumptions underlying the current discourse on choice under normative uncertainty. (shrink)

This paper addresses the debate surrounding epistocracy. While many discussions of epistocracy focus on its instrumental defenses, this paper aims to critically examine the non-instrumental jury argument offered by Jason Brennan. Brennan’s argument equates the rights of individuals in political decisions to their rights in jury decisions, asserting that just as individuals have a right to a competent jury, they likewise have a right to a competent electorate. We disagree. By juxtaposing the costs of enforcing such rights and the severity (...) of the harm prevented by their enforcement, this paper argues in favor of maintaining the right to a competent jury while denying the existence of a right to a competent electorate. The central claim is that while securing a competent jury is feasible and vital, attempting to secure a competent electorate poses significant challenges and may prevent less significant harm than perceived. (shrink)

This Special Issue titled "Severe Uncertainty in Science, Medicine and Technology" aims to shed new light on the understanding of severe uncertainty and its multifaceted implications. The main idea of the papers of this collection is that, despite possible sophisticated statistical judgments towards future risks in science, medicine, and technology, severe forms of uncertainty still exist.While ignorance is usually assumed to be a total absence of knowledge, uncertainty often refers to the incompleteness of knowledge or information. In its extreme form, (...) this is called "severe uncertainty" but is also known as "fundamental," "radical," "deep," "great," or "genuine" uncertainty. A common characteristic of these... (shrink)

A popular model of normative decision-making under uncertainty suggests choosing the option with the maximum expected moral choice-worthiness (MEC), where the choice-worthiness values from each moral theory, which are assumed commensurable, are weighted by credence and combined. This study adds descriptive uncertainty about the non-moral facts of a situation into the model by treating choice-worthiness as a random variable. When agents face greater descriptive uncertainty, the choice-worthiness random variable will have a greater spread and a larger standard deviation. MEC, as (...) a decision rule, is sensitive only to the expected value of the random variable and not to its standard deviation. For example, MEC is insensitive to cases where an option with a large degree of descriptive uncertainty may have a higher probability of being below some threshold of impermissibility than does an option with less dispersion, even though the latter has a higher expected choice-worthiness. When applied to the same situation, similar moral theories will have statistically correlated choice-worthiness values. This correlation affects the dispersion of the credence-weighted sum of the random variables but not its expected value. Thus, MEC is insensitive to aspects of the normative situation to which a good decision-rule should be sensitive. (shrink)

Nick Bostrom and others have suggested treating decision-making under moral uncertainty as analogous to parliamentary decision-making. The core suggestion of this “parliamentary approach” is that the competing moral theories function like delegates to the parliament, and that these delegates then make decisions by some combination of bargaining and voting. There seems some reason to hope that such an approach might avoid standard objections to existing approaches (for example, the “maximise expected choiceworthiness” (MEC) and “my favourite theory” approaches). However, the parliamentary (...) approach is so far extremely underspecified, making it largely indeterminate how such a model will in fact behave in the respects that those concerned with moral uncertainty care about. This paper explores one way of making it precise. We treat predicaments of moral uncertainty as analogous to bargaining situations alone (setting aside voting), and apply a version of the Nash solution that is standard in bargaining theory. The resulting model does indeed perform in many of the hoped-for ways. However, so also does a version of MEC that employs a structural approach to intertheoretic comparisons. It seems to us an open question which, regarding this version of MEC and the bargaining-theoretic approach, is superior to the other. We identify the key points on which the two differ. (shrink)

In this thesis, I discuss the aggregation problem for T. M. Scanlon’s “contractualism”. I argue that Scanlonian contractualists have the following two options when it comes to the aggregation problem. First, they can choose to limit aggregation directly via a specific version of the Relevance View, “Sequential Claims-Matching”. Second, Scanlonian contractualists can adopt a so-called “mixed solution” of which I propose a specific version. My mixed solution does not limit aggregation. Rather, it either avoids some of the counterintuitive results in (...) certain cases, or at least makes these same results look a lot less counterintuitive by putting them into perspective thanks to a plausible plurality of precisely specified values. The stakes between these two options are as follows. The first option justifies Scanlon’s intuition in one vs. many cases of which his “World Cup Case” is one example, but it comes at the cost of a seemingly quite limited range of application: any version of the Relevance View only works in cases in which we have no more than two groups of competing claimants. The second option has the exact opposite implications: my mixed solution cannot justify Scanlon’s intuition in one vs. many cases, but it could be applied to any case with any number of groups of competing claimants. In this choice between pre-theoretical intuition and feasibility, I would urge Scanlonian contractualists to choose feasibility. (shrink)

This paper explores the role of moral uncertainty in explaining the morally disruptive character of new technologies. We argue that existing accounts of technomoral change do not fully explain its disruptiveness. This explanatory gap can be bridged by examining the epistemic dimensions of technomoral change, focusing on moral uncertainty and inquiry. To develop this account, we examine three historical cases: the introduction of the early pregnancy test, the contraception pill, and brain death. The resulting account highlights what we call “differential (...) disruption” and provides a resource for fields such as technology assessment, ethics of technology, and responsible innovation. (shrink)

Every scientific or intellectual movement rests on central premises and assumptions that shape its philosophy. The purpose of this study is to review a brief account of the main philosophical bases of grey systems theory (GST) and the paradigm governing its principles. So, the recent studies on the philosophical foundations of GST have been reviewed and tried to pay attention to some key ambiguities in the previous studies and give more and clearer explanations in this paper. Also, this paper tries (...) to fill the gap among the previous studies and to provide a purposeful connection between them by expressing two key concepts: complete information and imbalance knowledge. Primarily, the study addresses the theoretical foundations of uncertainty and the concept of greyness. Next, it focuses on the notion of “complete information” and challenges to it. Then, it reviews such processes as perception, cognition, and understanding, as well as their dynamic nature. It explains how knowledge is produced through understanding and interpreting information/data and the dynamics governing the whole process. Also, the study describes any dataset, no matter how large it may be, will remain incomplete, imperfect, and grey, so humans only rely on incomplete datasets to interpret the world. As such, information and knowledge are always grey and uncertain because they are basically contingent on subjective understandings and interpretations and imperfect inputs and data. Finally, as a key development, this study also demonstrates that human grey knowledge remains imbalanced across different disciplines and spheres. In the end, a brief overview of the philosophical paradigm of GST is also provided. GST is depicted as an anti-realistic, anti-positivistic, and non-deterministic approach, which is inherently pluralistic and ideographic. According to GST principles, dynamicity and change are essential parts of human narratives of the world and systems, and human knowledge is constantly reproduced through collecting new information. As a result, knowledge, theories, narratives, and scientific laws dynamically change. Given this premise, one could argue that GST is considerably compatible with the postulates of post-modern thinking. (shrink)

This article is about the role of factual uncertainty for moral decision-making as it concerns the ethics of machine decision-making. The view that is defended here is that factual uncertainties require a normative evaluation and that ethics of machine decision faces a triple-edged problem, which concerns what a machine ought to do, given its technical constraints, what decisional uncertainty is acceptable, and what trade-offs are acceptable to decrease the decisional uncertainty.