In this article we explore an argumentative pattern that provides a normative justification for expected utility functions grounded on empirical evidence, showing how it worked in three different episodes of their development. The argument claims that we should prudentially maximize our expected utility since this is the criterion effectively applied by those who are considered wisest in making risky choices (be it gamblers or businessmen). Yet, to justify the adoption of this rule, it should be proven that (...) this is empirically true: i.e. that a given function allows us to predict the choices of that particular class of agents. We show how expected utility functions were introduced and contested in accordance with this pattern in the 18th century and how it recurred in the 1950s when Allais made his case against the neo-Bernoullians. (shrink)

This chapter of the Handbook of Utility Theory aims at covering the connections between utility theory and social ethics. The chapter first discusses the philosophical interpretations of utility functions, then explains how social choice theory uses them to represent interpersonal comparisons of welfare in either utilitarian or non-utilitarian representations of social preferences. The chapter also contains an extensive account of John Harsanyi's formal reconstruction of utilitarianism and its developments in the later literature, especially when society faces (...) uncertainty rather than probabilistic risk. (shrink)

It is widely held that under ordinal utility, utility differences are ill-defined. Allegedly, for these to be well-defined (without turning to choice under risk or the like), one should adopt as a new kind of primitive quaternary relations, instead of the traditional binary relations underlying ordinal utility functions. Correlatively, it is also widely held that the key structural properties of quaternary relations are entirely arbitrary from an ordinal point of view. These properties would be, in a (...) nutshell, the hallmark of cardinal utility. While much is obviously true in these two tenets, this note explains why, as stated, they should be abandoned. Any ordinal utility function induces a rich quaternary relation. There is such a thing as ordinal utility differences. Furthermore, this induced quaternary relation respects, apart from completeness, the most standard structural properties of quaternary relations. These properties are, from an ordinal point of view, anything but arbitrary; from a quaternary perspective only completeness should be considered the hallmark—if any—of cardinal utility. These facts are explained to be especially relevant to the critical appreciation of the ordinalist methodology. (shrink)

We reexamine some of the classic problems connected with the use of cardinal utility functions in decision theory, and discuss Patrick Suppes's contributions to this field in light of a reinterpretation we propose for these problems. We analytically decompose the doctrine of ordinalism, which only accepts ordinal utility functions, and distinguish between several doctrines of cardinalism, depending on what components of ordinalism they specifically reject. We identify Suppes's doctrine with the major deviation from ordinalism that conceives (...) of utility functions as representing preference differences, while being non- etheless empirically related to choices. We highlight the originality, promises and limits of this choice-based cardinalism. (shrink)

Suppose that it is rational to choose or intend a course of action if and only if the course of action maximizes some sort of expectation of some sort of value. What sort of value should this definition appeal to? According to an influential neo-Humean view, the answer is “Utility”, where utility is defined as a measure of subjective preference. According to a rival neo-Aristotelian view, the answer is “Choiceworthiness”, where choiceworthiness is an irreducibly normative notion of a (...) course of action that is good in a certain way. The neo-Humean view requires preferences to be measurable by means of a utility function. Various interpretations of what exactly a “preference” is are explored, to see if there is any interpretation that supports the claim that a rational agent’s “preferences” must satisfy the “axioms” that are necessary for them to be measurable in this way. It is argued that the only interpretation that supports the idea that the rational agent’s preferences must meet these axioms interprets “preferences” as a kind of value-judgment. But this turns out to be version of the neo-Aristotelian view, rather than the neo-Humean view. Rational intentions maximize expected choiceworthiness, not expected utility. (shrink)

Matthew Adler's Measuring Social Welfare is an introduction to the social welfare function (SWF) methodology. This essay questions some ideas at the core of the SWF methodology having to do with the relation between the SWF and the measure of well-being. The facts about individual well-being do not single out a particular scale on which well-being must be measured. As with physical quantities, there are multiple scales that can be used to represent the same information about well-being; no one scale (...) is special. Like physical laws, the SWF and its ranking of distributions cannot depend on exactly which of these scales we use. Adler and other theorists in the SWF tradition have used this idea to derive highly restrictive constraints on the shape of the SWF. These constraints rule out seemingly plausible views about distributive justice and population ethics. I argue, however, that these constraints stem from a simple but instructive mistake. The SWF should not be applied to vectors of numbers such as 1 and 2, but rather to vectors of dimensioned quantities such as 1 util and 2 utils. This seemingly pedantic suggestion turns out to have far-reaching consequences. Unlike the orthodox SWF approach, treating welfare levels as dimensioned quantities lets us distinguish between real changes in well-being and mere changes in the unit of measurement. It does this without making the SWF depend on the scale on which welfare is measured, and in a way that avoids the restrictive constraints on the shape of the SWF. (shrink)

Epistemic rationality is typically taken to be immodest at least in this sense: a rational epistemic state should always take itself to be doing at least as well, epistemically and by its own light, than any alternative epistemic state. If epistemic states are probability functions and their alternatives are other probability functions defined over the same collection of proposition, we can capture the relevant sense of immodesty by claiming that epistemic utility functions are (strictly) proper. In (...) this paper I examine what happens if we allow for the alternatives to an epistemic state to include probability functions with different domains. I first prove an impossibility result: on minimal assumptions, I show that there is no way of vindicating strong immodesty principles to the effect that any probability function should take itself to be doing at least as well than any alternative probability function, regardless of its domain. I then consider alternative, weaker generalizations of the traditional immodesty principle and prove some characterization results for some classes of epistemic utility functions satisfying each of the relevant principles. (shrink)

Among recent objections to Pascal's Wager, two are especially compelling. The first is that decision theory, and specifically the requirement of maximizing expected utility, is incompatible with infinite utility values. The second is that even if infinite utility values are admitted, the argument of the Wager is invalid provided that we allow mixed strategies. Furthermore, Hájek has shown that reformulations of Pascal's Wager that address these criticisms inevitably lead to arguments that are philosophically unsatisfying and historically unfaithful. (...) Both the objections and Hájek's philosophical worries disappear, however, if we represent our preferences using relative utilities rather than a one-place utility function. Relative utilities provide a conservative way to make sense of infinite value that preserves the familiar equation of rationality with the maximization of expected utility. They also provide a means of investigating a broader class of problems related to the Wager. (shrink)

Savage's framework of subjective preference among acts provides a paradigmatic derivation of rational subjective probabilities within a more general theory of rational decisions. The system is based on a set of possible states of the world, and on acts, which are functions that assign to each state a consequence€. The representation theorem states that the given preference between acts is determined by their expected utilities, based on uniquely determined probabilities (assigned to sets of states), and numeric utilities assigned to (...) consequences. Savage's derivation, however, is based on a highly problematic well-known assumption not included among his postulates: for any consequence of an act in some state, there is a "constant act" which has that consequence in all states. This ability to transfer consequences from state to state is, in many cases, miraculous -- including simple scenarios suggested by Savage as natural cases for applying his theory. We propose a simplification of the system, which yields the representation theorem without the constant act assumption. We need only postulates P1-P6. This is done at the cost of reducing the set of acts included in the setup. The reduction excludes certain theoretical infinitary scenarios, but includes the scenarios that should be handled by a system that models human decisions. (shrink)

Differential privacy (DP) aims to confer data processing systems with inherent privacy guarantees, offering strong protections for personal data. But DP’s approach to privacy carries with it certain assumptions about how mathematical abstractions will be translated into real-world systems, which—if left unexamined and unrealized in practice—could function to shield data collectors from liability and criticism, rather than substantively protect data subjects from privacy harms. This article investigates these assumptions and discusses their implications for using DP to govern data-driven systems. In (...) Parts 1 and 2, we introduce DP as, on one hand, a mathematical framework and, on the other hand, a kind of real-world sociotechnical system, using a hypothetical case study to illustrate how the two can diverge. In Parts 3 and 4, we discuss the way DP frames privacy loss, data processing interventions, and data subject participation, arguing it could exacerbate existing problems in privacy regulation. In part 5, we conclude with a discussion of DP’s potential interactions with the endogeneity of privacy law, and we propose principles for best governing DP systems. In making such assumptions and their consequences explicit, we hope to help DP succeed at realizing its promise for better substantive privacy protections. (shrink)

Some propositions are more epistemically important than others. Further, how important a proposition is is often a contingent matter—some propositions count more in some worlds than in others. Epistemic Utility Theory cannot accommodate this fact, at least not in any standard way. For EUT to be successful, legitimate measures of epistemic utility must be proper, i.e., every probability function must assign itself maximum expected utility. Once we vary the importance of propositions across worlds, however, normal measures of (...) epistemic utility become improper. I argue there isn’t any good way out for EUT. (shrink)

We study whether it is possible to generalise Seidenfeld et al.’s representation result for coherent choice functions in terms of sets of probability/utility pairs when we let go of Archimedeanity. We show that the convexity property is necessary but not sufficient for a choice function to be an infimum of a class of lexicographic ones. For the special case of two-dimensional option spaces, we determine the necessary and sufficient conditions by weakening the Archimedean axiom.

In his article “Utility Cascades”, Max Khan Hayward argues that act-utilitarians should sometimes either ignore evidence about the effectiveness of their actions or fail to apportion their support to an action's effectiveness. His conclusions are said to have particular significance for the effective altruism movement, which centers seeking and being guided by evidence. Hayward's argument is that act-utilitarians are vulnerable to succumbing to “utility cascades”, that these cascades function to frustrate the ultimate goals of act-utilitarians, and that one (...) apposite way to avoid them is by “ostriching”: ignoring relevant evidence. If true, this conclusion would have remarkable consequences for act-utilitarianism and the effective altruism movement. However, Hayward is mistaken – albeit in an interesting way and with broader significance for moral philosophy. His argument trades on a subtle mischaracterization of act-utilitarianism. Act-utilitarians are not especially vulnerable to utility cascades, and they shouldn't ostrich. (shrink)

This paper develops and explores a new framework for theorizing about the measurement and aggregation of well-being. It is a qualitative variation on the framework of social welfare functionals developed by Amartya Sen. In Sen’s framework, a social or overall betterness ordering is assigned to each profile of real-valued utility functions. In the qualitative framework developed here, numerical utilities are replaced by the properties they are supposed to represent. This makes it possible to characterize the measurability and interpersonal (...) comparability of well-being directly, without the use of invariance conditions, and to distinguish between real changes in well-being and merely representational changes in the unit of measurement. The qualitative framework is shown to have important implications for a range of issues in axiology and social choice theory, including the characterization of welfarism, axiomatic derivations of utilitarianism, the meaningfulness of prioritarianism, the informational requirements of variable-population ethics, the impossibility theorems of Arrow and others, and the metaphysics of value. (shrink)

Tsui and Weymark (Economic Theory, 1997) have shown that the only continuous social welfare orderings on the whole Euclidean space which satisfy the weak Pareto principle and are invariant to individual-specific similarity transformations of utilities are strongly dictatorial. Their proof relies on functional equation arguments which are quite complex. This note provides a simpler proof of their theorem.

We provide conditions under which an incomplete strongly independent preorder on a convex set X can be represented by a set of mixture preserving real-valued functions. We allow X to be infi nite dimensional. The main continuity condition we focus on is mixture continuity. This is sufficient for such a representation provided X has countable dimension or satisfi es a condition that we call Polarization.

Conscious mental states are states we are in some way aware of. I compare higher-order theories of consciousness, which explain consciousness by appeal to such higher-order awareness (HOA), and first-order theories, which do not, and I argue that higher-order theories have substantial explanatory advantages. The higher-order nature of our awareness of our conscious states suggests an analogy with the metacognition that figures in the regulation of psychological processes and behaviour. I argue that, although both consciousness and metacognition involve higher-order psychological (...) states, they have little more in common. One thing they do share is the possibility of misrepresentation; just as metacognitive processing can misrepresent one’s cognitive states and abilities, so the HOA in virtue of which one’s mental states are conscious can, and sometimes does, misdescribe those states. A striking difference between the two, however, has to do with utility for psychological processing. Metacognition has considerable benefit for psychological processing; in contrast, it is unlikely that there is much, if any, utility to mental states’ being conscious over and above the utility those states have when they are not conscious. (shrink)

According to the orthodox treatment of risk preferences in decision theory, they are to be explained in terms of the agent's desires about concrete outcomes. The orthodoxy has been criticised both for conflating two types of attitudes and for committing agents to attitudes that do not seem rationally required. To avoid these problems, it has been suggested that an agent's attitudes to risk should be captured by a risk function that is independent of her utility and probability functions. (...) The main problem with that approach is that it suggests that attitudes to risk are wholly distinct from people's (non-instrumental) desires. To overcome this problem, we develop a framework where an agent's utility function is defined over chance propositions (i.e., propositions describing objective probability distributions) as well as ordinary (non-chance) ones, and argue that one should explain different risk attitudes in terms of different forms of the utility function over such propositions. (shrink)

It is widely held that the influence of risk on rational decisions is not entirely explained by the shape of an agent’s utility curve. Buchak (Erkenntnis, 2013, Risk and rationality, Oxford University Press, Oxford, in press) presents an axiomatic decision theory, risk-weighted expected utility theory (REU), in which decision weights are the agent’s subjective probabilities modified by his risk-function r. REU is briefly described, and the global applicability of r is discussed. Rabin’s (Econometrica 68:1281–1292, 2000) calibration theorem strongly (...) suggests that plausible levels of risk aversion cannot be fully explained by concave utility functions; this provides motivation for REU and other theories. But applied to the synchronic preferences of an individual agent, Rabin’s result is not as problematic as it may first appear. Theories that treat outcomes as gains and losses (e.g. prospect theory and cumulative prospect theory) account for risk sensitivity in a way not available to REU. Reference points that mark the difference between gains and losses are subject to framing, many instances of which cannot be regarded as rational. However, rational decision theory may recognize the difference between gains and losses, without endorsing all ways of fixing the point of reference. In any event, REU is a very interesting theory. (shrink)

Pérez Carballo adopts an epistemic utility theory picture of epistemic norms where epistemic utility functions measure the value of degrees of belief, and rationality consists in maximizing expected epistemic utility. Within this framework he seeks to show that we can make sense of the intuitive idea that some true beliefs—say true beliefs about botany—are more valuable than other true beliefs—say true beliefs about the precise number of plants in North Dakota. To do so, however, Pérez Carballo (...) argues that we must think of the value of epistemic states as consisting in more than simply accuracy. This sheds light on which questions it is most epistemically valuable to pursue. (shrink)

The orthodox theory of instrumental rationality, expected utility (EU) theory, severely restricts the way in which risk-considerations can figure into a rational individual's preferences. It is argued here that this is because EU theory neglects an important component of instrumental rationality. This paper presents a more general theory of decision-making, risk-weighted expected utility (REU) theory, of which expected utility maximization is a special case. According to REU theory, the weight that each outcome gets in decision-making is not (...) the subjective probability of that outcome; rather, the weight each outcome gets depends on both its subjective probability and its position in the gamble. Furthermore, the individual's utility function, her subjective probability function, and a function that measures her attitude towards risk can be separately derived from her preferences via a Representation Theorem. This theorem illuminates the role that each of these entities plays in preferences, and shows how REU theory explicates the components of instrumental rationality. (shrink)

In this paper I offer an account of the normative dimension implicit in D. Bernoulli’s expected utility functions by means of an analysis of the juridical metaphors upon which the concept of mathematical expectation was moulded. Following a suggestion by the late E. Coumet, I show how this concept incorporated a certain standard of justice which was put in question by the St. Petersburg paradox. I contend that Bernoulli would have solved it by introducing an alternative normative criterion (...) rather than a positive model of decision making processes. (shrink)

Papers from the conference on AI Risk (published in JETAI), supplemented by additional work. --- If the intelligence of artificial systems were to surpass that of humans, humanity would face significant risks. The time has come to consider these issues, and this consideration must include progress in artificial intelligence (AI) as much as insights from AI theory. -- Featuring contributions from leading experts and thinkers in artificial intelligence, Risks of Artificial Intelligence is the first volume of collected chapters dedicated to (...) examining the risks of AI. The book evaluates predictions of the future of AI, proposes ways to ensure that AI systems will be beneficial to humans, and then critically evaluates such proposals. 1 Vincent C. Müller, Editorial: Risks of Artificial Intelligence - 2 Steve Omohundro, Autonomous Technology and the Greater Human Good - 3 Stuart Armstrong, Kaj Sotala and Sean O’Heigeartaigh, The Errors, Insights and Lessons of Famous AI Predictions - and What they Mean for the Future - 4 Ted Goertzel, The Path to More General Artificial Intelligence - 5 Miles Brundage, Limitations and Risks of Machine Ethics - 6 Roman Yampolskiy, Utility Function Security in Artificially Intelligent Agents - 7 Ben Goertzel, GOLEM: Toward an AGI Meta-Architecture Enabling Both Goal Preservation and Radical Self-Improvement - 8 Alexey Potapov and Sergey Rodionov, Universal Empathy and Ethical Bias for Artificial General Intelligence - 9 András Kornai, Bounding the Impact of AGI - 10 Anders Sandberg, Ethics and Impact of Brain Emulations 11 Daniel Dewey, Long-Term Strategies for Ending Existential Risk from Fast Takeoff - 12 Mark Bishop, The Singularity, or How I Learned to Stop Worrying and Love AI -. (shrink)

We investigate risk attitudes when the underlying domain of payoffs is finite and the payoffs are, in general, not numerical. In such cases, the traditional notions of absolute risk attitudes, that are designed for convex domains of numerical payoffs, are not applicable. We introduce comparative notions of weak and strong risk attitudes that remain applicable. We examine how they are characterized within the rank-dependent utility model, thus including expected utility as a special case. In particular, we characterize strong (...) comparative risk aversion under rank-dependent utility. This is our main result. From this and other findings, we draw two novel conclusions. First, under expected utility, weak and strong comparative risk aversion are characterized by the same condition over finite domains. By contrast, such is not the case under non-expected utility. Second, under expected utility, weak comparative risk aversion is characterized by the same condition when the utility functions have finite range and when they have convex range. By contrast, such is not the case under non-expected utility. Thus, considering comparative risk aversion over finite domains leads to a better understanding of the divide between expected and non-expected utility, more generally, the structural properties of the main models of decision-making under risk. (shrink)

I consider the problem of how to derive what an agent believes from their credence function and utility function. I argue the best solution of this problem is pragmatic, i.e. it is sensitive to the kinds of choices actually facing the agent. I further argue that this explains why our notion of justified belief appears to be pragmatic, as is argued e.g. by Fantl and McGrath. The notion of epistemic justification is not really a pragmatic notion, but it is (...) being applied to a pragmatically defined concept, i.e. belief. (shrink)

DRAFT In what follows, we will attempt to point out two technical loopholes in the proof of the “Impossibility Theorem for Welfarist Axiologies” as proposed by Gustaf Arrhenius (Arrhenius, 2000). The two problems arise from different principles, one from an application of the “Addition Principle” and one from an application of the “Dominance Principle” after a counterintuitive combination of the “No Repugnant/Anti-Egalitarian Conclusion” criteria, the former non-fatal and latter fatal. We will show that these problems arise via two methodologies. 1) (...) We appeal to intuitions with counterexample cases. 2) We replace ordinal priority-ordering with cardinal utility functions (von Neumann & Morgenstern, 1944). If correct, our critique shows that at least one avenue is available to welfarist axiology, which reopens the possibilities of the search for a “Theory X” (Parfit, 1984). DRAFT. (shrink)

This paper (first published under the same title in Journal of Mathematical Economics, 29, 1998, p. 331-361) is a sequel to "Consistent Bayesian Aggregation", Journal of Economic Theory, 66, 1995, p. 313-351, by the same author. Both papers examine mathematically whether the the following assumptions are compatible: the individuals and the group both form their preferences according to Subjective Expected Utility (SEU) theory, and the preferences of the group satisfy the Pareto principle with respect to those of the individuals. (...) While the 1995 paper explored these assumptions in the axiomatic context of Savage's (1954-1972) SEU theory, the present paper explores them in the context of Anscombe and Aumann's (1963) alternative SEU theory. We first show that the problematic assumptions become compatible when the Anscombe-Aumann utility functions are state-dependent and no subjective probabilities are elicited. Then we show that the problematic assumptions become incompatible when the Anscombe-Aumann utility functions are state-dependent, like before, but subjective probabilities are elicited using a relevant technical scheme. This last result reinstates the impossibilities proved by the 1995 paper, and thus shows them to be robust with respect to the choice of the SEU axiomatic framework. The technical scheme used for the elicitation of subjective probabilities is that of Karni, Schmeidler and Vind (1983). (shrink)

We argue that philosophers ought to distinguish epistemic decision theory and epistemology, in just the way ordinary decision theory is distinguished from ethics. Once one does this, the internalist arguments that motivate much of epistemic decision theory make sense, given specific interpretations of the formalism. Making this distinction also causes trouble for the principle called Propriety, which says, roughly, that the only acceptable epistemic utility functions make probabilistically coherent credence functions immodest. We cast doubt on this requirement, (...) but then argue that epistemic decision theorists should never have wanted such a strong principle in any case. (shrink)

Epistemic states of uncertainty play important roles in ethical and political theorizing. Theories that appeal to a “veil of ignorance,” for example, analyze fairness or impartiality in terms of certain states of ignorance. It is important, then, to scrutinize proposed conceptions of ignorance and explore promising alternatives in such contexts. Here, I study Lerner’s probabilistic egalitarian theorem in the setting of imprecise probabilities. Lerner’s theorem assumes that a social planner tasked with distributing income to individuals in a population is “completely (...) ignorant” about which utility functions belong to which individuals. Lerner models this ignorance with a certain uniform probability distribution, and shows that, under certain further assumptions, income should be equally distributed. Much of the criticism of the relevance of Lerner’s result centers on the representation of ignorance involved. Imprecise probabilities provide a general framework for reasoning about various forms of uncertainty including, in particular, ignorance. To what extent can Lerner’s conclusion be maintained in this setting? (shrink)

This review essay of Economics Rules situates Dani Rodrik’s contribution with respect to the 2007–2008 global economic crisis. This financial meltdown, which the eurozone did not fully recover from before the Covid-19 pandemic, led to soul- searching among economists as well as a call for heterodox economic approaches. Yet, over the past decade, instead the economics profession has maintained its orthodoxy. Rodrik’s Economics Rules offers a critique of the economics profession that is castigating but mild. It calls for economists to (...) use more and diverse models without becoming wedded to any single model or an overarching vision. Yet Rodrik ratifies many of the benchmark models standard to orthodox economics and provides little ground for a fundamental rethinking of the discipline. This essay analyses the conservatism underlying Rodrik’s approach, which upholds general equilibrium theory and rational expectations underlying the efficient market hypothesis. It argues that the economics discipline’s scope-creep to maintain its applicability to all human decision-making, and its acceptance of all-inclusive utility functions, crowds out moral sentiments and civic virtue. Thus. it argues that rather than urging economists simply to be more cautious in their application of models to address particular social concerns, instead economists must recognise their discipline’s inherent limitations. (shrink)

This paper argues that the types of intention can be modeled both as modal operators and via a multi-hyperintensional semantics. I delineate the semantic profiles of the types of intention, and provide a precise account of how the types of intention are unified in virtue of both their operations in a single, encompassing, epistemic space, and their role in practical reasoning. I endeavor to provide reasons adducing against the proposal that the types of intention are reducible to the mental states (...) of belief and desire, where the former state is codified by subjective probability measures and the latter is codified by a utility function. I argue, instead, that each of the types of intention -- i.e., intention-in-action, intention-as-explanation, and intention-for-the-future -- has as its aim the value of an outcome of the agent's action, as derived by her partial beliefs and assignments of utility, and as codified by the value of expected utility in evidential decision theory. (shrink)

This paper takes issue with an influential interpretationist argument for physicalism about intentionality based on the possibility of radical interpretation. The interpretationist defends the physicalist thesis that the intentional truths supervene on the physical truths by arguing that it is possible for a radical interpreter, who knows all of the physical truths, to work out the intentional truths about what an arbitrary agent believes, desires, and means without recourse to any further empirical information. One of the most compelling arguments for (...) the possibility of radical interpretation, associated most closely with David Lewis and Donald Davidson, gives a central role to decision theoretic representation theorems, which demonstrate that if an agent’s preferences satisfy certain constraints, it is possible to deduce probability and utility functions that represent her beliefs and desires. We argue that an interpretationist who wants to rely on existing representation theorems in defence of the possibility of radical interpretation faces a trilemma, each horn of which is incompatible with the possibility of radical interpretation. (shrink)

Special Issue “Risks of artificial general intelligence”, Journal of Experimental and Theoretical Artificial Intelligence, 26/3 (2014), ed. Vincent C. Müller. http://www.tandfonline.com/toc/teta20/26/3# - Risks of general artificial intelligence, Vincent C. Müller, pages 297-301 - Autonomous technology and the greater human good - Steve Omohundro - pages 303-315 - - - The errors, insights and lessons of famous AI predictions – and what they mean for the future - Stuart Armstrong, Kaj Sotala & Seán S. Ó hÉigeartaigh - pages 317-342 - - (...) - The path to more general artificial intelligence - Ted Goertzel - pages 343-354 - - - Limitations and risks of machine ethics - Miles Brundage - pages 355-372 - - - Utility function security in artificially intelligent agents - Roman V. Yampolskiy - pages 373-389 - - - GOLEM: towards an AGI meta-architecture enabling both goal preservation and radical self-improvement - Ben Goertzel - pages 391-403 - - - Universal empathy and ethical bias for artificial general intelligence - Alexey Potapov & Sergey Rodionov - pages 405-416 - - - Bounding the impact of AGI - András Kornai - pages 417-438 - - - Ethics of brain emulations - Anders Sandberg - pages 439-457. (shrink)

Although expected utility theory has proven a fruitful and elegant theory in the finite realm, attempts to generalize it to infinite values have resulted in many paradoxes. In this paper, we argue that the use of John Conway's surreal numbers shall provide a firm mathematical foundation for transfinite decision theory. To that end, we prove a surreal representation theorem and show that our surreal decision theory respects dominance reasoning even in the case of infinite values. We then bring our (...) theory to bear on one of the more venerable decision problems in the literature: Pascal's Wager. Analyzing the wager showcases our theory's virtues and advantages. To that end, we analyze two objections against the wager: Mixed Strategies and Many Gods. After formulating the two objections in the framework of surreal utilities and probabilities, our theory correctly predicts that (1) the pure Pascalian strategy beats all mixed strategies, and (2) what one should do in a Pascalian decision problem depends on what one's credence function is like. Our analysis therefore suggests that although Pascal's Wager is mathematically coherent, it does not deliver what it purports to, a rationally compelling argument that people should lead a religious life regardless of how confident they are in theism and its alternatives. (shrink)

In this work we answer a long standing request for temporal embeddings of deontic STIT logics by introducing the multi-agent STIT logic TDS . The logic is based upon atemporal utilitarian STIT logic. Yet, the logic presented here will be neutral: instead of committing ourselves to utilitarian theories, we prove the logic TDS sound and complete with respect to relational frames not employing any utilitarian function. We demonstrate how these neutral frames can be transformed into utilitarian temporal frames, while preserving (...) validity. Last, we discuss problems that arise from employing binary utility functions in a temporal setting. (shrink)

According to a widely held view, rationality demands that the preferences of a person be transitive. The transitivity assumption is an axiom in standard theories of rational choice. It is also prima facie very plausible. I argue here that transitivity is not a necessary condition of rationality; it is a constraint only in some cases. The argument presented here is based on the non-linearity of differential utility functions. This paper has four parts. First, I present an argument against (...) the transitivity assumption (I, II). Second, I discuss some objections; this will also bring out some features of the view defended here, like the essentially comparative nature of choice (III). Then, I discuss certain proposals that promise to avoid my conclusion (IV). Finally, I go into some implications concerning the nature of rational decision making (V). To accept some intransitivities as rational does not make our life easier; it can even lead to hard choices. However, it does not lead to skepticism about rational choice. (shrink)

Contemporary mainstream economics cannot be seen as disconnected from philosophical concerns. On the contrary, it should be understood as a defence for a specific philosophy, namely, crude quantitative hedonism where money would measure pleasure and pain. Disguised among a great mathematical apparatus involving utility functions, supply, and demand, lies a specific hedonist philosophy that every year is lectured to thousands of economic and business students around the world. This hedonist philosophy is much less sophisticated than that in ancient (...) hedonist philosophers as Epicurus or Lucretius. Furthermore, it does not solve any of the systematic difficulties regularly faced by hedonist philosophy. However, the argument that economics is detached from philosophy works as a rhetorical artifice to protect its dominant underlying philosophy: Philosophical disputes would have to be addressed within the biased mathematical apparatus of quantitative hedonism. Economists and business students must learn to identify the underlying philosophy in mainstream economics and alternative philosophical systems. (shrink)

Hun Chung’s recent article “Rawls’s Self-Defeat: A Formal Analysis” argues that the selection of results equivalent to justice as fairness can be derived by utilitarianism. Chung argues that these results can be achieved through the use of Rawls’s constructed utility function from his work Justice as Fairness. Although Chung’s article is finely argued and presented in great detail, this paper will show that Chung made three mistakes in the fundamentals of his argument. First, Chung mistakes Rawls’s constructed utility (...) function as actually utilitarian. Second, Chung confuses the motivations and the assumptions of those held by agents in the original position and those held by utilitarian agents. Last, Chung’s previous mistakes result in the incorrect conclusion that an equal distribution of resources will be selected by representative agents instead of the unequal distribution prescribed by the difference principle. (shrink)

In his classic book “the Foundations of Statistics” Savage developed a formal system of rational decision making. The system is based on (i) a set of possible states of the world, (ii) a set of consequences, (iii) a set of acts, which are functions from states to consequences, and (iv) a preference relation over the acts, which represents the preferences of an idealized rational agent. The goal and the culmination of the enterprise is a representation theorem: Any preference relation (...) that satisfies certain arguably acceptable postulates determines a (finitely additive) probability distribution over the states and a utility assignment to the consequences, such that the preferences among acts are determined by their expected utilities. Additional problematic assumptions are however required in Savage's proofs. First, there is a Boolean algebra of events (sets of states) which determines the richness of the set of acts. The probabilities are assigned to members of this algebra. Savage's proof requires that this be a σ-algebra (i.e., closed under infinite countable unions and intersections), which makes for an extremely rich preference relation. On Savage's view we should not require subjective probabilities to be σ-additive. He therefore finds the insistence on a σ-algebra peculiar and is unhappy with it. But he sees no way of avoiding it. Second, the assignment of utilities requires the constant act assumption: for every consequence there is a constant act, which produces that consequence in every state. This assumption is known to be highly counterintuitive. The present work contains two mathematical results. The first, and the more difficult one, shows that the σ-algebra assumption can be dropped. The second states that, as long as utilities are assigned to finite gambles only, the constant act assumption can be replaced by the more plausible and much weaker assumption that there are at least two non-equivalent constant acts. The second result also employs a novel way of deriving utilities in Savage-style systems -- without appealing to von Neumann-Morgenstern lotteries. The paper discusses the notion of “idealized agent" that underlies Savage's approach, and argues that the simplified system, which is adequate for all the actual purposes for which the system is designed, involves a more realistic notion of an idealized agent. (shrink)

When you are indifferent between two options, it’s rationally permissible to take either. One way to decide between two such options is to flip a fair coin, taking one option if it lands heads and the other if it lands tails. Is it rationally permissible to employ such a tie-breaking procedure? Intuitively, yes. However, if you are genuinely risk-averse—in particular, if you adhere to risk-weighted expected utility theory (Buchak in Risk and rationality, Oxford University Press, 2013) and have a (...) strictly convex risk-function—the answer will often be no: the REU of deciding by coin-flip will be lower than the REU of choosing one of the options outright (so long as at least one of the options is a nondegenerate gamble). This turns out to be a significant worry for risk-weighted expected utility theory. I argue that it adds real bite to established worries about diachronic consistency afflicting views, like risk-weighted expected utility theory, that violate Independence. And that, while these worries might be surmountable, surmounting them comes at a price. (shrink)

Whereas many others have scrutinized the Allais paradox from a theoretical angle, we study the paradox from an historical perspective and link our findings to a suggestion as to how decision theory could make use of it today. We emphasize that Allais proposed the paradox asa normative argument, concerned with ‘the rational man’ and not the ‘real man’, to use his words. Moreover, and more subtly, we argue that Allais had an unusual sense of the normative, being concerned not so (...) much with therationality of choicesas with therationality of the agent as a person. These two claims are buttressed by a detailed investigation – the first of its kind – of the 1952 Paris conference on risk, which set the context for the invention of the paradox, and a detailed reconstruction – also the first of its kind – of Allais’s specific normative argument from his numerous but allusive writings. The paper contrasts these interpretations of what the paradox historically represented, with how it generally came to function within decision theory from the late 1970s onwards: that is, as an empirical refutation of the expected utility hypothesis, and more specifically of the condition of von Neumann–Morgenstern independence that underlies that hypothesis. While not denying that this use of the paradox was fruitful in many ways, we propose another use that turns out also to be compatible with an experimental perspective. Following Allais’s hints on ‘the experimental definition of rationality’, this new use consists inletting the experiment itself speak of the rationality or otherwise of the subjects. In the 1970s, a short sequence of papers inspired by Allais implemented original ways of eliciting the reasons guiding the subjects’ choices, and claimed to be able to draw relevant normative consequences from this information. We end by reviewing this forgotten experimental avenue not simply historically, but with a view to recommending it for possible use by decision theorists today. (shrink)

This paper attempts to be a contribution to the epistemological project of explaining complex conceptual structures departing from more basic ones. The central thesis of the paper is that there are what I call “functionally structured concepts”, these are non-harmonic concepts in Dummett’s sense that might be legitimized if there is a function that justifies the tie between the inferential connection the concept allows us to trace. Proving this requires enhancing the russellian existential analysis of definite descriptions to apply to (...) functions and using this in proving the legitimacy of such concepts. The utility of the proposal is shown for the case of thick ethical terms and an attempt is made to use it in explaining the development of natural numbers. This last move could allow us to go one step lower in explaining the genesis of natural numbers while maintaining the notion of abstract numbers as higher order entities. (shrink)

The social welfare functional approach to social choice theory fails to distinguish a genuine change in individual well-beings from a merely representational change due to the use of different measurement scales. A generalization of the concept of a social welfare functional is introduced that explicitly takes account of the scales that are used to measure well-beings so as to distinguish between these two kinds of changes. This generalization of the standard theoretical framework results in a more satisfactory formulation of welfarism, (...) the doctrine that social alternatives are evaluated and socially ranked solely in terms of the well-beings of the relevant individuals. This scale-dependent form of welfarism is axiomatized using this framework. The implications of this approach for characterizing classes of social welfare orderings are also considered. (shrink)

Several areas of welfare economics seek to evaluate states of affairs as a function of interpersonally comparable individual utilities. The aim is to map each state of affairs onto a vector of individual utilities, and then to produce an ordering of these vectors that can be represented by a mathematical function assigning a real number to each. When this approach is used in intertemporal contexts, a central theoretical question concerns the evaluative weight to be applied to utility coming at (...) different times. This question concerns the rate of pure time preference, which is one key determinant of the social discount rate. This article argues that the standard philosophical account of pure time preference is mistaken, because it ascribes to economists a methodological commitment they need not, and often do not, accept. This in turn undercuts the most common philosophical objection to pure time preference, which traces at least to Rawls’s A Theory of Justice. The article then evaluates three further objections to pure time preference, concluding that it might still be defensible under certain circumstances. The article closes by articulating a final argument that is suggested by the “Social, Economic and Ethical Concepts and Methods” chapter of the most recent Intergovernmental Panel on Climate Change report. If this further argument is sound, it would constitute a decisive objection to pure time preference as it currently figures in much intertemporal welfare economics. (shrink)

Intelligent neurotechnology is an emerging field that combines neurotechnologies like brain-computer interface (BCI) with artificial intelligence. This paper introduces a capability framework to assess the responsible use of intelligent BCI systems and provide practical ethical guidance. It proposes two tests, the threshold and flourishing tests, that BCI applications must meet, and illustrates them in a series of cases. After a brief introduction (Section 1), Section 2 sets forth the capability view and the two tests. It illustrates the threshold test using (...) examples from clinical medicine of BCI applications that enable patients with profound disabilities to function at a threshold level through computer mediation. Section 3 illustrates the flourishing test by exploring possible future applications of BCI involving neuroenhancements for healthy people, using examples adapted from research currently underway in the US military. Section 3 applies a capability lens to a complex case involving dual effects, both therapeutic and non-therapeutic, showing how the threshold and flourishing tests resolve the case. Section 4 replies to three objections: neurorights are the best tool for assessing BCI; the two tests are moving targets; and the analysis utilizes a capability view to do work it is not designed for. The paper concludes that a capability view offers unique advantages and gives practical guidance for evaluating the responsible use of present and future BCI applications. Extrapolating from our analysis may help guide other emerging technologies, such as germline gene editing, expected to impact central human capabilities. (shrink)

Reductionist accounts of objective chance rely on a notion of fit, which ties the chances at a world to the frequencies at that world. Here, I criticize extant measures of the fit of a chance system and draw on recent literature in epistemic utility theory to propose a new model: chances fit a world insofar as they are accurate at that world. I show how this model of fit does a better job of explaining the normative features of chance, (...) its role in the laws of nature, and its status as an expert function than do previous accounts. (shrink)

ABSTRACT A new ‘political philosophy’ is indispensable to the ‘post-Corona world,’ and this paper tries to analyze the future of ‘liberal democracy’ in it. It shows that ‘liberal democracy’ faces a ‘global crisis’ that has begun before, but the ‘novel Coronavirus pandemic,’ as a setback for it, strongly encourages that crisis. ‘Liberalism’ and ‘democracy,’ which had long been assumed by ‘political philosophers’ to go together, are now becoming decoupled, and the ‘liberal values’ of ‘democracy’ are eroding. To find why and (...) how, this paper analyzes ‘authoritarianism,’ ‘totalitarianism,’ and the evils and propensities of ‘democracy’ that bring about further erosions of ‘liberal values.’ There may be difficult trade-offs to be made between ‘liberal’ and ‘authoritarian’ ‘values’ - and, after the experience of ‘Coronavirus,’ this paper shows the ‘illiberal or authoritarian democracy’ may become stronger. -/- KEYWORDS: democracy, liberalism, liberal democracy, illiberal democracy, the novel Coronavirus pandemic, Covid-19, authoritarianism, totalitarianism. -/- EXTENDED ABSTRACT For almost a century in West, ‘democracy’ has meant ‘liberal democracy’-a political system marked not only by ‘free and fair elections,’ but also by ‘liberal values.’ ‘Liberalism’ applied to the problem of the limits of the criminal law would require commitment to the presumption in favor of ‘liberty.’ If the word ‘liberal’ is to have any utility in this context, it should refer to one who has so powerful a commitment to ‘liberty’ that she is motivated to limit the number of acknowledged liberty-limiting or coercion-legitimizing principles as narrowly as possible. So, she only believes in ‘harm-principle’ as the morally relevant reason for criminal prohibitions. ‘Offense principle’ and ‘paternalistic’ and ‘moralistic’ considerations, when introduced as support for penal legislation, have no weight at all. So, it means ‘autonomy’ and ‘self-regarding vs. other-regarding actions distinction’ based on which the human is the owner of her mind and body and everything consenting adults do is beyond the realm of morality and law. It leads to ‘individual’ ‘basic rights and liberties’ such as ‘basic rights and liberties’ of ‘speech,’ ‘religion,’ and ‘property’ and ‘collective’ ‘basic rights and liberties’ such as ‘basic rights and liberties of assembly,’ ‘civil society,’ ‘political pluralism,’ ‘democratic institutions,’ and ‘non-governmental organizations.’ ‘Liberalism’ also believes in ‘equality of conditions, ‘equal and free participatory rights in political decision making,’ and ‘collective self-governance.’ It also believes in the ‘rule of law,’ a ‘separation of powers,’ and ‘checks and balances.’ However, even under fair and free elections, the elected leaders can be ‘populists,’ ‘ultra-nationalists,’ ‘racists,’ ‘fascists,’ and ‘authoritarians’ who do not respect ‘inviolable basic rights and liberties,’ and suppress ‘minorities.’ Democratically elected leaders can routinely ignore constitutional limits on their power and deprive their citizens of ‘basic rights and liberties.’ The two strands of ‘liberal democracy’ have been coming apart in the world, and the ‘liberal’ elements of ‘democracy’ have been fraying and eroding even before the ‘novel Coronavirus pandemic.’ Even before the Coronavirus hit, there was already much discussion of a crisis of ‘liberal democracy.’ In particular, there has been a debate about whether ‘liberalism’ and ‘democracy,’ which had long been assumed to go together, were becoming decoupled. This paper shows ‘liberal democracy’ faces a ‘global crisis’ that had begun before the ‘novel Coronavirus pandemic.’ It also analyzes ‘authoritarianism,’ ‘totalitarianism,’ ‘liberalism,’ and ‘democracy’ and shows that Covid-19 is a setback for ‘liberal democracy.’ What is striking about the current moment is that many of the ‘liberal’ elements of ‘democracy’ are so far holding up under immense pressure. ‘Illiberal democracies’ seemed to be emerging in many countries. This model of ‘illiberal democracy,’ in which elections continue to be held but some individual rights and liberties are curtailed, may emerge stronger from this new crisis. In that sense, the ‘pandemic’ may become a challenge not only to ‘democracy’ as such but also to ‘liberal democracy’ in particular – in other words, a system of popular sovereignty together with guaranteed basic rights, such as freedom of association and expression and checks and balances on executive power. ‘Authoritarian’ procedures may succeed in mitigating the spread of the ‘Coronavirus,’ but the world now faces another problem: that when the virus recedes, many ‘liberal democracies’ will be far less ‘liberal’ or ‘democratic’ than they were before. In times of crisis, ‘liberal values’ have been ignored temporarily in the name of executive power. However, the ‘temporary’ can become ‘permanent.’ In addition, if citizens lose their faith in the legitimacy of ‘liberal democracy’ as the best form of government, and think ‘liberal democracy’ cannot function effectively during a crisis, and ‘authoritarian regimes’ manage the crisis more decisively, the world will slide towards ‘illiberal or authoritarian democracy,’ and many ‘liberal democracies’ will be at grave risk of failure. There may now be difficult trade-offs to be made between those basic rights and security - and, after the experience of Covid-19, many citizens may choose security. (shrink)

Default positions, predetermined starting points that aid in complex decision-making, are common in clinical medicine. In this article, we identify and critically examine common default positions in clinical ethics practice. Whether default positions ought to be held is an important normative question, but here we are primarily interested in the descriptive, rather than normative, properties of default positions. We argue that default positions in clinical ethics function to protect and promote important values in medicine—respect for persons, utility, and justice. (...) Further, default positions in clinical ethics may also guard against harm. Where default positions exist, there are epistemic burdens to overturn them. The person wishing to reject the default position, rather than the person endorsing it, bears this burden. The person who bears the burden of meeting the epistemic requirements must provide evidence proportional to the degree of harm the default position protects against. Default positions that protect against significant harm impose significant epistemic requirements to overturn. This asymmetry not only makes medical decision-making more economical but also serves to promote and protect certain values. The identification and analysis of common and recognizable default positions can help to identify other default positions and the conditions under which their associated epistemic requirements are met. The article concludes with considerations of potential problems with the use of default positions in clinical ethics. (shrink)

This paper utilizes Robert Smithson's philosophy as a kind of counterpoint, rather than refutation, to many of Hegel's convictions on the nature and function of art in world historical spirit.

Critique of prevailing textbook conception of sufficient conditions and necessary conditions as a truth functional relation of material implication (p->q)/(~q->~p). Explanation of common sense conception of condition as correlative of consequence, involving dependence. Utility of this conception exhibited in resolving puzzles regarding ontology, truth, and fatalism.