Accelerating global climate change drives new climate risks. People around the world are researching, designing, and implementing strategies to manage these risks. Identifying and implementing sound climate risk management strategies poses nontrivial challenges including (a) linking the required disciplines, (b) identifying relevant values and objectives, (c) identifying and quantifying important uncertainties, (d) resolving interactions between decision levers and the system dynamics, (e) quantifying the trade-offs between diverse values under deep and dynamic uncertainties, (f) communicating to inform decisions, and (g) (...) learning from the decision-making needs to inform research design. Here we review these challenges and avenues to overcome them. ▪ People and institutions are confronted with emerging and dynamic climate risks. ▪ Stakeholder values are central to defining the decision problem. ▪ Mission-oriented basic research helps to improve the design of climate risk management strategies. (shrink)

This volume bridges contemporary philosophical conceptions of risk and responsibility and offers an extensive examination of the topic. It shows that risk and responsibility combine in ways that give rise to new philosophical questions and problems. Philosophical interest in the relationship between risk and responsibility continues to rise, due in no small part due to environmental crises, emerging technologies, legal developments, and new medical advances. Despite such interest, scholars are just now working out how to conceive of (...) the links between risk and responsibility, the implications that risks may have to conceptions of responsibility (and vice versa), as well as how such theorizing might play out in applied cases. With contributions from leading scholars, this volume brings together new work examining the interplay between risk and responsibility, exploring its varied philosophical aspects and applications to contemporary issues in law, bioethics, technology, and environmental ethics. Risk and Responsibility in Context will be of interest to philosophers working in ethics, bioethics, philosophy of law, and philosophy of technology, as well as scholars and practitioners in law, health and science management, public policy, and environmental studies. (shrink)

A growing number of decision theorists have, in recent years, defended the view that rationality is permissive under risk: Different rational agents may be more or less risk-averse or risk-inclined. This can result in them making different choices under risk even if they value outcomes in exactly the same way. One pressing question that arises once we grant such permissiveness is what attitude to risk we should implement when choosing on behalf of other people. Are (...) we permitted to implement any of the rationally permissible risk attitudes, is there some specific risk attitude that is required when choosing for others, or are we required to defer to the risk attitudes of the people on whose behalf we are choosing? This article elaborates on this question, explains its wider practical and theoretical significance, provides an overview of existing answers, and explores how to go about providing a more systematic account of how to choose on behalf of others in risky contexts. (shrink)

In addition to protecting agents’ autonomy, consent plays a crucial social role: it enables agents to secure partners in valuable interactions that would be prohibitively morally risk otherwise. To do this, consent must be observable: agents must be able to track the facts about whether they have received a consent-based permission. I argue that this morally justifies a consent-practice on which communicating that one consents is sufficient for consent, but also generates robust constraints on what sorts of behaviors can (...) be taken as consent- communicating. (shrink)

Failing to recognise sentience in AI systems (false negatives) poses a far greater risk of potentially astronomical suffering compared to mistakenly attributing sentience to non-sentient systems (false positives). This paper analyses the issue through the moral frameworks of longtermism, utilitarianism, and deontology, concluding that all three assign greater urgency to avoiding false negatives. Given the astronomical number of AIs that may exist in the future, even a small chance of overlooking sentience is an unacceptable risk. To address this, (...) the paper proposes a comprehensive approach including research, field-building, and tentative policy development. Humanity must take steps to ensure the well-being of all sentient minds, both biological and artificial. (shrink)

This article argues that Lara Buchak’s risk-weighted expected utility (REU) theory fails to offer a true alternative to expected utility theory. Under commonly held assumptions about dynamic choice and the framing of decision problems, rational agents are guided by their attitudes to temporally extended courses of action. If so, REU theory makes approximately the same recommendations as expected utility theory. Being more permissive about dynamic choice or framing, however, undermines the theory’s claim to capturing a steady choice disposition in (...) the face of risk. I argue that this poses a challenge to alternatives to expected utility theory more generally. (shrink)

A natural view in distributive ethics is that everyone's interests matter, but the interests of the relatively worse off matter more than the interests of the relatively better off. I provide a new argument for this view. The argument takes as its starting point the proposal, due to Harsanyi and Rawls, that facts about distributive ethics are discerned from individual preferences in the "original position." I draw on recent work in decision theory, along with an intuitive principle about risk-taking, (...) to derive the view. (shrink)

Why can testimony alone be enough for findings of liability? Why statistical evidence alone can’t? These questions underpin the “Proof Paradox” (Redmayne 2008, Enoch et al. 2012). Many epistemologists have attempted to explain this paradox from a purely epistemic perspective. I call it the “Epistemic Project”. In this paper, I take a step back from this recent trend. Stemming from considerations about the nature and role of standards of proof, I define three requirements that any successful account in line with (...) the Epistemic Project should meet. I then consider three recent epistemic accounts on which the standard is met when the evidence rules out modal risk (Pritchard 2018), normic risk (Ebert et al. 2020), or relevant alternatives (Gardiner 2019 2020). I argue that none of these accounts meets all the requirements. Finally, I offer reasons to be pessimistic about the prospects of having a successful epistemic explanation of the paradox. I suggest the discussion on the proof paradox would benefit from undergoing a ‘value-turn’. (shrink)

I have claimed that risk-weighted expected utility maximizers are rational, and that their preferences cannot be captured by expected utility theory. Richard Pettigrew and Rachael Briggs have recently challenged these claims. Both authors argue that only EU-maximizers are rational. In addition, Pettigrew argues that the preferences of REU-maximizers can indeed be captured by EU theory, and Briggs argues that REU-maximizers lose a valuable tool for simplifying their decision problems. I hold that their arguments do not succeed and that my (...) original claims still stand. However, their arguments do highlight some costs of REU theory. (shrink)

The standard argument to the conclusion that artificial intelligence (AI) constitutes an existential risk for the human species uses two premises: (1) AI may reach superintelligent levels, at which point we humans lose control (the ‘singularity claim’); (2) Any level of intelligence can go along with any goal (the ‘orthogonality thesis’). We find that the singularity claim requires a notion of ‘general intelligence’, while the orthogonality thesis requires a notion of ‘instrumental intelligence’. If this interpretation is correct, they cannot (...) be joined as premises and the argument for the existential risk of AI turns out invalid. If the interpretation is incorrect and both premises use the same notion of intelligence, then at least one of the premises is false and the orthogonality thesis remains itself orthogonal to the argument to existential risk from AI. In either case, the standard argument for existential risk from AI is not sound.—Having said that, there remains a risk of instrumental AI to cause very significant damage if designed or used badly, though this is not due to superintelligence or a singularity. (shrink)

Ramsey (1926) sketches a proposal for measuring the subjective probabilities of an agent by their observable preferences, assuming that the agent is an expected utility maximizer. I show how to extend the spirit of Ramsey's method to a strictly wider class of agents: risk-weighted expected utility maximizers (Buchak 2013). In particular, I show how we can measure the risk attitudes of an agent by their observable preferences, assuming that the agent is a risk-weighted expected utility maximizer. Further, (...) we can leverage this method to measure the subjective probabilities of a risk-weighted expected utility maximizer. (shrink)

Critical race theorists and standpoint epistemologists argue that agents who are members of dominant social groups are often in a state of ignorance about the extent of their social dominance, where this ignorance is explained by these agents' membership in a socially dominant group (e.g., Mills 2007). To illustrate this claim bluntly, it is argued: 1) that many white men do not know the extent of their social dominance, 2) that they remain ignorant as to the extent of their dominant (...) social position even where this information is freely attainable, and 3) that this ignorance is due in part to the fact that they are white men. We argue that on Buchak's (2010, 2013) model of risk averse instrumental rationality, ignorance of one's privileges can be rational. This argument yields a new account of elite-group ignorance, why it may occur, and how it might be alleviated. (shrink)

A moderately risk averse person may turn down a 50/50 gamble that either results in her winning $200 or losing $100. Such behaviour seems rational if, for instance, the pain of losing $100 is felt more strongly than the joy of winning $200. The aim of this paper is to examine an influential argument that some have interpreted as showing that such moderate risk aversion is irrational. After presenting an axiomatic argument that I take to be the strongest (...) case for the claim that moderate risk aversion is irrational, I show that it essentially depends on an assumption that those who think that risk aversion can be rational should be skeptical of. Hence, I conclude that risk aversion need not be irrational. (shrink)

Violence risk assessment tools are increasingly used within criminal justice and forensic psychiatry, however there is little relevant, reliable and unbiased data regarding their predictive accuracy. We argue that such data are needed to (i) prevent excessive reliance on risk assessment scores, (ii) allow matching of different risk assessment tools to different contexts of application, (iii) protect against problematic forms of discrimination and stigmatisation, and (iv) ensure that contentious demographic variables are not prematurely removed from risk (...) assessment tools. (shrink)

This book examines the moral luck paradox, relating it to Kantian, consequentialist and virtue-based approaches to ethics. It also applies the paradox to areas in medical ethics, including allocation of scarce medical resources, informed consent to treatment, withholding life-sustaining treatment, psychiatry, reproductive ethics, genetic testing and medical research. If risk and luck are taken seriously, it might seem to follow that we cannot develop any definite moral standards, that we are doomed to moral relativism. However, Dickenson offers strong counter-arguments (...) to this view that enable us to think in terms of universal standards. (shrink)

In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart’s Law as “Virtually any goal specification, pursued to the extreme, will result in the extinction of humanity”, and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart’s Law holds or not. As our key contribution, we identify a set of conditions that are necessary for a model that aims to be (...) informative for evaluating specific arguments for Extinction-level Goodhart’s Law. Since each of the conditions seems to significantly contribute to the complexity of the resulting model, formally evaluating the hypothesis might be exceedingly difficult. This raises the possibility that whether the risk of extinction from artificial intelligence is real or not, the underlying dynamics might be invisible to current scientific methods. (shrink)

There has been a movement aiming to teach agents about their privilege by making the information about their privilege as costless as possible. However, some argue that in risk-sensitive frameworks, such as Lara Buchak’s (2013), it can be rational for privileged agents to shield themselves from learning about their privilege, even if the information is costless and relevant. This threatens the efficacy of these information-access efforts in alleviating the problem of elite-group ignorance. In response, I show that even within (...) the same framework, in this case David Kinney and Liam Kofi Bright’s (2021), the rationality of this information avoidance rests on shaky ground in practice. In this framework, whether an agent should avoid information depends on the precise details of (1) how relevant they expect the information to be, (2) their priors about the value of various options, and (3) their risk attitudes. The model suggests that the rationality of elite-group ignorance is a function of structural factors that are pervasive but nonetheless not insurmountable, thus offering a way out of pessimism about elite group education. (shrink)

A small but growing number of studies have aimed to understand, assess and reduce existential risks, or risks that threaten the continued existence of mankind. However, most attention has been focused on known and tangible risks. This paper proposes a heuristic for reducing the risk of black swan extinction events. These events are, as the name suggests, stochastic and unforeseen when they happen. Decision theory based on a fixed model of possible outcomes cannot properly deal with this kind of (...) event. Neither can probabilistic risk analysis. This paper will argue that the approach that is referred to as engineering safety could be applied to reducing the risk from black swan extinction events. It will also propose a conceptual sketch of how such a strategy may be implemented: isolated, self-sufficient, and continuously manned underground refuges. Some characteristics of such refuges are also described, in particular the psychosocial aspects. Furthermore, it is argued that this implementation of the engineering safety strategy safety barriers would be effective and plausible and could reduce the risk of an extinction event in a wide range of possible scenarios. Considering the staggering opportunity cost of an existential catastrophe, such strategies ought to be explored more vigorously. (shrink)

The EU Artificial Intelligence Act (AIA) defines four risk categories for AI systems: unacceptable, high, limited, and minimal. However, it lacks a clear methodology for the assessment of these risks in concrete situations. Risks are broadly categorized based on the application areas of AI systems and ambiguous risk factors. This paper suggests a methodology for assessing AI risk magnitudes, focusing on the construction of real-world risk scenarios. To this scope, we propose to integrate the AIA with (...) a framework developed by the Intergovernmental Panel on Climate Change (IPCC) reports and related literature. This approach enables a nuanced analysis of AI risk by exploring the interplay between (a) risk determinants, (b) individual drivers of determinants, and (c) multiple risk types. We further refine the proposed methodology by applying a proportionality test to balance the competing values involved in AI risk assessment. Finally, we present three uses of this approach under the AIA: to implement the Regulation, to assess the significance of risks, and to develop internal risk management systems for AI deployers. (shrink)

Philosophy &Public Affairs, Volume 51, Issue 4, Page 373-412, Fall 2023.

How much does rationality constrain what we should believe on the basis of our evidence? According to this book, not very much. For most people and most bodies of evidence, there is a wide range of beliefs that rationality permits them to have in response to that evidence. The argument, which takes inspiration from William James' ideas in 'The Will to Believe', proceeds from two premises. The first is a theory about the basis of epistemic rationality. It's called epistemic utility (...) theory, and it says that what it is epistemically rational for you to believe is what it would be rational for you to choose if you were given the chance to pick your beliefs and, when picking them, you were to care only about their epistemic value. So, to say which beliefs are permitted, we must say how to measure epistemic value, and which decision rule to use when picking your beliefs. The second premise is a claim about attitudes to epistemic risk, and it says that rationality permits many different such attitudes. These attitudes can show up in epistemic utility theory in two ways: in the way you measure epistemic value; and in the decision rule you use to pick beliefs. This book explores the latter. The result is permissivism about epistemic rationality: different attitudes to epistemic risk lead to different choices of prior beliefs; given most bodies of evidence, different priors lead to different posteriors; and even once we fix your attitudes to epistemic risk, if they are at all risk-inclined, there is a range of different priors and therefore different posteriors they permit. (shrink)

When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk avoidant risk function. This, in turn, has been claimed to require the use of a risk avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the significant (...) risks associated with continued human survival. I raise objections to the claim that respect for others’ risk attitudes requires risk avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others’ risk attitudes in fixed population cases. GPI Working Paper No . 20-2022. (shrink)

Can it be rational to be risk-averse? It seems plausible that the answer is yes—that normative decision theory should accommodate risk aversion. But there is a seemingly compelling class of arguments against our most promising methods of doing so. These long-run arguments point out that, in practice, each decision an agent makes is just one in a very long sequence of such decisions. Given this form of dynamic choice situation, and the (Strong) Law of Large Numbers, they conclude (...) that those theories which accommodate risk aversion end up delivering the same verdicts as risk-neutral theories in nearly all practical cases. If so, why not just accept a simpler, risk-neutral theory? The resulting practical verdicts seem to be much the same. In this paper, I show that these arguments do not in fact condemn those risk-aversion-accommodating theories. Risk aversion can indeed survive the long run. (shrink)

Catastrophic risk raises questions that are not only of practical importance, but also of great philosophical interest, such as how to define catastrophe and what distinguishes catastrophic outcomes from non-catastrophic ones. Catastrophic risk also raises questions about how to rationally respond to such risks. How to rationally respond arguably partly depends on the severity of the uncertainty, for instance, whether quantitative probabilistic information is available, or whether only comparative likelihood information is available, or neither type of information. Finally, (...) catastrophic risk raises important ethical questions about what to do when catastrophe avoidance conflicts with equity promotion. (shrink)

This paper examines a recent argument in favour of strong precautionary action—possibly including working to hasten human extinction—on the basis of a decision-theoretic view that accommodates the risk-attitudes of all affected while giving more weight to the more risk-averse attitudes. First, we dispute the need to take into account other people’s attitudes towards risk at all. Second we argue that a version of the non-identity problem undermines the case for doing so in the context of future people. (...) Lastly, we suggest that we should not work to hasten human extinction, even if significant risk aversion is warranted. (shrink)

It is widely held that the influence of risk on rational decisions is not entirely explained by the shape of an agent’s utility curve. Buchak (Erkenntnis, 2013, Risk and rationality, Oxford University Press, Oxford, in press) presents an axiomatic decision theory, risk-weighted expected utility theory (REU), in which decision weights are the agent’s subjective probabilities modified by his risk-function r. REU is briefly described, and the global applicability of r is discussed. Rabin’s (Econometrica 68:1281–1292, 2000) calibration (...) theorem strongly suggests that plausible levels of risk aversion cannot be fully explained by concave utility functions; this provides motivation for REU and other theories. But applied to the synchronic preferences of an individual agent, Rabin’s result is not as problematic as it may first appear. Theories that treat outcomes as gains and losses (e.g. prospect theory and cumulative prospect theory) account for risk sensitivity in a way not available to REU. Reference points that mark the difference between gains and losses are subject to framing, many instances of which cannot be regarded as rational. However, rational decision theory may recognize the difference between gains and losses, without endorsing all ways of fixing the point of reference. In any event, REU is a very interesting theory. (shrink)

The primary responsibility of the US Food and Drug Administration is to protect public health by ensuring the safety of the food supply. To that end, it sometimes conducts risk assessments of novel food products, such as genetically modified food. The FDA describes its regulatory review of GM food as a purely scientific activity, untainted by any normative considerations. This paper provides evidence that the regulatory agency is not justified in making that claim. It is argued that the FDA’s (...) policy stance on GM food is shaped by neoliberal considerations. The agency’s review of a genetically engineered animal, the AquAdvantage salmon, is used as a case study to track the influence of neoliberalism on its regulatory review protocol. After that, an epistemic argument justifying public engagement in the risk assessment of new GM food is outlined. It is because risk evaluations involve normative judgments, in a democracy, layperson representatives of informal epistemic communities that could be affected by a new GM food should have the opportunity to decide the ethical, political or other normative questions that arise during the regulatory review of that entity. (shrink)

Many philosophers assume that, when making moral decisions under uncertainty, we should choose the option that has the greatest expected moral value, regardless of how risky it is. But their arguments for maximizing expected moral value do not support it over rival, risk-averse approaches. In this paper, I present a novel argument for maximizing expected value: when we think about larger series of decisions that each decision is a part of, all but the most risk-averse agents would prefer (...) that we consistently choose the option with the highest expected value. To the extent that what we choose on a given occasion should be guided by the entire series of choices we prefer, then on each occasion, we should choose the option with the highest expected moral value. (shrink)

Under what conditions does machine learning (ML) model opacity inhibit the possibility of explaining and understanding phenomena? In this article, I argue that nonepistemic values give shape to the ML opacity problem even if we keep researcher interests fixed. Treating ML models as an instance of doing model-based science to explain and understand phenomena reveals that there is (i) an external opacity problem, where the presence of inductive risk imposes higher standards on externally validating models, and (ii) an internal (...) opacity problem, where greater inductive risk demands a higher level of transparency regarding the inferences the model makes. (shrink)

Almost all philosophers agree that a necessary condition on lying is that one says what one believes to be false. But, philosophers haven’t considered the possibility that the true requirement on lying concerns, rather, one’s degree-of-belief. Liars impose a risk on their audience. The greater the liar’s confidence that what she asserts is false, the greater the risk she’ll think she’s imposing on the dupe, and, therefore, the greater her blameworthiness. From this, I arrive at a dilemma: either (...) the belief requirement is wrong, or lying isn’t interesting. I suggest an alternative necessary condition for lying on a degree-of-belief framework. (shrink)

Uncertainty, insufficient information or information of poor quality, limited cognitive capacity and time, along with value conflicts and ethical considerations, are all aspects thatmake risk managementand riskcommunication difficult. This paper provides a review of different risk concepts and describes how these influence risk management, communication and planning in relation to forest ecosystem services. Based on the review and results of empirical studies, we suggest that personal assessment of risk is decisive in the management of forest ecosystem (...) services. The results are used together with a reviewof different principles of the distribution of risk to propose an approach to risk communication that is effective aswell as ethically sound. Knowledge of heuristics and mutual information on both beliefs and desires are important in the proposed risk communication approach. Such knowledge provides an opportunity for relevant information exchange, so that gaps in personal knowledge maps can be filled in and effective risk communication can be promoted. (shrink)

The orthodox theory of instrumental rationality, expected utility (EU) theory, severely restricts the way in which risk-considerations can figure into a rational individual's preferences. It is argued here that this is because EU theory neglects an important component of instrumental rationality. This paper presents a more general theory of decision-making, risk-weighted expected utility (REU) theory, of which expected utility maximization is a special case. According to REU theory, the weight that each outcome gets in decision-making is not the (...) subjective probability of that outcome; rather, the weight each outcome gets depends on both its subjective probability and its position in the gamble. Furthermore, the individual's utility function, her subjective probability function, and a function that measures her attitude towards risk can be separately derived from her preferences via a Representation Theorem. This theorem illuminates the role that each of these entities plays in preferences, and shows how REU theory explicates the components of instrumental rationality. (shrink)

Risk communication has been generally categorized as a warning act, which is performed in order to prevent or minimize risk. On the other side, risk analysis has also underscored the role played by information in reducing uncertainty about risk. In both approaches the safety aspects related to the protection of the right to health are on focus. However, it seems that there are cases where a risk cannot possibly be avoided or uncertainty reduced, this is (...) for instance valid for the declaration of side effects associated with pharmaceutical products or when a decision about drug approval or retirement must be delivered on the available evidence. In these cases, risk communication seems to accomplish other tasks than preventing risk or reducing uncertainty. The present paper analyzes the legal instruments which have been developed in order to control and manage the risks related to drugs – such as the notion of “development risk” or “residual risk” – and relates them to different kinds of uncertainty. These are conceptualized as epistemic, ecological, metric, ethical, and stochastic, depending on their nature. By referring to this taxonomy, different functions of pharmaceutical risk communication are identified and connected with the legal tools of uncertainty management. The purpose is to distinguish the different functions of risk communication and make explicit their different legal nature and implications. (shrink)

In this paper we aim to demonstrate the enormous ethical complexity that is prevalent in child obesity cases. This complexity, we argue, favors a cautious approach. Against those perhaps inclined to blame neglectful parents, we argue that laying the blame for child obesity at the feet of parents is simplistic once the broader context is taken into account. We also show that parents not only enjoy important relational prerogatives worth defending, but that children, too, are beneficiaries of that relationship in (...) ways difficult to match elsewhere. Finally, against the backdrop of growing public concern and pressure to intervene earlier in the life cycle, we examine the perhaps unintended stigmatizing effects that labeling and intervention can have and consider a number of risks and potential harms occasioned by state interventions in these cases. (shrink)

An axiom of medical research ethics is that a protocol is moral only if it has a “favorable risk-benefit ratio”. This axiom is usually interpreted in the following way: a medical research protocol is moral only if it has a positive expected value -- that is, if it is likely to do more good (to both subjects and society) than harm. I argue that, thus interpreted, the axiom has two problems. First, it is unusable, because it requires us to (...) know more about the potential outcomes of research than we ever could. Second, it is false, because it conflicts with the so-called “soft paternalist” principles of liberal democracy. In place of this flawed rule I propose a new way of making risk-benefit assessments, one that does comport with the principles of liberalism. I argue that a protocol is moral only if it would be entered into by competent subjects who are informed about the protocol. The new rule this eschews all pseudo-utilitarian calculation about the protocol’s likely harms and benefits. (shrink)

If the intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity. Time has come to consider these issues, and this consideration must include progress in AI as much as insights from the theory of AI. The papers in this volume try to make cautious headway in setting the problem, evaluating predictions on the future of AI, proposing ways to ensure that AI systems will be beneficial to humans – and (...) critically evaluating such proposals. (shrink)

In “Risk Based Passenger Screening in Aviation Security: Implications and Variants of a New Paradigm”, Sebastian Weydner-Volkmann describes the current paradigm shift from ‘traditional’ forms of screening to ‘risk based passenger screening’ (RBS) in aviation security. This paradigm shift is put in the context of the wider historical development of risk management approaches. Through a discussion of Michel Foucault, Herfried Münkler and Ulrich Beck, Weydner-Volkmann analyses the shortcomings of such approaches in public security policies, which become especially (...) evident in the aviation security context. As he shows, the turn towards methods of RBS can be seen as an attempt to address a trade-off ‘trilemma’ between the effective provision of security, the implied costs for industry and passengers, and the ethical, legal and societal implications of the screening procedures. In order to analyse foreseeable outcomes of embracing RBS, he differentiates three prototypical variants of the new paradigm on the basis of their main referent and rationale. For each variant, he then subsequently assesses the implications for the ‘trilemma’, after having unveiled the criteria of analysis that will necessarily have to be followed within a serious appraisal of RBS methods. (shrink)

Special Issue “Risks of artificial general intelligence”, Journal of Experimental and Theoretical Artificial Intelligence, 26/3 (2014), ed. Vincent C. Müller. http://www.tandfonline.com/toc/teta20/26/3# - Risks of general artificial intelligence, Vincent C. Müller, pages 297-301 - Autonomous technology and the greater human good - Steve Omohundro - pages 303-315 - - - The errors, insights and lessons of famous AI predictions – and what they mean for the future - Stuart Armstrong, Kaj Sotala & Seán S. Ó hÉigeartaigh - pages 317-342 - - (...) - The path to more general artificial intelligence - Ted Goertzel - pages 343-354 - - - Limitations and risks of machine ethics - Miles Brundage - pages 355-372 - - - Utility function security in artificially intelligent agents - Roman V. Yampolskiy - pages 373-389 - - - GOLEM: towards an AGI meta-architecture enabling both goal preservation and radical self-improvement - Ben Goertzel - pages 391-403 - - - Universal empathy and ethical bias for artificial general intelligence - Alexey Potapov & Sergey Rodionov - pages 405-416 - - - Bounding the impact of AGI - András Kornai - pages 417-438 - - - Ethics of brain emulations - Anders Sandberg - pages 439-457. (shrink)

I defend normative externalism from the objection that it cannot account for the wrongfulness of moral recklessness. The defence is fairly simple—there is no wrong of moral recklessness. There is an intuitive argument by analogy that there should be a wrong of moral recklessness, and the bulk of the paper consists of a response to this analogy. A central part of my response is that if people were motivated to avoid moral recklessness, they would have to have an unpleasant sort (...) of motivation, what Michael Smith calls “moral fetishism”. (shrink)

There are few moral principles less controversial than “don’t touch people’s private parts without consent.” Though the principle doesn’t make explicit that there are exceptions, there clearly are some. Parents must wipe their infants. If an unconscious patient is admitted to the emergency room with a profusely bleeding laceration on their genitals, a doctor must give them stitches. The researchers who proposed the study in question, which would look for a connection between burn patients’ microbiomes and their clinical outcomes, presumably (...) believed they had identified an additional exception to the aforementioned principle. I argue that they did not. Rather, because of their tremendous inherent risks, we ought only to perform intimate procedures on those who can’t consent when the procedure is for their own good. -/- Intimate violations have been overlooked in both philosophy and medicine. Because of this, we have lacked an adequate conceptual framework for identifying certain kinds of harms research can inflict. When we understand these special risks, we see that the IRB was right to deny a waiver of consent. (shrink)

Trust is a core feature of the physician-patient relationship, and risk is central to trust. Patients take risks when they trust their providers to care for them effectively and appropriately. Not all patients take these risks: some medical relationships are marked by mistrust and suspicion. Empirical evidence suggests that some patients and families of color in the United States may be more likely to mistrust their providers and to be suspicious of specific medical practices and institutions. Given both historical (...) and ongoing oppression and injustice in American medical institutions, such mistrust can be apt. Yet it can also frustrate patient care, leading to family and provider distress. In this paper, I propose one way that providers might work to reestablish trust by taking risks in signaling their own trustworthiness. This interpersonal step is not meant to replace efforts to remedy systemic injustice, but is an immediate measure for addressing mistrust in occurrent cases. (shrink)

When you are indifferent between two options, it’s rationally permissible to take either. One way to decide between two such options is to flip a fair coin, taking one option if it lands heads and the other if it lands tails. Is it rationally permissible to employ such a tie-breaking procedure? Intuitively, yes. However, if you are genuinely risk-averse—in particular, if you adhere to risk-weighted expected utility theory (Buchak in Risk and rationality, Oxford University Press, 2013) and (...) have a strictly convex risk-function—the answer will often be no: the REU of deciding by coin-flip will be lower than the REU of choosing one of the options outright (so long as at least one of the options is a nondegenerate gamble). This turns out to be a significant worry for risk-weighted expected utility theory. I argue that it adds real bite to established worries about diachronic consistency afflicting views, like risk-weighted expected utility theory, that violate Independence. And that, while these worries might be surmountable, surmounting them comes at a price. (shrink)

This is the editorial for a special volume of JETAI, featuring papers by Omohundro, Armstrong/Sotala/O’Heigeartaigh, T Goertzel, Brundage, Yampolskiy, B. Goertzel, Potapov/Rodinov, Kornai and Sandberg. - If the general intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity – so even if we estimate the probability of this event to be fairly low, it is necessary to think about it now. We need to estimate what progress we can expect, (...) what the impact of superintelligent machines might be, how we might design safe and controllable systems, and whether there are directions of research that should best be avoided or strengthened. (shrink)

The paper re-expresses arguments against the normative validity of expected utility theory in Robin Pope (1983, 1991a, 1991b, 1985, 1995, 2000, 2001, 2005, 2006, 2007). These concern the neglect of the evolving stages of knowledge ahead (stages of what the future will bring). Such evolution is fundamental to an experience of risk, yet not consistently incorporated even in axiomatised temporal versions of expected utility. Its neglect entails a disregard of emotional and financial effects on well-being before a particular (...) class='Hi'>risk is resolved. These arguments are complemented with an analysis of the essential uniqueness property in the context of temporal and atemporal expected utility theory and a proof of the absence of a limit property natural in an axiomatised approach to temporal expected utility theory. Problems of the time structure of risk are investigated in a simple temporal framework restricted to a subclass of temporal lotteries in the sense of David Kreps and Evan Porteus (1978). This subclass is narrow but wide enough to discuss basic issues. It will be shown that there are serious objections against the modification of expected utility theory axiomatised by Kreps and Porteus (1978, 1979). By contrast the umbrella theory proffered by Pope that she has now termed SKAT, the Stages of Knowledge Ahead Theory, offers an epistemically consistent framework within which to construct particular models to deal with particular decision situations. A model by Caplin and Leahy (2001) will also be discussed and contrasted with the modelling within SKAT (Pope, Leopold and Leitner 2007). (shrink)

The essay title is also the overarching content of the article published in Conservation Letters, August 2023. One of the article’s most notable findings is mentioned as follows: “Specifically, threatened vertebrate fauna with habitat capable of supporting highvalue productive lands received less protection and experienced greater habitat loss.”.

We investigate risk attitudes when the underlying domain of payoffs is finite and the payoffs are, in general, not numerical. In such cases, the traditional notions of absolute risk attitudes, that are designed for convex domains of numerical payoffs, are not applicable. We introduce comparative notions of weak and strong risk attitudes that remain applicable. We examine how they are characterized within the rank-dependent utility model, thus including expected utility as a special case. In particular, we characterize (...) strong comparative risk aversion under rank-dependent utility. This is our main result. From this and other findings, we draw two novel conclusions. First, under expected utility, weak and strong comparative risk aversion are characterized by the same condition over finite domains. By contrast, such is not the case under non-expected utility. Second, under expected utility, weak comparative risk aversion is characterized by the same condition when the utility functions have finite range and when they have convex range. By contrast, such is not the case under non-expected utility. Thus, considering comparative risk aversion over finite domains leads to a better understanding of the divide between expected and non-expected utility, more generally, the structural properties of the main models of decision-making under risk. (shrink)

The epistemology of risk examines how risks bear on epistemic properties. A common framework for examining the epistemology of risk holds that strength of evidential support is best modelled as numerical probability given the available evidence. In this essay I develop and motivate a rival ‘relevant alternatives’ framework for theorising about the epistemology of risk. I describe three loci for thinking about the epistemology of risk. The first locus concerns consequences of relying on a belief for (...) action, where those consequences are significant if the belief is false. The second locus concerns whether beliefs themselves—regardless of action—can be risky, costly, or harmful. The third locus concerns epistemic risks we confront as social epistemic agents. I aim to motivate the relevant alternatives framework as a fruitful approach to the epistemology of risk. I first articulate a ‘relevant alternatives’ model of the relationship between stakes, evidence, and action. I then employ the relevant alternatives framework to undermine the motivation for moral encroachment. Finally, I argue the relevant alternatives framework illuminates epistemic phenomena such as gaslighting, conspiracy theories, and crying wolf, and I draw on the framework to diagnose the undue skepticism endemic to rape accusations. (shrink)

Which risks are bad? This is not an easy question to answer in any non-circular way. Not only are risks sought out for various reasons, but risks are plainly discounted in many situations. What may seem "risky" when examined all by itself, may not seem risky when encountered in a real lived situation. Thus risks that are imposed by others, in particular, might seem horrendous when considered in abstraction, but quite acceptable when encountered in life. What we need to do, (...) among other things, is to examine this "seeming." Are people being deluded or distracted when they fail to take seriously the riskiness of activities undertaken by others, or does this make some rational sense? There can be no doubt that people sometimes are deluded, of course. We are all fallible. But risks are not unalloyed evils. They are typically taken because of their association with something positive. So questions about whether people are making wise judgments when they don’t care about certain risks that they take, or, on the other side of the coin, when they become absolutely hysterical when confronted with a risk that appears to others to be insignificant, hinge on a web of further factors. (shrink)

In this paper, we argue that our moral concern for future well-being should reduce over time due to important practical considerations about how humans interact with spacetime. After surveying several of these considerations (around equality, special duties, existential contingency, and overlapping moral concern) we develop a set of core principles that can both explain their moral significance and highlight why this is inherently bound up with our relationship with spacetime. These relate to the equitable distribution of (1) moral concern in (...) a universe that we can only causally affect in one temporal direction; (2) access to the benefits from using spacetime as a resource; and (3) the burdens of care given (1) and (2). We conclude by considering the practical implications of our argument and find that, while it is often assumed that a preference for present over future well-being weakens the case for existential risk mitigation, this likely is not the case. (shrink)