A growing number of decision theorists have, in recent years, defended the view that rationality is permissive under risk: Different rational agents may be more or less risk-averse or risk-inclined. This can result in them making different choices under risk even if they value outcomes in exactly the same way. One pressing question that arises once we grant such permissiveness is what attitude to risk we should implement when choosing on behalf of other people. Are we permitted to implement any (...) of the rationally permissible risk attitudes, is there some specific risk attitude that is required when choosing for others, or are we required to defer to the risk attitudes of the people on whose behalf we are choosing? This article elaborates on this question, explains its wider practical and theoretical significance, provides an overview of existing answers, and explores how to go about providing a more systematic account of how to choose on behalf of others in risky contexts. (shrink)

Catastrophic risk raises questions that are not only of practical importance, but also of great philosophical interest, such as how to define catastrophe and what distinguishes catastrophic outcomes from non-catastrophic ones. Catastrophic risk also raises questions about how to rationally respond to such risks. How to rationally respond arguably partly depends on the severity of the uncertainty, for instance, whether quantitative probabilistic information is available, or whether only comparative likelihood information is available, or neither type of information. Finally, catastrophic (...) risk raises important ethical questions about what to do when catastrophe avoidance conflicts with equity promotion. (shrink)

How much does rationality constrain what we should believe on the basis of our evidence? According to this book, not very much. For most people and most bodies of evidence, there is a wide range of beliefs that rationality permits them to have in response to that evidence. The argument, which takes inspiration from William James' ideas in 'The Will to Believe', proceeds from two premises. The first is a theory about the basis of epistemic rationality. It's called epistemic utility (...) theory, and it says that what it is epistemically rational for you to believe is what it would be rational for you to choose if you were given the chance to pick your beliefs and, when picking them, you were to care only about their epistemic value. So, to say which beliefs are permitted, we must say how to measure epistemic value, and which decision rule to use when picking your beliefs. The second premise is a claim about attitudes to epistemic risk, and it says that rationality permits many different such attitudes. These attitudes can show up in epistemic utility theory in two ways: in the way you measure epistemic value; and in the decision rule you use to pick beliefs. This book explores the latter. The result is permissivism about epistemic rationality: different attitudes to epistemic risk lead to different choices of prior beliefs; given most bodies of evidence, different priors lead to different posteriors; and even once we fix your attitudes to epistemic risk, if they are at all risk-inclined, there is a range of different priors and therefore different posteriors they permit. (shrink)

Philosophy &Public Affairs, Volume 51, Issue 4, Page 373-412, Fall 2023.

This paper examines a recent argument in favour of strong precautionary action—possibly including working to hasten human extinction—on the basis of a decision-theoretic view that accommodates the risk-attitudes of all affected while giving more weight to the more risk-averse attitudes. First, we dispute the need to take into account other people’s attitudes towards risk at all. Second we argue that a version of the non-identity problem undermines the case for doing so in the context of future people. Lastly, we suggest (...) that we should not work to hasten human extinction, even if significant risk aversion is warranted. (shrink)

In “Risk Based Passenger Screening in Aviation Security: Implications and Variants of a New Paradigm”, Sebastian Weydner-Volkmann describes the current paradigm shift from ‘traditional’ forms of screening to ‘risk based passenger screening’ (RBS) in aviation security. This paradigm shift is put in the context of the wider historical development of risk management approaches. Through a discussion of Michel Foucault, Herfried Münkler and Ulrich Beck, Weydner-Volkmann analyses the shortcomings of such approaches in public security policies, which become especially evident in the (...) aviation security context. As he shows, the turn towards methods of RBS can be seen as an attempt to address a trade-off ‘trilemma’ between the effective provision of security, the implied costs for industry and passengers, and the ethical, legal and societal implications of the screening procedures. In order to analyse foreseeable outcomes of embracing RBS, he differentiates three prototypical variants of the new paradigm on the basis of their main referent and rationale. For each variant, he then subsequently assesses the implications for the ‘trilemma’, after having unveiled the criteria of analysis that will necessarily have to be followed within a serious appraisal of RBS methods. (shrink)

In addition to protecting agents’ autonomy, consent plays a crucial social role: it enables agents to secure partners in valuable interactions that would be prohibitively morally risk otherwise. To do this, consent must be observable: agents must be able to track the facts about whether they have received a consent-based permission. I argue that this morally justifies a consent-practice on which communicating that one consents is sufficient for consent, but also generates robust constraints on what sorts of behaviors can be (...) taken as consent- communicating. (shrink)

International regulation of autonomous weapon systems (AWS) is increasingly conceived as an exercise in risk management. This requires a shared approach for assessing the risks of AWS. This paper presents a structured approach to risk assessment and regulation for AWS, adapting a qualitative framework inspired by the Intergovernmental Panel on Climate Change (IPCC). It examines the interactions among key risk factors—determinants, drivers, and types—to evaluate the risk magnitude of AWS and establish risk tolerance thresholds through a risk matrix informed (...) by background knowledge of event likelihood and severity. Further, it proposes a methodology to assess community risk appetite, emphasizing that such assessments and resulting tolerance levels should be determined through deliberation in a multistakeholder forum. The paper highlights the complexities of applying risk-based regulations to AWS internationally, particularly the challenge of defining a global community for risk assessment and regulatory legitimization. (shrink)

Uncertainty, insufficient information or information of poor quality, limited cognitive capacity and time, along with value conflicts and ethical considerations, are all aspects thatmake risk managementand riskcommunication difficult. This paper provides a review of different risk concepts and describes how these influence risk management, communication and planning in relation to forest ecosystem services. Based on the review and results of empirical studies, we suggest that personal assessment of risk is decisive in the management of forest ecosystem services. The results are (...) used together with a reviewof different principles of the distribution of risk to propose an approach to risk communication that is effective aswell as ethically sound. Knowledge of heuristics and mutual information on both beliefs and desires are important in the proposed risk communication approach. Such knowledge provides an opportunity for relevant information exchange, so that gaps in personal knowledge maps can be filled in and effective risk communication can be promoted. (shrink)

This chapter discusses how we should think about experiences that threaten to radically transform our understanding of the world. While it can be rational to treat the “doxastically transformative” potential of an experience as a reason to choose against it, such a decision must be based in something more than the fact that this experience would alter one’s current beliefs. It only in light of knowledge of how things are that a person can choose rationally against transformative processes that would (...) destroy this knowledge. (shrink)

Risk propagation encompasses a plethora of techniques for analyzing how risk “spreads” in a given system. Albeit commonly used in technical literature, the very notion of risk propagation turns out to be a conceptually imprecise and overloaded one. This might also explain the multitude of modeling solutions that have been proposed in the lit- erature. Having a clear understanding of what exactly risk is, how it be quantified, and in what sense it can be propagated is fundamental for devising high-quality (...) risk assessment and decision-making solutions. In this paper, we exploit a previous well-established work about the nature of risk and related notions with the goal of providing a proper interpre- tation of the different notions of risk propagation, as well as revealing and harmonizing the alternative semantics for the links used in common risk propagation graphs. Finally, we discuss how these results can be leveraged in practice to model risk propagation scenarios. (shrink)

I have claimed that risk-weighted expected utility maximizers are rational, and that their preferences cannot be captured by expected utility theory. Richard Pettigrew and Rachael Briggs have recently challenged these claims. Both authors argue that only EU-maximizers are rational. In addition, Pettigrew argues that the preferences of REU-maximizers can indeed be captured by EU theory, and Briggs argues that REU-maximizers lose a valuable tool for simplifying their decision problems. I hold that their arguments do not succeed and that my original (...) claims still stand. However, their arguments do highlight some costs of REU theory. (shrink)

Crispin Wright maintains that the architecture of perceptual justification is such that we can acquire justification for our perceptual beliefs only if we have antecedent justification for ruling out any sceptical alternative. Wright contends that this principle doesn’t elicit scepticism, for we are non-evidentially entitled to accept the negation of any sceptical alternative. Sebastiano Moruzzi has challenged Wright’s contention by arguing that since our non-evidential entitlements don’t remove the epistemic risk of our perceptual beliefs, they don’t actually enable us to (...) acquire justification for these beliefs. In this paper I show that Wright’s responses to Moruzzi are ineffective and that Moruzzi’s argument is validated by probabilistic reasoning. I also suggest that Wright cannot answer Moruzzi’s challenge without weakening the support available for his conception of the architecture of perceptual justification. (shrink)

The author argues for a theory of responsibility for outcomes of imposed risk, based on whether it was permissible to impose the risk. When one tries to apply this persuasive model of responsibility for outcomes of risk imposition to procreation, which is a risk imposing act, one finds that it doesn’t match one’s intuitions about responsibility for outcomes of procreative risk. This mismatch exposes a justificatory gap for procreativity, namely, that procreation cannot avail itself of the shared vulnerability to (...) class='Hi'>risks and their constraints—to the balance one is forced to strike between one’s interest in being free to impose risks on others and one’s interest in being safe from harm resulting from the risk imposed by others—which serves to justify risk imposition, generally. Whereas most risk impositions involve trade-offs of liberty and security among people who share the vulnerabilities associated with the taking, imposing, or being constrained from imposing risks, procreation involves the introduction of people into that position of vulnerability in the first place. Thus, when one procreates, one imposes risks in the absence of the shared vulnerability that usually serves as a justification for risk imposition. Procreative risks may not be wrongfully imposed, but they aren’t permissibly imposed in a manner fully comparable to other permissibly imposed risks. This makes procreation a unique form of risk imposition, with unique implications for its justification and for one’s responsibility for its outcomes. This insight can help explain several puzzling procreative asymmetries. (shrink)

This paper aims to scrutinize how the notion of risk should be understood and applied to possibly catastrophic cases. I begin with clarifying the standard usage of the notion of risk, particularly emphasizing the conceptual relation between risk and probability. Then, I investigate how to make decisions in the case of seemingly catastrophic disasters by contrasting the precautionary principle with the preventive (prevention) principle. Finally, I examine what kind of causal thinking tends to be actually adopted when we make decisions (...) on emergent cases. My arguments are mainly based on Japan's 2011 massive earthquake and nuclear power station accident. Masaki Ichinose. (shrink)

This article argues that Lara Buchak’s risk-weighted expected utility (REU) theory fails to offer a true alternative to expected utility theory. Under commonly held assumptions about dynamic choice and the framing of decision problems, rational agents are guided by their attitudes to temporally extended courses of action. If so, REU theory makes approximately the same recommendations as expected utility theory. Being more permissive about dynamic choice or framing, however, undermines the theory’s claim to capturing a steady choice disposition in the (...) face of risk. I argue that this poses a challenge to alternatives to expected utility theory more generally. (shrink)

According to the orthodox treatment of risk preferences in decision theory, they are to be explained in terms of the agent's desires about concrete outcomes. The orthodoxy has been criticised both for conflating two types of attitudes and for committing agents to attitudes that do not seem rationally required. To avoid these problems, it has been suggested that an agent's attitudes to risk should be captured by a risk function that is independent of her utility and probability functions. The main (...) problem with that approach is that it suggests that attitudes to risk are wholly distinct from people's (non-instrumental) desires. To overcome this problem, we develop a framework where an agent's utility function is defined over chance propositions (i.e., propositions describing objective probability distributions) as well as ordinary (non-chance) ones, and argue that one should explain different risk attitudes in terms of different forms of the utility function over such propositions. (shrink)

Vaccination complacency occurs when perceived risks of vaccine-preventable diseases are sufficiently low so that vaccination is no longer perceived as a necessary precaution. Disease outbreaks can once again increase perceptions of risk, thereby decrease vaccine complacency, and in turn decrease vaccine hesitancy. It is not well understood, however, how change in perceived risk translates into change in vaccine hesitancy. -/- We advance the concept of vaccine propensity, which relates a change in willingness to vaccinate with a change in perceived (...) risk of infection—holding fixed other considerations such as vaccine confidence and convenience. (shrink)

This chapter addresses a general question: What is responsible risking? It explores the notion of "responsible risking" as a thick moral concept, and it argues that the notion can be given moral content that could be action-guiding and add an important tool to our moral toolbox. To impose risks responsibly, on this view, is to take on responsibility in a good way. A core part of responsible risking, this chapter argues, is some version of a Forethought Condition. Such a (...) condition requires us to not make decisions or plans such that we could not deliver on our responsibility obligations. This chapter argues that three moral obligations can be derived from descriptive notions of responsibility. They are (1) the control condition, (2) the responsibility to fix condition, and (3) the responsibility to explain condition. The idea is that in order to impose risks responsibly we ought not act in ways that undermine our ability to remain in control over the risks, undermine our ability to sort out negative outcomes by reparations or compensation, or undermine our prospects of providing an explanation for our actions that also makes sense in hindsight and to those affected. This idea is then tried out on the case of germline gene editing. It is argued that such a moral notion of responsibility is also action-guiding when we do not know for certain what is the morally right or obligatory action all-things-considered due to epistemic gaps. (shrink)

When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk avoidant risk function. This, in turn, has been claimed to require the use of a risk avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the significant risks associated with continued (...) human survival. I raise objections to the claim that respect for others’ risk attitudes requires risk avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others’ risk attitudes in fixed population cases. GPI Working Paper No . 20-2022. (shrink)

A natural view in distributive ethics is that everyone's interests matter, but the interests of the relatively worse off matter more than the interests of the relatively better off. I provide a new argument for this view. The argument takes as its starting point the proposal, due to Harsanyi and Rawls, that facts about distributive ethics are discerned from individual preferences in the "original position." I draw on recent work in decision theory, along with an intuitive principle about risk-taking, to (...) derive the view. (shrink)

The EU Artificial Intelligence Act (AIA) defines four risk categories for AI systems: unacceptable, high, limited, and minimal. However, it lacks a clear methodology for the assessment of these risks in concrete situations. Risks are broadly categorized based on the application areas of AI systems and ambiguous risk factors. This paper suggests a methodology for assessing AI risk magnitudes, focusing on the construction of real-world risk scenarios. To this scope, we propose to integrate the AIA with a framework (...) developed by the Intergovernmental Panel on Climate Change (IPCC) reports and related literature. This approach enables a nuanced analysis of AI risk by exploring the interplay between (a) risk determinants, (b) individual drivers of determinants, and (c) multiple risk types. We further refine the proposed methodology by applying a proportionality test to balance the competing values involved in AI risk assessment. Finally, we present three uses of this approach under the AIA: to implement the Regulation, to assess the significance of risks, and to develop internal risk management systems for AI deployers. (shrink)

Perhaps the topic of acceptable risk never had a sexier and more succinct introduction than the one Edward Norton, playing an automobile company executive, gave it in Fight Club: “Take the number of vehicles in the field (A), multiply it by the probable rate of failure (B), and multiply the result by the average out of court settlement (C). A*B*C=X. If X is less than the cost of the recall, we don’t do one.” Of course, this dystopic scene also gets (...) to the heart of the issue in another way: acceptable risk deals with mathematical calculations about the value of life, injury, and emotional wreckage, making calculation a difficult matter ethically, politically, and economically. This entry will explore the history of this idea, focusing on its development alongside statistics into its wide importance today. (shrink)

Why can testimony alone be enough for findings of liability? Why statistical evidence alone can’t? These questions underpin the “Proof Paradox” (Redmayne 2008, Enoch et al. 2012). Many epistemologists have attempted to explain this paradox from a purely epistemic perspective. I call it the “Epistemic Project”. In this paper, I take a step back from this recent trend. Stemming from considerations about the nature and role of standards of proof, I define three requirements that any successful account in line with (...) the Epistemic Project should meet. I then consider three recent epistemic accounts on which the standard is met when the evidence rules out modal risk (Pritchard 2018), normic risk (Ebert et al. 2020), or relevant alternatives (Gardiner 2019 2020). I argue that none of these accounts meets all the requirements. Finally, I offer reasons to be pessimistic about the prospects of having a successful epistemic explanation of the paradox. I suggest the discussion on the proof paradox would benefit from undergoing a ‘value-turn’. (shrink)

Population axiology concerns how to evaluate populations in terms of their moral goodness, that is, how to order populations by the relations “is better than” and “is as good as”. The task has been to find an adequate theory about the moral value of states of affairs where the number of people, the quality of their lives, and their identities may vary. So far, this field has largely ignored issues about uncertainty and the conditions that have been discussed mostly pertain (...) to the ranking of risk-free outcomes. Most public policy choices, however, are decisions under uncertainty, including policy choices that affect the size of a population. Here, we shall address the question of how to rank population prospects—that is, alternatives that contain uncertainty as to which population they will bring about—by the relations “is better than” and “is as good as”. We start by illustrating how well-known population axiologies can be extended to population prospect axiologies. And we show that new problems arise when extending population axiologies to prospects. In particular, traditional population axiologies lead to prospect-versions of the problems that they praised for avoiding in the risk-free settings. Finally, we identify an intuitive adequacy condition that, we contend, should be satisfied by any population prospect axiology, and show how given this condition, the impossibility theorems in population axiology can be extended to (non-trivial) impossibility theorems for population prospect axiology. (shrink)

The co-evolutionary concept of three-modal stable evolutionary strategy of Homo sapiens is developed. The concept based on the principle of evolutionary complementarity of anthropogenesis: value of evolutionary risk and evolutionary path of human evolution are defined by descriptive (evolutionary efficiency) and creative-teleological (evolutionary correctness) parameters simultaneously, that cannot be instrumental reduced to other ones. Resulting volume of both parameters define the vectors of biological, social, cultural and techno-rationalistic human evolution by two gear mechanism — genetic and cultural co-evolution and techno-humanitarian (...) balance. Explanatory model and methodology of evaluation of creatively teleological evolutionary risk component of NBIC technological complex is proposed. Integral part of the model is evolutionary semantics (time-varying semantic code, the compliance of the biological, socio-cultural and techno-rationalist adaptive modules of human stable evolutionary strategy). (shrink)

It is widely held that the influence of risk on rational decisions is not entirely explained by the shape of an agent’s utility curve. Buchak (Erkenntnis, 2013, Risk and rationality, Oxford University Press, Oxford, in press) presents an axiomatic decision theory, risk-weighted expected utility theory (REU), in which decision weights are the agent’s subjective probabilities modified by his risk-function r. REU is briefly described, and the global applicability of r is discussed. Rabin’s (Econometrica 68:1281–1292, 2000) calibration theorem strongly suggests that (...) plausible levels of risk aversion cannot be fully explained by concave utility functions; this provides motivation for REU and other theories. But applied to the synchronic preferences of an individual agent, Rabin’s result is not as problematic as it may first appear. Theories that treat outcomes as gains and losses (e.g. prospect theory and cumulative prospect theory) account for risk sensitivity in a way not available to REU. Reference points that mark the difference between gains and losses are subject to framing, many instances of which cannot be regarded as rational. However, rational decision theory may recognize the difference between gains and losses, without endorsing all ways of fixing the point of reference. In any event, REU is a very interesting theory. (shrink)

We investigate risk attitudes when the underlying domain of payoffs is finite and the payoffs are, in general, not numerical. In such cases, the traditional notions of absolute risk attitudes, that are designed for convex domains of numerical payoffs, are not applicable. We introduce comparative notions of weak and strong risk attitudes that remain applicable. We examine how they are characterized within the rank-dependent utility model, thus including expected utility as a special case. In particular, we characterize strong comparative risk (...) aversion under rank-dependent utility. This is our main result. From this and other findings, we draw two novel conclusions. First, under expected utility, weak and strong comparative risk aversion are characterized by the same condition over finite domains. By contrast, such is not the case under non-expected utility. Second, under expected utility, weak comparative risk aversion is characterized by the same condition when the utility functions have finite range and when they have convex range. By contrast, such is not the case under non-expected utility. Thus, considering comparative risk aversion over finite domains leads to a better understanding of the divide between expected and non-expected utility, more generally, the structural properties of the main models of decision-making under risk. (shrink)

Sources of evolutionary risk for stable strategy of adaptive Homo sapiens are an imbalance of: (1) the intra-genomic co-evolution (intragenomic conflicts); (2) the gene-cultural co-evolution; (3) inter-cultural co-evolution; (4) techno-humanitarian balance; (5) inter-technological conflicts (technological traps). At least phenomenologically the components of the evolutionary risk are reversible, but in the aggregate they are in potentio irreversible destructive ones for biosocial, and cultural self-identity of Homo sapiens. When the actual evolution is the subject of a rationalist control and/or manipulation, the magnitude (...) of the 4th and 5th components of the evolutionary risk reaches a level of existential significance. (shrink)

Many researchers have argued that humanity will create artificial general intelligence (AGI) within the next twenty to one hundred years. It has been suggested that AGI may inflict serious damage to human well-being on a global scale ('catastrophic risk'). After summarizing the arguments for why AGI may pose such a risk, we review the fieldʼs proposed responses to AGI risk. We consider societal proposals, proposals for external constraints on AGI behaviors and proposals for creating AGIs that are safe due to (...) their internal design. (shrink)

The standard argument to the conclusion that artificial intelligence (AI) constitutes an existential risk for the human species uses two premises: (1) AI may reach superintelligent levels, at which point we humans lose control (the ‘singularity claim’); (2) Any level of intelligence can go along with any goal (the ‘orthogonality thesis’). We find that the singularity claim requires a notion of ‘general intelligence’, while the orthogonality thesis requires a notion of ‘instrumental intelligence’. If this interpretation is correct, they cannot be (...) joined as premises and the argument for the existential risk of AI turns out invalid. If the interpretation is incorrect and both premises use the same notion of intelligence, then at least one of the premises is false and the orthogonality thesis remains itself orthogonal to the argument to existential risk from AI. In either case, the standard argument for existential risk from AI is not sound.—Having said that, there remains a risk of instrumental AI to cause very significant damage if designed or used badly, though this is not due to superintelligence or a singularity. (shrink)

Violence risk assessment tools are increasingly used within criminal justice and forensic psychiatry, however there is little relevant, reliable and unbiased data regarding their predictive accuracy. We argue that such data are needed to (i) prevent excessive reliance on risk assessment scores, (ii) allow matching of different risk assessment tools to different contexts of application, (iii) protect against problematic forms of discrimination and stigmatisation, and (iv) ensure that contentious demographic variables are not prematurely removed from risk assessment tools.

This book examines the moral luck paradox, relating it to Kantian, consequentialist and virtue-based approaches to ethics. It also applies the paradox to areas in medical ethics, including allocation of scarce medical resources, informed consent to treatment, withholding life-sustaining treatment, psychiatry, reproductive ethics, genetic testing and medical research. If risk and luck are taken seriously, it might seem to follow that we cannot develop any definite moral standards, that we are doomed to moral relativism. However, Dickenson offers strong counter-arguments to (...) this view that enable us to think in terms of universal standards. (shrink)

In this paper, we argue that our moral concern for future well-being should reduce over time due to important practical considerations about how humans interact with spacetime. After surveying several of these considerations (around equality, special duties, existential contingency, and overlapping moral concern) we develop a set of core principles that can both explain their moral significance and highlight why this is inherently bound up with our relationship with spacetime. These relate to the equitable distribution of (1) moral concern in (...) a universe that we can only causally affect in one temporal direction; (2) access to the benefits from using spacetime as a resource; and (3) the burdens of care given (1) and (2). We conclude by considering the practical implications of our argument and find that, while it is often assumed that a preference for present over future well-being weakens the case for existential risk mitigation, this likely is not the case. (shrink)

Critical race theorists and standpoint epistemologists argue that agents who are members of dominant social groups are often in a state of ignorance about the extent of their social dominance, where this ignorance is explained by these agents' membership in a socially dominant group (e.g., Mills 2007). To illustrate this claim bluntly, it is argued: 1) that many white men do not know the extent of their social dominance, 2) that they remain ignorant as to the extent of their dominant (...) social position even where this information is freely attainable, and 3) that this ignorance is due in part to the fact that they are white men. We argue that on Buchak's (2010, 2013) model of risk averse instrumental rationality, ignorance of one's privileges can be rational. This argument yields a new account of elite-group ignorance, why it may occur, and how it might be alleviated. (shrink)

A moderately risk averse person may turn down a 50/50 gamble that either results in her winning $200 or losing $100. Such behaviour seems rational if, for instance, the pain of losing $100 is felt more strongly than the joy of winning $200. The aim of this paper is to examine an influential argument that some have interpreted as showing that such moderate risk aversion is irrational. After presenting an axiomatic argument that I take to be the strongest case for (...) the claim that moderate risk aversion is irrational, I show that it essentially depends on an assumption that those who think that risk aversion can be rational should be skeptical of. Hence, I conclude that risk aversion need not be irrational. (shrink)

In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart’s Law as “Virtually any goal specification, pursued to the extreme, will result in the extinction of humanity”, and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart’s Law holds or not. As our key contribution, we identify a set of conditions that are necessary for a model that aims to (...) be informative for evaluating specific arguments for Extinction-level Goodhart’s Law. Since each of the conditions seems to significantly contribute to the complexity of the resulting model, formally evaluating the hypothesis might be exceedingly difficult. This raises the possibility that whether the risk of extinction from artificial intelligence is real or not, the underlying dynamics might be invisible to current scientific methods. (shrink)

A small but growing number of studies have aimed to understand, assess and reduce existential risks, or risks that threaten the continued existence of mankind. However, most attention has been focused on known and tangible risks. This paper proposes a heuristic for reducing the risk of black swan extinction events. These events are, as the name suggests, stochastic and unforeseen when they happen. Decision theory based on a fixed model of possible outcomes cannot properly deal with this (...) kind of event. Neither can probabilistic risk analysis. This paper will argue that the approach that is referred to as engineering safety could be applied to reducing the risk from black swan extinction events. It will also propose a conceptual sketch of how such a strategy may be implemented: isolated, self-sufficient, and continuously manned underground refuges. Some characteristics of such refuges are also described, in particular the psychosocial aspects. Furthermore, it is argued that this implementation of the engineering safety strategy safety barriers would be effective and plausible and could reduce the risk of an extinction event in a wide range of possible scenarios. Considering the staggering opportunity cost of an existential catastrophe, such strategies ought to be explored more vigorously. (shrink)

An axiom of medical research ethics is that a protocol is moral only if it has a “favorable risk-benefit ratio”. This axiom is usually interpreted in the following way: a medical research protocol is moral only if it has a positive expected value -- that is, if it is likely to do more good (to both subjects and society) than harm. I argue that, thus interpreted, the axiom has two problems. First, it is unusable, because it requires us to know (...) more about the potential outcomes of research than we ever could. Second, it is false, because it conflicts with the so-called “soft paternalist” principles of liberal democracy. In place of this flawed rule I propose a new way of making risk-benefit assessments, one that does comport with the principles of liberalism. I argue that a protocol is moral only if it would be entered into by competent subjects who are informed about the protocol. The new rule this eschews all pseudo-utilitarian calculation about the protocol’s likely harms and benefits. (shrink)

In this paper, I examine the decision-theoretic status of risk attitudes. I start by providing evidence showing that the risk attitude concepts do not play a major role in the axiomatic analysis of the classic models of decision-making under risk. This can be interpreted as reflecting the neutrality of these models between the possible risk attitudes. My central claim, however, is that such neutrality needs to be qualified and the axiomatic relevance of risk attitudes needs to be re-evaluated accordingly. Specifically, (...) I highlight the importance of the conditional variation and the strengthening of risk attitudes, and I explain why they establish the axiomatic significance of the risk attitude concepts. I also present several questions for future research regarding the strengthening of risk attitudes. (shrink)

There has been a movement aiming to teach agents about their privilege by making the information about their privilege as costless as possible. However, some argue that in risk-sensitive frameworks, such as Lara Buchak’s (2013), it can be rational for privileged agents to shield themselves from learning about their privilege, even if the information is costless and relevant. This threatens the efficacy of these information-access efforts in alleviating the problem of elite-group ignorance. In response, I show that even within the (...) same framework, in this case David Kinney and Liam Kofi Bright’s (2021), the rationality of this information avoidance rests on shaky ground in practice. In this framework, whether an agent should avoid information depends on the precise details of (1) how relevant they expect the information to be, (2) their priors about the value of various options, and (3) their risk attitudes. The model suggests that the rationality of elite-group ignorance is a function of structural factors that are pervasive but nonetheless not insurmountable, thus offering a way out of pessimism about elite group education. (shrink)

Abstract: This research paper presents a novel approach to predict audit risks using a neural network model. The dataset used for this study was obtained from Kaggle and comprises 774 samples with 18 features, including Sector_score, PARA_A, SCORE_A, PARA_B, SCORE_B, TOTAL, numbers, marks, Money_Value, District, Loss, Loss_SCORE, History, History_score, score, and Risk. The proposed neural network architecture consists of three layers, including one input layer, one hidden layer, and one output layer. The neural network model was trained and validated, (...) achieving an impressive accuracy of 100% and an average error of 0.000015, indicating its robust predictive capability. Moreover, we conducted feature importance analysis to identify the most influential features for predicting audit risk. The key features found to be critical for classifying fraudulent activities in audit risk prediction are Sector_score, PARA_A, SCORE_A, PARA_B, SCORE_B, TOTAL, numbers, marks, Money_Value, District, Loss, Loss_SCORE, History, and History_score. This research contributes to the field of audit risk prediction by demonstrating the effectiveness of a neural network-based approach and highlighting the importance of specific features in detecting fraudulent activities. The findings have significant implications for auditors and organizations seeking to enhance their audit risk assessment processes, ultimately leading to improved financial transparency and fraud detection. (shrink)

To clarify the proper role of values in science, focusing on controversial expert responses to Covid-19, this article examines the status of (in)convenient hypotheses. Polarizing cases like health experts downplaying mask efficacy to save resources for healthcare workers, or scientists dismissing “accidental lab leak” hypotheses in view of potential xenophobia, plausibly involve modifying evidential standards for (in)convenient claims. Societies could accept that scientists handle (in)convenient claims just like nonscientists, and give experts less political power. Or societies could hold scientists to (...) a higher bar, by expecting them not to modify evidential standards to avoid costs only incidentally tied to error. (shrink)

Can it be rational to be risk-averse? It seems plausible that the answer is yes—that normative decision theory should accommodate risk aversion. But there is a seemingly compelling class of arguments against our most promising methods of doing so. These long-run arguments point out that, in practice, each decision an agent makes is just one in a very long sequence of such decisions. Given this form of dynamic choice situation, and the (Strong) Law of Large Numbers, they conclude that those (...) theories which accommodate risk aversion end up delivering the same verdicts as risk-neutral theories in nearly all practical cases. If so, why not just accept a simpler, risk-neutral theory? The resulting practical verdicts seem to be much the same. In this paper, I show that these arguments do not in fact condemn those risk-aversion-accommodating theories. Risk aversion can indeed survive the long run. (shrink)

The guide explains the basic concepts of natural, social and evolutionary components and methods of risk management and control of modern biotechnologies stemming from the general theory of human evolution. A a transdisciplinary approach is a feature of the presentation of the material The risk is considered from the point of view of evolutionary anthropology, as a basic element of a stable evolutionary strategy of our biological species and the basis of a combination of natural-scientific and humanitarian forms of presentation (...) and interpretation of the material. CONTENT BIOSAFETY AND BIOETHICS IN BIOTECHNOLOGY (Introduction). Part 1. BIOTECHNOLOGY AS A FACTOR OF EVOLUTION AND A SOURCE OF TECHNOGENIC RISK. Chapter 1. GENERAL THEORY OF TECHNOLOGICAL RISK. Chapter 2. RISK AS AN ATTRIBUTE OF STABLE HUMAN EVOLUTIONARY STRATEGY. Chapter 3. BIOTECHNOLOGY AS A SOURCE OF BIORISKS Chapter 4. COMPONENTS OF THE "BIOLOGICAL FACTOR" OF RISK. Chapter 5. BIOLOGICAL WEAPONS and BIOTERRORISM. Part 2. BIOLOGICAL, SOCIAL AND EVOLUTIONARY COMPONENTS OF THE RISK OF MODERN BIOTECHNOLOGIES Chapter 6. RISK MANAGEMENT DURING BIOTECHNOLOGICAL RESEARCH.. Chapter 7. BIOTECHNOLOGY AS A SOURCE OF SOCIAL INSTABILITY. Chapter 8. BIOTECHNOLOGY AS A SOURCE OF CIVILIZATIONAL RISK. -/- . (shrink)

Rowe has recently argued that pure risk of harm cannot itself be a harm. I respond to Rowe and argue that given an appropriate understanding of objective probabilities, pure objective risk of harm can itself be a harm.

Under what conditions does machine learning (ML) model opacity inhibit the possibility of explaining and understanding phenomena? In this article, I argue that nonepistemic values give shape to the ML opacity problem even if we keep researcher interests fixed. Treating ML models as an instance of doing model-based science to explain and understand phenomena reveals that there is (i) an external opacity problem, where the presence of inductive risk imposes higher standards on externally validating models, and (ii) an internal opacity (...) problem, where greater inductive risk demands a higher level of transparency regarding the inferences the model makes. (shrink)

Stable adaptive strategy of Homo sapiens (SASH) is a result of the integration in the three-module fractal adaptations based on three independent processes of generation, replication, and the implementation of adaptations — genetic, socio-cultural and symbolic ones. The evolutionary landscape SASH is a topos of several evolutionary multi-dimensional vectors: 1) extraversional projective-activity behavioral intention (adaptive inversion 1), 2) mimesis (socio-cultural inheritance), 3) social (Machiavellian) intelligence, 4) the extension of inter-individual communication beyond their own social groups and their own species in (...) the rest of the world, 5) the symbolic system of communication (symbolic inheritance), 6) spiritualistic trans- formation of emotionally-shaped components of mentality, 7) the dominance of the rationalist thought mentality (enhancer of adaptive inverse 1), 8) a recursive distribution of projective-activity intentions on the man himself his genome, psyche and culture (Adaptive Inversion 2), 9) introversional reorientation of the vector of cognitive activity (adaptive inversion 3). (shrink)

Human civilisation faces a range of existential risks, including nuclear war, runaway climate change and superintelligent artificial intelligence run amok. As we show here with calculations for the New Zealand setting, large numbers of currently living and, especially, future people are potentially threatened by existential risks. A just process for resource allocation demands that we consider future generations but also account for solidarity with the present. Here we consider the various ethical and policy issues involved and make a (...) case for further engagement with the New Zealand public to determine societal values towards future lives and their protection. (shrink)