This paper aims to scrutinize how the notion of risk should be understood and applied to possibly catastrophic cases. I begin with clarifying the standard usage of the notion of risk, particularly emphasizing the conceptual relation between risk and probability. Then, I investigate how to make decisions in the case of seemingly catastrophic disasters by contrasting the precautionary principle with the preventive (prevention) principle. Finally, I examine what kind of causal thinking tends to be actually adopted when (...) we make decisions on emergent cases. My arguments are mainly based on Japan's 2011 massive earthquake and nuclear power station accident. Masaki Ichinose. (shrink)

In a recent paper in this journal, Carter and Peterson raise two distinctly epistemological puzzles that arise for anyone aspiring to defend the precautionary principle. The first puzzle trades on an application of epistemic contextualism to the precautionary principle; the second puzzle concerns the compatibility of the precautionary principle with the de minimis rule. In this note, I argue that neither puzzle should worry defenders of the precautionary principle. The first puzzle can be shown to be (...) an instance of the familiar but conceptually harmless challenge of adjudicating between relevant interests to reach assessments of threats when applying the precautionary principle. The second puzzle can be shown to rely on a subtle but crucial misrepresentation of the relevant probabilities at play when applying the precautionary principle. (shrink)

A challenge faced by defenders of the precautionary principle is to clarify when the evidence that a harmful event might occur suffices to regard this prospect as a real possibility. Plausible versions of the principle must articulate some epistemic threshold, or de minimis requirement, which specifies when precautionary measures are justified. Critics have argued that formulating such a threshold is problematic in the context of the precautionary principle. First, this is because the precautionary principle appears to (...) be ambiguous about the distinction between risk and uncertainty: should the principle merely be invoked when evidential probabilities are absent, or also when probabilities have low epistemic credentials? Secondly, defenders of the precautionary principle face an aggregation puzzle: in judging whether or not the de minimis requirement has been met, how should first-order evidential probabilities and their second-order epistemic standing be aggregated? This article argues that the ambiguity can be resolved, and the epistemological puzzle can be solved. Focusing on decisions in the context of climate uncertainty, I advance a version of the precautionary principle that serves as a plausible decision rule, to be adopted in situations where its main alternative—cost–benefit analysis—does not deliver. (shrink)

The precautionary principle is invoked in a number of important personal and policy decision contexts. Peterson shows that certain ways of making the principle precise are inconsistent with other criteria of decision-making. Some object that the results do not apply to cases of deep uncertainty or value incommensurability which are alleged to be in the principle’s wheelhouse. First, I show that Peterson’s impossibility results can be generalized considerably to cover cases of both deep uncertainty and incommensurability. Second, I contrast (...) an alternative way of giving voice to the precautionary impulse. (shrink)

Risk communication has been generally categorized as a warning act, which is performed in order to prevent or minimize risk. On the other side, risk analysis has also underscored the role played by information in reducing uncertainty about risk. In both approaches the safety aspects related to the protection of the right to health are on focus. However, it seems that there are cases where a risk cannot possibly be avoided or uncertainty reduced, this is for instance valid for the (...) declaration of side effects associated with pharmaceutical products or when a decision about drug approval or retirement must be delivered on the available evidence. In these cases, risk communication seems to accomplish other tasks than preventing risk or reducing uncertainty. The present paper analyzes the legal instruments which have been developed in order to control and manage the risks related to drugs – such as the notion of “development risk” or “residual risk” – and relates them to different kinds of uncertainty. These are conceptualized as epistemic, ecological, metric, ethical, and stochastic, depending on their nature. By referring to this taxonomy, different functions of pharmaceutical risk communication are identified and connected with the legal tools of uncertainty management. The purpose is to distinguish the different functions of risk communication and make explicit their different legal nature and implications. (shrink)

The precautionary principle in public decision making concerns situations where following an assessment of the available scientific information, there are reasonable grounds for concern for the possibility of adverse effects on the environment or human health, but scientific uncertainty persists. In such cases provisional risk management measures may be adopted, without having to wait until the reality and seriousness of those adverse effects become fully apparent. This is the definition of the precautionary principle as operationalized under EU law. (...) The precautionary principle is a deliberative principle. Its application involves deliberation on a range of normative dimensions which need to be taken into account while making the principle operational in the public policy context. Under EU law, any risk management measures to be adopted while implementing the precautionary principle, have to be proportionate to ensure the chosen high level of protection in the European Community. This articlewill illustrate the established practice concerning the release of genetically modified organisms into the environment and how the principle is implemented under hard law. The article also provides an outlook on what this may imply for the relative new case of nanotechnology and the use of precautionary principle within the context of soft law (use of codes of conduct). (shrink)

The precautionary principle (PP) aims to anticipate and minimize potentially serious or irreversible risks under conditions of scientific uncertainty. Thus it preserves the potential for future developments. It has been incorporated into many international treaties and pieces of national legislation for environmental protection and sustainable development. In this article, we outline an interpretation of the PP as a framework of orientation for a sustainable information society. Since the risks induced by future information and communication technologies (ICT) are social risks (...) for the most part, we propose to extend the PP from mainly environmental to social subjects of protection. From an ethical point of view, the PP and sustainability share the principle of intergenerational justice, which can be used as an argument to preserve free space for the decisions of future generations. Applied to technical innovation and to ICT issues in particular, the extended PP can serve as a framework of orientation to avoid socio-economically irreversible developments. We conclude that the PP is a useful approach for: (i) policy makers to reconcile information society and sustainability policies and (ii) ICT companies to formulate sustainability strategies. (shrink)

We begin by showing that every theory of the value of uncertain prospects must have one of three unpalatable properties. _Reckless_ theories recommend giving up a sure thing, no matter how good, for an arbitrarily tiny chance of enormous gain; _timid_ theories permit passing up an arbitrarily large potential gain to prevent a tiny increase in risk; _non-transitive_ theories deny the principle that, if A is better than B and B is better than C, then A must be better than (...) C. Having set up this trilemma, we study its horns. Non-transitivity has been much discussed; we focus on drawing out the costs and benefits of recklessness and timidity when it comes to axiology, decision theory, and normative uncertainty. (shrink)

The Precautionary Principle (PP) is an influential principle of risk management. It has been widely introduced into environmental legislation, and it plays an important role in most international environmental agreements. Yet, there is little consensus on precisely how to understand and formulate the principle. In this paper I prove some impossibility results for two plausible formulations of the PP as a decision-rule. These results illustrate the difficulty in making the PP consistent with the acceptance of any trade-offs between catastrophic (...) risks and more ordinary goods. (shrink)

Bartha and DesRoches (2021) and Steel and Bartha (2023) argue that we should understand the precautionary principle as the injunction to maximise lexical utilities. They show that the lexical utility model has important pragmatic advantages. Moreover, the model has the theoretical advantage of satisfying all axioms of expected utility theory except continuity. In this paper I raise a trilemma for any attempt at modelling the precautionary principle with lexical utilities: it permits choice cycles or leads to paralysis or (...) implies that the smallest value difference that is possible in a context has extreme axiological implications. (shrink)

In this paper we present two distinctly epistemological puzzles that arise for one who aspires to defend some plausible version of the precautionary principle. The first puzzle involves an application of contextualism in epistemology; and the second puzzle concerns the task of defending a plausible version of the precautionary principle that would not be invalidated by de minimis.

In a recent paper in this journal, we proposed two novel puzzles associated with the precautionary principle. Both are puzzles that materialise, we argue, once we investigate the principle through an epistemological lens, and each constitutes a philosophical hurdle for any proponent of a plausible version of the precautionary principle. Steglich-Petersen claims, also in this journal, that he has resolved our puzzles. In this short note, we explain why we remain skeptical.

This Article aims to demonstrate that the WTO jurisprudence on science-related trade disputes has become imbued with a specific vision of science that has prevented any possible application of the precautionary principle. This situation is due both to the WTO’s specific dispute settlement procedures and to the substantive nature of precautionary measures. Indeed, such measures’ foundation on “insufficient scientific evidence” dramatically undermines the probative value of science in WTO adjudication and creates a seeming contradiction: The system requires defendants (...) to provide legal evidence of the absence of sufficient scientific evidence. The reasoning of the Panel on the EC–Biotech case was riddled with this apparent paradox. For the first time, the US–Continued Suspension case has opened a gateway to address this fundamental issue. (shrink)

The dissemination of information in times of crisis or emergency is distinctive since the affected individuals may take, process, and act on information differently. As the Centers for Disease Control and Prevention noted “the right message at the right time from the right person can save lives.” This study elaborates on the principles of crisis and emergency risk communication (CERC) in the realistic narrative, and notes that a successful CERC should be prompt, accurate, veracious, empathetic, respectful, and promote meaningful (...) action. Since pandemics are not only biomedical but also social, political, economic, and cultural incidents, critical communication in the time of a pandemic is of crucial significance. Hence, the Coronavirus (COVID-19) pandemic is assessed as a recent and important case study. This pandemic had four chief challenges; including that since the disease is caused by the novel SARS-CoV-2 virus the assumptions about the disease have been revised numerously and rapidly. In addition, the cumulative scale and scope of COVID-19 which is in excess of 627 million confirmed cases, and approximately 6.58 million deaths worldwide remained a strain. Another concern was the overwhelming uncertainty and evolving trajectory of the pandemic about how and when it ends or enters the endemic stage of the outbreak. Finally, the overabundance of amplification and dissemination of both accurate and inaccurate information led to an infodemic. Long before the COVID-19 pandemic, many individuals used the internet as an important resource for seeking health-related information. However, the infodemic and the other aforementioned challenges made distinguishing up-to-date accurate information a more arduous task, and CERC and pandemic-related communication are even more crucial. Hence, a multi-dimensional set of skills is required to overcome these challenges whether amidst or after the pandemic. Therefore, a multi-stakeholder approach towards an overarching and comprehensive metaliteracy that “provides an integrated and inclusive framework to cultivate critical thinking and proficiency in comprehending data, information, visuals and the media” is essential. This study recommends and elaborates on how to incorporate metaliteracy into crisis and risk communication at governmental, organizational, and individual levels. (shrink)

Lockdowns, or modern quarantines, involve the use of novel restrictive non-pharmaceutical interventions (NPIs) to suppress the transmission of COVID-19. In this paper, I aim to critically analyze the emerging history and philosophy of lockdowns, with an emphasis on the communication of health evidence and risk for informing policy decisions. I draw a distinction between evidence-based and modeling-based decision-making. I argue that using the normative framework of evidence-based medicine would have recommended against the use of lockdowns. I first review the World (...) Health Organization’s evidence-based pandemic preparedness plans for respiratory viruses. I then provide a very brief history of COVID-19 modeling, which was cited as justification for the use of lockdowns in the U.K., the U.S., and much of the world. I focus on the so-called Imperial College model designed by Neil Ferguson et al. as well as the so-called Oxford model designed by José Lourenço et al. I analyze the evidence-based pandemic response known as ‘mitigation’, and I compare it with Ferguson et al.’s experimental strategy known as ‘suppression’. I summarize the strengths and weaknesses of these strategies based on their diametric aims and each model’s parametric assumptions. Based on my critical analysis of the suppression strategy, I attempt to expose what has been called the ‘logic of lockdowns’, which Sunetra Gupta of the Oxford model group has suggested is flawed. Finally, I consider Trisha Greenhalgh’s objection to evidence-based policy based on the precautionary principle, and I attempt to offer a response. I conclude with a brief narrative review of the emerging randomized evidence on restrictive NPIs, which seems to support my claim that mitigation was the strategy that would have been recommended by evidence-based medicine. If this is true, then COVID-19 modeling may serve as an important reminder of the enduring lesson of evidence-based medicine: that one should always ‘Trust the Evidence!’ for better health policy. (shrink)

Preston Greene (2020) argues that we should not conduct simulation investigations because of the risk that we might be terminated if our world is a simulation designed to research various counterfactuals about the world of the simulators. In response, we propose a sequence of arguments, most of which have the form of an "even if” response to anyone unmoved by our previous arguments. It runs thus: (i) if simulation is possible, then simulators are as likely to care about simulating simulations (...) as they are likely to care about simulating basement (i.e. non-simulated) worlds. But (ii) even if simulations are interested only in simulating basement worlds the discovery that we are in a simulation will have little or no impact on the evolution of ordinary events. But (iii) even if discovering that we are in a simulation impacts the evolution of ordinary events, the effects of seeming to do so could also happen in a basement world, and might be the subject of interesting counterfactuals in the basement world Finally, (iv) there is little reason to think that there is a catastrophic effect from successful simulation probes, and no argument from the precautionary principle can be used to leverage the negligible credence one ought have in this. Thus, if we do develop a simulation probe, then let’s do it. (shrink)

My paper builds on the conceptual tools from three interrelated philosophical debates that—as I believe—may help structure important if chaotic discussions about surrogates for living human brains and resolve some practical issues related to regulatory matters. In particular, I refer to the discussions about the “moral precautionary principle” in research ethics (Koplin and Wilkinson 2019); about normative uncertainty in ethics (MacAskill, Bykvist, and Ord 2020), and about the inductive risk problem for animal welfare scientists (Birch 2018). I elucidate upon (...) the possible meanings of the phrase “a too good human brain surrogate” used by Henry T. Greely (2021), and I demonstrate that the evaluation of the practical and regulatory implications of the “goodness” of such surrogates created for research purposes should be sensitive to the possible consequences of two types of errors: the under-attribution and over-attribution of moral status to such beings. Many authors writing about this topic (including Greely 2021, but see also, e.g., Koplin and Savulescu 2019) concentrate only on the first type of error, neglecting the negative consequences of the second type, i.e., over-attribution. (shrink)

Utility discounting in intertemporal economic modelling has been viewed as problematic, both for descriptive and normative reasons. However, positive utility discount rates can be defended normatively; in particular, it is rational for future utility to be discounted to take into account model-independent outcomes when decision-making under risk. The resultant values will tend to be smaller than descriptive rates under most probability assignments. This also allows us to address some objections that intertemporal considerations will be overdemanding. A principle for utility discount (...) rates is suggested which is rooted in probability discounting. Utility discounting is defended against objections from Parfit (1984) and Broome (2005); Broome (2012). A sample utility discount rate is estimated. [Open access]. (shrink)

Existential risks threaten the future of humanity, but they are difficult to measure. However, to communicate, prioritize and mitigate such risks it is important to estimate their relative significance. Risk probabilities are typically used, but for existential risks they are problematic due to ambiguity, and because quantitative probabilities do not represent some aspects of these risks. Thus, a standardized and easily comprehensible instrument is called for, to communicate dangers from various global catastrophic and existential risks. In this article, inspired by (...) the Torino scale of asteroid danger, we suggest a color coded scale to communicate the magnitude of global catastrophic and existential risks. The scale is based on the probability intervals of risks in the next century if they are available. The risks’ estimations could be adjusted based on their severities and other factors. The scale covers not only existential risks, but smaller size global catastrophic risks. It consists of six color levels, which correspond to previously suggested levels of prevention activity. We estimate artificial intelligence risks as “red”, while “orange” risks include nanotechnology, synthetic biology, full scale nuclear war and a large global agricultural shortfall (caused by regional nuclear war, coincident extreme weather, etc.) The risks of natural pandemic, supervolcanic eruption and global warming are marked as “yellow” and the danger from asteroids is “green”. -/- Keywords: global catastrophic risks; existential risks; Torino scale; policy; risk probability . (shrink)

When faced with an urgent and credible threat of grave harm, we should take proportionate precautions. But what is it for a precaution to be “proportionate”? I construct a pragmatic analysis of consisting of four tests—permissibility-in-principle, adequacy, reasonable necessity and consistency—that could realistically be applied by a citizens’ assembly meeting online or in person. I apply these tests retrospectively to two examples from the COVID-19 pandemic—border closures and school closures—arguing that my account captures the key questions on which it is (...) both feasible and important to integrate expert input with democratic input. I then consider how we might try to manage the risk of future pandemics in a proportionate way. (shrink)

Translating risk estimates derived from epidemiologic study into evidence of causality for a particular patient is problematic. The difficulty of this process is not unique to the medical context; rather, courts are also challenged with the task of using risk estimates to infer evidence of cause in particular cases. Thus, an examination of how this is done in a legal context might provide insight into when and how it is appropriate to use risk information as evidence of cause in a (...) medical context. A careful study of the case of Goodman v. Viljoen, a medical malpractice suit litigated in the Ontario Superior Court of Justice in 2011, reveals different approaches to how risk information is used as or might be considered a substitute for evidence of causation, and the pitfalls associated with these approaches. Achieving statistical thresholds, specifically minimizing the probability of falsely rejecting the null hypothesis, and exceeding a relative risk of 2, plays a significant role in establishing causality of the particular in the legal setting. However, providing a reasonable explanation or establishing “biological plausibility” of the causal association also seems important, and (to some) may even take precedent over statistical thresholds for a given context. (shrink)

Within the causal modeling literature, debates about the Causal Faithfulness Condition have concerned whether it is probable that the parameters in causal models will have values such that distinct causal paths will cancel. As the parameters in a model are fixed by the probability distribution over its variables, it is initially puzzling what it means to assign probabilities to these parameters. I propose that to assign a probability to a parameter in a model is to treat (...) that parameter as a function of a variable in an augmented model. By combining this proposal with widely adopted principles regarding which variables must be included in a model, I argue that the various proposed counterexamples to CFC involving coordinated parameters are not genuine counterexamples. I then consider the cases in which CFC fails due not to coordination, but by coincidence, and propose explanatory and predictive bases for ruling out such coincidences without presuming that they are improbable. The aim of the proposed defenses is not to show that CFC never fails, but rather to argue that its use in a particular context may be defended using general modeling assumptions rather than by relying on claims about how often it fails. (shrink)

Many global catastrophic and existential risks (X-risks) threaten the existence of humankind. There are also many ideas for their prevention, but the meta-problem is that these ideas are not structured. This lack of structure means it is not easy to choose the right plan(s) or to implement them in the correct order. I suggest using a “Plan A, Plan B” model, which has shown its effectiveness in planning actions in unpredictable environments. In this approach, Plan B is a backup option, (...) implemented if Plan A fails. In the case of global risks, Plan A is intended to prevent a catastrophe and Plan B to survive it, if it is not avoided. Each plan has similar stages: analysis, planning, funding, low-level realization and high-level realization. Two variables—plans and stages—provide an effective basis for classification of all possible X-risks prevention methods in the form of a two-dimensional map, allowing the choice of optimal paths for human survival. I have created a framework for estimating the utility of various prevention methods based on their probability of success, the chances that they will be realized in time, their opportunity cost, and their risk. I also distinguish between top-down and bottom-up approaches. (shrink)

According to Agnosticism with a capital A, even if we don’t see how any reason we know of would justify God in permitting all the evil in the world and even if we lack evidential and non-evidential warrant for theism, we should not infer that there probably is no reason that would justify God. That’s because, under those conditions, we should be in doubt about whether the goods we know of constitute a representative sample of all the goods there are, (...) among relevantly similar things. In my "Epistemic Humility, Arguments from Evil, and Moral Skepticism" (2009), I defended Agnosticism against the charge that it leaves us in doubt about whether we are obligated to intervene to prevent horrific suffering we can prevent at no risk to ourselves. In his “Agnosticism, Skeptical Theism, and Moral Obligation” (2014), Stephen Maitzen argues that, in light of my defense, Agnosticism is at odds with commonsense morality’s insistence that we have an obligation to intervene in such cases. In the present essay, I argue that the moral principle Maitzen seems to impute to commonsense (it's hard to tell what principle he has in mind) is false and that a moral principle much more in keeping with commonsense is compatible with Agnosticism and my defense of it. Along the way, I mention multiple misrepresentations Maitzen makes of Agnosticism and my defense of it. (shrink)

There is widespread belief in a tension between quantum theory and special relativity, motivated by the idea that quantum theory violates J. S. Bell’s criterion of local causality, which is meant to implement the causal structure of relativistic space-time. This paper argues that if one takes the essential intuitive idea behind local causality to be that probabilities in a locally causal theory depend only on what occurs in the backward light cone and if one regards objective probability as (...) what imposes constraints on rational credence along the lines of David Lewis’ Principal Principle, then one arrives at the view that whether or not Bell’s criterion holds is irrelevant for whether or not local causality holds. The assumptions on which this argument rests are highlighted, and those that may seem controversial are motivated. (shrink)

The Covid-19 pandemic raises the need for an ethical framework addressing unique questions of airborne infectious disease. In particular, are you ethically obliged to wear a face mask? If so, why and when? The Respiratory Ethics Framework (REF) herein derives answers from ethical norms. Always covering coughs and sneezes just in case you might be infectious is an ethical norm. But if you are infectious with an airborne illness, you are probably spreading germs even with every breath and vocalization. Therefore, (...) given that respiratory covering should be done to prevent spreading germs, you should wear a mask if you believe you are infectious because it also provides covering while breathing and vocalizing. REF is grounded in the non-harm principle. Why should we avoid spreading infections to others? Because they can cause bodily physical harm and we should act to prevent causing such harm to others without justifiable reason. Under REF, the magnitude of your obligation to mask is proportional to the magnitude of risk you pose to others by way of airborne germs. This functional relationship is logically modeled and visually mapped with a decision matrix in this essay. (shrink)

What proper role should considerations of risk, particularly to research subjects, play when it comes to conducting research on human enhancement in the military context? We introduce the currently visible military enhancement techniques (1) and the standard discussion of risk for these (2), in particular what we refer to as the ‘Assumption’, which states that the demands for risk-avoidance are higher for enhancement than for therapy. We challenge the Assumption through the introduction of three categories of enhancements (3): therapeutic, (...) class='Hi'>preventive, and pure enhancements. This demands a revision of the Assumption (4), alongside which we propose some further general principles bearing on how to balance risks and benefits in the context of military enhancement research. We identify a particular type of therapeutic enhancements as providing a more responsible path to human trials of the relevant interventions than pure enhancement applications. Finally, we discuss some possible objections to our line of thought (5). While acknowledging their potential insights, we ultimately find them to be unpersuasive, at least provided that our proposal is understood as fully non-coercive towards the candidates for such therapeutic enhancement trials. (shrink)

What proper role should considerations of risk, particularly to research subjects, play when it comes to conducting research on human enhancement in the military context? We introduce the currently visible military enhancement techniques and the standard discussion of risk for these, in particular what we refer to as the ‘Assumption’, which states that the demands for risk-avoidance are higher for enhancement than for therapy. We challenge the Assumption through the introduction of three categories of enhancements : therapeutic, preventive, and (...) pure enhancements. This demands a revision of the Assumption, alongside which we propose some further general principles bearing on how to balance risks and benefits in the context of military enhancement research. We identify a particular type of therapeutic enhancements as providing a more responsible path to human trials of the relevant interventions than pure enhancement applications. Finally, we discuss some possible objections to our line of thought. While acknowledging their potential insights, we ultimately find them to be unpersuasive, at least provided that our proposal is understood as fully non-coercive towards the candidates for such therapeutic enhancement trials. (shrink)

With the fast pace of AI development, the problem of preventing its global catastrophic risks arises. However, no satisfactory solution has been found. From several possibilities, the confinement of AI in a box is considered as a low-quality possible solution for AI safety. However, some treacherous AIs can be stopped by effective confinement if it is used as an additional measure. Here, we proposed an idealized model of the best possible confinement by aggregating all known ideas in the field of (...) AI boxing. We model the confinement based on the principles used in the safety engineering of nuclear power plants. We show that AI confinement should be implemented in several levels of defense. These levels include 1) AI design in fail-safe manner 2) limiting its capabilities, preventing self-improving and circuit breakers on treacherous turn 3) isolation from the outside world and, lastly, as the final hope 4) outside measures oriented on stopping AI in the wild. We demonstrate that the substantial number (more than 50 ideas listed in the article) of mutually independent measures could provide a relatively high probability of the containment of a human-level AI but may be not sufficient to preserve runaway of superintelligent AI. Thus, these measures will work only if they are used to prevent superintelligent AI creation, but not for containing superintelligence. We suggest that there could be a safe operation threshold, on which AI is useful, but is not able to hack containment system from the inside, the same way as a safe level of chain reaction inside nuclear power plants is maintained. However, overall, a failure of the confinement is inevitable, so we need to use the full AGI limited number of times (AI-ticks). (shrink)

Since the release of ChatGPT, there has been a lot of debate about whether AI systems pose an existential risk to humanity. This paper develops a general framework for thinking about the existential risk of AI systems. We analyze a two-premise argument that AI systems pose a threat to humanity. Premise one: AI systems will become extremely powerful. Premise two: if AI systems become extremely powerful, they will destroy humanity. We use these two premises to construct a taxonomy of ‘survival (...) stories’, in which humanity survives into the far future. In each survival story, one of the two premises fails. Either scientific barriers prevent AI systems from becoming extremely powerful; or humanity bans research into AI systems, thereby preventing them from becoming extremely powerful; or extremely powerful AI systems do not destroy humanity, because their goals prevent them from doing so; or extremely powerful AI systems do not destroy humanity, because we can reliably detect and disable systems that have the goal of doing so. We argue that different survival stories face different challenges. We also argue that different survival stories motivate different responses to the threats from AI. Finally, we use our taxonomy to produce rough estimates of ‘P(doom)’, the probability that humanity will be destroyed by AI. (shrink)

When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk avoidant risk function. This, in turn, has been claimed to require the use of a risk avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the significant risks associated with continued human (...) survival. I raise objections to the claim that respect for others’ risk attitudes requires risk avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others’ risk attitudes in fixed population cases. GPI Working Paper No . 20-2022. (shrink)

When probability discounting (or probability weighting), one multiplies the value of an outcome by one's subjective probability that the outcome will obtain in decision-making. The broader import of defending probability discounting is to help justify cost-benefit analyses in contexts such as climate change. This chapter defends probability discounting under risk both negatively, from arguments by Simon Caney (2008, 2009), and with a new positive argument. First, in responding to Caney, I argue that small costs and benefits need to be evaluated, (...) and that viewing practices at the social level is too coarse-grained. Second, I argue for probability discounting using a distinction between causal responsibility and moral responsibility. Moral responsibility can be cashed out in terms of blameworthiness and praiseworthiness, while causal responsibility obtains in full for any effect which is part of a causal chain linked to one's act. With this distinction in hand, unlike causal responsibility, moral responsibility can be seen as coming in degrees. My argument is, given that we can limit our deliberation and consideration to that which we are morally responsible for and that our moral responsibility for outcomes is limited by our subjective probabilities, our subjective probabilities can ground probability discounting. (shrink)

Risk assessment is one of the prerequisites for understanding its causes and possible consequences. We base our risk assessment on the principles described in the European standard EN 31000 - Risk Management Process. This standard comprehensively describes the continuous activities that are necessary in managing risks and minimizing their possible adverse effects on the operation of the system under investigation. In this activity, it is necessary to first identify the existing risks, then analyze and evaluate the identified risks. In the (...) analysis of existing risks, it is possible to use both qualitative and quantitative analytical methods, or combine them. We use qualitative methods in cases where we do not have a sufficient amount of input information, these are more subjective. Quantitative methods are more accurate, but also more demanding on input information and time. The choice of a suitable analytical method is a basic prerequisite for knowledge of risks and their evaluation. The values of individual risks obtained in this way are the basis for determining the measures that are ecessary to minimize them, i.e., to adjust them to an acceptable level. The draft measures are always based on the value of the individual components used to calculate the risk number, as well as on the value of the asset , which needs to be protected. Appropriately chosen analytical methods are one of the basic prerequisites for the consistent application of the principles of risk management, as a continuous process aimed at increasing the overall security of the system under study. In the article, the author describes the procedures used in risk assessment, as well as specific analytical methods that can be used in working with risks. The aim of identifying risk factors is to create a list of events that could cause undesirable disruption to ongoing processes. At this stage, we define all the risks that will be subsequently analyzed and evaluated. When identifying, we can use methods such as, e.g. SWOT, PHA (Preliminary Hazard Analysis) or CA (Checklist Analysis). Methods suitable for determining the causes and creating scenarios for the course of a risk event are ETA (Event Tree Analysis) or FTA (Fault Tree Analysis). The basic analysis of the system can be performed using the FMEA method (Failure Mode and Effect Analysis), which provides a numerical risk assessment. By comparison with the numerical value of the risk that we are willing to accept, we obtain 2 groups of risks. Acceptable, which will be given regular attention and unacceptable, which we will focus on in risk management and we will try to minimize its negative affect on the functioning of the system under study. (shrink)

We examine two competing solutions to Benardete paradoxes: causal finitism, according to which nothing can have infinitely many causes, and the unsatisfiable pair diagnosis (UPD), according to which such paradoxes are logically impossible and no metaphysical thesis need be adopted to avoid them. We argue that the UPD enjoys notable theoretical advantages over causal finitism. Causal finitists, however, have levelled two main objections to the UPD. First, they urge that the UPD requires positing a ‘mysterious force’ that (...) prevents paradoxes from arising. Since such a force is implausible, the UPD is in trouble. Second, they employ recombination or patchwork principles to argue that paradoxical situations would be possible if causal finitism were false. Since such situations are not possible, causal finitism is true, and so a substantive metaphysical thesis is needed to avoid the paradoxes. We argue that the UPD proponent can successfully respond to these objections. (shrink)

Epistemic closure under known implication is the principle that knowledge of "p" and knowledge of "p implies q", together, imply knowledge of "q". This principle is intuitive, yet several putative counterexamples have been formulated against it. This paper addresses the question, why is epistemic closure both intuitive and prone to counterexamples? In particular, the paper examines whether probability theory can offer an answer to this question based on four strategies. The first probability-based strategy rests on the accumulation of risks. The (...) problem with this strategy is that risk accumulation cannot accommodate certain counterexamples to epistemic closure. The second strategy is based on the idea of evidential support, that is, a piece of evidence supports a proposition whenever it increases the probability of the proposition. This strategy makes progress and can accommodate certain putative counterexamples to closure. However, this strategy also gives rise to a number of counterintuitive results. Finally, there are two broadly probabilistic strategies, one based on the idea of resilient probability and the other on the idea of assumptions that are taken for granted. These strategies are promising but are prone to some of the shortcomings of the second strategy. All in all, I conclude that each strategy fails. Probability theory, then, is unlikely to offer the account we need. (shrink)

Abstract: Global chemical contamination is an underexplored source of global catastrophic risks that is estimated to have low a priori probability. However, events such as pollinating insects’ population decline and lowering of the human male sperm count hint at some toxic exposure accumulation and thus could be a global catastrophic risk event if not prevented by future medical advances. We identified several potentially dangerous sources of the global chemical contamination, which may happen now or could happen in the future: autocatalytic (...) reactions, exposure to multiple subthreshold sources, and long-term unintended consequences, arising from both natural and bioengineered sources. We list several especially dangerous chemicals—dioxin, organiс compounds, and toxic heavy metals. We also discuss the features of such dangerous chemicals—molecules that can stay in the biosphere for a long time and affect it over time. We explore several social processes and scenarios where global chemical contamination becomes possible: large natural catastrophe like meteorite impact, supervolcano eruption, new ways of predicting properties of the chemicals via machine learning and their manufacturing via synthetic biology, uncontrolled “capitalistic” economic development with a corresponding large waste production, quick adoption of many chemicals with unknown long-term properties and unintended side-effects. These are all low probability, so work on other global catastrophic risks should be prioritized, but chemical risks could exacerbate other types of catastrophe contributing to social collapse. (shrink)

In this paper, we argue that our moral concern for future well-being should reduce over time due to important practical considerations about how humans interact with spacetime. After surveying several of these considerations (around equality, special duties, existential contingency, and overlapping moral concern) we develop a set of core principles that can both explain their moral significance and highlight why this is inherently bound up with our relationship with spacetime. These relate to the equitable distribution of (1) moral concern in (...) a universe that we can only causally affect in one temporal direction; (2) access to the benefits from using spacetime as a resource; and (3) the burdens of care given (1) and (2). We conclude by considering the practical implications of our argument and find that, while it is often assumed that a preference for present over future well-being weakens the case for existential risk mitigation, this likely is not the case. (shrink)

The article is a plea for ethicists to regard probability as one of their most important concerns. It outlines a series of topics of central importance in ethical theory in which probability is implicated, often in a surprisingly deep way, and lists a number of open problems. Topics covered include: interpretations of probability in ethical contexts; the evaluative and normative significance of risk or uncertainty; uses and abuses of expected utility theory; veils of ignorance; Harsanyi’s aggregation theorem; population size problems; (...) equality; fairness; giving priority to the worse off; continuity; incommensurability; nonexpected utility theory; evaluative measurement; aggregation; causal and evidential decision theory; act consequentialism; rule consequentialism; and deontology. (shrink)

Powerful, technically complex international compliance regimes have developed recently in certain professions that deal with risk: banking (the Basel II regime), accountancy (IFRS) and the actuarial profession. The need to deal with major risks has acted as a strong driver of international co-operation to create enforceable international semilegal systems, as happened earlier in such fields as international health regulations. This regulation in technical fields contrasts with the failure of an international general-purpose political and legal regime to develop. We survey the (...) new global regulatory systems in the actuarial, banking and accounting fields, with a view to showing how the need to deal reasonably with risk has resulted in an international de facto law solidly based on correct abstract principles of probability. (shrink)

The ecosystem services argument is a highly publicised instrumental argument for protecting biodiversity. I develop a new objection to this argument based on the lack of a causal connection from global species losses to local ecosystem changes. I survey some alternative formulations of services arguments, including ones incorporating option value or a precautionary principle, and show that they do not fare much better than the standard version. I conclude that environmental thinkers should rely less on ecosystem services as (...) a means to defend biodiversity, and that attention should be focused on additional types of value which might be attributed to global biodiversity. (shrink)

Deontological theories face difficulties in accounting for situations involving risk; the most natural ways of extending deontological principles to such situations have unpalatable consequences. In extending ethical principles to decision under risk, theorists often assume the risk must be incorporated into the theory by means of a function from the product of probability assignments to certain values. Deontologists should reject this assumption; essentially different actions are available to the agent when she cannot know that a certain act is in her (...) power, so we cannot simply understand her choice situation as a “risk-weighted” version of choice under certainty. (shrink)

Most current techniques to deal with invasive species are ineffective or have highly damaging side effects. To this end suppression-drives based on clustered regularly inter-spaced short palindromic repeats (CRISPR/Cas9) have been touted as a potential silver bullet for the problem, allowing for a highly focused, humane and cost-effective means of removing a target species from an environment. Suppression-drives come with serious risks, however, such that the precautionary principle seems to warrant us not deploying this technology. The focus of this (...) paper is on one such risk – the danger of a suppression-drive escaping containment and wiping out the target species globally. Here, I argue that in most cases this risk is significant enough to warrant not using a gene-drive. In some cases, however, we can bypass the precautionary principle by using an approach that hinges on what I term the ‘Worst-Case Clause’. This clause, in turn, provides us with a litmus test that can be fruitfully used to determine what species are viable targets for suppression-drives in the wild. Using this metric in concert with other considerations, I suggest that only three species are currently possible viable targets – the European rabbit, ship rat and Caribbean Tree Frog. (shrink)

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk, often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent AI both contributes to (...) existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks. (shrink)

It is generally recognised that the potential positive and negative impacts of geoengineering will be distributed unevenly both geographically and temporally. The question of distributive justice in geoengineering thus is one of the major ethical issues associated with geoengineering. Currently, the question of distributive justice in geoengineering is framed in terms of who gets what (potential) benefits and harms from geoengineering, i.e. it is about the distribution of the outcomes of geoengineering. In this paper, I argue that the discussions on (...) distributive justice in geoengineering should not be outcome-based. Instead, it should be risk-based. I identify two problems for framing the question of distributive justice in geoengineering in terms of the distribution of its outcomes, i.e. the ‘if and then’ syndrome and the limited applicability of distributive principles in geoengineering policy, and suggest risk is a more proper object of distribution in the case of geoengineering. Following Hayenhjelm, I argue that the object of distribution in the case of fair distribution of risk should be (i) sources of risks and (ii) precautionary measures. I shall then demonstrate how it can be applied to the question of distributive justice in geoengineering. Finally, I end this paper by exploring the possible responses to the question of distributive justice in geoengineering by three major accounts of distributive justice, i.e. egalitarianism, prioritarianism, and sufficientarianism. (shrink)

In Western societies many immigrants live in difficult social and working conditions. Together with other factors, this state of affairs represents a risk for the well being of their children. This article will consider the principle risk factors for child psychopathology and/or distress, with a distinction between temporary and permanent factors and with a peculiar attention to the interplay between risk and protective factors. Risk factors can be ordered in cultural, social, familiar/parental and individual factors. Some of these are general (...) risk factors, applying to child and adolescent psychopathology and distress independently from the status of immigrants’ offspring. Other factors are specific of migration, some of them being related to: a) different ways of immigrated families to situate themselves within the host society ; b) cultural/familiar attitudes in child’s nurture and education; c) the family role of women as well as factors specific of the pregnancy period in immigrants; d) the ability of the school system to enhance and support children’s abilities to integrate within the new society; e) the political/bureaucratic facilitation/impediment to the regularization of VISA, with the consequent effect on the sense of identity/rejection within/from the host society. In conclusion, the programs for monitoring immigrants’ living and health conditions should also include: the assessment of parental skills, the dynamic indicators of risk and protection indexes, the assessment of living conditions and social school environment, with a careful attention to those early signs of discomfort that might precede possible later onset of psychopathology and/or social distress. (shrink)

I will argue that physicians have an ethical obligation to justify their conscientious objection and the most reliable interpretation of the Polish legal framework claims that conscientious objection is permissible only when the justification shows the genuineness of the judgment of conscience that is not based on false beliefs and arises from a moral norm that has a high rank. I will demonstrate that the dogma accepted in the Polish doctrine that the reasons that lie behind conscientious objection in medicine (...) cannot be evaluated or controlled by anyone is based either on a mistaken interpretation of the Constitution or on the unreliable concept of conscience. I will refer to the legal regulations concerning military refusals that require from objectors to reveal and justify their views. Finally, I will demonstrate why conscientious objection under uncertainty does not deserve acceptance, because it is based on a specific version of the precautionary principle. (shrink)

When does a model explain? When does it promote understanding? A dominant approach to scientific explanation is the interventionist view. According to this view, when X explains Y, intervening on X can produce, prevent or alter Y in some predictable way. In this paper, I argue for two claims. First, I reject a position that many interventionist theorists endorse. This position is that to explain some phenomenon by providing a model is also to understand that phenomenon. While endorsing the interventionist (...) view, I argue that explaining and understanding are distinct scientific achievements. Second, I defend a novel theory of scientific understanding. According to this view, when some model M promotes understanding, M makes available a distinctive mental state. This state is of the same psychological kind as when we grasp events in a narrative as bearing on some ultimate conclusion. To conclude, I show that, given this view, mechanistic explanations often provide a powerful source of understanding that many causal-historical models lack. This paper will be of interest to both philosophers of science and epistemologists engaged in the topics of explanation and understanding. (shrink)

Some experts have argued that patients should routinely be told the specific magnitude and absolute probability of potential risks and benefits of screening tests. This position is motivated by the idea that framing risk information in ways that are less precise violates the ethical principle of respect for autonomy and its application in informed consent or shared decisionmaking. In this Perspective, we consider a number of problems with this view that have not been adequately addressed. The most important challenges stem (...) from the danger that patients will misunderstand the information or have irrational responses to it. Any initiative in this area should take such factors into account and should consider carefully how to apply the ethical principles of respect for autonomy and beneficence. (shrink)