In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart’s Law as “Virtually any goal specification, pursued to the extreme, will result in the extinction of humanity”, and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart’s Law holds or not. As our key contribution, we identify a set of conditions that are necessary for a model that (...) aims to be informative for evaluating specific arguments for Extinction-level Goodhart’s Law. Since each of the conditions seems to significantly contribute to the complexity of the resulting model, formally evaluating the hypothesis might be exceedingly difficult. This raises the possibility that whether the risk of extinction from artificial intelligence is real or not, the underlying dynamics might be invisible to current scientific methods. (shrink)

This paper examines a recent argument in favour of strong precautionary action—possibly including working to hasten human extinction—on the basis of a decision-theoretic view that accommodates the risk-attitudes of all affected while giving more weight to the more risk-averse attitudes. First, we dispute the need to take into account other people’s attitudes towards risk at all. Second we argue that a version of the non-identity problem undermines the case for doing so in the context of future (...) people. Lastly, we suggest that we should not work to hasten human extinction, even if significant risk aversion is warranted. (shrink)

Our Earth’s climate is changing. Any species living in the Earth’s ecosystem need to thrive to adapt to the new living conditions. Otherwise, extinction will be their outcome. In the race for adaptation, waterbirds (Aequorlitornithes), such as penguins, cormorants, and alcids, seem disadvantageous.

“Existential risks” are risks that threaten the destruction of humanity’s long-term potential: risks of nuclear wars, pandemics, supervolcano eruptions, and so on. On standard utilitarianism, it seems, the reduction of such risks should be a key global priority today. Many effective altruists agree with this verdict. But how should the importance of these risks be assessed on a Christian moral theory? In this paper, I begin to answer this question – taking Thomas Aquinas as a reference, and the risks of (...) anthropogenic extinction as an example. I’ll suggest that on a Thomist framework, driving ourselves extinct would constitute a complete failure to fulfil our God-given nature, a radical form of hubris, and a loss of cosmological significance. So the reduction of such risks seems vital for Thomists, or Christians more broadly. Indeed, I end with the thought that these considerations are not parochially Christian, or indeed specifically religious: everyone with a similar view of mankind – as immensely valuable yet fallen and fragile – should agree that existential risks are a pressing moral concern. (shrink)

Longtermism is the view that the most urgent global priorities, and those to which we should devote the largest portion of our resources, are those that focus on (i) ensuring a long future for humanity, and perhaps sentient or intelligent life more generally, and (ii) improving the quality of the lives that inhabit that long future. While it is by no means the only one, the argument most commonly given for this conclusion is that these interventions have greater expected goodness (...) per unit of resource devoted to them than each of the other available interventions, including those that focus on the health and well-being of the current population. In this paper, I argue that, even if we grant the consequentialist ethics upon which this argument depends, and even if we grant one of the axiologies that are typically paired with that ethics to give the argument, we are not morally required to choose an option that maximises expected utility; indeed, we might not even be permitted to do so. Instead, I will argue, if the argument's consequentialism is correct, we should choose using a decision theory that is sensitive to risk, and allows us to give greater weight to worse-case outcomes than expected utility theory does. And, I will show, such decision theories do not always recommend longtermist interventions. Indeed, sometimes, they recommend exactly the opposite: sometimes, they recommend hastening human extinction. Many, though not all, will take this as a reductio of the consequentialism or the axiology of the argument. I remain agnostic on the conclusion we should draw. (shrink)

This review provides an overview of the ethics of extinctions with a focus on the Western analytical environmental ethics literature. It thereby gives special attention to the possible philosophical grounds for Michael Soulé’s assertion that the untimely ‘extinction of populations and species is bad’. Illustrating such debates in environmental ethics, the guiding question for this review concerns why – or when – anthropogenic extinctions are bad or wrong, which also includes the question of when that might not be the (...) case (i.e. which extinctions are even desirable). After providing an explanation of the disciplinary perspective taken (section “Introduction”), the concept of extinction and its history within that literature are introduced (section “Understanding extinction”). Then, in section “Why (or when) might anthropogenic extinctions be morally problematic?”, different reasons for why anthropogenic extinctions might be morally problematic are presented based on the loss of species’ value, harm to nonhuman individuals, the loss of valuable biological variety and duties to future generations. This section concludes by also considering cases where anthropogenic extinctions might be justified. Section “How to respond to extinctions?” then addresses a selection of topics concerning risks and de-extinction technologies. Finally, the section on “Extinction studies” introduces other viewpoints on the ethics of extinction from the extinction studies literature, followed by the “Conclusion”. (shrink)

When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk avoidant risk function. This, in turn, has been claimed to require the use of a risk avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the (...) significant risks associated with continued human survival. I raise objections to the claim that respect for others’ risk attitudes requires risk avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others’ risk attitudes in fixed population cases. GPI Working Paper No . 20-2022. (shrink)

A small but growing number of studies have aimed to understand, assess and reduce existential risks, or risks that threaten the continued existence of mankind. However, most attention has been focused on known and tangible risks. This paper proposes a heuristic for reducing the risk of black swan extinction events. These events are, as the name suggests, stochastic and unforeseen when they happen. Decision theory based on a fixed model of possible outcomes cannot properly deal with this kind (...) of event. Neither can probabilistic risk analysis. This paper will argue that the approach that is referred to as engineering safety could be applied to reducing the risk from black swan extinction events. It will also propose a conceptual sketch of how such a strategy may be implemented: isolated, self-sufficient, and continuously manned underground refuges. Some characteristics of such refuges are also described, in particular the psychosocial aspects. Furthermore, it is argued that this implementation of the engineering safety strategy safety barriers would be effective and plausible and could reduce the risk of an extinction event in a wide range of possible scenarios. Considering the staggering opportunity cost of an existential catastrophe, such strategies ought to be explored more vigorously. (shrink)

Most current techniques to deal with invasive species are ineffective or have highly damaging side effects. To this end suppression-drives based on clustered regularly inter-spaced short palindromic repeats (CRISPR/Cas9) have been touted as a potential silver bullet for the problem, allowing for a highly focused, humane and cost-effective means of removing a target species from an environment. Suppression-drives come with serious risks, however, such that the precautionary principle seems to warrant us not deploying this technology. The focus of this paper (...) is on one such risk – the danger of a suppression-drive escaping containment and wiping out the target species globally. Here, I argue that in most cases this risk is significant enough to warrant not using a gene-drive. In some cases, however, we can bypass the precautionary principle by using an approach that hinges on what I term the ‘Worst-Case Clause’. This clause, in turn, provides us with a litmus test that can be fruitfully used to determine what species are viable targets for suppression-drives in the wild. Using this metric in concert with other considerations, I suggest that only three species are currently possible viable targets – the European rabbit, ship rat and Caribbean Tree Frog. (shrink)

Existence value refers to the value humans ascribe to the existence of something, regard­less of whether it is or will be of any particular use to them. This existence value based on preference satisfaction should be taken into account in evaluating activities that come with a risk of species extinction. There are two main objections. The first is that on the preference satisfaction interpretation, the concept lacks moral importance because satisfying people’s preferences may involve no good or well-being (...) for them. However, existence value can be based on a restricted version of the preference satisfaction theory, which is not vulnerable to the skeptical arguments about the link between preference satisfaction and well-being. The second objection is that even if preference satisfaction can be linked to well-being, understanding existence value in terms of individual preference satisfaction is incoherent, because existence value reflects disinterested preferences that involve no benefits to the individual. However, the fact that existence value may involve disinterested preferences does not threaten the coherence of the concept, but suggests that it does not fit smoothly into the “utilitarian” or “welfarist” framework it is commonly considered within. A pluralistic normative approach based on prima facie duties can be an alternative to standard utilitarian-style approaches for considering existence value in concrete cases involving a risk of species extinction, such as through deep sea mining. (shrink)

In this article, a classification of the global catastrophic risks connected with the possible existence (or non-existence) of extraterrestrial intelligence is presented. If there are no extra-terrestrial intelligences (ETIs) in our light cone, it either means that the Great Filter is behind us, and thus some kind of periodic sterilizing natural catastrophe, like a gamma-ray burst, should be given a higher probability estimate, or that the Great Filter is ahead of us, and thus a future global catastrophe is high probability. (...) If ETIs exist in our light cone, it is most likely they exist in the form of ET artificial general intelligence (ETAI). ETAI may be traveling at near-light speed as an explosive wave of intelligence in the form of some nano-mechanisms which could reach us at any moment; may send us potentially dangerous messages with AI-virus payloads; or be near or on Earth in some dormant form. This dormant form could be nanobots, which could act as berserkers, and be triggered when humans unwittingly cross some threshold. If ETIs are nearby and less advanced, they could find our messaging extra-terrestrial intelligence (METI) messages and send their “fleet” to or direct weapons toward Earth. Even extinct ETIs could have left dangerous remnants in the form of AI systems that our civilization may encounter. If our existence is a simulation, this could be run by ETIs. Even a false belief in the existence of ETIs could potentially fuel millennial sects. If humanity is the first intelligence to emerge, it may kill or prevent the existence of all future potential ETI, a different type of catastrophe. Several options to prevent catastrophic risks are connected with ETI: sending requests for help, using random strategies to escape the Fermi paradox, Great Filter prediction, or the hope that ETIs will find the remains of our extinct civilization and resurrect it. (shrink)

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk, often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent (...) AI both contributes to existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks. (shrink)

Pandemics have been suggested as global risks many times, but it has been shown that the probability of human extinction due to one pandemic is small, as it will not be able to affect and kill all people, but likely only half, even in the worst cases. Assuming that the probability of the worst pandemic to kill a person is 0.5, and assuming linear interaction between different pandemics, 30 strong pandemics running simultaneously will kill everyone. Such situations cannot happen (...) naturally, but because biotechnology is developing analogously to Moore’s law, it may become possible in the near future (10-50 years from now), because of biohackers, CRISPR, bioprinters, AI-assisted DNA-programing, and weapons of “knowledge-enabled mass destruction” published on the Internet. It could also happened in case of large-scale biological war, or if a rogue country released its entire biological arsenal simultaneously. We also will address other scenarios and risk increasing factors as well as mitigation and adaptation strategies. (shrink)

Scientists have been arguing for more than 25 years about whether it is a good idea to collect voucher specimens from particularly vulnerable biological populations. Some think that, obviously, scientists should not be harvesting organisms from, for instance, critically endangered species. Others think that, obviously, it is the special job of scientists to collect precisely such information before any chance of retrieving it is forever lost. The character, extent, longevity, and span of the ongoing disagreement indicates that this is likely (...) to be a hard problem to solve. Nonetheless, the aim of this paper is to help field biologists figure out what do to when collecting a voucher specimen risks extinction. Here I present and assess varying practices of specimen collection for both extant and extinct species in order to compare and contrast cases where extinction risk both is and is not a problem. When it comes to taking vouchers from extant species at some risk of extinction, I determine that those advocating for conservative approaches to collection as well as those advocating for liberal information-gathering practices have good reasons to assess things in the way they each do. This means that there is unlikely to be a decisive, one-size-fits-all response to this problem. Still, progress can be made. We can acknowledge the risks of proceeding in either manner, as well as the uncertainty about how best to proceed. We can proceed as thoughtfully as possible, and be ready to articulate a rationale for whichever procedure is used in any particular case. (shrink)

According to an intriguing though somewhat enigmatic line of thought first proposed by Jonathan Bennett, if humanity went extinct any time soon this would be unfortunate because important business would be left unfinished. This line of thought remains largely unexplored. I offer an interpretation of the idea that captures its intuitive appeal, is consistent with plausible constraints, and makes it non-redundant to other views in the literature. The resulting view contrasts with a welfare-promotion perspective, according to which extinction would (...) be bad because it would come at a massive opportunity cost in terms of feasibly achievable welfare over the lifetime of our species. (shrink)

Current planetary defense policy prioritizes a probability assessment of risk of Earth impact by an asteroid or a comet in the planning of detection and mitigation strategies and in setting the levels of urgency and budgeting to operationalize them. The result has been a focus on asteroids of Tunguska size, which could destroy a city or a region, since this is the most likely sort of object we would need to defend against. However a complete risk assessment would (...) consider not only the probability of an impact but also the magnitude of its consequences, which in the case of an object of Chicxulub size could be the end of civilization or even human extinction. This paper argues that a planetary defense policy based on a complete (or one could say genuine) risk assessment would justify expenditures much higher than at present. (shrink)

Recently many methods for reducing the risk of human extinction have been suggested, including building refuges underground and in space. Here we will discuss the perspective of using military nuclear submarines or their derivatives to ensure the survival of a small portion of humanity who will be able to rebuild human civilization after a large catastrophe. We will show that it is a very cost-effective way to build refuges, and viable solutions exist for various budgets and timeframes. Nuclear (...) submarines are surface independent, and could provide energy, oxygen, fresh water and perhaps even food for their inhabitants for years. They are able to withstand close nuclear explosions and radiation. They are able to maintain isolation from biological attacks and most known weapons. They already exist and need only small adaptation to be used as refuges. But building refuges is only “Plan B” of existential risk preparation; it is better to eliminate such risks than try to survive them. (shrink)

One of Derek Parfit’s greatest legacies was the search for Theory X, a theory of population ethics that avoided all the implausible conclusions and paradoxes that have dogged the field since its inception: the Absurd Conclusion, the Repugnant Conclusion, the Non-Identity Problem, and the Mere Addition Paradox. In recent years, it has been argued that this search is doomed to failure and no satisfactory population axiology is possible. This chapter reviews Parfit’s life’s work in the field and argues that he (...) provided all the necessary components necessary for a Theory X. It then shows how these components can be combined together and applied to the global challenges Parfit argued matter most: preventing human extinction, managing catastrophic risks, and eradicating global poverty and suffering. Finally, it identifies a number of challenges facing his theory and suggests how these may be overcome. (shrink)

Abstract: Four main forms of Doomsday Argument (DA) exist—Gott’s DA, Carter’s DA, Grace’s DA and Universal DA. All four forms use different probabilistic logic to predict that the end of the human civilization will happen unexpectedly soon based on our early location in human history. There are hundreds of publications about the validity of the Doomsday argument. Most of the attempts to disprove the Doomsday Argument have some weak points. As a result, we are uncertain about the validity of DA (...) proofs and rebuttals. In this article, a meta-DA is introduced, which uses the idea of logical uncertainty over the DA’s validity estimated based on a virtual prediction market of the opinions of different scientists. The result is around 0.4 for the validity of some form of DA, and even smaller for “Strong DA”, which predicts the end of the world in the near term. We discuss many examples of the validity of the DA in real life as an instrument to prove it “experimentally”. We also show that DA becomes strongest if it is based on the idea of the “natural reference class” of observers, that is, the observers who know about the DA (i.e. a Self-Referenced DA). Such a DA predicts that there is a high probability of a global catastrophe with human extinction in the 21st century, which aligns with what we already know based on analysis of different technological risks. (shrink)

This chapter makes the case for strong longtermism: the claim that, in many situations, impact on the long-run future is the most important feature of our actions. Our case begins with the observation that an astronomical number of people could exist in the aeons to come. Even on conservative estimates, the expected future population is enormous. We then add a moral claim: all the consequences of our actions matter. In particular, the moral importance of what happens does not depend on (...) when it happens. That pushes us toward strong longtermism. We then address a few potential concerns, the first of which is that it is impossible to have any sufficiently predictable influence on the course of the long-run future. We argue that this is not true. Some actions can reasonably be expected to improve humanity’s long-term prospects. These include reducing the risk of human extinction, preventing climate change, guiding the development of artificial intelligence, and investing funds for later use. We end by arguing that these actions are more than just extremely effective ways to do good. Since the benefits of longtermist efforts are large and the personal costs are comparatively small, we are morally required to take up these efforts. (shrink)

Abstract. The presumptuous philosopher (PP) thought experiment lends more credence to the hypothesis which postulates the existence of a larger number of observers than other hypothesis. The PP was suggested as a purely speculative endeavor. However, there is a class of real world observer-selection effects where it could be applied, and one of them is the possibility of interstellar panspermia (IP). There are two types of anthropic reasoning: SIA and SSA. SIA implies that my existence is an argument that larger (...) total number of observers exists; SSA implies that I should find myself in a region with larger number of observers. However, as S. Armstrong showed, SIA can’t distinguish between different ways how larger number of observers appeared, so it can’t favor IP compared with other ways to get many observers. SSA application here is less controversial as it tells only about relation between regions size: e.g. I am more likely to live in a larger country than in a small one, conditioning that the total number of small countries is also small. Anthropic considerations suggest that the universes in the multiverse with interstellar panspermia will have orders of magnitude more civilizations than universes without IP, and thus we are likely to be in such a universe. This is a strong counterargument against a variant of the Rare Earth hypothesis based on the difficulty of abiogenesis: even if abiogenesis is difficult, IP will disseminate life over billions of planets, meaning we are likely to find ourselves in a universe where IP has happened. This implies that there should be many planets with life in our galaxy, and renders the Fermi paradox even sharper. Either the Great Filter is ahead of us and there are high risks of anthropogenic extinction, or there are many alien civilizations nearby and of the same age as ours—which is itself a global catastrophic risk, as in that case the wave of alien colonization could arrive between 25-500 ky from now. -/- . (shrink)

Plausibly, one important part of a good life is doing work that makes a contribution, or a positive difference to the world. In this paper, however, I explore contribution pessimism, the view that people will not always have adequate opportunities for making contributions. I distinguish between three interestingly different and at least initially plausible reasons why this view might be true: in slogan form, things might become too easy, they might become too good, or we might be too late. Now, (...) one response to these problems might be to deliberately attempt to undo them. However, I claim that this solution is intuitively misguided, and argue that explaining this supports a holistic approach to the value of contributions. Finally, I argue that if contribution pessimism is correct, it could provide an explanation of some widely held intuitions about issues in population ethics, with implications about the practical issue of how much priority we should give to addressing risks of human extinction. (shrink)

Person-affecting views in population ethics state that (in cases where all else is equal) we’re permitted but not required to create people who would enjoy good lives. In this paper, I present an argument against every possible variety of person-affecting view. The argument takes the form of a dilemma. Narrow person-affecting views must embrace at least one of three implausible verdicts in a case that I call ‘Expanded Non-Identity.’ Wide person-affecting views run into trouble in a case that I call (...) ‘Two-Shot Non-Identity.’ One plausible practical upshot of my argument is as follows: we individuals and our governments should be doing more to reduce the risk of human extinction this century. (shrink)

Henry Shue’s latest book, The Pivotal Generation: Why We Have a Moral Responsibility to Slow Climate Change Right Now, is an excellent read, both clear and comprehensive. It is written in a way that makes it accessible to philosophers and non-philosophers alike. The book argues persuasively that the people alive today must take immediate and drastic action to tackle climate change, as the current decade will be crucial for determining how severe the impacts will become. Shue warns how a sharp (...) division into past, present, and future is misleading when it comes to climate change and can obfuscate the extent of the responsibility that the current generations bear. For us to acknowledge our responsibility, we must recognise how deeply intertwined our lives are with both the past and the future. One of these innate deep connections between generations is the extensive time period that carbon emissions can continue to contribute to climate change, for dozens of centuries. -/- The current generations have been bequeathed an energy system that relies on fossil fuels and that continues to add greenhouse gases to the atmosphere. For over two centuries now, the planet has been getting warmer as an unintended side effect of industrialisation. We are the first humans to understand the dynamics of the Earth’s climate and how human actions cause anthropogenic climate change. With the rapidly advanced scientific knowledge, it has become apparent that there is a need to quickly transition to a different energy regime. Shue points out that this knowledge has made us the first humans to recognise that action is required, but the urgency of the problem also makes us possibly the last to still be able to act before certain major threats are aggravated. With many irreversible climate change impacts, the date-of-last-opportunity to take action to prevent them might be very soon. According to Shue (2021, p. 6), “This gives us an awesome responsibility. Humans have accidentally set our own house on fire, and if we do not douse the flames while they are no more extensive than they are now, it may not be possible ever to extinguish them”. That is why we alive now are the pivotal generation. -/- Although a sense of urgency runs through the book and the seriousness of the situation is made very clear, Shue steers away from fearmongering. He does not think that human extinction is around the corner, but underlines that we cannot rule out such threats until we stop feeding the beast. Unless we cap emissions at a relatively safe level and transform our infrastructure to net zero, we will keep on increasing risks to future people by “adding continuously to the mushrooming danger” (p. 23). The book stresses that the possibility of passing several tipping points adds to the urgency and the risks because when positive feedback mechanisms are triggered, climate change accelerates. These positive feedbacks can feed into each other, such as when warming seawater melts even more ice, with the possibility of a cascade of feedbacks if enough tipping points are passed. Such changes are irreversible and could themselves become further positive feedbacks. Shue (pp. 24–25) warns that it is “likely that the near future is the last chance to avoid passing significant tipping points and entirely possible that the near future is the last chance to avoid provoking a cascade of tipping points” (emphasis in original). -/- While I applaud the general message of Shue’s book, I find that the brushstrokes he uses when identifying those responsible are a little too broad. The reason for this is twofold. Firstly, it is questionable as to how many of us really know enough about the risks we are leaving future generations with. Secondly, discussion in terms of generations underplays the big differences between the responsibility of different groups of actors within them. In what follows, I will elaborate on these points. I should note from the outset, however, that although I think that these are important issues, I find them to be points of clarification in an impressive and ambitious book on why our actions have such significance and why objections to urgent, large-scale climate action are misplaced. (shrink)

Critical examination of possible socio-political Anthropocene consequences leads to the conclusion that the sustainable development concept is not an adequate answer for current threats and risks. An effort to implement the sustainable development concept can even make climate changes and other forms of nature devastation worse, as it turns out on ongoing greenhouse gas concentrations growth in the atmosphere, despite obligations that result to all states of the world from Paris agreement. The climate change rate and range of plant and (...) animal species extinction confirms J. Lovelock´s thesis that humanity does not have enough time and, most likely, enough sources to implement the sustainable development concept and, as B. Latour points out, we do not have a big enough planet. Therefore, it is necessary to think about a retreat – to areas that are likely to stay livable even after the sea level rise and desertification; it is necessary to think about how to move millions of people from areas flooded by sea or parched by desert; to think about a change of our current way of life, of production of food and energy so that not only humankind as animal species would continue, but also our civilization and its technological and organizational knowledge. Even basic theses based on this concept indicate a need to reassess most political concepts, mainly the concepts of sovereignty, citizenship, but also those of freedom of movement and residence, freedom of business and consume, which work as fundamentals of the neoliberal economic-political system. Retreat expects complex and organized migration to beforehand chosen and arranged areas. Opposite to retreat is mass escape chaos that reduces human relations to fight for survival. As J. Lovelock proposed in his sustainable retreat concept, relocating people from areas affected by climate changes to climate oases would mean that rich northern countries give up their current, in fact, isolationistic politics of refusing climate migrants. Therefore, the concept of sustainable retreat requires much greater competence and especially willingness to cooperate not only nationally, but also internationally and globally, in addition to what humanity is capable of doing today. (shrink)

Longtermists claim that what we ought to do is mainly determined by how our actions might affect the very long-run future. A natural objection to longtermism is that these effects may be nearly impossible to predict — perhaps so close to impossible that, despite the astronomical importance of the far future, the expected value of our present actions is mainly determined by near-term considerations. This paper aims to precisify and evaluate one version of this epistemic objection to longtermism. To that (...) end, I develop two simple models for comparing ‘longtermist’ and ‘neartermist’ interventions, incorporating the idea that it is harder to make a predictable difference to the further future. These models yield mixed conclusions: if we simply aim to maximize expected value, and don’t mind premising our choices on minuscule probabilities of astronomical payoffs, the case for longtermism looks robust. But on some prima facie plausible empirical worldviews, the expectational superiority of longtermist interventions depends heavily on these ‘Pascalian’ probabilities. So the case for longtermism may depend either on plausible but non-obvious empirical claims or on a tolerance for Pascalian fanaticism. (shrink)

This paper serves as an intuition building mechanism for understanding the basics of AI, misalignment, and the reasons for why strong AI is being pursued. The approach is to engage with both pro and anti AI development arguments to gain a deeper understanding of both views, and hopefully of the issue as a whole. We investigate the basics of misalignment, common misconceptions, and the arguments for why we would want to pursue strong AI anyway. The paper delves into various aspects (...) of the problem, including existential risk, deception, hedonic adaptation, and the potential for the complete extinction of humanity. By integrating multiple elements of philosophy, this paper aims to provide a holistic understanding of the alignment problem and its significance for the future of humanity. (shrink)

The field of AI safety aims to prevent increasingly capable artificially intelligent systems from causing humans harm. Research on moral alignment is widely thought to offer a promising safety strategy: if we can equip AI systems with appropriate ethical rules, according to this line of thought, they'll be unlikely to disempower, destroy or otherwise seriously harm us. Deontological morality looks like a particularly attractive candidate for an alignment target, given its popularity, relative technical tractability and commitment to harm-avoidance principles. I (...) argue that the connection between moral alignment and safe behavior is more tenuous than many have hoped. In general, AI systems can possess either of these properties in the absence of the other, and we should favor safety when the two conflict. In particular, advanced AI systems governed by standard versions of deontology need not be especially safe. (shrink)

The main objective of this work is to achieve an understanding of Husserl's phenomenology as philosophical foundation for theology. It sustains, in the first place, that theology and philosophy do more than converge. It deepens, in second place, in the connection between phenomenology and theology, as well as in the Husserlian conception of God as entelechy and ἐνέργεια. This study concludes with a reflection, in third place, on the possibility of elaborating a theology from the phenomenological inspiration. The thesis that (...) is defended is that phenomenology means for theology, literally, an authentic Herausforderung, that is to say, a provocation that from outside interpellates a discourse, the theological one, that many judged already exhausted and at risk of extinction. In the conclusions are detailed the main contributions that the phenomenology makes to theology. El principal objetivo de este trabajo es lograr una comprensión de la la fenomenología de Husserl como fundamento filosófico para la teología. Sostiene, en primer lugar, la idea de que la teología y la filosofía hacen algo más que converger. Profundiza, en segundo lugar, en la conexión entre fenomenología y teología, así como en la concepción husserliana de Dios como entelequia y ἐνέργεια. Este estudio concluye con una reflexión, en tercer lugar, sobre la posibilidad de elaborar una teología de inspiración fenomenológica. La tesis que se defiende es que la fenomenología supone para la teología, literalmente, una auténtica Herausforderung, es decir, una provocación que desde fuera interpela a un discurso, el teológico, que muchos juzgaban ya agotado y en riesgo de extinción. En las conclusiones se detallan las principales aportaciones que la fenomenología hace a la teología. (shrink)

One of the possible solutions of the Fermi paradox is that all civilizations go extinct because they hit some Late Great Filter. Such a universal Late Great Filter must be an unpredictable event that all civilizations unexpectedly encounter, even if they try to escape extinction. This is similar to the “Death in Damascus” paradox from decision theory. However, this unpredictable Late Great Filter could be escaped by choosing a random strategy for humanity’s future development. However, if all civilizations act (...) randomly, this could be the actual cause of universal extinction, as no one will choose the most optimal strategy. Using a meta-random strategy, where the decision to start random actions is also made randomly, solves this. (shrink)

Some people feel distressed reflecting on human extinction. Some people even claim that our efforts and lives would be empty and pointless if humanity becomes extinct, even if this will not occur for millions of years. In this essay, I will attempt to demonstrate that this claim is false. The desire for long-lastingness or quasi-immortality is often unwittingly adopted as a standard for judging whether our efforts are significant. If we accomplish our goals and then later in life conclude (...) that these accomplishments were of no significance, then this is a sign that the desire for long-lastingness has crept into our standards. By recognizing this, and refraining from adopting an unreasonable standard to judge whether our efforts are significant, it will be to our advantage. Then, when we look back on life from an external perspective that encompasses times after humanity has become extinct, we will not conclude that our efforts amounted to nothing. Rather, we will conclude that many people made significant accomplishments that made their lives and the lives of other people better than they would have been if their goals had never been pursued. (shrink)

Some people think that the inevitability of human extinction renders life meaningless. Joshua Seachris has argued that naturalism can be conceptualized as a meta-narrative and that it narrates across important questions of human life, including what is the meaning of life and how life will end. How a narrative ends is important, Seachris argues. In the absence of God, and with knowledge that human extinction is a certainty, is there any way that humanity could be meaningful and have (...) a good ending? I will distinguish between two conceptions of how humanity could be meaningful: the traditional view and an alternative view, which I will outline. I will argue that this alternative view provides a plausible explanation for how humanity could become meaningful. I will also argue that coming to terms with our mortality and other limitations would add meaning to human life and provide humanity with a good ending. (shrink)

The cumulative impact of enhancement technologies may alter the human species in the very long-term future. In this article, I will start showing how radical genetic enhancements may accelerate the conversion into a novel species. I will also clarify the concepts of ‘biological species’, ‘transhuman’ and ‘posthuman’. Then, I will summarize some ethical arguments for creating a transhuman or posthuman species with a substantially higher level of well-being than the human one. In particular, I will present what I shall call (...) the Principle of the Best Interests of Posthumanity, which states that the enhancement of the human and transhuman species must be directed towards the creation of a posthuman existence that is substantially more valuable than its predecessors. I suggest that human extinction may be considered, within that principle, as one of the best interests of posthumanity. Finally, I will develop three objections that make that principle unattractive and that show that pursuing a full-blown programme of posthuman evolution is ethically flawed. (shrink)

The standard argument to the conclusion that artificial intelligence (AI) constitutes an existential risk for the human species uses two premises: (1) AI may reach superintelligent levels, at which point we humans lose control (the ‘singularity claim’); (2) Any level of intelligence can go along with any goal (the ‘orthogonality thesis’). We find that the singularity claim requires a notion of ‘general intelligence’, while the orthogonality thesis requires a notion of ‘instrumental intelligence’. If this interpretation is correct, they cannot (...) be joined as premises and the argument for the existential risk of AI turns out invalid. If the interpretation is incorrect and both premises use the same notion of intelligence, then at least one of the premises is false and the orthogonality thesis remains itself orthogonal to the argument to existential risk from AI. In either case, the standard argument for existential risk from AI is not sound.—Having said that, there remains a risk of instrumental AI to cause very significant damage if designed or used badly, though this is not due to superintelligence or a singularity. (shrink)

In English, as in many other languages, male-gendered pronouns are sometimes used to refer not only to men, but to individuals whose gender is unknown or unspecified, to human beings in general (as in ―mankind‖) and sometimes even to females (as when the casual ―Hey guys‖ is spoken to a group of women). These so-called he/man or masculine generics have come under fire in recent decades for being sexist, even archaic, and positively harmful to women and girls; and advocates of (...) gender-neutral (or nonsexist) language have put forward serious efforts to discourage their use. Have they been successful, and to what extent? In this paper, I review some of the main arguments in favor of abolishing sexist male generics. I then present three studies tracking the use of he/man terminology in academic, popular, and personal discourse over the past several decades. I show that the use of these terms has fallen dramatically in recent years, while nonsexist alternatives have gradually taken their place. We may be paying witness to the early stages of the ultimate extinction of masculine generics. (shrink)

While anthropogenic species extinctions can be considered morally problematic for a range of reasons, they can also be described as a problem of interspecies justice. That is the focus of this paper in which we argue that human-caused species extinctions can be integrated within a non-anthropocentric account of reparative justice that is significantly similar to how reparation is understood within political theory at large. An account such as this faces a series of difficulties, such as how to make right past (...) injustices against victims that do not exist any longer and which belonged to species now extinct. For that purpose, we offer an argument in favour of symbolic forms of reparation in the form of, amongst other things, the preservation and restoration of habitat to materially repair the interspecies justice relationship; as well as monuments and education to honour the nonhuman dead. (shrink)

This book defends an account of justice to nonhuman beings – i.e., to animals, plants etc. – also known as ecological or interspecies justice, and which lies in the intersection of environmental political theory and environmental ethics. More specifically, against the background of the current extinction crisis this book defends a global non-ranking biocentric theory of distributive ecological/interspecies justice to wild nonhuman beings, because the extinction crisis does not only need practical solutions, but also an account of how (...) it is the outcome of large-scale injustice. This book not only provides a novel theoretical framework that supports such a claim, but it also develops the theoretical tools to find just compromises between the entitlements of intraspecies and intra-human global distributive justice to ecological space for the purpose of working towards a vision of just conservation. -/- Content: 1. Introducing Ecological Justice 2. Political Non-Ranking Biocentrism 3. The Community of Justice 4. The Currency of Distributive Justice 5. The Principles of Distributive Justice 6. Ecological Justice and the Capabilities Approach 7. Biodiversity Loss: An Injustice? 8. Who Owns the Earth? 9. Visions of Just Conservation 10. Outlook for Implementation . (shrink)

This paper addresses two problems concerning the deferred ostension of extinct and fictive kinds. First, the sampled item, the fossil or the depiction, is not a sample of the referent. Nonetheless, the retained characteristic shape, understood via analogy with living creatures, enables the reference to be fixed. Second, though both extinct and fictive kinds are targets of deferred ostension, there is an important difference in the sample. Fossilization is a natural causal process that makes fossils to be reflections of their (...) originals. As reflections, fossils embed their referents in the primary existential world of perceived things. Images of artificial kinds, by contrast, leave their referents in the secondary existential world of mere appearance. In this way, the paper widens the scope of the Kripke-Putnam account of ostension for naming kinds by drawing on Quine’s concept of deferred ostension for absent referents. (shrink)

Many of us think human extinction would be a very bad thing, and that we have moral reasons to prevent it. But there is disagreement over what would make extinction so bad, and thus over what grounds these moral reasons. Recently, several theorists have argued that our reasons to prevent extinction stem not just from the value of the welfare of future lives, but also from certain additional values relating to the existence of humanity itself (for example, (...) humanity’s “final” value, or the value of humanity in itself). In this paper, I argue against these “additional value” views. Despite their initial appeal, these views will inevitably face conflicts between the additional values to which they appeal, and the value of the welfare of future lives. And, I argue, the views cannot plausibly resolve these conflicts. In contrast, these conflicts do not arise for a rival view, on which our reasons to prevent extinction stem just from the value of the welfare of future lives. I conclude that this gives us reason to prefer the latter view, despite the greater initial plausibility of additional value views. (shrink)

This article argues that Lara Buchak’s risk-weighted expected utility (REU) theory fails to offer a true alternative to expected utility theory. Under commonly held assumptions about dynamic choice and the framing of decision problems, rational agents are guided by their attitudes to temporally extended courses of action. If so, REU theory makes approximately the same recommendations as expected utility theory. Being more permissive about dynamic choice or framing, however, undermines the theory’s claim to capturing a steady choice disposition in (...) the face of risk. I argue that this poses a challenge to alternatives to expected utility theory more generally. (shrink)

De-extinction is the process through which extinct species can be brought back into existence. Although these projects have the potential to cause great harm to animal welfare, discussion on issues surrounding de-extinction have focussed primarily on other issues. In this paper, I examine the potential types of welfare harm that can arise through de-extinction programs, including problems with cloning, captive rearing and re-introduction. I argue that welfare harm should be an important consideration when making decisions on de- (...) class='Hi'>extinction projects. Though most of the proposed benefits of these projects are insufficient to outweigh the current potential welfare harm, these problems may be overcome with further development of the technology and careful selection of appropriate species as de-extinction candidates. (shrink)

In addition to protecting agents’ autonomy, consent plays a crucial social role: it enables agents to secure partners in valuable interactions that would be prohibitively morally risk otherwise. To do this, consent must be observable: agents must be able to track the facts about whether they have received a consent-based permission. I argue that this morally justifies a consent-practice on which communicating that one consents is sufficient for consent, but also generates robust constraints on what sorts of behaviors can (...) be taken as consent- communicating. (shrink)

A growing number of decision theorists have, in recent years, defended the view that rationality is permissive under risk: Different rational agents may be more or less risk-averse or risk-inclined. This can result in them making different choices under risk even if they value outcomes in exactly the same way. One pressing question that arises once we grant such permissiveness is what attitude to risk we should implement when choosing on behalf of other people. Are (...) we permitted to implement any of the rationally permissible risk attitudes, is there some specific risk attitude that is required when choosing for others, or are we required to defer to the risk attitudes of the people on whose behalf we are choosing? This article elaborates on this question, explains its wider practical and theoretical significance, provides an overview of existing answers, and explores how to go about providing a more systematic account of how to choose on behalf of others in risky contexts. (shrink)

Uncertainty, insufficient information or information of poor quality, limited cognitive capacity and time, along with value conflicts and ethical considerations, are all aspects thatmake risk managementand riskcommunication difficult. This paper provides a review of different risk concepts and describes how these influence risk management, communication and planning in relation to forest ecosystem services. Based on the review and results of empirical studies, we suggest that personal assessment of risk is decisive in the management of forest ecosystem (...) services. The results are used together with a reviewof different principles of the distribution of risk to propose an approach to risk communication that is effective aswell as ethically sound. Knowledge of heuristics and mutual information on both beliefs and desires are important in the proposed risk communication approach. Such knowledge provides an opportunity for relevant information exchange, so that gaps in personal knowledge maps can be filled in and effective risk communication can be promoted. (shrink)

The underreporting of suspected adverse drug reactions remains a primary issue for contemporary post-market drug surveillance or ‘pharmacovigilance.’ Pharmacovigilance pioneer W.H.W. Inman argued that ‘deadly sins’ committed by clinicians are to blame for underreporting. Of these ‘sins,’ ignorance and lethargy are the most obvious and impactful in causing underreporting. However, recent analyses show that diffidence, insecurity, and indifference additionally play a major role. I aim to augment our understanding of diffidence, insecurity, and indifference by arguing these sins are underwritten by (...) value judgments arising via epistemic risk. I contend that ‘evidence-based’ medicine codifies these sins. (shrink)

Powerful, technically complex international compliance regimes have developed recently in certain professions that deal with risk: banking (the Basel II regime), accountancy (IFRS) and the actuarial profession. The need to deal with major risks has acted as a strong driver of international co-operation to create enforceable international semilegal systems, as happened earlier in such fields as international health regulations. This regulation in technical fields contrasts with the failure of an international general-purpose political and legal regime to develop. We survey (...) the new global regulatory systems in the actuarial, banking and accounting fields, with a view to showing how the need to deal reasonably with risk has resulted in an international de facto law solidly based on correct abstract principles of probability. (shrink)

A natural view in distributive ethics is that everyone's interests matter, but the interests of the relatively worse off matter more than the interests of the relatively better off. I provide a new argument for this view. The argument takes as its starting point the proposal, due to Harsanyi and Rawls, that facts about distributive ethics are discerned from individual preferences in the "original position." I draw on recent work in decision theory, along with an intuitive principle about risk-taking, (...) to derive the view. (shrink)

The paper re-expresses arguments against the normative validity of expected utility theory in Robin Pope (1983, 1991a, 1991b, 1985, 1995, 2000, 2001, 2005, 2006, 2007). These concern the neglect of the evolving stages of knowledge ahead (stages of what the future will bring). Such evolution is fundamental to an experience of risk, yet not consistently incorporated even in axiomatised temporal versions of expected utility. Its neglect entails a disregard of emotional and financial effects on well-being before a particular (...) class='Hi'>risk is resolved. These arguments are complemented with an analysis of the essential uniqueness property in the context of temporal and atemporal expected utility theory and a proof of the absence of a limit property natural in an axiomatised approach to temporal expected utility theory. Problems of the time structure of risk are investigated in a simple temporal framework restricted to a subclass of temporal lotteries in the sense of David Kreps and Evan Porteus (1978). This subclass is narrow but wide enough to discuss basic issues. It will be shown that there are serious objections against the modification of expected utility theory axiomatised by Kreps and Porteus (1978, 1979). By contrast the umbrella theory proffered by Pope that she has now termed SKAT, the Stages of Knowledge Ahead Theory, offers an epistemically consistent framework within which to construct particular models to deal with particular decision situations. A model by Caplin and Leahy (2001) will also be discussed and contrasted with the modelling within SKAT (Pope, Leopold and Leitner 2007). (shrink)