What moral reasons, if any, do we have to ensure the long-term survival of humanity? This article contrastively explores two answers to this question: according to the first, we should ensure the survival of humanity because we have reason to maximize the number of happy lives that are ever lived, all else equal. According to the second, seeking to sustain humanity into the future is the appropriate response to the final value of humanity itself. Along the way, the article discusses (...) various issues in population axiology, particularly the so-called Intuition of Neutrality and John Broome’s ‘greediness objection’ to this intuition. (shrink)

In slogan form, the thesis of this dissertation is that shaping the far future is overwhelmingly important. More precisely, I argue that: Main Thesis: From a global perspective, what matters most is that we do what is best for the general trajectory along which our descendants develop over the coming millions, billions, and trillions of years. The first chapter introduces some key concepts, clarifies the main thesis, and outlines what follows in later chapters. Some of the key concepts include: existential (...) risk, the world's development trajectory, proximate benefits and ripple effects, speeding up development, trajectory changes, and the distinction between broad and targeted attempts to shape the far future. The second chapter is a defense of some methodological assumptions for developing normative theories which makes my thesis more plausible. In the third chapter, I introduce and begin to defend some key empirical and normative assumptions which, if true, strongly support my main thesis. In the fourth and fifth chapters, I argue against two of the strongest objections to my arguments. These objections come from population ethics, and are based on Person-Affecting Views and views according to which additional lives have diminishing marginal value. I argue that these views face extreme difficulties and cannot plausibly be used to rebut my arguments. In the sixth and seventh chapters, I discuss a decision-theoretic paradox which is relevant to my arguments. The simplest plausible theoretical assumptions which support my main thesis imply a view I call fanaticism, according to which any non-zero probability of an infinitely good outcome, no matter how small, is better than any probability of a finitely good outcome. I argue that denying fanaticism is inconsistent with other normative principles that seem very obvious, so that we are faced with a paradox. I have no solution to the paradox; I instead argue that we should continue to use our inconsistent principles, but we should use them tastefully. We should do this because, currently, we know of no consistent set of principles which does better. (shrink)

Many thinkers believe that the next transformational change in human organisation will be the onset of human-level artificial intelligence, and that the most likely method of achieving this will come through brain emulations or "ems": the ability to scan human brains and program their connections into ever faster computers. Taking this as his starting point, Hanson describes what a world dominated by these ems will be like.

A controversial scientific vision predicts a time in which humans and machines will merge and create a new form of non-biological intelligence, explaining how the occurrence will solve such issues as pollution, hunger, and aging.

This book aims to contribute significantly to the understanding of issues of value which repeatedly emerge in interdisciplinary discussions on space and society. Although a recurring feature of discussions about space in the humanities, the treatment of value questions has tended to be patchy, of uneven quality and even, on occasion, idiosyncratic rather than drawing upon a close familiarity with state-of-the-art ethical theory. One of the volume's aims is to promote a more robust and theoretically informed approach to the ethical (...) dimension of discussions on space and society. While the contributions are written in a manner which is accessible across disciplines, the book still withstands scrutiny by those whose work is primarily on ethics. At the same time it allows academics across a range of disciplines an insight into current approaches toward how the work of ethics gets done. The issues of value raised could be used to inform debates about regulation, space law and protocols for microbial discovery as well as longer-range policy debates about funding. (shrink)

Why should we care what happens to future generations? Samuel Scheffler argues that we are more invested in the fate of our descendants than we may realize. Implicit in our own attachments are powerful reasons for wanting the chain of human generations to persist into the indefinite future under conditions conducive to human flourishing.

Parfit's requirements for an ideal Theory X cannot be fully met since the Mere Addition Principle and Non-Antiegalitarianism imply the Repugnant Conclusion: Theory X does not exist. However, since the Repugnant Conclusion is really compelling, the Impersonal Total Principle should be adopted for impartial comparisons concerning future generations. Nevertheless, where our own interests are affected, we may yet choose to be partial, trading off our concern for future goodness with our self-interests. Theory X' meets all Parfit's requirements except the Mere (...) Addition Principle in less compelling cases. (shrink)

Philosophers concerned with procreative ethics have long been puzzled by Parfit’s Non-Identity Problem (NIP). Various solutions have been proposed, but I argue that we have not solved the problem on its own narrow person-affecting terms, i.e., in terms of the identified individuals affected by procreative decisions and acts, especially future children. Thus, the core problem remains unsolved. This is a nagging concern for all who hold the common intuition that actions that harm no one are permissible. I argue against Harmon’s (...) and Woodward’s direct, narrow person-affecting solutions, and in favor of a new solution to the NIP. My solution, or, rather, dissolution, is based on the argument that merely possible people, i.e., hypothetical people who could possibly, but will not actually, exist, are morally irrelevant. I show that the NIP only arises when we concern ourselves with merely possible people. Once we are careful to restrict our concerns to only those that do or will exist, the NIP is dissolved. (shrink)

The moral obligation to support space exploration follows from our obligations to protect the environment and to survive as a species. It can be justified through three related arguments: one supporting space exploration as necessary for acquiring resources, and two illustrating the need for space technology in order to combat extraterrestrial threats such as meteorite impacts. Three sorts of objections have been raised against this obligation. The first are objections alleging that supporting space exploration is impractical. The second is the (...) widely held notion that space exploration and environmentalism are at odds with one another. Finally, there are two objections to using space resources that Robert Sparrow has raised on the topic of terraforming. The obligation to support space exploration can be defended in at least three ways: (1) the "argument from resources," that space exploration is useful for amplifying our available resources; (2) the "argument from asteroids," that space exploration is necessary for protecting the environment and its inhabitants from extraterrestrial threats such as meteorite impacts; and (3) the "argument from solar burnout," that we are obligated to pursue interstellar colonization in order to ensure long-term human survival. (shrink)

A small but growing number of studies have aimed to understand, assess and reduce existential risks, or risks that threaten the continued existence of mankind. However, most attention has been focused on known and tangible risks. This paper proposes a heuristic for reducing the risk of black swan extinction events. These events are, as the name suggests, stochastic and unforeseen when they happen. Decision theory based on a fixed model of possible outcomes cannot properly deal with this kind of event. (...) Neither can probabilistic risk analysis. This paper will argue that the approach that is referred to as engineering safety could be applied to reducing the risk from black swan extinction events. It will also propose a conceptual sketch of how such a strategy may be implemented: isolated, self-sufficient, and continuously manned underground refuges. Some characteristics of such refuges are also described, in particular the psychosocial aspects. Furthermore, it is argued that this implementation of the engineering safety strategy safety barriers would be effective and plausible and could reduce the risk of an extinction event in a wide range of possible scenarios. Considering the staggering opportunity cost of an existential catastrophe, such strategies ought to be explored more vigorously. (shrink)

The view that the obligation to promote happiness is, as Popper puts it, "in any case much less urgent" than the obligation to eliminate unhappiness we might call the "Negative Doctrine". I know of no plausible form of the Negative Doctrine.

This paper defends a qualified version of moral vegetarianism. It defends a weak thesis and, more tentatively, a strong thesis, both from a very broad basis that assumes neither that animals have rights nor that they are entitled to equal consideration. The essay's only assumption about moral status, an assumption defended in the analysis of the wrongness of cruelty to animals, is that sentient animals have at least some moral status. One need not be a strong champion of animal protection, (...) then, to embrace moral vegetarianism. One need only assume some reasonable view of animals' moral status. (shrink)

This paper defends a qualified version of moral vegetarianism. It defends a weak thesis and, more tentatively, a strong thesis, both from a very broad basis that assumes neither that animals have rights nor that they are entitled to equal consideration. The essay's only assumption about moral status, an assumption defended in the analysis of the wrongness of cruelty to animals, is that sentient animals have at least some moral status. One need not be a strong champion of animal protection, (...) then, to embrace moral vegetarianism. One need only assume some reasonable view of animals' moral status. (shrink)

Cognitive enhancement takes many and diverse forms. Various methods of cognitive enhancement have implications for the near future. At the same time, these technologies raise a range of ethical issues. For example, they interact with notions of authenticity, the good life, and the role of medicine in our lives. Present and anticipated methods for cognitive enhancement also create challenges for public policy and regulation.

This paper argues that at least one of the following propositions is true: the human species is very likely to go extinct before reaching a "posthuman" stage; any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history ; we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor-simulations is false, unless we are currently (...) living in a simulation. A number of other consequences of this result are also discussed. (shrink)

I argue that at least one of the following propositions is true: the human species is very likely to become extinct before reaching a ’posthuman’ stage; any posthuman civilization is extremely unlikely to run a significant number of simulations of its evolutionary history ; we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we shall one day become posthumans who run ancestor-simulations is false, unless we are currently living (...) in a simulation. I discuss some consequences of this result. (shrink)

A recent exchange on Benatar’s book Better never to have been between Doyal and Benatar discusses Benatar’s bold claim that people should not be brought into existence. Here, I expand the discussion of original position that the exchange focused on. I also discuss the asymmetries, between benefit and harm and between existence and non-existence, upon which Benatar’s bold claim rests. In both discussions, I show how Benatar’s bold claim can be rejected.

A search is under way for a theory that can accommodate our intuitions in population axiology. The object of this search has proved elusive. This is not surprising since, as we shall see, any welfarist axiology that satisfies three reasonable conditions implies at least one of three counter-intuitive conclusions. I shall start by pointing out the failures in three recent attempts to construct an acceptable population axiology. I shall then present an impossibility theorem and conclude with a short discussion of (...) how it might be extended to pluralist axiologies, that is, axiologies that take more values than welfare into account. (shrink)

This chapter presents the text of a lecture on the Industrial Revolution in Great Britain given at the British Academy's 2009 Keynes Lecture in Economics. This text suggests that the Industrial Revolution was Britain's response to the global economy that emerged after 1500 and that Britain's success in world trade resulted in one of the most urbanised economies in Europe with unusually high wages and cheap energy prices. The text here also highlights the contribution of Britain in the invention of (...) the steam engine and the cotton spinning machines and in scientific discoveries relating to atmospheric pressure. (shrink)

Many researchers have argued that humanity will create artificial general intelligence (AGI) within the next twenty to one hundred years. It has been suggested that AGI may inflict serious damage to human well-being on a global scale ('catastrophic risk'). After summarizing the arguments for why AGI may pose such a risk, we review the fieldʼs proposed responses to AGI risk. We consider societal proposals, proposals for external constraints on AGI behaviors and proposals for creating AGIs that are safe due to (...) their internal design. (shrink)

In this century a number of events could extinguish humanity. The probability of these events may be very low, but the expected value of preventing them could be high, as it represents the value of all future human lives. We review the challenges to studying human extinction risks and, by way of example, estimate the cost effectiveness of preventing extinction-level asteroid impacts.

The assumption that environmental ethics must be nonanthropocentric in order to be adequate is mistaken. There are two forms of anthropocentrism, weak and strong, and weak anthropocentrism is adequate to support an environmental ethic. Environmental ethics is, however, distinctive vis-a-vis standard British and American ethical systems because, in order to be adequate, it must be nonindividualistic.Environmental ethics involves decisions on two levels, one kind of which differs from usual decisions affecting individual fairness while the other does not. The latter, called (...) allocational decisions, are not reducible to the former and govern the use of resources across extended time. Weak anthropocentrism provides a basis for criticizing individual, consumptive needs and can provide the basis for adjudicatingbetween these levels, thereby providing an adequate basis for environmental ethics without the questionable ontological commitments made by nonanthropocentrists in attributing intrinsie value to nature. (shrink)

Recently many methods for reducing the risk of human extinction have been suggested, including building refuges underground and in space. Here we will discuss the perspective of using military nuclear submarines or their derivatives to ensure the survival of a small portion of humanity who will be able to rebuild human civilization after a large catastrophe. We will show that it is a very cost-effective way to build refuges, and viable solutions exist for various budgets and timeframes. Nuclear submarines are (...) surface independent, and could provide energy, oxygen, fresh water and perhaps even food for their inhabitants for years. They are able to withstand close nuclear explosions and radiation. They are able to maintain isolation from biological attacks and most known weapons. They already exist and need only small adaptation to be used as refuges. But building refuges is only “Plan B” of existential risk preparation; it is better to eliminate such risks than try to survive them. (shrink)

The Matrix presents a version of an old philosophical fable: the brain in a vat. A disembodied brain is floating in a vat, inside a scientist’s laboratory. The scientist has arranged that the brain will be stimulated with the same sort of inputs that a normal embodied brain receives. To do this, the brain is connected to a giant computer simulation of a world. The simulation determines which inputs the brain receives. When the brain produces outputs, these are fed back (...) into the simulation. The internal state of the brain is just like that of a normal brain, despite the fact that it lacks a body. From the brain’s point of view, things seem very much as they seem to you and me. (shrink)

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk, often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent AI both contributes to (...) existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks. (shrink)

What kinds of fundamental limits are there in how capable artificial intelligence (AI) systems might become? Two questions in particular are of interest: (1) How much more capable could AI become relative to humans, and (2) how easily could superhuman capability be acquired? To answer these questions, we will consider the literature on human expertise and intelligence, discuss its relevance for AI, and consider how AI could improve on humans in two major aspects of thought and expertise, namely simulation and (...) pattern recognition. We find that although there are very real limits to prediction, it seems like AI could still substantially improve on human intelligence. (shrink)

Computers have far to go to match human strengths, and our estimates will depend on analogy and extrapolation. Fortunately, these are grounded in the first bit of the journey, now behind us. Thirty years of computer vision reveals that 1 MIPS can extract simple features from real-time imagery--tracking a white line or a white spot on a mottled background. 10 MIPS can follow complex gray-scale patches--as smart bombs, cruise missiles and early self-driving vans attest. 100 MIPS can follow moderately unpredictable (...) features like roads--as recent long NAVLAB trips demonstrate. 1,000 MIPS will be adequate for coarse-grained three-dimensional spatial awareness--illustrated by several mid-resolution stereoscopic vision programs, including my own. 10,000 MIPS can find three-dimensional objects in clutter--suggested by several "bin-picking" and high-resolution stereo-vision demonstrations, which accomplish the task in an hour or so at 10 MIPS. The data fades there--research careers are too short, and computer memories too small, for significantly more elaborate experiments. (shrink)

Humans have slowly built more productive societies by slowly acquiring various kinds of capital, and by carefully matching them to each other. Because disruptions can disturb this careful matching, and discourage social coordination, large disruptions can cause a “social collapse,” i.e., a reduction in productivity out of proportion to the disruption. For many types of disasters, severity seems to follow a power law distribution. For some of types, such as wars and earthquakes, most of the expected harm is predicted to (...) occur in extreme events, which kill most people on Earth. So if we are willing to worry about any war or earthquake, we should worry especially about extreme versions. If individuals varied little in their resistance to such disruptions, events a little stronger than extreme ones would eliminate humanity, and our only hope would be to prevent such events. If individuals vary a lot in their resistance, however, then it may pay to increase the variance in such resistance, such as by creating special sanctuaries from which the few remaining humans could rebuild society. (shrink)