The coronavirus pandemic, like its predecessors - AIDS, Ebola, etc., is evidence of the evolutionary instability of the socio-cultural and ecological niche created by mankind, as the main factor in the evolutionary success of our biological species and the civilization created by it. At least, this applies to the modern global civilization, which is called technogenic or technological, although it exists in several varieties. As we hope to show, the current crisis has less ontological as well as epistemological (...) roots; its reason lies in the main evolutionary trends in the development of science as a social institution. It was only later that epistemological factors were transformed into existential-ontological factors associated with the asymmetry of the existence of civilization and our biosocial nature. The perception or ignorance of a risk factor as a real fact is determined by the presence or absence of knowledge about it. In other words, risk is the result of the integration of the corresponding ontological concept into the general categorical structure. The plurality of such structures is a hallmark of multidisciplinary ontologies, each of which is associated with its own factual continuum. The aim of this study was to conceptually model for support relativistic parameter of evolutionary efficiency of stable evolutionary human strategy (SESH) in its techno-rationalistic module. The meaning of this term is equivalent to the category of scientific and technological development. (shrink)

From Chernobyl to Fukushima, it became clear that the technology is a system evolutionary factor, and the consequences of man-made disasters, as the actualization of risk related to changes in the social heredity (cultural transmission) elements. The uniqueness of the human phenomenon is a characteristic of the system arising out of the nonlinear interaction of biological, cultural and techno-rationalistic adaptive modules. Distribution emerging adaptive innovation within each module is in accordance with the two algorithms that are characterized by (...) the dominance of vertical (transgenerational) and horizontal (infection, contagion) adaptive streams of information, respectively. Evolutionary risk is the result of an imbalance of autonomous adaptive systems have an essential attribute of adaptibe strategy of Homo. Technological civilization inherent predisposition to overcome their dependence on biological and physical components. This feature serves as an enhancer of the evolutionary generating conjugate with the scientific and technological development risk We can assume the existence of an intention of Western mentality to a high priority (positive or negative) of technological modifications micro-social environment and post- Soviet (East Slavic) mentality to modification of macro-social system. (shrink)

Sources of evolutionary risk for stable strategy of adaptive Homo sapiens are an imbalance of: (1) the intra-genomic co-evolution (intragenomic conflicts); (2) the gene-cultural co-evolution; (3) inter-cultural co-evolution; (4) techno-humanitarian balance; (5) inter-technological conflicts (technological traps). At least phenomenologically the components of the evolutionary risk are reversible, but in the aggregate they are in potentio irreversible destructive ones for biosocial, and cultural self-identity of Homo sapiens. When the actual evolution is the subject of a rationalist control (...) and/or manipulation, the magnitude of the 4th and 5th components of the evolutionary risk reaches a level of existential significance. (shrink)

Attempt of trans-disciplinary analysis of the evolutionary value of bioethics is realized. Currently, there are High Tech schemes for management and control of genetic, socio-cultural and mental evolution of Homo sapiens (NBIC, High Hume, etc.). The biological, socio-cultural and technological factors are included in the fabric of modern theories and technologies of social and political control and manipulation. However, the basic philosophical and ideological systems of modern civilization formed mainly in the 17–18 centuries and are experiencing ever-increasing and destabilizing (...) risk-taking pressure from the scientific theories and technological realities. The sequence of diagnostic signs of a new era once again split into technological and natural sciences’ from one hand, and humanitarian and anthropological sciences’, from other. The natural sciences series corresponds to a system of technological risks be solved using algorithms established safety procedures. The socio-humanitarian series presented anthropological risk. Global bioethics phenomenon is regarded as systemic socio-cultural adaptation for technology-driven human evolution. The conceptual model for meta-structure of stable evolutionary strategy of Homo sapiens (SESH) is proposes. In accordance to model, SESH composed of genetic, socio-cultural and techno-rationalist modules, and global bioethics as a tool to minimize existential evolutionary risk. An existence of objectively descriptive and value-teleological evolutionary trajectory parameters of humanity in the modern technological and civilizational context (1), and the genesis of global bioethics as a system social adaptation to ensure self-identity (2) are postulated. -/- . (shrink)

Purpose of the present work is to attempt to give a glance at the problem of existential and anthropological risk caused by the contemporary man-made civilization from the perspective of comparison and confrontation of aesthetics, the substrate of which is emotional and metaphorical interpretation of individual subjective values and politics feeding by objectively rational interests of social groups. In both cases there is some semantic gap present between the represented social reality and its representation in perception of works (...) of art and in the political doctrines as well. Methodology of the research is evolutionary anthropologicalcomparativistics. Originality of the conducted analysis comes to the following: As the antithesis to biological and social reductionism in interpretation of the phenomenon of bio-power it is proposed a co-evolutionary semantic model in accordance with which the described semantic gap is of the substantial nature related to the complex module organization of a consistent and adaptive human strategy consisting of three associated but independently functional modules. Evolutionary trajectory of all anthropogenesis components including civilization cultural and social-political evolution is identified by the proportion between two macro variables – evolutionary effectiveness and evolutionary stability, i.e. preservation in the context of consequential transformations of some invariants of Homo sapiens species specificity organization. It should be noted that inasmuch as in respect to human, some modules of the evolutionary strategy assume self-reflection attributes, it would be more correctly to state about evolutionary correctness, i.e. correspondence to some perfection. As a result, the future of human nature depends not only on the rationalist principles of ethics of Homo species, but also on the holistic and emotionally aesthetic image of «Self». In conclusion it should be noted that there is a causal link between the development of High Hume technologies and the totality of the trend in the anthropological phenomenon of bio-power that permeates all the available human existence in modern civilization. As a result, there is a transformation of a contemporary social risk in the evolutionary civilization risk. (shrink)

The standard argument to the conclusion that artificial intelligence (AI) constitutes an existential risk for the human species uses two premises: (1) AI may reach superintelligent levels, at which point we humans lose control (the ‘singularity claim’); (2) Any level of intelligence can go along with any goal (the ‘orthogonality thesis’). We find that the singularity claim requires a notion of ‘general intelligence’, while the orthogonality thesis requires a notion of ‘instrumental intelligence’. If this interpretation is correct, they (...) cannot be joined as premises and the argument for the existential risk of AI turns out invalid. If the interpretation is incorrect and both premises use the same notion of intelligence, then at least one of the premises is false and the orthogonality thesis remains itself orthogonal to the argument to existential risk from AI. In either case, the standard argument for existential risk from AI is not sound.—Having said that, there remains a risk of instrumental AI to cause very significant damage if designed or used badly, though this is not due to superintelligence or a singularity. (shrink)

A small but growing number of studies have aimed to understand, assess and reduce existential risks, or risks that threaten the continued existence of mankind. However, most attention has been focused on known and tangible risks. This paper proposes a heuristic for reducing the risk of black swan extinction events. These events are, as the name suggests, stochastic and unforeseen when they happen. Decision theory based on a fixed model of possible outcomes cannot properly deal with this kind (...) of event. Neither can probabilistic risk analysis. This paper will argue that the approach that is referred to as engineering safety could be applied to reducing the risk from black swan extinction events. It will also propose a conceptual sketch of how such a strategy may be implemented: isolated, self-sufficient, and continuously manned underground refuges. Some characteristics of such refuges are also described, in particular the psychosocial aspects. Furthermore, it is argued that this implementation of the engineering safety strategy safety barriers would be effective and plausible and could reduce the risk of an extinction event in a wide range of possible scenarios. Considering the staggering opportunity cost of an existential catastrophe, such strategies ought to be explored more vigorously. (shrink)

Alan Turing, one of the fathers of computing, warned that Artificial Intelligence (AI) could one day pose an existential risk to humanity. Today, recent advancements in the field AI have been accompanied by a renewed set of existential warnings. But what exactly constitutes an existential risk? And how exactly does AI pose such a threat? In this chapter we aim to answer these questions. In particular, we will critically explore three commonly cited reasons for thinking (...) that AI poses an existential threat to humanity: the control problem, the possibility of global disruption from an AI race dynamic, and the weaponization of AI. (shrink)

I propose a novel model of the human ego (which I define as the tendency to measure one’s value based on extrinsic success rather than intrinsic aptitude or ability). I further propose the conjecture that ego so defined both is a non-adaptive by-product of evolutionary pressures, and has some evolutionary value as an adaptation (protecting self-interest). I explore ramifications of this model, including how it mediates individuals’ reactions to perceived and actual limits of their power, their ability to (...) cope with risk and uncertainty, and how this model may interpolate between rational choice models and cognitive psychology. I develop numerous examples and applications, including poverty traps, to demonstrate the model’s predictive power to elucidate a broad range of social phenomena. -/- [December 2018: Updated version to submit for publication. Expanded Sections 4 and 5.1, revised Section 5.7] -/- . (shrink)

Ian Stoner has recently argued that we ought not to colonize Mars because doing so would flout our pro tanto obligation not to violate the principle of scientific conservation, and there is no countervailing considerations that render our violation of the principle permissible. While I remain agnostic on, my primary goal in this article is to challenge : there are countervailing considerations that render our violation of the principle permissible. As such, Stoner has failed to establish that we ought not (...) to colonize Mars. I close with some thoughts on what it would take to show that we do have an obligation to colonize Mars and related issues concerning the relationship between the way we discount our preferences over time and projects with long time horizons, like space colonization. (shrink)

Stable adaptive strategy of Homo sapiens (SESH) is a superposition of three different adaptive data arrays: biological, socio-cultural and technological modules, based on three independent processes of generation and replication of an adaptive information – genetic, socio-cultural and symbolic transmissions (inheritance). Third component SESH focused equally to the adaptive transformation of the environment and carrier of SESH. With the advent of High Hume technology, risk has reached the existential significance level. The existential level of technical risk (...) is, by definition, an evolutionary risk as possible leads to the genesis of disappearance of humanity as a species. The emergence of bioethics has to consider as a form of modern (transdisciplinary) scientific concept and sociocultural adaptation for regulate human identity in the global-evolutionary transformation and performs the function of self-preservation. (shrink)

The monograph is an English, expanded and revised version of the book Cheshko, V. T., Ivanitskaya, L.V., & Glazko, V.I. (2018). Anthropocene. Philosophy of Biotechnology. Moscow, Course. The manuscript was completed by me on November 15, 2019. It is a study devoted to the development of the concept of a stable evolutionary human strategy as a unique phenomenon of global evolution. The name “An Evolutionary Metaphysics (Cheshko, 2012; Glazko et al., 2016). With equal rights, this study could be (...) entitled “Biotechnology as a result and factor of the evolutionary processˮ. The choice in favor of used “The Evolutionary Metaphysics of Human Enhancement Technologiesˮ was made in accordance with the basic principle of modern post-academician and human-sized science, a classic example of which is biotechnology. The “Metaphysics of Evolution” and “Evolutionary Metaphysics” concepts are used in several ways in modern philosophical discourse. In any case, the values contain a logical or associative reference to the teleological nature of the evolutionary process (Hull, 1967, 1989; Apel, 1995; Faye, 2016; Dupre, 2017; Rose, 2018, etc). In our study, the “evolutionary metaphysics” serves to denote the thesis of the rationalization and technologization of global evolution and anthropogenesis, in particular. At the same time, the postulate of an open future remains relevant in relation to the results of the evolutionary process. The theory of evolution of complex, including the humans system and algorithm for its constructing are а synthesis of evolutionary epistemology, philosophical anthropology and concrete scientific empirical basis in modern science. ln other words, natural philosophy is regaining the status bar element theoretical science in the era of technology-driven evolution. The co-evolutionary concept of 3-modal stable evolutionary strategy of Homo sapiens is developed. The concept based оn the principle of evolutionary complementarity of anthropogenesis: value of evolutionary risk and evolutionary path of human evolution are defined bу descriptive (evolutionary efficiency) and creative-teleological (evolutionary correctness) parameters simultaneously, that cannot bе instrumental reduced to others ones. Resulting volume of both parameters define the vectors of blological, social, cultural and techno-rationalistic human evolution Ьу two gear mechanism genetic and cultural co-evolution and techno-humanitarian balance. The resultant each of them сап estimated Ьу the ratio of socio-psychological predispositions of humanization / dehumanization in mentality. Explanatory model and methodology of evaluation of creatively teleological evolutionary risk component of NBIC technological complex is proposed. Integral part of the model is evolutionary semantics (time-varying semantic code, the compliance of the blological, socio-cultural and techno-rationalist adaptive modules of human stable evolutionary strategy). It is seem necessary to make three clarifications. First, logical construct, “evolutionary metaphysics” contains an internal contradiction, because it unites two alternative explanatory models. “Metaphysics”, as a subject, implies deducibility of the process from the initial general abstract principle, and, consequently, the outcome of the development of the object is uniquely determined by the initial conditions. Predicate, “evolutionary”, means stochastic mechanism of realizing the same principle by memorizing and replicating random choices in all variants of the post-Darwin paradigm. In philosophy, random choice corresponds to the category of “free will” of a reasonable agent. In evolutionary theory, the same phenomenon is reflected in the concept of “covariant replication”. Authors will attempt to synthesize both of these models in a single transdisciplinary theoretical framework. Secondly, the interpretation of the term “evolutionary (adaptive) strategyˮ is different from the classical definition. The difference is that the adaptive strategy in this context is equivalent to the survival, i.e. it includes the adaptation to the environment and the transformation (construction) of the medium in accordance with the objectives of survival. To emphasize this difference authors used verbal construction “adaptiveˮ (rather than “evolutionaryˮ) strategy as more adequate. In all other cases, the two terms may be regarded as synonymous. Thirdly, the initial two essays of this series were published in one book in 2012. Their main goal was the development of the logically consistent methodological concept of stable adaptive (evolutionary) strategy of hominines and the argumentation of its heuristic possibilities as a transdisciplinary scientific paradigm of modern anthropology. The task was to demonstrate the possibilities of the SESH concept in describing and explaining the evolutionary prospects for the interaction of social organization and technology (techno-humanitarian balance) and the associated biological and cultural mechanisms of the genesis of religion (gene-cultural co-evolution). In other words, it was related to the sphere of cultural and philosophical anthropology, i.e. to the axiological component of any theoretical constructions describing the behavior of self-organizing systems with human participation. In contrast, the present work is an attempt to introduce this concept into the sphere of biological anthropology and, consequently, its main goal is to demonstrate the possibility of verification of its main provisions by means of procedures developed by natural science, i.e. refers to the descriptive component of the same theoretical constructions. The result of this in the future should be methods for assessing, calculating and predicting the risk of loss of biological and cultural identity of a person, associated with a permanent and continuously deepening process of development of science and technology. (shrink)

In Chapter 1 I discuss the basic problem which made me undertake the issue of human death. That problem was the dualism in the depiction of human nature which has not been fully overcome yet, the dualism which leads to the emergence of new difficulties in contemporary attempts at adequately solving the problem of human death. They include the separation of soul from the body in the moment of death, and the borderline between the moment of death and the moment (...) of resurrection. Chapter 2 is devoted to the presentation of a new anthropological perspective, of an anthropology build “from the bottom up”, that is “from foundations”, and taking into account man’s rooting in the biological environment, and thus also in the genetic heritage taken over from our pre-human ancestors. That anthropology brings new aspects into traditional anthropology by extending it to include new ways of explaining the genesis of the human body and spirit, and new depictions of man’s death. In Chapter 2 I will also present the concept of evolution proposed by P.T. de Chardin, which I appreciate for its comprehensive evolutionary perception of the world. This concept has provoked various responses, but it certainly provides a creative inspiration for further attempts at applying the idea of evolution to various issues. The general framework of the evolutionary model proposed by the French researcher turns out to be useful also when applied to the evolutionary understanding of human death. Chapter 3 contains a discussion of the principal premises of the evolutionary concept of human death taken from analyses carried out by T. Wojciechowski concerning the issue of time, space, and the structure of variable material beings. A being is variable because it has a temporal structure, and the changes that occur are the derivatives of that structure. The temporal structure may be seen as endless, in which a being, even though its existence had a beginning, will exist for ever. Material substance should not be seen statically, but in dynamic and evolutionary terms. In evolution, time becomes a driver of development, an element shaping and directing the evolution of the entire cosmos. Man’s substance has been made capable of two levels of being and acting: the spatial and temporal level, and the endless and immanent level. Together, they form the one essence of man. Thus, the human person is partially immersed in the material cosmos, and partially elevated above it. Chapter 4 presents the evolutionary approach to the moment of man’s death. The views of T. Wojciechowski will be presented in which the moment of death is seen as a moment of transformation and perfection of the structure of the essence of human existence. Furthermore, the issue of the relationship existing, according to T. Wojciechowski, between man’s death and resurrection will be discussed. Understood in evolutionary terms, the moment of death is the time when the structure of human existence is transformed. Death transfers man fully into eternity and enables his life on a higher level of existential being. The element of flesh is retained, but changes its nature. Such view of death allows it to be understood as the moment of resurrection and the culmination of the evolutionary process of life. Death is therefore an “invention” of the evolution and should be seen as a natural and positive phenomenon. At this point, I also propose a solution to the question about reconciliation between temporal and endless existence. In Chapter 5 I take up the issue of consequences flowing from the postulated understanding of human death. They have been pointed to by T. Wojciechowski, but need to be further developed and clarified. He believed two lines in man’s development need to be recognized: that of natural and that of supernatural evolution. Both have the same direction, while occurring on different planes. In each of these lines, death is a natural element of life, pre-planned and, so to say, “encoded” in the cosmic matter. Therefore, the treatment of biological death as the consequence of sin is not only contrary to the evolutionary understanding of life, but in addition artificially combines that which is natural with that which is supernatural. Man’s authentic experience of his own present is a condition of his achieving full existential perfection in the moment of this death. Full engagement in the present is, in a way, a harbinger of the future; thus, the life “here” and “there” may not be inherently inconsistent. In the light of the evolutionary concept, death is not a punishment, but the most positive experience of man’s life in that it enables him to achieve full existential perfection. Chapter 6 is devoted to a substantiation of the possibility of evolutionary transformation of the structure of human existence in the moment of death. An adequate cause needs to be identified which could substantiate that transformation and achievement of perfection. I believe an appropriate substantiation can be found in God’s creative act, interpreted in evolutionary terms. The process of evolution, with its subsequent stages, is only a necessary condition for that “transformation” and “elevation”. The sufficient condition is fulfilled by the continuous creative intervention of God. The concept of God’s creative activity in the moment of man’s death finds its continuation in Chapter 7, where I present a detailed analysis of God’s activity at the time of man’s death. The concept of God’s creative activity extended over the entire process of evolution makes it possible to consider the moment of death as the moment of man’s resurrection. The moment of death is thus a moment of creation in evolutionary terms. Thus, as long as man is related to temporal and spatial elements, his development, or creation, continues. Death is only the completion of the final stage of creation, which may be painful and evoke fear, but in fact should give hope, for it involves resurrection. I also propose that the creative act in the moment of man’s death-resurrection be considered a basic action of God, which explains the manner in which God’s activity is contained within the framework of the creative act, encompassing also the moment of death-resurrection. The final Chapter 8 describes a didactic model illustrating the concept proposed here. I use the analogy of white light dispersion effect in the prism, and compare the incident ray to the creative act, and the dispersed rays going through the prism to the line of man’s natural and supernatural evolution. This model also shows the difference in the perspectives from which the phenomenon of death is seen by God and by man. (shrink)

Recent advances in natural language processing have given rise to a new kind of AI architecture: the language agent. By repeatedly calling an LLM to perform a variety of cognitive tasks, language agents are able to function autonomously to pursue goals specified in natural language and stored in a human-readable format. Because of their architecture, language agents exhibit behavior that is predictable according to the laws of folk psychology: they function as though they have desires and beliefs, and then make (...) and update plans to pursue their desires given their beliefs. We argue that the rise of language agents significantly reduces the probability of an existential catastrophe due to loss of control over an AGI. This is because the probability of such an existential catastrophe is proportional to the difficulty of aligning AGI systems, and language agents significantly reduce that difficulty. In particular, language agents help to resolve three important issues related to aligning AIs: reward misspecification, goal misgeneralization, and uninterpretability. (shrink)

Stable evolutionary strategy of Homo sapiens (SESH) is built in accordance with the modular and hierarchical principle and consists of the same type of self-replicating elements, i.e. is a system of systems. On the top level of the organization of SESH is the superposition of genetic, social, cultural and techno-rationalistic complexes. The components of this triad differ in the mechanism of cycles of generation - replication - transmission - fixing/elimination of adoptively relevant information. This mechanism is implemented either in (...) accordance with the Darwin-Weismann modus, or according to the Lamarck modus, the difference between them is clear from the title. The integral attribute of the system of systems including ESSH is the production of evolutionary risks. The sources of evolutionary risk for stable adaptive strategy of Homo sapiens are the imbalance of (1) the intra-genomic co-evolution (intragenomic conflicts); (2) the gene-cultural coevolution; (3) the inter-cultural co-evolution; (4) techno-humanitarian balance; (5) intertechnological conflicts (technological traps). At least phenomenologically the components of the evolutionary risk are reversible, but in the aggregate they are in potentio irreversible destructive ones for bio-social, and cultural self-identity of Homo sapiens. When the actual evolution is the subject of a rationalist control and/or manipulation, the magnitude of the 4th and 5th components of the evolutionary risk reaches the level of existential significance. (shrink)

Stable evolutionary strategy of Homo sapiens (SESH) is built in accordance with the modular and hierarchical principle and consists of the same type of self-replicating elements, i.e. is a system of systems. On the top level of the organization of SESH is the superposition of genetic, social, cultural and techno-rationalistic complexes. The components of this triad differ in the mechanism of cycles of generation - replication - transmission - fixing/elimination of adoptively relevant information. This mechanism is implemented either in (...) accordance with the Darwin-Weismann modus, or according to the Lamarck modus, the difference between them is clear from the title. The integral attribute of the system of systems including ESSH is the production of evolutionary risks. The sources of evolutionary risk for stable adaptive strategy of Homo sapiens are the imbalance of (1) the intra-genomic co-evolution (intragenomic conflicts); (2) the gene-cultural co- evolution; (3) the inter-cultural co-evolution; (4) techno-humanitarian balance; (5) inter- technological conflicts (technological traps). At least phenomenologically the components of the evolutionary risk are reversible, but in the aggregate they are in potentio irreversible destructive ones for bio-social, and cultural self-identity of Homo sapiens. When the actual evolution is the subject of a rationalist control and/or manipulation, the magnitude of the 4th and 5th components of the evolutionary risk reaches the level of existential significance. (shrink)

Purpose (metatask) of the present work is to attempt to give a glance at the problem of existential and anthropo- logical risk caused by the contemporary man-made civilization from the perspective of comparison and confronta- tion of aesthetics, the substrate of which is emotional and metaphorical interpretation of individual subjective values and politics feeding by objectively rational interests of social groups. In both cases there is some semantic gap pre- sent between the represented social reality and its representation (...) in perception of works of art and in the political doctrines as well. Methodology of the research is evolutionary anthropological comparativistics. Originality of the conducted analysis comes to the following: As the antithesis to biological and social reductionism in interpretation of the phenomenon of bio-power it is proposed a co-evolutionary semantic model in accordance with which the de- scribed semantic gap is of the substantial nature related to the complex module organization of a consistent and adaptive human strategy consisting of three associated but independently functional modules (genetic, cultural and techno-rational). Evolutionary trajectory of all anthropogenesis components including civilization cultural and so- cial-political evolution is identified by the proportion between two macro variables – evolutionary effectiveness and evolutionary stability (sameness), i.e. preservation in the context of consequential transformations of some invari- ants of Homo sapiens species specificity organization. It should be noted that inasmuch as in respect to human, some modules of the evolutionary (adaptive) strategy assume self-reflection attributes, it would be more correctly to state about evolutionary correctness, i.e. correspondence to some perfection. As a result, the future of human nature de- pends not only on the rationalist principles of ethics of Homo species (the archaism of Jurgen Habermas), but also on the holistic and emotionally aesthetic image of «Self». In conclusion it should be noted that there is a causal link between the development of High Hume (NBIC) technologies and the totality of the trend in the anthropological phenomenon of bio-power that permeates all the available human existence in modern civilization. As a result, there is a transformation of a contemporary social (man-made) risk in the evolutionary civilization risk. (shrink)

In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart’s Law as “Virtually any goal specification, pursued to the extreme, will result in the extinction of humanity”, and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart’s Law holds or not. As our key contribution, we identify a set of conditions that are necessary for a model that aims to (...) be informative for evaluating specific arguments for Extinction-level Goodhart’s Law. Since each of the conditions seems to significantly contribute to the complexity of the resulting model, formally evaluating the hypothesis might be exceedingly difficult. This raises the possibility that whether the risk of extinction from artificial intelligence is real or not, the underlying dynamics might be invisible to current scientific methods. (shrink)

Climate change is an existential risk reinforced by ordinary actions in afuent societies—often silently present in comfortable and enjoyable habits. This silence is sometimes broken, presenting itself as a nagging reminder of how our habits fuel a catastrophe. As a case in point, global warming has created a state of urgency among wine makers in Spain, as the alcohol level has risen to a point where it jeopardises wine quality and thereby Spanish viticulture. Eforts are currently being made (...) to solve this problem technologically by developing (non-GMO) strains of yeast that produce less alcohol. If successful, this could help save Spanish viticulture. This kind of technological solution is routinely criticised for ofering simplistic “techno-fx” solutions to solve complex societal problems. However, it is not clear what features are criticised by the notion of techno-fx and hence how to avoid this criticism. In our interpretation, the techno-fx notion is not exclusively used as a dismissive term. The notion points to a general virtue of engineering: to create technological solutions that work silently in the background. This asset of technological solutions might sometimes be problematic and sometimes not. Hence, it needs to be analysed case-by-case by paying attention to the moral relevance of the hidden implications of the technology and of the unrefective actions and habits that they facilitate. Such moral analysis will in turn inform strategies for foregrounding this technology to counteract silencing. We discuss what this means in the case of modifying yeast as a viticultural climate resilience strategy. (shrink)

Failing to recognise sentience in AI systems (false negatives) poses a far greater risk of potentially astronomical suffering compared to mistakenly attributing sentience to non-sentient systems (false positives). This paper analyses the issue through the moral frameworks of longtermism, utilitarianism, and deontology, concluding that all three assign greater urgency to avoiding false negatives. Given the astronomical number of AIs that may exist in the future, even a small chance of overlooking sentience is an unacceptable risk. To address this, (...) the paper proposes a comprehensive approach including research, field-building, and tentative policy development. Humanity must take steps to ensure the well-being of all sentient minds, both biological and artificial. (shrink)

Papers from the conference on AI Risk (published in JETAI), supplemented by additional work. --- If the intelligence of artificial systems were to surpass that of humans, humanity would face significant risks. The time has come to consider these issues, and this consideration must include progress in artificial intelligence (AI) as much as insights from AI theory. -- Featuring contributions from leading experts and thinkers in artificial intelligence, Risks of Artificial Intelligence is the first volume of collected chapters dedicated (...) to examining the risks of AI. The book evaluates predictions of the future of AI, proposes ways to ensure that AI systems will be beneficial to humans, and then critically evaluates such proposals. 1 Vincent C. Müller, Editorial: Risks of Artificial Intelligence - 2 Steve Omohundro, Autonomous Technology and the Greater Human Good - 3 Stuart Armstrong, Kaj Sotala and Sean O’Heigeartaigh, The Errors, Insights and Lessons of Famous AI Predictions - and What they Mean for the Future - 4 Ted Goertzel, The Path to More General Artificial Intelligence - 5 Miles Brundage, Limitations and Risks of Machine Ethics - 6 Roman Yampolskiy, Utility Function Security in Artificially Intelligent Agents - 7 Ben Goertzel, GOLEM: Toward an AGI Meta-Architecture Enabling Both Goal Preservation and Radical Self-Improvement - 8 Alexey Potapov and Sergey Rodionov, Universal Empathy and Ethical Bias for Artificial General Intelligence - 9 András Kornai, Bounding the Impact of AGI - 10 Anders Sandberg, Ethics and Impact of Brain Emulations 11 Daniel Dewey, Long-Term Strategies for Ending Existential Risk from Fast Takeoff - 12 Mark Bishop, The Singularity, or How I Learned to Stop Worrying and Love AI -. (shrink)

Many global catastrophic and existential risks (X-risks) threaten the existence of humankind. There are also many ideas for their prevention, but the meta-problem is that these ideas are not structured. This lack of structure means it is not easy to choose the right plan(s) or to implement them in the correct order. I suggest using a “Plan A, Plan B” model, which has shown its effectiveness in planning actions in unpredictable environments. In this approach, Plan B is a backup (...) option, implemented if Plan A fails. In the case of global risks, Plan A is intended to prevent a catastrophe and Plan B to survive it, if it is not avoided. Each plan has similar stages: analysis, planning, funding, low-level realization and high-level realization. Two variables—plans and stages—provide an effective basis for classification of all possible X-risks prevention methods in the form of a two-dimensional map, allowing the choice of optimal paths for human survival. I have created a framework for estimating the utility of various prevention methods based on their probability of success, the chances that they will be realized in time, their opportunity cost, and their risk. I also distinguish between top-down and bottom-up approaches. (shrink)

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk, often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent AI (...) both contributes to existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks. (shrink)

The utopian character of modern scientific theories, with the human nature as a subject, is an inevitable consequence of the presence of an imperative component of transdisciplinary human dimensional scientific knowledge. Its social function is the adaptation of the descriptive component of the theory to the given socio-cultural type that simplifies the passage of the process of social verification of the theory. The genesis of bioethics can be seen as one of the basic premises for the actualization of the anthropic (...) principle of ontology, which thus acquires the axiological and epistemological sense. (shrink)

Humanity may underestimate the rate of natural global catastrophes because of the survival bias (“anthropic shadow”). But the resulting reduction of the Earth’s future habitability duration is not very large in most plausible cases (1-2 orders of magnitude) and thus it looks like we still have at least millions of years. However, anthropic shadow implies anthropic fragility: we are more likely to live in a world where a sterilizing catastrophe is long overdue and could be triggered by unexpectedly small human (...) actions. In the same way, an over-inflated toy balloon, which will soon burst, is very fragile. Anthropic fragility can manifest itself in the higher chances of runaway global warming. It has often been suggested that the Earth's atmosphere remained life-supporting for billions of years by sheer chance. Therefore, the survival bias can be strong. It is also known that Earth-like water worlds could experience transitions into deadly moisture greenhouse (mean T = 65C). This means that relatively small anthropogenic actions could put the climate above an unpredictable tipping point, which could lead to the moisture greenhouse. Thus, it is necessary to carry out urgent geoengineering studies and prepare to prevent an unexpected climate catastrophe. There are three main counterarguments against the existence of the anthropic shadow: self-indication assumption (SIA), past observers and the Gaia hypothesis; we show that they fail. It was proposed that SIA exactly compensates the anthropic shadow as an observer unlikely to find herself in a world with a strong anthropic shadow; however, there is a baseline level of the anthropic shadow for all habitable planets, similar to the rate of evolutionary transitions like abiogenesis. There are no “past observers” as qualified observers appeared only 50 years ago. Gaia hypothesis assumes existence of self-stabilizing feedback in climate, but new types of events like quick CO2 growth could override its coping ability. We present a list of other catastrophes that may have been underestimated because of the anthropic shadow, including collider catastrophes, nuclear war and even an alien invasion. We also hypothesized that human intelligence is more likely to emerge in an unstable world that is nearing its end and thus we get a new form of Doomsday argument. (shrink)

The Evolutionary Manifesto shows that evolution is directional and demonstrates that this has major implications for humanity. The Manifesto shows that humanity must align its social systems and behaviour with the trajectory of evolution if we are to survive and thrive into the future. The Manifesto goes on to demonstrate that humanity has an essential role to play in the future evolution of life on this planet. It demonstrates that life on Earth has reached a critical stage in evolution’s (...) trajectory. Up to now evolution has moved along its trajectory blindly and unconsciously. But if evolution on Earth is to continue to advance successfully in the future, the process will have to be driven intentionally and consciously by humanity. In this way, a proper understanding of evolution shows us how we can live a life that matters in a larger scheme of things. An evolutionary worldview answers the fundamental existential question that faces us all: “What should I do with my life?” The Manifesto outlines in broad terms what humanity must do to advance the evolutionary process—how we will intentionally have to re-make our political, social and economic systems, and how we will need to develop our cognition and consciousness. As such, the Manifesto presents a detailed case for evolutionary activism--coordinated action directed at advancing the evolutionary process intentionally. (shrink)

The co-evolutionary concept of tri-modal stable evolutionary strategy (SESH) of Homo sapiens is developed. The concept based on the principle of evolutionary complementarity of anthropogenesis: value of evolutionary risk and evolutionary path of human evolution are defined by descriptive (evolutionary efficiency) and creative-teleological (evolutionary correctness) parameters simultaneously, that cannot be instrumental reduced to others ones. Resulting volume of both parameters define the vectors of human evolution by two gear mechanism ˗ genetic and (...) cultural co-evolution and techno-humanitarian balance. Explanatory model and methodology of evaluation of creatively teleological evolutionary risk component of NBIC technological complex is proposed. The mechanism of influence of each module on the evolution of the two remaining modules of SESH a priori can be twofold: (1) informational co-evolution (direct selective pressure); (2) semantic co-evolution (time-varying semantic code, the compliance of the biological, socio- cultural and techno-rationalist adaptive modules of human stable evolutionary strategy). More rapidly evolving autonomous element of the co-evolutionary pair becomes making sense factor for the partner. Semantic co-evolution is the discrete acquisition of adaptive significance of individual alleles by changing socio-cultural types, manifested as an increase in the genetic variability of populations of Homo sapiens, and domesticated species parallel to socio-culturogenesis. The socio- cultural landscape leads to drastically difference of direction in biological evolution of Atlantic and East-Slavic ethnicities/civilizations. (shrink)

The guide explains the basic concepts of natural, social and evolutionary components and methods of risk management and control of modern biotechnologies stemming from the general theory of human evolution. A a transdisciplinary approach is a feature of the presentation of the material The risk is considered from the point of view of evolutionary anthropology, as a basic element of a stable evolutionary strategy of our biological species and the basis of a combination of natural-scientific (...) and humanitarian forms of presentation and interpretation of the material. CONTENT BIOSAFETY AND BIOETHICS IN BIOTECHNOLOGY (Introduction). Part 1. BIOTECHNOLOGY AS A FACTOR OF EVOLUTION AND A SOURCE OF TECHNOGENIC RISK. Chapter 1. GENERAL THEORY OF TECHNOLOGICAL RISK. Chapter 2. RISK AS AN ATTRIBUTE OF STABLE HUMAN EVOLUTIONARY STRATEGY. Chapter 3. BIOTECHNOLOGY AS A SOURCE OF BIORISKS Chapter 4. COMPONENTS OF THE "BIOLOGICAL FACTOR" OF RISK. Chapter 5. BIOLOGICAL WEAPONS and BIOTERRORISM. Part 2. BIOLOGICAL, SOCIAL AND EVOLUTIONARY COMPONENTS OF THE RISK OF MODERN BIOTECHNOLOGIES Chapter 6. RISK MANAGEMENT DURING BIOTECHNOLOGICAL RESEARCH.. Chapter 7. BIOTECHNOLOGY AS A SOURCE OF SOCIAL INSTABILITY. Chapter 8. BIOTECHNOLOGY AS A SOURCE OF CIVILIZATIONAL RISK. -/- . (shrink)

In this article, a classification of the global catastrophic risks connected with the possible existence (or non-existence) of extraterrestrial intelligence is presented. If there are no extra-terrestrial intelligences (ETIs) in our light cone, it either means that the Great Filter is behind us, and thus some kind of periodic sterilizing natural catastrophe, like a gamma-ray burst, should be given a higher probability estimate, or that the Great Filter is ahead of us, and thus a future global catastrophe is high probability. (...) If ETIs exist in our light cone, it is most likely they exist in the form of ET artificial general intelligence (ETAI). ETAI may be traveling at near-light speed as an explosive wave of intelligence in the form of some nano-mechanisms which could reach us at any moment; may send us potentially dangerous messages with AI-virus payloads; or be near or on Earth in some dormant form. This dormant form could be nanobots, which could act as berserkers, and be triggered when humans unwittingly cross some threshold. If ETIs are nearby and less advanced, they could find our messaging extra-terrestrial intelligence (METI) messages and send their “fleet” to or direct weapons toward Earth. Even extinct ETIs could have left dangerous remnants in the form of AI systems that our civilization may encounter. If our existence is a simulation, this could be run by ETIs. Even a false belief in the existence of ETIs could potentially fuel millennial sects. If humanity is the first intelligence to emerge, it may kill or prevent the existence of all future potential ETI, a different type of catastrophe. Several options to prevent catastrophic risks are connected with ETI: sending requests for help, using random strategies to escape the Fermi paradox, Great Filter prediction, or the hope that ETIs will find the remains of our extinct civilization and resurrect it. (shrink)

Abstract: Global chemical contamination is an underexplored source of global catastrophic risks that is estimated to have low a priori probability. However, events such as pollinating insects’ population decline and lowering of the human male sperm count hint at some toxic exposure accumulation and thus could be a global catastrophic risk event if not prevented by future medical advances. We identified several potentially dangerous sources of the global chemical contamination, which may happen now or could happen in the future: (...) autocatalytic reactions, exposure to multiple subthreshold sources, and long-term unintended consequences, arising from both natural and bioengineered sources. We list several especially dangerous chemicals—dioxin, organiс compounds, and toxic heavy metals. We also discuss the features of such dangerous chemicals—molecules that can stay in the biosphere for a long time and affect it over time. We explore several social processes and scenarios where global chemical contamination becomes possible: large natural catastrophe like meteorite impact, supervolcano eruption, new ways of predicting properties of the chemicals via machine learning and their manufacturing via synthetic biology, uncontrolled “capitalistic” economic development with a corresponding large waste production, quick adoption of many chemicals with unknown long-term properties and unintended side-effects. These are all low probability, so work on other global catastrophic risks should be prioritized, but chemical risks could exacerbate other types of catastrophe contributing to social collapse. (shrink)

Abstract: This article examines risks associated with the program of passive search for alien signals (Search for Extraterrestrial Intelligence, or SETI) connected with the possibility of finding of alien transmission which includes description of AI system aimed on self-replication (SETI-attack). A scenario of potential vulnerability is proposed as well as the reasons why the proportion of dangerous to harmless signals may be high. The article identifies necessary conditions for the feasibility and effectiveness of the SETI-attack: ETI existence, possibility of AI, (...) small size of the Seed AI, small speed of physical interstellar travel and large distance between civilizations. Needed additions to the SETI protocol are considered: keep the signal existence, its content and source location in secret, don’t run alien programs, jam dangerous signals, wait until creation of humanity’s own AI. In addition, scenarios in which it may be reasonable to start an alien AI (if it has been downloaded) are explored. (shrink)

The importance of addressing the existential threat to humanity, climate change, has grown remarkedly in recent years while conflicting views and interests in societies exist. Therefore, climate change agendas have been weaponized to varying degrees, ranging from the international level between countries to the domestic level among political parties. In such contexts, climate change agendas are predominantly driven by political or economic ambitions, sometimes unconnected to concerns for environmental sustainability. Consequently, it can result in an environment that fosters antagonism (...) and disputes over power and position and increases the risk of prolonged confrontations, hindering the collective global efforts to mitigate and adapt to climate change. Through the current discourse, we aim to provide a preliminary definition of the weaponization of climate change and environmental degradation and examine its risks and consequences on international relations, political dynamics, public perception, and the comprehensive integrity of climate action. We also recommend embracing a globally coordinated, scientifically substantiated approach to circumvent climate change by building an eco-surplus cultural value system. (shrink)

Many global catastrophic risks are threatening human civilization, and a number of ideas have been suggested for preventing or surviving them. However, if these interventions fail, society could preserve information about the human race and human DNA samples in the hopes that the next civilization on Earth will be able to reconstruct Homo sapiens and our culture. This requires information preservation of an order of magnitude of 100 million years, a little-explored topic thus far. It is important that a potential (...) future civilization will discover this information as early as possible, thus a beacon should accompany the message in order to increase visibility. The message should ideally contain information about how humanity was destroyed, perhaps including a continuous recording until the end. This could help the potential future civilization to survive. The best place for long-term data storage is under the surface of the Moon, with the beacon constructed as a complex geometric figure drawn by small craters or trenches around a central point. There are several cost-effective options for sending the message as opportunistic payloads on different planned landers. (shrink)

In this essay I seek to critically evaluate some forms of holism and organicism in biological thought, as a more deflationary echo to Gilbert and Sarkar's reflection on the need for an 'umbrella' concept to convey the new vitality of holistic concepts in biology (Gilbert and Sarkar 2000). Given that some recent discussions in theoretical biology call for an organism concept (from Moreno and Mossio’s work on organization to Kirschner et al.’s research paper in Cell, 2000, building on chemistry to (...) articulate what they called “molecular vitalism,” studying the “vitalistic” properties of molecular, cellular, and organismal function, and Pepper and Herron’s suggestion in their 2008 paper that organisms define a category that evolutionary biology cannot do without), the question, what concept of organicism are they calling for? To what extent are such claims philosophically committed to a non-naturalistic concept of organism as organizing centre, as a foundational rather than heuristic concept – or possibly a “biochauvinism,” to use Di Paolo’s term (Di Paolo 2009)? My aim in this paper is to conceptually clarify the forms of holism and organicism that are involved in these cases (and I acknowledge that the study of early 20th-century holisms [Peterson 2010] indicates that not all of them were in fact ‘organicist’ or ‘biologistic’). I suggest that contemporary holists are still potentially beholden to a certain kind of vitalism or “biochauvinism”; but that when they reduce their claims to mere heuristics, conversely, they risk losing sight of a certain kind of organizational “thickness”, a “vital materiality” (Wheeler 2010) which is characteristic of biological systems (Bechtel 2007). And I ask if it is possible to articulate a concept of biological holism or organicism which is neither an empirical ‘biochauvinism’ nor a metaphysical ‘vitalism’? (shrink)

It is pretty obvious that language and human consciousness entertain tight relations. We could not really be conscious of ourselves without the possibility to say “I” or “me”. And language is a key contributor in our capability to identify ourselves as conscious entities existing in the environment. But the relations linking language and consciousness are complex and difficult to analyze. Evolutionary origins of language are unknown as no fossil traces have been left by our ancestors. Sciences of consciousness however (...) begin to make available some possible evolutionary scenarios about the nature of human consciousness. We want here to propose a link between language and consciousness by using such an evolutionary scenario and also introduce the usage of a systemic approach to meaning generation. In the first part of the presentation we will use an existing scenario about the evolutionary nature of self-consciousness where the development of language has a role (1). We will highlight this role in order to identify language and self-consciousness as inter-dependant in their nature through a possible common evolutionary origin. Self-consciousness and language could then be considered as tightly inter-dependant through a common build up of human nature during evolution. The scenario presents an evolutionary nature of self-consciousness as resulting of the capability for pre-human primates to identify with their conspecifics (1, 2). The conspecifics are represented as existing in the environment, and such identification brought our pre-human ancestors to consider themselves as also existing in the environment. The scenario takes this event as being a first step for a conscious self-representation within pre-human primates, which progressively evolved toward our today human consciousness (3). But such identification with conspecifics was not for free at times of survival of the fittest. Identifying with conspecifics meant for our pre-human ancestor to also identify with their sufferings or encountered dangers. These came in addition to the dangers or sufferings naturally encountered and created a significant anxiety increase (1). The resulting level of anxiety had to be limited. One possibility for that was to develop psychological or physical tools that could have reduced the risks of occurrences and developments of such dangers and sufferings. Among these tools is the performance of language which can induce significant evolutionary advantages. We will propose a first scheme about how these evolutionary advantages could have reduced the dangers and sufferings encountered by our pre-human ancestors. We will also show how the development of language produced by itself a positive feedback on the development of inter-subjectivity in the evolutionary scenario, and so participated directly to the development of human consciousness. Other evolutionary advantages have existed like the development of imitation and synergy through experience (4). Language played a role there also, and has to be taken into account. The second part of the presentation will propose the usage of an existing systemic approach to meaning generation in terms of constraints satisfaction (5, 6). Constraints for pre-human primates, ranging from a basic “stay alive” to highly elaborated “limit anxiety”, were source of multiple meaning generations in which language has played (and still plays) a key role. Several continuations will be proposed linked to the here above thread on a co-evolutions of language and human consciousness, as based on the evolutionary scenario. (shrink)

Recently criticisms against autonomous weapons were presented in a video in which an AI-powered drone kills a person. However, some said that this video is a distraction from the real risk of AI—the risk of unlimitedly self-improving AI systems. In this article, we analyze arguments from both sides and turn them into conditions. The following conditions are identified as leading to autonomous weapons becoming a global catastrophic risk: 1) Artificial General Intelligence (AGI) development is delayed relative to (...) progress in narrow AI and manufacturing. 2) The potential for very cheap manufacture of drones, with prices below 1 USD each. 3) Anti-drone defense capabilities lagging offensive development. 4) Special global military posture encouraging development of drone swarms as a strategic offensive weapon, able to kill civilians. We conclude that while it is unlikely that drone swarms alone will become existential risk, lethal autonomous weapons could contribute to civilizational collapse in case of new world war. (shrink)

The “Value Alignment Problem” is the challenge of how to align the values of artificial intelligence with human values, whatever they may be, such that AI does not pose a risk to the existence of humans. Existing approaches appear to conceive of the problem as "how do we ensure that AI solves the problem in the right way", in order to avoid the possibility of AI turning humans into paperclips in order to “make more paperclips” or eradicating the human (...) race to “solve climate change”. This paper proposes an approach to Alignment rooted in the Enactive theory of mind that reconceptualises it as "how do we make relevant to AI what it relevant to humans". This conceptualisation is supported with a discussion of 4E cognition and goes on to suggest that the Alignment problem and the Frame problem are the same problem. The paper concludes with a discussion of the tradeoffs of the different approaches to value alignment. (shrink)

Firstly, the "Metaverse" possesses two distinctive features, "thickness" and "imagination," promising the public a structure of unknown scenarios but with unclear definitions. Attempts to establish an open framework through incompleteness, however, fail to facilitate interactions between humans and the scenario. Due to the dilemma of "digital twinning," the "Metaverse" cannot be realized as "another universe". Hence, the "Metaverse" is, in fact, only a virtual experiential territory created by aggregating technologies that offer immersion and interactivity. Secondly, when artificial intelligence serves as (...) a key supporting technology for the "Metaverse," it presents multifaceted existential risks, manifested as safety, cognitive, and ethical dilemmas. Finally, we propose that the framework for responsible "Metaverse" governance should be constructed through three pathways: collective responsibility, realism, and the co-governance of the virtual and real. (shrink)

When our choice affects some other person and the outcome is unknown, it has been argued that we should defer to their risk attitude, if known, or else default to use of a risk avoidant risk function. This, in turn, has been claimed to require the use of a risk avoidant risk function when making decisions that primarily affect future people, and to decrease the desirability of efforts to prevent human extinction, owing to the significant (...) risks associated with continued human survival. I raise objections to the claim that respect for others’ risk attitudes requires risk avoidance when choosing for future generations. In particular, I argue that there is no known principle of interpersonal aggregation that yields acceptable results in variable population contexts and is consistent with a plausible ideal of respect for others’ risk attitudes in fixed population cases. GPI Working Paper No . 20-2022. (shrink)

This article presents a challenge that those philosophers who deny the causal interpretation of explanations provided by population genetics might have to address. Indeed, some philosophers, known as statisticalists, claim that the concept of natural selection is statistical in character and cannot be construed in causal terms. On the contrary, other philosophers, known as causalists, argue against the statistical view and support the causal interpretation of natural selection. The problem I am concerned with here arises for the statisticalists because the (...) debate on the nature of natural selection intersects the debate on whether mathematical explanations of empirical facts are genuine scientific explanations. I argue that if the explanations provided by population genetics are regarded by the statisticalists as non-causal explanations of that kind, then statisticalism risks being incompatible with a naturalist stance. The statisticalist faces a dilemma: either she maintains statisticalism but has to renounce naturalism; or she maintains naturalism but has to content herself with an account of the explanations provided by population genetics that she deems unsatisfactory. This challenge is relevant to the statisticalists because many of them see themselves as naturalists. (shrink)

Human civilisation faces a range of existential risks, including nuclear war, runaway climate change and superintelligent artificial intelligence run amok. As we show here with calculations for the New Zealand setting, large numbers of currently living and, especially, future people are potentially threatened by existential risks. A just process for resource allocation demands that we consider future generations but also account for solidarity with the present. Here we consider the various ethical and policy issues involved and make a (...) case for further engagement with the New Zealand public to determine societal values towards future lives and their protection. (shrink)

In this paper, we argue that our moral concern for future well-being should reduce over time due to important practical considerations about how humans interact with spacetime. After surveying several of these considerations (around equality, special duties, existential contingency, and overlapping moral concern) we develop a set of core principles that can both explain their moral significance and highlight why this is inherently bound up with our relationship with spacetime. These relate to the equitable distribution of (1) moral concern (...) in a universe that we can only causally affect in one temporal direction; (2) access to the benefits from using spacetime as a resource; and (3) the burdens of care given (1) and (2). We conclude by considering the practical implications of our argument and find that, while it is often assumed that a preference for present over future well-being weakens the case for existential risk mitigation, this likely is not the case. (shrink)

Longtermists have recently argued that it is overwhelmingly important to do what we can to mitigate existential risks to humanity. I consider three mistakes that are often made in calculating the value of existential risk mitigation. I show how correcting these mistakes pushes the value of existential risk mitigation substantially below leading estimates, potentially low enough to threaten the normative case for existential risk mitigation. I use this discussion to draw four positive lessons (...) for the study of existential risk. -/- . (shrink)

The precarious rights of senior citizens, especially those who are highly educated and who are expected to counsel and guide the younger generations, has stimulated the creation internationally of advocacy associations and opinion leader groups. The strength of these groups, however, varies from country to country. In some countries, they are supported and are the focus of intense interest; in others, they are practically ignored. For this is reason we believe that the creation of a network of all these associations (...) is essential. The proposed network would act as a support for the already-existing policies of the United Nations’ High Commission for Human Rights, of independent experts, and of the Global Alliance for the Rights of Older People. All three have long ago recommended the creation of a recognized instrument for uniting presently scattered efforts. The proposed network, therefore, will seek to promote the international exchange of relevant expertise, and it will reinforce the commitments and actions that single countries are currently taking to meet these objectives. For example, informative public events can be organised to promote particular support initiatives and to provide an opportunity for new members of the network to be presented. The network will promote health for senior citizens, disease prevention, senior mobility, safe free time for seniors, alimentary education, protection against new risks and dangers, as well as equity in the services necessary for seniors to adopt new information and communication technologies. In the case of retired academic members, the network will promote equality with respect to continuing use of digital technologies (particularly email), continuing access to research libraries, and the guaranteed ability for seniors to fund their own research programs and to deliver free seminars. (shrink)

This paper serves as an intuition building mechanism for understanding the basics of AI, misalignment, and the reasons for why strong AI is being pursued. The approach is to engage with both pro and anti AI development arguments to gain a deeper understanding of both views, and hopefully of the issue as a whole. We investigate the basics of misalignment, common misconceptions, and the arguments for why we would want to pursue strong AI anyway. The paper delves into various aspects (...) of the problem, including existential risk, deception, hedonic adaptation, and the potential for the complete extinction of humanity. By integrating multiple elements of philosophy, this paper aims to provide a holistic understanding of the alignment problem and its significance for the future of humanity. (shrink)

The effective altruism movement aims to save lives in the most cost-effective ways. In the future, technology will allow radical life extension, and anyone who survives until that time will gain potentially indefinite life extension. Fighting aging now increases the number of people who will survive until radical life extension becomes possible. We suggest a simple model, where radical life extension is achieved in 2100, the human population is 10 billion, and life expectancy is increased by simple geroprotectors like metformin (...) or nicotinamide mononucleotide by three more years on average, so an additional 750 million people survive until “immortality”. The cost of clinical trials to prove that metformin is a real geroprotector is $65 million. In this simplified case, the price of a life saved is around eight cents, 10 000 times cheaper than saving a life from malaria by providing bed nets. However, fighting aging should not be done in place of fighting existential risks, as they are complementary causes. (shrink)

“Existential risks” are risks that threaten the destruction of humanity’s long-term potential: risks of nuclear wars, pandemics, supervolcano eruptions, and so on. On standard utilitarianism, it seems, the reduction of such risks should be a key global priority today. Many effective altruists agree with this verdict. But how should the importance of these risks be assessed on a Christian moral theory? In this paper, I begin to answer this question – taking Thomas Aquinas as a reference, and the risks (...) of anthropogenic extinction as an example. I’ll suggest that on a Thomist framework, driving ourselves extinct would constitute a complete failure to fulfil our God-given nature, a radical form of hubris, and a loss of cosmological significance. So the reduction of such risks seems vital for Thomists, or Christians more broadly. Indeed, I end with the thought that these considerations are not parochially Christian, or indeed specifically religious: everyone with a similar view of mankind – as immensely valuable yet fallen and fragile – should agree that existential risks are a pressing moral concern. (shrink)

Existential risks threaten the future of humanity, but they are difficult to measure. However, to communicate, prioritize and mitigate such risks it is important to estimate their relative significance. Risk probabilities are typically used, but for existential risks they are problematic due to ambiguity, and because quantitative probabilities do not represent some aspects of these risks. Thus, a standardized and easily comprehensible instrument is called for, to communicate dangers from various global catastrophic and existential risks. In (...) this article, inspired by the Torino scale of asteroid danger, we suggest a color coded scale to communicate the magnitude of global catastrophic and existential risks. The scale is based on the probability intervals of risks in the next century if they are available. The risks’ estimations could be adjusted based on their severities and other factors. The scale covers not only existential risks, but smaller size global catastrophic risks. It consists of six color levels, which correspond to previously suggested levels of prevention activity. We estimate artificial intelligence risks as “red”, while “orange” risks include nanotechnology, synthetic biology, full scale nuclear war and a large global agricultural shortfall (caused by regional nuclear war, coincident extreme weather, etc.) The risks of natural pandemic, supervolcanic eruption and global warming are marked as “yellow” and the danger from asteroids is “green”. -/- Keywords: global catastrophic risks; existential risks; Torino scale; policy; risk probability . (shrink)

. Grant Ramsey and Michael Deem argue that appreciating the role that empathy plays in posttransgression guilt leads to a more promising account of the emotion’s evolutionary origins. But because their proposal fails to adequately distinguish guilt from shame, we cannot say which of the two emotions we are actually getting an evolutionary account of. Moreover, a closer look at the details suggests both that empathy may be more relevant for our understanding of shame’s evolutionary origins than (...) guilt’s, and that guilt is unlikely to be an adaptation. (shrink)

Which artificial intelligence (AI) systems are agents? To answer this question, I propose a multidimensional account of agency. According to this account, a system's agency profile is jointly determined by its level of goal-directedness and autonomy as well as is abilities for directly impacting the surrounding world, long-term planning and acting for reasons. Rooted in extant theories of agency, this account enables fine-grained, nuanced comparative characterizations of artificial agency. I show that this account has multiple important virtues and is more (...) informative than alternatives. More speculatively, it may help to illuminate two important emerging questions in AI ethics: 1. Can agency contribute to the moral status of non-human beings, and how? 2. When and why might AI systems exhibit power-seeking behaviour and does this pose an existential risk to humanity? (shrink)