Evolutionary development is sometimes thought of as exhibiting an inexorable trend towards higher, more complex, and normatively worthwhile forms of life. This paper explores some dystopian scenarios where freewheeling evolutionary developments, while continuing to produce complex and intelligent forms of organization, lead to the gradual elimination of all forms of being that we care about. We then consider how such catastrophic outcomes could be avoided and argue that under certain conditions the only possible remedy would be a globally coordinated policy (...) to control human evolution by modifying the fitness function of future intelligent life forms. (shrink)

We describe a significant practical consequence of taking anthropic biases into account in deriving predictions for rare stochastic catastrophic events. The risks associated with catastrophes such as asteroidal/cometary impacts, supervolcanic episodes, and explosions of supernovae/gamma-ray bursts are based on their observed frequencies. As a result, the frequencies of catastrophes that destroy or are otherwise incompatible with the existence of observers are systematically underestimated. We describe the consequences of the anthropic bias for estimation of catastrophic risks, and suggest some directions for (...) future work. (shrink)

Die Übernahme des KI-Unternehmens DeepMind durch Google für rund eine halbe Milliarde US-Dollar signalisierte vor einem Jahr, dass von der KI-Forschung vielversprechende Ergebnisse erwartet werden. Spätestens seit bekannte Wissenschaftler wie Stephen Hawking und Unternehmer wie Elon Musk oder Bill Gates davor warnen, dass künstliche Intelligenz eine Bedrohung für die Menschheit darstellt, schlägt das KI-Thema hohe Wellen. Die Stiftung für Effektiven Altruismus (EAS, vormals GBS Schweiz) hat mit der Unterstützung von Experten/innen aus Informatik und KI ein umfassendes Diskussionspapier zu den Chancen (...) und Risiken der künstlichen Intelligenz verfasst und geht darin auf aktuelle, mittel- und langfristige Herausforderungen im Bereich der KI-Entwicklung ein. (shrink)

In 2000 Bill Joy proposed that the best way to prevent technological apocalypse was to "relinquish" emerging bio-, info- and nanotechnologies. His essay introduced many watchdog groups to the dangers that futurists had been warning of for decades. One such group, ETC, has called for a moratorium on all nanotechnological research until all safety issues can be investigated and social impacts ameliorated. In this essay I discuss the differences and similarities of regulating bio- and nanotechnological innovation to the efforts to (...) regulate nuclear and biological weapons of mass destruction. I then suggest the creation of a global technology regulatory regime to ensure the safe and equitable diffusion of genetic, molecular and information technologies, and point out the principal political obstacles to implementing such a regime. (shrink)

Discussions about the possible consequences of creating superintelligence have included the possibility of existential risk, often understood mainly as the risk of human extinction. We argue that suffering risks (s-risks) , where an adverse outcome would bring about severe suffering on an astronomical scale, are risks of a comparable severity and probability as risks of extinction. Preventing them is the common interest of many different value systems. Furthermore, we argue that in the same way as superintelligent AI both contributes to (...) existential risk but can also help prevent it, superintelligent AI can both be a suffering risk or help avoid it. Some types of work aimed at making superintelligent AI safe will also help prevent suffering risks, and there may also be a class of safeguards for AI that helps specifically against s-risks. (shrink)

In this essay I will explore an understanding of the potential moral agency of robots; arguing that the key characteristics of physical embodiment; adaptive learning; empathy in action; and a teleology toward the good are the primary necessary components for a machine to become a moral agent. In this context; other possible options will be rejected as necessary for moral agency; including simplistic notions of intelligence; computational power; and rule-following; complete freedom; a sense of God; and an immaterial soul. I (...) argue that it is likely that such moral machines may be able to be built; and that this does not diminish humanity or human personhood. (shrink)

The human desire to acquire new capacities is as ancient as our species itself. We have always sought to expand the boundaries of our existence, be it socially, geographically, or mentally. There is a tendency in at least some individuals always to search for a way around every obstacle and limitation to human life and happiness.

There exists a real dearth of literature available to Anglophones dealing with philosophical connections between transhumanism and Marxism. This is surprising, given the existence of works on just this relation in the other major European languages and the fact that 47 per cent of people surveyed in the 2007 Interests and Beliefs Survey of the Members of the World Transhumanist Association identified as “left,” though not strictly Marxist (Hughes 2008). Rather than seeking to explain this dearth here, I aim to (...) contribute to its being filled in by identifying three fundamental areas of similarity between transhumanism and Marxism. These are: the importance of material conditions, and particularly technological advancement, for revolution; conceptions of human nature; and conceptions of nature in general. While it is true that both Marxism and (especially) transhumanism are broad fields that encompass diverse positions, even working with somewhat generalized characterizations of the two reveals interesting parallels and dissimilarities fruitful for future work. -/- This comparison also shows that transhumanism and Marxism can learn important lessons from one another that are complementary to their respective projects. I suggest that Marxists can learn from transhumanists two lessons: that some “natural” forces may become reified forces and the extent to which the productive apparatus is now relevant to revolution. Transhumanists, on the other hand, can learn from Marxist theory the essentially social nature of the human being and the ramifications this has for the transformation of the human condition and for the forms of social organization compatible with transhumanist aims. Transhumanists can also benefit from considering the relevance of Marx’s theory of alienation to their goals of technological advancement. (shrink)

Este trabajo versa sobre la aplicación del principio de precaución a un caso concreto: los organismos modificados genéticamente (OMG). Está estructurado del modo siguiente. En primer lugar, nos centramos en los OMG, intentando aclarar el significado de los términos “organismo modificado genéticamente” y “transgénico”, ofreciendo un breve recorrido por su historia y dibujando a grandes rasgos los riesgos y beneficios que pueden procurar. A continuación, nos ocupamos del tratamiento de los OMG en algunas de las regulaciones principales en las que (...) se apela al principio de precaución y las distintas opiniones que críticos y defensores de la nueva tecnología tienen sobre su aplicación. Tras argumentar que la diversidad de opiniones se debe, al menos en parte, a la ambigüedad del propio principio, analizamos las diversas variantes en las que este se presenta. Para concluir, intentamos establecer qué tipo de precaución resulta adecuada en el caso de los OMG. (shrink)

One of the strands of the Transhumanist movement, Singulitarianism, studies the possibility that high-level artificial intelligence may be created in the future, debating ways to ensure that the interaction between human society and advanced artificial intelligence can occur safely and beneficially. But how can we guarantee this safe interaction? Are there any indications that a Singularity may be on the horizon? In trying to answer these questions, We'll make a small introduction to the area of security research in artificial intelligence. (...) We'll review some of the current paradigms in the development of autonomous intelligent systems and evidence that we can use to prospect the coming of a possible technological Singularity. Finally, we will present a reflection using the COVID-19 pandemic, something that showed that our biggest problem in managing existential risks is our lack of coordination skills as a global society. (shrink)

What does Cyberpunk and AI Ethics have to do with each other? Cyberpunk is a sub-genre of science fiction that explores the post-human relationships between human experience and technology. One similarity between AI Ethics and Cyberpunk literature is that both seek a dialogue in which the reader may inquire about the future and the ethical and social problems that our technological advance may bring upon society. In recent years, an increasing number of ethical matters involving AI have been pointed and (...) debated, and several ethical principles and guides have been suggested as governance policies for the tech industry. However, would this be the role of AI Ethics? To serve as a soft and ambiguous version of the law? I would like to promote in this article a more Cyberpunk way of doing AI Ethics, whit a more anarchic way of governance. In this study, I will seek to expose some of the deficits of the underlying power structures of our society, and suggest that AI governance be subject to public opinion, so that ‘good AI’ can become ‘good AI for all’. (shrink)

The use of advanced technologies, even of so-called transhuman technology, does not make militaries transhuman. Transhumanism includes dimensions of ethics that are themselves in direct conflict with many transhuman capabilities of soldiers in warfare. The use of advanced weapons of mass destruction represents an anti-humanism that undermines the modern, open, and high-tech nation state.

A probable solution of the Fermi paradox; and a necessary step in humanity’s cosmic development; is the construction of a Bracewell-Von Neumann Artificial General Intelligence. The use of BN probes is the most plausible method of initial galactic exploration and communication for advanced ET civilizations; and our own cosmic evolution lies firmly in the utilization of; and cooperation with; AGI agents. To establish these claims; I explore the most credible developmental path from carbon-based life forms to planetary civilizations and AI (...) creation. I consider the likely physical characteristics of extraterrestrial AI probes and propose ways to predict their behavior. Lastly; I ponder the possible trajectories for humanity’s cosmic future. (shrink)

The greatest existential threats to humanity stem from increasingly powerful advanced technologies. Yet the “risk potential” of such tools can only be realized when coupled with a suitable agent who; through error or terror; could use the tool to bring about an existential catastrophe. While the existential risk literature has provided many accounts of how advanced technologies might be misused and abused to cause unprecedented harm; no scholar has yet explored the other half of the agent-tool coupling; namely the agent. (...) This paper aims to correct this failure by offering a comprehensive overview of what we could call “agential riskology.” Only by studying the unique properties of different agential risk types can one acquire an accurate picture of the existential danger before us. (shrink)