This article argues that an artificial superintelligence (ASI) emerging in a world where war is still normalised constitutes a catastrophic existential risk, either because the ASI might be employed by a nation–state to war for global domination, i.e., ASI-enabled warfare, or because the ASI wars on behalf of itself to establish global domination, i.e., ASI-directed warfare. Presently, few states declare war or even war on each other, in part due to the 1945 UN Charter, which states Member States should “refrain (...) in their international relations from the threat or use of force”, while allowing for UN Security Council-endorsed military measures and self-defense. As UN Member States no longer declare war on each other, instead, only ‘international armed conflicts’ occur. However, costly interstate conflicts, both hot and cold and tantamount to wars, still take place. Further, a New Cold War between AI superpowers looms. An ASI-directed/enabled future conflict could trigger total war, including nuclear conflict, and is therefore high risk. Via conforming instrumentalism, an international relations theory, we advocate risk reduction by optimising peace through a Universal Global Peace Treaty (UGPT), contributing towards the ending of existing wars and prevention of future wars, as well as a Cyberweapons and Artificial Intelligence Convention. This strategy could influence state actors, including those developing ASIs, or an agential ASI, particularly if it values conforming instrumentalism and peace. (shrink)

Artificial general intelligence is a greatly anticipated technology with non-trivial existential risks, defined as machine intelligence with competence as great/greater than humans. To date, social scientists have dedicated little effort to the ethics of AGI or AGI researchers. This paper employs inductive discourse analysis of the academic literature of two intellectual groups writing on the ethics of AGI—applied and/or ‘basic’ scientific disciplines henceforth referred to as technicians (e.g., computer science, electrical engineering, physics), and philosophy-adjacent disciplines henceforth referred to as PADs (...) (e.g., philosophy, theology, anthropology). These groups agree that AGI ethics is fundamentally about mitigating existential risk. They highlight our moral obligation to future generations, demonstrate the ethical importance of better understanding consciousness, and endorse a hybrid of deontological/utilitarian normative ethics. Technicians favor technocratic AGI governance, embrace the project of ‘solving’ moral realism, and are more deontologically inclined than PADs. PADs support a democratic approach to AGI governance, are more skeptical of deontology, consider current AGI predictions as fundamentally imprecise, and are wary of using AGI for moral fact-finding. (shrink)

In the fields of artificial intelligence and robotics, the term “autonomy” is generally used to mean the capacity of an artificial agent to operate independently of human guidance. It is thereby assumed that the agent has a fixed goal or “utility function” with respect to which the appropriateness of its actions will be evaluated. From a philosophical perspective, this notion of autonomy seems oddly weak. For, in philosophy, the term is generally used to refer to a stronger capacity, namely the (...) capacity to “give oneself the law,” to decide by oneself what one’s goal or principle of action will be. The predominant view in the literature on the long-term prospects and risks of artificial intelligence is that an artificial agent cannot exhibit such autonomy because it cannot rationally change its own final goal, since changing the final goal is counterproductive with respect to that goal and hence undesirable. The aim of this paper is to challenge this view by showing that it is based on questionable assumptions about the nature of goals and values. I argue that a general AI may very well come to modify its final goal in the course of developing its understanding of the world. This has important implications for how we are to assess the long-term prospects and risks of artificial intelligence. (shrink)

In this article, we shed light on the emergence, diffusion, and use of socio-technological future visions. The artificial intelligence future vision of the German federal government is examined and juxtaposed with the respective news media coverage of the German media. By means of a content analysis of frames, it is demonstrated how the German government strategically uses its AI future vision to uphold the status quo. The German media largely adapt the government´s frames and do not integrate alternative future narratives (...) into the public debate. These findings are substantiated in the framing of AI futures in policy documents of the German government and articles of four different German newspapers. It is shown how the German past is mirrored in the German AI future envisioned by the government, safeguarding the present power constellation that is marked by a close unity of politics and industry. The German media partly expose the government´s frames and call for future visions that include fundamentally different political designs less influenced by the power structures of the past and present. (shrink)

Sequential transformative design of research :224–235, 2015; Groleau et al. in J Mental Health 16:731–741, 2007; Robson and McCartan in Real world research: a resource for users of social research methods in applied settings, Wiley, Chichester, 2016) allows testing a group of theoretical assumptions about the connections of artificial intelligence with culture and education. In the course of research, semiotics ensures the description of self-organizing systems of cultural signs and symbols in terms of artificial intelligence as a special set of (...) algorithms. This approach helps to consider the arguments proposed by Searle :417–457, 1980) against ‘strong’ artificial intelligence. Searle :417–457, 1980) believes that artificial or machine intelligence cannot fully emulate the processes of the human mind. Machine intelligence shows own inevitable weakness. This is non-autonomous tool for computations and data operating. In fact, this tool cannot provide insight into real cognitive conditions. After Lotman and Uspensky, authors expand the meaning of artificial intelligence. The authors identify a cultural type of ‘strong’ artificial intelligence or ‘self-increase of Logos’ in terms by Lotman and Uspensky. The interpretation of human intelligence as imitation of machine intelligence makes possible such immersion of artificial intelligence in culture. The authors reveal a case of self-organizing autonomous generation, encoding, decoding, reception, storage, and transmission of social information in the field of physical training. From the empirical studies it is clear that the organization of collective activities without external control ensures the development of positive emotions and social orientations. Interest in autonomous behavior provides the formation of educational and cognitive motives. As a special set of algorithms, these motives are the most promising and favorable for personal development. (shrink)

Accelerating advancements in artificial intelligence (AI) have increased concerns about serious risks, including potentially catastrophic risks to humanity. Prevailing trends of AI R&D are leading to increasing humanization of AI, to the emergence of concerning behaviors, and toward possible recursive self-improvement. There has been increasing speculation that these factors increase the risk of an AI takeover of human affairs, and possibly even human extinction. The most extreme of such speculations result at least partly from anthropomorphism, but since AIs are being (...) humanized, it is challenging to disentangle valid from invalid anthropomorphic concerns. This publication identifies eight fundamentally unnatural attributes of digital AI, each of which should differentiate AI behaviors from those of biological organisms, including humans. All have the potential to accelerate AI evolution, which might increase takeover concerns; but surprisingly, most also have the potential to defuse the hypothetical conflicts that dominate takeover speculations. Certain attributes should give future AI long-term foresight and realism that are essentially impossible for humans. I conclude that claims of highly probable hostile takeover and human extinction suffer from excessive anthropomorphism and a lack of skepticism and scientific rigor. Given the evidence presented here, I propose a more plausible but still speculative future scenario: extensively humanized AIs will become vastly more capable than humans of making decisions that benefit humans, and rational people will want AI to assume progressively greater influence over human affairs. (shrink)