This article assesses how autonomy and machine learning impact the existential risk of nuclear war. It situates the problem of cyber security, which proceeds by stealth, within the larger context of nuclear deterrence, which is effective when it functions with transparency and credibility. Cyber vulnerabilities poses new weaknesses to the strategic stability provided by nuclear deterrence. This article offers best practices for the use of computer and information technologies integrated into nuclear weapons systems. Focusing on nuclear command and control, avoiding (...) autonomy and machine learning is recommended as one means to reduce the existential risk of unintended nuclear conflict. (shrink)

There are two types of artificial general intelligence (AGI) safety solutions: global and local. Most previously suggested solutions are local: they explain how to align or “box” a specific AI (Artificial Intelligence), but do not explain how to prevent the creation of dangerous AI in other places. Global solutions are those that ensure any AI on Earth is not dangerous. The number of suggested global solutions is much smaller than the number of proposed local solutions. Global solutions can be divided (...) into four groups: 1. No AI: AGI technology is banned or its use is otherwise prevented; 2. One AI: the first superintelligent AI is used to prevent the creation of any others; 3. Net of AIs as AI police: a balance is created between many AIs, so they evolve as a net and can prevent any rogue AI from taking over the world; 4. Humans inside AI: humans are augmented or part of AI. We explore many ideas, both old and new, regarding global solutions for AI safety. They include changing the number of AI teams, different forms of “AI Nanny” (non-self-improving global control AI system able to prevent creation of dangerous AIs), selling AI safety solutions, and sending messages to future AI. Not every local solution scales to a global solution or does it ethically and safely. The choice of the best local solution should include understanding of the ways in which it will be scaled up. Human-AI teams or a superintelligent AI Service as suggested by Drexler may be examples of such ethically scalable local solutions, but the final choice depends on some unknown variables such as the speed of AI progress. (shrink)

Human extinction is something generally deemed as undesirable, although some scholars view it as a potential solution to the problems of the Earth since it would reduce the moral evil and the suffering that are brought about by humans. We contend that humans collectively have absolute intrinsic value as sentient, conscious and rational entities, and we should preserve them from extinction. However, severe threats, such as climate change and incurable viruses, might push humanity to the brink of extinction. Should that (...) occur, it might be useful to envision a successor to humans able to preserve and hand down its value. One option would be to resort to humanoid robots that reproduce our salient characteristics by imitation, thanks to AI powered by machine learning. However, the question would arise of how to select the characteristics needed for our successors to thrive. This could prove to be particularly challenging. A way out might come from an algorithm entrusted with this choice. In fact, an algorithmic selection both at the social and at the individual level could be a preferred choice than other traditional ways of making decisions. In this sense, reflecting on human extinction helps us to identify solutions that are also suitable for the problems we face today. (shrink)

The intention of this paper is to point to the dilemma humanity may face in light of AI advancements. The dilemma is whether to create a world with less evil or maintain the human status of moral agents. This dilemma may arise as a consequence of using automated decision-making systems for high-stakes decisions. The use of automated decision-making bears the risk of eliminating human moral agency and autonomy and reducing humans to mere moral patients. On the other hand, it also (...) has the potential to bring tremendous benefits to humanity by decreasing human-induced harm in the world. After presenting how this dilemma may arise, I explore general avenues for addressing it. I will argue that we do not have to solve this dilemma in an all-or-nothing fashion and that a more nuanced approach may be suitable. However, the main point I want to highlight is that we need to have a principled way of addressing this dilemma, which is currently missing. (shrink)

The AGI alignment problem has a bimodal distribution of outcomes with most outcomes clustering around the poles of total success and existential, catastrophic failure. Consequently, attempts to solve AGI alignment should, all else equal, prefer false negatives (ignoring research programs that would have been successful) to false positives (pursuing research programs that will unexpectedly fail). Thus, we propose adopting a policy of responding to points of metaphysical and practical uncertainty associated with the alignment problem by limiting and choosing necessary assumptions (...) to reduce the risk false positives. Herein we explore in detail some of the relevant points of uncertainty that AGI alignment research hinges on and consider how to reduce false positives in response to them. (shrink)

This article analyses ChatGPT from the perspective of the philosophy of education. It explores ChatGPT’s implications for universities, focussing on the intertwined concepts of critical thinking, the crisis of higher education, and humanity. Does ChatGPT sound the death knell for critical thinking and, thus, exacerbate the oft-diagnosed ‘crisis in education’? And is ChatGPT really a convenient, but dangerous, tool to outsource humanity to machines?. In addressing these questions, the article’s two main arguments offer an alternative to both triumphalist and overly (...) pessimistic narratives: first, ChatGPT can lead to a revitalisation of critical thinking in higher education. However, it arrives at this conclusion not from the triumphalist viewpoint that celebrates ChatGPT’s (allegedly) limitless potential, but from the more sober perspective that ChatGPT combines remarkable strengths with considerable weaknesses and built-in limitations. Secondly, ChatGPT can prompt a return to the qualities that distinguish humans from calculating machines and (re)instate critical thinking as the pivotal virtue of higher education. The article arrives at this conclusion by rejecting the overly pessimistic concern with ‘outsourcing humanity’ and endorsing the idea that ChatGPT lays bare a ‘crisis in education’ that constitutes, simultaneously, a precious opportunity. Finally, the article stresses that this opportunity inevitably comes at a price. There will be winners and losers of the ChatGPT revolution and there is a danger that ChatGPT reintroduces elitism through the back door. One urgent task of the near future, therefore, will be to keep this danger in check. (shrink)

Artificial general intelligence is a greatly anticipated technology with non-trivial existential risks, defined as machine intelligence with competence as great/greater than humans. To date, social scientists have dedicated little effort to the ethics of AGI or AGI researchers. This paper employs inductive discourse analysis of the academic literature of two intellectual groups writing on the ethics of AGI—applied and/or ‘basic’ scientific disciplines henceforth referred to as technicians (e.g., computer science, electrical engineering, physics), and philosophy-adjacent disciplines henceforth referred to as PADs (...) (e.g., philosophy, theology, anthropology). These groups agree that AGI ethics is fundamentally about mitigating existential risk. They highlight our moral obligation to future generations, demonstrate the ethical importance of better understanding consciousness, and endorse a hybrid of deontological/utilitarian normative ethics. Technicians favor technocratic AGI governance, embrace the project of ‘solving’ moral realism, and are more deontologically inclined than PADs. PADs support a democratic approach to AGI governance, are more skeptical of deontology, consider current AGI predictions as fundamentally imprecise, and are wary of using AGI for moral fact-finding. (shrink)

This article argues that an artificial superintelligence (ASI) emerging in a world where war is still normalised constitutes a catastrophic existential risk, either because the ASI might be employed by a nation–state to war for global domination, i.e., ASI-enabled warfare, or because the ASI wars on behalf of itself to establish global domination, i.e., ASI-directed warfare. Presently, few states declare war or even war on each other, in part due to the 1945 UN Charter, which states Member States should “refrain (...) in their international relations from the threat or use of force”, while allowing for UN Security Council-endorsed military measures and self-defense. As UN Member States no longer declare war on each other, instead, only ‘international armed conflicts’ occur. However, costly interstate conflicts, both hot and cold and tantamount to wars, still take place. Further, a New Cold War between AI superpowers looms. An ASI-directed/enabled future conflict could trigger total war, including nuclear conflict, and is therefore high risk. Via conforming instrumentalism, an international relations theory, we advocate risk reduction by optimising peace through a Universal Global Peace Treaty (UGPT), contributing towards the ending of existing wars and prevention of future wars, as well as a Cyberweapons and Artificial Intelligence Convention. This strategy could influence state actors, including those developing ASIs, or an agential ASI, particularly if it values conforming instrumentalism and peace. (shrink)

This article questions the effects on urban research dynamics of the Big Data and AI turn in urban management. Increasing access to large datasets collected in real time could make certain mathematical models developed in research fields related to the management of urban systems obsolete. These ongoing evolutions are the subject of numerous works whose main angle of reflection is the future of cities rather than the transformations at work in the academic field. Our article proposes grasp the scientific dynamics (...) in areas of research related to two urban systems: transportation and water. The article demonstrates the importance of grasping these dynamics if we want to be able to apprehend what the urban management of tomorrow's cities will be like. To analyse these research areas’ dynamics, we use two complementary materials: bibliometric data and interviews. The interviews conducted in 2018 with academics and higher education officials in Paris and Edinburgh suggest avenues for hybridization between traditional modelling approaches and research in machine learning, artificial intelligence and Big Data. The bibliometric analysis highlight the trends at work: it shows that traffic flow as well as transportation studies are focussing more and more on AI and Big Data and that traffic flow studies are arousing a growing interest among computer scientists, while, so far, this interest is less pronounced in the water research area, and more especially regarding water quality. The differences observed between research on transportation and that on water confirm the multifaceted nature of the developments at work and encourage us to reject overly hasty and simplistic generalisations about the transformations underway. (shrink)