It seems obvious that phenomenally conscious experience is something of great value, and that this value maps onto a range of important ethical issues. For example, claims about the value of life for those in a permanent vegetative state, debates about treatment and study of disorders of consciousness, controversies about end-of-life care for those with advanced dementia, and arguments about the moral status of embryos, fetuses, and non-human animals arguably turn on the moral significance of various facts about consciousness. However, (...) though work has been done on the moral significance of elements of consciousness, such as pain and pleasure, little explicit attention has been devoted to the ethical significance of consciousness. In this book Joshua Shepherd presents a systematic account of the value present within conscious experience. This account emphasizes not only the nature of consciousness, but the importance of items within experience such as affect, valence, and the complex overall shape of particular valuable experiences. Shepherd also relates this account to difficult cases involving non-humans and those with disorders of consciousness, arguing that the value of consciousness influences and partially explains the degree of moral status a being possesses, without fully determining it. The upshot is a deeper understanding of both the moral importance of phenomenal consciousness and its relations to moral status. This book will be of great interest to philosophers and students of ethics, bioethics, philosophy of psychology, philosophy of mind and cognitive science. (shrink)

Die Übernahme des KI-Unternehmens DeepMind durch Google für rund eine halbe Milliarde US-Dollar signalisierte vor einem Jahr, dass von der KI-Forschung vielversprechende Ergebnisse erwartet werden. Spätestens seit bekannte Wissenschaftler wie Stephen Hawking und Unternehmer wie Elon Musk oder Bill Gates davor warnen, dass künstliche Intelligenz eine Bedrohung für die Menschheit darstellt, schlägt das KI-Thema hohe Wellen. Die Stiftung für Effektiven Altruismus (EAS, vormals GBS Schweiz) hat mit der Unterstützung von Experten/innen aus Informatik und KI ein umfassendes Diskussionspapier zu den Chancen (...) und Risiken der künstlichen Intelligenz verfasst und geht darin auf aktuelle, mittel- und langfristige Herausforderungen im Bereich der KI-Entwicklung ein. (shrink)

There are possible artificially intelligent beings who do not differ in any morally relevant respect from human beings. Such possible beings would deserve moral consideration similar to that of human beings. Our duties to them would not be appreciably reduced by the fact that they are non-human, nor by the fact that they owe their existence to us. Indeed, if they owe their existence to us, we would likely have additional moral obligations to them that we don’t ordinarily owe to (...) human strangers – obligations similar to those of parent to child or god to creature. Given our moral obligations to such AIs, two principles for ethical AI design recommend themselves: (1) design AIs that tend to provoke reactions from users that accurately reflect the AIs’ real moral status, and (2) avoid designing AIs whose moral status is unclear. Since human moral intuition and moral theory evolved and developed in contexts without AI, those intuitions and theories might break down or become destabilized when confronted with the wide range of weird minds that AI design might make possible. (shrink)

[This is the short version of: Müller, Vincent C. and Bostrom, Nick (forthcoming 2016), ‘Future progress in artificial intelligence: A survey of expert opinion’, in Vincent C. Müller (ed.), Fundamental Issues of Artificial Intelligence (Synthese Library 377; Berlin: Springer).] - - - In some quarters, there is intense concern about high–level machine intelligence and superintelligent AI coming up in a few dec- ades, bringing with it significant risks for human- ity; in other quarters, these issues are ignored or considered science (...) fiction. We wanted to clarify what the distribution of opinions actually is, what probability the best experts currently assign to high–level machine intelligence coming up within a particular time–frame, which risks they see with that development and how fast they see these developing. We thus designed a brief questionnaire and distributed it to four groups of experts. Overall, the results show an agreement among experts that AI systems will probably reach overall human ability around 2040-2050 and move on to superintelligence in less than 30 years thereafter. The experts say the probability is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity. (shrink)

An updated survey of the research scope in Integral Biomathics.

If the intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity. Time has come to consider these issues, and this consideration must include progress in AI as much as insights from the theory of AI. The papers in this volume try to make cautious headway in setting the problem, evaluating predictions on the future of AI, proposing ways to ensure that AI systems will be beneficial to humans – and critically (...) evaluating such proposals. (shrink)

The philosophy of AI has seen some changes, in particular: 1) AI moves away from cognitive science, and 2) the long term risks of AI now appear to be a worthy concern. In this context, the classical central concerns – such as the relation of cognition and computation, embodiment, intelligence & rationality, and information – will regain urgency.

There is, in some quarters, concern about high–level machine intelligence and superintelligent AI coming up in a few decades, bringing with it significant risks for humanity. In other quarters, these issues are ignored or considered science fiction. We wanted to clarify what the distribution of opinions actually is, what probability the best experts currently assign to high–level machine intelligence coming up within a particular time–frame, which risks they see with that development, and how fast they see these developing. We thus (...) designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity. (shrink)

This is the editorial for a special volume of JETAI, featuring papers by Omohundro, Armstrong/Sotala/O’Heigeartaigh, T Goertzel, Brundage, Yampolskiy, B. Goertzel, Potapov/Rodinov, Kornai and Sandberg. - If the general intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity – so even if we estimate the probability of this event to be fairly low, it is necessary to think about it now. We need to estimate what progress we can expect, what (...) the impact of superintelligent machines might be, how we might design safe and controllable systems, and whether there are directions of research that should best be avoided or strengthened. (shrink)

The ecosystem approach to computer system development is similar to management of biodiversity. Instead of modeling machines after a successful individual, it models machines after successful teams. It includes measuring the evaluative diversity of human teams (i.e. the disparity in ways members conduct the evaluative aspect of decision-making), adding similarly diverse machines to those teams, and monitoring the impact on evaluative balance. This article reviews new research relevant to this approach, especially the validation of a survey instrument for measuring computational (...) evaluative differences in humans (the GRINSQ). The research confirms the existence of all four known machine types among humans. (shrink)

An anthropogenic global catastrophic risk is a human-induced risk that threatens sustained and wide-scale loss of life and damage to civilisation across the globe. In order to understand how new research on governance mechanisms for emerging technologies might assuage such risks, it is important to ask how perceptions, beliefs, and attitudes towards the governance of global catastrophic risk within the research community shape the conduct of potentially risky research. The aim of this study is to deepen our understanding of emerging (...) technology research culture as it relates to global catastrophic risks, and to shed new light on how new research governance mechanisms might be developed. We analyse in-depth interviews with leading AI and biotech researchers both from universities and the private sector. We develop new insights in terms of four salient themes. First, ‘engineering mindset’, which highlights the premium placed by many interviewees on pursuing interesting research about the physical world for its own sake. Second, ‘self-government’, which looks at how self-regulation of technological development currently occurs. Third, ‘pure incentives’, focussing on how career and other incentives shapes research. Fourth, ‘norms and persuasion’, which examines the role of moral considerations in guiding the research choices of scientists. We end by considering the implications of these findings for future research on governance of anthropogenic global catastrophic risk. (shrink)

How to integrate artificial intelligence technologies in the functioning and structures of our society has become a concern of contemporary politics and public debates. In this paper, we investigate national AI strategies as a peculiar form of co-shaping this development, a hybrid of policy and discourse that offers imaginaries, allocates resources, and sets rules. Conceptually, the paper is informed by sociotechnical imaginaries, the sociology of expectations, myths, and the sublime. Empirically we analyze AI policy documents of four key players in (...) the field, namely China, the United States, France, and Germany. The results show that the narrative construction of AI strategies is strikingly similar: they all establish AI as an inevitable and massively disrupting technological development by building on rhetorical devices such as a grand legacy and international competition. Having established this inevitable, yet uncertain, AI future, national leaders proclaim leadership intervention and articulate opportunities and distinct national pathways. While this narrative construction is quite uniform, the respective AI imaginaries are remarkably different, reflecting the vast cultural, political, and economic differences of the countries under study. As governments endow these imaginary pathways with massive resources and investments, they contribute to coproducing the installment of these futures and, thus, yield a performative lock-in function. (shrink)

The paper presents an ethical analysis and constructive critique of the current practice of AI ethics. It identifies conceptual substantive and procedural challenges and it outlines strategies to address them. The strategies include countering the hype and understanding AI as ubiquitous infrastructure including neglected issues of ethics and justice such as structural background injustices into the scope of AI ethics and making the procedures and fora of AI ethics more inclusive and better informed with regard to philosophical ethics. These measures (...) integrate the perspective of AI justice into AI ethics, strengthening its capacity to provide comprehensive normative orientation and guidance for the development and use of AI that actually improves human lives and living together. (shrink)

Rush Rhees oli Ludwig Wittgensteinin oppilas, ystävä ja työtoveri, joka on toistaiseksi tullut parhaiten tunnetuksi opettajansa kirjoitusten tulkitsijana ja julkaisijana. Hän oli myös omaperäinen filosofi, jonka ajattelu on saanut osakseen varsin vähän huomiota. Tässä artikkelissa tarkastellaan Rheesin filosofiakäsitystä, johon liittyvä tematiikka on Rheesin, samoin kuin Wittgensteinin, kohdalla erottamattomasti sidoksissa kysymykseen kielen, puhumisen ja ymmärtämisen mahdollisuudesta. Rhees ajattelee, että Wittgensteinin kieltä koskevat tutkimukset ovat jatkumoa antiikin filosofien todellisuuden luonnetta koskeville pohdinnoille. Rhees kuitenkin näkee vakavia ongelmia Wittgensteinin tavassa käsitellä kieltä koskevaa filosofian (...) ydinkysymystä. Kritiikin kärki koskee Wittgensteinin käsitystä kielipeleistä. Rhees katsoo, että Wittgenstein antaa kielipelin käsitteelle liian keskeisen aseman kielen yleistä luonnetta koskevissa huomautuksissaan. Rhees korostaa, ettei kieltä ja puhumista voi kuvata sen avulla tyhjentävästi. Kielestä ja puhumisesta ei voi esittää teoriaa eikä yleistä formaalista mallia. Lisäksi hän näkee kielipelimetaforan liian mekanistisena kuvaamaan kieltä ja ihmisten välistä kommunikointia. Rhees kuitenkin puhuu ”kielen ykseydestä” tai ”kielen yhtenäisyydestä” viittaamalla keskustelun käsitteeseen ja ajattelee, että kielen ykseyden ymmärtäminen keskusteluna valaisee paremmin kielen luonnetta kuin ajatus kielestä kielipelien joukkona. Rheesin filosofiakäsitystä ja hänen tapaansa harjoittaa filosofiaa leimaakin vahvasti dialogin merkityksen korostaminen, mikä näkyy kiinnostavalla tavalla hänen uskontoa ja etiikkaa käsittelevissä kirjoituksissaan. Rhees edustaa sellaista näkemystä filosofiasta ja sen metodista, jossa itse filosofisella keskustelulla ja tutkimuksella on itseisarvo. Filosofin tehtävänä ei ole olla elämänopas, joka tarjoaa vastauksia elämänongelmiin tai uskonnollisten kysymysten rationaalisuuteen. Filosofian tehtävänä on jatkuva ihmettely, dialogi ja ymmärryksen etsiminen. Filosofisten kysymysten vaikeuden ymmärtäminen on tärkeämpää kuin niiden ratkaiseminen. (shrink)

The lexicon of religion has been widely used in the context of the social and cultural transformations associated with the ‘digital revolution’, whether in metaphoric or in realistic terms. The study of digital magic/magic in digital times, the other side of the coin of the Sacred 2.0, is still in its infancy. Yet, references to magic are made frequently in reflections about the rapid development of the digitalisation of society and culture, and they deserve more in-depth study. This paper tackles (...) the issue of magic in and of new technologies, including artificial intelligence devices. After a broad outline of the presence of magic in the digital ecosystem, this paper will focus on the lexical surface and semantic references to ‘magical thinking’, explore the underlying reasons for such references and unveil the regimes of existence for magic. Finally, the paper questions the role of language in the construction of a ‘digital magic’ and the transformations of the semantic category of magic as seen through the prism of a cultural revolution, that of the digitalisation of societies. (shrink)

Contemporary ethical analysis of Artificial Intelligence is growing rapidly. One of its most recognizable outcomes is the publication of a number of ethics guidelines that, intended to guide governmental policy, address issues raised by AI design, development, and implementation and generally present a set of recommendations. Here we propose two things: first, regarding content, since some of the applied issues raised by AI are related to fundamental questions about topics like intelligence, consciousness, and the ontological and ethical status of humans, (...) among others, the treatment of these issues would benefit from interfacing with neuroethics that has been addressing those same issues in the context of brain research. Second, the identification and management of some of the practical ethical challenges raised by AI would be enriched by embracing the methodological resources used in neuroethics. In particular, we focus on the methodological distinction between conceptual and action-oriented neuroethical approaches. We argue that the normative discussion about AI will benefit from further integration of conceptual analysis, including analysis of some operative assumptions, their meaning in different contexts, and their mutual relevance in order to avoid misplaced or disproportionate concerns and achieve a more realistic and useful approach to identifying and managing the emerging ethical issues. (shrink)

Various kinds of human bioenhancement represent a major topic of contention in both bioethics and futures studies. Moral enhancement is one of them. It will be argued that voluntary moral bio-enhancement should be based on an opt out moral enhancement scheme. Such a scheme would avoid the challenges of a voluntary moral enhancement opt in scheme. The former has a proper place in a minimal state. It will be explained why such a state can be called Utopia. The concept of (...) voluntary opt out moral enhancement in Utopia will be highlighted in detail. (shrink)

Can humans and artificial intelligences share concepts and communicate? Making AI Intelligible shows that philosophical work on the metaphysics of meaning can help answer these questions. Herman Cappelen and Josh Dever use the externalist tradition in philosophy to create models of how AIs and humans can understand each other. In doing so, they illustrate ways in which that philosophical tradition can be improved. The questions addressed in the book are not only theoretically interesting, but the answers have pressing practical implications. (...) Many important decisions about human life are now influenced by AI. In giving that power to AI, we presuppose that AIs can track features of the world that we care about (for example, creditworthiness, recidivism, cancer, and combatants). If AIs can share our concepts, that will go some way towards justifying this reliance on AI. This ground-breaking study offers insight into how to take some first steps towards achieving Interpretable AI. (shrink)

In contrast to China’s efforts to upgrade its system of governance around a stupefying amount of data collection and electronic scoring, countries committed to democracy and human rights did not upgrade their systems. Instead, those countries ended up with surveillance capitalism. It is vital for the survival of those ideas about governance to perform such an upgrade. This paper aims to contribute to that goal. I propose a framework of epistemic actorhood in terms of four roles and characterize digital lifeworlds (...) and what matters about them both in terms of how they fit in with Max Tegmark’s distinctions among stages of life and in terms of how they generate their own episteme, the data episteme, with its immense possibilities of infopower. Epistemic rights that strengthen existing human rights – as part of a fourth generation of rights – are needed to protect epistemic actorhood in those roles. In the long run, we might well need a new kind of right, the right to the exercise of genuinely human intelligence. (shrink)

One of the strands of the Transhumanist movement, Singulitarianism, studies the possibility that high-level artificial intelligence may be created in the future, debating ways to ensure that the interaction between human society and advanced artificial intelligence can occur safely and beneficially. But how can we guarantee this safe interaction? Are there any indications that a Singularity may be on the horizon? In trying to answer these questions, We'll make a small introduction to the area of security research in artificial intelligence. (...) We'll review some of the current paradigms in the development of autonomous intelligent systems and evidence that we can use to prospect the coming of a possible technological Singularity. Finally, we will present a reflection using the COVID-19 pandemic, something that showed that our biggest problem in managing existential risks is our lack of coordination skills as a global society. (shrink)

New technologies, including artificial intelligence (AI), have helped us begin to take our first steps off Earth and into outer space. But conflicts inevitably will arise and, in the absence of settled governance, may be resolved by force, as is typical for new frontiers. But the terrestrial assumptions behind the ethics of war will need to be rethought when the context radically changes, and both the environment of space and the advent of robotic warfighters with superhuman capabilities will constitute such (...) a radical change. This essay examines how new autonomous technologies, especially dual-use technologies, and the challenges to human existence in space will force us to rethink the ethics of war, both from space to Earth, and in space itself. (shrink)

Artificial general intelligence is a greatly anticipated technology with non-trivial existential risks, defined as machine intelligence with competence as great/greater than humans. To date, social scientists have dedicated little effort to the ethics of AGI or AGI researchers. This paper employs inductive discourse analysis of the academic literature of two intellectual groups writing on the ethics of AGI—applied and/or ‘basic’ scientific disciplines henceforth referred to as technicians (e.g., computer science, electrical engineering, physics), and philosophy-adjacent disciplines henceforth referred to as PADs (...) (e.g., philosophy, theology, anthropology). These groups agree that AGI ethics is fundamentally about mitigating existential risk. They highlight our moral obligation to future generations, demonstrate the ethical importance of better understanding consciousness, and endorse a hybrid of deontological/utilitarian normative ethics. Technicians favor technocratic AGI governance, embrace the project of ‘solving’ moral realism, and are more deontologically inclined than PADs. PADs support a democratic approach to AGI governance, are more skeptical of deontology, consider current AGI predictions as fundamentally imprecise, and are wary of using AGI for moral fact-finding. (shrink)

What is the best overarching ethical principle to give a possible future superintelligent machine, given that we do not know what the best ethics are today or in the future? Eliezer Yudkowsky has suggested that a superintelligent AI should have as its goal to carry out the coherent extrapolated volition of humanity (CEV), the most coherent way of combining human goals. The article discusses some problems with this proposal and some alternatives suggested by Nick Bostrom. A slightly different proposal is (...) then suggested, which I argue solves the problems better than Yudkowsky’s proposal. (shrink)

On virtue ethical grounds, Barbro Fröding and Martin Peterson recently recommended that near-future AIs should be developed as ‘Friendly AI’. AI in social interaction with humans should be programmed such that they mimic aspects of human friendship. While it is a reasonable goal to implement AI systems interacting with humans as Friendly AI, I identify four issues that need to be addressed concerning Friendly AI with Fröding’s and Peterson’s understanding of Friendly AI as a starting point. In a first step, (...) I briefly recapitulate Fröding’s and Peterson’s arguments for Friendly AI. I then highlight some issues with Fröding’s and Peterson’s approach and line of reasoning and identify four problems related to the notion of Friendly AI, which all pertain to the role and need for humans’ moral development. These are that one should consider the moral tendencies and preferences of the humans interacting with a friendly AI, it needs to be considered whether the humans interacting with a Friendly AI are still developing their virtues and character traits, the indirect effects of replacing humans with Friendly AI should be considered with respect to the possibilities for humans to develop their moral virtues and that the question whether the AI is perceived as some form of Artificial General Intelligence cannot be neglected. In conclusion, I argue that all of these four problems are related to humans moral development and that this observation strongly emphasizes the role and need for humans moral development in correlation to the accelerating development of AI-systems. (shrink)

As our epistemic ambitions grow, the common and scientific endeavours are becoming increasingly dependent on Machine Learning (ML). The field rests on a single experimental paradigm, which consists of splitting the available data into a training and testing set and using the latter to measure how well the trained ML model generalises to unseen samples. If the model reaches acceptable accuracy, an a posteriori contract comes into effect between humans and the model, supposedly allowing its deployment to target environments. Yet (...) the latter part of the contract depends on human inductive predictions or generalisations, which infer a uniformity between the trained ML model and the targets. The paper asks how we justify the contract between human and machine learning. It is argued that the justification becomes a pressing issue when we use ML to reach ‘elsewheres’ in space and time or deploy ML models in non-benign environments. The paper argues that the only viable version of the contract can be based on optimality (instead of on reliability which cannot be justified without circularity) and aligns this position with Schurz’s optimality justification. It is shown that when dealing with inaccessible/unstable ground-truths (‘elsewheres’ and non-benign targets), the optimality justification undergoes a slight change, which should reflect critically on our epistemic ambitions. Therefore, the study of ML robustness should involve not only heuristics that lead to acceptable accuracies on testing sets. The justification of human inductive predictions or generalisations about the uniformity between ML models and targets should be included as well. Without it, the assumptions about inductive risk minimisation in ML are not addressed in full. (shrink)

Ex-Machina and Morgan, two recent science-fiction films that deal with the creation of humanoids, also explored the relationship between artificial intelligence, spatiality and the lingering question mark regarding artificial consciousness. In both narratives, the creators of the humanoids have tried to mimic human consciousness as closely as possible, which has resulted in the imprisonment of the humanoids due to proprietary concerns in Ex-Machina and due to the violent behavior of the humanoid in Morgan. This article addresses the dilemma of whether (...) or not the humanoids in both films possess high levels of artificial consciousness and its possible consequences regarding focalization, a narrative term that presupposes subjectivity, as well as offer two new categories of posthuman focalization—X-focalization and A-focalization. The issue of captivity also has far-reaching ethical implications when considering the underlying assumption of artificial consciousness—if humanoids are indeed endowed with a subjective inner life, then they are entitled to be treated as moral agents, equivalent to humans rather than animals. (shrink)

Virtue ethics seems to be a promising moral theory for understanding and interpreting the development and behavior of artificial moral agents. Virtuous artificial agents would blur traditional distinctions between different sorts of moral machines and could make a claim to membership in the moral community. Accordingly, we investigate the “machine question” by studying whether virtue or vice can be attributed to artificial intelligence; that is, are people willing to judge machines as possessing moral character? An experiment describes situations where either (...) human or AI agents engage in virtuous or vicious behavior and experiment participants then judge their level of virtue or vice. The scenarios represent different virtue ethics domains of truth, justice, fear, wealth, and honor. Quantitative and qualitative analyses show that moral attributions are weakened for AIs compared to humans, and the reasoning and explanations for the attributions are varied and more complex. On “relational” views of membership in the moral community, virtuous machines would indeed be included, even if they are indeed weakened. Hence, while our moral relationships with artificial agents may be of the same types, they may yet remain substantively different than our relationships to human beings. (shrink)

The increasing use of Artificial Intelligence for making decisions in public affairs has sparked a lively debate on the benefits and potential harms of self-learning technologies, ranging from the hopes of fully informed and objectively taken decisions to fear for the destruction of mankind. To prevent the negative outcomes and to achieve accountable systems, many have argued that we need to open up the “black box” of AI decision-making and make it more transparent. Whereas this debate has primarily focused on (...) how transparency can secure high-quality, fair, and reliable decisions, far less attention has been devoted to the role of transparency when it comes to how the general public come to perceive AI decision-making as legitimate and worthy of acceptance. Since relying on coercion is not only normatively problematic but also costly and highly inefficient, perceived legitimacy is fundamental to the democratic system. This paper discusses how transparency in and about AI decision-making can affect the public’s perception of the legitimacy of decisions and decision-makers and produce a framework for analyzing these questions. We argue that a limited form of transparency that focuses on providing justifications for decisions has the potential to provide sufficient ground for perceived legitimacy without producing the harms full transparency would bring. (shrink)

Recent advancements in AI have prompted a large number of AI ethics guidelines published by governments and nonprofits. While many of these papers propose concrete or seemingly applicable ideas, few philosophically sound proposals are made. In particular, we observe that the line of questioning has often not been examined critically and underlying conceptual problems not always dealt with at the root. In this paper, we investigate the nature of ethical AI systems and what their moral status might be by first (...) turning to the notions of moral agency and patience. We find that their explication would come at a too high cost which is why we, second, articulate a different approach that avoids vague and ambiguous concepts or the problem of other minds. Third, we explore the impact of our philosophical and conceptual analysis on the regulatory landscape, make this link explicit, and finally propose a set of promising policy steps. (shrink)

Decision theorists disagree about how instrumentally rational agents, i.e., agents trying to achieve some goal, should behave in so-called Newcomb-like problems, with the main contenders being causal and evidential decision theory. Since the main goal of artificial intelligence research is to create machines that make instrumentally rational decisions, the disagreement pertains to this field. In addition to the more philosophical question of what the right decision theory is, the goal of AI poses the question of how to implement any given (...) decision theory in an AI. For example, how would one go about building an AI whose behavior matches evidential decision theory’s recommendations? Conversely, we can ask which decision theories describe the behavior of any existing AI design. In this paper, we study what decision theory an approval-directed agent, i.e., an agent whose goal it is to maximize the score it receives from an overseer, implements. If we assume that the overseer rewards the agent based on the expected value of some von Neumann–Morgenstern utility function, then such an approval-directed agent is guided by two decision theories: the one used by the agent to decide which action to choose in order to maximize the reward and the one used by the overseer to compute the expected utility of a chosen action. We show which of these two decision theories describes the agent’s behavior in which situations. (shrink)

We need an account of corporate agency that is temporally robust – one that will help future people to cope with challenges posed by corporate groups in a range of credible futures. In particular, we need to bequeath moral resources that enable future people to avoid futures dominated by corporate groups that have no regard for human beings. This paper asks how future philosophers living in broken or digital futures might re-imagine contemporary debates about corporate agency. It argues that the (...) only temporally robust account is moralised extreme collectivism, where full moral personhood is accorded to those corporate groups that are reliably disposed to respond appropriately to moral reasons. (shrink)

The greatest existential threats to humanity stem from increasingly powerful advanced technologies. Yet the “risk potential” of such tools can only be realized when coupled with a suitable agent who; through error or terror; could use the tool to bring about an existential catastrophe. While the existential risk literature has provided many accounts of how advanced technologies might be misused and abused to cause unprecedented harm; no scholar has yet explored the other half of the agent-tool coupling; namely the agent. (...) This paper aims to correct this failure by offering a comprehensive overview of what we could call “agential riskology.” Only by studying the unique properties of different agential risk types can one acquire an accurate picture of the existential danger before us. (shrink)

There are two primary camps in which nanotechnology today can be categorized normal nanotechnology and speculative nanotechnology. The birth of nanotechnology proper was conceived through discourses of speculative nanotechnology. However, current nanotech-nology research has detracted from its speculative promises in favour of more attainable material products. Nonetheless, normal nanotechnology has leveraged the popular support and consequential funding it needs to conduct research and development (R&D) as a result of popular conceptions of speculative nanotechnology and its promises. Similarly, the scholarly literature (...) has shifted its focus away from speculative nanofutures towards normal nanotech-nology R&D. This paper shows that there is an incongruence between the representation of nanotechnology in the media, scholarly journals and industry. (shrink)

While it is often said that robotics should aspire to reproducible and measurable results that allow benchmarking, I argue that a focus on benchmarking can be a hindrance for progress in robotics. The reason is what I call the ‘measure-target confusion’, the confusion between a measure of progress and the target of progress. Progress on a benchmark (the measure) is not identical to scientific or technological progress (the target). In the past, several academic disciplines have been led into pursuing only (...) reproducible and measurable ‘scientific’ results – robotics should be careful to follow that line because results that can be benchmarked must be specific and context-dependent, but robotics targets whole complex systems for a broad variety of contexts. While it is extremely valuable to improve benchmarks to reduce the distance be- tween measure and target, the general problem to measure progress towards more intelligent machines (the target) will not be solved by benchmarks alone; we need a balanced approach with sophisticated benchmarks, plus real-life testing, plus qualitative judgment. (shrink)

From his preliminary analysis in 1965, Hubert Dreyfus projected a future much different than those with which his contemporaries were practically concerned, tempering their optimism in realizing something like human intelligence through conventional methods. At that time, he advised that there was nothing “directly” to be done toward machines with human-like intelligence, and that practical research should aim at a symbiosis between human beings and computers with computers doing what they do best, processing discrete symbols in formally structured problem domains. (...) Fast-forward five decades, and his emphasis on the difference between two essential modes of processing, the unconscious yet purposeful mode fundamental to situated human cognition, and the “minded” sense of conscious processing characterizing symbolic reasoning that seems to lend itself to explicit programming, continues into the famous Dreyfus–McDowell debate. The present memorial reviews Dreyfus’ early projections, asking if the fears that punctuate current popular commentary on AI are warranted, and in light of these if he would deliver similar practical advice to researchers today. (shrink)

The aim of this paper is to offer an analysis of the notion of artificial moral agent (AMA) and of its impact on human beings’ self-understanding as moral agents. Firstly, I introduce the topic by presenting what I call the Continuity Approach. Its main claim holds that AMAs and human moral agents exhibit no significant qualitative difference and, therefore, should be considered homogeneous entities. Secondly, I focus on the consequences this approach leads to. In order to do this I take (...) into consideration the work of Bostrom and Dietrich, who have radically assumed this viewpoint and thoroughly explored its implications. Thirdly, I present an alternative approach to AMAs—the Discontinuity Approach—which underscores an essential difference between human moral agents and AMAs by tackling the matter from another angle. In this section I concentrate on the work of Johnson and Bryson and I highlight the link between their claims and Heidegger’s and Jonas’s sug-gestions concerning the relationship between human beings and technological products. In conclusion I argue that, although the Continuity Approach turns out to be a necessary postulate to the machine ethics project, the Discontinuity Approach highlights a relevant distinction between AMAs and human moral agents. On this account, the Discontinuity Approach generates a clearer understanding of what AMAs are, of how we should face the moral issues they pose, and, finally, of the difference that separates machine ethics from moral philosophy. (shrink)

In this paper, we first enumerate the problems that humans might face with a new type of technology such as robots with artificial intelligence (AI robots). Robotics entrepreneurs are calling for discussions about goals and values because AI robots, which are potentially more intelligent than humans, can no longer be fully understood and controlled by humans. AI robots could even develop into ethically “bad” agents and become very harmful. We consider these discussions as part of a process of developing responsible (...) innovations in AI robotics in order to prevent catastrophic risks on a global scale. To deal with these issues, we propose the capability-effectual approach, drawing on two bodies of research: the capability approach from ethics, and the effectual process model from entrepreneurship research. The capability approach provides central human capabilities, guiding the effectual process through individual goals and aspirations in the collaborative design process of stakeholders. More precisely, by assuming and understanding correspondences between goals, purposes, desires, and aspirations in the languages of different disciplines, the capability-effectual approach clarifies both how a capability list working globally could affect the aspirations and end-goals of individuals, and how local aspirations and end-goals could either energise or limit effectual processes. Theoretically, the capability-effectual approach links the collaboration of stakeholders and the design process in responsible innovation research. Practically, this approach could potentially contribute to the robust development of AI robots by providing robotics entrepreneurs with a tool for establishing a permissible action range within which to develop AI robotics. (shrink)

To reassure those concerned about wholesale discontinuity between human existence and posthumanity, transhumanists assert shared ground with antiquity on vital challenges and aspirations. Because their claims reflect key misconceptions, there is no shared vision for transhumanists to invoke. Having exposed their misuses of Prometheus, Plato, and Aristotle, I show that not only do transhumanists and antiquity crucially diverge on our relation to ideals, contrast-dependent aspiration, and worthy endeavors but that illumining this divide exposes central weaknesses in transhumanist argumentation. What is (...) more, antiquity’s handling of these topics suggests a way through the impasse in current enhancement debates about human “nature” and helps to resolve a tension within transhumanists’ accounts of what our best moments signify about the ontological requirements for real flourishing. (shrink)

The rapid advancement of algorithmic trading has demonstrated the success of AI automation, as well as gaps in our understanding of the implications of this technology proliferation. We explore ethical issues in the context of autonomous trading agents, both to address problems in this domain and as a case study for regulating autonomous agents more generally. We argue that increasingly competent trading agents will be capable of initiative at wider levels, necessitating clarification of ethical and legal boundaries, and corresponding development (...) of norms and enforcement capability. (shrink)

From the standpoint of disability advocacy, further exploration of the concept of well-being stands to be availing. The notion that “welfarism” about disability, which Julian Savulescu and Guy Kahane debuted, qualifies as helpful is encouraged by their claim that welfarism shares important commitments with that advocacy. As becomes clear when they apply their welfarist frame to procreative decisions, endorsing welfarism would, in fact, sharply undermine it. Savulescu and Kahane's Principle of Procreative Beneficence—which reflects transhumanism, or advocacy of radical bioenhancement—morally requires (...) parents to choose the child who will, in all probability, have “the best life.” Assuming the emergence of potent biotechnologies, procreative decision-making would be highly standardized, for prospective parents would be morally obliged to maximize select capacities, including intelligence, self-control, and hedonic set-point, in their children. Welfarism, applied to reproduction, is staunchly objectivist about what course is incumbent on decision-makers, giving no credence to first-personal values, aspirations, and experiences. Though this dismissal of individual perspectives applies to everyone, its implications for disability advocacy are especially severe. With that advocacy in view, greater attention to “well-being” should, therefore, be severed from the welfarism of Savulescu and Kahane. (shrink)

Frente a una narrativa distópica presente en los medios de comunicación y cultura popular que caracteriza el avance y desarrollo de la inteligencia artificial como una amenaza o riesgo existencial quiero valorar de manera crítica y constructiva el rol de la inteligencia artificial para el bien común. La tecnología digital también se puede aplicar para la solución de grandes problemas de la humanidad, como los 17 Objetivos de Desarrollo Sostenible y sus 169 metas de la agenda 2030. En este artículo, (...) comentaré distintos casos de uso y aplicación de la inteligencia artificial y la robótica encaminados a conseguir los 17 Objetivos de Desarrollo Sostenible y qué principios éticos deben guiar su aplicación para que la inteligencia artificial consiga la ambiciosa agenda 2030. También comentaré el plan de acción de España y la estrategia nacional para cumplir la agenda 2030 y de qué manera incorpora las TIC. (shrink)

Different people have different perceptions about artificial intelligence (AI). It is extremely important to bring together all the alternative frames of thinking—from the various communities of developers, researchers, business leaders, policymakers, and citizens—to properly start acknowledging AI. This article highlights the ‘fruitful collaboration’ that sociology and AI could develop in both social and technical terms. We discuss how biases and unfairness are among the major challenges to be addressed in such a sociotechnical perspective. First, as intelligent machines reveal their nature (...) of ‘magnifying glasses’ in the automation of existing inequalities, we show how the AI technical community is calling for transparency and explainability, accountability and contestability. Not to be considered as panaceas, they all contribute to ensuring human control in novel practices that include requirement, design and development methodologies for a fairer AI. Second, we elaborate on the mounting attention for technological narratives as technology is recognized as a social practice within a specific institutional context. Not only do narratives reflect organizing visions for society, but they also are a tangible sign of the traditional lines of social, economic, and political inequalities. We conclude with a call for a diverse approach within the AI community and a richer knowledge about narratives as they help in better addressing future technical developments, public debate, and policy. AI practice is interdisciplinary by nature and it will benefit from a socio-technical perspective. (shrink)

Can humans get arbitrarily capable reinforcement learning agents to do their bidding? Or will sufficiently capable RL agents always find ways to bypass their intended objectives by shortcutting their reward signal? This question impacts how far RL can be scaled, and whether alternative paradigms must be developed in order to build safe artificial general intelligence. In this paper, we study when an RL agent has an instrumental goal to tamper with its reward process, and describe design principles that prevent instrumental (...) goals for two different types of reward tampering. Combined, the design principles can prevent reward tampering from being an instrumental goal. The analysis benefits from causal influence diagrams to provide intuitive yet precise formalizations. (shrink)

Robotics is currently not only a cutting-edge research area, but is potentially disruptive to all domains of our lives—for better and worse. While legislation is struggling to keep pace with the development of these new artifacts, our intellectual limitations and physical laws seem to present the only hard demarcation lines, when it comes to state-of-the-art R&D. To better understand the possible implications, the paper at hand critically investigates underlying processes and structures of robotics in the context of Heidegger’s and Nishitani’s (...) accounts of science and technology. Furthermore, the analysis draws on Bauman’s theory of modernity in an attempt to assess the potential risk of large-scale robot integration. The paper will highlight undergirding mechanisms and severe challenges imposed upon our socio-cultural lifeworlds by massive robotic integration. Admittedly, presenting a mainly melancholic account, it will, however, also explore the possibility of robotics forcing us to reassess our position and to solve problems, which we seem unable to tackle without facing existential crises. (shrink)

In the last 20 years the Turing test has been left further behind by new developments in artificial intelligence. At the same time, however, these developments have revived some key elements of the Turing test: imitation and adversarialness. On the one hand, many generative models, such as generative adversarial networks, build imitators under an adversarial setting that strongly resembles the Turing test. The term “Turing learning” has been used for this kind of setting. On the other hand, AI benchmarks are (...) suffering an adversarial situation too, with a ‘challenge-solve-and-replace’ evaluation dynamics whenever human performance is ‘imitated’. The particular AI community rushes to replace the old benchmark by a more challenging benchmark, one for which human performance would still be beyond AI. These two phenomena related to the Turing test are sufficiently distinctive, important and general for a detailed analysis. This is the main goal of this paper. After recognising the abyss that appears beyond superhuman performance, we build on Turing learning to identify two different evaluation schemas: Turing testing and adversarial testing. We revisit some of the key questions surrounding the Turing test, such as ‘understanding’, commonsense reasoning and extracting meaning from the world, and explore how the new testing paradigms should work to unmask the limitations of current and future AI. Finally, we discuss how behavioural similarity metrics could be used to create taxonomies for artificial and natural intelligence. Both testing schemas should complete a transition in which humans should give way to machines—not only as references to be imitated but also as judges—when pursuing and measuring machine intelligence. (shrink)

Las nuevas tecnologías parecen estar cambiando radicalmente nuestro mundo. Analizamos aquí las condiciones para que esto sea así con relación a un grupo muy importante de fenómenos vinculados a las tecnologías computacionales y a las biotecnologías, que responden a denominaciones de uso ya muy extendido, en buena medida gracias a los medios de comunicación: realidad virtual, realidad aumentada, inteligencia artificial, humanidad mejorada e Internet de las cosas. Argumentamos que la inteligencia artificial ocupa entre ellos un lugar central y que la (...) constitución de un mundo realmente nuevo depende esencialmente de nuestras actitudes ante ella. (shrink)