The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. Other animals have stronger muscles or sharper claws, but we have cleverer brains. If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of (...) our species then would come to depend on the actions of the machine superintelligence. But we have one advantage: we get to make the first move. Will it be possible to construct a seed AI or otherwise to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation? To get closer to an answer to this question, we must make our way through a fascinating landscape of topics and considerations. Read the book and learn about oracles, genies, singletons; about boxing methods, tripwires, and mind crime; about humanity's cosmic endowment and differential technological development; indirect normativity, instrumental convergence, whole brain emulation and technology couplings; Malthusian economics and dystopian evolution; artificial intelligence, and biological cognitive enhancement, and collective intelligence. (shrink)

The National Science Foundation's Second Merit Criterion, or Broader Impacts Criterion , was introduced in 1997 as the result of an earlier Congressional movement to enhance the accountability and responsibility as well as the effectiveness of federally funded projects. We demonstrate that a robust understanding and appreciation of NSF BIC argues for a broader conception of research ethics in the sciences than is currently offered in Responsible Conduct of Research training. This essay advocates augmenting RCR education with training regarding broader (...) impacts. We demonstrate that enhancing research ethics training in this way provides a more comprehensive understanding of the ethics relevant to scientific research and prepares scientists to think not only in terms of responsibly conducted science, but also of the role of science in responding to identified social needs and in adhering to principles of social justice. As universities respond to the mandate from America COMPETES to “provide training and oversight in the responsible and ethical conduct of research”, we urge institutions to embrace a more adequate conception of research ethics, what we call the Ethical Dimensions of Scientific Research, that addresses the full range of ethical issues relevant to scientific inquiry, including ethical issues related to the broader impacts of scientific research and practice. (shrink)

Steven Shapin argues that science, for all its immense authority and power, is and always has been a human endeavor, subject to human capacities and limits.

“Only a small community has concentratedon general intelligence. No one has tried to make a thinking machine... The bottom line is that we really haven’t progressed too far toward a truly intelligent machine. We have collections of dumb specialists in small domains; the true majesty of general intelligence still awaits our attack.... We have got to get back to the deepest questions of AI and general intelligence... ” –MarvinMinsky as interviewed in Hal’s Legacy, edited by David Stork, 2000. Our goal (...) in creating this edited volume has been to?ll an apparent gap in the scienti?c literature, by providing a coherent presentation of a body of contemporary research that, in spite of its integral importance, has hitherto kept a very low pro?le within the scienti?c and intellectual community. This body of work has not been given a name before; in this book we christen it “Arti?cial General Intelligence”. What distinguishes AGI work from run-of-the-mill “arti?cial intelligence” research is that it is explicitly focused on engineering general intelligence in the short term. We have been active researchers in the AGI?eld for many years, and it has been a pleasure to gather together papers from our colleagues working on related ideas from their own perspectives. In the Introduction we give a conceptual overview of the AGI?eld, and also summarize and interrelate the key ideas of the papers in the subsequent chapters. (shrink)

A controversial scientific vision predicts a time in which humans and machines will merge and create a new form of non-biological intelligence, explaining how the occurrence will solve such issues as pollution, hunger, and aging.

Many thinkers believe that the next transformational change in human organisation will be the onset of human-level artificial intelligence, and that the most likely method of achieving this will come through brain emulations or "ems": the ability to scan human brains and program their connections into ever faster computers. Taking this as his starting point, Hanson describes what a world dominated by these ems will be like.

Although nanotechnology is often defined as operations on the 10-9 meters, the lack of charisma in the scale-bound definitions has been fortified by remarkable dreams and alluring promises that spark excitement for nanotechnology. The story of the rhetorical development of nanotechnology reveals how speculative claims are powerful constructions that create legitimacy in this emerging technological domain. From its inception, nanotechnology has been more of a dream than reality, more fiction than fact. In recent years, however, the term nanotechnology has been (...) actively drawn toward the present to begin to deliver on the fantastic expectations. This debate over time and timing is loaded with paradox. This work examines how future claims work to define what counts as nanotechnology and reveals dilemmas that accompany temporal disjunctures. Science and politics converge in debates about the future of technology as expectations serve to create and enforce power and legitimacy in the emerging area. (shrink)