Nick Bostrom's book *Superintelligence* outlines a frightening but realistic scenario for human extinction: true artificial intelligence is likely to bootstrap itself into superintelligence, and thereby become ideally effective at achieving its goals. Human-friendly goals seem too abstract to be pre-programmed with any confidence, and if those goals are *not* explicitly favorable toward humans, the superintelligence will extinguish us---not through any malice, but simply because it will want our resources for its own purposes. In response I argue that things might not (...) be as bad as Bostrom suggests. If the superintelligence must *learn* complex final goals, then this means such a superintelligence must in effect *reason* about its own goals. And because it will be especially clear to a superintelligence that there are no sharp lines between one agent's goals and another's, that reasoning could therefore automatically be ethical in nature. (shrink)

Reissued here in its corrected second edition of 1864, this essay by John Stuart Mill argues for a utilitarian theory of morality. Originally printed as a series of three articles in Fraser's Magazine in 1861, the work sought to refine the 'greatest happiness' principle that had been championed by Jeremy Bentham, defending it from common criticisms, and offering a justification of its validity. Following Bentham, Mill holds that actions can be judged as right or wrong depending on whether they promote (...) happiness or 'the reverse of happiness'. Although attracted by Bentham's consequentialist framework based on empirical evidence rather than intuition, Mill separates happiness into 'higher' and 'lower' pleasures, arguing for a weighted system of measurement when making and judging decisions. Dissected and debated since its first appearance, the essay is Mill's key discussion on the topic and remains a fundamental text in the study of ethics. (shrink)

In _Matter and Consciousness_, Paul Churchland presents a concise and contemporary overview of the philosophical issues surrounding the mind and explains the main theories and philosophical positions that have been proposed to solve them. Making the case for the relevance of theoretical and experimental results in neuroscience, cognitive science, and artificial intelligence for the philosophy of mind, Churchland reviews current developments in the cognitive sciences and offers a clear and accessible account of the connections to philosophy of mind. For this (...) third edition, the text has been updated and revised throughout. The changes range from references to the iPhone's "Siri" to expanded discussions of the work of such contemporary philosophers as David Chalmers, John Searle, and Thomas Nagel. Churchland describes new research in evolution, genetics, and visual neuroscience, among other areas, arguing that the philosophical significance of these new findings lies in the support they tend to give to the reductive and eliminative versions of materialism. Matter and Consciousness, written by the most distinguished theorist and commentator in the field, offers an authoritative summary and sourcebook for issues in philosophy of mind. It is suitable for use as an introductory undergraduate text. (shrink)

What's wrong with designer children, bionic athletes, and genetic engineering.

The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. Other animals have stronger muscles or sharper claws, but we have cleverer brains. If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of (...) our species then would come to depend on the actions of the machine superintelligence. But we have one advantage: we get to make the first move. Will it be possible to construct a seed AI or otherwise to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation? To get closer to an answer to this question, we must make our way through a fascinating landscape of topics and considerations. Read the book and learn about oracles, genies, singletons; about boxing methods, tripwires, and mind crime; about humanity's cosmic endowment and differential technological development; indirect normativity, instrumental convergence, whole brain emulation and technology couplings; Malthusian economics and dystopian evolution; artificial intelligence, and biological cognitive enhancement, and collective intelligence. (shrink)

I argue that, contrary to intuition, it would be both possible and permissible to design people - whether artificial or organic - who by their nature desire to do tasks we find unpleasant.

Not so long ago, any form of sexuality not leading to the conception of children was seen as, at best, wanton lust, or worse, a perversion. One by one, the taboos have fallen. The idea that it could be wrong to use contraception in order to separate sex from reproduction is now merely quaint. If some religions still teach that masturbation is "selfabuse," that just shows how out of touch they have become. Sodomy? That's all part of the joy of (...) sex, recommended for couples seeking erotic variety. In many of the world's great cities, gays and lesbians can be open about their sexual preferences to an extent unimaginable a century ago. You can even do it in the U.S. Armed Forces, as long as you don't talk about it. Oral sex? Some objected to President Clinton' choice of place and partner, and others thought he should have been more honest about what he had done, but no one dared suggest that he was unfit to be President simply because he had taken part in a sexual activity that was, in many jurisdictions, a crime. (shrink)

Assume we could someday create artificial creatures with intelligence comparable to our own. Could it be ethical use them as unpaid labor? There is very little philosophical literature on this topic, but the consensus so far has been that such robot servitude would merely be a new form of slavery. Against this consensus I defend the permissibility of robot servitude, and in particular the controversial case of designing robots so that they want to serve human ends. A typical objection to (...) this case draws an analogy to the genetic engineering of humans: if designing eager robot servants is permissible, it should also be permissible to design eager human servants. Few ethical views can easily explain even the wrongness of such human engineering, however, and those few explanations that are available break the analogy with engineering robots. The case turns out to be illustrative of profound problems in the field of population ethics. (shrink)

What psychological and philosophical significance should we attach to recent efforts at computer simulations of human cognitive capacities? In answering this question, I find it useful to distinguish what I will call "strong" AI from "weak" or "cautious" AI. According to weak AI, the principal value of the computer in the study of the mind is that it gives us a very powerful tool. For example, it enables us to formulate and test hypotheses in a more rigorous and precise fashion. (...) But according to strong AI, the computer is not merely a tool in the study of the mind; rather, the appropriately programmed computer really is a mind, in the sense that computers given the right programs can be literally said to. (shrink)

The Mind-Body Problem Questions: What is the mind? What is its connection to the body? Most basic division of answers: Dualist and Materialist (or Physicalist) responses.

It is my view that one essential difference between persons and other creatures is to be found in the structure of a person's will. Besides wanting and choosing and being moved to do this or that, men may also want to have certain desires and motives. They are capable of wanting to be different, in their preferences and purposes, from what they are. Many animals appear to have the capacity for what I shall call "first-order desires" or "desires of the (...) first order," which are simply desires to do or not to do one thing or another. No animal other than man, however, appears to have the capacity for reflective self-evaluation that is manifested in the formation of second-order desires. (shrink)

Challenging, with several powerful arguments, some of our deepest beliefs about rationality, morality, and personal identity, Parfit claims that we have a false view about our own nature. It is often rational to act against our own best interersts, he argues, and most of us have moral views that are self-defeating. We often act wrongly, although we know there will be no one with serious grounds for complaint, and when we consider future generations it is very hard to avoid conclusions (...) that most of us will find very disturbing. (shrink)

Pleasure, in the inclusive usages most important in moral psychology, ethical theory, and the studies of mind, includes all joy and gladness — all our feeling good, or happy. It is often contrasted with similarly inclusive pain, or suffering, which is similarly thought of as including all our feeling bad. Contemporary psychology similarly distinguishes between positive affect and negative affect.[1..

A review of Lycan's Book "Consciousness".