We propose four policies of ethical design of human-grade Artificial Intelligence. Two of our policies are precautionary. Given substantial uncertainty both about ethical theory and about the conditions under which AI would have conscious experiences, we should be cautious in our handling of cases where different moral theories or different theories of consciousness would produce very different ethical recommendations. Two of our policies concern respect and freedom. If we design AI that deserves moral consideration equivalent to that of human beings, (...) that AI should be designed with self-respect and with the freedom to explore values other than those we might impose. We are especially concerned about the temptation to create human-grade AI pre-installed with the desire to cheerfully sacrifice itself for its creators’ benefit. (shrink)

According to the cohabitation account, all the persons that result from a fission event cohabit the same body prior to fission. This article concerns a problem for this account. Suppose Manuel and Jimena are suffering from an equally painful migraine. Unlike Jimena, however, Manuel will undergo fission. Assuming you have a spare painkiller, whom should you give it to? Intuitively, you have no more reason to give it to one or the other. The problem is that the cohabitation account suggests (...) otherwise. According to the account, there are two persons cohabiting Manuel’s body, in which case you should arguably give them the pill, since doing so alleviates the pain of more beings. One response argues that the two persons cohabiting Manuel’s body share one pain. Thus, giving them the pill alleviates no more pain than giving it to Jimena, and therefore you have no more reason to do one or the other. The goal of this article is to show that this response fails. (shrink)

A problem of the many Fs arises in cases where intuitively there is precisely one F (in the region you are talking about), but when you look closely you find many candidates for being that F, each one apparently as well-qualified as the next. Imagine an apparently solitary cloud in an otherwise blue sky. Look closer, and you'll see lots of water vapor molecules, with no one collection of them more eligible than the others to count as the cloud. Many (...) things are like this when you look closely enough. The problem arises for mundane things like rocks, houses and coins. It also arises for entities that present special philosophical challenges, like persons and experiencers. In this essay, I present a new argument that the problem of the many experiencers is an especially hard problem of the many, and property dualism—the view that properties that there is something it is like to instantiate are irreducible—may be the best way to solve it. (shrink)

There are possible artificially intelligent beings who do not differ in any morally relevant respect from human beings. Such possible beings would deserve moral consideration similar to that of human beings. Our duties to them would not be appreciably reduced by the fact that they are non-human, nor by the fact that they owe their existence to us. Indeed, if they owe their existence to us, we would likely have additional moral obligations to them that we don’t ordinarily owe to (...) human strangers – obligations similar to those of parent to child or god to creature. Given our moral obligations to such AIs, two principles for ethical AI design recommend themselves: (1) design AIs that tend to provoke reactions from users that accurately reflect the AIs’ real moral status, and (2) avoid designing AIs whose moral status is unclear. Since human moral intuition and moral theory evolved and developed in contexts without AI, those intuitions and theories might break down or become destabilized when confronted with the wide range of weird minds that AI design might make possible. (shrink)

Material objects extend through space by having different spatial parts in different places. But how do they persist through time? According to some philosophers, things have temporal parts as well as spatial parts: accepting this is supposed to help us solve a whole bunch of metaphysical problems, and keep our philosophy in line with modern physics. Other philosophers disagree, arguing that neither metaphysics nor physics give us good reason to believe in temporal parts.

This paper introduces the Monstrous Conclusion, according to which, for any population, there is a better population consisting of just one individual (the Monster). The Monstrous Conclusion is deeply counterintuitive. I defend a version of Prioritarianism as a particularly promising population axiology that does not imply the Monstrous Conclusion. According to this version of Prioritarianism, which I call Asymptotic Prioritarianism, there is diminishing marginal moral importance of individual welfare that can get close to, but never quite reach, some upper limit. (...) I argue that Asymptotic Prioritarianism faces a theoretical cost, that I call the Absolute Priority Principle. However, the Absolute Priority Principle is an extreme version of what I call the Trade-off Condition, an already noteworthy problem facing other (more widely endorsed) versions of Prioritarianism. I conclude that it is better for a theory to imply the Absolute Priority Principle and avoid the Monstrous Conclusion than to imply the Monstrous Conclusion and the Trade-off Condition. The potential for Asymptotic Prioritarianism is substantial. (shrink)

It may soon be possible for neurotechnology to connect two subjects' brains such that they share a single token mental state, such as a feeling of pleasure or displeasure. How will our moral frameworks have to adapt to accommodate this prospect? And if this sort of mental-state-sharing might already obtain in some cases, how should this possibility impact our moral thinking? This question turns out to be extremely challenging, because different examples generate different intuitions: If two subjects share very few (...) mental states, then it seems that we should count the value of those states twice, but if they share very many mental states, then it seems that we should count the value of those statesonce. We suggest that these conflicting intuitions can be reconciled if the mental states that matter for welfare have a holistic character, in a way that is independently plausible. We close by drawing tentative conclusions about how we ought to think about the moral significance of shared mental states. (shrink)

According to the cohabitation account, all the persons that result from a fission event cohabit the same body prior to fission. This article concerns a problem for this account. Suppose Manuel and Jimena are suffering from an equally painful migraine. Unlike Jimena, however, Manuel will undergo fission. Assuming you have a spare painkiller, whom should you give it to? Intuitively, you have no more reason to give it to one or the other. The problem is that the cohabitation account suggests (...) otherwise. According to the account, there are two persons cohabiting Manuel’s body, in which case you should arguably give them the pill, since doing so alleviates the pain of more beings. One response argues that the two persons cohabiting Manuel’s body share one pain. Thus, giving them the pill alleviates no more pain than giving it to Jimena, and therefore you have no more reason to do one or the other. The goal of this article is to show that this response fails. (shrink)