Without needing to commit to any specific claims about what states of affairs have most agent-neutral value, we can nevertheless predict that states of affairs which are relatively valuable are also relatively likely to occur—on the grounds that, all else equal, at least some other agents are likely to recognize the value of those states of affairs, pursue them because they are valuable, and successfully bring them about as a consequence of that pursuit. This gives us a way to promote (...) value as such, rather than promoting it under some more tendentious description. We can predict that actions which help other people—or our own future selves—to recognize valuable states of affairs, actions which motivate them to pursue whatever states of affairs they believe to be valuable, or actions which help them succeed at their pursuits will, all else equal, have positive consequences. So we have a pro tanto reason to take such actions, and the subjective justification of that reason is independent of other moral claims. (shrink)

There is no strong reason to believe that human-level intelligence represents an upper limit of the capacity of artificial intelligence, should it be realized. This poses serious safety issues, since a superintelligent system would have great power to direct the future according to its possibly flawed motivation system. Solving this issue in general has proven to be considerably harder than expected. This paper looks at one particular approach, Oracle AI. An Oracle AI is an AI that does not act in (...) the world except by answering questions. Even this narrow approach presents considerable challenges. In this paper, we analyse and critique various methods of controlling the AI. In general an Oracle AI might be safer than unrestricted AI, but still remains potentially dangerous. (shrink)

How should we act when uncertain about the moral truth, or when trying to remain neutral between competing moral theories? This dissertation argues that some types of actions and policies are relatively likely to be approved by a very wide range of moral theories—even theories which have never yet been formulated, or which appear to cancel out one another's advice. For example, I argue that actions and policies which increase a moral agent's access to primary goods also tend to increase (...) that agent's likelihood of bringing about good consequences, even under varying and mutually incompatible hypotheses about what consequences count as "good". We therefore have a subjective, pro tanto moral reason to perform such actions and enact such policies—one whose justification does not require treating any particular theory as especially probable, but instead merely requires treating at least one at-least-partly consequentialist moral theory as an open hypothesis, and is therefore applicable even under conditions of moral uncertainty or moral neutrality. My discussion begins abstractly, but as it progresses it gradually applies its framework to increasingly concrete issues. I find that the justification of some liberal policies—in the classical sense of "liberal"—can be accomplished with significantly fewer moral assumptions than have traditionally been relied upon. (shrink)

I discuss the realizability and the ethical ramifications of Machine Ethics, from a number of different perspectives: I label these the anthropocentric, infocentric, biocentric and ecocentric perspectives. Each of these approaches takes a characteristic view of the position of humanity relative to other aspects of the designed and the natural worlds—or relative to the possibilities of ‘extra-human’ extensions to the ethical community. In the course of the discussion, a number of key issues emerge concerning the relation between technology and ethics, (...) and the nature of what it is to have moral status. Some radical challenges to certain technological presuppositions and ramifications of the infocentric approach will be discussed. Notwithstanding the obvious tensions between the infocentric perspective on one side and the biocentric and ecocentric perspectives on the other, we will see that there are also striking parallels in the way that each of these three approaches generates challenges to an anthropocentric ethical hegemony, and possible scope for some degree of convergence. (shrink)