New AI technologies have the potential to cause unintended harms in diverse domains including warfare, judicial sentencing, biomedicine and governance. One strategy for realising the benefits of AI whilst avoiding its potential dangers is to ensure that new AIs are properly ‘aligned’ with some form of ‘alignment target.’ One danger of this strategy is that – dependent on the alignment target chosen – our AIs might optimise for objectives that reflect the values only of a certain subset of society, and (...) that do not take into account alternative views about what constitutes desirable and safe behaviour for AI agents. In response to this problem, several AI ethicists have suggested alignment targets that are designed to be sensitive to widespread normative disagreement amongst the relevant stakeholders. Authors inspired by voting theory have suggested that AIs should be aligned with the verdicts of actual or simulated ‘moral parliaments’ whose members represent the normative views of the relevant stakeholders. Other authors inspired by decision theory and the philosophical literature on moral uncertainty have suggested that AIs should maximise socially expected choiceworthiness. In this paper, I argue that both of these proposals face several important problems. In particular, they fail to select attractive ‘compromise options’ in cases where such options are available. I go on to propose and defend an alternative, bargaining-theoretic alignment target, which avoids the problems associated with the voting- and decision-theoretic approaches. (shrink)

Considering public moral attitudes is a hallmark of the anticipatory governance of emerging biotechnologies, such as heritable human genome editing. However, such anticipatory governance often overlooks that future morality is open to change and that future generations may perform different moral assessments on the very biotechnologies we are trying to govern in the present. In this article, we identify an ’anticipatory gap’ that has not been sufficiently addressed in the discussion on the public governance of heritable genome editing, namely, uncertainty (...) about the moral visions of future generations about the emerging applications that we are currently attempting to govern now. This paper motivates the relevance of this anticipatory gap, identifying the challenges it generates and offering various recommendations so that moral uncertainty does not lead to governance paralysis with regard to human germline genome editing. (shrink)