As the capabilities of artificial intelligence systems improve, it becomes important to constrain their actions to ensure their behaviour remains beneficial to humanity. A variety of ethical, legal and safety-based frameworks have been proposed as a basis for designing these constraints. Despite their variations, these frameworks share the common characteristic that decision-making must consider multiple potentially conflicting factors. We demonstrate that these alignment frameworks can be represented as utility functions, but that the widely used Maximum Expected Utility paradigm provides insufficient (...) support for such multiobjective decision-making. We show that a Multiobjective Maximum Expected Utility paradigm based on the combination of vector utilities and non-linear action–selection can overcome many of the issues which limit MEU’s effectiveness in implementing aligned AI. We examine existing approaches to multiobjective AI, and identify how these can contribute to the development of human-aligned intelligent agents. (shrink)

The present paper examines the notion of agency using a model from artificial intelligence. The main thesis of the paper is that learning is a necessary condition for agency: Agency presupposes control, and control is acquired in a learning process. This thesis is explored using the so-called PS model. After substantiation the thesis, the paper explores the relation between agency and different kinds of learning using the PS model.