Our preferences determine how we act and think, but exactly what the mechanics are and how they work is a central cause of concern in many disciplines. This book uses techniques from modern logics of information flow and action to develop a unified new theory of what preference is and how it changes. The theory emphasizes reasons for preference, as well as its entanglement with our beliefs. Moreover, the book provides dynamic logical systems which describe the explicit triggers driving preference (...) change, including new information, suggestions, and commands. In sum, the book creates new bridges between many fields, from philosophy and computer science to economics, linguistics, and psychology. For the experienced scholar access to a large body of recent literature is provided and the novice gets a thorough introduction to the action and techniques of dynamic logic. (shrink)

The aim of this paper is to propose a formal approach to reasoning about desires, understood as logical propositions which we would be pleased to make true, also acknowledging the fact that desire is a matter of degree. It is first shown that, at the static level, desires should satisfy certain principles that differ from those to which beliefs obey. In this sense, from a static perspective, the logic of desires is different from the logic of beliefs. While the accumulation (...) of beliefs tend to reduce the remaining possible worlds they point at, the accumulation of desires tends to increase the set of states of affairs tentatively considered as satisfactory. Indeed beliefs are expected to be closed under conjunctions, while, in the positive view of desires developed here, one can argue that endorsing \ as a desire means to desire \ and to desire \. However, desiring \ and \ at the same time is not usually regarded as rational, since it does not make much sense to desire one thing and its contrary at the same time. Thus when a new desire is added to the set of desires of an agent, a revision process may be necessary. Just as belief revision relies on an epistemic entrenchment relation, desire revision is based on a hedonic entrenchment relation satisfying other properties, due to the different natures of belief and desire. While epistemic entrenchment relations are known to be qualitative necessity relations, hedonic relations obeying a set of reasonable postulates correspond to another set-function in possibility theory, called guaranteed possibility, that drive well-behaved desire revision operations. Then the general framework of possibilistic logic provides a syntactic setting for encoding desire change. The paper also insists that desires should be carefully distinguished from goals. (shrink)

The dominant practice of AI alignment assumes (1) that preferences are an adequate representation of human values, (2) that human rationality can be understood in terms of maximizing the satisfaction of preferences, and (3) that AI systems should be aligned with the preferences of one or more humans to ensure that they behave safely and in accordance with our values. Whether implicitly followed or explicitly endorsed, these commitments constitute what we term a preferentist approach to AI alignment. In this paper, (...) we characterize and challenge the preferentist approach, describing conceptual and technical alternatives that are ripe for further research. We first survey the limits of rational choice theory as a descriptive model, explaining how preferences fail to capture the thick semantic content of human values, and how utility representations neglect the possible incommensurability of those values. We then critique the normativity of expected utility theory (EUT) for humans and AI, drawing upon arguments showing how rational agents need not comply with EUT, while highlighting how EUT is silent on which preferences are normatively acceptable. Finally, we argue that these limitations motivate a reframing of the targets of AI alignment: Instead of alignment with the preferences of a human user, developer, or humanity-writ-large, AI systems should be aligned with normative standards appropriate to their social roles, such as the role of a general-purpose assistant. Furthermore, these standards should be negotiated and agreed upon by all relevant stakeholders. On this alternative conception of alignment, a multiplicity of AI systems will be able to serve diverse ends, aligned with normative standards that promote mutual benefit and limit harm despite our plural and divergent values. (shrink)

The young Marx once remarked that political economy finds itself in an estranged form and is therefore in desperate need of a critical reconstruction of its object [Gegenstand]. He proposed a complete deconstruction of economic objectivity and its categories, hoping to recover the true species-life of man. In the article, we assert that contemporary economic theory remains confined by this estrangement, despite managing to ‘revolutionize’ itself out of the grip of classical political economy. The subjectivist-marginalist reliance on ‘measurable’ consumer preferences (...) not only solidified the discipline’s estrangement, but also wrested away any remaining basic principles of economics through neoclassical reconceptualization. A break with estrangement would require novel critical economic thinking that would do away with the discontinuity between classical and neoclassical economics. It would therefore need a rich enough framework to scrutinize its principal categories. We argue that Alain Badiou’s objective phenomenology possesses a complementary synchronic structure able to conform to basic economic tenets, allowing for a comparative and synthetic approach. This would then be the basis for a new model of economic theorizing. We conclude the article with Marx’s value form, seeing it as a possible central category of a newly proposed economic framework. (shrink)

Changing preferencesis a phenomenonoften invoked but rarely properlyaccounted for. Throughout the history of the social sciences, researchers have come against the possibility that their subjects’ preferenceswere affected by the phenomenato be explainedor by otherfactorsnot taken into accountin the explanation.Sporadically, attempts have been made to systematically investigate these in uences, but none of these seems to have had a lasting impact. Today we are still not much further with respect to preference change than we were at the middle of the last (...) century. This anthology hopes to provide a new impulse for research into this important subject. In particular, we have chosen two routes to amplify this impulse. First, we stress the use of modellingtechniquesfamiliar from economicsand decision theory. Instead of constructing complex, all-encompassing theories of preference change, the authors of this volume start with very simple, formal accounts of some possible and hopefully plausible mechanism of preference change. Eventually, these models may nd their way into larger, empirically adequate theories, but at this stage, we think that the most importantwork lies in building structure.Secondly,we stress the importance of interdisciplinary exchange. Only by drawing together experts from different elds can the complex empirical and theoretical issues in the modelling of preference change be adequately investigated. (shrink)

Existence value refers to the value humans ascribe to the existence of something, regard­less of whether it is or will be of any particular use to them. This existence value based on preference satisfaction should be taken into account in evaluating activities that come with a risk of species extinction. There are two main objections. The first is that on the preference satisfaction interpretation, the concept lacks moral importance because satisfying people’s preferences may involve no good or well-being for them. (...) However, existence value can be based on a restricted version of the preference satisfaction theory, which is not vulnerable to the skeptical arguments about the link between preference satisfaction and well-being. The second objection is that even if preference satisfaction can be linked to well-being, understanding existence value in terms of individual preference satisfaction is incoherent, because existence value reflects disinterested preferences that involve no benefits to the individual. However, the fact that existence value may involve disinterested preferences does not threaten the coherence of the concept, but suggests that it does not fit smoothly into the “utilitarian” or “welfarist” framework it is commonly considered within. A pluralistic normative approach based on prima facie duties can be an alternative to standard utilitarian-style approaches for considering existence value in concrete cases involving a risk of species extinction, such as through deep sea mining. (shrink)

With characteristic incisiveness Georg Henrik von Wright identifies pro­ haireticIogic (i. e. the logic of preference) as the core of a general theory of value concepts. Essentially, this nucleus involves the logical study of acts from the point of view of their preferability. 1 (italics added) Though the term prohairesis is found in Plato, as well as in Aristotle's treatment of the relations of preference, it is von Wright who introduces this word into contemporary analytical philoso­ phy, and succinctly specifies (...) the philosophical dimensions it encompasses. The above emphasis upon the philosophical study of the formalization of preferences is a matter of utmost importance for understanding the type of in­ quiry this investigation attempts to initiate. Over the past one hundred years the literature on general theories of subjective utility has become massive, where one considers the work done in psychometrics, econometrics, statistical theories, probability theories, etc., etc. Histories in these areas are strong in tracing various evolutions in the development of the concept of preference in decision-making. However, what has not been investigated with sustained at­ tention are the fundamentally philosophical inquiries into the formalization of preference-relations. (shrink)

Expected Utility Theory (EUT) has anomalies when interpreted descriptively and tested empirically. Experiments show that the way in which options are formulated is, in most cases, relevant for decision-making. This kind of anomaly is directly related, however, not with a proper axiom of EUT but rather with the logical principle of extensionality and its decision theoretic version: the principle of invariance. This paper focuses on the phenomenon of framing effects (FE) and the associated failures of invariance. FE arise when different (...) descriptions of the same object lead to different preferences, beliefs, or choices. A formal reconstruction of the Gain–Loss (G-L) frame is proposed, expanding the domain of standard decision theory to incorporate not only prospects but also descriptions of prospects. The system is extended by adding non-monotonic logical resources so as to account for the ceteris paribus character of FE. Finally, the same method is applied to sketching a formal representation for an anomalous case that involves subjective probabilities. The logical method is a general one that can be applied to most cases of FE. The philosophical consequences of the strategy pursued are also discussed. (shrink)

A possible world semantics for preference is developed. The remainder operator () is used to give precision to the notion that two states of the world are as similar as possible, given a specified difference between them. A general structure is introduced for preference relations between states of affairs, and three types of such preference relations are defined. It is argued that one of them, actual preference, corresponds closely to the concept of preference in informal discourse. Its logical properties are (...) studied and shown to be plausible. (shrink)

We show that in infinite worlds the following three conditions are incompatible: The spatiotemporal ordering of individuals is morally irrelevant. All else being equal, the act of bringing about a good outcome with a high probability is better than the act of bringing about the same outcome with a low probability. One act is better than another only if there is a nonzero probability that it brings about a better outcome. The impossibility of combining these conditions shows that it is (...) more costly to endorse than has been previously acknowledged. (shrink)

Kurt Gödel’s version of the Ontological Proof derives rather than assumes the crucial Possibility Claim: the claim that it is possible that something God-like exists. Gödel’s derivation starts off with a proof of the Possible Instantiation of the Positive: the principle that, if a property is positive, it is possible that there exists something that has that property. I argue that Gödel’s proof of this principle relies on some implausible axiological assumptions but it can be patched so that it only (...) relies on plausible principles. But Gödel’s derivation of the Possibility Claim also needs a substantial axiological assumption, which is still open to doubt. (shrink)

This chapter presents a new semantics for inductive empirical knowledge. The epistemic agent is represented concretely as a learner who processes new inputs through time and who forms new beliefs from those inputs by means of a concrete, computable learning program. The agent’s belief state is represented hyper-intensionally as a set of time-indexed sentences. Knowledge is interpreted as avoidance of error in the limit and as having converged to true belief from the present time onward. Familiar topics are re-examined within (...) the semantics, such as inductive skepticism, the logic of discovery, Duhem’s problem, the articulation of theories by auxiliary hypotheses, the role of serendipity in scientific knowledge, Fitch’s paradox, deductive closure of knowability, whether one can know inductively that one knows inductively, whether one can know inductively that one does not know inductively, and whether expert instruction can spread common inductive knowledge—as opposed to mere, true belief—through a community of gullible pupils. (shrink)

A general format is introduced for deriving preferences over states of affairs from preferences over a set of contextually complete alternatives. Formal results are given both for this general format and for a specific instance of it that is a plausible explication of ceteris paribus preference.

In this paper we introduce a formalism for single-agent decision making that is based on Dynamic Argumentation Frameworks. The formalism can be used to justify a choice, which is based on the current situation the agent is involved. Taking advantage of the inference mechanism of the argumentation formalism, it is possible to consider preference relations, and conflicts among the available alternatives for that reasoning. With this formalization, given a particular set of evidence, the justified conclusions supported by warranted arguments will (...) be used by the agent’s decision rules to determine which alternatives will be selected. We also present an algorithm that implements a choice function based on our formalization. Finally, we complete our presentation by introducing formal results that relate the proposed framework with approaches of classical decision theory. (shrink)

The aim of the work is to provide a language to reason about Closed Interactions, i.e. all those situations in which the outcomes of an interaction can be determined by the agents themselves and in which the environment cannot interfere with they are able to determine. We will see that two different interpretations can be given of this restriction, both stemming from Pauly Representation Theorem. We will identify such restrictions and axiomatize their logic. We will apply the formal tools to (...) reason about games and their regulation. (shrink)