There are plenty of classic paradoxes about conditional obligations, like the duty to be gentle if one is to murder, and about “supererogatory” deeds beyond the call of duty. But little has been said about the intersection of these topics. We develop the first general account of conditional supererogation, with the power to solve familiar puzzles as well as several that we introduce. Our account, moreover, flows from two familiar ideas: that conditionals restrict quantification and that supererogation emerges from a (...) clash between justifying and requiring reasons. (shrink)

To consequentialize a deontological moral theory is to give a theory which issues the same moral verdicts, but explains those verdicts in terms of maximizing or satisficing value. There are many motivations for consequentializing: to reconcile plausible ideas behind deontology with plausible ideas behind consequentialism, to help us better understand deontological theories, or to extend deontological theories beyond what intuitions alone tell us. It has proven difficult to consequentialize theories that allow for moral dilemmas or that deny that “ought” implies (...) “can.” This article argues that the problem is best solved by allowing impossible actions as inputs into consequentializations. It shows that all other approaches that have been advocated are inadequate. It also argues that progress in consequentialization, and in formal ethics more generally, requires thinking about more than just wrongness and permissibility; we should think about contrary-to-duty obligations and degrees of wrongness as well. (shrink)

When a novel scientific theory conflicts with otherwise plausible moral assumptions, we do not treat that as evidence against the theory. We may scrutinize the empirical data more keenly and take extra care over its interpretation, but science is in some core sense immune to moral refutation. Can the same be said of philosophical theories (or the non-ethical, ‘metaphysical’ ones at least)? If a position in the philosophy of mind, for example, is discovered to have eye-widening moral import, does that (...) count against it at all? Actual responses by philosophers to the question of whether unanticipated moral consequences of metaphysical theories have evidential force are scattered, implicit, divergent, under-argued, and sometimes even self-undermining. The present discussion is, most immediately, an attempt to sort out the confusion. Beyond that, it exploits the new perspective this question gives us on a familiar topic: the relation of philosophy to science. (shrink)

There has been a recent surge of work on deontic modality within philosophy of language. This work has put the deontic logic tradition in contact with natural language semantics, resulting in significant increase in sophistication on both ends. This chapter surveys the main motivations, achievements, and prospects of this work.

In this paper, I will discuss some examples of the so-called contrary-to-duty paradox, a well-known puzzle in deontic logic. A contrary-to-duty obligation is an obligation telling us what ought to be the case if something forbidden is true, for example: ‘If she is guilty, she should confess’. Contrary-to-duty obligations are important in our moral and legal thinking. Therefore, we want to be able to find an adequate symbolisation of such obligations in some logical system, a task that has turned out (...) to be difficult. This is shown by the so-called contrary-to-duty paradox. I will investigate and evaluate one kind of solution to this problem that has been suggested in the literature, which I will call the ‘counterfactual solution’. I will use some recent systems that combine not only counterfactual logic and deontic logic, but also temporal logic, in my analysis of the paradox. I will argue that the counterfactual solution has many attractive features and that it can give a fairly satisfactory answer to some examples of the contrary-to-duty paradox, but that it nevertheless has some serious problems. The conclusion is that, notwithstanding the many attractive features of the solution, there seem to be other approaches to the paradox that are more promising. (shrink)

This chapter provides a selective survey of prominent theories of the semantics of deontic modals in logic and natural language. We focus on Kratzer’s (1977; 1981; 1991) semantics and extensions to this analysis. Kratzer’s semantics has been far and away the most influential theory of deontic modals, which provide a base case for the interpretation of normative language in general. Understanding the logic and truth-conditions of normative language is one of the core areas of metaethics. It informs our understanding of (...) normative arguments and normative reasoning. As this chapter will emphasize, some forms of normative language don’t allow for the inferences that classical logic trains philosophers to expect. Understanding what inferences are valid for normative language should impact our under-standing of how we reason, and should reason, about the normative. We will first look at how deontic modals are understood in the context of modal logic and natural language. Then we’ll survey some recent debates and discoveries in the literature on deontic modals in natural language. We close with some considerations about the relevance of natural language to metaethics. (shrink)

Kiesewetter defends the normativity of rationality by presenting a new solution to the problems that arise from the common assumption that we ought to be rational. He provides a defence of a reason-response conception of rationality, an evidence-relative account of reason, and an explanation of structural irrationality in relation to these accounts.

Maximalism is the view that if an agent is permitted to perform a certain type of action (say, baking), this is in virtue of the fact that she is permitted to perform some instance of this type (say, baking a pie), where φ-ing is an instance of ψ-ing if and only if φ-ing entails ψ-ing but not vice versa. Now, the point of this paper is not to defend maximalism, but to defend a certain account of our options that when (...) combined with maximalism results in a theory that both avoids the sorts of objections that have typically been levelled against maximalism and accommodates the plausible idea that a moral theory must be collectively successful in the sense that everyone’s satisfying the theory guarantees that our theory-given aims will be best achieved. I argue that, for something to count as an option for an agent, it must, in the relevant sense, be under her control. And I argue that the relevant sort of control is the sort that we exercise over our reasons-responsive attitudes (e.g., our beliefs, desires, and intentions) by being both receptive and reactive to reasons. I call this sort of control rational control, and I call the view that φ-ing is an option for an agent if and only if she has rational control over whether she φs rationalism. When we combine this view with maximalism, we get rationalist maximalism, which I argue is a promising moral theory. (shrink)

Vít Gvoždiak published a reconciliatory analysis of Searle’s social ontology with semiotics in Gvoždiak (2012). Without prior knowledge of his paper, an analysis of the same subject appeared in Msimang (2014). Even though Searle’s social ontology is a common point of reference in the formulation of semiotics in these papers, it also serves as a point of departure in their understanding of semiotics and its development. The semiotic theory expressed in Gvoždiak (2012) is an inherently linguistic (speech act centred) theory, (...) whereas the semiotic theory presented in Msimang (2014) tends more towards a general theory of communicative systems in which social ontology, which follows from speech act theory, is an interesting part. It is my purpose in this note to contrast the two positions of semiotic theory as they appear in the aforementioned papers in reference to their appropriation of Searle’s social ontology. (shrink)

A dynamic semantics for iffy oughts offers an attractive alternative to the folklore that Chisholm's paradox enforces an unhappy choice between the intuitive inference rules of factual and deontic detachment. The first part of the story told here shows how a dynamic theory about ifs and oughts gives rise to a nonmonotonic perspective on deontic discourse and reasoning that elegantly removes the air of paradox from Chisholm's puzzle without sacrificing any of the two detachment principles. The second part of the (...) story showcases two bonus applications of the framework suggested here: it offers a response to Forrester's gentle murder paradox and avoids Kolodny and MacFarlane's miners paradox about deontic reasoning under epistemic uncertainty. A comparison between the dynamic semantic proposal made in this paper and a more conservative approach combining a static semantics with a dynamic pragmatics is provided. (shrink)

Overview of fundamental work in deontic logic.

Moti Mizrahi (2013) presents some novel counterexamples to Hypothetical Syllogism (HS) for indicative conditionals. I show that they are not compelling as they neglect the complicated ways in which conditionals and modals interact. I then briefly outline why HS should nevertheless be rejected.

A paradox is presented that the poses new problems for both the truth norm and the knowledge norm of belief.

“S ought (not) to see to it that p at t” is true iff an intention on the part of S to see to it that p at t is (in) correct. From this truth condition follows an understanding of the conceptual role of ought-claims in practical inference: ought-claims are interchangeable with intentions having the same content. From this conceptual role, it is quite clear why first-person, present-tense ought-judgments, and just those, motivate: failure to be motivated is a failure of (...) rationality. The point and purpose of 'ought' is mainly to express the results of practical reasoning performed on premises held hypothetically, an exercise of which there are many varieties. Our capacity for normative thought and language is a consequence of, and intimately related to, our capacity for practical thought. (shrink)

We investigate under what conditions contrary-to-duty (CTD) structures lacking temporal and action elements can be given a coherent reading. We argue, contrary to some recent proposals, that CTD is not an instance of defeasible reasoning, and that methods of nonmonotonic logics are inadequate since they are unable to distinguish between defeasibility and violation of primary obligations. We propose a semantic framework based on the idea that primary and CTD obligations are obligations of different kinds: a CTD obligation pertains to, or (...) pre-supposes, a certain context in which a primary obligation is already violated. This framework is presented initially as an extension of Standard Deontic Logic (SDL), a normal modal logic of type KD, and is illustrated by application to a series of examples. The concluding section is concerned with some resemblances between CTD and defeasible reasoning. We show first that the SDL-based framework contains a flaw and must be adjusted. A discussion of possible adjustments, including an alternative treatment in terms of a preference-based semantics, reveals difficulties that are reminiscent of problems in defeasible reasoning and intensional accounts of defeasible conditionals. (shrink)

This paper presents a nonmonotonic deontic logic based on commonsense entailment. It establishes criteria a successful account of obligation should satisfy, and develops a theory that satisfies them. The theory includes two conditional notions of prima facie obligation. One is constitutive; the other is epistemic, and follows nonmonotonically from the constitutive notion. The paper defines unconditional notions of prima facie obligation in terms of the conditional notions.

This article proposes a systematic application of recent developments in the logic of preference to a number of topics in deontic logic. The key junction is the well‐known Hansson conditional for dyadic obligations. These conditionals are generalized by pairing them with reasoning about syntactic priority structures. The resulting two‐level approach to obligations is tested first against standard scenarios of contrary‐to‐duty obligations, leading also to a generalization for the Kanger‐Anderson reduction of deontic logic. Next, the priority framework is applied to model (...) two intuitively different sorts of deontic dynamics of obligations, based on information changes and on genuine normative events. In this two‐level setting, we also offer novel takes on vexed issues such as the Chisholm paradox and modelling strong permission. Finally, the priority framework is shown to provide a unifying setting for the study of operations on norms as such, in particular, adding or deleting individual norms, and even merging whole norm systems in different manners. (shrink)

This paper concerns a thorny problem posed by conditional requirements: we expect some modal conditionals of the form ‘if p, then it must be that p’ to be false, yet they all come out true given two basic assumptions needed to account for ordinary conditional requirement like ‘If the light turns red, then cars must stop’. The first assumption is a semantic expectation linking conditional requirements with absolute ones, the second is the possible-worlds-based definition of modal sentences. Keeping the former (...) means giving up the latter, a result with profound implications for Kratzer’s formal semantics, modifications thereof, as well as deontic logics. (shrink)

Diagnosis theory reasons about incomplete knowledge and only considers the past. It distinguishes between violations and non-violations. Qualitative decision theory reasons about decision variables and considers the future. It distinguishes between fulfilled goals and unfulfilled goals. In this paper we formalize normative diagnoses and decisions in the special purpose formalism DIO(DE)2 as well as in extensions of the preference-based deontic logic PDL. The DIagnostic and DEcision-theoretic framework for DEontic reasoning DIO(DE)2 formalizes reasoning about violations and fulfillments, and is used to (...) characterize the distinction between normative diagnosis theory and (qualitative) decision theory. The extension of the preference-based deontic logic PDL shows how normative diagnostic and decision-theoretic reasoning — i.e. reasoning about violations and fulfillments — can be formalized as an extension of deontic reasoning. (shrink)

This article proposes a systematic application of recent developments in the logic of preference to a number of topics in deontic logic. The key junction is the well-known Hansson conditional for dyadic obligations. These conditionals are generalized by pairing them with reasoning about syntactic priority structures. The resulting two-level approach to obligations is tested first against standard scenarios of contrary-to-duty obligations, leading also to a generalization for the Kanger-Anderson reduction of deontic logic. Next, the priority framework is applied to model (...) two intuitively different sorts of deontic dynamics of obligations, based on information changes and on genuine normative events. In this two-level setting, we also offer novel takes on vexed issues such as the Chisholm paradox and modelling strong permission. Finally, the priority framework is shown to provide a unifying setting for the study of operations on norms as such, in particular, adding or deleting individual norms, and even merging whole norm systems in different manners. (shrink)

Conditionals that contain a modality in the consequent give rise to a particular semantic phenomenon whereby the antecedent of the conditional blocks possibilities when interpreting the modality in the consequent. This explains the puzzling logical behaviour of constructions like "If you don't buy a lottery ticket, you can't win", "If you eat that poison, it is unlikely that you will survive the day" and "If you kill Harry, you ought to kill him gently". In this paper it is argued that (...) a semantic version of the Ramsey Test provides a key in the analysis of such constructions. The logic for this semantics is axiomatized and some examples are studied, among them a well-known puzzle for contrary-to-duty obligations. (shrink)

This thesis is about indicative conditionals and apparent counterexamples to classically valid argument forms. Specifically, it applies the following four theories: - material (inspired by Grice (1961, 1975 and 1989)); - possible-worlds (inspired by Stalnaker (1981); Lewis (1976); and Kratzer (2012)), - suppositional (inspired by Adams (1975) and Edgington (1995 and 2014)); and - hybrid (inspired by Jackson (1987)) to try and solve the following two counterexamples: - Vann McGee’s to modus ponens (1985); and - Lewis Carroll’s to modus tollens (...) (1894). I argue that none of the theories I consider can explain – without facing any problems – the three individually plausible but jointly inconsistent theses that give rise to the apparent counterexamples. The three theses of the trilemma are the following: (1) the argument is invalid; (2) the argument is an instance of modus ponens or modus tollens; and (3) modus ponens or modus tollens is valid. (shrink)

This book offers a state-of-the-art introduction to the basic techniques and results of neighborhood semantics for modal logic. In addition to presenting the relevant technical background, it highlights both the pitfalls and potential uses of neighborhood models – an interesting class of mathematical structures that were originally introduced to provide a semantics for weak systems of modal logic. In addition, the book discusses a broad range of topics, including standard modal logic results ; bisimulations for neighborhood models and other model-theoretic (...) constructions; comparisons with other semantics for modal logic ; neighborhood semantics for first-order modal logic, applications in game theory ; applications in epistemic logic ; and non-normal modal logics with dynamic modalities. The book can be used as the primary text for seminars on philosophical logic focused on non-normal modal logics; as a supplemental text for courses on modal logic, logic in AI, or philosophical logic ; or as the primary source for researchers interested in learning about the uses of neighborhood semantics in philosophical logic and game theory. (shrink)

Our preferences determine how we act and think, but exactly what the mechanics are and how they work is a central cause of concern in many disciplines. This book uses techniques from modern logics of information flow and action to develop a unified new theory of what preference is and how it changes. The theory emphasizes reasons for preference, as well as its entanglement with our beliefs. Moreover, the book provides dynamic logical systems which describe the explicit triggers driving preference (...) change, including new information, suggestions, and commands. In sum, the book creates new bridges between many fields, from philosophy and computer science to economics, linguistics, and psychology. For the experienced scholar access to a large body of recent literature is provided and the novice gets a thorough introduction to the action and techniques of dynamic logic. (shrink)

Many philosophers think that requirements of rationality are “wide-scope”. That is to say: they are requirements to satisfy some material conditional, such that one counts as satisfying the requirement iff one either makes the conditional’s antecedent false or makes its consequent true. These contrast with narrow-scope requirements, where the requirement takes scope only over the consequent of the conditional. Many of the philosophers who have preferred wide-scope requirements to narrow-scope requirements have also endorsed a corresponding semantic claim, namely that ordinary (...) talk about rationality, despite appearances to the contrary, expresses wide-scope claims. In doing so, they seek to avoid attributing massive error to ordinary speakers. However, it is becoming increasingly clear that the wide-scope semantics inadequately captures the meaning of ordinary talk about rationality. It seems, then, that we are left with a dilemma: either give up the view that requirements of rationality are wide-scope, or accept an implausible semantics for ordinary talk about rationality, or attribute massive error to speakers. In this paper, I argue that this dilemma is only apparent, since we can appeal to a standard kind of contextualist semantics for modals to explain why narrow-scope talk comes out true in virtue of the wide-scope requirements. My view, then, combines wide-scoping about the explanatorily fundamental requirements of rationality with a contextualist variant of a narrow-scope semantics. I argue that this view gives us the best of both worlds, as well as solving related puzzles and challenges for the extant views in the literature. (shrink)

Lou Goble proposed powerful conditional deontic logics (CDPM) that are able to deal with deontic conflicts by means of restricting the inheritance principle. One of the central problems for dyadic deontic logics is to properly treat the restricted applicability of the principle “strengthening the antecedent”. In most cases it is desirable to derive from an obligation A under condition B, that A is also obliged under condition B and C. However, there are important counterexamples. Goble proposed a weakened rational monotonicity (...) principle to tackle this problem. This solution is suboptimal as it is for some examples counter-intuitive or even leads to explosion. The paper identifies also other problems of Goble’s systems. For instance, to make optimal use of the restricted inheritance principle, in many cases the user has to manually add certain statements to the premises. An adaptive logic framework based on CDPM is proposed which is able to tackle these problems. It allows for certain rules to be applied as much as possible. In this way counter-intuitive consequences as well as explosion can be prohibited and no user interference is required. Furthermore, for non-conflicting premise sets the adaptive logics are equivalent to Goble’s dyadic version of standard deontic logic. (shrink)

This paper offers a two dimensional variation of Standard Deontic Logic SDL, which we call 2SDL. Using 2SDL we can show that we can overcome many of the difficulties that SDL has in representing linguistic sets of Contrary-to-Duties (known as paradoxes) including the Chisholm, Ross, Good Samaritan and Forrester paradoxes. We note that many dimensional logics have been around since 1947, and so 2SDL could have been presented already in the 1970s. Better late than never! As a detailed case study (...) illustrating the power of 2SDL, we examine the system DL of Deontic Logic of Andrew Jones and Ingmar Pörn offered in 1985 to solve the Chisholm paradox of Contrary to Duties. The critical examination is done using logics and methods available in 1985 and solutions are proposed using what was available in 1985. (shrink)

This paper defends a counterexample to Modus Tollens, and uses it to draw some conclusions about the logic and semantics of indicative conditionals and probability operators in natural language. Along the way we investigate some of the interactions of these expressions with 'knows', and we call into question the thesis that all knowledge ascriptions have truth-conditions.

The crucial feature of obligation sentences to which the puzzles point is that such sentences, and evaluative sentences more generally, are defeasible. They may be warranted, given some information, only to be defeated by further information. A theory that recognizes this no longer needs to see conditional obligation as anything more than a simple combination of unary obligation and the conditional.

G3-style sequent calculi for the logics in the cube of non-normal modal logics and for their deontic extensions are studied. For each calculus we prove that weakening and contraction are height-preserving admissible, and we give a syntactic proof of the admissibility of cut. This implies that the subformula property holds and that derivability can be decided by a terminating proof search whose complexity is in Pspace. These calculi are shown to be equivalent to the axiomatic ones and, therefore, they are (...) sound and complete with respect to neighbourhood semantics. Finally, a Maehara-style proof of Craig’s interpolation theorem for most of the logics considered is given. (shrink)

(See also the separate entry for the volume itself.) This introduction has three parts. The first providing an overview of some main lines of research in deontic logic: the emergence of SDL, Chisholm's paradox and the development of dyadic deontic logics, various other puzzles/challenges and areas of development, along with philosophical applications. The second part focus on some actual and potential fruitful interactions between deontic logic, computer science and artificial intelligence. These include applications of deontic logic to AI knowledge representation (...) in legal systems, to modelling computer systems where it is expected that sub-ideal states will emerge and require countermeasures, to norm-governed human interactions with computer systems, and to the representation of some features of multi-agent systems where different agent-like computer systems interact with one another. The third and final part briefly groups and previews the papers in the anthology. (shrink)

There has been considerable debate regarding the relative merits of two theses about moral obligation known as actualism and possibilism. Both theses seek to give expression to the general idea that one ought to do the best one can. According to actualism, one’s obligations turn on what would happen if one chose some course of action, whereas, according to possibilism, they turn on what could happen if one chose some course of action. There are two strands to the debate: the (...) substantive verdicts that the two theses render in particular cases, and the accounts that they yield of the conceptual structure of moral obligation. Possibilism is conceptually appealing, whereas actualism is not, but the latter may seem to render superior substantive verdicts. In this paper, it is argued that, by turning from the objectivist’s emphasis on what is actually best to the prospectivist’s emphasis on what one’s evidence indicates is best, possibilists can provide an account of moral obligation that is both conceptually and substantively attractive. (shrink)

A new deontic logic, Action Type Deontic Logic, is presented. To motivate this logic, a number of benchmark cases are shown, representing inferences a deontic logic should validate. Some of the benchmark cases are singled out for further comments and some formal approaches to deontic reasoning are evaluated with respect to the benchmark cases. After that follows an informal introduction to the ideas behind the formal semantics, focussing on the distinction between action types and action tokens. Then the syntax and (...) semantics of Action Type Deontic Logic is presented and it is shown to meet the benchmarks. Finally, possibilities for further research are indicated. In the appendix, decidability of the satisfiability of formulas is proved via a technique known from monadic First Order Logic. (shrink)

Conciliationism is a view—well, a family of views—in the epistemology of disagreement. The idea, simply put, is that, in a wide range of cases where you find yourself in disagreement with another reasoner about the truth of some proposition, you are rationally obliged to adjust your credence in the direction of hers. Conciliationism enjoys a fair bit of prima facie plausibility. Most versions of it, however, suffer from a common (and rather obvious) problem: self-incrimination. Although there is some recognition in (...) the recent literature on disagreement that there is a problem in this vicinity, it, more often than not, is completely ignored. When it is not completely ignored, both the conciliationists and their critics tend to frame the problem incorrectly. As a result, they end up focusing most of their attention on pseudo-problems rather than the genuine worry. Furthermore, even when they have hit on the genuine worry, conciliationists have tended to give surprising and implausible responses to it. Even by their own lights, conciliationists aren’t justified in believing their conciliationist theses. In this paper, I argue that it’s time to come clean. Time to look for a view on the epistemic significance of disagreement that we can actually be justified in believing. (shrink)

There is a renewed debate about modus ponens. Strikingly, the recent counterexamples in Cantwell, Dreier and MacFarlane and Kolodny are generated by restricted readings of the ‘if’-clause. Moreover, it can be argued on general grounds that the restrictor view of conditionals developed in Kratzer and Lewis leads to counterexamples to modus ponens. This paper provides a careful analysis of modus ponens within the framework of the restrictor view. Despite appearances to the contrary, there is a robust sense in which modus (...) ponens is valid, owing to the fact that conditionals do not only allow for restricted readings but have bare interpretations, too. (shrink)

The standard semantics for modality, together with the influential restrictor analysis of conditionals (Kratzer, 1986, 2012) renders conditional ought claims like “If John’s stealing, he ought to be stealing” trivially true. While this might seem like a problem specifically for the restrictor analysis, the issue is far more general. Any account must predict that modals in the consequent of a conditional sometimes receive obligatorily unrestricted interpretation, as in the example above, but sometimes appear restricted, as in, e.g., “If John’s speeding, (...) he ought to pay the fine.” And the problem runs deeper, for there are non-conditional variants of the data. Thus, the solution cannot lie in adopting a particular analysis of conditionals, nor a specific account of the interaction between conditionals and modals. Indeed, with minimal assumptions, the standard account of modality will render a myriad of claims about what one ought to, must, or may, do trivially true. Worse, the problem extends to a wide range of non-deontic modalities, including metaphysical modality. But the disaster has a remedy. I argue that the source of the problem for the standard account lies in its failure to capture an inferential evidence constraint encoded in the meaning of a wide range of modal constructions. I offer an account that captures this constraint, and show it provides a general and independently motivated solution to the problem. (shrink)

In this paper we present a Gentzen system for reasoning with contrary-to-duty obligations. The intuition behind the system is that a contrary-to-duty is a special kind of normative exception. The logical machinery to formalise this idea is taken from substructural logics and it is based on the definition of a new non-classical connective capturing the notion of reparational obligation. Then the system is tested against well-known contrary-to-duty paradoxes.

Centered around the analysis of the prescriptive portion of the Vedas, the Sanskrit philosophical school of Mīmāṃsā provides a treasure trove of normative investigations. We focus on the leading Mīmāṃsā authors Prabhākara, Kumārila and Maṇḍana, and discuss three modal logics that formalize their deontic theories. In the first part of this paper, we use logic to analyze, compare and clarify the various solutions to the _śyena_ controversy, a two-thousand-year-old problem arising from seemingly conflicting commands in the Vedas. In the second (...) part, the formalized Mīmāṃsā theories are analyzed and employed to provide alternative perspectives on well-known paradoxes from the contemporary field of deontic logic. Thus, we go from logic to Mīmāṃsā and back again. (shrink)

Recent years have yielded many discussions on how to endow autonomous agents with the ability to make ethical decisions, and the need for explicit ethical reasoning and transparency is a persistent theme in this literature. We present a modular and transparent approach to equip autonomous agents with the ability to comply with ethical prescriptions, while still enacting pre-learned optimal behaviour. Our approach relies on a normative supervisor module, that integrates a theorem prover for defeasible deontic logic within the control loop (...) of a reinforcement learning agent. The supervisor operates as both an event recorder and an on-the-fly compliance checker w.r.t. an external norm base. We successfully evaluated our approach with several tests using variations of the game Pac-Man, subject to a variety of “ethical” constraints. (shrink)