Formal Social Epistemology, Misc

There is a commonly made distinction between two types of scientists: risk-taking, trailblazing mavericks and detail-oriented followers. A number of recent papers have discussed the question what a desirable mixture of mavericks and followers looks like. Answering this question is most useful if a scientific community can be steered toward such a desirable mixture. One attractive route is through credit incentives: manipulating rewards so that reward-seeking scientists are likely to form the desired mixture of their own accord. Here I argue (...) that this idea is less straightforward than it may seem. Interpreting mavericks as scientists who prioritize rewards over speed and risk, I show in a deliberatively simple model that there is a fixed mixture which is not particularly likely to be desirable and which credit incentives cannot alter. I consider a way around this result, but this has some major drawbacks. I conclude that credit incentives are not as promising a way to create a desirable mixture of mavericks and followers as one might have thought. (shrink)

Recent philosophical work has praised the reward structure of science, while recent empirical work has shown that many scientific results may not be reproducible. I argue that the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work, thus contributing to the so-called reproducibility crisis. I use a rational choice model to identify a set of sufficient conditions for this problem to arise, and I argue that these conditions plausibly (...) apply to a wide range of research situations. Currently proposed solutions will not fully address this problem. Philosophical commentators should temper their optimism about the reward structure of science. (shrink)

In this paper, we explain and showcase the promising methodology of testimonial network analysis and visualization for experimental epistemology, arguing that it can be used to gain insights and answer philosophical questions in social epistemology. Our use case is the epistemic community that discusses vaccine safety primarily in English on Twitter. In two studies, we show, using both statistical analysis and exploratory data visualization, that there is almost no neutral or ambivalent discussion of vaccine safety on Twitter. Roughly half the (...) accounts engaging with this topic are pro-vaccine, while the other half are con-vaccine. We also show that these two camps rarely engage with one another, and that the con-vaccine camp has greater epistemic reach and receptivity than the pro-vaccine camp. In light of these findings, we question whether testimonial networks as they are currently constituted on popular fora such as Twitter are living up to their promise of delivering the wisdom of crowds. We conclude by pointing to directions for further research in digital social epistemology. (shrink)

In everyday life and in science we acquire evidence of evidence and based on this new evidence we often change our epistemic states. An assumption underlying such practice is that the following EEE Slogan is correct: 'evidence of evidence is evidence' (Feldman 2007, p. 208). We suggest that evidence of evidence is best understood as higher-order evidence about the epistemic state of agents. In order to model evidence of evidence we introduce a new powerful framework for modelling epistemic states, Dyadic (...) Bayesianism. Based on this framework, we then discuss characterizations of evidence of evidence and argue for one of them. Finally, we show that whether the EEE Slogan holds, depends on the specific kind of evidence of evidence. (shrink)

According to an influential Enlightenment ideal, one shouldn't rely epistemically on other people's say-so, at least not if one is in a position to evaluate the relevant evidence for oneself. However, in much recent work in social epistemology, we are urged to dispense with this ideal, which is seen as stemming from a misguided focus on isolated individuals to the exclusion of groups and communities. In this paper, I argue that that an emphasis on the social nature of inquiry should (...) not lead us to entirely abandon the Enlightenment ideal of epistemically autonomous agents. Specifically, I suggest that it is an appropriate ideal for those who serve as experts in a given epistemic community, and develop a notion of expert acceptance to make sense of this. I go on to show that, all other things being equal, this kind of epistemic autonomy among experts makes their joint testimony more reliable, which in turn brings epistemic benefits both to laypeople and to experts in other fields. (shrink)

Merging of opinions results underwrite Bayesian rejoinders to complaints about the subjective nature of personal probability. Such results establish that sufficiently similar priors achieve consensus in the long run when fed the same increasing stream of evidence. Initial subjectivity, the line goes, is of mere transient significance, giving way to intersubjective agreement eventually. Here, we establish a merging result for sets of probability measures that are updated by Jeffrey conditioning. This generalizes a number of different merging results in the literature. (...) We also show that such sets converge to a shared, maximally informed opinion. Convergence to a maximally informed opinion is a (weak) Jeffrey conditioning analogue of Bayesian “convergence to the truth” for conditional probabilities. Finally, we demonstrate the philosophical significance of our study by detailing applications to the topics of dynamic coherence, imprecise probabilities, and probabilistic opinion pooling. (shrink)

It has been argued that an epistemically rational agent’s evidence is subjectively mediated through some rational epistemic standards, and that there are incompatible but equally rational epistemic standards available to agents. This supports Permissiveness, the view according to which one or multiple fully rational agents are permitted to take distinct incompatible doxastic attitudes towards P (relative to a body of evidence). In this paper, I argue that the above claims entail the existence of a unique and more reliable epistemic standard. (...) My strategy relies on Condorcet’s Jury Theorem. This gives rise to an important problem for those who argue that epistemic standards are permissive, since the reliability criterion is incompatible with such a type of Permissiveness. (shrink)

The communist norm requires that scientists widely share the results of their work. Where did this norm come from, and how does it persist? Michael Strevens provides a partial answer to these questions by showing that scientists should be willing to sign a social contract that mandates sharing. However, he also argues that it is not in an individual credit-maximizing scientist's interest to follow this norm. I argue against Strevens that individual scientists can rationally conform to the communist norm, even (...) in the absence of a social contract or other ways of socially enforcing the norm, by proving results to this effect in a game-theoretic model. This shows that the incentives provided to scientists through the priority rule are sufficient to explain both the origins and the persistence of the communist norm, adding to previous results emphasizing the benefits of the incentive structure created by the priority rule. (shrink)

Several recent results on the aggregation of judgments over logically connected propositions show that, under certain conditions, dictatorships are the only propositionwise aggregation functions generating fully rational (i.e., complete and consistent) collective judgments. A frequently mentioned route to avoid dictatorships is to allow incomplete collective judgments. We show that this route does not lead very far: we obtain oligarchies rather than dictatorships if instead of full rationality we merely require that collective judgments be deductively closed, arguably a minimal condition of (...) rationality, compatible even with empty judgment sets. We derive several characterizations of oligarchies and provide illustrative applications to Arrowian preference aggregation and Kasher and Rubinsteinís group identification problem. (shrink)

Should editors of scientific journals practice triple-anonymous reviewing? I consider two arguments in favor. The first says that insofar as editors’ decisions are affected by information they would not have had under triple-anonymous review, an injustice is committed against certain authors. I show that even well-meaning editors would commit this wrong and I endorse this argument. The second argument says that insofar as editors’ decisions are affected by information they would not have had under triple-anonymous review, it will negatively affect (...) the quality of published papers. I distinguish between two kinds of biases that an editor might have. I show that one of them has a positive effect on quality and the other a negative one, and that the combined effect could be either positive or negative. Thus I do not endorse the second argument in general. However, I do endorse this argument for certain fields, for which I argue that the positive effect does not apply. (shrink)

I argue elsewhere (Roche 2014) that evidence of evidence is evidence under screening-off. Tal and Comesaña (2017) argue that my appeal to screening-off is subject to two objections. They then propose an evidence of evidence thesis involving the notion of a defeater. There is much to learn from their very careful discussion. I argue, though, that their objections fail and that their evidence of evidence thesis is open to counterexample.

I show that the social stratification of academic science can arise as a result of academics’ preference for reading work of high epistemic value. This is consistent with a view on which academic superstars are highly competent academics, but also with a view on which superstars arise primarily due to luck. I argue that stratification is beneficial if most superstars are competent, but not if most superstars are lucky. I also argue that it is impossible to tell whether most superstars (...) are in fact competent or lucky, or which group a given superstar belongs to, and hence whether stratification is overall beneficial. (shrink)

The majority rule has caught much attention in recent debate about the aggregation of judgments. But its role in finding the truth is limited. A majority of expert judgments is not necessarily authoritative, even if all experts are equally competent, if they make their judgments independently of each other, and if all the judgments are based on the same source of (good) evidence. In this paper I demonstrate this limitation by presenting a simple counterexample and a related general result. I (...) pave the way for this argument by introducing a Bayesian model of evidence and expert judgment in order to give a precise account of the basic problem. (shrink)

How can different individuals' probability assignments to some events be aggregated into a collective probability assignment? Classic results on this problem assume that the set of relevant events -- the agenda -- is a sigma-algebra and is thus closed under disjunction (union) and conjunction (intersection). We drop this demanding assumption and explore probabilistic opinion pooling on general agendas. One might be interested in the probability of rain and that of an interest-rate increase, but not in the probability of rain or (...) an interest-rate increase. We characterize linear pooling and neutral pooling for general agendas, with classic results as special cases for agendas that are sigma-algebras. As an illustrative application, we also consider probabilistic preference aggregation. Finally, we compare our results with existing results on binary judgment aggregation and Arrovian preference aggregation. This paper is the first of two self-contained, but technically related companion papers inspired by binary judgment-aggregation theory. (shrink)

A part of the scientific literature consists of intermediate results within a longer project. Scientists often publish a first result in the course of their work, while aware that they should soon achieve a more advanced result from this preliminary result. Should they follow the proverb “a bird in the hand is worth two in the bush”, and publish any intermediate result they get? This is the normative question addressed in this paper. My aim is to clarify, to refine, and (...) to assess informal arguments about the choice whether to publish intermediate results. To this end, I adopt a rational decision framework, supposing some utility or preferences, and I propose a formal model. The best publishing strategy turns out to depend on the research situation. In some simple circumstances, even selfish and short-minded scientists should publish their intermediate results, and should thus behave like their altruistic peers, i. e. like society would like them to behave. In other research situations, with inhomogeneous reward or difficulty profiles, the best strategy is opposite. These results suggest qualified philosophical morals. (shrink)

In this paper we propose and analyze a game-theoretic model of the epistemology of peer disagreement. In this model, the peers' rationality is evaluated in terms of their probability of ending the disagreement with a true belief. We find that different strategies---in particular, one based on the Steadfast View and one based on the Conciliatory View---are rational depending on the truth-sensitivity of the individuals involved in the disagreement. Interestingly, the Steadfast and the Conciliatory Views can even be rational simultaneously in (...) some circumstances. We tentatively provide some reasons to favor the Conciliatory View in such cases. We argue that the game-theoretic perspective is a fruitful one in this debate, and this fruitfulness has not been exhausted by the present paper. (shrink)

This paper explores two non-standard supermajority rules in the context of judgment aggregation over multiple logically connected issues. These rules set the supermajority threshold in a local, context sensitive way—partly as a function of the input profile of opinions. To motivate the interest of these rules, I prove two results. First, I characterize each rule in terms of a condition I call ‘Block Preservation’. Block preservation says that if a majority of group members accept a judgment set, then so should (...) the group. Second, I show that one of these rules is, in a precise sense, a judgment aggregation analogue of a rule for connecting qualitative and quantitative belief that has been recently defended by Hannes Leitgeb. The structural analogy is due to the fact that Leitgeb sets thresholds for qualitative beliefs in a local, context sensitive way—partly as a function of the given credence function. (shrink)

In solving judgment aggregation problems, groups often face constraints. Many decision problems can be modelled in terms the acceptance or rejection of certain propositions in a language, and constraints as propositions that the decisions should be consistent with. For example, court judgments in breach-of-contract cases should be consistent with the constraint that action and obligation are necessary and sufficient for liability; judgments on how to rank several options in an order of preference with the constraint of transitivity; and judgments on (...) budget items with budgetary constraints. Often more or less demanding constraints on decisions are imaginable. For instance, in preference ranking problems, the transitivity constraint is often contrasted with the weaker acyclicity constraint. In this paper, we make constraints explicit in judgment aggregation by relativizing the rationality conditions of consistency and deductive closure to a constraint set, whose variation yields more or less strong notions of rationality. We review several general results on judgment aggregation in light of such constraints. (shrink)

Collaboration is increasingly popular across academia. Collaborative work raises certain ethical questions, however. How will the fruits of collaboration be divided? How will the work for the collaborative project be split? In this paper, we consider the following question in particular. Are there ways in which these divisions systematically disadvantage certain groups? -/- We use evolutionary game theoretic models to address this question. First, we discuss results from O'Connor and Bruner (unpublished). In this paper, we show that underrepresented groups in (...) academia can be disadvantaged in such situations by dint of their small numbers. Second, we present novel results exploring how the hierarchical structure of academia can lead to bargaining disadvantage. We investigate models where one actor has a higher baseline of academic success, less to lose if collaboration goes south, or greater rewards for non-collaborative work. We show that in these situations, the less powerful partner is disadvantaged in bargaining over collaboration. (shrink)

How should a group with different opinions (but the same values) make decisions? In a Bayesian setting, the natural question is how to aggregate credences: how to use a single credence function to naturally represent a collection of different credence functions. An extension of the standard Dutch-book arguments that apply to individual decision-makers recommends that group credences should be updated by conditionalization. This imposes a constraint on what aggregation rules can be like. Taking conditionalization as a basic constraint, we gather (...) lessons from the established work on credence aggregation, and extend this work with two new impossibility results. We then explore contrasting features of two kinds of rules that satisfy the constraints we articulate: one kind uses fixed prior credences, and the other uses geometric averaging, as opposed to arithmetic averaging. We also prove a new characterisation result for geometric averaging. Finally we consider applications to neighboring philosophical issues, including the epistemology of disagreement. (shrink)

In this paper, I introduce the emerging theory of judgment aggregation as a framework for studying institutional design in social epistemology. When a group or collective organization is given an epistemic task, its performance may depend on its ‘aggregation procedure’, i.e. its mechanism for aggregating the group members’ individual beliefs or judgments into corresponding collective beliefs or judgments endorsed by the group as a whole. I argue that a group’s aggregation procedure plays an important role in determining whether the group (...) can meet two challenges: the ‘rationality challenge’ and the ‘knowledge challenge’. The rationality challenge arises when a group is required to endorse consistent beliefs or judgments; the knowledge challenge arises when the group’s beliefs or judgments are required to track certain truths. My discussion seeks to identify those properties of an aggregation procedure that affect a group’s success at meeting each of the two challenges. (shrink)

This paper generalises the classical Condorcet jury theorem from majority voting over two options to plurality voting over multiple options. The paper further discusses the debate between epistemic and procedural democracy and situates its formal results in that debate. The paper finally compares a number of different social choice procedures for many-option choices in terms of their epistemic merits. An appendix explores the implications of some of the present mathematical results for the question of how probable majority cycles (as in (...) Condorcet's paradox) are in large electorates. (shrink)

Suppose that the members of a group each hold a rational set of judgments on some interconnected questions, and imagine that the group itself has to form a collective, rational set of judgments on those questions. How should it go about dealing with this task? We argue that the question raised is subject to a difficulty that has recently been noticed in discussion of the doctrinal paradox in jurisprudence. And we show that there is a general impossibility theorem that that (...) difficulty illustrates. Our paper describes this impossibility result and provides an exploration of its significance. The result naturally invites comparison with Kenneth Arrow's famous theorem (Arrow, 1963 and 1984; Sen, 1970) and we elaborate that comparison in a companion paper (List and Pettit, 2002). The paper is in four sections. The first section documents the need for various groups to aggregate its members' judgments; the second presents the discursive paradox; the third gives an informal statement of the more general impossibility result; the formal proof is presented in an appendix. The fourth section, finally, discusses some escape routes from that impossibility. (shrink)

Meta-induction, in its various forms, is an imitative prediction method, where the prediction methods and the predictions of other agents are imitated to the extent that those methods or agents have proven successful in the past. In past work, Schurz demonstrated the optimality of meta-induction as a method for predicting unknown events and quantities. However, much recent discussion, along with formal and empirical work, on the Wisdom of Crowds has extolled the virtue of diverse and independent judgment as essential to (...) maintenance of 'wise crowds'. This suggests that meta-inductive prediction methods could undermine the wisdom of the crowd inasmuch these methods recommend that agents imitate the predictions of other agents. In this article, we evaluate meta-inductive methods with a focus on the impact on a group's performance that may result from including meta-inductivists among its members. In addition to considering cases of global accessibility (i.e., cases where the judgments of all members of the group are available to all of the group's members), we consider cases where agents only have access to the judgments of other agents within their own local neighborhoods. (shrink)

We evaluate the asymptotic performance of boundedly-rational strategies in multi-armed bandit problems, where performance is measured in terms of the tendency (in the limit) to play optimal actions in either (i) isolation or (ii) networks of other learners. We show that, for many strategies commonly employed in economics, psychology, and machine learning, performance in isolation and performance in networks are essentially unrelated. Our results suggest that the appropriateness of various, common boundedly-rational strategies depends crucially upon the social context (if any) (...) in which such strategies are to be employed. (shrink)

In the latter half of the twentieth century, philosophers of science have argued (implicitly and explicitly) that epistemically rational individuals might compose epistemically irrational groups and that, conversely, epistemically rational groups might be composed of epistemically irrational individuals. We call the conjunction of these two claims the Independence Thesis, as they together imply that methodological prescriptions for scientific communities and those for individual scientists might be logically independent of one another. We develop a formal model of scientific inquiry, define four (...) criteria for individual and group epistemic rationality, and then prove that the four definitions diverge, in the sense that individuals will be judged rational when groups are not and vice versa. We conclude by explaining implications of the inconsistency thesis for (i) descriptive history and sociology of science and (ii) normative prescriptions for scientific communities. (shrink)

While a large social-choice-theoretic literature discusses the aggregation of individual judgments into collective ones, there is much less formal work on the transformation of judgments in group communication. I develop a model of judgment transformation and prove a baseline impossibility theorem: Any judgment transformation function satisfying some initially plausible conditions is the identity function, under which no opinion change occurs. I identify escape routes from this impossibility and argue that the kind of group communication envisaged by deliberative democats must be (...) "holistic": It must focus on webs of connected propositions, not on one proposition at a time, which echoes the Duhem-Quine "holism thesis" on scientific theory testing. My approach provides a map of the logical space in which different possible group communication processes are located. (shrink)

Suppose several individuals (e.g., experts on a panel) each assign probabilities to some events. How can these individual probability assignments be aggregated into a single collective probability assignment? This article reviews several proposed solutions to this problem. We focus on three salient proposals: linear pooling (the weighted or unweighted linear averaging of probabilities), geometric pooling (the weighted or unweighted geometric averaging of probabilities), and multiplicative pooling (where probabilities are multiplied rather than averaged). We present axiomatic characterisations of each class of (...) pooling functions (most of them classic, but one new) and argue that linear pooling can be justified "procedurally" but not "epistemically", while the other two pooling methods can be justified "epistemically". The choice between them, in turn, depends on whether the individuals' probability assignments are based on shared information or on private information. We conclude by mentioning a number of other pooling methods. (shrink)

There is a substantial class of collective decision problems whose successful solution requires interdependence among decision makers at the agenda-setting stage and independence at the stage of choice. We define this class of problems and describe and apply a search-and-decision mechanism theoretically modeled in the context of honeybees and identified in earlier empirical work in biology. The honeybees’ mechanism has useful implications for mechanism design in human institutions, including courts, legislatures, executive appointments, research and development in firms, and basic research (...) in the sciences. Our paper offers a fresh perspective on the idea of “biomimicry” in institutional design and raises the possibility of comparative institutional analysis across species. (shrink)

The aim of this article is to introduce the theory of judgment aggregation, a growing interdisciplinary research area. The theory addresses the following question: How can a group of individuals make consistent collective judgments on a given set of propositions on the basis of the group members' individual judgments on them? I begin by explaining the observation that initially sparked the interest in judgment aggregation, the so-called "doctinal" and "discursive paradoxes". I then introduce the basic formal model of judgment aggregation, (...) which allows me to present some illustrative variants of a generic impossibility result. I subsequently turn to the question of how this impossibility result can be avoided, going through several possible escape routes. Finally, I relate the theory of judgment aggregation to other branches of aggregation theory. Rather than offering a comprehensive survey of the theory of judgment aggregation, I hope to introduce the theory in a succinct and pedagogical way, providing an illustrative rather than exhaustive coverage of some of its key ideas and results. (shrink)

How can the propositional attitudes of several individuals be aggregated into overall collective propositional attitudes? Although there are large bodies of work on the aggregation of various special kinds of propositional attitudes, such as preferences, judgments, probabilities and utilities, the aggregation of propositional attitudes is seldom studied in full generality. In this paper, we seek to contribute to filling this gap in the literature. We sketch the ingredients of a general theory of propositional attitude aggregation and prove two new theorems. (...) Our first theorem simultaneously characterizes some prominent aggregation rules in the cases of probability, judgment and preference aggregation, including linear opinion pooling and Arrovian dictatorships. Our second theorem abstracts even further from the specific kinds of attitudes in question and describes the properties of a large class of aggregation rules applicable to a variety of belief-like attitudes. Our approach integrates some previously disconnected areas of investigation. (shrink)

Under the independence and competence assumptions of Condorcet’s classical jury model, the probability of a correct majority decision converges to certainty as the jury size increases, a seemingly unrealistic result. Using Bayesian networks, we argue that the model’s independence assumption requires that the state of the world (guilty or not guilty) is the latest common cause of all jurors’ votes. But often – arguably in all courtroom cases and in many expert panels – the latest such common cause is a (...) shared ‘body of evidence’ observed by the jurors. In the corresponding Bayesian network, the votes are direct descendants not of the state of the world, but of the body of evidence, which in turn is a direct descendant of the state of the world. We develop a model of jury decisions based on this Bayesian network. Our model permits the possibility of misleading evidence, even for a maximally competent observer, which cannot easily be accommodated in the classical model. We prove that (i) the probability of a correct majority verdict converges to the probability that the body of evidence is not misleading, a value typically below 1; (ii) depending on the required threshold of ‘no reasonable doubt’, it may be impossible, even in an arbitrarily large jury, to establish guilt of a defendant ‘beyond any reasonable doubt’. (shrink)

According to a common view, prejudice always involves some form of epistemic culpability, i.e., a failure to respond to evidence in the appropriate way. I argue that the common view wrongfully assumes that prejudices always involve universal generalizations. After motivating the more plausible thesis that prejudices typically involve a species of generic judgment, I show that standard examples provide no grounds for positing a strong connection between prejudice and epistemic culpability. More generally, the common view fails to recognize the extent (...) to which prejudices are epistemically insidious: once they are internalized as background beliefs, they quite reasonably come to control the assessment and interpretation of new evidence. This property of insidiousness helps explain why prejudices are so recalcitrant to empirical counterevidence and also why they are frequently invisible to introspective reflection. (shrink)