Critiques of Experimental Philosophy

In some domains experts perform better than novices, and in other domains experts do not generally perform better than novices. According to empirical studies of expert performance, this is because the former but not the latter domains make available to training practitioners a direct form of learning feedback. Several philosophers resource this empirical literature to cast doubt on the quality of philosophical expertise. They claim that philosophy is like the dubious domains in that it does not make available the good, (...) direct kind of learning feedback, and thus there are empirical grounds for doubting the epistemic quality of philosophical expertise. I examine the empirical studies that are purportedly bad news for professional philosophers. On the basis of that examination, I provide three reasons why the empirical study of non-philosophical expertise does not undermine the status of philosophical expertise. First, the non-philosophical task-types from which the critics generalize are unrepresentative of relevant philosophical task-types. Second, empirical critiques of non-philosophical experts are often made relative to the performance of linear models—a comparison that is inapt in a philosophical context. Third, the critics fail to discuss findings from the empirical study of non-philosophical expertise that have more favorable implications for the epistemic status of philosophical expertise. In addition to discussing implications for philosophical expertise, this article makes progress in the philosophical analysis of the science of expertise and expert development. (shrink)

This is the general introduction to the edited collection "A companion to Experimental Philosophy".

This paper examines a constellation of ethical and editorial issues that have arisen since philosophers started to conduct, submit and publish empirical research. These issues encompass concerns over responsible authorship, fair treatment of human subjects, ethicality of experimental procedures, availability of data, unselective reporting and publishability of research findings. This study aims to assess whether the philosophical community has as yet successfully addressed such issues. To do so, the instructions for authors, submission process and published research papers of 29 main (...) journals in philosophy have been considered and analyzed. In light of the evidence reported here, it is argued that the philosophical community has as yet failed to properly tackle such issues. The paper also delivers some recommendations for authors, reviewers and editors in the field. (shrink)

Providing valid responses to a self-report survey requires cognitive effort. Subjects engaging in insufficient effort responding (IER) are unwilling to take this effort. Compared to psychologists, experimental philosophers so far seem to have paid less attention to IER. This paper is an attempt to begin to alleviate this shortcoming. First, I explain IER’s nature, prevalence and negative effects in self-report surveys in general. Second, I argue that IER might also affect experimental philosophy studies. Third, I develop recommendations as to how (...) experimental philosophers should (and should not) try to prevent IER. Fourth, I develop recommendations as to how experimental philosophers should (and should not) try to detect IER. Fifth, I sketch how experimental philosophers ought to proceed once a subject has been identified as an insufficient effort responder. And finally, I report the results of an online survey that addresses experimental philosophers’ current knowledge, consideration and assessment of IER. (shrink)

A growing body of psychological research seeks to understand how people's thinking comports with long-standing philosophical theories, such as whether they view ethical or aesthetic truths as subjective or objective. Yet such research can be critically undermined if it fails to accurately characterize the philosophical positions in question and fails to ensure that subjects understand them appropriately. We argue that a recent article by Rabb et al. (2020) fails to meet these demands and propose several constructive solutions for future research.

In domains like stock brokerage, clinical psychiatry, and long‐term political forecasting, experts generally fail to outperform novices. Empirical researchers agree on why this is: experts must receive direct or environmental learning feedback during training to develop reliable expertise, and these domains are deficient in this type of feedback. A growing number of philosophers resource this consensus view to argue that, given the absence of direct or environmental philosophical feedback, we should not give the philosophical intuitions or theories of expert philosophers (...) greater credence than those of novice philosophers. This article has three objectives. The first is to explore several overlooked issues concerning the strategy of generalizing from empirical studies of non‐philosophical expertise to the epistemic status of philosophical expertise. The second is to explain why empirical research into a causal relationship between direct learning feedback and enhanced expert performance does not provide good grounds for abandoning a default optimism about the epistemic superiority of expert philosophical theories. The third is to sketch a positive characterization of learning feedback that addresses developmental concerns made salient by the empirical literature on expert performance for specifically theory‐driven or “armchair” domains like philosophy. (shrink)

Recent empirical studies of philosophers by Eric Schwitzgebel and others have seriously called into question whether professional ethicists have any useful expertise with thought experiments, given that their intuitions appear to be no more reliable than those of lay subjects. Drawing on such results, sceptics like Edouard Machery argue that normative ethics as it is currently practiced is deeply problematic. In this paper, I present two main arguments in defense of the standard methodology of normative ethics. First, there is strong (...) reason to believe that expertise with thought experiments requires considering scenarios in their proper theoretical context and in parallel with other pertinent situations, so that we should not expect philosophers to be better than lay folk at responding to decontextualized cases. Second, skeptical views underestimate the epistemic benefits of the actual practices of post-processing initial verdicts both at individual and social levels. Contrary to a mythical conception of ‘the method of cases’, philosophers are frequently sensitive to the quality of intuitive evidence, reject and revise their verdicts on the basis of independently supported principles or interpersonal criticism, and defer to recognized specialists. (shrink)

This is the second book by Baz that aims to show that a big chunk of contemporary philosophy is fundamentally misguided. His first book, When Words Are Called For: A Defense of Ordinary Language Philosophy (2012) adopted a therapeutic approach (in the Wittgensteinian style) to problems in contemporary epistemology, arguing that when properly thought through, the way philosophers talk about ‘knowing’ that something is the case ultimately does not make sense. Baz’s goal in his second book is less therapeutic and (...) more constructive: he aims to start a methodological revolution (in the Kuhnian sense)—to shake contemporary philosophers out of the unconscious habits of normal science and provoke them into making a radical change in the methods they use to do philosophy and the basic assumptions that motivate those methods. (shrink)

This paper defends a challenge, inspired by arguments drawn from contemporary ordinary language philosophy and grounded in experimental data, to certain forms of standard philosophical practice. There has been a resurgence of philosophers who describe themselves as practicing "ordinary language philosophy". The resurgence can be divided into constructive and critical approaches. The critical approach to neo-ordinary language philosophy has been forcefully developed by Baz (2012a,b, 2014, 2015, 2016, forthcoming), who attempts to show that a substantial chunk of contemporary philosophy is (...) fundamentally misguided. I describe Baz's project and argue that while there is reason to be skeptical of its radical conclusion, it conveys an important truth about discontinuities between ordinary uses of philosophically significant expressions ("know", e.g.) and their use in philosophical thought experiments. I discuss some evidence from experimental psychology and behavioral economics indicating that there is a risk of overlooking important aspects of meaning or misinterpreting experimental results by focusing only on abstract experimental scenarios, rather than employing more diverse and more ecologically valid experimental designs. I conclude by presenting a revised version of the critical argument from ordinary language. (shrink)

The file associated with this record is under embargo while permission to archive is sought from the publisher. The full text may be available through the publisher links provided above.

Examines the relevance of empirical studies of responsibility judgments for traditional philosophical concerns about free will and moral responsibility. We argue that experimental philosophy is relevant to the traditional debates, but that setting up experiments and interpreting data in just the right way is no less difficult than negotiating traditional philosophical arguments. Both routes are valuable, but so far neither promises a way to secure significant agreement among the competing parties. To illustrate, we focus on three sorts of issues. For (...) illustration, we discuss an error theory for incompatibilist intuitions proposed by Eddy Nahmias and colleagues, the role that empirical studies might have in the assessment of manipulation arguments for incompatibilism, and the suggestion that empirical studies reveal that core criteria for moral responsibility ought not to be applied invariantly across different sorts of cases. (shrink)

In recent years, some defenders of traditional philosophical methodology have argued that certain critiques of armchair methods are mistaken in assuming that intuitions play central evidential roles in traditional philosophical methods. According to this kind of response, experimental philosophers attack a straw man; it doesn’t matter whether intuitions are reliable, because philosophers don’t use intuitions in the way assumed. Deutsch (2010), Williamson (2007), and Cappelen (2012) all defend traditional methods in something like this way. I also endorsed something like this (...) line in Ichikawa (2014a). -/- In this contribution, I will follow up on this sort of defence of traditional philosophical methods in three ways. In §1, I will rehearse and extend some of my reasons for challenging the idea that traditional methods depend on intuitions in an evidential role. (My reasons are very different from those discussed in (Cappelen, 2012).) I will also engage with some recent more sophisticated attempts to establish the idea that intuitions play evidential roles in philosophy, such as that of Chudnoff (2013). In §2, I will consider and argue against a dismissive response to such positions from experimental philosophers, who consider the question of philosophical reliance on intuitions to be irrelevant to the experimentalist critique. But in §3, I will argue that it would also be a mistake to conclude (as Herman Cappelen does) that the critique is rendered totally irrelevant by the denial of the evidential role of intuitions; I defend a more moderate view on which the bearing of experimental studies of philosophical intuitions is relevant for philosophical methodology, but only in a relatively limited way. (shrink)

Much of the recent movement organized under the heading “Experimental Philosophy” has been concerned with the empirical study of responses to thought experiments drawn from the literature on philosophical analysis. I consider what bearing these studies have on the traditional projects in which thought experiments have been used in philosophy. This will help to answer the question what the relation is between Experimental Philosophy and philosophy, whether it is an “exciting new style of [philosophical] research”, “a new interdisciplinary field that (...) uses methods normally associated with psychology to investigate questions normally associated with philosophy” (Knobe et al. 2012), or whether its relation to philosophy consists, as some have suggested, in no more than the word ‘philosophy’ appearing in its title, or whether the truth lies somewhere in between these two views. I first distinguishes different strands in Experimental Philosophy, negative and positive x-phi, and x-phi pursuing philosophy as opposed to x-phi as cognitive science. Next I review some ways in which Experimental Philosophy has been criticized. Finally, I consider what would have to be true for Experimental Philosophy to have one or another sort of relevance to philosophy, whether the assumptions required are true, how we could know it, and the ideal limits of the usefulness Experimental Philosophy to philosophy. I conclude x-phi cannot in principle be a replacement for traditional first person approaches because it yields the wrong kind of knowledge and that it can nonetheless be a practical aid in conducting philosophical thought experiments. n. (shrink)

Once symbolized by a burning armchair, experimental philosophy has in recent years shifted away from its original hostility to traditional methods. Starting with a brief historical review of the experimentalist challenge to traditional philosophical practice, this chapter looks at research undercutting that challenge, and at ways in which experimental work has evolved to complement and strengthen traditional approaches to philosophical questions.

Do philosophers use intuitions? Should philosophers use intuitions? Can philosophical methods (where intuitions are concerned) be improved upon? In order to answer these questions we need to have some idea of how we should go about answering them. I defend a way of going about methodology of intuitions: a metamethodology. I claim the following: (i) we should approach methodological questions about intuitions with a thin conception of intuitions in mind; (ii) we should carve intuitions finely; and, (iii) we should carve (...) to a grain to which we are sensitive in our everyday philosophising. The reason is that, unless we do so, we don’t get what we want from philosophical methodology. I argue that what we want is information that will aid us in formulating practical advice concerning how to do philosophy responsibly/well/better. (shrink)

Recent empirical work appears to suggest that the moral intuitions of professional philosophers are just as vulnerable to distorting psychological factors as are those of ordinary people. This paper assesses these recent tests of the ‘expertise defense’ of philosophical intuition. I argue that the use of familiar cases and principles constitutes a methodological problem. Since these items are familiar to philosophers, but not ordinary people, the two subject groups do not confront identical cognitive tasks. Reflection on this point shows that (...) these findings do not threaten philosophical expertise—though we can draw lessons for more effective empirical tests. (shrink)

The rise of experimental philosophy has placed metaphilosophical questions, particularly those concerning concepts, at the center of philosophical attention. X-phi offers empirically rigorous methods for identifying conceptual content, but what exactly it contributes towards evaluating conceptual content remains unclear. We show how x-phi complements Rudolf Carnap’s underappreciated methodology for concept determination, explication. This clarifies and extends x-phi’s positive philosophical import, and also exhibits explication’s broad appeal. But there is a potential problem: Carnap’s account of explication was limited to empirical and (...) logical concepts, but many concepts of interest to philosophers are essentially normative. With formal epistemology as a case study, we show how x-phi assisted explication can apply to normative domains. (shrink)

A number of philosophers have recently suggested that the role of intuitions in the epistemology of armchair philosophy has been exaggerated. This suggestion is rehearsed and endorsed. What bearing does the rejection of the centrality of intuition in armchair philosophy have on experimentalist critiques of the latter? I distinguish two very different kinds of experimentalist critique: one critique requires the centrality of intuition; the other does not.

One of the more visible recent developments in philosophical methodology is the experimental philosophy movement. On its surface, the experimentalist challenge looks like a dramatic threat to the apriority of philosophy; ‘experimentalist’ is nearly antonymic with ‘aprioristic’. This appearance, I suggest, is misleading; the experimentalist critique is entirely unrelated to questions about the apriority of philosophical investigation. There are many reasons to resist the skeptical conclusions of negative experimental philosophers; but even if they are granted—even if the experimentalists are right (...) to claim that we must do much more careful laboratory work in order legitimately to be confident in our philosophical judgments— the apriority of philosophy is unimpugned. The kinds of scientific investigation that experimental philosophers argue to be necessary involve merely enabling sensory experiences. Although they are not enabling in the sense of permitting concept acquisition, they are enabling in another epistemically significant way that is also consistent with the apriority of philosophy. (shrink)

The evidential value of moral intuitions has been challenged by psychological work showing that the intuitions of ordinary people are affected by distorting factors. One reply to this challenge, the expertise defence, claims that training in philosophical thinking confers enhanced reliability on the intuitions of professional philosophers. This defence is often expressed through analogy: since we do not allow doubts about folk judgments in domains like mathematics or physics to undermine the plausibility of judgments by experts in these domains, we (...) also should not do so in philosophy. In this paper I clarify the logic of the analogy strategy, and defend it against recent challenges by Jesper Ryberg. The discussion exposes an interesting divide: while Ryberg’s challenges may weaken analogies between morality and domains like mathematics, they do not affect analogies to other domains, such as physics. I conclude that the expertise defence can be supported by analogical means, though care is required in selecting an appropriate analog. I discuss implications of this conclusion for the expertise defence debate and for study of the moral domain itself. (shrink)

This chapter considers Ernest Sosa’s contributions to philosophical methodology. In Section 1, Sosa’s approach to the role of intuitions in the epistemology of philosophy is considered and related to his broader virtue-theoretic epistemological framework. Of particular focus is the question whether false or unjustified intuitions may justify. Section 2 considers Sosa’s response to sceptical challenges about intuitions, especially those deriving from experimental philosophy. I argue that Sosa’s attempt to attribute apparent disagreement in survey data to difference in meaning fails, but (...) that some of his other, more general, responses to experimentalist sceptics succeed. (shrink)

Experimental philosophy is often presented as a new movement that avoids many of the difficulties that face traditional philosophy. This article distinguishes two views of experimental philosophy: a narrow view in which philosophers conduct empirical investigations of intuitions, and a broad view which says that experimental philosophy is just the colocation in the same body of (i) philosophical naturalism and (ii) the actual practice of cognitive science. These two positions are rarely clearly distinguished in the literature about experimental philosophy, both (...) pro and con. The article argues, first, that the broader view is the only plausible one; discussions of experimental philosophy should recognize that the narrow view is a caricature of experimental philosophy as it is currently done. It then shows both how objections to experimental philosophy are transformed and how positive recommendations can be provided by adopting a broad conception of experimental philosophy. (shrink)

One critique of experimental philosophy is that the intuitions of the philosophically untutored should be accorded little to no weight; instead, only the intuitions of professional philosophers should matter. In response to this critique, “experimentalists” often claim that the intuitions of professional philosophers are biased. In this paper, we explore this question of whose intuitions should be disqualified and why. Much of the literature on this issue focuses on the question of whether the intuitions of professional philosophers are reliable. In (...) contrast, we instead focus on the idea of “muddled” intuitions—i.e. intuitions that are misdirected and about notions other than the ones under discussion. We argue that the philosophically untutored are likely to have muddled intuitions and that professional philosophers are likely to have unmuddled intuitions. Although being umuddled does not, by itself, establish the reliability of the intuitions of professional philosophers, being muddled is enough to disqualify the intuitions of the philosophically untutored. We then turn to the charge that, despite being unmuddled, professional philosophers still have biased intuitions. To evaluate this charge, we switch focus from the general notion of biased intuition to the more specific notion of theory-laden intuition. We argue that there is prima facie evidence—in the form of the presence of conflicts of intuition—for thinking that at least some of the intuitions of professional philosophers are theory-laden. In summary, we conclude that that there is no clean and easy answer to the question of whose intuitions should matter. (shrink)

In “Against Arguments from Reference” (Mallon et al., 2009), Ron Mallon, Edouard Machery, Shaun Nichols, and Stephen Stich (hereafter, MMNS) argue that recent experiments concerning reference undermine various philosophical arguments that presuppose the correctness of the causal-historical theory of reference. We will argue three things in reply. First, the experiments in question—concerning Kripke’s Gödel/Schmidt example—don’t really speak to the dispute between descriptivism and the causal-historical theory; though the two theories are empirically testable, we need to look at quite different data (...) than MMNS do to decide between them. Second, the Gödel/Schmidt example plays a different, and much smaller, role in Kripke’s argument for the causal-historical theory than MMNS assume. Finally, and relatedly, even if Kripke is wrong about the Gödel/Schmidt example—indeed, even if the causal-historical theory is not the correct theory of names for some human languages—that does not, contrary to MMNS’s claim, undermine uses of the causalhistorical theory in philosophical research projects. (shrink)

Many epistemologists use intuitive responses to particular cases as evidence for their theories. Recently, experimental philosophers have challenged the evidential value of intuitions, suggesting that our responses to particular cases are unstable, inconsistent with the responses of the untrained, and swayed by factors such as ethnicity and gender. This paper presents evidence that neither gender nor ethnicity influence epistemic intuitions, and that the standard responses to Gettier cases and the like are widely shared. It argues that epistemic intuitions are produced (...) by the natural ‘mindreading’ capacity that underpins ordinary attributions of belief and knowledge in everyday social interaction. Although this capacity is fallible, its weaknesses are similar to the weaknesses of natural capacities such as sensory perception. Experimentalists who do not wish to be skeptical about ordinary empirical methods have no good reason to be skeptical about epistemic intuitions. (shrink)

Some recent researches in experimental philosophy have posed a problem for philosophers’ appeal to intuition (hereinafter referred to as PAI); the aim of this paper is to offer an answer to this challenge. The thesis against PAI implies that, given some experimental results, intuition does not seem to be a reliable epistemic source, and —more importantly— given the actual state of knowledge about its operation, we do not have sufficient resources to mitigate its errors and thus establish its reliability. That (...) is why PAI is hopeless. Throughout this paper I will defend my own conception of PAI, which I have called the Deliberative Conception, and consequently, I will defend intersubjective agreement as a means to mitigate PAI errors, offering empirical evidence from recent studies on the Argumentative Theory of Reason that favor the conception I defend here. Finally, I will reply to some objections that might arise against the Deliberative Conception, which will lead me to discuss some metaphilosophical issues that are significantly relevant for the future of the dispute about the appeal to intuition. (shrink)

Despite well-established results in survey methodology, many experimental philosophers have not asked whether and in what way conclusions about folk intuitions follow from people’s responses to their surveys. Rather, they appear to have proceeded on the assumption that intuitions can be simply read off from survey responses. Survey research, however, is fraught with difficulties. I review some of the relevant literature—particularly focusing on the conversational pragmatic aspects of survey research—and consider its application to common experimental philosophy surveys. I argue for (...) two claims. First, that experimental philosophers’ survey methodology leaves the facts about folk intuitions massively underdetermined; and second, that what has been regarded as evidence for the instability of philosophical intuitions is, at least in some cases, better accounted for in terms of subjects’ reactions to subtle pragmatic cues contained in the surveys. (shrink)

Today, experimental philosophers challenge traditional appeals to intuition; they empirically collect folk intuitions and then use their findings to attack philosophers' intuitions. However this movement is not uniform. Radical experimentalists criticize the use of intuitions in philosophy altogether and they have been mostly attacked. Contrariwise, moderate experimentalists imply that laypersons' intuitions are somehow relevant to philosophical problems. Sometimes they even use folk intuitions in order to advance theoretical theses. In this paper I will try to challenge the so-called moderate experimental (...) attempts to rely on intuition in order to promote philosophical theses. Hoy en día, los filósofos experimentales ponen en duda la tradición de apelar a la intuición; coleccionan intuiciones empíricas populares y después utilizan sus descubrimientos para atacar las intuiciones de los filósofos. Sin embargo, esta estrategia no es uniforme. Los experimentalistas radicales critican cualquier uso de intuiciones en filosofía y han sido objeto de los mayores ataques. A diferencia de ellos, los experimentalistas moderados sugieren que las intuiciones del vulgo son de alguna manera pertinentes en los problemas filosóficos. Algunas veces incluso usan intuiciones populares para defender tesis teóricas. En este trabajo pretendo poner en duda los intentos de los llamados experimentalistas moderados de apelar a intuiciones para promover tesis filosóficas. (shrink)

I address a criticism of the use of thought experiments in conceptual analysis advanced on the basis of the survey method of so-called experimental philosophy. The criticism holds that surveys show that intuitions are relative to cultures in a way that undermines the claim that intuition-based investigation yields any objective answer to philosophical questions. The crucial question is what intuitions are as philosophers have been interested in them. To answer this question we look at the role of intuitions in philosophical (...) inquiry. When we have done this, we can see that it is impossible for intuitions properly understood to be relative in the way that has been suggested as they are conceived of as expressions of competence in the concepts deployed in their contents. The remaining methodological issues, though not to be dismissed, present no in principle objection to the method of thought experiments. (shrink)

In disputes about conceptual analysis, each side typically appeals to pre-theoretical 'intuitions' about particular cases. Recently, many naturalistically oriented philosophers have suggested that these appeals should be understood as empirical hypotheses about what people would say when presented with descriptions of situations, and have consequently conducted surveys on non-specialists. I argue that this philosophical research programme, a key branch of what is known as 'experimental philosophy', rests on mistaken assumptions about the relation between people's concepts and their linguistic behaviour. The (...) conceptual claims that philosophers make imply predictions about the folk's responses only under certain demanding, counterfactual conditions. Because of the nature of these conditions, the claims cannot be tested with methods of positivist social science. We are, however, entitled to appeal to intuitions about folk concepts in virtue of possessing implicit normative knowledge acquired through reflective participation in everyday linguistic practices. (shrink)

Recent third person approaches to thought experiments and conceptual analysis through the method of surveys are motivated by and motivate skepticism about the traditional first person method. I argue that such surveys give no good ground for skepticism, that they have some utility, but that they do not represent a fundamentally new way of doing philosophy, that they are liable to considerable methodological difficulties, and that they cannot be substituted for the first person method, since the a priori knowledge which (...) is our object in conceptual analysis can be acquired only from the first person standpoint. (shrink)