Media attention to retracted research suggests that a substantial number of papers are corrupted by misinformation. In reality, every paper contains misinformation; at issue is whether the balance of correct versus incorrect information is acceptable. This paper postulates that analysis of retracted research papers can provide insight into medical misinformation, although retracted papers are not a random sample of incorrect papers. Error is the most common reason for retraction and error may be the principal cause of misinformation as well. Still, (...) one-quarter of retracted papers are fraudulent, and misinformation may also arise through fraud. This paper hypothesises that error and fraud are the main sources of misinformation and that error is more common than fraud. Retraction removes misinformation from the literature; bias is non-retracted misinformation. Bias arises when scientific impropriety results in false research findings. Impropriety can involve experimental design, data collection, data analysis, or data presentation. Yet impropriety also arises through earnest error or statistical naiveté; not all bias is fraud. Several measures are proposed to minimise misinformation in the medical literature, including: greater detail in the clinical trial registry, with rigorous definition of inclusion and exclusion criteria and primary endpoints; clear statistical criteria for every aspect of clinical trials, especially sample size; responsibility for data integrity that accrues to all named authors; increased transparency as to how the costs of research were paid; and greater clarity as to the reasons for retraction. Misinformation can arise without malicious intent; authors of incorrect papers are owed a presumption of incompetence, not malice. (shrink)

Evidence hierarchies are widely used to assess evidence in systematic reviews of medical studies. I give several arguments against the use of evidence hierarchies. The problems with evidence hierarchies are numerous, and include methodological shortcomings, philosophical problems, and formal constraints. I argue that medical science should not employ evidence hierarchies, including even the latest and most-sophisticated of such hierarchies.

Subjective Bayesianism is a major school of uncertain reasoning and statistical inference. It is often criticized for a lack of objectivity: it opens the door to the influence of values and biases, evidence judgments can vary substantially between scientists, it is not suited for informing policy decisions. My paper rebuts these concerns by connecting the debates on scientific objectivity and statistical method. First, I show that the above concerns arise equally for standard frequentist inference with null hypothesis significance tests. Second, (...) the criticisms are based on specific senses of objectivity with unclear epistemic value. Third, I show that Subjective Bayesianism promotes other, epistemically relevant senses of scientific objectivity—most notably by increasing the transparency of scientific reasoning. (shrink)

For model-based frequentist statistics, based on a parametric statistical model ${{\cal M}_\theta }$, the trustworthiness of the ensuing evidence depends crucially on the validity of the probabilistic assumptions comprising ${{\cal M}_\theta }$, the optimality of the inference procedures employed, and the adequateness of the sample size to learn from data by securing –. It is argued that the criticism of the postdata severity evaluation of testing results based on a small n by Rochefort-Maranda is meritless because it conflates [a] misuses (...) of testing with [b] genuine foundational problems. Interrogating this criticism reveals several misconceptions about trustworthy evidence and estimation-based effect sizes, which are uncritically embraced by the replication crisis literature. (shrink)

Plagiarism by college and university faculty members has become a growing issue and concern in academia. This paper presents a case study of an extreme and clear case of such plagiarism. Yet an analysis of the legal and ethical contexts of such plagiarism, and the specific chronicle of this case, illustrate the complexities and difficulties in dealing with such situations. Implications for researchers, for colleges and universities, and for academic publishers and journals are offered.

The cross-cultural differences in epistemic intuitions reported by Weinberg, Nichols and Stich (2001; hereafter: WNS) laid the ground for the negative program of experimental philosophy. However, most of WNS’s findings were not corroborated in further studies. The exception here is the study concerning purported differences between Westerners and Indians in knowledge ascriptions concerning the Zebra Case, which was never properly replicated. Our study replicates the above-mentioned experiment on a considerably larger sample of Westerners (n = 211) and Indians (n = (...) 204). The analysis found a significant difference between the ethnic groups in question in the predicted direction: Indians were more likely to attribute knowledge in the Zebra Case than Westerners. In this paper, we offer an explanation of our result that takes into account the fact that replications of WNS’s other experiments did not find any cross-cultural differences. We argue that the Zebra Case is unique among the vignettes tested by WNS since it should not be regarded as a Gettier case but rather as a scenario exhibiting skeptical pressure concerning the reliability of sense-perception. We argue that skepticism towards perception as a means of gaining knowledge is a trope that is deeply rooted in Western epistemology but is very much absent from Classical Indian philosophical inquiry. This line of reasoning is based on a thorough examination of the skeptical scenarios discussed by philosophers of the Indian Nyaya tradition and their adversaries. (shrink)

Recent work has demonstrated that academic research faces serious challenges. Incentives to defend publishable ideas often lead researchers astray. Despite their tendency to produce error, efforts to publish erroneous results typically help a researcher's career. In addition, errors often arise from seemingly innocent methodological assumptions that allow researchers to believe their research is sound. This article discusses this research, as well as research into difficulties facing epistemic rationality caused by nonepistemic incentives. It then applies the lessons of this research to (...) philosophical practice. It explains why philosophy likely suffers from these problems. It then provides an example of a widely shared methodological assumption that allows such research to be pursued and regularly published. It claims that the significance philosophers place on arguments is inappropriate, and that typical evaluations of philosophical arguments involve an instance of the base-rate fallacy. It concludes by discussing whether or not this article is self-defeating. (shrink)

The results of forensic science are believed to be reliable, and are widely used in support of verdicts around the world. However, due to the lack of suitable empirical studies, we actually know very little about the reliability of such results. In this paper, I argue that phenomena analogous to the main culprits for the replication crisis in psychology are also present in forensic science. Therefore forensic results are significantly less reliable than is commonly believed. I conclude that in order (...) to obtain reliable estimates for the reliability of forensic results, we need to conduct studies analogous to the large-scale replication projects in psychology. Additionally, I point to some ways for improving the reliability of forensic science, inspired by the reforms proposed in response to the replicability crisis. (shrink)

The epistemic attitudes of scientists, such as epistemic tolerance and authoritarianism, play important roles in the discourse about rivaling theories. Epistemic tolerance stands for the mental attitude of an epistemic agent, e.g., a scientist, who is open to opposing views, while epistemic authoritarianism represents the tendency to uncritically accept views of authorities. Another relevant epistemic factor when it comes to the epistemic decisions of scientists is the skepticism towards the scientific method. However, the question is whether these epistemic attitudes are (...) influenced by their sociopolitical counterparts, such as the researcher’s degree of conservatism. To empirically investigate the interplay between epistemic and sociopolitical attitudes of scientists, we conducted a survey with researchers (N = 655) across different disciplines. We propose scales for measuring epistemic tolerance and epistemic authoritarianism, as well as a scale for detecting the participants' readiness to question the scientific method. Furthermore, we investigate the relationship between epistemic tolerance and epistemic authoritarianism on the one hand, and career stage and sociopolitical views on the other hand. Interestingly, our study found only small correlations between the participants' degree of conservatism and their epistemic attitudes. This suggests that political views, against common argumentation, actually do not play an important role in one’s scientific decisions. Moreover, social scientists scored higher on the epistemic tolerance and lower on the epistemic authoritarianism scale than natural scientists. Finally, the results indicate that natural scientists question the scientific method less than social scientists. (shrink)

Historical inductions, that is, the pessimistic metainduction and the problem of unconceived alternatives, are critically analyzed via John D. Norton’s material theory of induction and subsequently rejected as noncogent arguments. It is suggested that the material theory is amenable to a local version of the pessimistic metainduction, for example, in the context of some medical studies.

In their paper titled “Gender and philosophical intuition,” Buckwalter and Stich argue that the intuitions of women and men differ significantly on various types of philosophical questions. Furthermore, men's intuitions, so the authors claim, are more in line with traditionally accepted solutions of classical problems. This inherent bias, so the argument goes, is one of the factors that leads more men than women to pursue degrees and careers in philosophy. These findings have received a considerable amount of attention and the (...) paper is to appear in the second edition of Experimental Philosophy edited by Knobe and Nichols , which itself is an influential outlet. Given the exposure of these results, we attempted to replicate three of the classes of questions that Buckwalter and Stich review in their paper and for which they report significant differences. We failed to replicate the results using several different sources for data collection (one being identical to the original procedures.. (shrink)

In one of the earlier influential papers in the field of experimental philosophy titled Normativity and Epistemic Intuitions published in 2001, Jonathan M. Weinberg, Shaun Nichols and Stephen Stich reported that respondents answered Gettier type questions differently depending on their ethnic background as well as socioeconomic status. There is currently a debate going on, on the significance of the results of Weinberg et al. (2001) and its implications for philosophical methodology in general and epistemology in specific. Despite the debates, however, (...) to our knowledge, there has not been a replication attempt of the experiments of the original paper. We collected data from four different sources (two on-line and two in-person) to replicate the experiments. Despite several different data sets and in various cases larger sample sizes and hence greater power to detect differences, we failed to detect significant differences between the above-mentioned ethnic and socioeconomic groups. Our results suggest that epistemic intuitions are more robust across ethnic and socioeconomic groups than Weinberg et al. (2001) indicates. Given our data, we believe that the notion of differences in epistemic intuitions among different ethnic and socioeconomic groups that follows from Weinberg et al. (2001) needs to be corrected. (shrink)

Previous research indicates that Euro-American women are more upset by imagining their male partners committing homosexual infidelities than heterosexual ones. The present studies sought to replicate these findings and extend them to two non-Western cultures wherein masculine men frequently engage in sexual interactions with feminine third-gender males. Across six studies in three cultural locales, women were asked to rate their degree of upset when imagining that their partner committed infidelity that was heterosexual in nature, as well as infidelity that was (...) homosexual. In two Canadian undergraduate samples, women reported greater upset at imagining partner infidelity with a female, whereas a community sample of middle-aged women reported equal upset across infidelity types. Samoan women reported substantially less upset at the thought of partner infidelity with a third-gender male than with a female. Istmo Zapotec women reported equal upset toward infidelity with a female or a third-gender male, whereas a second Zapotec sample reported slightly greater upset at the thought of infidelity with a muxe. Results illustrate how cultural contexts moderate the degree to which same-sex infidelity scenarios are upsetting to women. (shrink)

Multiple simultaneous independent discoveries, so well enunciated by Robert K. Merton in the early 1960s but already discussed for several hundreds of years, is a classic concept in the sociology of science. In this paper, the concept of multiple simultaneous independent errors is proposed, analyzed, and discussed. The concept of Selective Pessimistic Induction is proposed and used to connect MIDs with MIEs. Five types of MIEs are discussed: multiple errors in the interpretation of experimental data or computational results; multiple misjudgments (...) of the value of another’s research results or conclusions; multiple cases of false anticipation of achieving a certain experimental result; multiples of ignoring or omitting relevant precedents; and multiple instances of failure due to a not-yet-conceived scientific concept or principle. Causal MIDs and MIEs are those that can be traced directly to antecedent knowledge. Acausal MIDs and MIEs are those involving a consequential and identifiable leap from antecedent knowledge. Examples of causal and acausal MIEs are provided, mostly but not exclusively from the discipline of chemistry. Comparisons are made between MIDs and MIEs. Topics for future research are discussed and implications of these concepts are proposed. (shrink)

Funerals are a reflective practice to bid farewell to the departed soul. Different religions, cultural traditions, rituals, and social beliefs guide how funeral practices take place. Family and friends gather together to support each other in times of grief. However, during the coronavirus pandemic, the way funerals are taking place is affected by the country's rules and region to avoid the spread of infection. The present study explores the media portrayal of public sentiments over funerals. In particular, the present study (...) tried to identify linguistic dimensions associated with lexical components of social processes, affective processes, fear, and disgust. An exhaustive search of newspaper coverage of funeral and related articles was made for a specific corona period. After an initial screening for the details and language used, a total of 46 newspaper articles on funerals were finalized for the analysis. Linguistic Inquiry and Word Count (LIWC) software was used to determine the association between linguistic dimensions of function words and words related to social and affective processes, as presented in the newspaper articles. Sentiment Analysis and Cognition Engine (SEANCE) was applied for the analysis of sentiment, social cognition, and social order. Bayesian correlation analysis and regression revealed positive and significant associations between function words and affective processes, between pronouns and social processes, and between negative adjectives and psychological processes of fear and disgust. Also, significant negative associations were found between polarity nouns and psychological processes of fear and disgust and between polarity verbs and psychological processes of fear and disgust. Bayes factor 10 provides strong evidence in favor of the study hypotheses. The media is influenced by the prevailing sentiments in society and reflects their perception of the current social order and beliefs. The findings provide a glimpse into the prevailing sentiment of society through the lens of media coverage. These understandings are expected to enhance our observations of how people express their feelings over the loss of their loved ones and help mental health professionals develop their therapeutic protocols to treat the coronavirus disease 2019 (COVID-19)-affected population. (shrink)

Data which constitute satisfactory evidence in other contexts are sometimes not treated as valid evidence in the context of scientic conrmation. I give a justicatory explanation of this fact, appealing to the incentives, biases, and social situatedness of scientists.

The study of human behavior is severely hampered by logistical problems, ethical and legal constraints, and funding shortfalls. However, the biggest difficulty of conducting social and behavioral research is the extraordinary complexity of the study phenomena. In this article, we review the impact of complexity on research design, hypothesis testing, measurement, data analyses, reproducibility, and the communication of findings in psychological science. The systematic investigation of the world often requires different approaches because of the variability in complexity. Confirmatory testing, multi-factorial (...) designs, survey methods, large samples, and modeling are frequently needed to study complex social and behavioral topics. Complexity impedes the measurement of general constructs, the reproducibility of results and scientific reporting, and the general rigor of research. Many of the benchmarks established by classic work in physical science are not attainable in studies of more complex phenomena. Consequently, the standards used to evaluate scientific research should be tethered to the complexity of the study topic. (shrink)

A science war is in full swing which has taken science's reproducibility crisis as a battleground. While conservatives and corporate interests use the crisis to weaken regulations, their opponent deny the existence of a science's crisis altogether. Thus, for the conservative National Association of Scholars NAS the crisis is real and due to the progressive assault on higher education with ideologies such as neo-Marxism, radical feminism, historicism, post-colonialism, deconstructionism, post-modernism, liberation theology. In the opposite field, some commentators claim that there (...) is no crisis in science and that saying the opposite is irresponsible. These positions are to be seen in the context of the ongoing battle against regulation, of which the new rules proposed at the US Environmental Protection Agency are but the last chapter. In this optic, Naomi Oreskes writes on Nature that what constitutes the crisis is the conservatives' attack on science. This evident right-left divide in the reading of the crisis is unhelpful and dangerous to the survival of science itself. An alternative reading ignored by the contendents would suggest that structural contradictions have emerged in modern science, and that addressing these should be the focus of our attention. (shrink)

Current Perspectives in Philosophy of Biology.

Populists are often cast as deniers of rationality, creators of a climate of “post-truth,” and valuing tribe over truth and the rigors of science. Their critics claim the authority of rationality and empirical facts. Yet the critics no less than populists enable an environment of spurious claims and defective argumentation. This is especially true in the realm of science. An important case study is the account of scientific trust offered by a leading public intellectual and historian of science, Naomi Oreskes, (...) and the misapplication of that theory during the coronavirus pandemic. (shrink)

Critical Data Studies explore the unique cultural, ethical, and critical challenges posed by Big Data. Rather than treat Big Data as only scientifically empirical and therefore largely neutral phenomena, CDS advocates the view that Big Data should be seen as always-already constituted within wider data assemblages. Assemblages is a concept that helps capture the multitude of ways that already-composed data structures inflect and interact with society, its organization and functioning, and the resulting impact on individuals’ daily lives. CDS questions the (...) many assumptions about Big Data that permeate contemporary literature on information and society by locating instances where Big Data may be naively taken to denote objective and transparent informational entities. In this introduction to the Big Data & Society CDS special theme, we briefly describe CDS work, its orientations, and principles. (shrink)

Kyle Stanford claims we have historical evidence that there likely are plausible unconceived alternatives in fundamental domains of science, and thus evidence that our best theories in these domains are probably false. Accordingly, we should adopt a form of instrumentalism. Elsewhere, I have argued that in fact we do not have historical evidence for the existence of plausible unconceived alternatives in particular domains of science, and that the main challenge to scientific realism is rather to provide evidence that there are (...) likely not plausible unconceived alternatives. In the present paper, I contend that we may come to have such evidence in the long run of science. I then investigate the epistemic consequences of the claim that we presently do not have evidence for or against the existence of plausible unconceived alternatives, but that in the future we may come to have evidence against the existence of plausible unconceived alternatives. I argue there are prima facie reasons to endorse a form of voluntarism in this situation according to which scientists and others may rationally be more optimistic or more pessimistic about the truth of our best theories, on the grounds that the widespread acceptance of an obligation to be an instrumentalist threatens to disrupt the proper functioning of science, in part because the domain of application of the problem of unconceived alternatives is unclear. (shrink)

The global pessimistic meta-induction argues from the falsity of scientific theories accepted in the past to the likely falsity of currently accepted scientific theories. I contend that this argument commits a statistical error previously unmentioned in the literature and is self-undermining. I then compare the global pessimistic meta-induction to a local pessimistic meta-induction based on recent negative assessments of the reliability of medical research. If there is any future in drawing pessimistic conclusions from the history of science, it lies in (...) local meta-inductions, but these meta-inductions will not result in global distrust of the results of science. (shrink)

Researchers increasingly rely on large data sets of health information, often linked with biological specimens. In recent years, the argument has been made that obtaining informed consent for conducting records-based research is unduly burdensome and results in ?consent bias.? As a type of selection bias, consent bias is said to exist when the group giving researchers access to their data differs from the group denying access. Therefore, to promote socially beneficial research, it is argued that consent should be unnecessary. After (...) analyzing the biostatistics evidence and bioethics arguments, the article concludes that (1) claims about the amount of consent bias are overstated; (2) commonly used statistical methods usually can account for consent bias; and (3) any residual effects of consent bias are below an acceptable level of imprecision and constitute a reasonable social cost for conducting ethically responsible research. (shrink)

The enduring replication crisis in many scientific disciplines casts doubt on the ability of science to estimate effect sizes accurately, and in a wider sense, to self-correct its findings and to produce reliable knowledge. We investigate the merits of a particular countermeasure—replacing null hypothesis significance testing with Bayesian inference—in the context of the meta-analytic aggregation of effect sizes. In particular, we elaborate on the advantages of this Bayesian reform proposal under conditions of publication bias and other methodological imperfections that are (...) typical of experimental research in the behavioral sciences. Moving to Bayesian statistics would not solve the replication crisis single-handedly. However, the move would eliminate important sources of effect size overestimation for the conditions we study. (shrink)

The enduring replication crisis in many scientific disciplines casts doubt on the ability of science to estimate effect sizes accurately, and in a wider sense, to self-correct its findings and to produce reliable knowledge. We investigate the merits of a particular countermeasure—replacing null hypothesis significance testing with Bayesian inference—in the context of the meta-analytic aggregation of effect sizes. In particular, we elaborate on the advantages of this Bayesian reform proposal under conditions of publication bias and other methodological imperfections that are (...) typical of experimental research in the behavioral sciences. Moving to Bayesian statistics would not solve the replication crisis single-handedly. However, the move would eliminate important sources of effect size overestimation for the conditions we study. (shrink)

Replicability is widely taken to ground the epistemic authority of science. However, in recent years, important published findings in the social, behavioral, and biomedical sciences have failed to replicate, suggesting that these fields are facing a “replicability crisis.” For philosophers, the crisis should not be taken as bad news but as an opportunity to do work on several fronts, including conceptual analysis, history and philosophy of science, research ethics, and social epistemology. This article introduces philosophers to these discussions. First, I (...) discuss precedents and evidence for the crisis. Second, I discuss methodological, statistical, and social-structural factors that have contributed to the crisis. Third, I focus on the philosophical issues raised by the crisis. Finally, I discuss proposed solutions and highlight the gaps that philosophers could focus on. (shrink)

Advocates of the self-corrective thesis argue that scientific method will refute false theories and find closer approximations to the truth in the long run. I discuss a contemporary interpretation of this thesis in terms of frequentist statistics in the context of the behavioral sciences. First, I identify experimental replications and systematic aggregation of evidence (meta-analysis) as the self-corrective mechanism. Then, I present a computer simulation study of scientific communities that implement this mechanism to argue that frequentist statistics may converge upon (...) a correct estimate or not depending on the social structure of the community that uses it. Based on this study, I argue that methodological explanations of the “replicability crisis” in psychology are limited and propose an alternative explanation in terms of biases. Finally, I conclude suggesting that scientific self-correction should be understood as an interaction effect between inference methods and social structures. (shrink)

Computing is no longer the sole domain of professionals, educated and trained through traditional routes to service public and private sector organisations under paid contracts. Computing has been democratised with the advent of economically accessible hardware, a multitude of software tools and the internet. Computing is by everyone for everyone.

For the Greeks, O King, who make logical demonstrations, use words emptied of power, and this very activity is what constitutes their philosophy, a mere noise of words. But we [Egyptians] do not use words [logoi] but sounds [phōnai] which are full of effects.If I could tell you what it meant, there would be no point in dancing it.The Eleatic thinker Zeno was a friend, perhaps adopted son, and student of Parmenides. He is famous for his many paradoxes on space (...) and motion, such as Achilles and the tortoise. He was more than just an intellectual, however, having many pursuits including politics, which led to his gruesome death. Running arms to the Liparans, who were resisting the growing power of the Athenians, he .. (shrink)

This paper looks at an appeal to the authority of biomedical research that has recently been used by empirical economists to motivate and justify their methods. I argue that those who make this appeal mistake the nature of biomedical research. Randomised trials, which are said to have revolutionised biomedical research, are a central methodology, but according to only one paradigm. There is another paradigm at work in biomedical research, the inferentialist paradigm, in which randomised trials play no special role. I (...) outline the inferentialist alternative in broad strokes, apply it to a recent controversy in econometrics and draw some general conclusions concerning econometric methodology. (shrink)

Two approaches to evidential reasoning compete in the biomedical and social sciences: the experimental and the pragmatist. Whereas experimentalism has received considerable philosophical analysis and support since the times of Bacon and Mill, pragmatism about evidence has been neither articulated nor defended. The overall aim is to fill this gap and develop a theory that articulates the latter. The main ideas of the theory will be illustrated and supported by a case study on the smoking/lung cancer controversy in the 1950s.

Francesco Guala once wrote that ‘The problem of extrapolation is a minor scandal in the philosophy of science’. This paper agrees with the statement, but for reasons different from Guala’s. The scandal is not, or not any longer, that the problem has been ignored in the philosophy of science. The scandal is that framing the problem as one of external validity encourages poor evidential reasoning. The aim of this paper is to propose an alternative—an alternative which constitutes much better evidential (...) reasoning about target systems of interest, and which makes do without consideration of external validity. (shrink)

20 years ago, the Report of the Commission on Research Integrity was submitted to the Secretary of the Department of Health and Human Services and to House and Senate Committees. As directed in enabling legislation, the Commission had provided recommendations on a new definition of research misconduct, oversight of scientific practices, and development of a regulation to protect whistleblowers. Reflecting the ethos of the time, the Commission recommended that institutions receiving Public Health Service research funding should provide oversight of all (...) but the most egregious misconduct. The suggested definition of research misconduct was organized around misappropriation, interference and misrepresentation, which would have addressed collaborative/authorship disputes and sabotage in scientific laboratories, both of which remain unaddressed in current policy. The Commission also recommended the Whistleblower Bill of Rights and Responsibilities which would have authorized remedies for whistleblowers who experienced retaliation and sanctions against retaliators. Response from the scientific community was highly critical, and none of the Commission’s recommendations was accepted. No new body has examined issues within the Commission’s charge, there has been no significant Congressional or public pressure to do so, institutions have not been able to sustain standards that would have avoided current concerns about bias and irreproducibility in research, and there is still no entity in science capable of addressing issues assigned to the Commission and other urgent issues. (shrink)

P-hacking or data dredging involves manipulation of the research data in order to obtain a statistically significant result. The reasons behind P-hacking and the consequences of the same are discussed in the present manuscript.

The debates between Bayesian, frequentist, and other methodologies of statistics have tended to focus on conceptual justifications, sociological arguments, or mathematical proofs of their long run properties. Both Bayesian statistics and frequentist (“classical”) statistics have strong cases on these grounds. In this article, we instead approach the debates in the “Statistics Wars” from a largely unexplored angle: simulations of different methodologies’ performance in the short to medium run. We conducted a large number of simulations using a straightforward decision problem based (...) around tossing a coin with unknown bias and then placing bets. In this simulation, we programmed four players, inspired by Bayesian statistics, frequentist statistics, Jon Williamson’s version of Objective Bayesianism, and a player who simply extrapolates from observed frequencies to general frequencies. The last player functions as a benchmark: a statistical methodology should at least outperform a crude form of induction. We focused on the performance of these methodologies in guiding the players towards good decisions. Unlike an earlier simulation study of this type, we found no systematic difference in performance between the Bayesian and frequentist players, provided the Bayesian used a flat prior and the frequentist used a low confidence level. Unlike that study, we were able to use Big Data methods to mitigate problems of random error in the simulation results. The Williamsonian player, who is a novel element of our study, also had no systematic differences in their performance, provided that they use a low confidence level. These players performed similarly even in the very short run, when players were making different decisions. Our study indicates that all three methodologies should be taken seriously by philosophers and practitioners of statistics. However, the frequentist and Williamsonian performed poorly when their confidence levels were high, and the Bayesian was surprisingly harmed by biased priors, providing some unexpected lessons for these methodologies when facing this type of decision problem. (shrink)

Many people no longer trust mainstream science. It seems reasonable to assume that this unfortunate state of affairs - which...

Recent replication crises in psychology and other fields have led to intense reflection about the validity of common research practices. Much of this reflection has focussed on reporting standards, and how they may be related to the questionable research practices that could underlie a high proportion of irreproducible findings in the published record. As a developing field, it is particularly important for Experimental Philosophy to avoid some of the pitfalls that have beset other disciplines. To this end, here we provide (...) a detailed, comprehensive assessment of current reporting practices in Experimental Philosophy. We focus on the quality of statistical reporting and the disclosure of information about study methodology. We assess all the articles using quantitative methods that were published over the years 2013–2016 in 29 leading philosophy journals. We find that null hypothesis significance testing is the prevalent statistical practice in Experimental Philosophy, although relying solely on this approach has been criticised in the psychological literature. To augment this approach, various additional measures have become commonplace in other fields, but we find that Experimental Philosophy has adopted these only partially: 53% of the papers report an effect size, 28% confidence intervals, 1% examined prospective statistical power and 5% report observed statistical power. Importantly, we find no direct relation between an article’s reporting quality and its impact. We conclude with recommendations for authors, reviewers and editors in Experimental Philosophy, to facilitate making research statistically-transparent and reproducible. (shrink)

Arguments discrediting the value-free ideal of science have left us with the question of how to distinguish desirable values from biases that compromise the reliability of research. In this paper, I argue for a characterization of cognitive biases as deviations of thought processes that systematically lead scientists to the wrong conclusions. In particular, cognitive biases could help us understand a crucial issue in science today: how systematic error is introduced in research outcomes, even when research is evaluated as of good (...) quality. To conclude, I suggest that some debiasing mechanisms have great potential for countering implicit methodological biases in science. (shrink)