Recent replication crises in psychology and other fields have led to intense reflection about the validity of common research practices. Much of this reflection has focussed on reporting standards, and how they may be related to the questionable research practices that could underlie a high proportion of irreproducible findings in the published record. As a developing field, it is particularly important for Experimental Philosophy to avoid some of the pitfalls that have beset other disciplines. To this end, here we provide (...) a detailed, comprehensive assessment of current reporting practices in Experimental Philosophy. We focus on the quality of statistical reporting and the disclosure of information about study methodology. We assess all the articles using quantitative methods that were published over the years 2013–2016 in 29 leading philosophy journals. We find that null hypothesis significance testing is the prevalent statistical practice in Experimental Philosophy, although relying solely on this approach has been criticised in the psychological literature. To augment this approach, various additional measures have become commonplace in other fields, but we find that Experimental Philosophy has adopted these only partially: 53% of the papers report an effect size, 28% confidence intervals, 1% examined prospective statistical power and 5% report observed statistical power. Importantly, we find no direct relation between an article’s reporting quality and its impact. We conclude with recommendations for authors, reviewers and editors in Experimental Philosophy, to facilitate making research statistically-transparent and reproducible. (shrink)

This paper examines a constellation of ethical and editorial issues that have arisen since philosophers started to conduct, submit and publish empirical research. These issues encompass concerns over responsible authorship, fair treatment of human subjects, ethicality of experimental procedures, availability of data, unselective reporting and publishability of research findings. This study aims to assess whether the philosophical community has as yet successfully addressed such issues. To do so, the instructions for authors, submission process and published research papers of 29 main (...) journals in philosophy have been considered and analyzed. In light of the evidence reported here, it is argued that the philosophical community has as yet failed to properly tackle such issues. The paper also delivers some recommendations for authors, reviewers and editors in the field. (shrink)

Naturalistic philosophers rely on literature search and review in a number of ways and for different purposes. Yet this article shows how processes of literature search and review are likely to be affected by widespread and systematic biases. A solution to this problem is offered here. Whilst the tradition of systematic reviews of literature from scientific disciplines has been neglected in philosophy, systematic reviews are important tools that minimize bias in literature search and review and allow for greater reproducibility and (...) transparency. If naturalistic philosophers wish to reduce bias in their research, they should then supplement their traditional tools for literature search and review by including systematic methodologies. (shrink)

This paper examines a constellation of ethical and editorial issues that have arisen since philosophers started to conduct, submit and publish empirical research. These issues encompass concerns over responsible authorship, fair treatment of human subjects, ethicality of experimental procedures, availability of data, unselective reporting and publishability of research findings. This study aims to assess whether the philosophical community has as yet successfully addressed such issues. To do so, the instructions for authors, submission process and published research papers of 29 main (...) journals in philosophy have been considered and analyzed. In light of the evidence reported here, it is argued that the philosophical community has as yet failed to properly tackle such issues. The paper also delivers some recommendations for authors, reviewers and editors in the field. (shrink)