This paper focuses on the distinction between methods which are mathematically "clever", and those which are simply crude, typically repetitive and computer intensive, approaches for "crunching" out answers to problems. Examples of the latter include simulated probability distributions and resampling methods in statistics, and iterative methods for solving equations or optimisation problems. Most of these methods require software support, but this is easily provided by a PC. The paper argues that the crunchier methods often have substantial advantages from the perspectives (...) of user-friendliness, reliability (in the sense that misuse is less likely), educational efficiency and realism. This means that they offer very considerable potential for simplifying the mathematical syllabus underlying many areas of applied mathematics such as management science and statistics: crunchier methods can provide the same, or greater, technical power, flexibility and insight, while requiring only a fraction of the mathematical conceptual background needed by their cleverer brethren. (shrink)

This paper summarizes recent and ongoing experimental work regarding the reality, nature, effects, and causes of the underrepresentation of women in academic philosophy. We first present empirical data on several aspects of underrepresentation, and then consider various reasons why this gender imbalance is problematic. We then turn to the published and preliminary results of empirical work aimed at identifying factors that might explain it.

Saul_Jennifer, Scepticism and Implicit Bias.

Policy disputes arise at all scales of governance: in clubs, non-profits, firms, nations, and alliances of nations. Both the means and ends of policy are disputed. While many, perhaps most, policy disputes arise from conflicting ends, important disputes also arise from differing beliefs on how to achieve shared ends. In fact, according to many experts in economics and development, governments often choose policies that are “inefficient” in the sense that most everyone could expect to gain from other feasible policies. Many (...) other kinds of experts also see existing policies as often clearly inferior to known alternatives. (shrink)

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications (...) of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to countenance the complexity of social interactions among actors involved directly and indirectly in peer review. (shrink)

Peer review is often taken to be the main form of quality control on academic research. Usually journals carry this out. However, parts of maths and physics appear to have a parallel, crowd-sourced model of peer review, where papers are posted on the arXiv to be publicly discussed. In this paper we argue that crowd-sourced peer review is likely to do better than journal-solicited peer review at sorting papers by quality. Our argument rests on two key claims. First, crowd-sourced peer (...) review will lead on average to more reviewers per paper than journal-solicited peer review. Second, due to the wisdom of the crowds, more reviewers will tend to make better judgments than fewer. We make the second claim precise by looking at the Condorcet Jury Theorem as well as two related jury theorems developed specifically to apply to peer review. (shrink)

Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. (...) We conclude that on present evidence abolishing peer review weakly dominates the status quo. (shrink)

Business ethics journals have appeared on a few ranked lists that are specific to this niche discipline. As with more traditional academic disciplines, these rankings are used for academic rewards such as faculty tenure and promotion, along with department and school ratings. Journal ranking has been subject to considerable criticism even as its administrative use persists. Among the criticisms are that journal quality is a poor proxy for article quality, citation rate is an imperfect reflection of article influence, and bias (...) may be introduced into rankings by visibility characteristics such as journal age, size, circulation, and experience of the rater with a journal. This research note studies the effect of journal age and size on the rankings of business ethics journals compiled by Beets, Lewis, and Brower, by Albrecht, Thompson, Hoopes, and Rodrigo, and by Serenko and Bontis. Significant correlation was found for journal age with the administratively derived Beets et al. ranking. No significant correlation was found for size in any ranking study. Results were not significant for the Albrecht et al. and the Serenko and Bontis rankings representing the perspectives from surveys of active researchers or citation analysis. Perhaps sometimes a journal’s reputation precedes it, as perception of journal quality may be biased by journal visibility, either because it has been published and available for a number of years, or because it is well known and likely to be cited. (shrink)

It has been argued that implicit biases are operative in philosophy and lead to significant epistemic costs in the field. Philosophers working on this issue have focussed mainly on implicit gender and race biases. They have overlooked ideological bias, which targets political orientations. Psychologists have found ideological bias in their field and have argued that it has negative epistemic effects on scientific research. I relate this debate to the field of philosophy and argue that if, as some studies suggest, the (...) same bias also exists in philosophy then it will lead to hitherto unrecognised epistemic hazards in the field. Furthermore, the bias is epistemically different from the more familiar biases in respects that are important for epistemology, ethics, and metaphilosophy. (shrink)

In this essay we introduce the idea of an epistemic triangle, with factual, theoretical, and conceptual investigations at its vertices, and argue that whereas scientific progress requires a balance among the three types of investigations, psychology's epistemic triangle is stretched disproportionately in the direction of factual investigations. Expressed by a variety of different problems, this unbalance may be created by a main operative theme—the obsession of psychology with a narrow and mechanical view of the scientific method and a misguided aversion (...) to conceptual inquiries. Hence, to redress psychology's epistemic triangle, a broader and more realistic conception of method is needed and, in particular, conceptual investigations must be promoted. Using examples from different research domains, we describe the nature of conceptual investigations, relate them to theoretical investigations, and illustrate their purposes, forms, and limitations. (shrink)

This paper reviews the published empirical evidence concerning journal peer review consisting of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, (...) early-acceptance procedures, author nominations of reviewers, structured rating sheets, open peer review, results-blind review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. (shrink)

This article reports the results of a study of confirmational response bias among social work journals. A contrived research paper with positive findings and its negative mirror image were submitted to two different groups of social work journals and to two comparison groups of journals outside social work. The quantitative results, suggesting bias, are tentative; but the qualitative findings based upon an analysis of the referee comments are clear and consistent. Few referees from prestigious or nonprestcgrous social work journals prepared (...) reviews that were knowledgeable, scientifically astute, or objective. The best reviews came from journals outside of soccal work or from journals that are accepted as social work journals but originate with other disciplines. (shrink)

The fallacies, as traditionally conceived, are wrong ways of reasoning that nevertheless appear attractive to us. Recently, however, Woods (2013) has argued that they don’t merit such a title, and that what we take to be fallacies are instead largely virtuous forms of reasoning. This reformation of the fallacies forms part of Woods’ larger project to naturalize logic. In this paper I will look to his analysis of the argumentum ad hominem as a case study for the prospects of this (...) project. I will argue that the empirical literature on implicit bias presents a difficulty for the reformation of the ad hominem as cognitively virtuous. Cases where implicit bias influences our assessment of the truth or claim or argument are instances of ad hominem reasoning, and these qualify as fallacious on Woods’ own definition. (shrink)

An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino’s theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community’s knowledge claims, but on the beliefs and (...) norms of the culture of the knowledge community itself. To illustrate how socializing cognition can bias evaluations, we focus on peer-review practices, with some discussion of peer-review practices in philosophy. Data include responses to surveys by editors from general philosophy journals, as well as analyses of reviews and editorial decisions for the 2007 Cognitive Science Society Conference. (shrink)

This study seeks to examine the impact of alumni connections between the evaluators and evaluatees on the results of peer review ratings for the Korean national R&D project and selection success rate. Specifically, this study analyzed the evaluation results of 8,402 research proposal entries submitted between 2007 and 2011 for the “general researchers support project,” all in the Natural Science and Engineering areas and sponsored by the National Research Foundation of Korea. Each proposal entry was evaluated by three evaluators, and (...) approximately 39 percent of the proposals had at least one evaluator from the same university. The results of this study showed that evaluators have a tendency to give relatively high scores to research proposals submitted by the alumni of the same universities as their alma mater. Also, when an evaluator from the same university as an evaluatee was included in the evaluator group, the results of this study showed that the percentage of entry submissions was higher compared to the contrary. Such results show that in the process of peer review–based research proposal evaluations for national R&D projects, alumni connections have significant influence on evaluation results in South Korea. (shrink)

“Implicit bias” is a term of art referring to relatively unconscious and relatively automatic features of prejudiced judgment and social behavior. While psychologists in the field of “implicit social cognition” study “implicit attitudes” toward consumer products, self-esteem, food, alcohol, political values, and more, the most striking and well-known research has focused on implicit attitudes toward members of socially stigmatized groups, such as African-Americans, women, and the LGBTQ community.[1] For example, imagine Frank, who explicitly believes that women and men are equally (...) suited for careers outside the home. Despite his explicitly egalitarian belief, Frank might nevertheless implicitly associate women with the home, and this implicit association might lead him to behave in any number of biased ways, from trusting feedback from female co-workers less to hiring equally qualified men over women. Psychological research on implicit bias is relatively recent (§1), but a host of metaphysical (§2), epistemological (§3), and ethical questions (§4) about implicit bias are pressing.[2]. (shrink)

This book does a splendid job of describing and documenting the dysfunctional features of contemporary science mentioned in the book’s subtitle. Were I still teaching, I would have my students read this book as the basis for many productive class discussions. The margins of my copy overflow with notes, comments, and cues for further reading. The 80 pages of endnotes, for some 260 pages of text, are the best and most interesting documentation that I can recall ever finding in such (...) a book. At any rate, I recommend this book wholeheartedly; I doubt that anyone interested in the nature of contemporary science will fail to be informed and to find stimulation for further thought and reading. The Preface already promises that this will be a page-turner. Many will be astonished and disheartened by the fully documented cases of outwardly distinguished academics whose work was largely or completely fraudulent, as with Diederik Stapel (pp. 4–5 and later). Ritchie quite appropriately sees replication as the essence of science (p. 5): “If it won’t replicate, then it’s hard to describe what you’ve done as scientific at all.” Note that this is an empirical statement, not the Popperian criterion that theories must be falsifiable in principle if they are to be regarded as scientific. If a claimed observable phenomenon cannot be repeated, then we cannot know that it was real, that it happened even once, when first claimed. That’s the continuing dilemma for parapsychology, cryptozoology, for anomalistics in general. Ritchie points out that the scientific community failed to handle appropriately the issue of replication in the case of Stapel, and also with Daryl Bem’s claimed evidence of precognition. Overall, peer review and journal publication practices have not saved science from “a dizzying array of incompetence, delusion, lies, and self-deception” (p. 7). (shrink)

After a short overview of arguments pro and contra peer reviews, examples of gross misjudgement are compiled, followed by an attempt to identify some frequent, recurrent patterns of unjustified rejection of scientific manuscripts. A few specific questions are studied in more detail: the claim for still more precise and comprehensive definitions, the right way of handling ‘‘parallel theories’’, and the frequent misuse of the term ‘‘pseudoscience’’. Finally, practical rules to improve refereeing and ‘‘basic rights of authors’’ are proposed, together with (...) a word of encouragement for future authors. Keywords: Peer reviews—cases of misjudgement—patterns of unjustified rejection—misuse of the term ‘‘pseudoscience’’—proposals to improve refereeing. (shrink)

A growing body of literature has identified potential problems that can compromise the quality, fairness, and integrity of journal peer review, including inadequate review, inconsistent reviewer reports, reviewer biases, and ethical transgressions by reviewers. We examine the evidence concerning these problems and discuss proposed reforms, including double-blind and open review. Regardless of the outcome of additional research or attempts at reforming the system, it is clear that editors are the linchpin of peer review, since they make decisions that have a (...) significant impact on the process and its outcome. We consider some of the steps editors should take to promote quality, fairness and integrity in different stages of the peer review process and make some recommendations for editorial conduct and decision-making. (shrink)

We examine the criticisms and subsequent changes that arise in the course of peer review. Fifty-two scholars who had recently published in Administrative Science Quarterly were surveyed regarding their peer review experience and how their article changed from initial journal submission to eventual publication. Papers that challenged theoretical perspectives faced distinctively high levels of criticism and change, particularly with attention to methodology, while those that offered a new perspective or that extended or combined established perspectives were less criticized and changed. (...) The number of challenge-oriented publications was small as well, suggesting that either few such submissions survive the review process or few are submitted in the first place. Overall, peer review appears open to expansion of the variety of theoretical argument but does little to aid in the winnowing out of established perspectives. (shrink)

Prediction markets, which trade contracts based on the results of predictions, have been remarkably successful in predicting the results of political events. A number of proposals have been made to extend prediction markets to scientific questions, and some small-scale science prediction markets have been implemented. Advocates for science prediction markets argue that they could alleviate problems in science such as bias in peer review and epistemically unjustified consensus. I argue that bias in peer review and epistemically unjustified consensuses are genuine (...) problems in science, and that current attempts to solve them fail in practical circumstances. Prediction markets do show some promise for answering scientific questions while avoiding the pitfalls of consensus and peer review. However, there are strong reasons to believe that science prediction markets will not perform nearly so well as election prediction markets, and implementing prediction markets on a large scale could have significant unintended consequences for the organization of scientific research. The promise of prediction markets to solve current problems in assessing scientific claims is therefore largely illusory. (shrink)

BackgroundTriggered by a series of controversies and diversifying expectations of editorial practices, several innovative peer review procedures and supporting technologies have been proposed. However, adoption of these new initiatives seems slow. This raises questions about the wider conditions for peer review change and about the considerations that inform decisions to innovate. We set out to study the structure of commercial publishers’ editorial process, to reveal how the benefits of peer review innovations are understood, and to describe the considerations that inform (...) the implementation of innovations.MethodsWe carried out field visits to the editorial office of two large academic publishers housing the editorial staff of several hundreds of journals, to study their editorial process, and interviewed editors not affiliated with large publishers. Field notes were transcribed and analysed using coding software.ResultsAt the publishers we analysed, the decision-making structure seems to show both clear patterns of hierarchy and layering of the different editorial practices. While information about new initiatives circulates widely, their implementation depends on assessment of stakeholder’s wishes, impact on reputation, efficiency and implementation costs, with final decisions left to managers at the top of the internal hierarchy. Main tensions arise between commercial and substantial arguments. The editorial process is closely connected to commercial practices of creating business value, and the very specific terms in which business value is understood, such as reputation considerations and the urge to increase efficiency. Journals independent of large commercial publishers tend to have less hierarchically structured processes, report more flexibility to implement innovations, and to a greater extent decouple commercial and editorial perspectives.ConclusionOur study demonstrates that peer review innovations are partly to be understood in light of commercial considerations related to reputation, efficiency and implementations costs. These arguments extend beyond previously studied topics in publishing economics, including publishers’ choice for business or publication models and reach into the very heart of the editorial and peer review process. (shrink)

Over the past hundred years, a number of scientific investigators claim to have adduced experimental evidence for phenomena information” seems to behave like a weak signal that has to compete for the information-processing resources of the organism, a reduction of ongoing sensorimotor activity may facilitate ESP detection. Such a meaningful convergence of results suggests that psi phenomena may represent a unitary, coherent process whose nature and compatibility with current physical theory have yet to be determined. The theoretical implications and potential (...) practical applications of psi could be significant, irrespective of the small magnitude of psi effects in laboratory settings. (shrink)