Researchers increasingly rely on large data sets of health information, often linked with biological specimens. In recent years, the argument has been made that obtaining informed consent for conducting records-based research is unduly burdensome and results in ?consent bias.? As a type of selection bias, consent bias is said to exist when the group giving researchers access to their data differs from the group denying access. Therefore, to promote socially beneficial research, it is argued that consent should be unnecessary. After (...) analyzing the biostatistics evidence and bioethics arguments, the article concludes that (1) claims about the amount of consent bias are overstated; (2) commonly used statistical methods usually can account for consent bias; and (3) any residual effects of consent bias are below an acceptable level of imprecision and constitute a reasonable social cost for conducting ethically responsible research. (shrink)

The enduring replication crisis in many scientific disciplines casts doubt on the ability of science to estimate effect sizes accurately, and in a wider sense, to self-correct its findings and to produce reliable knowledge. We investigate the merits of a particular countermeasure—replacing null hypothesis significance testing with Bayesian inference—in the context of the meta-analytic aggregation of effect sizes. In particular, we elaborate on the advantages of this Bayesian reform proposal under conditions of publication bias and other methodological imperfections that are (...) typical of experimental research in the behavioral sciences. Moving to Bayesian statistics would not solve the replication crisis single-handedly. However, the move would eliminate important sources of effect size overestimation for the conditions we study. (shrink)

The enduring replication crisis in many scientific disciplines casts doubt on the ability of science to estimate effect sizes accurately, and in a wider sense, to self-correct its findings and to produce reliable knowledge. We investigate the merits of a particular countermeasure—replacing null hypothesis significance testing with Bayesian inference—in the context of the meta-analytic aggregation of effect sizes. In particular, we elaborate on the advantages of this Bayesian reform proposal under conditions of publication bias and other methodological imperfections that are (...) typical of experimental research in the behavioral sciences. Moving to Bayesian statistics would not solve the replication crisis single-handedly. However, the move would eliminate important sources of effect size overestimation for the conditions we study. (shrink)

Replicability is widely taken to ground the epistemic authority of science. However, in recent years, important published findings in the social, behavioral, and biomedical sciences have failed to replicate, suggesting that these fields are facing a “replicability crisis.” For philosophers, the crisis should not be taken as bad news but as an opportunity to do work on several fronts, including conceptual analysis, history and philosophy of science, research ethics, and social epistemology. This article introduces philosophers to these discussions. First, I (...) discuss precedents and evidence for the crisis. Second, I discuss methodological, statistical, and social-structural factors that have contributed to the crisis. Third, I focus on the philosophical issues raised by the crisis. Finally, I discuss proposed solutions and highlight the gaps that philosophers could focus on. (shrink)

Advocates of the self-corrective thesis argue that scientific method will refute false theories and find closer approximations to the truth in the long run. I discuss a contemporary interpretation of this thesis in terms of frequentist statistics in the context of the behavioral sciences. First, I identify experimental replications and systematic aggregation of evidence (meta-analysis) as the self-corrective mechanism. Then, I present a computer simulation study of scientific communities that implement this mechanism to argue that frequentist statistics may converge upon (...) a correct estimate or not depending on the social structure of the community that uses it. Based on this study, I argue that methodological explanations of the “replicability crisis” in psychology are limited and propose an alternative explanation in terms of biases. Finally, I conclude suggesting that scientific self-correction should be understood as an interaction effect between inference methods and social structures. (shrink)

Computing is no longer the sole domain of professionals, educated and trained through traditional routes to service public and private sector organisations under paid contracts. Computing has been democratised with the advent of economically accessible hardware, a multitude of software tools and the internet. Computing is by everyone for everyone.

For the Greeks, O King, who make logical demonstrations, use words emptied of power, and this very activity is what constitutes their philosophy, a mere noise of words. But we [Egyptians] do not use words [logoi] but sounds [phōnai] which are full of effects.If I could tell you what it meant, there would be no point in dancing it.The Eleatic thinker Zeno was a friend, perhaps adopted son, and student of Parmenides. He is famous for his many paradoxes on space (...) and motion, such as Achilles and the tortoise. He was more than just an intellectual, however, having many pursuits including politics, which led to his gruesome death. Running arms to the Liparans, who were resisting the growing power of the Athenians, he .. (shrink)

This paper looks at an appeal to the authority of biomedical research that has recently been used by empirical economists to motivate and justify their methods. I argue that those who make this appeal mistake the nature of biomedical research. Randomised trials, which are said to have revolutionised biomedical research, are a central methodology, but according to only one paradigm. There is another paradigm at work in biomedical research, the inferentialist paradigm, in which randomised trials play no special role. I (...) outline the inferentialist alternative in broad strokes, apply it to a recent controversy in econometrics and draw some general conclusions concerning econometric methodology. (shrink)

Two approaches to evidential reasoning compete in the biomedical and social sciences: the experimental and the pragmatist. Whereas experimentalism has received considerable philosophical analysis and support since the times of Bacon and Mill, pragmatism about evidence has been neither articulated nor defended. The overall aim is to fill this gap and develop a theory that articulates the latter. The main ideas of the theory will be illustrated and supported by a case study on the smoking/lung cancer controversy in the 1950s.

Francesco Guala once wrote that ‘The problem of extrapolation is a minor scandal in the philosophy of science’. This paper agrees with the statement, but for reasons different from Guala’s. The scandal is not, or not any longer, that the problem has been ignored in the philosophy of science. The scandal is that framing the problem as one of external validity encourages poor evidential reasoning. The aim of this paper is to propose an alternative—an alternative which constitutes much better evidential (...) reasoning about target systems of interest, and which makes do without consideration of external validity. (shrink)

20 years ago, the Report of the Commission on Research Integrity was submitted to the Secretary of the Department of Health and Human Services and to House and Senate Committees. As directed in enabling legislation, the Commission had provided recommendations on a new definition of research misconduct, oversight of scientific practices, and development of a regulation to protect whistleblowers. Reflecting the ethos of the time, the Commission recommended that institutions receiving Public Health Service research funding should provide oversight of all (...) but the most egregious misconduct. The suggested definition of research misconduct was organized around misappropriation, interference and misrepresentation, which would have addressed collaborative/authorship disputes and sabotage in scientific laboratories, both of which remain unaddressed in current policy. The Commission also recommended the Whistleblower Bill of Rights and Responsibilities which would have authorized remedies for whistleblowers who experienced retaliation and sanctions against retaliators. Response from the scientific community was highly critical, and none of the Commission’s recommendations was accepted. No new body has examined issues within the Commission’s charge, there has been no significant Congressional or public pressure to do so, institutions have not been able to sustain standards that would have avoided current concerns about bias and irreproducibility in research, and there is still no entity in science capable of addressing issues assigned to the Commission and other urgent issues. (shrink)

P-hacking or data dredging involves manipulation of the research data in order to obtain a statistically significant result. The reasons behind P-hacking and the consequences of the same are discussed in the present manuscript.

The debates between Bayesian, frequentist, and other methodologies of statistics have tended to focus on conceptual justifications, sociological arguments, or mathematical proofs of their long run properties. Both Bayesian statistics and frequentist (“classical”) statistics have strong cases on these grounds. In this article, we instead approach the debates in the “Statistics Wars” from a largely unexplored angle: simulations of different methodologies’ performance in the short to medium run. We conducted a large number of simulations using a straightforward decision problem based (...) around tossing a coin with unknown bias and then placing bets. In this simulation, we programmed four players, inspired by Bayesian statistics, frequentist statistics, Jon Williamson’s version of Objective Bayesianism, and a player who simply extrapolates from observed frequencies to general frequencies. The last player functions as a benchmark: a statistical methodology should at least outperform a crude form of induction. We focused on the performance of these methodologies in guiding the players towards good decisions. Unlike an earlier simulation study of this type, we found no systematic difference in performance between the Bayesian and frequentist players, provided the Bayesian used a flat prior and the frequentist used a low confidence level. Unlike that study, we were able to use Big Data methods to mitigate problems of random error in the simulation results. The Williamsonian player, who is a novel element of our study, also had no systematic differences in their performance, provided that they use a low confidence level. These players performed similarly even in the very short run, when players were making different decisions. Our study indicates that all three methodologies should be taken seriously by philosophers and practitioners of statistics. However, the frequentist and Williamsonian performed poorly when their confidence levels were high, and the Bayesian was surprisingly harmed by biased priors, providing some unexpected lessons for these methodologies when facing this type of decision problem. (shrink)

Recent replication crises in psychology and other fields have led to intense reflection about the validity of common research practices. Much of this reflection has focussed on reporting standards, and how they may be related to the questionable research practices that could underlie a high proportion of irreproducible findings in the published record. As a developing field, it is particularly important for Experimental Philosophy to avoid some of the pitfalls that have beset other disciplines. To this end, here we provide (...) a detailed, comprehensive assessment of current reporting practices in Experimental Philosophy. We focus on the quality of statistical reporting and the disclosure of information about study methodology. We assess all the articles using quantitative methods that were published over the years 2013–2016 in 29 leading philosophy journals. We find that null hypothesis significance testing is the prevalent statistical practice in Experimental Philosophy, although relying solely on this approach has been criticised in the psychological literature. To augment this approach, various additional measures have become commonplace in other fields, but we find that Experimental Philosophy has adopted these only partially: 53% of the papers report an effect size, 28% confidence intervals, 1% examined prospective statistical power and 5% report observed statistical power. Importantly, we find no direct relation between an article’s reporting quality and its impact. We conclude with recommendations for authors, reviewers and editors in Experimental Philosophy, to facilitate making research statistically-transparent and reproducible. (shrink)

Arguments discrediting the value-free ideal of science have left us with the question of how to distinguish desirable values from biases that compromise the reliability of research. In this paper, I argue for a characterization of cognitive biases as deviations of thought processes that systematically lead scientists to the wrong conclusions. In particular, cognitive biases could help us understand a crucial issue in science today: how systematic error is introduced in research outcomes, even when research is evaluated as of good (...) quality. To conclude, I suggest that some debiasing mechanisms have great potential for countering implicit methodological biases in science. (shrink)

Philosophy of science is expanding via the introduction of new digital data and tools for their analysis. The data comprise digitized published books and journal articles, as well as heretofore unpublished material such as images, archival text, notebooks, meeting notes, and programs. The growth in available data is matched by the extensive development of automated analysis tools. The variety of data sources and tools can be overwhelming. In this article, we survey the state of digital work in the philosophy of (...) science, showing what kinds of questions can be answered and how one can go about answering them. (shrink)

Clinical trials play a prominent role today in medicine, but are not without controversy. These issues start from the day physicians begin their specialization process in medical school and continues onto their day-to-day practice as attendings with referral patterns and resulting financial incentives. This combined with the lack of training in basic issues of epistemology and statistics, allows poor interpretations of clinical trials to reign free. A proposal to integrate the notion of severity to help remedy these issues are made (...) by ensuring that studies are tested “As Severe As Reasonably Possible” will put P-values, multiplicity adjustments, and model checking in their rightful place. In light of this, multiple clinical trials are examined with an in-depth analysis of erroneous statistical inferences taken from phase III randomized trials, non-inferiority, early phase, as well as observational studies. Severe testing in the setting of medicine will help with alleviate some of the issues with the current misinterpretation of clinical studies. (shrink)

Citation-based metrics are increasingly used as a proxy to define representative, considerable, or significant papers. We challenge this belief by taking into account factors that may play a role in providing citations to a manuscript and whether/how those highly cited studies could shape a scientific field. A different approach to summarisation of relevant core publications within a topic is proposed.

In experimental cognitive psychology, objects of inquiry are typically operationalized with psychological tasks. When interpreting results from such tasks, we focus primarily on behavioral measures such as reaction times and accuracy rather than experiences – i.e., phenomenology – associated with the task, and posit that the tasks elicit the desired cognitive phenomenon. Evaluating whether the tasks indeed elicit the desired phenomenon can be facilitated by understanding the experience during task performance. In this paper we explore the breadth of experiences that (...) are elicited by and accompany task performance using in-depth phenomenological and qualitative methodology to gather subjective reports during the performance of a visuo-spatial change detection task. Thirty-one participants were asked to remember either colors, orientations or positions of the presented stimuli and recall them after a short delay. Qualitative reports revealed rich experiential landscapes associated with the task-performance, suggesting a distinction between two broad classes of experience: phenomena at the front of consciousness and background feelings. The former includes cognitive strategies and aspects of metacognition, whereas the latter include more difficult-to-detect aspects of experience that comprise the overall sense of experience. We focus primarily on the background feelings, since strategies of task-performance to a large extent map onto previously identified cognitive processes and discuss the methodological implications of our findings. (shrink)

Over recent years, the research community has been increasingly using preprint servers to share manuscripts that are not yet peer-reviewed. Even if it enables quick dissemination of research findings, this practice raises several challenges in publication ethics and integrity. In particular, preprints have become an important source of information for stakeholders interested in COVID19 research developments, including traditional media, social media, and policy makers. Despite caveats about their nature, many users can still confuse pre-prints with peer-reviewed manuscripts. If unconfirmed but (...) already widely shared first-draft results later prove wrong or misinterpreted, it can be very difficult to “unlearn” what we thought was true. Complexity further increases if unconfirmed findings have been used to inform guidelines. To help achieve a balance between early access to research findings and its negative consequences, we formulated five recommendations: (a) consensus should be sought on a term clearer than ‘pre-print’, such as ‘Unrefereed manuscript’, “Manuscript awaiting peer review” or ‘’Non-reviewed manuscript”; (b) Caveats about unrefereed manuscripts should be prominent on their first page, and each page should include a red watermark stating ‘Caution—Not Peer Reviewed’; (c) pre-print authors should certify that their manuscript will be submitted to a peer-review journal, and should regularly update the manuscript status; (d) high level consultations should be convened, to formulate clear principles and policies for the publication and dissemination of non-peer reviewed research results; (e) in the longer term, an international initiative to certify servers that comply with good practices could be envisaged. (shrink)

Recent reports and papers on chimeric research highlight the promise of chimeric models of human neuropsychiatric disorders to ameliorate human suffering due to autism spectrum disorders, depression, and schizophrenia. These calls, however, typically do not acknowledge, much less address, criticisms of model creation and validation, or concerns about scientific conduct more generally. The ethical justification for the use of nonhuman animals in research depends on the production of benefits to humans based on such research. But the assessment and production of (...) benefits are highly uncertain and rife with both practical and conceptual challenges. This essay provides a general framework for classifying the benefits of biomedical research and then focuses on two factors that directly impact—and threaten—the production of knowledge in research that models neuropsychiatric disorders. (shrink)

A standard view would suggest that research is a neutral apolitical activity. It neutralizes external pressures by its fidelity to robust scientific methods. However, politics is an inevitable part of human knowledge. Our knowledge of the world is always mediated by human priorities. What matters is therefore a contested and political debate rather a neutral accumulation of factual data. How researchers manage this varies. Research paradigms are one way in which research engages with knowledge. They frame knowledge within epistemological and (...) ontological philosophies. In this paper, I will explore this view in relation to neo‐positivism, qualitative research, Foucault and critical realism. I will argue that if nursing knowledge is to be effective it needs to acknowledge the political, particularly in the context of neoliberalism. Healthcare systems are having to cope with a social world increasingly dominated by market fundamentalism, extreme levels of inequality and a rise in xenophobia. These forces are undermining the provision of ethically sound health care, misdirecting research practice and contributing to a discourse of dehumanization. These forces need to be challenged politically and I will argue that epistemologically diverse approaches, alongside a realist ontology can provide a way forward for nursing research. (shrink)

Patients who leave the hospital prior to their medically recommended endpoint (i.e., a discharge against medical advice, or “AMA”) typically prompt considerable consternation among physicians and o...

One of the critical issues in the philosophy of science is to understand scientific knowledge. This paper proposes a novel approach to the study of reflection on science, called “cognitive metascience”. In particular, it offers a new understanding of scientific knowledge as constituted by various kinds of scientific representations, framed as cognitive artifacts. It introduces a novel functional taxonomy of cognitive artifacts prevalent in scientific practice, covering a huge diversity of their formats, vehicles, and functions. As a consequence, toolboxes, conceptual (...) frameworks, theories, models, and individual hypotheses can be understood as artifacts supporting our cognitive performance. It is also shown that by empirically studying how artifacts function, we may discover hitherto undiscussed virtues and vices of these scientific representations. This paper relies on the use of language technology to analyze scientific discourse empirically, which allows us to uncover the metascientific views of researchers. This, in turn, can become part of normative considerations concerning virtues and vices of cognitive artifacts. (shrink)

There is much discussion in the media and some of the scientific literature of how many of the conclusions from scientific research should be doubted. These critiques often focus on studies – typically in non‐experimental spheres of biomedical and social sciences – that search large datasets for novel correlations, with a risk that inappropriate statistical evaluations might yield dubious conclusions. By contrast, results from experimental biological research can often be interpreted largely without statistical analysis. Typically: novel observation(s) are reported, and (...) an explanatory hypothesis is offered; multiple labs undertake experiments to test the hypothesis; interpretation of the results may refute the hypothesis, support it or provoke its modification; the test/revise sequence is reiterated many times; and the field moves forward. I illustrate this experimental/non‐experimental dichotomy by examining the contrasting recent histories of: (a) our remarkable and growing understanding of how several inositol‐containing phospholipids contribute to the lives of eukaryote cells; and (b) the difficulty of achieving any agreed mechanistic understanding of why consuming dietary supplements of inositol is clinically beneficial in some metabolic diseases. (shrink)

Fairness in machine learning (ML) is an ever-growing field of research due to the manifold potential for harm from algorithmic discrimination. To prevent such harm, a large body of literature develops new approaches to quantify fairness. Here, we investigate how one can divert the quantification of fairness by describing a practice we call “fairness hacking” for the purpose of shrouding unfairness in algorithms. This impacts end-users who rely on learning algorithms, as well as the broader community interested in fair AI (...) practices. We introduce two different categories of fairness hacking in reference to the established concept of p-hacking. The first category, intra-metric fairness hacking, describes the misuse of a particular metric by adding or removing sensitive attributes from the analysis. In this context, countermeasures that have been developed to prevent or reduce p-hacking can be applied to similarly prevent or reduce fairness hacking. The second category of fairness hacking is inter-metric fairness hacking. Inter-metric fairness hacking is the search for a specific fair metric with given attributes. We argue that countermeasures to prevent or reduce inter-metric fairness hacking are still in their infancy. Finally, we demonstrate both types of fairness hacking using real datasets. Our paper intends to serve as a guidance for discussions within the fair ML community to prevent or reduce the misuse of fairness metrics, and thus reduce overall harm from ML applications. (shrink)

The current strategy of searching for an effective treatment for COVID-19 relies mainly on repurposing existing therapies developed to target other diseases. Conflicting results have emerged in regard to the efficacy of several tested compounds but later results were negative. The number of conducted and ongoing trials and the urgent need for a treatment pose the risk that false-positive results will be incorrectly interpreted as evidence for treatments’ efficacy and a ground for drug approval. Our purpose is twofold. First, we (...) show that the number of drug-repurposing trials can explain the false-positive results. Second, we assess the evidence for treatments’ efficacy from the perspective of evidential pluralism and argue that considering mechanistic evidence is particularly needed in cases when the evidence from clinical trials is conflicting or of low quality. Our analysis is an application of the program of Evidence Based Medicine Plus to the drug repurposing trials for COVID. Our study shows that if decision-makers applied EBM+, authorizing the use of ineffective treatments would be less likely. We analyze the example of trials assessing the efficacy of hydroxychloroquine as a treatment for COVID-19 and mechanistic evidence in favor of and against its therapeutic power to draw a lesson for decision-makers and drug agencies on how excessive hypothesis testing can lead to spurious findings and how studying negative mechanistic evidence can be helpful in discriminating genuine from spurious results. (shrink)

The crisis of replication has led many to blame statistical significance tests for making it too easy to find impressive looking effects that do not replicate. However, the very fact it becomes difficult to replicate effects when features of the tests are tied down actually serves to vindicate statistical significance tests. While statistical significance tests, used correctly, serve to bound the probabilities of erroneous interpretations of data, this error control is nullified by data-dredging, multiple testing, and other biasing selection effects. (...) Arguments claiming to vitiate statistical significance tests attack straw person variants of tests that commit well-known fallacies and misinterpretations. There is a tension between popular calls for preregistration – arguably, one of the most promising ways to boost replication – and accounts that downplay error probabilities: Bayes Factors, Bayesian posteriors, likelihood ratios. By underscoring the importance of error control for well testedness, the replication crisis points to reformulating tests so as to avoid fallacies and report the extent of discrepancies that are and are not indicated with severity. (shrink)

While the common procedure of statistical significance testing and its accompanying concept of p-values have long been surrounded by controversy, renewed concern has been triggered by the replication crisis in science. Many blame statistical significance tests themselves, and some regard them as sufficiently damaging to scientific practice as to warrant being abandoned. We take a contrary position, arguing that the central criticisms arise from misunderstanding and misusing the statistical tools, and that in fact the purported remedies themselves risk damaging science. (...) We argue that banning the use of p-value thresholds in interpreting data does not diminish but rather exacerbates data-dredging and biasing selection effects. If an account cannot specify outcomes that will not be allowed to count as evidence for a claim—if all thresholds are abandoned—then there is no test of that claim. The contributions of this paper are: To explain the rival statistical philosophies underlying the ongoing controversy; To elucidate and reinterpret statistical significance tests, and explain how this reinterpretation ameliorates common misuses and misinterpretations; To argue why recent recommendations to replace, abandon, or retire statistical significance undermine a central function of statistics in science: to test whether observed patterns in the data are genuine or due to background variability. (shrink)

Moral, social, political, and other “nonepistemic” values can lead to bias in science, from prioritizing certain topics over others to the rationalization of questionable research practices. Such values might seem particularly common or powerful in the social sciences, given their subject matter. However, I argue first that the well-documented phenomenon of motivated reasoning provides a useful framework for understanding when values guide scientific inquiry (in pernicious or productive ways). Second, this analysis reveals a parity thesis: values influence the social and (...) natural sciences about equally, particularly because both are so prominently affected by desires for social credit and status, including recognition and career advancement. Ultimately, bias in natural and social science is both natural and social— that is, a part of human nature and considerably motivated by a concern for social status (and its maintenance). Whether the pervasive influence of values is inimical to the sciences is a separate question. (shrink)

There are currently serious concerns that published scientific findings often fail to be reproducible, and that some solutions may be gleaned by attending the several methodological and sociological recommendations that could be found in the literature. However, researchers would also arrive at some answers by considering the advice of the philosophy of science, particularly semantics, about theses on truth related to scientific realism. Sometimes scientists understand the correspondence thesis of truth as asserting that the next unique empirical confirmation of a (...) hypothesis suffices to attribute truth to it provisionally. Such empiricist bias is not necessarily at the core of CTT, but Mario Bunge proposed the synthetic thesis of truth, based on CTT, to explicitly avoid the bias. STT requires considering a hypothesis corroborated, both by purely empirical confirmation and external consistency or compatibility with the bulk of existing background knowledge. While a capricious understanding of CTT could be rigged to recommend the “one shot game” in hypothesis testing, STT clearly demands the use of multiple approaches, empirical as well as theoretical, and it asserts that a scientific test is effective to the extent to which it is neither purely empirical, nor viewed in isolation. Pattern consistency together with an understanding of causal relations make confirmed hypotheses robust and more reliable. The militancy of the double mechanism of hypothesis control of STT can help mitigate the reproducibility crisis. Earl Werner’s research program in community ecology is an example of the use of STT criteria, which leads to the development of reliable, cross-checked, ecological results, with high predictive capacity. (shrink)

There are currently serious concerns that published scientific findings often fail to be reproducible, and that some solutions may be gleaned by attending the several methodological and sociological recommendations that could be found in the literature. However, researchers would also arrive at some answers by considering the advice of the philosophy of science, particularly semantics, about theses on truth related to scientific realism. Sometimes scientists understand the correspondence thesis of truth as asserting that the next unique empirical confirmation of a (...) hypothesis suffices to attribute truth to it provisionally. Such empiricist bias is not necessarily at the core of CTT, but Mario Bunge proposed the synthetic thesis of truth, based on CTT, to explicitly avoid the bias. STT requires considering a hypothesis corroborated, both by purely empirical confirmation and external consistency or compatibility with the bulk of existing background knowledge. While a capricious understanding of CTT could be rigged to recommend the “one shot game” in hypothesis testing, STT clearly demands the use of multiple approaches, empirical as well as theoretical, and it asserts that a scientific test is effective to the extent to which it is neither purely empirical, nor viewed in isolation. Pattern consistency together with an understanding of causal relations make confirmed hypotheses robust and more reliable. The militancy of the double mechanism of hypothesis control of STT can help mitigate the reproducibility crisis. Earl Werner’s research program in community ecology is an example of the use of STT criteria, which leads to the development of reliable, cross-checked, ecological results, with high predictive capacity. (shrink)

While some form of evaluation has always been employed in science (e.g. peer review, hiring), formal systems of evaluation of research and researchers have recently come to play a more prominent role in many countries because of the adoption of new models of governance. According to such models, the quality of the output of both researchers and their institutions is measured, and issues such as eligibility for tenure or the allocation of public funding to research institutions crucially depends on the (...) outcomes of such measures. However, concerns have been raised over the risk that such evaluation may be threatening epistemic pluralism by penalizing the existent heterodox schools of thought and discouraging the pursuit of new ones. It has been proposed that this may happen because of epistemic bias favouring mainstream research programmes. In this paper, I claim that (1) epistemic pluralism is desirable and should be preserved; (2) formal evaluation exercises may threaten epistemic pluralism because they may be affected by some form of epistemic bias; therefore, (3) to preserve epistemic pluralism, we need some strategy to actively dampen epistemic bias. (shrink)

Universities are seldom lauded publicly for maintaining good processes and practices; instead, media stories commonly focus on shortcomings. Furthermore, universities, even when doing everything right, sometimes are unfairly targeted for criticism in circumstances in which making a public defence is difficult. A prominent case at the University of Wollongong shows how defending a university’s integrity can be hampered by confidentiality requirements, lack of public understanding of thesis examination processes and of disciplinary expectations, and university procedures not designed for extraordinary attacks. (...) The implication is that there can be value in fostering greater awareness of the ways that universities and disciplinary fields operate, and reconsidering procedures with an eye towards possible attacks, both external and internal. (shrink)

Digital Pills are an innovative drug-device technology that permits to combine traditional medications with a monitoring system that automatically records data about medication adherence as well as patients’ physiological data. Although DP are a promising innovation in the field of digital medicine, their use has also raised a number of ethical concerns. These ethical concerns, however, have been expressed principally from a theoretical perspective, whereas an ethical analysis with a more empirically oriented approach is lacking. There is also a lack (...) of clarity about the empirical evidence available concerning the application of this innovative digital medicine. To map the studies where DP have been tested on patients and discuss the ethically relevant issues evident therein, we performed a scoping review of the empirical literature concerning DP. Our search allowed us to identify 18 papers reporting on studies where DP were tested on patients. These included studies with different designs and involving patients with a variety of conditions. In the empirical literature, a number of issues with ethical relevance were evident. At the patient level, the ethical issues include users’ interaction with DP, personal sphere, health-related risks and patients’ benefits. At the provider level, ethically relevant issues touch upon the doctor-patient relationship and the question of data access. At the societal level, they concern the benefits to society, the quality of evidence and the dichotomy device-medicine. We conclude that evidence concerning DP is not robust and that more research should be performed and study results made available to evaluate this digital medicine. Moreover, our analysis of the ethically relevant aspects within empirical literature underscores that there are concrete and specific open questions that should be tackled in the ethical discussion about this new technological solution. (shrink)

‘Epistemic’ arguments for conservatism typically claim that given the limits of human reason, we are better off accepting some particular social practice or institution rather than trying to consciously improve it. I critically examine and defend here one such argument, claiming that there are some domains of social life in which, given the limits of our knowledge and the complexity of the social world, we ought to defer to those institutions that have robustly endured in a wide variety of circumstances (...) in the past while not being correlated with intolerable outcomes. These are domains of social life in which our ignorance of optimal institutions is radical, and there is uncertainty about the costs of error. This is an argument for the preservation of particular institutions, not particular policies or outcomes, and it specifically identifies these with the institutions that John Rawls called ‘the basic structure of society.’ The argument further implies that to the extent that there is any reason to change these institutions, changes should be calculated as far as possible to increase their ‘epistemic power.’. (shrink)

I draw lessons from experimental economics. I argue that the lack of mathematical formalism cannot be usefully thought as the cause of the underappreciation of contextual and generalizability considerations. Instead, this lack is problematic because it hinders a clear relationship between theory and quantitative predictions. I also advocate a pragmatic policy-focused approach as a partial remedy to the generalizability problem.