Flawed clinical practice guidelines may compromise patient care. Commercial conflicts of interest on panels that write treatment guidelines are particularly problematic, because panelists may have conflicting agendas that influence guideline recommendations. Historically, there has been no legal remedy for conflicts of interest on guidelines panels. However, in May 2008, the Attorney General of Connecticut concluded a ground-breaking antitrust investigation into the development of Lyme disease treatment guidelines by one of the largest medical societies in the United States, the Infectious Diseases (...) Society of America (IDSA). Although the investigation found significant flaws in the IDSA guidelines development process, the subsequent review of the guidelines mandated by the settlement was compromised by a lack of impartiality at various stages of the IDSA review process. This article will examine the interplay between the recent calls for guidelines reform, the ethical canons of medicine, and due process considerations under antitrust laws as they apply to the formulation of the IDSA Lyme disease treatment guidelines. The article will also discuss pitfalls in the implementation of the IDSA antitrust settlement that should be avoided in the future. (shrink)

Biomedical innovation and translation are increasingly emphasizing research using “big data.” The hope is that big data methods will both speed up research and make its results more applicable to “real-world” patients and health services. While big data research has been embraced by scientists, politicians, industry, and the public, numerous ethical, organizational, and technical/methodological concerns have also been raised. With respect to technical and methodological concerns, there is a view that these will be resolved through sophisticated information technologies, predictive algorithms, (...) and data analysis techniques. While such advances will likely go some way towards resolving technical and methodological issues, we believe that the epistemological issues raised by big data research have important ethical implications and raise questions about the very possibility of big data research achieving its goals. (shrink)

ABSTRACTDespite the extensive literature on what questionable research practices are and how to measure them, the normative underpinnings of such practices have remained less explored. QRPs often fall into a grey area of justifiable and unjustifiable practices. Where to precisely draw the line between such practices challenges individual scholars and this harms science. We investigate QRPs from a normative perspective using the theory of communicative action. We highlight the role of the collective in assessing individual behaviours. Our contribution is a (...) framework that allows identification of when particular actions cross over from acceptable to unacceptable practice. Thus, this article provides grounds for developing scientific standards to raise the quality of scientific research. (shrink)

The paper deals with the introduction of nanotechnology in biochips. Based on interviews and theoretical reflections, it explores blind spots left by technology assessment and ethical investigations. These have focused on possible consequences of increased diffusability of a diagnostic device, neglecting both the context of research as well as increased accuracy, despite it being a more essential feature of nanobiochip projects. Also, rather than one of many parallel aspects in innovation processes, ethics is considered here as a ubiquitous system of (...) choices between sometimes antagonistic values. Thus, the paper investigates what is at stake when accuracy is balanced with other practical values in different contexts. Dramatic nanotechnological increase of accuracy in biochips can raise ethical issues, since it is at odds with other values such as diffusability and reliability. But those issues will not be as revolutionary as is often claimed: neither in diagnostics, because accuracy of measurements is not accuracy of diagnostics; nor in research, because a boost in measurement accuracy is not sufficient to overcome significance-chasing malpractices. The conclusion extends to methodological recommendations. (shrink)

The aim of this work is to identify and analyze a set of challenges that are likely to be encountered when one embarks on fieldwork in linguistic communities that feature small, young, and/or non-standard languages with a goal to elicit big sets of rich data. For each challenge, we (i) explain its nature and implications, (ii) offer one or more examples of how it is manifested in actual linguistic communities, and (iii) where possible, offer recommendations for addressing it effectively. Our (...) list of challenges involves static characteristics (e.g., absence of orthographic conventions and how it affects data collection), dynamic processes (e.g., speed of language change in small languages and how it affects longitudinal collection of big amounts of data), and interactive relations between non-dynamic features that are nevertheless subject to potentially rapid change (e.g., absence of standardized assessment tools or estimates for psycholinguistic variables). The identified challenges represent the domains of data collection and handling, participant recruitment, and experimental design. Among other issues, we discuss population limits and degree of power, inter- and intraspeaker variation, absence of metalanguage and its implications for the process of eliciting acceptability judgments, and challenges that arise from absence of local funding, conflicting regulations in relation to privacy issues, and exporting large samples of data across countries. Finally, the ten experimental challenges presented cover a broad typological spectrum, encompassing both spoken and sign, extant and nearly extinct languages. (shrink)

Under the traditional system of peer-reviewed publication, the degree of prestige conferred to authors by successful publication is tied to the degree of the intellectual rigor of its peer review process: ambitious scientists do well professionally by doing well epistemically. As a result, we should expect journal editors, in their dual role as epistemic evaluators and prestige-allocators, to have the power to motivate improved author behavior through the tightening of publication requirements. Contrary to this expectation, I will argue that the (...) publication bias literature in academic medicine demonstrates that editor interventions have had limited effectiveness in improving the health of the publication and trial registration record, suggesting that much stronger interventions are needed. (shrink)

Evidence-based medicine frequently uses statistical hypothesis testing. In this paradigm, data can only disconfirm a research hypothesis’ competitors: One tests the negation of a statistical hypothesis that is supposed to correspond to the research hypothesis. In practice, these hypotheses are often misaligned. For instance, directional research hypotheses are often paired with non-directional statistical hypotheses. Prima facie, one cannot gain proper evidence for one’s research hypothesis employing a misaligned statistical hypothesis. This paper sheds lights on the nature of and the reasons (...) for such misalignments and it provides a thorough analysis of whether they pose a threat to evidence-based medicine. The upshots are that the misalignments are often hidden for clinicians and that although some cases of misalignments can be partially counterbalanced, the overall threat is non-negligible. The counterbalances either lead to methodological inadequacy, loss of statistical power, or involve a lack of information that could be crucial for decision making. This result casts doubt on various findings of medical studies in addition to issues associated with under-powered studies or the replication crisis. (shrink)

While it is widely acknowledged that psychology is in the throes of a replication ‘crisis’, relatively little attention has been paid to the role theory plays in our evaluation of replications as ‘failed’ or ‘successful’. This paper applies well-known arguments in philosophy of science about the interplay between theory and experiment to a contemporary case study of infants’ understanding of false belief (Onishi and Baillargeon [2005]), and attempts to replicate it. It argues that the lack of consensus about over-arching theories (...) informing both the concepts under study and the methodologies used to track them means that researchers disagree over which experiments constitute replications of the original. The second part of the paper places this specific debate within a broader discussion of the replication crisis as a crisis of ‘theory’, developing work by Muthukrishna and Henrich ([2018]) and Bird ([2018]). Bird argues that the lack of agreed over-arching theories in psychology means that a high rate of replication failure is to be expected; this paper agrees with his diagnosis but challenges his proposal that more replication will resolve the problem. (shrink)

We may well be witnessing a decisive event in the history of knowledge as diversity is becoming one of the premier values of late modern societies. We seek to preserve and foster biodiversity, neurodiversity, racial diversity, ethnic diversity, gender diversity, linguistic diversity, cultural diversity, and perspectival diversity. Perspectival diversity has become the passage point through which other forms of diversity must pass to become epistemically consequential. This article examines how two of its varieties, viewpoint diversity and educational diversity, have come (...) to transform the moral economy of science. Both aim at multiplying perspectives on a given subject, but their political subtexts differ markedly. The valorization of educational diversity followed a US Supreme Court decision in 1978 that enabled universities to advance social justice, if they justified race-conscious admissions in terms of the pedagogic benefits of a more diverse student body for all. By contrast, the proponents of viewpoint diversity aim at the reform of scientific knowledge production and distribution rather than the reallocation of status and power among different social groups. We examine the political epistemology of viewpoint diversity by analyzing a controversy between social psychologists who, amid the American culture wars of the 2010s, debated how to rein in their political biases in a scientific field supposedly lacking political diversity. Out of this scientific controversy grew Heterodox Academy, an activist organization promoting viewpoint diversity in higher education. By relating and comparing viewpoint and educational diversity, we clarify what is at stake epistemically in the US-centric moral economy of diversity. (shrink)

Despite all the attention to Big Data and the claims that it represents a “paradigm shift” in science, we lack understanding about what are the qualities of Big Data that may contribute to this revolutionary impact. In this paper, we look beyond the quantitative aspects of Big Data and examine it from a sociotechnical perspective. We argue that a key factor that distinguishes “Big Data” from “lots of data” lies in changes to the traditional, well-established “control zones” that facilitated clear (...) provenance of scientific data, thereby ensuring data integrity and providing the foundation for credible science. The breakdown of these control zones is a consequence of the manner in which our network technology and culture enable and encourage open, anonymous sharing of information, participation regardless of expertise, and collaboration across geographic, disciplinary, and institutional barriers. We are left with the conundrum—how to reap the benefits of Big Data while re-creating a trust fabric and an accountable chain of responsibility that make credible science possible. (shrink)

Research misconduct remains an important problem in health research despite decades of local, national, regional, and international efforts to eliminate it. The ultimate goal of every health research project, irrespective of setting, is to produce trustworthy findings to address local as well as global health issues. To be able to lead or participate meaningfully in international research collaborations, individual and institutional capacities for research integrity are paramount. Accordingly, this paper concerns itself not only with individuals’ research skills but also with (...) institutional and national policies and governance. Such policies and governance provide an ethical scaffold for the production of knowledge and structure incentives. This paper’s operational definition of research therefore draws from Institute of Medicine’s articulation of health research as an inquiry that aims to produce knowledge about the structure, processes, or effects of personal health services; and from an existing health systems framework. The paper reviews the research regulatory environment and the ethics apparatus in Ghana, and describes a project jointly undertaken by Ghanaian researchers in collaboration with New York University to assess the perceived adequacy of current institutional practices, opportunities, and incentives for promoting RI. (shrink)

In this work, we explore the epistemic import of the value-ladenness of Neyman-Pearson’s Theory of Testing Hypotheses by reconstructing and extending Daniel Steel’s argument for the legitimate influence of pragmatic values on scientific inference. We focus on how to properly understand N-P’s pragmatic value-ladenness and the epistemic reliability of N-P. We develop an account of the twofold influence of pragmatic values on N-P’s epistemic reliability and replicability. We refer to these two distinguished aspects as “direct” and “indirect”. We discuss the (...) replicability of experiments in terms of the indirect aspect and the replicability of outcomes in terms of the direct aspect. We argue that the influence of pragmatic values is beneficial to N-P’s epistemic reliability and replicability indirectly. We show that while the direct influence of pragmatic values can be beneficial, its negative effects on reliability and replicability are also unavoidable in some cases, with the direct and indirect aspects possibly being incongruent. (shrink)

We show that if among the tested hypotheses the number of true hypotheses is not equal to the number of false hypotheses, then Neyman-Pearson theory of testing hypotheses does not warrant minimal epistemic reliability. We also argue that N-P does not protect from the possible negative effects of the pragmatic value-laden unequal setting of error probabilities on N-P’s epistemic reliability. Most importantly, we argue that in the case of a negative impact no methodological adjustment is available to neutralize it, so (...) in such cases the discussed pragmatic-value-ladenness of N-P inevitably compromises the goal of attaining truth. (shrink)

In light of the widespread existence of financial and non-financial issues that contribute to the appearance or fact of conflict of interest, it is proposed that conflict of interest should generally be assumed, no matter the source of financial support or the expressed declarations of conflicts and even with respect to one’s own work. No new model is advanced for modification of peer-review processes or for elaboration of author declarations of interest. Researchers should be assessing the quality of published work (...) as best they can and make their own decisions on the appropriate use of the work. While some apparent sources of conflict are likely more obvious and serious than others, even subtler biases can influence scientific reports. Ignoring peer-reviewed contributions because of conflict-of-interest concerns is discouraged. Listening skeptically to all sources, including yourself, is encouraged. (shrink)

Journal peer review is an essential part of academic practices.1 But how well does it serve its purpose and which factors have an influence on how close it comes to achieving its aims? Peer review has been widely discussed in empirical literature: it has been studied both qualitatively and quantitatively (e.g., by Cole, who in his 1992 book uses data on how grant applications submitted to National Science Foundation were...

The promises of precision medicine are often heralded in the medical and lay literature, but routine integration of genomics in clinical practice is still limited. While the “last mile” infrastructure to bring genomics to the bedside has been demonstrated in some healthcare settings, a number of challenges remain — both in the receptivity of today's health system and in its technical and educational readiness to respond to this evolution in care. To improve the impact of genomics on health and disease (...) management, we will need to integrate both new knowledge and new care processes into existing workflows. This change will be onerous and time-consuming, but hopefully valuable to the provision of high quality, economically feasible care worldwide. (shrink)

Antidepressants, in particular newer agents, are among the most widely prescribed medications worldwide with annual sales of billions of dollars. The introduction of these agents in the market has passed through seemingly strict regulatory control. Over a thousand randomized trials have been conducted with antidepressants. Statistically significant benefits have been repeatedly demonstrated and the medical literature is flooded with several hundreds of.

Objectivity, as one of the key attributes of science, has become an indispensable part of its ethos and a central theme of the philosophy of science. As such, it has been a subject of philosophical reflection by a number of authors. In our project – in which both philosophers of science and scientists participate – we examine the concept of objectivity in the natural sciences with the tools of experimental philosophy. We aim to identify specific operational dimensions of objectivity, those (...) with which current scientist actually work, thus making them accessible for further theoretical analysis and research. In this text, we present results from the first, qualitative phase of our research based on in-depth interviews and focus groups with scientists working in the Czech Republic. Further, we confront this empirical data with theoretical notions of objectivity. We also present other findings from the research and formulate hypotheses for the research’s subsequent phases. (shrink)

Recently, many scientists have become concerned about an excessive number of failures to reproduce statistically significant effects. The situation has become dire enough that the situation has been named the ‘reproducibility crisis’. After reviewing the relevant literature to confirm the observation that scientists do indeed view replication as currently problematic, I explain in philosophical terms why the replication of empirical phenomena, such as statistically significant effects, is important for scientific progress. Following that explanation, I examine various diagnoses of the reproducibility (...) crisis, and argue that for the majority of scientists the crisis is due, at least in part, to a form of publication bias. This conclusion sets the stage for an assessment of the view that evidential relations in science are inherently value-laden, a view championed by Heather Douglas and Kevin Elliott. I argue, in response to Douglas and Elliott, and as motivated by the meta-scientific resistance scientists harbour to a publication bias, that if we advocate the value-ladenness of science the result would be a deepening of the reproducibility crisis. (shrink)

Fake news and alternative facts have become commonplace in these so-called “post-factual times.” What about medical research - are scientific facts fake as well? Many recent disclosures have fueled the claim that scientific facts are suspect and that science is in crisis. Scientists appear to engage in facting interests instead of revealing interesting facts. This can be observed in terms of what has been called polarised research, where some researchers continuously publish positive results while others publish negative results on the (...) same issue – even when based on the same data. In order to identify and address this challenge, the objective of this study is to investigate how polarised research produce “polarised facts.” Mammography screening for breast cancer is applied as an example. The main benefit with mammography screening is the reduced breast cancer mortality, while the main harm is overdiagnosis and subsequent overtreatment. Accordingly, the Overdiagnosis to Mortality Reduction Ratio is an estimate of the risk-benefit-ratio for mammography screening. As there are intense interests involved as well as strong opinions in debates on mammography screening, one could expect polarisation in published results on OMRR. A literature search identifies 8 studies publishing results for OMRR and reveals that OMRR varies 25-fold, from 0.4 to 10. Two experts in polarised research were asked to rank the attitudes of the corresponding authors to mammography screening of the identified publications. The results show a strong correlation between the OMRR and the authors’ attitudes to screening. Mammography screening for breast cancer appears as an exemplary field of strongly polarised research. This is but one example of how scientists’ strong professional interests can polarise research. Instead of revealing interesting facts researchers may come to fact interests. In order to avoid this and sustain trust in science, researchers should disclose professional and not only financial interests when submitting and publishing research. (shrink)

_Helsevesenet har blitt en sentral samfunnsaktør og medisinen dets førende fag. Hvorfor har det blitt slik? En grunn er at medisinen fører sammen sfærer som ellers har vært adskilt: det sanne, det gode og det skjønne. Medisinen som fag og helsevesenet som institusjon, har blitt et fascinerende skjæringspunkt nettopp mellom evidens, etikk og estetikk. Her kobles kunnskap til det som er vondt og skjønnhet til det som er friskt. Kunnskapsproduksjonen dirigeres ut fra ønsket om å gjøre det gode ved å (...) unngå det vonde. Samtidig styres helsevesenets normer også av estetikkens normer. Koblingene mellom evidens, estetikk og etikk gir noen grunnleggende utfordringer, som det er viktig å være oppmerksom på. Riktig håndtert gjør de helsevesenet generelt og medisinen spesielt til et fascinerende og fruktbart skjæringspunkt mellom det sanne, det gode og det skjønne. _ _Nøkkelord:_ Evidens, etikk, estetikk, sannhete, godhet, skjønnhet _English Title:_ Evidence, ethics and aesthetics: Medicine at the intersection of the true, the good and the beautiful _Abstract:_ While Western history of ideas has tried to differentiate truth, goodness, and beauty, medicine is a fascinating meeting place precisely between evidence, ethics, and aesthetics. In medicine, knowledge is connected to pain and suffering and beauty to health. The production of knowledge is directed towards doing good by avoiding bad. At the same time, medicine is also governed by aesthetics. A range of challenges follow from the connection between the three spheres, but if we are attentive to the interconnectedness medicine becomes a fascinating and fruitful rendezvous of truth, goodness, and beauty. _Keywords_: Evidence, ethics, aesthetics, truth, goodness, beauty. (shrink)

We provide a novel articulation of the epistemic peril of p-hacking using three resources from philosophy: predictivism, Bayesian confirmation theory, and model selection theory. We defend a nuanced position on p-hacking: p-hacking is sometimes, but not always, epistemically pernicious. Our argument requires a novel understanding of Bayesianism, since a standard criticism of Bayesian confirmation theory is that it cannot represent the influence of biased methods. We then turn to pre-analysis plans, a methodological device used to mitigate p-hacking. Some say that (...) pre-analysis plans are epistemically meritorious while others deny this, and in practice pre-analysis plans are often violated. We resolve this debate with a modest defence of pre-analysis plans. Further, we argue that pre-analysis plans can be epistemically relevant even if the plan is not strictly followed—and suggest that allowing for flexible pre-analysis plans may be the best available policy option. (shrink)

Most discussions of the reproducibility crisis focus on its epistemic aspect: the fact that the scientific community fails to follow some norms of scientific investigation, which leads to high rates of irreproducibility via a high rate of false positive findings. The purpose of this paper is to argue that there is a heretofore underappreciated and understudied dimension to the reproducibility crisis in experimental psychology and neuroscience that may prove to be at least as important as the epistemic dimension. This is (...) the communication dimension. The link between communication and reproducibility is immediate: independent investigators would not be able to recreate an experiment whose design or implementation were inadequately described. I exploit evidence of a replicability and reproducibility crisis in computational science, as well as research into quality of reporting to support the claim that a widespread failure to adhere to reporting standards, especially the norm of descriptive completeness, is an important contributing factor in the current reproducibility crisis in experimental psychology and neuroscience. (shrink)

Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. (...) We conclude that on present evidence abolishing peer review weakly dominates the status quo. (shrink)

Cognitive neuroscience increasingly relies on complex data analysis methods. Researchers in this field come from highly diverse scientific backgrounds, such as psychology, engineering, and medicine. This poses challenges with respect to acquisition of appropriate scientific computing and data analysis skills, as well as communication among researchers with different knowledge and skills sets. Are researchers in cognitive neuroscience adequately equipped to address these challenges? Here, we present evidence from an online survey of methods skills. Respondents (n= 307) mainly comprised students and (...) post-doctoral researchers working in the cognitive neurosciences. Multiple choice questions addressed a variety of basic and fundamental aspects of neuroimaging data analysis, such as signal analysis, linear algebra, and statistics. We analyzed performance with respect to the following factors: undergraduate degree (grouped into Psychology, Methods, and Biology), current researcher status (undergraduate student, PhD student, and post-doctoral researcher), gender, and self-rated expertise levels. Overall accuracy was 72%. Not surprisingly, the Methods group performed best (87%), followed by Biology (73%) and Psychology (66%). Accuracy increased from undergraduate (59%) to PhD (74%) level, but not from PhD to post-doctoral (74%) level. The difference in performance for the Methods vs. non-methods (Psychology/Biology) groups was especially striking for questions related to signal analysis and linear algebra, two areas particularly relevant to neuroimaging research. Self-rated methods expertise was not strongly predictive of performance. The majority of respondents (93%) indicated they would like to receive at least some additional training on the topics covered in this survey. In conclusion, methods skills among junior researchers in cognitive neuroscience can be improved, researchers are aware of this, and there is strong demand for more skills-oriented training opportunities. We hope that this survey will provide an empirical basis for the development of bespoke skills-oriented training programs in cognitive neuroscience institutions. We will provide practical suggestions on how to achieve this. (shrink)

Researchers are concerned about whether manipulations have the intended effects. Many journals and reviewers view manipulation checks favorably, and they are widely reported in prestigious journals. However, the prototypical manipulation check is a verbal (rather than behavioral) measure that always appears at the same point in the procedure (rather than its order being varied to assess order effects). Embedding such manipulation checks within an experiment comes with problems. While we conceptualize manipulation checks as measures, they can also act as interventions (...) which initiate new processes that would otherwise not occur. The default assumption that manipulation checks do not affect experimental conclusions is unwarranted. They may amplify, undo, or interact with the effects of a manipulation. Further, the use of manipulation checks in mediational analyses does not rule out confounding variables, as any unmeasured variables that correlate with the manipulation check may still drive the relationship. Alternatives such as nonverbal and behavioral measures as manipulation checks and pilot testing are less problematic. Reviewers should view manipulation checks more critically, and authors should explore alternative methods to ensure the effectiveness of manipulations. (shrink)

In a recent editorial published in Nature, the journal's editors comment on a new automated software that has been used to check findings in psychology publications. The editors express concern with the way in which the anonymous fact-checkers have proceeded, but at the same time, they underscore the crucial role of peer criticism for scientific progress and insist: "self-correction is at the heart of science." Brief as it is, the editorial showcases that peer criticism and the application of norms of (...) good research practice are very thorny issues indeed.It is essential for the functioning of science that empirical findings, arguments, as well as methodological approaches are scrutinized... (shrink)

Social cognition is critical for successfully navigating social relationships. Current evidence suggests that older adults exhibit poorer performance in several core social-cognitive domains compared to younger adults. Neurocognitive decline is commonly discussed as one of the key arbiters of age-related decline in social-cognitive abilities. While evidence supports this notion, age effects are likely attributable to multiple factors. This paper aims to recontextualize past evidence by focusing issues of motivation, task design, and representative samples. In light of these issues, we identify (...) directions for future research to aide our understanding of social-cognitive aging. (shrink)

Industry involvement in the machine learning (ML) community seems to be increasing. However, the quantitative scale and ethical implications of this influence are rather unknown. For this purpose, we have not only carried out an informed ethical analysis of the field, but have inspected all papers of the main ML conferences NeurIPS, CVPR, and ICML of the last 5 years—almost 11,000 papers in total. Our statistical approach focuses on conflicts of interest, innovation, and gender equality. We have obtained four main (...) findings. (1) Academic–corporate collaborations are growing in numbers. At the same time, we found that conflicts of interest are rarely disclosed. (2) Industry papers amply mention terms that relate to particular trending machine learning topics earlier than academia does. (3) Industry papers are not lagging behind academic papers with regard to how often they mention keywords that are proxies for social impact considerations. (4) Finally, we demonstrate that industry papers fall short of their academic counterparts with respect to the ratio of gender diversity. We believe that this work is a starting point for an informed debate within and outside of the ML community. (shrink)

Objectivity in the sciences is a much-touted yet problematic concept. It is sometimes held up as characterising scientific knowledge, yet operational definitions are diverse and call for such paradoxical genius as the ability to see without a perspective, to predict repeatability, to elicit nature’s own self-revelation, or to discern the structure of reality with inerrancy. Here we propose a positive and general definition of objectivity based on work in the Reformational philosophy tradition. We recognise a suite of relation-frames–ways in which (...) things function and relate to each other, which can be analytically distinguished in the process of conceptual abstraction. These relation-frames also ground the diverse aspects of scientific analysis within which relationships and properties may be abstracted from entities and systems. We argue that objectivity can be understood as characteristic of representations that attempt to portray a subject in an earlier relation-frame than that in which it characteristically functions. In short, objectivity is projection. This proposal is exemplified from mathematics and the natural sciences and some possible objections to it are considered, as well as its extension to the social sciences. (shrink)

This paper develops an account of the opacity problem in medical machine learning (ML). Guided by pragmatist assumptions, I argue that opacity in ML models is problematic insofar as it potentially undermines the achievement of two key purposes: ensuring generalizability and optimizing clinician–machine decision-making. Three opacity amelioration strategies are examined, with explainable artificial intelligence (XAI) as the predominant approach, challenged by two revisionary strategies in the form of reliabilism and the interpretability by design. Comparing the three strategies, I argue that (...) interpretability by design is most promising to overcome opacity in medical ML. Looking beyond the individual opacity amelioration strategies, the paper also contributes to a deeper understanding of the problem space and the solution space regarding opacity in medical ML. (shrink)

Science traditionally is taught as a linear process based on logic and carried out by objective researchers following the scientific method. Practice of science is a far more nuanced enterprise, one in which intuition and passion become just as important as objectivity and logic. Whether the activity is committing to study a particular research problem, drawing conclusions about a hypothesis under investigation, choosing whether to count results as data or experimental noise, or deciding what information to present in a research (...) paper, ethical challenges inevitably will arise because of the ambiguities inherent in practice. Unless these ambiguities are acknowledged and their sources understood explicitly, responsible conduct of science education will not adequately prepare the individuals receiving the training for the kinds of decisions essential to research integrity that they will have to make as scientists. (shrink)

Scientific publication is immensely important to the scientific endeavour. There is, however, concern that rewarding scientists chiefly on publication creates a perverse incentive, allowing careless and fraudulent conduct to thrive, compounded by the predisposition of top-tier journals towards novel, positive findings rather than investigations confirming null hypothesis. This potentially compounds a reproducibility crisis in several fields, and risks undermining science and public trust in scientific findings. To date, there has been comparatively little modelling on factors that influence science trustworthiness, despite (...) the importance of quantifying the problem. We present a simple phenomenological model with cohorts of diligent, careless and unethical scientists, with funding allocated by published outputs. This analysis suggests that trustworthiness of published science in a given field is influenced by false positive rate, and pressures for positive results. We find decreasing available funding has negative consequences for resulting trustworthiness, and examine strategies to combat propagation of irreproducible science. (shrink)

Concern about the integrity of empirical research has arisen in recent years in the light of studies showing the vast majority of publications in academic journals report positive results, many of these results are false and cannot be replicated, and many positive results are the product of data dredging and the application of flexible data analysis practices coupled with selective reporting. While a number of potential solutions have been proposed, the effects of these are poorly understood and empirical evaluation of (...) each would take many years. We propose that methods from the systems sciences be used to assess the effects, both positive and negative, of proposed solutions to the problem of declining research integrity such as study registration, Registered Reports, and open access to methods and data. In order to illustrate the potential application of systems science methods to the study of research integrity, we describe three broad types of models: one built on the characteristics of specific academic disciplines; one a diffusion of research norms model conceptualizing researchers as susceptible, “infected” and recovered; and one conceptualizing publications as a product produced by an industry comprised of academics who respond to incentives and disincentives. (shrink)