El conocimiento utilizado para armar e implementar políticas socioeconómicas refiere por lo general a aseveraciones causales que pueden ser conceptuadas de diferentes maneras; sin embargo, estas suelen omitir un tema central: las consecuencias alternativas o los desvíos que emergen en caso de fracasar, lo que puede acarrear consecuencias negativas. Se argumenta que, para una buena implementación, es fundamental tener en cuenta dichas consecuencias alternativas, lo cual implica un cambio sustancial en el modo de tomar decisiones, donde la asimetría de resultados (...) es relevante. (shrink)

RESUMEN El conocimiento utilizado para armar e implementar políticas socioeconómicas refiere por lo general a aseveraciones causales que pueden ser conceptuadas de diferentes maneras; sin embargo, estas suelen omitir un tema central: las consecuencias alternativas o los desvíos que emergen en caso de fracasar, lo que puede acarrear consecuencias negativas. Se argumenta que, para una buena implementación, es fundamental tener en cuenta dichas consecuencias alternativas, lo cual implica un cambio sustancial en el modo de tomar decisiones, donde la asimetría de (...) resultados es relevante. ABSTRACT The knowledge used to structure and implement socioeconomic policies generally refers to causal assertions that can be conceptualized differently; however, they tend to disregard the central issue of alternative consequences or deviations that arise in case of failure, which can cause negative effects. The paper argues that a successful implementation entails taking these alternative consequences into account. This implies a substantial change in the way decisions are made, in which the asymmetry of results becomes relevant. (shrink)

The explanatory/pragmatic-trial distinction enjoys a burgeoning philosophical and medical literature and a significant contingent of support among philosophers and healthcare stakeholders as an important way to assess the design and results of randomized controlled trials. A major motivation has been the need to provide relevant, generalizable data to drive healthcare decisions. While talk of pragmatic and explanatory trials could be seen as convenient shorthand, the distinction can also be seen as harboring deeper issues related to inferential strategies used to evaluate (...) causal claims regarding medical treatments. A comprehensive, critical analysis of the distinction and underlying epistemological framework upon which the distinction is based, particularly with respect to treatment effectiveness, has yet to be forthcoming. I provide this, analyzing the distinction’s relationship to generalizability and cognate distinctions between ideal conditions and real-world practice, internal and external validity, and efficacy and effectiveness. I also analyze recent philosophical work that relies on the explanatory/pragmatic-trial distinction and that advocates for more pragmatic trials. I conclude that as an organizing principle for trial-design decisions and trial evaluation, the explanatory/pragmatic-trial distinction is conceptually problematic and not as useful as its proponents seem to think. Since some pragmatic-trial features can be inimical to establishing treatment effectiveness, pragmatic-trial features should not be conflated with pragmatic trials’ avowed goals. If the distinction is to be useful, it and some associated concepts, including generalizability, should be reformulated, lest they continue to underlie a medical epistemology that could contribute to methodologically flawed and potentially unethical advice for the design and interpretation of trials. (shrink)

Making sense of why something succeeded or failed is central to scientific practice: it provides an interpretation of what happened, i.e. an hypothesized explanation for the results, that informs scientists’ deliberations over their next steps. In philosophy, the realism debate has dominated the project of making sense of scientists’ success and failure claims, restricting its focus to whether truth or reliability best explain science’s most secure successes. Our aim, in contrast, will be to expand and advance the practice-oriented project sketched (...) by Arthur Fine in his work on the Natural Ontological Attitude. An important obstacle to articulating a positive program, we suggest, has been overlooking how scientists adopt standardized rules and procedures in order to define and operationalize meanings for success and failure relative to their situated goals. To help fill this gap, we introduce two new ideas, design specifications and track records, and show how they advance our ability to make sense of scientific modeling practices while maintaining a deflationary stance toward the realism debate. (shrink)

To be effective, a medical intervention must improve one's health by targeting a disease. The concept of disease, though, is controversial. Among the leading accounts of disease-naturalism, normativism, hybridism, and eliminativism-I defend a version of hybridism. A hybrid account of disease holds that for a state to be a disease that state must both (i) have a constitutive causal basis and (ii) cause harm. The dual requirement of hybridism entails that a medical intervention, to be deemed effective, must target either (...) the constitutive causal basis of a disease or the harms caused by the disease (or ideally both). This provides a theoretical underpinning to the two principle aims of medical treatment: care and cure. (shrink)

There are two competing views regarding the role of mechanistic knowledge in inferences about the effectiveness of interventions. One view holds that inferences about the effectiveness of interventions should be based only on data from population-level studies (often statistical evidence from randomised trials). The other view holds that such inferences must be based in part on mechanistic evidence. The competing views are local principles of inference, the plausibility of which can be assessed by a more general normative principle of inference. (...) Bayesianism tells us to base inferences on both the ‘likelihood’ and the ‘prior’. The likelihood represents statistical evidence. One influence on the prior probability of a hypothesis like ‘d causes x’ is mechanistic knowledge of how d causes x. Thus, reasoning about such inferences by appealing to both statistical and mechanistic evidence is vindicated by our best general theory of inference. The primary contribution of this paper is to assess the merits and weaknesses of the arguments on both sides of the debate, using the Bayesian framework. This analysis lends support to those who argue that we should base our causal inferences about interventions in part on mechanistic evidence. (shrink)

Philosophers have debated at length the epistemological status of scientific thought experiments. I contend that the literature on this topic still lacks a common conceptual framework, a lacuna that produces radical disagreement among the participants in this debate. To remedy this problem, I suggest focusing on the distinction between the internal and the external validity of an experiment, which is also crucial for thought experiments. I then develop an account of both kinds of validity in the context of thought experiments. (...) I show that we can naturally conceptualise internal validity in terms of games of make-believe. Then, I argue that external validity is best defined as accurate representation of a target system. Finally, I turn back to the current debate on thought experiments and show that my diarchic account provides a general framework that can be shared by the competing philosophical views, as well as a fruitful guide for their reconciliation. (shrink)

Explainable artificial intelligence (XAI) is an emerging, multidisciplinary field of research that seeks to develop methods and tools for making AI systems more explainable or interpretable. XAI researchers increasingly recognise explainability as a context-, audience- and purpose-sensitive phenomenon, rather than a single well-defined property that can be directly measured and optimised. However, since there is currently no overarching definition of explainability, this poses a risk of miscommunication between the many different researchers within this multidisciplinary space. This is the problem we (...) seek to address in this paper. We outline a framework, called Explanatory Pragmatism, which we argue has two attractive features. First, it allows us to conceptualise explainability in explicitly context-, audience- and purpose-relative terms, while retaining a unified underlying definition of explainability. Second, it makes visible any normative disagreements that may underpin conflicting claims about explainability regarding the purposes for which explanations are sought. Third, it allows us to distinguish several dimensions of AI explainability. We illustrate this framework by applying it to a case study involving a machine learning model for predicting whether patients suffering disorders of consciousness were likely to recover consciousness. (shrink)

This paper addresses the problem of how scientific knowledge, which is always locally generated, becomes accepted in other sites. The analysis suggests that there are a small number of strategies that enable scientists to resituate knowledge and that these strategies are generic: they are not restricted to specific disciplines or modes of doing science but rather are found in a variety of different forms across the sciences.

This article explores the characteristics of research sites that scientists have called “natural experiments” to understand and develop usable distinctions for the social sciences between “Nature’s or Society’s experiments” and “natural experiments.” In this analysis, natural experiments emerge as the retro-fitting by social scientists of events that have happened in the social world into the traditional forms of field or randomized trial experiments. By contrast, “Society’s experiments” figure as events in the world that happen in circumstances that are already sufficiently (...) “controlled” to be open for direct analysis without reconstruction work. (shrink)

It has recently been argued that successful evidence-based policy should rely on two kinds of evidence: statistical and mechanistic. The former is held to be evidence that a policy brings about the desired outcome, and the latter concerns how it does so. Although agreeing with the spirit of this proposal, we argue that the underlying conception of mechanistic evidence as evidence that is different in kind from correlational, difference-making or statistical evidence, does not correctly capture the role that information about (...) mechanisms should play in evidence-based policy. We offer an alternative account of mechanistic evidence as information concerning the causal pathway connecting the policy intervention to its outcome. Not only can this be analyzed as evidence of difference-making, it is also to be found at any level and is obtainable by a broad range of methods, both experimental and observational. Using behavioral policy as an illustration, we draw the implications of this revised understanding of mechanistic evidence for debates concerning policy extrapolation, evidence hierarchies, and evidence integration. (shrink)

Deidealization as a topic in its own right has attracted remarkably little philosophical interest despite the extensive literature on idealization. One reason for this is the often implicit assumption that idealization and deidealization are, potentially at least, reversible processes. We question this assumption by analyzing the challenges of deidealization within a menu of four broad categories: deidealizing as recomposing, deidealizing as reformulating, deidealizing as concretizing, and deidealizing as situating. On closer inspection, models turn out much more inflexible than the reversal (...) thesis would have us believe, and deidealization emerges as a creative part of modeling. (shrink)

This paper argues that an important type of experiment-target inference, extrapolating causal effects, requires models to be successful. Focusing on extrapolation in Evidence-Based Policy, it is ar...

I consider recent strategies proposed by econometricians for extrapolating causal effects from experimental to target populations. I argue that these strategies fall prey to the extrapolator’s circle: they require so much knowledge about the target population that the causal effects to be extrapolated can be identified from information about the target alone. I then consider comparative process tracing as a potential remedy. Although specifically designed to evade the extrapolator’s circle, I argue that CPT is unlikely to facilitate extrapolation in typical (...) econometrics and evidence-based policy applications. To argue this, I offer a distinction between two kinds of extrapolation, attributive and predictive, the latter being prevalent in econometrics and evidence-based policy. I argue that CPT is not helpful for predictive extrapolation when using the kinds of evidence that econometricians and evidence-based policy researchers prefer. I suggest that econometricians may need to consider qualitative evidence to overcome this problem. (shrink)

In this paper a critique to philosophical approaches that presuppose invariant knowledge for policy purposes is carried out. It is shown that socioeconomic processes do not fit to the logic of stable causal factors, but they are more suited to the logic of "open-ended results". On the basis of this ontological variation it is argued that ex-ante interventions are not appropriate in the socioeconomic realm. On the contrary, they must be understood in a “dynamic” sense. Finally, derivational robustness analysis is (...) proposed as a useful tool for overcoming the problem of “overconstraint”, a typical problem of economic models. (shrink)

RESUMEN: En el presente trabajo se hará una crítica al enfoque de máquinas nomológicas desarrollado por Nancy Cartwright para el caso de las ciencias sociales. Se argumentará que los fenómenos sociales no responden a una lógica de "capacidades", sino de "árboles de posibilidades" o "resultados de final abierto". La estabilidad o invarianza, si bien es posible en el reino de lo social, parece ser más la excepción que la regla. Esto tiene consecuencias importantes para los propósitos de intervención y control. (...) De acuerdo con las corrientes manipulabilistas tradicionales la invarianza es una herramienta fundamental para cumplir cualquier propósito intervencionista, ya que permite prever con un buen grado de confianza el resultado de una manipulación. Sin embargo, en este trabajo se intentará mostrar que, a pesar de que no sea del todo plausible obtener un conocimiento invariante en el reino de lo social, las intervenciones pueden ser igualmente eficaces. Para ello, será fundamental reemplazar la noción "ex-ante" de intervención por una que involucre continuidad o sistematicidad. También será clave tener en cuenta aquellas cláusulas o condiciones especificadas dentro de los modelos, ya que estos últimos cumplen el rol de ser anteproyectos para la generación de resultados. ABSTRACT: In the present paper, a critique of Nancy Cartwright's nomological machines approach in the social sciences will be made. It is argued that social phenomena do not correspond to the logic of "capacities", but to the logic of "possibility trees" or "open-ended results". Stability or invariance, although possible in the social realm, seem to be more the exception than the rule. This has important implications for the purposes of intervention and control. According to traditional manipulabilistic accounts, invariance is essential for achieving any interventionist purpose, because it allows predicting the result of a manipulation with a high degree of confidence. However, in this paper it will be shown that even though the obtaining of invariant knowledge is not entirely plausible in the social realm, interventions may be effective. To do this, it will be necessary to replace the "ex-ante" notion of intervention with another notion that involves continuity. It will also be crucial to take into account the clauses or conditions specified in models, since such models play the role of being blueprints for generating results. (shrink)

This brief note aims to reply to a comment made by Menno Rol abou...

Resumen: En el presente trabajo se examinarán las distintas clases de conocimientos que se requieren para una correcta implementación de política. Cartwright y Hardie han criticado el enfoque de la política basada en la evidencia por estar fundamentado en inferencias inductivas, y han propuesto un enfoque alternativo donde, por un lado, el principio causal o variable política se abstraiga lo máximo posible como para superar cualquier problema de validez externa, y por el otro, que este principio sea complementado con un (...) acervo de factores coadyuvantes. Si bien este enfoque proporciona una buena base para el armado de políticas, omite un tema central: las consecuencias alternativas que se derivan de una política. Más precisamente, en el enfoque de Cartwright y Hardie se apunta a recolectar información a fin de armar una política que funcione aquí, pero no dice nada si la política se desvía de su curso deseado. El problema de esto emerge cuando los desvíos conllevan consecuencias trascendentalmente negativas. Se argumentará que el conocimiento de las consecuencias alternativas de una política es tan importante como el conocimiento de factores centrales y de factores coadyuvantes. La toma de decisión de un hacedor de política no dependerá solo entonces de las chances de éxito de una política, sino también de la asimetría de sus resultados.: The present paper examines the different kinds of knowledge required for correct policy implementation. Cartwright and Hardie have criticized the evidence-based policy approach for being grounded on inductive inferences. They propose an alternative approach where, on the one hand, the causal principle or policy variable is abstracted as much as possible in order to overcome external validity problems, and, on the other hand, this principle is complemented with a set of supporting factors which refer to very specific aspects of a target system. While this approach provides a good basis for formulating a policy, it omits a central issue: the alternative consequences that derive from a policy. More precisely, Cartwright and Hardie’s approach aims to collect information in order to assemble a policy that works here. However, it says nothing about the cases where a policy deviates from its desired path. The problem arises when such deviations lead to transcendentally negative consequences. It will be argued that knowledge of a policy’s alternative consequences are just as important as the knowledge of central and support factors. A policy-maker’s decision will then depend not only on a policy’s chances of success, but also on the asymmetry of its results. (shrink)

Proponents of evidence-based medicine and some philosophers of science seem to agree that knowledge of mechanisms can help solve the problem of applying results of controlled studies to target populations (‘the problem of extrapolation’). We describe the problem of extrapolation, characterize mechanisms, and outline how mechanistic knowledge might be used to solve the problem. Our main thesis is that there are four often overlooked problems with using mechanistic knowledge to solve the problem of extrapolation. First, our understanding of mechanisms is (...) often (and arguably, likely to remain) incomplete. Secondly, knowledge of mechanisms is not always applicable outside the tightly controlled laboratory conditions in which it is gained. Thirdly, mechanisms can behave paradoxically. Fourthly, as Daniel Steel points out, using mechanistic knowledge faces the problem of the ‘extrapolator’s circle’. At the same time, when the problems with mechanistic knowledge have been addressed, such knowledge can and should be used to mitigate (nothing can entirely solve) the problem of extrapolation. (shrink)

This paper examines the scientific controversy over the yields of genetically modified [GM] crops as a case study in epistemologically deep disagreements. Appeals to “the evidence” are inadequate to resolve such disagreements; not because the interlocutors have radically different metaphysical views (as in cases of incommensurability), but instead because they assume rival epistemological frameworks and so have incompatible views about what kinds of research methods and claims count as evidence. Specifically, I show that, in the yield debate, proponents and opponents (...) of GM crops cite two different sets of claims as evidence, which correspond to two rival epistemological frameworks, classical experimental epistemology and Nancy Cartwright's evidence for use. I go on to argue that, even if both sides of the debate accepted Cartwright's view, they might still disagree over what counts as evidence, because evidence for use ties standards of evidence to what is sometimes called the “context of application.”. (shrink)

Cognitive values like simplicity, broad scope, and easy handling are properties of a scientific representation that result from the idealization which is involved in the construction of a representation. These properties may facilitate the application of epistemic values to credibility assessments, which provides a rationale for assigning an auxiliary function to cognitive values. In this paper, I defend a further rationale for cognitive values which consists in the assessment of the usefulness of a representation. Usefulness includes the relevance of a (...) representation regarding the investigation of a given problem and its practicability for the users. This rationale builds on the claim that any evaluation of scientific representations should pursue two aims: providing information about their credibility and providing information about their usefulness. Cognitive values relating to the usefulness of a representation and epistemic values relating to its credibility both perform a first-order function. Cognitive values are abstract, and several values with first-order functions may conflict in their application. Thus, in order for cognitive values to account for the sort of problem that is to be investigated by means of a representation, they need to be appropriately specified and weighed. Comprehensiveness, complexity, high resolution, and easy handling, for instance, may be required in a first-order function for model-based prediction of regional climate impacts but not for explaining how the global climate system works. Specifying and weighing cognitive and epistemic values relative to a given problem is a legitimate second-order function of social values. (shrink)

:Proponents of behavioural policies seek to justify them as ‘evidence-based’. Yet they typically fail to show through which mechanisms these policies operate. This paper shows – at the hand of examples from economics and psychology – that without sufficient mechanistic evidence, one often cannot determine whether a given policy in its target environment will be effective, robust, persistent or welfare-improving. Because these properties are important for justification, policies that lack sufficient support from mechanistic evidence should not be called ‘evidence-based’.

:In this paper, we analyse the difference between two types of behavioural policies – nudges and boosts. We distinguish them on the basis of the mechanisms through which they are expected to operate and identify the contextual conditions that are necessary for each policy to be successful. Our framework helps judging which type of policy is more likely to bring about the intended behavioural outcome in a given situation.

We track and analyze the re-situation of scientific knowledge in the field of human population genomics ancestry studies. We understand re-situation as a process of accommodating the direct or indirect transfer of objects of knowledge from one site/situation to other sites/situations. Our take on the concept borrows from Mary S. Morgan’s work on facts traveling while expanding it to include other objects of knowledge such as models, data, software, findings, and visualizations. We structure a specific case study by tracking the (...) re-situation of these objects between three research projects studying human population diversity reported in three articles in Science, Genome Research and PLoS Genetics between 2002 and 2005. We characterize these three engagements as a unit of analysis, a “skirmish,” in order to compare: the divergence of interests in how life-scientists answer similar research questions and to track the challenging transformation of workflows in research laboratories as these scientific objects are re-situated individually or in bundles. Our analysis of the case study shows that an accurate understanding of re-situation requires tracking the whole bundle of objects in a project because they interact in particular key ways. The absence or dismissal of these interactions opens the door to unforeseen trade-offs, misunderstandings and misrepresentations about research design and workflow and what these say about the questions asked and the findings produced. (shrink)

In this paper we identify six theses that constitute core results of philosophical investigation into the nature of mechanisms, and of the role that the search for and identification of mechanisms play in the sciences. These theses represent the fruits of the body of research that is now often called New Mechanism. We concisely present the main arguments for these theses. In the literature, these arguments are scattered and often implicit. Our analysis can guide future research in many ways: it (...) provides critics of New Mechanism with clear targets, it can reduce misunderstandings, it can clarify differences of opinion among New Mechanists and it helps to define a research agenda for New Mechanists. (shrink)

With the ascent of modern epidemiology in the Twentieth Century came a new standard model of prediction in public health and clinical medicine. In this article, we describe the structure of the model. The standard model uses epidemiological measures-most commonly, risk measures-to predict outcomes (prognosis) and effect sizes (treatment) in a patient population that can then be transformed into probabilities for individual patients. In the first step, a risk measure in a study population is generalized or extrapolated to a target (...) population. In the second step, the risk measure is particularized or transformed to yield probabilistic information relevant to a patient from the target population. Hence, we call the approach the Risk Generalization-Particularization (Risk GP) Model. There are serious problems at both stages, especially with the extent to which the required assumptions will hold and the extent to which we have evidence for the assumptions. Given that there are other models of prediction that use different assumptions, we should not inflexibly commit ourselves to one standard model. Instead, model pluralism should be standard in medical prediction. (shrink)

Simple extrapolation is the orthodox approach to extrapolating from clinical trials in evidence-based medicine: extrapolate the relative effect size from the trial unless there is a compelling reason not to do so. I argue that this method relies on a myth and a fallacy. The myth of simple extrapolation is the idea that the relative risk is a ‘golden ratio’ that is usually transportable due to some special mathematical or theoretical property. The fallacy of simple extrapolation is an unjustified argument (...) from ignorance: we conclude that the relative effect size is transportable in the absence of evidence to the contrary. In short, simple extrapolation is a deeply problematic solution to the problem of extrapolation. (shrink)

It is sometimes thought that randomized study group allocation is uniquely proficient at producing comparison groups that are evenly balanced for all confounding causes. Philosophers have argued that in real randomized controlled trials this balance assumption typically fails. But is the balance assumption an important ideal? I run a thought experiment, the CONFOUND study, to answer this question. I then suggest a new account of causal inference in ideal and real comparative group studies that helps clarify the roles of confounding (...) variables and randomization. 1Confounders and Causes2The Balance Assumption3The CONFOUND Study 3.1CONFOUND 13.2CONFOUND 24Disjunction C and the Ideal Study 4.1The ultimate ‘other cause’: C4.2The ideal comparative group study4.3Required conditions for causal inference5Confounders as Causes, Confounders as Correlates6Summary. (shrink)

How should we understand scientific progress? Kuhn famously discussed science as its own internally driven venture, structured by paradigms. He also famously had a problem describing progress in science, as problem-solving ability failed to provide a clear rubric across paradigm change—paradigm changes tossed out problems as well as solving them. I argue here that much of Kuhn’s inability to articulate a clear view of scientific progress stems from his focus on pure science and a neglect of applied science. I trace (...) the history of the distinction between pure and applied science, showing how the distinction came about, the rhetorical uses to which the distinction has been put, and how pure science came to be both more valued by scientists and philosophers. I argue that the distinction between pure and applied science does not stand up to philosophical scrutiny, and that once we relinquish it, we can provide Kuhn with a clear sense of scientific progress. It is not one, though, that will ultimately prove acceptable. For that, societal evaluations of scientific work are needed. (shrink)

Las corrientes manipulabilistas tradicionales consideran que una implementación de política efectiva debe estar fundamentada en un conocimiento invariante. Este modo de pensamiento está basado en el supuesto ontológico de que existen factores causales estables tanto en las ciencias naturales como en las sociales. También asumen que un sólo tipo de intervención ex-ante es pertinente para una implementación efectiva. En el presente artículo se realizará una crítica a estos enfoques. En particular, se mostrará que los procesos sociales responden a una lógica (...) de “árboles de posibilidades” o “resultados de final abierto”. Sobre la base de esta variante ontológica se argumentará que más que un conocimiento de factores invariantes, el sustento de una buena política debe buscarse en los condicionamientos explicitados dentro de un anteproyecto y en la factibilidad de que los mismos se satisfagan en el sistema objetivo. Asimismo, se mostrará cómo las típicas intervenciones ex-ante no son adecuadas en el reino de lo social. Contrario a ellos, las intervenciones deberán ser sistemáticas. (shrink)