The explanatory/pragmatic-trial distinction enjoys a burgeoning philosophical and medical literature and a significant contingent of support among philosophers and healthcare stakeholders as an important way to assess the design and results of randomized controlled trials. A major motivation has been the need to provide relevant, generalizable data to drive healthcare decisions. While talk of pragmatic and explanatory trials could be seen as convenient shorthand, the distinction can also be seen as harboring deeper issues related to inferential strategies used to evaluate (...) causal claims regarding medical treatments. A comprehensive, critical analysis of the distinction and underlying epistemological framework upon which the distinction is based, particularly with respect to treatment effectiveness, has yet to be forthcoming. I provide this, analyzing the distinction’s relationship to generalizability and cognate distinctions between ideal conditions and real-world practice, internal and external validity, and efficacy and effectiveness. I also analyze recent philosophical work that relies on the explanatory/pragmatic-trial distinction and that advocates for more pragmatic trials. I conclude that as an organizing principle for trial-design decisions and trial evaluation, the explanatory/pragmatic-trial distinction is conceptually problematic and not as useful as its proponents seem to think. Since some pragmatic-trial features can be inimical to establishing treatment effectiveness, pragmatic-trial features should not be conflated with pragmatic trials’ avowed goals. If the distinction is to be useful, it and some associated concepts, including generalizability, should be reformulated, lest they continue to underlie a medical epistemology that could contribute to methodologically flawed and potentially unethical advice for the design and interpretation of trials. (shrink)

External validity is typically regarded as the downside of case study research by methodologists and social scientists; case studies, however, are often aimed at drawing lessons that are generalizable to new contexts. The gap between the generalizability potential of case studies and the research goals demands closer scrutiny. I suggest that the conclusion that case study research is weak in external validity follows from a set of assumptions that I term the "traditional view," which are disputable at best. In this (...) view, external validity is treated as a matter of mere representativeness. I argue that it is best understood instead as a problem of inference and that the emphasis should be placed on the comparability of the study rather than on the typicality of the case. By making case studies highly comparable, their external validity can be reliably and efficiently assessed and, in this way, their generalizability potential enhanced. (shrink)

A shared problem across the sciences is to make sense of correlational data coming from observations and/or from experiments. Arguably, this means establishing when correlations are causal and when they are not. This is an old problem in philosophy. This paper, narrowing down the scope to quantitative causal analysis in social science, reformulates the problem in terms of the validity of statistical models. Two strategies to make sense of correlational data are presented: first, a 'structural strategy', the goal of which (...) is to model and test causal structures that explain correlational data; second, a 'manipulationist or interventionist strategy', that hinges upon the notion of invariance under intervention. It is argued that while the former can offer a solution the latter cannot. (shrink)

Mentalist view began to lose its standing among psychologists mainly during the first half of the twentieth century. As a result, the enthusiasm to build an objective science began to grow among behaviourists and ethologists. The rise of cognitive sciences around the 1960s, however, revived the debates over the importance of cognitive intervening variables in explaining behaviours that could not be explained by clinging solely to a pure behavioural approach. Nevertheless, even though cognitive functions in nonhuman animals have been identified (...) in many studies, attributing human mental properties to animals is still being criticized as anthropomorphism. Such anthropomorphic attributions have been considered as an open door to the return of subjective methodology. Representation of anthropomorphic attributions as scientific activities within the context of discovery as opposed to the context of justification casts a new light on this problem. The present analysis proposes that anthropomorphic attributions are formed based on top-down idiosyncratic intuitions on the cusp of hypothesis building, outside the context of scientific justification. In other words, an anthropomorphic attribution is a potential creative link between pure observations of behaviours and building testable hypotheses about cognition in the process of scientific discovery. Thus, serving a function within the context of discovery, anthropomorphism does not motivate the return of subjective methodology, simply because it is not inconsistent with the rule-governed path in the context of scientific justification. (shrink)

Over the past seven decades, randomised controlled trials (RCTs) have revolutionised clinical research and achieved a gold standard status. However, extending their use to evaluate complex interventions is problematic. In this paper we will demonstrate that complex intervention RCTs violate the necessary premises that govern the RCTs logic and underpin their rigour. The lack of blinding, heterogeneity of participants, as well as poor treatment standardisation and difficulty of controlling for confounders, which characterise complex intervention RCTs, can potentially be profoundly detrimental (...) to their integrity. Proponents of this approach argue that matching “real world” circumstances, while maintaining the randomised design, enhances external validity. We counter this argument by pointing out that an inverted U relation exists between internal and external validity, and thus relaxing the experimental conditions beyond a certain threshold can potentially paradoxically render the RCT externally invalid, i.e. its results cannot be used anywhere. We shall illustrate the inappropriate use of RCTs to evaluate community treatment orders and propose an alternative epistemic model that is based on mechanistic reasoning and Cartwright's capacity concept. (shrink)

Although research on corporate social responsibility has grown steadily, little research has focused on CSR at the individual level. In addition, research on the role of environmental friendly organizational citizenship behaviors within CSR initiatives is scarce. In response to this gap and recent calls for further research on both individual and organizational variables of employees’ environmentally friendly, or green, behaviors, this article sheds light on the influence of these variables on three types of green employee behaviors simultaneously: recycling, energy savings, (...) and printing reduction. An initial theoretical model identifies both individual and organizational variables that affect different types of green behaviors as a stepping stone for further research. The results reveal managerial implications and future research directions on the design of effective social marketing interventions that motivate different types of OCBs in the workplace. In particular, the results suggest that creating separate interventions for each type of environmental behavior, as well as for each organization, sector, and type of organization , is necessary. In addition, this research illustrates patterns of attitudes, perceptions, and behaviors by exploring individual and organizational variables and behaviors across seven different organizations belonging to different sectors. (shrink)

In the biomedical, behavioural and social sciences, the leading method used to estimate causal effects is commonly randomised controlled trials (RCTs) that are generally viewed as both the source and justification of the most valid evidence. In studying the foundation and theory behind RCTs, the existing literature analyses important single issues and biases in isolation that influence causal outcomes in trials (such as randomisation, statistical probabilities and placebos). The common account of biased causal inference is described in a general way (...) in terms of probabilistic imbalances between trial groups. This paper expands the common account of causal bias by distinguishing between the range of biases arising between trial groups but also within one of the groups or across the entire sample during trial design, implementation and analysis. This is done by providing concrete examples from highly influential RCT studies. In going beyond the existing RCT literature, the paper provides a broader, practice-based account of causal bias that specifies the between-group, within-group and across-group biases that affect the estimated causal results of trials – impacting both the effect size and statistical significance. Within this expanded framework, we can better identify the range of different types of biases we face in practice and address the central question about the overall validity of the RCT method and its causal claims. A study can face several smaller biases (related simultaneously to a smaller sample, smaller estimated effect, greater unblinding etc.) that generally add up to greater aggregate bias. Though difficult to measure precisely, it is important to assess and provide information in studies on how much different sources of bias, combined, can explain the estimated causal effect. The RCT method is thereby often the best we have to inform our policy decisions – and the evidence is strengthened when combined with multiple studies and other methods. Yet there is room for continually improving trials and identifying ways to reduce biases they face and to increase their overall validity. Implications are discussed. (shrink)

Evidence-Based Medicine has little consideration for mechanisms and philosophers of science and medicine have recently made pleas to increase the place of mechanisms in the medical evidence hierarchy. However, in this debate the notions of mechanisms seem to be limited to 'mechanistic processes' and 'complex-systems mechanisms,' understood as 'componential causal systems'. I believe that this will not do full justice to how mechanisms are used in biological, psychological and social sciences and, consequently, in a more biopsychosocial approach to medicine. Here, (...) I propose, following (Kuorikoski, 2009), to pay more attention to 'abstract forms of interaction' mechanisms. The present work scrutinized review articles on depression and medically unexplained pain, which are considered to be of multifactorial pathogenesis, for their use of mechanisms. In review articles on these disorders there seemed to be a range of uses between more 'abstract forms of interaction' and 'componential causal system' mechanisms. I therefore propose to expand the notions of mechanisms considered in medicine to include that of more 'abstract forms of interaction' to better explain and manage biopsychosocial disorders. (shrink)

A series of recent debates in experimental economics have associated demand effects with the artificiality of the experimental setting and have linked it to the problem of external validity. In this paper, we argue that these associations can be misleading, partly because of the ambiguity with which “artificiality” has been defined, but also because demand effects and external validity are related in complex ways. We argue that artificiality may be directly as well as inversely correlated with demand effects. We also (...) distinguish between the demand effects of experimentation and the reactions that they may trigger and that might endanger experimental validity. We conclude that economists should pay more attention to the way in which subjects construe the experimental task and learn to exploit subjects’ reactivity to expectations in their experiments. (shrink)

In this paper a critique to philosophical approaches that presuppose invariant knowledge for policy purposes is carried out. It is shown that socioeconomic processes do not fit to the logic of stable causal factors, but they are more suited to the logic of "open-ended results". On the basis of this ontological variation it is argued that ex-ante interventions are not appropriate in the socioeconomic realm. On the contrary, they must be understood in a “dynamic” sense. Finally, derivational robustness analysis is (...) proposed as a useful tool for overcoming the problem of “overconstraint”, a typical problem of economic models. (shrink)

While the term “reactivity” has come to be associated with specific phenomena in the social sciences, having to do with subjects’ awareness of being studied, this paper takes a broader stance on this concept. I argue that reactivity is a ubiquitous feature of the psychological subject matter and that this fact is a precondition of experimental research, while also posing potential problems for the experimenter. The latter are connected to the worry about distorted data and experimental artifacts. But what are (...) experimental artifacts and what is the most productive way of dealing with them? In this paper, I approach these questions by exploring the ways in which experimenters in psychology simultaneously exploit and suppress the reactivity of their subject matter in order to produce experimental data that speak to the question or subject matter at hand. Highlighting the artificiality of experimental data, I raise the question of what distinguishes a genuine experimental result from an experimental artifact. My analysis construes experimental results as the outcomes of inferences from the data that take material background assumptions as auxiliary premises. Artifacts occur when one or more of these background assumptions are false, such that the data do not reliably serve the purposes they were generated for. I conclude by laying out the ways in which my analysis of data quality is relevant to, and informed by, recent debates about the replicability of experimental results. (shrink)

Can an ugly experiment be a good experiment? Philosophers have identified many beautiful experiments and explored ways in which their beauty might be connected to their epistemic value. In contrast, the present chapter seeks out (and celebrates) ugly experiments. Among the ugliest are those being designed by AI algorithms. Interestingly, in the contexts where such experiments tend to be deployed, low aesthetic value correlates with high epistemic value. In other words, ugly experiments can be good. Given this, we should conclude (...) that beauty is not generally necessary or sufficient for epistemic value, and increasing beauty will not generally tend to increase epistemic value. (shrink)