What kinds of evidence reliably support predictions of effectiveness for health and social care interventions? There is increasing reliance, not only for health care policy and practice but also for more general social and economic policy deliberation, on evidence that comes from studies whose basic logic is that of JS Mill's method of difference. These include randomized controlled trials, case–control studies, cohort studies, and some uses of causal Bayes nets and counterfactual-licensing models like ones commonly developed in econometrics. The topic (...) of this paper is the 'external validity' of causal conclusions from these kinds of studies. We shall argue two claims. Claim, negative: external validity is the wrong idea; claim, positive: 'capacities' are almost always the right idea, if there is a right idea to be had. If we are right about these claims, it makes big problems for policy decisions. Many advice guides for grading policy predictions give top grades to a proposed policy if it has two good Mill's-method-of difference studies that support it. But if capacities are to serve as the conduit for support from a method-of-difference study to an effectiveness prediction, much more evidence, and much different in kind, is required. We will illustrate the complexities involved with the case of multisystemic therapy, an internationally adopted intervention to try to diminish antisocial behaviour in young people. (shrink)

Patient-reported outcome measures (PROMs) are increasingly used to assess multiple facets of healthcare, including effectiveness, side effects of treatment, symptoms, health care needs, quality of care, and the evaluation of health care options. There are thousands of these measures and yet there is very little discussion of their theoretical underpinnings. In her 2008 Presidential address to the Society for Quality of Life Research (ISOQoL), Professor Donna Lamping challenged researchers to grapple with the theoretical issues that arise from these measures. In (...) this paper, I attempt to do so by arguing for an analogy between PROMs and Hans-Georg Gadamer’s logic of question and answer. While researchers readily admit that the constructs involved in PROMs are imperfectly understood and lack a gold standard, they often ignore the consequences of this fact. Gadamer’s work on questions and their importance to philosophical hermeneutics helps to show that the questions researchers ask about such constructs are also imperfectly understood. I argue that these questions should not be standardized, and I instead propose a theoretical framework that understands PROMs as posing genuine questions to respondents—questions that are open to reinterpretation. (shrink)

The evidence from randomized controlled trials (RCTs) is widely regarded as supplying the ‘gold standard’ in medicine—we may sometimes have to settle for other forms of evidence, but this is always epistemically second-best. But how well justified is the epistemic claim about the superiority of RCTs? This paper adds to my earlier (predominantly negative) analyses of the claims produced in favour of the idea that randomization plays a uniquely privileged epistemic role, by closely inspecting three related arguments from leading contributors (...) to the burgeoning field of probabilistic causality—Papineau, Cartwright and Pearl. It concludes that none of these further arguments supplies any practical reason for thinking of randomization as having unique epistemic power. IntroductionWhy the issue is of great practical importance—the ECMO casePapineau on the ‘virtues of randomization’Cartwright on causality and the ‘ideal’ randomized experimentPearl on randomization, nets and causesConclusion. (shrink)

Health-related quality of life measures aim to assess patients’ subjective experience in order to gauge an increasingly wide variety of health care issues such as patient needs; satisfaction; side effects; quality of care; disease progression and cost effectiveness. Their popularity is undoubtedly due to a larger initiative to provide patient-centered care. The use of patient perspectives to guide health care improvements and spending is rooted in the idea that we must respect patients as self-determining agents. In this paper I look (...) at the two main orientations to quality of life measurement: standardized and individualized measures. I argue that while these measures are attempts to provide for patient self-determination, they both fail to do so. In their place I suggest a new approach which overcomes their respective difficulties: a dialogic approach. (shrink)

Patient-reported outcomes are increasingly used as dependent variables in studies regarding the effectiveness of clinical interventions. But patient-reported outcome measures (PROMs) do not provide intuitively meaningful data. For instance, it is not clear what a five point increase or decrease on a particular scale signifies. Establishing ‘interpretability’ involves making changes in outcomes meaningful. Attempts to interpret PROMs have led to the development of methods for identifying a minimum important difference (MID). In this paper, however, I draw on Charles Taylor’s distinction (...) between weak and strong evaluations to suggest that identifying a MID, specifically, a MID that uses a patient-reported reference group, may not provide an adequate interpretation of these measures. Moreover, I argue that the difficulty with interpreting these measures is tied to a larger problem concerning their validity. If researchers wish to interpret PROMs, they may first need to know more about the constructs they attempt to measure, namely, quality of life. (shrink)

Robert Truog describes the controversial randomized controlled trials (RCTs) of extracorporeal membrane oxygenation (ECMO) therapy in newborns. Because early results with ECMO indicated that it might be a great advance, saving many lives, Truog argues that ECMO should not have been tested using RCTs, but that a long-term, large-scale observational study of actual clinical practice should have been conducted instead. Central to Truog’s argument, however, is the idea that ECMO is an unusual case. Thus, it is an open question whether (...) Truog’s conclusions can be extended to other areas of medical research. In this paper, I look at epistemological and ethical issues arising in the care of patients with chronic diseases, using ECMO as a starting point. Both the similarities and the dissimilarities of these two cases highlight important issues in biomedical research and support a conclusion similar to Truog’s. Observational studies of clinical practice provide the best evidence to inform the treatment of patients with chronic disease. (shrink)