Rationale, Aims, and Objectives: Confidence (or belief) that a therapy is effective is essential to practicing clinical medicine. GRADE, a popular framework for developing clinical recommendations, provides a means for assigning how much confidence one should have in a therapy's effect estimate. One's level of confidence (or “degree of belief”) can also be modelled using Bayes theorem. In this paper, we look through both a GRADE and Bayesian lens to examine how one determines confidence in the effect estimate. Methods: Philosophical (...) examination. Results: The GRADE framework uses a criteria‐based method to assign a quality of evidence level. The criteria pertain mostly to considerations of methodological rigour, derived from a modified evidence‐based medicine evidence hierarchy. The four levels of quality relate to the level of confidence one should have in the effect estimate. The Bayesian framework is not bound by a predetermined set of criteria. Bayes theorem shows how a rational agent adjusts confidence (ie, degree of belief) in the effect estimate on the basis of the available evidence. Such adjustments relate to the principles of incremental confirmation and evidence proportionism. Use of the Bayesian framework reveals some potential pitfalls in GRADE's criteria‐based thinking on confidence that are out of step with our intuitions on evidence. Conclusions: A rational thinker uses all available evidence to formulate beliefs. The GRADE criteria seem to suggest that we discard some of that information when other, more favoured information (eg, derived from clinical trials) is available. The GRADE framework should strive to ensure that the whole evidence base is considered when determining confidence in the effect estimate. The incremental value of such evidence on determining confidence in the effect estimate should be assigned in a manner that is theoretically or empirically justified, such that confidence is proportional to the evidence, both for and against it. (shrink)

Translating risk estimates derived from epidemiologic study into evidence of causality for a particular patient is problematic. The difficulty of this process is not unique to the medical context; rather, courts are also challenged with the task of using risk estimates to infer evidence of cause in particular cases. Thus, an examination of how this is done in a legal context might provide insight into when and how it is appropriate to use risk information as evidence of cause in a (...) medical context. A careful study of the case of Goodman v. Viljoen, a medical malpractice suit litigated in the Ontario Superior Court of Justice in 2011, reveals different approaches to how risk information is used as or might be considered a substitute for evidence of causation, and the pitfalls associated with these approaches. Achieving statistical thresholds, specifically minimizing the probability of falsely rejecting the null hypothesis, and exceeding a relative risk of 2, plays a significant role in establishing causality of the particular in the legal setting. However, providing a reasonable explanation or establishing “biological plausibility” of the causal association also seems important, and (to some) may even take precedent over statistical thresholds for a given context. (shrink)

The evidence based medicine movement has championed the need for objective and transparent methods of clinical guideline development. The Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) framework was developed for that purpose. Central to this framework is criteria for assessing the quality of evidence from clinical studies and the impact that body of evidence should have on our confidence in the clinical effectiveness of a therapy under examination. Grades of Recommendation, Assessment, Development, and Evaluation has been adopted by a (...) number of professional medical societies and organizations as a means for orienting the development of clinical guidelines. As a result, the method of GRADE has implications on how health care is delivered and patient outcomes. In this paper, we reveal several issues with the underlying logic of GRADE that warrant further discussion. First, the definitions of the “grades of evidence” provided by GRADE, while explicit, are functionally vague. Second, the “criteria for assigning grade of evidence” is seemingly arbitrary and arguably logically incoherent. Finally, the GRADE method is unclear on how to integrate evidence grades with other important factors, such as patient preferences, and trade‐offs between costs, benefits, and harms when proposing a clinical practice recommendation. Much of the GRADE method requires judgement on the part of the user, making it unclear as to how the framework reduces bias in recommendations or makes them more transparent—both goals of the programme. It is our view that the issues presented in this paper undermine GRADE's justificatory scheme, thereby limiting the usefulness of GRADE as a tool for developing clinical recommendations. (shrink)