The problem of evaluating automated large-scale evidence aggregators

Synthese (8):3083-3102 (2019)
Download Edit this record How to cite View on PhilPapers
In the biomedical context, policy makers face a large amount of potentially discordant evidence from different sources. This prompts the question of how this evidence should be aggregated in the interests of best-informed policy recommendations. The starting point of our discussion is Hunter and Williams’ recent work on an automated aggregation method for medical evidence. Our negative claim is that it is far from clear what the relevant criteria for evaluating an evidence aggregator of this sort are. What is the appropriate balance between explicitly coded algorithms and implicit reasoning involved, for instance, in the packaging of input evidence? In short: What is the optimal degree of ‘automation’? On the positive side: We propose the ability to perform an adequate robustness analysis as the focal criterion, primarily because it directs efforts to what is most important, namely, the structure of the algorithm and the appropriate extent of automation. Moreover, where there are resource constraints on the aggregation process, one must also consider what balance between volume of evidence and accuracy in the treatment of individual evidence best facilitates inference. There is no prerogative to aggregate the total evidence available if this would in fact reduce overall accuracy.
Reprint years
2017, 2019
PhilPapers/Archive ID
Revision history
Archival date: 2018-01-05
View upload history
References found in this work BETA
Mechanisms and the Evidence Hierarchy.Clarke, Brendan; Gillies, Donald; Illari, Phyllis; Russo, Federica & Williamson, Jon
Interpreting Causality in the Health Sciences.Russo, Federica & Williamson, Jon
Economic Modelling as Robustness Analysis.Kuorikoski, Jaakko; Lehtinen, Aki & Marchionni, Caterina

View all 15 references / Add more references

Citations of this work BETA

No citations found.

Add more citations

Added to PP index

Total views
191 ( #22,451 of 50,373 )

Recent downloads (6 months)
36 ( #16,948 of 50,373 )

How can I increase my downloads?

Downloads since first upload
This graph includes both downloads from PhilArchive and clicks to external links.