Managing “Deep” Uncertainty in Global Health: The Case of TB Testing

Policymakers and clinicians in global health often face considerable uncertainty when making decisions. Statistical uncertainty—arising from the fact that analysis and modelling typically use samples rather than whole populations—can be accounted for by placing confidence intervals on the estimated impacts of different policies and treatment regimes. However, “deep” uncertainty—which may arise from issues including missing data or limited external validity from existing trials—poses a more fundamental challenge to decision-making. In a newly published article in The Proceedings of the National Academy of Sciences, we examine such “deep” uncertainty in the context of a new diagnostic test for Tuberculosis (TB), including discussions of diversification, and ask what a reasonable policy response might be for public health agencies combatting TB.

“Deep” uncertainty can create challenges when there is incomplete knowledge of the prevalence of a disease in patients with specific characteristics, due to underreporting, misdiagnosis, or limited granularity in available data. Similarly, there may be “deep” uncertainty about the effectiveness of testing or treatment for patients that differ from the populations studied in trials – e.g., if trials are conducted with otherwise healthy adults, but a decision must be taken about how to treat children who also have other health conditions.

TB remains a serious global health problem, and was the cause of 1.6 million deaths worldwide in 2017. In 2010, the World Health Organization endorsed a new, faster, and more accurate test for diagnosis—Xpert MTB/RIF (Cepheid, Sunnyvale, CA, USA). However, trials showed that introducing the test did not lead to reductions in TB-related morbidity or deaths.

To help understand why, we model how a clinician might decide whether to order tests for TB and whether to treat a patient for TB, with or without test results. We highlight the role of deep uncertainty about the prevalence of TB and the accuracy of different tests, for patients with different characteristics.

We first show how, for some patient populations, it may be optimal for the clinician to treat without testing, even after the new test is introduced. This may partly explain the muted impact of introducing a new, improved diagnostic on patient outcomes.

We then show that, given this “deep” uncertainty, a reasonable policy is for clinicians to pursue diversification. That is, we show that within groups of patients with the same characteristics, clinicians may want to randomly test and treat some fraction of patients but not others, in proportions that can be calculated from available data. Diversification has the immediate benefit of minimising the chance that clinicians make large errors; i.e., choose one testing or treatment regime when another turns out to be better. Over time, diversification also generates new evidence on the accuracy of new diagnostic tests and on treatment response, similar to the evidence produced by randomised controlled trials, but on a potentially much larger scale.

This article was originally published by the Center for Global Development.