Avoiding verification bias in screening test evaluation in resource poor settings: a case study from Zimbabwe

Abstract

Background Verification bias occurs when the percentage of subjects receiving disease verification differs according to the test result. Statistical adjustment yields unbiased sensitivity and specificity under a missing at random (MAR) assumption.

Purpose To use an example from an international study to show how the assumptions needed for unbiased statistical adjustment for verification bias can be undermined by conditions on the ground, and that accuracy of estimates is also compromised by too low a sampling fraction of subjects who test negative.

Methods A study in Zimbabwe assessed the accuracy of a screening test for cervical cancer screening, visual inspection with acetic acid (VIA). The study was conducted in two phases, Phase I, where only 10% of subjects with negative tests received verification, and Phase II, in which nearly all subjects were verified. Unadjusted, simple- and covariate-adjusted estimates were compared to investigate factors affecting differences. Bootstrap simulations were used to illustrate the effect of varying test negative sampling fractions.

Results Phase I unadjusted sensitivity and specificity were 0.66 (0.61—0.70) and 0.34 (0.31—0.36), respectively. Simple-weighted adjusted estimators accounting only for VIA status were 0.20 (0.17—0.23) and 0.80 (0.78—0.81), respectively, suggesting the test to be useless. It was found that verification (colposcopy) capacity in-country had been exceeded, and that random selection of test negative patients for colposcopy had been compromised. Phase II estimates of sensitivity and specificity were 0.77 and 0.64, respectively. With 9% disease prevalence, a VIA test-negative sampling fraction >50% was necessary for the confidence intervals for sensitivity to have more than a 90% probability of including the true value. Limitations Phase I statistical adjustment was not made for MAR deviations unexplained by the two auxiliary factors, Pap results and STD history. Adjustment was not possible for other unmeasured co-factors.

Conclusions While there are standard formulae for correcting for verification bias, these will be biased if the MAR assumption is not met, which can occur through the actions of study personnel or subjects. Design of such studies in low resource environments needs to either require 100% verification, or employ procedures ensuring that the sample of test negatives who receive verification is indeed random. In addition, required test negative sampling fractions need to incorporate information on both disease prevalence and overall sample size. Clinical Trials 2008; 5: 496—503. http://ctj.sagepub.com