A critique on “A randomized evaluation of on-site monitoring nested in a multinational randomized trial”

We read with interest the study entitled “A randomized evaluation of on-site monitoring nested in a multinational randomized trial,” which has potential to guide future on-site clinical trial monitoring. ¹ Wyman Engan et al. ¹ evaluated monitoring strategies by nesting a cluster randomized study within the Strategic Timing of AntiRetroviral Treatment (START) trial. ² START trial was an international, open-label randomized trial comparing immediate and deferred anti-retroviral therapy (ART). ² The nested study randomized 196 sites to either on-site monitoring in addition to central and local monitoring (i.e. “on-site group”) or central and local monitoring (i.e. “no on-site group”), and the primary outcome was a participant-level composite outcome (including eligibility violations, informed consent violations, use of ART not permitted by protocol, late reporting of START primary and secondary endpoints, and data alternation).

Using intention-to-treat (ITT) analysis was a strength in the study. All the randomized monitoring sites were included in the data analysis, even if data for that site was incomplete, preserving the groups generated through randomization. Another strength was comparing the study to other monitoring studies: the Optimisation of Monitoring for Clinical Research Studies (OPTIMON), the ADApted MONitoring study (ADAMON), and the TargetEd Monitoring: Prospective Evaluation and Refinement (TEMPER) trials. OPTIMON concluded that the risk-adapted strategy was inferior to on-site monitoring. Similarly, the ADAMON trial showed a small significant difference between extensive on-site monitoring and risk-based monitoring, whereas TEMPER concluded that triggered monitoring did not significantly distinguish sites with monitoring findings.

The results in the authors’ Table 3 showed a significantly greater number of participants in the on-site group (n = 134) with a primary composite outcome compared to the no on-site group (n = 85), with an odds ratio (OR) of 1.7 (95% confidence interval (CI): 1.1–2.7; p = 0.03). However, only two components of the primary endpoint reached statistical significance. Eligibility violations had a very high OR in favor of the on-site group and had a large impact on the overall result, despite a wide CI (12.2, 95% CI: 1.8–85.2; p = 0.01). START serious events reported more than 6 months from occurrence were also greater on the on-site group (2.0, 95% CI: 1.1–3.7; p = 0.02). The other four components did not differ between the two groups. Despite these positive results, the authors conclude that the addition of on-site monitoring would be unlikely to influence the results of START and thus is not recommended when considering the overall costs. We suggest that future trials should be designed to assess the efficiency of various monitoring strategies using a formal prespecified framework with a non-inferiority boundary such as the one reported by Freemantle et al. ³ before widespread adoption of remote monitoring. Further appropriately designed trials are needed to reach clinically convincing conclusions.