The Potential for a Benefit-Estimating Algorithm to Improve Recommendations for Preventive Services: Comparing Algorithm Recommendations with Those of Primary Care Providers

Abstract

Purpose

To compare an established benefit-estimating algorithm for recommending and prioritizing preventive services for a patient (Individualized Precision Prevention; IPP) with concordant rankings from primary care providers (PCPs).

Methods

We developed 12 realistic routine patient care scenarios focused on preventive services and recruited 40 PCPs to rank the priority of recommended preventive services. Our analysis compared the benefit-estimating algorithm’s rankings of preventive services for each of the 12 patient scenarios to the PCPs’ rankings using length-dependent rank-biased overlap (LDRBO) calculations. Moderate concordance would suggest that the computer algorithm presented an opportunity to improve preventive care, whereas very high or low concordance would call into question what the algorithm could contribute to clinical practice.

Results

For all 12 patient care scenarios, comparing the benefit-estimating algorithm’s output to the combined priority rankings from all PCPs yields a mean value of 0.45, corresponding to a moderate level of concordance or agreement between the numeric rankings of the algorithm and the expert provider rankings. This study illustrates the potential importance of having computed IPP recommendations readily available for point-of-care decision making by PCPs.

Conclusion

We demonstrate that this approach aligned with the overall judgment of clinical experts and may help providers prioritize preventive services in time-constrained clinical contexts. The modest correlation between the benefit-estimating algorithm and expert providers suggests that, in some cases, the algorithm has the potential to provide useful advice about preventive services during care.

Highlights

• Using scenarios, we compared how primary care providers and an algorithm prioritized recommendations for preventive services based on individual information about a patient.

• The providers’ rankings of the clinical importance of preventive services were moderately concordant with rankings produced by the algorithm, suggesting that the algorithm presents an opportunity to improve the effects of preventive care.

• For half of the scenarios, the algorithm recommended one preventive service that was not in the PCPs’ consensus top 3, suggesting that the algorithm may raise provider awareness of services that may be beneficial to specific patients.

• An algorithm-driven approach to individualized precision prevention that uses a patient’s data to generate personalized recommendations of preventive services can help providers and patients identify and prioritize high-priority preventive services together.