Five-Year Cost-Effectiveness Modeling of Primary Care-Based,Nonmydriatic Automated Retinal Image Analysis Screening Among Low-Income Patients With Diabetes

Abstract

Background:

Artificial intelligence-based technology systems offer an alternative solution for diabetic retinopathy (DR) screening compared with standard, in-office dilated eye examinations. We performed a cost-effectiveness analysis of Automated Retinal Image Analysis System (ARIAS)-based DR screening in a primary care medicine clinic that serves a low-income patient population.

Methods:

A model-based, cost-effectiveness analysis of two DR screening systems was created utilizing data from a recent study comparing adherence rates to follow-up eye care among adults ages 18 or older with a clinical diagnosis of diabetes. In the study, the patients were prescreened with an ARIAS-based, nonmydriatic (undilated), point-of-care tool in the primary care setting and were compared with patients with diabetes who were referred for dilated retinal screening without prescreening, as is the current standard of care. Using a Markov model with microsimulation resulting in a total of 600 000 simulated patient experiences, we calculated the incremental cost-utility ratio (ICUR) of the two screening approaches, with regard to five-year cost-effectiveness of DR screening and treatment of vision-threatening DR.

Results:

At five years, ARIAS-based screening showed similar utility as the standard of care screening systems. However, ARIAS reduced costs by 23.3%, with an ICUR of $258 721.81 comparing the current practice to ARIAS.

Conclusions:

Primary care-based ARIAS DR screening is cost-effective when compared with standard of care screening methods.

Keywords

artificial intelligence cost-effectiveness analysis diabetic retinopathy healthcare economics machine learning technology public health

Introduction

Visual impairment secondary to diabetic retinopathy (DR) can be mitigated by regular screening eye examinations, which aids timely diagnosis and appropriate treatment.¹ For patients with DR, their disease course closely tracks their environment, access to healthcare, and economic resources. Improvements in screening and treatment of DR have led to an overall decrease in the prevalence of blindness caused by diabetic eye disease in recent years. However, these benefits are primarily observed in high-resource settings.² In the United States, rates of screening for DR in high-resource settings can be as high as 60%,³ while some underserved patient populations are screened at rates less than 20%.⁴ This discrepancy in rates of DR screening highlights the need to develop strategies that extend the benefits of monitoring and treatment for DR to all patients.

Efforts leveraging artificial intelligence (AI)-based technology are increasingly being used to improve the availability of DR screening. Automated Retinal Image Analysis Systems (ARIAS) can detect DR with high specificity and sensitivity. To date, the literature investigating ARIAS primarily focuses on its diagnostic accuracy as compared with manual grading of fundus photographs, and two ARIAS systems—IDx-DR and most recently EyeArt 2.0—have been approved by the FDA for clinical use in the United States.^5-10

Although ARIAS are highly reliable tools for detecting DR,^5,11 their value—or cost-effectiveness—to health care systems is less well understood. We sought to model short-term cost-effectiveness using data from a recent prospective study conducted within our institution’s health system analyzing adherence to recommended follow-up ophthalmic care among low-income adults with diabetes, after the implementation of an ARIAS-based screening program in a primary care clinic. We hypothesized that implementation of ARIAS in this low-income primary care setting would be cost-effective compared with the current standard of care in which all patients with diabetes are referred for annual dilated retinal examination.

Subjects, Materials, and Methods

Overview and Participant Data

We constructed a decision tree to conduct a cost-effectiveness analysis comparing two screening strategies via Markov modeling with microsimulation. The experimental strategy involved ARIAS screening of all adult patients with diabetes performed in the primary care clinic, whereas the reference strategy was current standard of care: referring all patients with diabetes for a dilated screening eye examination.^12-14 Our model used customized data for our institution derived from a prospective study performed by our group to examine the effects of an ARIAS-based primary care clinic DR screen on patient adherence to ophthalmic follow-up recommendations.¹⁵ Briefly, 179 adults with diabetes were screened with ARIAS, performed using nonmydriatic (undilated) retinal photography and image grading with EyeArt 2.0 (Eyenuk, Inc., Woodland Hills, CA, USA; of note, EyeArt 2.0 recently received FDA clearance for clinic use of autonomous detection of DR, including vision-threatening DR [vtDR]). After screening using the system, most patients received their results immediately during the visit, including the severity of their DR, a picture of their fundus photograph, and recommended follow-up interval with an eye care specialist. Patients with referable DR (moderate DR or greater), vtDR (any central diabetic macular edema [DME] or severe DR or greater), or who had inconclusive screening results were referred for ophthalmic consultation. As is typical of many referrals primary care providers in our health care system make to specialists, a follow-up referral examination was scheduled by phone subsequent to the initial PCP encounter. Rates of adherence to these recommendations were then compared with historical adherence rates to ophthalmic consultation among adults with diabetes from the same clinic. We used the baseline demographic data, referral rates and adherence rates (Table 1), and DR severity distributions described in the referenced study for the present economic modeling analysis.¹⁵

Table 1.

Referral and Adherence Rates and Diabetic Retinopathy (DR) Severity Prevalence between the Current Practice and ARIAS Cohorts.

	Current practice	ARIAS	Source
% Referred for dilated eye examination	100	50.8	Liu et al.¹⁵
% Adherent with eye examination referral	11.5	54.9
% Of all screened patients with completed eye examination	11.5	27.9
% Mild-moderate diabetic retinopathy	35.7	26.0
% Vision-threatening diabetic retinopathy	7.5	10.0

ARIAS, Automated Retinal Image Analysis System.

DR Disease States

We defined vtDR as the presence of any central diabetic macular edema (DME), presence of severe nonproliferative DR (NPDR), or presence of proliferative DR (PDR) (Figure 1). After adjusting published epidemiologic data to reflect the racial/ethnic demographics among patients at our institution, we estimated the prevalence of each possible vtDR subtype or “state” (Table 2) and used these estimates in the economic model.^16-19 As described in greater detail in the next section, treatment and complication rates during year one and years two to five for possible treatment approaches, as well as the rates of progression to severe vision loss (SVL) among patients who were treated, were based on the Diabetic Retinopathy Clinical Research Network (DRCR) Protocol I and S studies.^18-22 The rate of progression to SVL for patients with vtDR who did not receive treatment was modeled based on epidemiologic data. These figures can be found in Table 3.^18-23

Figure 1.

Truncated and simplified decision tree comparing Automated Retinal Image Analysis System screening to standard of care.

Table 2.

Modeled Prevalence of vtDR Subtypes with Associated Treatment Approach.

Subtype of vtDR	Prevalence among all adults with vtDR (%)	Treatment approach	Source
Severe NPDR without DME	91.0	Close monitoring three times annually	Varma et al¹⁶
No or mild-moderate DR with DME	5.8	DRCR Protocol I, ranibizumab with deferred laser arm	Zhang et al¹⁷
Severe NPDR with DME	0.6	DRCR Protocol I, ranibizumab with deferred laser arm	Gross et al¹⁸
PDR without DME	2.1	DRCR Protocol S, PRP arm	Elman et al¹⁹
PDR with DME	0.6	DRCR Protocol S, ranibizumab arm	Elman et al¹⁹

Note: Figures weighted to reflect the WUSTL primary care patient population.

DME, diabetic macular edema; DR, diabetic retinopathy; DRCR, Diabetic Retinopathy Clinical Research Network; NPDR, nonproliferative DR; PDR, proliferative DR; vtDR, vision-threatening DR.

Table 3.

Modeled Per-Patient Costs of Treatment and Complications for Year One and Years Two to Five for DME, PDR, and PDR with DME.

Year 1 DME only per-patient treatment cost	Number	Cost ($)	Per-patient cost ($)	Source
No. of visits year one	10			Expert opinion, more conservative than the number listed in Qin²⁴
Dilated examination	10	101.27	1012.70
OCT	10	41.81	418.10
No. of injections	9	101.99	917.91	Qin²⁴
% Focal laser	32.0	512.12	163.88	Qin²⁴
Subtotal			2512.59
0.5-mg ranibizumab	9	2114.06	19 026.54	# of injections: Qin²⁴ price: Hutton et al.¹⁴, adjusted to 2019 dollars
Total cost with ranibizumab			21 539.13
Year one DME only per-patient complications cost	Proportion (%)	Cost ($)	Per-patient cost ($)	Source
% Retinal detachment	0.5	2556.44	12.78	Elman et al.¹⁹
% Vitrectomy	2.0	3715.83	74.32
% Endophthalmitis	0.5	3240.00	16.20
% Cataract surgery	6.5	3229.80	209.94
% Elevated IOP	2.0	767.00	15.34
Total			328.58
Years two to five DME only annual average per-patient treatment cost	Number	Cost ($)	Per-patient cost ($)	Source
No. of visits per year	7			Expert opinion, more conservative than the number listed in Elman et al.²¹; Qin²⁴
Dilated examination	7	101.27	708.89
OCT	7	41.81	292.67
No. of injections	4.25	101.99	433.46	Elman et al.²¹; Qin²⁴
% Focal laser	12.8	512.12	65.30	Elman et al.²¹; Qin²⁴
Subtotal			1500.31
0.5-mg ranibizumab	4.25	2114.06	8984.76	No. of injections: Elman et al.²¹; Qin²⁴ price: Hutton et al.¹⁴, adjusted to 2019 dollars
Total			10 485.07
Years two to five DME only average annual per-patient complication cost	Proportion (%)	Cost ($)	Per-patient cost ($)	Source
% Retinal detachment	0.3	2556.44	6.46	Elman et al.¹⁹; Gross et al.²⁰; Elman et al.²¹
% Vitrectomy	0.7	3715.83	27.78	Elman et al.¹⁹; Gross et al.²⁰; Elman et al.²¹
% Endophthalmitis	0.3	3240.00	10.13	Elman et al.¹⁹; Gross et al.²⁰ estimating 0.25% per year for years three to five
% Cataract surgery	6.4	3229.80	205.09	Elman et al.¹⁹; Gross et al.²⁰; Elman et al.²¹
% Elevated IOP	2.8	767.00	21.09	Elman et al.¹⁹; Gross et al.²⁰; Elman et al.²¹
Total			270.54
Year one PDR per-patient treatment cost (PRP arm)	Number	Cost ($)	Per-patient cost ($)	Source
No. of visits year one	5			Expert opinion, more conservative than the number listed in Gross et al.¹⁸
Dilated examination	5	101.27	506.35
OCT	5	41.81	209.05
No. of initial PRP sessions	1.55	314.26	485.69	Gross et al.¹⁸
% Follow-up PRP	34.8%	314.26	109.46
No. of injections	1.4	101.99	142.79
% Focal laser	4.0%	512.12	20.48
Subtotal			1473.82
0.5-mg ranibizumab	1.4	2114.06	2959.68	No. of injections: Gross et al.¹⁸ price: Hutton et al.¹⁴, adjusted to 2019 dollars
Total with ranibizumab			4433.50
Year one PDR per-patient complications cost (PRP arm)	Proportion (%)	Cost ($)	Per-patient cost ($)	Source
% Retinal detachment	5.0	2556.44	127.82	Gross et al.¹⁸
% Vitrectomy	7.5	3715.83	278.69
% Endophthalmitis	0.0	3240.00	0.00
% Cataract surgery	3.0	3229.80	96.89
% Elevated IOP	6.5	767.00	49.86
Total			553.26
Years two to five PDR annual per-patient treatment cost (PRP arm)	Number	Cost ($)	Per-patient cost ($)	Source
No. of visits per year	3			Expert opinion, more conservative than the number listed in Bressler et al.²²
Dilated examination	3	101.27	303.81
OCT	3	41.81	125.43
% Follow-up PRP	4.6	314.26	14.58	Bressler et al.²²
No. of injections	1	101.99	101.99	Gross et al.¹⁸
% Focal laser	0.25	512.12	1.29	Gross et al.¹⁸
Subtotal			547.10
0.5-mg ranibizumab	1	2114.06	2114.06	No. of injections: Gross et al.¹⁸ price: Hutton et al.¹⁴, adjusted to 2019 dollars
Total			2661.16
Years two to five PDR annual per-patient complications cost (PRP arm)	Proportion (%)	Cost ($)	Per-patient cost ($)	Source
% Retinal detachment	3.3	2556.44	84.36	Bressler et al.²²
% Vitrectomy	2.9	3715.83	106.83
% Endophthalmitis	0.0	3240.00	0.00
% Cataract surgery	4.0	3229.80	129.19
% Elevated IOP	2.9	767.00	22.05
Total			342.44
Year one PDR and DME treatment cost (ranibizumab arm)	Number	Cost ($)	Per-patient cost ($)	Source
No. of visits year one	10			Expert opinion, more conservative than the number listed in Gross et al.¹⁸
Dilated examination	10	101.27	1012.70
OCT	10	41.81	418.10
No. of initial PRP sessions	1.5455	314.26	485.69	Gross et al.¹⁸
% Follow-up PRP	6.0	314.26	29.14
No. of injections	8.9	101.99	907.71
% Focal laser	9.5	512.12	48.65
Subtotal			2901.99
0.5-mg ranibizumab	8.9	2114.06	18 815.13	No. of injections: Gross et al.¹⁸ price: Hutton et al.¹⁴, adjusted to 2019 dollars
Total with ranibizumab			21 717.13
Year one PDR and DME complications cost (ranibizumab arm)	Proportion (%)	Cost ($)	Per-patient cost ($)	Source
% Retinal detachment	3.0	2556.44	76.69	Gross et al.¹⁸
% Vitrectomy	2.0	3715.83	74.32
% Endophthalmitis		3240.00	0.00
% Cataract surgery	1.0	3229.80	32.30
% Elevated IOP	4.5	767.00	34.52
Total			217.82
Years two to five PDR and DME annual per-patient treatment cost (ranibizumab arm)	Number	Cost ($)	Per-patient cost	Source
No. of visits per year	5			Expert opinion, more conservative than the number listed in Bressler et al.²²
Dilated examination	5	101.27	506.35
OCT	5	41.81	209.05
% Follow-up PRP	2.0	314.26	6.29	Bressler et al.²², expert opinion
No. of injections	2.65	101.99	270.27	Bressler et al.²²
% Focal laser	3.88	512.12	19.84	Bressler et al.²²
Subtotal			1011.80
0.5-mg ranibizumab	2.65	2114.06	5602.26	No. of injections: Bressler et al.²² price: Hutton et al.¹⁴, adjusted to 2019 dollars
Total			6614.06
Years two to five PDR and DME annual per-patient complications cost (ranibizumab arm)	Proportion (%)	Cost ($)	Per-patient cost ($)	Source
% Retinal detachment	0.75	2556.44	19.17	Bressler et al.²²
% Vitrectomy	2.3	3715.83	83.61
% Endophthalmitis	0.0	3240.00	0.00
% Cataract surgery	4.3	3229.80	137.27
% Elevated IOP	2.88	767.00	22.05
Total			262.10

	Rate	Source
Treated	2.0%	Elman et al.²¹; Bressler et al.²²
Untreated	7.4%	Wong et al.²³

DME, diabetic macular edema; IOP, intraocular pressure; OCT, optical coherence tomography; PDR, proliferative diabetic retinopathy; PRP, pan-retinal photocoagulation.

Costs and Other Data

Costs were drawn from Medicare 2019 allowable charges using current procedural terminology (CPT) codes and other cost estimates in the recently published literature, adjusted to reflect 2019 dollars.^12-14,25 Per-patient treatment and complication costs for year one were as outlined in the respective studies for the relevant vtDR pathology. Annual per-patient treatment and complication costs for years two to five were modeled as the average for all treatments/complications reported between years two to five in the respective protocols for the patient’s corresponding retinopathy. These figures attempted to capture the overall cost from the payor’s perspective for the diagnosis and treatment of different vtDR states. In rare instances where the data were unclear, most notably regarding the number of annual visits to an ophthalmologist for vtDR treatment and monitoring (ie, the number of patient visits in DRCR Protocols I and S likely overestimate those of actual clinical practice), appeal to conservative expert opinion was made. All variables used in the model are listed in Tables 3 and 4.^{14,18-22,24,26,27} For disease that was below the threshold for treatment (severe NPDR without DME), we performed modeling based on an assumption that such patients would be examined three times per year. For treatable disease, three approaches were modeled: (1) DME in no DR or any nonproliferative DR was treated per the deferred laser arm of DRCR Protocol I; (2) PDR without DME was treated per the PRP arm of DRCR Protocol S; and (3) concurrent PDR and DME were treated per the ranibizumab arm of DRCR Protocol S (Table 2). Per-patient treatment and complication costs among the three treatment approaches were calculated for year one and years two to five (Figure 2A and B). Future outcomes were discounted at a future rate of 3% to reflect their present value.

Table 4.

List of Treatment and Complication Costs.

Item	Cost ($)	Source
Dilated eye examination	101.27	2019 Medicare CPT: 92004 and 92014
Optical coherence tomography	41.81	2019 Medicare CPT: 92134
Intravitreal injection	101.99	2019 Medicare CPT: 67028
Focal laser	512.12	2019 Medicare CPT: 67210
Pan-retinal photocoagulation	314.26	2019 Medicare CPT: 67228
0.5-mg ranibizumab medication	2114.06	Hutton et al.¹⁴
Retinal detachment repair	2556.44
Vitrectomy	3715.83
Endophthalmitis	3240.00	Schmier et al.²⁶
Cataract extraction with intraocular lens placement	3229.80	Hutton et al.¹⁴
Elevated intraocular pressure	767.00	Li et al.²⁷

Note: Published costs before 2019 were adjusted to reflect 2019 US dollars.

CPT: current procedural terminology.

Figure 2.

Estimated per-patient costs by vtDR subtype during year one and years two to five. A, with or B, without medication costs included. Shown are adverse events costs (purple), clinic visit costs (orange), cost of pan-retinal photocoagulation (gray), cost of focal laser therapy (yellow), intravitreous injection procedure costs (light blue), and medication costs for 0.5-mg ranibizumab (green).

As ARIAS is primarily a secondary prevention strategy, all patients were modeled with good initial visual acuity of 20/30 or better in the better-seeing eye. SVL was defined as progression to visual acuity worse than 20/200 in the better-seeing eye, and rates of development of SVL in treated and untreated patients were based on the published literature as described above (Table 3).^23,28

Primary and Secondary Outcome Measures

The primary outcome measure was the incremental cost-utility ratio (ICUR) of current screening practice compared with ARIAS screening, a value generally used to quantify the incremental cost associated with a one-unit increase in utility of one health intervention compared to another.²⁹ Using Markov modeling with microsimulation over a five-year period, the ICUR was determined by dividing the difference in average direct per-patient costs between the two screening strategies by the difference in mean health utility values between the strategies. For each of the two screening strategies, secondary outcome measures included the predicted proportion of patients with vtDR who were adherent with eye examination referral, and incident development of SVL after five years.

Health Economic Analysis

Health utility is a measure of quality of life with a time component that is ultimately totaled to calculate quality-adjusted life years (QALY). Utility values used in our model were obtained from previous studies converting visual acuity to QALY.^12-14,28,30 When two sources reported different QALY approximations for the same visual acuity, the more conservative, that is, higher, utility value was chosen.^28,30 For health utility changes associated with treatment by either ranibizumab and/or pan-retinal photocoagulation (PRP), we used utility values previously published (Table 5).¹⁴

Table 5.

Utility Values for Vision States and Disease Treatments.

State	Utility	Range tested	Source
Baseline	0.87	0.8 to 1.0	Brown et al.³⁰
Severe vision loss	0.66	0.5 to 0.77	Brown et al.³⁰
Utility change, PRP treatment if no DME	−0.0075	−0.2 to +0.2	Hutton et al.¹⁴
Utility change, PRP treatment if DME	−0.019	−0.2 to +0.2
Utility change, ranibizumab treatment if no DME	0.0095	−0.2 to +0.2)
Utility change, ranibizumab treatment if DME	0.0335	−0.2 to +0.2
Death	0	N/A

Note: Severe vision loss defined as VA 20/200 or worse in the better-seeing eye.

DME, diabetic macular edema; DR, diabetic retinopathy; PRP, pan-retinal photocoagulation.

As the primary care clinic at our institution serves approximately 3000 patients with diabetes each year, the relevant per-patient ARIAS screening costs were calculated using this number (Table 6). Published long-term outcome data from Protocols I and S extended to year five, allowing us to use these data in our five-year model.^20,22 Furthermore, these data guided our analysis of the model using a two-stage Monte Carlo simulation consisting of 100 iterations of bootstrap sampling of 3000 individual patients modeled over five years, in each branch of the decision tree allowing a comparison of 3000 patients in the current practice branch to be compared with 3000 patients in the ARIAS branch. This resulted in a total of 600 000 simulated patient experiences (100 × 3000 × 2 decision tree branches). Treatment costs and utility for the two management strategies across each of the trials were summed to create means and SDs permitting calculation of the ICUR.

Table 6.

Estimated Per-Patient ARIAS Screening Costs.

	Value ($)	Per-patient cost ($)	Source
CR-2 fundus photo camera	16 000.00	5.33	Internet search
Camera maintenance per year	1000.00	0.33	Expert opinion
Eyenuk annual software subscription	5000.00	1.67	Expert opinion
Staff labor to complete screening		10.00	2019, “Patient care technician” (https://www.indeed.com/cmp/Barnes–jewish-Hospital/salaries)
ARIAS Medicare reimbursement		13.69	2019 Medicare CPT: 92228-TC
Total		31.02

Note: Per-patient cost based on 3000 patients screened per year.

ARIAS, Automated Retinal Image Analysis System; CPT, current procedural terminology.

Sensitivity Analyses

One-way sensitivity analyses were performed for key variables. Subsequently, the variables were substituted with distributions to facilitate probability sensitivity analyses. Use of distributions instead of variable point estimates allowed for exploration of variables that may be exerting excessive influence while accounting for changes in multiple variables (as compared with a one-way sensitivity analysis where all other variables remain constant). Variables were tested over a distribution range (Table 7). The willingness-to-pay, or the dollar amount that health care decision-makers would be willing to pay for a one-unit increase in effectiveness, was set at $100 000.^13,14

Table 7.

Model Distributions and Range Tested.

Variable	Point estimate	SD
Mean patient age	57	5
Baseline utility	0.84	0.21
SVL utility	0.66	0.165
Cost of 0.5-mg ranibizumab medication ($)	2114.06	528.52
Total cost for ARIAS screening ($)	31.02	7.76
Cost of DME treatment for year one ($)	2512.59	628.15
Cost of DME treatment for years two to five ($)	1500.31	375.08
Cost of DME complications for year one ($)	328.58	82.14
Cost of DME complications for years two to five ($)	270.54	67.64
Cost of eye exam (OCT + DFE) ($)	143.08	35.77
Cost of DME and PDR complications for year one ($)	217.82	54.46
Cost of DME and PDR complications for years two to five ($)	262.10	65.52
Cost of DME and PDR treatment for year one ($)	2901.99	725.50
Cost of DME and PDR treatment for years two to five ($)	1011.80	252.95
Cost of PDR only complications for year one ($)	553.26	138.31
Cost of PDR only complications for years two to five ($)	342.44	85.61
Cost of PDR only treatment for year one ($)	1473.82	368.45
Cost of PDR only treatment for years two to five ($)	582.87	145.72
Probability adherent, ARIAS (%)	54.90	13.73
Probability adherent, current practice (%)	11.50	2.88
Probability referred, ARIAS (%)	50.80	12.70
Probability vtDR, ARIAS (%)	10.00	2.50
Probability vtDR progress to SVL, non-adherent (%)	7.40	1.85
Probability vtDR, current practice (%)	7.46	1.86
Probability adherent patients develop SVL	2.00	0.50
Probability patient has DR after ARIAS screening (%)	26.00	6.50
Probability patient has DR among all patients in the current practice group (%)	35.70	8.93
Probability patient has NPDR with DME if they have vtDR	5.96	1.49
Probability patient has DME only if screen positive for vtDR (%)	5.82	1.46
Probability patient has PDR only if they have vtDR (%)	2.07	0.52
Probability patient has PDR with DME if they have vtDR (%)	0.56	0.14

ARIAS, Automated Retinal Image Analysis System; DFE, dilated fundus examination; DME, diabetic macular edema; DR, diabetic retinopathy; NPDR, nonproliferative DR; OCT, optical coherence tomography; PDR, proliferative DR; SVL, severe vision loss; vtDR, vision-threatening DR.

Statistical Software

Modeling was performed using TreeAge Pro software (TreeAge Pro 2019, R1. TreeAge Software, Williamstown, MA, USA) using methods similar to those in previously published cost-effectiveness literature.^12-14 Additional statistical analysis was performed using STATA/IC software version 14.2 (StataCorp LP, College Station, TX, USA) with alpha set at 0.05.

Results

Main Outcome Measures

We determined the absolute and incremental costs and utility between the current practice and ARIAS groups (Table 8). At five years, current practice showed comparable utility as ARIAS, but at a much greater cost. We calculated a 23.3% reduction in cost in the ARIAS group compared with standard practice (P < .001). Comparing current practice to ARIAS screening, we computed an ICUR of $258 721.81.

Table 8.

Cost-effectiveness Comparison of ARIAS Versus the Current Practice.

Group	Cost, 2019 US$	Utility, QALYs	Cost utility ratio	Incremental cost ($)	Incremental utility	ICUR ($/QALY)
ARIAS	1596.99	4.942	323.1
Current practice	2082.91	4.944	421.3	485.92	0.00188	258 721.81
Statistical testing using Monte Carlo simulation (1000 samples)
Variable	Current practice	ARIAS	Percentage of increase or decrease	P value
Cost ($)	2082.91	1596.99	23.3	<.001
Cost SD ($)	583.90	526.31
Effect	4.944	4.942	0.04	.989
Effect SD	1.017	1.023

ARIAS, Automated Retinal Image Analysis System; ICUR, incremental cost-utility ratio; QALYs, quality-adjusted life years.

Sensitivity Analysis

In one-way sensitivity analysis, the cost-effectiveness of ARIAS was robust across a wide range of variability in individual model inputs (Table 9). The model favored ARIAS until the price of 0.5-mg ranibizumab fell below $80.26 (not including the cost of the injection procedure). Additionally, ARIAS was cost-effective up to a combined cost of ARIAS screening of $161.14 per screening and up to the standard of care eye examination adherence rate of 71.5%. The full list of the nine variables that had a threshold value leading to a reversal of the cost-effectiveness analysis from favoring ARIAS to favoring current practice, in addition to select variables that notably did not have a threshold value, are included in Table 9. All other variables not included in Table 9 had no significant effects on cost-effectiveness.

Table 9.

One-way Sensitivity Threshold Testing with Notable Variables without Threshold Values.

Variable	Modeled point estimate	Sensitivity threshold value (favors ARIAS up to [or down to])
Baseline utility	0.88	0.68
Utility after development of SVL	0.60	0.816
Cost of ARIAS screening (camera, maintenance, labor, software) ($)	31.02	161.14
Ranibizumab per-injection medication cost ($)	2114.06	80.26
Probability referred after ARIAS screening (%)	50.8	69.8
Probability adherent in the current practice group (%)	11.50	71.5
Probability of having vtDR after a positive ARIAS screening (%)	10.0	17.3
Probability of progressing to SVL among non-adherent patients (%)	7.4	3.0
Probability of vtDR in the current practice group (%)	7.5	3.2
Notable variables that do not alter model favorability of ARIAS
Cost of treatment for DME only (year one or years two to five) ($)	2512.59	N/A
Cost of treatment for PDR with DME (year one or years two to five) ($)	2416.30	N/A
Cost of treatment for PDR without DME (year one or years two to five) ($)	1473.82	N/A
Cost of eye examination ($)	143.08	N/A
Probability of having vtDR at baseline in the refer everyone group	7.46	N/A
Probability adherent in the ARIAS group (%)	54.9	N/A
Probability of progressing to SVL among adherent patients (%)	2.0	N/A

ARIAS, Automated Retinal Image Analysis System; DME, diabetic macular edema; DR, diabetic retinopathy; NPDR, nonproliferative DR; PDR, proliferative DR; SVL, severe vision loss; vtDR, vision-threatening DR.

Following one-way sensitivity testing, each variable was converted from a point estimate to a distribution in the Markov model. This permitted probabilistic sensitivity analysis which identified the proportion of iterations favoring ARIAS versus standard of care at varying willingness to pay thresholds. As illustrated in Figure 3, at a willingness to pay of $50 000, the model favored ARIAS in 66% of samples, whereas at a willingness to pay of $100 000, the model favored ARIAS in 59% of instances.

Figure 3.

Cost-effectiveness acceptability curve. Acceptability curve for DR screening using ARIAS in a primary care setting (orange line) compared against rates for the standard of care, in which all individuals with diabetes are referred for dilated retinal examination (blue line). ARIAS, Automated Retinal Image Analysis System; DR, diabetic retinopathy.

Similar to findings in other cost-effectiveness studies, the per-patient price of 0.5-mg ranibizumab accounts for the greatest proportion of the annual cost for any vtDR (Figure 2A).¹³ As illustrated in Figure 2B with the medication cost of 0.5-mg ranibizumab removed, the second largest per-patient cost was the expense of ongoing clinic visits (eye examination with optical coherence tomography).

Secondary Outcome Measures

Compared with current practice, our five-year decision tree simulation showed that implementation of ARIAS increased the proportion of all patients with vtDR who were adherent with eye examination referral from 33.8% in the reference group to 59.4% in the ARIAS group. In this model, ARIAS also decreased the incidence of SVL from 3.1% to 1.1% (Table 10). These proportions were calculated from the decision tree in a deterministic fashion, precluding the use of “random walk” simulations, as we performed for primary outcome measures. Subsequently, we could not conduct statistical testing or sensitivity analyses to further analyze these estimates.

Table 10.

Secondary Outcomes of ARIAS Implementation after Five Years.

	Current practice	ARIAS	Percentage of increase or decrease
Percentage of all vtDR patients adherent	33.8	59.4	75.7
Percentage of patients who develop SVL	3.1	1.1	64.4

ARIAS, Automated Retinal Image Analysis System; SVL, severe vision loss; vtDR, vision-threatening diabetic retinopathy.

Discussion

Five-year economic modeling based on our preliminary data of low-income patients with diabetes suggests that primary care-based ARIAS implementation for DR screening is cost-effective. The calculated ICUR of $258 721.81 comparing the current practice to ARIAS screening is well beyond the willingness-to-pay threshold of $100 000.^13,14 ARIAS screening is not associated with greater short-term utility but is substantially less expensive than our institution’s historical practice of referring all patients with diabetes for annual for an in-office screening eye examinations (Table 8). The cost-effectiveness of ARIAS likely derives from the reduction in unnecessary referrals in patients without evidence of DR, the increase in adherence to follow-up ophthalmic care recommendations in patients who have evidence of vtDR as determined by ARIAS technology, and subsequent reduction in incident SVL due to identification and treatment of vtDR (Table 10). Future studies should explore the long-term costs and benefits of ARIAS implementation among low-income primary care patients with diabetes.

While ARIAS screening demonstrated its cost-effectiveness primarily by its reductions in costs, five-year QALY gains were comparable in both the ARIAS and current practice groups. This may be due to the short time frame of our economic modeling as well as the fact that there were relatively few incident cases of SVL in either screening arm (Table 10). Another finding worth greater exploration is that one-way sensitivity analysis demonstrated a change in the cost-effectiveness model once the per-injection medication price of ranibizumab dropped below $80.26 (Table 9). As a one-way sensitivity testing result, it is limited by the fact that all other model variables are held constant. However, it is substantially below the value tested using a distribution wherein the 0.5-mg ranibizumab price was modeled at $2114.06 with an SD of $528.51 (Table 7); if the price of ranibizumab changes dramatically in the United States, repeat modeling with variable distributions and probability sensitivity analyses may be necessary.

This study is limited by the fact that our model analyzes cost-effectiveness over a relatively short time period, a time period chosen primarily due to the follow-up data available via the Protocol I and S studies tracking vtDR treatment and complication rates and visual acuity outcomes. Longer-term studies should be conducted to evaluate differences in cost or QALY. Second, because our modeled treatment relied on the Protocol I and S study results, any limitations in these studies could affect our results as well. This includes evaluating the cost-effectiveness of DR treatment using 0.5-mg ranibizumab, as an economic analysis, including 0.3-mg ranibizumab, aflibercept, or bevacizumab may yield different results, as discussed above. As for why our model used 0.5-mg ranibizumab for modeling purposes, there were several reasons: treating DME with bevacizumab is still off-label, using 0.5-mg ranibizumab for DR treatment cost-effectiveness modeling is commonplace in the published literature,^12-14 and in real-life clinical practice, there is still significant use of ranibizumab/aflibercept as first-line treatment for DME, especially for patients with moderate vision loss from DME at baseline (<20/50).^31,32 These considerations were the driving factors for our decision to use 0.5-mg ranibizumab as the standard-of-care treatment for patients with DME in our economic model, as it was the modeled treatment paradigm most valid for current recommended practices across all organizations.

A further limitation of our study includes converting visual acuity to QALY figures–QALYis an imperfect measure of a patient’s quality of life. Nevertheless, it is well established in the health economics literature and does provide for a means of comparing competing management strategies of varying costs. As our model was built on the local data obtained from prospective analysis of ARIAS implementation,¹⁵ any limitations to that study, including confounding factors that influenced patient adherence differently between the ARIAS and current practice groups, could impact our cost-effectiveness outcomes as well. However, as detailed in that study, although small demographic differences between the pre-ARIAS and post-ARIAS cohorts were found, it was the post-ARIAS group that had features traditionally associated with nonadherence to medical advice (ie, higher glycated hemoglobin index). Lastly, our model only included direct costs of screening and treatment from the payor’s perspective and did not account for indirect costs to patients such as missed work for appointments or transportation costs—costs that would have likely made ARIAS even more cost-effective, especially among low-income patients that are particularly impacted by cost and transportation barriers.

The cost-effectiveness of telemedicine-based digital retinal imaging (TMDRI) systems, which are distinguished from ARIAS by the involvement of human graders, is well established despite their poor uptake in the United States health care system at large.³³ Current estimates suggest that between 80% and 85% of ophthalmology practices in the United States do not currently employ any form of teleophthalmology or automated retinal screening systems.³⁴ The lack of such systems exposed weaknesses in health care systems brought on by stressors such as the ongoing COVID-19 pandemic. As all fields in medicine move toward methods to reduce patient risk to nosocomial disease, TMDRI or ARIAS systems offer highly effective and cost-saving methods of screening, as indicated by the results of our analyses and many others. Although assessment of the relative cost-effectiveness of TMDRI and ARIAS is beyond the scope of the current study, such a comparison could be important for many health systems that are currently not employing such screening tools.

A final important point is that the analysis in our study took into consideration screening for DR only, not several other ocular comorbidities for which low-income patients are at high risk. Although EyeArt 2.0 can potentially flag ocular abnormalities such as visually significant cataracts or retinal detachments,¹⁵ the software was designed for the singular purpose of detecting DR. A limitation to our analysis is that the impact of missed screening for ocular comorbidities, such as glaucoma, in patients with diabetes was not taken into consideration. Indeed, this concern points to the potential dangers of allowing an AI platform to “clear” patients from annual dilated ophthalmic examinations by a trained human. However, as noted in numerous epidemiologic studies, many—if not most—patients with diabetes do not currently follow guidelines for annual screening examinations. Such adherence rates can be as low as 10%-30% in especially vulnerable populations.^4,35 ARIAS, as a point-of-care test, effectively removes a major barrier for achieving eye care screening adherence, namely, the burden of an additional medical visit in a population that heavily skews toward being low-income and actively employed and in which time away from work demands can be difficult to arrange. Moreover, the algorithms driving ARIAS screening are already being modified to allow for the detection of many ocular conditions, including optic disc edema, glaucoma, and macular degeneration.^36-38 Therefore, we anticipate that AI-based screening will not only improve eye screening in populations that typically do not seek eye care but also eventually provide comprehensive screening for many common causes of blindness.

Conclusion

When modeled over five years, primary care-based ARIAS screening among low-income patients with diabetes is substantially less costly and equally as effective as the more traditional practice of referring all patients for annual dilated eye examination. The cost-effectiveness of ARIAS may result from the reduction in unnecessary referrals for annual dilated eye examination, increased rates of adherence to follow-up ophthalmic care recommendations among patients with evidence of vtDR, and the resulting reduction in incident development of SVL. These findings provide support for the financial viability and sustainable use of ARIAS screening systems in the primary care setting.

Footnotes

Abbreviations

ARIAS, automated retinal image analysis system; CPT, current procedural terminology; DME, diabetic macular edema; DR, diabetic retinopathy; DRCR, Diabetic Retinopathy Clinical Research Network; ICUR, incremental cost-utility ratio; NPDR, non-proliferative diabetic retinopathy; PRP, panretinal photocoagulation; PDR, proliferative diabetic retinopathy; QALY, quality-adjusted life years; SVL, severe vision loss; vtDR, vision threatening diabetic retinopathy.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: the screening software described in this study was provided by Eyenuk, Inc. Additional funding was provided to Dr. Rajagopal by a Research to Prevent Blindness Career Development Award, and to Dr. Fuller by the Mentors in Medicine Program, Department of Medicine at Washington University School of Medicine in St. Louis.

ORCID iDs

Spencer D. Fuller

Martin Gregory

Jessica Kuo

References

Wong

Sun

Kawasaki

, et al. Guidelines on diabetic eye care: the international council of ophthalmology recommendations for screening, follow-up, referral, and treatment based on resource settings. Ophthalmology. 2018;125(10):1608-1622.

Sabanayagam

Yip

Ting

DSW

Tan

Wong

TY.

Ten emerging trends in the epidemiology of diabetic retinopathy. Ophthalmic Epidemiol. 2016;23(4):209-222.

Klein

BEK

. Screening for diabetic retinopathy, revisited. Am J Ophthalmol. 2002;134(2):261-263.

Daskivich

Vasquez

Martinez

Tseng

C-H

Mangione

CM.

Implementation and evaluation of a large-scale teleretinal diabetic retinopathy screening program in the Los Angeles County Department of Health Services. JAMA Intern Med. 2017;177(5):642-649.

Tufail

Rudisill

Egan

, et al. Automated diabetic retinopathy image assessment software: diagnostic accuracy and cost-effectiveness compared with human graders. Ophthalmology. 2017;124(3):343-351.

Soto-Pedre

Navea

Millan

, et al. Evaluation of automated image analysis software for the detection of diabetic retinopathy to reduce the ophthalmologists’ workload. Acta Ophthalmol (Copenh). 2015;93(1):e52-e56.

Welikala

Fraz

Foster

, et al. Automated retinal image quality assessment on the UK Biobank dataset for epidemiological studies. Comput Biol Med. 2016;71:67-76.

Ting

DSW

Cheung

CY-L

Lim

, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017;318(22):2211-2223.

Nielsen

Lautrup

Andersen

JKH

Savarimuthu

Grauslund

. Deep learning–based algorithms in screening of diabetic retinopathy: a systematic review of diagnostic performance [published online ahead of print November 3, 2018]. Ophthalmol Retina. doi:10.1016/j.oret.2018.10.014

10.

IDx. Homepage. Available at: https://www.eyediagnosis.net/. Accessed November 21, 2019.

11.

Bhaskaranand

Ramachandra

Bhat

, et al. Automated diabetic retinopathy screening and monitoring using retinal fundus image analysis. J Diabetes Sci Technol. 2016;10(2):254-261.

12.

Dewan

Lambert

Edler

Kymes

Apte

RS.

Cost-effectiveness analysis of ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema. Ophthalmology. 2012;119(8):1679-1684.

13.

Ross

Hutton

Stein

Bressler

Jampol

Glassman

AR.

Cost-effectiveness of aflibercept, bevacizumab, and ranibizumab for diabetic macular edema treatment: analysis From the Diabetic Retinopathy Clinical Research Network comparative effectiveness trial. JAMA Ophthalmol. 2016;134(8):888-896.

14.

Hutton

Stein

Bressler

Jampol

Browning

Glassman

AR.

Cost-effectiveness of intravitreous ranibizumab compared with panretinal photocoagulation for proliferative diabetic retinopathy: secondary analysis from a diabetic retinopathy clinical research network randomized clinical trial. JAMA Ophthalmol. 2017;135(6):576-584.

15.

Liu

Gibson

Ramchal

Shankar

Rajagopal

. Diabetic retinopathy screening in a primary care clinic with automated retinal image analysis improves adherence to follow-up ophthalmic care [published online ahead of print June 2020]. Ophthalmol Retina. doi:10.1016/j.oret.2020.06.016

16.

Varma

Bressler

Doan

, et al. Prevalence of and risk factors for diabetic macular edema in the United States. JAMA Ophthalmol. 2014;132(11):1334-1340.

17.

Zhang

Saaddine

Chou

C-F

, et al. Prevalence of diabetic retinopathy in the United States, 2005-2008. JAMA. 2010;304(6):649-656.

18.

Gross

Glassman

Jampol

, et al. Panretinal photocoagulation vs intravitreous ranibizumab for proliferative diabetic retinopathy: a randomized clinical trial. JAMA. 2015;314(20):2137-2146.

19.

Elman

Aiello

Beck

, et al. Randomized trial evaluating ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema. Ophthalmology. 2010;117(6):1064-1077.e35.

20.

Gross

Glassman

Liu

, et al. Five-year outcomes of panretinal photocoagulation vs intravitreous ranibizumab for proliferative diabetic retinopathy: a randomized clinical trial. JAMA Ophthalmol. 2018;136(10):1138-1148.

21.

Elman

Bressler

Qin

, et al. Expanded 2-year follow-up of ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema. Ophthalmology. 2011;118(4):609-614.

22.

Bressler

Glassman

Almukhtar

, et al. Five-year outcomes of ranibizumab with prompt or deferred laser versus laser or triamcinolone plus deferred ranibizumab for diabetic macular edema. Am J Ophthalmol. 2016;164:57-68.

23.

Wong

Mwamburi

Klein

, et al. Rates of progression in diabetic retinopathy during different time periods: a systematic review and meta-analysis. Diabetes Care. 2009;32(12):2307-2313.

24.

Qin

Intravitreal ranibizumab for diabetic macular edema with prompt vs deferred laser treatment: 3-year randomized trial results. Ophthalmology. 2012;119(11):2312-2318.

25.

Medicare Coverage Download Database. https://www.cms.gov/medicare-coverage-database/downloads/downloadable-databases.aspx. Accessed October 17, 2019.

26.

Schmier

Hulme-Lowe

Covert

Lau

EC.

An updated estimate of costs of endophthalmitis following cataract surgery among Medicare patients: 2010-2014. Clin Ophthalmol Auckl NZ. 2016;10:2121-2127.

27.

Tham

Chi

Lam

DS.

Cost-effectiveness of treating normal tension glaucoma. Invest Ophthalmol Vis Sci. 2013;54(5):3394-3399.

28.

Sharma

Brown

Hollands

Shah

GK.

The cost-effectiveness of photodynamic therapy for fellow eyes with subfoveal choroidal neovascularization secondary to age-related macular degeneration. Ophthalmology. 2001;108(11):2051-2059.

29.

Detsky

AS.

A clinician’s guide to cost-effectiveness analysis. Ann Intern Med. 1990;113(2):147.

30.

Brown

Sharma

Landy

Health care economic analyses and value-based medicine. Surv Ophthalmol. 2003;48(2):204-223.

31.

Wells

Glassman

Ayala

, et al. Aflibercept, bevacizumab, or ranibizumab for diabetic macular edema: two-year results from a comparative effectiveness randomized clinical trial. Ophthalmology. 2016;123(6):1351-1359.

32.

Baker

Glassman

Beaulieu

, et al. Effect of initial management with aflibercept vs laser photocoagulation vs observation on vision loss among patients with diabetic macular edema involving the center of the macula and good visual acuity: a randomized clinical trial. JAMA. 2019;321(19):1880-1894.

33.

Whited

Datta

Aiello

, et al. A modeled economic analysis of a digital teleophthalmology system as used by three federal healthcare agencies for detecting proliferative diabetic retinopathy. Telemed E-Health. 2005;11(6):641-651.

34.

Starr

Israilevich

Zhitnitsky

, et al. Practice patterns and responsiveness to simulated common ocular complaints among US ophthalmology centers during the COVID-19 pandemic. JAMA Ophthalmol. 2020;138(9):981-988.

35.

Kuo

Liu

Gibson

, et al. Factors associated with adherence to screening guidelines for diabetic retinopathy among low-income metropolitan patients. Mo Med. 2020;117(3):258-264.

36.

Milea

Najjar

Jiang

, et al. Artificial intelligence to detect papilledema from ocular fundus photographs. N Engl J Med. 2020;382(18):1687-1695.

37.

Zheng

Johnson

Garg

Boland

MV.

Artificial intelligence in glaucoma. Curr Opin Ophthalmol. 2019;30(2):97-103.

38.

Pead

Megaw

Cameron

, et al. Automated detection of age-related macular degeneration in color fundus photography: a systematic review. Surv Ophthalmol. 2019;64(4):498-511.