The Challenge of Diabetic Retinopathy Standardization in an Ophthalmological Dataset

Big data analysis and artificial intelligence (AI) algorithms change how medical care is driven. They provide more accurate diagnoses, better follow-up of medical conditions, risk classification and prediction, and workflow optimization. ¹ In ophthalmology, multimodal examinations are essential for diagnosis and follow-up. ¹ Efficient and predictable management could be provided with big data and AI. ²

To achieve optimal AI development, dataset formation is critical. Reliable datasets with many entries, multiple ethnicities, and secure and trustworthy labels are fundamental. Nevertheless, labeling process standardization and homogenization remain challenges. ²

Diabetic retinopathy (DR) is a significant cause of preventable blindness in working-age adults worldwide, responsible for more than 24,000 annual cases of blindness. ³ There is increased risk in patients with chronic diabetes mellitus, especially those with poor disease control. ⁴ AI technology provides unprecedented diagnostic accuracy, triage intelligence, risk stratification, and workflow optimization with accuracy equivalent to health-care professionals ⁵ and more cost-effectiveness in DR detection. ³ Standardizing and homogenizing the classification of DR in ophthalmological datasets is challenging.

We compared applied DR scales in publicly available DR retinal fundus photos datasets. (Table 1)

Table 1.

Publicly Available DR Datasets.

	EYEPACS	ODIR	Asian Pacific tele ophthalmology society dataset	DIARETDB0 and 1	DR1 D2	E-ophtha	Hamilton eye	Indian diabetic retinopathy image dataset	Jichi DR	Rotterdam ophthalmic data repository	Messidor 1 and 2	Tsukazaki dataset
Country	United States	China	India	Finland	Brazil	France	United States	India	Japan	Netherland	France	Japan
Images	88702	8000	5590	220	1597	463	169	516	9939	1120	1748	13047
Diabetic retinopathy classification	ICDR	None	Modified ICDR	None	None	None	None	ICDR	Modified Davis	Own 5 stage classification	ICDR	None

Abbreviations: DIARETDB0/1 = Standard Diabetic Retinopathy Dataset Calibration Levels 0 and 1; DR1 and 2 = diabetic retinopathy; EYEPACS = Eye Picture Archive Communication System; Jichi DR = diabetic retinopathy dataset from Jichi Medical University; ODIR = Ocular Disease Intelligent Recognition.

A significant quality-labeling by retina specialists could improve algorithm performance, ² and reliable datasets are fundamental to AI development; however, labeling process standardization and homogenization remain challenges. ² In some datasets, the DR classification is not explicit (eg, ROD Rep). There was no DR classification method in datasets such as Tsukazaki, HEI- MED, E-ophtha, DR1, DR2, DIARETDB0/1, and ODIR.

The most often-applied classification in open-access ophthalmological datasets is ICDR (73.64%), which stratifies DR in four stages and is based on the classic ETDRS classification. We proposed the Scottish Diabetic Retinopathy (SDR) as appropriate for diagnosis thought retinal images, with one single macular centered retinal exam. SDR grading is more sensitive for grading moderate and severe cases than ICDR classification. In DR datasets, the SDR grading is an adequate classification due to greater sensitivity in detecting moderate and severe DR using a single macula-centered image.

When choosing the classification method applied in the dataset, the image field of view and the number of images must be considered. Classical ETDRS and ICDR classifications tend to underestimate DR classification in retinal photographic images due to limited image view areas compared to retinal fundus examinations. Scottish Diabetic Retinopathy Grading (SDRG) is the only classification based on a single fovea-centered retinal photograph. For this reason, it is a valuable alternative to be considered in DR dataset classifications.

It is fundamental to standardize DR grading in datasets to develop algorithms and ensure proper patient referral. AI can provide a more accurate diagnosis ¹ and be more cost-effective than human assessment. ⁶

The various DR labeling systems generate a fundamental problem for AI datasets. Possible solutions are retinal-finding identifications in supervised learning or standardization of a DR classification. Reliable labeling methods also need to be considered in datasets, with more trustworthy labeling, including multiple-specialist analysis.