AI Effective in Screening for Referable Diabetic Retinopathy

Using artificial intelligence (AI) in real-world settings is an effective way to screen for referable diabetic retinopathy (RDR), according to a meta-analysis published in PLOS Global Public Health. Existing guidelines and improving current practice may be consequences of these results.

DR is the most common complication of diabetes in adults of working age, with an estimated 103.12 million adults estimated to have the disease in 2020. DR screening aims to distinguish between patients who require a referable for ophthalmological intervention and those who can continue routine eye care services. AI has been a promising solution to challenges of development in health care services.

This review assessed the accuracy of AI to diagnose RDR compared with trained human graders in the real world.

Digital eye visualizing Artificial Intelligence | Image credit: Patrick Helmholz - stock.adobe.com

Cochrane Central Register of Controlled Trials, MEDLINE, Cumulative Index to Nursing and Allied Health Literature, Scopus, and Web of Science were all searched on February 9, 2023. A manual search of British Journal of Ophthalmology, American Journal of Ophthalmology, Ophthalmology and Retina, JAMA Ophthalmology, and Investigative Ophthalmology and Visual Science was also conducted.

Randomized control trials and observational analytical studies were included. Studies that were based on retrospective validation of existing images, review articles, editorials, case series, case reports, and qualitative studies were all excluded from the meta-analysis. Studies that had data on true positives, false positives, true negatives, and false negatives were included.

There were 15 studies with 17 datasets included for the meta-analysis. Patient-level analysis was used as the main meta-analysis. Patient-level information on diagnostic accuracy was used solely in 8 studies, 5 studies based diagnostic accuracy on eye-level information, and 2 used both patient- and eye-level information.

A forest plot of the patient-level analysis of 45,785 patients found that there was minimal variation in accuracy estimates. Good test accuracy was found in the hierarchical summary receiver operating characteristics (HSROC) plot. Sensitivity was 95.33% (95% CI, 90.60%-100%) and specificity was 92.01% (95% CI, 87.61%-96.42%). The eye-level analysis also found minimal variation in specificity and moderate variation in sensitivity in a forest plot. The HSROC plot found good test accuracy. Specificity of data was 91.24% (95% CI, 79.15%-100%) and 93.90% for specificity (95% CI, 90.63%-97.16%).

Sources of heterogeneity were also looked for on the patient level. The level of economic development in a country was found to be a major indicator of heterogeneity, with high-income countries such as Australia having specificity of 95.61% and sensitivity of 90.82% compared with low-middle income countries, such as China and India, at 95.38% and 92.21% respectively. Sensitivity and specificity were also slightly higher in primary-level health care settings compared with tertiary-level (99.35% vs 94.71% and 93.72% vs 90.88%).

There were some limitations to this study. The definition of RDR in the review was based on how the authors of the studies defined it, which could have made fundus images less clear in diagnosing RDR. All included studies had an unclear or high risk of bias. There was minimal representation of countries of different economic development.

The authors concluded that their review provided “evidence that AI could effectively screen for RDR even in real-world settings.” Sensitivity and specificity was high regardless of level of economic development in certain countries.

Reference

Uy H, Fielding C, Hohlfeld A, et al. Diagnostic test accuracy of artificial intelligence in screening for referable diabetic retinopathy in real-world settings: a systematic review and meta-analysis. PLOS Glob Public Health. Published online September 20, 2023. doi:10.1371/journal.pgph.0002160