Validating DevCheck: a semi-automated computer algorithm for measuring foveal immaturity in premature infants

Introduction

The fovea allows for high-acuity vision and color discrimination due to densely packed cone photoreceptors. In addition, displacement of the innermost layers allows light to more directly reach the photoreceptors. During foveal development, the inner retina progressively displaces peripherally in a centrifugal pattern (1). Premature birth is frequently associated with an immature fovea, characterized by persistent inner retinal layers and a shallow foveal depression (2-6).

Detecting foveal immaturity in the premature population has potential clinical applications. Foveal immaturity, seen on optical coherence tomography (OCT), is a known biomarker for retinopathy of prematurity (ROP) diagnosis and severity in infants screened for ROP and ex-premature children (2,3,7,8). Detecting foveal immaturity may therefore contribute to the development of an OCT ROP screening tool. Given its correlation with zone of peripheral avascularity (7), foveal immaturity has the potential to serve as a biomarker for peripheral avascular retina in infants with spontaneous or pharmacologic regression of ROP. Further research is needed to determine whether this biomarker can predict recently reported long-term risk of retinal detachment and blindness occurring decades later (9,10). In that case, foveal immaturity detection would guide the selection of infants who would most benefit from additional prophylactic laser (9,10). Nonetheless, the association between foveal immaturity and visual acuity is unclear, with conflicting results among ex-preterm children and adults in the literature (2,11-14).

Semi-automated analysis of foveal immaturity has been validated using investigational handheld swept source optical coherence tomography (SS-OCT) images in the infant population, with good to excellent agreement between different graders (15,16). Foveal immaturity parameters include fovea/parafovea (F/P) thickness ratio and foveal angle (15,16). The purpose of this study is to validate a newer program, called DevCheck, optimized for commercially available handheld spectral domain optical coherence tomography (SD-OCT) (Envisu C2300, Leica Microsystems, Inc., Wetzlar, Germany) performed on a cohort of awake premature infants at the time of routine ROP examinations. This tool may provide both research and clinical applications in the future.

Methods

This is a prospective, observational study of premature infants screened for ROP. Handheld SD-OCT was performed on premature infants undergoing routine ROP examinations at the University of Washington and Seattle Children’s Hospital neonatal intensive care units (NICUs). The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by Institutional Review Board of Seattle Children’s Hospital and The University of Washington (No. PIROSTUDY15464) and informed consent was taken from all individual participants.

Participants

Consecutive patients being screened for ROP were recruited between 2015 and 2018, and informed consent from the infants’ parents or legal guardians was obtained prior to study participation. A sample size of 49 was calculated based on a minimum acceptable reliability intraclass correlation coefficient (ICC) of 0.6, expected reliability 0.8, and 2-tailed significance level of 0.05 at 80% power for two graders. Premature infants were recruited based on standard screening criteria for risk of ROP (birth weight of <1,500 g and/or gestational age of <30 weeks) (17). Infants were excluded if they were deemed too unstable by the NICU medical team for safe examination or imaging. In addition to OCT imaging, all participants also underwent standard binocular indirect ophthalmoscopy, which was used to determine the clinical management and follow-up for ROP.

Imaging/image selection

One to 2 drops of 1% phenylephrine hydrochloride and 0.2% cyclopentolate ophthalmic solution were used to dilate all the participants’ eyes. Images from both eyes were taken using an Envisu C2300 handheld SD-OCT device on the same day as the routine ROP examinations, obtaining multiple volumes to capture high-quality foveal and optic nerve images. Infants were imaged in the supine position without sedation, per usual clinical workflow. No eyelid speculum was used, and no ocular contact occurred. Eyelids were held open by the imagers’ fingers.

Trained graders (H.J. and C.J.P.) selected the highest-quality foveal B scans for each eye. B scans were exported as TIFF files, which are compatible with both programs used in this study. As a convenience sample taken for a separate study, images taken closest to a postmenstrual age (PMA) of 36 weeks were used. The two trained graders independently evaluated all B scans for image quality, defining inadequate quality as distortion, low contrast, or decentering felt to interfere with semi-automated segmentation analysis. The two graders also independently evaluated images for significantly distorting macular edema, defined as at least one foveal intraretinal hyporeflective space inducing distortion of the natural foveal contour. For disagreements, the two graders reevaluated images together and reached consensus for a final decision. Based on these evaluations, images that were of inadequate quality for analysis or had clearly identifiable, significantly distorting macular edema were excluded.

Image analysis

Two different semi-automated computer programs for foveal developmental analysis were used in this validation study: a novel, Python (RRID:SCR_008394) algorithm (DevCheck) and a previously validated MATLAB (RRID:SCR_001622) algorithm (16).

One of the authors (J.B.) created DevCheck (https://github.com/jasonbunk/napari-oct-segmentation) by modifying the previously validated MATLAB program (Figure 1) (16) to measure the same parameters with greater automation, calibrated specifically for handheld SD-OCT images from the Leica device (Figure 2). Briefly, the calculations made by the MATLAB program were translated into Python. Then, additional features, such as the use of image processing algorithms including edge detection and ridge detection, were added for better automation. Compared to its predecessor, the new program streamlines the user interface, allowing users to erase and re-draw certain segments of each segmented layer without having to start over when errors are identified. This is due to the implementation of an interactive image viewer, Napari (RRID: SCR_022765), that allows for direct user correction. The choroidal thickness measurements were abandoned due to inadequate choroidal imaging quality in handheld SD-OCT images. Two independent, trained, masked graders (authors C.J.P. and J.Y.S.) analyzed the selected SD-OCT images using DevCheck. Upon loading the image, the program automatically segmented the internal limiting membrane (ILM), outer border of the outer plexiform layer (OPL), and the retinal pigment epithelium (RPE) of the macula. The graders then adjusted the segmented RPE layer for accuracy as needed before having the program align the B-scan with the RPE as the reference. The graders then manually adjusted the remainder of the automatically segmented layers as needed to correct segmentation errors. When measuring the foveal angle, users manually identified the center of the fovea (lowest point of the fovea) as well as the location where the retinal contour begins to flatten (on each side of the foveal center). The program then calculated the foveal angle in degrees. The parafovea was automatically identified at 2.5 mm away from the manually selected foveal center on each side. The program then measured the inner and outer retinal thickness at the fovea and parafovea, averaging the two sides for the parafoveal measurements. These measurements were used to generate the F/P ratio of the inner and outer retina.

Figure 1 A sample segmentation performed by the previously validated MATLAB program for foveal immaturity analysis. This program was applied to SD-OCT images to serve as a gold standard to validate DevCheck. (A) Original handheld SD-OCT image. (B) As previously described in Lawson et al. (16), the MATLAB program required that the user place 3 dots (blue and red) near the fovea to designate the foveal angle and then the user must circle each layer (internal limiting membrane—green, outer plexiform layer—red, retinal pigment epithelium—blue, and choroidal/scleral junction—white) before segmentation. The program then calculated the foveal angle, the inner and outer retinal thickness and the choroidal thickness at the fovea and parafovea (2.5 mm from the foveal center—yellow dots). Ultimately, choroidal thickness measurements were not used due to inadequate image quality. SD-OCT, spectral domain optical coherence tomography.

Figure 2 A sample segmentation by DevCheck for foveal immaturity analysis validated for handheld SD-OCT in this study. (A) Original handheld SD-OCT image. (B) DevCheck measured foveal immaturity by automatically segmenting layers. The user had an opportunity to adjust the segmentation for any errors. The user then designated the foveal angle (3 white dots). The program provided the following: inner and outer retinal thickness at the fovea and parafovea (2.5 mm from the foveal center—vertical yellow bands) as well as foveal angle. The choroidal thickness measurements were ultimately not included in this study due to inadequate image quality. C/S Junction, choroidal/scleral junction; ILM, internal limiting membrane; OPL, outer plexiform layer; RPE, retinal pigment epithelium; SD-OCT, spectral domain optical coherence tomography.

For disagreements between the two graders, a third grader (M.T.C.) was used as a tie breaker. Disagreements were defined by the prior Lawson et al. study (16) as differences in thickness measurements greater than 50 microns or differences in foveal angle greater than 15 degrees. The tiebreaker’s measurement and the measurement that was closest to the tiebreaker’s measurement were averaged together for the final result. Otherwise, when agreement was achieved, the final result was calculated as an average between the first two graders’ results.

One trained grader (author C.J.P.) also performed MATLAB program analysis on a subset of 10 randomly selected images (convenience sample) using previously described methods on a separate day (Figure 1) (16). The program required the user to manually circle each layer (ILM, RPE, and OPL) and then mark the center of the fovea. The program then segmented the layers and aligned the image in reference to the RPE. The grader then marked the locations where the foveal contour started to flatten on each side of the foveal center. The parafovea was also automatically defined as 2.5 mm from the foveal center on each side. The same foveal angle, thickness measurements and F/P ratios were calculated by the program as described above.

Statistical analysis

The primary analysis for this study was a validation study for the functionality of the DevCheck program between two different graders and compared to a previously validated program (MATLAB). ICCs were calculated for all measurements of the complete dataset between the two graders (J.Y.S. and C.J.P.) who performed the SD-OCT analysis using DevCheck. ICCs were also calculated to compare the measurements for a subset of 10 images that were evaluated using both the Python-based DevCheck and MATLAB-based programs. All ICCs were reported with 95% confidence intervals (CIs). An ICC value between 0.75 and 0.9 was considered good reliability/agreement, and an ICC value greater than 0.90 was considered excellent reliability/agreement.

All statistical analyses were performed using IBM SPSS Statistics for Windows, Version 29.0 (IBM Corp., Armonk, NY, USA). A generalized linear mixed model was used to account for non-independence of repeated measurements per infant. A P value of less than 0.05 was considered statistically significant.

ROP diagnosis was recorded in this study based on worst diagnosis among binocular indirect ophthalmoscopy results from all study visits for each eye, rather than at the single time point where image analysis was performed. As a secondary analysis to illustrate DevCheck’s potential use, demographic data and ROP clinical characteristics were analyzed for their associations with the final foveal immaturity parameters based on DevCheck.

Results

Participants

A total of 71 images from 47 awake premature infants (46.8% male; mean gestational age 27.99±2.71 weeks; mean birthweight 974.30±279.54 grams; 37.1% ROP) were analyzed in this study. Demographic data for participants are presented in Table 1.

Intergrader agreement for DevCheck

The two graders performing DevCheck analysis agreed on 64 out of 71 images (90.1%; see Figure 3). Among the 7 images that required tiebreaking, all 7 (100%) disagreed based on foveal angle, and no disagreements occurred in foveal thickness measurements. All ICCs demonstrated good to excellent agreement between the two graders (Table 2). The ICCs for inner and outer retinal thicknesses at the fovea were 0.91 and 0.75, respectively. At the parafovea, they were 0.83 and 0.81, respectively. The ICC for the foveal angle was 0.82. Scatter plots comparing foveal angle and inner and outer F/P ratio measurements between the two graders are shown in Figure 3, illustrating the degree of correlation.

Figure 3 Comparing semi-automated measurements of foveal immaturity between Grader 1 and Grader 2 using DevCheck. (A) The R² value for inner F/P ratio measurements between Graders 1 and 2 was 0.67. (B) The R² value for outer F/P ratio measurements between Graders 1 and 2 was 0.29. (C) The R² value for foveal angle measurements between Graders 1 and 2 was 0.49. F/P, fovea/parafovea.

Interprogram agreement

Comparing DevCheck to the previously validated MATLAB program (n=10) (Table 3 and Figure 4), the ICCs for inner and outer retinal thicknesses at the fovea were 0.98 and 0.83, respectively. At the parafovea, they were 0.93 and 0.48, respectively. The foveal angle ICC was 0.95.

Figure 4 Comparing semi-automated measurements of foveal immaturity between DevCheck and MATLAB programs, performed by grader C.J.P. (A) The R² value for inner F/P ratio measurements between the two programs was 0.91. (B) The R² value for outer F/P ratio measurements between the two programs was 0.67. (C) The R² value for foveal angle measurements between the two programs was 0.83. F/P, fovea/parafovea.

Scatter plots comparing foveal angle and inner and outer F/P ratio measurements between the two programs are shown in Figure 3. The R² values ranged from 0.67 to 0.91, indicating moderate to strong correlation.

Utilizing final measurements from the Python-based DevCheck program after tie-breaker analysis was performed, demographic and ROP severity characteristics were analyzed for their relationship to the foveal immaturity parameters. Older gestational age was significantly associated with decreasing inner retinal F/P ratio (P=0.004) while no significant correlation was seen with outer retinal F/P ratio (P=0.14). No other parameters’ relationships with demographic characteristics or ROP severity reached statistical significance (all P>0.10; Table 4).

Discussion

This study validates a novel semi-automated computer program, DevCheck, for potential research and clinical use to analyze foveal immaturity from handheld SD-OCT images from the commercially available ENVISU C2300 device performed in awake premature infants.

We identified good to excellent intergrader agreement for DevCheck. The new algorithm also had comparable results to the previously validated MATLAB program (16), except for outer retinal thickness measurements at the parafovea. However, these differences may stem from image quality, which was limited in the parafoveal region for some handheld SD-OCT images.

Current standard ROP screening examinations include techniques that are distressing to infants. Previous studies identified cardiorespiratory impact associated with the use of an eyelid speculum, scleral depressor, and bright light from binocular indirect ophthalmoscopy (18-20). Using handheld OCT imaging as a screening tool for ROP could reduce the number of distressing traditional ROP screening examinations an infant might be subjected to and may even help predict risk for peripheral avascular retina (7). Legocki et al. combined various OCT biomarkers into an algorithm with high accuracy (area under the receiver operating characteristic curve of 0.94) to identify early referral-warranted ROP using non-contact handheld SD-OCT (21), Similarly, Bazvand et al. identified parameters including gestational age, foveal pit depth, and central foveal thickness that could accurately predict need for treatment in infants with ROP (22). Finally, Tam et al. found that higher F/P ratio at the inner retina and shallower foveal angle were indicators of ROP severity (7). Further study is needed before widespread clinical application of handheld OCT as a screening tool.

This study confirms the previously reported correlations between inner retinal F/P ratio and gestational age found in Lawson et al.’s study (16). Other demographic and ROP (7) associations were not seen. With a limited sample size and a single PMA time point, the present study was not designed to assess DevCheck as an ROP screening tool.

Outside of sampling limitations, this study was also limited by lower image quality for handheld SD-OCT compared to SS-OCT images previously used to validate the MATLAB program. Therefore, choroidal thickness measurements could not be obtained reliably, and insufficient parafoveal measurement quality may have degraded agreement analysis. Because the MATLAB program was not optimized for SD-OCT images, it served as an imperfect gold standard for comparison to DevCheck. This limitation was mitigated by including additional validation through intergrader agreement. Furthermore, agreement analysis between the two programs, performed on SD-OCT images in the present study, was satisfactory with rare exception. Finally, as the first commercially available handheld OCT system, the ENVISU C2300 SD-OCT device remains the most widely used handheld OCT in clinical settings despite its limitations.

Conclusions

We have developed a user-friendly Python-based algorithm, DevCheck, for measuring foveal development from commercial handheld SD-OCT images. The tool demonstrated good to excellent agreement between graders and comparable results to a prior validated MATLAB program. DevCheck is feasible for future research and clinical applications exploring infant foveal development. This study serves as proof-of-concept for DevCheck’s future potential to elucidate visual development, aid in identifying the risk of peripheral avascular retina, and measure biomarkers of ROP severity.

Acknowledgments

This study was presented as a poster at the annual Association for Research in Vision and Ophthalmology (ARVO) Conference in May 2024.

Footnote

Data Sharing Statement: Available at https://qims.amegroups.com/article/view/10.21037/qims-2025-1460/dss

Funding: This work was supported by Violet Sees to M.T.C.; the Alcon Research Institute Young Investigator Award to M.T.C.; Latham UW Medicine Vision Research Innovation Award to M.T.C.; Knights Templar Eye Foundation Career Starter Research Grant to M.T.C.; unrestricted grants from Research to Prevent Blindness; and the National Institute of Health (grant No. P30 EY001730) to the University of Washington Department of Ophthalmology.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-2025-1460/coif). L.E.G. is currently a member of a for-profit private practice, Ophthalmic Surgeons & Physicians, Ltd., which has no particular financial interest in this publication. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by Institutional Review Board of Seattle Children’s Hospital and The University of Washington (No. PIROSTUDY15464) and informed consent was taken from all individual participants.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Hendrickson A, Possin D, Vajzovic L, Toth CA. Histologic development of the human fovea from midgestation to maturity. Am J Ophthalmol 2012;154:767-778.e2. [Crossref] [PubMed]
2. Wu WC, Lin RI, Shih CP, Wang NK, Chen YP, Chao AN, Chen KJ, Chen TL, Hwang YS, Lai CC, Huang CY, Tsai S. Visual acuity, optical components, and macular abnormalities in patients with a history of retinopathy of prematurity. Ophthalmology 2012;119:1907-16. [Crossref] [PubMed]
3. Akerblom H, Larsson E, Eriksson U, Holmström G. Central macular thickness is correlated with gestational age at birth in prematurely born children. Br J Ophthalmol 2011;95:799-803. [Crossref] [PubMed]
4. Wang J, Spencer R, Leffler JN, Birch EE. Critical period for foveal fine structure in children with regressed retinopathy of prematurity. Retina 2012;32:330-9. [Crossref] [PubMed]
5. He Y, Pettenkofer M, Chu A, Sadda SR, Corradetti G, Tsui I. Characterization of Foveal Development in Treatment-Naïve Extremely Preterm Infants. Transl Vis Sci Technol 2022;11:11. [Crossref] [PubMed]
6. Rosén R, Sjöstrand J, Nilsson M, Hellgren K. A methodological approach for evaluation of foveal immaturity after extremely preterm birth. Ophthalmic Physiol Opt 2015;35:433-41. [Crossref] [PubMed]
7. Tam EK, Lee KE, Lawson SE, Monger TR, Legocki AT, Kelly JP, Liu T, Zheng Y, Ding L, Tarczy-Hornoch K, Wang R, Cabrera MT. Semiautomated measures of foveal immaturity from handheld swept-source optical coherence tomography are associated with retinopathy of prematurity severity. Br J Ophthalmol 2025;109:549-53. [Crossref] [PubMed]
8. Akula JD, Arellano IA, Swanson EA, Favazza TL, Bowe TS, Munro RJ, Ferguson RD, Hansen RM, Moskowitz A, Fulton AB. The Fovea in Retinopathy of Prematurity. Invest Ophthalmol Vis Sci 2020;61:28. [Crossref] [PubMed]
9. Hamad AE, Moinuddin O, Blair MP, Schechet SA, Shapiro MJ, Quiram PA, et al. Late-Onset Retinal Findings and Complications in Untreated Retinopathy of Prematurity. Ophthalmol Retina 2020;4:602-12. [Crossref] [PubMed]
10. Hsu HT, Yu-Chuan Kang E, Blair MP, Shapiro M, Komati R, Hubbard BG, et al. Late Vitreoretinal Complications of Regressed Retinopathy of Prematurity: Retinal Break, Vitreous Hemorrhage, and Retinal Detachment. Ophthalmol Retina 2023;7:72-80. [Crossref] [PubMed]
11. Yanni SE, Wang J, Chan M, Carroll J, Farsiu S, Leffler JN, Spencer R, Birch EE. Foveal avascular zone and foveal pit formation after preterm birth. Br J Ophthalmol 2012;96:961-6. [Crossref] [PubMed]
12. Molnar AEC, Rosén RM, Nilsson M, Larsson EKB, Holmström GE, Hellgren KM. Central macular thickness in 6.5-year-old children born extremely preterm is strongly associated with gestational age even when adjusted for risk factors. Retina 2017;37:2281-8. [Crossref] [PubMed]
13. Balasubramanian S, Beckmann J, Mehta H, Sadda SR, Chanwimol K, Nassisi M, Tsui I, Marlow N, Jain S. Relationship between Retinal Thickness Profiles and Visual Outcomes in Young Adults Born Extremely Preterm: The EPICure@19 Study. Ophthalmology 2019;126:107-12. [Crossref] [PubMed]
14. Lee YS, Teh WM, Tseng HJ, Hwang YS, Lai CC, Wu WC. Comparison of foveal thickness in preschool children with a history of retinopathy of prematurity and laser photocoagulation or anti-vascular endothelial growth factor treatment: a prospective, longitudinal study. Br J Ophthalmol 2022;106:106-12. [Crossref] [PubMed]
15. O'Sullivan ML, Ying GS, Mangalesh S, Tai V, Divecha HR, Winter KP, Toth CA, Chen X. BabySTEPS Group. Foveal Differentiation and Inner Retinal Displacement Are Arrested in Extremely Premature Infants. Invest Ophthalmol Vis Sci 2021;62:25. [Crossref] [PubMed]
16. Lawson SE, Tam EK, Zheng Y, Liu T, Monger TR, Lee KE, Legocki A, Kelly J, Ding L, Wang RK, Tarczy-Hornoch K, Cabrera MT. Semi-Automated Analysis of Foveal Maturity in Premature and Full-Term Infants Using Handheld Swept-Source Optical Coherence Tomography. Transl Vis Sci Technol 2023;12:5. [Crossref] [PubMed]
17. Fierson WMAmerican Academy of Pediatrics Section on Ophthalmology. American Academy of Ophthalmology; American Association for Pediatric Ophthalmology and Strabismus; American Association of Certified Orthoptists. Screening Examination of Premature Infants for Retinopathy of Prematurity. Pediatrics 2018;142:e. Erratum in: Pediatrics 2019;143:e20183810. [Crossref] [PubMed]
18. Mangalesh S, Sarin N, McGeehan B, Prakalapakorn SG, Tran-Viet D, Cotten CM, Freedman SF, Maguire MG, Toth CA. BabySTEPS Group. Preterm Infant Stress During Handheld Optical Coherence Tomography vs Binocular Indirect Ophthalmoscopy Examination for Retinopathy of Prematurity. JAMA Ophthalmol 2021;139:567-74. [Crossref] [PubMed]
19. Mitchell AJ, Green A, Jeffs DA, Roberson PK. Physiologic effects of retinopathy of prematurity screening examinations. Adv Neonatal Care 2011;11:291-7. [Crossref] [PubMed]
20. Mukherjee AN, Watts P, Al-Madfai H, Manoj B, Roberts D. Impact of retinopathy of prematurity screening examination on cardiorespiratory indices: a comparison of indirect ophthalmoscopy and retcam imaging. Ophthalmology 2006;113:1547-52. [Crossref] [PubMed]
21. Legocki AT, Lee AY, Ding L, Moshiri Y, Zepeda EM, Gillette TB, Grant LE, Shariff A, Touch P, Lee CS, Tarczy-Hornoch K, Cabrera MT. Multivariate Models to Diagnose Early Referral-Warranted Retinopathy of Prematurity With Handheld Optical Coherence Tomography. Transl Vis Sci Technol 2023;12:26. [Crossref] [PubMed]
22. Bazvand F, Salari F, Barzi NH, Ghassemi F, Maleki A, Dastjani Farahani A, Imani Fooladi M, Taleghani NT, Shariati MK, Harper CA 3rd, Mehrabi Bahar M. Potential application of foveal structural measurements in treatment decision for retinopathy of prematurity: an OCT-based study. Int J Retina Vitreous 2023;9:63. [Crossref] [PubMed]