Real-time automated thickness measurement of the in vivo human tympanic membrane using optical coherence tomography
Introduction
Otitis media (OM) is a middle ear infection that is extremely common and most prevalent in children under the age of three (1). This disease initially presents as an acute infection, but can progress to chronic OM. Effusions, or fluid within the middle ear cavity, may also be present. The current gold standard for diagnosis of OM is visual inspection via an otoscope to examine the surface of the tympanic membrane (TM) (2,3). Previously, our lab developed an optical coherence tomography (OCT) system with a handheld probe to noninvasively image the in vivo human TM in cross-section, as well as any structures behind the TM and within the middle ear (4,5). Depth-resolved OCT imaging has provided a unique opportunity to quantify the in vivo thickness of the TM, and correlate thickness with the different types of OM.
OCT is a noninvasive, high-resolution, real-time imaging modality that has been extensively used in ophthalmology (6-8) as well as in a large number of other medical and surgical specialties (9). In the specialties of otolaryngology (otology), pediatrics, and primary care, this technology can also be applied to improve the visualization of the TM and the structures behind it in the middle ear (10,11). While otoscopy is the gold standard for diagnosis of OM by visualizing the surface of the TM and assessing it qualitatively (2,3), this technique and others, such as acoustic reflectometry and tympanometry, do not enable the physician to visualize if a middle-ear biofilm is present behind the TM (2,12). A biofilm is a protective microenvironment containing one or more bacterial species that are prone to develop resistance toward antibiotics and re-seed infections in chronic OM.
To facilitate the use of OCT for diagnosing and quantifying OM, and for the implementation of OCT in the clinic, an automatic and real-time algorithm to quantify the thickness of the TM is desired. While several studies have imaged the ear with OCT, both ex vivo and in vivo (13-15), there are no published algorithms for segmentation and measurement of an in vivo TM using OCT. Notably, Nguyen et al. developed a classification algorithm to characterize TMs based on OCT depth scans (A-scans), however, the algorithm was not real-time and did not account for the angular orientation of the TM (16). Furthermore, a quantitative measurement of the TM thickness can have clinical importance, as there is an association with having a biofilm behind the TM and chronic OM (17-19). Additionally, it has been shown that the in vivo TM thickness in humans differs significantly between normal healthy subjects and those with acute and chronic OM (20).
For this study, we have optimized our existing portable OCT system and handheld probe for imaging the TM and the structures immediately behind it. We have developed a real-time segmentation algorithm which detects the TM and calculates its thickness while correcting for any tilt of the TM. Our algorithm utilizes a Hough transform to detect the boundaries of the TM. The Hough transform has been extensively utilized in image processing because it allows for detection of known shapes from images (21-25). Together, the OCT system with the handheld probe, and the real-time segmentation algorithm, will allow for immediate quantification of TM thickness, provide image and segmentation data for patient records, and enable a new way to screen for and visualize OM noninvasively within the human ear.
Materials and methods
The system used for this study was a portable spectral-domain (SD) OCT system shown in Figure 1. This system and the design of the handheld probe represents an advancement over our previous design (4). Figure 1A shows a photo of the portable imaging system. The system is contained within a custom fabricated cart (35 cm in depth, 62 cm in width, 150 cm in height) that enables reliable transport between and within clinical sites.
Light is delivered and collected through a handheld imaging probe modeled after a traditional otoscope (Figure 1B). The familiar form factor facilitates ease of use for physicians and other health professionals. The probe is ergonomically inspired (3.8 cm in depth, 10 cm in width, 20 cm in height) and lightweight (approximately 250 grams). The near-infrared beam for OCT imaging is scanned across the surface of the tissue using a micro-electro-mechanical-system (MEMS)-based mirror element (Advanced MEMS, San Francisco, CA, USA). A high-resolution color charge-coupled device (CCD)-based camera is integrated alongside the OCT optical system to provide surface images of tissue simultaneously with the OCT cross-sectional images. Illumination for the surface images is provided by an integrated fiber bundle and high-power LED, which are both located within the nose cone of the probe.
The OCT imaging engine is composed of a Michelson-based interferometer with accompanying broadband light source and high-resolution spectrometer. A schematic of this system is shown in Figure 1C. A broadband light source (T-860HP, Superlum), centered at 860 nm, provides 12 mW of output power with a full-width-half-maximum (FWHM) bandwidth of 135 nm. The spectrometer unit (Wasatch Photonics, UT) has a resolution of 0.04 nm paired with a 12-bit high speed line-scan camera (spl4096-140km, Basler), capable of recording 70,000 lines per second. Axial and transverse resolution was measured to be approximately 2.4 and 15 µm in air, respectively. Video-rate processing and display is enabled by using a graphics processing unit (GPU)-based OCT processing algorithm, which enables up to 100 frames per second, depending on number of A-lines used. The system has a measured SNR of approximately 105 dB, when using a perfect reflector in the sample arm.
The data is displayed through a custom-designed graphical user interface (LabView). Users can control important imaging parameters and easily save both OCT and standard otoscopic surface image data. Data can be saved by depressing a trigger button on the probe handle or by touching a button on the touch screen monitor. Data is “pre-triggered” from a data queue, enabling the saving of the previous N number of frames prior to pressing the save button. This ensures that desirable data seen by the user on the screen in real-time will not be lost.
Once the signal was acquired, all processing was performed using LabView 2013 and CUDA v5.5. on a GPU (GTX 680, NVIDIA). An outline of the processing steps is shown in Figure 2. First, standard OCT processing was performed involving a λ-to-k linearization, followed by a Fourier transform, and gamma intensity correction. Once the cross-sectional OCT data was processed, a noise threshold was applied to aid in feature extraction. The algorithm calculated the average intensity of the image and only segmented the images that were above a predetermined average intensity. This reduced the processing time because only frames that had enough data for analysis were processed.
Before proceeding to the Hough transform, the frames were subdivided into three predefined regions-of-interest (ROI), as shown in Figure 2. This enabled the algorithm to compensate for slight bends in the TM by fitting it with different segmentations in each ROI. The total region analyzed consisted of 2,048 pixels in depth, but only 1,000 pixels were used to reduce processing times and to ensure that the sample was near the focus of the probe.
Next, as shown in the center column of Figure 2, a Hough transform was applied to each ROI using (Eq. [1]):
Each pixel in the Hough space corresponds to the sum of intensities along a line with slope, m, and intercept, b. Therefore, the pixel in the Hough space with the highest intensity corresponds to a line in the frame with the highest cumulative intensity. This line was used to determine the slope, mmax, of the TM in each individual ROI. The boundaries of the TM were found by determining which lines of the same slope had one tenth the cumulative intensity of the maximum line in the Hough space, and were characterized by b1 and b2. For each ROI, the lines y = mmaxx + b1 and y = mmaxx + b2 were then superimposed in red and green onto the display.
The thickness within each ROI was calculated as the orthogonal distance between the two lines in each ROI. To convert mmax, b1, and b2 into a quantitative measurement, the equations shown in the right column of Figure 2 were used. These equations corrected for the refractive index of the TM, assumed to be 1.44 from Van der Jeught, et al. (26), voxel size, and the angle of the TM. The final thickness in each ROI was determined by forming a right triangle where the hypotenuse was the final thickness, T. In the equations, the system-dependent constants cx and cz converted from units of pixels to microns along the x and z dimensions, respectively. The median thickness of the ROIs was calculated and displayed to give the user a final readout. The median was used to reduce the effect of outliers in the measurements.
Results
The algorithm performed at a rate of 30 frames per second with a frame size of 1,000×1,000 pixels2. To test the algorithm, a phantom consisting of layers of tape was imaged as it was rotated through different angles and translated vertically to test the stability of the tilt correction capabilities. Figure 3 shows the real-time display as the phantom was rotated (from –7 to 5 degrees) and Figure 4 shows the display as the phantom was translated vertically (over 650 µm). Figure 5 shows still frame images and the median thickness of the ROIs as a function of time for each dataset. The peak-to-peak variance was approximately 5 μm within both data sets. The small area of missing points in the median thicknesses during phantom rotation was a result of the phantom surface approaching a tilt of zero degrees. Near zero degrees, spurious normal incidence reflections from within the sample arm caused the algorithm to fail. These reflections were artifacts of a non-anti-reflection coated piece of glass and should be corrected in future handheld probe designs.
Next, under an Institutional Review Board protocol approved by both the University of Illinois at Urbana-Champaign and Carle Foundation Hospital, Urbana, IL, the algorithm was tested on image data from both healthy human volunteers and volunteers with ear infections to evaluate how it performed during in vivo imaging sessions. Figure 6 and the top of Figure 7 show how the probe was used in vivo and in real-time for a normal patient to obtain the data shown in Figure 7. Figure 8 shows the onscreen display in real-time. The data from Figure 8 was acquired during the filming of Figure 6 via the finger trigger button on the handheld probe. The still shots in the middle of Figure 7 show the data before and after processing. On the right side, the three ROIs can be seen with the median thickness displayed at the base of each section. The bottom of Figure 7 shows the thickness of each ROI as a function of time. Two distinct regions in time can be seen: the time when the user was still positioning the probe to bring the TM into view, and after the TM was located within the field-of-view. These two periods can also be observed in the video (Figure 6). The median thicknesses of each ROI are shown after the positioning has been achieved. Within this field-of-view, the spatially-varying thickness of the TM can be observed. On the left, the TM thickness is measured to be approximately 60 μm, while on the right it is observed to be approximately 80 μm.
In Figure 9, the algorithm was applied to still images from patients with either acute or chronic OM. These images from subjects with OM were obtained with a prior version of the handheld probe, which exhibited a lower signal-to-noise ratio (4). Therefore, for these images, one ROI was selected manually. The algorithm finds the boundary of the TM on one side and includes the biofilm in the calculation of the TM thickness. As expected, the TM thicknesses in the OM cases are thicker than the cases of a normal TM. Additionally, the thicknesses measured with this algorithm corresponded the ranges of TM thicknesses found in acute and chronic OM, as reported by Monroy et al. (20).
Discussion
The primary goal of this project was to improve the technologies used by physicians and healthcare providers for caring for their patients with OM. It is expected that the thickness of the TM in normal healthy subjects and those with acute and chronic OM will vary (20). However, prior to the use of OCT, no technology has been able to measure the thickness of the in vivo human TM with micron resolution, and no algorithm has been previously developed to make this quantitative measurement in real-time, such as in a busy healthcare provider’s office. The rapid assessment of TM thickness has the potential to be a new diagnostic metric in OM, and for guiding treatment strategy. Because chronic OM has been highly associated with biofilms in the middle ear, and because bacteria within biofilms are known to be highly resistant to standard antibiotic regimens, the ability to noninvasively identify the presence of biofilms has the potential to be an important clinical biomarker for directing more effective antibiotic use and for determining the optimal time to recommend surgical intervention.
To address this goal in this study, we first improved the existing handheld probe and OCT system, as discussed previously. One example of the performance improvement can be seen by comparing the images in Figures 7,9. In Figure 9, the algorithm was applied to patient images obtained with an older version of the probe. With the current probe and system, and an improved signal-to-noise ratio, the images in Figure 7 could be automatically analyzed by the algorithm and did not require manual selection of the ROIs; hence, the algorithm was fully automated. Additionally, the newer probe and system design provided more convenient data acquisition because it saved pre-trigger data that could be accessed via a finger trigger built into the probe, or by a foot trigger on the ground. This allowed the user to capture higher-quality and segmented images and videos of the in vivo TM.
The main advantages of the algorithm presented here are: (I) real-time processing and display; (II) automated segmentation and quantitative thickness measurements; and (III) functionality that accounts for various angles of the TM. This was achieved by rejecting images without significant signal in them and by limiting the amount of steps in the algorithm. After noise thresholding, the images could be directly analyzed by the Hough transform. The slope obtained by the Hough transform was then used to correct for the tilt of the sample before calculating the thickness. Implementing three separate ROIs allowed the algorithm to overcome one of the inherent limitations of the Hough transform; the shape of interest needs to be known prior to analysis of the image. In our algorithm, we assumed the TM was fairly straight, and met this criterion by dividing the image into several ROIs where the algorithm could then adapt to a slight curvature of the TM. Determining the number of ROIs to implement is a tradeoff between processing speed and the potential for a more accurate segmentation. Three ROIs were used in this study as the best compromise between speed and accuracy.
Imaging the TM with a handheld probe is an inherently unstable process because of natural motion between both the subject and the operator. The operator needs to locate the TM within the imaging field-of-view and hold the probe steady during imaging and segmentation. For this reason, we analyzed the stability of the algorithm when a tape phantom (representing an artificial TM) was tilted to several angles and was vertically translated. For both tests, the algorithm had an acceptable peak-to-peak variability of 5 µm. However, when the sample was close to horizontal, the algorithm often failed due to artifacts in the data from normal incident reflections off of the surface of the phantom or TM. When tested on human subjects, the measurements generated by the algorithm varied due to the movements of the probe, user, and/or subject, but once the desired region of the TM was within the field-of-view, the measurements stabilized in less than 1 second. The normal TM is known to vary in thickness across its diameter, and as standard use protocols are developed, it may be necessary to scan multiple regions of the TM to more accurately assess these variations in thickness and to look for any sporadic distribution of biofilm growth that can be found across the TM.
It is important to note that the algorithm does not specifically discriminate between the multiple boundaries that may be associated with the TM, such as the boundary between the ear canal space and TM, the boundary between the TM and the middle ear space/effusion, and in the case of a biofilm, the boundary between the TM and biofilm or between the biofilm and middle ear space/effusion. Instead, it finds the boundaries with the highest intensity within the image. Therefore, when analyzing TMs with biofilms, the algorithm measured the thickness of the TM and the biofilm together. This, however, is advantageous because measurements that exceed the standard thickness of the normal TM are highly indicative of pathology, such as in acute OM or chronic OM with a biofilm, as previously recognized (20). Future improvements to this algorithm and approach will investigate segmentation strategies to identify and quantify all boundaries and layers that may be present, particularly in OM with effusions and/or biofilms.
Some limitations of our algorithm exist. In addition to failing when there is a normal incidence reflection from the TM, the algorithm is also problematic when imaging regions of the TM that are highly curved or close to the ear canal wall. To address this, more ROIs could be used; however, with more ROIs, the processing and display time will be slower, and there will be less data within each ROI for comparison, making the algorithm more susceptible to noise.
Conclusions
In conclusion, we have developed a novel, automatic, and quantitative algorithm that has enabled the real-time segmentation and thickness measurement of the in vivo human TM. This algorithm and methodology, in conjunction with OCT imaging of the TM and middle ear, has been shown to be both stable and reliable for measuring the thickness of the TM, even when it may be tilted or vertically displaced within the field-of-view. Applying this algorithm to OCT images from both healthy subjects and subjects with OM, robust detection of the TM boundaries and thickness measurements were obtained. The use of OCT and this algorithm offers physicians and healthcare providers a potentially new diagnostic biomarker to rapidly distinguish normal TMs and middle ear conditions from those involving acute and chronic OM, where the thickness measurements change in response to inflammation or the presence of a middle ear biofilm. By providing a new quantitative and diagnostic biomarker, these technologies offer the future potential to improve the detection, assessment, and treatment strategies for this very common disease.
Acknowledgements
We would like to thank Dr. Michael Novak, Dr. Malcolm Hill, and their clinical and research staff from Carle Foundation Hospital for their continued collaboration and assistance in the acquisition of clinical OCT data. We thank Eric Chaney and Darold Spillman for their assistance with IRB protocol management and logistical and information technology support. We would especially like to thank the additional contributions of Darold Spillman in the design, construction, and integration of the OCT system used in these studies.
Funding: This research was supported in part by grants from the National Institutes of Health (NIBIB, 1 R01 EB013723), the National Science Foundation (CBET 14-45111), the Illinois Proof-of-Concept Fund, and the Center for Integration of Medicine & Innovative Technology (CIMIT). Additional information can be found at: http://biophotonics.illinois.edu.
Authors’ contribution: Zita Hubler and Nathan D. Shemonski co-developed and implemented the algorithm. Ryan L. Shelton, Guillermo L. Monroy, and Ryan M. Nolan designed, constructed, and used the OCT system and handheld probe. Zita Hubler, Guillermo L. Monroy, Nathan D. Shemonski, and Ryan L. Shelton wrote the paper. Stephen A. Boppart conceived and supervised this study, obtained funding for this study, and edited the paper.
Disclosures: Stephen A. Boppart and Ryan L. Shelton are co-founders of PhotoniCare, Inc., which is licensing intellectual property from the University of Illinois at Urbana-Champaign related to OCT imaging of the ear and eye.
References
- Klein JO. Otitis media. Clin Infect Dis 1994;19:823-33. [PubMed]
- Shekelle P, Takata G, Chan LS, Mangione-Smith R, Corley PM, Morphew T, Morton S. Diagnosis, natural history, and late effects of otitis media with effusion. Evid Rep Technol Assess (Summ) 2002;1-5. [PubMed]
- Lieberthal AS, Carroll AE, Chonmaitree T, Ganiats TG, Hoberman A, Jackson MA, Joffe MD, Miller DT, Rosenfeld RM, Sevilla XD, Schwartz RH, Thomas PA, Tunkel DE. The diagnosis and management of acute otitis media. Pediatrics 2013;131:e964-99. [PubMed]
- Shelton RL, Jung W, Sayegh SI, McCormick DT, Kim J, Boppart SA. Optical coherence tomography for advanced screening in the primary care office. J Biophotonics 2014;7:525-33. [PubMed]
- Jung W, Kim J, Jeon M, Chaney EJ, Stewart CN, Boppart SA. Handheld optical coherence tomography scanner for primary care diagnostics. IEEE Trans Biomed Eng 2011;58:741-4. [PubMed]
- Huang D, Swanson EA, Lin CP, Schuman JS, Stinson WG, Chang W, Hee MR, Flotte T, Gregory K, Puliafito CA, Fujimoto JG. Optical coherence tomography. Science 1991;254:1178-81. [PubMed]
- Alasil T, Keane PA, Sim DA, Tufail A, Rauser ME. Optical coherence tomography in pediatric ophthalmology: current roles and future directions. Ophthalmic Surg Lasers Imaging Retina 2013;44:S19-29. [PubMed]
- Wojtkowski M, Leitgeb R, Kowalczyk A, Bajraszewski T, Fercher AF. In vivo human retinal imaging by Fourier domain optical coherence tomography. J Biomed Opt 2002;7:457-63. [PubMed]
- Fujimoto JG, Brezinski ME, Tearney GJ, Boppart SA, Bouma B, Hee MR, Southern JF, Swanson EA. Optical biopsy and imaging using optical coherence tomography. Nat Med 1995;1:970-2. [PubMed]
- Pitris C, Saunders KT, Fujimoto JG, Brezinski ME. High-resolution imaging of the middle ear with optical coherence tomography: a feasibility study. Arch Otolaryngol Head Neck Surg 2001;127:637-42. [PubMed]
- Djalilian HR, Ridgway J, Tam M, Sepehr A, Chen Z, Wong BJ. Imaging the human tympanic membrane using optical coherence tomography in vivo. Otol Neurotol 2008;29:1091-4. [PubMed]
- Hildebrandt HM, Blackwood A, Cooke J, Harrison RV, Harmes KM, Passamani PP, Klein KC. Otitis media: Guideline for clinical care ambulatory. Available online: http://www.med.umich.edu/1info/fhp/practiceguides/om/OM.pdf
- Burkhardt A, Kirsten L, Bornitz M, Zahnert T, Koch E. Investigation of the human tympanic membrane oscillation ex vivo by Doppler optical coherence tomography. J Biophotonics 2014;7:434-41. [PubMed]
- Subhash HM, Davila V, Sun H, Nguyen-Huynh AT, Nuttall AL, Wang RK. Volumetric in vivo imaging of intracochlear microstructures in mice by high-speed spectral domain optical coherence tomography. J Biomed Opt 2010;15:036024. [PubMed]
- Djalilian HR, Rubinstein M, Wu EC, Naemi K, Zardouz S, Karimi K, Wong BJ. Optical coherence tomography of cholesteatoma. Otol Neurotol 2010;31:932-5. [PubMed]
- Nguyen CT, Jung W, Kim J, Chaney EJ, Novak M, Stewart CN, Boppart SA. Noninvasive in vivo optical detection of biofilm in the human middle ear. Proc Natl Acad Sci U S A 2012;109:9529-34. [PubMed]
- Hall-Stoodley L, Hu FZ, Gieseke A, Nistico L, Nguyen D, Hayes J, Forbes M, Greenberg DP, Dice B, Burrows A, Wackym PA, Stoodley P, Post JC, Ehrlich GD, Kerschner JE. Direct detection of bacterial biofilms on the middle-ear mucosa of children with chronic otitis media. JAMA 2006;296:202-11. [PubMed]
- Post JC. Direct evidence of bacterial biofilms in otitis media. Laryngoscope 2001;111:2083-94. [PubMed]
- Bakaletz LO. Bacterial biofilms in otitis media: evidence and relevance. Pediatr Infect Dis J 2007;26:S17-9. [PubMed]
- Monroy GM, Shelton RL, Nolan RM, Nguyen CT, Novak MA. Non-invasive depth-resolved optical measurements of the middle ear for differentiating otitis media. Laryngoscope 2014;
- Illingworth J, Kittler J. A survey of the Hough transform. Computer Vision, Graphics & Image Processing 1988;44:87-116.
- Le DS, Thoma GR, Wechsler H. Automatic page orientation and skew angle detection for binary document images. Pattern Recogn 1994;27:1325-44.
- Ecabert O, Peters J, Schramm H, Lorenz C, von Berg J, Walker MJ, Vembar M, Olszewski ME, Subramanyan K, Lavi G, Weese J. Automatic model-based segmentation of the heart in CT images. IEEE Trans Med Imaging 2008;27:1189-201. [PubMed]
- Oloumi F, Rangayyan RM, Ells AL. Computer-aided diagnosis of proliferative diabetic retinopathy via modeling of the major temporal arcade in retinal fundus images. J Digit Imaging 2013;26:1124-30. [PubMed]
- Chen M, Bai J, Siochi RA. A new method of diaphragm apex motion detection from 2D projection images of mega-voltage cone beam CT. Phys Med Biol 2013;58:715-33. [PubMed]
- Van der Jeught S, Dirckx JJ, Aerts JR, Bradu A, Podoleanu AG, Buytaert JA. Full-field thickness distribution of human tympanic membrane obtained with optical coherence tomography. J Assoc Res Otolaryngol 2013;14:483-94. [PubMed]
- Hubler Z, Shemonski ND, Shelton RL, Monroy GL, Nolan RM, Boppart SA. Real-time OCT imaging and automated segmentation and thickness measurement of a tape phantom under user rotation and at various incident angles of imaging. Asvide 2014;1:353. Available online: http://www.asvide.com/articles/386
- Hubler Z, Shemonski ND, Shelton RL, Monroy GL, Nolan RM, Boppart SA. Real-time OCT imaging and automated segmentation and thickness measurement of a tape phantom under user translation along the direction of imaging. Asvide 2014;1:354. Available online: http://www.asvide.com/articles/387
- Hubler Z, Shemonski ND, Shelton RL, Monroy GL, Nolan RM, Boppart SA. In vivo OCT imaging of the human tympanic membrane using a portable OCT system with a handheld probe. Asvide 2014;1:355. Available online: http://www.asvide.com/articles/388
- Hubler Z, Shemonski ND, Shelton RL, Monroy GL, Nolan RM, Boppart SA. Real-time OCT imaging and automated segmentation and thickness measurement of the in vivo human tympanic membrane (normal). Asvide 2014;1:356. Available online: http://www.asvide.com/articles/389