Deep learning-based bronchial tree-guided semi-automatic segmentation of pulmonary segments in computed tomography images

Introduction

Lung cancer is a leading cause of mortality worldwide, and resulted in approximately 1.8 million deaths in 2020 (1). Computed tomography (CT) currently serves as the most suitable imaging modality for early detection examinations of lung cancer due to its exceptional spatial resolution and contrast resolution that enable the precise visualization of the chest’s anatomical structures.

Radiologists, pulmonologists, and surgeons use pulmonary segments as a reference to determine the location of lung lesions. The accurate identification and visualization of these segmental structures enhance the reliability and precision of individualized treatment plans (e.g., resection by lobectomy or segmentectomy). Several studies have shown that segmentectomy helps to maintain lung parenchyma and preserves 2–3.5% higher lung function in patients with early-stage lung cancer while achieving a similar overall survival rate to that of lobectomy (2,3). Additionally, segmentation of pulmonary segments has the potential to be useful in radiation therapy. Baisden et al. (4) found that the likely maximum acceptable dose (MAD) for patients considered for stereotactic body radiotherapy can be predicted based on the planned treatment volume and lung volume. However, the ratio of the affected segments to the lung volume could provide additional information for MAD prediction, as the dose to the target could affect the whole volume of the related segments rather than the planning tumor volume regions in the segments. Additionally, the segmentation of pulmonary segments could also help in functional lung avoidance radiotherapy (FLART). Research indicates that certain regions may regain function after radiotherapy in patients with central tumors (5). This phenomenon occurs when the tumor obstructs the airway prior to treatment, subsequently shrinking post-treatment and allowing for the re-ventilation of the corresponding region. The identification of segments on a patient’s diagnostic high-resolution CT (HRCT) can help to identify temporarily hypo-ventilated regions. The segmentation is then deformed to four-dimensional CT (4DCT) could provide more precise guidance for FLART through the integration of CT-based ventilation imaging methods (6-9).

Despite the extensive clinical benefits related to the segmentation of pulmonary segments, its practical application is limited by the time-intensive process of delineating segment boundaries within three-dimensional CT (3DCT) images. Typically, no distinct physical boundaries separate these segments. The complexity of segmenting these segments further compounds this issue. van Rikxoort et al. (10) developed a fully automatic method that subdivides each lobe into segments using voxel classification. This approach considers voxel features based on their relative positions within the lobe and their spatial relationship to detected lobar fissures, and had 77% accuracy in identifying tumor locations, albeit without validating the segment boundaries. Kuang et al. (11) developed the ImPulSe deep-learning model for the segmentation of pulmonary segments, which had an overall segmentation dice similarity coefficient (DSC) of 0.846. However, in certain cases, the existing models used for the segmentation of pulmonary segments may not be suitable. For example, the upper part of the left upper lobe (LUL) can be divided into three segments (apical, anterior and posterior) rather than the common two segments (apico-posterior and anterior) (12,13). In other instances, it may be necessary to further divide a segment into two to three sub-segments to achieve a more precise analysis. In these situations, the standard models for segment segmentation may not adequately capture the anatomical variability or provide the level of detail required. Consequently, manual revision on a slice-by-slice basis may be necessary.

Some anatomical-based methods have been developed to address the limitations of existing models. These methods take into account the specific anatomical characteristics of the lung, such as the distribution of the bronchial tree (BT) or pulmonary arteries (PAs). Kuhnigk et al. (14) proposed a BT-based method to extract pulmonary segments from 3DCT data. After lobe segmentation, the method approximated lung segments by assigning each lung voxel to the nearest point of the segmented BT within the same lobe. Validation studies reported an accuracy DSC of 0.8 on two in-vitro left lungs. Stoecker et al. (15) introduced a PA-based segment segmentation approach, achieving a mean surface distance (MSD) of 2–3 mm compared to the ground truth. However, the manual generation of PAs in Stoecker et al.’s study took 4–6 hours, which poses a barrier to the widespread adoption of this method in clinical settings.

In addition, recent studies have demonstrated that the generation of 4DCT-ventilation images differs when using lung segmentations that exclude airways and pulmonary vasculature (referred to as ‘advanced lungs’) and when using lung segmentations that include airways and pulmonary vasculature (referred to as ‘standard lungs’) (16-18). The advanced lungs-based method is assumed to exhibit fewer image-generation artifacts. By applying anatomical-based pulmonary segment methods, advanced segments can be generated, which can then be used to create 4DCT-ventilation images for each segment and subsequently to form the final 4DCT-ventilation image.

The rapid advancement of deep learning has led to its increasing prevalence in the segmentation of the lung, lobes, airways, and vessels within the lungs (19-21). These advancements hold promise in assisting with the segmentation of pulmonary segments. In this study, we aimed to construct a BT-based method to identify lung segments in CT images by combining multiple deep-learning models for the automatic segmentation of the lung, lobes, and airways. Subsequently, the BTs are automatically grouped into their corresponding pulmonary segments based on their anatomical positions. The labeling results require checking and correction if mislabeling occurs. After labeling the lobar pulmonary airways, each voxel within the lobe is assigned to the nearest pulmonary airway, establishing the pulmonary segments. The resulting segmentation is validated by comparing it to manual segmentations performed by a physician.

Unlike previous methods, our method has three key advantages. First, the developed method automatically identifies bronchial branches based on the anatomical position. Second, it uses multiple deep-learning models to facilitate the automatic contouring of the lungs, lobes, and BT. Third, unlike other BT-based methods, our results were validated using real patient data.

Methods

Scheme overview

The proposed method for the segmentation of pulmonary segments is a BT-based method (22). As shown in Figure 1, this method comprises five sequential steps: (I) lung segmentation; (II) lobe segmentation; (III) BT segmentation; (IV) BT branches labeling; and (V) pulmonary segment segmentation. In this study, the lung segmentation and BT segmentation were conducted using pre-trained deep-learning models available to the public that did not require additional training. For the lobe segmentation, a deep-learning model was developed specifically for this study using a publicly accessible data set (23). These steps are explained in further detail in subsequent sections. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and was approved by the Departmental Research Committee of The Hong Kong Polytechnic University. The data set used in this study was acquired from a public data set, the requirement of informed consent for this analysis was waived.

Figure 1 The workflow for generating the pulmonary segments using the bronchial tree-based method. The U-Net R231 and BronchiNet models (which are publicly accessible) from other published studies were used without any modifications. The (self-developed) V-Net model was developed for this study.

Patient data

Due to the time-consuming nature of manually segmenting 18 segments, only 14 low-dose documented breath-hold HRCT scans without serious large disease regions were used for the segmentation and evaluation. These scans were obtained from the VIA/I-ELCAP Public Access Research Database (https://veet.via.cornell.edu/cgi-bin/datac/signon.cgi). Each scan was acquired in a single breath hold and had a slice thickness of 1.25 mm, resulting in minimal visible motion artifacts on the CT image. For evaluation purposes, a physician manually delineated the reference structures, including the lungs, lobes, and pulmonary segments for 14 patients. The accuracy of the BT-based method for the segmentation of the pulmonary segments was assessed using the DSC and MSD.

Segmentation of the lung and lung lobes

The conventional threshold-based method fails to accurately delineate the lung mask due to the diversity of CT images, patient modalities, and lung patterns associated with diseases (24). Thus, in this study, a well-trained U-Net (R231) model was used for the automatic segmentation of the lung mask without any fine tuning (24). The model was trained using diverse data sets of patients with various diseases, and had a mean DSC of 0.98±0.03; for further details on the model design (see https://github.com/JoHof/lungmask). After the segmentation of the lung mask, a V-Net deep-learning model (25) was developed in this study to outline the lung lobes. For more details about the model’s design and optimization settings (see Appendix 1).

The training and validation data were obtained from a publicly accessible database (23), which consisted of 51 randomly selected CT images from the LUNA16 data set, each with manually labeled lung-lobe masks. Of the 51 CT images, 41 were randomly selected for training, and the remaining 10 were used for validation. The accuracy of the V-Net model on the validation data set was evaluated using the DSC and MSD. The DSC measures the overlap between two volumes; that is, the V-Net lobe mask (A) and the manual lobe mask (B). The Hausdorff distance represents the maximum distance among the surface distances of all points in surface A. The surface distance from a point in surface A to surface B is the distance from that point to its nearest corresponding point on surface B. The MSD corresponds to the average surface distance of all points in surface A. The accuracy rate of the U-Net (R231) and V-Net models were estimated by comparing them to the segmentations performed by the physician using the 14 patients.

Segmentation of the airway

In this study, the respiratory airway was divided into the central airway and the BT branches. The central airway includes the trachea, the left main bronchus (LMB), the right main bronchus (RMB), and the following five bronchi connected to five different lobes: right upper lobe (RUL), right middle lobe (RML), right lower lobe (RLL), LUL, and left lower lobe (LLL). The contouring of the central airway was performed automatically using a differential geometric approach method (26). This method relies on the disparities in principal curvatures and directions between the airways and surrounding tissues in geometric space. For the segmentation of the BT branches, we used a well-trained three-dimensional (3D) convolutional neural network, BronchiNet. This model was directly applied without any modifications to segment the peripheral airways (27). A description of this model can be found at https://github.com/antonioguj/bronchinet. This method involved cropping the full-size images into bounding boxes around the lung region. These boxes could overlap, and each box had dimensions of 252×252×252 as input for the model. Before cropping, all the CT images were re-sampled to a resolution of 1 mm × 1 mm × 1 mm. These re-sampled data were also used for subsequent analyses and the segmentation of the pulmonary segments. The total respiratory airway was obtained by combining the results of the two methods described above. The evaluation metrics included the total BT lengths in the lungs (excluding the trachea) and the maximum generation numbers (starting from the trachea, generation 0). For the left lung, the LUL was further divided into the upper lobe and the lingular lobe for analysis. The lingular lobe corresponds to the middle lobe in the right lung. Due to the time-consuming nature of manual airway segmentation, the airways of the 14 patients were not manually contoured to verify the accuracy and completeness of the BronchiNet model in this study.

Labeling the BT branches

To facilitate the segmentation of each pulmonary segment, it is imperative to first identify each branch of the airway as shown in Figure 2. For this study, the lung was divided into the left and right lung. After the airway segmentation, a 3D image skeletonization method (28) was employed to extract the airway skeleton from the airway mask as depicted in Figure 2A,2B. The automated lobe-based airway labeling method (29) was used to identify the central airways, including the trachea, LMB, RMB, RUL, RML, RLL, LUL, and LLL, based on the anatomical position of the airway distribution. The outcome of identifying the main bronchi in the right lung is shown in Figure 2C. Each branch of the airway was then identified to ascertain the subsequent generations of bronchi.

Figure 2 The five steps of the bronchial tree branches labeling in the right lung. (A) The airway mask. (B) The outcome of the skeletonization of the airway mask. (C) The identification of the main bronchi of the right lung. (D) The consequence of the recognition of the represented branches in the right upper lobe. (E) The result obtained from the classification of the remaining branches to the represented branches. (F) The culminating result of the bronchial tree for the right lung, having executed steps (D) and (E) on the other lobes. In this figure, the X-axis represents the left-right direction, the Y-axis denotes the anterior-posterior direction, and the Z-axis indicates the superior-inferior direction.

The labeling of each branch was primarily based on the position of the airway’s distal end point. As illustrated in the top left quadrant of Figure 3, the RUL was divided into the following three pulmonary segments: the apical segment, posterior segment, and anterior segment. In relation to the branches connected to the RUL, the branch with the highest z-value of the endpoint was selected as the representative branch of the apical segment, the branch with the highest y-value of the endpoint was chosen as the representative branch of the posterior segment, and the branch with the lowest y-value of the endpoint was designated as the representative branch of the anterior segment. The result is depicted in Figure 2D. However, in some instances, two branches may initially share some of the same sections. In such cases, the point at which the two branches begin to separate was considered the dividing node. As Figure 2E shows, any other branches sharing the same section after the dividing node, along with the representative branch, were assigned to the same segment. This process ensured that all branches were allocated to their respective segments. Figure 3 displays the distribution of a representative branch airway in the corresponding lobes. A similar approach was employed to assign the remaining branches to the corresponding segments in other lobes. The final result of the BT branches labeling in the right lung is presented in Figure 2F. This methodology was also applied to the left lung to label the BT branches. For the LUL, the lobe was first separated into the upper part and lingular part, and each part was then further divided into two segments as shown in Figure 3.

Figure 3 The disposition of the representative branch airway within their respective lobes. The red lines represent the representative branch airway, while the dash lines outline the boundary of the segments in the lobes, and the large black dots indicate the dividing nodes. It should be noted that the superior segments of both the right and left lower lobes has been omitted from the figure to avoid confusion. Each representative branch airway can be distinguished based on the position of its endpoints, which have the highest or lowest value along the X-, Y-, or Z-axis. In this figure, the X-axis represents the left-right direction, the Y-axis denotes the anterior-posterior direction, and the Z-axis indicates the superior-inferior direction. RUL, right upper lobe; RML, right middle lobe; RLL, right lower lobe; LUL, left upper lobe; LLL, left lower lobe.

Due to the variability in airway distribution, the automated BT branches labeling method mentioned above may not achieve 100% accuracy. Thus, some manual interventions may be necessary to construct an accurate BT model. Each representative branch must be carefully examined, and any mislabeling must be rectified. For instance, Figure 4 illustrates a case in which the left upper bronchus exhibited a trifurcated type (30). The dividing node between the apico-posterior and anterior branches coincided with the dividing node between the upper part and the lingular part. As a result, the automatic identification method initially assigned the anterior segment to the lingular part, leading to the division of the true apico-posterior segment into two segments—false apico-posterior and false anterior—as depicted in Figure 4A. After a visual inspection, we manually reassigned the representative branches of the apico-posterior, anterior, and lingular parts. This adjustment enabled us to achieve the corrected segmentation results as depicted in Figure 4B.

Figure 4 A variant example of the airway distribution in the left upper lobe with airway branch auto-identification (A) and manual correction (B).

Segmentation of pulmonary segments

The pulmonary segments, which are surrounded by connective tissue septa (31) that are not identifiable on CT images, were segmented separately for each lobe. The labeled BT branches were used to delineate the pulmonary segments within each lobe. The segmentation of pulmonary segments was performed in each lobe. In the case of the RUL, the distance between each voxel and the segmental bronchial branches of the apical segment, posterior segment, and anterior segment was calculated. The voxel was then assigned to the corresponding segment based on the shortest distance. To address the issue of certain segmental bronchial branches being longer than others due to the incomplete segmentation of the BT and its potential bias in the segmentation of pulmonary segments, a principal direction was computed and used to create a virtual branch for each segment. The principal direction was determined by normalizing the sum of all vectors, with each vector defined as the path from the divided node to another point on the branches. The virtual branch was represented by the vector originating from the divided node and extending along the principal direction to the boundary of the lobe. The outcome of the BT branches labeling with and without the virtual branch in the left lung is shown in Figure 5. Further, we conducted a Spearman correlation analysis of the delineation length of the bronchial branches and the DSC of the pulmonary segment.

Figure 5 An example of the bronchial tree distribution in the left lung without (A) and with (B) the virtual branch.

Results

Segmentation of lung lobes

In relation to the lobe segmentation, the V-Net model yielded a mean DSC and a mean MSD of 0.94±0.06 and 0.76±0.73 mm, respectively, in the validation data set. For the left lung, the mean DSC was 0.97±0.02, while for the right lung, it was 0.92±0.07. Notably, the middle lobe of the right lung had the lowest DSC (0.86±0.09). The MSD values for the middle lobe and upper lobe of the right lung were 1.49±0.99 and 0.82±0.42 mm, respectively, which were higher than those of the other lobes for which the MSD values were less than 0.80 mm. This difference may be due to the difficulty in identifying the fissure between the upper and middle lobes of the right lung in some patients’ images. Figure 6 provides an example of the coronal view of a CT image, lung masks, and lobe masks generated by the model for the same patient. The evaluation results of the U-Net (R231) and V-Net models are presented in Table 1. These results compared the auto-segmentations of the lungs and lobes for the 14 patients with the corresponding manual segmentations. These 14 patients were not part of the training data set for the models; however, the performance of both models on these patients was comparable to the performance on the validation data set in terms of DSC. Notably, there was a slight increase in the MSD for the lobes, which had a mean MSD of 0.94±0.87 mm, compared to the validation data set.

Figure 6 The coronal view of a patient with the CT image (A), lung masks (B), and lobe masks (C). CT, computed tomography.

Segmentation and labeling of the BT branches

The combination of the differential geometric approach and the 3D convolutional neural network model proved effective in identifying the majority of airways on the CT images for the airway segmentation. However, as the segmentation results might have included disconnected airways, any “isolated” branches in the segmentation results were removed to ensure accuracy. The average total length of the segmented airway trees per CT image was 1,902.8±502.1 mm, with an average of 8.5±1.3 maximum generations of airway trees. There was no simple direct relationship between the generation numbers and the average total airway lengths. The segments in the upper lobe exhibited the longest average airway tree lengths, with values of 106.8±60.1 and 135.3±56.3 mm for the right and left lungs, respectively. Conversely, the segments in the middle lobe had the shortest average airway tree lengths, with values of 51.9±35.5 and 39.9±33.3 mm for the right and left lungs, respectively. For the lower lobe, the average airway tree lengths of the segments were 91.8±51.9 and 105.3±58.2 mm for the right and left lungs, respectively. Figure 7 presents the average length of the airway tree for each segment. Figure 2 illustrates the performance of a sample for airway segmentation and labeling. Among the 14 patients, five patients had accurate airway labeling that did not require any corrections, seven patients required re-labeling in one lobe, one patient required re-labeling in two lobes, and one patient required re-labeling in three lobes. Overall, only 17% of the lobes required re-labeling of the BT branches by manually selecting the represented branch of individual pulmonary segments within the lobe.

Figure 7 The average airway tree length of the 18 segments. The median length is shown as the line inside the box. The bottom and top edges of the box indicate the 25th and 75th percentiles, respectively.

The performance of the semi-automatic BT-based method

Figure 8 presents a comparison between the BT-based segmentation and manual segmentation of two representative results, one from the right lung and the other from the left lung. The performance evaluation yielded a mean DSC of 0.72±0.10 for the right lung and 0.73±0.12 for the left lung. The mean DSC for the entire lung was 0.73±0.11. Figure 9A presents an evaluation of the DSC for each segment. Similar to the average airway tree length, the segments in the upper lobe exhibited the highest average DSCs with values of 0.76±0.08 and 0.79±0.10 for the right and left lungs, respectively. The segments in the middle lobe had the lowest average DSCs with values of 0.66±0.10 and 0.68±0.13 for the right and left lungs, respectively. For the lower lobe, the average DSCs of the segments were 0.73±0.09 and 0.71±0.11 for the right and left lungs, respectively. The MSD was 6.3±3.0 mm for the right lung and 6.0±2.8 mm for the left lung. The MSD for the entire lung was 6.1±2.9 mm. The RML exhibited the highest MSD with a value of 8.0±2.8 mm, while the MSD values for the other lobes were less than 6.3 mm. The MSD evaluation result for each segment is shown in Figure 9B.

Figure 8 The comparisons of the segmentation for two selected scans (the rows), presented in the sagittal view to display as many segments on each slice as possible. The first row depicts the left lung, while the second row depicts the right lung. (A,B) The CT images with manual segmentation. (C,D) The CT images with BT-based segmentation. The choice of slices ensures that a maximum number of pulmonary segments are visible in each image. CT, computed tomography; BT, bronchial tree.

Figure 9 The evaluation of the semi-automatic bronchial tree-based method for each segment. The median value is shown as the line inside the box. The bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. (A) The DSC for each segment; (B) the MSD for each segment. DSC, dice similarity coefficient; MSD, mean surface distance.

In terms of the lobes, the average airway tree length in the segments appeared to reflect the quality of the segmentation performance. The analysis revealed a moderate correlation (Spearman correlation coefficient =0.68) between the airway lengths and the DSCs for the pulmonary segments. This relationship is visually represented in Figure 10, which depicts the observed association between the mean DSC and the average total airway lengths across the 18 segments.

Figure 10 The correlation between the airway length and the DSC value of the segments. DSC, dice similarity coefficient.

Table 2 compares the results of our current segmentation method for the pulmonary segments with the results of the other existing methods. It is important to note that these methods were evaluated using their own data sets and evaluation parameters. Among them, model-based methods, such as voxel classification (10) and ImPulSe (11), have the advantage of automatically performing segment segmentation. Notably, the ImPulSe model achieved a DSC of 0.846. Conversely, anatomical-based methods, such as the BT-based (14) and PA-based (15) methods, required the manual contouring of partial accessory organs and the identification of the corresponding airway or artery branches for each segment. The PA-based method had the highest accuracy with a MSD of 2–3 mm. In our approach, we combined existing publicly available segmentation models with our self-developed model and method to automate the segmentation of the lungs, lobes, and airway, and the identification of airway branches for segment segmentation. The accuracy of our method was lower than that of the ImPulSe and PA-based method. Thus, further improvements are still needed. In cases of airway distribution variability, manual interventions may be required to correct misidentifications.

Incorporating the virtual branch to improve segmentation

The incorporation of the virtual branch helps address the imbalance caused by incomplete airway segmentation. Figure 11 provides two examples of the segmentation of the pulmonary segments in the left lingular lobe, illustrating the effect of virtual branches. For the first patient shown in Figure 11A-11C, the length of the airway in the superior segment of the LUL lingular part (LUL_lingular_sup) branch was shorter (length: 7.1 mm) than the inferior segment of the LUL lingular part (LUL_lingular_inf) branch (length: 21.9 mm). Consequently, the segmentation result for the LUL_lingular_sup segment was considerably smaller than that of the LUL_lingular_inf segment for this patient. However, by including the virtual branch, the DSCs of LUL_lingular_sup and LUL_lingular_inf segments increased from 0.19 and 0.65 to 0.72 and 0.71, respectively. In the second patient (Figure 11D-11F), both the airway branches of the LUL_lingular_sup and LUL_lingular_inf segments were short. Despite the inclusion of virtual branches, the accuracy of the segmentation did not improve. The DSCs of LUL_lingular_sup and LUL_lingular_inf segments changed from 0.57 and 0.34 to 0.62 and 0.29, respectively. Notably, among all the patients, this particular patient had the lowest DSC for the LUL_lingular_inf segment.

Figure 11 The results of the segmentation for the LUL_lingular_sup and LUL_lingular_inf segments, both with and without the virtual branch, are compared to the manual segmentation. 3D view screenshots of the image are provided from different angles to enhance the visibility of the bronchial branches and the boundary between the two pulmonary segments. (A,D) The BT-based method without the virtual branch. (B,E) The BT-based method with the virtual branch. (C,F) The manual segmentation with the virtual branch. LUL, left upper lobe; LUL_lingular_sup, superior of the LUL lingular; LUL_lingular_ inf, inferior of the LUL lingular; LUL, left upper lobe; 3D, three-dimensional; BT, bronchial tree.

Discussion

In this study, we developed a BT-based method to perform the segmentation of pulmonary segments on 14 HRCT images. The workflow involved the segmentation of the lung, lobes, and airways, as well as the labeling of the airways. Deep-learning models were used to automatically segment the lung, lobes, and airways, while the anatomical position was used to label the airway branches. As this method successfully automated the segmentation of lung segments, the physician was were only required to verify and rectify any misidentified bronchial branches resulting from incomplete airway segmentation or variations. Corrections were necessary for 64% of the patients, but most adjustments were confined to a single lobe. Consequently, this method significantly reduced the time burden associated with the segmentation of pulmonary segments. Additionally, a virtual branch method was proposed to improve results in cases of substantial incomplete airway contouring, which achieved an accuracy of 0.73±0.11 in terms of DSC and a MSD of 6.1±2.9 mm. Despite advances in the BT-based segmentation method, the results obtained still lag behind those obtained for the established PA-based method and state-of-the-art deep-learning techniques (11,15). Further enhancements are necessary to improve the accuracy of the method, particularly in cases in which specific patients exhibit suboptimal results for certain segments.

To the best of our knowledge, this is the first assessment of the segmentation of pulmonary segments on clinical CT images using the BT-based method. With a DSC of 0.846 (11), our method had lower accuracy than existing deep-learning methods; however, our method has the potential to be extended to segment sub-segments and account for segment variations (30,32-34). The combination of deep learning with our methods may enable the strengths of each approach to be leveraged. Deep-learning methods have the ability to learn from large data sets and capture intricate patterns, while anatomical-based methods consider the underlying anatomical features of the lungs. This synergy could lead to more accurate and robust segment segmentation, particularly in scenarios involving segment variations or the need for sub-segment segmentation.

The PA-based method described by Stoecker et al. (15) achieved a MSD accuracy of 2–3 mm, which is significantly less than our result of 6.1 mm. However, the manual generation of PAs in their study required 4–6 hours, making it impractical for daily clinical use. It may be worth exploring the application of deep learning to expedite the delineation process, which may also require the identification of veins. The feasibility and effectiveness of such an approach need to be further investigated.

It is important to note that the proposed method relies directly on each patient’s BT, making it sensitive to variations in the segmental BT. These variations pose challenges to the full automation of the segmentation process using this method. Additionally, the quality of the BT segmentation greatly affects the final segmentation outcome. The virtual branch introduced in this study improved results in cases of incomplete airway contouring within the segments. However, it primarily represents the airway growth trend near the lung hilum and may not adequately account for the distal regions of the airway if the segmentation of the branch airway is very short as shown in Figure 11D-11F. Special attention should be paid to the quality of BT segmentation, especially in cases involving tumor-bearing lobes where a large central tumor may result in the absence of the entire BT. Moreover, the airway segmentation method employed in this study may not perform well for 4DCT or CT scans with a significant disease burden, as certain BT branches in specific segments may not be segmented at all. Thus, the current method is only applicable to breath-hold HRCT scans and patients without extensive disease involvement. In the future, we intend to enhance the performance of the airway segmentation by fine tuning the current model using a larger data set that includes CT data with a variety of lung diseases (35). By incorporating diverse cases of lung diseases, we will be able to effectively address variations and abnormalities commonly encountered in clinical scenarios, resulting in a more robust and reliable airway segmentation model.

To enhance the method further, the branch lines (instead of the branch airway) could be manually drawn to improve the representation of the BT, particularly in regions with insufficient segmentation by the deep-learning model. Our proposed semi-automatic segmentation method requires physicians to manually correct certain branch recognitions, as it is not entirely automated. Currently, the results of airway segmentation and labeling are imported into the third-party software, Slicer, along with the CT images for visual inspection. To enhance usability for clinical purposes, it would be beneficial to develop a dedicated software program that enables interactive corrections. In such a software program, a feature could be implemented to allow users to draw virtual branch lines that represent the airway segmentation for the interested segments. The inclusion of these virtual branch lines would compensate for any potential limitations or areas of insufficiency in the deep-learning model’s segmentation. By incorporating this interactive correction functionality and the use of virtual branch lines, the performance of the airway segmentation would be significantly improved. Notably, valuable results can be achieved in just a few minutes, making the process more efficient and time-effective for clinical practitioners. Further, applying a smoothing technique to the boundaries of the segmented regions in a final post-processing step may enhance the overall results. The segmentation of the pulmonary segments will be combined with the CT-based ventilation imaging method to identify temporarily hypo-ventilated regions for better FLART in the future.

Conclusions

In conclusion, this study successfully demonstrated the feasibility of integrating multiple deep-learning models for the segmentation of pulmonary segments using a BT-based method on HRCT scans. The results highlighted the potential of the BT-based method for the automatic segmentation of pulmonary segments.

Acknowledgments

Funding: This work was supported by funding from the General Research Fund of The University Grants Committee (No. 15103520), the Health and Medical Research Fund of The Health Bureau (Nos. HMRF 07183266 and HMRF 09200576), The Government of the Hong Kong Special Administrative Regions, the PolyU (UGC) Start-Up Fund for RAPs under the Strategic Hiring Scheme (No. P0038378), and the Project of RISA of The Hong Kong Polytechnic University (No. P0043001).

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-23-1251/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and was approved by the Departmental Research Committee of The Hong Kong Polytechnic University. The data set used in this study was acquired from a public data set, the requirement of informed consent for this analysis was waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Wild C, Weiderpass E, Stewart BW. World cancer report: cancer research for cancer prevention. IARC Press; 2020.
2. Saji H, Okada M, Tsuboi M, Nakajima R, Suzuki K, Aokage K, et al. Segmentectomy versus lobectomy in small-sized peripheral non-small-cell lung cancer (JCOG0802/WJOG4607L): a multicentre, open-label, phase 3, randomised, controlled, non-inferiority trial. Lancet 2022;399:1607-17. [Crossref] [PubMed]
3. Altorki N, Wang X, Kozono D, Watt C, Landrenau R, Wigle D, et al. Lobar or Sublobar Resection for Peripheral Stage IA Non-Small-Cell Lung Cancer. N Engl J Med 2023;388:489-98. [Crossref] [PubMed]
4. Baisden JM, Romney DA, Reish AG, Cai J, Sheng K, Jones DR, Benedict SH, Read PW, Larner JM. Dose as a function of lung volume and planned treatment volume in helical tomotherapy intensity-modulated radiation therapy-based stereotactic body radiation therapy for small lung tumors. Int J Radiat Oncol Biol Phys 2007;68:1229-37. [Crossref] [PubMed]
5. Yuan ST, Frey KA, Gross MD, Hayman JA, Arenberg D, Cai XW, Ramnath N, Hassan K, Moran J, Eisbruch A, Ten Haken RK, Kong FM. Changes in global function and regional ventilation and perfusion on SPECT during the course of radiotherapy in patients with non-small-cell lung cancer. Int J Radiat Oncol Biol Phys 2012;82:e631-8. [Crossref] [PubMed]
6. Huang YH, Teng X, Zhang J, Chen Z, Ma Z, Ren G, Kong FS, Ge H, Cai J. Respiratory Invariant Textures From Static Computed Tomography Scans for Explainable Lung Function Characterization. J Thorac Imaging 2023;38:286-96. [Crossref] [PubMed]
7. Chen Z, Huang YH, Kong FM, Ho WY, Ren G, Cai J. A super-voxel-based method for generating surrogate lung ventilation images from CT. Front Physiol 2023;14:1085158. [Crossref] [PubMed]
8. Ren G, Lam SK, Zhang J, Xiao H, Cheung AL, Ho WY, Qin J, Cai J. Investigation of a Novel Deep Learning-Based Computed Tomography Perfusion Mapping Framework for Functional Lung Avoidance Radiotherapy. Front Oncol 2021;11:644703. [Crossref] [PubMed]
9. Ren G, Zhang J, Li T, Xiao H, Cheung LY, Ho WY, Qin J, Cai J. Deep Learning-Based Computed Tomography Perfusion Mapping (DL-CTPM) for Pulmonary CT-to-Perfusion Translation. Int J Radiat Oncol Biol Phys 2021;110:1508-18. [Crossref] [PubMed]
10. van Rikxoort EM, de Hoop B, van de Vorst S, Prokop M, van Ginneken B. Automatic segmentation of pulmonary segments from volumetric chest CT scans. IEEE Trans Med Imaging 2009;28:621-30. [Crossref] [PubMed]
11. Kuang K, Zhang L, Li J, Li H, Chen J, Du B, Yang J. What makes for automatic reconstruction of pulmonary segments. In: Wang L, Dou Q, Fletcher PT, Speidel S, Li S. editors. Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. Lecture Notes in Computer Science, vol 13431. Springer, Cham; 2022.
12. Zhao X, Ju Y, Liu C, Li J, Huang M, Sun J, Wang T. Bronchial anatomy of left lung: a study of multi-detector row CT. Surg Radiol Anat 2009;31:85-91. [Crossref] [PubMed]
13. Kandathil A, Chamarthy M. Pulmonary vascular anatomy & anatomical variants. Cardiovasc Diagn Ther 2018;8:201-7. [Crossref] [PubMed]
14. Kuhnigk JM, Dicken V, Zidowitz S, et al. Informatics in radiology (infoRAD): new tools for computer assistance in thoracic CT. Part 1. Functional analysis of lungs, lung lobes, and bronchopulmonary segments. Radiographics 2005;25:525-36. [Crossref] [PubMed]
15. Stoecker C, Welter S, Moltz JH, Lassen B, Kuhnigk JM, Krass S, Peitgen HO. Determination of lung segments in computed tomography images using the Euclidean distance to the pulmonary artery. Med Phys 2013;40:091912. [Crossref] [PubMed]
16. Chen Y, Pahlavian SH, Jacobs P, Neupane T, Forghani-Arani F, Castillo E, Castillo R, Vinogradskiy Y. Systematic Evaluation of the Impact of Lung Segmentation Methods on 4-Dimensional Computed Tomography Ventilation Imaging Using a Large Patient Database. Int J Radiat Oncol Biol Phys 2024;118:242-52. [Crossref] [PubMed]
17. Guerrero T, Sanders K, Castillo E, Zhang Y, Bidaut L, Pan T, Komaki R. Dynamic ventilation imaging from four-dimensional computed tomography. Phys Med Biol 2006;51:777-91. [Crossref] [PubMed]
18. Castillo R, Castillo E, Martinez J, Guerrero T. Ventilation from four-dimensional computed tomography: density versus Jacobian methods. Phys Med Biol 2010;55:4661-85. [Crossref] [PubMed]
19. Carmo D, Ribeiro J, Dertkigil S, Appenzeller S, Lotufo R, Rittner L. A Systematic Review of Automated Segmentation Methods and Public Datasets for the Lung and its Lobes and Findings on Computed Tomography Images. Yearb Med Inform 2022;31:277-95. [Crossref] [PubMed]
20. Tan W, Liu P, Li X, Xu S, Chen Y, Yang J. Segmentation of lung airways based on deep learning methods. IET Image Process 2022;16:1444-56.
21. Tan W, Zhou L, Li X, Yang X, Chen Y, Yang J. Automated vessel segmentation in lung CT and CTA images via deep neural networks. J Xray Sci Technol 2021;29:1123-37. [Crossref] [PubMed]
22. Busayarat S, Zrimec T. Detection of Bronchopulmonary Segments on High-Resolution CT--Preliminary Results. Twentieth IEEE International Symposium on Computer-Based Medical Systems (CBMS'07); 20-22 June 2007; Maribor, Slovenia. IEEE; 2007:199-204.
23. Tang H, Zhang C, Xie X. Automatic pulmonary lobe segmentation using deep learning. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019); 08-11 April 2019; Venice, Italy. IEEE; 2019:1225-8.
24. Hofmanninger J, Prayer F, Pan J, Röhrich S, Prosch H, Langs G. Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem. Eur Radiol Exp 2020;4:50. [Crossref] [PubMed]
25. Milletari F, Navab N, Ahmadi SA. V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth International Conference on 3D Vision (3DV); 25-28 October 2016; Stanford, CA, USA. IEEE; 2016:565-71.
26. Pu J, Fuhrman C, Good WF, Sciurba FC, Gur D. A differential geometric approach to automated segmentation of human airway tree. IEEE Trans Med Imaging 2011;30:266-78. [Crossref] [PubMed]
27. Garcia-Uceda A, Selvan R, Saghir Z, Tiddens HAWM, de Bruijne M. Automatic airway segmentation from computed tomography using robust and efficient 3-D convolutional neural networks. Sci Rep 2021;11:16001. [Crossref] [PubMed]
28. Kerschnitzki M, Kollmannsberger P, Burghammer M, Duda GN, Weinkamer R, Wagermaier W, Fratzl P. Architecture of the osteocyte network correlates with bone material quality. J Bone Miner Res 2013;28:1837-45. [Crossref] [PubMed]
29. Gu S, Wang Z, Siegfried JM, Wilson D, Bigbee WL, Pu J. Automated lobe-based airway labeling. Int J Biomed Imaging 2012;2012:382806. [Crossref] [PubMed]
30. He H, Wang F, Wang PY, Chen P, Li WWL, Perroni G, Liu SY. Anatomical analysis of variations in the bronchus pattern of the left upper lobe using three-dimensional computed tomography angiography and bronchography. Ann Transl Med 2022;10:305. [Crossref] [PubMed]
31. Ugalde P, Camargo Jde J, Deslauriers J. Lobes, fissures, and bronchopulmonary segments. Thorac Surg Clin 2007;17:587-99. [Crossref] [PubMed]
32. Smith BM, Traboulsi H, Austin JHM, Manichaikul A, Hoffman EA, Bleecker ER, et al. Human airway branch variation and chronic obstructive pulmonary disease. Proc Natl Acad Sci U S A 2018;115:E974-81. [Crossref] [PubMed]
33. Sirajuddin A. Normal Thoracic Anatomy and Common Variants. In: Kanne J. editor. Clinically Oriented Pulmonary Imaging. Respiratory Medicine. Humana Press; 2012:1-17. doi: 10.1007/978-1-61779-542-8_1.
34. Martín-Ruiz S, Gutiérrez-Collar C, Forcén Vicente De Vera E, Bernabé-Barrios MJ, de Blas CS, Konschake M, Ramón Sañudo J, Maranillo E. The bronchial segmentation and its anatomical variations. A clinical-anatomic and bronchoscopy study. Ann Anat 2021;235:151677. [Crossref] [PubMed]
35. Zhang M, Wu Y, Zhang H, Qin Y, Zheng H, Tang W, et al. Multi-site, Multi-domain Airway Tree Modeling. Med Image Anal 2023;90:102957. [Crossref] [PubMed]