Hemodynamic property incorporated brain tumor segmentation by deep learning and density-based analysis of dynamic susceptibility contrast-enhanced magnetic resonance imaging (MRI)

Introduction

Magnetic resonance imaging (MRI) offers superb resolution and exquisite soft tissue contrast for delineating tumors and lesions. Segmentation of tumors from magnetic resonance images is an essential process to derive quantitative information that can be used for diagnosis, tumor classification, and treatment monitoring (1). Most tumor segmentation approaches are based on the spatial information and signal intensity or contrast highlighted in images obtained with different MRI data acquisition sequences (2), which provide specific tissue contrast based on their unique MRI properties, such as longitudinal T₁ and transverse T₂ relaxation times and magnetic susceptibility. Typically, T₁ or T₂ weighted MRI sequences and a fluid-attenuated inversion recovery (FLAIR) sequence with additional gadolinium (Gd) contrast-enhanced T_1-weighted imaging are used in the clinical protocol to image brain tumors. Based on the corresponding contrast of a tumor appearing in these different MRI methods in comparison to the normal tissues, segmentation of tumors can be performed using various signal processing approaches, including most state-of-the-art deep learning networks (3,4). For example, convolutional neural networks (CNN) are typically used to perform recognition tasks in images by learning complex features of tumors to define tumor regions for segmentation in MRI (5). Architectures that have found success include U-Net, V-Net, and Attention U-Net, which enable tumor segmentation in a wide variety of anatomical structures, including the brain, liver, lung, and breast (6-9). However, the majority of current tumor segmentation approaches, if not all, are mostly based on morphological images and image features to determine the boundary and volume of a tumor (10). More recently, radiomics features obtained from morphological images are used to detail subtle differences in tumor tissues and intra-tumor heterogeneity to improve the segmentation (11-13). Nevertheless, segmentation results with those methods are mostly dependent on spatial information captured in the structural MRI scans. Importantly, physiological information of tumors, especially hemodynamic properties of the tumor revealed by dynamic susceptibility contrast (DSC) enhanced MRI, is completely overlooked. DSC MRI is commonly used in brain tumor imaging to record a four-dimensional (4D) dataset with an additional temporal dimension for capturing the blood supply to different regions of the tumor (14,15). Hemodynamic parameters of the tumor and normal tissue, including blood volume and blood flow, can be derived and calculated from the time-dependent signal changes when the intravenously injected bolus of the MRI contrast agent, typically a Gd-based contrast agent, is passing through the tumor vasculature (16). Therefore, DSC MRI provides important and quantifiable hemodynamic properties of tumor tissue that is more sensitive to tumor progression, recurrence, and treatment responses than structural and morphological MRI (14). For example, a significant increase in blood flow is indicative of the progression of a tumor that is likely driven by angiogenesis, as the formation of new blood vessels is essential to sustain the metabolic requirements of the tumor (17). Clinically, DSC perfusion MRI has been increasingly used for diagnosis and monitoring of glioblastoma (GBM), which is the most common class of malignant brain tumors (18,19). Previous investigations have consistently demonstrated a positive correlation between elevated relative cerebral blood flow (rCBF) and higher grades of gliomas (20,21). Furthermore, blood vessels in GBM are distorted, evidenced by traits such as incomplete basement membranes, irregular diameters, increased permeability, and tortuous branching patterns in the vasculature (22). GBM are extremely aggressive brain tumors characterized by heterogeneous components, including hemorrhage, ischemia, gelatinous regions, and necrosis, leading to diverse enhancement patterns (23,24). Thus, DSC perfusion MRI offers tumor tissue hemodynamic information to differentiate such heterogeneity (25,26). Clearly, conventional approaches of segmenting tumors solely based on morphological and structural images, such as T₁-weighted, T₂-weighted, and FLAIR images, are unable to capture the hemodynamic features when differentiating tumor tissues from others.

The current study attempts to include tissue hemodynamic properties derived from time-dependent DSC MRI data into tumor segmentation processes to “tag” the segmented tumor tissue with physiological information that is better correlated with tumor biology and pathology. As such, we developed a new method taking advantage of both deep learning spatial segmentation techniques and hemodynamic information from perfusion MRI. By utilizing a density-based machine learning approach to analyze the DSC MRI time course profiles within a region of interest (ROI) segmented by a deep neural network, we can use the information captured by 4D DSC perfusion MRI data to further improve tumor segmentation for more accurately monitoring tumor progression and treatment responses.

Methods

We acquired MRI data from a cohort of 513 GBM patients in the University of Pennsylvania glioblastoma database (UPENN-GBM). Because of retrospective use of de-identified data from this open source available through the Cancer Imaging Archive (TCIA) at the National Cancer Institute (27-29), obtaining written informed consent from subjects was done at the original study site and not required for this study. All individuals in this database had been diagnosed with high-grade GBM and scanned at the University of Pennsylvania Health System between 2006 and 2018. Table S1 summarizes demographic and clinical information on these patients. We also selected a subset of 54 patients who had both pre-treatment scores of the Karnofsky Performance Scale (KPS), which measures neurological functions of the patients, and survival time after the 1^st tumor resection. Other clinical data, including partial or total tumor resection, isocitrate dehydrogenase 1 (IDH1) mutation status, methylguanine-DNA methyltransferase (MGMT) promoter methylation status, and predicted pseudo-progression index, were too sparse in the selected cohort of these 54 patients, thus lacking sample size and power for appropriate correlation analysis. In the deep learning analysis of the cohort of 513 patients, the test set contained the specified subset of 54 patients, while the remaining 459 patients were randomly allocated to adhere to an overall 80–20 training-test split. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

All MRI data were obtained prior to surgery using 3T MRI scanners manufactured by various companies, including Siemens (Tim/Trio, Verio, SkyraFit, MAGNETOM Vida, Numaris 4, Trio, Espree, and AvantoFit) as well as GE (Signa Discovery MR 750). Image acquisition followed the standard clinical brain tumor protocol, encompassing T₁ and T₂ spin echo imaging, T₂ FLAIR, post-contrast T₁ spin echo imaging, and DSC MRI scans. Detailed information about the specific scanners and scan parameters is available in Table S2.

We preprocessed data according to a standardized image preprocessing protocol as used by UPENN-GBM, encompassing the preprocessing of both three-dimensional (3D) structural images and 4D DSC perfusion MRI data. The images were skull-stripped, resampled to the 1-mm isotropic resolution, and diffeomorphically co-registered to the SRI24 normal adult brain atlas (30) to ensure uniform image dimension and voxel size across images collected with different acquisition sequences.

Workflow

As illustrated in Figure 1, our tumor segmentation workflow entails a multi-step process. The overall algorithm can be interpreted in two main sections: first, applying supervised machine learning techniques on structural 3D MRI data, followed by incorporating 4D perfusion data using unsupervised machine learning techniques. Data samples for the proposed algorithm are available upon request.

Figure 1 Overview of the algorithm workflow. (A) FLAIR images with tumors enhanced were used as input. (B,C) Initial tumor segmentation performed on structural data with 2D U-Net to generate the tumor mask (or ROIs) in each slice. (D) Diffeomorphic co-registration of ROI onto DSC perfusion MRI data. (E) Dimensionality reduction of DSC perfusion MRI data along time points dimension using PCA. (F) Density-based analysis of hemodynamics in resulting low-dimensional form of DSC perfusion MRI data within ROI. (G) Output segmentation after density-based analysis. 3D, three-dimensional; FLAIR, fluid-attenuated inversion recovery; 2D, two-dimensional; 4D, four-dimensional; DSC, dynamic susceptibility contrast; MRI, magnetic resonance imaging; PCA, principal component analysis; ROI, region of interest.

Initially, we performed tumor segmentation on T₂ FLAIR images from 513 patients (Figure 1A) employing a Keras-based implementation of the U-Net CNN (3). This U-Net variant was meticulously trained to identify tumor regions specifically from T₂ FLAIR images (Figure 1B) and generated a segmentation mask (Figure 1C) that was then co-registered using diffeomorphic co-registration to time-course images from DSC MRI (Figure 1D) to ensure that the tumor regions identified by the U-Net network on the T₂ FLAIR images were accurately located within the images collected with DSC MRI.

Upon establishing a ROI within the DSC MRI data, we proceeded to analyze and refine the tumor region, leveraging the temporal information embedded within the DSC MRI data using the Hierarchical Density-Based Scan (HDBScan) algorithm, a data-driven, density-based clustering method that is used to cluster data points within a collection based on a selected distance metric (9). In this experiment, each data point corresponded to a time course profile with 45 time-steps originating from individual voxels within the ROI in DSC MRI images. To optimize for computational efficiency without compromising data integrity, we applied dimensional reduction through principal component analysis (PCA) using the SciKit-Learn library (31), reducing the number of time steps per data point to 3–6 steps per data point by projecting each of 45 data points onto a lower coordinate basis while preserving >97% data variance (Figure 1E). Subsequently, using the HDBScan Python library (32), we conducted two HDBScan passes on this lower coordinate basis: the first using the Excess of Mass (EOM) method and Euclidean distance for cohesive and optimal clusters, and the second using the Leaf method and Euclidean distance for finer clustering (Figure 1F). To harmonize the outcomes of these two passes, we synthesized a new cluster by constructing cluster centroids derived from the Leaf pass of HDBScan, which established the number of clusters in the final segmentation. Data points were assigned to these resulting clusters using a nearest-neighbor algorithm, drawing from the EOM pass of HDBScan. This integration approach unified the strengths of both passes, enhancing precision by removing the outliers identified during cluster generation. It also ensured that the resulting segmentation reflects the true tumor region according to the hemodynamically active tissue informed by DSC perfusion MRI data (Figure 1G).

U-Net model

We used a CNN-based U-Net architecture to make up the structural segmentation backbone of our algorithm. Figure 2 presents the model that uses the standard encoder-decoder framework with skip connections between the corresponding feature maps of the encoder and decoder layers (6). The encoder network consists of convolutional layers in tandem with pooling operations to extract hierarchical features from the input image and output a latent feature representation. The decoder network up-samples this latent feature representation and utilizes the skip connections to combine the up-sampled features with the corresponding hierarchical features collected from the encoder network to output the backbone segmentation. We implemented this modified two-dimensional (2D) U-Net to segment each axial slice image through the entire 3D volume and then employ conventional multi-planar reconstruction to form a 3D segmentation mask of the target volume. Training was performed using the Adam optimizer and categorical cross-entropy loss (33). Intermediate evaluation of the accuracy of the segmentation output was done and presented using the Dice coefficient.

Figure 2 Diagram of the U-Net architecture used in the study.

Density-based analysis

Upon identifying the ROI output by the proposed U-Net model, we applied a density-based clustering method on the corresponding ROI to 4D DSC MRI data to refine the tumor segmentation by extracting significant data and excluding outliers, like noise, motion artifacts, or partial volume effect. Density-based clustering methods operate by identifying densely connected data points according to a selected distance, resulting in a more data-driven method as opposed to the model-driven U-Net approach. Furthermore, density-based methods are largely deterministic depending on data, minPts, ϵ, and distance function, in contrast to other methods such as k-means where different initialization commonly results in different outputs (34). ϵ refers to the size of the neighborhood for each data point; each data point in a cluster exists within the ϵ-neighborhood of its neighbors. minPts refers to the minimum number of points required for a prospective cluster to be considered a dense region. Traditional density-based spatial clustering of applications with noise (DBSCAN) algorithm is too dependent on parameters ϵ and minPts for reliable use across MRI scans without having standardization protocols, much less across clinical settings (34). To address this, we used a variation of DBSCAN called HDBScan as a self-adjusting algorithm that uses a range of ϵ distances for varying cluster densities (35).

According to HDBScan, the algorithm first computed the mutual reachability distance between each pair of data points, defined as the maximum between the core distances of each point and the distance between the points. The core distance is defined as the distance of the data point to its minPts nearest neighbor. With these mutual reachability distances, we constructed a graph representation of the data and a minimum spanning tree (MST). Sorting the edges by mutual reachability distance subsequently turns the MST into a hierarchical tree structure (dendrogram) with the largest ϵ value corresponding to the root and the smaller ϵ values corresponding to the leaf nodes. Next, the dendrogram is pruned to remove outliers. Starting from the root, each successive cluster split is retained only if both child clusters maintain a minimum of minPts points. If only one child cluster meets this criterion, the parent cluster keeps the points while pruning outliers within the child cluster that don’t satisfy the minPts condition, removing them from the dendrogram.

Given the pruned dendrogram, choosing an epsilon value would result in outputting the clusters present at the corresponding mutual reachability distance level. This doesn’t achieve clusters of varying densities; we need a metric to select clusters present at different levels of the dendrogram. To do so, as part of the EOM method, we selected clusters in a manner that maximizes stability, defined by Campello et al. (9) to be

S(Ci)=∑xj∈Ci(λmax(xj,Ci)−λmin(Ci))=∑xj∈Ci1εmin(xj,Ci)−1εmax(Ci)[1]

where λ is set to 1∈. This equation inherently prioritizes clusters that exhibit greater persistence across a broader spectrum of ϵ values as they are ostensibly stable. The algorithm then traversed the dendrogram from the leaf nodes upwards. It assessed the stability at each node with the objective of maximizing the cumulative sum of cluster stabilities. This process culminated in the selection of clusters that correspond optimally to significant data points within the 4D DSC MRI volume. Simultaneously, it efficiently pruned extraneous noise or less significant data points in accordance with the principles of the HDBScan algorithm.

Contour complexity

To compare the precision and localized predictions between the reference standard and the refined segmentation mask output by our algorithm, we compared the boundary complexities from each resultant mask. Boundary complexity can be roughly quantified using the surface area-to-volume ratio, where a higher ratio indicates a more intricate boundary structure. However, surface area-to-volume ratios increase as the effective radius becomes smaller for the same shape. Therefore, we used the effective radius of the contours, calculated by approximating the contour as a nonuniform sphere and deriving from the volume of the contour, to attenuate this intrinsic decrease in surface area to volume ratios. The effective radius is calculated as:

R=3V4π3[2]

This can be quickly derived from the equation for the volume of a sphere. We defined the increase in contour complexity (ICC) as the ratio of the surface area to volume ratios of the two segmented masks weighted by the effective radii of the segmented masks as follows:

ICC=(SA2V2SA1V1⋅R2R1)−1[3]

SA₁ and V₁ refer to the surface area and volume of the initial segmentation mask output by U-Net, respectively, while SA₂ and V₂ refer to the same for the refined segmentation mask output by the algorithm. R₂ and R₁ refer to the effective radius of the initial segmentation mask and the refined segmentation mask, respectively.

Performance evaluation and Pearson correlation analysis

To compare with current standard segmentation methodologies, we used the automated reference standard supplemented by manual revisions as provided by the UPENN-GBM dataset. While this reference standard is constrained by its reliance solely on 3D structural data and is susceptible to intra- and inter-observer variability, it does reflect the prevailing clinical practice for segmentation rooted in structural MRI. We also used nnU-Net and standard 2D U-Net models trained using the reference standard as ground truth for quantitative comparison to our proposed approach (36). Quantitative assessment was conducted using the Jaccard Index, or Intersection over Union (IoU), defined as the intersection of two regions divided by their union, and precision, defined as the ratio of true positives to the sum of the true positives plus false positives. We used these metrics to characterize the overlap between the proposed segmentation and the reference standard masks.

To assess the clinical relevance and possible improvement of the proposed segmentation method, we also examined the relationship between the tumor volume from the segmented tumor regions and the selected clinical outcome measurements available in the UPENN-GBM dataset. Using 54 cases that have both MRI data and documented information on KPS and post-surgery survival days, we performed a bivariate Pearson correlation analysis using the software IBM Statistical Product and Service Solutions (SPSS) Statistics 26.0 (RRID:SCR_016479) between segmented tumor volumes obtained from either our approach or the reference standard provided by the UPENN-GBM dataset with KPS and post-surgery survival days. Statistically significant relationships between these clinical data and segmented tumor volumes can be found from the Pearson correlation coefficient (r) defined as follows:

rx,y=covx,yσxσy[4]

where cov_x,y is the sample covariance of x and y, and σ_x, σ_y are the sample standard deviations (SDs) of x and y. A two-tailed significance level of P<0.05 was applied. Correlation between KPS and duration of survival after surgery was performed as a control to validate that the correlation analysis would detect a significant correlation between the two conditions.

Results

The reported approach by incorporating signal changes derived from the DSC-MRI time course data yielded not only tumor region segmentation akin to conventional methods but also accentuated tumor tissue by providing voxel-level hemodynamic information and distinctions, as the example shows in Figure 3. Notably, the tumor regions segmented using the reported approach for including time-dependent hemodynamic data from DSC perfusion MRI are different from those obtained using the standard 2D U-Net method without including the hemodynamic information (Figure 3A,3B). This is not surprising but anticipated as segmentation relying solely on traditional T₁, T₂-weighted, and FLAIR images is predominantly based on morphological features, which provides the spatial framework for the reported methods. Importantly, intratumoral heterogeneity, which is a hallmark of GBM and many other aggressive solid tumors, is delineated in the segmented tumors (Figure 3C).

Figure 3 An example of a U-Net segmented tumor region on the axial slice using a case of diagnosed IDH1-wildtype GBM (female, 64-year-old). The tumor is in the left frontal lobe with an indistinct mass effect. Tumor region before refinement (A) after refinement with HDBScan (B), and (C) 3D projection of time courses corresponding to voxels where purple points represent outlier voxels present in tumor region (A) but not (B). IDH1, isocitrate-dehydrogenase 1 gene; GBM, glioblastoma; HDBScan, Hierarchical Density-Based Scan; 3D, three-dimensional.

U-Net and density-based analysis

Compared with the 2D U-Net backbone, we retained an average of 67.93% of the voxels within the ROIs with a SD of 13.98% across the cohort of 513 subjects in total using our current density-based approach (Table 1). This suggests that compared to just using the U-Net architecture, which relies on 3D spatial information and contrast seen in FLAIR images, the reported approach, which utilizes 4D spatiotemporal information present in DSC MRI imaging, enhances the segmentation by reducing the size of the backbone segmentation by 32.05% on average by eliminating voxels with insignificant hemodynamic activity seen in perfusion MRI data. According to the automated reference standard provided by the UPENN-GBM dataset, the final segmentation output by our approach reached a 74.83% IoU score on average compared to 83.16% IoU score using the U-Net approach alone and 87.004% IoU using nnU-Net with 5-fold cross-validation (Table 1). We improved the precision from 92.63% with U-Net alone and 92.76% with nnU-Net to 94.57% using our reported approach, indicating an overall enhancement of precision in segmentation when combining 3D structural MRI data with additional hemodynamic information of the tumor from 4D DSC perfusion MRI.

Interestingly, and perhaps significantly, the obtained IoU is lower than the IoUs resulting from both the initial U-Net segmentation and the nnU-Net segmentation, signifying that perfusion MRI discerns information that is either absent or, at the very least, unique from what is acquired through 3D structural MRI. This observation is significant as prevailing tumor segmentation methods, which underpin the reference standard, lack hemodynamic analysis. This information could be instrumental in further refining the existing reference standard for segmentations reliant on structural images or vice versa.

A notable feature of tumors segmented with additional hemodynamic information from DSC perfusion MRI is better delineation of intratumoral heterogeneity, as shown in Figure 4. In addition to the morphological heterogeneity highlighted by FLAIR images, the new segmentation approach revealed intratumoral physiological or hemodynamic heterogeneity based on the differences in blood perfusion and supplies to tumor tissues. The number of subregions can be highlighted based on the differences between the time course profiles of the DSC MRI data collected in tumors, which are unique for individual cases. Consistent with previous studies (23-26), our findings support the notion that blood supply to the tumor is diversely different and dependent on the type of tumor tissues and regions, evidenced by the multiple subregions identified by our algorithm. Each subregion corresponds to a separate and dense cluster within the time course data of perfusion MRI, signifying discernable patterns of blood perfusion or hemodynamic differences within the tumor.

Figure 4 An example of the intratumoral heterogeneity revealed by the segmentation approach based on a IDH1-wildtype GBM tumors in a 64-year-old patient (female), also referenced in Figure 3. (A) Results of segmentation with tumor tissue subtypes highlighted by the cluster ID on the FLAIR image. (B) Average time course profiles of each cluster highlighted in (A) matched by cluster ID corresponding to tumor tissue subtypes found after analyzing hemodynamics in DSC MRI data. Notably, cluster 3 represents a cluster that would be classified as insignificant/inconclusive. IDH1, isocitrate-dehydrogenase 1 gene; GBM, glioblastoma; FLAIR, fluid-attenuated inversion recovery; DSC, dynamic susceptibility contrast; MRI, magnetic resonance imaging.

Contour complexity

Using our current segmentation approach, the generated masks featured intricate, non-linear boundaries rather than smoother, convex hull-like boundaries using the reference standard, as exemplified in Figure 5. When measuring the contour complexity of the final segmentation masks as defined in our methods, we find that the final segmentation had an average of 146.92% ICC when compared to the reference standard, indicating a quantifiable increase in the complexity of the segmentation boundaries. While greater contour complexity does not inherently equate to higher accuracy in tumor segmentation, it does indicate a more refined level of tumor segmentation based on the added hemodynamic information included in our proposed method. In Figure 5A, 5B, and 5E, the tumor mask exhibits a relatively smooth, convex hull-like structure in the reference standard derived from 3D structural MRI. Our proposed approach, on the other hand, can exclude tissues within the convex region and yields more precise and localized predictions by leveraging hemodynamic information from 4D DSC MRI. This refinement, particularly evident within central regions of the tumors, leads to increased segmentation boundary complexity compared to the reference standard.

Figure 5 Selected axial slices from nine different patients with final tumor segmentation mask and reference standard segmentation mask overlaid on FLAIR images. Note case (D) and (F), where the reported algorithm excludes the necrotic core (arrows indicated), and case (G), where the algorithm focuses on the enhancing tumor region (arrow indicated). (A,D,F,G-I) are IDH1-wildtype GBM. (B,C,E) and IDH1-NOS/NEC GBM. FLAIR, fluid-attenuated inversion recovery; IDH1, isocitrate-dehydrogenase 1 gene; GBM, glioblastoma; NOS, not otherwise specified; NEC, not elsewhere classified.

In our case, the reference standard masks that act as the “ground truth” exhibited a preference for contiguous ROIs. Consequently, the supervised learning models, including nnU-Net, U-Net, and other clinically accepted 3D structural-based segmentation networks, may inadvertently encompass insignificant data points within the ROI due to inherent neural network bias aligned with the characteristics of the ground truth they are trained on, whereas our proposed algorithm, density-based, data-driven, and utilizing hemodynamics from 4D perfusion data, avoided this bias. Notably, it was challenging for our approach to segment necrotic core portions, as shown in Figure 5D,5F, which could be due to the absence of functional blood vessels within the necrotic core of GBM and, thus, no signal response collected from perfusion MRI. The algorithm also appeared to emphasize enhancing tumor regions (Figure 5G), indicating a corresponding hemodynamic response in 4D DSC perfusion data that may not be easily identified in 3D FLAIR data.

Pearson correlation analysis

In our Pearson correlation analysis with 54 subjects included in the current study, we found a significant correlation between KPS and the total tumor volume measured by our segmentation method (r=−0.309, P=0.023), while no significance was observed in comparison with the reference standard (Table 2, Figure 6A). The observed negative correlation suggests that patients with larger tumors suffered a more pronounced functional decline. Neither segmented tumor volumes showed statistically significant correlations with the reported survival days after tumor resection surgery. When we examined the bivariate correlation between KPS and survival days after surgery, which was performed as a control for testing the sensitivity of the correlation analysis with KPS, we observed a significant correlation between these two clinical outcome measurements (r=0.375, P=0.005), indicating high relevancy and reliability of KPS for evaluating the outcome of GBM (Figure 6B).

Figure 6 Graphical representation of significant results from correlation analysis on 54 subjects with line of best fit and corresponding Pearson correlation (r) and significance (P). (A) Correlation between KPS and tumor volume produced by proposed method. (B) Correlation between KPS and days of survival from surgery (control). KPS, Karnofsky Performance Scale.

Discussion

We demonstrated that using the temporal contrast changes in 4D DSC MRI data adds tumor tissue hemodynamic responses into the tumor segmentation. Conventional tumor segmentation approaches and most new deep learning-based methods typically use spatial and morphological information of structural MRI with different contrast, leaving out DSC MRI that has been routinely included in the clinical brain tumor imaging protocol. GBM typically presents high intra-tumoral heterogeneity, which is further complicated after the resection or chemo- or radiation treatment (25,26). Sole reliance on structural MRI, such as T₁-weighted, T₂-weighted, and FLAIR images, may result in overestimation and inaccurate evaluation of tumor growth and infiltration, given the presence of various forms of brain edema, including cytotoxic, vasogenic, and interstitial edema (11,12). The increased contour complexity using our proposed approach suggests that the inclusion of tissue physiological information from perfusion MRI effectively segregates or excludes voxels lacking hemodynamically active tissue. Furthermore, our algorithm excels at eliminating extraneous data points within the segmentation boundary, resulting in intricate segmentation masks characterized by greater precision and localized predictions, a departure from the smoother, more contiguous segmentation masks produced by prior methods. These extraneous data points may stem from factors such as excised tumor regions, visual artifacts during data acquisition, and other sources of noise impacting image quality. While DSC MRI data does not have high enough resolution to structurally identify most of these abnormal traits, the increased permeability disrupts the blood-brain barrier, resulting in extravasation of contrast agents, which is quantitatively measurable with DSC MRI data (17). The hemodynamic information provided by DSC MRI is especially useful in distinguishing radiation necrosis vs. tumor recurrence and progression (24,37). Therefore, the reported hemodynamics-incorporated segmentation approach potentially provides accurate differentiation of viable tumor tissues from other tissue types for quantitative measurements of tumor size/volume for monitoring tumor progression and treatment responses.

Our algorithm uses a 2D U-Net implementation instead of a 3D U-Net for initial structural segmentation on 3D data. Valid concerns include performance degradation and inconsistent segmentation across adjacent slices, whereas 3D U-Net would output a well-formed mask across slices. However, variable slice thickness across clinical applications in MRI makes a 2D U-Net preferable, as 2D U-Net can accommodate variable slice thickness if the 2D slice resolutions are consistent. Additionally, as spatial resolution increases with recent advances pushing sub-millimeter resolution, training a 2D U-Net is much more efficient when compared to training a 3D U-Net. Given the need for applicability in various clinical settings, the practical benefits of a 2D U-Net implementation, particularly in addressing variable slice thickness, are still viable and adequate.

The current proof-of-concept study needs further optimization. In our study, initial segmentation is conducted using a U-Net implementation based on the information available in 3D structural images. While the addition of hemodynamic information from 4D perfusion MRI data enabled the identification of physiologically active tumor tissues and intratumor hemodynamic heterogeneity, these regions are situated within the ROIs delineated by the 3D structural data using conventional intensity-only based segmentation. As such, there exists a compelling need to optimize the backbone segmentation process to provide more dependable ROIs for the incorporation of 4D DSC perfusion MRI data. Optimizations should consider the challenges associated with working with dynamic image datasets, which are more susceptible to variations in data quality and artifacts. Given increased computational demands in dealing with complex 4D data, a simple approach is to expand U-Net’s initial ROI boundaries perpendicularly. However, further research is necessary to validate the effectiveness of artificially increasing boundaries solely based on spatial considerations and determine the optimal method for expanding the boundary while preserving tumor segmentation integrity.

Worth noting is that we opted to use unsupervised learning techniques over well-established supervised learning techniques due to consideration of the inherent constraints of supervised techniques, which heavily rely on the prior ground truth annotations. Our focus is on harnessing blood perfusion information from DSC MRI data to improve tumor segmentation. Through the utilization of unsupervised techniques, which operate independently of using a reference standard as a guiding truth, we can analyze hemodynamic information to surpass the limitations of the reference standard provided by UPENN-GBM, which are rooted in 3D structural MRI imaging. Using 3D structural MRI as a foundational basis is a pattern consistent with annotations of other publicly available datasets, such as those provided as part of the Brain Tumor Segmentation Challenge (BraTS), which is the leading benchmark for evaluation and comparison of new brain segmentation techniques.

The current study used GBM perfusion MRI cases from the open-source data from the Cancer Imaging Archive. Thus, the retrospectively collected clinical data naturally are diverse. Reference standard segmentation methods used by the UPENN-GBM database were based on 3D structural images with manual revisions. They are generally well annotated but prone to intra- and inter-observer variability (27). Our algorithm is designed to refine tumor segmentation masks from existing methods and clarify uncertainties such as tissue boundaries or mitigating difficulties arising from visual assessment. While our proposed algorithm correlates well with clinical data, such as KPS scores, compared to the reference standard, robust validation against a “ground truth” is necessary to reinforce the significance of the advantages brought by the incorporation of perfusion data before warranting adoption in clinical settings. Density-based clustering can identify outliers throughout the tumor region output by the backbone structural segmentation network, not just at the boundaries, making reconciliation with asynchronous expert review difficult as manual expert review likely favors single convex bounded volumes over complex volumes with high granularity.

Intratumoral heterogeneity is a feature of solid tumors that can be considered as tissue level variations of tumor structural and biological properties. Intratumoral heterogeneity may reflect tissues or cells with distinct genotypes and phenotypes that exhibit divergent biological behaviors and functions and, therefore, need to be diagnosed and characterized for targeted or precision treatment. From a cancer imaging point of view and normally in radiology clinical practice, intratumoral heterogeneity is typically reported by observations of tissue-type specific or related different imaging contrast, for example, features shown in FLAIR images. However, those contrast differences or heterogeneity are based on structural imaging that only can show the steady state morphological differences of tumor tissue. On the other hand, all morphological differences should have their physiological bases or correlates, e.g., differences in blood supply as reported by DSC MRI. We, as well as others, have proposed and shown the hemodynamic heterogeneity of brain tumors (38-40) in previous studies. The dynamic and complex information from DSC MRI is difficult to incorporate into tumor segmentation. However, it is more physiologically relevant to tumor biological properties. We anticipate that future studies and method development will lead to tools and solutions for comprehensive imaging characterizations of tumor heterogeneity morphologically and physiologically.

The slices presented in the figures were chosen to showcase the segmentation output of our proposed algorithm and highlight drawbacks identified during our analysis. For a more comprehensive assessment, processed data from patients not featured in the manuscript can be provided upon request from the corresponding authors.

Conclusions

By harnessing the hemodynamic information from DSC perfusion MRI data, the newly reported segmentation approach can effectively detect and identify voxels that contain physiologically distinct tumor tissue, which are not distinguishable by conventional segmentation methods that only use structural or morphological images. This new capability enables tissue physiology-informed tumor segmentation, thus enhancing the segmentation accuracy when combined with morphologic feature-based segmentation. Our findings indicate that exploring the tumor hemodynamic characteristics may provide quantitative MRI measurements that are more sensitive to tumor progression and treatment response, evidenced by outperforming the reference standard when correlated with clinical measurements of patient physiological and functional conditions. This approach has the potential to change clinical practices on brain tumor imaging, and furthermore, can be applied to other clinical indications that use perfusion MRI to assess hemodynamic properties of the tissue, such as stroke.

Acknowledgments

Funding: This work was supported in part by a grant from the National Institutes of Health (No. R01CA169937 to H.M.).

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-23-1471/coif). H.M. reports that this study was supported in part by the National Institutes of Health (No. R01CA169937). The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).Because of retrospective use of these open source and de-identified data from the UPENN-GBM dataset, obtaining written informed consent from subjects was done at the original study site and not required for this study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Wadhwa A, Bhardwaj A, Singh Verma V. A review on brain tumor segmentation of MRI images. Magn Reson Imaging 2019;61:247-59. [Crossref] [PubMed]
2. Fawzi A, Achuthan A, Belaton B. Brain Image Segmentation in Recent Years: A Narrative Review. Brain Sci 2021;11:1055. [Crossref] [PubMed]
3. Shaukat Z, Farooq QUA, Tu S, Xiao C, Ali S. A state-of-the-art technique to perform cloud-based semantic segmentation using deep learning 3D U-Net architecture. BMC Bioinformatics 2022;23:251. [Crossref] [PubMed]
4. Saeedi S, Rezayi S, Keshavarz H. R Niakan Kalhori S. MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques. BMC Med Inform Decis Mak 2023;23:16. [Crossref] [PubMed]
5. Yamashita R, Nishio M, Do RKG, Togashi K. Convolutional neural networks: an overview and application in radiology. Insights Imaging 2018;9:611-29. [Crossref] [PubMed]
6. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Lecture Notes in Computer Science 2015;9351:234-41.
7. Milletari F, Navab N, Ahmadi SA. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, 2016:565-71.
8. OktayOSchlemperJFolgocLLLeeMHeinrichMMisawaKMoriKMcDonaghSHammerlaNYKainzBGlockerBRueckertD. Attention U-Net: Learning Where to Look for the Pancreas. arXiv:1804.03999.
9. Campello RJGB, Moulavi D, Sander J. Density-Based Clustering Based on Hierarchical Density Estimates. Pacific-Asia conference on knowledge discovery and data mining. Lecture Notes in Computer Science 2013;7819:160-72.
10. Mohammed YMA, El Garouani S, Jellouli I. A survey of methods for brain tumor segmentation-based MRI images. J Comput Des Eng 2023;10:266-93.
11. Eloyan A, Yue MS, Khachatryan D. Tumor heterogeneity estimation for radiomics in cancer. Stat Med 2020;39:4704-23. [Crossref] [PubMed]
12. Fan M, Chen H, You C, Liu L, Gu Y, Peng W, Gao X, Li L. Radiomics of Tumor Heterogeneity in Longitudinal Dynamic Contrast-Enhanced Magnetic Resonance Imaging for Predicting Response to Neoadjuvant Chemotherapy in Breast Cancer. Front Mol Biosci 2021;8:622219. [Crossref] [PubMed]
13. Moradmand H, Aghamiri SMR, Ghaderi R. Impact of image preprocessing methods on reproducibility of radiomic features in multimodal magnetic resonance imaging in glioblastoma. J Appl Clin Med Phys 2020;21:179-90. [Crossref] [PubMed]
14. Boxerman JL, Quarles CC, Hu LS, Erickson BJ, Gerstner ER, Smits M, et al. Consensus recommendations for a dynamic susceptibility contrast MRI protocol for use in high-grade gliomas. Neuro Oncol 2020;22:1262-75. [Crossref] [PubMed]
15. Schmainda KM, Prah MA, Rand SD, Liu Y, Logan B, Muzi M, et al. Multisite Concordance of DSC-MRI Analysis for Brain Tumors: Results of a National Cancer Institute Quantitative Imaging Network Collaborative Project. AJNR Am J Neuroradiol 2018;39:1008-16. [Crossref] [PubMed]
16. Zhou Z, Lu ZR. Gadolinium-based contrast agents for magnetic resonance cancer imaging. Wiley Interdiscip Rev Nanomed Nanobiotechnol 2013;5:1-18. [Crossref] [PubMed]
17. Henriksen OM, Del Mar Álvarez-Torres M, Figueiredo P, Hangel G, Keil VC, Nechifor RE, Riemer F, Schmainda KM, Warnert EAH, Wiegers EC, Booth TC. High-Grade Glioma Treatment Response Monitoring Biomarkers: A Position Statement on the Evidence Supporting the Use of Advanced MRI Techniques in the Clinic, and the Latest Bench-to-Bedside Developments. Part 1: Perfusion and Diffusion Techniques. Front Oncol 2022;12:810263. [Crossref] [PubMed]
18. van Dijken BRJ, van Laar PJ, Smits M, Dankbaar JW, Enting RH, van der Hoorn A. Perfusion MRI in treatment evaluation of glioblastomas: Clinical relevance of current and future techniques. J Magn Reson Imaging 2019;49:11-22. [Crossref] [PubMed]
19. Schmainda KM, Zhang Z, Prah M, Snyder BS, Gilbert MR, Sorensen AG, Barboriak DP, Boxerman JL. Dynamic susceptibility contrast MRI measures of relative cerebral blood volume as a prognostic marker for overall survival in recurrent glioblastoma: results from the ACRIN 6677/RTOG 0625 multicenter trial. Neuro Oncol 2015;17:1148-56. [Crossref] [PubMed]
20. Hakyemez B, Erdogan C, Ercan I, Ergin N, Uysal S, Atahan S. High-grade and low-grade gliomas: differentiation by using perfusion MR imaging. Clin Radiol 2005;60:493-502. [Crossref] [PubMed]
21. Aydin S, Fatihoglu E, Kosar PN, Ergun E. Perfusion and permeability MRI in glioma grading. Egypt J Radiol Nucl Med 2020;51:2.
22. Cribaro GP, Saavedra-López E, Romarate L, Mitxitorena I, Díaz LR, Casanova PV, Roig-Martínez M, Gallego JM, Perez-Vallés A, Barcia C. Three-dimensional vascular microenvironment landscape in human glioblastoma. Acta Neuropathol Commun 2021;9:24. [Crossref] [PubMed]
23. Maral H, Ertekin E, Tunçyürek Ö, Özsunar Y. Effects of Susceptibility Artifacts on Perfusion MRI in Patients with Primary Brain Tumor: A Comparison of Arterial Spin-Labeling versus DSC. AJNR Am J Neuroradiol 2020;41:255-61. [Crossref] [PubMed]
24. Jafari-Khouzani K, Emblem KE, Kalpathy-Cramer J, Bjørnerud A, Vangel MG, Gerstner ER, Schmainda KM, Paynabar K, Wu O, Wen PY, Batchelor T, Rosen B, Stufflebeam SM. Repeatability of Cerebral Perfusion Using Dynamic Susceptibility Contrast MRI in Glioblastoma Patients. Transl Oncol 2015;8:137-46. [Crossref] [PubMed]
25. Broka A, Hysenaj Z, Sharma S, Rehmani R. Lion in Sheep's Clothing: Glioblastoma Mimicking Intracranial Hemorrhage. Cureus 2021;13:e14212. [Crossref] [PubMed]
26. Liao W, Liu Y, Wang X, Jiang X, Tang B, Fang J, Chen C, Hu Z. Differentiation of primary central nervous system lymphoma and high-grade glioma with dynamic susceptibility contrast-enhanced perfusion magnetic resonance imaging. Acta Radiol 2009;50:217-25. [Crossref] [PubMed]
27. Bakas S, Sako C, Akbari H, Bilello M, Sotiras A, Shukla G, et al. The University of Pennsylvania glioblastoma (UPenn-GBM) cohort: advanced MRI, clinical, genomics, & radiomics. Sci Data 2022;9:453. [Crossref] [PubMed]
28. Bakas S, Sako C, Akbari H, Bilello M, Sotiras A, Shukla G, et al. Multi-parametric magnetic resonance imaging (mpMRI) scans for de novo Glioblastoma (GBM) patients from the University of Pennsylvania Health System (UPENN-GBM). The Cancer Imaging Archive (TCIA) Public Access 2021. Available online: https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=70225642
29. Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M, Tarbox L, Prior F. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 2013;26:1045-57. [Crossref] [PubMed]
30. Rohlfing T, Zahr NM, Sullivan EV, Pfefferbaum A. The SRI24 multichannel atlas of normal adult human brain structure. Hum Brain Mapp 2010;31:798-819. [Crossref] [PubMed]
31. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E. Scikit-learn: Machine Learning in Python. J Mach Learn Res 2011;12:2825-30.
32. McInnes L, Healy J, Astels S. hdbscan: Hierarchical density based clustering. J Open Source Softw 2017;2:205.
33. KingmaDPBaJ. Adam: A Method for Stochastic Optimization. arXiv:1412.6980.
34. Ester M, Kriegel HP, Sander J, Xu X. A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, 1996:226-31.
35. Malzer C, Baum M. A Hybrid Approach To Hierarchical Density-based Cluster Selection. 2020 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Karlsruhe, Germany, 2020:223-8.
36. Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 2021;18:203-11. [Crossref] [PubMed]
37. Shiroishi MS, Castellazzi G, Boxerman JL, D'Amore F, Essig M, Nguyen TB, Provenzale JM, Enterline DS, Anzalone N, Dörfler A, Rovira À, Wintermark M, Law M. Principles of T2 *-weighted dynamic susceptibility contrast MRI technique in brain tumor imaging. J Magn Reson Imaging 2015;41:296-313. [Crossref] [PubMed]
38. Ji B, Wang S, Liu Z, Weinberg BD, Yang X, Liu T, Wang L, Mao H. Revealing hemodynamic heterogeneity of gliomas based on signal profile features of dynamic susceptibility contrast-enhanced MRI. Neuroimage Clin 2019;23:101864. [Crossref] [PubMed]
39. Jeong J, Wang L, Ji B, Lei Y, Ali A, Liu T, Curran WJ, Mao H, Yang X. Machine-learning based classification of glioblastoma using delta-radiomic features derived from dynamic susceptibility contrast enhanced magnetic resonance images: Introduction. Quant Imaging Med Surg 2019;9:1201-13. [Crossref] [PubMed]
40. Liu Z, Yao B, Wen J, Wang M, Ren Y, Chen Y, Hu Z, Li Y, Liang D, Liu X, Zheng H, Luo D, Zhang N. Voxel-wise mapping of DCE-MRI time-intensity-curve profiles enables visualizing and quantifying hemodynamic heterogeneity in breast lesions. Eur Radiol 2024;34:182-92. [Crossref] [PubMed]