Automated and quantitative segmentation of colorectal polyps via an information-guided deep learning network with boundary perception

Introduction

Colorectal cancer (CRC) remains a formidable challenge in global healthcare, consistently ranking as a top contributor to cancer-related death worldwide (1,2). The consensus among clinicians is that the most effective intervention for CRC involves the early identification and removal of its precursors—colorectal polyps. Although colonoscopy is the gold standard for implementing this strategy (3), it entails certain drawbacks. In real-world clinical settings, the physician’s view is often obscured by lighting effects or intestinal debris. Moreover, polyps vary wildly in shape and size, and their borders often blend into the surrounding tissue. This makes diagnosis heavily reliant on the perceptiveness and experience of the physician, which, unfortunately, can lead to a significant number of polyps being missed. Automated segmentation offers a solution to this issue, serving as a much-needed digital safety net to enhance diagnostic accuracy.

In the development of reliable automated tools, there have been two primary eras: the early period of hand-crafted features and the modern shift toward deep learning. Initially, researchers in this field attempted to define polyps using fixed geometric rules, considering factors such as shape, texture, and color. For example, Hwang et al. (4) focused on detecting polyps using watershed algorithms and ellipse fitting, while Tajbakhsh et al. (5) and Sánchez-González et al. (6) applied shape-context descriptors and edge-color cues. These methods were groundbreaking at the time, but neither provided sufficient flexibility and often failed to provide satisfactory performance in segmenting polyps in real-world clinical contexts. The emergence of deep learning (7), specifically the U-Net architecture (8), marked a significant breakthrough in the field. In this approach, skip connections are applied to bridge the gap between high-level features and fine-grained spatial detail. Since its initial development, several notable iterations have been devised: Patel et al. (9) added semantic and adaptive modules to capture and emphasize challenging fine-grained details, and Dai et al. (10) experimented with dual-path designs to more effectively utilize feature information. More recently, the development of SSFormer (11) further innovated this field by exchanging traditional convolutional neural networks for vision transformers, allowing the model to capture global dependencies and fine-grained details simultaneously.

Despite the performance in improvement, the boundary problem remains a persistent issue for most models. Although a significant number of methods are highly capable of identifying the polyp’s core, discerning where the polyp ends and healthy tissue begins remains challenging. Because these edges are often extremely faint (low contrast), models frequently over- or underestimate the boundaries, resulting in erroneous edge predictions. In clinical contexts, in which millimeter-level precision is needed for safe biopsy or resection, this lack of accuracy is highly problematic. To address this, parallel reverse attention network (PraNet) (12) was recently developed and introduced a reverse attention mechanism to help the model explicitly analyze the relationship between the object and its edge. This idea was refined in the construction of context-aware residual attention network (CaraNet) (13)—which uses axial attention, boundary refinement network (BRNet) (14)—which includes a dedicated module to generate explicit boundary maps for guidance, and boundary uncertainty aware network (BUNet) (15)—which employs uncertainty-aware regions. Despite these advances, a robust means to handling extremely adverse scenarios, in which boundaries are nearly invisible or the morphology is exceptionally complex, remains lacking. Indeed, the currently available models still tend to underperform when the visual cues are highly ambiguous.

We therefore developed the information-guided deep learning network (IGBP-Net) to address these specific issues. Unlike previous methods that treat boundary extraction as an isolated auxiliary task, IGBP-Net provides guidance that links spatial localization and boundary perception, ensuring that boundary cues are strictly extracted from target-adjacent regions and do not include background noise. The principal contributions of our study are as follows:

• Information-guided boundary perception: we developed a novel mechanism that explicitly couples spatial localization with boundary extraction. An information aggregation module (AGG) generates initial localization cues to dynamically guide the boundary-perception module (BPM). Equipped with multiscale strip convolutions, the BPM geometrically aligns with irregular polyp contours, capturing fine-grained edges exclusively from target-adjacent regions and preventing interference from background noise.
• Uncertainty-aware region fusion: to bridge boundary and region features, we designed the region fusion module (RFM). In contrast to simple feature concatenation, this module mathematically determines correlation between boundary priors and internal region semantics. By specifically targeting and reinforcing uncertainty-prone areas, the RFM iteratively rectifies structural errors and refines the final segmentation maps from deep to shallow layers.
• Contextual feature refinement: to robustly support the aforementioned boundary-region interactions, we introduced the global and local feature extraction module (GL) and the channel multiscale module (CMM) as supporting mechanisms. By balancing the global semantic context with local details and mining features along the channel dimension, these modules efficiently suppress background noise, ensuring the intermediate representations provide a highly reliable foundation for the core network.

We present this article in accordance with the TRIPOD+AI reporting checklist (available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0197/rc).

Methods

Datasets

The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. To assess the performance of IGBP-Net, experiments were conducted on four public datasets: Kvasir (16), Computer Vision Center Clinic Database (CVC-ClinicDB) (17), Computer Vision Center Colon Database (CVC-ColonDB) (5), and ETIS-Larib Polyp DB (ETIS) (18). The data setup in our study included the same standard protocol used in PraNet (12) and other recent studies (19,20). The training set consisted of 1,450 randomly selected images, with 900 from Kvasir and 550 from CVC-ClinicDB. The test set included the remaining images from these two groups, along with the full CVC-ColonDB and ETIS datasets, for a total of 798 images. Because image resolutions vary, all inputs were resized to 352×352 pixels to match the parameters of PraNet (11,12,21) and ensure a fair comparison. Finally, to increase diversity and limit overfitting, random data augmentation techniques—including Gaussian blur, flipping, and color jittering—were applied during training. The boundary ground truths were consistently derived offline from the regional masks via a standard morphological pipeline. Specifically, we applied a Canny edge detector (with minimum and maximum thresholds set to 50 and 150, respectively) to extract the initial single-pixel contour. This was followed by a morphological dilation operation for one iteration with a 4×4 square kernel to create the final thickened boundary region, providing stable spatial supervision for the network.

Overall structure of the model

Figure 1 illustrates the overall architecture of the proposed IGBP-Net. Built upon a classic encoder–decoder layout with a multistage deep supervision strategy, the network is driven by our core modeling insight: the need for guidance of spatial localization and boundary perception. Pyramid vision transformer version 2 (PVTv2) (22) serves as the backbone and extracts hierarchical multiscale features, denoted as Fi (i=1,2,3,4), from the input image. Rather than treating region and boundary extraction as parallel, isolated tasks, we establish an interaction mechanism. First, the AGG integrates deep-level features (F₂, F₃, F₄) to synthesize a coarse segmentation map S₅, which provides initial spatial localization for the polyp targets. Crucially, this localization explicitly guides the BPM. By combining this spatial guidance with the shallow feature F₁ for structural details, the BPM captures precise boundary cues (S_b) strictly from target-adjacent regions, thereby preventing background noise interference. For robust support of this core boundary-region interaction, the feature representations undergo hierarchical contextual refinement. Specifically, the GL module processes F₂ and F₃ from complementary perspectives to suppress background noise, yielding F_2.1 and F_3.1. Meanwhile, the CMM operates on the deepest feature F₄ along the channel dimension to fully exploit the global semantic context, producing F_4.1. Furthermore, distinct composite operations—M₁, M₂, and M₃—are constructed from the blending of convolution, upsampling, and channel concatenation. These operations hierarchically fuse features from adjacent layers (F_2.1, F_3.1, F_4.1) to refine the representations, resulting in F_2.2, F_3.2, and F_4.2. Finally, to mathematically connect the boundary and regional semantics, the RFM is deployed at each decoding stage. Using the boundary cues, current stage features, and the prediction map from the preceding RFM as inputs, it determines the correlation between boundary and interior regions. This explicitly rectifies and reinforces highly uncertain areas, iteratively refining the final segmentation results from coarse to fine.

Figure 1 Overall architecture of the proposed IGBP-Net. The framework consists of a PVTv2 backbone in an encoder-decoder setup, supported by multistage deep supervision. The network features several key modules, including the AGG, BPM, GL, and the CMM. Composite fusion operations are applied to blend features across different levels. Finally, the RFM handles the iterative refinement of the segmentation results. AGG, information aggregation module; BN, batch normalization; BPM, boundary-perception module; CMM, channel multiscale module; conv, convolution; GL, global and local feature extraction module; IGBP-Net, information-guided deep learning network; PVTv2, pyramid vision transformer version 2; ReLu, rectified linear unit; RFM, region fusion module.

AGG

The detailed architecture of the AGG is illustrated in Figure 2. The primary purpose of this module is to combine deep-level features and locate the main polyp regions. This location data acts as a critical guide, helping the BPM focus on the target areas to draw accurate boundaries. Additionally, since shallow features have a high resolution, processing them directly requires a substantial amount computing power. Consequently, we use deep-level features as the main inputs for the AGG to handle the information. The formulae for these processes are as follows:

F2&3=conv(conv(F2⊗F3)⊕F3)[1]

F3&4=conv(conv(F3⊗F4)⊕F4)[2]

S′5=conv(conv(F2&3⊗F3&4)⊕F3&4)[3]

Figure 2 Structure of the AGG. AGG, information aggregation module; BN, batch normalization; conv, convolution; ReLu, rectified linear unit.

where Conv (∙) denotes a composite block consisting of a 3×3 convolution, followed by Batch normalization and the rectified linear unit (ReLU) activation function.

During the fusion process, element-wise multiplication is used to highlight the most important features. This step boosts the weight of target regions, helping the model locate them even amidst complex background clutter. To balance this, element-wise addition reintroduces the original feature information. Acting as a residual connection, it prevents data loss that might occur during fusion. To implement this, the AGG module adopts a progressive fusion strategy, which is mathematically formulated in Eqs. [1-3]. Specifically, Eqs. [1,2] define the generation of intermediate representations (F_2&3 and F_3&4) by integrating features from adjacent layers. Subsequently, Eq. [3] computes the final aggregated feature map. This hierarchical approach bridges the “semantic gap” that often occurs when features from highly disparate levels are directly mixed. The result is a gradual refinement of information, which improves the model’s sensitivity for target localization.

BPM

We introduced the BPM to accurately segment polyp boundaries in complex conditions. This module efficiently locates and extracts detailed edge clues. The detailed architecture of the BPM is illustrated in Figure 3.

Figure 3 Structure of the BPM. Green dashed region, the initial phase. Red dashed region, the multi-scale feature extraction stage. Blue dashed region, the final stage. BN, batch normalization; BPM, boundary-perception module; conv, convolution; DWconv, depthwise convolution; RA, residual attention; ReLu, rectified linear unit; SA, spatial attention.

Shallow feature maps F₁ capture rich spatial details that help the model discern sharp boundary contours. However, these features lack the global context needed to locate objects clearly. Relying only on them can confuse the target with background noise, leading to false positives. The localization maps from the AGG solves this problem. Acting as spatial guides, they direct the BPM to focus on the target area, ensuring it extracts boundary information only from the object’s surroundings. The process works as follows.

• As highlighted by the green dashed region in Figure 3, the initial phase manages feature interaction. The module accepts the localization feature map S'₅ and the shallow feature map F₁ as inputs. S'₅ corresponds to the penultimate tensor from the AGG, positioned just before the final prediction layer, and carries the global semantic context. The process for this stage is expressed as follows:

  $$X=conv\left(conv\left(cat\left({S^{\prime }}_5,RA\left(F_1\right)\right)\right)\oplus {S^{\prime }}_5\oplus RA\left(F_1\right)\right)$$[4]

  where Conv (∙) denotes a composite block consisting of a 3×3 convolution, followed by batch normalization and the ReLU activation function; Cat (∙) denotes the channel-wise concatenation operation; and RA (∙) represents the residual attention mechanism, the structure of which is detailed in the leftmost part of Figure 3; and SA (∙) refers to the spatial attention mechanism (23). The spatial attention mechanism handles the shallow feature map F₁. Its principal function is to highlight key local details while blocking the background noise often found in shallow layers. As shown in Eq. [4], the process uses channel concatenation followed by residual connections. This setup allows the model to focus on the target areas step by step. Meanwhile, the residual links keep the original features intact, preventing data loss during the mix.
• As indicated by the red dashed region in Figure 3, the multi-scale feature extraction stage extracts specific boundary cues. We use a parallel setup with multiscale depth-wise separable strip convolutions. Polyps often appear highly similar to the surrounding mucosa and have complex shapes, causing their boundaries to be displayed as blurred, elongated strips. Strip convolutions align naturally with this geometry, making them ideal for the efficient modeling of such features. Moreover, the parallel branches offer diversity receptive fields, allowing the model to capture subtle differences between adjacent regions for precise boundary localization.
• Represented by the blue dashed region in Figure 3, the final stage focuses on refinement. We adapted the residual attention mechanism (12) to specifically target and sharpen boundary regions. This approach improves the model’s sensitivity to edge details. This process can be expressed as follows:

  $${S^{\prime }}_b=conv\left(Y\otimes \left(1-sigmoid\left(S_5\right)\right)\right)$$[5]

  where Y denotes the output feature map from the multi-scale feature extraction stage. This map holds both the precise boundary cues and the internal semantics of the target object. To isolate the boundary information, element-wise multiplication of Y by the inverted coarse segmentation map, S₅ (from the AGG) is conducted. This step effectively suppresses activations inside the object, allowing the model to focus strictly on the edges. As a result, the operation further sharpens the polyp contours, ultimately producing the final high-fidelity boundary prediction map, S_b.

The internal working mechanism of the BPM is depicted in Figure 4, which includes visualizations of key intermediate features generated via channel-wise averaging.

Figure 4 Step-wise visualization of feature evolution within the BPM. Column 2 displays shallow features. These capture rich local details but lack the global context needed to locate objects. Adding localization cues (Column 3) focuses attention on the target and reduces background noise. Next, strip convolutions (Column 4) sharpen the boundary structures. Subsequently, the reverse Attention mechanism (Column 5) suppresses the interior regions to isolate specific edge cues. Finally, the predictions in Column 6 closely match the boundary ground truths (Column 7). This confirms the module’s precision in complex scenarios. BPM, boundary-perception module; GT, ground truth.

Global and local feature extraction

Inside the hierarchical encoder (Figure 1), the features F_i (i=1,2,3,4) change gradually. As information flows through the deeper layers of the network, local spatial details decrease, while the global semantic context grows. The intermediate features, F₂ and F₃, offer a unique advantage. They provide a hybrid representation, balancing low-level textures with high-level semantics. To leverage this dual nature, we built the GL, whose design is presented in Figure 5.

Figure 5 Structure of the GL. BN, batch normalization; conv, convolution; ECA, efficient channel attention; GL, global and local feature extraction module; RDA, residual dual attention; ReLu, rectified linear unit; SA, spatial attention.

First, the input features pass through two convolutional layers to reduce the channel dimension. Next, a parallel multibranch structure is applied. This setup combines the axial attention mechanism (13) with a multiscale feature extraction block. Together, they capture both global context and fine details. Axial attention is a variant of standard self-attention that uses query (Q), key (K), and value (V) projections to calculate interactions. This efficiently identifies long-range dependencies across different regions. The formula for this process is as follows:

Attention(Q,K,V)=Softmax(QKTD)V[6]

where Q, K, and V represent the query, key, and value matrices derived from the input features, respectively; D is the dimension of the queries and keys; and Softmax represents the normalized activation function.

Unlike standard self-attention, the axial attention mechanism calculates attention separately for the width and height axes. Standard self-attention flattens the spatial dimensions of Q and K into (W × H) × C. Axial attention, however, reshapes these into H × (C × W) or W × (C × H). This approach significantly lowers computational complexity. It also allows the model to capture long-range dependencies in both the vertical and horizontal directions, efficiently modeling the global semantic context. To complement this global view and capture fine details, we use a multiscale feature extraction module.

The module operates through three primary stages: multiscale extraction, hierarchical connection, and feature fusion. (I) In the multiscale extraction stage, the module utilizes four parallel branches. Dilated convolutions are stacked from the bottom up to progressively expand the receptive field starting from 3×3, which captures image details at various scales without the heavy computational cost of standard convolutions. (II) During the hierarchical connection stage, each upper branch integrates the output from the branch directly below it. This early interaction between nearby scales prevents the loss of local information that can occur when highly disparate branches are merged simultaneously. (III) Finally, the fusion stage concatenates the outputs from all branches to create the final representation. The complete mathematical formulation of these three sequential operations is defined in Eq. [7]:

x=conv(conv(input))x1=conv(x)x2=conv(convd=3(x))+x1x3=conv(convd=5(convd=3(x)))+x2x4=conv(convd=7(convd=5(convd=3(x))))+x3Output=conv(conv(cat(x1,x2,x3,x4)))[7]

where Conv (∙) denotes the channel-wise concatenation operation.

Subsequently, the enhanced features output by the parallel axial attention and multiscale branches are concatenated. This yields a hybrid representation that encapsulates both global semantic context and local spatial details. Furthermore, a residual connection is employed to fortify feature representation and prevent potential performance degradation. These refined features are then fed into the residual dual attention (RDA) module, the architecture of which is detailed in the bottom-right corner of Figure 5. Here, SA (∙) denotes the spatial attention mechanism. It is designed to accentuate salient spatial details while effectively suppressing irrelevant background information. Additionally, the efficient channel attention (ECA) mechanism (24), by leveraging local cross-channel interactions, adaptively recalibrates channel importance, emphasizing informative channels while diminishing redundant ones. Similar to the previous stages, a residual connection is integrated within the RDA module to facilitate gradient flow and maintain feature identity. In summary, the GL module effectively enhances both global and local feature representations. By filtering out noise, it ensures that the model’s attention is explicitly concentrated on the lesion regions.

CMM

The deepest feature map F₄ (with dimensions 512×11×11) inherently encodes rich global semantic information, which is primarily distributed across its high-dimensional channels (C=512). To effectively exploit this, we propose the CMM, designed to model multiscale global semantics explicitly at the channel level. Its architecture is illustrated in Figure 6.

Figure 6 Structure of the CMM. BN, batch normalization; CMM, channel multiscale module; conv, convolution; ECA, efficient channel attention; ReLu, rectified linear unit.

First, the module utilizes three parallel convolutional branches, each employing distinct channel reduction rates to extract global features. Specifically, the channel dimensions in these branches are transformed as follows: 512→32, 512→320→32, and 512→320→128→32. These diverse channel transformations allow the model to capture varied feature representations and thoroughly examine interchannel relationships. Next, the outputs from these branches are concatenated and subsequently processed via two parallel pathways. (I) The first path employs a standard convolution to compress the channel dimension and fuse features. (II) The second path incorporates a channel shuffle operation followed by a convolution. This strategy facilitates cross-group information flow and interaction, ensuring deep feature integration and enhancing representational capacity. Crucially, the first path functions as a complementary residual-like connection. It preserves original information integrity, thereby mitigating the potential performance degradation caused by improper fusion in the second path. Finally, features from both paths are concatenated fused via convolution and refined with the ECA mechanism for feature enhancement and noise filtration. In summary, the CMM effectively examines global polyp features within the channel dimension, significantly aiding the model in target identification and localization.

Region fusion

The detailed architecture of the RFM is illustrated in Figure 7. This module is strategically deployed across the second, third, and fourth stages of the network. The RFM accepts three primary inputs: (I) the extracted boundary cues; (II) the feature maps generated by the M_i (i=2,3,4) operations; and (III) the saliency segmentation map from the preceding stage (specifically, for the fourth stage, the input serves as the coarse segmentation map derived from the AGG module). The primary objective of the RFM is to establish a progressive deep-to-shallow fusion pathway. By iteratively integrating and supplementing feature information, the module progressively refines the segmentation output to achieve optimal performance. The specific information-processing workflow is formulated as follows:

X=conv(Fi.2⊗Si+1⊕Fi.2)[8]

Y=conv(X⊗S′b+X)[9]

Su=E(Si+1⊗Sb⊕Si+1)[10]

Si=conv(conv(Y⊗Su⊕Y))[11]

Figure 7 Structure of the RFM. BN, batch normalization; conv, convolution; ReLu, rectified linear unit; RFM, region fusion module.

In Eq. [8], F_i,2 denotes the feature representations extracted by the M_i operation, and S_i+1 is the segmentation map generated by the preceding RFM stage. First, the element-wise tensor multiplication is employed to accentuate salient features shared by both inputs. Crucially, this operation also upweights the polyp-specific features that might be overlooked in F_i,2 but that are highly activated in S_i+1. Subsequently, element-wise tensor addition reintroduces the original F_i,2. This residual-like step prevents the potential suppression of unique polyp features intrinsic to F_i,2 that could be diminished by the multiplication. In Eq. [9], S'_b denotes the boundary-rich feature map output by the BPM, which is channel-aligned with the intermediate feature X. Although tensor multiplication effectively enhances the weights of boundary cues lacking in X, it risks suppressing nonboundary semantic information. Therefore, tensor addition is applied to compensate for this loss, ensuring a balanced feature representation. In Eq. [10], where S_b represents the boundary segmentation map from the BPM, the combination of tensor multiplication and addition facilitates the complementary integration of recognized cues from both S_b and S_i+1. Finally, to further refine the segmentation, we introduced an uncertainty-aware enhancement operation, denoted as E(x). This purpose of this operation is to heighten the model’s sensitivity to ambiguous or hard-to-classify regions. The specific formulation of E(x) is as follows:

E(x)=1−|sigmoid(x)−T|T[12]

where sigmoid (x) denotes the standard sigmoid activation function.

“Uncertain regions” typically manifest as blurred boundaries or areas compromised by external interference. In these regions, the model struggles to distinguish the lesion from the background. Consequently, the predicted probability scores for these ambiguous pixels tend to fluctuate around 0.5, indicating low confidence. Therefore, we set the threshold to 0.5. This could maximize the reweighting coefficient for these hard-to-classify areas, thereby explicitly enhancing the model’s sensitivity to uncertainty. Finally, as shown in Eq. [11], the element-wise multiplication intensifies the focus on these uncertain regions, while the addition operation preserves the features of regions with established confidence to prevent information loss. Following a final convolutional layer, the refined polyp segmentation map for the current stage, denoted as S_i, is generated.

Evaluation metrics

To quantitatively evaluate the segmentation performance, we employed several standard metrics that are widely adopted in medical image segmentation. These metrics included the mean dice similarity coefficient, intersection over union (IoU), precision, and recall. To ensure statistical rigor and account for experimental variance, all metrics are reported alongside their 95% confidence intervals (CIs). The formulae for these metrics are as follows:

Dice=2|F∩G||F|+|G|[13]

IoU=TPTP+FP+FN[14]

Precision=TPTP+FP[15]

Recall=TPTP+FN[16]

where F and G denote the predicted segmentation map and the corresponding ground truth mask, respectively; TP is the number of pixels correctly identified as polyp tissue (i.e., the intersection of the prediction and the ground truth); FP is the number of background pixels erroneously classified as polyp tissue; and FN is the number of actual polyp pixels missed (misclassified as background) by the model.

Experimental design

We implemented the method using PyTorch. Experiments were run on a workstation with a GeForce RTX 3090 GPU (Nvidia, Santa Clara, CA, USA) and Windows 10 (Microsoft Corp., Redmond, WA, USA). We set the batch size to 8 and trained for 120 epochs. For optimization, we employed the Adam optimizer with an initial learning rate of 1×10⁻⁴ (β₁=0.9, β₂=0.999). To stabilize the training process, the gradient clipping norm was set to 2.0. We also adopted a deep supervision strategy to improve training stability. During training, the total loss was computed as a weighted sum of the predictions from multiple deep supervision stages. To explicitly penalize boundary errors and enforce contour alignment, the structural loss for the boundary prediction was assigned a weight of 2.0. For the regional predictions, a progressive weight decay strategy was applied from shallow to deep layers: the final segmentation map received a weight of 1.0, while the intermediate lateral maps and the initial coarse saliency map were weighted at 0.8, 0.6, and 0.4, respectively. By integrating both the weighted IoU loss and the weighted binary cross-entropy loss across all these supervision stages, the final total objective function is mathematically formulated in Eq. [17]:

L(x,G)=LwIou(x,G)+Lwbce(x,G)Loss=∑i=25L(Si,G)+L(Sb,Gb)[17]

where G and G_b denote the ground truth masks for the global polyp regions and the boundaries, respectively; and S_i(i∈{b,2,3,4,5}) represents the predicted segmentation maps generated at the corresponding stages, as indicated in Figure 1.

To validate the segmentation performance of our model, we compared it with nine mainstream algorithms: U-Net++ (25), residual UNet (ResUNet)++ (26), PraNet (12), duplex contextual relation network (DCRNet) (27), SSFormer (11), CaraNet (13), BRNet (14), multiscale spatial reverse attention network for polyp segmentation (MSRAformer) (28), and polyp pyramid vision transformer (PolypPVT) (19). U-Net++ and ResUNet++ are improved versions of the classic U-Net and are standard benchmarks in medical image segmentation. PraNet, MSRAformer, and CaraNet use reverse attention mechanisms, while BRNet incorporates boundary generation and loss constraints during training. We selected these four specifically because they represent key efforts in examining polyp boundary information. The other models are all recognized as top-tier algorithms in the field of polyp segmentation. To ensure a strictly rigorous and fair comparison, all benchmark models were retrained under identical experimental settings, dataset splits, and preprocessing pipelines.

Results

Comparison study

Tables 1,2 summarize the quantitative performance of the proposed IGBP-Net compared to other methods on the Kvasir and CVC-ClinicDB datasets. On the Kvasir dataset, our model achieved the best performance across major metrics, yielding a Dice score of 93.26%, an IoU of 88.92%, and a precision of 93.62%. Notably, compared to the second-best method, IGBP-Net achieved improvements of 1.19% in Dice score and 1.82% in IoU. Similarly, on the CVC-ClinicDB dataset, IGBP-Net demonstrated superior segmentation capability, securing top-tier results, with a Dice score of 94.52%, an IoU of 90.01%, and a precision of 95.40%, representing a performance gain of 0.72% in Dice score and 1.00% in IoU over the nearest competitor.

Tables 3,4 present the comparative results on the more challenging CVC-ColonDB and ETIS datasets. On CVC-ColonDB, IGBP-Net consistently outperformed the approaches, achieving optimal values of 80.96%, 73.57%, and 84.06% for the Dice score, IoU, and precision, respectively. Specifically, it surpassed the next-best method, PolypPVT, by margins of 1.74%, 1.28%, and 1.40%, respectively. The advantages of IGBP-Net were most pronounced on the ETIS dataset, which is known for its complexity. Our method achieved state-of-the-art results across all metrics. Compared to CaraNet, IGBP-Net provided substantial improvements, boosting performance by 4.84%, 5.98%, 5.12%, and 3.71%, respectively.

Three key outcomes of IGBP-Net should be noted. First, it achieved top scores on the Kvasir and CVC-ClinicDB datasets. Since these sources formed the training set, this success confirms that the model effectively learns the data distribution. Second, the model maintained its high performance on the CVC-ColonDB and ETIS. Because these datasets were completely excluded from training, these results prove that IGBP-Net generalizes well to unseen data domains. Third, the Kvasir dataset is particularly challenging due to the inclusion of large polyps with varied shapes and complex surroundings. Our model’s strong performance here demonstrates its precise boundary perception, which enables it to define edges accurately even in these difficult scenarios. Notably, alongside achieving the high average scores, our model exhibited narrower CIs, demonstrating superior stability and robustness across a variety of test cases.

Figure 8 is a visual comparison of IGBP-Net with the competing models. Most of the competitors struggle with blurred or unclear boundaries (see columns 1, 2, 3, and 7). They also failed to capture the details of complex, irregular shapes (see columns 1, 2, and 4). This often leads to oversegmentation or undersegmentation, resulting in imprecise predictions. For small polyps (columns 5 and 6), some models missed the target entirely, while others mistook normal tissue for polyps. In contrast, IGBP-Net proved to be much more precise. It accurately traced the margins of complex polyps, producing clean segmentation maps. It also located small targets effectively and ignored background noise. Overall, IGBP-Net handles the varied shapes and complexities of polyps better than do existing models, solving the common issues in the field.

Figure 8 Visual comparison of the segmentation results produced by the different models (A-J). GT, ground truth.

Ablation study

To further validate the efficacy of each individual component within the proposed IGBP-Net, we conducted comprehensive ablation studies. Experiments were performed on the Kvasir and CVC-ClinicDB datasets, and the results are summarized in Tables 5,6. The configurations were as follows: D1 (baseline), SSFormer; D2, D1 + GL + CMM; D3, D2 + BPM; D4, D3 + AGG; and D5 (proposed model), D4 + RFM.

Compared to the baseline (D1), D2 achieved average improvements of 0.73% in Dice, 1.57% in IoU, 1.12% in precision, and 0.43% in recall. These gains indicate that the GL and CMM effectively mine latent cues from the encoder outputs, thereby enhancing feature representation. Building upon D2, D3 yielded further increases of 0.59%, 0.58%, and 0.89% in Dice, IoU, and precision, respectively. This demonstrates that the BPM effectively refines polyp boundary localization and improves segmentation quality along object margins. For D4, the average IoU and recall increased by 0.38% and 0.51%, respectively, compared to D3. This suggests that the AGG effectively integrates deep features to capture global semantic information. By guiding the model to focus on the polyp regions, AGG ensures that the boundary refinement operations of the BPM are explicitly targeted at the lesion margins. Finally, D5 (the proposed IGBP-Net) outperformed D4 with average gains of 0.59% for Dice, 0.71% for IoU, 1.08% for precision, and 0.35% for recall. Notably, the performance improvement observed in D5 was more significant compared to the incremental gains obtained in the previous iterations. This improvement can be attributed to the RFM, which facilitates mutual guidance between boundary cues and multistage saliency maps. By complementarily fusing these features and reinforcing uncertain regions, RFM effectively rectifies errors, pushing the model toward optimal performance.

Comparison with alternative designs

To further justify the architectural necessity of our specific formulations and ensure the gains were not merely attributable to additive complexity, we tested a competing alternative design for the BPM. We replaced the multiscale strip convolutions with standard square convolutions (substituting 17×5 with 9×9, 11×3 with 5×5, and 7×3 with 3×3) of equivalent receptive fields. As shown in Tables 7,8, the Dice score and IoU of this alternative solution were both lower on the Kvasir and CVC-ClinicDB datasets. This comparison demonstrates that standard square convolutions inherently incorporate excessive background noise when processing edges. Our specific multiscale strip convolutions are architecturally necessary because they geometrically align with the elongated, irregular topological boundary features of polyps, thereby isolating edge cues more effectively than simpler formulations.

Furthermore, for the RFM, simpler formulations such as basic channel concatenation lack the explicit mathematical capacity to model the uncertainty between boundaries and internal regions, which is strictly required for addressing low-contrast ambiguity.

Failure mode analysis

Although IGBP-Net demonstrated strong performance in segmenting polyps with blurred boundaries and complex shapes via information guidance and boundary perception, limitations remain. As illustrated in Figure 9, the model struggled in a few scenarios: extremely flat polyps with reflective surfaces (Column 1), polyps with heavy reflection near the edges (Column 2), and highly camouflaged polyps (Column 3). These failures primarily occurred for three reasons: (I) some polyps closely resembled normal tissue, making it difficult for the model to detect subtle differences amidst environmental noise; (II) strong light reflection near the edges obscured key boundary feature cues; and (III) the limited size of the training dataset restricted the model’s ability to effectively generalize to these rare, extreme cases.

Figure 9 Visualization of model segmentation failure (A-C). GT, ground truth.

Discussion

Recent advances in medical imaging are being increasingly driven by large vision foundation models. For example, generalist models trained with contextual mixture of adapter experts (29) and the 3D medical segment anything model 2 (30) offer versatile, task-agnostic solutions. With open-world capabilities, task-specific hierarchical networks leveraging scale-aware pyramidal feature learning (31)—aligning with our multiscale designs—remain highly efficient for dense predictions in boundary-ambiguous tasks such as colonoscopy. Additionally, environmental noise in endoscopic videos could be mitigated via general-purpose video reconstruction that synergizes grid-splicing diffusion models and large language models (32).

Beyond segmentation, recognizing polyps as primary precursors to CRC is a clinical necessity for downstream multimodal risk stratification. Future research should integrate our boundary-aware segmentation masks into multimodal pipelines. For instance, it has been found that employing hierarchical multimodal co-attention transformers to combine histology and radiology allows for robust survival prediction (33). Furthermore, applying counterfactual bidirectional co-attention transformers for integrative histology-genomic analysis enables comprehensive cancer risk stratification and personalized drug response predictions (34), linking automated polyp detection with precision oncology.

Limitations

Our study involved two primary limitations. First, prioritizing boundary precision via multiple interacting modules incurs considerable computational overhead. Evaluated on an RTX 3090 GPU (352×352 input), IGBP-Net has 49.75 million parameters and requires 130.3 giga floating-point operations per second. Its inference speed of ~28.79 frames per second meets real-time video requirements, but leaves a small computational margin for concurrent software processes in standard endoscopic systems. Second, our evaluation relied on standard vision metrics, which do not fully reflect downstream clinical utility. Qualitative failure mode analysis revealed performance degradation under severe clinical conditions, such as in extremely flat polyps (near-zero contrast) or prominent imaging artifacts such as specular highlights and motion blur. In these situations, obscured boundary cues cause ambiguous delineations that could affect the precise assessment of resection margins.

Future work

To address these limitations, our future research will focus on two primary directions. Technically, we will explore model compression strategies, such as knowledge distillation and structural pruning, to significantly reduce the parameter count and improve inference efficiency for seamless deployment. Furthermore, we plan to conduct comprehensive clinical-style evaluations. This will involve testing the framework on datasets explicitly stratified by specific polyp morphologies, sizes, and artifact types, which is a necessary prerequisite before the model can be considered for standalone clinical deployment.

Conclusions

In this study, we developed IGBP-Net to address the challenges of low interclass contrast and morphological complexity in colorectal polyp segmentation. Rather than simply assembling existing feature extractors, our framework is driven by a core modeling insight of guidance connecting spatial localization and boundary perception. By mathematically coupling boundary priors with internal region semantics and supporting this interaction with hierarchical contextual refinement, the network progressively rectifies structural errors in highly uncertain areas. The experimental results demonstrate that IGBP-Net delivers highly competitive segmentation precision, particularly for polyps with blurred boundaries, complex contours, and small sizes. To address the limitations in extreme low-contrast or camouflaged scenarios, future work will include the investigation of uncertainty-aware shape reasoning and semisupervised learning. Additionally, balance algorithmic accuracy and the capacity for real-time clinical deployment, we plan to examine model compression techniques, such as knowledge distillation and structural pruning, to optimize inference efficiency. Furthermore, as polyps are primary precursors to CRC, our goal is to integrate our boundary-aware masks into downstream multimodal pipelines and thus facilitate comprehensive cancer risk stratification and personalized clinical treatment.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the TRIPOD+AI reporting checklist. Available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0197/rc

Funding: This work was supported by the Guangxi Key Research and Development Plan Project (grant No. Guike AB24010338), the National Natural Science Foundation of China (NSFC) (grant No. 61862019), and Guangxi Key Laboratory Fund of Embedded Technology and Intelligent System.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0197/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Cao J, Wang X, Qu Z, Zhuo L, Li X, Zhang H, Yang Y, Wei W. WDFF-Net: Weighted Dual-Branch Feature Fusion Network for Polyp Segmentation With Object-Aware Attention Mechanism. IEEE J Biomed Health Inform 2024;28:4118-31. [Crossref] [PubMed]
2. Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer statistics, 2022. CA Cancer J Clin 2022;72:7-33. [Crossref] [PubMed]
3. Xiao B, Hu J, Li W, Pun CM, Bi X. CTNet: Contrastive Transformer Network for Polyp Segmentation. IEEE Trans Cybern 2024;54:5040-53. [Crossref] [PubMed]
4. Hwang S, Oh JH, Tavanapong W, Wong J, de Groen PC. Polyp detection in colonoscopy video using elliptical shape feature. 2007 IEEE International Conference on Image Processing; 2007 Sep 16-19; San Antonio, TX, USA. IEEE; 2007:II-465-8.
5. Tajbakhsh N, Gurudu SR, Liang J. Automated Polyp Detection in Colonoscopy Videos Using Shape and Context Information. IEEE Trans Med Imaging 2016;35:630-44. [Crossref] [PubMed]
6. Sánchez-González A, García-Zapirain B, Sierra-Sosa D, Elmaghraby A. Automatized colon polyp segmentation via contour region analysis. Comput Biol Med 2018;100:152-64. [Crossref] [PubMed]
7. Zhang P, Yang H, Shen X, Wang L. Polyp segmentation model based on fusion of local and global features. Electron Meas Technol 2024;47:100-9.
8. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM, editors. Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015); 2015 Oct 5-9; Munich, Germany. Springer; 2015:234-41.
9. Patel K, Bur AM, Wang G. Enhanced U-Net: A Feature Enhancement Network for Polyp Segmentation. Proc Int Robot Vis Conf 2021;2021:181-8.
10. Dai D, Dong C, Yan Q, Sun Y, Zhang C, Li Z, Xu S I. (2)U-Net: A dual-path U-Net with rich information interaction for medical image segmentation. Med Image Anal 2024;97:103241. [Crossref] [PubMed]
11. Wang J, Huang Q, Tang F, Meng J, Su J, Song S. Stepwise feature fusion: Local guides global. In: Medical Image Computing and Computer Assisted Intervention (MICCAI 2022); 2022 Sep 18-22; Singapore. Springer; 2022:110-20.
12. Fan DP, Ji GP, Zhou T, Chen G, Fu H, Shen J, Shao L. PraNet: Parallel Reverse Attention Network for Polyp Segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI 2020); 2020 Oct 4-8; Lima, Peru. Springer; 2020:263-73.
13. Lou A, Guan S, Loew M. CaraNet: context axial reverse attention network for segmentation of small medical objects. J Med Imaging (Bellingham) 2023;10:014005. [Crossref] [PubMed]
14. Yue G, Li Y, Jiang W, Zhou W, Zhou T. Boundary refinement network for colorectal polyp segmentation in colonoscopy images. IEEE Signal Process Lett 2024;31:954-8.
15. Yue G, Zhuo G, Yan W, Zhou T, Tang C, Yang P, Wang T. Boundary uncertainty aware network for automated polyp segmentation. Neural Netw 2024;170:390-404. [Crossref] [PubMed]
16. Jha D, Smedsrud PH, Riegler MA, Halvorsen P, de Lange T, Johansen D, Johansen HD. Kvasir-SEG: A Segmented Polyp Dataset. In: Multimedia Modeling (MMM 2020); 2020 Jan 5-8; Daejeon, Korea. Springer; 2020:451-62.
17. Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño F. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imaging Graph 2015;43:99-111.
18. Silva J, Histace A, Romain O, Dray X, Granado B. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. Int J Comput Assist Radiol Surg 2014;9:283-93. [Crossref] [PubMed]
19. Dong B, Wang W, Fan DP, Li J, Fu H, Shao L. Polyp-PVT: Polyp segmentation with pyramid vision transformers. CAAI Artif Intell Res 2023;2:9150015.
20. Huang X, Zhuo L, Zhang H, Yang Y, Li X, Zhang J, Wei W. Polyp segmentation network with hybrid channel-spatial attention and pyramid global context guided feature fusion. Comput Med Imaging Graph 2022;98:102072. [Crossref] [PubMed]
21. Zhang Y, Liu H, Hu Q. TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation. In: Proceedings of the Medical Image Computing and Computer Assisted Intervention (MICCAI 2021); 2021 Sep 27-Oct 1; Strasbourg, France. Springer; 2021:14-24.
22. Wang W, Xie E, Li X, Fan DP, Song K, Liang D, Lu T, Luo P, Shao L. PVT v2: Improved baselines with pyramid vision transformer. Comput Vis Media 2022;8:415-24.
23. Guo C, Szemenyei M, Yi Y, Wang W, Chen B, Fan C. SA-UNet: Spatial Attention U-Net for Retinal Vessel Segmentation. In: Proceedings of the 25th International Conference on Pattern Recognition (ICPR); 2021 Jan 10-15; Milan, Italy. IEEE; 2021:1236-42.
24. Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2020 Jun 13-19; Seattle, WA, USA. IEEE; 2020:11531-9.
25. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support 2018;2018:3-11. [Crossref] [PubMed]
26. Jha D, Smedsrud PH, Riegler MA, Johansen D, De Lange T, Halvorsen P, Johansen HD. ResUNet++: An advanced architecture for medical image segmentation. In: Proceedings of the 2019 IEEE International Symposium on Multimedia (ISM); 2019 Dec 9-11; San Diego, CA, USA. IEEE; 2019:225-2255.
27. Yin Z, Liang K, Ma Z, Guo J. Duplex contextual relation network for polyp segmentation. In: Proceedings of the 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI); 2022 Mar 28-31; Kolkata, India. IEEE; 2022:1-5.
28. Wu C, Long C, Li S, Yang J, Jiang F, Zhou R. MSRAformer: Multiscale spatial reverse attention network for polyp segmentation. Comput Biol Med 2022;151:106274. [Crossref] [PubMed]
29. Zhang S, Zhang Q, Zhang S, Liu X, Yue J, Lu M, Xu H, Yao J, Wei X, Cao J, Zhang X, Gao M, Shen J, Hao Y, Wang Y, Zhang X, Wu S, Zhang P, Cui S, Wang G. A generalist foundation model and database for open-world medical image segmentation. Nat Biomed Eng 2025; Epub ahead of print. [Crossref]
30. Ma J, Yang Z, Kim S, Chen B, Baharoon M, Fallahpour A, Asakereh R, Li Z. MedSAM2: Segment Anything in 3D Medical Images and Videos. arXiv:2504.03600 [Preprint]. 2025. Available online: https://arxiv.org/abs/2504.03600
31. Liu X, Liang J, Zhang J, Qian Z, Xing P, Chen T, Yang S, Chukwudi C, Qiu L, Liu D, Zhao J. Advancing hierarchical neural networks with scale-aware pyramidal feature learning for medical image dense prediction. Comput Methods Programs Biomed 2025;265:108705. [Crossref] [PubMed]
32. Liu J, Zhang J, Yang S, Wang R, Li T, Zhou H, Chen P, Wu G. Toward General-Purpose Video Reconstruction Through Synergy of Grid-Splicing Diffusion and Large Language Models. IEEE Trans Circuits Syst Video Technol 2026;36:1328-40.
33. Li Z, Jiang Y, Lu M, Li R, Xia Y. Survival Prediction via Hierarchical Multimodal Co-Attention Transformer: A Computational Histology-Radiology Solution. IEEE Trans Med Imaging 2023;42:2678-89. [Crossref] [PubMed]
34. Ji Z, Ge Y, Chukwudi C. U K, Zhang SM, Peng Y, Zhu J, Zaki H, Zhang X, Yang S, Wang X, Chen Y, Zhao J. Counterfactual Bidirectional Co-Attention Transformer for Integrative Histology-Genomic Cancer Risk Stratification. IEEE J Biomed Health Inform 2025;29:5862-74. [Crossref] [PubMed]