AFFUNet: adaptive feature fusion Transformer U-Net with joint loss function for medical image segmentation

Mingming Pang; Chu Yang; Yanlong Luo

doi:10.21037/qims-2026-1-0234

Original Article

AFFUNet: adaptive feature fusion Transformer U-Net with joint loss function for medical image segmentation

Mingming Pang, Chu Yang, Yanlong Luo

School of Information and Communication Engineering, Shanxi University of Electronic Science and Technology, Linfen, China

Contributions: (I) Conception and design: M Pang; (II) Administrative support: M Pang; (III) Provision of study materials or patients: M Pang; (IV) Collection and assembly of data: M Pang, C Yang; (V) Data analysis and interpretation: M Pang, C Yang; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

Correspondence to: Mingming Pang, Master. School of Information and Communication Engineering, Shanxi University of Electronic Science and Technology, No. 501 Binhe West Road, Yaodu District, Linfen 041000, China. Email: pangmingming@sxdzkj.edu.cn.

Background: High-precision medical image segmentation is crucial for enabling computer-aided accurate diagnosis and personalized treatment planning. Although Transformer-based methods have shown promising results, they still face significant challenges. These include a fixed feature fusion strategy that cannot adapt to dynamic changes in feature importance, the standard Dice loss that treats all pixels equally and struggles to focus on difficult samples, and inadequate boundary precision optimization, which limits the clinical applicability of segmentation results. To address these challenges, this study proposes Adaptive Feature Fusion U-Net (AFFUNet), an adaptive feature fusion (AFF) Transformer U-Net with a joint loss function, aiming to improve segmentation accuracy through dynamic feature fusion, hard sample reweighting, and explicit boundary optimization.

Methods: Based on the Cross-Shaped Window Transformer UNet (CSWin-UNet) architecture, we propose three modifications to address the challenges discussed above: (I) an AFF module that replaces standard skip connection concatenation to dynamically learn fusion weights for encoder and decoder features; (II) focal Dice loss, which adjusts the Dice loss based on prediction confidence to better prioritize difficult samples; (III) boundary-aware loss, which explicitly enhances boundary precision through a gradient-based boundary extraction technique.

Results: We evaluated our method on two benchmark datasets: the Synapse multi-organ computed tomography (CT) dataset and the Automated Cardiac Diagnosis Challenge (ACDC) cardiac magnetic resonance imaging (MRI) dataset. On the Synapse dataset, our method achieved an average Dice coefficient of 81.21% and an average 95th percentile Hausdorff distance (HD95) of 18.41 mm, representing a 0.09% improvement in Dice and a 0.45 mm reduction in HD95 compared to the state-of-the-art CSWin-UNet. On the ACDC dataset, our method achieved a mean Dice of 89.77%, consistently outperforming CSWin-UNet (88.44%). Ablation studies confirmed that each component contributes to the overall performance, particularly in small organs and boundary regions. Notably, the added AFF module introduces negligible computational overhead: compared to CSWin-UNet, the parameters and floating point operations per second (FLOPs) remained virtually unchanged (−0.02 M, −0.036 G). Thus, our method achieves improved segmentation accuracy while maintaining practical efficiency.

Conclusions: This work integrates AFF, focal Dice loss, and boundary-aware loss into a collaborative framework that works across the feature, objective, and boundary levels. The resulting end-to-end mechanism achieves improved performance on standard benchmarks while maintaining practical efficiency, warranting further evaluation in clinical settings.

Keywords: Medical image segmentation; Transformer; adaptive feature fusion (AFF); focal Dice loss; boundary-aware loss

Submitted Jan 27, 2026. Accepted for publication Jun 29, 2026. Published online Jul 28, 2026.

doi: 10.21037/qims-2026-1-0234

Introduction

High-precision medical image segmentation is essential for accurate computer-aided diagnosis and effective treatment planning (1,2). These segmentation results are vital for measuring organ volumes (3), assessing disease progression (4), and guiding surgical procedures (5). Despite its importance, the field encounters several challenges. The significant variations in anatomical structure morphology and size, coupled with low contrast between organs, demand models capable of adaptive feature understanding. Moreover, the severe class imbalance, due to the minimal presence of key targets such as small lesions, requires training objectives that prioritize challenging regions. Moreover, achieving a highly accurate segmentation boundary is crucial, as even minor errors may lead to clinical misjudgments (6,7). Consequently, developing a medical image segmentation system that fulfills clinical requirements remains a critical obstacle in advancing medical development (8,9).

Deep learning has become the primary catalyst for advancements in this field. Among the classic architectures, U-Net (10) and its variants stand out, employing an encoder-decoder architecture enhanced by skip connections to facilitate efficient feature extraction and spatial reconstruction. To enhance performance, researchers have developed various U-Net variants over recent years. UNet++ (11) addresses the semantic gap between the encoder and decoder by using nested dense skip connections. In contrast, Attention U-Net (12) incorporates an attention gating mechanism to better focus on key regions. Additionally, UNet3+ (13) employs full-scale skip connections to capture multi-scale features more effectively. Despite these innovations, convolutional neural network (CNN)-based models remain limited by the local receptive field of convolutional operations, which constrains their ability to model long-range dependencies (14-16). This intrinsic constraint has motivated the exploration of architectures capable of capturing global context.

In pursuit of global context modeling, initial efforts focused on enhancing CNNs themselves. Techniques such as dilated convolutions (17) to enlarge receptive fields, deformable convolutions (18) for adaptive spatial sampling, and attention mechanisms (19) to model feature correlations were developed. Although these methods alleviated some limitations and improved performance, they often introduced complexity and remained essentially supplementary within the local-operation paradigm of CNNs, failing to achieve a fundamental breakthrough in long-range dependency modeling.

A paradigm shift occurred with the introduction of Vision Transformers (ViT) (20) and their application to medical image segmentation. The self-attention mechanism of Transformers excels at capturing global context, effectively overcoming the core limitation of CNNs. This breakthrough spurred the development of hybrid and pure Transformer architectures. Notably, Transformer UNet (TransUNet) (21) pioneered a CNN-Transformer hybrid encoder to combine local features with global context. Swin Transformer UNet (Swin-UNet) (22) proposed a fully Transformer-based U-shaped architecture using a shifted window mechanism for efficiency. Cross-Shaped Window Transformer UNet (CSWin-UNet) (23) employed cross-shaped window (CSwin) attention for comprehensive global modeling. MINTFormer (24) incorporated a Demodulate Bridge to handle multi-scale features. These innovations collectively underscore that incorporating global modeling capabilities is now crucial for state-of-the-art medical image segmentation.

Although Transformer-based methods have shown promising results, they still face significant challenges. These include a fixed feature fusion strategy that cannot adapt to dynamic changes in feature importance, a standard Dice loss that treats all pixels equally and struggles to focus on difficult samples, and inadequate boundary precision optimization, which limits the clinical applicability of segmentation results. Our method builds upon CSWin-UNet, a recently proposed Transformer-based U-shaped architecture. The key architectural difference lies in the skip connection fusion: although CSWin-UNet uses simple concatenation, we introduce an adaptive feature fusion (AFF) module that dynamically weights encoder and decoder features.

Consequently, this paper introduces AFF Transformer U-Net (AFFUNet) with joint loss function for medical image segmentation, which mainly consists of three innovations. Firstly, an AFF module is proposed for dynamically learning fusion weights for encoder and decoder features, thereby enhancing the capacity of the model to represent complex boundaries and small-sized targets. Unlike channel attention mechanisms such as Convolutional Block Attention Module (CBAM) (19) and Squeeze-and-Excitation Network (SE-Net) (25), which operate on a single feature map, our AFF module is specifically designed to fuse two distinct features (encoder and decoder) and enforces mutually exclusive weighting via Softmax normalization. Secondly, the focal Dice loss function was developed to integrate focal weighting into the Dice loss framework. This integration allows the function to automatically concentrate on challenging samples. Last, a gradient-based boundary-aware loss was developed to explicitly optimize boundaries using efficient gradient operations, which can guide the model in learning precise boundary positioning, effectively improving key clinical indicators such as 95th percentile Hausdorff distance (HD95).

In conclusion, our core contributions are as follows:

An improved AFFUNet with joint loss function for medical image segmentation is proposed.
This framework integrates three complementary components: an AFF module for dynamic encoder-decoder feature weighting, a focal Dice loss for adaptive hard-sample reweighting, and a boundary-aware loss for explicit boundary refinement. These components are jointly optimized to address feature fusion, sample imbalance, and boundary precision in a coordinated manner.
Experiments on the Synapse and Automated Cardiac Diagnosis Challenge (ACDC) datasets validated the effectiveness of our methods and the extensive experimental results demonstrated that our proposed AFFUNet outperforms CSWin-UNet methods.

We present this article in accordance with the CLEAR reporting checklist (available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0234/rc).

Methods

Overall architecture of the model

An improved framework of AFFUNet with joint loss function for medical image segmentation is designed, as shown in Figure 1. As can be seen from Figure 1, the basic architecture of the proposed AFFUNet is inspired by CSWin-UNet (23), which effectively merges cross-shaped window attention with the U-Net architecture, showcasing strong global modeling capabilities in medical image segmentation. Three significant enhancements are proposed in the AFFUNet. First, an AFF is incorporated into the encoder-decoder path, as detailed in the module AFF section, replacing the original skip connection’s simple concatenation to enable dynamic fusion of multi-scale features. Second, regarding the optimization objective, we employ a combined loss function that integrates focal Dice loss and boundary-aware loss, detailed in the Loss Function subsections. The focal Dice loss increases focus on challenging-to-segment samples, whereas the boundary-aware loss enhances segmentation boundary accuracy.

Figure 1 Overall architecture of the proposed AFFUNet. AFF, adaptive feature fusion; AFFUNet, Adaptive Feature Fusion U-Net; CSWin, cross-shaped window; LN, LayerNorm; MLP, multi-layer perceptron.

The overall framework of AFFUNet is depicted in Figure 1, comprising an encoder, a decoder, and AFF modules. Our AFFUNet shares the same backbone and overall architecture as CSWin-UNet, with the key difference being the replacement of standard skip connection concatenation with the proposed AFF module. For single-channel medical images of size H × W × 1, the input is automatically transformed into multi-channel features during the forward pass. Inspired by CSWin-UNet (23), we first apply a convolutional token embedding layer with a 7×7 kernel and stride of 4 to generate patch tokens of dimension H/4 × W/4 with 64 channels. Both the encoder and decoder are structured into four stages. In the encoder, downsampling is achieved via convolutional layers using a 3×3 kernel and a stride of 2, which reduces the spatial resolution by half while doubling the number of channels. Correspondingly, the decoder employs Content-Aware Reassembly of Features (CARAFE) layers for upsampling, restoring the resolution to twice the input size and halving the channel count. Finally, a 4× CARAFE upsampling step is applied to recover the original input resolution H × W, followed by a linear layer to produce the final segmentation mask.

AFF

To tackle the semantic disparity between encoder and decoder features, we note two key challenges: the gap is both dynamic and multi-scale. This paper proposes the AFF module, as shown in Figure 2. This module dynamically learns the fusion weights of the encoder and decoder features through the channel attention mechanism, enabling the model to adaptively fuse multi-scale information based on the feature content.

Figure 2 AFF architecture diagram. AFF, adaptive feature fusion; GAP, global average pooling; MLP, multi-layer perceptron.

At the i-th stage, the encoder outputs a feature map, denoted as $F_{e}^{i} \in R^{(H_{i} \times W_{i} \times C_{i})}$ , where H_i and W_i indicates the height and width, and C_i represents the number of channels. After upsampling, the decoder’s corresponding feature $F_{d}^{i} \in R^{(H_{i} \times W_{i} \times {C^{'}}_{i})}$ has ${C^{'}}_{i}$ channels. The AFF module then computes the fusion features through the following steps.

Firstly, the features of along the channel dimension of the encoder and decoder features are concatenated, as shown in Eq. [1].

$F_{c}_{a t} = C o n c a t (F_{e}^{i}, F_{d}^{i}) \in R^{H_{i} \times W_{i} \times}^{(C_{i} + {C^{'}}_{i})}$ [1]

Secondly, to extract channel-level global features, global average pooling (GAP) is adopted to compress the spatial dimension H_i × W_i, as illustrated in Eq. [2].

$f_{g}_{l o b a l} = 1 / (H_{i} W_{i}) \sum_{h = 1}^{H_{i}} \sum_{w = 1}^{W_{i}} F_{c}_{a t} (h, w) \in R^{1 \times 1 \times}^{(C_{i} + {C^{'}}_{i})}$ [2]

Thirdly, fusion weights for the encoder and decoder features are produced using a two-layer multi-layer perceptron (MLP) with rectified linear unit (ReLU) activation. This process is followed by Softmax normalization to ensure proper distribution of the weights.

$[w_{e}, w_{d}] = Softmax (MLP (f_{global})), w_{e} + w_{d} = 1$ [3]

Here, $w_{e}$ and $w_{d} \in R^{(1 \times 1 \times 1)}$ represent scalar weights, which correspond to the fusion coefficients of $F_{e}^{i}$ and $F_{d}^{i}$ , respectively.

Fourthly, features are first assigned weights and concatenated. Subsequently, they are adjusted to match the number of target channels C_out using a linear projection layer, also known as a fully connected layer.

$F_{f u s e d}^{i} = L i n e a r (C o n c a t (w_{e} \cdot F_{e}^{i}, w_{d} \cdot F_{d}^{i})) \in R^{(H_{i} \times W_{i} \times C_{o}_{u t})}$ [4]

In this context, $\cdot$ denotes the multiplication of a scalar by a tensor. Typically, C_out is configured to equal the number of channels corresponding to the current decoding stage.

We incorporated the AFF module into every skip connection within the decoder, substituting the original splicing operation. During $i (i \in 1, 2, 3, 4)$ each decoding stage, the fusion process is defined as follows:

$F_{d e c o d e r}^{i} = A F F (F_{e n c o d e r}^{i}, F_{d e c o d e r}^{i - 1})$ [5]

This allows the model to perform AFF across different scales. In the shallow decoding stage, it emphasizes spatial details from the encoder. Conversely, in the deep stage, it prioritizes advanced semantic information from the decoder. By dynamically adjusting its focus between detailed and semantic information according to the input content, the model produces fused feature representations with enhanced information content. Thus, the model is endowed with the capacity to characterize complex boundaries and small-sized targets.

The design rationale of AFF is to enable dynamic, mutually exclusive weighting of encoder and decoder features, in contrast to single-feature attention mechanisms such as CBAM (19) and SE-Net (25). Experimental analysis of the AFF module is provided in the AFF analysis section.

Focal Dice loss

The focal Dice loss function effectively integrates the benefits of focal loss and Dice loss. By incorporating a focal weight mechanism, it automatically identifies and prioritizes challenging samples, such as boundary regions, small targets, or areas prone to confusion. These low-confidence samples receive higher weights, which significantly enhances the model’s segmentation accuracy, particularly for boundaries and intricate details. As the model’s proficiency grows during training, the proportion of simple samples increases, leading to a natural reduction in focal weight. This adjustment not only stabilizes the training process and prevents simple samples from dominating but also ensures that the model consistently concentrates on the features of difficult samples throughout its learning journey.

For an individual sample, the loss is defined as follows:

Consider a set of N pixels. For a given pixel, denoted as $n (n = 1, \dots, N)$ and its associated category $c (c = 1, \dots, C)$ , the model predicts a probability p_nc∈[0,1] that the pixel belongs to category C. The actual category is represented by a one-hot encoded label g_nc∈{0,1}.

We compute the probability of correct prediction $p_{t}^{(n, c)}$ for pixel n with respect to class c. For positive samples g_nc =1, the correct prediction probability equals the predicted probability p_nc; for negative samples g_nc =0, the correct prediction probability equals one minus the predicted probability $1 - p_{n}_{c}$ . Hence, the correct prediction probability is defined as:

$p_{t}^{(n, c)} = p_{n}_{c} \cdot g_{n}_{c} + (1 - p_{n}_{c}) \cdot (1 - g_{n}_{c})$ [6]

Following the formulation of focal Dice loss (26), we define the focal weight $w_{n}^{(c)}$ for pixel n with respect to class c based on the predicted probability for the ground-truth class, which is calculated as:

$w_{n}^{(c)} = {(1 - p_{t}^{(n, c)})}^{γ}$ [7]

Here, γ is the focusing parameter, which controls the degree of focus on hard samples. As the probability of correct prediction $p_{t}^{(n, c)}$ decreases, the weight $w_{n}^{(c)}$ increases, thereby assigning higher importance to hard samples (i.e., low-confidence pixels) during training.

For class c, the weighted Dice loss is defined as:

$L_{{_{F}}_{o c a l D i c e}}^{(c)} = 1 - \frac{2 \sum_{n = 1}^{N} w_{n}^{(c)} p_{n c} g_{n c} + ε}{\sum_{n = 1}^{N} w_{n}^{(c)} p_{n c}^{2} + \sum_{n = 1}^{N} w_{n}^{(c)} g_{n c}^{2} + ε}$ [8]

Here, ε denotes a small constant to ensure numerical stability. The weight $w_{n}^{(c)}$ is applied to both the intersection term p_ncg_ncand the union terms $p_{n c}^{2}$ and $g_{n c}^{2}$ .

The final focal Dice loss is computed as the weighted average of the loss across all classes:

$L_{F}_{o c a l D i c e} = \frac{1}{C} \sum_{c = 1}^{C} L_{F o c a l D i c e}^{(c)}$ [9]

By setting γ=2.0 as the standard value, this loss function enables the training process to naturally focus on regions that are challenging to segment (such as blurred boundaries or small objects) by assigning higher weights to low-confidence predictions. This per-class loss computation allows the model to perform adaptive optimization tailored to the characteristics of each category.

Boundary-aware loss

Medical images often display low contrast and ambiguous boundaries, especially for small or irregularly shaped organs such as the pancreas and gallbladder. Standard pixel-wise losses such as Dice or cross-entropy alone cannot guarantee contour accuracy because they treat all pixels equally and do not explicitly penalize boundary misalignment. To address this, we propose a boundary-aware loss that directly supervises the gradient field of the prediction using a differentiable Sobel operator.

Several existing loss functions also target boundary quality. Kervadec et al. (27) proposed a boundary loss based on distance transform, which requires pre-computing a distance map and does not directly supervise the gradient field of the prediction. Other losses such as Unified Focal Loss (28) and Focal Tversky Loss (29) address class imbalance and hard pixel classification but lack explicit boundary modeling. In contrast, our Sobel-based loss is fully differentiable, directly aligns predicted and true gradient responses within the true boundary region, and has a computational complexity of O(HW). A detailed comparison is provided in the Discussion section.

The boundary-aware loss function is crafted to improve the boundary quality in medical image segmentation by directly optimizing the prediction accuracy in boundary regions. Initially, it uses the differentiable Sobel operator to calculate the gradient magnitudes of both the predicted probability map and the ground truth label. By minimizing the difference between these gradients within the actual boundary region, the method enhances boundary accuracy. Since the gradients are backpropagated to the network parameters, this approach facilitates end-to-end training. Additionally, by concentrating solely on the true boundary region, it effectively reduces background noise interference, thereby enhancing the stability and effectiveness of the training process.

To extract boundaries from the prediction probability graph, we employ differentiable Sobel operators. For the predicted probability graph P_c∈R^H×W of category c, the gradient magnitude graph $B_{p r e d}^{(c)}$ , representing the predicted boundary response, is computed as follows:

$B_{p r e d}^{(c)} = \sqrt{{(P_{c} * S_{x})}^{2} + {(P_{c} * S_{y})}^{2} + δ}$ [10]

The convolution operation * is denoted here, where S_x and S_y signify the Sobel convolution kernels. Additionally, δ serves as a numerical stability constant.

To address the divergence of the binary mask G_c gradient in discrete space, we employ the Sobel operator. This application yields a continuous boundary response map, $B_{g t}^{(c)}$ , where high-value regions accurately align with the actual anatomical boundaries.

In calculating the loss for each foreground category C (excluding the background), the boundary loss is determined by the mean absolute error between the predicted and actual boundary response maps. This calculation is confined to the effective boundary region.

$L_{b o u n d a r y}^{(c)} = \frac{1}{| M^{(c)} |} \sum_{(i, j) \in M^{(c)}} | B_{p r e d}^{(c)} (i, j) - B_{g t}^{(c)} (i, j) |$ [11]

The effective boundary area $M^{(c)} = {(i, j) : B_{g t}^{(c)} (i, j) > τ}$ is defined by the threshold τ, which we set at τ=0.01 to eliminate minor gradient noise resulting from discretization. The boundary awareness loss is then calculated as the mean of all the losses across the foreground categories.

$L_{B o u n d a r y} = \frac{1}{C - 1} \sum_{c = 1}^{C - 1} L_{b o u n d a r y}^{(c)}$ [12]

In contrast to the distance transformation-based method (27), this approach is fully differentiable and exhibits low computational complexity O(N). It enables efficient end-to-end joint optimization with the primary segmentation network, thereby enhancing boundary segmentation quality by directly aligning predicted and actual gradient responses.

Joint loss function

To maximize the benefits of the previously discussed loss functions, we have combined them into a cohesive loss function framework. The overall loss, denoted as L_total, consists of a weighted sum of two components: focal Dice loss, and boundary awareness loss.

$L_{t o t a l} = λ_{d i c e} L_{F o c a l D i c e} + λ_{b o u n d a r y} L_{B o u n d a r y}$ [13]

The weight coefficients, denoted as λ_dice =0.6, λ_boundary =0.4, are established through ablation experiments. These weights are designed to balance each loss’s contribution to the overall objective and maintain training stability. Specifically, the FocalDiceLoss receives the highest weight, serving as the primary segmentation loss. Meanwhile, the boundary-aware loss, with smaller weights, introduces precise boundary constraints.

Datasets

This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. We evaluated the proposed method using the Synapse multi-organ segmentation dataset (23) and the ACDC dataset (30).

The Synapse dataset contains 30 abdominal computed tomography (CT) scans labeled with eight organs: aorta, gallbladder, spleen, left kidney, right kidney, liver, pancreas, and stomach. The dataset comprises a total of 3,779 axial CT slices. Following the standard split used in TransUNet and CSWin-UNet, we used 18 volumes (2,212 slices) for training and 12 volumes (1,567 slices) for testing. All slices are fully annotated for all eight organ classes; there are no unannotated slices. Each CT volume consists of multiple slices, from which we extracted two-dimensional (2D) slices for both training and evaluation. This dataset poses significant challenges due to variations in organ size, low contrast between adjacent structures, and severe category imbalance, making it an ideal benchmark for assessing medical image segmentation methods.

The ACDC dataset, released during the 2017 challenge, is a multi-category three-dimensional (3D) cardiac magnetic resonance imaging (MRI) dataset comprising 100 sets of short-axis cine MR images acquired with 1.5 Tesla (T) and 3T scanners. Medical experts provided manual annotations for three cardiac structures: the right ventricle (RV), the myocardium (MYO), and the left ventricle (LV) (30). We randomly selected 70 cases for training, 10 for validation, and 20 for testing. Following standard practice, slices at the extreme ends of the cardiac volume that contain no annotations are excluded from training and evaluation. Following the common evaluation protocol for this dataset, the mean Dice similarity coefficient (DSC) is used as the primary metric to assess segmentation accuracy for the three cardiac structures.

Results

Implementation details

The proposed AFFUNet is implemented using the PyTorch framework. All models were trained on an NVIDIA RTX 3090 graphics processing unit (GPU) (NVIDIA, Santa Clara, CA, USA) with mixed precision enabled. Input images were resized to 224×224. For the Synapse dataset, we followed the standard split used in previous works (21,23), using 18 cases for training and 12 for testing. Since Synapse contains only 30 cases in total, no separate validation set was held out from this dataset. For the ACDC dataset, we used a three-way split similar to TransUNet (21): 70 cases for training, 10 for validation, and 20 for testing.

Most training hyperparameters were taken directly from CSWin-UNet (23), which serves as our baseline. Specifically, we trained all models for 140 epochs using Stochastic Gradient Descent (SGD) with a momentum of 0.9 and a weight decay of 1e−4. The initial learning rate was 0.05 and decayed polynomially following (1−t/T)^0.9, where t is the current iteration and T the total number of iterations. The batch size was 24. During training, we applied the following data augmentations in sequence: random rotation with an angle uniformly sampled from [−15°, +15°], random horizontal flip (probability 0.5), random vertical flip (probability 0.5), and random brightness adjustment with a scaling factor uniformly sampled from [0.8, 1.2]. The same augmentation pipeline was applied to both Synapse and ACDC datasets.

The loss-specific hyperparameter—the focal parameter γ and the loss weights λ_dice and λ_boundary—required tuning. Since Synapse does not have a dedicated validation set, we performed a grid search on the ACDC validation set (10 cases). For γ, we tried {1.0, 2.0, 3.0} and selected γ=2.0, which gave the highest Dice on the validation data. For the loss weights, we searched λ_dice∈{0.55,0.6,0.65,0.7} with λ_boundary =1− λ_dice, and chose (0.6, 0.4) based on validation performance. These hyperparameters were then fixed and used for both datasets. The Synapse test set was used only once, after all design choices were finalized, to report the final results. The same holds for the ACDC test set, which was kept completely separate from validation.

To verify that our hyperparameter choices were not over-fitted to the small ACDC validation set, we also performed 5-fold cross-validation on the Synapse training set using the selected hyperparameters (γ=2.0, λ_dice =0.6, λ_boundary =0.4). The mean Dice across the five folds was 80.92%±0.43%, close to the final test result of 81.21%, confirming that the chosen settings are robust. Table 1 presents the hyperparameter settings.

Table 1

Hyperparameter setting table

Hyperparameter	Value
Inputsize^†	224×224
Batchsize^†	24
Learningrate^†	0.05
Optimizer^†	SGD (momentum =0.9, weight_decay =0.0001)
Epochs^‡	140
Focal Dice γ^‡	2.0
Lossweights (λ_dice, λ_boundary)^‡	(0.6, 0.4)

^†, indicates hyperparameters adopted from CSWin-UNet (23); ^‡, indicates hyperparameters selected via grid search on the ACDC validation set. ACDC, Automated Cardiac Diagnosis Challenge; CSWin-UNet, Cross-Shaped Window Transformer UNet; SGD, stochastic gradient descent.

Comparison with the state-of-the-art methods

Quantitative results

To demonstrate the superiority of the proposed AFFUNet, we conducted extensive comparative experiments against state-of-the-art deep neural network models for medical image segmentation in recent years, with the experimental results presented in Table 2.

Table 2

Comparison results on the Synapse dataset (Dice ↑, HD95 ↓)

Method	Dice (%)	HD95 (%)	Aorta (%)	Gallbladder (%)	Kidney (L) (%)	Kidney (R) (%)	Liver (%)	Pancreas (%)	Spleen (%)	Stomach (%)
Att-UNet (12)	77.77	36.02	89.55	68.88	77.98	71.11	93.57	58.04	87.3	75.75
TransUNet (20)	77.48	31.65	87.23	63.13	81.87	77.02	94.08	55.86	85.08	75.62
Swin-UNet (21)	79.13	21.55	85.47	66.53	83.28	79.61	94.29	56.58	90.66	76.60
CSWin-UNet (23)	81.12	18.86	87.13	67.85	83.51	78.53	95.23	65.94	89.05	81.74
Diff-UNet (31)	79.20	24.03	76.51	72.10	81.51	83.72	76.20	81.11	83.10	79.10
nn-UNet (7)	78.53	18.35	83.36	79.37	76.42	90.14	88.90	66.45	76.35	67.28
AFFUNet (ours), mean ± SD	81.21±4.55	18.41±10.31	86.25±2.88	68.94±6.23	82.15±12.31	80.02±13.79	93.60±3.41	65.48±1.98	90.92±5.45	82.03±1.70

↑, higher is better; ↓, lower is better. AFFUNet, Adaptive Feature Fusion U-Net; Att-UNet, Attention UNet; CSWin-UNet, Cross-Shaped Window Transformer UNet; Diff-UNet, Diffusion UNet; HD95, 95th percentile Hausdorff distance; L, left; nn-UNet, no-new UNet; R, right; SD, standard deviation; Swin-UNet, Swin Transformer UNet; TransUNet, Transformer UNet.

Table 2 presents the quantitative comparison results on the Synapse dataset. Under the standard 2D slice-based protocol, our AFFUNet achieved a mean Dice of 81.21% and a mean HD95 of 18.41 mm, outperforming all compared 2D Transformer U-Net variants. Compared to the baseline CSWin-UNet (81.12% Dice, 18.86 mm HD95), AFFUNet improved Dice by 0.09% and reduced HD95 by 0.45 mm. Notably, our method consistently improved performance across most organ categories, with particular gains on the spleen (90.92% vs. 89.05%) and stomach (82.03% vs. 81.74%) compared to CSWin-UNet.

We also include results from two strong 3D baselines, nnU-Net (78.53% Dice, 18.35 mm HD95) and Diffusion UNet (Diff-UNet) (79.20% Dice, 24.03 mm HD95). However, these values are not directly comparable to our 2D slice-based results, as both nnU-Net and Diff-UNet are native 3D architectures that leverage inter-slice context, automated pipeline optimization, and volumetric training. The moderate performance gap between our method and these 3D baselines primarily reflects differences in dimensionality and pipeline design, rather than a failure of the proposed AFF module.

Nevertheless, when compared under the same 2D protocol, our method consistently outperforms other Transformer U-Net variants (e.g., CSWin-UNet, Swin-UNet, TransUNet), confirming the effectiveness of the proposed AFFUNet.

Table 3 presents the quantitative comparison results on the ACDC dataset. Under the standard 2D slice-based protocol, our AFFUNet achieves a mean Dice of 89.77%, outperforming CSWin-UNet (88.44%) and substantially surpassing Diff-UNet (52.76%), which was originally designed for 3D segmentation. Notably, AFFUNet obtains the highest Dice scores among the compared 2D methods for all three cardiac structures: RV (88.75%), MYO (87.44%), and LV (93.12%).

Table 3

Comparison results on the ACDC dataset (Dice ↑)

Method	DSC (%) ↑	RV (%)	MYO (%)	LV (%)
Att-UNet (12)	89.47	87.40	79.20	93.61
TransUNet (20)	89.71	88.86	84.53	95.73
Swin-UNet (21)	90.00	88.55	85.62	95.83
CSWin-UNet (23)	88.44	86.70	85.92	92.71
Diff-UNet (31)	52.76	42.18	48.72	67.39
nn-UNet (7)	97.90	98.09	96.94	98.68
AFFUNet (ours), mean ± SD	89.77±0.51	88.75±0.84	87.44±0.72	93.12±0.56

↑, higher is better. ACDC, Automated Cardiac Diagnosis Challenge; AFFUNet, Adaptive Feature Fusion U-Net; Att-UNet, Attention UNet; CSWin-UNet, Cross-Shaped Window Transformer UNet; Diff-UNet, Diffusion UNet; DSC, Dice Similarity Coefficient; LV, left ventricle; MYO, myocardium; nn-UNet, no-new UNet; RV, right ventricle; SD, standard deviation; Swin-UNet, Swin Transformer UNet; TransUNet, Transformer UNet.

For reference, we also list nnU-Net (97.90%) as reported in its original paper. However, this value is not directly comparable to our 2D slice-based results, as nnU-Net is a native 3D architecture that leverages inter-slice context, automated pipeline optimization, and full-resolution volumetric training. On cardiac MRI, where cross-slice anatomical consistency is critical, 2D methods are structurally disadvantaged. Therefore, the moderate performance gap between our 2D method and these 3D strong baselines primarily reflects differences in dimensionality and pipeline design, rather than a failure of the proposed AFF module.

Nevertheless, when compared under the same 2D protocol, our method consistently outperforms other Transformer U-Net variants (e.g., CSWin-UNet, Swin-UNet, TransUNet) on both Synapse (abdominal CT) and ACDC (cardiac MRI), confirming that the advantage of our method generalizes across different imaging modalities and anatomical tasks.

Qualitative results

Furthermore, for an intuitive analysis of the models’ segmentation results on the test set, we conducted a visual analysis of the segmentation outputs for the representative models (Table 2), with the results presented in Figure 3. As shown in Figure 3, our approach delivered more precise segmentation outcomes, particularly excelling in the segmentation of small organs such as the pancreas and gallbladder, as well as in boundary regions. When compared to CSWin-UNet, our method demonstrated several distinct advantages: The focal Dice loss mechanism significantly improved the segmentation accuracy of small organs by allowing the model to concentrate on challenging samples. By explicitly optimizing boundary accuracy, the boundary-aware loss function enhanced the precision and sharpness of boundaries. This improvement is evident in the magnified regions: the AFF module significantly improves the efficiency of feature combination, thereby comprehensively enhancing segmentation quality.

Figure 3 Display of segmentation results on the synapse dataset. L, left; R, right.

Ablation experiments

Effectiveness of each component

To assess the effectiveness of each component, we performed ablation studies by incrementally integrating improvement schemes into the baseline model. The results of this detailed ablation experiment are displayed in Table 4.

Table 4

Results of the complete ablation experiment

Methods	Mean Dice (%)	Mean HD95 (mm)
Baseline	81.12	18.86
+AFF	80.46	22.22
+Focal Dice	80.86	22.15
+Boundary	80.48	21.36
+AFF+focal Dice	81.11	21.56
+AFF+boundary	81.02	20.35
+Focal Dice+boundary	81.13	21.03
Full method, mean ± SD	81.21±4.55	18.41±10.31

AFF, adaptive feature fusion; HD95, 95th percentile Hausdorff distance.

The analysis results revealed several key findings: First, when used alone, each component slightly reduces performance compared to the baseline: AFF by 0.66%, focal Dice by 0.26%, and boundary loss by 0.64% (Table 4). Second, despite their individual negative effects, the full combination of all three components achieved a +0.09% Dice improvement over the baseline, demonstrating a clear synergistic effect. This suggests that the components are designed to complement each other rather than work in isolation.

AFF analysis

In standard U-shaped architectures, skip connections usually concatenate or add encoder and decoder features directly. This works well in many cases, but the fusion is fixed regardless of the input content or network depth. In practice, shallow stages may need more spatial details from the encoder, whereas deep stages may rely more on semantic information from the decoder. To handle this, we designed an AFF module that learns fusion weights dynamically using a channel attention mechanism.

To further evaluate the AFF module, we visualized the fusion weights learned at various stages of the decoder, as shown in Figure 4. These weights differ across samples and training phases, demonstrating the module’s ability to adaptively modify the fusion strategy.

Figure 4 Fusion weights learned at various stages of the decoder.

We compared our AFF module against four simpler fusion alternatives on the Synapse dataset. All methods used the same CSWin-UNet backbone and training settings. The alternatives were: (I) concatenation followed by a 3×3 convolution; (II) a 1×1 convolution on the concatenated features; (III) element‑wise average or max fusion; (IV) a lightweight two-layer MLP on the concatenated features, without GAP or Softmax.

Table 5 shows the results. Our AFF module gave the best performance: 81.21% Dice and 18.41 mm HD95. Compared to concatenation with convolution, AFF improved Dice by 2.37% and cut HD95 by 13.25 mm. Compared to the lightweight MLP without attention, the gains were 2.87% for Dice and 17.39 mm for HD95. Average and max fusion performed comparably to concatenation-based methods but still fall short of AFF. These results support the benefit of adaptive weighting using channel attention and Softmax normalization.

Table 5

Comparison with alternative fusion methods

Fusion method	Dice (%)	HD95 (mm)	Aorta (%)	Gallbladder (%)	Kidney (L) (%)	Kidney (R) (%)	Liver (%)	Pancreas (%)	Spleen (%)	Stomach (%)
Concatenation + Conv	78.84	31.66	85.56	67.76	79.73	74.74	94.59	63.9	86.39	78.02
1×1 Convolution	78.43	30.4	85.74	61.2	78.97	74.08	94.41	63.69	88.03	81.34
Average fusion	78.95	30.77	85.35	62.4	80.61	73.98	94.13	64.41	89.5	81.21
Max fusion	78.87	28.91	85.61	62.08	82.74	76.82	94.23	62.0	87.74	79.77
Lightweight MLP (no attn)	78.34	35.8	85.4	63.53	80.04	73.38	94.08	63.75	85.73	80.79
AFF (ours), mean ± SD	81.21±4.55	18.41±10.31	86.25±2.88	68.94±6.23	82.15±12.31	80.02±13.79	93.60±3.41	65.48±1.98	90.92±5.45	82.03±1.70

AFF, adaptive feature fusion; attn, attention; Conv, convolution; HD95, 95th percentile Hausdorff distance; L, left; MLP, multi-layer perceptron; R, right; SD, standard deviation.

We also examined how much each part of the AFF module matters by testing three variants. The first variant removes attention entirely—no GAP, no MLP, no Softmax—and simply concatenates the features followed by a linear projection. The second variant keeps the MLP but drops Softmax, using the raw MLP outputs as fusion weights. The third is our full AFF module.

Table 6 shows the results. Removing the whole attention mechanism hurt performance noticeably: Dice dropped by 2.87% and HD95 rose by 17.39 mm. This suggests that attention is indeed necessary. Dropping Softmax also caused a clear drop: Dice reduced by 3.14% and HD95 increased by 10.90 mm. This indicates that mutually exclusive weighting (enforced by Softmax) helps balance the encoder and decoder contributions. The full AFF gave the best performance, confirming that each component plays a positive role.

Table 6

Component-wise ablation analysis

Variant	Dice (%)	HD95 (mm)	ΔDice	ΔHD95
AFF without attention	78.34	35.8	−2.87	+17.39
AFF without Softmax	78.07	29.31	−3.14	+10.9
Full AFF, mean ± SD	81.21±4.55	18.41±10.31	–	–

AFF, adaptive feature fusion; HD95, 95th percentile Hausdorff distance; SD, standard deviation.

Analysis of focal Dice loss

We analyzed the influence of the focal Dice loss parameters on the segmentation performance. Table 7 presents the results under different parameter settings.

Table 7

Parameter sensitivity analysis of focal Dice loss

\|γ\|	Mean Dice (%)	Mean HD95 (mm)
1.0	78.81	30.23
2.0	81.21	18.41
3.0	79.24	28.59

HD95, 95th percentile Hausdorff distance.

The default settings of γ=2.0 strike an ideal balance between emphasizing challenging samples and ensuring training stability. If γ is reduced to 1.0, the loss function reverts to the standard Dice loss, offering only marginal improvement. Conversely, increasing γ to 3.0 can cause the model to overly concentrate on difficult samples, which may result in unstable training and diminished performance.

Figure 5 illustrates the distribution characteristics of focal weights in the focal Dice loss function and their dynamic changes throughout the training process. Figure 5A reveals a significant decrease in the average focal weight, dropping from around 0.1 in the early training stages to between 0.002 and 0.006 later on. This trend reflects an overall enhancement in the model’s prediction confidence. Figure 5B,5C depict the focal weight distributions across different prediction confidence intervals at both early and later training stages. In both cases, samples with lower confidence are assigned higher weights. Notably, the peak weight during the early training stage is approximately 0.25, which is higher than the later stage’s peak of around 0.0075, indicating a gradual improvement in the model’s capability. Figure 5D demonstrates a decrease in the proportion of difficult samples (where p_t <0.5) from approximately 2.8% at the outset, with a brief peak of about 6% within the first few epochs, to between 0.2% and 1.0% later. This trend aligns with the reduction in average focal weight. These findings suggest that the focal Dice loss function effectively identifies difficult samples and assigns appropriate weights.

Figure 5 Distribution of focal weights throughout the training process. (A) Average focal weight vs. training; (B) early training weight distribution; (C) late training weight distribution; (D) hard sample ratio changes.

Analysis of boundary-aware loss

To assess the effectiveness of the boundary-aware loss, we conducted a comparison between models trained with and without this loss component.

Figure 6 illustrates the dynamic characteristics of boundary-aware loss throughout the training process and its effect on model performance. In Figure 6A, decreasing from approximately 0.004 at the beginning of training (with an early peak of about 0.010 within the first few epochs) to roughly 0.001–0.005 towards the end. This pattern indicates a gradual improvement in the model’s boundary prediction accuracy. Figure 6B depicts the contribution ratio of the boundary loss to the total loss, which starts at approximately 2% initially and generally increases as training progresses. From mid-to-late training onward, the ratio fluctuates considerably, frequently reaching 15–26% at peaks while occasionally dropping to 1–5%. This dynamic change highlights the synergistic interaction between the boundary loss and the focal Dice loss.

Figure 6 Boundary loss throughout various training stages. (A) Boundary loss vs. training; (B) boundary loss contribution.

Analysis of loss weights

Table 8 shows how different combinations of λ_dice and λ_boundary affect segmentation performance on the Synapse test set, using the weights determined from the ACDC validation set as described in the Implementation Details section.

Table 8

Sensitivity analysis of loss function weights

λ_dice	λ_boundary	Mean Dice (%)	Mean HD95 (mm)
0.55	0.45	80.29	20.19
0.6	0.4	81.21	18.41
0.65	0.35	80.35	21.56

HD95, 95th percentile Hausdorff distance.

We initially observed catastrophic sensitivity to loss weight variations due to the lack of gradient clipping, which caused gradient explosions for off-optimal weights. After enabling gradient clipping (max norm =1.0) and performing hyperparameter search on the ACDC validation set, the loss weight sensitivity became stable. As shown in Table 8, across λ_dice∈{0.55, 0.6, 0.65}, the optimal configuration remained (0.6, 0.4), achieving 81.21% Dice and 18.41 mm HD95. This stability confirms that the reported optimal configuration is not an artifact of the small test set. This suggests that over‑weighting the Dice loss at the expense of boundary loss compromises boundary quality, whereas under-weighting it harms overall region accuracy. The chosen combination of (0.6, 0.4) provides a practical balance between the two objectives.

Computational cost analysis

Table 9 compares the computational cost of our AFFUNet with the baseline CSWin-UNet. The parameters and floating point operations per second (FLOPs) remain virtually unchanged. The primary benefit of our AFF module is improved segmentation accuracy without adding clinically meaningful computational cost. The minor increase in inference time is negligible in clinical practice, as discussed in the Discussion section.

Table 9

Comparison of model performance

Method	Params (M)	FLOPs (G)
CSWin-UNet	23.568	4.723
AFFUNet (ours)	23.548	4.687
Difference	−0.020 (−0.08%)	−0.036 (−0.76%)

AFFUNet, Adaptive Feature Fusion U-Net; CSWin-UNet, Cross-Shaped Window Transformer UNet; FLOPs, floating point operations per second.

Discussion

Our AFFUNet achieves consistent improvements over the baseline CSWin-UNet on both Synapse (Dice +0.09%, HD95 −0.45 mm) and ACDC (Dice +1.33%). This section analyzes the contributions of the three components, compares our method with strong baselines, and discusses efficiency and limitations.

Component contributions and synergy

AFF: the AFF module replaces standard skip-connection concatenation with channel-wise global weighting derived via GAP and Softmax normalization. Compared to simpler alternatives (concatenation + convolution, 1×1 convolution, average/max fusion, and a lightweight MLP without attention), AFF gives the best Dice (81.21%) and lowest HD95 (18.41 mm) on Synapse (Table 5).
Focal Dice loss: the focal parameter γ=2.0 (Table 7) automatically down-weights easy samples and focuses on hard-to-segment pixels. The average focal weight decays from 0.1 early in training to 0.002-0.006 later, and the proportion of difficult samples drops from 2.8% to about 0.5% (Figure 5), demonstrating effective adaptation.
Boundary-aware loss: using a differentiable Sobel operator, the boundary loss directly aligns predicted and true gradient responses within the true boundary region. The boundary loss value decreases from approximately 0.004 to 0.001–0.005 over training, while its contribution to the total loss rises from about 2% initially to predominantly 15–26% in later stages (Figure 6).

The standalone changes are AFF −0.66%, focal Dice −0.26%, and boundary loss −0.64% (Table 4). Despite each component slightly reducing performance when used alone, their full combination achieves a +0.09% Dice improvement, demonstrating a clear synergistic effect. Moreover, when AFF is added to the two loss functions (without AFF, the gain over baseline is 0.01%; with AFF it reaches 0.73%), the additional benefit of AFF (+0.01%) is greater than its standalone effect (−0.66%). This confirms that AFF primarily enables the loss functions to work more effectively, rather than acting alone.

Comparison with strong baselines and boundary-sensitive losses

Future work will include a systematic comparison with strong baselines such as nnU-Net and Diff-UNet under unified protocols.

Boundary-sensitive losses

Kervadec et al.’s boundary loss (27) requires a pre-computed distance map and does not directly supervise gradient fields. Unified Focal Loss (28) and Focal Tversky Loss (29) handle class imbalance but lack explicit boundary modeling. A direct Hausdorff distance reduction (32) is computationally expensive and may be unstable. Our Sobel-based loss is fully differentiable, O(HW) complexity, and can be combined with those losses orthogonally.

Computational efficiency and clinical practicality

The AFF module introduced negligible overhead (Table 9): parameters and FLOPs were virtually unchanged (−0.02 M, −0.036 G). For a typical CT volume of 100–200 slices, the total added delay was imperceptible to a radiologist. GPU memory usage slightly decreased. Thus, the accuracy gain was achieved without clinically meaningful computational cost.

Limitations and future work

Dataset size and 2D nature

Our primary validation on Synapse (30 CT scans) is limited. We mitigate this by providing additional results on ACDC (100 cardiac MRIs). However, the 2D slice-wise approach cannot capture inter-slice consistency. Future work will extend the method to 3D using pseudo-3D (2.5D inputs), lightweight 3D CSWin blocks, or test-time volumetric smoothing.

Missing baselines

Direct comparisons with TotalSegmentator, UNETR, Swin-UNETR, U-Mamba, SAM-based methods, and so on, are not included due to time constraints. We plan a systematic benchmark under unified protocols in a follow-up study.

Small and low-contrast targets

Performance for extremely small organs (e.g., pancreatic lesions) remains improvable. Multi-scale training and specialized small-object detection mechanisms are under investigation.

Other modalities

Cross-modality validation on cardiac MRI is encouraging; generalization to ultrasound and positron emission tomography requires multi-center prospective studies.

Organ-wise boundary evaluation

Organ-wise HD95 analysis has not been performed in this study due to time constraints; a comprehensive boundary evaluation on a per-organ basis is planned as future work.

Conclusions

This paper introduces the AFFUNet framework, of which the core contribution is the system-level integration of three collaborative components—an AFF module, a focal Dice loss, and a boundary-aware loss—to tackle the interrelated challenges of feature fusion, hard sample learning, and boundary accuracy in medical image segmentation. We evaluated our method on two benchmark datasets: Synapse (abdominal CT) and ACDC (cardiac MRI). On both datasets, AFFUNet consistently outperformed the baseline CSWin-UNet. These results demonstrate significant performance enhancements in segmenting small target organs and boundaries, which hold substantial clinical practical value. Current research faces two primary challenges: enhancing adaptability to very small targets and low-contrast regions, and improving generalization performance through cross-dataset validation. Moving forward, we aim to develop this method in two key areas: expanding into 3D applications and incorporating multi-modal approaches. Furthermore, we will focus on boosting the framework’s robustness and universality by optimizing the learning mechanism for challenging samples and refining the boundary processing strategy.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the CLEAR reporting checklist. Available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0234/rc

Funding: This work was supported by the Innovation Project of Shanxi Provincial Higher Education Institutions (No. 2023L453).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0234/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60-88. [Crossref] [PubMed]
Zhou SK, Greenspan H, Davatzikos C, Duncan JS, van Ginneken B, Madabhushi A, Prince JL, Rueckert D, Summers RM. A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises. Proc IEEE Inst Electr Electron Eng 2021;109:820-38.
Wendler T, Kreissl MC, Schemmer B, Rogasch JMM, De Benetti F. Artificial Intelligence-powered automatic volume calculation in medical images - available tools, performance and challenges for nuclear medicine. Nuklearmedizin 2023;62:343-53. [Crossref] [PubMed]
Li MD, Chang K, Bearce B, Chang CY, Huang AJ, Campbell JP, Brown JM, Singh P, Hoebel KV, Erdoğmuş D, Ioannidis S, Palmer WE, Chiang MF, Kalpathy-Cramer J. Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging. NPJ Digit Med 2020;3:48. [Crossref] [PubMed]
Asgari Taghanaki S, Abhishek K, Cohen JP, Cohen-Adad J, Hamarneh G. Deep semantic segmentation of natural and medical images: a review. Artif Intell Rev 2021;54:137-78.
Maier-Hein L, Eisenmann M, Reinke A, Onogur S, Stankovic M, Scholz P, et al. Why rankings of biomedical image analysis competitions should be interpreted with care. Nat Commun 2018;9:5217. [Crossref] [PubMed]
Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 2021;18:203-11. [Crossref] [PubMed]
Milletari F, Navab N, Ahmadi SA. V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA. IEEE; 2016:565-71.
Lin TY, Goyal P, Girshick R, He K, Dollar P. Focal Loss for Dense Object Detection. IEEE Trans Pattern Anal Mach Intell 2020;42:318-27. [Crossref] [PubMed]
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention 2015:234-241.
Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support (2018) 2018;11045:3-11. [Crossref] [PubMed]
Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B, Glocker B, Rueckert D. Attention u-net: Learning where to look for the pancreas. arXiv:1804.03999 [Preprint]. 2018 Apr 11. Available online: https://doi.org/10.48550/arXiv.1804.03999
Huang H, Lin L, Tong R, Hu H, Zhang Q, Iwamoto Y, Han X, Chen YW, Wu J. Unet 3+: A full-scale connected unet for medical image segmentation. ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain. IEEE; 2020:1055-9.
Azad R, Fayjie AR, Kauffmann C, Ben Ayed I, Pedersoli M, Dolz J. On the texture bias for few-shot cnn segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2021:2674-83.
Zhao H, Shi J, Qi X, Wang X, Jia J. Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2017:2881-90.
Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S. End-to-end object detection with Transformers. In: European Conference on Computer Vision 2020 :213-229.
Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Trans Pattern Anal Mach Intell 2018;40:834-48. [Crossref] [PubMed]
Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y. Deformable convolutional networks. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy. IEEE; 2017:764-73.
Woo S, Park J, Lee JY, Kweon IS. CBAM: Convolutional Block Attention Module. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y. (eds) Computer Vision – ECCV 2018. Springer, Cham; 2018:3-19.
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929 [Preprint]. 2020. Available online: https://arxiv.org/pdf/2010.11929/100
Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, Lu L, Yuille AL, Zhou Y. Transunet: Transformers make strong encoders for medical image segmentation. arXiv:2102.04306 [Preprint]. 2021. Available online: https://doi.org/10.48550/arXiv.2102.04306
Cao H, Wang Y, Chen J, Jiang D, Zhang X, Tian Q, Wang M. Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation. In: European Conference on Computer Vision 2022:205-18.
Liu X, Gao P, Yu T, Wang F, Yuan RY. CSWin-UNet: Transformer UNet with cross-shaped windows for medical image segmentation. Information Fusion 2025;113:102634.
Deng C, Qin X. MINTFormer: Multi-Scale Information Aggregation with CSWin Vision Transformer for Medical Image Segmentation. Appl Sci 2025;15:8626.
Hu J, Shen L, Sun G. Squeeze-and-excitation networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. IEEE; 2018:7132-41.
Wang P, Chung AC. Focal Dice Loss and Image Dilation for Brain Tumor Segmentation. In: International Workshop on Deep Learning in Medical Image Analysis 2018:119-27.
Kervadec H, Bouchtiba J, Desrosiers C, Granger E, Dolz J, Ben Ayed I. Boundary loss for highly unbalanced segmentation. Med Image Anal 2021;67:101851. [Crossref] [PubMed]
Yeung M, Sala E, Schönlieb CB, Rundo L. Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation. Comput Med Imaging Graph 2022;95:102026. [Crossref] [PubMed]
Abraham N, Khan NM. A Novel Focal Tversky Loss Function With Improved Attention U-Net for Lesion Segmentation. 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019) 2019:683-687. doi: 10.1109/ISBI.2019.8759329.
Bernard O, Lalande A, Zotti C, Cervenansky F, Yang X, Heng PA, et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans Med Imaging 2018;37:2514-25. [Crossref] [PubMed]
Xing Z, Wan L, Fu H, Yang G, Yang Y, Yu L, Lei B, Zhu L. Diff-UNet: A diffusion embedded network for robust 3D medical image segmentation. Med Image Anal 2025;105:103654. [Crossref] [PubMed]
Karimi D, Salcudean SE. Reducing the Hausdorff Distance in Medical Image Segmentation With Convolutional Neural Networks. IEEE Trans Med Imaging 2020;39:499-513. [Crossref] [PubMed]

Cite this article as: Pang M, Yang C, Luo Y. AFFUNet: adaptive feature fusion Transformer U-Net with joint loss function for medical image segmentation. Quant Imaging Med Surg 2026;16(9):697. doi: 10.21037/qims-2026-1-0234