Weak redundancy U-shaped network and heatmap-based object prompt method for real-time medical image processing

Introduction

In the contemporary era characterized by the rapid advancement of artificial intelligence (AI), the integration of novel technologies and algorithms can improve medical quality and patient experience, reduce the workload of doctors, and address the current shortage of radiology professionals. The application of computer-aided lesion detection, segmentation, and organ recognition can significantly enhance the intelligence level of traditional medical equipment and provide better assistance to doctors in their work. Hence, the development of smart medical care has emerged as a major trend in the field of medical equipment (1,2). In numerous medical scenarios, rapid, efficient, and low-resource-consumption object detection and automatic segmentation functionalities are requisite, such as real-time detection during ultrasound scanning, rapid identification, and three-dimensional (3D) reconstruction of organs in computed tomography (CT) images. Additionally, along with the advancement of embedded technology, a multitude of portable medical apparatuses have emerged, which are designed to be used in any location, at any time, and are not restricted to the hospital setting. For these portable medical devices, AI is also requisite for their functionality. Nevertheless, given the constrained local computing power of portable devices, lightweight networks are particularly indispensable. Hence, the research on lightweight medical image detection and segmentation networks holds significant practical importance.

In the field of object detection, existing algorithms can be categorized into two primary types: two-stage algorithms and single-stage algorithms. The two-stage algorithm, exemplified by the region-based convolutional neural network (R-CNN) (3), mainly consists of extracting regions of interests (ROIs), extracting features, bounding box regression, and classification. For example, at the very beginning, there were the spatial pyramid pooling network (SPP-Net) (4) and Fast R-CNN (5). Then, Faster R-CNN (6) emerged, which uses a region proposal network (RPN) to generate ROIs, enabling end-to-end training and detection. Region-based fully convolutional network (R-FCN) (7) uses position-sensitive score maps to address a dilemma between translation-invariance in image classification and translation-variance. The single-stage object detection algorithm eliminates the need for ROI extraction, transforming the task into a direct regression problem. For example, initially, You Only Look Once (YOLO) (8) emerged as a quintessential single-stage detection algorithm that achieves real-time and efficient target detection by simultaneously localizing and classifying objects through a single network architecture. The most recent iteration in this series is YOLOv10 (9). Unlike YOLO, single shot detector (SSD) (10) directly uses convolution layers for detection on different feature maps. Furthermore, with Focal Loss, RetinaNet (11) effectively addresses the challenge of highly imbalanced positive and negative samples, attaining accuracy levels comparable to those of two-stage networks.

Algorithms in the field of image segmentation can be categorized into two types: semantic segmentation and instance segmentation. Semantic segmentation associates each pixel in an image with a corresponding class label. FCN (12) serves as a foundational network for semantic segmentation. SegNet (13) uses max pooling with indices to reduce the number of parameters. U-Net (14) is introduced as a solution for biomedical image segmentation; the key innovation of which is the incorporation of skip connections within the decoder. VNet (15) incorporates residual connections along with convolutional layers, making it more suitable for 3D image segmentation. UNet3+ (16) uses an extensive framework of skip connections and deep supervision. TransUNet (17) introduces the transformer to enhance the use of global context information. VM-UNet (18) integrates visual state space (VSS) blocks with the U-Net architecture. In addition, the DeepLab series algorithms are deep learning frameworks for semantic segmentation proposed by Google, aiming to improve the accuracy and simplify the network design and training process. The initial DeepLabv1 (19) algorithm introduces atrous convolution and conditional random fields to address the multi-scale challenges and pixel-level localization. Currently, DeepLabv3+ (20) represents the latest iteration of the DeepLab series. PSPNet (21) uses the pyramid pooling module to effectively capture contextual information across various scales. Furthermore, some bilateral networks divide the feature extraction process into two distinct branches: the detail branch and the semantic branch. Notable examples include Bilateral Segmentation Network version 1 (BiSeNetv1) (22), BiSeNetv2 (23), BiSeNetv3 (24), and Fast Semantic Segmentation Network (Fast-SCNN) (25).

The instance segmentation algorithm integrates object detection and semantic segmentation techniques to assign distinct labels to individual instances of the same class. Currently, these algorithms can be categorized into two types: two-stage methods and single-stage methods. The flow of the two-stage method involves identifying each instance through object detection, followed by performing semantic segmentation within the detected bounding boxes. For example, Mask R-CNN (26) is a typical two-stage algorithm. Conversely, single-stage methods achieve both object detection and instance segmentation using a unified network architecture. An example is You Only Look At CoefficienTs (YOLACT) (27).

Designing a lightweight network necessitates the reduction of parameters and computational demands while preserving performance. Specific strategies include the implementation of lightweight modules, elimination of redundant layers and parameters, and optimization of network architecture. For example, SqueezeNet (28) uses 1×1 convolutional kernels in place of traditional 3×3 kernels, significantly reducing the number of parameters. MobileNet (29) introduces depthwise separable convolution that decomposes standard convolutions into two stages: depthwise and pointwise convolution. Inception (30) proposes a parallel structure using filters of varying sizes to concurrently process input data, capturing richer contextual information without escalating computational complexity. In terms of medical images, for example, Muksimova et al. (31) integrated an Enhanced Spatial Attention (ESA) layer on YOLOv5m.

Although there are many algorithms mentioned above, the unique characteristics of medical images render it impractical to directly apply networks designed for natural images. Therefore, this study aimed to develop a specialized lightweight detection and segmentation network tailored for medical imaging. We analyze the shortcomings of existing algorithms for medical images and propose an innovative network architecture.

The current algorithms are not fully adapted to medical images, and there are numerous redundant calculations, including the following aspects.

• Most medical images are in grayscale. The texture of various organs and tissues is difficult to distinguish, and there is an anatomical relationship among them. Therefore, a large receptive field is necessary to obtain sufficient contextual information; otherwise, misidentification is prone to occur. Hence, compared with natural images, they are more dependent on advanced semantic information.
• Current object detection algorithms primarily rely on anchors of varying scales. However, not all medical images contain targets across all scales, rendering the detection of certain scales redundant.
• Medical images typically feature a single scene with relatively simple structures, whereas networks for natural images are designed to address complex problems, often characterized by large depth and width. Directly applying these networks to medical image tasks may introduce considerable complexity and redundancy.
• Medical image datasets are generally limited in size, which renders them particularly susceptible to overfitting when employing networks with excessive parameters or undue complexity.

Considering the aforementioned differences between medical image processing and natural image processing, existing methods for real-time medical image segmentation have the shortcomings of complex architectures, a large number of parameters, and feature redundancy. Therefore, this paper aimed to develop a lightweight network structure with fast speed and high segmentation accuracy, specifically tailored for medical images. In this study, we address the issue of redundant computing by designing a U-shaped segmentation network that incorporates redundancy reduction. It boasts a compact parameter size of 130.67 KB and achieves a segmentation speed of 488 frames per second (FPS). Within this architecture, we minimize feature redundancy through channel limitation, thereby compelling the network to learn more efficient features and fully utilize multi-level features. Additionally, we introduce auxiliary information flows to enhance information utilization.

Secondly, in this article, we use “object prompt” to represent visualizing the position of the target object in an image to guide the viewer’s attention. In response to the requirement for real-time object prompts in medical applications, we propose an innovative method based on heatmaps, different from standard object detection tasks, which use bounding boxes to prompt target objects. This approach aims to further reduce task complexity and consequently minimize the model size. We have developed a method for generating heatmaps and designed the network architecture for heatmap prediction. The resulting model has a compact size of 24.75 KB and can achieve a processing speed of up to 9,862.13 FPS on graphics processing unit (GPU) and 107.93 FPS on central processing unit (CPU).

The rest of this article is arranged as follows. Section II reports the proposed method. Section III introduces the experiment results and our discussion. Finally, in Section IV, we conclude this article.

Methods

Datasets

The data used in this study were sourced from multiple publicly available datasets (detailed datasets were listed in Appendix 1) and encompass the following tasks: segmentation of the lungs in chest X-rays (32,33), placental vessel segmentation task in fetoscopic images (34), segmentation of gliomas, tumors, and edema in brain magnetic resonance imaging (MRI) images, left atrium segmentation task in heart MRI images, liver segmentation task in CT images, and spleen segmentation in CT images (35). In addition, we also used a dataset of thyroid nodule ultrasound images provided by Ruijin Hospital. This dataset comprises a total of 1,522 thyroid ultrasound images, each of which has a thyroid nodule mask label, and all of them are marked by professional physicians. The acquisition and use of the thyroid ultrasound image dataset in this study strictly adhere to ethical standards. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the institutional Ethics Committee of Ruijin Hospital, Shanghai Jiao Tong University (No. LWEC2024011), and informed consent was provided by all the patients.

We conducted preprocessing on the images in each dataset. First, irrelevant information and manual labels were removed, extracting only the valid image portions from the Digital Imaging and Communications in Medicine (DICOM) files that contained all the necessary data. Second, for several CT datasets, we screened out slice images containing target objects to address potential data imbalance issues caused by a large number of images without targets. Third, we standardized the sizes of the images and masks. In this study, all images and masks were resized to 224×224 pixels. Finally, the processed dataset was randomly split into training, validation, and test sets in an 8:1:1 ratio.

Weak redundancy U-Net (WRU-Net) for segmentation

Overall architecture

For medical image segmentation tasks, we developed a lightweight network: WRU-Net. The concept of lightweight network design primarily focuses on minimizing parameters and computational workload, optimizing network structure, and striking a balance between computational efficiency and performance. Furthermore, the underlying principle is to minimize redundancy and optimize the utilization of existing features. The architecture of our WRU-Net is illustrated in Figure 1. Taking 32 channels and an input size of 256×256 as an illustrative example, the specific architecture of the network, along with the output dimensions for each component, is systematically outlined in Table 1.

Figure 1 WRU-Net detailed network structure. “Conv” incorporates a convolutional layer followed by batch normalization and a ReLU activation function; “DWConv” represents a depthwise separable convolutional layer followed by batch normalization and a ReLU activation function; “Avg Pool” indicates an average pooling layer; “DeConv” represents deconvolution (transposed convolution) with batch normalization and ReLU activation; “Up” refers to bilinear interpolation for 2× up-sampling; “ConvSig” stands for a convolutional layer followed by batch normalization and a sigmoid activation function; “Cat” represents concatenate. ReLU, rectified linear unit; WRU-Net, weak redundancy U-Net.

This network structure is adapted from the traditional encoder-decoder structure, and based on this, we developed the following three design proposals.

• The number of convolutional layer channels at various depths of the network consistently remains at 32.
• In the decoder stage, the features are up-sampled through deconvolution-bilinear interpolation-deconvolution. Additionally, direct addition is adopted to fuse the features from the encoder and decoder stages.
• By incorporating auxiliary information flows, the efficiency of vertical information transmission can be enhanced without significantly increasing complexity or the number of parameters.

In this section, we will elaborate on each of the aforementioned three points.

Channels

We maintained the number of channels at 32 rather than doubling the number of channels as the feature graph size decreases for several reasons. Generally, in network design, as the depth of the convolutional layer increases and the scale of the feature graph decreases, the number of channels is typically designed to increase. This is because the size of the feature map gradually reduces after successive down-sampling, which subsequently requires more channels to preserve the information. However, this leads to significant information redundancy, resulting in an excessive number of parameters. Additionally, given the typically limited size of medical image datasets, this can easily lead to overfitting. Furthermore, considering that the U-shaped network uses long-distance skip connections to reuse information, it supplements the encoder’s information in the decoding process. Therefore, certain information can be derived from the encoding features, and the decoder does not need to retain excessive information. We aimed to enable the network to learn the most effective features at each scale, ensuring that the features optimized for a specific scale are indeed acquired. During the decoding process, information from the encoder can be used comprehensively, allowing us to extrapolate a considerable amount of redundant information.

Up-sampling and feature fusion

Up-sampling is commonly achieved through bilinear interpolation or deconvolution. Deconvolution operations can further process convolutional information, where each pixel in the depth feature map integrates information from a broader surrounding area, allowing the deconvolution process to interpret this comprehensive data. However, setting the stride of the deconvolution layer to 2 and using a kernel size of 3 can lead to checkerboard artifacts, which will complicate the task. To mitigate this issue, we set the deconvolution stride to 1 and maintained a kernel size of 3. Subsequently, we increased the feature map size using bilinear interpolation and then applied deconvolution for further understanding. This approach minimizes the impact of artifacts and allows for more effective processing of our features.

In the feature fusion phase, we opted for the direct addition of the two feature maps instead of using concatenation. Given that concatenation necessitates additional convolution operations for effective fusion, our experimental results demonstrated that direct addition yields comparable performance. Therefore, to reduce network complexity and computational load, we adopted this approach.

Auxiliary information flows

In the U-Net architecture, skip connections play a crucial role by facilitating information exchange between corresponding encoder and decoder layers. Prior studies have demonstrated that feature exchange across different scales can enhance network performance [as seen in UNet++ (36) and UNet3+ (16)]. The key characteristic of UNet++ is its dense skip connections among feature maps at each stage, which aim to strengthen cross-scale interaction. UNet3+, on the other hand, uses full-scale skip connections and deep supervision mechanisms. Full-scale skip connections integrate high-level semantics with low-level semantics from feature maps of different scales directly. However, excessive cross-scale information exchange significantly increases network complexity, posing substantial computational challenges. Furthermore, directly resizing lower-layer feature maps for concatenation with upper layers can result in redundant information, leading to unnecessary reprocessing of already understood features. The architecture proposed in this paper circumvents the use of excessively dense feature map connections by leveraging phased results from the auxiliary information flow to enhance information exchange across scales, rather than directly using the feature maps of each layer. As illustrated in Figure 1, we use convolutional operations to compress information, followed by a sigmoid activation function to extract the compressed information of each scale. This compressed information is then up-sampled and concatenated with the next layer, repeating this process across layers. This approach enables more effective information exchange across different scales, thereby establishing an auxiliary information flow that supports the final classification. Additionally, Figure 1 includes supplementary infographics for each scale, providing insight into the network’s internal processes.

Heatmap-based object prompt method

Heatmap-based object prompt

Above, we introduced a segmentation algorithm. However, in many practical scenarios, object detection can adequately meet requirements while conserving computational resources. For instance, real-time detection and visualization of nodules in ultrasound scans, and identification of target organs or lesion regions in CT images. Similar applications can provide physicians with an indication of the approximate location of targets without necessitating precise segmentation. To further optimize computational efficiency, we propose an object prompt method based on a heatmap. This inspiration comes from attention maps and class activation maps. Although heatmaps cannot perfectly delineate the edges and shapes of objects, they can effectively indicate the approximate location and size of objects. The difference is that typical attention maps or class activation maps are internally generated by the network, derived from soft supervision signals, and primarily function as intermediate attention modules, or tools for network attention visualization, rather than direct network outputs. In contrast, our proposed heatmap prediction method involves calculating and generating heatmaps via masks, which are then used as supervisory signals for direct network learning.

The generation method of the heatmap is shown in Figure 2. By translating the mask based on its gravity center, the heat value at the center after translation is determined as the ratio of the intersection area between the translated mask and the original mask to the total area of the mask. The formula is as follows:

H(Center′)=Area(Intersection)Area(Mask)[1]

Figure 2 Diagram of the heatmap generation method.

Heatmaps reduce the difficulty of object prompts, blur the boundaries of objects, and only require a coarse prediction of the object’s position and shape. In network design, the demand for detailed spatial information is diminished, allowing direct prediction on low-resolution feature maps. The heatmap at the desired resolution can be obtained through straightforward linear up-sampling.

Heatmap prediction network

Based on the aforementioned concepts, we developed a heatmap prediction network, as illustrated in Figure 3. The encoder architecture uses the same module as that of WRU-Net’s encoder, maintaining a low channel count. Given that heatmap prediction is less complex compared to precise segmentation, this network uses only 16 channels. For the input image, average pooling is first used for down-sampling, reducing the size and thus decreasing the computational complexity. In the decoder, the deconvolution-bilinear interpolation-deconvolution module mentioned above is used for up-sampling. Heatmap prediction is conducted on the 32×32 feature map. In this scale, a concatenated feature fusion approach is used to provide more high-level semantic information. Finally, the prediction results at the 32×32 scale are progressively up-sampled to generate the final heatmap, which serves as the network’s output.

Figure 3 The detailed structure of the heatmap prediction network. “Conv” incorporates a convolutional layer followed by batch normalization and a ReLU activation function; “DWConv” represents a depthwise separable convolutional layer followed by batch normalization and a ReLU activation function; “Avg Pool” indicates an average pooling layer; “DeConv” represents deconvolution (transposed convolution) with batch normalization and ReLU activation; “Up” refers to bilinear interpolation for 2× up-sampling; “Cat” represents concatenate. c, channels; k, kernel size; ReLU, rectified linear unit; s, stride.

The loss function of the network is as follows:

HMLoss=Cosine(hmpre,hmtrue)+SmoothL1(hmpre,hmtrue)[2]

Compared to traditional bounding box object detection methods, using heatmaps for object prompts can also achieve the purpose, and the visualization effect is more obvious. In this way, a model with fewer parameters can be achieved, thereby reducing computational resource consumption and achieving a lightweight, rapid object prompt.

Evaluation and experiment

Segmentation results evaluation method

In this study, we used Dice similarity coefficient (DSC), intersection over union (IoU), precision, and recall to evaluate the segmentation outcomes, and their formulations are presented as follows.

DSC(Mask,gtMaskpre)=2TPFP+2TP+FN[3]

IoU(Maskgt,Maskpre)=TPFP+TP+FN[4]

Precision(Maskgt,Maskpre)=TPTP+FP[5]

Recall(Maskgt,Maskpre)=TPTP+FN[6]

Where TP, TN, FP, and FN represent the number of pixels of true positive, true negative, false positive, and false negative, respectively. Maskgt represents the ground truth mask. Maskpre represents the predicted mask.

Heatmap evaluation method

To evaluate the prediction accuracy of heatmaps, we set a threshold to generate masks for both the true and predicted heatmaps, isolating regions exceeding this threshold. Subsequently, we calculated the DSC and IoU between these two masks as evaluation metrics, as shown in Eqs. [7] to [10]. Furthermore, we use the residual standard deviation (RSD) as a metric to evaluate the prediction result of pixel values in the heatmap, as shown in Eq. [11]. In this study, we examined the evaluation outcomes across various thresholds, with detailed results presented in the subsequent section.

HmMaskgt(t)=Where(hmgt>t)[7]

HmMaskpre(t)=Where(hmpre>t)[8]

DSChm(t)=DSC(HmMaskgt(t),HmMaskpre(t))[9]

IoUhm(t)=Iou(HmMaskgt(t),HmMaskpre(t))[10]

RSD=∑i=1n(yi−y^i)2n−2[11]

Where t represents the set threshold. HmMaskgt represents the region where the true heatmap exceeds the threshold. HmMaskpre represents the region where the predicted heatmap exceeds the threshold. yi denotes the true value of each pixel in the heatmap, whereas y^i represents the corresponding predicted value, and n indicates the total number of pixels.

Experiment

We carried out experiments on CPU (Intel i7-7700, Intel, Santa Clara, CA, USA) and GPU (NVIDIA GeForce RTX 2080, NVIDIA, Santa Clara, CA, USA), and conducted training and testing on diverse data sets, using parameters, floating point operations per second (FLOPs), and FPS to assess the model size and its computational efficiency. We trained the network using the training dataset, selected the optimal model based on the validation set performance, and ultimately evaluated the model’s performance on the test set.

Results

Results of segmentation

Evaluation results

We conducted experiments on the datasets detailed in section “Datasets”. Figure 4 illustrates representative segmentation outcomes from our WRU-Net, with two examples provided for each dataset. Each example includes the original image, the ground truth mask, and the predicted mask. As evident from the figure, it achieves satisfactory segmentation performance.

Figure 4 Examples of WRU-Net outcomes across diverse datasets. FN, false negative; FP, false positive; TN, true negative; TP, true positive; WRU-Net, weak redundancy U-Net.

Quantitatively, we compared our proposed method with several established networks, including VNet (15), U-Net (14), nnU-Net (37), TransUNet (17), BRAU-Net++ (38), Swin-Unet (39), and DeepLabV3+ (20). We calculated the evaluation indicators for each network and dataset (see Table 2 for the results). Furthermore, we conducted a comparative analysis of the number of model parameters, FLOPs, and FPS. The FPS data presented in the table represent the average values obtained from testing conducted on each dataset. It can be observed from the experimental results across various datasets that, although the proposed WRU-Net demonstrates superiority in terms of speed compared to other networks, there are noticeable gaps in segmentation accuracy on certain datasets. In the domain of image segmentation, a significant and nearly inherent trade-off exists between speed and performance. This trade-off constitutes one of the key challenges in both network design and practical application deployment. Consequently, a prudent decision must be made by considering the real-time requirements of the target application, the constraints of hardware resources, and the acceptable level of accuracy degradation.

It can be inferred from Table 2 that the network’s performance varies across different datasets. This variation arises due to the distinct characteristics of each dataset. For instance, the Chest dataset features a balanced and large object scale. The Thyroid Nodule dataset contains substantial high-frequency noise, as it consists of ultrasound images with blurred object edges and significant scale differences between objects. The brain tumor dataset includes numerous small and complex edges, along with small objects. The placental vessel dataset demands the network to focus more on distinguishing foreground and background differences, while also addressing fuzzy and indistinguishable targets. No single network achieves optimal performance across all datasets. Network design must balance the unique characteristics of various datasets to achieve satisfactory results.

According to the experimental results, it is evident that WRU-Net exhibits significantly lower performance compared to other networks on the brain tumor dataset. This can primarily be attributed to the relatively high complexity of the brain tumor segmentation task, coupled with the limited number of parameters, which leads to underfitting on this dataset. In the ablation study, we observed that using 64 channels could substantially enhance the performance in this task.

The data in the table indicate that WRU-Net’s segmentation performance on each dataset is comparable to, or even surpasses, that of existing state-of-the-art networks. Additionally, WRU-Net has a significantly lower parameter count and achieves superior segmentation speed relative to other networks. This performance is especially well-suited for application scenarios with constrained computing resources or stringent real-time requirements, such as embedded systems, mobile devices (e.g., smartphones), and real-time ultrasonic scanning scenarios. Specifically, in scenarios such as ultrasonic scanning, the calculation ability to support ultrasonic devices is often limited, whereas the real-time nature of imaging necessitates the capability to efficiently process high-speed data streams.

Ablation study

The WRU-Net proposed in this paper represents a universal structure. The fixed number of channels can be determined based on specific datasets and tasks. For the task of medical image segmentation, we conducted ablation experiments and tested performance in three cases with the number of channels of 16, 32, and 64, respectively, on all datasets. Similarly, we conducted experiments in scenarios without auxiliary information flow to further validate the effectiveness of incorporating it.

The experimental results are presented in Table 3. If the number of channels was reduced to 16, the performance across all datasets was inferior to that of the 32-channel network. For simpler segmentation tasks, such as the chest, heart, and spleen datasets, the decline in DSC and IoU was relatively minor. However, for other datasets, the evaluation metrics exhibited more pronounced differences. If the number of channels was increased to 64, the model demonstrated slightly improved performance on thyroid nodule, placental vessel, and brain tumors datasets compared to the 32-channel model, yet its performance on other datasets remained consistent with the 32-channel configuration. Based on these experimental findings, we selected 32 as the fixed number of channels after comprehensively evaluating both model performance and parameter count. Further, the performance on the thyroid nodule, placental vessel, and heart datasets has decreased significantly without auxiliary information flow.

Results of heatmap

We trained and evaluated the heatmap prediction network proposed in this paper across multiple datasets. Figure 5 illustrates the comparison between some of the predicted results and the true values, demonstrating a satisfactory predictive performance. Additionally, we conducted quantitative evaluations under various thresholds, as detailed in Table 4, to select appropriate thresholds for different datasets. It can be observed that, for different datasets, individual thresholds should be selected to maximize the DSC and IoU. This variation is primarily attributed to the differences in object scales across datasets. For instance, a threshold of 0.05 is optimal for the thyroid nodule, liver, and spleen datasets; a threshold of 0.01 is recommended for the brain tumor dataset; and a threshold of 0.1 is most suitable for the Heart dataset. Our model contains only 24.75 KB parameters, achieving 9,862.13 FPS on GPU and 107.93 FPS on CPU.

Figure 5 Examples of heatmap results.

Discussion

Firstly, the WRU-Net was introduced to address the challenge of real-time segmentation in medical imaging. It uses a U-shaped encoder-decoder architecture and maintains a minimal number of channels in the feature maps at each stage to minimize redundant computation. To enhance information utilization, we have also designed auxiliary information flows within the decoder stage to propagate the intermediate results across different scales. According to the above experimental results, our network significantly enhances the segmentation speed of medical images while maintaining exceptional segmentation quality even with a very low number of parameters. This corroborates the limitations and redundant computations identified in existing networks for medical image segmentation tasks as analyzed in this study. Furthermore, it demonstrates the feasibility and effectiveness of the proposed segmentation framework for medical images. In designing the network architecture, we intentionally refrained from employing various attention mechanisms, deformable convolutions, or other advanced networks to enhance network performance; instead, we used simple foundational units to validate the efficacy of our network framework.

The heatmap object prompt method proposed in this paper is designed for real-time detection and visual cueing tasks. Given the unique requirements of medical application scenarios, it is more appropriate to conduct semantic detection rather than the bounding box prediction methods commonly used in natural image object detection. Heatmaps effectively simplify the segmentation task by softening object boundaries while preserving their positions and general shapes. This approach allows us to achieve the desired outcomes using a streamlined architecture. The experimental results presented in the preceding section demonstrate that our heatmap prediction network yields satisfactory results. Notably, it features a very small parameter count and exhibits high computational efficiency.

Although the method proposed in this paper yields promising results, it also exhibits certain limitations. Specifically, due to its small number of parameters, WRU-Net may inherently exhibit reduced fitting capabilities for datasets, particularly when the number of object classes is large. In such cases, it becomes necessary to increase the number of channels accordingly. This adjustment, however, leads to an increase in the number of parameters and a slowdown in speed, thereby diminishing its competitive advantages over other networks. Consequently, WRU-Net is better suited for scenarios with a smaller number of segmentation classes, which more effectively highlights its strengths in segmentation performance and speed. Further, the method of using heatmaps for visualizing objects, while effectively reducing the network’s complexity, inevitably introduces certain limitations. Firstly, heatmaps are incapable of providing precise object boundaries; they can only indicate the position and scale of the object. Secondly, heatmaps tend to lose detailed shape information. Additionally, the heatmap approach does not support instance-level differentiation. The direct extraction of object counts and center point locations is not feasible, necessitating secondary processing to obtain such information. In application scenarios where accurate object numbers and center locations are critical, this approach may be less suitable.

Considering the small number of parameters, it is necessary to pay attention to its generalization. An excessive number of parameters may result in overfitting, whereas an insufficient number of parameters can lead to underfitting. Both scenarios are detrimental to the performance on unseen data. The network presented in this paper, due to its limited parameter count, is not well-suited for large and complex datasets. Instead, it is better suited for relatively singular tasks, such as medical image segmentation. Consequently, it is essential to comprehensively evaluate and select the appropriate network by considering factors such as task requirements and dataset quality to ensure optimal generalization.

Finally, the network proposed in this paper is characterized by a small number of parameters and high speed, which are its most prominent features. Consequently, it is especially well-suited for applications on mobile devices, real-time ultrasound systems, and embedded systems. For instance, medical image-assisted diagnostic software deployed on mobile devices, such as smartphones, can perform local secure computations without depending on network connectivity or cloud computing resources.

Conclusions

In this paper, regarding the issue of medical image segmentation, we analyze the various sources of redundant computation present in existing networks and propose a weak redundancy network called WRU-Net. The network is characterized by a U-shaped encoder-decoder architecture, a limited number of channels, and auxiliary information flows. Experimental results across diverse medical datasets demonstrated that our WRU-Net significantly enhanced segmentation speed while maintaining effective performance even with a few parameters.

Furthermore, we introduced a heatmap-based object prompt method. This approach is primarily designed for real-time visualization of object prompts. We have detailed the methodology for heatmap generation, the predictive modeling framework, and the evaluation method. The experimental outcomes demonstrated that our network exhibits high efficacy, with a remarkably low parameter count and rapid processing speed.

In summary, this study represents an exploratory effort in the domain of medical image processing, focusing on lightweight solutions. The lightweight architecture introduced in this paper can also be extended to the 3D segmentation task of medical images. Indeed, compared with two-dimensional (2D) image segmentation, medical images such as CT images are more suitable for 3D segmentation and visualization. However, 3D image segmentation faces challenges such as an extremely high computational load and substantial memory requirements, imposing stringent demands on equipment. Consequently, the architecture explored in this paper holds considerable potential in the field of 3D image segmentation.

Acknowledgments

The authors would like to thank the doctors of Ruijin Hospital affiliated with Shanghai Jiao Tong University for their help and all the authors of the open-source codes and database used in this study.

Footnote

Data Sharing Statement: Available at https://qims.amegroups.com/article/view/10.21037/qims-2025-567/dss

Funding: This work was greatly supported by Shanghai Huangpu District Health Commission (Grant Nos. 2023XD02 and 2023GG01), the National Natural Science Foundation of China (Grant No. 61973210), and the Medical-engineering Cross Projects of SJTU (Grant No. YG2019ZDA17 and No. ZH2018QNB23).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-2025-567/coif). Z.F. reports that he has received research grants from Shanghai Huangpu District Health Commission (Grant No. 2023XD02 and No. 2023GG01), the National Natural Science Foundation of China (Grant No. 61973210), and the Medical-engineering Cross Projects of SJTU (Grant No. YG2019ZDA17 and No. ZH2018QNB23). The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the institutional Ethics Committee of Ruijin Hospital, Shanghai Jiao Tong University (No. LWEC2024011) and informed consent was taken from all the patients.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Preum SM, Munir S, Ma M, Yasar MS, Stone DJ, Williams R, Alemzadeh H, Stankovic JA. A Review of Cognitive Assistants for Healthcare: Trends, Prospects, and Future Directions. ACM Comput Surv 2021;53:1-37.
2. Ma B, Yang J, Wong FKY, Wong AKC, Ma T, Meng J, Zhao Y, Wang Y, Lu Q. Artificial intelligence in elderly healthcare: A scoping review. Ageing Res Rev 2023;83:101808. [Crossref] [PubMed]
3. Girshick R, Donahue J, Darrell T, Malik J. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proc IEEE Comput Soc Conf Comput Vision Pattern Recognit 2014. Available online: https://openaccess.thecvf.com/content_cvpr_2014/html/Girshick_Rich_Feature_Hierarchies_2014_CVPR_paper.html
4. He K, Zhang X, Ren S, Sun J. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. IEEE Trans Pattern Anal Mach Intell 2015;37:1904-16. [Crossref] [PubMed]
5. Girshick R. Fast R-CNN. Computer science. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). 2015:1440-8. Available online: https://ieeexplore.ieee.org/document/7410526
6. Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans Pattern Anal Mach Intell 2017;39:1137-49. [Crossref] [PubMed]
7. Dai J, Li Y, He K, Sun J. R-FCN: Object detection via region-based fully convolutional networks. Advances in neural information processing systems 2016. Available online: https://proceedings.neurips.cc/paper/2016/hash/577ef1154f3240ad5b9b413aa7346a1e-Abstract.html
8. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Las Vegas, NV, USA. IEEE; 2016:779-88.
9. Wang A, Chen H, Liu L, Chen K, Lin Z, Han J, Ding G. YOLOv10: Real-Time End-to-End Object Detection. arXiv:2405.14458 [Preprint]. 2024. Available online: https://doi.org/10.48550/arXiv.2405.14458
10. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC. SSD: Single Shot MultiBox Detector. In: Leibe B, Matas J, Sebe N, Welling M, editions. Computer Vision–ECCV 2016. Lecture Notes in Computer Science. Cham: Springer; 2016:21-37.
11. Lin TY, Goyal P, Girshick R, He K, Dollar P. Focal Loss for Dense Object Detection. IEEE Trans Pattern Anal Mach Intell 2020;42:318-27. [Crossref] [PubMed]
12. Shelhamer E, Long J, Darrell T. Fully Convolutional Networks for Semantic Segmentation. IEEE Trans Pattern Anal Mach Intell 2017;39:640-51. [Crossref] [PubMed]
13. Badrinarayanan V, Kendall A, Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Trans Pattern Anal Mach Intell 2017;39:2481-95. [Crossref] [PubMed]
14. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells W, Frangi A, editors. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015. Cham: Springer; 2015:234-41.
15. Milletari F, Navab N, Ahmadi SA. V-Net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth International Conference on 3D Vision (3DV); Stanford, CA, USA. 2016:565-71.
16. Huang H, Lin L, Tong R, Hu H, Zhang Q, Iwamoto Y, Han X, Chen YW, Wu J. UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation. CASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); Barcelona, Spain. IEEE; 2020:1055-9.
17. Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, Lu L, Yuille AL, Zhou Y. TransUNet: Transformers make strong encoders for medical image segmentation. arXiv:2102.04306 [Preprint]. 2021. Available online: https://doi.org/10.48550/arXiv.2102.04306
18. Ruan J, Xiang S. VM-UNet: Vision Mamba UNet for Medical Image Segmentation. arXiv:2402.02491 [Preprint]. 2024. Available online: https://doi.org/10.48550/arXiv.2402.02491
19. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL. Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv:1412.7062 [Preprint]. 2015. Available online: https://doi.org/10.48550/arXiv.1412.7062
20. Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, editors. Computer Vision–ECCV 2018. Cham: Springer; 2018:833-51.
21. Zhao H, Shi J, Qi X, Wang X, Jia J. Pyramid scene parsing network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Honolulu, HI, USA. IEEE; 2017:6230-39.
22. Yu C, Wang J, Peng C, Gao C, Yu G, Sang N. BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, editors. Computer Vision–ECCV 2018. Cham: Springer; 2018:334-49.
23. Yu C, Gao C, Wang J, Yu G, Shen C, Sang N. BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation. Int J Comput Vis 2021;129:3051-68.
24. Tsai TH, Tseng YW. BiSeNet V3: Bilateral segmentation network with coordinate attention for real-time semantic segmentation. Neurocomputing 2023;532:33-42.
25. Poudel RPK, Liwicki S, Cipolla R. Fast-SCNN: Fast semantic segmentation network. arXiv:1902.04502 [Preprint]. 2019. Available online: https://doi.org/10.48550/arXiv.1902.04502
26. He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. IEEE Trans Pattern Anal Mach Intell 2020;42:386-97. [Crossref] [PubMed]
27. Bolya D, Zhou C, Xiao F, Lee YJ. YOLACT: Real-time instance segmentation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV); Seoul, Korea (South). IEEE; 2019:9156-65
28. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K. Squeezenet: alexnet-level accuracy with 50x fewer parameters and <0.5mb model size. arXiv:1602.07360 [Preprint]. 2018. Available online: https://doi.org/10.48550/arXiv.1602.07360
29. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861 [Preprint]. 2017. Available online: https://doi.org/10.48550/arXiv.1704.04861
30. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Boston, MA, USA. IEEE; 2015:1-9.
31. Muksimova S, Umirzakova S, Mardieva S, Iskhakova N, Sultanov M, Cho YI. A lightweight attention-driven YOLOv5m model for improved brain tumor detection. Comput Biol Med 2025;188:109893. [Crossref] [PubMed]
32. Candemir S, Jaeger S, Palaniappan K, Musco JP, Singh RK, Xue Zhiyun, Karargyris A, Antani S, Thoma G, McDonald CJ. Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Trans Med Imaging 2014;33:577-90. [Crossref] [PubMed]
33. Jaeger S, Karargyris A, Candemir S, Folio L, Siegelman J, Callaghan F, Xue Zhiyun, Palaniappan K, Singh RK, Antani S, Thoma G, Wang Yi-Xiang, Lu Pu-Xuan, McDonald CJ. Automatic tuberculosis screening using chest radiographs. IEEE Trans Med Imaging 2014;33:233-45. [Crossref] [PubMed]
34. Bano S, Vasconcelos F, Shepherd LM, Vander Poorten E, Vercauteren T, Ourselin S, David AL, Deprest J, Stoyanov D. Deep Placental Vessel Segmentation for Fetoscopic Mosaicking. In: Martel AL, editor. Medical Image Computing and Computer Assisted Intervention–MICCAI 2020. Cham: Springer; 2020:763-73.
35. Ye J, Cheng J, Chen J, Deng Z, Li T, Wang H, Su Y, Huang Z, Chen J, Jiang L, Sun H, Zhu M, Zhang S, He J, Qiao Y. SA-Med2D-20M Dataset: Segment Anything in 2D Medical Imaging with 20 Million masks. arXiv:2311.11969 [Preprint]. 2023. Available online: https://doi.org/10.48550/arXiv.2311.11969
36. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support 2018;11045:3-11. [Crossref] [PubMed]
37. Isensee F, Petersen J, Klein A, Zimmerer D, Maier-Hein KH. Abstract: nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation. In: Handels H, Deserno T, Maier A, Maier-Hein K, Palm C, Tolxdorff T, editors. Bildverarbeitung für die Medizin. Wiesbaden: Springer Vieweg; 2019:22.
38. Lan L, Cai P, Jiang L, Liu X, Li Y, Zhang Y. BRAU-Net++: U-Shaped Hybrid CNN-Transformer Network for Medical Image Segmentation. arXiv:2401.00722 [Preprint]. 2024. Available online: https://doi.org/10.48550/arXiv.2401.00722
39. Cao H, Wang Y, Chen J, Jiang D, Zhang X, Tian Q, Wang M. Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation. In: Karlinsky L, Michaeli T, Nishino K, editors. Computer Vision–ECCV 2022 Workshops. Cham: Springer; 2023:205-18.