Analyzing explainability of YOLO-based breast cancer detection using heat map visualizations

Introduction

Breast cancer is the most common worldwide cause of death among women. In 2022, breast cancer is the second most diagnostic cancer (11.6%) followed by lung cancer (12.4%) for both genders. However, breast cancer is the topmost founded incidence (23.8%) and mortality (15.4%) in females. From 1980–2000, the incidence rate in high-income countries was increasing whereas the mortality rate decreased since the early 1990s. This is the result of a screening mammogram. World Health Organization (WHO) recommends that women at average risk for breast cancer aged 50–69 years to do screening mammograms every 2 years (1). In Thailand, breast cancer is rapidly increasing, with over 19,452 new cases expected in 2025. The National Cancer Control Program (NCCP) of Thailand also recommends screening mammograms every 2 years for women aged over 45 years old and recommends breast self-examination (BSE) for women aged over 30 years old (2). Early detection via mammographic screening is a well-established method for improving breast cancer survival rates. However, emerging evidence suggests that modifiable lifestyle factors, including diet, physical activity, and body weight, significantly impact patient prognosis. Research has shown that adherence to cancer prevention strategies—including maintaining a healthy weight, exercising, and following a balanced diet—reduces recurrence risk by 37% and mortality by 58% in high-risk patients. This underscores the importance of integrating advanced imaging with risk factor analysis for improved diagnostic precision and patient care (3).

Mammography is a diagnostic technology that is frequently used in clinical practice to check for breast cancer. To enhance the survival rate, early detection, and accurate diagnosis are crucial. Errors in breast cancer detection via mammography often stem from several factors. Dense breast tissue can obscure abnormalities, a phenomenon known as the “masking effect”. The inherent characteristics of tumors can complicate detection, with benign masses sometimes appearing suspicious and malignant masses occasionally resembling benign lesions. Additionally, human factors such as radiologist perception and decision-making can contribute to misinterpretations. Compared to Western women, Asian women tend to have a higher breast density, which makes it even harder in Thailand to analyze and classify types of breast cancer. Hence, in order to give doctors a second view to help and support their decisions, computer-aided detection (CADe) and diagnosis (CADx), or computer-aided system (CAD), is crucial (4).

Leveraging technological advancements in the medical field, many diseases can be detected using CADs, including CADe and CADx. CADe will be used to detect the highly dense masses in mammography, while CADx will be used to classify those lesions. Both together is a computer-based system helping the radiologist to interpret medical images. Despite this, CAD systems have been used for many decades, and there still be improvement all the way long (5). Various object detection models such as Faster R-CNN, RetinaNet, and You Only Look Once (YOLO) have been used to improve the detection performance of CAD systems. Among them, YOLO was regarded as an outstanding approach due to its efficiency in object detection (6). Unlike YOLO, other object detections, such as deformation parts models and R-CNN, rely on region proposal methods, generating potential bounding boxes on the image and running classifiers on those proposals. As a disadvantage, this process often requires post-processing steps to refine boxes and eliminate duplicates, resulting in slower performance and difficulty in optimization. Whereas YOLO treats detection as a regression problem, predicting bounding boxes and class probabilities directly from image pixels. By training on full images and optimizing performance holistically, YOLO operates faster and more efficiently than conventional methods, reducing background errors by considering the entire image during both the training and testing phases (7). For this reason, different versions of YOLO have been widely applied in the medical AI field for various tasks, including breast cancer diagnosis. However, to the best of our knowledge, the most recent versions of YOLO, such as YOLO version 9 (YOLOv9), YOLO version 10 (YOLOv10), and YOLO version 11 (YOLO11) have yet to be widely explored for breast cancer image detection. These versions introduced significant advancements and achieved better performance in other object detection tasks. Therefore, it is worth applying these models to breast cancer diagnosis.

YOLOv9, in particular, incorporates two notable innovations: the programmable gradient information (PGI) and the generalized efficient layer aggregation network (GELAN). PGI addresses data loss issues commonly associated with deep neural networks by generating stable gradients through an auxiliary reversible branch. Simultaneously, GELAN improves the efficiency of lightweight models by implementing gradient path planning, which optimizes the flow of gradients during the training process. These improvements allow YOLOv9 to achieve exceptional performance, combining the advantages of being lightweight, fast, and highly accurate (8). Following the successes of YOLOv9, YOLOv10 introduces additional improvements aimed at refining the model’s post-processing and architectural design. Traditionally, YOLO models have relied on non-maximum suppression (NMS) to eliminate redundant outputs during the post-processing phase, which can increase computational overhead and impact accuracy. YOLOv10 addresses this limitation by introducing an NMS-free approach, utilizing dual label assignments and a consistent matching metric to reduce redundancy more effectively. Furthermore, regarding model architecture, YOLOv10 adopts a holistic efficiency-accuracy-driven design strategy, which optimizes the balance between computational efficiency and predictive accuracy, further advancing the model’s overall performance (9). The latest version of the YOLO model, YOLO11, was released in September 2024 by Ultralytics, a platform dedicated to simplifying AI training for researchers and developers. YOLO11 has been designed for exceptional adaptability across diverse environments, offering state-of-the-art accuracy, speed, and efficiency. This update serves as a versatile solution for a wide range of computer vision tasks, making it a valuable tool for various applications (10).

Nevertheless, almost all of the AI-based diagnosis methods, including both classification and detection models, have significant limitations regarding user trustworthiness. A primary challenge in AI, particularly in fields such as medicine that demand high accuracy, is the lack of transparency and explainability. As illustrated in Figure 1, when AI systems produce incorrect predictions, they do not explain the error. This issue arises from the nature of deep learning networks, which autonomously calculate and adjust internal parameters (weights) during the learning process. This self-adjusting mechanism operates in a manner that is not readily interpretable, often referred to as the “black box” problem, reflecting the limited understanding of the AI decision-making process (11).

Figure 1 The black box nature of AI models. AI, artificial intelligence.

Explainable artificial intelligence (XAI) addresses these challenges by providing comprehensible insights into how AI systems operate. By elucidating the underlying processes, XAI helps users evaluate the reliability of AI outputs, identify potential biases, and ensure accountability. This transparency not only fosters trust and collaboration between AI systems and humans across various domains but also highlights weaknesses in AI models, facilitating their continuous improvements. The main contributions to this study are as follows:

• We examined three state-of-the-art YOLO versions—YOLOv9, YOLOv10, and YOLO11—and trained them on a breast cancer dataset. Given the dependency of model performance on dataset characteristics, our objective was to identify the most suitable model for breast cancer image classification and detection applications.
• We investigated seven XAI methodologies, including Grad-CAM, Grad-CAM++, Eigen-CAM, EigenGrad-CAM, XGrad-CAM, LayerCAM, and HiResCAM, applying them to the best-performing YOLO model in this research. These methodologies were utilized to identify model weaknesses for future improvement and to determine the most effective XAI approach for enhancing YOLO model performance.

Related works

In recent years, the application of YOLO-based CADe systems has gained significant attention in the field of medical imaging to enhance the accuracy and efficiency of breast cancer diagnosis. YOLO is a real-time object detection that can see the entire image and create a bounding box to identify regions of interest (ROIs) and classification (6). YOLO’s integration with CAD systems is to improve patient outcomes and diagnostic workflows for radiologists by offering fast and accurate detection of lesions. Numerous researches have shown the effectiveness of these systems. For instance, Al-Masni et al. (12) proposed a YOLO-based CAD system for diagnosis using mammograms. Their approach classified mass abnormality into benign and malignant and contained four main stages: mammogram preprocessing, feature extraction utilizing multi-convolutional deep layers, mass detection with a confidence model, and finally mass classification using a fully connected neural network (FC-NN). The evolution of YOLO has spurred subsequent iterations, and numerous studies have utilized the updated versions of YOLO to further enhance breast cancer detection and classification methodologies. Bal et al. utilized YOLOv3 as a region proposal network (RPN) (13). After YOLOv3 detected the ROI, the convolutional neural network (CNN) was used to classify benign and malignant lesions, aiding in the diagnosis of Ductal Carcinoma in breast tissue. YOLOv4 integration was presented in (14) by Hamed et al. to locate lesions and classify the pathology type. They also compared the performance of feature extractions such as ResNet, Visual Geometry Group (VGG), Inception, etc. against YOLO. Subsequent research has continued to explore and refine YOLO-based approaches. For instance, Su et al. proposed a double shot model for mass detection and segmentation simultaneously using a combination of YOLO version 5 (YOLOv5) and local-global (LOGO) architectures (15). Baccouche et al. utilized a YOLO-based model to identify the lesion into mass, calcification, or Architectural distortion (16). Additionally, they used image-to-image translation techniques (CycleGAN and Pix2Pix) to analyze current and prior mammograms, helping to predict earlier missed abnormalities. Frank SJ employed YOLOv5 to identify masses within mammogram images and integrated it with a deep CNN (17). EfficientNet, which was trained by divided subregion tiles of mammograms, has been used to highlight the specific region of masses on the image to potential breast abnormalities. EfficientNet, which was trained by divided subregion tiles of mammograms, has been used to highlight the specific region of masses on the image to potential breast abnormalities.

Whilst there have been notable achievements in YOLO-based breast cancer detection and classification, the reliability of these systems continues to be a point of contention among users. In response to the ‘black box’ challenge, XAI has been introduced to serve the purpose of elucidating the decision-making processes of these models through visualization. Eigen-CAM, a prominent XAI technique, was employed in a study by Prinzi et al. (18), where the performance of several YOLO-based architectures—namely YOLOv3, YOLOv5 (nano, small, medium, and large variants), and YOLOv5-transformer—was trained using the CBIS-DDSM dataset for pretraining in breast cancer classification. Subsequently, transfer learning of the previous step was applied to these YOLO variants, which were trained by the Inbreast dataset and a proprietary dataset. Their study revealed that the YOLOv5 small version is the standout performer among all versions, prompting further experiments to find the best training protocol for the YOLOv5. The result shows the Adam optimizer, coupled with extensive data augmentation and a 16-batch size, yielded the most promising results with YOLOv5s (YOLO version 5 small size). Consequently, YOLOv5s was applied to a testing set from the proprietary dataset and visualized with two XAI methodologies, namely Eigen-CAM and occlusion sensitivity to provide insightful explanations for model behavior. Notably, Eigen-CAM visualization was the preferred choice for both clarity and heightened accuracy in model interpretation. Eigen-CAM was employed to enhance interpretability and explain model outputs by highlighting all suspicious ROIs within the mammogram. Even when the YOLOv5 model produced incorrect predictions, Eigen-CAM continued to identify regions that could potentially contain abnormalities, thereby ensuring interpretability and valuable insights even in failure cases. This capacity is instrumental in reducing false negatives (FNs), decreasing the risk of missing cancerous regions. However, this benefit is offset by a higher incidence of false positives (FPs), resulting in some normal regions being mistakenly labeled as suspicious. Another XAI technique employed for visualizing breast cancer is Grad-CAM. For instance, Talaat et al. (19) introduce BCaXAI, an AI-based model designed to diagnose breast cancer using mammograms, thereby minimizing the need for invasive biopsies and reducing radiologist bias. The model, built on the Inception-ResNet V2 architecture with Grad-CAM for enhanced explainability, demonstrated superior accuracy compared to models like ResNet50 and VGG16. The integration of Grad-CAM heatmaps allows radiologists to correlate the model’s predictions with their clinical expertise, enhancing interpretability and trust. Visual analysis confirms that the model accurately identifies critical diagnostic features, such as masses and calcifications, while minimizing false associations, thereby ensuring high diagnostic precision. This alignment between predicted features and radiological expertise confirms that the model accurately captures medically pertinent information, avoiding irrelevant correlations and ensuring high diagnostic precision.

Moreover, in the study of Munshi et al. (20), SHAP (one of the XAI techniques) was utilized as the foundational component to establish a comprehensive framework for breast cancer analysis. They presented a novel methodology combining image and numerical data features with XAI for breast cancer identification. Their approach featured a U-NET transfer learning model for image-based prediction and an ensemble model that integrated characteristics from a customized CNN with random forest (RF) and support vector machine (SVM) classifiers. We present this article in accordance with the CLEAR reporting checklist (available at https://qims.amegroups.com/article/view/10.21037/qims-2024-2911/rc).

Methods

In this section, we delineate the methodology employed in this research and critically evaluate each tool relevant to our work. As illustrated in Figure 2, the initial phase of this study involves the application of YOLOv9, YOLOv10, and YOLO11 to the dataset, aiming to detect benign and malignant lesions. Subsequently, we will discuss contrast limited adaptive histogram equalization (CLAHE) and elucidate its functionality to improve YOLO efficiency. Finally, we will examine seven different XAIs methodologies, including Grad-CAM, Grad-CAM++, Eigen-CAM, EigenGrad-CAM, XGrad-CAM, LayerCAM, and HiResCAM detailing their operational principles. Additionally, this section will outline the criteria employed to select the superior model and to interpret the results generated by XAIs.

Figure 2 Pipeline of the proposed methodology. CAM, class activation map; CLAHE, contrast limited adaptive histogram equalization; MIAS, Mammographic Image Analysis Society; YOLO, You Only Look Once.

Dataset

The dataset utilized in this study is composed of two distinct sources: the INbreast: Toward a Full-field Digital Mammographic Database (21) and the Mammographic Image Analysis Society (MIAS) databases (22). This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. INbreast 2012 was collected at the Centro Hospitalar de S. João (CHSJ), Breast Centre, Porto, Portugal, with approval from the Portuguese National Committee of Data Protection and Hospital’s Ethics Committee. The dataset was created to support the development of algorithms for the detection and diagnosis of mammographic lesions, as the availability of a robust mammography database is critical for this purpose. INbreast contains full-field digital mammograms (FFDMs) with highly precise annotations provided by expert radiologists, making it a valuable resource for breast cancer research and algorithm development. The database includes 115 cases, with 90 cases from women with both breasts affected (four images per case) and 25 cases from mastectomy patients (two images per case).

The MIAS database, on the other hand, was collected through the UK National Breast Screening Programme using a Joyce-Loebl scanning microdensitometer. The mammograms, originally digitized at 50 microns, were resampled to 200 microns with a resolution of 1,024×1,024 pixels. It includes 161 cases, with paired images for the left (even-numbered) and right (odd-numbered) breasts of each patient. The MIAS dataset is accessible through the Pilot European Image Processing Archive (PEIPA) at the University of Essex.

In this research, the dataset focuses on the mass lesions for 1,720 images. The image size is 640×640, which is the standard size for YOLOv9. Auto-orients were applied in the data preprocessing process without augmentation. The dataset was divided into two main groups: training and testing. The training set is around 90% of the original dataset and an international validation set is also accounted in it. The testing set is around 10% of the original data set.

Ultralytic

Ultralytics is an advanced AI training platform designed to support researchers and developers in deploying AI solutions. The platform emphasizes the provision of open-source tools for a wide range of applications, simplifying the AI training process. By integrating technical excellence with a steadfast commitment to its core values, Ultralytics aims to democratize AI, making it accessible to individuals irrespective of their background or resources. In this study, Ultralytics was utilized to facilitate the training of YOLOv9, YOLOv10, and YOLO11. While YOLO11 was designed and developed by Ultralytics, the earlier models were integrated into the platform to enhance their usability (23).

In this research, we employed the Ultralytics platform to train three YOLO models—YOLOv9, YOLOv10, and YOLO11—for two-class classification of breast cancer images, distinguishing between benign and malignant cases. If no lesion is present in the input images, YOLO treats the image as background and does not generate any bounding boxes. This approach enabled a comparative analysis of model performance to identify the most effective method for accurate cancer diagnosis.

Preprocessing

CLAHE

CLAHE is a well-known image preprocessing technique employed to enhance the contrast of an image. CLAHE is an extension of adaptive histogram equalization (AHE) designed to mitigate the risk of excessive contrast enhancement in regions of the image where pixel values are nearly uniform (near-constant regions), which can cause noise in regions that have relatively uniform pixel values. By limiting the degree of contrast amplification, CLAHE addresses this issue effectively. AHE itself is a refinement of histogram equalization (HE), which enhances contrast across the entire image uniformly. However, HE can inadvertently introduce noise throughout the image due to its global approach. In contrast, AHE computes multiple histograms, each corresponding to a different section of the image, and adjusts the lightness values, accordingly, making it more suitable for improving local contrast and enhancing edge definitions in specific regions. In this study, we utilized CLAHE to enhance the contrast of low-quality images, thereby improving the accuracy of object detection and visualization.

Detection models

YOLOv9

YOLOv9 worked on the loss of information during the feedforward process, known as the information bottleneck problem. The deeper neuron network can easily lose the information, where the data moves through the layers of feature extraction and spatial transformation of a deep learning network. The result is less able to retain complete information, making it possible for the model to train on unreliable gradients and poor convergence. To address this problem, advanced architecture such as PreAct ResNet has employed reversible functions. The reversible functions preserve the original input data at each layer. This method is effective in very deep networks with a thousand network layers or more, while shallower and more lightweight networks are less applicable. In response, YOLOv9 introduced a novel auxiliary supervision framework known as PGI. This framework consists of three key components: the main branch, the auxiliary reversible branch, and the multi-level auxiliary information. During the inference process, there is one main branch, ensuring no additional inference cost. The other two components are used to solve the loss of information problem.

The auxiliary reversible branch adds reliable gradients and updates network parameters by mapping data to the target. This reversible architecture will be applied only to train the process and not used in prediction to reduce the inference cost. The last component is multi-level auxiliary information, which is used to fix the problem caused by the auxiliary reversible branch.

Due to the versatility of YOLO, it can do a lot of tasks through different heads. The multi-level auxiliary information is used to aggregate the information from the previous layers that contain all target objects to the main branch. This approach helps prevent the loss of crucial information across all target sizes, as YOLO is designed to handle objects of varying scales.

Another improvement of YOLOv9 is the new network architecture, Generalized ELAN or GELAN. GELAN is a combination of two neuron network architectures, CPSNet and ELAN. GELAN was specifically developed to optimize the balance between lightweight design, inference speed, and accuracy, further enhancing YOLOv9’s performance across multiple tasks (8).

YOLOv10

In recent years, YOLO models have gained significant popularity due to their balanced trade-off between performance and efficiency. However, YOLO models still face limitations regarding the accuracy-latency trade-offs during both the forward pass and the post-processing stage. Typically, YOLO employs a one-to-many label assignment strategy during training, which can lead to redundancy in predictions. In this approach, each ground truth object (the actual object in the image) is assigned to multiple predicted bounding boxes or “positive samples”. This strategy allows the model to learn from diverse feedback and richer supervisory signals but generates redundant object detections, necessitating the use of NMS to select the optimal prediction during inference. NMS as an additional post-processing step prevents the model from end-to-end deployment, slows down inference speed, and makes the model’s performance sensitive to NMS hyperparameters. This can result in suboptimal detection outcomes and limit the model’s robustness in various scenarios. YOLOv10 aims to address these limitations by pushing the boundaries of the accuracy-speed trade-off through improvements in both post-processing and model architecture throughout the detection pipeline.

In post-processing, they introduce an NMS-free training strategy for YOLOs with dual-label assignments and consistent matching metrics. The dual-label assignment merges the traditional one-to-many strategy used in YOLO with the one-to-one approach typically employed by CNN-based detectors. In the one-to-one assignment, each ground truth object is matched with only one prediction, eliminating the need for NMS during inference. As a result, the model outputs a single, non-redundant prediction for each object, leading to faster inference times. By combining these two strategies, YOLOv10 retains the rich supervisory signal of the one-to-many approach while benefiting from the non-redundant predictions of the one-to-one method. To ensure seamless integration of these two assignment strategies, YOLOv10 applies consistent matching metrics across both branches, enabling the model to learn uniformly and produce more reliable and efficient predictions.

In model architecture, they introduced a Holistic Efficiency-Accuracy Driven Model Design. This model design improved both efficiency and accuracy perspectives. For efficiency-driven components include changes to the lightweight classification head, spatial-channel decoupled downsampling, and rank-guided block design, all of which reduce computational overhead and enhance model speed. For accuracy-driven components enhancements such as large-kernel convolutions and PSA (Position-Sensitive Attention) ensure that the model maintains or improves its accuracy, particularly for detecting objects in complex environments (9).

YOLO11

In September 2024, Ultralytics introduced the latest iteration of its YOLO model, YOLO11. This version represents a significant advancement, featuring enhanced feature extraction that enables the model to operate with fewer parameters and increased processing efficiency. The improved architecture of YOLO11 incorporates optimized training methodologies and an augmented data pipeline, which facilitates adaptability to a diverse range of tasks, from small-scale projects to large-scale applications. Remarkably, YOLO11m demonstrates a higher mean average precision (mAP) on the Common Objects in Context (COCO) dataset while using fewer parameters than its predecessor, YOLOv8m.

YOLO11 is designed for exceptional efficiency in terms of computational resources, making it versatile for deployment across various environments, including NVIDIA GPUs, edge devices, and cloud platforms. This adaptability establishes YOLO11 as a transformative tool for developers and researchers in the field of computer vision, supporting a range of applications from real-time object detection to image classification (10).

YOLOv9, YOLOv10, and YOLO11 are available in multiple model sizes, which directly impact prediction performance. In this study, we selected YOLOv9c (25.5 million parameters), YOLOv10l (24.4 million parameters), and YOLO11l (25.3 million parameters). These model sizes were chosen to be approximately equal to facilitate a fair comparison of performance across different versions while also considering computational cost constraints.

XAIs

Grad-CAM

One of the most effective ways to explain AI models’ predictions is through visual explanations, by identifying prediction errors and offering opportunities to enhance performance while fostering confidence in these deep learning models. One such method, class activation maps (CAMs), visualizes the decision-making areas by modifying the model architecture. CAM achieves this by replacing fully connected layers with a global average pooling (GAP) layer, which computes a weighted average of the features in the last convolutional layer. Grad-CAM or gradient weight-CAM is an extension of the CAM technique, incorporating two significant advancements. First, it introduces gradient-based weights, which enable the generation of class-specific importance maps. Second, it applies a rectified linear unit (ReLU) activation function after combining the weights with the feature maps, ensuring the focus remains on positively contributing regions. These modifications allow Grad-CAM to produce heatmaps without requiring modifications to the model architecture or retraining to obtain class-specific weights (24).

Unlike CAMs, methods like Grad-CAM and Grad-CAM++ do not require altering the model architecture. Grad-CAM was developed to serve two characteristics of what makes a good visualization: class-discriminative (localize the category in the image) and high-resolution (capture fine-grained detail). It generates a heatmap of the important regions of the image by using the gradient information flowing into the last convolutional layer to assign the importance feature, which is used to make a decision in a particular class. The neuron importance weights (wkc) were calculated using the gradient of the score for class c (Yc) with respect to feature map activations (Ak). These gradients flow to the global average pooled over width (1Z) and height (j) dimensions. Finally, normalization was applied through 1Z, where Z is the total number of pixels in the feature map, as shown in the equation below:

wkc=1z∑i∑j∂yc∂yijk[1]

The class-discriminative localization map Grad-CAM, LGrad-CAMc is constructed as a linear combination of gradient weights that quantify the importance of feature maps. To focus on the positive contributions to the class of interest, a ReLU activation function is applied, ensuring that only positively influential regions are highlighted. Without the application of ReLU, the resulting map may include irrelevant areas, leading to suboptimal outcomes for the target class.

LGrad-CAMc=ReLU(∑kωkcAk)[2]

Grad-CAM++

Grad-CAM++ was inspired by two algorithms, CAM and Grad-CAM, with an improvement in overcoming the limitation of Grad-CAM which is multiple occurrences of the same object class in the same image and partial object coverage, Grad-CAM mostly highlights only the most discriminative regions instead of the entire object. Grad-CAM++ attempts to overcome Grad-CAM’s shortcomings by refining the way gradient weights are calculated. The normalization Z used at the end of the calculation is the cause of spatial footprint bias (smaller active pixels get lower importance). Grad-CAM++ suggested taking a weighted average of the pixel-wise gradients. The modification equation for calculating gradient weight in Grad-CAM++ is shown below:

ωkc=∑i∑jαijkc⋅ReLU(∂Yc∂Aijk)[3]

There are two main modifications in Grad-CAM++. The first one is the application of ReLU to gain only positive gradients, which supports the model’s prediction. The second one is weighting (αijkc) which balances the contributions of all feature maps, addressing the bias toward larger spatial footprints seen in Grad-CAM. This weight can be considered as a spatial variability with its ability to balance the contributions of all feature maps, ensuring smaller or less prominent regions are not ignored (25).

Eigen-CAM

All class discriminative methods, including Grad-CAM and Grad-CAM++, rely on gradient values. Whenever the model prediction is incorrect, it can lead to wrong or distorted visualization. Furthermore, these methods tend to add extra computational complexity. To address those problems, Eigen-CAM offers a simpler and more intuitive alternative for generating CAMs. It does not depend on class relevance scores or weights, making it independent of the specific class being predicted. Eigen-CAM utilizes principal component analysis (PCA) through singular value decomposition to find its principal components. The CAM is generated by selecting the first eigenvector (which corresponds to the largest variation) and projecting the output feature maps onto this eigenvector. This highlights the most relevant areas of the image contributing to CNN’s decision. Importantly, Eigen-CAM achieves robust object localization without requiring modifications to the CNN architecture or gradient backpropagation, and it outperforms previous methods, such as Grad-CAM and Grad-CAM++, showing up to a 12% improvement in weakly supervised object localization (26).

EigenGrad-CAM

EigenGrad-CAM enhances the existing Eigen-CAM technique by incorporating an element of class discrimination, allowing for more precise and interpretable visual explanations in neural networks. By leveraging the first principal component, or eigenvector, EigenGrad-CAM effectively reintroduces and advances upon the Grad-CAM method. This improvement not only refines the visualization but also strengthens the interpretability of the generated heatmaps, making the model’s decision-making process more transparent and aligned (27).

XGrad-CAM

XGrad-CAM can be considered an advanced version of Grad-CAM. Recognizing the lack of theoretical foundations in both CAM and Grad-CAM, the authors introduced XGrad-CAM to address these limitations by incorporating two essential axiomatic properties and providing enhanced visualization. Axiomatic properties refer to fundamental characteristics deemed necessary for a robust visualization method, including continuity, implementation invariance, sensitivity, and conservation.

Continuity ensures the robustness of the method to minor variations in input, evaluating the impact of small perturbations of input on the generated explanation. Implementation invariance guarantees that the explanations are not influenced by architectural or implementation differences in the models, focusing on the relationship between model equivalence and explanation consistency. Sensitivity ensures that the explanation correctly reflects the contribution of each feature to the model’s decision. Conservation ensures that the explanation fully accounts for the model’s decision without over or underestimating the contributions.

XGrad-CAM enhances Grad-CAM by incorporating sensitivity and conservation into its design. Specifically, XGrad-CAM modifies the gradient-weight calculation by combining gradient values with feature map activations, whereas Grad-CAM relies solely on the gradient feature. This modification results in more accurate and interpretable visualizations (28).

LayerCAM

CAMs have been widely utilized for weakly supervised tasks in recent years. However, CAM often locates coarse regions of the target objects, a limitation attributed to a small spatial resolution of the final convolutional layer. To address this issue, LayerCAM was introduced as a method to generate more fine-grained object localization information from CAMs, thereby improving the accuracy of object localization. This improvement arises from a re-examination of the relationship between feature maps and their corresponding gradients.

LayerCAM enables the generation of reliable CAMs from different layers of a CNN. By combining CAMs from multiple layers, LayerCAM can identify more precise object locations, thereby enhancing the performance of weakly supervised tasks. Furthermore, LayerCAM can be seamlessly applied to any CNN-based image classifier without requiring architectural modifications and the backpropagation way (29).

HiResCAM

HiResCAM extends the Grad-CAM methodology, addressing its limitations in accurately identifying the regions relevant to a model’s decisions. Grad-CAM sometimes highlights locations that are not directly related to the model decision the model’s decision-making process and tends to expand attention maps beyond the regions where the model truly focuses. This issue arises from the calculation of feature importance weights, which utilize averaging gradients across spatial dimensions. Such averaging can lead to the loss of essential explanatory details by blurring gradient information and constraining the explanation to align with the relative magnitudes and positive-negative patterns of the feature maps. This limitation reduces its ability to accurately reflect the locations the model uses for predictions.

HiResCAM overcomes these challenges by avoiding gradient averaging. Instead, it directly multiplies the feature maps by their respective gradients, enabling element-wise rescaling and preserving sign changes. This approach produces high-resolution attention maps that more accurately capture the computations and reasoning of the model. Unlike Grad-CAM, HiResCAM does not rely on global averaging; its attention maps are computed through element-wise multiplication of gradients and feature maps. This method ensures a more precise representation of the locations involved in the model’s predictions.

Furthermore, HiResCAM is versatile and capable of being applied to CNNs of any dimensionality, including 2 dimension (2D), 3 dimension (3D), or higher. By design, HiResCAM guarantees that attention maps highlight the exact regions the CNN utilizes for its predictive tasks, offering a more accurate and detailed explanation of the model’s behavior (30).

Performance measurements

In the domain of object detection, the accuracy of a model like YOLO is multi-faceted and can be assessed through several key metrics. These metrics collectively provide a holistic view of the model’s performance in terms of detection, classification, and localization of objects. The primary components of accuracy in YOLO can be measured using the following:

Confusion matrix

A confusion matrix is a table used to evaluate the performance of a classification algorithm, offering a visual and quantitative summary by comparing predicted and actual class labels. In this paper, we use a normalized confusion matrix, which presents the proportional distribution of classification results. The matrix consists of two classes: benign [0] and malignant [1]. It includes four key values true positive (TP), representing correct predictions of class 0 (benign); true negative (TN), correct predictions of class 1 (malignant); FP, incorrect predictions of class 1 (malignant) when the actual label is class 0 (benign); and FN, incorrect predictions of class 0 (benign) when the actual label is class 1 (malignant).

In the context of healthcare, FNs are often more concerning than FPs. Misclassifying a malignant tumor as benign can have severe consequences for the patient’s health or life. Therefore, relying on accuracy alone is insufficient to understand the model’s tendencies toward FPs or FNs. Metrics like Precision and Recall provide better insights into this aspect. Therefore, relying on accuracy alone is insufficient to understand the model’s tendencies toward FPs or FNs. Metrics like Precision and Recall provide better insights into this aspect.

Precision and recall

Precision quantifies the proportion of correct positive detections among all positive detections made by the model. A high precision indicates a low FP rate, meaning the model rarely misidentifies background or other objects as the target objects. It is defined as:

Precision=TPTP+FP[4]

Recall measures the proportion of actual positive cases that are correctly identified by the model. A high recall indicates that the model successfully detects most of the relevant objects in the dataset. In medical applications, achieving a recall closer to 1 is crucial, as it reduces the chances of incorrectly predicting malignant cases. It is expressed as:

Recall=TPTP+FN[5]

In classification models, a trade-off typically exists between precision and recall; as one increases, the other often decreases. This precision-recall (P-R) balance can reveal potential biases within the model. For instance, when precision (i.e., correct identification of positive cases) is high but recall (the model’s ability to identify actual positives) is low, it suggests that the model effectively captures positive instances but may overlook instances of the negative class, indicating a bias and imbalance. In medical diagnostics, high recall is generally prioritized over high precision, as the consequences of FNs are more critical than those of FPs. For example, if the positive class is benign and the negative class is malignant, misclassifying malignant cases as benign could delay treatment, increasing the risk of cancer metastasis and, in severe cases, posing a life-threatening risk to the patient.

Intersection over Union (IoU)

IoU, or Intersection over Union, is a performance metric used to assess how effectively XAI can illustrate decision-making areas. IoU measures the degree of overlap between the predicted bounding box and the ground truth bounding box. IoU is frequently used to evaluate object detection performance, reflecting the accuracy of detection. IoU was calculated by the following equation:

IOU=AreaofOverlapAreaofUnion[6]

mAP

mAP is a crucial metric widely used to measure the performance of models that focus on object detection tasks and information retrieval on images. It serves as a standard evaluation criterion in various benchmark challenges, such as COCO, ImageNET challenge, Google Open Image Challenge, etc. The mAP extends the concept of AP. The mAP metric builds upon the concept of AP, which is defined as the area under the P-R curve.

AP=∫01p(r)dr[7]

where p is the precision and r is the recall.

The P-R curve is a graphical representation of the trade-off between precision and recall for different classification thresholds. It helps visualize how well the model distinguishes between TPs and FPs across varying recall levels.

The mAP is then computed as the mean of the average precision values across all object classes, defined as follows:

mAP=1n∑k=1nAPk[8]

where n represents the total number of classes, and AP_k is the average precision for class k.

For object detection tasks, mAP calculations rely on the IoU threshold, determining whether a predicted bounding box is classified as a TP or FP. This classification directly influences precision and recall values, and consequently, the AP and mAP scores. The metric mAP@0.50 measures precision at an IoU threshold of 0.50, focusing on the model’s ability to detect objects correctly. In contrast, mAP@0.50:0.95 or averages precision across a range of IoU thresholds from 0.50 to 0.95, providing a comprehensive assessment of detection performance. Higher mAP scores indicate that the model achieves a favorable balance between precision and recall, reflecting its effectiveness in detecting and localizing objects accurately across varying classes and IoU thresholds (21). Averages precision across a range of IoU thresholds from 0.50 to 0.95, providing a comprehensive assessment of detection performance. Higher mAP scores indicate that the model achieves a favorable balance between precision and recall, reflecting its effectiveness in detecting and localizing objects accurately across varying classes and IoU thresholds.

The visualizations of the decision-making processes of the underlying model facilitate a better understanding of its limitations, particularly in identifying biases within datasets and enhancing model robustness against adversarial attacks. The deployment of XAI methods in this study establishes user confidence in the reliability of classification and object detection models, especially for practitioners such as doctors and clinicians. To measure the explainability of the XAI models, we calculate the following indexes:

Matching ground truth (mGT)

The metric mGT serves as a measure of how accurately the saliency map or heat map mask (p) aligns with the ground truth mask (n) in highlighting the significant regions in an image. The metric mGT provides a ratio that reflects the extent to which ground truth pixels overlap with the most salient regions of the saliency map, as defined by the equation below:

mGT=np[9]

Where p is the count of the number of ones in the ground truth mask, which is also the pixels marked as important in the ground truth mask binary ones in the ground truth mask, and n is the number of pixels the saliency map’s brightest regions that match the important regions in the GT mask. A higher mGT score indicates that the heat map accurately highlights the truly important areas, thereby serving as an indicator of the performance of XAI methods (31).

Pearson correlation coefficient (PCC)

The PCC assesses the linear correlation between two variables, evaluating whether variations in pixel intensities within the highlighted regions of both the ground truth and the heat map are aligned, thereby indicating a shared emphasis on specific areas. PCC values range from −1 to 1, where a value of 1 signifies a perfect positive correlation, meaning the two variables vary in the same direction. Conversely, a value of -1 denotes a perfect negative correlation, indicating that the two variables vary in opposite directions. A PCC value of 0 indicates no correlation, signifying no relationship between the two variables. The PCC is calculated using the equation below:

PCC(u1,u2)=u1Tu2||u1||||u2||[10]

In this context, u₁ represents the flattened ground truth mask, while u₂ denotes the flattened heat map mask. A high PCC value implies that the model’s interpretation of salient regions within the image closely corresponds to the reference ground truth, an essential factor in validating XAI methods. This alignment is particularly critical in fields such as medical imaging, where ensuring that the model’s focus matches the ground truth can foster confidence in its predictions.

Root mean square error (RMSE)

RMSE measures the average magnitude of the error between two sets of values. It is the square root of the average of squared differences.

RMSE=1HW∑i=1H∑j=1W(yij−y′ij)2[11]

In this context, H and W are height and width of an image while y_ij is the ground truth mask and y′ij is the heat map mask. Lower RMSE means the small differences between ground truth and heat map, suggesting that the model’s interpretation aligns closely with the ground truth, thereby implying higher accuracy. Conversely, a higher RMSE reflects greater discrepancies, indicating that the model’s focus is less consistent with the ground truth, suggesting that the model’s interpretation aligns closely with the ground truth, thereby implying higher accuracy. Conversely, a higher RMSE reflects greater discrepancies, indicating that the model’s focus is less consistent with the ground truth.

Results

The results presented in this section focus on two critical aspects of the research: the outcomes of the YOLO models and the effectiveness of XAI approaches. To evaluate the detection performance using three versions of YOLO, we employed precision, recall, and mAP as performance metrics, which were derived from the confusion matrix. These metrics served as criteria for evaluating and selecting the superior model based on its overall performance.

The performance of three YOLO models was evaluated using the normalized confusion matrix and mAP, as presented in Figure 3 and Table 1. The normalized confusion matrix highlights the progressive improvements across the YOLO models. Although we performed a two-class classification, the YOLO model generates a 3×3 confusion matrix. This occurs because YOLO inherently separates the background from the objects of interest to generate bounding boxes, effectively treating the background as an additional class during the detection process. YOLOv9 achieved an accuracy of 80% for class 0 and 75% for class 1. YOLOv10 improved accuracy, reaching 93% for class 0 and 80% for class 1, with a notable reduction in the confusion rate, particularly for class 0. Amongst all three versions, YOLO11 achieved the highest classification performance, with accuracies of 95% for class 0 and 85% for class 1.

Figure 3 Normalized confusion metrics for three versions of YOLO models. YOLO, You Only Look Once.

The mAP results further corroborate this trend, with YOLO11 achieving the highest value of 0.935, compared to 0.926 for YOLOv10 and 0.868 for YOLOv9. These progressive improvements across the YOLO versions highlight the effectiveness of the model enhancements. The superior mAP values and reduced misclassification rates of YOLO11 make it a strong candidate for reliable breast cancer image classification. Consequently, YOLO11 was selected for further experiments.

From Figures 4-6, illustrating the training and validation accuracy and loss curves, it can be observed that YOLOv9 exhibits relatively high training and validation losses, accompanied by significant fluctuations, suggesting a lack of model stability. YOLOv10, on the other hand, shows marked improvements with smoother training curves and enhanced generalization, minimizing fluctuations in validation losses. Among the three models, YOLO11 demonstrates the best performance, characterized by the lowest losses and the highest mAP values, indicating superior optimization and efficiency. These results indicate a clear progression in model performance, driven by architectural and training improvements across versions.

Figure 4 YOLOv9 training and validation results. The x-axis in all plots represents the number of training epochs, while the y-axis denotes the corresponding loss or evaluation metric. The first row shows results for the training dataset: box_loss (Bounding Box Regression Loss), cls_loss (Classification Loss), and dfl_loss (Distribution Focal Loss). The last two plots in the first-row display precision(B) and recall(B) on the validation dataset, where (B) stands for bounding box. The second row presents validation metrics: box_loss, cls_loss, and dfl_loss for the validation dataset, followed by mAP50(B) (mean average precision at IoU threshold 0.50) and mAP50-95(B) (mean average precision averaged over IoU thresholds from 0.50 to 0.95). IoU, Intersection over Union; YOLO, You Only Look Once.

Figure 5 YOLOv10 training and validation results. The x-axis in all plots represents the number of training epochs, while the y-axis denotes the corresponding loss or evaluation metric. The first row shows training losses: box_om (Objectness Matching Box Loss), cls_om (Objectness Matching Class Loss), dfl_om (Objectness Matching Distribution Focal Loss), box_oo (Objectness Overlap Box Loss), and cls_oo (Objectness Matching Class Loss) The second row presents: recall(B), mAP50(B) (mean average precision at IoU threshold 0.50), and mAP50-95(B) (mean average precision averaged over IoU thresholds from 0.50 to 0.95), dfl_oo (Objectness Overlap Distribution Focal Loss), precision(B), where (B) stands for bounding box. These plots collectively demonstrate the model’s training progress, with decreasing losses and improving detection performance over epochs. YOLO, You Only Look Once.

Figure 6 YOLO11 training and validation results. The x-axis in all plots represents the number of training epochs, while the y-axis denotes the corresponding loss or evaluation metric. The first row shows results for the training dataset: box_loss (Bounding Box Regression Loss), cls_loss (classification loss), and dfl_loss (Distribution Focal Loss). The last two plots in the first-row display precision(B) and recall(B) on the validation dataset, where (B) stands for bounding box. The second row presents validation metrics: box_loss, cls_loss, and dfl_loss for the validation dataset, followed by mAP50(B) (mean average precision at IoU threshold 0.50) and mAP50-95(B) (mean average precision averaged over IoU thresholds from 0.50 to 0.95). IoU, Intersection over Union; YOLO, You Only Look Once.

All XAI methodologies were applied to visualize the black box decision-making process of the YOLO model. We expected that XAIs would highlight the important regions that influenced the model’s prediction. However, as most XAI techniques are specifically designed for CNN-based architectures, the unique and modified iterations of YOLO seem to pose additional challenges. YOLO’s complexity and customizations for diverse applications present compatibility challenges. Consequently, not all XAI methods are compatible with YOLO. Figure 7 compares the heatmaps generated by the six XAI methods we analyzed. As shown in the figure, the heatmap outputs of Grad-CAM, Grad-CAM++, Eigen-CAM, and XGrad-CAM are less effective compared to EigenGrad-CAM, LayerCAM, and HiResCAM. Notably, all these XAI methods were applied to the same YOLO model, which was trained on the training dataset. This issue is due to the different gradient computation approaches used in each XAI method, as discussed in the “Methodology section (III) XAI”. Based on this comparison, EigenGrad-CAM, LayerCAM, and HiResCAM demonstrated the most effective results.

Figure 7 Breast cancer image after applying 7 different XAIs. CAM, class activation map; XAI, explainable artificial intelligence.

To identify the most suitable XAI methodology for YOLO11 trained on breast cancer image datasets, five performance metrics were employed: mGT, precision, recall, PCC, and RMSE.

From Figure 7, it is clear that the differences between the three XAIs—Eigen-CAM, LayerCAM, and HiResCAM—are indistinguishable through visual observation only. Therefore, a quantitative evaluation using the performance metrics presented in Tables 2-5 is necessary. The results indicate that HiResCAM achieved the highest average mGT score (0.4908), surpassing EigenGrad-CAM (0.4531) and LayerCAM (0.4266). Additionally, HiResCAM recorded the highest number of images with mGT scores exceeding 0.5. It also exhibited higher precision and recall compared to the other two methods, along with the lowest RMSE. While the PCC of HiResCAM is slightly lower than that of EigenGrad-CAM, these findings collectively suggest that HiResCAM provides superior heatmap localization quality in regions of high relevance.

The results further demonstrate that the heatmaps generated by HiResCAM closely align with the ground truth, accurately highlighting areas of interest and reflecting the model’s predictive accuracy. By leveraging both the mGT metric and the XAI methodology, the trustworthiness of the model is enhanced, providing users with greater confidence in its predictions and interpretability.

Discussion

This section presents the rationale for selecting each model size and the key hyperparameters pertinent to the training process. As detailed in the Results section, among the three state-of-the-art YOLO versions evaluated, YOLO11 demonstrated the most effective performance for the classification and detection of breast cancer images within our dataset. It achieved superior accuracy while maintaining a balanced trade-off between predictive performance and computational efficiency. Subsequently, combining XAI with YOLO11 aimed to focus on model interpretability through heatmap visualizations. As discussed in the Introduction, XAI plays a critical role not only in enhancing user trust by elucidating the opaque decision-making processes of AI models, but also in identifying potential shortcomings within the training process, thereby guiding future improvements. Therefore, this section will also discuss the key insights derived from the XAI visualizations and highlights notable observations that inform both model evaluation and refinement.

During the training process, YOLOv9c, comprising 25.5 million parameters, was selected to optimize the computational resources. Based on this reason YOLOv10l and YOLO11l were chosen with parameters 24.4 and 25.3 million in order, which is closely aligned with that of YOLOv9c. The hyperparameters for both YOLO models were largely identical, with notable exceptions in augmentation techniques and overlap masking strategies. All models were trained over 100 epochs, employing a batch size of 16 and an image size of 640×640 pixels. Key hyperparameters included an initial learning rate of 0.01, momentum of 0.937, weight decay of 0.0005, and the utilization of automatic mixed precision (AMP) to facilitate expedited training.

The next phase of this research involves visualizing the decision-making process using seven different XAI methodologies, including Grad-CAM, Grad-CAM++, Eigen-CAM, XGrad-CAM, EigenGrad-CAM, LayerCAM, and HiResCAM. The interpretations provided by these XAI methods reveal that a few of the images for which the YOLO model fails to generate bounding boxes correspond to the same images, where the XAI techniques fail to highlight any pixels. These images are notably of low quality. Based on this observation, we employed CLAHE to enhance the image quality. The application of CLAHE demonstrated promising results: 50% of the previously undetectable images were successfully identified by the YOLO model, and the XAI methods were able to highlight important regions within these images. CLAHE enhances image quality by increasing contrast, thereby facilitating lesion detection in breast cancer images using YOLO. Experimental results indicate that applying CLAHE specifically to blurred or low-quality images prior to detection and classification significantly improves performance. Conversely, applying CLAHE indiscriminately to all images may result in a slight decline in performance rather than an enhancement.

As illustrated in Figure 8, the two images represent a mammogram before and after the application of CLAHE, demonstrating notable differences in contrast and detail visibility. The enhanced image (right) exhibits superior contrast compared to the original image (left), facilitating improved visualization of anatomical structures, including tissue details and blood vessels. This enhancement renders faint structures more discernible, which is particularly advantageous for detecting abnormalities such as microcalcifications or tumors.

Figure 8 Breast cancer image before and after applying CLAHE. CLAHE, contrast limited adaptive histogram equalization.

Conclusions

Although numerous studies have focused on integrating classification and detection models with medical imaging, the latest YOLO iterations—YOLOv9, YOLOv10, and YOLO11—have yet to be widely explored for breast cancer image detection. This study addresses this gap by exploring these state-of-the-art YOLO versions to identify the most suitable model for breast cancer imaging tasks. Given the lack of transparency in AI decision-making processes, often referred to as the “black box” problem, XAI methodologies were employed to visualize the decision-making process of the CNN models used in this study. Seven XAI methods—Grad-CAM, Grad-CAM++, Eigen-CAM, XGrad-CAM, EigenGrad-CAM, LayerCAM, and HiResCAM—were applied to enhance the interpretability of the YOLO models. The best-performing YOLO version identified in the initial analysis was subsequently paired with these XAI techniques to determine the most effective visualization method.

The YOLO11 model demonstrated superior performance with the highest mAP score of 0.926. However, low-quality images presented detection challenges, as XAI analysis revealed the absence of bounding boxes in such cases. By applying CLAHE for image enhancement, 50% of these previously undetected images were successfully identified. For this reason, applying CLAHE after training, specifically during prediction, is more effective for addressing low-quality images. Additional image enhancement techniques will be explored in future research to further improve detection accuracy. Among the seven XAI methods evaluated, EigenGrad-CAM, LayerCAM, and HiResCAM stood out for their clarity and qualitative accuracy. Using quantitative metrics such as mGT, PCC, and RMSE, HiResCAM emerged as the best-performing XAI technique, with an average mGT score of 0.490793 and the largest number of images exceeding the mGT threshold of 0.5. These results demonstrate the synergy between the YOLO11 model and HiResCAM, underscoring their potential for enhancing the accuracy, transparency, and reliability of breast cancer detection systems.

The integration of XAI methodologies enhances the trustworthiness of AI models such as YOLO by providing insights into their decision-making processes. Furthermore, XAIs can identify potential weaknesses within the model, such as areas of bias or regions of focus during decision-making, thereby offering developers valuable information for model refinement. For instance, in this study, the interpretation of XAI results highlighted issues with low-quality images that lacked detection bounding boxes. By applying the CLAHE technique to enhance image quality, the number of images with detectable bounding boxes was significantly increased, demonstrating the utility of XAIs in improving both model performance and interpretability.

Acknowledgments

Authors express our sincere gratitude to King Mongkut’s Institute of Technology Ladkrabang (KMITL) and the KMITL Research & Innovation Services (KRIS) for their generous financial support, without which this research would not have been possible. We also gratefully acknowledge the creators of the INbreast and MIAS datasets for making the images publicly available, which was essential to this research.

Footnote

Reporting Checklist: The authors have completed the CLEAR reporting checklist. Available at https://qims.amegroups.com/article/view/10.21037/qims-2024-2911/rc

Funding: This study was supported by King Mongkut’s Institute of Technology Ladkrabang (KMITL) Research and Innovative Service (KRIS) (grant No. KREF016616).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-2024-2911/coif). Both authors report that this research was supported by King Mongkut’s Institute of Technology Ladkrabang (KMITL) Research and Innovative Service (KRIS) (No. KREF016616). The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Bray F, Laversanne M, Sung H, Ferlay J, Siegel RL, Soerjomataram I, Jemal A. Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2024;74:229-63. [Crossref] [PubMed]
2. Insamran W, Sangrajrang S. National Cancer Control Program of Thailand. Asian Pac J Cancer Prev 2020;21:577-82. [Crossref] [PubMed]
3. Cannioto RA, Attwood KM, Davis EW, Mendicino LA, Hutson A, Zirpoli GR, Tang L, Nair NM, Barlow W, Hershman DL, Unger JM, Moore HCF, Isaacs C, Hobday TJ, Hortobagyi GN, Gralow JR, Albain KS, Budd GT, Ambrosone CB. Adherence to Cancer Prevention Lifestyle Recommendations Before, During, and 2 Years After Treatment for High-risk Breast Cancer. JAMA Netw Open 2023;6:e2311673. [Crossref] [PubMed]
4. Ekpo EU, Alakhras M, Brennan P. Errors in Mammography Cannot be Solved Through Technology Alone. Asian Pac J Cancer Prev 2018;19:291-301. [Crossref] [PubMed]
5. Halalli B, Makandar A. Computer Aided Diagnosis - Medical Image Analysis Techniques | IntechOpen. Available online: https://www.intechopen.com/chapters/56615
6. Darma IWAS, Suciati N, Siahaan D. A Performance Comparison of Balinese Carving Motif Detection and Recognition using YOLOv5 and Mask R-CNN. In: 2021 5th International Conference on Informatics and Computational Sciences (ICICoS) 2021:52-7. Available online: https://ieeexplore.ieee.org/document/9651855
7. Redmon J, Divvala S, Girshick R, Farhadi A. You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016:779-88. Available online: https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/Redmon_You_Only_Look_CVPR_2016_paper.html
8. Wang CY, Yeh IH, Mark Liao HY. YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information. In: Leonardis A, Ricci E, Roth S, et al. editors. Computer Vision – ECCV 2024. Cham: Springer; 2025:1-21.
9. Wang A, Chen H, Liu L, et al. YOLOv10: Real-Time End-to-End Object Detection. Available online: https://openreview.net/pdf?id=tz83Nyb71l
10. YOLO11: Future of AI Vision | Ultralytics [Internet]. [cited 2024 November 2]. Available online: https://www.ultralytics.com/blog/ultralytics-yolo11-has-arrived-redefine-whats-possible-in-ai
11. Samek W, Müller KR. Towards Explainable Artificial Intelligence. In: Samek W, Montavon G, Vedaldi A, et al. editors. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Cham: Springer International Publishing; 2019:5-22.
12. Al-masni MA, Al-antari MA, Park JM, Gi G, Kim TY, Rivera P, et al. Detection and classification of the breast abnormalities in digital mammograms via regional Convolutional Neural Network. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2017:1230-3. Available online: https://ieeexplore.ieee.org/abstract/document/8037053?casa_token=b6qYCplq9K0AAAAA:dDn_wozD-w6It6qPPU8iMGkdUKZ-cM0c3P1rPvdVtIMOJ5xtSrO_ZbIcBmH8kQN4yqraHCWZ
13. Bal A, Das M, Satapathy SM. YOLO as a Region Proposal Network for Diagnosing Breast Cancer. In: 2021 Grace Hopper Celebration India (GHCI) 2021:1-6. Available online: https://ieeexplore.ieee.org/abstract/document/9513988?casa_token=rlzuSWDZ394AAAAA:hK02ACr4OpeegyTiI97TBxaoROn0dPsSuSyhJan4RNvNFsfo8seTx3isDuruvAUM2n_zBk25
14. Hamed G, Marey M, Amin SE, Tolba MF. Automated Breast Cancer Detection and Classification in Full Field Digital Mammograms Using Two Full and Cropped Detection Paths Approach. IEEE Access 2021;9:116898-913.
15. Su Y, Liu Q, Xie W, Hu P. YOLO-LOGO: A transformer-based YOLO segmentation model for breast mass detection and segmentation in digital mammograms. Comput Methods Programs Biomed 2022;221:106903. [Crossref] [PubMed]
16. Baccouche A, Garcia-Zapirain B, Zheng Y, Elmaghraby AS. Early detection and classification of abnormality in prior mammograms using image-to-image translation and YOLO techniques. Comput Methods Programs Biomed 2022;221:106884. [Crossref] [PubMed]
17. Frank SJ. A deep learning architecture with an object-detection algorithm and a convolutional neural network for breast mass detection and visualization. Healthcare Analytics 2023;3:100186.
18. Prinzi F, Insalaco M, Orlando A, Gaglio S, Vitabile S. A Yolo-Based Model for Breast Cancer Detection in Mammograms. Cognitive Computation 2024;16:107-20.
19. Talaat FM, Gamel SA, El-Balka RM, Shehata M. ZainEldin H. Grad-CAM Enabled Breast Cancer Classification with a 3D Inception-ResNet V2: Empowering Radiologists with Explainable Insights. Cancers (Basel) 2024;16:3668. [Crossref] [PubMed]
20. Munshi RM, Cascone L, Alturki N, Saidani O, Alshardan A, Umer M. A novel approach for breast cancer detection using optimized ensemble learning framework and XAI. Image and Vision Computing 2024;142:104910.
21. Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS. INbreast: toward a full-field digital mammographic database. Acad Radiol 2012;19:236-48. [Crossref] [PubMed]
22. Suckling J. The Mammographic Image Analysis Society Digital Mammogram Database. In: Excerpta Medica International Congress Series. York, England; 1994:375-8.
23. Ultralytics Team. [cited 2024 November 2]. Available online: https://www.ultralytics.com/about
24. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. Int J Comput Vis 2020;128:336-59.
25. Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN. Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) 2018:839-47.
26. Muhammad MB, Yeasin M. Eigen-CAM: Class Activation Map using Principal Components. In: 2020 International Joint Conference on Neural Networks (IJCNN) 2020:1-7. Available online: https://ieeexplore.ieee.org/abstract/document/9206626?casa_token=4yeoc1jtccIAAAAA:BIn-NVDhg5hZ-fY6dFW1wYKhD7npIw-Z7Ky5AQBGD58hbBoZzbiw_MD8_35VNan3rBvlfW3jATQ
27. YOLOv8-Explainer: Python packages that enable XAI methods for YOLOv8. [cited 2024 Oct 27]. Available online: https://spritan.github.io/YOLOv8_Explainer/
28. Fu R, Lay N, Roth H, et al. A visual interpretability for pulmonary disease classification using chest X-ray images. In: Proceedings of the British Machine Vision Conference (BMVC). 2020. Available online: https://www.bmvc2020-conference.com/assets/papers/0631.pdf
29. Jiang PT, Zhang CB, Hou Q, Cheng MM, Wei Y. LayerCAM: Exploring Hierarchical Class Activation Maps for Localization. IEEE Trans Image Process 2021;30:5875-88. [Crossref] [PubMed]
30. Draelos RL, Carin L. Explainable multiple abnormality classification of chest CT volumes. Artificial Intelligence in Medicine 2022;132:102372. [Crossref] [PubMed]
31. Szczepankiewicz K, Popowicz A, Charkiewicz K, Nałęcz-Charkiewicz K, Szczepankiewicz M, Lasota S, Zawistowski P, Radlak K. Ground truth based comparison of saliency maps algorithms. Sci Rep 2023;13:16887. [Crossref] [PubMed]