Modeling long-range dependencies for weakly supervised disease classification and localization on chest X-ray

Introduction

Chest X-ray (CXR) imaging is one of the most commonly available and widely applied radiological examinations at present, and is essential for screening and clinical diagnosis (1,2). In clinical practice, the reading of CXRs heavily depends on manual observation by radiologists with professional knowledge and experience. However, due to the complex pathologies of different lung lesions, even radiologists with long-term clinical training and professional guidance can make mistakes (3). The emergence and rapid progress of deep learning can improve disease diagnosis based on CXRs and lessen the chances of mistakes being made (4-6). As a fundamental task in medical image analysis, image detection allows for disease classification and lesion localization, which helps radiologists to work more efficiently and accurately. Therefore, it is crucial that a computer-aided diagnosis system for thoracic disease classification and localization on CXR images is developed (7).

The development of deep learning methods for disease detection is heavily dependent on supervised learning with high-quality disease annotations, such as pixel-level labels, which can be costly. Large public chest radiography datasets such as ChestX-ray14 (8) and CheXpert (9), which do not provide pixel-wise annotations or coarse bounding boxes for most CXR images, are not suitable for training supervised disease detection models. Consequently, several unsupervised and weakly supervised learning methods have been proposed to handle tasks without pixel-level annotations (8,10,11). However, the established workflow of unsupervised methods is complex, challenging, and inefficient in clinical-aided analysis (12). Therefore, the development of automatic detection models for thoracic diseases must rely on weakly supervised methods with image-level labels and a minimal amount of bounding box annotations (3,8).

Recently, convolutional neural networks (CNNs) have been widely adopted for weakly supervised thoracic disease detection (8,13-15). In CNNs, each kernel slides over the image and detects the same local pattern in each field. Convolution layers learn image features from small input patches and preserve the spatial relations between pixels. A recent study showed that CNNs can learn object detectors effectively, even when they are trained as classifiers using only global labels (16). The localization ability of CNNs is achieved by identifying the regions used for classification in the last convolutional layer of the network. However, CNN-based approaches generally carry limitations for the modeling of long-range dependencies that are present in an image; such limitations have been noted in computer vision tasks (17,18). In brain ventricle segmentation, for example, Valanarasu et al. (18) found that CNNs misclassified the background as a mask but long-range dependency learning could help to prevent the segmentation network from making this mistake. Due to the intrinsic locality of convolution operations, each CNN convolutional kernel pays attention to only several local pixels in the whole image, which forces the CNN to concentrate on local characteristics rather than learning the global patterns. Figure 1 visualizes an example of lesion localization results of a CNN-based method and our method in the ChestX-ray14 dataset. In Figure 1A, the CNN misidentifies the lower lobe of the left lung as effusion. For a network to provide efficient localization, it should activate the class-specific image regions. However, due to the limitations in the modeling of long-range dependencies, the “lesions” identified by the CNN are only local abnormalities and not true lesions. Our proposed method does not make this mistake because it learns the long-range dependencies of the global context of the image, which helps the network to discriminate true abnormal regions and reduce false positives.

Figure 1 Visual localization results of a CXR from a case diagnosed with “effusion”. The bounding boxes in green represent the published ground truth, and the red represents the predicted results of our proposed method. (A) The lesion localized by DenseNet121 (a CNN-based method). (B) The lesion localized by our proposed method. CXR, chest X-ray; CNN, convolutional neural network.

To address the problem of long-range dependencies, previous studies have proposed combining CNNs with, for example, attention mechanisms and bag-of-visual-words model, to further input features from CNNs into the attention module or use such features to define visual dictionaries for extracting the semantic context (19-21). However, these studies focused on the semantic context and spatial relationship between regions of interest (ROIs) and the features of CNNs rather than global attention on full-size images. Moreover, no previous study has focused on modeling long-range dependencies to improve the performance of medical image localization tasks. Using a transformer derived from natural language processing (NLP) for sequence-to-sequence prediction is a possible solution with an innate global self-attention mechanism (22). Following its success in NLP, the transformer has very recently been applied to computer vision applications (17,23). Dosovitskiy et al. (23) proposed the vision transformer (ViT), which achieved the state of the art in ImageNet (24) classification. However, a ViT trained on patches alone limits the network’s ability to learn any dependencies for inter-patch pixels, which is insufficient for medical image detection (18).

Some existing research shows that the localization ability of a network is improved when the feature maps before the global pooling layer have high spatial resolution (25). However, most previous weakly supervised approaches have typically applied deep network structures. The feature maps generated by these approaches usually lack sufficient spatial resolution, which prevents accurate lesion localization. Therefore, CNN-based deep networks generally have weak performances, especially when target structures in pathological images exhibit significant inter-patient variation in texture, shape, and size.

To address the above limitations, this work proposes a weakly supervised deep learning model with the following merits. First, to obtain fine-grained saliency maps in which both large and small lesions can be accurately delineated, we used a pyramid structure to upsample and combine the low-resolution features from multiple layers of DenseNet, a CNN-based network (26). Second, we adjusted the ViT to generate features that capture the global context of the whole image. To integrate the features from the pyramid DenseNet and the adaptive ViT, we designed a fusion module, which forced the network to focus on patch-wise and inter-patch dependencies simultaneously. Our model jointly generates disease classification and lesion localization by relying on saliency map detectors (8,27) and Log-Sum-Exp (LSE) pooling functions. Based on the official training and test set split, we experimented with our proposed method on the ChestX-ray14 dataset. Our proposed method achieved state-of-the-art disease classification and localization results in both quantitative and qualitative visual assessments. We also evaluated the generalization performance of our method in an external CXR dataset from the validation set of CheXpert. Our results suggest that weakly supervised object detection benefits from the transformer’s ability to model long-range dependencies. We present the following article in accordance with the TRIPOD reporting checklist (available at https://qims.amegroups.com/article/view/10.21037/qims-21-1117/rc).

Methods

Datasets

This study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

To train and evaluate our proposed model in disease classification and lesion localization, we performed our experiments on the large public CXR dataset in the United States, ChestX-ray14 (8). The ChestX-ray14 dataset contains 112,120 frontal-view CXRimages of 30,805 patients collected from the clinical picture archiving and communication system (PACS) database at the National Institutes of Health Clinical Center between 1992 and 2015. The dataset has 14 disease image labels that were text-mined from the associated radiological reports using NLP (where each image can have multi-labels). The 14 most common thoracic pathologies in the dataset are atelectasis, cardiomegaly, effusion, infiltration, mass, nodule, pneumonia, pneumothorax, consolidation, edema, emphysema, fibrosis, pleural thickening, and hernia. In the test set of this dataset, 880 images are annotated with bounding boxes relating to the first eight diseases mentioned above.

To externally evaluate the generalization performance of our method, we collected 123 CXR images from the validation set of another public CXR dataset, CheXpert (9). The CheXpert dataset consists of 224,316 chest radiographs of 65,240 patients that were collected from Stanford Hospital between October 2002 and July 2017. The dataset has 14 disease image labels which differ slightly from those of ChestX-ray14, including no finding, atelectasis, cardiomegaly, pleural effusion, pneumonia, pneumothorax, consolidation, edema, enlarged cardiomediastinum, lung opacity, lung lesion, pleural other, fracture, and support devices. The images selected for our study needed to contain at least one positive result for five diseases: atelectasis, cardiomegaly, pleural effusion, pneumonia, and pneumothorax, which overlapped with the first eight diseases for the ChestX-ray14 dataset. Also, based on the ground truth of the cases from the CheXpert validation set, two board-certified chest radiologists by Shanghai Municipal Health Commission [with 8 and 2 years of experience in computed tomography (CT) scan interpretation] performed a double-blind review to obtain bounding box annotations using the MarkMan software (http://www.getmarkman.com/).

Method overview

This work aimed to establish a weakly supervised detection model for thoracic disease classification and lesion localization which performs localization by drawing bounding boxes on CXRs. The model was trained on CXRs with only image-wise labels of 14 diseases. Our proposed approach was shaped by three ideas. First, to solve the problem of insufficient resolution caused by the deep network, we designed a DenseNet with a feature pyramid structure to generate fine-resolution feature maps. The reasons for choosing DenseNet as our CNN-based network branch were as follows: (I) due to feature reuse, DenseNet has a small number of parameters and efficient calculation, which was crucial to the parallel training of our proposed two network branches (28); (II) DenseNet improves the information and gradient propagation with dense connections, which makes the training of very deep networks tractable (28). Second, we adjusted the ViT to encode the global context features of the whole image and combined them with the features from the pyramid DenseNet for training together. Third, we encoded the features from previous stages into class-specific saliency maps and used an LSE pooling structure to pool each saliency map into a single disease probability score. Disease localization was achieved by calculating saliency maps in the forward propagation of the trained model. Therefore, our proposed method generated the features with fine resolution, modelled the long-range and inter-patch pixels dependencies, and combined the context of two branches for training. The network structure is illustrated in Figure 2.

Figure 2 An overview of our proposed method. Our model comprises two branches and a fusion module. The classification score was acquired from the LSE layer, and the localization was obtained from the saliency maps before the LSE layer. ViT, vision transformer; CNN, convolutional neural network; LSE, Log-Sum-Exp.

Pyramid DenseNet

Huang et al. (28) proposed the DenseNet, which introduces direct connections from any preceding layers to all subsequent layers with the same feature map size. For each layer, the inputs consist of feature maps from all previous convolution blocks, and the outputs generated by this layer are passed to all subsequent layers. However, the repeated downsampling processes in dense blocks diminish the spatial resolution of feature maps. After several dense blocks in the basic structure have encoded, the output feature maps significantly decrease in size compared to the input image, losing the detailed semantic information.

To improve the localization ability of the network, we designed a pyramid DenseNet to generate fine-resolution feature maps. We retained the structure of DenseNet before the fully connected layer as our basic structure and introduced the feature pyramid structure. Specifically, we upsampled the feature maps of the fourth dense block of DenseNet and fused them with the outputs of the third dense block. When the input image size was 512×512, the pyramid DenseNet output the feature maps with the spatial size of 64×64, which was much larger than the feature maps obtained after the final dense block of DenseNet (16×16). Figure 3 shows the pyramid DenseNet in detail. We used DenseNet121 as our primary structure in this study. Note that we did not continue to upsample the fused feature maps of previous outputs, because a size increase in the feature maps of the ViT—another branch of our model introduced in the next section—could lead to huge resource consumption. With the pyramid DenseNet, we obtained fine-resolution feature maps.

Figure 3 The architecture of the pyramid DenseNet. The network receives the input image and passes it via the main structure, which consists of four dense blocks interspersed with three transition blocks. After being generated by the last dense block, low-resolution feature maps were upsampled and fused with the outputs of the previous third dense block to obtain the fine resolution feature maps.

Adaptive ViT

ViTs use a pure transformer model for image classification by embedding image patches that take on the role of words and extracting deep features of images (23). However, the patch-wise training of ViTs captures only the semantic information between patches and loses the information or dependencies for inter-patch pixels.

To model the patch-wise and inter-patch dependencies of the whole image in the network, we designed an adaptive ViT to capture global features and a fusion module to integrate the local features from the pyramid DenseNet and the global features from the adaptive ViT. Details of our adaptive ViT and fusion module are shown in Figures 2,4. According to the work of Dosovitskiy et al. (23), several variants of ViT have been defined, and we used ViT-Base in our work. ViT-Base consists of a patch embedding layer, 12 transformer encoder layers comprising an attention layer and a feed-forward layer, and a classification head implemented by a multilayer perceptron (MLP) with a single hidden layer (23). The adaptive ViT maintains the ViT-Base structure before the classification head as the basic structure to obtain the embedded patch sequence. For the purpose of localization, our adaptive ViT recovers the spatial order of the patch sequence by reshaping it into two-dimension feature maps.

Figure 4 The architecture of the adaptive ViT. Norm, normalization; MLP, multilayer perceptron; ViT, vision transformer.

In the first stage, the adaptive ViT divides the image into non-overlapping square patches of size p² pixels, where p refers to the side length of the square patch and is designated as 16 in our work. Then, the adaptive ViT flattens the patches into vectors and linearly projects them to embedded patches. Along with the position embedding, the embedded patches are passed through a transformer encoder with 12 layers and obtained an embedded patch sequence. Finally, the patch sequence is reshaped into two-dimensional feature maps. Given the input image of size H×W×3, our adaptive ViT obtains the feature maps of size H/16×W/16×C, where C refers to the number of feature map channels.

Our designed fusion module receives the feature maps from the pyramid DenseNet and adaptive ViT as inputs. As shown in Figure 2, the fused features are passed through a convolutional block that produces K saliency maps (one map for each class). The global branch adaptive ViT focuses on deep context information, and the local branch pyramid DenseNet focuses on fine-grained information. Our proposed method models patch-wise and inter-patch dependencies and combines the features of two branches for training.

LSE pooling

A specific weakly supervised learning technique was introduced into our method to accomplish accurate disease classification and lesion localization. The presence or absence of each pathology is denoted by y. As shown in Figure 2, we applied LSE pooling to pool each of the saliency maps into a single disease probability score, as denoted by y˜. We trained our network with binary cross-entropy loss between the prediction and the ground truth. The label distribution of our dataset was unbalanced, and there were many cases with no pathologies; therefore, we used positive and negative weight factors W_P,W_N∊R^K to balance the label distribution for each disease. The loss function for each sample i is:

l(y˜i,yi)=−WPyilog(y˜i)−WN(1−yi)log(1−y˜i)[1]

Where WP(c)=1−Pc/|G|, WN(c)=Pc/|G|, Pc indicates the number of positive cases for class c and |G| indicates the training set size.

To obtain the bounding box of the disease, a thresholding technique was applied to segment the class-specific saliency maps generated by the CNN module in Figure 2. Based on ground-truth bounding box annotations, the thresholds used for segmentation were empirically determined and followed the criterion of previous approaches in this field (8,15). Therefore, our method is proposed for disease classification and lesion localization and requires only CXRs and coarse labeling of image-level annotations for network training.

Experiment

Dataset partition and experimental set-up

To maintain a fair comparison, we tested our proposed method in the ChestX-ray14 dataset and followed the official training-test set partitioning. For the selection of model parameters, we selected 10% of the training set as the validation set and repeated the random split 10 times to improve the reliability of the results. For localization assessment, all 880 CXRs and their ground-truth boundary boxes were used. The label distribution of the experimental data used for training and evaluation is shown in Table 1.

At the training stage, we used a learning rate of 0.00001 and the optimization of Adam (29), with a momentum of 0.9, weight decay of 0.001, and a minibatch size of 16. To improve the computational performance, inputs of 1,024×1,024 were downsampled to 512×512 and normalized using the ImageNet mean and standard deviation (24). During training, we applied simple image augmentation, in which each image was zoomed between 0 and 0.1, translated in four directions between −50 and 50 pixels, and rotated between −10 and 10 degrees.

Evaluation metrics

To evaluate the classification performance of our method, we calculated the accuracy, precision, sensitivity, and F1-score using the following equations:

Accuracy=TP+TNTP+TN+FP+FN[2]

Precision=TPTP+FP[3]

Sensitivity=TPTP+FN[4]

F1score=2×Presision×SensitivityPresision+Sensitivity[5]

We also calculated the area under the curve (AUC) for each class and the average of all classes to compare our classification results with those of other state-of-the-art methods (8,30,31). To evaluate the localization accuracy, we computed the intersection of the union (IoU) between the predicted bounding boxes of our proposed approach and the provided ground truth.

Results

Table 2 shows the classification performance of our proposed method for 14 diseases. Table 3 compares the AUCs for classification between our approach and several baselines. For all 14 diseases, our model achieved better classification results than the model of Wang et al. (8). Our model also performed significantly better than that of Ma et al. (31) in 11 out of 14 diseases, and it outperformed Yao et al.’s (30) model in 9 out of 14 diseases. Furthermore, the AUCs of our model were more than 2% higher than those of the Yao et al. (30) and Ma et al. (31) models for “atelectasis”, “infiltration”, “nodule”, “pneumonia”, “pneumothorax”, “emphysema”, and “pleural thickening”.

Table 4 compares the localization performance of our method with that of previously published methods under different IoU thresholds (8,32). Our method achieved better localization results than that of Wang et al. (8) in 6 of 8 diseases, which showed that the saliency maps were well fitted with the ground truth bounding boxes. For “atelectasis” and “nodule”, our model was only slightly inferior to the existing CNN-based methods, which indicated that focusing on more local information may activate more accurate image regions for those two diseases. Furthermore, when compared to the model of Zhou et al. (32), which is based on a CNN and consists of a customized pooling structure and an adaptive DenseNet front-end, our model showed significant improvements in the localization of all 14 diseases. The localization results of the above methods and our proposed method are visualized in Figure 5 (8,32). The “abnormalities” identified by Wang et al.’s method were usually multiple and covered large areas, and the results of Zhou et al. were small but incorrect. With CNNs as their backbone, these methods focus on local patterns, which caused multiple of incorrect abnormalities to be identified. Both quantitative and qualitative experimental results showed that our method could effectively capture the lesions due to its learning of inter-patch and patch-wise dependencies and activation of the imaging regions used for classification globally. The localization results of our proposed method for some examples are visualized in Figure 6.

Figure 5 Several visualized localization results of Wang et al. (8), Zhou et al. (32), and our proposed method. The bounding boxes in green represent the published ground truth, and the red represents the predicted results using different methods.

Figure 6 Examples of visual localization outcomes for eight diseases in the ChestX-ray14 dataset. The images on the left are the input CXRs and are labeled with two bounding boxes: the green shows the published ground truth, and the red is the predicted result obtained by a simple threshold method on the class-specific saliency maps, which are shown on the right. The IoU results calculated by ground truth and predicted bounding boxes are shown on the left. CXR, chest X-ray; IoU, intersection of the union.

Discussion

In clinical procedures, it is insufficient that computer-aided chest radiography diagnostic systems only predict disease classification. Spatial localization of disease lesions provides visual evidence for the classification result, which is indispensable for auxiliary diagnosis. Disease localization potentially decreases the number of false positives and improves the diagnostic efficiency. Our weakly supervised disease detection method was trained only on image-level annotations but can generate disease classifications and lesion spatial localization simultaneously. In contrast to the previous state of the art, our method can identify global abnormalities in whole images by integrating ViT and DenseNet121 with a pyramid structure, which improves the localization performance.

To evaluate the generalization performance of our method, we calculated the AUCs for disease classification and IoU localization results in 123 external test cases (Table 5). The overall results showed that our method performed well in the external dataset, indicating its significant detection performance and excellent generalization ability. Interestingly, “pneumonia” only achieved an AUC of 0.571, which can be explained by our external dataset having only eight positive pneumonia cases; this makes this result biased and unreliable.

Table 6 shows the ablation study for the quantitative localization results of different feature encoders with DenseNet121, ViT, and our network (DenseNet121+ViT) as backbones. Qualitative evaluation of these three networks is shown in Figure 7. The predicted bounding boxes show that our network captures long-range dependencies and generates fine-grained feature maps extremely well. For example, as shown in the first row of Figure 7, DenseNet121 cannot detect real lesions because it only identifies locally suspected “abnormalities” and is confused about real lesions. However, as the ViT backbone encodes long-range dependencies, it learns to successfully localize in the whole image. As shown in the second and third rows, the ViT backbone network captures the lesion and its coarse location as the patch-wise training restricts the network in learning inter-patch and fine-grained information. As shown in the last column of Figure 7, our method takes long-range and inter-patch dependencies into account by integrating ViT and DenseNet121 with a pyramid structure, which makes its predictions accurate and detailed.

Figure 7 Several qualitative results of different feature encoder backbones in the ChestX-ray14 dataset: DenseNet121, ViT, and our network (DenseNet121+ViT). The bounding boxes in green represent the published ground truth, and the red represents the predicted results. ViT, vision transformer; DenseNet121+ViT, our proposed method which includes two network branches, DenseNet121 and ViT.

However, our method has some notable shortcomings. First, the image-level annotations in ChestX-ray14 were obtained using NLP technology rather than through manual observation by radiologists. The annotations used in our method training and validation are not completely correct. Therefore, we will explore advanced text mining methods to improve the accuracy of automatic tagging in our subsequent work (9). Second, due its integration of two network branches, our method requires much computing resources and exhibits low prediction efficiency. Therefore, another future improvement of our method is to develop a cascade network to encode patch-wise and inter-patch dependencies. Fewer parameters and simpler networks can improve the diagnosis efficiency of computer-aided diagnosis systems. Third, only a small external dataset with 123 CXR images was used for model validation, and the generalization ability of our method could not be effectively evaluated. Therefore, in our future work, we will collect more CXRs with annotations from clinical scenarios to expand the dataset for model validation and to evaluate the generalization of the model comprehensively and effectively.

Conclusions

Both quantitative and qualitative visual results showed that our proposed method achieves the new state-of-the-art for disease classification and weakly supervised localization. Compared to previous methods, our method produced superior results by using ViT to learn long-range dependencies between patches of the whole images, activating image regions used for classification globally, and avoiding mislocalization caused by the intrinsic locality of convolution operations. Furthermore, because ViTs can only learn inter-patch dependencies, we applied DenseNet121 along with a feature pyramid structure to encode high-resolution spatial information and learn inter-patch dependencies in our network. As an alternative framework to the dominant CNN-based approaches for medical image detection, our method achieves a superior performance.

Acknowledgments

We would like to thank the Shanghai Double First-Class University Construction and Development of High-Level Local Universities: Intelligent Medicine Emerging Interdisciplinary Cultivation Project and the Medical Research Data Center of Fudan University for providing financial assistance and technical advice.

Funding: This work was supported by the National Natural Science Foundation of China (No. 91846302) and the S&T Program of Hebei (No. 21377734D).

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://qims.amegroups.com/article/view/10.21037/qims-21-1117/rc

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-21-1117/coif). FL has a pending patent (No. 202110640995.8). The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The datasets used in this study are all public CXRs. No ethical approval and written informed consent were required.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Körner M, Weber CH, Wirth S, Pfeifer KJ, Reiser MF, Treitl M. Advances in digital radiography: physical principles and system overview. Radiographics 2007;27:675-86. [Crossref] [PubMed]
2. Hui TCH, Khoo HW, Young BE, Haja Mohideen SM, Lee YS, Lim CJ, Leo YS, Kaw GJL, Lye DC, Tan CH. Clinical utility of chest radiography for severe COVID-19. Quant Imaging Med Surg 2020;10:1540-50. [Crossref] [PubMed]
3. Yan C, Yao J, Li R, Xu Z, Huang J. Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays. Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics; Washington, DC, USA: Association for Computing Machinery; 2018:103-4.
4. Kim HG, Lee KM, Kim EJ, Lee JS. Improvement diagnostic accuracy of sinusitis recognition in paranasal sinus X-ray using multiple deep learning models. Quant Imaging Med Surg 2019;9:942-51. [Crossref] [PubMed]
5. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60-88. [Crossref] [PubMed]
6. Setio AAA, Traverso A, de Bel T, Berens MSN, Bogaard CVD, Cerello P, et al. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge. Med Image Anal 2017;42:1-13. [Crossref] [PubMed]
7. Qin C, Yao D, Shi Y, Song Z. Computer-aided detection in chest radiography based on artificial intelligence: a survey. Biomed Eng Online 2018;17:113. [Crossref] [PubMed]
8. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM. editors. ChestX-Ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases. Proceedings of the IEEE conference on computer vision and pattern recognition; 2017:3462-71.
9. Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Marklund H, Haghgoo B, Ball R, Shpanskaya K, Seekins J, Mong DA, Halabi SS, Sandberg JK, Jones R, Larson DB, Langlotz CP, Patel BN, Lungren MP, Ng AY. CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. Proceedings of the AAAI Conference on Artificial Intelligence; 2019:590-7.
10. Li P, Sun M, Wang Z, Chai B. OPTICS-based Unsupervised Method for Flaking Degree Evaluation on the Murals in Mogao Grottoes. Sci Rep 2018;8:15954. [Crossref] [PubMed]
11. Shahid KT, Schizas I. Unsupervised Mitral Valve Tracking for Disease Detection in Echocardiogram Videos. J Imaging 2020;6:93. [Crossref] [PubMed]
12. Zadeh Shirazi A, Seyyed Mahdavi Chabok SJ, Mohammadi Z. A novel and reliable computational intelligence system for breast cancer detection. Med Biol Eng Comput 2018;56:721-32. [Crossref] [PubMed]
13. Kumar P, Grewal M, Srivastava MM, editors. Boosted Cascaded Convnets for Multilabel Classification of Thoracic Diseases in Chest Radiographs. In: International Conference Image Analysis and Recognition; Springer; 2018:546-52.
14. Guan Q, Huang Y, Zhong Z, Zheng Z, Zheng L, Yang Y. Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification. arXiv preprint arXiv:1801.09927v1 2018.
15. Guan Q, Huang Y, Zhong Z, Zheng Z, Zheng L, Yang Y. Thorax disease classification with attention guided convolutional neural network. Pattern Recognition Letters 2020;131:38-45. [Crossref]
16. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A. Object Detectors Emerge in Deep Scene CNNs. Proceedings of the 2015 International Conference on Learning Representations; 2015.
17. Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, Lu L, Yuille AL, Zhou Y. TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv preprint arXiv:2102.04306v1 2021.
18. Valanarasu JMJ, Oza P, Hacihaliloglu I, Patel VM, editors. Medical Transformer: Gated Axial-Attention for Medical Image Segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention; Springer; 2021:36-46.
19. Sitaula C, Hossain MB. Attention-based VGG-16 model for COVID-19 chest X-ray image classification. Appl Intell (Dordr) 2021;51:2850-63. [Crossref] [PubMed]
20. Sitaula C, Aryal S. New bag of deep visual words based features to classify chest x-ray images for COVID-19 diagnosis. Health Inf Sci Syst 2021;9:24. [Crossref] [PubMed]
21. Huang Z, Wang X, Wei Y, Huang L, Shi H, Liu W, Huang TS. CCNet: Criss-Cross Attention for Semantic Segmentation. IEEE Trans Pattern Anal Mach Intell 2020; Epub ahead of print. [Crossref] [PubMed]
22. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems; 2017:30.
23. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv preprint arXiv:2010.11929v2 2020.
24. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2017;60:84-90. [Crossref]
25. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A. editors. Learning Deep Features for Discriminative Localization. Proceedings of the IEEE conference on computer vision and pattern recognition; 2016:2921-9.
26. Lin T, Dollár P, Girshick R, He K, Hariharan B, Belongie S, editors. Feature Pyramid Networks for Object Detection. Proceedings of the IEEE conference on computer vision and pattern recognition; 2017:2117-25.
27. Yao L, Prosky J, Poblenz E, Covington B, Lyman K. Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutions. arXiv preprint arXiv:1803.07703v1 2018.
28. Huang G, Liu Z, Maaten LVD, Weinberger KQ, editors. Densely Connected Convolutional Networks. Proceedings of the IEEE conference on computer vision and pattern recognition; 2017:4700-8.
29. Kingma DP, Ba J. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980v5 2014.
30. Yao L, Poblenz E, Dagunts D, Covington B, Bernard D, Lyman K. Learning to diagnose from scratch by exploiting dependencies among labels. arXiv preprint arXiv:1710.10501v2 2017.
31. Ma C, Wang H, Hoi SCH, editors. Multi-label Thoracic Disease Image Classification with Cross-Attention Networks. arXiv preprint arXiv:2007.10859v1 2020.
32. Zhou B, Li Y, Wang J. A Weakly Supervised Adaptive DenseNet for Classifying Thoracic Diseases and Identifying Abnormalities. arXiv preprint arXiv:1807.01257v2 2018.