A new nomogram for predicting the malignant diagnosis of Breast Imaging Reporting and Data System (BI-RADS) ultrasonography category 4A lesions in women with dense breast tissue in the diagnostic setting

Introduction

The performance of mammography (MG) is markedly lower in women with dense breast tissue (1). The incidence of dense breast tissue is higher in Asians than in White and other races (83.22% vs. 61.37%) (2). Ultrasonography (US) plays an indispensable role in dense breast examinations in both the screening and diagnostic settings (3-5).

A chapter on the US was added to the fourth edition of the Breast Imaging Reporting and Data System (BI-RADS) atlas by the American College of Radiology (ACR) in 2003 (6). BI-RADS category 4 lesions have a high likelihood of cancer diagnosis (2–95%). In the fifth edition of BI-RADS updated in 2013, category 4 was further divided into three subcategories: 4A, 4B, and 4C (7). Biopsy has been the recommended management approach for all 3 subcategories because of the suspicion of malignancy. However, the malignancy rate of category 4A lesions is rather low (2–10%), and the proportion of category 4A lesions was 44.5% amongst category 4-5 lesions (8). The average reported rate of BI-RADS category 4A lesions on the diagnostic US in our center is only 6.7% and is much lower for patients under 35 years old (2.7%) (8). Since most biopsies of category 4A lesions are benign, the exclusive use of biopsy can cause unnecessary health care costs and patient anxiety.

Although the ACR BI-RADS guideline has significantly improved the management of breast abnormalities, radiologists still make the final BI-RADS category assessment by weighing up the individual imaging features intuitively in general clinical practice. As such, an objective approach combining individual imaging features and patient information is required to reduce the number of unnecessary biopsies, especially for low suspicion BI-RADS category 4A lesions.

This study aimed to establish a nomogram, incorporating US and MG features and patient clinical information, to quantitatively predict the individual likelihood of malignancy of BI-RADS US category 4A lesions in women with dense breast tissue in the diagnostic setting. We also aimed to identify patients at lower risk of malignancy and eventually prevent unnecessary biopsies.

Methods

Patients

This study’s training set data was compiled from 418 patients who met the inclusion criteria and were diagnosed and treated at Sun Yat-sen memorial hospital from January 2014 to June 2017. The inclusion criteria included female patients aged 18 or older who underwent whole breast examinations (US and MG examinations) in the diagnostic setting after experiencing symptoms such as palpable abnormalities, breast pain, and nipple discharge. All breasts were assessed as category 4A by radiologists using the same BI-RADS-US 2013 edition criteria and had dense tissue on MG images (breast composition category c or d according to the 2013 edition of the BI-RADS-MG criteria). All patients had a pathological diagnosis by breast biopsy and/or surgical excision. To test the nomograms’ generalizability, an external validation cohort of 97 patients provided by the Second Affiliated Hospital of Guangzhou Medical University between January 2015 and December 2017 was used. Only 1 4A lesion, which was the most likely to be malignant, was selected from 1 patient. All patients were fully informed about the overall variables in the developed nomogram. Ethical approval was provided by Sun Yat-sen Memorial Hospital Ethics Committee (SYSEC-KY-KS-2019-151). As this was a retrospective study, informed consent from the study participants was exempted.

A standardized data form was created to retrieve all relevant information, including demographic data (age, family history of breast cancer, history of benign breast biopsy), clinical examination of US imaging features (lesion size, margin, shape, orientation, lymph nodes, vascularity, echo, posterior features, calcification, and multiple solid lesions), and MG imaging features (i.e., BI-RADS assessment, suspicious mass/calcifications, architectural distortion, and asymmetry).

US and MG

Four dedicated breast radiologists with >5 years experience (RG, FTL, HLW, JSM)in our center performed the breast US examinations on patients using a high-frequency broad bandwidth linear array transducer equipped with a high-resolution US unit (S2000/S1000 and 18L6 transducer with a center frequency of 15 MHz; Siemens Medical Solutions, Erlangen, Germany). Imaging quality-related parameters were set according to the ACR guidelines (7,9). US was performed and interpreted by a single breast radiologist simultaneously according to the 2013 ACR BI-RADS-US criteria. Lesions were assigned as category 4A if the likelihood of malignancy was assessed to be low (2–10%). Such cases usually had a single suspicious mass with features such as no circumscribed margin and irregular shape or nonparallel orientation. Some cases had low suspicious features of axillary lymph nodes (cortical thickness of 2.3–3 mm and short axis length less than 1 cm) (10-12). The BI-RADS-US assignment was independent of the MG findings, which is usual practice in US departments in China. US imaging features of patients from the training and validation cohorts were reviewed by another independent radiologist (YYP). The radiologist remained blinded to the patients’ MG imaging and pathological information.

MG was performed by a technician with 10 years of experience. Each examination was carried out with the Digital MG machine (Planmed Nuance Excel; Planmed, Helsinki, Finland). Two dedicated radiologists, YH and SYS, with >5 years experience in breast imaging diagnosis, reviewed all MG images from the training and validation cohorts according to the 2013 ACR BI-RADS-MG criteria independently, with patients’ US imaging and pathological information unknown beforehand.

According to the second edition of the BI-RADS-US guidelines, patients with category ≥4A lesions are recommended to undergo biopsy. However, the final decision is generally made by patients and referring physicians. We chose to perform a US-guided biopsy and/or surgical excision in our center to obtain the pathological diagnosis. In this study, the US-guided breast biopsy was conducted with a core instrument (with a 14- or 16-gauge needle) or vacuum-assisted biopsy machine (with an 8- or 11-gauge needle). More than 4 tissue samples were obtained and placed in formalin solution, then processed for histopathology by standard procedures. Open breast surgery was suggested for patients with indefinite or malignant histological results. The pathological results were classified as benign or malignant according to the 2013 ACR BI-RADS (7). Atypical ductal/lobular hyperplasia, peripheral duct papilloma and phyllodes tumor results were defined as benign.

Statistical analysis

The training set and validation set results were compared using the χ² test or Fisher’s exact test. Risk factors were identified by univariate analysis based on the χ² test or Fisher’s exact test, and the statistically significant variables (P<0.05) were included in the final models by applying multivariate logistic regression to the training set. A nomogram was formulated based on the multivariate analysis results and by using the rms package in R version 3.6.3. The nomogram was then subjected to the bootstrap resample method of 1,000 times for both the internal validation of the training cohort and the external validation of the validation cohort. Bootstrap resampling for internal validation was performed to reduce the model’s over-fitting bias and obtain more reliable prediction accuracy. External validation was also performed with an independent cohort of 97 patients from the Second Affiliated Hospital of Guangzhou Medical University between 2014 and 2017. The nomogram’s predictive accuracy and discriminative ability were determined with the concordance index (C-index) and calibration curves. The C-index is a generalization of the area under the receiver operating characteristic (ROC) curve (AUC) (13). A larger C-index corresponds to more accuracy in prediction. Calibration curves were plotted to assess the calibration of the nomogram. The risk score for each patient was calculated from the regression coefficients, and the numerical variables of risk scores were divided into low- or high-risk groups according to the threshold determined by the maximal Youden index for predicting the malignancy rates. Then, we divided patients into a high-risk group or low-risk group according to the cutoff point. Histological type, molecular subtype, and the ki67 status of the malignant lesions were compared between the low-risk and high-risk groups using the χ² test or Fisher’s exact test. The statistical analyses and modeling were performed using STATA (version 13; Stata Co., College Station, TX) and R software packages. All statistical tests were two-sided, with statistical significance defined as P<0.05.

Results

Study population

The training group comprised 418 female patients (median age, 44 years; range, 19–84 years) with BI-RADS-US category 4A lesions and dense breasts treated at Sun Yat-sen memorial hospital. Among them, 42 patients (10%) had malignant histology, while 376 patients (90%) had benign histology. An independent validation dataset was composed of 97 female patients (median age, 43 years; range, 24–64 years) diagnosed with BI-RADS-US category 4A lesions and dense breasts at the Second Affiliated Hospital of Guangzhou Medical University. A total of 10 patients (10.3%) presented with malignant histology, while 87 patients (89.7%) had benign histology in the validation population. Demographic data and MG and US imaging features of patients in the training and validation cohorts are listed in Table 1. There were no differences in the demographic and imaging features between the training data and validation data.

Table 1 Demographic and imaging features of patients in the training and validation cohorts
Full table

Analysis of histological findings in the training set

In the training set, risk factors were determined by univariate analysis with the χ² test or Fisher’s exact test for different demographic and imaging features. These variables comprised age, family history of breast cancer, history of benign breast biopsy, clinical examination of the mammary gland, US imaging features (lesion size, margin, shape, orientation, lymph nodes, vascularity, echo, posterior features, calcification, and multiple solid lesions), and MG imaging features (BI-RADS assessment, mass, calcifications, architectural distortion, and asymmetry) (Table 2). Among the malignant patients, the rate of family history of breast cancer was much higher than that of benign patients (19.0% vs. 4.5%, P=0.001). In the malignant patients, the lesion margin was predominantly not circumscribed (81.0% vs. 39.1%, P<0.001), the shapes of lesions were significantly more irregular (61.9% vs. 33.0%, P<0.001), and the orientation was significantly nonparallel (52.4% vs. 22.9%, P<0.001). Moreover, the rate of low suspicious lymph nodes was higher in malignant patients (7.1% vs. 0.8%, P=0.009). Nine lesions (21.4%) in the 42 malignant patients were found to be suspicious for MG calcifications, while in benign patients, the rate was 2.1% (P<0.001).

Table 2 Analysis of the histological findings in training set in US 4A patients
Full table

Development of the prediction model

The final univariate and multivariate logistic regression model variables were composed of family history of breast cancer, US imaging features (i.e., margin, shape, orientation, lymph nodes), and calcifications on MG imaging (Table 3). A nomogram incorporating all significant predictive factors was established (Figure 1). The nomogram identified low suspicious lymph nodes on US and suspicious calcifications on MG as the most significant factors contributing to prediction, followed by a family history of breast cancer. Other US imaging features (i.e., margin, shape, orientation) had a moderate effect on prediction.

Table 3 Univariate and multivariate analyses of the training set for histological findings
Full table

Figure 1 Prognostic nomogram for predicting patients’ histological findings with Breast Imaging Reporting and Data System (BI-RADS) ultrasonography category 4A lesions in dense breast tissue. The prediction model was developed with the training dataset, which incorporated the family history of breast cancer, ultrasonography imaging features (shape, margin, orientation, and low suspicious lymph nodes), suspicious calcifications on MG imaging, and presented as a nomogram. The sum of the scores is located on the Total Points axis and corresponds to the malignancy risk of 4A lesions.

Validation and calibration of the nomogram

In the primary dataset, Harrell’s C-index produced by the prediction model was 0.813 (95% CI, 0.733 to 0.893) for the established nomogram. In the validation set, the nomogram’s C-index for the prediction of malignant lesions was 0.765 (95% CI, 0.584 to 0.946). The calibration plots presented excellent agreement in the training cohort and good agreement in the validation cohort between the nomogram prediction and the real observation made for the histological findings (Figure 2A,B).

Figure 2 Calibration plots for histological findings between the nomogram prediction model and real observations in the (A) training cohort and (B) validation cohort. The calibration plots presented excellent agreement in the training cohort and good agreement in the validation cohort between the nomogram prediction and real observations made for the histological findings. The Hosmer and Lemeshow Test were 0.578 for the training cohort and 0.789 for the validation cohort.

Performance of the nomogram in stratifying the risk of patients

We assigned risk scores for each risk factor based on the regression coefficients from the final model that represented the histological findings’ prediction (Table 4). The overall risk scores of all patients ranged between 0–304, and we chose 100 as the cutoff score with the maximal Youden index for predicting the malignancy rates. Then, patients were divided into a high-risk group and a low-risk group according to the cutoff point, and the AUC was 0.744 (95% CI, 0.664 to 0.824). In the low-risk group, 95.4% had benign pathology (395/414), while only 4.6% (19/414) had malignant pathology. 67.3% (68/101) had benign pathology in the high-risk group, and 32.7% (33/101) had malignant pathology.

Table 4 Point assignment and prediction scores for histological findings
Full table

In total, 52 breast lesions were malignant in the training and validation cohorts, and 50 of the 52 breast lesions underwent immunohistochemical testing. Histological type, molecular subtype, and the ki67 status of malignant lesions were compared between the low-risk and high-risk groups. There were no significant differences between the low-risk and high-risk groups in these clinicopathological characteristics (Table S1).

Discussion

Higher breast density reduces the sensitivity of MG and affects the results (14,15). Moreover, women with denser tissue, not fattier tissue, have a higher risk of breast cancer (16). The proportion of dense breast lesions in China is much higher than in any other country (17). The US has become an indispensable breast examination method for patients with dense breast lesions. In our center, patients with dense breasts undergo both MG and US examinations. The ACR BI-RADS guideline recommends biopsy for patients with category 4 lesions, causing unnecessary health care costs and patient anxiety, despite the very low malignancy rate of category 4A lesions (2–10%). Here, we developed and validated a nomogram based on demographic data and clinical examination images for predicting the malignant diagnosis of BI-RADS US category 4A lesions in women with dense breast tissue.

Patients with dense breast tissue and US BI-RADS 4A lesions are difficult to manage, and risk stratification is also challenging. For these cases, our nomogram incorporates 6 factors, composed of a family history of breast cancer, 4 US imaging features (margin, shape, orientation, lymph nodes), and calcifications on MG imaging. Harrell’s C-index was used to predict the malignancy rate, which was 0.813 (95% CI, 0.733 to 0.893) in the training set and 0.765 (95% CI, 0.584 to 0.946) in the validation set.

Our nomogram score’s cutoff point was 100, which divided the US of category 4A patients into two groups: high risk and low risk. In the low-risk group, 95.4% (395/414) had benign pathology, and 4.6% had malignant pathology (19/414), while in the high-risk group, 67.3% (68/101) had benign pathology and 32.7% (33/101) had malignant pathology. The prediction of our nomogram was more accurate than the ACR BI-RADS category 4A lesions and other studies. In terms of the strain elastography features for BI-RADS 4A lesions, the malignancy rate was found to be 4.5% (4/88) for soft elastography and 12.6% (21/167) for not soft elastography (18). Menezes et al. used optoacoustic US to help downgrade benign masses classified as BI-RADS 4with the US, with 11.1% (3/27) BI-RADS 4b being correctly downgraded to BI-RADS 3 or 2, and 28.6% (2/7) malignant masses classified as BI-RASD 4a being incorrectly downgraded (19). Our nomogram of the low-risk group was able to help 95.4% of patients avoid a biopsy, based only on a few essential factors, i.e., family history of breast cancer, US imaging features, and calcifications on MG imaging. A study by Patterson et al. showed that breast cancer risk was higher in women aged >40 with BI-RADS 4A US lesions; however, this was not observed in our study (20).

In the multivariate analyses, 6 variables were found to be independent predictors for malignant diagnosis, including a family history of breast cancer, US imaging features (i.e., margin, shape, orientation, lymph nodes), and calcifications on MG imaging. Benndorf et al. reported that breast cancer’s family history was an independent risk factor (odds ratio: 5.53) in the setting of an MG BI-RADS 3 assessment (21). Other previous studies have shown that the US features that most likely predict a malignant diagnosis are irregular shapes, nonparallel orientations, and no circumscribed margin (22-25). Our results were consistent with these findings, suggesting that other US features, such as internal echo, posterior features, calcification, and vascularity, cannot be used as significant predictors for malignancy (24,25). The US is a useful imaging modality to evaluate axillary nodes using morphological criteria, such as cortical thickening, hilar effacement, and nonhilar cortical blood flow. It can be highly specific in the identification of metastases (12). Although our data indicated that the rate of lymph node metastasis gradually increased as the BI-RADS US category steadily increased, 12.3% (9/73) of patients with a category 4A malignant lesion also presented a positive axillary status (8). In this study, the US lymph node status had the most important contribution in predicting malignancy in the nomogram, followed by suspicious calcifications on MG. Other MG features did not show distinct predictive value in this study. Our study, focused on women with dense breast tissue, showed that the US’s role in dense breast tissue is more important, and noncalcified lesions can be obscured by normal dense tissue, which has also been reported in previous studies (3,4,26).

According to the second edition of the BI-RADS-US guidelines, patients with category ≥4A lesions are suggested to undergo biopsy. However, physicians in China may give other medical advice based on their own experience after clinical consultations and physical and imaging examinations. The rate of biopsy for BI-RADS-US 4A lesions may be less than 70–80%. Our nomogram provides a convenient and feasible tool to predict the malignancy risk of patients. For the high-risk group of patients, active biopsy or surgery is recommended, while for patients in the low-risk group who are not willing to undergo biopsy, intensive follow-up can be suggested.

There are several primary limitations in this study. First, this was a retrospective study design which is limited by the deficient collection of risk factors related to breast cancer in patients, such as menstruation, fertility, and smoking, amongst others. Second, we did not have the patients’ genetic mutation (BRCA 1/2, etc.) data. This information is relevant to the dense breast tissue population and can indicate a higher likelihood of malignancy. Lastly, our nomogram did not include MRI imaging because the examination is expensive and inaccessible. The use of MRI is optional in patients with dense breast tissue, but its use is not universally recommended. Moreover, the determination of BI-RADS-US category 4A lesions is an uncertain process dependent on the institution and radiologist training. Therefore, cohorts of 4A patients may look different between different institutions/countries.

In future studies, we will carry out prospective multicenter radiomics investigations combining the risk factors associated with breast cancer in patients with radiomics data to establish a more sensitive and specific breast cancer prediction model.

In conclusion, we have developed a well-discriminated and calibrated nomogram which provides an individual prediction of malignancy of BI-RADS US category 4A lesions in dense breast tissue based entirely upon the family history of breast cancer, US imaging features (i.e., margin, shape, and orientation), low suspicious lymph nodes on the US, and suspicious calcifications on MG. Our nomogram may help clinicians identify patients with a lower risk of malignancy, thus preventing unnecessary and expensive biopsies.

Acknowledgments

Funding: This study is jointly funded by the National Key R&D Program of China (Grant No. 2017YFC1309103, 2017YFC1309104), Natural Science Foundation of China (Grant No. 82003311, 82072907, 81672594, 81772836, and 81872139), National Science Foundation of Guangdong Province (Grant No. 2014A030306003), Sun Yat-sen Memorial Hospital cultivation project for clinical research (Grant No. SYS-C-201805 and SYS-Q-202004), Sun Yat-sen Memorial Hospital Yat-Sen scientific research launch project (Grant No. YXQH201920), Medical Science and Technology Research Fund of Guangdong Province (No. A2020391), Clinical Innovation Research Program of Guangzhou Regenerative Medicine and Health Guangdong Laboratory (Grant No. 2018GZR0201004), Project of The Beijing Xisike Clinical Oncology Research Foundation (Y-Roche2019/2-0078), Key Projects of The National Natural Science Foundation of China (51861125203) and Guangzhou Science and Technology key projects (201804020076).

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/qims-20-1203). The authors have no conflicts of interest to declare.

Ethical Statement: Ethical approval was provided by Sun Yat-sen Memorial Hospital Ethics Committee (SYSEC-KY-KS-2019-151). As this was a retrospective study, informed consent from the study participants was exempted.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Mandelson MT, Oestreicher N, Porter PL, White D, Finder CA, Taplin SH, White E. Breast density as a predictor of mammographic detection: comparison of interval- and screen-detected cancers. J Natl Cancer Inst 2000;92:1081-7. [Crossref] [PubMed]
2. del Carmen MG, Halpern EF, Kopans DB, Moy B, Moore RH, Goss PE, Hughes KS. Mammographic breast density and race. AJR Am J Roentgenol 2007;188:1147-50. [Crossref] [PubMed]
3. Berg WA, Bandos AI, Mendelson EB, Lehrer D, Jong RA, Pisano ED. Ultrasound as the Primary Screening Test for Breast Cancer: Analysis From ACRIN 6666. J Natl Cancer Inst 2015;108:djv367 [Crossref] [PubMed]
4. Ohuchi N, Suzuki A, Sobue T, Kawai M, Yamamoto S, Zheng YF, Shiono YN, Saito H, Kuriyama S, Tohno E, Endo T, Fukao A, Tsuji I, Yamaguchi T, Ohashi Y, Fukuda M, Ishida T. Sensitivity and specificity of mammography and adjunctive ultrasonography to screen for breast cancer in the Japan Strategic Anti-cancer Randomized Trial (J-START): a randomised controlled trial. Lancet 2016;387:341-8. [Crossref] [PubMed]
5. Shen S, Zhou Y, Xu Y, Zhang B, Duan X, Huang R, Li B, Shi Y, Shao Z, Liao H, Jiang J, Shen N, Zhang J, Yu C, Jiang H, Li S, Han S, Ma J, Sun Q. A multi-centre randomised trial comparing ultrasound vs mammography for screening breast cancer in high-risk Chinese women. Br J Cancer 2015;112:998-1004. [Crossref] [PubMed]
6. Mendelson E, Baum J, Berg W, Merritt C, Rubin E. Breast imaging reporting and data system, BI-RADS: ultrasound. Reston VA: American College of Radiology 2003.
7. Sickles E, D’Orsi C, Bassett L, Appleton C, Berg W, Burnside E. ACR BI-RADS® Atlas Breast Imaging Reporting and Data System. Reston, VA: American College of Radiology 2013.
8. Hu Y, Yang Y, Gu R, Jin L, Shen S, Liu F, Wang H, Mei J, Jiang X, Liu Q, Su F. Does patient age affect the PPV3 of ACR BI-RADS Ultrasound categories 4 and 5 in the diagnostic setting? Eur Radiol 2018;28:2492-8. [Crossref] [PubMed]
9. American College of Radiology. ACR practice parameter for theperformance of a breast ultrasound examination. Available online: https://www.acr.org/-/media/ACR/Files/Practice-Parameters/USBreast.pdf?la=en. Res.38-2016. Accessed Oct 20, 2020.
10. Lee B, Lim AK, Krell J, Satchithananda K, Coombes RC, Lewis JS, Stebbing J. The efficacy of axillary ultrasound in the detection of nodal metastasis in breast cancer. AJR Am J Roentgenol 2013;200:W314-20 [Crossref] [PubMed]
11. Deurloo EE, Tanis PJ, Gilhuijs KG, Muller SH, Kröger R, Peterse JL, Rutgers EJ, Valdés Olmos R, Schultze Kool LJ. Reduction in the number of sentinel lymph node procedures by preoperative ultrasonography of the axilla in breast cancer. Eur J Cancer 2003;39:1068-73. [Crossref] [PubMed]
12. Ecanow JS, Abe H, Newstead GM, Ecanow DB, Jeske JM. Axillary staging of breast cancer: what the radiologist should know. Radiographics 2013;33:1589-612. [Crossref] [PubMed]
13. Harrell FE, Jr. Regression Modeling Strategies. With Applications to Linear Models, Logistic Regression, and Survival Analysis. Springer -Verlag New York; 2001.
14. Philpotts LE. Density Variation among Mammographic Modalities Will Likely Impact Imaging Management and Risk Stratification. Radiology 2019;291:328-9. [Crossref] [PubMed]
15. Evans DG, van Veen EM, Howell A, Astley S. Heritability of mammographic breast density. Quant Imaging Med Surg 2020;10:2387-91. [Crossref] [PubMed]
16. Boyd NF, Rommens JM, Vogt K, Lee V, Hopper JL, Yaffe MJ, Paterson AD. Mammographic breast density as an intermediate phenotype for breast cancer. Lancet Oncol 2005;6:798-808. [Crossref] [PubMed]
17. Yang Y, Liu J, Gu R, Hu Y, Liu F, Yun M, Xiao Q, Wu M, Liu Q, Su F. Influence of factors on mammographic density in premenopausal Chinese women. Eur J Cancer Prev 2016;25:306-11. [Crossref] [PubMed]
18. Koh J, Kim EK, Kim MJ, Yoon JH, Park VY, Moon HJ. Role of elastography for downgrading BI-RADS category 4a breast lesions according to risk factors. Acta Radiol 2019;60:278-85. [Crossref] [PubMed]
19. Menezes GLG, Pijnappel RM, Meeuwis C, Bisschops R, Veltman J, Lavin PT, van de Vijver MJ, Mann RM. Downgrading of Breast Masses Suspicious for Cancer by Using Optoacoustic Breast Imaging. Radiology 2018;288:355-65. [Crossref] [PubMed]
20. Patterson SK, Neal CH, Jeffries DO, Joe A, Klein K, Bailey J, Pinsky R, Paramagul C, Watcharotone K. Outcomes of solid palpable masses assessed as BI-RADS 3 or 4A: a retrospective review. Breast Cancer Res Treat 2014;147:311-6. [Crossref] [PubMed]
21. Benndorf M, Wu Y, Burnside ES. A history of breast cancer and older age allow risk stratification of mammographic BI-RADS 3 ratings in the diagnostic setting. Clin Imaging 2016;40:200-4. [Crossref] [PubMed]
22. Rahbar G, Sie AC, Hansen GC, Prince JS, Melany ML, Reynolds HE, Jackson VP, Sayre JW, Bassett LW. Benign versus malignant solid breast masses: US differentiation. Radiology 1999;213:889-94. [Crossref] [PubMed]
23. Hong AS, Rosen EL, Soo MS, Baker JA. BI-RADS for sonography: positive and negative predictive values of sonographic features. AJR Am J Roentgenol 2005;184:1260-5. [Crossref] [PubMed]
24. Bouzghar G, Levenback BJ, Sultan LR, Venkatesh SS, Cwanger A, Conant EF, Sehgal CM. Bayesian probability of malignancy with BI-RADS sonographic features. J Ultrasound Med 2014;33:641-8. [Crossref] [PubMed]
25. Elverici E, Barca AN, Aktas H, Ozsoy A, Zengin B, Cavusoglu M, Araz L. Nonpalpable BI-RADS 4 breast lesions: sonographic findings and pathology correlation. Diagn Interv Radiol 2015;21:189-94. [Crossref] [PubMed]
26. Nelson HD, Pappas M, Cantor A, Griffin J, Daeges M, Humphrey L. Harms of Breast Cancer Screening: Systematic Review to Update the 2009 US Preventive Services Task Force Recommendation. Ann Intern Med 2016;164:256-67. [Crossref] [PubMed]