Application of decision tree algorithms to predict central lymph node metastasis in well-differentiated papillary thyroid carcinoma based on multimodal ultrasound parameters: a retrospective study

Introduction

According to the latest Global Cancer Incidence, Mortality and Prevalence (GLOBOCAN) statistics provided by the International Agency for Research in 2020, it is estimated that 586,202 people were newly diagnosed with thyroid cancer, accounting for 3.0% of all new cancer cases (1). Papillary thyroid carcinoma (PTC) is the most typical pathological type of thyroid cancer. Although more than 90% of PTCs are well-differentiated tumors, central compartment lymph node metastasis (CLNM) can still occur in the early stage, which is closely related to local recurrence (2,3). The role of prophylactic central neck dissection (pCND) in patients with well-differentiated PTCs has become controversial in recent years (4-6). Unnecessary pCND may increase the probability of postoperative hypoparathyroidism and recurrent laryngeal nerve injury, while well-differentiated PTCs have limited recurrence incidence. Accurately assessing the risk of CLNM by preoperative imaging examination is vital to avoid the excessive physical and mental trauma caused by overdiagnosis and treatment (7).

Ultrasound (US) is a safe, convenient, and cost-effective examination method. Even though high-resolution probes have been widely used to diagnose abnormal cervical lymph nodes and have achieved good diagnostic efficiency, there are still limitations in distinguishing CLNM from non-CLNM due to the deep anatomical position and the complex surrounding relationship. A previous meta-analysis showed that the sensitivity of conventional US for direct diagnosis of CLNM was only 38–64%, which is much lower than that of lateral compartment cervical lymph node metastasis (LLNM) (8). Advances in US technology, including contrast-enhanced ultrasound (CEUS), shear wave elastography (SWE), and superb microvascular imaging (SMI) for tumors have immensely improved its diagnostic efficiency. Several studies have shown that the SWE and CEUS characteristics of PTCs are closely related to the extrathyroidal extension (ETE) and lymph node metastasis of PTCs, suggesting that multimodal US may have great potential in predicting CLNM (9-13).

There remains an existing need for improved CLNM prediction performance. Current studies have established a prediction model using a regression algorithm and nomogram, making it possible to forecast CLNM risk in clinical application (14-16). Machine learning (ML) algorithms have also been widely used in image diagnosis over the last few years. Prior limited studies have shown that ML models can predict CLNM based on clinical information and conventional US features (17,18). However, the applicability of existing models in this field is restricted due to a lack of multimodal US characteristics of PTCs. The present study aimed to predict CLNM more accurately than the currently performed conventional US. Therefore, we developed a prediction model for CLNM by integrating the clinical and multimodal US features of PTCs as model input parameters and using a decision tree (DT) as a classification method. We present the following article in accordance with the TRIPOD reporting checklist (available at https://qims.amegroups.com/article/view/10.21037/qims-22-650/rc).

Methods

Study population

Patients who underwent multimodal US examination of thyroid nodules at the Beijing Tiantan Hospital, Capital Medical University from January 2020 to September 2021 were analyzed retrospectively. Consecutive patients satisfying the inclusion and exclusion criteria were collected. The inclusion criteria are as follows: (I) adults over 18 years old, (II) patients who were examined preoperatively through multimodal US examination [if the patient had multiple nodules on 1 side, only 1 of the nodules with the highest Thyroid Imaging Reporting & Data System (TI-RADS) level was included], and (III) patients who underwent thyroidectomy and central compartment lymph node dissection (area VI) in our hospital. After the operation, we determined whether the pathological diagnosis was well-differentiated PTC (pathological type was classic or follicular subtype PTC) and whether the central lymph nodes were metastatic. The exclusion criteria were as follows: (I) patients lacking important information, (II) patients with other head or neck malignancies, and (III) patients who had previously undergone thyroidectomy.

The scope of thyroid surgery was mainly determined according to the 2015 management guidelines of the American Thyroid Association (ATA) (5). This study was reviewed and approved by the ethics committee of Beijing Tiantan Hospital, Capital Medical University, and was performed in accordance with the Declaration of Helsinki (as revised in 2013). Written informed consent was obtained from all participants of this study. Among the 852 thyroid nodules examined using multimodal US preoperatively, 173 nodules received thyroidectomy in our hospital, 163 nodules were pathologically confirmed as well-differentiated PTCs, 10 pathologically confirmed as other types of thyroid cancer or benign nodules were excluded, and 15 were excluded due to having higher TI-RADS grade nodules on the same side or a lack of pivotal data. Finally, 148 PTCs from 142 patients were included in this retrospective study (Figure 1). All cases had undergone pCND, and CLNM was confirmed by pathology.

Figure 1 Flow diagram of patient selection. PTC, papillary thyroid carcinoma.

Clinical characteristics

The baseline clinical characteristics, including age, gender, body weight, body mass index (BMI), thyroid-stimulating hormone (TSH), thyroglobulin (Tg), thyroglobulin antibody (TgAb), and thyroid peroxidase antibody (TPOAb) were collected from the patients’ medical records within 1 month before surgery. The tumor number, location, ETE, and Hashimoto thyroiditis (HT) status were confirmed using the pathological results.

Multimodal ultrasonographical features

An Aplio i900 (Toshiba, Tokyo, Japan) equipped with a linear array i18LX5 probe at a transmitting frequency of 5–18 MHz was used for all US examinations. All patients were in a supine position with head flexion backward. US examination was performed by examiners with more than 10 years of working experience. During the procedure, we recorded all of the raw data in real time. The use of precision imaging, ApliPure composite imaging, and second-generation differential harmonic imaging technologies improved the image quality and penetration depth while reducing artifacts and speckle noise. The machine settings for multimodal US examination are summarized in Table 1.

Conventional US is considered to include gray-scale US and color Doppler flow imaging (CDFI). Three diameter lines (longitudinal, transverse, and thick) of the thyroid nodules were measured on gray-scale US, and the following characteristics were evaluated: shape (regular, irregular), margin (well defined, poorly defined, and unable to define; nodule margins covered by macrocalcification with acoustic shadows were classified as unable to define), echogenicity (isoechoic, hyperechoic, or hypoechoic), type of calcification (absent, macrocalcification, or microcalcification; microcalcification was defined as strong echo spots less than 1 mm in the nodule), microcalcification quantities (less than 5, greater than or equal to 5), peripheral halo (absent, present), taller-than-wide (anteroposterior/transverse ratio >1 or ≤1), contact capsule (absent, present, and whether the nodules contact the hyperechoic line of the capsule; Figure 2), and diffused background (absent, present).

Figure 2 A 24-year-old female with postoperative pathology confirmed papillary thyroid carcinoma in the right lobe with central compartment lymph node metastasis. (A) Gray-scale ultrasound showed that the nodule was in contact with the thyroid capsule (white arrow heads). (B) Contrast-enhanced ultrasound showed diffuse isoenhancement 10 seconds after injection. (C) Hematoxylin and eosin-stained thyroidectomy of the specimen (original magnification ×4; scale bar 600 µm) confirmed that the nodule had extrathyroidal extension.

CDFI grading was divided into 3 levels based on the Alder standard: (I) none, no blood flow inside or around the nodule; (II) moderate, fewer than 5 punctate blood flows or 2 strip blood flows in the nodule; and (III) marked, 5 or more punctate blood flows or more than 2 main vessels in the nodule (19). The characteristics of abnormal cervical lymph nodes on conventional US (ACLN) and abnormal central lymph nodes on conventional US (ACCLN) included microcalcifications, cystic aspect, hyperechogenicity, and round shape or peripheral vascularity, according to the 2015 ATA guidelines (5). The color SMI and monochrome SMI were employed to assess the vascular distribution of the nodules on the section with the most abundant blood flow. The SMI blood flow estimation standard was similar to that of CDFI (Figure 3). The SMI vascular index was obtained by calculating the percentage of color pixels divided by the total pixels in the region of interest.

Figure 3 Superb microvascular imaging grading based on Adler classification. (A) None, absent microvascular flow inside the nodule. (B) Moderate, fewer than 5 punctate blood flows or 2 strip blood flows in the nodule. (C) Marked, 5 or more punctate blood flows or more than 2 main vessels in the nodule.

CEUS was conducted using the same equipment as that used in the conventional US. The mechanical index was set to 0.08–0.10. SonoVue (Bracco Imaging, Milan, Italy) was mixed well with 5 mL of saline as the contrast agent of CEUS. After this, 1.6–1.8 mL of the suspension was rapidly injected into the patient’s vena intermedia cubiti, which was followed immediately by 5.0 mL of 0.9% saline. The CEUS imaging lasted 2 minutes and was stored in the device digitally for analysis. Next, the time-intensity curve was obtained for nodules and the surrounding parenchyma. Quantitative parameters were automatically calculated, including the peak intensity, time to peak, mean transit time, and area under the time-intensity curve. Meanwhile, the examiners evaluated the qualitative parameters, including the enhancement homogeneity (homogeneous or heterogeneous), enhancement direction (centripetal or noncentripetal), enhancement intensity (mostly no enhancement, hypoenhancement, isoenhancement, hyperenhancement), peripheral enhancement (absent, present), and enhancement type (wash-in and wash-out order, which are classified as earlier and synchronous or later).

Shear wave elasticity imaging was applied to evaluate tumor stiffness. After the waveform was stable, 3 measurements of shear wave velocity (SWV) were taken by placing a region of interest inside the nodule and in the surrounding parenchyma. Soon afterward, the average SWV (SWV mean) of the nodules and the SWV ratio (SWV ratio = SWV of the nodules/surrounding parenchyma) were calculated.

Statistical analysis

Normally distributed data are expressed as the mean ± standard deviation and analyzed with a t-test, while nonnormally distributed data are shown as the median with interquartile range (IQR) and compared with the Mann-Whitney test. The χ² test or Fisher exact probability method was used to compare categorical variables, while Pearson chi-squared test was applied for stratified or matched categorical data. Based on previous studies and the relevant formula, the sample size required was calculated to be 144. Finally, all available data on the database were used to maximize the power and generalizability of the results (20).

The chi-squared automatic interaction detector (CHAID) growth method was used to build the prediction model based on conventional and multimodal US individually. The CHAID algorithm selects the independent variable and displays them as “parent nodes” in the tree structure. Parent nodes represent the choices that will result in the splitting of samples. Therefore, a “branch” on the tree is formed. At end of the branch, no further split can be made, a “terminal node” is created, and these samples are classified as CLNM or non-CLNM (21). The maximum depth was set to 3, and the minimum number of cases of the parent and child nodes were 10 and 5, respectively. The significance level of the CHAID DT node splitting was P<0.05. Ten-fold cross-validation was used to verify the misjudgment rate of the model.

R software (version 4.1.2, The Foundation for Statistical Computing, Vienna, Austria) with the “caret” package was used to build a predictive model based on the random forest (RF) algorithm. Receiver operating characteristic (ROC) and precision-recall curves were used to manifest the diagnostic capability between the models. The diagnostic efficacy of the models was compared by calculating the sensitivity, specificity, positive predictive value, negative predictive value, accuracy, and the area under the ROC curve (AUROC) with 95% confidence intervals (CIs) of the different models. All statistical tests were performed using SPSS software (version 26.0, IBM Corporation, Armonk, NY, USA). For all tests, P<0.05 was considered statistically different.

Results

Demographic characteristics

A total of 142 patients were enrolled in this study, including 31 (21.8%) males and 111 (78.2%) females, aged from 19 to 72 years, with an average age of 42.2 years. The median body weight and BMI of all patients were 65.0 (IQR: 56.0–75.0) and 23.9 (IQR: 21.5–27.6), respectively. Among the 142 patients, 94 (66.2%) had a single PTC and 48 (33.8%) had multiple PTCs (Table 2).

Analysis of the clinical and multimodal ultrasonic parameters

Among the 148 nodules that met the inclusion requirements, 75 (50.7%) had accompanying CLNM status (as shown in Table 3). There were significant differences in age, ETE, ipsilateral gland with HT, and Tg levels between the CLNM and non-CLNM groups (P<0.05). In contrast, there were no significant differences in gender, body weight, BMI, tumor location, tumor multifocality, TSH, TgAb, and TPOAb levels (P>0.05). The correlation between multimodal ultrasonic characteristics and CLNM is shown in Table 4. Univariate analysis indicated that CLNM was positively correlated with maximum size, taller-than-wide, the number of microcalcifications greater than or equal to 5, capsule contact, ACLN, ACCLN, noncentripetal perfusion, delayed clearance on CEUS, higher SWV mean, and SWV ratio on SWE (P<0.05).

Predictive performance of the DT models

A total of 33 preoperative clinical features and US parameters of PTC nodules were used to determine the correlation with CLNM. Finally, clinical characteristics (including age and Tg level), 6 conventional US features (including tumor size, taller-than-wide, the number of microcalcifications, capsule contact, ACLN, and ACCLN), and 4 multimodal US features (including centripetal enhancement, delayed clearance, SWV mean, and SWV ratio) showed a significant correlation with CLNM (as shown in Tables 3,4). Three conventional US parameters and four multimodal US parameters were identified as independent variables to train the conventional US and multimodal US prediction models (Figures 4,5). The independent variables of conventional US DT were ACLN (adj. P<0.001), taller-than-wide (adj. P=0.001), and contact with capsule (adj. P=0.046). The conventional US DT model included a total of 6 nodes with 4 terminal nodes (number 1, 4, 5, and 6). Meanwhile, the independent variables of multimodal US DT were ACLN (adj. P<0.001), centripetal perfusion (adj. P=0.001), contact with capsule (adj. P<0.001), and taller-than-wide (adj. P=0.004). The multimodal US DT model included a total of 8 nodes with 5 terminal nodes (number 1, 5, 6, 7, and 8). Other variables that did not reach the adjusted significance level of 0.05 were not included in the models. The 10-fold cross-validation results showed that the misjudgment rates of the multimodal and conventional US models were 28.4% and 31.8%, respectively. Taking postoperative pathology as the gold standard, the AUROC of multimodal US DT in predicting CLNM was 0.837 (95% CI: 0.771–0.902), the sensitivity was 80.0%, the specificity was 76.7%, and the accuracy was 78.4%; the AUROC of CLNM predicted by conventional US DT was 0.773 (95% CI: 0.696–0.849), the sensitivity was 73.3%, the specificity was 74.0%, and the accuracy was 73.6%; the AUROC of CLNM predicted by ACLN was 0.666 (95% CI: 0.578–0.754), the sensitivity was 40.0%, the specificity was 93.2%, and the accuracy was 66.2%. The ROC and precision-recall curves were analyzed to compare the effectiveness of these models (Figure 6). The AUROC of the multimodal US DT was remarkably greater than that of the conventional US DT and ACLN (P=0.009; P<0.001), whose prediction efficiency was the best among the 3 models (Table 5). To determine the final model for predicting CLNM, Table S1 and Figure S1 provided a performance comparison of the multimodal RF and DT models; the multimodal DT algorithm had a higher AUROC compared to the multimodal RF algorithm (P=0.017).

Figure 4 Decision tree obtained using parameters from the gray-scale ultrasound. Each node contains 3 statistical values (category, %, n): “n” stands for the CLNM or non-CLNM population in this category, while “%” is the percentage of the CLNM or non-CLNM population. CLNM, central compartment lymph node metastasis; ACLN, abnormal cervical lymph node on conventional ultrasound; adj. P value, adjusted P value; df, degree of freedom.

Figure 5 Decision tree obtained using parameters from the multimodal ultrasound. Each node contains 3 statistical values (category, %, n): “n” stands for the CLNM or non-CLNM population in this category, while “%” is the percentage of the CLNM or non-CLNM population. CLNM, central compartment lymph node metastasis; ACLN, abnormal cervical lymph node on conventional ultrasound; adj. P value, adjusted P value; df, degree of freedom.

Figure 6 Evaluation of the predictive models. (A) The left picture shows the receiver operating characteristic curve analysis for predicting central compartment lymph node metastasis. The area under the curve for the multimodal US decision tree was the largest among the 3 models. (B) The right picture shows the precision-recall curves of the 3 models. US, ultrasound; ACLN, abnormal cervical lymph node on conventional ultrasound.

Discussion

Traditional methods to evaluate CLNM

The central lymph node is the first stop of thyroid lymphatic drainage. The methods of preoperative diagnosis of CLNM include US, computed tomography (CT), magnetic resonance imaging (MRI), and fine needle aspiration (FNA), among others. According to the ATA guidelines, preoperative US and US-guided FNA are recommended for assessing lymph node involvement in PTC. It has been reported that dual-energy CT reliably distinguishes metastatic and nonmetastatic lymph nodes smaller than 0.5 cm in patients with PTC (22). However, CT scans are expensive and may present a radiation risk, and contrast-enhanced CT cannot be applied for severe hepatic or renal insufficiency. MRI may be an alternative or complementary imaging modality for the detection of CLNM in PTC in addition to US and CT, but its sensitivity is limited (23). F-18 fluorodeoxyglucose positron emission tomography (F-18 FDG PET)-CT is sensitive to neck or mediastinal involvement and may reveal distant metastasis. It has been proven useful in the detection of cervical lymph node metastasis by visualizing abnormal glucose metabolism in the abnormal cells, showing great potential in patients with negative US and CT scans (24). Similar to CT, the radiation risk of F-18 FDG PET-CT is unfavorable compared to US; meanwhile, FNA is an invasive operation. When the abnormal lymph nodes are located in the deep layer of the neck or adjacent to the carotid sheath, the risk of serious complications is increased.

US is a safe, convenient, and cost-effective examination method. However, considering the complicated anatomical structure of the central compartment and the shelter of bone and the strong echo of gas, the diagnostic accuracy of conventional US for CLNM is much lower than that of LLNM. Our study showed that only a minority of CLNM can be directly identified by conventional US. Moreover, some CLNM is too minimal to be found in conventional US. Therefore, researchers have exerted efforts to study the association between US characteristics of PTCs and CLNM and have discovered that it may be a promising strategy to predict CLNM preoperatively to avoid unnecessary pCND (13,25). Previous studies have shown that the most important contributing factors for CLNM include the patient’s age, male gender, multifocality, tumor size, location, margin, microcalcification, ETE, and concomitant HT (14,26,27). In this study, some conventional US characteristics of PTCs were found to be predictors of CLNM, including larger maximum size, taller-than-wide, capsule contact, and ACLN. We also found that a large maximum diameter and capsule contact had a marked positive correlation with ETE and capsule invasion (Table 6). Moreover, large size or contact with the capsule, which reveals aggressive biological behaviors, also increased the likelihood of LLNM (P<0.001; P=0.056; Table 6). Only 7.4–22.5% of PTCs are accompanied by “jump metastasis” (28-30). Thus, ACLN often suggests CLNM-positive status. When ACLN was added as the independent variable of DT, the maximum size was no longer included in the model due to interactions between the variables.

Multimodal ultrasonic parameters predicting CLNM

Microvascular Doppler modalities, CEUS, and SWE combined with grayscale and color Doppler US procedures have made decent progress in tumor diagnosis (31). SMI is very valuable for displaying low-speed blood flow and perforating vessels (32,33). However, in the present study, none of the SMI blood flow grading or the microvascular density of PTC nodules was correlated with CLNM (Table 4). Previous reports have pointed out that the iso- or hyperenhancement CEUS pattern is an independent risk factor for CLNM (9-11). Instead, PTCs presented centripetal enhancement as a protective factor for CLNM in this study, which is different from the findings of previous studies (34,35). It is considered that the blood perfusion of nodules is insufficient presenting centripetal perfusion, which restricts tumor progression. Further research is needed to clarify whether and how centripetal perfusion affects CLNM. Real-time SWE employs an acoustic pulse generated by the probe to measure tissue stiffness, which quantifies the elasticity as SWV or Young’s modulus (36). The finding in this study that a larger SWV mean and SWV ratio are associated with CLNM is similar to that reported in previous literature (P=0.005; P=0.004) (13,37). It may be that the invasive tumor leads to interstitial collagen remodeling, making the nodule stiffer in the CLNM group (38).

In recent years, ML has been widely used in the field of medicine, especially in medical imaging. It takes full advantage of high-throughput imaging features by extracting crucial features from complex clinical contexts to assist in diagnosis and individualized treatment (39). As for the methods to predict the CLNM of thyroid cancer, the AUROC varied between 0.73 to 0.83 in previously reported ML algorithms that used clinical and conventional US features (17,18,40). However, multimodal US information has not been collected in this context. DT is one of the earliest and most commonly used supervised learning classification methods that can directly reflect the data characteristics and militate in favor of clinical interventions. It is a tree-structured classification scheme based on the “if…then” principle (41). The nodes represent the independent variables, while the leaves determine the final classification, which represented the CLNM or non-CLNM in our research.

The best DT model based on multimodal US characteristics that included the ACLN, taller-than-wide, capsule contact, and CEUS centripetal perfusion as independent variables suggested that these above characteristics of well-differentiated PTC contribute to CLNM. These key factors are grouped into 2 classification options to make diagnosis easier. In addition, CEUS is usually performed for suspicious patients; in our study, it was not only used for CLNM prediction but also for PTC diagnosis. CEUS characteristics may be the clue to further improving the accuracy of CLNM prediction in large-scale ML models. The sensitivity and specificity of the multimodal US DT model proposed in our research were more than 70%, and the AUROC was 0.837, indicating that its diagnostic performance was significantly better than that of the conventional US DT and some other risk-scoring models published previously (16,25,27). Unlike other ML models, such as support vector machines and RF, the results of DT can be interpreted very quickly (42). Thus, the multimodal US DT model does not increase the time cost and improves the accuracy of CLNM diagnosis. Making clinical decisions through our proposed model can relieve doctors of a heavy burden and reduce the time of diagnosis.

We also developed a model based on the RF algorithm. RF is a representative ML algorithm that uses bootstrap aggregation, which integrates all of the basic DT voting results (43). It has the advantages of efficient management of large databases and high prediction accuracy (44,45). However, we did not find that the RF algorithm performance was better than that of DT when applied in our study (Table S1, Figure S1). We speculated that the simple data structure (only 12 multimodal features showed a significant difference between the groups) cannot take full advantage of the RF algorithm. The multimodal US DT proposed in this study has good sensitivity and specificity in predicting CLNM, which indicates that the model has good predictive ability not only for identifying CLNM but also for identifying non-CLNM. Thus, our study may help reduce the incidence of pCND.

Limitations

One of the limitations of this study is that patients with multifocal PTCs (n=54) were not excluded, as there was no significant correlation between multifocality and CLNM (P=0.828). Moreover, since the incidence of LLNM was only 13.5% (20/148), we did not analyze the relevance of US characteristics and LLNM. As with any retrospective study, selection bias can affect the data. Additionally, we did not use multimodal US to evaluate the CLNM directly. Further research should be performed to understand the value of multimodal US in detecting central lymph node abnormalities. Finally, this single-center study was a preliminary exploration of DT to predict CLNM. Only the 10-fold crossover method was used for internal verification, and without sufficient external testing, the generalizability of the model is restricted. In the future, research with a larger sample size conducted in multiple centers using various equipment types is required for model establishment and real-world verification. With the rapid growth of big data, establishing artificial intelligence models based on multimodal clinical and radiomic signatures will be critical to improving the prediction precision and clinical relevancy of CLNM (20,46).

Conclusions

The preoperative multimodal US parameter DT prediction model established in this study has reliable predictive ability and good specificity for predicting CLNM in well-differentiated PTCs. As a result, this model may help clinicians identify those patients with a high risk of CLNM to avoid unnecessary pCND.

Acknowledgments

We appreciate the assistance provided by Hangjian Zhong during the preparation of this manuscript.

Funding: This work was supported by the National Natural Science Foundation of China (No. 81901744) and the Natural Science Foundation of Beijing (No. 7204255).

Footnote

Reporting Checklist: The authors have completed the TRIPOD checklist. Available at https://qims.amegroups.com/article/view/10.21037/qims-22-650/rc

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-22-650/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was reviewed and approved by the ethics committee of Beijing Tiantan Hospital, Capital Medical University, and was performed in accordance with the Declaration of Helsinki (as revised in 2013). Written informed consent was obtained from all participants of this study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin 2021;71:209-49. [Crossref] [PubMed]
2. Sakorafas GH, Sampanis D, Safioleas M. Cervical lymph node dissection in papillary thyroid cancer: current trends, persisting controversies, and unclarified uncertainties. Surg Oncol 2010;19:e57-70. [Crossref] [PubMed]
3. Lang BH, Wong CK, Yu HW, Lee KE. Postoperative nomogram for predicting disease-specific death and recurrence in papillary thyroid carcinoma. Head Neck 2016;38:E1256-63. [Crossref] [PubMed]
4. Gyorki DE, Untch B, Tuttle RM, Shaha AR. Prophylactic central neck dissection in differentiated thyroid cancer: an assessment of the evidence. Ann Surg Oncol 2013;20:2285-9. [Crossref] [PubMed]
5. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, Pacini F, Randolph GW, Sawka AM, Schlumberger M, Schuff KG, Sherman SI, Sosa JA, Steward DL, Tuttle RM, Wartofsky L. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 2016;26:1-133. [Crossref] [PubMed]
6. Kim SK, Woo JW, Lee JH, Park I, Choe JH, Kim JH, Kim JS. Prophylactic Central Neck Dissection Might Not Be Necessary in Papillary Thyroid Carcinoma: Analysis of 11,569 Cases from a Single Institution. J Am Coll Surg 2016;222:853-64. [Crossref] [PubMed]
7. Li M, Dal Maso L, Vaccarella S. Global trends in thyroid cancer incidence and the impact of overdiagnosis. Lancet Diabetes Endocrinol 2020;8:468-70. [Crossref] [PubMed]
8. Zhao H, Li H. Meta-analysis of ultrasound for cervical lymph nodes in papillary thyroid cancer: Diagnosis of central and lateral compartment nodal metastases. Eur J Radiol 2019;112:14-21. [Crossref] [PubMed]
9. Zhan J, Zhang LH, Yu Q, Li CL, Chen Y, Wang WP, Ding H. Prediction of cervical lymph node metastasis with contrast-enhanced ultrasound and association between presence of BRAF(V600E) and extrathyroidal extension in papillary thyroid carcinoma. Ther Adv Med Oncol 2020;12:1758835920942367. [Crossref] [PubMed]
10. Zhan J, Diao X, Chen Y, Wang W, Ding H. Predicting cervical lymph node metastasis in patients with papillary thyroid cancer (PTC) – Why contrast-enhanced ultrasound (CEUS) was performed before thyroidectomy. Clin Hemorheol Microcirc 2019;72:61-73. [Crossref] [PubMed]
11. Hong YR, Yan CX, Mo GQ, Luo ZY, Zhang Y, Wang Y, Huang PT. Conventional US, elastography, and contrast enhanced US features of papillary thyroid microcarcinoma predict central compartment lymph node metastases. Sci Rep 2015;5:7748. [Crossref] [PubMed]
12. Xu JM, Xu XH, Xu HX, Zhang YF, Guo LH, Liu LN, Liu C, Bo XW, Qu S, Xing M, Li XL. Prediction of cervical lymph node metastasis in patients with papillary thyroid cancer using combined conventional ultrasound, strain elastography, and acoustic radiation force impulse (ARFI) elastography. Eur Radiol 2016;26:2611-22. [Crossref] [PubMed]
13. Li T, Li H, Xue J, Miao J, Kang C. Shear wave elastography combined with gray-scale ultrasound for predicting central lymph node metastasis of papillary thyroid carcinoma. Surg Oncol 2021;36:1-6. [Crossref] [PubMed]
14. Feng JW, Hong LZ, Wang F, Wu WX, Hu J, Liu SY, Jiang Y, Ye J. A Nomogram Based on Clinical and Ultrasound Characteristics to Predict Central Lymph Node Metastasis of Papillary Thyroid Carcinoma. Front Endocrinol (Lausanne) 2021;12:666315. [Crossref] [PubMed]
15. Li N, He JH, Song C, Yang LC, Zhang HJ, Li ZH. Nomogram Including Elastography for Prediction of Contralateral Central Lymph Node Metastasis in Solitary Papillary Thyroid Carcinoma Preoperatively. Cancer Manag Res 2020;12:10789-97. [Crossref] [PubMed]
16. Huang C, Cong S, Liang T, Feng Z, Gan K, Zhou R, Guo Y, Luo S, Liang K, Wang Q. Development and validation of an ultrasound-based nomogram for preoperative prediction of cervical central lymph node metastasis in papillary thyroid carcinoma. Gland Surg 2020;9:956-67. [Crossref] [PubMed]
17. Zhu J, Zheng J, Li L, Huang R, Ren H, Wang D, Dai Z, Su X. Application of Machine Learning Algorithms to Predict Central Lymph Node Metastasis in T1-T2, Non-invasive, and Clinically Node Negative Papillary Thyroid Carcinoma. Front Med (Lausanne) 2021;8:635771. [Crossref] [PubMed]
18. Wu Y, Rao K, Liu J, Han C, Gong L, Chong Y, Liu Z, Xu X. Machine Learning Algorithms for the Prediction of Central Lymph Node Metastasis in Patients With Papillary Thyroid Cancer. Front Endocrinol (Lausanne) 2020;11:577537. [Crossref] [PubMed]
19. Adler DD, Carson PL, Rubin JM, Quinn-Reid D. Doppler ultrasound color flow imaging in the study of breast cancer: preliminary findings. Ultrasound Med Biol 1990;16:553-9. [Crossref] [PubMed]
20. Yu J, Deng Y, Liu T, Zhou J, Jia X, Xiao T, Zhou S, Li J, Guo Y, Wang Y, Zhou J, Chang C. Lymph node metastasis prediction of papillary thyroid carcinoma based on transfer learning radiomics. Nat Commun 2020;11:4807. [Crossref] [PubMed]
21. Rakowski W, Clark MA. Do groups of women aged 50 to 75 match the national average mammography rate? Am J Prev Med 1998;15:187-97. [Crossref] [PubMed]
22. Zou Y, Zheng M, Qi Z, Guo Y, Ji X, Huang L, Gong Y, Lu X, Ma G, Xia S. Dual-energy computed tomography could reliably differentiate metastatic from non-metastatic lymph nodes of less than 0.5 cm in patients with papillary thyroid carcinoma. Quant Imaging Med Surg 2021;11:1354-67. [Crossref] [PubMed]
23. Cho SJ, Suh CH, Baek JH, Chung SR, Choi YJ, Lee JH. Diagnostic performance of MRI to detect metastatic cervical lymph nodes in patients with thyroid cancer: a systematic review and meta-analysis. Clin Radiol 2020;75:562.e1-562.e10. [Crossref] [PubMed]
24. Kim K, Shim SR, Lee SW, Kim SJ. Diagnostic values of F-18 FDG PET or PET/CT, CT, and US for Preoperative Lymph Node Staging in Thyroid Cancer: A Network Meta-Analysis. Br J Radiol 2021;94:20201076. [Crossref] [PubMed]
25. Luo X, Wang J, Xu M, Zou X, Lin Q, Zheng W, Guo Z, Li A, Han F. Risk model and risk stratification to preoperatively predict central lymph node metastasis in papillary thyroid carcinoma. Gland Surg 2020;9:300-10. [Crossref] [PubMed]
26. Liu C, Xiao C, Chen J, Li X, Feng Z, Gao Q, Liu Z. Risk factor analysis for predicting cervical lymph node metastasis in papillary thyroid carcinoma: a study of 966 patients. BMC Cancer 2019;19:622. [Crossref] [PubMed]
27. Liu W, Cheng R, Ma Y, Wang D, Su Y, Diao C, Zhang J, Qian J, Liu J. Establishment and validation of the scoring system for preoperative prediction of central lymph node metastasis in papillary thyroid carcinoma. Sci Rep 2018;8:6962. [Crossref] [PubMed]
28. Zhao H, Huang T, Li H. Risk factors for skip metastasis and lateral lymph node metastasis of papillary thyroid cancer. Surgery 2019;166:55-60. [Crossref] [PubMed]
29. Feng JW, Qin AC, Ye J, Pan H, Jiang Y, Qu Z. Predictive Factors for Lateral Lymph Node Metastasis and Skip Metastasis in Papillary Thyroid Carcinoma. Endocr Pathol 2020;31:67-76. [Crossref] [PubMed]
30. Park JH, Lee YS, Kim BW, Chang HS, Park CS. Skip lateral neck node metastases in papillary thyroid carcinoma. World J Surg 2012;36:743-7. [Crossref] [PubMed]
31. Kloth C, Kratzer W, Schmidberger J, Beer M, Clevert DA, Graeter T. Ultrasound 2020 – Diagnostics & Therapy: On the Way to Multimodal Ultrasound: Contrast-Enhanced Ultrasound (CEUS), Microvascular Doppler Techniques, Fusion Imaging, Sonoelastography, Interventional Sonography. Rofo 2021;193:23-32. [Crossref] [PubMed]
32. Zhang L, Gu J, Zhao Y, Zhu M, Wei J, Zhang B. The role of multimodal ultrasonic flow imaging in Thyroid Imaging Reporting and Data System (TI-RADS) 4 nodules. Gland Surg 2020;9:1469-77. [Crossref] [PubMed]
33. Kratzer W, Güthle M, Dobler F, Seufferlein T, Graeter T, Schmidberger J, Barth TF, Klaus J. Comparison of superb microvascular imaging (SMI) quantified with ImageJ to quantified contrast-enhanced ultrasound (Qceus) in liver metastases-a pilot study. Quant Imaging Med Surg 2022;12:1762-74. [Crossref] [PubMed]
34. Wang Y, Nie F, Wang G, Liu T, Dong T, Sun Y. Value of Combining Clinical Factors, Conventional Ultrasound, and Contrast-Enhanced Ultrasound Features in Preoperative Prediction of Central Lymph Node Metastases of Different Sized Papillary Thyroid Carcinomas. Cancer Manag Res 2021;13:3403-15. [Crossref] [PubMed]
35. Zhan J, Zhang LH, Yu Q, Li CL, Chen Y, Wang WP, Ding H. Prediction of cervical lymph node metastasis with contrast-enhanced ultrasound and association between presence of BRAF(V600E) and extrathyroidal extension in papillary thyroid carcinoma. Ther Adv Med Oncol 2020;12:1758835920942367. [Crossref] [PubMed]
36. Swan KZ, Nielsen VE, Bonnema SJ. Evaluation of thyroid nodules by shear wave elastography: a review of current knowledge. J Endocrinol Invest 2021;44:2043-56. [Crossref] [PubMed]
37. Guo JN, Song LH, Yu PY, Yu SY, Deng SH, Mao XH, Xiu C, Sun J. Ultrasound Elastic Parameters Predict Central Lymph Node Metastasis of Papillary Thyroid Carcinoma. J Surg Res 2020;253:69-78. [Crossref] [PubMed]
38. Yi L, Qiong W, Yan W, Youben F, Bing H. Correlation between Ultrasound Elastography and Histologic Characteristics of Papillary Thyroid Carcinoma. Sci Rep 2017;7:45042. [Crossref] [PubMed]
39. Kourou K, Exarchos TP, Exarchos KP, Karamouzis MV, Fotiadis DI. Machine learning applications in cancer prognosis and prediction. Comput Struct Biotechnol J 2014;13:8-17. [Crossref] [PubMed]
40. Zou Y, Shi Y, Liu J, Cui G, Yang Z, Liu M, Sun F. A Comparative Analysis of Six Machine Learning Models Based on Ultrasound to Distinguish the Possibility of Central Cervical Lymph Node Metastasis in Patients With Papillary Thyroid Carcinoma. Front Oncol 2021;11:656127. [Crossref] [PubMed]
41. Nobre C, Martineli E, Braga A, de Carvalho A, Rezende S, Braga JL, Ludermir T. Knowledge extraction: a comparison between symbolic and connectionist methods. Int J Neural Syst 1999;9:257-64. [Crossref] [PubMed]
42. Pandhita S G, Sutrisna B, Wibowo S, Adisasmita AC, Rahardjo TBW, Amir N, Rustika R, Kosen S, Syarif S, Wreksoatmodjo BR. Decision Tree Clinical Algorithm for Screening of Mild Cognitive Impairment in the Elderly in Primary Health Care: Development, Test of Accuracy, and Time-Effectiveness Analysis. Neuroepidemiology 2020;54:243-50. [Crossref] [PubMed]
43. Steffens M, Lamina C, Illig T, Bettecken T, Vogler R, Entz P, et al. SNP-based analysis of genetic substructure in the German population. Hum Hered 2006;62:20-9. [Crossref] [PubMed]
44. Matsumoto K, Nohara Y, Soejima H, Yonehara T, Nakashima N, Kamouchi M. Stroke Prognostic Scores and Data-Driven Prediction of Clinical Outcomes After Acute Ischemic Stroke. Stroke 2020;51:1477-83. [Crossref] [PubMed]
45. Chiofolo C, Chbat N, Ghosh E, Eshelman L, Kashani K. Automated Continuous Acute Kidney Injury Prediction and Surveillance: A Random Forest Model. Mayo Clin Proc 2019;94:783-92. [Crossref] [PubMed]
46. Zhou SC, Liu TT, Zhou J, Huang YX, Guo Y, Yu JH, Wang YY, Chang C. An Ultrasound Radiomics Nomogram for Preoperative Prediction of Central Neck Lymph Node Metastasis in Papillary Thyroid Carcinoma. Front Oncol 2020;10:1591. [Crossref] [PubMed]