Development and validation of an ultrasound-based deep learning radiomics nomogram for risk assessment of lymph node metastasis in papillary thyroid microcarcinoma
Original Article

Development and validation of an ultrasound-based deep learning radiomics nomogram for risk assessment of lymph node metastasis in papillary thyroid microcarcinoma

Ziyi Liu1# ORCID logo, Wenwu Lu1# ORCID logo, Ming Ge1 ORCID logo, Ji Zhou1 ORCID logo, Hui Jiang2 ORCID logo, Xin Wu3 ORCID logo, Chaoxue Zhang1 ORCID logo

1Department of Ultrasound, The First Affiliated Hospital of Anhui Medical University, Hefei, China; 2Department of Ultrasound, The Second Affiliated Hospital of Anhui Medical University, Hefei, China; 3Department of Ultrasound, Affiliated Hospital of Integrated Traditional Chinese and Western Medicine, Nanjing University of Chinese Medicine, Nanjing, China

Contributions: (I) Conception and design: Z Liu, W Lu, C Zhang; (II) Administrative support: C Zhang; (III) Provision of study materials or patients: M Ge, J Zhou, H Jiang, X Wu; (IV) Collection and assembly of data: Z Liu, W Lu, C Zhang; (V) Data analysis and interpretation: Z Liu, W Lu, M Ge, J Zhou, H Jiang, X Wu; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

#These authors contributed equally to this work.

Correspondence to: Prof. Chaoxue Zhang, MD, PhD. Department of Ultrasound, The First Affiliated Hospital of Anhui Medical University, No. 218 Jixi Road, Shushan District, Hefei 230022, China. Email: zcxay@fy.ahmu.edu.cn.

Background: Accurate preoperative prediction of cervical lymph node metastasis (LNM) in papillary thyroid microcarcinoma (PTMC) remains challenging, particularly because occult nodal disease is common and conventional ultrasound (US) assessment is operator-dependent. This study aimed to develop and validate a multicenter US-based deep learning radiomics nomogram integrating intratumoral, peritumoral, and clinical features for individualized LNM risk assessment in patients with PTMC.

Methods: We retrospectively and prospectively collected multi-center data from three hospitals. Patients who underwent total thyroidectomy or lobectomy with lymph node dissection were allocated to a training set (763 cases), an external test set (118 cases), and a prospective validation set (94 cases). Radiomics features from within the tumor and deep learning (DL) features from the peritumoral region were extracted from the largest cross-sectional US image. Twelve machine learning (ML) models were built using the integrated features and evaluated on the test set to identify the optimal one. The machine learning prediction score (ML-score) was incorporated with clinical factors into a nomogram, and its performance was evaluated using area under the receiver operating characteristic curve (AUC), decision curve analysis (DCA), and clinical impact curves (CICs) analysis.

Results: The support vector machine (SVM) model showed the best overall performance among the twelve ML algorithms and was used to generate the SVM-score. Multivariable logistic regression identified age, sex, maximum tumor diameter, genetic mutation status, and SVM-score as independent predictors of LNM. The integrated nomogram achieved AUCs of 0.894 in the training cohort, 0.842 in the external test cohort, and 0.856 in the prospective validation cohort. Calibration curves showed good agreement between predicted and observed LNM risk, with Hosmer-Lemeshow test P values of 0.189, 0.383, and 0.254 in the three cohorts, respectively. DCA and CIC demonstrated that the nomogram provided greater clinical net benefit than the clinical model or SVM-score model alone.

Conclusions: The US-based DL radiomics nomogram integrating imaging features and clinical factors showed good performance for preoperative prediction of LNM in patients with PTMC. This non-invasive tool may assist individualized cervical lymph node risk stratification and support clinical decision-making.

Keywords: Lymph node metastasis (LNM); deep learning (DL); radiomics; nomogram; papillary thyroid microcarcinoma (PTMC)


Submitted Jan 19, 2026. Accepted for publication Jul 08, 2026. Published online Aug 05, 2026.

doi: 10.21037/qims-2026-1-0135


Introduction

The widespread use of imaging and fine-needle biopsy has markedly increased the detection rate of thyroid cancer, with sub-centimeter tumors (diameter <10 mm) representing a substantial proportion of cases (1,2). In China, these tumors occur frequently, with the majority being papillary carcinomas (3). Although papillary thyroid microcarcinomas (PTMC) are small in size, accumulating evidence shows that some of them possess invasive biological behavior and recurrence risk similar to those of larger carcinomas (4). A considerable subset of PTMC is associated with early cervical lymph node metastasis (LNM) (3,5). LNM is a key prognostic factor and a major determinant of surgical planning. The presence of LNM significantly increases the risk of locoregional recurrence and is associated with unfavorable long-term survival (6,7). Therefore, accurate preoperative evaluation of cervical lymph node status is critical for making individualized clinical decisions.

Fine-needle aspiration cytology (FNAC) is commonly used to evaluate suspicious lymph nodes, though its accuracy in assessing cervical node involvement is limited. For instance, FNAC has poor sensitivity in detecting lymph node micrometastases, often resulting in false-negative outcomes (8). Furthermore, while prophylactic central neck dissection may lower reoperation risks, evidence indicates that it does not significantly reduce local recurrence rates. Additionally, surgical complications such as recurrent laryngeal nerve injury and hypoparathyroidism may outweigh its potential benefits (9,10).

Ultrasound (US) remains the preferred imaging modality for the preoperative evaluation of thyroid cancer and LNM, owing to its accessibility, low cost, lack of radiation, and ease of use (11). However, its findings are operator-dependent and influenced by examiner experience (12). Some studies have found that when thyroid cancer is ≤10 mm in size, the rate of occult LNM is high (13). Moreover, in patients with coexisting Hashimoto’s thyroiditis (HT) and small thyroid cancers, it is challenging to distinguish between benign nodal enlargement and malignant metastasis preoperatively (8,14). Therefore, this study focuses on analyzing features of the thyroid nodule itself to explore factors influencing LNM.

Artificial intelligence, particularly radiomics and deep learning (DL), is increasingly applied in medical imaging. Radiomics enables high-throughput extraction of quantitative features from medical images to identify complex patterns, and has demonstrated potential in predicting tumor LNM (15,16). Advances in image acquisition and processing have expanded its role beyond diagnosis to include prognosis prediction and personalized treatment planning (17,18). DL supports end-to-end analysis by automatically extracting relevant features directly from images and building integrated models for tasks such as tumor classification (19).

Beyond intratumoral regions, radiomics and DL can also capture information from the peritumoral area (including the tumor microenvironment and microvasculature), which has been shown to improve predictive performance in various clinical contexts (20,21). Growing evidence suggests that models should not be confined to the tumor alone; the surrounding tissue may provide valuable insights into tumor heterogeneity, angiogenesis, lymphovascular invasion, and stromal response (20,21). Microscopic infiltration in apparently normal peritumoral tissue may also elevate the risk of distant metastasis.

This study aims to develop a nomogram for predicting LNM in PTMC. By integrating DL-derived radiomics features from both intratumoral and peritumoral US images with relevant clinical factors, the nomogram is designed as a non-invasive tool for overall preoperative cervical nodal risk stratification to support personalized treatment strategies for patients with PTMC. We present this article in accordance with the TRIPOD+AI reporting checklist (available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0135/rc).


Methods

Research subjects

This study was conducted in three phases: retrospective model development, retrospective external testing, and prospective external validation.

This multicenter study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Ethics Committee of The First Affiliated Hospital of Anhui Medical University, Hefei, China (No. PJ2023-07-11 for the multicenter retrospective study); the prospective study was approved by the Ethics Committee of The Second Affiliated Hospital of Anhui Medical University, Hefei, China (No. PJ2024-12-72 for the prospective study). The additional participating hospitals were informed of the approved study protocol and agreed to participate in the study and provide anonymized clinical and US data during the approved study period. The requirement for informed consent was waived for the study because of its retrospective nature and the use of anonymized data. Written informed consent was obtained from all participants in the prospective validation cohort. The clinical trial registration number was ChiCTR2500096950. For the retrospective phases, consecutive patients with PTMC who underwent thyroidectomy with central and/or lateral lymph node dissection were enrolled. The cohort allocation was center-based rather than randomly split. The training cohort comprised patients treated at The First Affiliated Hospital of Anhui Medical University (Medical Center 1) between January 2020 and January 2025, whereas the external testing cohort included patients treated at The Affiliated Hospital of Integrated Traditional Chinese and Western Medicine, Nanjing University of Chinese Medicine (Medical Center 2) between January 2022 and December 2024. A total of 763 patients were included in the training set and 118 patients in the external testing set. For prospective validation, patients with suspected PTMC who underwent thyroidectomy with central and/or lateral lymph node dissection at The Second Affiliated Hospital of Anhui Medical University (Medical Center 3) between February and September 2025 were consecutively recruited, yielding 94 eligible patients for the prospective validation cohort. All patients underwent preoperative cervical US evaluation followed by thyroid surgery. Therapeutic compartment-oriented neck dissection was performed for clinically or radiologically suspected nodal metastasis, whereas central neck dissection in clinically node-negative patients was undertaken selectively in certain cases according to institutional practice and surgeon judgment. Postoperative histopathology served as the reference standard for determining cervical LNM.

Inclusion and exclusion criteria for the training set, external test set, and validation set are detailed in Appendix 1 and Figure S1.

Clinical pathology data and US image screening

All clinical data and US images were retrospectively retrieved from the Picture Archiving and Communication System (PACS), including pathological reports, age, sex, tumor size, location, genetic mutations, and HT status. A summary of the prospective US image acquisition protocol is provided in Appendix 1. Based on histopathological results, patients were classified into LNM-positive and -negative groups.

Thyroid US images underwent quality control by two senior radiologists to exclude studies with significant artifacts, insufficient resolution, or mismatch between nodule location and pathological findings. All included images were confirmed to represent the largest longitudinal section of the nodule within surrounding normal tissue. To ensure comparability of two-dimensional (2D) US features across different devices, images were standardized to a uniform format with normalized pixel spacing. Grayscale normalisation was applied, supplemented where necessary by histogram matching, to minimize intermanufacturer variability. Detailed steps for image screening and preprocessing are outlined in Appendix 2.

Region of interest (ROI) sketching and feature extraction

Radiologist A delineated ROI around the entire tumor on its largest cross-sectional image using ITK-SNAP software (v3.8.0). Radiomic features were subsequently extracted using the PyRadiomics package (v3.0.1). To fully encompass the tumor, a square ROI was generated by extending 3 mm outward from the center of the original contour. DL features were extracted from these square ROIs using a DenseNet201 transfer learning model. The overall feature extraction pipeline is illustrated in Figure 1.

Figure 1 Flowchart of this study. ROI, region of interest; US, ultrasound.

To assess feature reproducibility, 50 random samples from the training set were independently segmented by two radiologists, each with five years of experience (Radiologist A performed the segmentation twice, with a two-week interval; Radiologist B performed it once). Features demonstrating high inter- and intra-observer agreement [intraclass correlation coefficient (ICC) >0.80] were selected. The final feature set comprised the intersection of features meeting this ICC threshold across all three segmentation analyses. Detailed extraction methodology is provided in Appendix 3.

Feature selection and ML-score construction

Radiomic and DL features extracted from the training set were integrated, retaining only those with an ICC >0.80. The feature data were then standardized using Z-score normalisation. Following the removal of zero-variance features, 768 significant features (P<0.05) were selected using independent t-tests (for normally distributed data) and Mann-Whitney U tests. Subsequent least absolute shrinkage and selection operator (LASSO) regression with five-fold cross-validation was applied to further reduce the feature set and identify key predictors. The detailed feature selection procedure is described in Appendix 4.

Using the final feature set from the training data, twelve ML algorithms—including AdaBoost, decision tree, multilayer perceptron, K-nearest neighbor, gradient boosting, CatBoost, LightGBM, extreme gradient boosting (XGBoost), bagging, ExtraTrees, random forest, and support vector machine (SVM)—were employed to construct LNM risk prediction models. The area under the receiver operating characteristic curve (AUC) on the test set was used to evaluate model performance and select the optimal algorithm, from which a corresponding machine learning prediction score (ML-score) was derived. The contribution of key predictive features to LNM was further visualized using Shapley Additive Explanations (SHAP) analysis. Full details of the model selection process are provided in Appendix 5.

Construction and functional evaluation of nomogram

Univariate logistic regression was used to screen clinical features and ML-score. Variables with P<0.05 were entered into multivariate logistic regression to construct the predictive nomogram. Calibration curves and the Hosmer-Lemeshow test were used to assess calibration. The optimal cutoff value was determined by the Youden index.

Model performance was quantified using AUC, sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV). Decision curve analysis (DCA) and clinical impact curves (CICs) were used to evaluate clinical utility. Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were calculated to compare the incremental value of the integrated model.

The regression formulas for the clinical model and the integrated nomogram model are provided in Appendix 6.

Statistical analyses

Statistical analyses were performed using R (v4.4.1), SPSS (v26.0), and MedCalc (v20.1). Continuous variables were expressed as mean ± standard deviation and compared using t-tests or Mann-Whitney U tests. Categorical variables were presented as counts and percentages. A two-sided P value <0.05 was considered statistically significant.


Results

Clinical and pathological characteristics of patients

The number of PTMC patients in the training set, test set and validation set was 763, 118 and 94, respectively. Postoperative pathology verification confirmed that LNM (+) patients accounted for 43.8% (334/763), 51.7% (61/118) and 50.0% (47/94) of the sample in the training, testing and validation sets, respectively. The clinical pathological characteristics of patients are summarised in Table 1. The univariate analysis identified several factors significantly related to LNM (P<0.05), including gender, age, maximum tumor diameter, HT and genetic mutation, as shown in Table 2. The clinical model based on these factors yielded AUCs of 0.694, 0.771, and 0.669 in the three cohorts.

Table 1

Clinicopathological characteristics associated with LNM in PTMC patients

Characteristics Training cohort (n=763) Test cohort (n=118) Validation cohort (n=94)
Positive (n=334) Negative (n=429) Positive (n=61) Negative (n=57) Positive (n=47) Negative (n=47)
Age, years 42.10±11.16 46.80±11.16 39.59±10.13 45.62±10.20 41.84±10.00 46.95±10.03
Sex
   Female 238 (71.3) 362 (84.4) 38 (62.3) 46 (80.7) 42 (89.4) 37 (78.7)
   Male 96 (28.7) 67 (15.6) 23 (37.7) 11 (19.3) 5 (10.6) 10 (21.3)
Size, mm 6.30±1.66 5.70±1.66 6.24±1.60 5.17±1.61 6.60±1.45 6.36±1.46
Location
   Left lobe 178 (53.3) 217 (50.6) 29 (47.5) 26 (45.6) 14 (29.8) 14 (29.8)
   Right lobe 143 (42.8) 208 (48.5) 28 (37.7) 29 (50.8) 31 (66.0) 31 (66.0)
   Isthmus 13 (3.9) 4 (0.9) 4 (6.5) 2 (3.5) 2 (4.2) 2 (4.2)
HT
   Presence 58 (17.4) 100 (23.3) 22 (36.1) 24 (42.1) 14 (29.8) 14 (29.8)
   Absence 276 (82.6) 329 (76.7) 39 (63.9) 33 (57.9) 33 (70.2) 33 (70.2)
Genetic mutation
   Positive 218 (65.3) 221 (51.5) 43 (70.5) 41 (71.9) 22 (46.8) 25 (53.2)
   Negative 116 (34.7) 208 (48.5) 18 (29.5) 16 (28.1) 25 (53.2) 22 (46.8)
SVM-score 0.66 0.26 0.61 0.38 0.68 0.46

Data are presented as mean ± standard deviation or n (%). HT, Hashimoto’s thyroiditis; LNM, lymph node metastasis; PTMC, papillary thyroid microcarcinoma; SVM, support vector machine.

Table 2

Univariate analysis of factors associated with LNM of PTMC patients in the training cohort

Characteristics LNM (+) (n=334) LNM (−) (n=429) P value
Age, years 42.10±11.16 46.80±11.16 <0.01*
Sex <0.01*
   Female 238 (71.3) 362 (84.4)
   Male 96 (28.7) 67 (15.6)
Size, mm 6.30±1.66 5.70±1.66 <0.01*
Tumor location 0.051
   Left lobe 178 (53.3) 217 (50.6)
   Right lobe 143 (42.8) 208 (48.5)
   Isthmus 13 (3.9) 4 (6.5)
HT 0.04*
   Presence 58 (17.4) 100 (23.3)
   Absence 276 (82.6) 329 (76.7)
Genetic mutation <0.01*
   Positive 218 (65.3) 221 (51.5)
   Negative 116 (34.7) 208 (48.5)
SVM-score 0.66 0.26 <0.01*

Data are presented as mean ± standard deviation or n (%). *, P<0.05. HT, Hashimoto’s thyroiditis; LNM, lymph node metastasis; PTMC, papillary thyroid microcarcinoma; SVM, support vector machine.

Feature selection and ML-score model construction

After ICC evaluation of the ROI of the 2D US image in the training set, 686 handcrafted radiomics features and 128 DL features were extracted. Through redundancy analysis and LASSO regression screening of residual features, 10 stable features were finally obtained to build ML-score (Figure S2).

Based on the AUC value on the test set of 12 LNM ML risk prediction models (Figure 2A), SVM is selected as the optimal ML model, thus constructing the SVM-score model. The detailed performance indicators of the twelve ML models across the training, test and validation sets are shown in Tables S1-S3. As shown in Figure 2B, the SHAP diagram quantifies the contribution of 10 key features in the SVM-score model to the prediction results, showing that DL features play a key role in LNM diagnosis.

Figure 2 Performance of machine learning algorithms. (A) Pie charts of 12 machine learning models for the training set and external test set. (B) The contributions of the top 10 features in the SVM-score model using SHAP plots. Each row signifies a single feature. The magnitude of a feature’s importance is indicated by the breadth of its distribution on the plot. Elevated feature values (shown in red) contribute positively to the prediction, while reduced values (shown in blue) contribute negatively to the predictive model. AUC, area under the curve; KNN, K-nearest neighbor; MLP, multilayer perceptron; SHAP, Shapley Additive Explanations; SVM, support vector machine; XGBoost, extreme gradient boosting.

Univariate analysis showed that the SVM-score was significantly associated with LNM in PTMC. The AUC values of the relevant SVM-score model in the training, test and validation sets are 0.875, 0.807 and 0.789.

Construction and evaluation of nomogram

Multivariable analysis identified SVM-score, age, maximum tumor diameter, genetic mutation status, and sex as independent predictors of LNM in PTMC (all P<0.05) (Table 3). These five variables were used to construct a diagnostic nomogram. The optimal threshold for the nomogram-derived score (Nomogram score) for evaluating LNM status was determined to be 0.577, based on maximizing the Youden index in the training set.

Table 3

Multivariable logistic regression analysis of factors associated with LNM in 763 PTMC patients from the training cohort

Intercept and Variable Clinical model Nomogram model
β Odds ratio (95% CI) P value β Odds ratio (95% CI) P value
Intercept −0.45 −2.23
Age −0.04 0.96 (0.95–0.98) <0.01* −0.04 0.96 (0.94–0.97) <0.01*
Sex 0.80 2.22 (1.53–3.22) <0.01* 0.68 1.98 (1.21–3.26) <0.01*
Size 0.24 1.27 (1.16–1.40) <0.01* 0.14 1.15 (1.01–1.30) 0.03*
Genetic mutation 0.51 1.66 (1.22–2.27) <0.01* 0.67 1.95 (1.29–2.95) <0.01*
HT NA NA 0.36 NA NA 0.96
Tumor location NA NA 0.16 NA NA 0.45
SVM-score NA NA NA 0.60 1.82 (1.68–1.97) <0.01*

*, P<0.05. CI, confidence interval; HT, Hashimoto’s thyroiditis; LNM, lymph node metastasis; NA, not available; PTMC, papillary thyroid microcarcinoma; SVM, support vector machine.

Calibration curves showed good agreement between predicted and observed LNM risk, with Hosmer-Lemeshow P values of 0.189, 0.383, and 0.254 in the training, test, and validation sets (Figure 3). As shown in Figure 4, the integrated nomogram achieved AUCs of 0.894 (training set), 0.842 (test set), and 0.856 (validation set), significantly outperforming the clinical model and SVM-score model alone (Table 4).

Figure 3 Nomogram and calibration curves of the training set, external test set, and prospective validation set. (A) Nomogram of predicting the risk of LNM in PTMC. (B) Calibration plots for internal validation of the LNM nomograms. LNM, lymph node metastasis; PTMC, papillary thyroid microcarcinoma; SVM, support vector machine.
Figure 4 ROC performance of predicting the PTMC LNM nomogram. (A) ROC curve for predicting LNM in the training cohort. (B) ROC curve for predicting LNM in the external test cohort. (C) ROC curve for predicting LNM in the prospective validation cohort. AUC, area under the ROC curve; LNM, lymph node metastasis; PTMC, papillary thyroid microcarcinoma; ROC, receiver operating characteristic; SVM, support vector machine.

Table 4

Performance of different models for evaluating LNM status of PTMC patients

Parameters Clinical model SVM-score model Comprehensive model
Training set Test set Validation set Training set Test set Validation set Training set Test set Validation set
AUC (95% CI) 0.694
(0.656–0.730)
0.771
(0.682–0.851)
0.669
(0.554–0.779)
0.875
(0.848–0.900)
0.807
(0.723–0.880)
0.789
(0.693–0.867)
0.894
(0.871–0.918)
0.842
(0.767–0.912)
0.856
(0.774–0.929)
Sensitivity 0.536 0.689 0.639 0.787 0.934 0.893 0.878 0.951 0.849
Specificity 0.755 0.685 0.619 0.847 0.385 0.427 0.794 0.561 0.512
PPV 0.630 0.702 0.628 0.800 0.619 0.609 0.769 0.700 0.637
NPV 0.676 0.671 0.630 0.836 0.844 0.800 0.893 0.914 0.771
PLR 2.201 2.283 1.742 5.203 1.534 1.584 4.304 2.220 1.783
NLR 0.615 0.458 0.592 0.252 0.177 0.257 0.154 0.089 0.301
ACC 0.659 0.687 0.629 0.820 0.669 0.660 0.831 0.763 0.681

ACC, accuracy; AUC, area under the receiver operating characteristic curve; CI, confidence interval; LNM, lymph node metastasis; NLR, negative likelihood ratio; NPV, negative predictive value; PLR, positive likelihood ratio; PPV, positive predictive value; PTMC, papillary thyroid microcarcinoma; SVM, support vector machine.

The clinical utility and net benefit of the nomogram were evaluated using DCA and CIC. The DCA indicated that, across all datasets, the integrated nomogram provided a greater net benefit for guiding decisions regarding LNM assessment compared to both the clinical model and the SVM-score model (Figure 5A-5C). CIC analysis revealed that at threshold probabilities exceeding 0.6, the number of LNM-positive cases predicted by the nomogram closely aligned with the actual number of events, supporting its clinical practicality (Figure 5D-5F).

Figure 5 Decision curve analysis and clinical impact curves of the integrated nomogram. (A-C) Decision curve analysis of the PTMC LNM in the training set (A), test set (B), and external validation (C). (D-F) Clinical impact curve for the risk model of the PTMC LNM in the training set (D), test set (E), and external validation (F). LNM, lymph node metastasis; PTMC, papillary thyroid microcarcinoma; SVM, support vector machine.

Both categorical and continuous NRI indices were positive across all datasets, indicating superior reclassification performance of the composite model over the single models. The IDI results confirmed that the nomogram had a significantly enhanced ability to discriminate between LNM-positive and LNM-negative patients. These findings collectively demonstrate that the integrated nomogram model significantly improves the predictive efficacy for LNM in PTMC compared to models based solely on clinical risk factors or the SVM-score alone. Detailed data are presented in Table 5.

Table 5

Evaluation of comparing the comprehensive nomogram model with the clinical model and the SVM-score model respectively through NRI and IDI

Characteristic NRI (categorical) NRI (continuous) IDI
Estimate (95% CI) P value Estimate (95% CI) P value Estimate (95% CI) P value
Train set
     Clinical vs. comprehensive 0.90 (0.81–1.00) <0.01* 1.24 (1.12–1.35) <0.01* 0.38 (0.34–0.41) <0.01*
     SVM-score vs. comprehensive 0.10 (0.05–0.15) <0.01* 0.91 (0.78–1.04) <0.01* 0.09 (0.08–0.10) <0.01*
Test set
     Clinical vs. comprehensive 0.27 (0.04–0.50) 0.02* 0.88 (0.55–1.20) <0.01* 0.13 (0.06–0.21) <0.01*
     SVM-score vs. comprehensive 0.42 (0.21–0.64) <0.01* 0.88 (0.56–1.21) <0.01* 0.17 (0.12–0.23) <0.01*
Validation set
     Clinical vs. comprehensive 0.87 (0.60–1.14) <0.01* 1.15 (0.82–1.48) <0.01* 0.35 (0.27–0.44) <0.01*
     SVM-score vs. comprehensive 0.25 (0.02–0.49) 0.03* 1.11 (0.77–1.44) <0.01* 0.22 (0.15–0.30) <0.01*

*, P<0.05. CI, confidence interval; IDI, integrated discrimination improvement; NRI, net reclassification improvement; SVM, support vector machine.


Discussion

Accurate preoperative assessment of cervical LNM remains a major challenge in the management of PTMC. In clinical practice, conventional US and cytology may fail to detect occult nodal disease, particularly in small tumors, leading to uncertainty in balancing oncologic adequacy against the potential morbidity of more extensive surgery (8,13). In this context, we developed and validated a multicenter, US-based prediction nomogram that integrates ML-derived imaging signatures with readily available clinical factors to provide individualized LNM risk stratification before surgery. By benchmarking twelve algorithms and selecting an optimal SVM classifier, the proposed model achieved consistently good discrimination across the training, external testing, and prospective validation cohorts (AUCs 0.894, 0.842, and 0.856, respectively), with favorable calibration and clinically meaningful net benefit. Collectively, these findings suggest that this model can effectively screen patients at high risk of hidden LNM and reduce the overuse of uniform lymph node handling strategies.

Previous studies have identified several clinical and US risk factors for LNM in microcarcinoma (3,22). Furthermore, some research suggests that coexisting HT may be associated with a higher risk of LNM (14). However, the precise impact of HT on LNM remains controversial; previous studies and reviews have reported inconsistent conclusions, suggesting that its role may vary according to tumor and patient characteristics (23,24). Clarifying the influence of HT is particularly important given that LNM status directly determines the surgical extent in PTMC. In our univariate analysis, HT was associated with LNM (P=0.04), but it was not an independent predictor in the multivariate model, suggesting that its effect may be confounded by other factors, such as sample size limitations, geographical variation, or ethnic differences.

To enhance the accuracy of LNM prediction, this study incorporates a radiomics approach. Radiomics non-invasively characterises tumor heterogeneity by converting medical images into high-throughput, quantifiable features that encapsulate rich pathological, biomarker, genomic, and prognostic information (15,16). Growing evidence suggests that an effective predictive model should not be restricted to the tumor region alone; the peritumoral area may also provide critical supplementary information regarding tumor heterogeneity (20,21). At present, substantial progress has been made in US radiomics research on papillary thyroid carcinoma (25,26); however, US-based radiomics studies focused specifically on PTMC remain limited (27), with prior work largely confined to intratumoral regions. Therefore, this study innovatively utilises a convolutional neural network to extract features from the peritumoral microenvironment and integrates these with traditional handcrafted radiomics and DL features derived from the tumor interior. This approach aims to reduce inter-observer variability and combine multi-dimensional information from the tumor’s core, margin, and surrounding tissue, thereby providing a more comprehensive characterisation of tumor biology for improved LNM prediction (17-19).

Consistent with previous evidence on clinical and molecular predictors of LNM in papillary thyroid carcinoma (28), multivariable analysis confirmed that age, maximum tumor diameter, sex, and BRAF V600E mutation status were independent predictors of LNM in the clinical model. To develop a clinically applicable decision-making tool, a comprehensive predictive model was established for more thorough risk assessment. This integrated model demonstrated strong discriminative ability and calibration across the training, external test, and external validation sets, with AUC values of 0.894, 0.842, and 0.856, respectively. Its predictive performance significantly surpassed that of the clinical model, which utilized only conventional clinical and US risk variables. Furthermore, the significant improvements in both the net reclassification index (NRI) and IDI confirmed that the incorporation of the SVM-score substantially enhanced the LNM prediction capability of the final model.

Compared with models based solely on clinical features or single-modality imaging (22,29-32), the integrated nomogram demonstrated superior predictive performance, supporting the value of combining multi-source data for LNM prediction. From a clinical perspective, the primary contribution of this model is to support preoperative risk stratification rather than to directly determine the extent of neck dissection. In patients with PTMC, particularly those with clinically node-negative or indeterminate findings, the nomogram may help identify individuals at higher risk of nodal metastasis. This may prompt more detailed evaluation of both central and lateral neck compartments, additional imaging when appropriate, or multidisciplinary discussion regarding surgical strategy. Conversely, in low-risk patients, the model may help avoid unnecessary aggressive management, thereby contributing to more balanced and individualized treatment decisions.

In practical implementation, the reliability of the model depends on the quality and completeness of input data. Poor image quality, inaccurate tumor segmentation, or missing clinical variables may reduce predictive accuracy. Therefore, input data should be carefully assessed before model application, and predictions based on low-quality or incomplete data should be interpreted with caution. In addition, the current workflow requires manual ROI delineation and expert interpretation, meaning that the model is intended for use by trained clinicians rather than non-specialists.

This study has several limitations. First, the analysis was confined to static images from the tumor’s largest cross-sectional diameter. Second, no random split or internal cross-validation was used, and selection bias may exist. Third, in this prediction model, LNM was analyzed as a binary patient‑level outcome.

Subsequent investigations will concentrate on enhancing the model’s clinical utility. Such efforts involve external validation in larger and more demographically varied cohorts, integration of multi‑modal imaging information, design of automated segmentation and quality assurance workflows, and development of region-specific and tumor-burden-adapted predictive tools. Further prospective clinical studies are also warranted to verify whether application of this model can optimize clinical decision-making and ultimately improve patient prognosis.


Conclusions

This study developed and validated a US-based DL radiomics nomogram that integrates imaging features and clinical factors for the individualised prediction of LNM in patients with PTMC. The proposed model may help identify patients at high risk of occult LNM, guide decisions regarding whether more detailed cervical lymph node evaluation is required, and reduce reliance on a one-size-fits-all approach to lymph node management. As a non-invasive tool, it may support preoperative LNM risk stratification and provide useful guidance for personalized clinical decision-making and future refined risk assessment.


Acknowledgments

None.


Footnote

Reporting Checklist: The authors have completed the TRIPOD+AI reporting checklist. Available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0135/rc

Data Sharing Statement: Available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0135/dss

Funding: This study was supported by Natural Science Foundation of Anhui Province of China (No. 2308085MH278, to C.Z.).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-2026-1-0135/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Ethics Committee of The First Affiliated Hospital of Anhui Medical University, Hefei, China (No. PJ2023-07-11 for the multicenter retrospective study); the prospective study was approved by the Ethics Committee of The Second Affiliated Hospital of Anhui Medical University, Hefei, China (No. PJ2024-12-72 for the prospective study). The additional participating hospitals were informed of and agreed to the study. Informed consent was waived for the retrospective study. All prospective study participants signed informed consent forms.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Boucai L, Zafereo M, Cabanillas ME. Thyroid Cancer: A Review. JAMA 2024;331:425-35. [Crossref] [PubMed]
  2. Pellegriti G, Frasca F, Regalbuto C, Squatrito S, Vigneri R. Worldwide increasing incidence of thyroid cancer: update on epidemiology and risk factors. J Cancer Epidemiol 2013;2013:965212. [Crossref] [PubMed]
  3. Wang Y, Guan Q, Xiang J. Nomogram for predicting central lymph node metastasis in papillary thyroid microcarcinoma: A retrospective cohort study of 8668 patients. Int J Surg 2018;55:98-102. [Crossref] [PubMed]
  4. Nie X, Tan Z, Ge M, Jiang L, Wang J, Zheng C. Risk factors analyses for lateral lymph node metastases in papillary thyroid carcinomas: a retrospective study of 356 patients. Arch Endocrinol Metab 2016;60:492-9. [Crossref] [PubMed]
  5. Wei Q, Wu D, Luo H, Wang X, Zhang R, Liu Y. Features of lymph node metastasis of papillary thyroid carcinoma in ultrasonography and CT and the significance of their combination in the diagnosis and prognosis of lymph node metastasis. J BUON 2018;23:1041-8.
  6. Guo K, Wang Z. Risk factors influencing the recurrence of papillary thyroid carcinoma: a systematic review and meta-analysis. Int J Clin Exp Pathol 2014;7:5393-403.
  7. Ito Y, Tomoda C, Uruno T, Takamura Y, Miya A, Kobayashi K, Matsuzuka F, Kuma K, Miyauchi A. Ultrasonographically and anatomopathologically detectable node metastases in the lateral compartment as indicators of worse relapse-free survival in patients with papillary thyroid carcinoma. World J Surg 2005;29:917-20. [Crossref] [PubMed]
  8. Jun HH, Kim SM, Kim BW, Lee YS, Chang HS, Park CS. Overcoming the limitations of fine needle aspiration biopsy: detection of lateral neck node metastasis in papillary thyroid carcinoma. Yonsei Med J 2015;56:182-8. [Crossref] [PubMed]
  9. Sippel RS, Robbins SE, Poehls JL, Pitt SC, Chen H, Leverson G, Long KL, Schneider DF, Connor NP. A Randomized Controlled Clinical Trial: No Clear Benefit to Prophylactic Central Neck Dissection in Patients With Clinically Node Negative Papillary Thyroid Cancer. Ann Surg 2020;272:496-503. [Crossref] [PubMed]
  10. Conzo G, Calò PG, Sinisi AA, De Bellis A, Pasquali D, Iorio S, Tartaglia E, Mauriello C, Gambardella C, Cavallo F, Medas F, Polistena A, Santini L, Avenia N. Impact of prophylactic central compartment neck dissection on locoregional recurrence of differentiated thyroid cancer in clinically node-negative patients: a retrospective study of a large clinical series. Surgery 2014;155:998-1005. [Crossref] [PubMed]
  11. Wong KT, Ahuja AT. Ultrasound of thyroid cancer. Cancer Imaging 2005;5:157-66. [Crossref] [PubMed]
  12. Alyami J, Almutairi FF, Aldoassary S, Albeshry A, Almontashri A, Abounassif M, Alamri M. Interobserver variability in ultrasound assessment of thyroid nodules. Medicine (Baltimore) 2022;101:e31106. [Crossref] [PubMed]
  13. Tang L, Qu RW, Park J, Simental AA, Inman JC. Prevalence of Occult Central Lymph Node Metastasis by Tumor Size in Papillary Thyroid Carcinoma: A Systematic Review and Meta-Analysis. Curr Oncol 2023;30:7335-50. [Crossref] [PubMed]
  14. Song E, Jeon MJ, Park S, Kim M, Oh HS, Song DE, Kim WG, Kim WB, Shong YK, Kim TY. Influence of coexistent Hashimoto's thyroiditis on the extent of cervical lymph node dissection and prognosis in papillary thyroid carcinoma. Clin Endocrinol (Oxf) 2018;88:123-8. [Crossref] [PubMed]
  15. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, Zegers CM, Gillies R, Boellard R, Dekker A, Aerts HJ. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 2012;48:441-6. [Crossref] [PubMed]
  16. Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology 2016;278:563-77. [Crossref] [PubMed]
  17. Avanzo M, Wei L, Stancanello J, Vallières M, Rao A, Morin O, Mattonen SA, El Naqa I. Machine and deep learning methods for radiomics. Med Phys 2020;47:e185-202. [Crossref] [PubMed]
  18. Chan HP, Hadjiiski LM, Samala RK. Computer-aided diagnosis in the era of deep learning. Med Phys 2020;47:e218-27. [Crossref] [PubMed]
  19. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal 2017;42:60-88. [Crossref] [PubMed]
  20. Lin P, Xie W, Li Y, Zhang C, Wu H, Wan H, Gao M, Liang F, Han P, Chen R, Cheng G, Liu X, Fan S, Huang X. Intratumoral and peritumoral radiomics of MRIs predicts pathologic complete response to neoadjuvant chemoimmunotherapy in patients with head and neck squamous cell carcinoma. J Immunother Cancer 2024;12:e009616. [Crossref] [PubMed]
  21. Ding J, Chen S, Serrano Sosa M, Cattell R, Lei L, Sun J, Prasanna P, Liu C, Huang C. Optimizing the Peritumoral Region Size in Radiomics Analysis for Sentinel Lymph Node Status Prediction in Breast Cancer. Acad Radiol 2022;29:S223-8. [Crossref] [PubMed]
  22. Feng JW, Hong LZ, Wang F, Wu WX, Hu J, Liu SY, Jiang Y, Ye J. A Nomogram Based on Clinical and Ultrasound Characteristics to Predict Central Lymph Node Metastasis of Papillary Thyroid Carcinoma. Front Endocrinol (Lausanne) 2021;12:666315. [Crossref] [PubMed]
  23. Lai X, Xia Y, Zhang B, Li J, Jiang Y. A meta-analysis of Hashimoto's thyroiditis and papillary thyroid carcinoma risk. Oncotarget 2017;8:62414-24. [Crossref] [PubMed]
  24. Lee I, Kim HK, Soh EY, Lee J. The Association Between Chronic Lymphocytic Thyroiditis and the Progress of Papillary Thyroid Cancer. World J Surg 2020;44:1506-13. [Crossref] [PubMed]
  25. Chen Z, Wang JJ, Du JB, Li JF, Zheng RT, Yuan SM, Wu T, Guo DM, Zhai YX. Development and validation of a dynamic nomogram for predicting central lymph node metastasis in papillary thyroid carcinoma patients based on clinical and ultrasound features. Quant Imaging Med Surg 2025;15:1555-70. [Crossref] [PubMed]
  26. Zou Y, Shi Y, Bi H, Tan J, Guo Q, Qin Y, Lu X, Ma X, Yang S, Liu J. A nomogram for risk stratification of central cervical lymph node metastasis in patients with papillary thyroid carcinoma. Quant Imaging Med Surg 2024;14:5084-98. [Crossref] [PubMed]
  27. Mou Y, Han X, Li J, Yu P, Wang C, Song Z, Wang X, Zhang M, Zhang H, Mao N, Song X. Development and Validation of a Computed Tomography-Based Radiomics Nomogram for the Preoperative Prediction of Central Lymph Node Metastasis in Papillary Thyroid Microcarcinoma. Acad Radiol 2024;31:1805-17. [Crossref] [PubMed]
  28. Zhou SL, Guo YP, Zhang L, Deng T, Xu ZG, Ding C, Sun WC, Zhao YW, Kong LF. Predicting factors of central lymph node metastasis and BRAF(V600E) mutation in Chinese population with papillary thyroid carcinoma. World J Surg Oncol 2021;19:211. [Crossref] [PubMed]
  29. Feng JW, Liu SQ, Qi GF, Ye J, Hong LZ, Wu WX, Jiang Y. Development and Validation of Clinical-Radiomics Nomogram for Preoperative Prediction of Central Lymph Node Metastasis in Papillary Thyroid Carcinoma. Acad Radiol 2024;31:2292-305. [Crossref] [PubMed]
  30. Dong L, Han X, Yu P, Zhang W, Wang C, Sun Q, Song F, Zhang H, Zheng G, Mao N, Song X CT. Radiomics-Based Nomogram for Predicting the Lateral Neck Lymph Node Metastasis in Papillary Thyroid Carcinoma: A Prospective Multicenter Study. Acad Radiol 2023;30:3032-46. [Crossref] [PubMed]
  31. Wang Y, Zhang S, Zhang M, Zhang G, Chen Z, Wang X, Yang Z, Yu Z, Ma H, Wang Z, Sang L. Prediction of lateral lymph node metastasis with short diameter less than 8 mm in papillary thyroid carcinoma based on radiomics. Cancer Imaging 2024;24:155. [Crossref] [PubMed]
  32. Wu X, Li J, Mou Y, Yao Y, Cui J, Mao N, Song X. Radiomics Nomogram for Identifying Sub-1 cm Benign and Malignant Thyroid Lesions. Front Oncol 2021;11:580886. [Crossref] [PubMed]
Cite this article as: Liu Z, Lu W, Ge M, Zhou J, Jiang H, Wu X, Zhang C. Development and validation of an ultrasound-based deep learning radiomics nomogram for risk assessment of lymph node metastasis in papillary thyroid microcarcinoma. Quant Imaging Med Surg 2026;16(9):730. doi: 10.21037/qims-2026-1-0135

Download Citation