Development and multicenter validation of a CT-based radiomics signature for discriminating histological grades of pancreatic ductal adenocarcinoma

Introduction

As of 2018, pancreatic cancer is the seventh leading cause of cancer death, having a high mortality rate, with almost as many deaths (n=432,000) as cases (n=459,000) (1). Pancreatic ductal adenocarcinoma (PDAC) is the main histopathological type of pancreatic cancer. Currently, surgery remains the only curative option for PDAC (2) but obtains poor outcomes. The histological grade of pancreatic cancer is an important independent predictor of outcome (3-10). Low-grade PDAC has been associated with long-term postoperative survival (5,8). High-grade PDAC tends to predict shorter survival and increased mortality compared with a low-grade disease (3,7). Even patients with the high-grade disease who undergo surgery did not have a significantly improved prognosis, but they might be affected by surgical complications and have a significantly worse quality of life. To some extent, the histological grade of PDAC might affect the treatment options (11). In order to improve a patient's quality of life, it is critical to avoid unnecessary surgical complications, select the best treatment plan, and prolong survival; furthermore, an accurate preoperative assessment of tumor grade could be essential for individual therapy. To date, the methods for grading pancreatic cancer involve invasive biopsies, which include endoscopic ultrasound-guided fine-needle biopsy (EUS-FNB) and ultrasound/computed tomography (CT)-guided percutaneous biopsy; however, patients who undergo these procedures risk suffering complications such as pancreatitis (12). In addition, because of the heterogeneity of pancreatic tumors, the histopathological evaluation of a specimen obtained by puncture biopsy is not equivalent to the whole tumor (13). A safe and accurate method that can provide a histopathological grade of the tumor before surgery is needed. Traditional imaging examinations are widely used as noninvasive preoperative evaluation methods for patients with PDAC and can provide information, including tumor size, site, and relationship with vessels. We attempted to clarify the differentiation of PDAC from CT images further.

Radiomics (14-17) is a quantitative imaging method that can extract more data from digital medical images to improve the accuracy of clinical diagnosis, prediction, and prognosis, and can support personalized clinical decisions and improve individualized treatment options. Radiomics based on CT and magnetic resonance imaging (MRI) has been used to discern the histological grades of colorectal cancers (18), bladder cancers (19), hepatocellular carcinomas (20), non-small cell lung cancers (21), and gliomas (22). In our study, we attempted to perform radiomics for patients with PDAC by extracting and selecting the features on preoperative contrast-enhanced CT (CE-CT) images that were highly correlated with histological tumor grade, to develop a radiomics signature in the training dataset, and to validate the signature in the test dataset. An external validation dataset was also used for further testing of the accuracy of the model. The aim of our study was to build a predictive model for differentiating low-grade from high-grade PDAC.

Methods

Patients

Ethical approval was obtained for this retrospective study, and the requirement for informed consent was waived. The inclusion criteria were as follows: (I) patients were histopathologically confirmed to have PDAC postoperatively; (II) the defined histological grade was on patients’ reports; (III) CE-CT included the pancreatic parenchymal phase; and (IV) CE-CT was performed within 1 month before surgery. The exclusion criteria were as follows: (I) poor image quality, including artifacts (respiratory movements or metallic artifacts associated with a stent) in the tumor or difficulty in identifying the tumor on the CT scans of every phase; or (II) preoperative anticancer treatment. A total of 301 patients admitted to Shengjing Hospital of China Medical University from January 2005 to December 2018, were finally enrolled in this study. They were randomly divided into a training group with 151 patients (89 men, 62 women; age, 60.6±9.4 years) and a test group with 150 patients (94 men, 56 women; age, 60.0±9.6 years). An additional 100 patients (55 men, 45 women; age, 59.3±8.9 years) from The First Affiliated Hospital of China Medical University and Liaoning Cancer Institute and Hospital were recruited as an external validation group from January 2010 to December 2018, with the same criteria (Figure 1). Clinical data such as age, gender, tumor location, and preoperative carbohydrate antigen 19-9 (CA19-9) level were recorded, with CA19-9 level described as normal (0–37 U/mL) or abnormal (>37 U/mL).

Figure 1 Flow chart of the patients enrolled in the training, test, and external validation datasets. PDAC, pancreatic ductal adenocarcinoma; CE-CT, contrast-enhanced computed tomography.

Histological grading

All surgically removed tumors were formalin-fixed, paraffin-embedded, and hematoxylin-eosin stained. According to the WHO classification of tumors of the digestive system (4^th edition) (23), the classification of the gland is as follows: well, moderately, poorly differentiated, and undifferentiated. In our study, dichotomous statistics were used to classify well-differentiated (n=56) and moderately differentiated (n=104) versus poorly differentiated (n=133) and undifferentiated (n=8) cancers into low-grade (n=160) versus high-grade groups (n=141), respectively.

Imaging acquisition

The participants in our study underwent CE-CT via multiple CT devices (Brilliance iCT; Philips Healthcare, USA; Aquillion ONE; Toshiba Medical Systems Corporation, Japan; and Somatom Definition AS; Siemens, Germany) at the three participating centers. The imaging conditions satisfied the following requirements: multislice spiral CT scanners; scanning field of view, 35 cm × 35 cm–40 cm × 40 cm; matrix, 512×512; thickness of reconstructed image, 1–3 mm; three-phase scan, preinjection phase, pancreatic parenchymal phase (40–45 s after injection), and portal phase (70–75 s after injection). The images from the pancreatic parenchymal phase were used for feature extraction in our study. The dose of iodine contrast agent was based on the patient’s weight (1 mL/kg), and the flow rate was 2.5–3.5 mL/s.

Tumor segmentation and feature extraction

The segmentation of every tumor on the training, test, and validation datasets was performed by the same junior radiologist with a working experience of 8 years. Imaging Biomarker Explorer (IBEX) software (V1.0β; MD Anderson Cancer Center, Houston, TX, USA) (24) was used. Regions of interest (ROIs) were manually drawn along the borders of the lesions on images of the pancreatic parenchymal phase. Two weeks after the first ROIs were delineated, 30 cases were randomly selected for repeat delineation of the ROIs by the same radiologist, and the extracted features were evaluated by calculating the intraclass correlation coefficient (ICC). Another senior radiologist with a diagnostic experience of 20 years then performed ROI delineation and feature extraction on the same 30 cases. The extracted features from the ROIs of the first less experienced radiologist were compared to those from the ROIs of the experienced radiologist by calculating the ICC. An ICC >0.75 was considered to be excellent for the stability and reproducibility of features. The radiomics features included morphology, intensity-based statistics, intensity histogram, gray level co-occurrence matrix (GLCM), gray level run length matrix, and neighborhood grey tone difference matrix. The parameters of the features are provided in Table S1.

Table S1 Features extracted by IBEX
Full table

Feature selection and development of radiomics signatures

A support vector machine (SVM) method (kernel = rbf, C =0.78, gamma =0.00069) based on recursive feature elimination (RFE) was used to select features for predicting PDAC grade, which trained the classifier using all features, computed the ranking criterion for all features, and removed the features with smallest ranking criterion (15). The procedure was iterated until the number of features was equal to the number of predefined features. All the data of the selected features were normalized with z-score normalization in the training dataset. The test and external validation datasets were also normalized using the mean and standard deviation derived from the features of the training dataset. Redundant features that referred to low discrimination features with variances lower than the threshold were removed. The threshold in our study was 0.5, which was obtained by grid search.

The least absolute shrinkage and selection operator (LASSO) regression model was used to build a predictive classifier machine and was trained by a ten-fold cross-validation method. We calculated a radiomics score (Radscore) for each patient with the following formula:

Radscore =${\displaystyle {\sum }_{i=1}^N{w}_i\cdot x_i+b}$ , where N is the feature number, w_i is the weight of the ith feature, x_i is the value of the ith selected feature, and b is biased.

Model evaluation

The model that was built from the training dataset was applied to the test dataset and the external validation dataset to assess accuracy. Receiver operating characteristic (ROC) curve analysis and calculations of areas under the curve (AUCs) were performed to determine the discriminative performance of the radiomics signature. The optimal cutoff value of the radiomics score was determined by the Youden index and was determined only in the training dataset, and then directly applied to the testing and external validation dataset. The corresponding sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy were also determined.

Statistical analysis

SPSS software (version 22.0.0.0, IBM Corporation, Armonk, NY, USA) was used for statistical analysis. Continuous variables were expressed as means ± SD or medians and interquartile range (IQR), and categorical variables were expressed as numbers and percentages. Differences between variables of the patient groups were assessed by the independent t-test or Mann-Whitney U test for continuous variables and Fisher’s exact test or chi-squared test for categorical variables. A two-sided P value <0.05 was considered statistically significant.

Multivariate logistic regression analysis was performed to identify independent predictors of PDAC grade and included items such as the Radscore and clinical variables (age, gender, CA19-9 levels, and tumor location).

The SVM model was used to select features, and the LASSO regression model was used to build a radiomics signature. They were all performed by PYTHON (version 3.7.0; www.python.org).

Results

Clinicopathological characteristics

The clinicopathological characteristics of the study patients are shown in Tables 1,2. The differences between age, gender, tumor location, and CA19-9 level of the low-grade versus the high-grade patients were not significant. The P values of age, gender, tumor location, and CA19-9 level were 0.080, 0.420, 0.206, and 0.278 respectively in the training dataset; 0.557, 0.107, 0.674, and 0.271 in the test dataset; and 0.551, 0.511, 0.227, and 0.275 in the external validation dataset. The difference between the histologic grades in the training and external validation datasets was significant (P=0.005). The differences between age, gender, tumor location, and CA19-9 level in the training and external validation datasets were not significant, and P values were 0.301, 0.537, 0.287, and 0.474 respectively.

Table 1 Clinicopathological characteristics and radiomics scores of patients in the training, testing, and external validation datasets
Full table

Table 2 Clinicopathological characteristics between the training and external validation datasets
Full table

Construction of radiomics signatures

A total of 1,452 features were extracted automatically by IBEX software. The interested reader can find the features in Table S1. Among these features, 614 were considered to have excellent robustness and reproducibility with ICCs >0.75. Finally, 80 important features with correspondingly weighted coefficients were selected from the 614 features and calculated by the model. The top 10 features with the highest weights included homogeneity, entropy, inverse variance, max probability, information measure correlation, and intensity. They are shown in Table S2. The formula for the radiomics score was generated by the LASSO method. The selected features with their corresponding weights and biases are shown in Table S3.

Table S2 The top 10 features with their corresponding weight of the selected 80 features
Full table

Table S3 The features with their corresponding weight and bias in radiomics score formula
Full table

Prediction performance of the radiomics signature

The median radiomics scores of the low-grade PDAC patients in the training and testing groups were 0.115 (–0.087 to 0.329) and 0.304 (0.106 to 0.413), respectively. The median radiomics scores of the high-grade PDAC patients in the training and testing groups were 0.690 (0.516 to 0.908) and 0.695 (0.559 to 0.852), respectively. The differences between the radiomics scores of the low-grade versus the high-grade PDAC patients in both the training and test groups were significant (P<0.001; Table 1, Figure 2).

Figure 2 Boxplots of radiomics scores between the patients grouped according to low-grade and high-grade tumors in the training dataset (A), test dataset (B), and external validation dataset (C).

The AUCs of radiomics scores as predictors of tumor differentiation were 0.961 and 0.910 for the training and test groups, respectively (Figure 3). The cutoff value of the radiomics score was 0.426. The sensitivity, specificity, PPV, NPV, and accuracy of the radiomics score are shown in Table 3.

Figure 3 ROC curves. (A) ROC curve of the training group. The cutoff value of radiomics score was 0.426, and the AUC was 0.961; (B) ROC curve of the test group, with the AUC being 0.910; (C) ROC curve of the external validation group, with AUC being 0.770. ROC, receiver operating characteristic; AUC, area under the curve.

Table 3 Predictive performance of radiomics signatures
Full table

External validation of the predictive performance of the radiomics signature

In the external validation group, the median radiomics scores of the low-grade and high-grade patients were 0.311 (0.147 to 0.427) and 0.535 (0.147 to 0.427), respectively, which was significantly different (P<0.001; Table 1, Figure 2). The AUC of the radiomics score for predicting histological grade was 0.770 (Figure 3). The sensitivity, specificity, PPV, NPV, and accuracy of the radiomics score are shown in Table 3.

Independent predictors of PDAC grade

After multivariate logistic regression analysis, the Radscore [odds ratio (OR) =0.003, P<0.001] was included in the predictive model. The P values of the remaining clinicopathological characteristics, including age, gender, CA19-9 level, and tumor site were 0.951, 0.388, 0.066, and 0.709, respectively, and were therefore not included.

Discussion

Radiomics signatures have been widely used in radiomics research. As an important radiomics predictive index, the radiomics signature shows good discrimination performance (32). It combines many features of the image, allowing a more comprehensive analysis that results in a more sensitive and accurate diagnosis of a lesion than a few or single image indicators would (33). In our study, the radiomics signature based on CE-CT showed good performance for predicting the histological grade of PDAC. Our result is consistent with the findings of Huang et al. (18) for colorectal cancers and Chen et al. (21) for non-small cell lung cancers. Our results also validated the conclusions of Choi (34) and Cassinotto (35), who assert that texture analysis is an important component of radiomics that reflects the histological differentiation of PDAC and correlates with patient outcomes.

In our study, the top 10 most important features belonged to the areas of homogeneity, entropy, inverse variance, max probability, information measure correlation, and intensity. Except for intensity, which belongs to the first-order feature that can reflect the gray intensity value of voxels, all the other features were second-order texture features, belonging to the GLCM. The GLCM refers to the spatial correlation characteristics of voxel gray values and is used to reflect texture (26). Homogeneity is a reflection of the uniformity of the distribution of voxels. Entropy refers to the randomness of the distribution of gray values which indicates the complexity of image texture: the more complex the image texture, the larger the entropy value. Inverse variance reflects local changes in image texture. If the image texture was relatively uniform and changed slowly between different regions, the inverse variance was increased. The max probability represents the texture feature that appears most frequently in the image. The features reflected the complexity and heterogeneity of the different aspects of the ROIs in the images and were combined to provide the radiomics signature for predicting the differentiation grade of each tumor.

To test the accuracy of our model further, we performed an external validation study (36-39). In the external validation dataset, the AUC of our model was 0.770, and the accuracy was 0.730, which was decreased in the prediction accuracy. This result might be related to the small sample size in the external validation dataset and the difference in sample distribution. The difference between the histological grades of the training and the external validation groups was significant (P=0.005), and the number of high-grade tumors in the external validation group was significantly lower than the number of low-grade tumors. The differences between the groups might account for the reduced quality of the model evaluation of the external validation group. Furthermore, our data were derived from multiple CT systems used at different medical centers. Differences between acquisition protocols might be the key reason for the reduced predictive power of the model in the external validation group. A multicenter prospective study using standardized acquisition and reconstruction protocols is our goal for the future.

In this study, we classified well-differentiated and moderately differentiated versus poorly differentiated and undifferentiated cancers into low-grade versus high-grade groups, respectively, which was consistent with the classification methods of Wasif and Rochefort et al. (3,4). This dichotomous method has been considered to reflect the biological behavior and outcomes of PDACs (23). Furthermore, there were relatively fewer cases in the well-differentiated group (n=56) and the undifferentiated group (n=8). This can be explained by the combined well and moderately differentiated cases versus the poorly differentiated and undifferentiated cases.

At present, histological grades are mainly obtained from FNB specimens before surgery. However, studies by Larghi et al. (40) have shown that the preoperative grading accuracy, sensitivity, and specificity of EUS-FNB were 56%, 41%, and 78%. This result was mainly attributed to the limited sample size and intra-observer inconsistency. The prediction model we built was based on preoperative CT images, which display the overall characteristics of the lesion, and the model objectively extracts the characteristics of the analyzed image, thereby avoiding the interference associated with subjective assessments and ultimately improving the accuracy of the prediction. Moreover, the risk of complications associated with a biopsy is avoided, and the predictive ability of pathological differentiation of pancreatic adenocarcinoma is improved before surgery.

In order to predict the histological grade of the lesion before surgery, some researchers have attempted to derive increased information from the traditional imaging examinations at the clinics. Elias et al. (41) evaluated the MRIs of 33 patients with pancreatic cancer to determine the degree of tumor differentiation based on the clarity of the margin of the tumor. The results showed that 71.4% of the poorly marginated lesions were well to moderately differentiated, while only 17.6% of focally defined cancers were well to moderately differentiated. In our study, the predictive accuracy of the radiomics signature was 73%, which was based on a larger sample size than the study by Elias et al., and avoided subjective judgment. Some investigators have also used CT perfusion parameters to predict the pathological differentiation of pancreatic cancer (42). The sensitivity and specificity for high-grade pancreatic cancer provided by the assessment of blood flow and blood volume were 79.2% and 82.4%, and 87.5% and 88.2%, respectively. The combination of CT perfusion assessments with radiomics might broaden the application of these techniques and provide improved results.

In addition, CA19-9 is an important tumor marker for pancreatic cancer and is closely related to outcomes of patients with pancreatic cancer (43-45). One study found CA19-9 produced in vitro by a pancreatic cancer cell line to be correlated with the histological grade of differentiation in vivo in nude mice (46). CA19-9 as a continuous variable might be more significant than a categorical variable. However, the data in our study came from multiple medical centers that evaluated patients over a long period of time, and the equipment and protocols were inconsistent. Thus, an accurate comparison between the variables of the internal and external validation sets was impossible. Therefore, we were only able to assess them as categorical variables. This issue needs to be resolved in future studies.

Our study has limitations. First, it only explored the relationship between a radiomics signature and the histological differentiation of PDACs but did not investigate the outcomes of the patients, about which patients and surgeons were more concerned, and for which further study is needed. Second, our study only analyzed the images of patients from the pancreatic parenchymal phase. Although PDAC lesions are well depicted in the pancreatic parenchymal phase, we still need to explore whether the noncontrast phase or portal vein phase will provide better results than the phase we investigated.

Conclusions

The CE-CT-based radiomics signature demonstrated moderate predictive accuracy for differentiating low-grade from high-grade PDAC and should become a new noninvasive method for the preoperative prediction of histological grades of PDAC.

Acknowledgments

We gratefully acknowledge the kind cooperation of Lingling Cui (The First Affiliated Hospital of China Medical University) and Yahong Luo (Liaoning Cancer Institute and Hospital) for providing patients datasets from the two other medical centers.

Funding: None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The study protocol was approved by the Institutional Review Board (IRB) of the Shengjing Hospital ethics committee (No. 2019PS186K). A waiver of informed consent was requested and approved by the IRB.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2018;68:394-424. [Crossref] [PubMed]
2. Vincent A, Herman J, Schulick R, Hruban RH, Goggins M. Pancreatic cancer. Lancet 2011;378:607-20. [Crossref] [PubMed]
3. Wasif N, Ko CY, Farrell J, Wainberg Z, Hines OJ, Reber H, Tomlinson JS. Impact of tumor grade on prognosis in pancreatic cancer: should we include grade in AJCC staging? Ann Surg Oncol 2010;17:2312-20. [Crossref] [PubMed]
4. Rochefort MM, Ankeny JS, Kadera BE, Donald GW, Isacoff W, Wainberg ZA, Hines OJ, Donahue TR, Reber HA, Tomlinson JS. Impact of tumor grade on pancreatic cancer prognosis: validation of a novel TNMG staging system. Ann Surg Oncol 2013;20:4322-9. [Crossref] [PubMed]
5. Stark AP, Sacks GD, Rochefort MM, Donahue TR, Reber HA, Tomlinson JS, Dawson DW, Eibl G, Hines OJ. Long-term survival in patients with pancreatic ductal adenocarcinoma. Surgery 2016;159:1520-7. [Crossref] [PubMed]
6. Mellon EA, Jin WH, Frakes JM, Centeno BA, Strom TJ, Springett GM, Malafa MP, Shridhar R, Hodul PJ, Hoffe SE. Predictors and survival for pathologic tumor response grade in borderline resectable and locally advanced pancreatic cancer treated with induction chemotherapy and neoadjuvant stereotactic body radiotherapy. Acta Oncol 2017;56:391-7. [Crossref] [PubMed]
7. Macías N, Sayagués JM, Esteban C, Iglesias M, González LM, Quiñones-Sampedro J, Gutiérrez ML, Corchete LA, Abad MM, Bengoechea O, Muñoz-Bellvis L. Histologic tumor grade and preoperative bilary drainage are the unique independent prognostic factors of survival in pancreatic ductal adenocarcinoma patients after pancreaticoduodenectomy. J Clin Gastroenterol 2018;52:e11-7. [Crossref] [PubMed]
8. Golan T, Sella T, Margalit O, Amit U, Halpern N, Aderka D, Shacham-Shmueli E, Urban D, Lawrence YR. Short- and long-term survival in metastatic pancreatic adenocarcinoma, 1993-2013. J Natl Compr Canc Netw 2017;15:1022-7. [Crossref] [PubMed]
9. Åkerberg D, Ansari D, Andersson R. Re-evaluation of classical prognostic factors in resectable ductal adenocarcinoma of the pancreas. World J Gastroenterol 2016;22:6424-33. [Crossref] [PubMed]
10. Garcea G, Dennison AR, Pattenden CJ, Neal CP, Sutton CD, Berry DP. Survival following curative resection for pancreatic ductal adenocarcinoma. A systematic review of the literature. JOP 2008;9:99-132. [PubMed]
11. Liu L, Xu HX, He M, Wang W, Wang WQ, Wu CT, Wei RQ, Liang Y, Gao HL, Liu C, Xu J, Long J, Ni QX, Shao CH, Wang J, Yu XJ. A novel scoring system predicts postsurgical survival and adjuvant chemotherapeutic benefits in patients with pancreatic adenocarcinoma: implications for AJCC-TNM staging. Surgery 2018;163:1280-94. [Crossref] [PubMed]
12. Eloubeidi MA, Tamhane A, Varadarajulu S, Wilcox CM. Frequency of major complications after EUS-guided FNA of solid pancreatic masses: a prospective evaluation. Gastrointest Endosc 2006;63:622-9. [Crossref] [PubMed]
13. Gerlinger M, Rowan AJ, Horswell S, Math M, Larkin J, Endesfelder D, Gronroos E, Martinez P, Matthews N, Stewart A, Tarpey P, Varela I, Phillimore B, Begum S, McDonald NQ, Butler A, Jones D, Raine K, Latimer C, Santos CR, Nohadani M, Eklund AC, Spencer-Dene B, Clark G, Pickering L, Stamp G, Gore M, Szallasi Z, Downward J, Futreal PA, Swanton C. Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. N Engl J Med 2012;366:883-92. [Crossref] [PubMed]
14. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, van Stiphout RG, Granton P, Zegers CM, Gillies R, Boellard R, Dekker A, Aerts HJ. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 2012;48:441-6. [Crossref] [PubMed]
15. Huang TM, Kecman V. Gene extraction for cancer diagnosis by support vector machines--an improvement. Artif Intell Med 2005;35:185-94. [Crossref] [PubMed]
16. Eilaghi A, Baig S, Zhang Y, Zhang J, Karanicolas P, Gallinger S, Khalvati F, Haider MA. CT texture features are associated with overall survival in pancreatic ductal adenocarcinoma - a quantitative analysis. BMC Med Imaging 2017;17:38. [Crossref] [PubMed]
17. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:420-8. [Crossref] [PubMed]
18. Huang X, Cheng Z, Huang Y, Liang C, He L, Ma Z, Chen X, Wu X, Li Y, Liang C, Liu Z. CT-based radiomics signature to discriminate high-grade from low-grade colorectal adenocarcinoma. Acad Radiol 2018;25:1285-97. [Crossref] [PubMed]
19. Zhang X, Xu X, Tian Q, Li B, Wu Y, Yang Z, Liang Z, Liu Y, Cui G, Lu H. Radiomics assessment of bladder cancer grade using texture features from diffusion-weighted imaging. J Magn Reson Imaging 2017;46:1281-8. [Crossref] [PubMed]
20. Wu M, Tan H, Gao F, Hai J, Ning P, Chen J, Zhu S, Wang M, Dou S, Shi D. Predicting the grade of hepatocellular carcinoma based on non-contrast-enhanced MRI radiomics signature. Eur Radiol 2019;29:2802-11. [Crossref] [PubMed]
21. Chen X, Fang M, Dong D, Wei X, Liu L, Xu X, Jiang X, Tian J, Liu Z. A radiomics signature in preoperative predicting degree of tumor differentiation in patients with non-small cell lung cancer. Acad Radiol 2018;25:1548-55. [Crossref] [PubMed]
22. Hwan-Ho Cho. Hyunjin Park. Classification of low-grade and high-grade glioma using multi-modal image radiomics features. Conf Proc IEEE Eng Med Biol Soc 2017;2017:3081-4. [PubMed]
23. Bosman FT, Carneiro F, Hruban RH, Theise ND. WHO classification of tumours of the digestive system. World Health Organization, 2010.
24. Zhang L, Fried DV, Fave XJ, Hunter LA, Yang J, Court LE. IBEX: an open infrastructure software platform to facilitate collaborative work in radiomics. Med Phys 2015;42:1341-53. [Crossref] [PubMed]
25. Soh LK, Tsatsoulis C. Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices. IEEE Transactions on Geoscience and Remote Sensing 1999;37:780-95. [Crossref]
26. Haralick RM, Shanmugam K, Dinstein IH. Textural features for image classification. IEEE Transactions on Systems, Man, and Cybernetics 1973;SMC-3:610-21. [Crossref]
27. Haralick RM, Shapiro LG. Computer and robot vision. Reading: Addison-Wesley Longman Publishing Company, 1992.
28. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, Bussink J, Monshouwer R, Haibe-Kains B, Rietveld D, Hoebers F, Rietbergen MM, Leemans CR, Dekker A, Quackenbush J, Gillies RJ, Lambin P. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 2014;5:4006. [Crossref] [PubMed]
29. Tang X. Texture information in run-length matrices. IEEE Transactions on Image Processing 1998;7:1602-9. [Crossref] [PubMed]
30. Amadasun M, King R. Textural Features Corresponding to Texural Properties. IEEE Transactions on Systems, Man and Cybernetics 1989;19:1264-74. [Crossref]
31. Basu S. Developing predictive models for lung tumor analysis. 2012. Available online: https://scholarcommons.usf.edu/etd/3963
32. Li Q, Bai H, Chen Y, Sun Q, Liu L, Zhou S, Wang G, Liang C, Li ZC. A fully-automatic multiparametric radiomics model: towards reproducible and prognostic imaging signature for prediction of overall survival in glioblastoma multiforme. Sci Rep 2017;7:14331. [Crossref] [PubMed]
33. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162:W1-73. [Crossref] [PubMed]
34. Choi MH, Lee YJ, Yoon SB, Choi JI, Jung SE, Rha SE. MRI of pancreatic ductal adenocarcinoma: texture analysis of T2-weighted images for predicting long-term outcome. Abdom Radiol (NY) 2019;44:122-30. [Crossref] [PubMed]
35. Cassinotto C, Chong J, Zogopoulos G, Reinhold C, Chiche L, Lafourcade JP, Cuggia A, Terrebonne E, Dohan A, Gallix B. Resectable pancreatic adenocarcinoma: role of CT quantitative imaging biomarkers for predicting pathology and patient outcomes. Eur J Radiol 2017;90:152-8. [Crossref] [PubMed]
36. Foley KG, Shi Z, Whybra P, Kalendralis P, Larue R, Berbee M, Sosef MN, Parkinson C, Staffurth J, Crosby TDL, Roberts SA, Dekker A, Wee L, Spezi E. External validation of a prognostic model incorporating quantitative PET image features in oesophageal cancer. Radiother Oncol 2019;133:205-12. [Crossref] [PubMed]
37. Kang D, Park JE, Kim YH, Kim JH, Oh JY, Kim J, Kim Y, Kim ST, Kim HS. Diffusion radiomics as a diagnostic model for atypical manifestation of primary central nervous system lymphoma: development and multicenter external validation. Neuro Oncol 2018;20:1251-61. [Crossref] [PubMed]
38. Leijenaar RT, Carvalho S, Hoebers FJ, Aerts HJ, van Elmpt WJ, Huang SH, Chan B, Waldron JN, O'Sullivan B, Lambin P. External validation of a prognostic CT-based radiomic signature in oropharyngeal squamous cell carcinoma. Acta Oncol 2015;54:1423-9. [Crossref] [PubMed]
39. Lucia F, Visvikis D, Vallieres M, Desseroit MC, Miranda O, Robin P, Bonaffini PA, Alfieri J, Masson I, Mervoyer A, Reinhold C, Pradier O, Hatt M, Schick U. External validation of a combined PET and MRI radiomics model for prediction of recurrence in cervical cancer patients treated with chemoradiotherapy. Eur J Nucl Med Mol Imaging 2019;46:864-77. [Crossref] [PubMed]
40. Larghi A, Correale L, Ricci R, Abdulkader I, Monges G, Iglesias-Garcia J, Giovannini M, Attili F, Vitale G, Hassan C, Costamagna G, Rindi G. Interobserver agreement and accuracy of preoperative endoscopic ultrasound-guided biopsy for histological grading of pancreatic cancer. Endoscopy 2015;47:308-14. [PubMed]
41. Elias J Jr, Semelka RC, Altun E, Tsurusaki M, Pamuklar E, Zapparoli M, Voultsinos V, Armao DM, Rubinas T. Pancreatic cancer: correlation of MR findings, clinical features, and tumor grade. J Magn Reson Imaging 2007;26:1556-63. [Crossref] [PubMed]
42. Kovač JD, Đurić-Stefanović A, Dugalić V, Lazić L, Stanisavljević D, Galun D, Mašulović D. CT perfusion and diffusion-weighted MR imaging of pancreatic adenocarcinoma: can we predict tumor grade using functional parameters? Acta Radiol 2019;60:1065-73. [Crossref] [PubMed]
43. Williams JL, Kadera BE, Nguyen AH, Muthusamy VR, Wainberg ZA, Hines OJ, Reber HA, Donahue TR. CA19-9 normalization during pre-operative treatment predicts longer survival for patients with locally progressed pancreatic cancer. J Gastrointest Surg 2016;20:1331-42. [Crossref] [PubMed]
44. Chen Y, Shao Z, Chen W, Xie H, Wu Z, Qin G, Zhao N. A varying-coefficient cox model for the effect of CA19-9 kinetics on overall survival in patients with advanced pancreatic cancer. Oncotarget 2017;8:29925-34. [PubMed]
45. Ni XG, Bai XF, Mao YL, Shao YF, Wu JX, Shan Y, Wang CF, Wang J, Tian YT, Liu Q, Xu DK, Zhao P. The clinical value of serum CEA, CA19-9, and CA242 in the diagnosis and prognosis of pancreatic cancer. Eur J Surg Oncol 2005;31:164-9. [Crossref] [PubMed]
46. Iwamura T, Taniguchi S, Kitamura N, Yamanari H, Kojima A, Hidaka K, Setoguchi T, Katsuki T. Correlation between CA19-9 production in vitro and histological grades of differentiation in vivo in clones isolated from a human pancreatic cancer cell line (SUIT-2). J Gastroenterol Hepatol 1992;7:512-9. [Crossref] [PubMed]