Identification of lesion activities in haptic cystic echinococcosis using machine learning model based on radiomics and clinical features

doi:10.12140/j.issn.1000-7423.2024.05.004

Abstract

Abstract:

Objective To develop machine learning models utilizing radiomic and clinical features to precisely identify the biological activity of haptic cystic echinococcosis (HCE). Methods The CT images and clinical data of 521 HCE patients treated at the Hepatobiliary and Pancreatic Surgery Department of Qinghai University Affiliated Hospital, along with 236 HCE patients treated at the General Surgery Departments of Guoluo Prefectural People’s Hospital and Yushu Prefectural People’s Hospital in 2018-2022, were collected. Radiomics features were extracted and screened accordingly. Univariate and multivariate logistic regression analyses were performed on the clinical data to select features for model construction. To construct radiomics and clinical models, seven machine learning algorithms were employed including Logistic Regression (LR), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Random Forest (RandomForest), Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), and Extra Trees. A clinical-image combined model was constructed based on the prediction from radiomics model combining clinical model, using soft voting method. DeLong’s test was used to compare the performances of the radiomics model, clinical model, and combined clinical-imaging model. In addition, external validation was utilized to assess the model’s performance. Results A total of 430 patients were included for model development and training, while 171 patients were designated for external validation. Fifty-one radiomics features and five clinical features were selected for model construction. Among the seven machine learning models, the XGBoost algorithm demonstrated the best performance, achieving area under the curve (AUC) values of 0.977 [95% confidence interval (CI): 0.964-0.990] and 0.839 (95% CI: 0.776-0.901) on the training and external validation sets, respectively. The radiomics model achieved AUC values of 0.998 (95% CI: 0.997-1.000) and 0.874 (95% CI: 0.822-0.927), while the combined model obtained AUC values of 1.000 (95% CI: 0.999-1.000) and 0.931 (95% CI: 0.894-0.968). The DeLong test results indicated that the performance of the combined model was superior to that of the clinical model in the training set (Z = 2.154, P < 0.05) and showed no statistically significant difference when compared to the radiomics model (Z = 0.562, P > 0.05); however, its performance on the external validation set was better than both the clinical and radiomics models (Z = 3.338, 3.331; P < 0.05). Calibration plots and decision curve analysis (DCA) indicated that the combined model exhibited the best calibration performance in both the training and external validation sets, yielding the highest net benefit, demonstrating consistent performance across different datasets, and displaying good generalizability and reliability in external validation. Conclusion The machine learning model, developed based on radiomic and clinical data, can precisely identify the biological activity of HCE lesions. The combined model exhibits higher diagnostic accuracy and clinical application potential, providing reference for making treatment plan for HCE patients.

Key words: Haptic cystic echinococcosis, Lesion activity, Machine learning model, Radiomics, Clinical features

CLC Number:

R532.32

WANG Zhanjin, CHEN Zhiheng, LI Fuyuan, CAI Junjie, XUE Zhangtuo, ZHOU Ying, CAO Yuntai, WANG Zhan. Identification of lesion activities in haptic cystic echinococcosis using machine learning model based on radiomics and clinical features[J]. CHINESE JOURNAL OF PARASITOLOGY AND PARASITIC DISEASES, 2024, 42(5): 582-593.

Figures/Tables 11

Table 1

Univariate and multivariate logistic regression analysis of the activity of the lesions and the basic information and clinical characteristics of the HCE patients

临床特征 Clinical features	无活性 Inactive （n = 272）	有活性 Active （n = 158）	单因素Logistic回归 Univariatelogistic regression analysis		多因素Logistic回归 Multivariate logistic regression analysis
临床特征 Clinical features	无活性 Inactive （n = 272）	有活性 Active （n = 158）	优势比（95% CI） OR（95% CI）	P	优势比（95% CI）OR（95% CI）	P
年龄/岁 Age/Year	48.0 (40.0-57.0）	41.5 (34.0-49.0）	0.991（0.988~0.933）	< 0.05	0.992（0.990-0.995）	< 0.05
性别/例 Gender/case			1.070（0.991~1.156）	> 0.05
男 Male	123（45.2%）^a	83（52.5%）^a
女 Famale	149（54.8%）^a	75（47.5%）^a
病灶位置/例 Lesion location/case			0.989（0.946~1.034）	> 0.05
肝右叶 Right liver	165（60.7%）^a	92（58.2%）^a
肝左叶 Left liver	75（27.6%）^a	34（21.5%）^a
肝左右叶 Both liver	32（11.8%）^a	32（20.3%）^a
病灶数量/例 No. lesion/case			1.072（0.991~1.160）	> 0.05
单发 Single lesion	171（62.9%）	88（55.7%）
多发 Multiple lesions	101（37.1%）	70（44.3%）
病灶最大直径/cm Lesion max diameter/cm	5.6（4.4-7.3）	7.9（5.9-9.9）	1.064（1.049~1.079）	< 0.05	1.049（1.036-1.062）	< 0.05
红细胞/ × 10⁹L Red blood cells/ × 10⁹L	6.1（5.2-7.2）	6.2（5.0-7.6）	1.211（1.141~1.284）	< 0.05	1.297（1.219-1.379）	< 0.05
白细胞/ × 10⁹L White blood cells/ × 10⁹L	4.6（4.2-5.0）	5.0（4.5-5.4）	1.018（1.000~1.037）	> 0.05
血红蛋白/ × 10⁹L Hemoglobin/ × 10⁹L	153.0（144.0-162.0）	151.0（133.0-163.0）	0.997（0.996~0.999）	< 0.05	0.995（0.993-0.997）	< 0.05
淋巴细胞/ × 10⁹L Lymphocytes/ × 10⁹L	1.8（1.5-2.1）	1.8（1.4-2.3）	1.047（0.984~1.114）	> 0.05
中性粒细胞/ × 10⁹L Neutrophils/ × 10⁹L	3.3（2.7-4.0）	3.5（2.7-4.7）	0.911（0.807~1.029）	> 0.05
单核细胞/ × 10⁹L Monocytes/ × 10⁹L	0.4（0.3-0.5）	0.4（0.3-0.4）	1.007（0.996~1.160）	> 0.05
血小板/ × 10⁹L Platelets/ × 10⁹L	226.0（185.5-256.0）	232.5（196.0-279.0）	1.001（1.000~1.001）	< 0.05	1.000（0.999-1.000）	> 0.05
丙氨酸转氨酶/U·L^-1 Alanine aminotransferase/U·L^-1	34.0（21.0-54.0）	27.0（18.0-44.0）	1.000（0.999~1.000）	> 0.05
总胆红素/U·L^-1 Total bilirubin/μmol U·L^-1	10.2（7.5-14.3）	10.9（7.5-14.7）	1.001（1.000~1.002）	< 0.05	1.001（1.000-1.002）	> 0.05
直接胆红素/U·L^-1 Direct bilirubin/μmol U·L^-1	3.4（2.6-4.6）	4.0（2.9-5.5）	1.001（1.000~1.003）	> 0.05
间接胆红素/U·L^-1 Indirect bilirubin/μmol U·L^-1	6.6（4.2-10.2）	6.2（4.2-8.9）	0.996（0.999~1.000）	> 0.05
总蛋白/g·L^-1 Total protein/g·L^-1	69.5（66.4-71.6）	68.6（65.0-72.0）	0.998（1.000~1.037）	> 0.05
白蛋白/g·L^-1 Albumin/g·L^-1	40.1（37.7-43.5）	39.6（37.1-42.1）	0.985（0.977~0.993）	< 0.05	0.988（0.981-0.995）	< 0.05
碱性磷酸酶/U·L^-1 Alkaline phosphatase/U·L^-1	91.5（68.0-122.5）	97.0（74.0-140.0）	1.000（1.000~1.000）	> 0.05
天冬氨酸转氨酶/U·L^-1 Aspartate aminotransferase/U·L^-1	25.0（18.0-38.5）	24.0（20.0-35.0）	1.000（0.999~1.001）	> 0.05
凝血酶原时间/s Prothrombin time/s	11.1（10.5-11.8）	10.9（10.4-11.7）	1.018（0.987~1.050）	> 0.05
国际标准化比率 International normalized ratio	0.9（0.9-1.0）	0.9（0.9-1.0）	1.414（0.978~1.114）	> 0.05
D-二聚体/µg·L^-1 D-Dimer/µg·L^-1	0.7（0.5-0.9）	0.6（0.4-0.9）	1.010（0.985~1.035）	> 0.05

Table 1

Fig. 1

Fig. 2

Table 2

The final selected 51 radiomics features

类型 Type	影像特征 Radiomics features
灰度共生矩阵（n = 9） Gray level co-occurrence matrix (n = 9)	gradient_glcm_InverseVariance、lbp_3D_m1_glcm_ClusterShade、lbp_3D_m2_glcm_ClusterShade、wavelet_HLH_glcm_Correlation、wavelet_HLL_glcm_Correlation、wavelet_LHL_glcm_Correlation、wavelet_LHL_glcm_Imc2、wavelet_LLH_glcm_Imc2、wavelet_LLL_glcm_MaximumProbability
形状特征（n = 2） Shape features (n = 2)	original_shape_MinorAxisLength、original_shape_Sphericity
灰度运行长度矩阵（n = 6） Gray level run length matrix(n = 6)	exponential_glrlm_GrayLevelNonUniformity、exponential_glrlm_RunVariance、lbp_3D_k_glrlm_RunVariance、lbp_3D_m1_glrlm_ShortRunLowGrayLevelEmphasis、lbp_3D_m2_glrlm_RunVariance、wavelet_HHH_glrlm_ShortRunLowGrayLevelEmphasis
灰度区域大小矩阵（n = 16） Gray level size zone matrix (n = 16)	exponential_glszm_GrayLevelNonUniformity、exponential_glszm_GrayLevelNonUniformityNormalized、exponential_glszm_ZoneEntropy、lbp_3D_k_glszm_GrayLevelNonUniformityNormalized、lbp_3D_k_glszm_GrayLevelVariance、lbp_3D_k_glszm_SmallAreaEmphasis、lbp_3D_k_glszm_SmallAreaLowGrayLevelEmphasis、lbp_3D_k_glszm_ZoneEntropy、lbp_3D_m1_glszm_GrayLevelNonUniformityNormalized、lbp_3D_m1_glszm_SmallAreaEmphasis、lbp_3D_m2_glszm_GrayLevelVariance、lbp_3D_m2_glszm_SmallAreaLowGrayLevelEmphasis、 log_sigma_2_0_mm_3D_glszm_LargeAreaHighGrayLevelEmphasis、original_glszm_SmallAreaHighGrayLevelEmphasis、wavelet_HHL_glszm_SmallAreaLowGrayLevelEmphasis、wavelet_LLH_glszm_SmallAreaLowGrayLevelEmphasis
灰度依赖矩阵（n = 6） Gray level dependence matrix (n = 6)	lbp_3D_k_gldm_DependenceEntropy、lbp_3D_k_gldm_DependenceVariance、lbp_3D_m1_gldm_SmallDependenceEmphasis、wavelet_HHL_gldm_LargeDependenceLowGrayLevelEmphasis、wavelet_LHH_gldm_LargeDependenceHighGrayLevelEmphasis、wavelet_LHL_gldm_LargeDependenceHighGrayLevelEmphasis
邻域灰度差矩阵（n = 6）Neighborhood gray-tone difference matrix (n = 2)	lbp_3D_m2_ngtdm_Complexity、log_sigma_2_0_mm_3D_ngtdm_Busyness、wavelet_HHH_ngtdm_Busyness、wavelet_LHH_ngtdm_Busyness、wavelet_LLH_ngtdm_Complexity、wavelet_LLL_ngtdm_Complexity
一阶统计特征（n = 6） First-order statistics features (n = 6)	lbp_3D_k_firstorder_Minimum、lbp_3D_m1_glszm_GrayLevelNonUniformityNormalized、lbp_3D_m2_glszm_GrayLevelVariance、wavelet_HLH_firstorder_Median、wavelet_LLH_firstorder_Skewness、wavelet_LLL_firstorder_10Percentile

Table 2

Table 3

Fig. 3

Table 4

Table 5

Table 6

Fig. 4

Fig. 5

References 40

[1]	Wen H, Vuitton L, Tuxun T, et al. Echinococcosis: advances in the 21st century[J]. Clin Microbiol Rev, 2019, 32(2): e00075-18.
[2]	Deplazes P, Rinaldi L, Alvarez Rojas CA, et al. Global distribution of alveolar and cystic echinococcosis[J]. Adv Parasitol, 2017, 95: 315-493.
[3]	Tamarozzi F, Akhan O, Cretu CM, et al. Prevalence of abdominal cystic echinococcosis in rural Bulgaria, romania, and Turkey: a cross-sectional, ultrasound-based, population study from the HERACLES project[J]. Lancet Infect Dis, 2018, 18(7): 769-778.
[4]	Rinaldi F, Brunetti E, Neumayr A, et al. Cystic echinococcosis of the liver: a primer for hepatologists[J]. World J Hepatol, 2014, 6(5): 293-305.
[5]	Chinese Doctor Association, Chinese College of Surgeons (CCS), Chinese Committee for Hadytidology (CCH). Expert consensus on diagnosis and treatment of hepatic cystic and alveolar echinococcosis (2019 edition)[J]. Chin J Dig Surg, 2019, 18(8): 711-721. (in Chinese)
	(中国医师协会外科医师分会包虫病外科专业委员会. 肝两型包虫病诊断与治疗专家共识(2019版)[J]. 中华消化外科杂志, 2019, 18(8): 711-721.)
[6]	Li J, Peng XY, Yang HQ, et al. Study of different type of hepatic hydatid cyst internal environment[J]. J Nongken Med, 2014, 36(2): 97-100. (in Chinese)
	(李江, 彭心宇, 杨宏强, 等. 不同分型的囊型肝包虫囊内环境的研究[J]. 农垦医学, 2014, 36(2): 97-100.)
[7]	McManus DP, Zhang W, Li J, et al. Echinococcosis[J]. Lancet, 2003, 362(9392): 1295-1304.
[8]	WHO Informal Working Group on Echinococcosis. Guidelines for treatment of cystic and alveolar echinococcosis in humans[J]. Bull World Health Organ, 1996, 74(3): 231-242.
[9]	Menezes Silva A. Hydatid cyst of the liver-criteria for the selection of appropriate treatment[J]. Acta Trop, 2003, 85(2): 237-242.
[10]	Apaer S, Ma HZ, Li T, et al. Prognostic value of plasma IL-27 on biological viability of hepatic cystic echinococcosis[J]. Int J Infect Dis, 2021, 109: 63-71.
[11]	Cao Y. The imaging diagnosis of echinococcosis: recent advances[J]. J Pract Radiol, 2018, 34(9): 1461-1464. (in Chinese)
	(曹源. 包虫病的影像学诊断进展[J]. 实用放射学杂志, 2018, 34(9): 1461-1464.)
[12]	Piccoli L, Meroni V, Genco F, et al. Serum cytokine profile by ELISA in patients with echinococcal cysts of the liver: a stage-specific approach to assess their biological activity[J]. Clin Dev Immunol, 2012, 2012: 483935.
[13]	Ciftci TT, Yabanoglu-Ciftci S, Unal E, et al. Metabolomic profiling of active and inactive liver cystic echinococcosis[J]. Acta Trop, 2021, 221: 105985.
[14]	Conchedda M, Caddori A, Caredda A, et al. Degree of calcification and cyst activity in hepatic cystic echinococcosis in humans[J]. Acta Trop, 2018, 182: 135-143.
[15]	Yasen A, Li WD, Aini A, et al. Th1/Th2/Th17 cytokine profile in hepatic cystic Echinococcosis patients with different cyst stages[J]. Parasite Immunol, 2021, 43(7): e12839.
[16]	Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data[J]. Radiology, 2016, 278(2): 563-577.
[17]	Polidori T, de Santis D, Rucci C, et al. Radiomics applications in cardiac imaging: a comprehensive review[J]. Radiol Med, 2023, 128(8): 922-933.
[18]	Fan M, Wang KL, Pan D, et al. Radiomic analysis reveals diverse prognostic and molecular insights into the response of breast cancer to neoadjuvant chemotherapy: a multicohort study[J]. J Transl Med, 2024, 22(1): 637.
[19]	Chen M, Copley SJ, Viola P, et al. Radiomics and artificial intelligence for precision medicine in lung cancer treatment[J]. Semin Cancer Biol, 2023, 93: 97-113.
[20]	Kang WD, Qiu X, Luo YG, et al. Application of radiomics-based multiomics combinations in the tumor microenvironment and cancer prognosis[J]. J Transl Med, 2023, 21(1): 598.
[21]	Xu WY, Zhang TL, Xia YW, et al. Predicting the surrounding zone of hepatic alveolar echinococcosis based on CT machine learning[J]. Chin Comput Med Imag, 2022, 28(3): 286-290. (in Chinese)
	(许文瑶, 张铁亮, 夏雨薇, 等. 基于CT影像的机器学习模型术前预测肝脏泡型包虫病边缘带浸润[J]. 中国医学计算机成像杂志, 2022, 28(3): 286-290.)
[22]	Yu YH, Liu WY, Zhao Y, et al. The value of CT imaging in distinguishing pulmonary cystic echinococcosis from pulmonary abscess[J]. J Clin Radiol, 2022, 41(11): 2041-2045. (in Chinese)
	(郁耀辉, 刘文亚, 赵圆, 等. CT影像组学鉴别肺囊性包虫病与肺脓肿的价值[J]. 临床放射学杂志, 2022, 41(11): 2041-2045.)
[23]	Hou J, Wen H, Wang MK, et al. Analysis of the influencing factors of lesion activity in hepatic cystic echinococcosis patients[J]. Chin J Parasitol Parasit Dis, 2022, 40(3): 309-315. (in Chinese)
	(侯娇, 温浩, 王明坤, 等. 肝细粒棘球蚴病手术患者病灶活性状态的影响因素分析[J]. 中国寄生虫学与寄生虫病杂志, 2022, 40(3): 309-315.)
[24]	Hosmer Jr DW, Lemeshow S, Sturdivant RX. Applied logistic regression[M]. Hoboken: John Wiley & Sons, 2013: 1-35.
[25]	Scholkopf B, Smola AJ. Learning with kernels: support vector machines, regularization, optimization, and beyond[J]. IEEE Trans Neural Netw, 2005, 16(3): 781.
[26]	Cover T, Hart P. Nearest neighbor pattern classification[J]. IEEE Trans Inf Theory, 1967, 13(1): 21-27.
[27]	Breiman L. Random forests[J]. Mach Learn, 2001, 45(1): 5-32.
[28]	Ogunleye A, Wang QG. XGBoost model for chronic kidney disease diagnosis[J]. IEEE/ACM Trans Comput Biol Bioinform, 2020, 17(6): 2131-2140.
[29]	GuolinKe QM, Finley T, Wang T, et al. Lightgbm: a highly efficient gradient boosting decision tree[C]. Cambridge: Adv Neural Inf Process Syst, 2017: 1-9.
[30]	Geurts P, Ernst D, Wehenkel L. Extremely randomized trees[J]. Mach Learn, 2006, 63(1): 3-42.
[31]	Pakala T, Molina M, Wu GY. Hepatic echinococcal cysts: a review[J]. J Clin Transl Hepatol, 2016, 4(1): 39-46.
[32]	Wen H, Xu MQ. Practical echinococcosis[M]. Beijing: Science Press, 2007: 76-154. (in Chinese)
	(温浩, 徐明谦. 实用包虫病学[M]. 北京: 科学出版社, 2007: 76-154.)
[33]	Shi R, Qiao F, Song J, et al. The value of MR IVIM-DWI in predicting the biological activity of hepatic cystic echinococcosis[J]. J Pract Radiol, 2020, 36(5): 826-830. (in Chinese)
	(石睿, 乔飞, 宋娟, 等. 磁共振体素内不相干运动扩散加权成像预测囊性肝包虫生物学活性的价值[J]. 实用放射学杂志, 2020, 36(5): 826-830.)
[34]	Li T, Bao HH. Evaluation of computer tomography and magnetic resonance imaging in the classification and viability of hepatic cystic echinococcosis[J]. Chin J Magn Reson Imag, 2021, 12(5): 25-29. (in Chinese)
	(李婷, 鲍海华. CT和MRI对肝囊型包虫病分型与活性的评价[J]. 磁共振成像, 2021, 12(5): 25-29.)
[35]	Bhutani N, Kajal P. Hepatic echinococcosis: a review[J]. Ann Med Surg, 2018, 36: 99-105.
[36]	Joliat GR, Melloul E, Petermann D, et al. Outcomes after liver resection for hepatic alveolar echinococcosis: a single-center cohort study[J]. World J Surg, 2015, 39(10): 2529-2534.
[37]	Balli O, Balli G, Cakir V, et al. Percutaneous treatment of giant cystic echinococcosis in liver: catheterization technique in patients with CE1 and CE3a[J]. Cardiovasc Intervent Radiol, 2019, 42(8): 1153-1159.
[38]	Cheng XY, Feng ZC, Pan BY, et al. Establishment and application of the BRP prognosis model for idiopathic pulmonary fibrosis[J]. J Transl Med, 2023, 21(1): 805.
[39]	He JX, Wang B, Tao JS, et al. Accurate classification of pulmonary nodules by a combined model of clinical, imaging, and cell-free DNA methylation biomarkers: a model development and external validation study[J]. Lancet Digit Health, 2023, 5(10): e647-e656.
[40]	Zhu ZC, Chen MJ, Hu G, et al. A pre-treatment CT-based weighted radiomic approach combined with clinical characteristics to predict durable clinical benefits of immunotherapy in advanced lung cancer[J]. Eur Radiol, 2023, 33(6): 3918-3930.

模型 Model	参数 Parameter
逻辑回归 LR	LogisticRegression (penalty = ‘l1’, solver = ‘saga’, max_iter = 1, random_state = 0）
支持向量机 SVM	SVC (kernel = ‘linear’, C = 0.1, probability = True, random_state = 0）
K-近邻 KNN	KNeighborsClassifier (algorithm = ‘kd_tree’, n_neighbors = 5）
随机森林 RandomForest	RandomForestClassifier (n_estimators = 10, max_depth = 3, min_samples_split = 4, random_state = 0）
极限梯度提升 XGBoost	XGBClassifier (n_estimators = 10, objective = ‘binary:logistic’, max_depth = 3, min_child_weight = 2, use_label_encoder = False, eval_metric = ‘error’）
轻量梯度提升 LightGBM	LGBMClassifier (n_estimators = 10, max_depth = 3, min_child_weight = 0.5）
极端随机树 ExtraTrees	ExtraTreesClassifier (n_estimators = 10, max_depth = 3, min_samples_split = 2, random_state = 0）

模型 Model	队列 Cohort	准确率 Accuracy	曲线下面积AUC	95%置信区间 95%CI	灵敏度 Sensitivity	特异度 Specificity	阳性预测值 PPV	阴性预测值 NPV	F1值 F1	阈值 Threshold
逻辑回归 LR	训练集 Train set	0.940	0.983	0.974~0.993	0.943	0.937	0.898	0.966	0.920	0.374
	验证集 Validation set	0.789	0.867	0.813~0.922	0.898	0.675	0.745	0.862	0.814	0.021
支持向量机SVM	训练集 Train set	0.963	0.984	0.974~0.994	0.943	0.974	0.955	0.967	0.949	0.464
支持向量机SVM	验证集 Validation set	0.807	0.852	0.792~0.912	0.795	0.819	0.824	0.791	0.809	0.225
K-近邻 KNN	训练集 Train set	0.921	0.975	0.965~0.986	0.835	0.971	0.943	0.910	0.886	0.500
	验证集 Validation set	0.789	0.864	0.810~0.917	0.739	0.843	0.833	0.753	0.783	0.250
随机森林 RandomForest	训练集 Train set	0.926	0.971	0.956~0.986	0.956	0.908	0.858	0.972	0.904	0.418
随机森林 RandomForest	验证集 Validation set	0.795	0.816	0.751~0.882	0.693	0.904	0.884	0.735	0.777	0.460
极限梯度提升 XGBoost	训练集 Train set	0.981	0.998	0.997~1.000	0.968	0.989	0.981	0.982	0.975	0.483
极限梯度提升 XGBoost	验证集 Validation set	0.813	0.874	0.822~0.927	0.841	0.783	0.804	0.823	0.822	0.190
轻量梯度提升LightGBM	训练集 Train set	0.921	0.984	0.976~0.992	0.956	0.901	0.848	0.972	0.899	0.346
轻量梯度提升LightGBM	验证集 Validation set	0.813	0.868	0.815~0.922	0.863	0.759	0.792	0.840	0.812	0.282
极端随机树 ExtraTrees	训练集 Train set	0.916	0.964	0.948~0.980	0.930	0.908	0.855	0.957	0.891	0.420
极端随机树 ExtraTrees	验证集 Validation set	0.784	0.870	0.818~0.922	0.636	0.940	0.918	0.709	0.752	0.467

模型 Model	队列 Cohort	临床模型 Clinical Models		影像模型 Radiomics Models		联合模型 Combined Models
模型 Model	队列 Cohort	曲线下面积 AUC	准确率 Accuracy	曲线下面积 AUC	准确率 Accuracy	曲线下面积 AUC	准确率 Accuracy
逻辑回归 LR	训练集 Train set	0.905	0.896	0.983	0.940	0.993	0.986
逻辑回归 LR	验证集 Validation set	0.812	0.813	0.867	0.789	0.886	0.836
支持向量机 SVM	训练集 Train set	0.923	0.904	0.963	0.984	0.987	0.991
支持向量机 SVM	验证集 Validation set	0.814	0.819	0.807	0.852	0.826	0.863
K-近邻 KNN	训练集 Train set	0.861	0.854	0.975	0.921	0.979	0.977
K-近邻 KNN	验证集 Validation set	0.764	0.783	0.864	0.789	0.889	0.816
随机森林 RandomForest	训练集 Train set	0.951	0.921	0.971	0.926	0.988	0.957
随机森林 RandomForest	验证集 Validation set	0.798	0.762	0.816	0.795	0.835	0.826
极度梯度提升 XGBoost	训练集 Train set	0.977	0.916	0.998	0.981	1.000	0.988
极度梯度提升 XGBoost	验证集 Validation set	0.839	0.789	0.874	0.813	0.931	0.871
轻量梯度提升 LightGBM	训练集 Train set	0.895	0.862	0.984	0.921	0.992	0.975
轻量梯度提升 LightGBM	验证集 Validation set	0.789	0.819	0.868	0.813	0.921	0.854
极端梯度树 ExtraTrees	训练集 Train set	0.912	0.919	0.964	0.916	0.978	0.964
极端梯度树 ExtraTrees	验证集 Validation set	0.834	0.824	0.870	0.784	0.905	0.865

模型 Model	队列 Cohort	准确率 Accuracy	曲线下面积AUC	95%置信区间 95% CI	灵敏度 Sensitivity	特异度 Specificity	阳性预测值 PPV	阴性预测值 NPV	F1值 F1	阈值 Threshold
临床模型 Clinical model	训练集 Train set	0.916	0.977	0.964~0.990	0.943	0.901	0.847	0.965	0.892	0.384
影像模型 Radiomics model	训练集 Train set	0.981	0.998	0.997~1.000	0.968	0.989	0.981	0.982	0.975	0.483
联合模型 Combined model	训练集 Train set	0.988	1.000	0.999~1.000	0.987	0.989	0.981	0.993	0.984	0.323
临床模型 Clinical model	验证集 Validation set	0.789	0.839	0.776~0.901	0.955	0.614	0.724	0.927	0.824	0.164
影像模型 Radiomics model	验证集 Validation set	0.813	0.874	0.822~0.927	0.841	0.783	0.804	0.823	0.822	0.190
联合模型 Combined model	验证集 Validation set	0.871	0.931	0.894~0.968	0.920	0.819	0.844	0.907	0.880	0.409