Evaluation of efficacy of visual intelligent recognition model for Oncomelania hupensis based on deep learning technology

doi:10.12140/j.issn.1000-7423.2021.06.006

Abstract

Abstract:

Objective To investigate the performance of a deep learning-based visual intelligent recognition model for the intermediate host of Schistosoma japonicum, the snail Oncomelania hupensis, and evaluate its efficacy in recognition and classification. Methods According to the distribution pattern of O. hupensis in the topographic type of lake marsh, hill and water network in Jiangsu Province, seven regions including Nanjing, Zhenjiang, Yangzhou, Suzhou, Changzhou, Wuxi and Yancheng were selected as the fields for sample collection from March 2019 to October 2020. From each of the fields, 3 snail breeding environment sites were randomly selected for collection of snail samples and images by smart phone. The image quality and morphologic features of the sampled snails were screened and classified by six experts of schistosomiasis control to establish a standard data set of snail image classification. The data set was compiled into training set and test set (comprised of internal and external test). Three major convolutional neural network models including MobilenetV2, ResNet50 and Inception-ResNet-V2 were used to exercise the model training in the training data set. Taking the standard snail image classification data set as the gold standard, the receiver operating characteristic (ROC) curves was applied to compare the sensitivity, specificity, diagnostic concordance rate (Kappa value), and Yorden index of three model types with the internal test data; comparison was performed between the best-fit recognition model and the findings of snail search staff through the external test set, evaluating the accuracy of models in snail recognition with the internal and external test set. Results Totally, 3 224 images of 4 types of snails similar to O. hupensis were collected: Semisulcospira cancellata, Opeas gracile, Euphaedus, and Tricula. After screening, 2 719 images were included into the standard snail image classification data set, among them 774 were of O. hupensis and 1 945 of 4 being similar to but not O. hupensis. The concordance Kappa value from the snail recognition models of MobilenetV2, ResNet50 and Inception-ResNet-V2 model with the gold standard was 0.78, 0.83 and 0.88 respectively. The sensitivity, specificity, accuracy, Yorden index, and area under the ROC curve (AUC) of the Inception-ResNet-V2 were found highest, being 92.00%, 97.16%, 96.13%, 0.89 and 0.95, respectively. There was no statistically significant difference in the sensitivity and specificity among the three models (χ² = 3.892, 4.948, P > 0.05), while the difference in accuracy was statistically significant (χ² = 8.607, P < 0.05). In the external test, the best-fit model Inception-ResNet-V2 and the findings by staff showed better concordance with the gold standard, having the Kappa values of 0.80 and 0.83, and the AUCs 0.88 and 0.92, respectively, of which the difference was not statistically significant (P > 0.05). Conclusion The deep learning-based visual intelligent recognition model for O. hupensis snailshowed evident accuracy.

Key words: Oncomelania hupensis, Intelligent recognition, Deep learning, Computer vision, Machine learning, Schistosomiasis, Surveillance

CLC Number:

R383.241

SHI Liang, XIONG Chun-rong, LIU Mao-mao, WEI Xiu-shen, ZHANG Jian-feng, WANG Xin-yao, WANG Tao, HANG De-rong, YANG Hai-tao, YANG Kun. Evaluation of efficacy of visual intelligent recognition model for Oncomelania hupensis based on deep learning technology[J]. CHINESE JOURNAL OF PARASITOLOGY AND PARASITIC DISEASES, 2021, 39(6): 764-770.

Figures/Tables 7

Fig. 1

Fig. 2

Table 1

Table 2

Fig. 3

Fig. 4

Table 3

References 33

[1]	Liao ZW, Wang SQ. Prevalence and prevention of major tropical diseases in China, 2000—2019[J]. Chin Trop Med, 2020, 20(3): 193-201. (in Chinese)
	(廖志武, 王善青. 我国2000—2019年主要热带病的流行与防治概况[J]. 中国热带医学, 2020, 20(3): 193-201.)
[2]	Deng WC, Li YS, Cheng XH, et al. Implications, spiritual characteristics and practical significance of Chinese schistosomiasis control culture[J]. Chin J Schisto Control, 2020, 32(3): 222-224, 229. (in Chinese)
	(邓维成, 李岳生, 程湘晖, 等. 论中国血防文化的内涵与精神特质及其现实意义[J]. 中国血吸虫病防治杂志, 2020, 32(3): 222-224, 229.)
[3]	Mao SB. The biology of schistosomiasis and the control of schistosomiasis[M]. Beijing: People’s Medical Publishing House, 1990: 699. (in Chinese)
	(毛守白. 血吸虫生物学与血吸虫病的防治[M]. 北京: 人民卫生出版社, 1990: 699.)
[4]	Zhou XN. Science on oncomelania snail[M]. Beijing: Science Press, 2005: 1. (in Chinese)
	(周晓农. 实用钉螺学[M]. 北京: 科学出版社, 2005: 1.)
[5]	Jiang TT, Yang K. Progresses of research on patterns and monitoring approaches of Oncomelania hupensis spread[J]. Chin J Schisto Control, 2020, 32(2): 208-212. (in Chinese)
	(蒋甜甜, 杨坤. 钉螺扩散规律与监测方法研究进展[J]. 中国血吸虫病防治杂志, 2020, 32(2): 208-212.)
[6]	Chinese Academy of Medical Sciences. Problems and improvement opinions on the method of Oncomelania survey[J]. Med Health Exp, 1960, 1: 18-19. (in Chinese)
	(中国医学科学院. 钉螺调查方法上存在的问题和改进意见[J]. 医药卫生快报, 1960, 1: 18-19.)
[7]	Chen M, Peng XW, Zhang HM, et al. Systematic field survey on the snail density by space sectioning method[J]. J Trop Dis Parasitol, 2015, 13(1): 23-25. (in Chinese)
	(陈美, 彭孝武, 张华明, 等. 按钉螺密度分级设定系统抽样查螺间距的探讨[J]. 热带病与寄生虫学, 2015, 13(1): 23-25.)
[8]	LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553): 436-444. doi: 10.1038/nature14539
[9]	He KM, Zhang XY, Ren SQ., et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification [C]//2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015: 1026-1034.
[10]	Carin L, Pencina MJ. On deep learning for medical image analysis[J]. JAMA, 2018, 320(11): 1192. doi: 10.1001/jama.2018.13316
[11]	Chae S, Kwon S, Lee D. Predicting infectious disease using deep learning and big data[J]. Int J Environ Res Public Health, 2018, 15(8): 1596. doi: 10.3390/ijerph15081596
[12]	Wallis C. How artificial intelligence will change medicine[J]. Nature, 2019, 576(7787): S48. doi: 10.1038/d41586-019-03845-1
[13]	National Health and Family Planning Commission of the People’s Republic of China. Survey of oncomelanid snails: WS/T 563—2017[S]. Beijing: China Standard Press, 2017. (in Chinese)
	(中华人民共和国国家卫生和计划生育委员会. 钉螺调查 WS/T 563—2017[S]. 北京: 中国标准出版社, 2017.)
[14]	Liu YY, Zhang WZ, Wang YX. Medical malacology[M]. Beijing: Ocean Press, 1993: 1-157. (in Chinese)
	(刘月英, 张文珍, 王耀先. 医学贝类学[M]. 北京: 海洋出版社, 1993: 1-157.)
[15]	Shi L, Xiong CR, Liu MM, et al. Establishment of a deep learning-visual model for intelligent recognition of Oncomelania hupensis[J]. Chin J Schisto Control, 2021, 33(5): 445-451. (in Chinese)
	(施亮, 熊春蓉, 刘毛毛, 等. 基于深度学习技术的湖北钉螺视觉智能识别模型的建立[J]. 中国血吸虫病防治杂志, 2021, 33(5): 445-451.
[16]	Wei XS. Analyzing deep learning: convolutional neural network principle and vision practice[M]. Beijing: Publishing House of Electronics Industry, 2018. (in Chinese)
	(魏秀参. 解析深度学习: 卷积神经网络原理与视觉实践[M]. 北京: 电子工业出版社, 2018.
[17]	Howard AG, Zhu ML, Chen B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv: 1704.04861, 2017.
[18]	He KM, Zhang XY, Ren SQ, et al. Deep residual learning for image recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[19]	Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, Inception-ResNet and the impact of residual connections on learning[C]. AAAI Conference on Artificial Intelligence, 2016.
[20]	Zhou ZH. Machine learning[M]. Beijing: Tsinghua University Press, 2016: 28-36. (in Chinese)
	(周志华. 机器学习[M]. 北京: 清华大学出版社, 2016: 28-36.)
[21]	Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks[J]. Inf Process Manag, 2009, 45(4): 427-437. doi: 10.1016/j.ipm.2009.03.002
[22]	Zhou YB, Zhao GM. Reliability of measurement and the methods of estimating reliability[J]. Chin J Epidemiol, 2003, 24(12): 1146-1149. (in Chinese)
	(周艺彪, 赵根明. 测量的可靠性及其估计方法[J]. 中华流行病学杂志, 2003, 24(12): 1146-1149.)
[23]	Zhang LJ, Xu ZM, Dang H, et al. Endemic status of schistosomiasis in People’s Republic of China in 2019[J]. Chin J Schisto Control, 2020, 32(6): 551-558. (in Chinese)
	(张利娟, 徐志敏, 党辉, 等. 2019年全国血吸虫病疫情通报[J]. 中国血吸虫病防治杂志, 2020, 32(6): 551-558.)
[24]	Zhang LJ, Zhu HQ, Wang Q, et al. Assessment of schistosomiasis transmission risk along the Yangtze River basin after the flood disaster in 2020[J]. Chin J Schisto Control, 2020, 32(5): 464-468, 475. (in Chinese)
	(张利娟, 祝红庆, 王强, 等. 2020年长江流域洪涝灾害后血吸虫病传播风险分析[J]. 中国血吸虫病防治杂志, 2020, 32(5): 464-468, 475.)
[25]	Zhang YE. Search of snail recognition and counting based on template matching of color and shape[J]. J Gannan Norm Univ, 2012, 33(6): 33-36. (in Chinese)
	(章银娥. 基于颜色和形状的模板匹配的钉螺识别计数研究[J]. 赣南师范学院学报, 2012, 33(6): 33-36.)
[26]	Yan SH, Huang XT. The Oncomelania digital image identification based on SIFT & SVM[J]. J Gannan Norm Univ, 2011, 32(6): 58-61. (in Chinese)
	(严深海, 黄贤通. 基于SIFT与SVM的钉螺数字图像识别[J]. 赣南师范学院学报, 2011, 32(6): 58-61.)
[27]	Wang H. Research on snail image recognition technology based on neural network[D]. Wuhan: Hubei University, 2010: 1-49. (in Chinese)
	(汪浩. 基于神经网络的钉螺图像识别技术研究[D]. 武汉: 湖北大学, 2010: 1-49.)
[28]	Wei XS, Xie CW, Wu JX, et al. Mask-CNN: localizing parts and selecting descriptors for fine-grained bird species categorization[J]. Pattern Recognit, 2018, 76: 704-714. doi: 10.1016/j.patcog.2017.10.002
[29]	Wei XS, Luo JH, Wu J, et al. Selective convolutional descriptor aggregation for fine-grained image retrieval[J]. IEEE Trans Image Process, 2017, 26(6): 2868-2881. doi: 10.1109/TIP.2017.2688133
[30]	Smith KP, Kirby JE. Image analysis and artificial intelligence in infectious disease diagnostics[J]. Clin Microbiol Infect, 2020, 26(10): 1318-1323. doi: 10.1016/j.cmi.2020.03.012
[31]	Shi F, Wang J, Shi J, et al. Review of artificial intelligence techniques in imaging data acquisition, segmentation, and diagnosis for COVID-19[J]. IEEE Rev Biomed Eng, 2021, 14: 4-15. doi: 10.1109/RBME.4664312
[32]	Song J, Gao C, Han Q, et al. Construction and clinical preliminary validation of an automaticbone age assessment model based on deep learning[J]. Chin J Radiol, 2019, 53(11): 974-978. (in Chinese)
	(宋娟, 高畅, 韩青, 等. 基于深度学习的儿童骨龄智能评估模型构建及初步临床验证[J]. 中华放射学杂志, 2019, 53(11): 974-978.)
[33]	Yang F, Poostchi M, Yu H, et al. Deep learning for smartphone-based malaria parasite detection in thick blood smears[J]. IEEE J Biomed Heal Informatics, 2019, 24(5): 1427-1438.

螺类 Snails	训练集 Training set	测试集Test set		合计 Total
螺类 Snails	训练集 Training set	内部测试集 Internal test set	外部测试集 External test set	合计 Total
钉螺 Oncomelania hupensis	594	150	30	774
非钉螺 Non-Oncomelania hupensis	1 316	599	30	1 945
合计Total	1 910	749	60	2 719

模型Model	灵敏度/%Sensitivity/%	特异性/%Specificity/%	准确率/% Accuracy/%	约登指数 Youden index	Kapaa值 Kappa value	ROC曲线下面积 AUC
MobilenetV2	84.67	94.66	92.66^a	0.79	0.78	0.90
Resnet50	88.00	96.16	94.53^ab	0.84	0.83	0.92
Inception-ResNet-V2	92.00	97.16	96.13^a	0.89	0.88	0.95
χ²	3.892	4.948	8.607
P值P value	> 0.05	> 0.05	< 0.05

分组 Group	灵敏度/% Sensitivity/%	特异性/% Specificity/%	准确率/% Accuracy/%	约登指数 Youden index	Kappa值 Kappa value	AUC
工作人员Staff	86.67	96.67	91.67	0.83	0.83	0.92
模型Model	86.67	93.33	90.00	0.80	0.80	0.88