Abstract
To address the issue of complex structure, various sizes and the interclass similarity of different lesions, this paper proposes a dual-input attentive RefineNet (DARNet) for automatic multiple lesion segmentation of diabetic retinopathy. DARNet includes a global image encoder, local image encoder and attention refinement decoder. The whole image and the patch image are used as the dual input and fed into ResNet50 and ResNet101 for down-sampling, respectively. The high-level attention refinement decoder adopts a dual attention mechanism to integrate the same-level features in the two encoders with the output of the low-level attention refinement module for multiscale feature fusion, which focuses the model on the lesion area to generate accurate predictions. We evaluated the segmentation performance of four lesions on three datasets, and the proposed method reached an average accuracy of 0.9582/0.9617/0.9578 and a dice score of 0.9521/0.9637/0.9508 on IDRiD, E-ophtha and DDR. Extensive experimental results demonstrate the proposed DARNet outperforms the state- of-the-art models and has better robustness and accuracy. It not only preserves the contour details and shape features of multiscale lesions, but also overcomes the interference of similar tissues and noises to realize accurate multi-lesion segmentation.
Similar content being viewed by others
Availability of Data and Materials
Data related to the current study are available from the corresponding author on reasonable request.
Code Availability
The codes used during the study are available from the corresponding author by request.
References
Ting DSW, Cheung GCM, Wong TY (2016) Diabetic retinopathy: global prevalence, major risk factors, screening practices and public health challenges: a review. Clin Exper Ophthalmol 44(4):260–277
Yau JWY, Rogers SL, Kawasaki R, Lamoureux EL, Kowalski JW, Bek T, Chen S-J, Dekker JM, Fletcher A, Grauslund J et al (2012) Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care 35(3):556–564
Salamat N, Missen MMS, Rashid A (2019) Diabetic retinopathy techniques in retinal images: A review. Artif Intell Med 97:168–188
Stolte S, Fang R (2020) A survey on medical image analysis in diabetic retinopathy. Med Image Anal 64:101742
Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316(22):2402–2410
Asiri N, Hussain M, Al Adel F, Alzaidi N (2019) Deep learning based computer-aided diagnosis systems for diabetic retinopathy: A survey. Artif Intell Med 99:101701
Zhang W, Dai Y, Liu M, Chen Y, Zhong J, Yi Z (2021) Deepuwf-plus: automatic fundus identification and diagnosis system based on ultrawide-field fundus imaging. Appl Intell 51:7533–7551
Koh JEW, Ng EYK, Bhandary SV, Laude A, Acharya UR (2018) Automated detection of retinal health using phog and surf features extracted from fundus images. Appl Intell 48:1379–1393
Acharya UR, Mookiah MRK, Koh Joel EW, Tan JH, Bhandary SV, Rao AK, Fujita H, Hagiwara Y, Chua CK, Laude A (2016) Automated screening system for retinal health using bi-dimensional empirical mode decomposition and integrated index. Comput Biol Med 75:54–62
Xu Y, Zhou Z, Li X, Zhang N, Zhang M, Wei P (2021) Ffu-net: Feature fusion u-net for lesion segmentation of diabetic retinopathy. BioMed Res Int 2021
Foo A, Hsu W, Lee ML, Lim G, Wong TY (2020) Multi-task learning for diabetic retinopathy grading and lesion segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 13267–13272
Yang Y, Li T, Li W, Wu H, Fan W, Zhang W (2017) Lesion detection and grading of diabetic retinopathy via two-stages deep convolutional neural networks. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 533–540
Sambyal N, Saini P, Syal R, Gupta V (2020) Modified u-net architecture for semantic segmentation of diabetic retinopathy images. Biocybern Biomed Eng 40(3):1094–1109
Guo S, Li T, Kang H, Li N, Zhang Y, Wang K (2019) L-seg: An end-to-end unified framework for multi-lesion segmentation of fundus images. Neurocomputing 349:52–63
Mo J, Zhang L, Feng Y (2018) Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks. Neurocomputing 290:161–171
Porwal P, Pachade S, Kamble R, Kokare M, Deshmukh G, Sahasrabuddhe V, Meriaudeau F (2018) Indian diabetic retinopathy image dataset (idrid): a database for diabetic retinopathy screening research. Data 3(3):25
Decenciere E, Cazuguel G, Zhang X, Thibault G, Klein J-C, Meyer F, Marcotegui B, Quellec G, Lamard M, Danno R et al (2013) Teleophta: Machine learning and image processing methods for teleophthalmology. Irbm 34(2):196–203
Li T, Gao Y, Wang K, Guo S, Liu H, Kang H (2019) Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Inf Sci 501:511–522
Wu B, Zhu W, Shi F, Zhu S, Chen X (2017) Automatic detection of microaneurysms in retinal fundus images. Comput Med Imaging Graph 55:106–112
Long S, Huang X, Chen Z, Pardhan S, Zheng D (2019) Automatic detection of hard exudates in color retinal images using dynamic threshold and svm classification: algorithm development and evaluation. BioMed Res Int 2019
Colomer A, Igual J, Naranjo V (2020) Detection of early signs of diabetic retinopathy based on textural and morphological information in fundus images. Sensors 20(4):1005
Kälviäinen RVJPH, Uusitalo H (2007) Diaretdb1 diabetic retinopathy database and evaluation protocol. In: Medical image understanding and analysis, vol 2007. Citeseer, p 61
Amin J, Sharif M, Yasmin M, Ali H, Fernandes S L (2017) A method for the detection and classification of diabetic retinopathy using structural predictors of bright lesions. J Comput Sci 19:153–164
Zhou Y, He X, Huang L, Liu L, Zhu F, Cui S, Shao L (2019) Collaborative learning of semi-supervised segmentation and classification for medical images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2079–2088
Xue J, Yan S, Qu J, Qi F, Qiu C, Zhang H, Chen M, Liu T, Li D, Liu X (2019) Deep membrane systems for multitask segmentation in diabetic retinopathy. Knowl-Based Syst 183:104887
Tan JH, Fujita H, Sivaprasad S, Bhandary SV, Rao AK, Chua KC, Acharya UR (2017) Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network. Inf Sci 420:66–76
Sun J, Peng Y, Guo Y, Li D (2021) Segmentation of the multimodal brain tumor image used the multi-pathway architecture method based on 3d fcn. Neurocomputing 423:34– 45
Ting DSW, Cheung CY-L, Lim G, Tan GSW, Quang ND, Gan A, Hamzah H, Garcia-Franco R, San Yeo IY, Lee S Y et al (2017) Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. Jama 318(22):2211–2223
Decencière E, Zhang X, Cazuguel G, Lay B, Cochener B, Trone C, Gain P, Ordonez R, Massin P, Erginay A et al (2014) Feedback on a publicly distributed image database: the messidor database. Image Anal Stereol 33(3):231–234
Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403
Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 801–818
Zhou Y, Wang B, Huang L, Cui S, Shao L (2021) A benchmark for studying diabetic retinopathy: Segmentation, grading, and transferability. IEEE Trans Med Imaging 40(3):818–828
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Lin G, Liu F, Milan A, Shen C, Reid I (2019) Refinenet: Multi-path refinement networks for dense prediction. IEEE Trans Pattern Anal Mach Intell 42(5):1228–1242
Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 248–255
Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241
Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations
Acknowledgements
The authors gratefully acknowledge the financial supports by the National Science Foundation of China under Grant number 61976126, as well as the Shandong Nature Science Foundation of China under Grant numbers ZR2019MF003, ZR2020MF132, ZR2020MH291.
Funding
This work was supported in part by the National Natural Science Foundation of China (Grant No. 61976126), Shandong Nature Science Foundation of China (No. ZR2019MF003, ZR2020MF132, ZR2020MH291).
Author information
Authors and Affiliations
Contributions
Yanfei Guo: Conceptualization, Methodology, Software, Writing the original draft. Yanjun Peng: Data curation, Supervision, Funding acquisition, Project administration.
Corresponding author
Ethics declarations
Conflicts of Interest
The authors declare there are no conflicts of interest regarding the publication of this paper.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Guo, Y., Peng, Y. Multiple lesion segmentation in diabetic retinopathy with dual-input attentive RefineNet. Appl Intell 52, 14440–14464 (2022). https://doi.org/10.1007/s10489-022-03204-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-022-03204-0