Skip to main content

Advertisement

SpringerLink
Log in

An Attention-Based Swin U-Net-Based Segmentation and Hybrid Deep Learning Based Diabetic Retinopathy Classification Framework Using Fundus Images

  • Research
  • Published:
Sensing and Imaging Aims and scope Submit manuscript

Abstract

Diabetes mellitus frequently occurs with diabetic retinopathy (DR), which results in lesions mostly in the retinas that damages the vision. Earlier DR identification and treatment can deeply lower the threat of visual loss. Compared to computer-aided diagnosis methods, the manual diagnosing of DR by ophthalmologists using fundus images of the retina leads to high-cost consumption, time consuming, and more effort. This study employs an attention-based deep learning framework for classifying DR using fundus images, and the severity level is estimated to provide better treatment. The required fundus images are obtained through real-world benchmark datasets. Then, the attained images are pre-processed to remove unnecessary distortions in the images. Subsequently, the resultant pre-processed image is sent through the attention-based swin U-net for segmentation, where the affected diseased part is clearly visualized using the segmentation process. Here, the hybrid optimization algorithm named spiral bacterial colony optimization (SBCO) is utilized to optimize the features within the U-net. Furthermore, fundus images are classified using deep structures, where the segmented image is directly undergone to the 3-dimensional convolutional neural network (3D-CNN), and the optimally selected features from the segmentation stage are applied to the dual attention recurrent neural network (DA-RNN). For instance, the parameters inside the deep structures are optimized via the same developed SBCO. Finally, the severity level of the DR is calculated. Several experimental observations are done to evaluate the effectiveness of the developed deep structure-based DR detection approach, considering numerous performance metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Qiao, L., Zhu, Y., & Zhou, H. (2020). Diabetic retinopathy detection using prognosis of microaneurysm and early diagnosis system for non-proliferative diabetic retinopathy based on deep learning algorithms. IEEE Access, 8, 104292–104302.

    Article  Google Scholar 

  2. Agurto, C. (2010). Multiscale AM–FM methods for diabetic retinopathy lesion detection. IEEE Transactions on Medical Imaging, 29(2), 502–512.

    Article  Google Scholar 

  3. Zhang, L., Li, Q., You, J., & Zhang, D. (2009). A modified matched filter with double-sided thresholding for screening proliferative diabetic retinopathy. IEEE Transactions on Information Technology in Biomedicine, 13(4), 528–534.

    Article  Google Scholar 

  4. Narasimha-Iyer, H. (2006). Robust detection and classification of longitudinal changes in color retinal fundus images for monitoring diabetic retinopathy. IEEE Transactions on Biomedical Engineering, 53(6), 1084–1098.

    Article  Google Scholar 

  5. Zeng, X., Chen, H., Luo, Y., & Ye, W. (2019). Automated diabetic retinopathy detection based on binocular siamese-like convolutional neural network. IEEE Access, 7, 30744–30753.

    Article  Google Scholar 

  6. Momeni Pour, A., Seyedarabi, H., Abbasi Jahromi, S. H., & Javadzadeh, A. (2020). Automatic detection and monitoring of diabetic retinopathy using efficient convolutional neural networks and contrast limited adaptive histogram equalization. IEEE Access, 8, 136668–136673.

    Article  Google Scholar 

  7. Majumder, S., & Kehtarnavaz, N. (2021). Multitasking deep learning model for detection of five stages of diabetic retinopathy. IEEE Access, 9, 123220–123230.

    Article  Google Scholar 

  8. Zhao, Y., et al. (2017). Intensity and compactness enabled saliency estimation for leakage detection in diabetic and malarial retinopathy. IEEE Transactions on Medical Imaging, 36(1), 51–63.

    Article  Google Scholar 

  9. Abdelsalam, M. M., & Zahran, M. A. (2021). A novel approach of diabetic retinopathy early detection based on multifractal geometry analysis for OCTA macular images using support vector machine. IEEE Access, 9, 22844–22858.

    Article  Google Scholar 

  10. Mateen, M., Wen, J., Hassan, M., Nasrullah, N., Sun, S., & Hayat, S. (2020). Automatic detection of diabetic retinopathy: A review on datasets, methods, and evaluation metrics. IEEE Access, 8, 48784–48811.

    Article  Google Scholar 

  11. Zhou, Y., Wang, B., Huang, L., Cui, S., & Shao, L. (2021). A benchmark for studying diabetic retinopathy: Segmentation, grading, and transferability. IEEE Transactions on Medical Imaging, 40(3), 818–828.

    Article  Google Scholar 

  12. Pires, R., Avila, S., Jelinek, H. F., Wainer, J., Valle, E., & Rocha, A. (2017). Beyond lesion-based diabetic retinopathy: A direct approach for referral. IEEE Journal of Biomedical and Health Informatics, 21(1), 193–200.

    Article  Google Scholar 

  13. Deperlioğlu, Ö., & Köse, U. (2018). Diagnosis of diabetic retinopathy using image processing and convolutional neural network. In 2018 Medical Technologies National Congress (TIPTEKNO) (pp. 1–4)

  14. Hua, C.-H., et al. (2021). Convolutional network with twofold feature augmentation for diabetic retinopathy recognition from multi-modal images. IEEE Journal of Biomedical and Health Informatics, 25(7), 2686–2697.

    Article  Google Scholar 

  15. Niu, Y., Gu, L., Zhao, Y., & Lu, F. (2022). Explainable diabetic retinopathy detection and retinal image generation. IEEE Journal of Biomedical and Health Informatics, 26(1), 44–55.

    Article  Google Scholar 

  16. Aujih, A. B., Shapiai, M. I., Meriaudeau, F., & Tang, T. B. (2022). EDR-Net: Lightweight deep neural network architecture for detecting referable diabetic retinopathy. IEEE Transactions on Biomedical Circuits and Systems, 16(3), 467–478.

    Article  Google Scholar 

  17. RobiulIslam, M., Abdulrazak, L. F., Nahiduzzaman, M., Goni, M. O. F., Anower, M. S., Ahsan, M., Haider, J., & Kowalski, M. (2022). Applying supervised contrastive learning for the detection of diabetic retinopathy and its severity levels from fundus images. Computers in Biology and Medicine, 146, 105602.

    Article  Google Scholar 

  18. Sugeno, A., YasuyukiIshikawa, Ohshima, T., & Muramatsu, R. (2021). Simple methods for the lesion detection and severity grading of diabetic retinopathy by image processing and transfer learning. Computers in Biology and Medicine, 137, 104795.

    Article  Google Scholar 

  19. Mary Dayana, A., & Sam Emmanuel, W. R. (2022). Deep learning enabled optimized feature selection and classification for grading diabetic retinopathy severity in the fundus image. Neural Computing and Applications, 34, 18663.

    Article  Google Scholar 

  20. Vasireddi, H. K., Suganya Devi, K., & Raja Reddy, G. N. V. (2022). Deep feed-forward neural-network–based screening system for diabetic retinopathy severity classification using the lion optimization algorithm. Graefe’s Archive for Clinical and Experimental Ophthalmology, 260, 1245–1263.

    Article  Google Scholar 

  21. Wang, J., Bai, Y., & Xia, B. (2020). Simultaneous diagnosis of severity and features of diabetic retinopathy in fundus photography using deep learning. IEEE Journal of Biomedical and Health Informatics, 24(12), 3397–3407.

    Article  Google Scholar 

  22. Roslinemary, A., & Kavitha, P. (2021). Automated diabetic retinopathy detection and classification using stochastic coordinate descent deep learning architectures. Materialstoday: PROCEEDINGS, 80, 43.

    Google Scholar 

  23. Qureshi, I., Ma, J., & Abbas, Q. (2021). Diabetic retinopathy detection and stage classification in eye fundus images using active deep learning. Multimedia Tools and Applications, 80, 11691–11721.

    Article  Google Scholar 

  24. Benasla, L., Belmadani, A., & Rahli, M. (2014). Spiral optimization algorithm for solving combined economic and emission dispatch. Electrical Power and Energy Systems, 62, 163–174.

    Article  Google Scholar 

  25. Niu, B., & Wang, H. (2012). Bacterial colony optimization. Discrete Dynamics in Nature and Society. https://doi.org/10.1155/2012/698057

    Article  Google Scholar 

  26. Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang X., Tian, Q., & Wang, M. (2021) Swin-Unet: Unet-like pure transformer for medical image segmentation. Arxiv

  27. Balaji, R., Gowtham Gowda, T., Aralere, H. K., & Priya, V. (2021). Spectral, spatial feature fusion using 3dcnn for hyperspectral image classification. International Research Journal of Modernization in Engineering Technology and Science, 3, 1.

    Google Scholar 

  28. Cecen, A., Dai, H., Yabansu, Y. C., Kalidindi, S. R., & Song, L. (2018). Material structure property linkages using three-dimensional convolutional neural networks. Acta Materialia, 146, 76–84. https://doi.org/10.1016/j.actamat.2017.11.053

    Article  Google Scholar 

  29. Vaiyapuri, T., Jothi, A., Narayanasamy, K., Kamatchi, K., Kadry, S., & Kim, J. (2022). Design of a honey badger optimization algorithm with a deep transfer learning-based osteosarcoma classification model. Cancers, 14(24), 6066.

    Article  Google Scholar 

  30. Vinoth Kumar, B., Zhang, S., Wu, T., Prakash, J., Zhou, L., & Li, K. (2022). A novel JAYA algorithm for optic disc localization in eye fundus images. International Journal of Computational Vision and Robotics, 12(3), 324–342.

    Article  Google Scholar 

  31. Amalia, R., Bustamam, A., & Sarwinda, D. (2021). Detection and description generation of diabetic retinopathy using convolutional neural network and long short-term memory. Journal of Physics Conference Series, 1722(1), 012010.

    Article  Google Scholar 

  32. Dua, M., Makhija, D., Manasa, P. Y. L., & Mishra, P. (2020). A CNN–RNN–LSTM based amalgamation for Alzheimer’s disease detection. Journal of Medical and Biological Engineering, 40, 688.

    Article  Google Scholar 

  33. Jasper Gnana Chandran, J., Jabez, J., & Srinivasulu, S. (2023). Auto-metric graph neural network optimized with capuchin search optimization algorithm for coinciding diabetic retinopathy and diabetic macular edema grading. Biomedical Signal Processing and Control, 80, 104386.

    Article  Google Scholar 

  34. Zulaikha Beevi, S. (2023). Multi-Level severity classification for diabetic retinopathy based on hybrid optimization enabled deep learning. Biomedical Signal Processing and Control, 84, 104736.

    Article  Google Scholar 

  35. Uppamma, P., & Bhattacharya, S. (2023). Diabetic retinopathy detection: A blockchain and African vulture optimization algorithm-based deep learning framework. Electronics, 12(3), 742.

    Article  Google Scholar 

  36. Zhang, C., Chen, P., & Lei, T. (2023). Multi-point attention-based semi-supervised learning for diabetic retinopathy classification. Biomedical Signal Processing and Control, 80, 104412.

    Article  Google Scholar 

  37. Alwakid, G., Gouda, W., & Humayun, M. (2023). Deep learning-based prediction of diabetic retinopathy using CLAHE and ESRGAN for enhancement. Healthcare, 11, 863.

    Article  Google Scholar 

  38. Balamurugan, N. M., Maithili, K., Rathish Babu, T. K. S., & Adimoolam, M. (2023). Stage-wise categorization and prediction of diabetic retinopathy using ensemble learning and 2D-CNN. Intelligent Automation and Soft Computing, 36(1), 499–514.

    Article  Google Scholar 

  39. Mondal, S. S., Mandal, N., Singh, K. K., Singh, A., & Izonin, I. (2023). EDLDR: An ensemble deep learning technique for detection and classification of diabetic retinopathy. Diagnostics, 13(1), 124.

    Article  Google Scholar 

  40. Jian, M., Chen, H., Tao, C., Li, X., & Wang, G. (2023). Triple-DRNet: A triple-cascade convolution neural network for diabetic retinopathy grading using fundus images. Computers in Biology and Medicine, 155, 106631.

    Article  Google Scholar 

  41. Desika Vinayaki, V., & Kalaiselvi, R. (2023). ESLO: Enhanced sea lion optimization based bi-directional CNN–RNN for accurate detection of diabetic retinopathy. Concurrency and Computation Practice and Experience, 35(1), e7391.

    Article  Google Scholar 

  42. Zhang, L., Liu, J., Ma, C., & Gui, H. (2023). Intelligent integrated framework towards high-accuracy machining. Engineering Science and Technology, an International Journal, 40, 101359.

    Article  Google Scholar 

  43. Khodabandelou, G., Moon, H., Amirat, Y., & Mohammed, S. (2023). A fuzzy convolutional attention-based GRU network for human activity recognition. Engineering Applications of Artificial Intelligence, 118, 105702.

    Article  Google Scholar 

  44. Rana, A., Reddy, A., Shrivastava, A., Verma, D., Ansari, M. S., & Singh D. (2022). Secure and smart healthcare system using IoT and deep learning models. In 2022 2nd international conference on technological advancements in computational sciences (ICTACS) (pp. 915–922).

Download references

Funding

This research did not receive any specific funding.

Author information

Authors and Affiliations

  1. School of Electronics and Communication Engineering, Dr. Vishwanath Karad MIT World Peace University, Pune, India

    Arti Khaparde, Manisha Kowdiki & Vaidehi Deshmukh

  2. School of Biology and Biotechnology, Dr. Vishwanath Karad MIT World Peace University, Pune, India

    Shilpa Chapadgaonkar

Authors
  1. Arti Khaparde
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shilpa Chapadgaonkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manisha Kowdiki
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vaidehi Deshmukh
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have made substantial contributions to conception and design, revising the manuscript, and the final approval of the version to be published. Also, all authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to Arti Khaparde.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khaparde, A., Chapadgaonkar, S., Kowdiki, M. et al. An Attention-Based Swin U-Net-Based Segmentation and Hybrid Deep Learning Based Diabetic Retinopathy Classification Framework Using Fundus Images. Sens Imaging 24, 20 (2023). https://doi.org/10.1007/s11220-023-00426-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11220-023-00426-5

Keywords

Search

Navigation