Skip to main content
SpringerLink
Log in

Hinge attention network: A joint model for diabetic retinopathy severity grading

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Diabetic Retinopathy is one of the prominent reasons for permanent blindness in working age, long term diabetic patients. With the prevalence in raise of diabetics, majority of the people are endangered to permanent vision loss. The advancements in medical imaging techniques enabled the research community to focus on developing automated and computerized systems for diagnosing retinopathy in early stages. But, it is a very complex challenge due to the presence of high intra-class variations and imbalanced data distribution for higher grades of severity. In recent years, various deep learning based models have been designed for automating the process of retinopathy severity classification. In this research work, we present a fascinating deep learning model with multiple attention stages called Hinge Attention Network (HA-Net). Proposed model consists of a pre-trained VGG16 base to extract initial spatial representation from retinal scan images, spatial attention autoencoder to learn lesion specific latent representations in spatial dimensions and a channel attention based hinge neural network to grab category based discriminative features in channel dimension and classify the severity grade of retinopathy. In addition to spatial and channel attention mechanism, we use Convolutional LSTM layer to prioritize highly important spatial maps before passing to hinge neural network. All these components of HA-Net, enabled it to make generalised and accurate predictions on unseen data. The effectiveness and acceptability of proposed model is proved by validating it using two benchmark datasets, Kaggle APTOS 2019 and ISBI IDRiD. Extensive experimental studies on these datasets reveal that, proposed HA-Net outstrip several existing models by achieving an accuracy of 85.54% on Kaggle APTOS, and an accuracy of 66.41% on IDRiD datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Abràmoff MD, Lou Y, Erginay A, Clarida W, Amelon R, Folk JC, Niemeijer M (2016) Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Investigative Ophthalmology & Visual Science 57(13):5206

    Google Scholar 

  2. Al-Antary MT, Arafa Y (2021) Multi-scale attention network for diabetic retinopathy classification. IEEE Access 9:54190–54200

    Article  Google Scholar 

  3. Amin J, Sharif M, Yasmin M (2016) A review on recent developments for detection of diabetic retinopathy. Scientifica

  4. Bhandary SV, Rao KA (2018) Automated screening system for retinal health using bi-dimensional empirical mode decomposition and integrated index. Computers in Biology and Medicine 75:54–62

    Google Scholar 

  5. Bodapati JD, Shaik NS, Naralasetti V (2021) Composite deep neural network with gated-attention mechanism for diabetic retinopathy severity classification. J Ambient Intell Humaniz Comput

  6. Bodapati JD, Shaik NS, Naralasetti V (2021) Deep convolution feature aggregation: an application to diabetic retinopathy severity level prediction. Signal Image Video Process:1–8

  7. Bodapati JD, Shaik NS, Naralasetti V, Mundukur NB (2020) Joint training of two-channel deep neural network for brain tumor classification. Signal Image Video Process:1–8

  8. Bodapati JD, Shareef SN, Naralasetti V, Mundukur NB (2021) Msenet: Multi-modal squeeze-and-excitation network for brain tumor severity prediction. Int J Pattern Recognit Artif Intell:2157005

  9. Bodapati JD, Veeranjaneyulu N, Shareef SN, Hakak S, Bilal M, Maddikunta PKR, Jo O (2020) Blended multi-modal deep convnet features for diabetic retinopathy severity prediction. Electronics 9(6):914

    Article  Google Scholar 

  10. Chen M, Shi X, Zhang Y, Wu D, Guizani M (2017) Deep features learning for medical image analysis with convolutional autoencoder neural network. IEEE Trans Big Data:1–1

  11. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

  12. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee, pp 248–255

  13. Dondeti V, Bodapati JD, Shareef SN, Naralasetti V (2020) Deep convolution features in non-linear embedding space for fundus image classification deep convolution features in non-linear embedding space for fundus image classification. Revue d’Intelligence Artificielle 34(3):307–313

    Article  Google Scholar 

  14. Eisenbarth GS (1986) Type i diabetes mellitus. New England Journal of Medicine 314(21):1360–1368

    Article  Google Scholar 

  15. Fukui H, Hirakawa T, Yamashita T, Fujiyoshi H (2019) Attention branch network: Learning of attention mechanism for visual explanation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR)

  16. Gao Z, Li J, Guo J, Chen Y, Yi Z, Zhong J (2019) Diagnosis of diabetic retinopathy using deep neural networks. IEEE Access 7:3360–3370

    Article  Google Scholar 

  17. Habib M, Welikala R, Hoppe A, Owen C, Rudnicka A, Barman S (2017) Detection of microaneurysms in retinal images using an ensemble classifier. Informatics in Medicine Unlocked 9:44–57

    Article  Google Scholar 

  18. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  19. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)

  20. International diabetes federation diabetes atlas (2019) https://www.diabetesatlas.org/en/. Accessed: 30-06-2021

  21. Janghorbani M, Jones RB, Allison SP (2000) Incidence of and risk factors for proliferative retinopathy and its association with blindness among diabetes clinic attenders. Ophthalmic Epidemiology 7(4):225–241

    Article  Google Scholar 

  22. Kaggle (2019) Aptos 2019 blindness detection challenge. https://www.kaggle.com/c/aptos2019-blindnes-detection. Accessed: 2019-12-30

  23. Kandel I, Castelli M (2020) Transfer learning with convolutional neural networks for diabetic retinopathy image classification. a review. Applied Sciences 10(6):2021

    Article  Google Scholar 

  24. Kassani SH, Kassani PH, Khazaeinezhad R, Wesolowski MJ, Schneider KA, Deters R (2019) Diabetic retinopathy classification using a modified xception architecture. In: 2019 IEEE International symposium on signal processing and information technology (ISSPIT). IEEE, pp 1–6

  25. Kaur N, Chatterjee S, Acharyya M, Kaur J, Kapoor N, Gupta S (2016) A supervised approach for automated detection of hemorrhages in retinal fundus images. In: 2016 5th International conference on wireless networks and embedded systems (WECON). IEEE, pp 1–5

  26. Li X, Hu X, Yu L, Zhu L, Fu C-W, Heng P-A (2019) Canet: Cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE Transactions on Medical Imaging 39(5):1483–1493

    Article  Google Scholar 

  27. Long S, Huang X, Chen Z, Pardhan S, Zheng D (2019) Automatic detection of hard exudates in color retinal images using dynamic threshold and svm classification: algorithm development and evaluation. BioMed Res Int

  28. Luong M-T, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. In: International conference on learning representations

  29. Martinez-Murcia FJ, Ortiz A, Ramírez J, Górriz JM, Cruz R (2021) Deep residual transfer learning for automatic diagnosis and grading of diabetic retinopathy. Neurocomputing 452:424–434

    Article  Google Scholar 

  30. Mateen M, Wen J, Song S, Huang Z et al (2019) Fundus image classification using vgg-19 architecture with pca and svd. Symmetry 11(1):1

    Article  Google Scholar 

  31. Mohammedhasan M, Uğuz H (2020) A new early stage diabetic retinopathy diagnosis model using deep convolutional neural networks and principal component analysis. Traitement du Signal 37(5):711–722

    Article  Google Scholar 

  32. Nanni L, Ghidoni S, Brahnam S (2017) Handcrafted vs. non-handcrafted features for computer vision classification. Pattern Recognition 71:158–172

    Article  Google Scholar 

  33. Noushin E, Pourreza M, Masoudi K, Ghiasi Shirazi E (2019) Microaneurysm detection in fundus images using a two step convolution neural network. Biomed Eng Online 18(1):67

    Article  Google Scholar 

  34. Pires R, Avila S, Wainer J, Valle E, Abramoff MD, Rocha A (2019) A data-driven approach to referable diabetic retinopathy detection. Artificial Intelligence in Medicine 96:93–106

    Article  Google Scholar 

  35. Porwal P, Pachade S, Kamble R, Kokare M, Deshmukh G, Sahasrabuddhe V, Meriaudeau F (2018) Indian diabetic retinopathy image dataset (idrid): a database for diabetic retinopathy screening research. Data 3(3):25

    Article  Google Scholar 

  36. Porwal P, Pachade S, Kokare M, Deshmukh G, Son J, Bae W, Liu L, Wang J, Liu X, Gao L et al (2020) Idrid: Diabetic retinopathy-segmentation and grading challenge. Medical Image Analysis 59:101561

    Article  Google Scholar 

  37. Prentašić P, Lončarić S (2016) Detection of exudates in fundus photographs using deep neural networks and anatomical landmark detection fusion. Computer Methods and Programs in Biomedicine 137:281–292

    Article  Google Scholar 

  38. Quellec G, Al Hajj H, Lamard M, Conze P-H, Massin P, Cochener B (2021) Explain: Explanatory artificial intelligence for diabetic retinopathy diagnosis. Med Image Anal:102118

  39. Qureshi I, Ma J, Abbas Q (2019) Recent development on detection methods for the diagnosis of diabetic retinopathy. Symmetry 11(6):749

    Article  Google Scholar 

  40. Rahim SS, Palade V, Holzinger A (2020) Image processing and machine learning techniques for diabetic retinopathy detection: A review. Artif Intell Mach Learn Digital Pathology:136–154

  41. Razzak MI, Naz S, Zaib A (2018) Deep learning for medical image processing: Overview, challenges and the future. In: Classification in BioApps. Springer, pp 323–350

  42. Reza AM (2004) Realization of the contrast limited adaptive histogram equalization (clahe) for real-time image enhancement. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology 38(1):35–44

    Article  Google Scholar 

  43. Shaban M, Ogur Z, Mahmoud A, Switala A, Shalaby A, Abu Khalifeh H, Ghazal M, Fraiwan L, Giridharan G, Sandhu H et al (2020) A convolutional neural network for the screening and staging of diabetic retinopathy. Plos one 15(6):e0233514

    Article  Google Scholar 

  44. Shaik NS, Cherukuri TK (2021) Lesion-aware attention with neural support vector machine for retinopathy diagnosis. Machine Vision and Applications 32(6):1–13

    Article  Google Scholar 

  45. Shaik NS, Cherukuri TK (2021) Multi-level attention network: application to brain tumor classification. Signal Image Video Process:1–8

  46. Shi X, Chen Z, Wang H, Yeung D-Y, Wong W-K, WOO, W-C (2015) Convolutional lstm network: A machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems. vol 28, Curran Associates, Inc., pp 802–810

  47. Sikder N, Masud M, Bairagi AK, Arif ASM, Nahid A-A, Alhumyani HA (2021) Severity classification of diabetic retinopathy using an ensemble learning algorithm through analyzing retinal images. Symmetry 13(4):670

    Article  Google Scholar 

  48. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  49. Srivastava RK, Greff K, Schmidhuber J (2015) Training very deep networks. In: Advances in neural information processing systems. vol 28, Curran Associates, Inc., pp 2377–2385

  50. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  51. Tsiknakis N, Theodoropoulos D, Manikis G, Ktistakis E, Boutsora O, Berto A, Scarpa F, Scarpa A, Fotiadis DI, Marias K (2021) Deep learning for diabetic retinopathy detection and classification based on fundus images: A review. Comput Biol Med:104599

  52. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. Journal of machine learning research 9:11

    MATH  Google Scholar 

  53. Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhudinov R, Zemel R, Bengio Y (2015) Show, attend and tell: Neural image caption generation with visual attention. In: Proceedings of machine learning research (Lille, France, 09 2015), vol 37, PMLR, pp 2048–2057

  54. Yau JW, Rogers SL, Kawasaki R, Lamoureux EL, Kowalski JW, Bek T, Chen S-J, Dekker JM, Fletcher A, Grauslund J et al (2012) prevalence and major risk factors of diabetic retinopathy. Diabetes care 35(3):556–564

    Article  Google Scholar 

  55. Zheng Y, He M, Congdon N (2012) The worldwide epidemic of diabetic retinopathy. Indian journal of ophthalmology 60(5):428

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. TATA Consultancy Services Limited, Hyderabad, India

    Nagur Shareef Shaik & Teja Krishna Cherukuri

Authors
  1. Nagur Shareef Shaik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Teja Krishna Cherukuri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nagur Shareef Shaik.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shaik, N.S., Cherukuri, T.K. Hinge attention network: A joint model for diabetic retinopathy severity grading. Appl Intell 52, 15105–15121 (2022). https://doi.org/10.1007/s10489-021-03043-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03043-5

Keywords

Search

Navigation