Skip to main content
SpringerLink
Log in

A novel optimized deep learning framework to spot keywords and query matching process in Devanagari scripts

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Character recognition is the process of translating scanned images of handwritten, printed, or typewritten text into machine-encoded text. The character recognition of scanned handwritten historical Devanagari documents is the most significant research in recent years. However, the existing classifier’s character recognition of historical Devanagari documents provided lower efficiency and less accuracy. Thus, to overcome these issues, the novel Spider Monkey-based Recurrent Framework (SMbRF) is developed in this research and used for Devanagari script character recognition and keyword spotting. In addition, the historical Devanagari script was collected from the library and scanned using an optical scanner. Moreover, the fitness of the spider monkey is utilized in the dense layer of the recurrent neural model that has tended to gain the finest performance. Here, the fitness function of the SMbRF is utilized to track and segment the lines and words. Also, keywords were tracked, indexed, and spotted by the SMbRF model. Additionally, the query-matching process was done by upgrading the fitness function of the spider monkey in the dense layer of the recurrent model. Finally, the developed approach was validated in the python environment and achieved the finest word spotting accuracy of 99.36%, F-measure of 98.26%, precision of 98.64, and recall of 97.865. Moreover, the recorded maximum error rate was only 2.5% compared to existing works; the proposed novel SMbRF has obtained outstanding results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Data availability

The datasets generated during and/or analysed during the current study are not publicly available due to [This database is collected from a library and scanned by an optical scanner] but are available from the corresponding author on reasonable request.

References

  1. Adolfo S, Marcelli A (2020) Using keyword spotting systems as tools for the transcription of historical handwritten documents: models and procedures for performance evaluation. Pattern Recogn Lett 131:329–335. https://doi.org/10.1016/j.patrec.2020.01.007

    Article  Google Scholar 

  2. Awotunde JB, Ogundokun RO, Ayo FE, Matiluko OE (2020) Speech segregation in background noise based on deep learning. IEEE Access 8:169568–169575. https://doi.org/10.1109/ACCESS.2020.3024077

    Article  Google Scholar 

  3. Bawa S, Kumar M (2021) A comprehensive survey on machine translation for English, Hindi and Sanskrit languages. J Ambient Intell Humaniz Comput 1–34. https://doi.org/10.1007/s12652-021-03479-0

  4. Benabdelaziz R, Gaceb D, Haddad M (2020) Word-spotting approach using transfer deep learning of a CNN network. 2020 1st international conference on communications, control systems and signal processing (CCSSP), pp 219-224. https://doi.org/10.1109/CCSSP49278.2020.9151583

  5. Bhunia AK, Roy PP, Mohta A, Pal U (2018) Cross-language framework for word recognition and spotting of Indic scripts. Pattern Recogn 79:12–31. https://doi.org/10.1016/j.patcog.2018.01.034

    Article  Google Scholar 

  6. Bhunia AK, Roy PP, Sain A (2020) Zone-based keyword spotting in Bangla and Devanagari documents. Multimed Tools Appl 79:27365–27389. https://doi.org/10.1007/s11042-019-08442-y

    Article  Google Scholar 

  7. Cheikhrouhou A, Kessentini Y, Kanoun S (2020) Hybrid HMM/BLSTM system for multi-script Keyword spotting in printed and handwritten documents with identification stage. Neural Comput & Appl 32:9201–9215. https://doi.org/10.1007/s00521-019-04429-w

    Article  Google Scholar 

  8. Cheikhrouhou A, Kessentin Y, Kanoun S (2021) Multi-task learning for simultaneous script identification and keyword spotting in document images. Pattern Recogn 113:107832. https://doi.org/10.1016/j.patcog.2021.107832

    Article  Google Scholar 

  9. Cilia ND, De Stefano C, Fontanella F, Marrocco C, Molinara M, Scotto Di Freca A (2020) An end-to-end deep learning system for medieval writer identification. Pattern Recogn Lett 129:137–143. https://doi.org/10.1016/j.patrec.2019.11.025

    Article  Google Scholar 

  10. Dargan S, Kumar M, Tuteja S (2021) PCA-based gender classification system using hybridization of features and classification techniques. Soft Comput 25(24):15281–15295. https://doi.org/10.1007/s00500-021-06118-0

    Article  Google Scholar 

  11. Das A, Bhunia AK, Roy PP, Pal U (2015) Handwritten word spotting in Indic scripts using foreground and background information. 2015 3rd IAPR Asian conference on pattern recognition (ACPR), IEEE. https://doi.org/10.1109/ACPR.2015.7486539

  12. Dheemanth Urs R, Chethan HK (2021) A study on identification and cleaning of struck-out words in handwritten documents. In: Jeena Jacob I, Kolandapalayam Shanmugam S, Piramuthu S, Falkowski-Gilski P (eds) Data intelligence and cognitive informatics. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-15-8530-2_6

    Chapter  Google Scholar 

  13. Elmansouri M, Makhfi NEL, Aghoutane B (2020) Toward classification of arabic manuscripts words based on the deep convolutional neural networks. 2020 International conference on intelligent systems and computer vision (ISCV). IEEE.

  14. Farooqui FF, Hassan M, Younis MS, Siddhu MK (2020) Offline hand written Urdu word spotting using random data generation. IEEE Access 8:131119–131136. https://doi.org/10.1109/ACCESS.2020.3010166

    Article  Google Scholar 

  15. Gao J, Guo X, Shang M, Sun J (2020) Page-level handwritten word spotting via discriminative feature learning. In Li G, Shen H, Yuan Y, Wang X, Liu H, Zhao X (eds) Knowledge Science, Engineering and Management. KSEM 2020. Lecture notes in computer science, vol 12274. Springer, Cham https://doi.org/10.1007/978-3-030-55130-8_32

  16. Gao Y, Mishchenko Y, Shah A, Matsoukas S, Vitaladevuni S (2020) Towards data-efficient modeling for wake word spotting. ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp 7479–7483. https://doi.org/10.1109/ICASSP40776.2020.9053313

  17. Kang S, Iwana BK, Uchida S (2021) Complex image processing with less data—document image binarization by integrating multiple pre-trained U-net modules. Pattern Recogn 109:107577. https://doi.org/10.1016/j.patcog.2020.107577

    Article  Google Scholar 

  18. Kang B, Han SS, Jeon YB, Jeong CS (2021) Deep learning based character recognition platform in complex situations. In park JJ, Fong SJ, Pan Y, sung Y (eds) advances in computer science and ubiquitous computing. Lecture Notes in Electrical Engineering, vol 715. Springer, Singapore. https://doi.org/10.1007/978-981-15-9343-7_60

  19. Kaur H, Kumar M (2021) On the recognition of offline handwritten word using holistic approach and AdaBoost methodology. Multimed Tools Appl 80:11155–11175. https://doi.org/10.1007/s11042-020-10297-7

    Article  Google Scholar 

  20. Kaur H, Kumar M (2021) Offline handwritten Gurumukhi word recognition using eXtreme gradient boosting methodology. Soft Comput 25(6):4451–4464. https://doi.org/10.1007/s00500-020-05455-w

    Article  Google Scholar 

  21. Kaur RP, Jindal MK, Kumar M (2021) Text and graphics segmentation of newspapers printed in Gurmukhi script: a hybrid approach. Vis Comput 37(7):1637–1659. https://doi.org/10.1007/s00371-020-01927-0

    Article  Google Scholar 

  22. Kumar M, Jindal MK, Sharma RK, Jindal SR, Singh H (2021) Improved recognition results of offline handwritten Gurumukhi characters using hybrid features and adaptive boosting. Soft Comput 25(17):11589–11601. https://doi.org/10.1007/s00500-021-06060-1

    Article  Google Scholar 

  23. Majumder S, Ghosh S, Malakar S, Sarkar R, Nasipuri M (2021) A voting-based technique for word spotting in handwritten document images. Multimed Tools Appl 80:12411–12434. https://doi.org/10.1007/s11042-020-10363-0

    Article  Google Scholar 

  24. Mathew M, Karatzas D, Jawahar CV (2021) DocVQA: a dataset for vqa on document images. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.

    Google Scholar 

  25. Mukherjee H, Majumder C, Dhar A, Sen S, Obaidullah SM, Roy K (2021) A deep learning approach with line drawing for isolated online Bangla character recognition. In: Giri D, Buyya R, Ponnusamy S, De D, Adamatzky a, Abawajy JH (eds) proceedings of the sixth international conference on mathematics and computing. Advances in intelligent systems and computing, vol 1262, springer, Singapore. https://doi.org/10.1007/978-981-15-8061-1_16

  26. Narang SR, Kumar M, Jindal MK (2021) DeepNetDevanagari: a deep learning model for Devanagari ancient character recognition. Multimed Tools Appl 80(13):20671–20686. https://doi.org/10.1007/s11042-021-10775-6

    Article  Google Scholar 

  27. Pabón OS, Torrente M, Provencio M, Rodríguez-Gonzalez A, Menasalvas E (2021) Integrating speculation detection and deep learning to extract lung Cancer diagnosis from clinical notes. Appl Sci 11(2):865. https://doi.org/10.3390/app11020865

    Article  Google Scholar 

  28. Roy PP, Rayar F, Ramel JY (2015) Word spotting in historical documents using primitive codebook and dynamic programming. Image Vis Comput 44:15–28. https://doi.org/10.1016/j.imavis.2015.09.006

    Article  Google Scholar 

  29. Roy PP, Bhunia AK, Das A et al (2017) Keyword spotting in doctor's handwriting on medical prescriptions. Expert Syst Appl 76:113–128. https://doi.org/10.1016/j.eswa.2017.01.027

    Article  Google Scholar 

  30. Roy PP, Kumar P, Patidar S (2021) 3D word spotting using leap motion sensor. Multimed Tools Appl 80:11671–11168. https://doi.org/10.1007/s11042-020-10229-5

    Article  Google Scholar 

  31. Sharada B, Sushma SN, Bharathlal (2019) Keyword spotting in historical devanagari manuscripts by word matching. In Nagabhushan P, Guru D, Shekar B, Kumar Y. Data Analytics and Learning. Lecture Notes in Networks and System, vol 43, Springer Singapore https://doi.org/10.1007/978-981-13-2514-4_6

  32. Sharma H, Hazrati G, Bansal JC (2019) Spider Monkey Optimization Algorithm. In: Spider monkey optimization algorithm. Evolutionary and swarm intelligence algorithms, Springer, Cham, pp 43–59. https://doi.org/10.1007/978-3-319-91341-4_4

    Chapter  Google Scholar 

  33. Singh H, Sharma RK, Singh VP, Kumar M (2021) Recognition of online handwritten Gurmukhi characters using recurrent neural network classifier. Soft Comput 25(8):6329–6338. https://doi.org/10.1007/s00500-021-05620-9

    Article  Google Scholar 

  34. Stauffer M, Fischer A, Riesen K (2018) Keyword spotting in historical handwritten documents based on graph matching. Pattern Recognit 81:240–253. https://doi.org/10.1016/j.patcog.2018.04.001

    Article  Google Scholar 

  35. Stauffer M, Fischer A, Riesen K (2020) Filters for graph-based keyword spotting in historical handwritten documents. Pattern Recogn Lett 134:125–134. https://doi.org/10.1016/j.patrec.2018.03.030

    Article  Google Scholar 

  36. Úbeda I, Saavedra JM, Nicolas S, Petitjean C, Heutte L (2020) Improving pattern spotting in historical documents using feature pyramid networks. Pattern Recognit Lett 131:398–404. https://doi.org/10.1016/j.patrec.2020.02.002

    Article  Google Scholar 

  37. Westphal F, Grahn H, Lavesson N (2020) Representative image selection for data efficient word spotting. International workshop on document analysis systems. Springer, Cham

    Google Scholar 

  38. Wolf F, Brandenbusch K, Fink GA (2020) Improving handwritten word synthesis for annotation-free word spotting. 2020 17th International Conference on Frontiers in Handwriting recognition (ICFHR). IEEE.

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, KBC North Maharashtra University, Jalgaon, Maharashtra, 425001, India

    Nilima Prakash Patil & R. J. Ramteke

Authors
  1. Nilima Prakash Patil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. J. Ramteke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nilima Prakash Patil.

Ethics declarations

Disclosure of potential conflict of interest

The authors declare that they have no potential conflict of interest.

Ethical approval

All applicable institutional and/or national guidelines for the care and use of animals were followed.

Informed consent

For this type of study formal consent is not required.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Patil, N.P., Ramteke, R.J. A novel optimized deep learning framework to spot keywords and query matching process in Devanagari scripts. Multimed Tools Appl 82, 30177–30199 (2023). https://doi.org/10.1007/s11042-023-14912-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14912-1

Keywords

Search

Navigation