Abstract
A sign language translator is a utilitarian in facilitating communication between the deaf community and the hearing majority. This paper proffers an innovative specialized convolutional neural network (CNN) model, Sign-Net, to recognize hand gesture signs by incorporating scale-space theory to deep learning framework. The proposed model is an ensemble of CNNs—a low resolution network (LRN) and a high resolution network (HRN). This architecture of the proposed model allows the ensemble to work at different spatial resolutions and at varying depths of CNN. The Sign-Net model was assessed with static signs of American Sign Language—alphabets and digits. Since there exists no sign dataset for deep learning, the ensemble performance is evaluated on the synthetic dataset which we have collected for this task. Assessment of the synthetic dataset by Sign-Net reported an impressive accuracy of 74.5%, notably superior to the other existing models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Williams E (1977) Experimental comparisons of face-to-face and mediated communication: a review. Psychol Bull 84:963
Cooper H, Holt B, Bowden R (2011) Sign language recognition. In: Visual analysis of humans. Springer, Berlin, pp 539–562
Nithin DK, Sivakumar PB (2015) Generic feature learning in computer vision. Procedia Comput Sci 58:202–209
Ramachandran R, Rajeev D, Krishnan S, Subathra P (2015) Deep learning—an overview. Int J Appl Eng Res 25433–25448
Nagi J, Ducatelle F, Di Caro GA, Ciresan D, Meier U, Giusti A, Nagi F, Schmidhuber J, Gambardella LM (2011) Max-pooling convolutional neural networks for vision-based hand gesture recognition. In: Signal and image processing applications (ICSIPA). IEEE, pp 342–347
Ma L, Huang W (2016) A static hand gesture recognition method based on the depth information. In: Intelligent human-machine systems and cybernetics (IHMSC), pp 136–139
Pigou L, Dieleman S, Kindermans PJ, Schrauwen B (2014) Sign language recognition using convolutional neural networks. In: Workshop at the European conference on computer vision. Springer, Berlin, pp 572–578
Liwicki S, Everingham M (2009) Automatic recognition of fingerspelled words in British sign language. In: Computer vision and pattern recognition workshops. IEEE, pp 50–57
Gattupalli S, Ghaderi A, Athitsos V (2016) Evaluation of deep learning based pose estimation for sign language recognition. In: Proceedings of the 9th ACM international conference on pervasive technologies related to assistive environments. ACM, p 12
Isaacs J, Foo S (2004) Hand pose estimation for American sign language recognition. In: Proceedings of the thirty-sixth southeastern symposium. IEEE, pp 132–136
Pugeault N, Bowden R (2011) Spelling it out: real-time ASL fingerspelling recognition. In: Computer vision workshops (ICCV workshops). IEEE, pp 1114–1119
Kang B, Tripathi S, Nguyen TQ (2015) Real-time sign language fingerspelling recognition using convolutional neural networks from depth map. In: 3rd IAPR Asian conference on pattern recognition. IEEE, pp 136–140
Ameen S, Vadera S (2017) A convolutional neural network to classify American sign language fingerspelling from depth and colour images. Expert Syst 34:e12197
Tao W, Leu MC, Yin Z (2018) American sign language alphabet recognition using convolutional neural networks with multiview augmentation and inference fusion. Eng Appl Artif Intell 76:202–213
Oyedotun OK, Khashman A (2016) Deep learning in vision-based static hand gesture recognition. Neural Comput Appl 1–11
Geetha M, Kaimal MR (2018) A 3d stroke based representation of sign language signs using key maximum curvature points and 3d chain codes. Multimedia Tools Appl 1–34
Geetha M, Aswathi PV (2013) Dynamic gesture recognition of Indian sign language considering local motion of hand using spatial location of key maximum curvature points. In: Intelligent computational systems (RAICS). IEEE, pp 86–91
Kuznetsova A, Leal-Taixe L, Rosenhahn B (2013) Real-time sign language recognition using a consumer depth camera. In: Proceedings of the IEEE international conference on computer vision workshops, pp 83–90
Dong C, Leu MC, Yin Z (2015) American sign language alphabet recognition using microsoft kinect. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 44–52
Lindeberg T (1994) Scale-space theory: a basic tool for analyzing structures at different scales. J Appl Stat 21:225–270
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Aloysius, N., Geetha, M. (2021). An Ensembled Scale-Space Model of Deep Convolutional Neural Networks for Sign Language Recognition. In: Chiplunkar, N.N., Fukao, T. (eds) Advances in Artificial Intelligence and Data Engineering. AIDE 2019. Advances in Intelligent Systems and Computing, vol 1133. Springer, Singapore. https://doi.org/10.1007/978-981-15-3514-7_29
Download citation
DOI: https://doi.org/10.1007/978-981-15-3514-7_29
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-3513-0
Online ISBN: 978-981-15-3514-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)