Skip to main content

Advertisement

SpringerLink
Log in

Gesture recognition based on skeletonization algorithm and CNN with ASL database

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the field of human-computer interaction, vision-based gesture recognition methods are widely studied. However, its recognition effect depends to a large extent on the performance of the recognition algorithm. The skeletonization algorithm and convolutional neural network (CNN) for the recognition algorithm reduce the impact of shooting angle and environment on recognition effect, and improve the accuracy of gesture recognition in complex environments. According to the influence of the shooting angle on the same gesture recognition, the skeletonization algorithm is optimized based on the layer-by-layer stripping concept, so that the key node information in the hand skeleton diagram is extracted. The gesture direction is determined by the spatial coordinate axis of the hand. Based on this, gesture segmentation is implemented to overcome the influence of the environment on the recognition effect. In order to further improve the accuracy of gesture recognition, the ASK gesture database is used to train the convolutional neural network model. The experimental results show that compared with SVM method, dictionary learning + sparse representation, CNN method and other methods, the recognition rate reaches 96.01%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Błaszczyk Ł (2016) Compressed sensing in MRI – mathematical preliminaries and basic examples. Nukleonika 61(1):41–43

    Article  Google Scholar 

  2. Bu X, Dong H, Han F et al (2018) Event-triggered distributed filtering over sensor networks with deception attacks and partial measurements. Int J Gen Syst 47:395–407

    Article  MathSciNet  Google Scholar 

  3. Caselli N, Sehyr Z, Cohen-Goldberg A et al (2017) ASL-LEX: A lexical database of American Sign Language. Behav Res Methods 49(2):784–801

    Article  Google Scholar 

  4. Chang W, Li G, Kong J et al (2018) Thermal Mechanical Stress Analysis of Ladle Lining with Integral Brick Joint. Arch Metall Mater 63(2):659–666

    Google Scholar 

  5. Chen D, Li G, Sun Y et al (2017) An interactive image segmentation method in hand gesture recognition. Sensors 17(2):253

    Article  Google Scholar 

  6. Fang Y, Liu H, Li G et al (2015) A multichannel surface emg system for hand motion recognition. International Journal of Humanoid Robotics 12(2). https://doi.org/10.1142/S0219843615500115

    Article  Google Scholar 

  7. Han F, Dong H, Wang Z et al (2018) Improved tobit kalman filtering for systems with random parameters via conditional expectation. Signal Process 147. https://doi.org/10.1016/j.sigpro.2018.01.015

    Article  Google Scholar 

  8. He Y, Li G, Liao Y et al (2017) Gesture recognition based on an improved local sparse representation classification algorithm. Clust Comput 1. https://doi.org/10.1007/s10586-017-1237-1

    Article  Google Scholar 

  9. He Y, Li G, Zhao Y et al (2018) Numerical simulation-based optimization of contact stress distribution and lubrication conditions in the straight worm drive. Strength of Materials 50(11):1–9

    Google Scholar 

  10. Jiang D, Zheng Z, Li G et al (2017) Gesture recognition based on binocular vision. Clust Comput 3. https://doi.org/10.1007/s10586-018-1844-5

    Article  Google Scholar 

  11. Jin KH, Lee D, Ye JC (2017) A general framework for compressed sensing and parallel mri using annihilating filter based low-rank Hankel matrix. IEEE Transactions on Computational Imaging 2(4):480–495

    Article  MathSciNet  Google Scholar 

  12. Li G, Gu Y, Kong J et al (2013) Intelligent control of air compressor production process. Applied Mathematics & Information Sciences 7(3):1051–1058

    Article  Google Scholar 

  13. Li G, Kong J, Jiang G et al (2012) Air-fuel ratio intelligent control in coke oven combustion process. International Journal on Information 12(11):4487–4494

    Google Scholar 

  14. Li G, Liu J, Jiang G et al (2015) Numerical simulation of temperature field and thermal stress field in the new type of ladle with the nanometer adiabatic material. Advances in Mechanical Engineering 7(4):1687814015575988

    Article  Google Scholar 

  15. Li G, Liu Z, Jiang G et al (2017) Numerical simulation of the influence factors for rotary kiln in temperature field and stress field and the structure optimization. Advances in Mechanical Engineering 7(6):1687814015589667

    Article  Google Scholar 

  16. Li G, Miao W, Jiang G et al (2015) Intelligent control model and its simulation of flue temperature in coke oven. Discrete and Continuous Dynamical Systems - Series S 8(6):1223–1237

    Article  MathSciNet  MATH  Google Scholar 

  17. Li G, Qu P, Kong J et al (2013) Coke oven intelligent integrated control system. Applied Mathematics & Information Sciences 7(3):1043–1050

    Article  Google Scholar 

  18. Li G, Qu P, Kong J et al (2013) Influence of working lining parameters on temperature and stress field of ladle. Applied Mathematics & Information Sciences 7(2):439–448

    Article  Google Scholar 

  19. Li B, Sun Y, Li G et al (2017) Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust Comput 1. https://doi.org/10.1007/s10586-017-1231-7

    Article  Google Scholar 

  20. Li G, Tang H, Zhao Y et al (2017) Hand gesture recognition based on convolution neural network. Clust Comput 3. https://doi.org/10.1007/s10586-017-1435-x

    Article  Google Scholar 

  21. Li G, Zhang L, Sun Y et al (2018) Towards the sEMG hand: internet of things sensors and haptic feedback application. Multimed Tools Applications 1:1–18

    Google Scholar 

  22. Liao Y, Sun Y, Li G et al (2017) Simultaneous Calibration: A Joint Optimization Approach for Multiple Kinect and External Cameras. Sensors 17(7):1491. https://doi.org/10.3390/s17071491

    Article  Google Scholar 

  23. Luzanin O, Plancak M (2014) Hand gesture recognition using low-budget data glove and cluster-trained probabilistic neural network. Assem Autom 34(1):94–105

    Article  Google Scholar 

  24. Miao W, Li G, Jiang G et al (2015) Optimal Grasp Planning of Multi-Fingered Robotic Hands: A Review. Applied and computational mathematics 14(3):238–247

    MathSciNet  MATH  Google Scholar 

  25. Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29(9):2352–2449

    Article  MathSciNet  MATH  Google Scholar 

  26. Saha P, Jin D, Liu Y et al (2018) Fuzzy object skeletonization: theory, algorithms, and applications. IEEE Trans Vis Comput Graph 24(8):2298–2314

    Article  Google Scholar 

  27. Shotton J, Fitzgibbon A, Cook M et al (2011) Real-Time Human Pose Recognition in Parts from Single Depth Images. Computer Vision and Pattern Recognition 56(1):1297–1304

    Google Scholar 

  28. Sun Y, Hu J, Li G et al (2018) Gear reducer optimal design based on computer multimedia simulation. J Supercomput 3:1–13. https://doi.org/10.1007/s11227-018-2255-3

    Article  Google Scholar 

  29. Sun Y, Li C, Li G et al (2018) Gesture Recognition Based on Kinect and sEMG Signal Fusion. Mobile Networks and Applications 23(4):797–805. https://doi.org/10.1007/s11036-018-1008-0

    Article  Google Scholar 

  30. Xiong H, Fan H, Jiang G et al (2017) A simulation-based study of dispatching rules in a dynamic job shop scheduling problem with batch release and extended technical precedence constraints. Eur J Oper Res 257(1):13–24

    Article  MathSciNet  MATH  Google Scholar 

  31. Xiong H, Fan H, Li G et al (2015) Research on steady-state simulation in dynamic job shop scheduling problem. Advances in Mechanical Engineering 7(9):1–11

    Article  Google Scholar 

  32. Yin Q, Li G, Zhu J (2017) Research on the method of step feature extraction for EOD robot based on 2D laser radar. Discrete and Continuous Dynamical Systems - Series S (DCDS-S) 8(6):1415–1421

    Article  MathSciNet  MATH  Google Scholar 

  33. Zhong X, Chen Y, Yu H et al (2018) Context-Aware Information Based Ultrasonic Gesture Recognition Method. Journal of Computer-Aided Design & Computer Graphics 30(1):173

    Article  Google Scholar 

  34. Cheng W, Sun Y, Li G et al (2018) Jointly network: a network based on CNN and RBM for gesture recognition. Neural Computing and Applications 1–17. https://doi.org/10.1007/s00521-018-3775-8

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by grants of National Natural Science Foundation of China (Grant No. 51575407, 51575338, 51575412, 61273106, 51505349) and the Grants of National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705). This paper is funded by Wuhan University of Science and Technology graduate students short-term study abroad special funds.

Author information

Authors and Affiliations

  1. Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan, 430081, China

    Du Jiang, Gongfa Li & Jianyi Kong

  2. Research Center for Biomimetic Robot and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, 430081, China

    Du Jiang & Gongfa Li

  3. Institute of Precision Manufacturing, Wuhan University of Science and Technology, Wuhan University of Science and Technology, Wuhan, 430081, China

    Gongfa Li & Bo Tao

  4. Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, 430081, China

    Ying Sun & Jianyi Kong

Authors
  1. Du Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gongfa Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianyi Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bo Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gongfa Li.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, D., Li, G., Sun, Y. et al. Gesture recognition based on skeletonization algorithm and CNN with ASL database. Multimed Tools Appl 78, 29953–29970 (2019). https://doi.org/10.1007/s11042-018-6748-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6748-0

Keywords

Search

Navigation