Skip to main content
SpringerLink
Log in

Research on gesture recognition of smart data fusion features in the IoT

  • Smart Data Aggregation Inspired Paradigm & Approaches in IoT Applns
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

With the rapid development of Internet of things technology, the interaction between people and things has become increasingly frequent. Using simple gestures instead of complex operations to interact with the machine, the fusion of smart data feature information and so on has gradually become a research hotspot. Considering that the depth image of the Kinect sensor lacks color information and is susceptible to depth thresholds, this paper proposes a gesture segmentation method based on the fusion of color information and depth information; in order to ensure the complete information of the segmentation image, a gesture feature extraction method based on Hu invariant moment and HOG feature fusion is proposed; and by determining the optimal weight parameters, the global and local features are effectively fused. Finally, the SVM classifier is used to classify and identify gestures. The experimental results show that the proposed fusion features method has a higher gesture recognition rate and better robustness than the traditional method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Chen D, Li G, Sun Y, Kong J, Jiang G, Tang H, Ju Z, Yu H, Liu H (2017) An interactive image segmentation method in hand gesture recognition. Sensors 17:253. https://doi.org/10.3390/s17020253

    Article  Google Scholar 

  2. Sun Y, Li C, Li G, Jiang G, Jiang D, Liu H, Zheng Z, Shu W (2018) Gesture recognition based on kinect and sEMG signal fusion. Mobile Netw Appl 23(4):797–805

    Article  Google Scholar 

  3. Li G, Zhang L, Sun Y, Kong J (2018) Towards the sEMG hand: internet of things sensors and haptic feedback application. Multimed Tools Appl. https://doi.org/10.1007/s11042-018-6293-x

    Article  Google Scholar 

  4. He Y, Li G, Sun Y, Zhao Y, Jiang G (2018) Numerical simulation-based optimization of contact stress distribution and lubrication conditions in the straight worm drive. Strength Mater 50(11):157. https://doi.org/10.1007/s11223-018-9955-z

    Article  Google Scholar 

  5. Jadooki S, Mohamad D, Saba T, Almazyad A, Rehman A (2017) Fused features mining for depth-based hand gesture recognition to classify blind human communication. Neural Comput Appl 28:3285. https://doi.org/10.1007/s00521-016-2244-5

    Article  Google Scholar 

  6. Li G, Tang H, Sun Y, Kong J, Jiang G, Jiang D, Tao B, Xu S, Liu H (2017) Hand gesture recognition based on convolution neural network. Clust Comput. https://doi.org/10.1007/s10586-017-1435-x

    Article  Google Scholar 

  7. Sun Y, Li G, Kong J, Jiang G, Jiang D, Tao B, Xu S, Liu H (2017) Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust Comput. https://doi.org/10.1007/s10586-017-1231-7

    Article  Google Scholar 

  8. Chang W, Li G, Kong J, Sun Y, Jiang G, Liu H (2018) Thermal mechanical stress analysis of ladle lining with integral brick joint. Arch Metall Mater 63(2):659–666

    Google Scholar 

  9. Liu H, Li G, Zhu X (2015) A multichannel surface EMG system for hand motion recognition. Int J Humanoid Robot. https://doi.org/10.1142/s0219843615500115

    Article  Google Scholar 

  10. Yin Q, Li G, Zhu J (2017) Research on the method of step feature extraction for EOD robot based on 2D laser radar. Discrete Contin Dyn Syst Ser S 8(6):1415–1421

    MathSciNet  MATH  Google Scholar 

  11. Chen D, Li G, Sun Y, Jiang G, Kong J, Li J, Liu H (2017) Fusion hand gesture segmentation and extraction based on CMOS sensor and 3D sensor. Int J Wirel Mobile Comput 12(3):305–312

    Article  Google Scholar 

  12. Sun Y, Hu J, Li G, Jiang G, Xiong H, Tao B, Zheng Z, Jiang D (2018) Gear reducer optimal design based on computer multimedia simulation. J Supercomput. https://doi.org/10.1007/s11227-018-2255-3

    Article  Google Scholar 

  13. He L, Xiong C, Liu K, Huang J, He C, Chen W (2018) Mechatronic design of a synergetic upper limb exoskeletal robot and wrench-based assistive control. J Bionic Eng 15:247. https://doi.org/10.1007/s42235-018-0019-7

    Article  Google Scholar 

  14. Cui Y, Xiong C (2014) Dynamic recurrent neural network based classification scheme for myoelectric control of upper limb rehabilitation robot. J Mech Med Biol. https://doi.org/10.1142/S021951941440017X

    Article  Google Scholar 

  15. Jiang D, Li G, Sun Y, Kong J, Tao B (2018) Gesture recognition based on skeletonization algorithm and CNN with ASL database. Multimed Tools Appl. https://doi.org/10.1007/s11042-018-6748-0

    Article  Google Scholar 

  16. Cheng W, Sun Y, Li G, Jiang G, Liu H (2018) Jointly network: a network based on CNN and RBM for gesture recognition. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3775-8

    Article  Google Scholar 

  17. Li J, Liu X, Ouyang G (2016) Using relevance feedback to distinguish the changes in EEG during different absence seizure phases. ClinicalEeg Neurosci. https://doi.org/10.1177/1550059414548721

    Article  Google Scholar 

  18. Li G, Qu P, Kong J, Jiang G, Xie L, Gao P, Wu Z, He Y (2013) Coke oven intelligent integrated control system. Appl Math Inf Sci 7(3):1043–1050

    Article  Google Scholar 

  19. Patvardhan C, Kumar P, Lakshmi CV (2018) Effective color image watermarking scheme using YCbCr color space and QR code. Multimed Tools Appl 77(10):12655. https://doi.org/10.1007/s11042-017-4909-1

    Article  Google Scholar 

  20. Li G, Miao W, Jiang G, Fang Y, Ju Z, Liu H (2015) Intelligent control model and its simulation of flue temperature in coke oven. Discrete Contin Dyn Syst Ser S 8(6):1223–1237

    MathSciNet  MATH  Google Scholar 

  21. Kundu A, Mazumder O, Lenka P, Bhaumik S (2017) Hand gesture recognition based omnidirectional wheelchair control Using IMU and EMG sensors. J Intell Robot Syst 91:529. https://doi.org/10.1007/s10846-017-0725-0

    Article  Google Scholar 

  22. Li G, Gu Y, Kong J, Jiang G, Xie L, Wu Z, Li Z, He Y, Gao P (2013) Intelligent control of air compressor production process. Appl Math Inf Sci 7(3):1051–1058

    Article  Google Scholar 

  23. Fei M, Li J, Liu H (2015) Visual tracking based on improved foreground detection and perceptual hashing. Neurocomputing 152:413–428

    Article  Google Scholar 

  24. Li G, Jiang D, Zhou Y, Jiang G, Kong J, Manogaran G (2019) Human lesion detection method based on image information and brain signal. IEEE Access. https://doi.org/10.1109/ACCESS.2019.2891749

    Article  Google Scholar 

  25. Li G, Kong J, Jiang G, Xie L, Jiang Z, Zhao G (2012) Air-fuel ratio intelligent control in coke oven combustion process. Int J Inf 12(11):4487–4494

    Google Scholar 

  26. He J, Zhu X (2017) Combining improved gray-level co-occurrence matrix with high density grid for myoelectric control robustness to electrode shift. IEEE Trans Neural Syst Rehabil Eng 99:1539–1548

    Article  Google Scholar 

  27. Zhu X, Liu J, Zhang D, Sheng X, Jiang N (2017) Cascaded adaptation framework for fast calibration of myoelectric control. IEEE Trans Neural Syst Rehabil Eng 25(3):254–264

    Article  Google Scholar 

  28. Hu C, Arvin F, Xiong C, Yue S (2017) Bio-inspired embedded vision system for autonomous micro-robots: the LGMD case. IEEE Trans Cognit Dev Syst 9(3):241–254

    Article  Google Scholar 

  29. Huang Y, Wang Y, Xiao L, Dong W (2014) Microfluidic serpentine antennas with designed mechanical tunability. Lab Chip 14(21):4205–4212

    Article  Google Scholar 

  30. Jiang D, Zheng Z, Li G, Sun Y, Kong J, Jiang G, Xiong H, Tao B, Xu S, Yu H, Liu H, Ju Z (2018) Gesture recognition based on binocular vision. Clust Comput. https://doi.org/10.1007/s10586-018-1844-5

    Article  Google Scholar 

  31. He Y, Li G, Liao Y, Sun Y, Kong J, Jiang G, Jiang D, Tao B, Xu S, Liu H (2018) Gesture recognition based on an improved local sparse representation classification algorithm. Clust Comput. https://doi.org/10.1007/s10586-017-1237-1

    Article  Google Scholar 

  32. Liao Y, Sun Y, Li G, Kong J, Jiang G, Jiang D, Cai H, Ju Z, Yu H, Liu H (2017) Simultaneous calibration: a joint optimization approach for multiple kinect and external cameras. Sensors 17(7):1491. https://doi.org/10.3390/s17071491

    Article  Google Scholar 

  33. Miao W, Li G, Sun Y, Jiang G, Kong J, Liu H (2016) Gesture recognition based on sparse representation. Int J Wirel Mobile Comput 11(4):348–356

    Article  Google Scholar 

  34. Li J, Wang J, Ju Z (2018) A novel hand gesture recognition based on high-level features. Int J Humanoid Rob. https://doi.org/10.1142/s0219843617500220

    Article  Google Scholar 

  35. Ju Z, Ji X, Li J, Liu H (2017) An integrative framework of human hand gesture segmentation for human-robot interaction. IEEE Syst J 11(3):1326–1336

    Article  Google Scholar 

  36. Li J, Allinson NM (2013) Building recognition using local oriented features. IEEE Trans Industr Inf 9(3):1697–1704

    Article  Google Scholar 

  37. Oyedotun OK, Khashman A (2017) Deep learning in vision-based static hand gesture recognition. Neural Comput Appl 28:3941. https://doi.org/10.1007/s00521-016-2294-8

    Article  Google Scholar 

  38. Miao W, Li G, Jiang G, Fang Y, Ju Z, Liu H (2015) Optimal grasp planning of multi-fingered robotic hands: a review. Appl Comput Math 14(3):238–247

    MathSciNet  MATH  Google Scholar 

  39. Liu H, Wu J, Fan S, Jin M, Fan C (2018) Integrated virtual impedance control based pose correction for a simultaneous three-fingered end-effector. Ind Robot. https://doi.org/10.1108/ir-09-2017-0173

    Article  Google Scholar 

  40. Li Z, Wang B, Liu H (2016) Target capturing control for space robots with unknown mass properties: a self-tuning method based on gyros and cameras. Sensors 16(9):1383. https://doi.org/10.3390/s16091383

    Article  Google Scholar 

  41. Cui M, Prasad S (2015) Class-dependent sparse representation classifier for robust hyperspectral image classification. IEEE Trans Geosci Remote Sens 53(5):2683–2695

    Article  Google Scholar 

  42. Liu H, Yang D, Jiang L, Fan S (2014) Development of a multi-DOF prosthetic hand with intrinsic actuation, intuitive control and sensory feedback. Ind Robot Int J 41(4):381–392

    Article  Google Scholar 

  43. Shull PB, Zhu X, Cutkosky MR (2017) Continuous movement tracking performance for predictable and unpredictable tasks with vibrotactile feedback. IEEE Trans Haptics 10(4):466–475

    Article  Google Scholar 

  44. Huang Y, Dong W, Huang T, Wang Y, Xiao L (2015) Self-similar design for stretchable wireless LC strain sensors. Sens Actuators A. https://doi.org/10.1016/j.sna.2015.01.004

    Article  Google Scholar 

  45. Li G, Wu H, Jiang G, Xu S, Liu H (2018) Dynamic gesture recognition in the internet of things. IEEE Access. https://doi.org/10.1109/ACCESS.2018.2887223

    Article  Google Scholar 

  46. Bellocerezo R, Bianconi F, Fernández A, González E, Maria FD (2016) Experimental comparison of color spaces for material classification. J Electron Imaging. https://doi.org/10.1117/1.jei.25.6.061406

    Article  Google Scholar 

  47. Chai G, Zhang D, Zhu X (2017) Developing non-somatotopic phantom finger sensation to comparable levels of somatotopic sensation through user training with electrotactile stimulation. IEEE Trans Neural Syst Rehabil Eng 25(5):469–480

    Article  Google Scholar 

  48. Li C, Li G, Jiang G, Chen D, Liu H (2018) Surface EMG data aggregation processing for intelligent prosthetic action recognition. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3909-z

    Article  Google Scholar 

  49. Satapathy S, Sri Madhava Raja N, Rajinikanth V, Ashour A (2018) Multi-level image thresholding using Otsu and chaotic bat algorithm. Neural Comput Appl 29:1285. https://doi.org/10.1007/s00521-016-2645-5

    Article  Google Scholar 

  50. Dong W, Gu G, Zhu X, Dong X (2015) Solving the boundary value problem of an under-actuated quadrotor with subspace stabilization approach. J Intell Rob Syst 80:299. https://doi.org/10.1007/s10846-014-0161-3

    Article  Google Scholar 

  51. Wu H, Huang Y, Xu F, Xu F, Duan Y, Yin Z (2016) Energy harvesters for wearable and stretchable electronics: from flexibility to stretchability. Adv Mater. https://doi.org/10.1002/adma.201602251

    Article  Google Scholar 

  52. Radman A, Zainal N, Suandi SA (2017) Automated segmentation of iris images acquired in an unconstrained environment using HOG-SVM and GrowCut. Digit Signal Proc 64:60–70

    Article  MathSciNet  Google Scholar 

  53. Jebril NA, Al-Zoubi HR, Al-Haija QA (2018) Recognition of handwritten arabic characters using histograms of oriented gradient (HOG). Pattern Recognit Image Anal 28:321. https://doi.org/10.1134/S1054661818020141

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by grants of the National Natural Science Foundation of China (Grant Nos. 51575407, 51575338, 51575412, 51505349 and 61273106) and Grants of the National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705).

Author information

Authors and Affiliations

  1. Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, 430081, China

    Chong Tan, Ying Sun & Gongfa Li

  2. Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, 430081, China

    Ying Sun, Gongfa Li & Guozhang Jiang

  3. Research Center of Biologic Manipulator and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, 430081, China

    Guozhang Jiang

  4. Institute of Precision Manufacturing, Wuhan University of Science and Technology, Wuhan, 430081, China

    Gongfa Li

  5. School of Computing, University of Portsmouth, Portsmouth, PO1 3HE, UK

    Disi Chen & Honghai Liu

Authors
  1. Chong Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gongfa Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guozhang Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Disi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Honghai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gongfa Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tan, C., Sun, Y., Li, G. et al. Research on gesture recognition of smart data fusion features in the IoT. Neural Comput & Applic 32, 16917–16929 (2020). https://doi.org/10.1007/s00521-019-04023-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04023-0

Keywords

Search

Navigation