Skip to main content
SpringerLink
Log in

Gesture recognition based on binocular vision

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

A convenient and effective binocular vision system is set up. Gesture information can be accurately extract from the complex environment with the system. The template calibration method is used to calibrate the binocular camera and the parameters of the camera are accurately obtained. In the phase of stereo matching, the BM algorithm is used to quickly and accurately match the images of the left and right cameras to get the parallax of the measured gesture. Combined with triangulation principle, resulting in a more dense depth map. Finally, the depth information is remapped to the original color image to realize three-dimensional reconstruction and three-dimensional cloud image generation. According to the cloud image information, it can be judged that the binocular vision system can effectively segment the gesture from the complex background.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Al-Helali, B.M., Mahmoud, S.A.: Arabic online handwriting recognition (AOHR): a survey. ACM Comput. Surv. (CSUR). 50(3), 33 (2017)

    Google Scholar 

  2. Sturman, D.J., David Z., Steve P.: Hands-on interaction with virtual environments. Proceedings of the 2nd Annual ACM SIGGRAPH Symposium on User Interface Software and Technology, pp. 19–24. ACM (1989)

  3. In-Cheol, K., Chien, S.-I.: Analysis of 3d hand trajectory gestures using stroke-based composite hidden markov models. Appl. Intell. 15(2), 131–143 (2001)

    MATH  Google Scholar 

  4. Noor, T., Shanableh, T., Assaleh, K.: Glove-based continuous Arabic sign language recognition in user-dependent mode. IEEE Trans. Hum.-Mach. Syst. 45(4), 526–533 (2015)

    Google Scholar 

  5. Fang, Y.L., Liu, H.L., Gongfa, Z.X.: A multichannel surface emg system for hand motion recognition. Int. J. Hum. Robot. 12(02), 1550011 (2015). https://doi.org/10.1142/S0219843615500115

    Article  Google Scholar 

  6. Frederic, K., Puhl, M., Krüger, A.: User-independent real-time hand gesture recognition based on surface electromyography. Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services, p. 36. ACM (2017)

  7. Tobely, T.E., Yoshiki, Y., Tsuda, R., Tsuruta, N., Amamiy, M.: Dynamic hand gesture recognition based on randomized self-organizing map algorithm. International Conference on Algorithmic Learning Theory, pp. 252–263. Springer, Berlin (2000)

    Google Scholar 

  8. Bhuyan, M.K., MacDorman, K.F., Kar, M.K., Neog, D.R., Lovell, B.C., Gadde, P.: Hand pose recognition from monocular images by geometrical and texture analysis. J. Vis. Lang. Comput. 28, 39–55 (2015)

    Google Scholar 

  9. Yin, Q., Li, G., Zhu, J.: Research on the method of step feature extraction for EOD robot based on 2d laser radar. Discret. Contin. Dyn. Syst. Ser. 8(6), 1415–1421 (2015)

    MathSciNet  MATH  Google Scholar 

  10. Li, Z., Li, G., Jiang, G., Fang, Y., Zhaojie, J., Liu, H.: Intelligent computation of grasping and manipulation for multi-fingered robotic hands. J. Comput. Theor. Nanosci. 12(12), 6192–6197 (2015)

    Google Scholar 

  11. Li, Z., Li, G., Sun, Y., Jiang, G., Kong, J., Liu, H.: Development of articulated robot trajectory planning. Int. J. Comput. Sci. Math. 8(1), 52–60 (2017)

    MathSciNet  Google Scholar 

  12. Ding, W., Li, G., Sun, Y., Jiang, G., Kong, J., Liu, H.: D-S evidential theory on semg signal recognition. Int. J. Comput. Sci. Math. 8(2), 138–145 (2017)

    MathSciNet  Google Scholar 

  13. He, Y.L., Gongfa, L.Y., Sun, Y.K., Jianyi, J., Guozhang, J., Du, T., Bo, X., Shuang, L.H.: Gesture recognition based on an improved local sparse representation classification algorithm. Clust. Comput. (2017). https://doi.org/10.1007/s10586-017-1237-1

    Google Scholar 

  14. Li, B.S., Ying, L.G., Kong, J.J., Guozhang, J.D., Tao, B.X., Shuang, L.H.: Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust. Comput. (2017). https://doi.org/10.1007/s10586-017-1231-7

    Google Scholar 

  15. Chen, D., Li, G., Sun, Y., Kong, J., Jiang, G., Tang, H., Zhaojie, J., Hui, Y., Liu, H.: An interactive image segmentation method in hand gesture recognition. Sensors 17(2), 253 (2017)

    Google Scholar 

  16. Chen, D., Li, G., Sun, Y., Kong, J., Jiang, G., Li, J., Liu, H.: Fusion hand gesture segmentation and extraction based on CMOS sensor and 3D sensor. Int. J. Wirel. Mob. Comput. 12(3), 305–312 (2017)

    Google Scholar 

  17. Li, G., Miao, W., Jiang, G., Fang, Y., Zhaojie, J., Liu, H.: Intelligent control model and its simulation of flue temperature in coke oven. Discret. Contin. Dyn. Syst. Ser. 8(6), 1223–1237 (2015)

    MathSciNet  MATH  Google Scholar 

  18. Ding, W., Li, G., Jiang, G., Fang, Y., Zhaojie, J., Liu, H.: Intelligent computation in grasping control of dexterous robot hand. J. Comput. Theor. Nanosci. 12(12), 6096–6099 (2015)

    Google Scholar 

  19. Wei, M., Li, G., Jiang, G., Fang, Y., Zhaojie, J., Liu, H.: Optimal grasp planning of multi-fingered robotic hands: a review. Appl. Comput. Math. 14(3), 238–247 (2015)

    MathSciNet  MATH  Google Scholar 

  20. Feng, L.B., Sheng, D.M., Liu, Y.: A gesture recognition method based on binocular vision system. International Conference on Computer Vision Systems, pp. 257-267. Springer, Cham (2017)

  21. Jadooki, S., Mohamad, D., Saba, T., Almazyad, A.S., Rehman, Amjad: Fused features mining for depth-based hand gesture recognition to classify blind human communication. Neural Comput. Appl. 28(11), 3285–3294 (2017)

    Google Scholar 

  22. Stroppa, L., Cristalli, C.: Stereo vision system for accurate 3D measurements of connector pins’ positions in production lines. Exp. Tech. 41(1), 69–78 (2017)

    Google Scholar 

  23. Miao, W., Li, G., Sun, Y., Jiang, G., Kong, J., Liu, H.: Gesture recognition based on sparse representation. Int. J. Wirel. Mob. Comput. 11(4), 348–356 (2016)

    Google Scholar 

  24. Mehryar, E., Evans, A.: Nasal patches and curves for expression-robust 3D face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(5), 995–1007 (2017)

    Google Scholar 

  25. Zhaojie, Ju, Ji, Xiaofei, Li, Jing, Liu, Honghai: An integrative framework of human hand gesture segmentation for human-robot interaction. IEEE Syst. J. 99, 1–11 (2015)

    Google Scholar 

  26. Li, G., Kong, J., Jiang, G., Xie, L., Jiang, Z., Zhao, G.: Air-fuel ratio intelligent control in coke oven combustion process. Inform. Int. Interdiscip. J. 15(11), 4487–4494 (2012)

    Google Scholar 

  27. Pop, D.O., Rogozan, A., Nashashibi, F., Bensrhair, A.: Fusion of stereo vision for pedestrian recognition using convolutional neural networks. ESANN 2017-25th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. (2017)

  28. Chen, D., Li, G., Jiang, G., Fang, Y., Zhaojie, J., Liu, H.: Intelligent computational control of multi-fingered dexterous robotic hand. J. Comput. Theor. Nanosci. 12(12), 6126–6132 (2015)

    Google Scholar 

  29. Li, G., Liu, J., Jiang, G., Liu, H.: Numerical simulation of temperature field and thermal stress field in the new type of ladle with the nanometer adiabatic material. Adv. Mech. Eng. 7(4), 1–13 (2015)

    Google Scholar 

  30. Starr, J.W., Lattimer, B.: Evidential sensor fusion of long-wavelength infrared stereo vision and 3D-LIDAR for rangefinding in fire environments. Fire Technol. 53, 1961–1983 (2017)

    Google Scholar 

  31. Li, G., Yuesheng, G., Kong, J., Jiang, G., Xie, L., Zehao, W., Li, Z., He, Y., Gao, P.: Intelligent control of air compressor production process. Appl. Math. Inform. Sci. 7(3), 1051–1058 (2013)

    Google Scholar 

  32. Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light computer vision and pattern recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on. IEEE. 1: I-I (2003)

  33. Mei, Q., Gao, J., Lin, H., Chen, Y., Yunbo, H., Wang, W., Zhang, G., Chen, X.: Structure light telecentric stereoscopic vision 3D measurement system based on Scheimpflug condition. Opt. Lasers Eng. 86, 83–91 (2016)

    Google Scholar 

  34. Aguilar, J.J., Torres, F., Lope, M.A.: Stereo vision for 3D measurement: accuracy analysis, calibration and industrial applications. Measurement 18(4), 193–200 (1996)

    Google Scholar 

  35. Li, G., Tang, H., Sun, Y., Kong, J., Guozhang J., Du, J., Bo T., Shuang, X., Liu, H.: Hand gesture recognition based on convolution neural network. Clust. Comput. (2017). https://doi.org/10.1007/s10586-017-1435-x

    Google Scholar 

  36. Zhang, Y., Wang, Z., Zou, L., Fang, H.: Event-based finite-time filtering for multi-rate systems with fading measurements. IEEE Trans. Aerosp. Electron. Syst. 53(3), 1431–1441 (2017)

    Google Scholar 

  37. Abdel-Aziz, Y., Karara, H.M.: Direct linear transformation into object space coordinates in close range photogrametry. Urbana-Champaign. pp. 1–18 (1971)

  38. Zhang, Y., Wang, Z., Ma, L.: Variance-constrained state estimation for networked multi-rate systems with measurement quantization and probabilistic sensor failures. Int. J. Robust Nonlinear Control 26(16), 3507–3523 (2016)

    MathSciNet  MATH  Google Scholar 

  39. Tsai, R.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 3(4), 323–344 (1987)

    Google Scholar 

  40. Moons, T., Van Gool, L., Proesmans, M., Pauwels, E.: Affine reconstruction from perspective image pairs with a relative object-camera translation in between. IEEE Trans. Pattern Anal. Mach. Intell. 18(1), 77–83 (1996)

    Google Scholar 

  41. Triggs, B.: Autocalibration and the absolute quadric. Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on. pp. 609–614. IEEE (1997)

  42. Liao, Y., Sun, Y., Li, G., Kong, J., Guozhang Jiang, Du, Jiang, H.C., Zhaojie, J., Hui, Y., Liu, H.: Simultaneous calibration: a joint optimization approach for multiple kinect and external cameras. Sensors 17(7), 1491 (2017). https://doi.org/10.3390/s17071491

    Article  Google Scholar 

  43. Tian, Z.: Face recognition from a single image per person using deep architecture neural networks. Clust. Comput. 19(1), 73–77 (2016)

    Google Scholar 

  44. Sun, Y., Li, C., Li, G., Jiang, G., Jiang, D., Liu, H., Zheng, Z., Shu, W.: Gesture recognition based on kinect and sEMG signal fusion. Mob Netw Appl. (2018). https://doi.org/10.1007/s11036-018-1008-0

    Google Scholar 

  45. Li, G., Peixin, Q., Kong, J., Jiang, G., Xie, L., Gao, P., Zehao, W., He, Y.: Coke oven intelligent integrated control system. Appl. Math. Inform. Sci. 7(3), 1043–1050 (2013)

    Google Scholar 

  46. Li, G., Peixin, Q., Kong, J., Jiang, G., Xie, L., Zehao, W., Gao, P., He, Y.: Influence of working lining parameters on temperature and stress field of ladle. Appl. Math. Inform. Sci. 7(2), 439–448 (2013)

    Google Scholar 

  47. Li, G., Liu, Z., Jiang, G., Xiong, H., Liu, H.: Numerical simulation of the influence factors for rotary kiln in temperature field and stress field and the structure optimization. Adv. Mech. Eng. 7(6), 1–15 (2015)

    Google Scholar 

  48. Wang, F., Jia, K., Feng, J.: The real-time depth map obtainment based on stereo matching. The Euro-China Conference on Intelligent Data Analysis and Applications, pp. 138–144, Springer International Publishing (2016)

  49. Yang, Q.: Stereo matching using tree filtering. IEEE Trans. Pattern Anal. Mach. Intell. 37(4), 834–846 (2015)

    Google Scholar 

  50. Mozerov, M.G., van de Weijer, J.: Accurate stereo matching by two-step energy minimization. IEEE Trans. Image Process. 24(3), 1153–1163 (2015)

    MathSciNet  MATH  Google Scholar 

  51. Hirschmuller, H., Scharstein, D.: Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans. Pattern Anal. Mach. Intell. 31(9), 1582–1599 (2009)

    Google Scholar 

  52. Li, J., Huang, W., Shao, L., Allinson, N.: Building recognition in urban environments: a survey of state-of-the-art and future challenges. Inform. Sci. 277(2), 406–420 (2014)

    Google Scholar 

Download references

Acknowledgements

This work was supported by grants of National Natural Science Foundation of China (Grant Nos. 51575407, 51575338, 51575412, 61733011) and the Grants of National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705).

Author information

Authors and Affiliations

  1. Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wuhan, 430081, China

    Du Jiang, Zujia Zheng, Gongfa Li, Jianyi Kong, Guozhang Jiang & Shuang Xu

  2. Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, 430081, China

    Ying Sun, Jianyi Kong, Guozhang Jiang, Hegen Xiong & Bo Tao

  3. Research Center of Biologic Manipulator and Intelligent Measurement and Control, Wuhan University of Science and Technology, Wuhan, 430081, China

    Gongfa Li

  4. School of Computing, University of Portsmouth, Portsmouth, PO1 3HE, UK

    Hui Yu, Honghai Liu & Zhaojie Ju

Authors
  1. Du Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zujia Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gongfa Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianyi Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guozhang Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hegen Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Bo Tao
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Shuang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Hui Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Honghai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Zhaojie Ju
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Du Jiang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, D., Zheng, Z., Li, G. et al. Gesture recognition based on binocular vision. Cluster Comput 22 (Suppl 6), 13261–13271 (2019). https://doi.org/10.1007/s10586-018-1844-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-018-1844-5

Keywords

Search

Navigation