Skip to main content
SpringerLink
Log in

Hand-Drawn Symbol Recognition in Immersive Virtual Reality Using Deep Extreme Learning Machines

  • Conference paper
  • First Online:
Recent Trends in Image Processing and Pattern Recognition (RTIP2R 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 709))

Abstract

Typical character and symbol recognition systems are based on images that are drawn on paper or on a tablet with actual physical contact between the pen and the surface. In this study, we investigate the recognition of symbols that are written while the user is immersed inside a room scale virtual reality experience using a consumer grade head-mounted display and related peripherals. A novel educational simulation was developed consisting of a virtual classroom with whiteboard where users can draw symbols. A database of 30 classes of hand-drawn symbols created from test subjects using this environment is presented. The performance of the symbol recognition system was evaluated with deep extreme learning machine classifiers, with accuracy rates of 94.88% with a single classifier and 95.95% with a multiple classifier approach. Further analysis of the results obtained support the conclusion that there are a number of challenges and difficulties related to drawing in this type of environment given the unique constraints and limitations imposed by virtual reality and in particular the lack of physical contact and haptic feedback between the controller and virtual space. Addressing the issues raised for these types of interfaces opens new challenges for both human-computer interaction and symbol recognition. Finally, the approach proposed in this paper creates a new avenue of research in the field of document analysis and recognition by exploring how texts and symbols can be analyzed and automatically recognized in virtual scenes of this type.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    TC-10: http://iapr-tc10.univ-lr.fr/.

References

  1. HTC Vive - Lighthouse Tracking Technology (2016). https://github.com/ValveSoftware/openvr, http://doc-ok.org/?p=1478

  2. Alhamdoosh, M., Wang, D.: Fast decorrelated neural network ensembles with random weights. Inf. Sci. 264, 104–117 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  3. Arica, N., Yarman-Vural, F.T.: An overview of character recognition focused on off-line handwriting. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 31(2), 216–233 (2001)

    Article  Google Scholar 

  4. Assenmacher, I., Hentschel, B., Cheng, N., Kuhlen, T., Christian, B.: Interactive data annotation in virtual environments. In: Proceedings of the Eurographics Symposium on Virtual Environments, pp. 119–126. ACM SIGGRAPH (2006)

    Google Scholar 

  5. Cecotti, H.: Deep random vector functional link network for handwritten character recognition. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2016)

    Google Scholar 

  6. Cecotti, H.: Hierarchical k-nearest neighbor with GPUs and a high performance cluster: application to handwritten character recognition. Int. J. Pattern Recogn. Artif. Intell. 31(2), 1–24 (2017)

    Article  Google Scholar 

  7. Chen, C.L.P.: A rapid supervised learning neural network for function interpolation and approximation. IEEE Trans. Neural Netw. 7(5), 1220–1230 (1996)

    Article  Google Scholar 

  8. Chen, C.L.P., Wan, J.Z.: A rapid learning and dynamic stepwise updating algorithm for flat neural networks and the application to time-series prediction. IEEE Trans. Syst. Man Cybern. Part B 29(1), 62–72 (1999)

    Article  Google Scholar 

  9. Chhabra, A.K.: Graphic symbol recognition: an overview. In: Tombre, K., Chhabra, A.K. (eds.) GREC 1997. LNCS, vol. 1389, pp. 68–79. Springer, Heidelberg (1998). doi:10.1007/3-540-64381-8_40

    Chapter  Google Scholar 

  10. Cordella, L.P., Vento, M.: Symbol recognition in documents: a collection of techniques? IJDAR 3(2), 73–88 (2000)

    Article  Google Scholar 

  11. Craig, A.B., Sherman, W.R.: Understanding Virtual Reality: Interface, Application, and Design. Morgan Kaufmann, San Francisco (2002)

    Google Scholar 

  12. Dosch, P., Valveny, E.: Report on the second symbol recognition contest. In: Liu, W., Lladós, J. (eds.) GREC 2005. LNCS, vol. 3926, pp. 381–397. Springer, Heidelberg (2006). doi:10.1007/11767978_35

    Chapter  Google Scholar 

  13. Ghosh, D., Dube, T., Shivaprasad, A.: Script recognition: a review. IEEE Trans. Pattern Anal. Mac. Intell. 32(12), 2142–2161 (2010)

    Article  Google Scholar 

  14. Huang, G.B., Saratchandran, P., Sundararajan, N.: A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Trans. Neural Netw. 16(1), 57–67 (2005)

    Article  Google Scholar 

  15. Huang, G.B., Siew, C.K.: Extreme learning machine with randomly assigned RBF kernels. Int. J. Inf. Technol. 11(1), 16–24 (2005)

    Google Scholar 

  16. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of the IEEE International Joint Conference on Neural Networks, vol. 2, pp. 985–990 (2004)

    Google Scholar 

  17. Igelnik, B., Pao, Y.H.: Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Trans. Neural Netw. 6(6), 1320–1329 (1995)

    Article  Google Scholar 

  18. Kasun, L.L.C., Zhou, H., Huang, G.B., Vong, C.M.: Representational learning with extreme learning machine for big data. IEEE Intell. Syst. 28(6), 31–34 (2013)

    Google Scholar 

  19. Ling, H., Jacobs, D.W.: Shape classification using the inner-distance. IEEE Trans. PAMI 29(2), 286–299 (2007)

    Article  Google Scholar 

  20. Liu, C., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: benchmarking of state-of-the-art techniques. Pattern Recogn. 36(10), 2271–2285 (2003)

    Article  MATH  Google Scholar 

  21. Nagy, G.: At the frontiers of OCR. Proc. of IEEE 80, 1093–1100 (1992)

    Article  Google Scholar 

  22. Nagy, N.: Twenty years of document image analysis in PAMI. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 38–62 (2000)

    Article  Google Scholar 

  23. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man. Cybern. 9(1), 62–66 (1979)

    Article  MathSciNet  Google Scholar 

  24. Pao, Y.H., Takefuji, Y.: Functional-link net computing: theory, system architecture, and functionalities. Computer 25(5), 76–79 (1992)

    Article  Google Scholar 

  25. Plamondon, R., Srihari, N.S.: On-line and off-line handwritten recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mac. Intell. 22(1), 63–84 (2000)

    Article  Google Scholar 

  26. Rahman, A., Fairhurst, M.: Multiple classifier decision combination strategies for character recognition: a review. Int. J. Doc. Anal. Recogn. (IJDAR) 5(4), 166–194 (2003)

    Article  Google Scholar 

  27. Rausch, D., Assenmacher, I., Kuhlen, T.: 3D sketch recognition for interaction in virtual environments. In: Workshop in Virtual Reality Interactions and Physical Simulation (VRIPHYS). The Eurographics Association (2010)

    Google Scholar 

  28. Santosh, K.C., Wendling, L.: Graphical Symbol Recognition. Wiley, Hoboken (2015)

    Google Scholar 

  29. Scardapane, S., Wang, D., Panella, M., Uncini, A.: Distributed learning for random vector functional-link networks. Inf. Sci. 301, 271–284 (2015)

    Article  MathSciNet  Google Scholar 

  30. Schmidt, W.F., Kraaijveld, M.A., Duin, R.P.W.: Feedforward neural networks with random weights. In: Proceedings of the 11th IAPR International Conference on Pattern Recognition, Conference B: Pattern Recognition Methodology and Systems, vol. 2, pp. 1–4 (1992)

    Google Scholar 

  31. Sezgin, T.M., Davis, R.: HMM-based efficient sketch recognition. In: Intelligent User Interfaces, pp. 281–283 (2005)

    Google Scholar 

  32. Sezgin, T.M., Davis, R.: Sketch interpretation using multiscale models of temporal patterns. IEEE Comput. Graph. Appl. 27(1), 28–37 (2007)

    Article  Google Scholar 

  33. Tang, J., Deng, C., Huang, G.B.: Extreme learning machine for multilayer perceptron. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 809–821 (2016)

    Article  MathSciNet  Google Scholar 

  34. Valveny, E., Delalandre, M., Raveaux, R., Lamiroy, B.: Report on the symbol recognition and spotting contest. In: Kwon, Y.-B., Ogier, J.-M. (eds.) GREC 2011. LNCS, vol. 7423, pp. 198–207. Springer, Heidelberg (2013). doi:10.1007/978-3-642-36824-0_19

    Chapter  Google Scholar 

  35. Wang, L.P., Wan, C.R.: Comments on the extreme learning machine. IEEE Trans. Neural Netw. 19(8), 1494–1495 (2008)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computing and Engineering, Ulster University, Magee Campus, Northland Road, Derry, Londonderry, BT48 7JL, Northern Ireland, UK

    Hubert Cecotti, Cyrus Boumedine & Michael Callaghan

Authors
  1. Hubert Cecotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cyrus Boumedine
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Callaghan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hubert Cecotti .

Editor information

Editors and Affiliations

  1. The University of South Dakota, Vermillion, South Dakota, USA

    K.C. Santosh

  2. Karnatak Arts, Science and Commerce College, Bidar, India

    Mallikarjun Hangarge

  3. Polytecnico di Bari, Bari, Italy

    Vitoantonio Bevilacqua

  4. University of Hyderabad, Hyderabad, India

    Atul Negi

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper

Cecotti, H., Boumedine, C., Callaghan, M. (2017). Hand-Drawn Symbol Recognition in Immersive Virtual Reality Using Deep Extreme Learning Machines. In: Santosh, K., Hangarge, M., Bevilacqua, V., Negi, A. (eds) Recent Trends in Image Processing and Pattern Recognition. RTIP2R 2016. Communications in Computer and Information Science, vol 709. Springer, Singapore. https://doi.org/10.1007/978-981-10-4859-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-4859-3_8

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-4858-6

  • Online ISBN: 978-981-10-4859-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation