Abstract
Typical character and symbol recognition systems are based on images that are drawn on paper or on a tablet with actual physical contact between the pen and the surface. In this study, we investigate the recognition of symbols that are written while the user is immersed inside a room scale virtual reality experience using a consumer grade head-mounted display and related peripherals. A novel educational simulation was developed consisting of a virtual classroom with whiteboard where users can draw symbols. A database of 30 classes of hand-drawn symbols created from test subjects using this environment is presented. The performance of the symbol recognition system was evaluated with deep extreme learning machine classifiers, with accuracy rates of 94.88% with a single classifier and 95.95% with a multiple classifier approach. Further analysis of the results obtained support the conclusion that there are a number of challenges and difficulties related to drawing in this type of environment given the unique constraints and limitations imposed by virtual reality and in particular the lack of physical contact and haptic feedback between the controller and virtual space. Addressing the issues raised for these types of interfaces opens new challenges for both human-computer interaction and symbol recognition. Finally, the approach proposed in this paper creates a new avenue of research in the field of document analysis and recognition by exploring how texts and symbols can be analyzed and automatically recognized in virtual scenes of this type.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
TC-10: http://iapr-tc10.univ-lr.fr/.
References
HTC Vive - Lighthouse Tracking Technology (2016). https://github.com/ValveSoftware/openvr, http://doc-ok.org/?p=1478
Alhamdoosh, M., Wang, D.: Fast decorrelated neural network ensembles with random weights. Inf. Sci. 264, 104–117 (2014)
Arica, N., Yarman-Vural, F.T.: An overview of character recognition focused on off-line handwriting. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 31(2), 216–233 (2001)
Assenmacher, I., Hentschel, B., Cheng, N., Kuhlen, T., Christian, B.: Interactive data annotation in virtual environments. In: Proceedings of the Eurographics Symposium on Virtual Environments, pp. 119–126. ACM SIGGRAPH (2006)
Cecotti, H.: Deep random vector functional link network for handwritten character recognition. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2016)
Cecotti, H.: Hierarchical k-nearest neighbor with GPUs and a high performance cluster: application to handwritten character recognition. Int. J. Pattern Recogn. Artif. Intell. 31(2), 1–24 (2017)
Chen, C.L.P.: A rapid supervised learning neural network for function interpolation and approximation. IEEE Trans. Neural Netw. 7(5), 1220–1230 (1996)
Chen, C.L.P., Wan, J.Z.: A rapid learning and dynamic stepwise updating algorithm for flat neural networks and the application to time-series prediction. IEEE Trans. Syst. Man Cybern. Part B 29(1), 62–72 (1999)
Chhabra, A.K.: Graphic symbol recognition: an overview. In: Tombre, K., Chhabra, A.K. (eds.) GREC 1997. LNCS, vol. 1389, pp. 68–79. Springer, Heidelberg (1998). doi:10.1007/3-540-64381-8_40
Cordella, L.P., Vento, M.: Symbol recognition in documents: a collection of techniques? IJDAR 3(2), 73–88 (2000)
Craig, A.B., Sherman, W.R.: Understanding Virtual Reality: Interface, Application, and Design. Morgan Kaufmann, San Francisco (2002)
Dosch, P., Valveny, E.: Report on the second symbol recognition contest. In: Liu, W., Lladós, J. (eds.) GREC 2005. LNCS, vol. 3926, pp. 381–397. Springer, Heidelberg (2006). doi:10.1007/11767978_35
Ghosh, D., Dube, T., Shivaprasad, A.: Script recognition: a review. IEEE Trans. Pattern Anal. Mac. Intell. 32(12), 2142–2161 (2010)
Huang, G.B., Saratchandran, P., Sundararajan, N.: A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Trans. Neural Netw. 16(1), 57–67 (2005)
Huang, G.B., Siew, C.K.: Extreme learning machine with randomly assigned RBF kernels. Int. J. Inf. Technol. 11(1), 16–24 (2005)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of the IEEE International Joint Conference on Neural Networks, vol. 2, pp. 985–990 (2004)
Igelnik, B., Pao, Y.H.: Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Trans. Neural Netw. 6(6), 1320–1329 (1995)
Kasun, L.L.C., Zhou, H., Huang, G.B., Vong, C.M.: Representational learning with extreme learning machine for big data. IEEE Intell. Syst. 28(6), 31–34 (2013)
Ling, H., Jacobs, D.W.: Shape classification using the inner-distance. IEEE Trans. PAMI 29(2), 286–299 (2007)
Liu, C., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten digit recognition: benchmarking of state-of-the-art techniques. Pattern Recogn. 36(10), 2271–2285 (2003)
Nagy, G.: At the frontiers of OCR. Proc. of IEEE 80, 1093–1100 (1992)
Nagy, N.: Twenty years of document image analysis in PAMI. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 38–62 (2000)
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man. Cybern. 9(1), 62–66 (1979)
Pao, Y.H., Takefuji, Y.: Functional-link net computing: theory, system architecture, and functionalities. Computer 25(5), 76–79 (1992)
Plamondon, R., Srihari, N.S.: On-line and off-line handwritten recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mac. Intell. 22(1), 63–84 (2000)
Rahman, A., Fairhurst, M.: Multiple classifier decision combination strategies for character recognition: a review. Int. J. Doc. Anal. Recogn. (IJDAR) 5(4), 166–194 (2003)
Rausch, D., Assenmacher, I., Kuhlen, T.: 3D sketch recognition for interaction in virtual environments. In: Workshop in Virtual Reality Interactions and Physical Simulation (VRIPHYS). The Eurographics Association (2010)
Santosh, K.C., Wendling, L.: Graphical Symbol Recognition. Wiley, Hoboken (2015)
Scardapane, S., Wang, D., Panella, M., Uncini, A.: Distributed learning for random vector functional-link networks. Inf. Sci. 301, 271–284 (2015)
Schmidt, W.F., Kraaijveld, M.A., Duin, R.P.W.: Feedforward neural networks with random weights. In: Proceedings of the 11th IAPR International Conference on Pattern Recognition, Conference B: Pattern Recognition Methodology and Systems, vol. 2, pp. 1–4 (1992)
Sezgin, T.M., Davis, R.: HMM-based efficient sketch recognition. In: Intelligent User Interfaces, pp. 281–283 (2005)
Sezgin, T.M., Davis, R.: Sketch interpretation using multiscale models of temporal patterns. IEEE Comput. Graph. Appl. 27(1), 28–37 (2007)
Tang, J., Deng, C., Huang, G.B.: Extreme learning machine for multilayer perceptron. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 809–821 (2016)
Valveny, E., Delalandre, M., Raveaux, R., Lamiroy, B.: Report on the symbol recognition and spotting contest. In: Kwon, Y.-B., Ogier, J.-M. (eds.) GREC 2011. LNCS, vol. 7423, pp. 198–207. Springer, Heidelberg (2013). doi:10.1007/978-3-642-36824-0_19
Wang, L.P., Wan, C.R.: Comments on the extreme learning machine. IEEE Trans. Neural Netw. 19(8), 1494–1495 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Cecotti, H., Boumedine, C., Callaghan, M. (2017). Hand-Drawn Symbol Recognition in Immersive Virtual Reality Using Deep Extreme Learning Machines. In: Santosh, K., Hangarge, M., Bevilacqua, V., Negi, A. (eds) Recent Trends in Image Processing and Pattern Recognition. RTIP2R 2016. Communications in Computer and Information Science, vol 709. Springer, Singapore. https://doi.org/10.1007/978-981-10-4859-3_8
Download citation
DOI: https://doi.org/10.1007/978-981-10-4859-3_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-4858-6
Online ISBN: 978-981-10-4859-3
eBook Packages: Computer ScienceComputer Science (R0)