Abstract
This paper presents a framework for gesture recognition by modeling a system based on Dynamic Bayesian Networks (DBNs) from a Marionette point of view. To incorporate human qualities like anticipation and empathy inside the perception system of a social robot remains, so far an open issue. It is our goal to search for ways of implementation and test the feasibility. Towards this end we started the development of the guide robot ’Nicole’ equipped with a monocular camera and an inertial sensor to observe its environment. The context of interaction is a person performing gestures and ’Nicole’ reacting by means of audio output and motion. In this paper we present a solution to the gesture recognition task based on Dynamic Bayesian Network (DBN). We show that using a DBN is a human-like concept of recognizing gestures that encompass the quality of anticipation through the concept of prediction and update. A novel approach is used by incorporating a marionette model in the DBN as a trade-off between simple constant acceleration models and complex articulated models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cohen, C.J., Conway, L., Koditschek, D.: Dynamical system representation, generation, and recognition of basic oscillatory motion gestures. In: International Conference on Automatic Face- and Gesture-Recognition (1996)
Kahn, R.E., Swain, M.J., Prokopowicz, P.N., Firby, R.J.: Gesture recognition using the perseus architecture. In: IEEE International Conference on Computer Vision and Pattern Recognition (1996)
Starner, T.: Visual recognition of american sign language using hidden markov models. Master’s thesis, MIT (1995)
Pavlovic, V.I.: Dynamic Bayesian Networks for Information Fusion with Applications to Human-Computer Interfaces. PhD thesis, Graduate College of the University of Illinois (1999)
Meltzoff, A.N., Moore, M.K.: Resolving the debate about early imitation. In: The Blackwell reader in developmental psychology, Oxford, pp. 151–155 (1999)
Nourbakhsh, I., Kunz, C., Willeke, T.: The mobot museum robot installations: A five year experiment. In: IROS 2003 (2003)
Burgard, W., Cremers, A.B., Fox, D., Hahnel, D., Lakemeyer, G., Schulz, D., Steiner, W., Thrun, S.: Experiences with an interactive museum tour-guide robot. Artificial Intelligence 114, 3–55 (1999)
Siegwart, R., et al.: Robox at expo.02: A large-scale installation of personal robots. Robotics and Autonomous Systems 42(3-4), 203–222 (2003)
Rossini, N.: The analysis of gesture: Establishing a set of parameters. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 124–131. Springer, Heidelberg (2004)
Kettebekov, S., Yeasin, M., Sharma, R.: Prosody based co-analysis for continuous recognition of coverbal gestures. In: International Conference on Multimodal Interfaces (ICMI 2002), Pittsburgh, USA, pp. 161–166 (2002)
Loeb, G.E.: Learning from the spinal cord. Journal of Physiology 533(1), 111–117 (2001)
Rett, J., Dias, J.: Visual based human motion analysis: Mapping gestures using a puppet model. In: Bento, C., Cardoso, A., Dias, G. (eds.) EPIA 2005. LNCS, vol. 3808, pp. 398–409. Springer, Heidelberg (2005)
Viola, P., Jones, M.J.: Rapid object detection using a boosted cascade of simple features. In: IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, p. 511 (2001)
Lienhart, R., Maydt, J.: An extended set of haar-like features for rapid object detection. In: IEEE International Conference on Image Processing, vol. 1, pp. 900–903 (2002)
Barreto, J., Menezes, P., Dias, J.: Human-robot interaction based on haar-like features and eigenfaces. In: IEEE International Conference on Robotics and Automation (2004)
Turk, M., Pentland, A.: Face recognition using eigenfaces. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 586–591 (1991)
Menezes, P., Barreto, J., Dias, J.: Face tracking based on haar-like features and eigenfaces. In: IFAC/EURON Symposium on Intelligent Autonomous Vehicles (2004)
Bradski, G.R.: Computer vision face tracking for use in a perceptual user interface. Intel Technology Journal, 15 (1998)
Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME—J.Basic Eng. 82, 35–45 (1960)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)
Ferreira, J., Dias, J.: A 3d scanner – three-dimensional recovery from reflected images. In: Proc. Controlo 2000 Conf. on Automatic Control, University of Minho, University of Minho, Portugal, pp. 690–695 (2000) (Student Forum)
Carpenter, H.: Movement of the eyes, 2nd edn. London Pion Limited, London (1988)
Soechting, J.F., Flanders, M.: Moving in three-dimensional space: Frames of reference, vectors, and coordinate systems. Annual Review of Neuroscience 15, 167–191 (1992)
Lobo, J., Dias, J.: Inertial sensed ego-motion for 3d vision. Journal of Robotic Systems 21, 3–12 (2004)
Lobo, J., Dias, J.: Vision and inertial sensor cooperation using gravity as a vertical reference. IEEE Trans. on PAMI 25, 1597–1608 (2003)
Lobo, J.: Inervis toolbox, http://www.deec.uc.pt/~jlobo/InerVis_WebIndex/InerVis_Toolbox.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rett, J., Dias, J. (2006). Gesture Recognition Using a Marionette Model and Dynamic Bayesian Networks (DBNs). In: Campilho, A., Kamel, M. (eds) Image Analysis and Recognition. ICIAR 2006. Lecture Notes in Computer Science, vol 4142. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11867661_7
Download citation
DOI: https://doi.org/10.1007/11867661_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44894-5
Online ISBN: 978-3-540-44896-9
eBook Packages: Computer ScienceComputer Science (R0)