Abstract
Fusing multiple continuous expert annotations is a crucial problem in machine learning and computer vision, particularly when dealing with uncertain and subjective tasks related to affective behaviour. Inspired by the concept of inferring shared and individual latent spaces in probabilistic CCA (PCCA), we firstly propose a novel, generative model which discovers temporal dependencies on the shared/individual spaces (DPCCA). In order to accommodate for temporal lags which are prominent amongst continuous annotations, we further introduce a latent warping process. We show that the resulting model (DPCTW) (i) can be used as a unifying framework for solving the problems of temporal alignment and fusion of multiple annotations in time, and (ii) that by incorporating dynamics, modelling annotation/sequence specific biases, noise estimation and time warping, DPCTW outperforms state-of-the-art methods for both the aggregation of multiple, yet imperfect expert annotations as well as the alignment of affective behavior.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Gunes, H., et al.: Emotion representation, analysis and synthesis in continuous space: A survey. In: Proc. of IEEE FG 2011 EmoSPACE WS, Santa Barbara, CA, USA, pp. 827–834 (2011)
Cowie, R., McKeown, G.: Statistical analysis of data from initial labelled database and recommendations for an economical coding scheme (2010), http://www.semaine-project.eu/
Wöllmer, M., et al.: Abandoning emotion classes. In: INTERSPEECH, pp. 597–600 (2008)
Nicolaou, M.A., et al.: Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Trans. on Affective Computing 2, 92–105 (2011)
Raykar, V.C., et al.: Learning from crowds. Journal of Machine Learning Research 11, 1297–1322 (2010)
Zhou, F., De la Torre, F.: Canonical time warping for alignment of human behavior. In: Advances in Neural Information Processing Systems, vol. 22, pp. 2286–2294 (2009)
Klami, A., Kaski, S.: Probabilistic approach to detecting dependencies between data sets. Neurocomput. 72, 39–46 (2008)
Bach, F.R., Jordan, M.I.: A Probabilistic Interpretation of Canonical Correlation Analysis. Technical report, University of California, Berkeley (2005)
Zeng, Z., et al.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. PAMI. 31, 39–58 (2009)
Kim, M., Pavlovic, V.: Discriminative Learning for Dynamic State Prediction. IEEE Trans. PAMI. 31, 1847–1861 (2009)
Ghahramani, Z., Roweis, S.T.: Learning nonlinear dynamical systems using an EM algorithm. In: Advances in NIPS, pp. 599–605. MIT Press (1999)
Roweis, S., Ghahramani, Z.: A unifying review of linear Gaussian models. Neural Computation 11, 305–345 (1999)
Ghahramani, Z., Jordan, M.I., Smyth, P.: Factorial hidden markov models. In: Machine Learning, vol. 29, pp. 245–273. MIT Press (1997)
Van der Merwe, R., Wan, E.: The square-root unscented Kalman filter for state and parameter-estimation. In: Proc. of IEEE ICASP 2001, vol. 6, pp. 3461–3464 (2001)
McKeown, G., et al.: The SEMAINE corpus of emotionally coloured character interactions. In: ICME, pp. 1079–1084 (2010)
Rabiner, L., Juang, B.H.: Fundamentals of Speech Recognition. United states edn. Prentice-Hall (1993)
Patras, I., Pantic, M.: Particle filtering with factorized likelihoods for tracking facial features. In: Proc. of IEEE FG 2004, pp. 97–102 (2004)
Tipping, M.E.: Sparse Bayesian Learning and the Relevance Vector Machine. Journal of Machine Learning Research 1, 211–244 (2001)
Pantic, M., et al.: Web-based database for facial expression analysis. In: Proc. of IEEE ICME, Amsterdam, The Netherlands, pp. 317–321 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nicolaou, M.A., Pavlovic, V., Pantic, M. (2012). Dynamic Probabilistic CCA for Analysis of Affective Behaviour. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7578. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33786-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-33786-4_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33785-7
Online ISBN: 978-3-642-33786-4
eBook Packages: Computer ScienceComputer Science (R0)