Skip to main content
SpringerLink
Log in

Correlation-optimized time warping for motion

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Retrieval and comparative editing/modeling of motion data require temporal alignment. In other words, for such processes to perform accurately, critical features of motion sequences need to occur simultaneously. In this paper, we propose correlation-optimized time warping (CoTW) for aligning motion data. CoTW utilizes a correlation-based objective function for characterizing alignment. The method solves an optimization problem to determine the optimum warping degree for different segments of the sequence. Using segment-wise interpolated warping, smooth motion trajectories are achieved that can be readily used for animation. Our method allows for manual tuning of the parameters, resulting in high customizability with respect to the number of actions in a single sequence as well as spatial regions of interest within the character model. Moreover, measures are taken to reduce distortion caused by over-warping. The framework also allows for automatic selection of an optimum reference when multiple sequences are available. Experimental results demonstrate the very accurate performance of CoTW compared to other techniques such as dynamic time warping, derivative dynamic time warping and canonical time warping. The mentioned customization capabilities are also illustrated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Aggarwal, J.K., Ryoo, M.S.: Human activity analysis: a review. ACM Comput. Surv. 43(3), 16 (2011)

    Article  Google Scholar 

  2. Etemad, S.A., Arya, A.: Modeling and transformation of 3D human motion. In: Proceedings of 5th international conference on computer graphics theory and applications, pp. 307–315 (2010)

  3. Amaya, K., Bruderlin, A., Calvert, T.: Emotion from motion. In: Proceedings of graphics interface, pp. 222–229 (1996)

  4. Rose, C., Cohen, M.F., Bodenheimer, B.: Verbs and adverbs: multidimensional motion interpolation. IEEE Comput. Graph. Appl. 18(5), 32–40 (1998)

    Article  Google Scholar 

  5. Etemad, S.A., Arya, A.: Extracting movement, posture, and temporal style features from human motion. Biol. Inspir. Cognit. Archit. 7, 15–25 (2013)

    Google Scholar 

  6. Etemad, S.A., Arya, A.: Classification and translation of style and affect in human motion using RBF neural networks. Neurocomputing 129, 585–595 (2013)

    Article  Google Scholar 

  7. Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech. Signal Process. 26(1), 43–49 (1978)

    Article  MATH  Google Scholar 

  8. Zhou, F., De la Torre, F.L.: Canonical time warping for alignment of human behavior. In: Proceedings NIPS, pp. 1–9 (2009)

  9. Hsu, E., Pulli, K., Popovic, F.: Style translation for human motion. ACM Trans. Graph. 24(3), 1082–1089 (2005)

    Article  Google Scholar 

  10. Vest Nielsen, N.-P., Carstensen, J.M., Smedsgaard, J.: Aligning of single and multiple wavelength chromatographic profiles for chemometric data analysis using correlation optimised warping. J. Chromatogr. A 805, 17–35 (1998)

    Article  Google Scholar 

  11. Etemad, S.A., Arya, A.: A customizable time warping method for motion alignment. In: Proceedings of the 7th IEEE international conference on semantic computing, pp. 387–388 (2013)

  12. Etemad, S.A., Arya, A., Parush, A.: Additivity in perception of affect from limb motion. Neurosci. Lett. 558, 132–136 (2014)

    Article  Google Scholar 

  13. Fu, A.W.-C., Keogh, E., Yung Lau, L., Ratanamahatana, C.A., Chi-Wing Wong, R.: Scaling and time warping in time series querying. Very Large Data Bases J. 17(4), 899–921 (2008)

    Article  Google Scholar 

  14. Keogh, E.J., Pazzani, M.J.: Derivative dynamic time warping. In 1st SIAM international conference on data mining. (2001)

  15. Rabiner, L., Juang, B.-H.: Fundamentals of speech recognition. Prentice Hall (1993)

  16. Myers, C., Rabiner, L., Rosenberg, A.: Performance tradeoffs in dynamic time warping algorithms for isolated word recognition. IEEE Trans. Acoust. Speech Signal Process. 28(6), 623–635 (1980)

    Article  MATH  Google Scholar 

  17. Lemire, D.: Faster retrieval with a two-pass dynamic-time-warping lower bound. Pattern Recognit. 42(9), 2169–2180 (2009)

    Article  MATH  Google Scholar 

  18. Zhou, F., De la Torre, F.: Generalized time warping for multi-modal alignment of human motion. In IEEE conference on computer vision and pattern recognition, pp. 1282–1289 (2012)

  19. Shapiro, A., Cao, Y., Faloutsos, P.: Style components. In graphics interface, pp. 33–39 (2006)

  20. Liu, G., Pan, Z., Lin, Z.: Style subspaces for character animation. J. Vis. Comput. Anim. 19(3–4), 199–209 (2008)

    Google Scholar 

  21. Heloir, A., Courty, N., Gibet, S., Multon, F.: Temporal alignment of communicative gesture sequences. J. Vis. Comput. Anim. 17(3–4), 347–357 (2006)

    Google Scholar 

  22. Listgarten, J., Neal, R.M., Roweis, S.T., Emili, A.: Multiple alignment of continuous time series. In NIPS, 817–824 (2005)

  23. Brand, M., Oliver, N., Pentland, A.: Coupled hidden Markov models for complex action recognition. In IEEE conference on computer vision and pattern recognition, pp. 994–999 (1997)

  24. Brand, M., Hertzmann, A.: Style machines. In SIGGRAPH, pp. 183–192 (2000)

  25. Taylor, G.W., Hinton, G.E., Roweis, S.T.: Modeling human motion using binary latent variables. In NIPS, pp. 1345 (2007)

  26. Bruderlin, A., Williams, A.: Motion signal processing. In: Proceedings of the 22nd annual conference on computer graphics and interactive techniques, pp. 97–104 (1995)

  27. Witkin, A., Popovic, Z.: Motion warping. In: Proceedings of the 22nd annual conference on computer graphics and interactive techniques, pp. 105–108 (1995)

  28. Gleicher, M.: Retargetting motion to new characters. In: Proceedings of the 25th annual conference on computer graphics and interactive techniques, pp. 33–42 (1998)

  29. Kovar, L., Gleicher, L.: Flexible automatic motion blending with registration curves. In: Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on computer animation, pp. 214–224 (2003)

  30. Kovar, L., Gleicher, M.: Automated extraction and parameterization of motions in large data sets. ACM Trans. Graph. 23(3), 559–568 (2004)

    Article  Google Scholar 

  31. Müller, M., Röder, T., Clausen, M.: Efficient content-based retrieval of motion capture data. ACM Trans. Graph. 24(3), 677–685 (2005)

    Article  Google Scholar 

  32. Müller, M., Röder, T.: Motion templates for automatic classification and retrieval of motion capture data. In: Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on computer animation, pp. 137–146 (2006)

  33. Zhou, F., De la Torre, F., Hodgins, J.K.: Aligned cluster analysis for temporal segmentation of human motion. In 8th IEEE international conference on automatic face and gesture recognition, pp. 1–7 (2008)

  34. Kim, M., Hyun, K., Kim, J., Lee, J.: Synchronized multi-character motion editing. ACM Trans. Graph. 28(3), 79 (2009)

    Google Scholar 

  35. Raptis, M., Kirovski, D., Hoppe, H.: Real-time classification of dance gestures from skeleton animation. In: Proceedings of the 2011 ACM SIGGRAPH/Eurographics symposium on computer animation, pp. 147–156 (2011)

  36. Cimen, G., Ilhan, H., Capin, T., Gurcay, H.: Classification of human motion based on affective state descriptors. Comput. Anim. Virtual Worlds 24(3–4), 355–363 (2013)

    Article  Google Scholar 

  37. Hsu, E., da Silva, M., Popović, J.: Guided time warping for motion editing. In: Proceedings of the 2007 ACM SIGGRAPH/Eurographics symposium on computer animation, pp. 45–52 (2007)

  38. Caspi, Y., Irani, M.: Aligning non-overlapping sequences. Int. J. Comput. Vis. 48(1), 39–51 (2002)

    Article  MATH  Google Scholar 

  39. Caspi, Y., Irani, M.: Spatio-temporal alignment of sequences. IEEE Trans. Pattern Anal. Mach. Intell. 24(11), 1409–1424 (2002)

    Article  Google Scholar 

  40. Padua, F.L.C., Carceroni, R.L., Santos, G.A.M.R., Kutulakos, K.N.: Linear sequence-to-sequence alignment. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 304–320 (2010)

    Article  Google Scholar 

  41. Junejo, I.N., Dexter, E., Laptev, I., Perez, P.: View-independent action recognition from temporal self-similarities. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 172–185 (2011)

    Article  Google Scholar 

  42. Lu, C., Mandal, M.: A robust technique for motion-based video sequences temporal alignment. IEEE Trans. Multimed. 15(1), 70–82 (2013)

    Article  Google Scholar 

  43. Pravdova, V., Walczak, B., Massart, D.L.: A comparison of two algorithms for warping of analytical signals. Anal. Chim. Acta 456, 77–92 (2002)

    Article  Google Scholar 

  44. Tomasi, G., van den Berg, F., Andersson, C.: Correlation optimized warping and dynamic time warping as preprocessing methods for chromatographic data. J. Chem. 18(5), 231–241 (2004)

    Article  Google Scholar 

  45. Tomasi, G.; Practical and computational aspects in chemometric data analysis. Ph.D. Thesis, The Royal Veterinary and Agricultural University (2006)

  46. Skov, T., van den Berg, F., Tomasi, G., Bro, R.: Automated alignment of chromatographic data. J. Chem. 20, 484–497 (2006)

    Article  Google Scholar 

  47. Wang, S., Yao, J., Liu, J., Petrick, N., Van Uitert, R.L., Periaswamy, S., Summers, R.M.: Registration of prone and supine CT colonography scans using correlation optimized warping and canonical correlation analysis. Med. Phys. 36(12), 5595–5603 (2009)

    Article  Google Scholar 

  48. Yu, T., Shen, X., Li, Q., Geng, W.: Motion retrieval based on movement notation language. Comp. Anim. Virtual Worlds 16(3–4), 273–282 (2005)

    Article  Google Scholar 

  49. Tang, J.K., Leung, H., Komura, T., Shum, H.P.: Emulating human perception of motion similarity. Comput. Anim. Virtual Worlds 19(3–4), 211–221 (2008)

    Article  Google Scholar 

  50. Gray, P.O.: Psychology, 5th edn. Worth, New York (2006)

    Google Scholar 

  51. Wolfe, J.M., Kluender, K.R., Levi, D.M., Bartoshuk, L.M., Herz, R.S., Klatzky, R.L., Lederman, S.J.: Gestalt Grouping Principles. Sensation and perception. 2nd ed., Sinauer Associates. pp. 78–80 (2008)

  52. Wang, J., Yagi, Y., Makihara, Y.: People tracking and segmentation using efficient shape sequences matching. In: Zha, H., Taniguchi, R., Maybank, S. (eds) 9th Asian Conferenceon Computer Vision, vol. 5995, pp. 204–213. Springer, Berlin (2009)

  53. Wang, J., Bodenheimer, B.: Synthesis and evaluation of linear motion transitions. ACM Trans. Graph. 27(1), 1 (2008)

    Google Scholar 

  54. Wang, Y.-S., Lin, H.-C., Sorkine, O., Lee, T.-Y.: Motion-based video retargeting with optimized crop-and-warp. ACM Trans. Graph. 29(4), 90 (2010)

    Google Scholar 

  55. Prazák, M., Hoyet, L., O’Sullivan, C., Perceptual evaluation of footskate cleanup. In symposium on computer animation, pp. 287–294 (2011)

  56. Bouillaguet, C., Chen, H.C., Cheng, C.M., Chou, T., Niederhagen, R., Shamir, A., Yang, B.Y.: Fast exhaustive search for polynomial systems in F2. Cryptographic hardware and embedded systems, pp. 203–218. Springer, Berlin (2010)

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Natural Sciences and Engineering Council of Canada (NSERC) and Ontario Centers of Excellence (OCE).

Author information

Authors and Affiliations

  1. School of Information Technology, Carleton University, Ottawa, ON, Canada

    S. Ali Etemad & Ali Arya

Authors
  1. S. Ali Etemad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Arya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Ali Etemad.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1 (WMV 34,781 KB)

ESM 2 (WMV 4,513 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Etemad, S.A., Arya, A. Correlation-optimized time warping for motion. Vis Comput 31, 1569–1586 (2015). https://doi.org/10.1007/s00371-014-1034-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-014-1034-2

Keywords

Search

Navigation