Skip to main content
SpringerLink
Log in

Design and Use of Linear Models for Image Motion Analysis

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Linear parameterized models of optical flow, particularly affine models, have become widespread in image motion analysis. The linear model coefficients are straightforward to estimate, and they provide reliable estimates of the optical flow of smooth surfaces. Here we explore the use of parameterized motion models that represent much more varied and complex motions. Our goals are threefold: to construct linear bases for complex motion phenomena; to estimate the coefficients of these linear models; and to recognize or classify image motions from the estimated coefficients. We consider two broad classes of motions: i) generic “motion features” such as motion discontinuities and moving bars; and ii) non-rigid, object-specific, motions such as the motion of human mouths. For motion features we construct a basis of steerable flow fields that approximate the motion features. For object-specific motions we construct basis flow fields from example motions using principal component analysis. In both cases, the model coefficients can be estimated directly from spatiotemporal image derivatives with a robust, multi-resolution scheme. Finally, we show how these model coefficients can be use to detect and recognize specific motions such as occlusion boundaries and facial expressions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ayer, S. and Sawhney, H. 1995. Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding. In Proc. IEEE International Conference on Computer Vision, Boston, MA, pp. 777–784.

  • Bab-Hadiashar, A. and Suter, D. 1998. Robust optical flow computation. International Journal of Computer Vision, 29:59–77.

    Google Scholar 

  • Barron, J.L., Fleet, D.J., and Beauchemin, S.S. 1994. Performance of optical flowtechniques. International Journal of Computer Vision, 12(1):43–77.

    Google Scholar 

  • Baumberg, A. and Hogg, D. 1994. Learning flexible models from image sequences. In Proc. European Conf. on Computer Vision, Stockholm, Sweden, J. Eklundh, (Ed.), LNCS-Series, Vol. 800, Springer-Verlag, pp. 299–308.

  • Beauchemin, S.S. and Barron, J.L. (2000). The local frequency structure of 1d occluding image signals. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(3).

  • Bell, A.J. and Sejnowski, T.J. 1997. The independent components of natural scenes are edge filters. Vision Research, 37(23):3327–3338.

    Google Scholar 

  • Bergen, J.R., Anandan, P., Hanna, K., and Hingorani, R. 1992a. Hierarchical model-based motion estimation. In Proc. European Conf. on Comp. Vis., Springer-Verlag, pp. 237–252.

  • Bergen, J.R., Burt, P.J., Hingorani, R., and Peleg, S. 1992b. A three-frame algorithm for estimating two-component image motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(9):886–896.

    Google Scholar 

  • Beymer, D. 1996. Feature correspondence by interleaving shape and texture computations. In Proc. IEEE Computer Vision and Pattern Recognition, San Francisco, pp. 921–928.

  • Black, M., Bérard, F., Jepson, A., Newman, W., Saund, E., Socher, G., and Taylor, M. 1998. The digital office: Overview. In AAAI Spring Symposium on Intelligent Environments, Stanford, CA, pp. 1–6.

  • Black, M.J. and Anandan, P. 1990. Constraints for the early detection of discontinuity from motion. In Proc. National Conf. on Artificial Intelligence, AAAI-90, Boston, MA, pp. 1060–1066.

  • Black, M.J. and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1):75–104.

    Google Scholar 

  • Black, M.J. and Jepson, A.D. 1998a. EigenTracking: Robust matching and tracking of articulated objects using a view-based representation. International Journal of Computer Vision, 26(1):63–84.

    Google Scholar 

  • Black, M.J. and Jepson, A.D. 1998b. A probabilistic framework for matching temporal trajectories: Condensation-based recognition of gestures and expressions. In Proc. European Conf. on Computer Vision, H. Burkhardt and B. Neumann (Eds.), Freiburg, Germany, LNCS-Series, Vol. 1406, Springer-Verlag, pp. 909–924.

  • Black, M.J. and Yacoob, Y. 1997. Recognizing facial expressions in image sequences using local parameterized models of image motion. International Journal of Computer Vision, 25(1):23–48.

    Google Scholar 

  • Black, M.J., Yacoob, Y., Jepson, A.D., and Fleet, D.J. 1997. Learning parameterized models of image motion. In Proc. IEEE Computer Vision and Pattern Recognition, Puerto Rico, pp. 561–567.

  • Black, M.J. and Fleet, D.J. 1999. Probabilistic detection and tracking of motion discontinuities. In Proc. IEEE International Conference on Computer Vision, Corfu, Greece, pp. 551–558.

  • Black, M.J., Fleet, D.J., and Yacoob, Y. (2000). Robustly estimating changes in image appearance. Computer Vision and Image Understanding, 78(1):8–31.

    Google Scholar 

  • Bregler, C. and Malik, J. 1998. Tracking people with twists and exponential maps. In Proc. IEEE Computer Vision and Pattern Recognition, Santa Barbara, pp. 8–15.

  • Bregler, C. and Omohundro, S.M. 1995. Nonlinear manifold learning for visual speech recognition. In Proc. IEEE International Conference on Computer Vision, Boston, MA, pp. 494–499.

  • Burt, P.J., Bergen, J.R., Hingorani, R., Kolczynski, R., Lee, W.A., Leung, A., Lubin, J., and Shvaytser, H. 1989. Object tracking with a moving camera: An application of dynamic motion analysis. In Proc. IEEE Workshop on Visual Motion, Irvine, CA, pp. 2–12.

  • Chou, G.T. 1995. A model of figure-ground segregation from kinetic occlusion. In Proc. IEEE International Conference on Computer Vision, Boston, MA, pp. 1050–1057.

  • Cootes, T.F., Edwards, G.J., and Taylor, C.J. 1995. Active appearance models. In Proc. European Conf. on Computer Vision, H. Burkhardt and B. Neumann (Eds.), Freiburg, Germany, LNCS Series, Vol. 1406, Springer-Verlag, pp. 484–498.

  • Darrell, T. and Pentland, A. 1995. Cooperative robust estimation using layers of support. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(5):474–487.

    Google Scholar 

  • Ezzat, T. and Poggio, T. 1996. Facial analysis and synthesis using image-based models. In Proc. International Conference on Automatic Face and Gesture Recognition, Killington, Vermont, pp. 116–121.

  • Fennema, C.L. and Thompson, W.B. 1979. Velocity determination in scenes containing several moving objects. Computer Vision, Graphics, and Image Processing, 9:301–315.

    Google Scholar 

  • Fleet, D.J. and Jepson, A.D. 1990. Computation of component image velocity from local phase information. International Journal of Computer Vision, 5:77–104.

    Google Scholar 

  • Fleet, D.J. 1992. Measurement of Image Velocity. Kluwer Academic Publ, Norwell.

    Google Scholar 

  • Fleet, D.J. and Langley, K. 1994. Computational analysis of nonfourier motion. Vision Research, 22:3057–3079.

    Google Scholar 

  • Freeman, W. and Adelson, E.H. 1991. The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13:891–906.

    Google Scholar 

  • Geman, S. and McClure, D.E. 1987. Statistical methods for tomographic image reconstruction. Bulletin of the International Statistical Institute, LII-4:5–21.

    Google Scholar 

  • Golub, G.H. and van Loan, C.F. 1983. Matrix Computations. Johns Hopkins University Press: Baltimore, Maryland.

    Google Scholar 

  • Hager, G. and Belhumeur, P. 1996. Real-time tracking of image regions with changes in geometry and illumination. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, pp. 403–410.

  • Hallinan, P. 1995. A deformable model for the recognition of human faces under arbitrary illumination. Ph.D. Thesis, Harvard University, Cambridge, MA.

    Google Scholar 

  • Harris, J.G., Koch, C., Staats, E., and Luo, J. 1990. Analog hardware for detecting discontinuities in early vision. Int. Journal of Comp. Vision, 4(3):211–223.

    Google Scholar 

  • Heitz, F. and Bouthemy, P. 1993. Multimodal motion estimation of discontinuous optical flow using markov random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(12):1217–1232.

    Google Scholar 

  • Isard, M. and Blake, A. 1998. Condensation–conditional density propagation for visual tracking. International Journal of Computer Vision, 29(1):2–28.

    Google Scholar 

  • Jepson, A. and Black, M.J. 1993. Mixture models for optical flow computation. In Partitioning Data Sets: With Applications to Psychology, Vision and Target Tracking, Ingmer Cox, Pierre Hansen, and Bela Julesz (Eds.), AMS Pub.: Providence, RI, pp. 271–286. DIMACS Workshop.

    Google Scholar 

  • Ju, S.X., Black, M.J., and Jepson, A.D. 1996. Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, pp. 307–314.

  • Kearney, J.K. and Thompson, W.B. 1987. An error analysis of gradient-based methods for optical flow estimation. IEEE Pattern Analysis and Machine Intelligence, 19(2):229–244.

    Google Scholar 

  • Lucas, B.D. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. International Joint Conference on Artificial Intelligence, Vancouver, pp. 674–679.

  • Nastar, C., Moghaddam, B., and Pentland, A. 1996. Generalized image matching: Statistical learning of physically-based deformations. In Proc. European Conf. on Computer Vision, Cambridge, UK, B. Buxton and R. Cipolla (Eds.), LNCS-Series, Vol. 1064, Springer-Verlag, pp. 589–598.

  • Nayar, S.K., Baker, S., and Murase, H. 1996. Parametric feature detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA. IEEE, pp. 471–477.

  • Nelson, R.C. and Polana, R. 1992. Qualitative recognition of motion using temporal texture. CVGIP: Image Understanding, 56(1):78–89.

    Google Scholar 

  • Niyogi, S.A. 1995. Detecting kinetic occlusion. In Proc. IEEE International Conference on Computer Vision, Boston, MA, pp. 1044–1049.

  • Ong, E.P. and Spann, M. 1999. Robust optical flow computation based on least-median-of-squares regression. International Journal of Computer Vision, 31:51–82.

    Google Scholar 

  • Perona, P. 1995. Deformable kernels for early vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17:488–499.

    Google Scholar 

  • Potter, J.L. 1980. Scene segmentation using motion information. IEEE Trans. S.M.C., 5:390–394.

    Google Scholar 

  • Sclaroff, S. and Isidoro, J. 1998. Active blobs. In Proc. International Conference on Computer Vision, Mumbai, India, pp. 1146–1153.

  • Sclaroff, S. and Pentland, A.P. 1994. Physically-based combinations of views: Representing rigid and nonrigid motion. In Proceedings of the Workshop on Motion of Non-rigid and Articulated Objects, Austin, Texas, pp. 158–164.

  • Shulman, D. and Herve, J.Y. 1989. Regularization of discontinuous flowfields. In Proc. IEEEWorkshop on Visual Motion, Irvine, CA, pp. 81–86.

  • Spoerri, A. and Ullman, S. 1987. The early detection of motion boundaries. In Proc. IEEE International Conference on Computer Vision, London, UK, pp. 209–218.

  • Szeliski, R. and Coughlan, J. 1997. Spline-based image registration. International Journal of Computer Vision, 22:199–213.

    Google Scholar 

  • Szeliski, R. and Shum, H. 1996. Motion estimation with quadtree splines. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(12):1199–1211.

    Google Scholar 

  • Thompson, W.B., Mutch, K.M., and Berzins, V.A. 1985. Dynamic occlusion analysis in optical flow fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 7:374–383.

    Google Scholar 

  • Vasconcelos, N. and Lippman, A. 1998. A spatiotemporal motion model for video summerization. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, pp. 361–366.

  • Vetter, T. 1996. Learning novel views to a single face image. In Proc. International Conference on Automatic Face and Gesture Recognition, Killington, Vermont, pp. 22–27.

  • Vetter, T., Jones, M.J., and Poggio, T. 1997. A bootstrapping algorithm for learning linear models of object classes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 40–46.

  • Wang, J.Y.A. and Adelson, E.H. 1994. Representing moving images with layers. IEEE Transactions on Image Processing, 3(5):625–638.

    Google Scholar 

  • Waxman, A.M. and Wohn, K. 1985. Contour evolution, neighbourhood deformation and global image flow: Planar surfaces in motion. International Journal of Robotics Research, 4:95–108.

    Google Scholar 

  • Weiss, Y. 1997. Smoothness in layers: Motion segmentation using nonparametric mixture estimation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 520–526.

  • Weiss, Y. and Adelson, E.H. 1996. A unified mixture framework for motion segmentation: Incorporating spatial coherence and estimating the number of models. In Proc. IEEE Computer Vision and Pattern Recognition, San Francisco, pp. 321–326.

  • Wu, Y., Kanade, T., Cohn, J., and Li, C. 1998. Optical flow estimation using wavelet motion model. In Proc. IEEE International Conference on Computer Vision, Mumbai, India, pp. 992–998.

  • Yamamoto, M., Sato, S., Kuwada, S., Kondo, T., and Osaki, Y. 1998. Incremental tracking of human actions from multiple views. In Proc. IEEE Computer Vision and Pattern Recognition, Santa Barbara, pp. 2–7.

Download references

Author information

Authors and Affiliations

  1. Department of Computing and Information Science, Queen's University, Kingston, Ontario, Canada, K7L 3N6

    David J. Fleet

  2. Xerox Palo Alto Research Center, 3333 Coyote Hill Road, Palo Alto, CA, 94304, USA

    David J. Fleet & Michael J. Black

  3. Computer Vision Laboratory, University of Maryland, College Park, MD, 20742, USA

    Yaser Yacoob

  4. Department of Computer Science, University of Toronto, Toronto, Ontario, Canada, M5S 1A4

    Allan D. Jepson

Authors
  1. David J. Fleet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael J. Black
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaser Yacoob
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Allan D. Jepson
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fleet, D.J., Black, M.J., Yacoob, Y. et al. Design and Use of Linear Models for Image Motion Analysis. International Journal of Computer Vision 36, 171–193 (2000). https://doi.org/10.1023/A:1008156202475

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008156202475

Search

Navigation