Skip to main content
SpringerLink
Log in

Reliable Estimation of Dense Optical Flow Fields with Large Displacements

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper we show that a classic optical flow technique by Nagel and Enkelmann (1986, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 8, pp. 565–593) can be regarded as an early anisotropic diffusion method with a diffusion tensor. We introduce three improvements into the model formulation that (i) avoid inconsistencies caused by centering the brightness term and the smoothness term in different images, (ii) use a linear scale-space focusing strategy from coarse to fine scales for avoiding convergence to physically irrelevant local minima, and (iii) create an energy functional that is invariant under linear brightness changes. Applying a gradient descent method to the resulting energy functional leads to a system of diffusion–reaction equations. We prove that this system has a unique solution under realistic assumptions on the initial data, and we present an efficient linear implicit numerical scheme in detail. Our method creates flow fields with 100 % density over the entire image domain, it is robust under a large range of parameter variations, and it can recover displacement fields that are far beyond the typical one-pixel limits which are characteristic for many differential methods for determining optical flow. We show that it performs better than the optical flow methods with 100 % density that are evaluated by Barron et al. (1994, Int. J. Comput. Vision, Vol. 12, pp. 43–47). Our software is available from the Internet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alvarez, L., Deriche, R., Sánchez, J., and Weickert, J. 2000. Dense disparity map estimation respecting image derivatives: A PDE and scale-space based approach. Report No. 3874, ROBOTVIS, INRIA, 2004 route des Lucioles, BP 93, 06902 Sophia-Antipolis Cedex, France.

  • Alvarez, L., Esclarin, J., Lefébure, M., and Sá0nchez, J. 1999. A PDE model for computing the optical flow. In Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, C.E.D.Y.A. XVI, Las Palmas de Gran Canaria, September 21-24, 1999, pp. 1349–1356.

  • Alvarez, L., Guichard, F., Lions, P.-L., and Morel, J.-M. 1993. Axioms and fundamental equations in image processing. Arch. Rational Mech. Anal., 123:199–257.

    Google Scholar 

  • Alvarez, L., Lions, P.-L., and Morel, J.-M. 1992. Image selective smoothing and edge detection by nonlinear diffusion II. SIAM J. Numer. Anal., 29:845–866.

    Google Scholar 

  • Alvarez, L. and Sánchez, J. 2000. 3-D geometry reconstruction using a color image stereo pair and partial differential equations. Report No. 6, Instituto Universitario de Ciencias y Tecnologias Cibernéticas, University of Las Palmas de Gran Canaria, Campus de Tafira, 35017 Las Palmas, Spain.

    Google Scholar 

  • Alvarez, L., Weickert, J., and Sánchez, J. 1999. A scale-space approach to nonlocal optical flow calculations. In Lecture Notes in Computer Science, Vol. 1682: Scale-Space Theories in Computer Vision, M. Nielsen, P. Johansen, O.F. Olsen, and J. Weickert (Eds.). Springer: Berlin, pp. 235–246.

    Google Scholar 

  • Anandan, P. 1989. A computational framework and an algorithm for the measurement of visual motion. Int. J. Comput. Vision, 2:283–310.

    Google Scholar 

  • Aubert, G., Deriche, R., and Kornprobst, P. to appear. Computing optical flow via variational techniques, in SIAM J. Math. Anal.

  • Barron, J.L., Fleet, D.J., and Beauchemin, S.S. 1994. Performance of optical flow techniques. Int. J. Comput. Vision, 12:43–77.

    Google Scholar 

  • Berger, M.-O., Deriche, R., Herlin, I., Jaffré, J., and Morel, J.-M. (Eds.). 1996. ICAOS’ 96: Images, wavelets and PDEs. Lecture Notes in Control and Information Sciences, Vol. 219. Springer: London.

  • Bergholm, F. 1987. Edge focusing, IEEE Trans. Pattern Anal. Mach. Intell., 9:726–741.

    Google Scholar 

  • Bertero, M., Poggio, T.A., and Torre, V. 1988. Ill-posed problems in early vision. Proc. IEEE, 76:869–889.

    Google Scholar 

  • Black, M.J. and Anandan, P. 1991. Robust dynamic motion estimation over time. In Proc. IEEE Comp. Soc. Conf. on Computer Vision and Pattern Recognition, CVPR’ 91, Maui, June 3-6, 1991, IEEE Computer Society Press, Los Alamitos, pp. 292–302.

    Google Scholar 

  • Black, M.J. and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise smooth flow fields. Computer Vision and Image Understanding, 63:75–104.

    Google Scholar 

  • Blake, A. and Zisserman, A. 1987. Visual Reconstruction. MIT Press: Cambridge, MA.

    Google Scholar 

  • Blanc-Féraud, L., Barlaud, M., and Gaidon, T. 1993. Motion estimation involving discontinuities in a multiresolution scheme. Optical Engineering, 32(7):1475–1482.

    Google Scholar 

  • Brezis, H. 1973. Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert. North Holland: Amsterdam.

    Google Scholar 

  • Caselles, V., Morel, J.M., Sapiro, G., and Tannenbaum, A. (Eds.). 1998. Special issue on partial differential equations and geometrydriven diffusion in image processing and analysis. IEEE Trans. Image Proc. 7(3).

  • Cohen, I. 1993. Nonlinear variational method for optical flow computation. In Proc. Eighth Scandinavian Conf. on Image Analysis, SCIA’ 93, Tromsø, May 25-28, 1993, Vol. 1, pp. 523–530.

    Google Scholar 

  • Deriche, R., Kornprobst, P., and Aubert, G. 1995. Optical-flow estimation while preserving its discontinuities: A variational approach. In Proc. Second Asian Conf. Computer Vision, ACCV’ 95, Singapore, December 5-8, 1995, Vol. 2, pp. 290–295.

    Google Scholar 

  • Enkelmann, W. 1988. Investigation of multigrid algorithms for the estimation of optical flow fields in image sequences. Computer Vision, Graphics and Image Processing, 43:150–177.

    Google Scholar 

  • Florack, L. 1997. Image Structure. Kluwer: Dordrecht.

    Google Scholar 

  • Florack, L.M.J., Niessen, W.J., and Nielsen, M. 1998. The intrinsic structure of the optic flow incorporating measurement duality. Int. J. Comput. Vision, 27:263–286.

    Google Scholar 

  • Galvin, B., McCane, B., and Novins, K. 1998. Recovering motion fields: An analysis of eight optical flow algorithms. In Proc. 1998 British Machine Vision Conference, BMVC’ 98, Southampton, September 14-17, 1998.

  • González, E., Trujillo, A., and MegaWave. A window oriented image processing software. Departamento de Informática y Sistemas, Universidad de Las Palmas. http://amiserver.dis.ulpgc.es/xmwgus/, 1994.

  • Guichard, F. and Rudin, L. 1996. Accurate estimation of discontinuous optical flow by minimizing divergence related functionals. In Proc. Third Int. Conf. on Image Processing, ICIP’ 96, Lausanne, September 1996, Vol. 1, pp. 497–500.

    Google Scholar 

  • Romeny, B. ter Haar, Florack, L., Koenderink, J., and Viergever, M. (Eds.). 1997. Scale-space theory in computer vision. Lecture Notes in Computer Science, Vol. 1252. Springer: Berlin.

  • Heitz, F. and Bouthemy, P. 1993. Multimodal estimation of discontinuous optical flow using Markov random fields. IEEE Trans. Pattern Anal. Mach. Intell., 15:1217–1232.

    Google Scholar 

  • Hinterberger, W. 1999. Generierung eines Films zwischen zwei Bildern mit Hilfe des optischen Flusses. M.Sc. Thesis, Industrial Mathematics Institute, University of Linz, Austria.

    Google Scholar 

  • Horn, B. and Schunck, B. 1981. Determining optical flow. Artificial Intelligence, 17:185–203.

    Google Scholar 

  • Iijima, T. 1962. Basic theory on normalization of pattern (in case of typical one-dimensional pattern). Bulletin of the Electrotechnical Laboratory (in Japanese), 26:368–388.

    Google Scholar 

  • Iijima, T. 1973. Pattern Recognition. (in Japanese) Corona-sha.

  • Iijima, T. 1989. Series of Basic Information Technology, Vol. 6: Theory of Pattern Recognition (in Japanese). Morishita Publishing.

  • Jähne, B. and Haussecker, H. 1998. Performance characteristics of low-level motion estimators in spatiotemporal images. In Proc. Workshop on Evaluation and Validation of Computer Vision Algorithms, Dagstuhl, March 16-20, 1998, R. Haralick, R. Klette, H.S. Stiehl, and M. Viergever (Eds.). Kluwer: Dordrecht. in press.

    Google Scholar 

  • Koenderink, J.J. 1984. The structure of images. Biological Cybernetics, 50:363–370.

    Google Scholar 

  • Kumar, A., Tannenbaum, A.R., and Balas, G.J. 1996. Optic flow: A curve evolution approach. IEEE Trans. Image Proc., 5:598–610.

    Google Scholar 

  • Lindeberg. 1994. Scale-Space Theory in ComputerVision. Kluwer: Boston.

    Google Scholar 

  • Mémin, E. and Pérez, P. 1998. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. Image Proc, 7:703–719.

    Google Scholar 

  • Mitiche, A. and Bouthemy, P. 1996. Computation and analysis of image motion: A synopsis of current problems and methods. Int. J. Comput. Vision, 19:29–55.

    Google Scholar 

  • Morton, K.W. and Mayers, L.M. 1994. Numerical Solution of Partial Differential Equations. Cambridge University Press: Cambridge.

    Google Scholar 

  • Nagel, H.-H. 1983. Constraints for the estimation of displacement vector fields from image sequences. In Proc. Eighth Int. Joint Conf. on Artificial Intelligence, IJCAI’ 83, Karls-ruhe, August 8-12, 1983, pp. 945–951.

  • Nagel, H.-H. 1987. On the estimation of optical flow: Relations between different approaches and some new results. Artificial Intelligence, 33:299–324.

    Google Scholar 

  • Nagel, H.-H. 1990. Extending the ‘oriented smoothness constraint’ into the temporal domain and the estimation of derivatives of optical flow. In Lecture Notes in Computer Science, Vol. 427: Computer Vision–ECCV’ 90, O. Faugeras (Ed.). Springer; Berlin, pp. 139–148.

    Google Scholar 

  • Nagel, H.-H. and Enkelmann, W. 1986. An investigation of smoothness constraints for the estimation of displacement vector fields from images sequences. IEEE Trans. Pattern Anal. Mach. Intell., 8:565–593.

    Google Scholar 

  • Nesi, P. 1993. Variational approach to optical flow estimation managing discontinuities. Image and Vision Computing, 11:419–439.

    Google Scholar 

  • Nielsen, M., Johansen, P., Olsen, O.F., and Weickert, J. (Eds.). 1999. Scale-space theories in computer vision. Lecture Notes in Computer Science, Vol. 1682. Springer: Berlin.

  • Otte, M. and Nagel, H.-H. 1995. Estimation of optical flow based on higher-order spatiotemporal derivatives in interlaced and noninterlaced image sequences. Artificial Intelligence, 78:5–43.

    Google Scholar 

  • Proesmans, M., Pauwels, E., and Van Gool, L. 1994a. Coupled geometry-driven diffusion equations for low-level vision. In Geometry-Driven Diffusion in Computer Vision, B.M. ter Haar Romeny (Ed.). Kluwer: Dordrecht, pp. 191–228.

    Google Scholar 

  • Proesmans, M., Van Gool, L., Pauwels, E., and Oosterlinck, A. 1994b. Determination of optical flow and its discontinuities using non-linear diffusion. In Lecture Notes in Computer Science, Vol. 801: Computer Vision–ECCV’ 94, J.-O. Eklundh (Ed.). Springer: Berlin, pp. 295–304.

    Google Scholar 

  • Schnörr, C. 1991a. Determining optical flowfor irregular domains by minimizing quadratic functionals of a certain class. Int. J. Comput. Vision, 6:25–38.

    Google Scholar 

  • Schnörr. 1991b.On functionals with greyvalue-controlled smoothness terms for determining optical flow. IEEE Trans. Pattern Anal. Mach. Intell., 15:1074–1079.

    Google Scholar 

  • Schnörr, C. 1994. Segmentation of visual motion by minimizing convex non-quadratic functionals. In Proc. 12th Int. Conf. Pattern Recognition, ICPR 12, Jerusalem, October 9-13, 1994, EEE Computer Society Press, Los Alamitos, Vol. A, pp. 661–663.

    Google Scholar 

  • Shah, J. 1993. A nonlinear diffusion model for discontinuous disparity and half-occlusions in stereo. In Proc. IEEE Comp. Soc. Conf. Computer Vision and Pattern Recognition, CVPR’ 93, New York, June 15-17, 1993, IEEE Computer Society Press, Los Alamitos, pp. 34–40.

    Google Scholar 

  • Snyder, M.A. 1991. On the mathematical foundations of smoothness constraints for the determination of optical flow and for surface reconstruction. IEEE Trans. Pattern Anal. Mach. Intell., 13:1105–1114.

    Google Scholar 

  • Sporring, J., Nielsen, M., Florack, L., and Johansen, P. (Eds.). 1997. Gaussian Scale-Space Theory. Kluwer: Dordrecht.

    Google Scholar 

  • Weickert, J. 1996. Theoretical foundations of anisotropic diffusion in image processing. Computing, Suppl. 11:221–236.

    Google Scholar 

  • Weickert, J. 1998. Anisotropic Diffusion in Image Processing. Teubner; Stuttgart.

    Google Scholar 

  • Weickert, J. 1998. On discontinuity-preserving optic flow. In Proc. Computer Vision and Mobile RoboticsWorkshop,CVMR'98, Santorini, September 17-18, 1998, S. Orphanoudakis, P. Trahanias, J. Crowley, and N. Katevas (Eds.). pp. 115–122.

  • Weickert, J., Ishikawa, S., and Imiya, A. 1999. Linear scale-space has first been proposed in Japan. J. Math. Imag. Vision, 10:237–252.

    Google Scholar 

  • Weickert, J. and Schnörr, C. 1999. Räumlich-zeitliche Berechnung des optischen Flusses mit nichtlinearen flußabhängigen Glattheitstermen. Mustererkennung 1999, W. Förstner, J.M. Buhmann, A. Faber, and P. Faber (Eds.). Springer: Berlin, pp. 317–324.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Informática y Sistemas, Universidad de Las Palmas, Campus de Tafira, SP-35017, Las Palmas, Spain

    Luis Alvarez & Javier Sánchez

  2. Computer Vision, Graphics, and Pattern Recognition Group, Department of Mathematics and Computer Science, University of Mannheim, D-68131, Mannheim, Germany

    Joachim Weickert

Authors
  1. Luis Alvarez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joachim Weickert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Javier Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Alvarez, L., Weickert, J. & Sánchez, J. Reliable Estimation of Dense Optical Flow Fields with Large Displacements. International Journal of Computer Vision 39, 41–56 (2000). https://doi.org/10.1023/A:1008170101536

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008170101536

Search

Navigation