Abstract
Using known camera motion to estimate depth from image sequences is an important problem in robot vision. Many applications of depth-from-motion, including navigation and manipulation, require algorithms that can estimate depth in an on-line, incremental fashion. This requires a representation that records the uncertainty in depth estimates and a mechanism that integrates new measurements with existing depth estimates to reduce the uncertainty over time. Kalman filtering provides this mechanism. Previous applications of Kalman filtering to depth-from-motion have been limited to estimating depth at the location of a sparse set of features. In this paper, we introduce a new, pixel-based (iconic) algorithm that estimates depth and depth uncertainty at each pixel and incrementally refines these estimates over time. We describe the algorithm and contrast its formulation and performance to that of a feature-based Kalman filtering algorithm. We compare the performance of the two approaches by analyzing their theoretical convergence rates, by conducting quantitative experiments with images of a flat poster, and by conducting qualitative experiments with images of a realistic outdoor-scene model. The results show that the new method is an effective way to extract depth from lateral camera translations. This approach can be extended to incorporate general motion and to integrate other sources of information, such as stereo. The algorithms we have developed, which combine Kalman filtering with iconic descriptions of depth, therefore can serve as a useful and general framework for low-level dynamic vision.
Similar content being viewed by others
References
P. Anandan, “Computing dense displacement fields with confidence measures in scenes containing occlusion,” Proc. DARPA Image Understanding Workshop, pp. 236–246, 1984.
N. Ayache and O.D. Faugeras, “Maintaining representations of the environment of a mobile robot,” Proc. 4th Intern. Symp. Robotics Res. 1987.
H.H. Baker, “Multiple-image computer vision,” Proc. 41st Photogrammetric Week, Stuttgart, West Germany, pp. 7–19, 1987.
R.C. Bolles, H.H. Baker, and D.H. Marimont, “Epipolar-plane image analysis: An approach to determining structure from motion.” Intern. J. Computer Vision 1:7–55, 1987.
T.J. Broida and R. Chellappa, “Kinematics and structure of a rigid object from a sequence of noisy images,” Proc. Workshop on Motion: Representation and Analysis. pp. 95–100. 1986.
A.R. Bruss and B.K.P. Horn, “Passive navigation,” Comput. Vision, Graphics, and Image Process. 21:3–20, 1983.
J. Canny, “A computational approach to edge detection.,” IEEE Trans. PAMI 8:679–698, 1986.
J.R. Wertz (ed.), Spacecraft Attitude Determination and Control. D. Reidel: Dordrecht, 1978.
O.D. Faugeras, N. Ayache, B. Faverjon, and F. Lustman, “Building visual maps by combining noisy stereo measurements.” Proc. IEEE Intern. Conf. Robotics and Automation, San Francisco, California pp. 1433–1438; 1986.
J. Hallam, “Resolving observer motion by object tracking,” Proc. 8th Intern. Joint Conf. Artif. Intelli. Karlsruhe, 1983.
D.J. Heeger, “Optical flow from spatiotemporal filters,” Proc. 1st Intern. Conf. Computer Vision, London, pp. 181–190. 1987.
B.K.P. Horn and B.G. Schunck, “Determining optical flow,” Artificial Intelligence, 17:185–203, 1981.
H.C. Longuet-Higgins and K. Prazdny, “The interpretation of a moving retinal image,” Proc. Roy. Soc. London B 208:385–397, 1980.
J. Marroquin, S. Mitter, and T. Poggio, “Probabilistic solution of ill-posed problems in computational vision,” J. Am. Stat. Assoc. 82:76–89, 1987.
L.H. Matthies, “Dynamic stereo.” Ph. D. thesis, Carnegie Mellon University, 1989.
L.H. Matthies and T. Kanade, “The cycle of uncertainty and constraint in robot perception,” Proc. Intern. Symp. Robotics Research, 1987.
L.H. Matthies and S.A. Shafer, “Error modeling in stereo navigation,” IEEE J. Robotics and Automation, pp. 239–248, 1987.
P.S. Maybeck, Stochastic Models, Estimation, and Control, vol. 2. Academic Press: New York, 1982.
P.S. Maybeck, Stochastic Models, Estimation, and Control, vol. 1. Academic Press, New York, 1979.
J.E.W. Mayhew and J.P. Frisby, “Psychophysical and computational studies towards a theory of human stereopsis,” Artificial Intelligence, 17:349–408, 1981.
E.M. Mikhail, Observations and Least Squares. University Press of America: Lanham, MD, 1976.
H.-H. Nagel and W. Enkelmann, “An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences,” IEEE Trans. PAMI 8:565–593, 1986.
V. Nalwa, “On detecting edges,” IEEE Trans. PAMI 8:699–714, 1986.
Y. Ohta and T. Kanade, “Stereo by intra-and inter-scanline search using dynamic programming,” IEEE Trans. PAMI 7:139–154, 1985.
P. Rives, E. Breuil, and B. Espiau, “Recursive estimation of 3d features using optical flow and camera motion,” Proc. Conf. Intelligent Autonomous Systems, pp. 522–532, 1986.
M.A. Snyder, “Uncertainty analysis of image measurements,” Proc. DARPA Image Understanding Workshop, Los Angeles, pp. 681–693, 1987.
I. Sobel, “On calibrating computer controlled cameras for perceiving 3-d scenes,” Artificial Intelligence 5:185–198, 1974.
R. Szeliski, Bayesian modeling of uncertainty in low-level vision. Ph.D. thesis, Carnegie Mellon University, 1988.
R. Szeliski, “Regularization uses fractal priors,” Proc. AAAI-87, Seattle, pp. 749–754, 1987.
Technical Staff, The Analytic Sciences Corporation, Applied Optimal Estimation, MIT Press: Cambridge, MA, 1974.
D. Terzopoulos, “Image analysis using multigrid relaxation methods,” IEEE Trans. PAMI 8:129–139, 1986.
D. Terzopoulos, “Regularization of inverse visual problems involving discontinuities,” IEEE Trans. PAMI 8:413–424, 1986.
A.M. Waxman and J.J. Duncan, “Binocular image flows,” Proc. Workshop on Motion: Representation and Analysis, Kiawah Island, SC, pp. 31–38, 1986.
G. Xu, S. Tsuji, and M. Asada, “Coarse-to-fine control strategy for matching motion stereo pairs,” Proc. IJCAI, Los Angeles, pp. 892–894, 1985.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Matthies, L., Kanade, T. & Szeliski, R. Kalman filter-based algorithms for estimating depth from image sequences. Int J Comput Vision 3, 209–238 (1989). https://doi.org/10.1007/BF00133032
Issue Date:
DOI: https://doi.org/10.1007/BF00133032