Abstract
We address the structure-from-motion problem in the context of head modeling from video sequences for which calibration data is not available. This task is made challenging by the fact that correspondences are difficult to establish due to lack of texture and that a quasi-euclidean representation is required for realism.
We have developed an approach based on regularized bundle-adjustment. It takes advantage of our rough knowledge of the head's shape, in the form of a generic face model. It allows us to recover relative head-motion and epipolar geometry accurately and consistently enough to exploit a previously-developed stereo-based approach to head modeling. In this way, complete and realistic head models can be acquired with a cheap and entirely passive sensor, such as an ordinary video camera, with minimal manual intervention.
We chose to demonstrate and evaluate our technique mainly in the context of head-modeling. We do so because it is the application for which all the tools required to perform the complete reconstruction are available to us. We will, however, argue that the approach is generic and could be applied to other tasks, such as body modeling, for which generic facetized models exist.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baratoff, G. and Aloimonos, Y. 1998. Changes in surface convexity and topography caused by distorsions of stereoscopic visual space. In European Conference on Computer Vision, Freiburg, Germany, pp. 188–202.
Beardsley, P.A., Zissermam, A., and Murray, D.W. 1997. Sequential update of projective and affine structure from motion. International Journal of Computer Vision, 23(3):235–259.
Beaton, A.E. and Turkey, J.W. 1974. The fitting of power series, meaning polynomials, illustrated on band-spectroscopic data. Technometric, 16:147–185.
Blanz, V. and Vetter, T. 1999. A morphable model for the synthesis of 3-D faces. In Computer Graphics, SIGGRAPH Proceedings, Log Angeles, CA, pp. 71–78.
Boulic, R., Capin, T., Huang, Z., Moccozet, L., Molet, T., Kalra, P., Lintermann, B., Magnenat-Thalmann, N., Pandzic, I., Saar, K., Schmitt, A., Shen, J., and Thalmann, D. 1995. Environment for interactive animation of multiple deformable human characters. In Eurographics, Maastricht, Netherlands, pp. 337–348.
DeCarlo, D. and Metaxas, D. 1998. Deformable model-based shape and motion analysis from images using motion residual error. In International Conference on Computer Vision, Bombay, India, pp. 113–119.
Devernay, F. and Faugeras, O.D. 1994. Computing differential properties of 3-D shapes from stereoscopic images without 3-D models. In Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 208–213.
Faugeras, O.D., Luong, Q.-T., and Maybank, S.J. 1992. Camera selfcalibration: theory and experiments. In European Conference on Computer Vision, Santa-Margerita, Italy, pp. 321–334.
Fitzgibbon, A.W. and Zisserman, A. 1998. Automatic camera recovery for closed or open image sequences. In European Conference on Computer Vision, Freiburg, Germany, pp. 311–326.
Fua, P. 1993. A parallel stereo algorithm that produces dense depth maps and preserves image features. Machine Vision and Applications, 6(1):35–49.
Fua, P. and Leclerc, Y.G. 1995. Object-centered surface reconstruction: Combining multi-image stereo and shading. International Journal of Computer Vision, 16:35–56.
Fua, P. and Miccio C. 1998. From regular images to animated heads: A least squares approach. In European Conference on Computer Vision, Freiburg, Germany, pp. 188–202.
Fua, P. and Miccio, C. 1999. Animated heads from ordinary images: A least squares approach. Computer Vision and Image Understanding, 75(3):247–259.
Gruen, A. and Beyer, H.A. 1992. System calibration through selfcalibration. In Calibration and Orientation of Cameras in Computer Vision, Washington D.C.
Hartley, R.I., Gupta, R., and Chang, T. 1992. Stereo from uncalibrated cameras. In Conference on Computer Vision and Pattern Recognition, pp. 761–764.
Jebara, T.S. and Pentland, A. 1997. Parametrized structure from motion for 3D adaptive feedback tracking of faces. In Conference on Computer Vision and Pattern-Recognition, Porto Rico, pp. 144–150.
Kalra, P., Mangili, A., Magnenat Thalmann, N., and Thalmann, D. 1992. Simulation of facial muscle actions based on rational free form deformations. In Eurographics.
Kang, S.B. 1997. A structure from motion approach using constrained deformable models and apperance prediction. Technical Report CRL 97/6, Digial, Cambridge Research Laboratory.
Lanitis, A., Taylor, C.J., and Cootes, T.F. 1995. A united approach of coding and interpreting face images. In International Conference on Computer Vision, Cambridge, MA, pp. 975–980.
Leclerc, Y.G. and Bobick, A.F. 1991. The direct computation of height from shading. In Conference on Computer Vision and Pattern Recognition, Lahaina, Maui, Hawaii.
Lee, W.S. and Magnenat Thalmann, N. 1998. From real faces to virtual faces: Problems and solutions. In 31A, Limoges, France.
Lee, Y. Terzopoulos, D., and Waters, K. 1995. Realistic modeling for facial animation. In Computer Graphics, SIGGRAPH Proceedings, Los Angeles, CA, pp. 191–198.
Luong, Q.-T. and Viéville, T. 1996. Canonical representations for the geometries of multiple projective views. Computer Vision and Image Understanding, 64(2):193–229.
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D.H. 1998. Synthesizing realistic facial expressions from photographs. In Computer Graphics, SIGGRAPH Proceedings, Vol. 26, pp. 75–84.
Pollefeys, M., Koch, R., and VanGool, L. 1998. Self-calibration amd metric reconstruction in spite of varying and unknown internal camera parameters. In International Conference on Computer Vision.
Press, W.H., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. 1986. Numerical Recipes, the Art of Scientific Computing. Cambridge U. Press: Cambridge, MA.
Proesmans, M., Van, Gool, L., and Oosterlinck, A. 1996. Active acquisition of 3D shape for moving objects. In International Conference on Image Processing, Lausanne, Switzerland.
Samaras, D. and Metaxas, D. 1998. Incorporating illumination constraints in deformable models. In Conference on Computer Vision and Pattern Recognition, Santa Barbara, pp. 322–329.
Sturm, P. 1997. Critical motion sequences for monocular self-calibration and uncalibrated euclidean reconstruction. In Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 1100–1105.
Tang, L. and Huang, T.S. 1996. Analysis-based facial expression synthesis. ICIP-III, 94:98–102.
Tarel, J.P. and Vezien, J.M. 1996. Camcal manual: A complete software solution for camera calibration. Technical Report 0196, INRIA.
Thalmann, D., Shen, J., and Chauvineau, E. 1996. Fast realistic human body deformations for animation and VR applications. In Computer Graphics International, Pohang, Korea.
Triggs, B. 1997. Autocalibration and the absolute quadric. In Conference on Computer Vision and Pattern Recognition, pp. 609–614.
Zhang, Z., Deriche, R., Faugeras, O., and Luong, Q. 1995. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence, 78:87–119.
Zienkiewicz, O.C. 1989. The Finite Element Method. McGraw-Hill.
Zisserman, A., Liebowitz, D., and Armstrong, M. 1998. Resolving ambiguities in auto-calibration. Philosophical Transactions: Mathematical, Physical and Engineering Sciences (A), 356(1740):11931–1211.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Fua, P. Regularized Bundle-Adjustment to Model Heads from Image Sequences without Calibration Data. International Journal of Computer Vision 38, 153–171 (2000). https://doi.org/10.1023/A:1008105802790
Issue Date:
DOI: https://doi.org/10.1023/A:1008105802790