Skip to main content
SpringerLink
Log in

From Images to Virtual and Augmented Reality

  • Chapter
Confluence of Computer Vision and Computer Graphics

Part of the book series: NATO Science Series ((ASHT,volume 84))

Abstract

We describe a method to completely automatically recover 3D scene structure together with a camera for each frame from a sequence of images acquired by an unknown camera undergoing unknown movement. Previous approaches have used calibration objects or landmarks to recover this information, and are therefore often limited to a particular scale. The approach of this paper is far more general, since the “landmarks” are derived directly from the imaged scene texture. The method can be applied to a large class of scenes and motions, and is demonstrated here for sequences of interior and exterior scenes using both controlled-motion and hand-held cameras.

We demonstrate two applications of this technology. The first is the automatic construction of a 3D graphical model of a piecewise planar scene; the second is the insertion of virtual objects into the original image sequence.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. C. Baillard and A. Zisserman. Automatic reconstruction of piecewise planar models from multiple views. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pages 559–565, June 1999.

    Google Scholar 

  2. P. Beardsley, P. Torr, and A. Zisserman. 3D model acquisition from extended image sequences. In Proc. European Conference on Computer Vision, LNCS 1064/1065, pages 683–695. Springer-Verlag, 1996.

    Google Scholar 

  3. O. Faugeras. What can be seen in three dimensions with an uncalibrated stereo rig? In Proc. European Conference on Computer Vision, LNCS 588, pages 563–578. Springer-Verlag, 1992.

    Google Scholar 

  4. O. D. Faugeras, Q. Luong, and S. Maybank. Camera self-calibration: Theory and experiments. In Proc. European Conference on Computer Vision, LNCS 588, pages 321–334. Springer-Verlag, 1992.

    Google Scholar 

  5. A. W. Fitzgibbon, G. Cross, and A. Zisserman. Automatic 3D model construction for turn-table sequences. In R. Koch and L. Van Gool, editors, 3D Structure from Multiple Images of Large-Scale Environments, LNCS 1506, pages 155–170. Springer-Verlag, June 1998.

    Chapter  Google Scholar 

  6. A. W. Fitzgibbon and A. Zisserman. Automatic camera recovery for closed or open image sequences. In Proc. European Conference on Computer Vision, pages 311–326. Springer-Verlag, June 1998.

    Google Scholar 

  7. C. J. Harris and M. Stephens. A combined corner and edge detector. In Proc. 4th Alvey Vision Conference, Manchester, pages 147–151, 1988.

    Google Scholar 

  8. R. I. Hartley. A linear method for reconstruction from lines and points. In Proc. International Conference on Computer Vision, pages 882–887, 1995.

    Chapter  Google Scholar 

  9. R. I. Hartley. Lines and points in three views and the trifocal tensor. International Journal of Computer Vision, 22(2): 125–140, 1997.

    Article  Google Scholar 

  10. R. I. Hartley, R. Gupta, and T. Chang. Stereo from uncalibrated cameras. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 1992.

    Google Scholar 

  11. R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2000. (to appear).

    MATH  Google Scholar 

  12. S. Laveau. Géométrie d’un système de N caméras. Théorie, estimation et applications. PhD thesis, INRIA, 1996.

    Google Scholar 

  13. H. C. Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133–135, September 1981.

    Article  Google Scholar 

  14. P. Milgram, S. Shumin, D. Drascic, and J. Grodski. Applications of augmented reality for human-robot communication. In International Conference on Intelligent Robots and Systems Proceedings, Yokohama, Japan, pages 1467–1472, 1993.

    Google Scholar 

  15. T. Moons, D. Frère, J. Vandekerckhove, and L. Van Gool. Automatic modelling and 3D reconstruction of urban house roofs from high resolution aerial imagery. In Proc. 5th European Conference on Computer Vision, Freiburg, Germany, pages 410–425, 1998.

    Google Scholar 

  16. S. Noronha and R. Nevatia. Detection and description of buildings from multiple images. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, pages 588–594, 1997.

    Google Scholar 

  17. M. Pollefeys, R. Koch, and L. Van Gool. Self calibration and metric reconstruction in spite of varying and unknown internal camera parameters. In Proc. 6th International Conference on Computer Vision, Bombay, India, pages 90–96, 1998.

    Google Scholar 

  18. P. Pritchett and A. Zisserman. Wide baseline stereo matching. In Proc. 6th International Conference on Computer Vision, Bombay, India, pages 754–760, January 1998.

    Google Scholar 

  19. C. Schmid and A. Zisserman. Automatic line matching across views. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pages 666–671, 1997.

    Google Scholar 

  20. C. Schmid and A. Zisserman. The geometry and matching of curves in multiple views. In Proc. European Conference on Computer Vision, pages 394–409. Springer-Verlag, June 1998.

    Google Scholar 

  21. A. Shashua. Trilinearity in visual recognition by alignment. In Proc. 3rd European Conference on Computer Vision, Stockholm, volume 1, pages 479–484, May 1994.

    Google Scholar 

  22. C. Slama. Manual of Photogrammetry. American Society of Photogrammetry, Falls Church, VA, USA, 4th edition, 1980.

    Google Scholar 

  23. M. E. Spetsakis and J. Aloimonos. Structure from motion using line correspondences. International Journal of Computer Vision, 4(3):171–183, 1990.

    Article  Google Scholar 

  24. C. Tomasi and T. Kanade. Shape and motion from image streams under orthography: A factorization approach. International Journal of Computer Vision, 9(2):137–154, November 1992.

    Article  Google Scholar 

  25. P. H. S. Torr and D. W. Murray. Statistical detection of independent movement from a moving camera. Image and Vision Computing, 1(4):180–187, May 1993.

    Article  Google Scholar 

  26. P. H. S. Torr and A. Zisserman. Robust parameterization and computation of the trifocal tensor. Image and Vision Computing, 15:591–605, 1997.

    Article  Google Scholar 

  27. P. H. S. Torr and A. Zisserman. Robust computation and parameterization of multiple view relations. In Proc. 6th International Conference on Computer Vision, Bombay, India, pages 727–732, January 1998.

    Google Scholar 

  28. U. Weidner and W. Förstner. Towards automatic building extraction from high-resolution digital elevation models. ISPRS j. of Photogrammetry and Remote Sensing, 50(4):38–49, August 1995.

    Article  Google Scholar 

  29. Z. Zhang, R. Deriche, O. D. Faugeras, and Q. Luong. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence, 78:87–119, 1995.

    Article  Google Scholar 

Download references

Authors
  1. Andrew Zisserman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew Fitzgibbon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Caroline Baillard
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Geoffrey Cross
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Computer and Information Science, University of Ljubljana, Tržaška 25, SI-1000, Ljubljana, Slovenia

    Aleš Leonardis Dipl.Ing. (1985), M.Sc. (1988) and Ph.D. (1993) (Associate Professor of computer and information science) & Franc Solina Dipl.Ing. (1979), M.Sc. (1982) and Ph.D. (1987) (Professor of computer and information science) (Associate Professor of computer and information science) &  (Professor of computer and information science)

  2. GRASP Laboratory, Department of Computer and Information Science, University of Pennsylvania, 19104-6389, Philadelphia, PA, USA

    Ruzena Bajcsy Ph.D. in 1967, Ph.D. in 1972 (Director of the Directorate for Computer and Information Science and Engineering (CISE) (Director of the Directorate for Computer and Information Science and Engineering (CISE)

  3. National Science Foundation, Washington, USA

    Ruzena Bajcsy Ph.D. in 1967, Ph.D. in 1972 (Director of the Directorate for Computer and Information Science and Engineering (CISE) (Director of the Directorate for Computer and Information Science and Engineering (CISE)

Rights and permissions

Reprints and permissions

Copyright information

© 2000 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Zisserman, A., Fitzgibbon, A., Baillard, C., Cross, G. (2000). From Images to Virtual and Augmented Reality. In: Leonardis, A., Solina, F., Bajcsy, R. (eds) Confluence of Computer Vision and Computer Graphics. NATO Science Series, vol 84. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-4321-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-94-011-4321-9_1

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-0-7923-6612-6

  • Online ISBN: 978-94-011-4321-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation