Skip to main content
SpringerLink
Log in

Methods for Volumetric Reconstruction of Visual Scenes

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we present methods for 3D volumetric reconstruction of visual scenes photographed by multiple calibrated cameras placed at arbitrary viewpoints. Our goal is to generate a 3D model that can be rendered to synthesize new photo-realistic views of the scene. We improve upon existing voxel coloring/space carving approaches by introducing new ways to compute visibility and photo-consistency, as well as model infinitely large scenes. In particular, we describe a visibility approach that uses all possible color information from the photographs during reconstruction, photo-consistency measures that are more robust and/or require less manual intervention, and a volumetric warping method for application of these reconstruction methods to large-scale scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Beardsley, P., Torr, P., and Zisserman, A. 1996. 3D model acquisition from extended image sequences. In Proc. European Conference on Computer Vision, pp. 683–695.

  • Bhotika, R., Fleet, D., and Kutulakos, K. 2002.Aprobabilistic theory of occupancy and emptiness. In Proc. European Conference on Computer Vision, vol. 3, pp. 112–132.

    Google Scholar 

  • Blinn, J. and Newell, M. 1976. Texture and reflection on computer generated images. Communications of ACM, 19(10):542–547.

    Google Scholar 

  • Bolles, R., Baker, H., and Marimont, D. 1987. Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1(1):7–55.

    Google Scholar 

  • Broadhurst, A. and Cipolla, R. 2001. A probabilistic framework for space carving. In Proc. International Conference on Computer Vision, pp. 388–393.

  • Carceroni, R. and Kutulakos, K. 2001. Multi-view scene capture by surfel sampling: From video streams to non-rigid 3D motion, shape & reflectance. In Proc. International Conference on Computer Vision, pp. 60–67.

  • Chhabra, V., 2001. Reconstructing specular objects withi imagebased rendering using color caching. Master's Thesis,Worchester Polytechnic Institute.

  • Colosimo, A., Sarti, A., and Tubaro, S. 2001. Image-based object modeling: A multi-resolution level-set approach. In Proc. International Conference on Image Processing, vol. 2, pp. 181–184.

    Google Scholar 

  • Culbertson, W.B., Malzbender, T., and Slabaugh, G. 1999. Generalized voxel coloring. In Proc. ICCV Workshop, Vision Algorithms Theory and Practice, Springer-Verlag Lecture Notes in Computer Science 1883, pp. 100–115.

  • De Bonet, J. and Viola, P. 1999. Roxels: Responsibility weighted 3D volume reconstruction. In Proc. International Conference on Computer Vision, pp. 418–425.

  • Debevec, P., Taylor, C., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry and imagebased approach. In Proc. SIGGRAPH, pp. 11–20.

  • Dyer, C. 2001.Volumetric scene reconstruction from multiple views. In Foundations of Image Analysis, L.S. Davis (ed.), Kluwer: Boston, MA, pp. 469–489.

    Google Scholar 

  • Eisert, P., Steinbach, E., and Girod, B. 1999. Multi-hypothesis, volumetric reconstruction of 3-D objects from multiple calibrated camera views. In Proc. of the International Conference on Acoustics, Speech, and Signal Processing, pp. 3509–3512.

  • Eisert, P., Steinbach, E., and Girod, B. 2000. Automatic reconstruction of 3-D stationary objects from multiple uncalibrated camera views. IEEE Transactions on Circuits and Systems for Video Technology, 10(2):261–277.

    Google Scholar 

  • Faugeras, O. and Lustman, F. 1988. Motion and structure from motion in a piecewise-planar environment. Journal of Pattern Recognition and Artificial Intelligence, 2(3):485–508.

    Google Scholar 

  • Faugeras, O. and Keriven, R. 1998. Complete dense stereovision using level set methods. IEEE Transactions on Image Processing, 7(3):336–344.

    Google Scholar 

  • Fua, P. and Leclerc, Y. 1995. Object-centered surface reconstruction: Combining multi-image stereo and shading. International Journal of Computer Vision, 16(1):35–56.

    Google Scholar 

  • Gortler, S., Grzeszczuk, R., Szeliski, R., and Cohen, M. 1996. The Lumigraph. In Proc. SIGGRAPH, pp. 43–54.

  • Greene, N. 1986. Environment mapping and other applications of world projections. IEEE Computer Graphics and Applications, 21–29.

  • Jebara, T., Azarbayejani, A., and Pentland, A. 1999. 3D structure from 2D motion. IEEE Signal Processing Magazine, 16(3):66–84.

    Google Scholar 

  • Johansson, B. 1999. View synthesis and 3D reconstruction of piecewise planar scenes using intersection lines between the planes. In Proc. International Conference on Computer Vision, vol. 1, pp. 54–59.

    Google Scholar 

  • Kimura, M., Satio, H., and Kanade, T. 1999. 3D voxel construction based on epipolar geometry. In Proc. International Conference on Image Processing, pp. 135–139.

  • Kutulakos, K. and Seitz, S. 1999.A theory of shape by space carving. International Conference on Computer Vision, vol. 1, pp. 307–314.

    Google Scholar 

  • Kutulakos, K. and Seitz, S. 2000.A theory of shape by space carving. International Journal of Computer Vision, 38(3):199–218.

    Google Scholar 

  • Laurentini, A. 1994. The visual Hull concept for Silhouette-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2):150–162.

    Google Scholar 

  • Levoy, M. and Hanrahan, P. 1996. Light field rendering. In Proc. SIGGRAPH, pp. 31–42.

  • Livingston, M. and Sobel, I. 2001. A new scale arbitration algorithm for image sequences applied to cylindrical photographs. HP Laboratories technical report HPL-2001-226R1.

  • Longuet-Higgins, H. 1981. A computer algorithm for reconstructing a scene from two projections. Nature, 293(10):133–135.

    Google Scholar 

  • Matusik, W., Buehler, C., Raskar, R., Gortler, S., and McMillan, L. 2000. Image-based visual Hulls. In Proc. SIGGRAPH, pp. 367–374.

  • Max, N. 1996. Hierarchical rendering of trees from precomputed multi-layer Z-buffers. In Proc. Eurographics Rendering Workshop, pp. 165–174.

  • McMillan, M. and Bishop, G. 1995. Plenoptic modeling: An imagebased rendering system. In Proc. SIGGRAPH, pp. 39–46.

  • Narayanan, P., Rander, P., and Kanade, T. 1998. Constructing virtual worlds using dense stereo. In Proc. International Conference on Computer Vision.

  • Özün, O. 2002. Comparison of photo-consistency measures used in the voxel coloring algorithm. Masters Thesis, Middle East Technical University.

  • Pfister, H., Zwicker, M., Van Baar, J., and Gross, M. 2000. Surfels: Surface elements as rendering primitives. In Proc. SIGGRAPH, pp. 335–342.

  • Pollefeys, M., Koch, R., Vergauwen, M., and Van Gool, L. 1999. Hand-held acquisition of 3D models with a video camera. In Proc. 2nd International Conference on 3-D Digital Imaging and Modeling, pp. 14–23.

  • Prock, A. and Dyer, C. 1998. Towards real-time voxel coloring. In Proc. DARPA Image Understanding Workshop, pp. 315–321.

  • Saito, H. and Kanade, T. 1999. Shape reconstruction in projective grid space from large number of images. In Proc. CVPR, pp. 49–54.

  • Savarese, S., Rushmeier, H., Bernardini, F., and Perona, P. 2001. Shadow carving. In Proc. International Conference on Computer Vision, vol 1, pp. 190–197.

    Google Scholar 

  • Seitz, S. and Dyer, C. 1999. Photorealistic scene reconstruction by voxel coloring. International Journal of Computer Vision, 35(2):151–173.

    Google Scholar 

  • Shade, J., Gortler, S., He, L., and Szeliski, R. 1998. Layered depth images. In Proc. SIGGRAPH, pp. 231–242.

  • Shum, H.Y., Han, M., and Szeliski, R. 1998. Interactive construction of 3D models from panoramic mosaics. In Proc. CVPR, pp. 427–433.

  • Slabaugh, G., Malzbender, T., and Culbertson, W.B. 2000. Volumetric warping for voxel coloring on an infinite domain. In Proc. 3D Structure from Multiple Images for Large Scale Environments (SMILE), pp. 41–50.

  • Slabaugh, G., Culbertson, W.B., Malzbender, T., and Schafer, R. 2001. A survey of volumetric scene reconstruction methods from photographs. In Proc. International Workshop on Volume Graphics, pp. 81–100.

  • Slabaugh, G., Schafer, R., and Hans, M. 2002. Image-based photo Hulls. In Proc. 1st International Symposium on 3D Processing, Visualization, and Transmission, pp. 704–708.

  • Slabaugh, G. 2002. Novel volumetric scene reconstruction methods for new view synthesis. Ph.D. Thesis, Georgia Institute of Technology.

  • Steinbach, E., Eisert, P., Betz, A., and Girod, B. 2000. 3-D reconstruction of real world objects using extended voxels. In Proc. International Conference on Image Processing, vol. I, pp. 569–572.

    Google Scholar 

  • Stevens, M., Culbertson, W.B., and Malzbender, T. 2002. A histogram-based color consistency test for voxel coloring. In Proc. International Conference on Pattern Recognition, vol. 4, pp. 118–121.

    Google Scholar 

  • Stevens, M. 1999. Reasoning about object appearance in the context of a scene. Ph.D. Thesis, Colorado State University.

  • Szeliski, R. 1993. Rapid octree construction from image sequences. Computer Vision, Graphics and Image Processing: Image Understanding, 58(1):23–32.

    Google Scholar 

  • Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Proc. International Conference on Computer Vision, pp. 517–524.

  • Tsai, R. 1987. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Journal of Robotics and Automation, RA-3(4):323–344.

    Google Scholar 

  • Vedula, S., Baker, S., Seitz, S., and Kanade, T. 2000 Shape and motion carving in 6D. In Proc. CVPR, vol. 2, pp. 592–598.

    Google Scholar 

  • Weghorst, H., Hooper, G., and Greenberg, D.P. 1984. Improving computational methods for ray tracing. ACM Transactions on Graphics, 3(1):52–69.

    Google Scholar 

  • Yezzi, A. and Soatto, S., 2001. Stereoscopic segmentation. In Proc. International Conference on Computer Vision, pp. 59–66.

Download references

Author information

Authors and Affiliations

  1. Intelligent Vision and Reasoning Department, Siemens Corporate Research, Princeton, NJ, 08540, USA

    Gregory G. Slabaugh

  2. Visual Computing Department, Hewlett-Packard Laboratories, Palo Alto, CA, 94304, USA

    W. Bruce Culbertson & Thomas Malzbender

  3. Charles River Analytics Inc., Cambridge, MA, 02138, USA

    Mark R. Stevens

  4. Center for Signal and Image Processing, Georgia Institute of Technology, Atlanta, GA, 30318, USA

    Ronald W. Schafer

Authors
  1. Gregory G. Slabaugh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. W. Bruce Culbertson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Malzbender
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mark R. Stevens
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ronald W. Schafer
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Slabaugh, G.G., Culbertson, W.B., Malzbender, T. et al. Methods for Volumetric Reconstruction of Visual Scenes. International Journal of Computer Vision 57, 179–199 (2004). https://doi.org/10.1023/B:VISI.0000013093.45070.3b

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/B:VISI.0000013093.45070.3b

Search

Navigation