Jin-Xiang Chai
ABSTRACT
This paper studies the problem of plenoptic sampling in image-based rendering (IBR). From a spectral analysis of light field signals and using the sampling theorem, we mathematically derive the analytical functions to determine the minimum sampling rate for light field rendering. The spectral support of a light field signal is bounded by the minimum and maximum depths only, no matter how complicated the spectral support might be because of depth variations in the scene. The minimum sampling rate for light field rendering is obtained by compacting the replicas of the spectral support of the sampled light field within the smallest interval. Given the minimum and maximum depths, a reconstruction filter with an optimal and constant depth can be designed to achieve anti-aliased light field rendering.
Plenoptic sampling goes beyond the minimum number of images needed for anti-aliased light field rendering. More significantly, it utilizes the scene depth information to determine the minimum sampling curve in the joint image and geometry space. The minimum sampling curve quantitatively describes the relationship among three key elements in IBR systems: scene complexity (geometrical and textural information), the number of image samples, and the output resolution. Therefore, plenoptic sampling bridges the gap between image-based rendering and traditional geometry-based rendering. Experimental results demonstrate the effectiveness of our approach.
- 1.E.H. Adelson and J. Bergen. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing, pages 3-20. MIT Press, Cambridge, MA, 1991.Google Scholar
- 2.M. Bass, editor. Handbook of Optics. McGraw-Hill, New York, 1995.Google Scholar
- 3.R.C. Bolles, H. H. Baker, and D. H. Marimont. Epipolar-plane image analysis: An approach to determining structure from motion. International Journal of Computer Vision, 1:7-55, 1987.Google ScholarCross Ref
- 4.E. Camahort, A. Lerios, and D. Fussell. Uniformly sampled light fields. In Proc. 9th Eurographics Workshop on Rendering, pages 117-130, 1998.Google ScholarCross Ref
- 5.J.-X. Chai and H.-Y. Shum. Parallel projections for stereo reconstruction. In Proc. CVPR 2000, 2000.Google Scholar
- 6.C. Chang, G. Bishop, and A. Lastra. Ldi tree: A hierarchical representation for image-based rendering. SIGGRAPH'99, pages 291-298, August 1999. Google ScholarDigital Library
- 7.S. Chen and L. Williams. View interpolation for image synthesis. Computer Graphics (SIGGRAPH'93), pages 279-288, August 1993. Google ScholarDigital Library
- 8.T. Feng and H.-Y. Shum. An optical analysis of light field rendering. Technical report, Microsoft Research, MSR-TR-2000-38, May 2000.Google Scholar
- 9.B. Girod. Motion compensation: visual aspects, accuracy, and fundamental limits. In Motion Analysis and Image Sequence Processing. Kluwer, 1995.Google Scholar
- 10.S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen. The lumigraph. In Computer Graphics Proceedings, Annual Conference Series, pages 43-54, Proc. SIGGRAPH'96 (New Orleans), August 1996. ACM SIGGRAPH. Google ScholarDigital Library
- 11.M. Halle. Holographic stereograms as discrete imaging systems. In Proc. SPIE Vol.2176, Practical Holography VIII, pages 73-84, May 1994.Google Scholar
- 12.V. Hlavac, A. Leonardis, and T. Werner. Automatic selection of reference views for image-based scene representations. In Proc. ECCV, pages 526-535, 1996. Google ScholarDigital Library
- 13.A. Isaksen, L. McMillan, and S. Gortler. Dynamically reparameterized light fields. Technical report, Technical Report MIT-LCS-TR-778, May 1999.Google Scholar
- 14.S. Kang. A survey of image-based rendering techniques. In VideoMetrics, SPIE Vol. 3641, pages 2-16, 1999.Google Scholar
- 15.J. Lengyel. The convergence of graphics and vision. Technical report, IEEE Computer, July 1998. Google ScholarDigital Library
- 16.M. Levoy and R Hanrahan. Light field rendering. In Computer Graphics Proceedings, Annual Conference Series, pages 31-42, Proc. SIGGRAPH'96 (New Orleans), August 1996. ACM SIGGRAPH. Google ScholarDigital Library
- 17.Z.-C. Lin and H.-Y. Shum. On the numbers of samples needed in light field rendering with constant-depth assumption. In Proc. CVPR 2000, 2000.Google Scholar
- 18.W. Mark, L. McMillan, and G. Bishop. Post-rendering 3d warping. In Proc. Symposium on I3D Graphics, pages 7-16, 1997. Google ScholarDigital Library
- 19.L. McMillan and G. Bishop. Plenoptic modeling: An image-based rendering system. Computer Graphics (SIGGRAPH'95), pages 39-46, August 1995. Google ScholarDigital Library
- 20.H. Schirmacher, W. Heidrich, and H. Seidel. Adaptive acquisition of lumigraphs from synthetic scenes. In Eurographics '99, pages 151-159, Sept 1999.Google ScholarCross Ref
- 21.S. M. Seitz and C. M. Dyer. View morphing. In Computer Graphics Proceedings, Annual Conference Series, pages 21-30, Proc. SIGGRAPH'96 (New Orleans), August 1996. ACM SIGGRAPH. Google ScholarDigital Library
- 22.J. Shade, S. Gortler, L.-W. He, and R. Szeliski. Layered depth images. In Computer Graphics (SIGGRAPH'98) Proceedings, pages 231-242, Orlando, July 1998. ACM SIGGRAPH. Google ScholarDigital Library
- 23.H.-Y. Shum and L.-W. He. Rendering with concentric mosaics. In Proc. SIG- GRAPH 99, pages 299-306, 1999. Google ScholarDigital Library
- 24.P. R Sloan, M. F. Cohen, and S. J. Gortler. Time critical lumigraph rendering. In Symposium on Interactive 3D Graphics, pages 17-23, Providence, RI, USA, 1997. Google ScholarDigital Library
- 25.A. Tekal. Digital Video Processing. Prentice Hall, 1996. Google ScholarDigital Library
Index Terms
- Plenoptic sampling
Recommendations
Local light field fusion: practical view synthesis with prescriptive sampling guidelines
We present a practical and robust deep learning solution for capturing and rendering novel views of complex real world scenes for virtual exploration. Previous approaches either require intractably dense view sampling or provide little to no guidance ...
Relighting with the Reflected Irradiance Field: Representation, Sampling and Reconstruction
Image-based relighting (IBL) is a technique to change the illumination of an image-based object/scene. In this paper, we define a representation called the reflected irradiance field which records the light reflected from a scene as viewed at a fixed ...
Spectral analysis of image-based rendering data with scene geometry
The plenoptic function is commonly used to model image-based rendering (IBR) in terms of sampling and reconstruction. In this paper, we study the influence of scene geometry on the spectrum of a plenoptic function using spectral analysis. As the scene ...
Comments