Johannes Kopf
Richard Szeliski
Michael Goesele
Skip Abstract Section
Abstract
We propose a novel image-based rendering algorithm for handling complex scenes that may include reflective surfaces. Our key contribution lies in treating the problem in the gradient domain. We use a standard technique to estimate scene depth, but assign depths to image gradients rather than pixels. A novel view is obtained by rendering the horizontal and vertical gradients, from which the final result is reconstructed through Poisson integration using an approximate solution as a data term. Our algorithm is able to handle general scenes including reflections and similar effects without explicitly separating the scene into reflective and transmissive parts, as required by previous work. Our prototype renderer is fully implemented on the GPU and runs in real time on commodity hardware.
- Beery, E., and Yeredor, A. 2008. Blind separation of superimposed shifted images using parameterized joint diagonalization. IEEE Transactions on Image Processing 17, 3, 340--353. Google Scholar
Digital Library
- Bergen, J. R., Burt, P. J., Hingorani, R., and Peleg, S. 1992. A three-frame algorithm for estimating two-component image motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 14, 9, 886--896. Google ScholarDigital Library
- Bhat, D. N., and Nayar, S. K. 1998. Stereo and specular reflection. International Journal of Computer Vision 26, 2, 91--106. Google ScholarDigital Library
- Buehler, C., Bosse, M., McMillan, L., Gortler, S. J., and Cohen, M. F. 2001. Unstructured Lumigraph rendering. Proc. SIGGRAPH 2001, 425--432. Google ScholarDigital Library
- Carceroni, R. L., and Kutulakos, K. N. 2002. Multi-view scene capture by surfel sampling: From video streams to nonrigid 3D motion, shape and reflectance. International Journal of Computer Vision 49, 2/3, 175--214. Google ScholarDigital Library
- Chaurasia, G., Duchene, S., Sorkine-Hornung, O., and Drettakis, G. 2013. Depth synthesis and local warps for plausible image-based navigation. ACM Transactions on Graphics 32, 3, Article no. 30. Google ScholarDigital Library
- Chen, S., and Williams, L. 1993. View interpolation for image synthesis. Proc. SIGGRAPH '93, 279--288. Google ScholarDigital Library
- Criminisi, A., Kang, S. B., Swaminathan, R., Szeliski, R., and Anandan, P. 2005. Extracting layers and analyzing their specular properties using epipolar-plane-image analysis. Computer Vision and Image Understanding 97, 1, 51--85. Google ScholarDigital Library
- Debevec, P. E., Taylor, C. J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. Proc. SIGGRAPH '96, 11--20. Google ScholarDigital Library
- Diamant, Y., and Schechner, Y. Y. 2008. Overcoming visual reverberations. Proc. Computer Vision and Pattern Recognition (CVPR'08), 1--8.Google Scholar
- Eisemann, M., De Decker, B., Magnor, M., Bekaert, P., de Aguiar, E., Ahmed, N., Theobalt, C., and Sellent, A. 2008. Floating textures. Computer Graphics Forum (Proc. Eurographics 2008) 27, 2, 409--418.Google Scholar
- Goesele, M., Ackermann, J., Fuhrmann, S., Haubold, C., Klowsky, R., Steedly, D., and Szeliski, R. 2010. Ambient point clouds for view interpolation. ACM Transactions on Graphics (Proc. SIGGRAPH 2010) 29, 4, Article no. 95. Google ScholarDigital Library
- Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The Lumigraph. Proc. SIGGRAPH '96, 43--54. Google ScholarDigital Library
- Hirschmüller, H. 2008. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 2, 328--341. Google ScholarDigital Library
- Irani, M., Rousso, B., and Peleg, S. 1994. Computing occluding and transparent motions. International Journal of Computer Vision 12, 1, 5--16. Google ScholarDigital Library
- Ju, S. X., Black, M. J., and Jepson, A. D. 1996. Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. Proc. Computer Vision and Pattern Recognition (CVPR '96), 307--314. Google ScholarDigital Library
- Kolmogorov, V., and Zabih, R. 2002. Multi-camera scene reconstruction via graph cuts. Proc. European Conference on Computer Vision 2002 (ECCV 2002), 82--96. Google ScholarDigital Library
- Levin, A., Zomet, A., and Weiss, Y. 2004. Separating reflections from a single image using local features. Proc. Computer Vision and Pattern Recognition 2004 (CVPR 2004) 1, 306--313.Google Scholar
- Levoy, M., and Hanrahan, P. 1996. Light field rendering. Proc. SIGGRAPH '96, 31--42. Google ScholarDigital Library
- Linz, C., Lipski, C., and Magnor, M. 2010. Multi-image interpolation based on graph-cuts and symmetric optical flow. Proc. Vision, Modeling and Visualization 2010 (VMV 2010), 115--122.Google Scholar
- Loop, C., and Zhang, Z. 1999. Computing rectifying homographies for stereo vision. Proc. Computer Vision and Pattern Recognition (CVPR '99), 125--131.Google Scholar
- Mahajan, D., Huang, F.-C., Matusik, W., Ramamoorthi, R., and Belhumeur, P. 2009. Moving gradients: a path-based method for plausible image interpolation. ACM Transactions on Graphics (Proc. SIGGRAPH 2009) 28, 3, Article no. 42. Google ScholarDigital Library
- Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics (Proc. SIGGRAPH 2003) 22, 3, 313--318. Google ScholarDigital Library
- Popescu, V., Mei, C., Dauble, J., and Sacks, E. 2006. Reflected-scene impostors for realistic reflections at interactive rates. Computer Graphics Forum (Proc. Eurographics 2006) 25, 3, 313--322.Google Scholar
- Schechner, Y. Y., Shamir, J., and Kiryati, N. 1999. Polarization-based decorrelation of transparent layers: The inclination angle of an invisible surface. Proc. International Conference on Computer Vision (ICCV '99), 814--819. Google ScholarDigital Library
- Schechner, Y. Y., Kiryati, N., and Shamir, J. 2000. Blind recovery of transparent and semireflected scenes. Proc. Computer Vision and Pattern Recognition (CVPR 2000), 38--43.Google Scholar
- Shade, J., Gortler, S., He, L., and Szeliski, R. 1998. Layered depth images. Proc. SIGGRAPH '98, 231--242. Google ScholarDigital Library
- Shewchuk, J. R. 1994. An introduction to the conjugate gradient method without the agonizing pain. Tech. rep., Carnegie Mellon University. Google ScholarDigital Library
- Shizawa, M., and Mase, K. 1991. A unified computational theory of motion transparency and motion boundaries based on eigenenergy analysis. Proc. Computer Vision and Pattern Recognition (CVPR '91), 289--295.Google Scholar
- Shum, H.-Y., Chan, S.-C., and Kang, S. B. 2007. Image-Based Rendering. Springer, New York, NY. Google ScholarDigital Library
- Sinha, S. N., Steedly, D., and Szeliski, R. 2009. Piecewise planar stereo for image-based rendering. Proc. International Conference on Computer Vision (ICCV 2009), 1881--1888.Google Scholar
- Sinha, S. N., Kopf, J., Goesele, M., Scharstein, D., and Szeliski, R. 2012. Image-based rendering for scenes with reflections. ACM Transactions on Graphics (Proc. SIGGRAPH 2012) 31, 4, Article no. 100. Google ScholarDigital Library
- Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3D. ACM Transactions on Graphics (Proc. SIGGRAPH 2006) 25, 3, 835--846. Google ScholarDigital Library
- Szeliski, R., Avidan, S., and Anandan, P. 2000. Layer extraction from multiple images containing reflections and transparency. Proc. Computer Vision and Pattern Recognition (CVPR 2000), 246--253.Google Scholar
- Tsin, Y., Kang, S. B., and Szeliski, R. 2006. Stereo matching with linear superposition of layers. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 2, 290--301. Google ScholarDigital Library
- Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM Transactions on Graphics (Proc. SIGGRAPH 2004) 23, 3, 600--608. Google ScholarDigital Library
Index Terms
- Image-based rendering in the gradient domain
Recommendations
Image-based rendering for scenes with reflections
We present a system for image-based modeling and rendering of real-world scenes containing reflective and glossy surfaces. Previous approaches to image-based rendering assume that the scene can be approximated by 3D proxies that enable view ...
Image-based rendering of diffuse, specular and glossy surfaces from a single image
SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniquesIn this paper, we present a new method to recover an approximation of the bidirectional reflectance distribution function (BRDF) of the surfaces present in a real scene. This is done from a single photograph and a 3D geometric model of the scene. The ...
An introduction to image-based rendering
Integrated image and graphics technologiesIn this chapter, we review the techniques for image-based rendering. Unlike traditional 3D computer graphics in which 3D geometry of the scene is known, image-based rendering (IBR) techniques render novel views directly from input images. IBR techniques ...
Comments