Abhijeet Ghosh
Paul Debevec
ABSTRACT
We present a novel process for acquiring detailed facial geometry with high resolution diffuse and specular photometric information from multiple viewpoints using polarized spherical gradient illumination. Key to our method is a new pair of linearly polarized lighting patterns which enables multiview diffuse-specular separation under a given spherical illumination condition from just two photographs. The patterns -- one following lines of latitude and one following lines of longitude -- allow the use of fixed linear polarizers in front of the cameras, enabling more efficient acquisition of diffuse and specular albedo and normal maps from multiple viewpoints. In a second step, we employ these albedo and normal maps as input to a novel multi-resolution adaptive domain message passing stereo reconstruction algorithm to create high resolution facial geometry. To do this, we formulate the stereo reconstruction from multiple cameras in a commonly parameterized domain for multiview reconstruction. We show competitive results consisting of high-resolution facial geometry with relightable reflectance maps using five DSLR cameras. Our technique scales well for multiview acquisition without requiring specialized camera systems for sensing multiple polarization states.
Supplemental Material
- Alexander, O., Rogers, M., Lambeth, W., Chiang, J.-Y., Ma, W.-C., Wang, C.-C., and Debevec, P. 2010. The Digital Emily Project: Achieving a photoreal digital actor. IEEE Computer Graphics and Applications 30 (July), 20--31. Google Scholar
Digital Library
- Beeler, T., Bickel, B., Beardsley, P., Sumner, B., and Gross, M. 2010. High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29 (July), 40:1--40:9. Google ScholarDigital Library
- Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Transactions on Graphics 26, 3, 33: 1--10. Google ScholarDigital Library
- Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. 29 (July), 41:1--41:10. Google ScholarDigital Library
- Davis, J., Nehab, D., Ramamoorthi, R., and Rusinkiewicz, S. 2005. Spacetime stereo: A unifying framework for depth from triangulation. PAMI 27, 2, 296--302. Google ScholarDigital Library
- Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proceedings of ACM SIGGRAPH 2000, 145--156. Google ScholarDigital Library
- Furukawa, Y., and Ponce, J. 2009. Dense 3D motion capture for human faces. In Proc. of CVPR 09.Google Scholar
- Fyffe, G., Hawkins, T., Watts, C., Ma, W.-C., and Debevec, P. 2011. Comprehensive facial performance capture. Computer Graphics Forum (Proc. EUROGRAPHICS) 30, 2.Google ScholarCross Ref
- Ghosh, A., Chen, T., Peers, P., Wilson, C. A., and Debevec, P. 2010. Circularly polarized spherical illumination reflectometry. ACM Trans. Graph. 29 (December), 162:1--162:12. Google ScholarDigital Library
- Hernandez, C., Vogiatzis, G., Brostow, G. J., Stenger, B., and Cipolla, R. 2007. Non-rigid photometric stereo with colored lights. In Proc. IEEE International Conference on Computer Vision, 1--8.Google Scholar
- Klaudiny, M., Hilton, A., and Edge, J. 2010. High-detail 3D capture of facial performance. In International Symposium 3D Data Processing, Visualization and Transmission (3DPVT).Google Scholar
- Kolmogorov, V. 2006. Convergent tree-reweighted message passing for energy minimization. IEEE Trans. Pattern Anal. Mach. Intell. 28 (October), 1568--1583. Google ScholarDigital Library
- Ma, W.-C., Hawkins, T., Peers, P., Chabert, C.-F., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Rendering Techniques, 183--194. Google ScholarDigital Library
- Ma, W.-C., Jones, A., Chiang, J.-Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., and Debevec, P. 2008. Facial performance synthesis using deformation-driven polynomial displacement maps. ACM TOG (Proc. SIGGRAPH Asia). Google ScholarDigital Library
- Malzbender, T., Wilburn, B., Gelb, D., and Ambrisco, B. 2006. Surface enhancement using real-time photometric stereo and reflectance transformation. In Rendering Techniques, 245--250. Google ScholarDigital Library
- Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3D geometry. ACM TOG 24, 3, 536--543. Google ScholarDigital Library
- Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3D model acquisition. ACM TOG 21, 3, 438--446. Google ScholarDigital Library
- Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. 2005. Performance relighting and reflectance transformation with time-multiplexed illumination. ACM TOG 24, 3, 756--764. Google ScholarDigital Library
- Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Jensen, H. W., and Gross, M. 2006. Analysis of human faces using a measurement-based skin reflectance model. ACM TOG 25, 3, 1013--1024. Google ScholarDigital Library
- Wilson, C. A., Ghosh, A., Peers, P., Chiang, J.-Y., Busch, J., and Debevec, P. 2010. Temporal upsampling of performance geometry using photometric alignment. ACM Trans. Graph. 29 (April), 17:1--17:11. Google ScholarDigital Library
- Woodford, O. J., Torr, P. H. S., Reid, I. D., and Fitzgibbon, A. W. 2009. Global stereo reconstruction under second order smoothness priors. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 12, 2115--2128. Google ScholarDigital Library
- Zhang, S., and Huang, P. 2006. High-resolution, real-time three-dimensional shape measurement. Optical Engineering 45, 12.Google Scholar
- Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM TOG 23, 3, 548--558. Google ScholarDigital Library
- Zhang, Z. 2000. A flexible new technique for camera calibration. PAMI 22, 11, 1330--1334. Google ScholarDigital Library
Index Terms
- Multiview face capture using polarized spherical gradient illumination
Recommendations
Multi-view facial capture using binary spherical gradient illumination
SIGGRAPH '19: ACM SIGGRAPH 2019 PostersHigh resolution facial capture has received significant attention in computer graphics due to its application in the creation of photorealistic digital humans for various applications ranging from film and VFX to games and VR. Here, the state of the art ...
Multiview face capture using polarized spherical gradient illumination
We present a novel process for acquiring detailed facial geometry with high resolution diffuse and specular photometric information from multiple viewpoints using polarized spherical gradient illumination. Key to our method is a new pair of linearly ...
Circularly polarized spherical illumination reflectometry
We present a novel method for surface reflectometry from a few observations of a scene under a single uniform spherical field of circularly polarized illumination. The method is based on a novel analysis of the Stokes reflectance field of circularly ...
Comments