Fu-Chung Huang
Abstract
Over the last few years, virtual reality (VR) has re-emerged as a technology that is now feasible at low cost via inexpensive cellphone components. In particular, advances of high-resolution micro displays, low-latency orientation trackers, and modern GPUs facilitate immersive experiences at low cost. One of the remaining challenges to further improve visual comfort in VR experiences is the vergence-accommodation conflict inherent to all stereoscopic displays. Accurate reproduction of all depth cues is crucial for visual comfort. By combining well-known stereoscopic display principles with emerging factored light field technology, we present the first wearable VR display supporting high image resolution as well as focus cues. A light field is presented to each eye, which provides more natural viewing experiences than conventional near-eye displays. Since the eye box is just slightly larger than the pupil size, rank-1 light field factorizations are sufficient to produce correct or nearly-correct focus cues; no time-multiplexed image display or gaze tracking is required. We analyze lens distortions in 4D light field space and correct them using the afforded high-dimensional image formation. We also demonstrate significant improvements in resolution and retinal blur quality over related near-eye displays. Finally, we analyze diffraction limits of these types of displays.
Supplemental Material
Available for Download
Supplemental files
- Akeley, K., Watt, S. J., Girshick, A. R., and Banks, M. S. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) 23, 804--813. Google Scholar
Digital Library
- Benton, S., and Bove, V. 2006. Holographic Imaging. John Wiley and Sons. Google ScholarDigital Library
- Brown, D. 2000. Decentering distortion of lenses. Photogrammetric Engineering 32, 3, 444462.Google Scholar
- Cakmakci, O., and Rolland, J. 2006. Head-worn displays: a review. Journal of Display Technology 2, 3, 199--216.Google ScholarCross Ref
- Clarberg, P., Toth, R., Hasselgren, J., Nilsson, J., and Akenine-Möller, T. 2014. Amfs: Adaptive multi-frequency shading for future graphics processors. ACM Trans. Graph. 33, 4 (July), 141:1--141:12. Google ScholarDigital Library
- Cutting, J. E., and Vishton, P. M. 1995. Perceiving layout and knowing distances: The interaction, relative potency, and contextual use of different information about depth. W. Epstein and S. Rogers (Eds.), Perception of space and motion, 69--117.Google Scholar
- Favalora, G. E. 2005. Volumetric 3D displays and application infrastructure. IEEE Computer 38, 37--44. Google ScholarDigital Library
- Hale, K., and Stanney, K. 2014. Handbook of Virtual Environments. CRC Press. Google ScholarDigital Library
- Heide, F., Lanman, D., Reddy, D., Kautz, J., Pulli, K., and Luebke, D. 2014. Cascaded displays: Spatiotemporal superresolution using offset pixel layers. ACM Trans. Graph. (SIGGRAPH) 33, 4, 60:1--60:11. Google ScholarDigital Library
- Held, R., Cooper, E., and Banks, M. 2012. Blur and Disparity Are Complementary Cues to Depth. Current Biology 22, R163--R165.Google ScholarCross Ref
- Hirsch, M., Wetzstein, G., and Raskar, R. 2014. A Compressive Light Field Projection System. ACM Trans. Graph. (Proc. SIGGRAPH) 33, 4, 1--12. Google ScholarDigital Library
- Hoffman, D. M., and Banks, M. S. 2010. Focus information is used to interpret binocular images. Journal of Vision 10, 5, 13.Google ScholarCross Ref
- Hua, H., and Javidi, B. 2014. A 3d integral imaging optical see-through head-mounted display. OSA Opt. Exp. 22, 11, 13484--13491.Google ScholarCross Ref
- Huang, F.-C., Wetzstein, G., Barsky, B. A., and Raskar, R. 2014. Eyeglasses-free display: Towards correcting visual aberrations with computational light field displays. ACM Trans. Graph. (SIGGRAPH) 33, 4, 59:1--59:12. Google ScholarDigital Library
- Jones, A., McDowall, I., Yamada, H., Bolas, M., and Debevec, P. 2007. Rendering for an interactive 360° light field display. ACM Trans. Graph. (SIGGRAPH) 26, 40:1--40:10. Google ScholarDigital Library
- Kelly, D. H. 1979. Motion and vision. ii. stabilized spatiotemporal threshold surface. Journal of the Optical Society of America 69, 1340--1349.Google ScholarCross Ref
- Kress, B., and Starner, T. 2013. A review of head-mounted displays (hmd) technologies and applications for consumer electronics. In Proc. SPIE, vol. 8720, 87200A--87200A--13.Google Scholar
- Lanman, D., and Luebke, D. 2013. Near-eye light field displays. ACM Trans. Graph. (SIGGRAPH Asia) 32, 6, 220:1--220:10. Google ScholarDigital Library
- Lanman, D., Hirsch, M., Kim, Y., and Raskar, R. 2010. Content-adaptive parallax barriers: Optimizing dual-layer 3D displays using low-rank light field factorization. ACM Trans. Graph. (SIGGRAPH Asia) 29, 163:1--163:10. Google ScholarDigital Library
- Levoy, M., and Hanrahan, P. 1996. Light Field Rendering. In ACM SGGRAPH, 31--42. Google ScholarDigital Library
- Liu, S., Cheng, D., and Hua, H. 2008. An optical see-through head mounted display with addressable focal planes. In Proc. ISMAR, 33--42. Google ScholarDigital Library
- Love, G. D., Hoffman, D. M., Hands, P. J., Gao, J., Kirby, A. K., and Banks, M. S. 2009. High-speed switchable lens enables the development of a volumetric stereoscopic display. OSA Optics Express 17, 18, 15716--15725.Google ScholarCross Ref
- MacKenzie, K. J., Hoffman, D. M., and Watt, S. J. 2010. Accommodation to multiplefocalplane displays: Implications for improving stereoscopic displays and for accommodation control. Journal of Vision 10, 8.Google ScholarCross Ref
- Maimone, A., and Fuchs, H. 2013. Computational augmented reality eyeglasses. In Proc. ISMAR, 29--38.Google Scholar
- Maimone, A., Wetzstein, G., Hirsch, M., Lanman, D., Raskar, R., and Fuchs, H. 2013. Focus 3d: Compressive accommodation display. ACM Trans. Graph. 32, 5, 153:1--153:13. Google ScholarDigital Library
- Maimone, A., Lanman, D., Rathinavel, K., Keller, K., Luebke, D., and Fuchs, H. 2014. Pinlight displays: Wide field of view augmented reality eyeglasses using defocused point light sources. ACM Trans. Graph. (SIGGRAPH) 33, 4. Google ScholarDigital Library
- Marshall, J. A., Marshall, J. A., Ariely, D., Burbeck, C. A., Aricly, T. D., Rolland, J. P., and Martin, K. E. 1996. Occlusion edge blur: A cue to relative visual depth. OSA JOSAA 13, 681--688.Google ScholarCross Ref
- Marwah, K., Wetzstein, G., Bando, Y., and Raskar, R. 2013. Compressive Light Field Photography using Overcomplete Dictionaries and Optimized Projections. ACM Trans. Graph. (Proc. SIGGRAPH). Google ScholarDigital Library
- Rothbaum, B., Hodges, L., Ready, D., Graap, K., and Alarcon, R. 2001. Virtual reality exposure therapy for vietnam veterans with posttraumatic stress disorder. Ann Surg 62, 8, 617--22.Google Scholar
- Ruch, T., and Fulton, J. F. 1960. Medical physiology and biophysics. W. B. Saunders Company.Google Scholar
- Rushton, S. K., and Riddell, P. M. 1999. Developing visual systems and exposure to virtual reality and stereo displays: some concerns and speculations about the demands on accommodation and vergence. Appl Ergonomics 30, 69--78.Google ScholarCross Ref
- Ryana, L., MacKenziea, K., and Watta, S. 2012. Multiple-focal-planes 3d displays: A practical solution to the vergence-accommodation conflict? In Proc. 3D Imaging (IC3D), 1--6.Google Scholar
- Schowengerdt, B. T., and Seibel, E. J. 2006. True 3-d scanned voxel displays using single or multiple light sources. Journal of the Society for Information Display 14, 2, 135--143.Google ScholarCross Ref
- Seetzen, H., Heidrich, W., Stuerzlinger, W., Ward, G., Whitehead, L., Trentacoste, M., Ghosh, A., and Vorozcovs, A. 2004. High dynamic range display systems. ACM Trans. Graph. (SIGGRAPH) 23, 3, 760--768. Google ScholarDigital Library
- Sullivan, A. 2003. A solid-state multi-planar volumetric display. In SID Digest, vol. 32, 207--211.Google Scholar
- Sutherland, I. 1968. A head-mounted three dimensional display. In Proc. AFIPS Fall Joint Computer Conference. Google ScholarDigital Library
- Takaki, Y., Tanaka, K., and Nakamura, J. 2011. Super multi-view display with a lower resolution flat-panel display. Opt. Express 19, 5, 4129--4139.Google ScholarCross Ref
- Takaki, Y. 2006. High-Density Directional Display for Generating Natural Three-Dimensional Images. Proc. IEEE 94, 3.Google ScholarCross Ref
- Watt, S., Akeley, K., Ernst, M., and Banks, M. 2005. Focus cues affect perceived depth. Journal of Vision 5, 10, 834--862.Google ScholarCross Ref
- Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic image synthesis for attenuation-based light field and high dynamic range displays. ACM Trans. Graph. (SIGGRAPH) 30, 1--11. Google ScholarDigital Library
- Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 1--11. Google ScholarDigital Library
- Wheatstone, C. 1838. Contributions to the physiology of vision. part the first. on some remarkable, and hitherto unobserved, phenomena of binocular vision. Philosophical Transactions of the Royal Society of London 128, 371--394.Google ScholarCross Ref
- Zhang, Z. 2000. A flexible new technique for camera calibration. IEEE Trans. PAMI 22, 11, 1330--1334. Google ScholarDigital Library
- Zwicker, M., Matusik, W., Durand, F., and Pfister, H. 2006. Antialiasing for automultiscopic 3D displays. In EGSR. Google ScholarDigital Library
Index Terms
- The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues
Recommendations
The making of welcome to light fields VR
SIGGRAPH '18: ACM SIGGRAPH 2018 TalksLight fields can provide transportive immersive experiences with a level of realism unsurpassed by any other imaging technology. Within a limited viewing volume, light fields accurately reproduce stereo parallax, motion parallax, reflections, ...
DeepFocus: learned image synthesis for computational displays
Addressing vergence-accommodation conflict in head-mounted displays (HMDs) requires resolving two interrelated problems. First, the hardware must support viewing sharp imagery over the full accommodation range of the user. Second, HMDs should accurately ...
A system for acquiring, processing, and rendering panoramic light field stills for virtual reality
We present a system for acquiring, processing, and rendering panoramic light field still photography for display in Virtual Reality (VR). We acquire spherical light field datasets with two novel light field camera rigs designed for portable and ...
Comments