Praneeth Chakravarthula
Skip Abstract Section
Abstract
Holographic displays promise to deliver unprecedented display capabilities in augmented reality applications, featuring a wide field of view, wide color gamut, spatial resolution, and depth cues all in a compact form factor. While emerging holographic display approaches have been successful in achieving large étendue and high image quality as seen by a camera, the large étendue also reveals a problem that makes existing displays impractical: the sampling of the holographic field by the eye pupil. Existing methods have not investigated this issue due to the lack of displays with large enough étendue, and, as such, they suffer from severe artifacts with varying eye pupil size and location.
We show that the holographic field as sampled by the eye pupil is highly varying for existing display setups, and we propose pupil-aware holography that maximizes the perceptual image quality irrespective of the size, location, and orientation of the eye pupil in a near-eye holographic display. We validate the proposed approach both in simulations and on a prototype holographic display and show that our method eliminates severe artifacts and significantly outperforms existing approaches.
- Kaan Akşit, Ward Lopes, Jonghyun Kim, Peter Shirley, and David Luebke. 2017. Near-eye varifocal augmented reality display using see-through screens. ACM Transactions on Graphics (TOG) 36, 6 (2017), 1--13.Google Scholar
Digital Library
- A Terry Bahill, Michael R Clark, and Lawrence Stark. 1975. The main sequence, a tool for studying human eye movements. Mathematical biosciences 24, 3--4 (1975), 191--204.Google Scholar
- S. Bahmani and J. Romberg. 2017. Phase Retrieval Meets Statistical Learning Theory: A Flexible Convex Relaxation. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research), Aarti Singh and Jerry Zhu (Eds.), Vol. 54. PMLR, Fort Lauderdale, FL, USA, 252--260.Google Scholar
- Heinz H Bauschke, Patrick L Combettes, and D Russell Luke. 2003. Hybrid projection-reflection method for phase retrieval. JOSA A 20, 6 (2003), 1025--1034.Google ScholarCross Ref
- Edward Buckley, Adrian Cable, Nic Lawrence, and Tim Wilkinson. 2006. Viewing angle enhancement for two-and three-dimensional holographic displays with random superresolution phase masks. Applied optics 45, 28 (2006), 7334--7341.Google Scholar
- Emmanuel J Candes, Thomas Strohmer, and Vladislav Voroninski. 2013. Phaselift: Exact and stable signal recovery from magnitude measurements via convex programming. Communications on Pure and Applied Mathematics 66, 8 (2013), 1241--1274.Google ScholarCross Ref
- Praneeth Chakravarthula, David Dunn, Kaan Akşit, and Henry Fuchs. 2018. Focusar: Auto-focus augmented reality eyeglasses for both real world and virtual imagery. IEEE transactions on visualization and computer graphics 24, 11 (2018), 2906--2916.Google Scholar
- Praneeth Chakravarthula, Yifan Peng, Joel Kollin, Henry Fuchs, and Felix Heide. 2019. Wirtinger holography for near-eye displays. ACM Transactions on Graphics (TOG) 38, 6 (2019), 213.Google ScholarDigital Library
- Praneeth Chakravarthula, Ethan Tseng, Henry Fuchs, and Felix Heide. 2022. Hogel-free Holography. ACM Transactions on Graphics (TOG) (2022).Google Scholar
- Praneeth Chakravarthula, Ethan Tseng, Tarun Srivastava, Henry Fuchs, and Felix Heide. 2020. Learned hardware-in-the-loop phase retrieval for holographic near-eye displays. ACM Transactions on Graphics (TOG) 39, 6 (2020), 186.Google ScholarDigital Library
- Praneeth Chakravarthula, Zhan Zhang, Okan Tursun, Piotr Didyk, Qi Sun, and Henry Fuchs. 2021. Gaze-contingent retinal speckle suppression for perceptually-matched foveated holographic displays. IEEE Transactions on Visualization and Computer Graphics 27, 11 (2021), 4194--4203.Google ScholarDigital Library
- Suyeon Choi, Manu Gopakumar, Yifan Peng, Jonghyun Kim, and Gordon Wetzstein. 2021. Neural 3D holography: Learning accurate wave propagation models for 3D holographic virtual and augmented reality displays. ACM Transactions on Graphics (TOG) 40, 6 (2021), 1--12.Google ScholarDigital Library
- Steffen Czolbe, Oswin Krause, Ingemar Cox, and Christian Igel. 2020. A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., 2051--2061.Google Scholar
- David Dunn, Cary Tippets, Kent Torell, Petr Kellnhofer, Kaan Akşit, Piotr Didyk, Karol Myszkowski, David Luebke, and Henry Fuchs. 2017. Wide field of view varifocal near-eye display using see-through deformable membrane mirrors. IEEE transactions on visualization and computer graphics 23, 4 (2017), 1322--1331.Google ScholarDigital Library
- M Hossein Eybposh, Nicholas W Caira, Mathew Atisa, Praneeth Chakravarthula, and Nicolas C Pégard. 2020. DeepCGH: 3D computer-generated holography using deep learning. Optics Express 28, 18 (2020), 26636--26650.Google ScholarCross Ref
- James R Fienup. 1982. Phase retrieval algorithms: a comparison. Applied optics 21, 15 (1982), 2758--2769.Google Scholar
- Ralph W Gerchberg. 1972. A practical algorithm for the determination of the phase from image and diffraction plane pictures. Optik 35 (1972), 237--246.Google Scholar
- Tom Goldstein and Christoph Studer. 2018. PhaseMax: Convex phase retrieval via basis pursuit. IEEE Transactions on Information Theory (2018).Google Scholar
- Joseph W Goodman. 2005. Introduction to Fourier optics. Roberts and Company Publishers.Google Scholar
- Ryoichi Horisaki, Ryosuke Takagi, and Jun Tanida. 2018. Deep-learning-generated holography. Applied optics 57, 14 (2018), 3859--3863.Google Scholar
- CK Hsueh and AA Sawchuk. 1978. Computer-generated double-phase holograms. Applied optics 17, 24 (1978), 3874--3883.Google Scholar
- Changwon Jang, Kiseung Bang, Gang Li, and Byoungho Lee. 2019. Holographic near-eye display with expanded eye-box. ACM Transactions on Graphics (TOG) 37, 6 (2019), 195.Google ScholarDigital Library
- Changwon Jang, Kiseung Bang, Seokil Moon, Jonghyun Kim, Seungjae Lee, and Byoungho Lee. 2017. Retinal 3D: augmented reality near-eye display via pupil-tracked light field projection on retina. ACM Transactions on Graphics (TOG) 36, 6 (2017), 190.Google ScholarDigital Library
- Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual Losses for Real-Time Style Transfer and Super-Resolution. In European Conference on Computer Vision (ECCV).Google Scholar
- Jonghyun Kim, Manu Gopakumar, Suyeon Choi, Yifan Peng, Ward Lopes, and Gordon Wetzstein. 2022. Holographic Glasses for Virtual Reality. In Proceedings of the ACM SIGGRAPH. 1--8.Google ScholarDigital Library
- Grace Kuo, Laura Waller, Ren Ng, and Andrew Maimone. 2020. High resolution étendue expansion for holographic displays. ACM Transactions on Graphics (TOG) 39, 4 (2020), 66.Google ScholarDigital Library
- LB Lesem, PM Hirsch, and JA Jordan. 1969. The kinoform: a new wavefront reconstruction device. IBM Journal of Research and Development 13, 2 (1969), 150--155.Google ScholarDigital Library
- Conny Lu, Praneeth Chakravarthula, Yujie Tao, Steven Chen, and Henry Fuchs. 2020. Improved vergence and accommodation via Purkinje Image tracking with multiple cameras for AR glasses. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 320--331.Google ScholarCross Ref
- D Russell Luke. 2004. Relaxed averaged alternating reflections for diffraction imaging. Inverse problems 21, 1 (2004), 37.Google Scholar
- Andrew Maimone, Andreas Georgiou, and Joel S Kollin. 2017. Holographic near-eye displays for virtual and augmented reality. ACM Transactions on Graphics (TOG) 36, 4 (2017), 85.Google ScholarDigital Library
- Andrew Maimone and Junren Wang. 2020. Holographic optics for thin and lightweight virtual reality. ACM Transactions on Graphics (TOG) 39, 4 (2020), 67--1.Google ScholarDigital Library
- Stefano Marchesini, Yu-Chao Tu, and Hau-tieng Wu. 2016. Alternating projection, ptychographic imaging and phase synchronization. Applied and Computational Harmonic Analysis 41, 3 (2016), 815--851.Google ScholarCross Ref
- Jae-Hyeung Park and Mehdi Askari. 2019. Non-hogel-based computer generated hologram from light field using complex field recovery technique from Wigner distribution function. Optics express 27, 3 (2019), 2562--2574.Google Scholar
- Yifan Peng, Suyeon Choi, Nitish Padmanaban, and Gordon Wetzstein. 2020. Neural holography with camera-in-the-loop training. ACM Transactions on Graphics (TOG) 39, 6 (2020), 185.Google ScholarDigital Library
- Yair Rivenson, Yibo Zhang, Harun Günaydın, Da Teng, and Aydogan Ozcan. 2018. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light: Science & Applications 7, 2 (2018), 17141.Google ScholarCross Ref
- Liang Shi, Beichen Li, Changil Kim, Petr Kellnhofer, and Wojciech Matusik. 2021. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591, 7849 (2021), 234--239.Google Scholar
- Koki Wakunami, Po-Yuan Hsieh, Ryutaro Oi, Takanori Senoh, Hisayuki Sasaki, Yasuyuki Ichihashi, Makoto Okui, Yi-Pai Huang, and Kenji Yamamoto. 2016. Projection-type see-through holographic three-dimensional display. Nature Communications 7 (2016).Google Scholar
- Andrew B Watson. 1993. DCT quantization matrices visually optimized for individual images. In Human vision, visual processing, and digital display IV, Vol. 1913. International Society for Optics and Photonics, 202--216.Google Scholar
- Zaiwen Wen, Chao Yang, Xin Liu, and Stefano Marchesini. 2012. Alternating direction methods for classical and ptychographic phase retrieval. Inverse Problems 28, 11 (2012), 115010.Google ScholarCross Ref
- Xinxing Xia, Yunqing Guan, Andrei State, Praneeth Chakravarthula, Tat-Jen Cham, and Henry Fuchs. 2020. Towards Eyeglass-style Holographic Near-eye Displays with Statically Expanded Eyebox. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 312--319.Google Scholar
- Jingzhao Zhang, Nicolas Pégard, Jingshan Zhong, Hillel Adesnik, and Laura Waller. 2017. 3D computer-generated holography by non-convex optimization. Optica 4, 10 (2017), 1306--1313.Google ScholarCross Ref
Index Terms
- Pupil-Aware Holography
Index terms have been assigned to the content through auto-classification.
Recommendations
Neural holography with camera-in-the-loop training
Holographic displays promise unprecedented capabilities for direct-view displays as well as virtual and augmented reality applications. However, one of the biggest challenges for computer-generated holography (CGH) is the fundamental tradeoff between ...
Hogel-Free Holography
Holography is a promising avenue for high-quality displays without requiring bulky, complex optical systems. While recent work has demonstrated accurate hologram generation of 2D scenes, high-quality holographic projections of 3D scenes has been out of ...
Interactive Holography: Pursuit of a Dream
Creating an interactive digital holographic display is a challenging task, requiring input from mathematics, physics and engineering. The main technological hurdle to develop holographic displays is sheer computational speed.
Comments