Sean Kelly
Sean Fanello
Paul Debevec
ABSTRACT
Motivated by recent availability of augmented and virtual reality platforms, we tackle the challenging problem of immersive storytelling experiences on mobile devices. In particular, we show an end-to-end system to generate 3D assets that enable real-time rendering of an opera on high end mobile phones. We call our system AR-ia and in this paper we walk through the main components and technical challenges of such a system, showing how to deliver an immersive mixed reality experience in every user’s living room.
- Alvaro Collet, Ming Chuang, Pat Sweeney, Don Gillett, Dennis Evseev, David Calabrese, Hugues Hoppe, Adam Kirk, and Steve Sullivan. 2015. High-quality Streamable Free-viewpoint Video. ACM TOG (2015).Google Scholar
- Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. 2000. Acquiring the Reflectance Field of a Human Face. In SIGGRAPH.Google Scholar
- Mingsong Dou, Philip Davidson, Sean Ryan Fanello, Sameh Khamis, Adarsh Kowdle, Christoph Rhemann, Vladimir Tankovich, and Shahram Izadi. 2017. Motion2Fusion: Real-time Volumetric Performance Capture. SIGGRAPH Asia (2017).Google ScholarDigital Library
- Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip Davidson, Sean Ryan Fanello, Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, Pushmeet Kohli, Vladimir Tankovich, and Shahram Izadi. 2016. Fusion4D: Real-time Performance Capture of Challenging Scenes. SIGGRAPH (2016).Google ScholarDigital Library
- Sean Ryan Fanello, Julien Valentin, Adarsh Kowdle, Christoph Rhemann, Vladimir Tankovich, Carlo Ciliberto, Philip Davidson, and Shahram Izadi. 2017a. Low Compute and Fully Parallel Computer Vision with HashMatch. In ICCV.Google Scholar
- Sean Ryan Fanello, Julien Valentin, Christoph Rhemann, Adarsh Kowdle, Vladimir Tankovich, Philip Davidson, and Shahram Izadi. 2017b. UltraStereo: Efficient Learning-based Matching for Active Stereo Systems. In CVPR.Google Scholar
- Kaiwen Guo, Peter Lincoln, Philip Davidson, Jay Busch, Xueming Yu, Matt Whalen, Geoff Harvey, Sergio Orts-Escolano, Rohit Pandey, Jason Dourgarian, Danhang Tang, Anastasia Tkach, Adarsh Kowdle, Emily Cooper, Mingsong Dou, Sean Fanello, Graham Fyffe, Christoph Rhemann, Jonathan Taylor, Paul Debevec, and Shahram Izadi. 2019. The Relightables: Volumetric Performance Capture of Humans with Realistic Relighting. In ACM TOG.Google ScholarDigital Library
- Adarsh Kowdle, Christoph Rhemann, Sean Fanello, Andrea Tagliasacchi, Jon Taylor, Philip Davidson, Mingsong Dou, Kaiwen Guo, Cem Keskin, Sameh Khamis, David Kim, Danhang Tang, Vladimir Tankovich, Julien Valentin, and Shahram Izadi. 2018. The Need 4 Speed in Real-Time Dense Visual Tracking. SIGGRAPH Asia (2018).Google Scholar
- Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L. Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A. Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, and Shahram Izadi. 2016. Holoportation: Virtual 3D Teleportation in Real-time. In UIST.Google ScholarDigital Library
- Kun Zhou, John Synder, Baining Guo, and Heung-Yeung Shum. 2004. Iso-charts: Stretch-driven Mesh Parameterization Using Spectral Analysis. In Eurographics.Google ScholarDigital Library
Index Terms
- AR-ia: Volumetric Opera for Mobile Augmented Reality
Recommendations
Augmented Winter Ski with AR HMD
AH '16: Proceedings of the 7th Augmented Human International Conference 2016At time of writing, several affordable Head-Mounted Displays (HMD) are going to be released to the mass market, most of them for Virtual Reality (VR with Oculus Rift, Samsung Gear...) but also for indoor Augmented Reality (AR) with Hololens. We have ...
Mobile collaborative augmented reality with real-time AR/VR switching
SIGGRAPH '15: ACM SIGGRAPH 2015 PostersThe recent boom in computing capabilities of mobile devices has led to the introduction of Virtual Reality into the mobile ecosystem. We demonstrate a framework for the Samsung Gear VR headset that allows developers to create a totally immersive AR & VR ...
Wearable mobile augmented reality: evaluating outdoor user experience
VRCAI '11: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in IndustryAugmented Reality (AR) technologies offer the potential to aid users in a number of professional areas. However, to date, most studies have been tested in controlled laboratory conditions. This paper outlines a user experience study of a wearable mobile ...
Comments