Sang-Uok Kum
Abstract
The goal of tele-immersion has long been to enable people at remote locations to share a sense of presence. A tele-immersion system acquires the 3D representation of a collaborator's environment remotely and sends it over the network where it is rendered in the user's environment. Acquisition, reconstruction, transmission, and rendering all have to be done in real-time to create a sense of presence. With added commodity hardware resources, parallelism can increase the acquisition volume and reconstruction data quality while maintaining real-time performance. However, this is not as easy for rendering since all of the data need to be combined into a single display.In this article, we present an algorithm to compress data from such 3D environments in real-time to solve this imbalance. We present a compression algorithm which scales comparably to the acquisition and reconstruction, reduces network transmission bandwidth, and reduces the rendering requirement for real-time performance. This is achieved by exploiting the coherence in the 3D environment data and removing them in real-time. We have tested the algorithm using a static office data set as well as a dynamic scene, the results of which are presented in the article.
- Advanced Network and Services, Inc. http://www.advanced.org.Google Scholar
- Baker, H. H., Tanguaya, D., Sobel, I., Gelb, D., Goss, M. E., Culbertson, W. B., and Malzbender, T. 2002. The Coliseum immersive teleconferencing system. In Proceedings of the International Workshop on Immersive Telepresence 2002 (Juan Les Pins, France).Google Scholar
- Chang, C.-F., Bishop, G., and Lastra, A. 1999. LDI tree: A hierarchical representation for image-based rendering. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., Los Angeles, CA. 291--298. Google Scholar
- Chen, W.-C., Towles, H., Nyland, L., Welch, G., and Fuchs, H. 2000. Toward a compelling sensation of telepresence: Demonstrating a portal to a distant (static) office. In Proceedings of the 11th IEEE Visualization Conference (Salt Lake City, Utah). 327--333. Google Scholar
- Cruz-Neira, C., Sandin, D. J., and DeFanti, T. A. 1993. Surround-screen projection-based virtual reality: The design and implementation of the cave. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (Anaheim, Calif.). ACM, New York, 135--142. Google Scholar
- Gross, M., W&uurl;rmlin, S., Naef, M., Lamboray, E., Spagno, C., Kunz, A., Koller-Meier, E., Svoboda, T., Van Gool, L., Lang, S., Strehlke, K., Moere, A. V., and Staadt, O. 2003. Blue-c: A spatially immersive display and 3D video portal for telepresence. ACM Transactions on Graphics, Special issue (Proceedings of ACM SIGGRAPH2003) 22, 3 (July), 819--827. Google Scholar
- Grossman, J. and Dally, W. J. 1998. Point sample rendering. In Proceedings of the 9th Eurographics Workshop on Rendering Techniques (Vienna, Austria). Springer-Verlag, New York, 181--192.Google Scholar
- Jain, A. K., Murty, M. N., and Flynn, P. J. 1999. Data clustering: a review. ACM Comput. Surv. 31, 3 (Sept.), 264--323. Google Scholar
- Kauff, P. and Schreer, O. 2002. An immersive 3D video-conferencing system using shared virtual team user environments. In Proceedings of the 4th ACM International Conference on Collaborative Virtual Environments (Bonn, Germany). ACM, New York, 105--112. Google Scholar
- Kelshikar, N., Zabulis, X., Mulligan, J., Daniilidis, K., Sawant, V., Sinha, S., Sparks, T., Larsen, S., Towles, H., Mayer-Patel, K., Fuchs, H., Urbanic, J., Benninger, K., Reddy, R., and Huntoon, G. 2003. Real-time terascale implementation of tele-immersion. In Proceedings of the the Terascale Performance Analysis Workshop at International Conference on Computational Science (Melbourne, Australia). Google Scholar
- Kum, S.-U., Mayer-Patel, K., and Fuchs, H. 2003. Real-time compression for dynamic 3D environments. In Proceedings of the 11th ACM International Conference on Multimedia (Berkeley, Calif.). ACM, New York, 185--194. Google Scholar
- Matusik, W., Buehler, C., Raskar, R., Gortler, S. J., and McMillan, L. 2000. Image-based visual hulls. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (New Orleans, La.). ACM Press/Addison-Wesley Publishing Co., 369--374. Google Scholar
- McMillan, L. and Bishop, G. 1995. Plenoptic modeling: An image-based rendering system. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (Los Angeles, Calif.). ACM, New York, 39--46. Google Scholar
- Molnar, S., Eyles, J., and Poulton, J. 1992. PixelFlow: High-speed rendering using image composition. In Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques (Chicago, Ill.). ACM, New York, 231--240. Google Scholar
- Office of the Future Project. http://www.cs.unc.edu/~ootf.Google Scholar
- Pe ña, J. M., Lozano, J. A., and Larra naga, P. 1999. An empirical comparison of four initialization methods for the k-means algorithm. Patt. Reco. Lett. 20, 10 (Oct.), 1027--1040. Google Scholar
- Pittsburgh Supercomputing Center. http://www.psc.edu.Google Scholar
- Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., and Koch, R. 2004. Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59, 3, 207--232. Google Scholar
- Rusinkiewicz, S. and Levoy, M. 2000. QSplat: A multiresolution point rendering system for large meshes. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (New Orleans, La.) ACM Press/Addison-Wesley Publishing Co., 343--352. Google Scholar
- Shade, J., Gortler, S., He, L., and Szeliski, R. 1998. Layered depth images. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (Orlando, Fla.). ACM, New York, 231--242. Google Scholar
- Stoll, G., Eldridge, M., Patterson, D., Webb, A., Berman, S., Levy, R., Caywood, C., Taveira, M., Hunt, S., and Hanrahan, P. 2001. Lightning-2: A high performance display subsystem for PC clusters. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (Los Angeles, Calif.). ACM, New York, 141--148. Google Scholar
- Towles, H., Chen, W.-C., Yang, R., Kum, S.-U., Fuchs, H., Kelshikar, N., Mulligan, J., Daniilidis, K., Holden, L., Zeleznik, B., Sadagic, A., and Lanier, J. 2002. 3D tele-immersion over internet2. In Proceedings of the International Workshop on Immersive Telepresence 2002 (Juan Les Pins, France).Google Scholar
- University of Pennsylvania GRASP Lab. http://www.grasp.upenn.edu.Google Scholar
- W&uurl;rmlin, S., Lamboray, E., and Gross, M. 2004. 3D video fragments: Dynamic point samples for real-time free-viewpoint video. Computers & Graphics, Special Issue on Coding, Compression and Streaming Techniques for 3D and Multimedia Data 28, 1 (Feb.), 3--14.Google Scholar
Index Terms
- Real-time multidepth stream compression
Recommendations
Real-time compression for dynamic 3D environments
MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on MultimediaThe goal of tele-immersion has long been to enable people at remote locations to share a sense of presence. A tele-immersion system acquires the 3D representation of a collaborator's environment remotely and sends it over the network where it is ...
Real-Time Global Illumination for VR Applications
Real-time global illumination in VR systems enhances scene realism by incorporating soft shadows, reflections of objects in the scene, and color bleeding. The Virtual Light Field (VLF) method enables real-time global illumination rendering in VR. The ...
Real Time Stable Haptic Rendering Of 3D Deformable Streaming Surface
MMSys'17: Proceedings of the 8th ACM on Multimedia Systems ConferenceIn recent years, many researches are focusing on the haptic interaction with streaming data like RGBD video / point cloud stream captured by commodity depth sensors. Most previous methods use partial streaming data from depth sensors and only ...
Comments