Nicolas Bonneel
Clara Suied
George Drettakis
Abstract
High-quality rendering of both audio and visual material properties is very important in interactive virtual environments, since convincingly rendered materials increase realism and the sense of immersion. We studied how the level of detail of auditory and visual stimuli interact in the perception of audio-visual material rendering quality. Our study is based on perception of material discrimination, when varying the levels of detail of modal synthesis for sound, and bidirectional reflectance distribution functions for graphics. We performed an experiment for two different models (a Dragon and a Bunny model) and two material types (plastic and gold). The results show a significant interaction between auditory and visual level of detail in the perception of material similarity, when comparing approximate levels of detail to a high-quality audio-visual reference rendering. We show how this result can contribute to significant savings in computation time in an interactive audio-visual rendering system. To our knowledge, this is the first study that shows interaction of audio and graphics representation in a material perception task.
- Ben-Artzi, A., Overbeck, R., and Ramamoorthi, R. 2006. Real-time brdf editing in complex lighting. In Proceedings of the SIGGRAPH Conference. ACM, New York, 945--954. Google Scholar
Digital Library
- Bonneel, N., Drettakis, G., Tsingos, N., Viaud-Delmon, I., and James, D. 2008. Fast modal sounds with scalable frequency-domain synthesis. In Proceedings of the SIGGRAPH Conference. ACM, New York. Google ScholarDigital Library
- Cook, R. L. and Torrance, K. E. 1982. A reflectance model for computer graphics. ACM Trans. Graph. 1, 1, 7--24. Google ScholarDigital Library
- Debevec, P. 1998. Rendering with natural light. In ACM SIGGRAPH 98 Electronic Art and Animation Catalog. ACM Press, New York, NY, 166. Google ScholarDigital Library
- Doel, K. V. D., Pai, D., Adam, T., Kortchmar, L., and Pichora-Fuller, K. 2002. Measurements of perceptual quality of contact sound models. In Proceedings of the International Conference on Auditory Display. 345--349.Google Scholar
- Donnelly, W. and Lauritzen, A. 2006. Variance shadow maps. In Proceedings of the 2006 Symposium on Interactive 3D Graphics and Games. ACM, New York, NY, 161--165. Google ScholarDigital Library
- Fleming, R. W., Dror, R. O., and Adelson, E. H. 2003. Real-world illumination and the perception of surface reflectance properties. J. Vision 3, 5, 347--368.Google ScholarCross Ref
- Fujisaki, W., Shimojo, S., Kashino, M., and Nishida, S. 2004. Recalibration of audiovisual simultaneity. Nat Neurosci 7, 7, 773--778.Google ScholarCross Ref
- Giordano, B. L. and McAdams, S. 2006. Material identification of real impact sounds: Effects of size variation in steel, glass, wood, and plexiglass plates. J. Acoustical Soc. Amer. 119, 2, 1171--1181.Google ScholarCross Ref
- Green, R. 2003. Spherical harmonic lighting: The gritty details. In Proceedings of the Game Developers Conference.Google Scholar
- Grelaud, D., Bonneel, N., Wimmer, M., Asselot, M., and Drettakis, G. 2009. Efficient and practical audio-visual rendering for games using crossmodal perception. InProceedings of the Symposium on Interactive 3D Graphics and Games. ACM, New York, NY. Google ScholarDigital Library
- James, D. L., Barbic, J., and Pai, D. K. 2006. Precomputed acoustic transfer: Output-sensitive, accurate sound generation for geometrically complex vibration sources. ACM Trans. Graph. 25, 3, 987--995. Google ScholarDigital Library
- Kautz, J., Sloan, P., and Snyder, J. 2002. Fast, arbitrary shading for low-frequency lighting using spherical harmonics. In Proceedings of the Eurographics Workshop on Rendering. 291--296. Google ScholarDigital Library
- Klatzky, R., Pai, D., and Krotkov, E. 2000. Perception of material from contact sounds. Presence: Teleoperators Virtual Environ. 9, 399--410. Google ScholarDigital Library
- Kristensen, A. W., Akenine-Möller, T., and Jensen, H. W. 2005. Precomputed local radiance transfer for real-time lighting design. ACM Trans. Graph. 24, 3, 1208--1215. Google ScholarDigital Library
- Mastoropoulou, G., Debattista, K., Chalmers, A., and Troscianko, T. 2005. The influence of sound effects on the perceived smoothness of rendered animations. In Proceedings of the Symposium on Applied Perception in Graphics and Visualization. ACM, New York, NY, 9--15. Google ScholarDigital Library
- Matusik, W., Pfister, H., Brand, M., and McMillan, L. 2003. A data-driven reflectance model. ACM Trans. Graph. 22, 3, 759--769. Google ScholarDigital Library
- McAdams, S., Chaigne, A., and Roussarie, V. 2004. The psycho-mechanics of simulated sound sources: Material properties of impacted bars. J. Acoustical Soc. Amer. 115, 1306--1320.Google ScholarCross Ref
- Ng, R., Ramamoorthi, R., and Hanrahan, P. 2003. All-frequency shadows using non-linear wavelet lighting approximation. ACM Trans. Graph. 22, 3, 376--381. Google ScholarDigital Library
- O'Brien, J. F., Shen, C., and Gatchalian, C. M. 2002. Synthesizing sounds from rigid-body simulations. In Proceedings of the Symposium on Computer Animation. ACM Press, 175--181. Google ScholarDigital Library
- Raghuvanshi, N. and Lin, M. C. 2007. Physically based sound synthesis for large-scale virtual environments. IEEE Comput. Graph. Appl. 27, 1, 14--18. Google ScholarDigital Library
- Ramamoorthi, R. and Hanrahan, P. 2001a. An efficient representation for irradiance environment maps. In Computer Graphics Proceedings, E. Fiume, Ed. 497--500. Google ScholarDigital Library
- Ramamoorthi, R. and Hanrahan, P. 2001b. On the relationship between radiance and irradiance: Determining the illumination from images of a convex object. J. Optical Soc. Amer.Google Scholar
- Ramamoorthi, R. and Hanrahan, P. 2002. Frequency space environment map rendering. ACM Trans. Graph. 21, 3, 517--526. Google ScholarDigital Library
- Ramanarayanan, G., Ferwerda, J., Walter, B., and Bala, K. 2007. Visual equivalence: Towards a new standard for image fidelity. ACM Trans. Graph. Google ScholarDigital Library
- Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. ACM Trans. Graph. 21, 3, 267--276. Google ScholarDigital Library
- Sloan, P.-P., Kautz, J., and Snyder, J. 2002. Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments. In Proceedings of ACM SIGGRAPH, 527--536. Google ScholarDigital Library
- Sloan, P.-P., Luna, B., and Snyder, J. 2005. Local, deformable precomputed radiance transfer. ACM Trans. Graph. 24, 3, 1216--1224. Google ScholarDigital Library
- Storms, R. L. and Zyda, M. 2000. Interactions in perceived quality of auditory-visual displays. Presence: Teleoperators Virtual Environ. 9, 6, 557--580. Google ScholarDigital Library
- van den Doel, K. and Pai, D. K. 2003. Modal synthesis for vibrating objects. Audio Anecdotes.Google Scholar
- Vangorp, P., Laurijssen, J., and DUTRÉ, P. 2007. The influence of shape on the perception of material reflectance. ACM Trans. Graph. Google ScholarDigital Library
Index Terms
- Bimodal perception of audio-visual material properties for virtual environments
Recommendations
Audio, visual, and audio-visual egocentric distance perception by moving subjects in virtual environments
We present a study on audio, visual, and audio-visual egocentric distance perception by moving subjects in virtual environments. Audio-visual rendering is provided using tracked passive visual stereoscopy and acoustic wave field synthesis (WFS). ...
Efficient and practical audio-visual rendering for games using crossmodal perception
I3D '09: Proceedings of the 2009 symposium on Interactive 3D graphics and gamesInteractive applications such as computer games, are inherently audio visual, requiring high-quality rendering of complex 3D audio soundscapes and graphics environments. A frequent source of audio events is impact sounds, typically generated with ...
Predicting Visual Perception of Material Structure in Virtual Environments
One of the most accurate yet still practical representation of material appearance is the Bidirectional Texture Function BTF. The BTF can be viewed as an extension of Bidirectional Reflectance Distribution Function BRDF for additional spatial ...
Comments