Georgia Mastoropoulou
ABSTRACT
The developers and users of interactive computer graphics (CG), such as 3D games and virtual reality, are demanding ever more realistic computer generated imagery delivered at high frame rates, to enable a greater perceptual experience for the user. As more computational power and/or transmission bandwidth are not always available, special techniques are applied that trade off fidelity in order to reduce computational complexity, while trying to minimise the perceptibility of the resulting visual defects. Research on human visual perception has promoted the development of perception driven CG techniques, where knowledge of the human visual system and its weaknesses are exploited when rendering/displaying 3D graphics. It is well known in the human perception community that many factors, including audio stimuli, may influence the amount of cognitive resources available to perform a visual task. In this paper we investigate the influence sound effects have on the perceptibility of motion smoothness in an animation (i.e. on the perception of delivered frame rate). Forty participants viewed pairs of computer-generated walkthrough animations (with the same visual content within the pair) displayed at five different frame rates, in all possible combinations. Both walkthroughs in each test pair were either silent or accompanied by sound effects and the participant had to detect the one that had a smoother motion (i.e. was delivered at higher frame rate). A significant effect of sound effects on the perceived smoothness was revealed. The participants who watched the audiovisual walkthroughs gave more erroneous answers while performing their task compared to the subjects in the "No Sound" group, regardless of their familiarity with animated CG. Especially the unfamiliar participants failed to notice motion smoothness variations which were apparent to them in the absence of sound. The effect of the type of camera movement in the scene (translation or rotation) on the viewers' perception of the motion smoothness/jerkiness was also investigated, but no significant association between them was found. Our results should lead to new insights in 3D graphics regarding the requirements for the delivered frame rate in a wide range of applications.
- Abbey, C. K., Eckstein, M. P., and Bochud, F. O. 1999. Estimation of human-observer templates for 2 alternative forced choice tasks. In Proceedings of SPIE, vol. 3663, 284--295.Google Scholar
- Apteker, R. T., Fisher, J. A., Kisimov, V. S., and Neishlos, H. 1995. Video acceptability and frame rate. IEEE Multimedia 2, 3 (Fall), 32--40. Google ScholarDigital Library
- Blauert, J. 1997. Spatial Hearing: The Psychophysics of Human Sound Localization. M.I.T. Press, Cambridge, MA.Google Scholar
- Box, G. E. P., Hunter, W. G., and Hunter, J. S. 1978. Statistics for Experimenters. New York: Wiley.Google Scholar
- Broadbent, D. 1958. Perception and Communication. London: Pergamon Press.Google Scholar
- Carlson, N. R. 1993. Psychology, The Science of Behavior. Needham Heights, MA: Allyn and Bacon.Google Scholar
- Cater, K., Chalmers, A., and Ward, G. 2003. Detail to attention: Exploiting visual tasks for selective rendering. In Proceedings Eurographics Rendering Symposium, Leuven. Google ScholarDigital Library
- Funkhouser, T., Jot, J., and Tsingos, N. 2002. Sounds good to me! computational sound for graphics, virtual reality, and interactive systems. In SIGGRAPH 2002 Course Notes, San Antonio, TX.Google Scholar
- Larsson, P., Vastfjall, D., and Kleiner, M. 2002. Better presence and performance in virtual environments by improved binaural sound rendering. In Proceedings of the AES 22nd Intl. Conf. on virtual, synthetic and entertainment audio, Espoo, Finland, 31--38.Google Scholar
- Luebke, D., and Hallen, B. 2001. Perceptually driven simplification for interactive rendering. In Proceedings of 12th Eurographics Workshop on Rendering, 221--223. Google ScholarDigital Library
- Massaro, D. W., and Warner, D. S. 1977. Dividing attention between auditory and visual perception. Perception & Psychophysics 21, 569--574.Google ScholarCross Ref
- Mastoropoulou, G., and Chalmers, A. 2004. The effect of music on the perception of display rate and duration of animated sequences: an experimental study. In Theory and Practice of Computer Graphics 2004 (TPCG'04), 128--134. Google ScholarDigital Library
- Myszkowski, K., Tawara, T., Akamine, H., and Seidel, H.-P. 2001. Perception-guided global illumination solution for animation rendering. In Proceedings of SIGGRAPH 2001, E. F. (Ed.), Ed., Computer Graphics Proceedings, Annual Conference Series. ACM Press / ACM SIGGRAPH, New York, 221--230. Google ScholarDigital Library
- O'Sullivan, C., Howlett, S., Morvan, Y., McDonnell, R., and O'Conor, K. 2004. Star report on perceptually adaptive graphics. In Eurographics 2004, Eurographics Association.Google Scholar
- Pattanaik, S., Tumblin, J., Yee, H., and Greenberg, D. 2000. Time-dependent visual adaptation for realistic real-time image display. In Proceedings of SIGGRAPH 2000, E. Fiume, Ed., Computer Graphics Proceedings, Annual Conference Series. ACM Press / ACM SIGGRAPH, New York, 47--54. Google ScholarDigital Library
- Ramasubramanian, M., Pattanaik, S., and Greenberg, D. 1999. A perceptually based physical error metric for realistic image synthesis. In Proceedings of SIGGRAPH 1999, A. Rockwood, Ed., Computer Graphics Proceedings, Annual Conference Series. ACM Press / ACM SIGGRAPH, New York, 73--82. Google ScholarDigital Library
- Recanzone, G. H. 2003. Auditory influences on visual temporal rate perception. Journal of Neurophysiology 89, 1078--1093.Google ScholarCross Ref
- Song, H., and Kuo, C. C. J. 2001. Rate control for low-bit-rate video via variable-encoding frame rates. IEEE Transactions on Circuits and Systems for Video Technology 11, 4, 512--521. Google ScholarDigital Library
- Storms, R. L. 1998. Auditory-Visual Cross-Modal Perception Phenomena. PhD thesis, Naval Postgraduate School, Monterey, California.Google Scholar
- Tellinghuisen, D. J., and Nowak, E. J. 2003. The inability to ignore auditory distractors as a function of visual task perceptual load. Perception & Psychophysics 65, 5, 817--828.Google ScholarCross Ref
- Watson, B., Friedman, A., and McGaffey, A. 2001. Measuring and predicting visual fidelity. In Proceedings of SIGGRAPH 2001, E. Fiume, Ed., Computer Graphics Proceedings, Annual Conference Series. ACM Press / ACM SIGGRAPH, New York, 213--220. Google ScholarDigital Library
- Welch, R. B., Duttonhurt, L. D., and Warren, D. H. 1986. Contributions of audition and vision to temporal rate perception. Perceptual Psychophysiology 39, 294--300.Google ScholarCross Ref
- Winkler, S., and Faller, C. 2005. Audiovisual quality evaluation of low-bitrate video. In SPIE/IS & T Human Vision and Electronic Imaging, vol. 5666, 139--148.Google ScholarCross Ref
- Yee, H., Pattanaik, S., and Greenberg, D. 2001. Spatiotemporal sensitivity and visual attention for efficient rendering of dynamic environments. In ACM Transactions on Computer Graphics, M. Cohen, Ed., Computer Graphics Proceedings, Annual Conference Series, vol. 20, 39--65. Google ScholarDigital Library
Index Terms
- The influence of sound effects on the perceived smoothness of rendered animations
Recommendations
Auditory bias of visual attention for perceptually-guided selective rendering of animations
GRAPHITE '05: Proceedings of the 3rd international conference on Computer graphics and interactive techniques in Australasia and South East AsiaThe developers and users of real-time graphics, such as games and virtual reality, are demanding ever more realistic computer generated images. Despite the availability of modern graphics hardware, such real-time high fidelity graphics is still not ...
Enabling portable animation browsing by transforming animations into comics
IMMPD '12: Proceedings of the 2nd ACM international workshop on Interactive multimedia on mobile and portable devicesThis paper presents a media transformation system that transforms animations into comics. The comic presentation not only enhances the efficiency of animation browsing, but also enables users to browse animations just like reading comic books, which is ...
Saliency in motion: selective rendering of dynamic virtual environments
SCCG '09: Proceedings of the 25th Spring Conference on Computer GraphicsA major obstacle for real-time rendering of high-fidelity graphics is computational complexity. A key point to consider in the pursuit of "realism in real-time" in computer graphics is that the Human Visual System (HVS) is a fundamental part of the ...
Comments