Rafal Mantiuk
Abstract
Due to rapid technological progress in high dynamic range (HDR) video capture and display, the efficient storage and transmission of such data is crucial for the completeness of any HDR imaging pipeline. We propose a new approach for inter-frame encoding of HDR video, which is embedded in the well-established MPEG-4 video compression standard. The key component of our technique is luminance quantization that is optimized for the contrast threshold perception in the human visual system. The quantization scheme requires only 10--11 bits to encode 12 orders of magnitude of visible luminance range and does not lead to perceivable contouring artifacts. Besides video encoding, the proposed quantization provides perceptually-optimized luminance sampling for fast implementation of any global tone mapping operator using a lookup table. To improve the quality of synthetic video sequences, we introduce a coding scheme for discrete cosine transform (DCT) blocks with high contrast. We demonstrate the capabilities of HDR video in a player, which enables decoding, tone mapping, and applying post-processing effects in real-time. The tone mapping algorithm as well as its parameters can be changed interactively while the video is playing. We can simulate post-processing effects such as glare, night vision, and motion blur, which appear very realistic due to the usage of HDR data.
Supplemental Material
Available for Download
- BOGART, R., KAINZ, F., AND HESS, D. 2003. OpenEXR image file format. In ACM SIGGRAPH 2003, Sketches & Applications.Google Scholar
- BORDER, P., AND GUILLOTEL, P. 2000. Perceptually adapted MPEG video encoding. In IS&T/SPIE Conf. on Hum. Vis. and Electronic Imaging V, Proc. of SPIE, volume 3959, 168--175.Google Scholar
- BURT, P., AND KOLCZYNSKI, R. 1993. Enhanced image capture through fusion. In Proc. of International Conference on Computer Vision (ICCV), 173--182.Google ScholarCross Ref
- CIE. 1981. An Analytical Model for Describing the Influence of Lighting Parameters Upon Visual Performance, vol. 1. Technical Foundations, CIE 19/2.1. International Organization for Standardization.Google Scholar
- DALY, S. 1993. The Visible Differences Predictor: An algorithm for the assessment of image fidelity. In Digital Image and Human Vision, Cambridge, MA: MIT Press, A. Watson, Ed., 179--206. Google ScholarDigital Library
- DEBEVEC, P., AND MALIK, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 369--378. Google ScholarDigital Library
- DEVLIN, K., CHALMERS, A., WILKIE, A., AND PURGATHOFER, W. 2002. Tone Reproduction and Physically Based Spectral Rendering. In Eurographics 2002: State of the Art Reports, Eurographics, 101--123.Google Scholar
- DRAGO, F., MYSZKOWSKI, K., ANNEN, T., AND CHIBA, N. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Computer Graphics Forum, proceedings of Eurographics 2003 22, 3, 419--426.Google Scholar
- DURAND, F., AND DORSEY, J. 2000. Interactive tone mapping. In Rendering Techniques 2000: 11th Eurographics Workshop on Rendering, 219--230. Google ScholarDigital Library
- FERWERDA, J., PATTANAIK, S., SHIRLEY, P., AND GREENBERG, D. 1996. A model of visual adaptation for realistic image synthesis. In Proceedings of SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 249--258. Google ScholarDigital Library
- GOODNIGHT, N., WANG, R., WOOLLEY, C., AND HUMPHREYS, G. 2003. Interactive time-dependent tone mapping using programmable graphics hardware. In Rendering Techniques 2003: 14th Eurographics Symposium on Rendering, 26--37. Google ScholarDigital Library
- HOOD, D., AND FINKELSTEIN, M. 1986. Sensitivity to light. In Handbook of Perception and Human Performance: 1. Sensory Processes and Perception, Wiley, New York, K. Boff, L. Kaufman, and J. Thomas, Eds., vol. 1.Google Scholar
- HUNT, R. 1995. The Reproduction of Colour in Photography, Printing and Television: 5th Edition. Fountain Press.Google Scholar
- ISO-IEC-14496-2. 1999. Information technology: Coding of audio-visual objects, Part 2: Visual. International Organization for Standardization, Geneva, Switzerland.Google Scholar
- KANG, S., UYTTENDAELE, M., WINDER, S., AND SZELISKI, R. 2003. High dynamic range video. ACM Transactions on Graphics 22, 3, 319--325. Google ScholarDigital Library
- LUBIN, J., AND PICA, A. 1991. A non-uniform quantizer matched to the human visual performance. Society of Information Display Int. Symposium Technical Digest of Papers, 22, 619--622.Google Scholar
- NADENAU, M. 2000. Integration of Human color vision Models into High Quality Image Compression. PhD thesis, École Polytechnique Fédéral Lausane.Google Scholar
- NAYAR, S., AND BRANZOI, V. 2003. Adaptive dynamic range imaging: Optical control of pixel exposures over space and time. In Proc. of IEEE International Conference on Computer Vision (ICCV 2003), 1168--1175. Google ScholarDigital Library
- NAYAR, S., AND MITSUNAGA, T. 2000. High dynamic range imaging: Spatially varying pixel exposures. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, 472--479.Google ScholarCross Ref
- PATTANAIK, S., TUMBLIN, J., YEE, H., AND GREENBERG, D. 2000. Time-dependent visual adaption for realistic image display. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 47--54. Google ScholarDigital Library
- PRESS, W., TEUKOLSKY, S., VETTERLING, W., AND FLANNERY, B. 1993. Numerical Recipes in C. Cambridge Univ. Press.Google Scholar
- REINHARD, E., STARK, M., SHIRLEY, P., AND FERWERDA, J. 2002. Photographic tone reproduction for digital images. ACM Transactions on Graphics 21, 3, 267--276. Google ScholarDigital Library
- SAITO, K. 1995. Electronic image pickup device. Japanese Patent 07-254965.Google Scholar
- SEETZEN, H., HEIDRICH, W., STUERZLINGER, W., WARD, G., WHITE-HEAD, L., TRENTACOSTE, M., GHOSH, A., AND VOROZCOVS, A. 2004. High dynamic range display systems. ACM Transactions on Graphics 23, 3. Google ScholarDigital Library
- SEZAN, M., YIP, K., AND DALY, S. 1987. Uniform perceptual quantization: Applications to digital radiography. IEEE Transactions on Systems, Man, and Cybernetics 17, 4, 622--634.Google ScholarCross Ref
- SHAPLEY, R., AND ENROTH-CUGELL, C. 1984. Visual adaption and retinal gain controls. In Progress in Retinal Research, Oxford: Pergamon Press, vol. 3, 263--346.Google Scholar
- SHEN, K., AND DELP, E. 1999. Wavelet based rate scalable video compression. IEEE Transactions on Circuits and Systems for Video Technology 9, 1, 109--122. Google ScholarDigital Library
- SPENCER, G., SHIRLEY, P., ZIMMERMAN, K., AND GREENBERG, D. 1995. Physically-based glare effects for digital images. In Proceedings of ACM SIGGRAPH 95, 325--334. Google ScholarDigital Library
- THOMSPON, W. B., SHIRLEY, P., AND FERWERDA, J. A. 2002. A spatial post-processing algorithm for images of night scenes. Journal of Graphics Tools 7, 1, 1--12. Google ScholarDigital Library
- VAN NES, F., AND BOUMAN, M. 1967. Spatial modulation transfer in the human eye. Journal of the Optical Society of America 57, 401--406.Google ScholarCross Ref
- WANG, Z., AND BOVIK, A. 2002. A universal image quality index. IEEE Signal Processing Letters 9, 3, 81--84.Google ScholarCross Ref
- WARD LARSON, G., RUSHMEIER, H., AND PIATKO, C. 1997. A visibility matching tone reproduction operator for high dynamic range scenes. IEEE Transactions on Visualization and Computer Graphics 3, 4, 291--306. Google ScholarDigital Library
- WARD LARSON, G. 1998. Logluv encoding for full-gamut, high-dynamic range images. Journal of Graphics Tools 3, 1, 815--30. Google ScholarDigital Library
- WARD, G. 1991. Real pixels. In Graphics Gems II, J. Arvo, Ed. Academic Press, 80--83.Google Scholar
Index Terms
- Perception-motivated high dynamic range video encoding
Recommendations
Backward compatible high dynamic range MPEG video compression
SIGGRAPH '06: ACM SIGGRAPH 2006 PapersTo embrace the imminent transition from traditional low-contrast video (LDR) content to superior high dynamic range (HDR) content, we propose a novel backward compatible HDR video compression (HDR MPEG) method. We introduce a compact reconstruction ...
Perception-motivated high dynamic range video encoding
SIGGRAPH '04: ACM SIGGRAPH 2004 PapersDue to rapid technological progress in high dynamic range (HDR) video capture and display, the efficient storage and transmission of such data is crucial for the completeness of any HDR imaging pipeline. We propose a new approach for inter-frame ...
Backward compatible high dynamic range MPEG video compression
To embrace the imminent transition from traditional low-contrast video (LDR) content to superior high dynamic range (HDR) content, we propose a novel backward compatible HDR video compression (HDR MPEG) method. We introduce a compact reconstruction ...
Comments