HDR-VQM: An objective quality measure for high dynamic range video
Introduction
The advent of better technologies in the field of visual signal capture and processing has fueled a paradigm shift in todays׳ multimedia communication systems. As a result, the notion of network-centric quality of service (QoS) in multimedia systems is being extended by relying on the concept of quality of experience (QoE) [1]. In this quest of increasing the immersive video experience and the overall QoE of the end user, newer technologies such as 3D, ultra high definition (UHD) and, more recently, high dynamic range (HDR) imaging have gained prominence within the multimedia signal processing community. HDR in particular has attracted attention since it in a way revisits the way we capture and display natural scenes. This is motivated by the fact that natural scenes often exhibit large ranges of illumination values. However, such high luminance values often exceed the capabilities of the traditional low dynamic range (LDR) capturing and display devices. Consequently, it is not possible to properly expose the dark and the bright areas simultaneously in one image (or video) during capture. This may lead to over-exposure (saturated pixels that are fully white) and/or under-exposure (very dark or noisy pixels as sensor׳s response falls below its noise threshold). In both cases, visual information is either lost or altered. HDR imaging focuses on minimizing such losses and therefore aims at improving the quality of the displayed pixels by incorporating higher contrast and luminance.
As a result, HDR imaging has attracted attention from both academia and industry, and there has been interest and effort to develop tools/algorithms for HDR video processing [2]. For instance, there have been recent efforts within the Moving Picture Experts Group (MPEG) for extending High Efficiency Video Coding (HEVC) to HDR. Likewise, the JPEG has announced extensions that will feature the original JPEG standard with support for HDR image compression. Despite of some work on evaluating quality of HDR images [3], [4] and video [5], there is overall lack of such efforts to quantify and measure the impact of such tools on HDR video quality using both subjective and objective approaches. The issue assumes further significance given that most of the existing objective methods may not be directly applicable for HDR quality estimation [6], [7], [8] (note that these studies only deal with HDR images and not video). It is therefore important to develop objective methods for HDR video quality measurement and benchmark their performance against subjective ground truth.
With regards to visual quality measurement, both subjective and objective approaches can be used. The former involves the use of human subjects to judge and rate the quality of the test stimuli. With appropriate laboratory conditions and a sufficiently large subject panel, it remains the most accurate method. The latter quality assessment method employs a computational (mathematical) model to provide estimates of the subjective video quality. While such objective models may not mimic subjective opinions accurately in a general scenario, they can be reasonably effective in specific conditions/applications. Hence, they can be an important tool towards automating the testing and standardization of HDR video processing algorithms such as HDR video compression, post-processing, inverse video tone mapping, etc., especially when subjective tests may not be feasible. In light of this, we present a computationally tractable HDR video quality estimation method based on HDR signal transformation and subsequent analysis of spatio-temporal segments, and also verify its prediction performance based on a test bed of 90 subjectively rated compressed HDR video sequences. To the best of our knowledge, our study is amongst the first few efforts towards the design and verification of an objective quality measurement method for HDR video, and is therefore of interest to the video signal processing community both from subjective and objective quality view points.
Section snippets
Background
Humans perceive the outside visual world through the interaction between luminance (measured in candela per square meter, cd/m2) and the eyes. Luminance first passes through the cornea, a transparent membrane. Then it enters the pupil, an aperture that is modified by the iris, a muscular diaphragm. Subsequently, light is refracted by the lens and hits the photoreceptors in the retina. There are two types of photoreceptors: cones and rods. The cones are located mostly in the fovea. They are more
The proposed objective HDR video quality measure
A block diagram outlining the major steps in the proposed HDR-VQM is shown in Fig. 1. It takes as input the source and the distorted HDR video sequences. Note that throughout the paper we use the notation src (source) and hrc (hypothetical reference circuit) to respectively denote reference and distorted video sequences. As shown in the figure, the first two steps are meant to convert the native input luminance to perceived luminance. These can therefore be seen as pre-processing steps, and the
HDR video dataset
To the best of our knowledge there are currently no publicly available subjectively annotated HDR video datasets dealing with the issue of visual quality. Therefore, for verifying the prediction performance of HDR-VQM and other objective methods, an in-house and comprehensive HDR video dataset was used. This section provides a brief description of the dataset.
Experimental results
Before we compute the experimental results, it is necessary to determine the parameter values in HDR-VQM. First, to compute the values of x and y (it may be recalled from Fig. 3 that these respectively denote the width and height of the considered block), we assume the central angle of the visual field in the fovea to be 2° [22]. Then, we can define a quantity W which denotes the length of the fixated window in terms of number of pixels and can be computed aswhere V is the viewing
Discussion
The previous sections proposed and verified the performance of an objective HDR video quality estimator HDR-VQM. An important point worth re-iterating is that in light of limitations imposed in HDR display technologies, the HDR video signal must be pre-processed. This, in turn implies that HDR video quality can be affected by the type of pre-processing used as well as the maximum displayable luminance since these can affect distortion visibility. Thus, the role of a display model is more
Concluding thoughts
HDR imaging is increasingly becoming popular in the multimedia signal processing community primarily as a tool towards enhancing the immersive video experience of the user. However, there are very few works that address the issue of assessing the impact of HDR video processing algorithms on the perceptual video quality both from subjective and objective angles. The study in this paper seeks to outline the first steps towards the design and verification of an objective HDR video quality
Acknowledgments
The authors wish to thank Romuald Pepion for his help in generating the subjective test results used in this paper. This work has been supported by the NEVEx Project FUI11 financed by the French Government.
References (25)
- et al.
Tone mapping based HDR compressiondoes it affect visual experience?
Signal Process.: Image Commun.
(2014) - P. Le Callet, S. Moller, A. Perkis, Qualinet white paper on definitions of quality of experience (2012), White Paper,...
- et al.
Advanced High Dynamic Range Imaging: Theory and Practice
(2011) - C. Mantel, S. Ferchiu, S. Forchhammer, Comparing subjective and objective quality assessment of HDR images compressed...
- P. Hanhart, M. Bernardo, P. Korshunov, M. Pereira, A. Pinheiro, T. Ebrahimi, HDR image compression: a new challenge for...
- M. Rerabek, P. Hanhart, P. Korsunov, T. Ebrahimi, Subjective and objective evaluation of HDR video compression, in:...
- et al.
Tone mapping-based high-dynamic-range image compressionstudy of optimization criterion and perceptual quality
Opt. Eng.
(2013) - G. Valenzise, F. Simone, P. Lauga, F. Dufaux, Performance evaluation of objective quality metrics for HDR image...
- M. Narwaria, M. Perreira Da Silva, P. Le Callet, R. Pepion, On improving the pooling in HDR-VDP-2 towards better HDR...
Foundations of Perception
(2006)
Cited by (155)
HDR video synthesis by a nonlocal regularization variational model
2023, Journal of Visual Communication and Image RepresentationNo-Reference Quality Assessment for HDR Images Based on Multi-scale Retinex Decomposition
2024, Communications in Computer and Information ScienceA Dataset and Model for the Visual Quality Assessment of Inversely Tone-Mapped HDR Videos
2024, IEEE Transactions on Image ProcessingA Study of Subjective and Objective Quality Assessment of HDR Videos
2024, IEEE Transactions on Image Processing