HDR-VQM: An objective quality measure for high dynamic range video

https://doi.org/10.1016/j.image.2015.04.009Get rights and content

Highlights

  • The paper presents one of the first objective method for high dynamic range video quality estimation.

  • It is based on analysis of short term video segments taking into account human viewing behavior.

  • The method described in the paper would be useful in scenarios where HDR video quality needs to be determined in an HDR video chain study.

Abstract

High dynamic range (HDR) signals fundamentally differ from the traditional low dynamic range (LDR) ones in that pixels are related (proportional) to the physical luminance in the scene (i.e. scene-referred). For that reason, the existing LDR video quality measurement methods may not be directly used for assessing quality in HDR videos. To address that, we present an objective HDR video quality measure (HDR-VQM) based on signal pre-processing, transformation, and subsequent frequency based decomposition. Video quality is then computed based on a spatio-temporal analysis that relates to human eye fixation behavior during video viewing. Consequently, the proposed method does not involve expensive computations related to explicit motion analysis in the HDR video signal, and is therefore computationally tractable. We also verified its prediction performance on a comprehensive, in-house subjective HDR video database with 90 sequences, and it was found to be better than some of the existing methods in terms of correlation with subjective scores (for both across sequence and per sequence cases). A software implementation of the proposed scheme is also made publicly available for free download and use.

Introduction

The advent of better technologies in the field of visual signal capture and processing has fueled a paradigm shift in todays׳ multimedia communication systems. As a result, the notion of network-centric quality of service (QoS) in multimedia systems is being extended by relying on the concept of quality of experience (QoE) [1]. In this quest of increasing the immersive video experience and the overall QoE of the end user, newer technologies such as 3D, ultra high definition (UHD) and, more recently, high dynamic range (HDR) imaging have gained prominence within the multimedia signal processing community. HDR in particular has attracted attention since it in a way revisits the way we capture and display natural scenes. This is motivated by the fact that natural scenes often exhibit large ranges of illumination values. However, such high luminance values often exceed the capabilities of the traditional low dynamic range (LDR) capturing and display devices. Consequently, it is not possible to properly expose the dark and the bright areas simultaneously in one image (or video) during capture. This may lead to over-exposure (saturated pixels that are fully white) and/or under-exposure (very dark or noisy pixels as sensor׳s response falls below its noise threshold). In both cases, visual information is either lost or altered. HDR imaging focuses on minimizing such losses and therefore aims at improving the quality of the displayed pixels by incorporating higher contrast and luminance.

As a result, HDR imaging has attracted attention from both academia and industry, and there has been interest and effort to develop tools/algorithms for HDR video processing [2]. For instance, there have been recent efforts within the Moving Picture Experts Group (MPEG) for extending High Efficiency Video Coding (HEVC) to HDR. Likewise, the JPEG has announced extensions that will feature the original JPEG standard with support for HDR image compression. Despite of some work on evaluating quality of HDR images [3], [4] and video [5], there is overall lack of such efforts to quantify and measure the impact of such tools on HDR video quality using both subjective and objective approaches. The issue assumes further significance given that most of the existing objective methods may not be directly applicable for HDR quality estimation [6], [7], [8] (note that these studies only deal with HDR images and not video). It is therefore important to develop objective methods for HDR video quality measurement and benchmark their performance against subjective ground truth.

With regards to visual quality measurement, both subjective and objective approaches can be used. The former involves the use of human subjects to judge and rate the quality of the test stimuli. With appropriate laboratory conditions and a sufficiently large subject panel, it remains the most accurate method. The latter quality assessment method employs a computational (mathematical) model to provide estimates of the subjective video quality. While such objective models may not mimic subjective opinions accurately in a general scenario, they can be reasonably effective in specific conditions/applications. Hence, they can be an important tool towards automating the testing and standardization of HDR video processing algorithms such as HDR video compression, post-processing, inverse video tone mapping, etc., especially when subjective tests may not be feasible. In light of this, we present a computationally tractable HDR video quality estimation method based on HDR signal transformation and subsequent analysis of spatio-temporal segments, and also verify its prediction performance based on a test bed of 90 subjectively rated compressed HDR video sequences. To the best of our knowledge, our study is amongst the first few efforts towards the design and verification of an objective quality measurement method for HDR video, and is therefore of interest to the video signal processing community both from subjective and objective quality view points.

Section snippets

Background

Humans perceive the outside visual world through the interaction between luminance (measured in candela per square meter, cd/m2) and the eyes. Luminance first passes through the cornea, a transparent membrane. Then it enters the pupil, an aperture that is modified by the iris, a muscular diaphragm. Subsequently, light is refracted by the lens and hits the photoreceptors in the retina. There are two types of photoreceptors: cones and rods. The cones are located mostly in the fovea. They are more

The proposed objective HDR video quality measure

A block diagram outlining the major steps in the proposed HDR-VQM is shown in Fig. 1. It takes as input the source and the distorted HDR video sequences. Note that throughout the paper we use the notation src (source) and hrc (hypothetical reference circuit) to respectively denote reference and distorted video sequences. As shown in the figure, the first two steps are meant to convert the native input luminance to perceived luminance. These can therefore be seen as pre-processing steps, and the

HDR video dataset

To the best of our knowledge there are currently no publicly available subjectively annotated HDR video datasets dealing with the issue of visual quality. Therefore, for verifying the prediction performance of HDR-VQM and other objective methods, an in-house and comprehensive HDR video dataset was used. This section provides a brief description of the dataset.

Experimental results

Before we compute the experimental results, it is necessary to determine the parameter values in HDR-VQM. First, to compute the values of x and y (it may be recalled from Fig. 3 that these respectively denote the width and height of the considered block), we assume the central angle of the visual field in the fovea to be 2° [22]. Then, we can define a quantity W which denotes the length of the fixated window in terms of number of pixels and can be computed asW=tan2°×V×R/DAwhere V is the viewing

Discussion

The previous sections proposed and verified the performance of an objective HDR video quality estimator HDR-VQM. An important point worth re-iterating is that in light of limitations imposed in HDR display technologies, the HDR video signal must be pre-processed. This, in turn implies that HDR video quality can be affected by the type of pre-processing used as well as the maximum displayable luminance since these can affect distortion visibility. Thus, the role of a display model is more

Concluding thoughts

HDR imaging is increasingly becoming popular in the multimedia signal processing community primarily as a tool towards enhancing the immersive video experience of the user. However, there are very few works that address the issue of assessing the impact of HDR video processing algorithms on the perceptual video quality both from subjective and objective angles. The study in this paper seeks to outline the first steps towards the design and verification of an objective HDR video quality

Acknowledgments

The authors wish to thank Romuald Pepion for his help in generating the subjective test results used in this paper. This work has been supported by the NEVEx Project FUI11 financed by the French Government.

References (25)

  • M. Narwaria et al.

    Tone mapping based HDR compressiondoes it affect visual experience?

    Signal Process.: Image Commun.

    (2014)
  • P. Le Callet, S. Moller, A. Perkis, Qualinet white paper on definitions of quality of experience (2012), White Paper,...
  • F. Banterle et al.

    Advanced High Dynamic Range Imaging: Theory and Practice

    (2011)
  • C. Mantel, S. Ferchiu, S. Forchhammer, Comparing subjective and objective quality assessment of HDR images compressed...
  • P. Hanhart, M. Bernardo, P. Korshunov, M. Pereira, A. Pinheiro, T. Ebrahimi, HDR image compression: a new challenge for...
  • M. Rerabek, P. Hanhart, P. Korsunov, T. Ebrahimi, Subjective and objective evaluation of HDR video compression, in:...
  • M. Narwaria et al.

    Tone mapping-based high-dynamic-range image compressionstudy of optimization criterion and perceptual quality

    Opt. Eng.

    (2013)
  • G. Valenzise, F. Simone, P. Lauga, F. Dufaux, Performance evaluation of objective quality metrics for HDR image...
  • M. Narwaria, M. Perreira Da Silva, P. Le Callet, R. Pepion, On improving the pooling in HDR-VDP-2 towards better HDR...
  • G. Mather

    Foundations of Perception

    (2006)
  • T. Aydin, R. Mantiuk, H. Seidel, Extending quality metrics to full luminance range images, in: Proceedings of the SPIE,...
  • A. Koz, F. Dufaux, Optimized tone mapping with perceptually uniform luminance values for backward-compatible high...
  • Cited by (155)

    • HDR video synthesis by a nonlocal regularization variational model

      2023, Journal of Visual Communication and Image Representation
    View all citing articles on Scopus
    View full text