Enhanced local tone mapping for detail preserving reproduction of high dynamic range images☆
Introduction
Luminance variation in real world scenes is very high and can span 10 orders of magnitude. While the human eye can simultaneously perceive the contrast of about 4 orders of magnitude, current consumer cameras are able to capture only a limited portion of this dynamic range, which is approximately 2–3 orders of magnitude. Recently devised sensors are able to capture more of the scenes’ dynamic range. These sensors are usually based on differently exposed sensor elements, which allows them to simultaneously capture the dark and bright parts of the scene. The high dynamic range (HDR) of the scene can also be captured with an ordinary low dynamic range (LDR) camera by combining multiple images taken with different exposure settings [1].
Although the aforementioned approaches are able to capture HDR scenes, there is a problem of reproduction of these images on standard display devices. Current commercial displays are able to reproduce images with a contrast of 2 orders of magnitude. There are prototype displays with much larger dynamic range, but this technology is still developing. Hence, to enable HDR image reproduction on ordinary LDR displays, an image needs to be processed using a tone mapping operator.
Tone mapping operators provide contrast reduction of HDR images while preserving image features as much as possible. These operators are, based on spatial adaptability, divided into two main categories: global and local operators.
Global operators apply the same mapping function to all image pixels, providing simplicity in implementation, and good global contrast. Unfortunately, global operators tend to suppress details in highlighted regions.
Local operators adapt to local luminance values, applying different mapping to each pixel, thus preserving details much better. Along with complexity local operators can produce several undesirable artifacts: halo artifacts, poor global contrast, surreal effect, etc.
In consumer electronics there is a constant need to provide users with the best subjective image quality. While some subjective image quality properties, such as global/local contrast, details strength, colorfulness, are defined, there are different opinions regarding which setting makes the best subjective impression. Hence, it is very important that the tone mapping operator intended for consumer electronics has the ability of easy tuning for desired image appearance. This enables companies or even the end user to adjust tone mapping settings in order to achieve the image quality goals for their specific device.
Enhanced local tone mapping operator (ELTM), presented in this paper, preserves and enhances details while providing good overall global contrast by maximally using the available output dynamic range. Thus, while it formally belongs to the group of local tone mapping operators, ELTM is actually positioned between global and local tone mapping operators, taking the best features from both approaches. ELTM also provides a set of decoupled tuning parameters, which give intuitive interface for achieving the desired output appearance.
Section snippets
Related work
Human visual system (HVS) has a highly nonlinear response to light intensity variations. The Weber-Fechner law defines this characteristic as logarithmic, which is why logarithm and power based functions are one of the most used function families for global tone mapping. Global tone mapping operators [2], [3], [4], [5], [6] apply the same mapping function to all image pixels. Although these operators have very efficient realizations and are robust, they do not perform well when it comes to
System overview
HDR irradiance map is usually obtained by combining several different exposures either by means of exposure varying sensors or by using multiple differently exposed frames. It is normalized to fit the range [0,1] by dividing each channel with maximal light level across all three color channels.
Although some color appearance models compress each of the color channels separately [32], [33] it is common to compress only the luminance channel because it contains achromatic information about
ELTM tuning
One of the main strengths of the ELTM is the ease of tuning for desired subjective appearance. There are several decoupled parameters that can be used for setting the desired appearance of an output image, as it is shown in Table 1, along with their default values.
Fig. 3 presents the tuning abilities of ELTM. It demonstrates appearance effects of changing each of the tuning parameters, one at a time. Since saturation modification is adjusted using standard procedures, it is excluded from this
ELTM brightness control
Although ELTM provides good mapping for various scenes, it is noticed that in some cases it can produce too bright or too dark output. This usually happens if the input scene has extremely low-key or high-key histogram. Namely, since the tone compression parameter p is the same for all scenes, fixed tone compression curve in the linear domain cannot fit all these extremes. To provide additional robustness, correction factor of the tuning parameter p is proposed. Average value of the logarithm
Results
ELTM was developed with the goal to enable display of HDR images obtained using a consumer electronics device such as a standard mobile phone camera. Since it should be used “in the wild” with various scenes and various devices ELTM was tested with a broad range of irradiance maps used in the literature and available online [38], [39], [40], [41], [42], [43], [44], [45], [46]. Results are presented for 50 scenes with different content, average brightness, resolution and origin. For comparison
Conclusion
In this paper the local tone mapping algorithm ELTM is presented. Although ELTM is based on the well-established foundations and techniques, it combines them in a novel way that brings consistently good results for various types of HDR scenes. It produces images with a good global and local contrast, preserving details in highlighted areas and shadows thanks to the proposed modification of tone compression function. Presented framework is universal in a sense that other tone compression
Acknowledgment
The authors would like to thank members of Intel Serbia team for support and cooperation during this research, especially Nemanja Tonic, Stojan Rakic and Aleksandar Sutic.
References (58)
- et al.
ICAM06: a refined image appearance model for HDR image rendering
J. Vis. Commun. Image Represent.
(2007) - P.E. Debevec, J. Malik, “Recovering high dynamic range radiance maps from photographs,” in: Proc. ACM SIGGRAPH97, New...
- et al.
A visibility matching tone reproduction operator for high dynamic range scenes
IEEE Trans. Visualiz. Comput. Graphics
(1997) - C. Schlick, “Quantization techniques for visualization of high dynamic range pictures,” in: Photorealistic Rendering...
- et al.
Display adaptive tone mapping
ACM Trans. Graphics
(2008) - et al.
Photographic tone reproduction for digital images
ACM Trans. Graphics
(2002) - et al.
Adaptive logarithmic mapping for displaying high contrast scenes
Comput. Graphics Forum
(2003) - S.N. Pattanaik, J.A. Ferwerda, M.D. Fairchild, D.P. Greenberg, “A multiscale model of adaptation and spatial vision for...
- J. Tumblin, G. Turk, “LCIS: A boundary hierarchy for detail-preserving contrast reduction,” in: Proc. ACM SIGGRAPH99,...
- et al.
Compressing and companding high dynamic range images with subband architectures
ACM Trans. Graphics
(2005)
Fast bilateral filtering for the display of high-dynamic-range images
ACM Trans. Graphics
Edge-preserving decompositions for multi-scale tone and detail manipulation
ACM Trans. Graphics
Local edge-preserving multiscale decomposition for high dynamic range image tone mapping
IEEE Trans. Image Process.
LLSURE: local linear SURE-based edge-preserving image filtering
IEEE Trans. Image Process.
Image smoothing via L0 gradient minimization
ACM Trans. Graphics
Local Laplacian filters: edge-aware image processing with a Laplacian pyramid
ACM Trans. Graphics
Guided image filtering
IEEE Trans. Pattern Anal. Mach. Intell.
Lightness and retinex theory
J. Opt. Soc. Amer.
Properties and performance of a center/surround retinex
IEEE Trans. Image Process.
High dynamic range image rendering with a retinex-based adaptive filter
IEEE Trans. Image Process.
Natural HDR image tone mapping based on retinex
IEEE Trans. Consumer Electron.
A novel tone mapping method for high dynamic range image by incorporating edge-preserving filter into method based on retinex
Appl. Math. Inf. Sci.
Dynamic range compression preserving local image contrast for digital video camera
IEEE Trans. Consumer Electron.
Tone mapping for high-dynamic-range images using localized gamma correction
J. Electron. Imag.
Cited by (15)
An enhancement framework based on gradient domain tone mapping and fuzzy logical for X-ray image of complex workpiece
2021, NDT and E InternationalCitation Excerpt :Global TMOs [9,10] apply a point-to-point mapping function to all pixels in a HDR image. Classical global operators include logarithmic and power based functions since they conform to the character of human's visual system [11]. Other common global TMOs are gamma correction and histogram equalization.
Industrial x-ray image enhancement network based on a ray scattering model
2023, Applied OpticsLow-illumination image enhancement with logarithmic tone mapping
2023, Open Computer ScienceAdaptive method for image dynamic range adjustment and detail enhancement
2022, Applied OpticsNonsubsampled contourlet transform based tone-mapping operator to optimize the dynamic range of diatom shells
2021, Microscopy Research and Technique
- ☆
This paper has been recommended for acceptance by Zicheng Liu.