Skip to main content
SpringerLink
Log in

Two-stage image decomposition and color regulator for low-light image enhancement

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Low-lighting is a common condition in data collection due to environmental restrictions. However, high-level pattern recognition tasks such as object detection require the datasets to be more clear. Thus, low-light image enhancement is necessary. Noise and color distortion are two major problems of the existing enhancement algorithms. This paper has proposed a low-light image enhancement algorithm that integrates denoising and color restoration. First, we propose a two-stage hybrid decomposition network, which can perform modified Retinex-decomposition on paired images, and then extract principal components of the decomposed low-light images to handle the nonlinear residuals, thereby obtaining reliable reflectance and illumination maps. Then, in order not to over-smooth the details and edges of the image, we use a flexible joint function to train the hybrid network. Finally, we create a color regulator in the HSI (Hue-Saturation-Intensity) space to correct the distortion in RGB space caused by coupling between pixels. Experimental results on public datasets show that the proposed method greatly enhanced the quality of low-light images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://sites.google.com/site/vonikakis/datasets.

References

  1. Pizer, S.M., Amburn, E.P., Austin, J.D., Cromartie, R., Geselowitz, A., Greer, T., ter Haar Romeny, B., Zimmerman, J.B., Zuiderveld, K.: Adaptive histogram equalization and its variations. Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987)

    Article  Google Scholar 

  2. Kim, Y.-T.: Contrast enhancement using brightness preserving bi-histogram equalization. IEEE Trans. Consum. Electron. 43(1), 1–8 (1997)

    Article  Google Scholar 

  3. Wei, C., Wang, W., Yang, W., Liu, J.: Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560 (2018)

  4. Li, M., Liu, J., Yang, W., Sun, X., Guo, Z.: Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans. Image Process. 27(6), 2828–2841 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  5. Park, S., Yu, S., Moon, B., Ko, S., Paik, J.: Low-light image enhancement using variational optimization-based retinex model. IEEE Trans. Consum. Electron. 63(2), 178–184 (2017)

    Article  Google Scholar 

  6. Lim, S., Kim, W.: DSLR: deep stacked Laplacian restorer for low-light image enhancement. IEEE Trans. Multimed. (2020). https://doi.org/10.1109/TMM.2020.3039361

    Article  Google Scholar 

  7. Cai, J., Gu, S., Zhang, L.: Learning a deep single image contrast enhancer from multi-exposure images. IEEE Trans. Image Process. 27(4), 2049–2062 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  8. Zhu, M., Pan, P., Chen, W., Yang, Y.: Eemefn: Low-light image enhancement via edge-enhanced multi-exposure fusion network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 13106–13113 (2020)

  9. Liu, R., Ma, L., Zhang, J., Fan, X., Luo, Z.: Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10561–10570 (2021)

  10. Liu, X., Li, H., Zhu, C.: Joint contrast enhancement and exposure fusion for real-world image dehazing. IEEE Trans. Multimed. (2021). https://doi.org/10.1109/TMM.2021.3110483

    Article  Google Scholar 

  11. Lu, K., Zhang, L.: Tbefn: a two-branch exposure-fusion network for low-light image enhancement. IEEE Trans. Multimed. (2020). https://doi.org/10.1109/TMM.2020.3037526

    Article  Google Scholar 

  12. Chen, B.-H., Huang, S.-C., Li, C.-Y., Kuo, S.-Y.: Haze removal using radial basis function networks for visibility restoration applications. IEEE Trans. Neural Netw. Learn. Syst. 29(8), 3828–3838 (2018). https://doi.org/10.1109/TNNLS.2017.2741975

    Article  Google Scholar 

  13. YU, N., LI, J., HUA, Z.: FLA-Net: multi-stage modular network for low-light image enhancement. Vis. Comput. 1–20 (2022)

  14. Guo, S., Wang, W., Wang, X., Xu, X.: Low-light image enhancement with joint illumination and noise data distribution transformation. Vis. Comput. 1–12 (2022)

  15. Song, X., Huang, J., Cao, J., Song, D.: Feature spatial pyramid network for low-light image enhancement. Vis. Comput. 1–11 (2022)

  16. Reza, A.M.: Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. J. VLSI Signal Process. Syst. Signal Image Video Technol. 38(1), 35–44 (2004)

    Article  Google Scholar 

  17. Land, E.H., McCann, J.J.: Lightness and retinex theory. Josa 61(1), 1–11 (1971)

    Article  Google Scholar 

  18. Jobson, D.J., Rahman, Z.-U., Woodell, G.A.: Properties and performance of a center/surround retinex. IEEE Trans. Image Process. 6(3), 451–462 (1997)

    Article  Google Scholar 

  19. Rahman, Z.-u., Jobson, D.J., Woodell, G.A.: Multi-scale retinex for color image enhancement. In: Proceedings of 3rd IEEE International Conference on Image Processing, vol. 3, pp. 1003–1006 (1996). IEEE

  20. Rahman, Z., Jobson, D.J., Woodell, G.A.: Retinex processing for automatic image enhancement. J. Electron. Imaging 13, 100–110 (2004)

    Article  Google Scholar 

  21. Joshi, P., Prakash, S.: Image enhancement with naturalness preservation. Vis. Comput. 36(1), 71–83 (2020)

    Article  Google Scholar 

  22. Li, M., Zhao, L., Zhou, D., Nie, R., Liu, Y., Wei, Y.: AEMS: an attention enhancement network of modules stacking for lowlight image enhancement. Vis. Comput. 1–17 (2021)

  23. Fu, X., Zeng, D., Huang, Y., Zhang, X.-P., Ding, X.: A weighted variational model for simultaneous reflectance and illumination estimation. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2782–2790 (2016). https://doi.org/10.1109/CVPR.2016.304

  24. Zhang, Y., Zhang, J., Guo, X.: Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 1632–1640 (2019)

  25. Guo, X., Li, Y., Ling, H.: Lime: low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26(2), 982–993 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  26. Lore, K.G., Akintayo, A., Sarkar, S.: Llnet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recognit. 61, 650–662 (2017)

    Article  Google Scholar 

  27. Sharma, V., Diba, A., Neven, D., Brown, M.S., Van Gool, L., Stiefelhagen, R.: Classification-driven dynamic image enhancement. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4033–4041 (2018)

  28. Jiang, Y., Gong, X., Liu, D., Cheng, Y., Fang, C., Shen, X., Yang, J., Zhou, P., Wang, Z.: Enlightengan: deep light enhancement without paired supervision. IEEE Trans. Image Process. 30, 2340–2349 (2021)

    Article  Google Scholar 

  29. Guo, C., Li, C., Guo, J., Loy, C.C., Hou, J., Kwong, S., Cong, R.: Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1780–1789 (2020)

  30. Yang, W., Wang, W., Huang, H., Wang, S., Liu, J.: Sparse gradient regularized deep retinex network for robust low-light image enhancement. IEEE Trans. Image Process. 30, 2072–2086 (2021)

    Article  Google Scholar 

  31. Jiang, Z., Li, H., Liu, L., Men, A., Wang, H.: A switched view of retinex: deep self-regularized low-light image enhancement. Neurocomputing 454, 361–372 (2021)

    Article  Google Scholar 

  32. Tian, C., Xu, Y., Zuo, W., Lin, C.-W., Zhang, D.: Asymmetric CNN for image superresolution. IEEE Trans. Syst. Man Cybern. Syst. 52(6), 3718–3730 (2022). https://doi.org/10.1109/TSMC.2021.3069265

    Article  Google Scholar 

  33. Zheng, M., Zhi, K., Zeng, J., Tian, C., You, L.: A hybrid CNN for image denoising. J. Artif. Intell. Technol. (2022)

  34. Tian, C., Xu, Y., Zuo, W., Du, B., Lin, C.-W., Zhang, D.: Designing and training of a dual CNN for image denoising. Knowl. Based Syst. 226, 106949 (2021)

    Article  Google Scholar 

  35. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    Article  MathSciNet  Google Scholar 

  36. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  37. Wang, S., Zheng, J., Hu, H.-M., Li, B.: Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans. Image Process. 22(9), 3538–3548 (2013)

    Article  Google Scholar 

  38. Allen, D.M.: Mean square error of prediction as a criterion for selecting variables. Technometrics 13(3), 469–475 (1971)

  39. Jobson, D.J., Rahman, Z.-U., Woodell, G.A.: A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Process. 6(7), 965–976 (1997)

    Article  Google Scholar 

  40. Lee, C., Lee, C., Kim, C.-S.: Contrast enhancement based on layered difference representation. In: 2012 19th IEEE International Conference on Image Processing, pp. 965–968 (2012). IEEE

  41. Lee, C., Lee, C., Lee, Y.-Y., Kim, C.-S.: Power-constrained contrast enhancement for emissive displays based on histogram equalization. IEEE Trans. Image Process. 21(1), 80–93 (2011)

    MathSciNet  MATH  Google Scholar 

  42. Fu, X., Zeng, D., Huang, Y., Zhang, X.-P., Ding, X.: A weighted variational model for simultaneous reflectance and illumination estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2782–2790 (2016)

Download references

Acknowledgements

The work presented in this paper is partially supported by the Guangdong-Hong Kong joint Project under Grant 2020A0505090005.

Author information

Authors and Affiliations

  1. State Key Laboratory of High Performance Complex Manufacturing, Central South University, Changsha, 410083, Hunan, China

    Xinyi Yu

  2. College of Mechanical and Electrical Engineering, Central South University, Changsha, 410083, Hunan, China

    Xinyi Yu

  3. Department of Advanced Design and Systems Engineering, City University of Hong Kong, Kowloon, 999077, Hong Kong, China

    Hanxiong Li

  4. Guangdong Engineering Research Center for Green Manufacturing and Energy Efficiency Optimization, Guangdong University of Technology, Guangzhou, 510006, Guangdong, China

    Haidong Yang

Authors
  1. Xinyi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanxiong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haidong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanxiong Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, X., Li, H. & Yang, H. Two-stage image decomposition and color regulator for low-light image enhancement. Vis Comput 39, 4165–4175 (2023). https://doi.org/10.1007/s00371-022-02582-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02582-3

Keywords

Search

Navigation