skip to main content
10.1145/3343031.3350983acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Progressive Retinex: Mutually Reinforced Illumination-Noise Perception Network for Low-Light Image Enhancement

Authors Info & Claims
Published:15 October 2019Publication History

ABSTRACT

Contrast enhancement and noise removal are coupled problems for low-light image enhancement. The existing Retinex based methods do not take the coupling relation into consideration, resulting in under or over-smoothing of the enhanced images. To address this issue, this paper presents a novel progressive Retinex framework, in which illumination and noise of low-light image are perceived in a mutually reinforced manner, leading to noise reduction low-light enhancement results. Specifically, two fully pointwise convolutional neural networks are devised to model the statistical regularities of ambient light and image noise respectively, and to leverage them as constraints to facilitate the mutual learning process. The proposed method not only suppresses the interference caused by the ambiguity between tiny textures and image noises, but also greatly improves the computational efficiency. Moreover, to solve the problem of insufficient training data, we propose an image synthesis strategy based on camera imaging model, which generates color images corrupted by illumination-dependent noises. Experimental results on both synthetic and real low-light images demonstrate the superiority of our proposed approaches against the State-Of-The-Art (SOTA) low-light enhancement methods.

References

  1. Yagiz Aksoy, Changil Kim, Petr Kellnhofer, Sylvain Paris, Mohamed Elgharib, Marc Pollefeys, and Wojciech Matusik. 2018. A Dataset of Flash and Ambient Illumination Pairs from the Crowd. In Proceedings of the European Conference on Computer Vision (ECCV). 634--649.Google ScholarGoogle ScholarCross RefCross Ref
  2. Neil DB Bruce. 2014. Expoblend: Information preserving exposure blending based on normalized log-domain entropy. Computers & Graphics , Vol. 39 (2014), 12--23.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Bolun Cai, Xianming Xu, Kailing Guo, Kui Jia, Bin Hu, and Dacheng Tao. 2017. A Joint Intrinsic-Extrinsic Prior Model for Retinex. In Proceedings of the IEEE International Conference on Computer Vision. 4000--4009.Google ScholarGoogle ScholarCross RefCross Ref
  4. Jianrui Cai, Shuhang Gu, and Lei Zhang. 2018. Learning a deep single image contrast enhancer from multi-exposure images. IEEE Transactions on Image Processing , Vol. 27, 4 (2018), 2049--2062.Google ScholarGoogle ScholarCross RefCross Ref
  5. Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. 2007. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on image processing , Vol. 16, 8 (2007), 2080--2095.Google ScholarGoogle ScholarCross RefCross Ref
  6. Michael Elad. 2005. Retinex by two bilateral filters. In International Conference on Scale-Space Theories in Computer Vision. Springer, 217--229.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. David A Forsyth. 1988. A novel approach to colour constancy. In 1988 Second International Conference on Computer Vision. IEEE, 9--18.Google ScholarGoogle ScholarCross RefCross Ref
  8. Xueyang Fu, Delu Zeng, Yue Huang, Xiao-Ping Zhang, and Xinghao Ding. 2016. A weighted variational model for simultaneous reflectance and illumination estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2782--2790.Google ScholarGoogle ScholarCross RefCross Ref
  9. Brian Funt and Lilong Shi. 2010. The rehabilitation of maxrgb. In Color and imaging conference , Vol. 2010. Society for Imaging Science and Technology, 256--259.Google ScholarGoogle Scholar
  10. Michael D Grossberg and Shree K Nayar. 2004. Modeling the space of camera response functions. IEEE transactions on pattern analysis and machine intelligence , Vol. 26, 10 (2004), 1272--1282.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Xiaojie Guo, Yu Li, and Haibin Ling. 2017. LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing , Vol. 26, 2 (2017), 982--993.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Nicolas Hautière, Jean-Philippe Tarel, Didier Aubert, and Eric Dumont. 2011. Blind contrast enhancement assessment by gradient ratioing at visible edges. Image Analysis & Stereology , Vol. 27, 2 (2011), 87--95.Google ScholarGoogle ScholarCross RefCross Ref
  13. Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided image filtering. IEEE transactions on pattern analysis & machine intelligence 6 (2013), 1397--1409.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision. 1026--1034.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Glenn E Healey and Raghava Kondepudy. 1994. Radiometric CCD camera calibration and noise estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 16, 3 (1994), 267--276.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia. ACM, 675--678.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Daniel J Jobson, Zia-ur Rahman, and Glenn A Woodell. 1997. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image processing , Vol. 6, 7 (1997), 965--976.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Hamid Reza Vaezi Joze, Mark S Drew, Graham D Finlayson, and Perla Aurora Troncoso Rey. 2012. The role of bright pixels in illumination estimation. In Color and Imaging Conference , Vol. 2012. Society for Imaging Science and Technology, 41--46.Google ScholarGoogle Scholar
  19. Fei Kou, Zhengguo Li, Changyun Wen, and Weihai Chen. 2017. Multi-scale exposure fusion via gradient domain guided image filtering. In 2017 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1105--1110.Google ScholarGoogle ScholarCross RefCross Ref
  20. Lin Li, Ronggang Wang, Wenmin Wang, and Wen Gao. 2015. A low-light image enhancement method for both denoising and contrast enlarging. In 2015 IEEE International Conference on Image Processing (ICIP). IEEE, 3730--3734.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Mading Li, Jiaying Liu, Wenhan Yang, Xiaoyan Sun, and Zongming Guo. 2018. Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model. IEEE Transactions on Image Processing , Vol. 27, 6 (2018), 2828--2841.Google ScholarGoogle ScholarCross RefCross Ref
  22. Ce Liu, Richard Szeliski, Sing Bing Kang, C Lawrence Zitnick, and William T Freeman. 2008. Automatic estimation and removal of noise from a single image. IEEE transactions on pattern analysis and machine intelligence , Vol. 30, 2 (2008), 299--314.Google ScholarGoogle Scholar
  23. Kin Gwn Lore, Adedotun Akintayo, and Soumik Sarkar. 2017. LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition , Vol. 61 (2017), 650--662.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Jean Michel Morel, Ana Belén Petro, and Catalina Sbert. 2010. A PDE formalization of Retinex theory. IEEE Transactions on Image Processing , Vol. 19, 11 (2010), 2825--2837.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Seonhee Park, Soohwan Yu, Minseo Kim, Kwanwoo Park, and Joonki Paik. 2018. Dual Autoencoder Network for Retinex-based Low-Light Image Enhancement. IEEE Access (2018).Google ScholarGoogle ScholarCross RefCross Ref
  26. Zia-ur Rahman, Daniel J Jobson, and Glenn A Woodell. 2004. Retinex processing for automatic image enhancement. Journal of Electronic imaging , Vol. 13, 1 (2004), 100--111.Google ScholarGoogle ScholarCross RefCross Ref
  27. Shanmuganathan Raman and Subhasis Chaudhuri. 2009. Bilateral Filter Based Compositing for Variable Exposure Photography.. In Eurographics (short papers). 1--4.Google ScholarGoogle Scholar
  28. Rajeev Ramanath, Wesley E Snyder, Griff L Bilbro, and William A Sander. 2002. Demosaicking methods for Bayer color arrays. Journal of Electronic imaging , Vol. 11, 3 (2002), 306--316.Google ScholarGoogle ScholarCross RefCross Ref
  29. Jianbing Shen, Ying Zhao, Shuicheng Yan, Xuelong Li, et almbox. 2014. Exposure fusion using boosting Laplacian pyramid. IEEE Trans. Cybernetics , Vol. 44, 9 (2014), 1579--1590.Google ScholarGoogle ScholarCross RefCross Ref
  30. Liang Shen, Zihan Yue, Fan Feng, Quan Chen, Shihao Liu, and Jie Ma. 2017. Msr-net: Low-light image enhancement using deep convolutional network. arXiv preprint arXiv:1711.02488 (2017).Google ScholarGoogle Scholar
  31. Rui Shen, Irene Cheng, Jianbo Shi, and Anup Basu. 2011. Generalized random walks for fusion of multi-exposure images. IEEE Transactions on Image Processing , Vol. 20, 12 (2011), 3634--3646.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Yanghai Tsin, Visvanathan Ramesh, and Takeo Kanade. 2001. Statistical calibration of CCD imaging process. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, Vol. 1. IEEE, 480--487.Google ScholarGoogle ScholarCross RefCross Ref
  33. Shuhang Wang and Gang Luo. 2018. Naturalness Preserved Image Enhancement Using a Priori Multi-Layer Lightness Statistics. IEEE transactions on image processing: a publication of the IEEE Signal Processing Society , Vol. 27, 2 (2018), 938--948.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et almbox. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing , Vol. 13, 4 (2004), 600--612.Google ScholarGoogle Scholar
  35. Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. 2018. Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560 (2018).Google ScholarGoogle Scholar
  36. Jing Zhang, Yang Cao, Shuai Fang, Yu Kang, and Chang Wen Chen. 2017. Fast Haze Removal for Nighttime Image Using Maximum Reflectance Prior. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) .Google ScholarGoogle Scholar
  37. Jing Zhang, Yang Cao, Yang Wang, Chenglin Wen, and Chang Wen Chen. 2018a. Fully Point-wise Convolutional Neural Network for Modeling Statistical Regularities in Natural Images. In 2018 ACM Multimedia Conference on Multimedia Conference. ACM, 984--992.Google ScholarGoogle Scholar
  38. Jing Zhang and Dacheng Tao. 2019. FAMED-Net: A Fast and Accurate Multi-scale End-to-end Dehazing Network. IEEE Transactions on Image Processing (2019), 1--13. https://doi.org/10.1109/TIP.2019.2922837Google ScholarGoogle Scholar
  39. Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. 2011. FSIM: A feature similarity index for image quality assessment. IEEE transactions on Image Processing , Vol. 20, 8 (2011), 2378--2386.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Qing Zhang, Ganzhao Yuan, Chunxia Xiao, Lei Zhu, and Wei-Shi Zheng. 2018b. High-Quality Exposure Correction of Underexposed Photos. In 2018 ACM Multimedia Conference on Multimedia Conference. ACM, 582--590.Google ScholarGoogle Scholar
  41. Wei Zhang and Wai-Kuen Cham. 2012. Gradient-directed multiexposure composition. IEEE Transactions on Image Processing , Vol. 21, 4 (2012), 2318--2323.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Xiangdong Zhang, Peiyi Shen, Lingli Luo, Liang Zhang, and Juan Song. 2012. Enhancement and noise reduction of very low light level images. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012). Ieee, 2034--2037.Google ScholarGoogle Scholar

Index Terms

  1. Progressive Retinex: Mutually Reinforced Illumination-Noise Perception Network for Low-Light Image Enhancement

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      MM '19: Proceedings of the 27th ACM International Conference on Multimedia
      October 2019
      2794 pages
      ISBN:9781450368896
      DOI:10.1145/3343031

      Copyright © 2019 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 15 October 2019

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      MM '19 Paper Acceptance Rate252of936submissions,27%Overall Acceptance Rate995of4,171submissions,24%

      Upcoming Conference

      MM '24
      MM '24: The 32nd ACM International Conference on Multimedia
      October 28 - November 1, 2024
      Melbourne , VIC , Australia

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader