Skip to main content
SpringerLink
Log in

EEDNet: Enhanced Encoder-Decoder Network for AutoISP

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 Workshops (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12537))

Included in the following conference series:

Abstract

Image Signal Processor (ISP) plays a core rule in camera systems. However, ISP tuning is highly complicated and requires professional skills and advanced imaging experiences. To skip the painful ISP tuning process, we introduce EEDNet in this paper, which directly transforms an image in the raw space to an image in the sRGB space (RAW-to-RGB). Data-driven RAW-to-RGB mapping is a grand new low-level vision task. In this work, we propose a hypothesis of the receptive field that large receptive field (LRF) is essential in high-level computer vision tasks, but not crucial in low-level pixel-to-pixel tasks. Besides, we present a ClipL1 loss, which simultaneously considers easy examples and outliers during the optimization process. Benefiting from the LRF hypothesis and ClipL1 loss, EEDNet can generate high-quality pictures with more details. Our method achieves promising results on Zurich RAW2RGB (ZRR) dataset and won the first place in AIM2020 ISP challenging.

Y. Zhu, Z. Guo, T. Liang, X. He—Equal Contribution

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abdelhamed, A., et al.: Ntire 2020 challenge on real image denoising: Dataset, methods and results. arXiv preprint arXiv:2005.04117 (2020)

  2. Araujo, A., Norris, W., Sim, J.: Computing receptive fields of convolutional neural networks. Distill (2019). https://doi.org/10.23915/distill.00021, https://distill.pub/2019/computing-receptive-fields

  3. Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv preprint arXiv:1607.06450 (2016)

  4. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  Google Scholar 

  5. Chen, L., Zhang, H., Xiao, J., Nie, L., Shao, J., Liu, W., Chua, T.S.: Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 5659–5667 (2017)

    Google Scholar 

  6. Fan, Y., Yu, J., Liu, D., Huang, T.S.: Scale-wise convolution for image restoration (2019)

    Google Scholar 

  7. Ignatov, A., Patel, J., Timofte, R.: Rendering natural camera bokeh effect with deep learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (June 2020)

    Google Scholar 

  8. Ignatov, A., Timofte, R., et al.: AIM 2020 challenge on learned image signal processing pipeline. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  9. Ignatov, A., Van Gool, L., Timofte, R.: Replacing mobile camera isp with a single deep learning model. arXiv preprint arXiv:2002.05509 (2020)

  10. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

  11. Liang, Z., Cai, J., Cao, Z., Zhang, L.: Cameranet: A two-stage framework for effective camera isp learning (2019)

    Google Scholar 

  12. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2980–2988 (2017)

    Google Scholar 

  13. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3431–3440 (2015)

    Google Scholar 

  14. Marnerides, D., Bashford-Rogers, T., Hatchett, J., Debattista, K.: Expandnet: A deep convolutional neural network for high dynamic range expansion from low dynamic range content. Comput. Graph. Forum 37(2), 37–49 (2017)

    Article  Google Scholar 

  15. Nah, S., Son, S., Timofte, R., Lee, K.M.: Ntire 2020 challenge on image and video deblurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2020)

    Google Scholar 

  16. Purohit, K., Suin, M., Kandula, P., Ambasamudram, R.: Depth-guided dense dynamic filtering network for bokeh effect rendering. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). pp. 3417–3426 (2019)

    Google Scholar 

  17. Ratnasingam, S.: Deep camera: A fully convolutional neural network for image signal processing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops (2019)

    Google Scholar 

  18. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems. pp. 91–99 (2015)

    Google Scholar 

  19. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  20. Sun, Y., Yu, Y., Wang, W.: Moiré photo restoration using multiresolution convolutional neural networks. IEEE Trans. Image Process. 27(8), 4160–4172 (2018)

    Article  MathSciNet  Google Scholar 

  21. Tao, X., Gao, H., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  22. Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016)

  23. Wang, X., et al.: Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops (2018)

    Google Scholar 

  24. Yan, Q., et al.: Deep hdr imaging via a non-local network. IEEE Trans. Image Process. 29, 4308–4322 (2020)

    Article  Google Scholar 

  25. Yuan, S., et al.: Aim 2019 challenge on image demoireing: Methods and results. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). pp. 3534–3545 (2019)

    Google Scholar 

  26. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    Article  MathSciNet  Google Scholar 

  27. Zhang, K., Zuo, W., Zhang, L.: Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the Advance Research Program (31511130301); National Key Research and Development Program (2017YFF0209806), and National Natural Science Foundation of China (No. 61906193; No. 61906195; No. 61702510).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Anhui University, Hefei, 230601, China

    Yu Zhu & Bo Jiang

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Zhenyu Guo, Tian Liang, Xiangyu He, Chenghua Li, Cong Leng, Yifan Zhang & Jian Cheng

  3. School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, 100049, China

    Zhenyu Guo & Tian Liang

  4. Nanjing Artificial Intelligence Chip Research, Institute of Automation, Chinese Academy of Sciences (AiRiA), Nanjing, China

    Chenghua Li, Cong Leng, Yifan Zhang & Jian Cheng

Authors
  1. Yu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenyu Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tian Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiangyu He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chenghua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cong Leng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bo Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yifan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jian Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Chenghua Li or Jian Cheng .

Editor information

Editors and Affiliations

  1. University of Clermont Auvergne, Clermont Ferrand, France

    Adrien Bartoli

  2. Università degli Studi di Udine, Udine, Italy

    Andrea Fusiello

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhu, Y. et al. (2020). EEDNet: Enhanced Encoder-Decoder Network for AutoISP. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12537. Springer, Cham. https://doi.org/10.1007/978-3-030-67070-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-67070-2_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-67069-6

  • Online ISBN: 978-3-030-67070-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation