Skip to main content
SpringerLink
Log in

AIM 2020 Challenge on Learned Image Signal Processing Pipeline

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 Workshops (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12537))

Included in the following conference series:

Abstract

This paper reviews the second AIM learned ISP challenge and provides the description of the proposed solutions and results. The participating teams were solving a real-world RAW-to-RGB mapping problem, where to goal was to map the original low-quality RAW images captured by the Huawei P20 device to the same photos obtained with the Canon 5D DSLR camera. The considered task embraced a number of complex computer vision subtasks, such as image demosaicing, denoising, white balancing, color and contrast correction, demoireing, etc. The target metric used in this challenge combined fidelity scores (PSNR and SSIM) with solutions’ perceptual results measured in a user study. The proposed solutions significantly improved the baseline results, defining the state-of-the-art for practical image signal processing pipeline modeling.

A. Ignatov and R. Timofte ({andrey,radu.timofte}@vision.ee.ethz.ch, ETH Zurich) are the challenge organizers, while the other authors participated in the challenge.

The Appendix A contains the authors’ teams and affiliations.

AIM 2020 webpage: https://data.vision.ee.ethz.ch/cvl/aim20/.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abdelhamed, A., Lin, S., Brown, M.S.: A high-quality denoising dataset for smartphone cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1692–1700 (2018)

    Google Scholar 

  2. Abdelhamed, A., Timofte, R., Brown, M.S., et al.: Ntire 2019 challenge on real image denoising: Methods and results. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2019)

    Google Scholar 

  3. Ancuti, C.O., Ancuti, C., Timofte, R., et al.: Ntire 2019 challenge on image dehazing: methods and results. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2019)

    Google Scholar 

  4. Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., Zelnik-Manor, L.: The 2018 pirm challenge on perceptual image super-resolution. In: The European Conference on Computer Vision (ECCV) Workshops (2018)

    Google Scholar 

  5. Cai, J., Gu, S., Timofte, R., Zhang, L., et al.: Ntire 2019 challenge on real image super-resolution: methods and results. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2019)

    Google Scholar 

  6. Cao, Y., Xu, J., Lin, S., Wei, F., Hu, H.: Gcnet: Non-local networks meet squeeze-excitation networks and beyond. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)

    Google Scholar 

  7. Chen, Y.S., Wang, Y.C., Kao, M.H., Chuang, Y.Y.: Deep photo enhancer: unpaired learning for image enhancement from photographs with gans. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  8. Dai, L., Liu, X., Li, C., Chen, J.: AWNet: attentive wavelet network for image isp. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  9. El Helou, M., Zhou, R., Süsstrunk, S., Timofte, R., et al.: AIM 2020: Scene relighting and illumination estimation challenge. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  10. Fuoli, D., Huang, Z., Gu, S., Timofte, R., et al.: AIM 2020 challenge on video extreme super-resolution: methods and results. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  11. Gu, S., Timofte, R.: A brief review of image denoising algorithms and beyond. In: Escalera, S., Ayache, S., Wan, J., Madadi, M., Guclu, U., Baro, X., (eds.) Inpainting and Denoising Challenges. The Springer Series on Challenges in Machine Learning. Springer, Cham (2019) https://doi.org/10.1007/978-3-030-25614-2_1

  12. Gu, S., Timofte, R., Zhang, R., et al.: Ntire 2019 challenge on image colorization: report. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2019)

    Google Scholar 

  13. Hui, Z., Gao, X., Yang, Y., Wang, X.: Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th ACM International Conference on Multimedia. pp. 2024–2032 (2019)

    Google Scholar 

  14. Hui, Z., Wang, X., Deng, L., Gao, X.: Perception-preserving convolutional networks for image enhancement on smartphones. In: European Conference on Computer Vision Workshops (2018)

    Google Scholar 

  15. Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., Van Gool, L.: Dslr-quality photos on mobile devices with deep convolutional networks. In: the IEEE International Conference on Computer Vision (ICCV) (2017)

    Google Scholar 

  16. Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., Van Gool, L.: Wespe: weakly supervised photo enhancer for digital cameras. arXiv preprint arXiv:1709.01118 (2017)

  17. Ignatov, A., Timofte, R.: Ntire 2019 challenge on image enhancement: methods and results. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  18. Ignatov, A.,et al.: Ai benchmark: Running deep neural networks on android smartphones. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

    Google Scholar 

  19. Ignatov, A., et al.: Aim 2019 challenge on raw to rgb mapping: Methods and results. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). pp. 3584–3590. IEEE (2019)

    Google Scholar 

  20. Ignatov, A., et al.: Ai benchmark: All about deep learning on smartphones in 2019. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). pp. 3617–3635. IEEE (2019)

    Google Scholar 

  21. Ignatov, A., Timofte, R., et al.: Pirm challenge on perceptual image enhancement on smartphones: Report. In: European Conference on Computer Vision Workshops (2018)

    Google Scholar 

  22. Ignatov, A., Timofte, R., et al.: AIM 2020 challenge on learned image signal processing pipeline. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  23. Ignatov, A., Timofte, R., et al.: AIM 2020 challenge on rendering realistic bokeh. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  24. Ignatov, A., Van Gool, L., Timofte, R.: Replacing mobile camera isp with a single deep learning model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 536–537 (2020)

    Google Scholar 

  25. Kim, B.H., Song, J., Ye, J.C., Baek, J.: PyNET-CA: enhanced PyNET with channel attention for end-to-end mobile image signal processing. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  26. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  27. Liu, H., Navarrete Michelini, P., Zhu, D.: Deep networks for image to image translation with mux and demux layers. In: European Conference on Computer Vision Workshops (2018)

    Google Scholar 

  28. Liu, P., Zhang, H., Zhang, K., Lin, L., Zuo, W.: Multi-level wavelet-cnn for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. pp. 773–782 (2018)

    Google Scholar 

  29. Luo, P., Ren, J., Peng, Z., Zhang, R., Li, J.: Differentiable learning-to-normalize via switchable normalization. arXiv preprint arXiv:1806.10779 (2018)

  30. Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Paul Smolley, S.: Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2794–2802 (2017)

    Google Scholar 

  31. Mertens, T., Kautz, J., Van Reeth, F.: Exposure fusion: A simple and practical alternative to high dynamic range photography. In: Computer graphics forum. vol. 28, pp. 161–171. Wiley Online Library (2009)

    Google Scholar 

  32. Nah, S., et al.: Ntire 2019 challenge on video deblurring and super-resolution: dataset and study. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2019)

    Google Scholar 

  33. Ntavelis, E., Romero, A., Bigdeli, S.A., Timofte, R., et al.: AIM 2020 challenge on image extreme inpainting. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  34. Pengfei, Z., et al.: Range scaling global u-net for perceptual image enhancement on mobile devices. In: European Conference on Computer Vision Workshops (2018)

    Google Scholar 

  35. Smith, L.N., Topin, N.: Super-convergence: Very fast training of neural networks using large learning rates. In: Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications. vol. 11006, p. 1100612. International Society for Optics and Photonics (2019)

    Google Scholar 

  36. Son, S., Lee, J., Nah, S., Timofte, R., Lee, K.M., et al.: AIM 2020 challenge on video temporal super-resolution. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  37. de Stoutz, E., Ignatov, A., Kobyshev, N., Timofte, R., Van Gool, L.: Fast perceptual image enhancement. In: European Conference on Computer Vision Workshops (2018)

    Google Scholar 

  38. Tao, X., Gao, H., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 8174–8182 (2018)

    Google Scholar 

  39. Timofte, R., Gu, S., Wu, J., Van Gool, L.: Ntire 2018 challenge on single image super-resolution: Methods and results. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2018)

    Google Scholar 

  40. Van Vu, T., Van Nguyen, C., Pham, T.X., Liu, T.M., Youu, C.D.: Fast and efficient image quality enhancement via desubpixel convolutional neural networks. In: European Conference on Computer Vision Workshops (2018)

    Google Scholar 

  41. Wei, P., Lu, H., Timofte, R., Lin, L., Zuo, W., et al.: AIM 2020 challenge on real image super-resolution. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  42. Zhang, K., Danelljan, M., Li, Y., Timofte, R., et al.: AIM 2020 challenge on efficient super-resolution: Methods and results. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

  43. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 286–301 (2018)

    Google Scholar 

  44. Zhu, Y., et al.: EEDNet: enhanced encoder-decoder network for autoisp. In: European Conference on Computer Vision Workshops (2020)

    Google Scholar 

Download references

Acknowledgments

We thank the AIM 2020 sponsors: Huawei, MediaTek, Qualcomm, NVIDIA, Google and Computer Vision Lab/ETH Zürich.

Author information

Authors and Affiliations

  1. Computer Vision Lab, ETH Zurich, Zurich, Switzerland

    Andrey Ignatov & Radu Timofte

  2. Harbin Institute of Technology, Harbin, China

    Zhilu Zhang, Ming Liu, Haolin Wang & Wangmeng Zuo

  3. SenseTime, Hong Kong, China

    Jiawei Zhang, Ruimao Zhang, Zhanglin Peng & Sijie Ren

  4. McMaster University, Hamilton, Canada

    Linhui Dai, Xiaohong Liu, Chengqi Li & Jun Chen

  5. Vermilion Vision, Abbeville, USA

    Yuichi Ito

  6. CVPR Unit, ISI Kolkata, Kolkata, India

    Bhavya Vasudeva, Puneesh Deora & Umapada Pal

  7. Nanjing Artificial Intelligence Chip Research, Institute of Automation Chinese Academy of Sciences (AiRiA), MAICRO, Nanjing, China

    Zhenyu Guo, Yu Zhu, Tian Liang, Chenghua Li & Cong Leng

  8. Baidu Research, Sunnyvale, USA

    Zhihong Pan & Baopu Li

  9. Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea

    Byung-Hoon Kim, Joonyoung Song & Jong Chul Ye

  10. Amazon Web Services, Seattle, South Korea

    JaeHyun Baek

  11. ST Unitas AI Research (STAIR), Narashino, South Korea

    Magauiya Zhussip & Hwechul Cho Ye

  12. Allganize, Oakland, USA

    Yeskendir Koishekenov

  13. SenseBrain, New York City, USA

    Xin Liu, Xueying Hu, Jun Jiang & Jinwei Gu

  14. Beijing University of Posts and Telecommunications, Beijing, China

    Kai Li & Pengliang Tan

  15. Santa Clara University, Santa Clara, USA

    Bingxin Hou

Authors
  1. Andrey Ignatov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Radu Timofte
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhilu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haolin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wangmeng Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jiawei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ruimao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Zhanglin Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sijie Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Linhui Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Xiaohong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Chengqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Jun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Yuichi Ito
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Bhavya Vasudeva
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Puneesh Deora
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Umapada Pal
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Zhenyu Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Yu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Tian Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Chenghua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Cong Leng
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Zhihong Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Baopu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Byung-Hoon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  27. Joonyoung Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  28. Jong Chul Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  29. JaeHyun Baek
    View author publications

    You can also search for this author in PubMed Google Scholar

  30. Magauiya Zhussip
    View author publications

    You can also search for this author in PubMed Google Scholar

  31. Yeskendir Koishekenov
    View author publications

    You can also search for this author in PubMed Google Scholar

  32. Hwechul Cho Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  33. Xin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  34. Xueying Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  35. Jun Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  36. Jinwei Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  37. Kai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  38. Pengliang Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  39. Bingxin Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrey Ignatov .

Editor information

Editors and Affiliations

  1. University of Clermont Auvergne, Clermont Ferrand, France

    Adrien Bartoli

  2. Università degli Studi di Udine, Udine, Italy

    Andrea Fusiello

A Appendix 1: Teams and affiliations

A Appendix 1: Teams and affiliations

AIM 2020 Learned ISP Challenge Team

Title: AIM 2020 Challenge on Learned Image Signal Processing Pipeline

Members: Andrey Ignatov  –  andrey@vision.ee.ethz.ch,

          Radu Timofte  –  radu.timofte@vision.ee.ethz.ch

Affiliations: Computer Vision Lab, ETH Zurich, Switzerland

MW-ISPNet

Title: Multi-level Wavelet ISP Network

Members: Zhilu Zhang \(^1\)  –  cszlzhang@outlook.com,

          Ming Liu \(^1\), Haolin Wang \(^1\), Wangmeng Zuo \(^1\)

          Jiawei Zhang \(^2\), Ruimao Zhang \(^2\), Zhanglin Peng \(^2\), Sijie Ren \(^2\)

Affiliations:\(^1\) – Harbin Institute of Technology, China

            \(^2\) – SenseTime, China

MacAI

Title: Attentive Wavelet Network for Image ISP [8]

Members: Linhui Dai  –  dail5@mcmaster.ca,

          Xiaohong Liu, Chengqi Li, Jun Chen

Affiliations: McMaster University, Canada

Vermilion Vision

Title: Scale Recurrent Deep Tone Mapping

Members: Yuichi Ito  –  yito@vermilionvision.net,

Affiliations: Vermilion Vision, United States

Eureka

Title: Local and Global Enhancement Network as Learned ISP

Members: Bhavya Vasudeva  –  bhavyavasudeva10@gmail.com,

          Puneesh Deora, Umapada Pal

Affiliations: CVPR Unit, ISI Kolkata, India

Airia_CG

Title 1: EEDNet: Enhanced Encoder-Decoder Network

Title 2: PUNet: Progressive U-Net via Contrast-aware Channel Attention

Members: Zhenyu Guo  –  guozhenyu2019@ia.ac.cn,

          Yu Zhu, Tian Liang, Chenghua Li, Cong Leng

Affiliations: Nanjing Artificial Intelligence Chip Research, Institute of Automation

            Chinese Academy of Sciences (AiRiA), MAICRO, China

Baidu Research Vision

Title: Learned Smartphone ISP using Mosaic-Adaptive Dense Residual Network

Members: Zhihong Pan  –  zhihongpan@baidu.com,

          Baopu Li

Affiliations: Baidu Research, United States

Skyb

Title: PyNet-CA: Enhanced PyNet with Channel Attention for Mobile ISP

Members: Byung-Hoon Kim \(^1\)  –  egyptdj@kaist.ac.kr,

          Joonyoung Song \(^1\), Jong Chul Ye \(^1\), JaeHyun Baek \(^2\)

Affiliations:\(^1\) – Korea Advanced Institute of Science and Technology (KAIST),

            \(^2\) – Amazon Web Services, South Korea

STAIR

Title: Recursive Residual Group Network for Image Mapping

Members: Magauiya Zhussip \(^1\)  –  magauiya173@gmail.com,

          Yeskendir Koishekenov \(^2\), Hwechul Cho Ye \(^1\)

Affiliations:\(^1\) – ST Unitas AI Research (STAIR), South Korea

            \(^2\) – Allganize, South Korea

SenseBrainer

Title: Multiscaled UNet

Members: Xin Liu  –  liuxin@sensebrain.site,

          Xueying Hu, Jun Jiang, Jinwei Gu

Affiliations: SenseBrain, United States

Bupt-mtc206

Title: RRGNet for Smartphone ISP

Members: Kai Li  –  492071523@qq.com,

          Pengliang Tan

Affiliations: Beijing University of Posts and Telecommunications, China

BingSoda

Title: Pixel-Wise Color Distance (PWCD model)

Members: Bingxin Hou  –  houbingxin@gmail.com,

Affiliations: Santa Clara University, United States

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ignatov, A. et al. (2020). AIM 2020 Challenge on Learned Image Signal Processing Pipeline. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12537. Springer, Cham. https://doi.org/10.1007/978-3-030-67070-2_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-67070-2_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-67069-6

  • Online ISBN: 978-3-030-67070-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation