Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Visual Saliency Detection via Convolutional Gated Recurrent Units

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 11954))

Included in the following conference series:

Abstract

Context is an important aspect for accurate saliency detection. However, the question of how to formally model image context within saliency detection frameworks is still an open problem. Recent saliency detection models designed using complex Deep Neural Networks to extract robust features, however often fail to select the right contextual features. These methods generally utilize physical attributes of objects for generating final saliency maps, but ignores scene contextual information. In this paper, we overcome such limitation using (i) a proposed novel end-to-end framework with a Contextual Unit (CTU) module that models the scene contextual information to give efficient saliency maps with the help of Convolutional GRU (Conv-GRU). This is the first work reported so far that utilizes Conv-GRU to generate image saliency maps. In addition, (ii) we propose a novel way of using the Conv-GRU that helps to refine saliency maps based on input image context. The proposed model has been evaluated on challenging benchmark saliency datasets, where it outperforms prominent state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://pytorch.org.

References

  1. Ballas, N., Yao, L., Pal, C.J., Courville, A.C.: Delving deeper into convolutional networks for learning video representations. CoRR abs/1511.06432 (2016)

    Google Scholar 

  2. Borji, A., Cheng, M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015). https://doi.org/10.1109/TIP.2015.2487833

    Article  MathSciNet  MATH  Google Scholar 

  3. Byeon, W., Breuel, T.M., Raue, F., Liwicki, M.: Scene labeling with LSTM recurrent neural networks. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3547–3555, June 2015. https://doi.org/10.1109/CVPR.2015.7298977

  4. Chen, S., Tan, X., Wang, B., Hu, X.: Reverse attention for salient object detection. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11213, pp. 236–252. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01240-3_15

    Chapter  Google Scholar 

  5. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734. Association for Computational Linguistics, Doha, Qatar, October 2014. https://doi.org/10.3115/v1/D14-1179, https://www.aclweb.org/anthology/D14-1179

  6. Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS 2014 Workshop on Deep Learning, December 2014 (2014)

    Google Scholar 

  7. Hou, Q., Cheng, M., Hu, X., Borji, A., Tu, Z., Torr, P.H.S.: Deeply supervised salient object detection with short connections. IEEE Trans. Pattern Anal. Mach. Intell. 41(4), 815–828 (2019). https://doi.org/10.1109/TPAMI.2018.2815688

    Article  Google Scholar 

  8. Hu, P., Shuai, B., Liu, J., Wang, G.: Deep level sets for salient object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 540–549, July 2017. https://doi.org/10.1109/CVPR.2017.65

  9. Krähenbühl, P., Koltun, V.: Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., Pereira, F., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 24, pp. 109–117. Curran Associates, Inc. (2011). http://papers.nips.cc/paper/4296-efficient-inference-in-fully-connected-crfs-with-gaussian-edge-potentials.pdf

  10. Kuen, J., Wang, Z., Wang, G.: Recurrent attentional networks for saliency detection. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    Google Scholar 

  11. Li, G., Yu, Y.: Visual saliency detection based on multiscale deep CNN features. IEEE Trans. Image Process. 25(11), 5012–5024 (2016). https://doi.org/10.1109/TIP.2016.2602079

    Article  MathSciNet  MATH  Google Scholar 

  12. Li, X., Yang, F., Cheng, H., Liu, W., Shen, D.: Contour knowledge transfer for salient object detection. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11219, pp. 370–385. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01267-0_22

    Chapter  Google Scholar 

  13. Li, Y., Hou, X., Koch, C., Rehg, J.M., Yuille, A.L.: The secrets of salient object segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 280–287, June 2014. https://doi.org/10.1109/CVPR.2014.43

  14. Liu, N., Han, J.: DHSNet: deep hierarchical saliency network for salient object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 678–686, June 2016. https://doi.org/10.1109/CVPR.2016.80

  15. Liu, N., Han, J., Yang, M.: PiCANet: learning pixel-wise contextual attention for saliency detection. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3089–3098, June 2018. https://doi.org/10.1109/CVPR.2018.00326

  16. Liu, Y., Wang, R., Shan, S., Chen, X.: Structure inference net: object detection using scene-level context and instance-level relationships. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    Google Scholar 

  17. Luo, Z., Mishra, A., Achkar, A., Eichel, J., Li, S., Jodoin, P.: Non-local deep features for salient object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6593–6601, July 2017. https://doi.org/10.1109/CVPR.2017.698

  18. Roy, S., Das, S.: Multi-criteria energy minimization with boundedness, edge-density and rarity, for object saliency in natural images. In: The Ninth Indian Conference on Computer Vision, Graphics, Image Processing (ICVGIP), Bangalore, India, 14–17 December 2014 (2014)

    Google Scholar 

  19. Shi, J., Yan, Q., Xu, L., Jia, J.: Hierarchical image saliency detection on extended CSSD. IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 717–729 (2016). https://doi.org/10.1109/TPAMI.2015.2465960

    Article  Google Scholar 

  20. Siam, M., Valipour, S., Jagersand, M., Ray, N.: Convolutional gated recurrent networks for video segmentation. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3090–3094, September 2017. https://doi.org/10.1109/ICIP.2017.8296851

  21. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556, September 2014

  22. Torralba, A., Castelhano, M.S., Oliva, A., Henderson, J.M.: Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychol. Rev. 113, 2006 (2006)

    Article  Google Scholar 

  23. Wang, L., et al.: Learning to detect salient objects with image-level supervision. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3796–3805, July 2017. https://doi.org/10.1109/CVPR.2017.404

  24. Wang, L., Wang, L., Lu, H., Zhang, P., Ruan, X.: Salient object detection with recurrent fully convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 1 (2018). https://doi.org/10.1109/TPAMI.2018.2846598

    Article  Google Scholar 

  25. Wang, T., Borji, A., Zhang, L., Zhang, P., Lu, H.: A stagewise refinement model for detecting salient objects in images. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4039–4048, October 2017. https://doi.org/10.1109/ICCV.2017.433

  26. Wang, T., et al.: Detect globally, refine locally: a novel approach to saliency detection. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3127–3135, June 2018. https://doi.org/10.1109/CVPR.2018.00330

  27. Wang, W., Shen, J., Dong, X., Borji, A.: Salient object detection driven by fixation prediction. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    Google Scholar 

  28. Xie, S., Tu, Z.: Holistically-nested edge detection. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1395–1403, December 2015. https://doi.org/10.1109/ICCV.2015.164

  29. Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.: Saliency detection via graph-based manifold ranking. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173, June 2013. https://doi.org/10.1109/CVPR.2013.407

  30. Zeng, Y., Lu, H., Zhang, L., Feng, M., Borji, A.: Learning to promote saliency detectors. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1644–1653, June 2018. https://doi.org/10.1109/CVPR.2018.00177

  31. Zhang, L., Dai, J., Lu, H., He, Y., Wang, G.: A bi-directional message passing model for salient object detection. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1741–1750, June 2018. https://doi.org/10.1109/CVPR.2018.00187

  32. Zhang, P., Wang, D., Lu, H., Wang, H., Ruan, X.: Amulet: aggregating multi-level convolutional features for salient object detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 202–211, October 2017. https://doi.org/10.1109/ICCV.2017.31

  33. Zhang, P., Wang, D., Lu, H., Wang, H., Yin, B.: Learning uncertain convolutional features for accurate saliency detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 212–221, October 2017. https://doi.org/10.1109/ICCV.2017.32

  34. Zhang, P., Liu, W., Lu, H., Shen, C.: Salient object detection by lossless feature reflection. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI 2018, pp. 1149–1155. AAAI Press (2018). http://dl.acm.org/citation.cfm?id=3304415.3304578

  35. Zhang, X., Wang, T., Qi, J., Lu, H., Wang, G.: Progressive attention guided recurrent network for salient object detection. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    Google Scholar 

  36. Zhao, R., Ouyang, W., Li, H., Wang, X.: Saliency detection by multi-context deep learning. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1265–1274, June 2015. https://doi.org/10.1109/CVPR.2015.7298731

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Technology Madras, Chennai, 600036, India

    Sayanti Bardhan & Sukhendu Das

  2. National Institute of Ocean Technology, Chennai, 600100, India

    Sayanti Bardhan & Shibu Jacob

Authors
  1. Sayanti Bardhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sukhendu Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shibu Jacob
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sayanti Bardhan .

Editor information

Editors and Affiliations

  1. Australian National University, Canberra, ACT, Australia

    Tom Gedeon

  2. Murdoch University, Murdoch, WA, Australia

    Kok Wai Wong

  3. Kyungpook National University, Daegu, Korea (Republic of)

    Minho Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bardhan, S., Das, S., Jacob, S. (2019). Visual Saliency Detection via Convolutional Gated Recurrent Units. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11954. Springer, Cham. https://doi.org/10.1007/978-3-030-36711-4_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-36711-4_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-36710-7

  • Online ISBN: 978-3-030-36711-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation