Skip to main content
SpringerLink
Log in

Multi-focus image fusion based on smooth and iteratively restore filter

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image fusion is the process of combining several images with different focus settings into a completely in-focus image. However, many state-of-the-art fusion methods cannot well preserve all the significant features of the source images to obtain an all-in-focus image. In this paper, an improved smooth and iteratively restore (SIR) filter is proposed to deal with the problem. The SIR filter can well smooth noise while retaining details of edges. First, SIR filtering is applied to decompose source images into base and detail layers. Second, saliency maps of the different layers of the sources are computed by a proposed salient feature filter. Third, pixel-wise maxima of the saliency maps are used to construct the binary decision maps for both source images. Then with the binary decision maps we fuse the base and detail layers respectively to yield the fused base and fused detail, which are then recombined to produce the final fused image. Spatial consistency is inherently guaranteed by this process. Tests on sets of grayscale and colour multi-focus images demonstrate that the proposed method achieves better performance than existing methods in terms of both subjective and objective evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Azarang A, Ghassemian H (2017) A new pansharpening method using multi resolution analysis framework and deep neural networks. 2017 3rd International Conference on Pattern Recognition and Image Analysis (IPRIA). IEEE

  2. Bavirisetti DP, Dhuli R (2018) Multi-focus image fusion using multi-scale image decomposition and saliency detection. Ain Shams Eng J 9(4):1103–1117

    Article  Google Scholar 

  3. Cai J, Cheng Q, Peng M et al (2017) Fusion of infrared and visible images based on nonsubsampled contourlet transform and sparse K-SVD dictionary learning. Infrared Phys Technol 82:85–95

    Article  Google Scholar 

  4. Cui G, Feng H, Xu Z et al (2015) Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition. Opt Commun 341:199–209

    Article  Google Scholar 

  5. CTD toolbox (2011) http://tag7.web.rice.edu/SplitBregmanfiles

  6. Du J, Li W, Xiao B et al (2016) Union Laplacian pyramid with multiple features for medical image fusion. Neurocomputing 194:326–339

    Article  Google Scholar 

  7. Du C, Gao S (2017) Image segmentation-based multi-focus image fusion through multi-scale convolutional neural network. IEEE access 5:15750–15761

    Article  Google Scholar 

  8. GFF toolbox (2013) http://xudongkang.weebly.com

  9. Guan J, Cham W (2017) Distortion based image quality index. Signal Information Processing Association Summit Conference. IEEE

  10. Huang W, Xiao L, Wei Z et al (2015) A new pan-sharpening method with deep neural networks. IEEE Geosci Remote Sens Lett 12(5):1037–1041

    Article  Google Scholar 

  11. Image fusion toolbox (2010) http://www.imagefusion.org

  12. Image fusion toolbox (2018) https://ww2.mathworks.cn/matlabcentral/fileexchange

  13. Jiang Y, Wang M (2014) Image fusion using multiscale edge-preserving decomposition based on weighted least squares filter. IET image Process (3):183–190

    Article  MathSciNet  Google Scholar 

  14. Kalantari NK, Ramamoorthi R (2017) Deep high dynamic range imaging of dynamic scenes. ACM Trans Graph 36(4):144:1–144:12

    Article  Google Scholar 

  15. Kou F, Li Z, Wen C et al (2018) Edge-preserving smoothing pyramid based multi-scale exposure fusion. J Vis Commun Image Represent 53:235–244

    Article  Google Scholar 

  16. Kumar BKS (2015) Image fusion based on pixel significance using cross bilateral filter. Signal Image Video Process 9(5):1193–1204

    Article  Google Scholar 

  17. Kniefacz P, Walter K (2015) Smooth and iteratively restore: a simple and fast edge-preserving smoothing model. arXiv:1505.06702

  18. Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    Article  Google Scholar 

  19. Li S, Kang X, Fang L et al (2017) Pixel-level image fusion: a survey of the state of the art. Inf Fusion 33:100–112

    Article  Google Scholar 

  20. Li W, Xie Y, Zhou H et al (2018) Structure-aware image fusion. Optik 172:1–11

    Article  Google Scholar 

  21. Liu S, Chen J (2016) A fast multi-focus image fusion algorithm by DWT and focused region decision map. Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). IEEE

  22. Liu S, Zhao J, Shi M (2015) Medical image fusion based on rolling guidance filter and spiking cortical model. Comput Math Methods Med 2015:1–9

  23. Liu Y, Chen X, Ward RK et al (2016) Image fusion with convolutional sparse representation. IEEE Signal Process Lett 23(12):1882–1886

    Article  Google Scholar 

  24. Liu Y, Chen X, Peng H et al (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36:191–207

    Article  Google Scholar 

  25. Liu Y, Chen X, Wang Z et al (2018) Deep learning for pixel-level image fusion: Recent advances and future prospects. Inf Fusion 42:158–173

    Article  Google Scholar 

  26. Ma J et al (2017) Multi-focus image fusion based on multi-scale focus measures and generalized random walk. 36th Chinese Control Conference (CCC). IEEE

  27. Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: a survey. Inf Fusion 45:153–178

    Article  Google Scholar 

  28. Multi-focus Image database (2014) http://www.imgfsr.com/sitebuilder/images

  29. Multi-focus Image database (2016) http://www.ece.lehigh.edu/SPCRL/IF/toy.htm

  30. Nejati M, Samavi S, Shirani S (2015) Multi-focus image fusion using dictionary-based sparse representation. Inf Fusion 25:72–84

    Article  Google Scholar 

  31. NSCT toolbox (2005) http://www.ifp.illinois.edu/minhdo/software

  32. Petrovic V, Xydeas C (2005) Objective Image Fusion Performance Characterisation. Tenth IEEE International Conference on Computer Vision. IEEE

  33. Qu X, Yan J, Xiao H et al (2008) Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Autom Sin 34(12):1508–1514

    Article  Google Scholar 

  34. Tello-Mijares S, Bescos J (2018) Region-based multifocus image fusion for the precise acquisition of Pap smear images. J Biomed Opt 23(5):056005

    Article  Google Scholar 

  35. Toet A (2016) Iterative guided image fusion. Peer J Comput Sci 2:e80

    Article  Google Scholar 

  36. Wang Z, Bovik AC, Sheikh HR et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  37. Yang B, Zhong J, Li Y et al (2017) Multi-focus image fusion and super-resolution with convolutional neural network. Int J Wave Multiresolution Inf Process 15(04):1750037

    Article  MathSciNet  Google Scholar 

  38. Yang Y, Que Y, Huang S et al (2017) Multiple visual features measurement with gradient domain guided filtering for multisensor image fusion. IEEE Trans Instrum Meas 66(4):691–703

    Article  Google Scholar 

  39. Zhang Y, Chen L, Jia J et al (2014) Multi-focus image fusion based on non-negative matrix factorization and difference images. Signal Process 105:84–97

    Article  Google Scholar 

  40. Zhang Y (2015) Multi-focus image fusion based on sparse decomposition. Int J Signal Process Image Process Pattern Recogn 8(2):157–164

    Google Scholar 

  41. Zhang Y, Chen L, Zhao Z et al (2016) Multi-focus image fusion based on cartoon-texture image decomposition. Optik-Int J Light Electron Opt 127(3):1291–1296

    Article  Google Scholar 

  42. Zhang Y, Bai X, Wang T (2017) Boundary finding based multi-focus image fusion through multi-scale morphological focus-measure. Inf Fusion 35:81–101

    Article  Google Scholar 

  43. Zhan K, Xie Y, Wang H, Min Y (2017) Fast filtering image fusion. J Electron Imaging J Electron Imaging 26(6):063004

    Google Scholar 

  44. Zhao J, Feng H, Xu Z et al (2013) Detail enhanced multi-source fusion using visual weight map extraction based on multi scale edge preserving decomposition. Opt Commun 287:45–52

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank Kang Xudong, Zhan kun, Zhou Zhiqiang, D. P. Bavirisetti, Qu Xiaobo, Alexander Toet for providing their codes. In addition, we would like to thank the editors and anonymous reviewers for their comments and suggestions. This work was supported by the Scientific and Technological Project of Henan Province (No.192102210122), the Postdoctoral Science Foundation of China (No.2015M582697), the Henan Province Basic and Cutting-Edge Technology Research Project of China (No. 17HASTIT024), the outstanding talents of scientific and technological innovation in Henan (No.184200510011), the National Natural Science Foundation of China (No. 61502219) and the International Science and Technology Cooperation Program of China (No. 2016YFE0104600).

Author information

Authors and Affiliations

  1. Luoyang Normal University, Luoyang, 471022, China

    Yongxin Zhang & Youzhong Ma

  2. School of Information Science, Northwest University, Xi’an, 710127, China

    Wenpeng Zhu & Xunli Fan

Authors
  1. Yongxin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenpeng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Youzhong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xunli Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongxin Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Zhu, W., Ma, Y. et al. Multi-focus image fusion based on smooth and iteratively restore filter. Multimed Tools Appl 78, 35027–35052 (2019). https://doi.org/10.1007/s11042-019-08127-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08127-6

Keywords

Search

Navigation