Skip to main content
Log in

Single image super resolution for texture images through neighbor embedding

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This article proposes an improved learning based super resolution scheme using manifold learning for texture images. Pseudo Zernike moment (PZM) has been employed to extract features from the texture images. In order to efficiently retrieve similar patches from the training patches, feature similarity index matrix (FSIM) has been used. Subsequently, for reconstruction of the high resolution (HR) patch, a collaborative optimal weight is generated from the least square (LS) and non-negative matrix factorization (NMF) methods. The proposed method is tested on some color texture, gray texture, and some standard images. Results of the proposed method on texture images advocate its superior performance over established state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Ahn IJ, Nam WH (2016) Texture enhancement via high-resolution style transfer for single-image super-resolution. arXiv:1612.00085

  2. Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183. https://doi.org/10.1109/TPAMI.2002.1033210

    Article  Google Scholar 

  3. Begin I, Ferrie FP (2004) Blind super-resolution using a learning-based approach. In: 2004 International conference on pattern recognition, pp 85–89. https://doi.org/10.1109/ICPR.2004.1334046

  4. Bevilacqua M, Roumy A, Guillemot C, Morel MLA (2013) Super-resolution using neighbor embedding of back-projection residuals. In: 18th International conference on digital signal processing, pp 1–8. https://doi.org/10.1109/ICDSP.2013.6622796

  5. Cao M, Gan Z, Zhu X (2012) Super-resolution algorithm through neighbor embedding with new feature selection and example training. In: 2012 IEEE International conference on signal processing, vol 2, pp 825–828. https://doi.org/10.1109/ICoSP.2012.6491708

  6. Chan TM, Zhang J (2006) An improved super-resolution with manifold learning and histogram matching. In: 2006 International conference on advances in biometrics, pp 756–762. https://doi.org/10.1007/11608288_101

  7. Chan TM, Zhang J, Pu J, Huang H (2009) Neighbor embedding based super-resolution algorithm through edge detection and feature selection. Pattern Recogn Lett 30(5):494–502. https://doi.org/10.1016/j.patrec.2008.11.008

    Article  Google Scholar 

  8. Chang X, Yang Y (2017) Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Transactions on Neural Networks and Learning Systems 28(10):2294–2305. https://doi.org/10.1109/TNNLS.2016.2582746

    Article  MathSciNet  Google Scholar 

  9. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: 2004 IEEE Computer society conference on computer vision and pattern recognition, vol 1, pp 275–28. https://doi.org/10.1109/CVPR.2004.1315043

  10. Chang X, Ma Z, Lin M, Yang Y, Hauptmann A (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 26 (8):3911–3920. https://doi.org/10.1109/TIP.2017.2708506

    Article  MathSciNet  Google Scholar 

  11. Chang X, Ma Z, Yang Y, Zeng Z, Hauptmann A G (2017) Bi-level semantic representation analysis for multimedia event detection. IEEE Trans Cybern 47(5):1180–1197. https://doi.org/10.1109/TCYB.2016.2539546

    Article  Google Scholar 

  12. Chang X, Yu YL, Yang Y, Xing EP (2017) Semantic pooling for complex event analysis in untrimmed videos. IEEE Trans Pattern Anal Mach Intell 39 (8):1617–1632. https://doi.org/10.1109/TPAMI.2016.2608901

    Article  Google Scholar 

  13. Chen X, Qi C (2014) Nonlinear neighbor embedding for single image super-resolution via kernel mapping. Signal Process 94:6–22. https://doi.org/10.1016/j.sigpro.2013.06.016

    Article  Google Scholar 

  14. Chong CW, Raveendran P, Mukundan R (2003) The scale invariants of pseudo-Zernike moments. Pattern Anal Appl 6(3):176–184. https://doi.org/10.1007/s10044-002-0183-5

    Article  MathSciNet  MATH  Google Scholar 

  15. Cruz C, Mehta R, Katkovnik V, Egiazarian K (2017) Single image super-resolution based on Wiener filter in similarity domain. arXiv:1704.04126

  16. Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630. https://doi.org/10.1109/TIP.2012.2235847

    Article  MathSciNet  MATH  Google Scholar 

  17. Fan W, Yeung DY (2007) Image hallucination using neighbor embedding over visual primitive manifolds. In: 2007 IEEE Conference on computer vision and pattern recognition, pp 1–7. https://doi.org/10.1109/CVPR.2007.383001

  18. Farsiu S, Robinson D, Elad M, Milanfar P (2004) Advances and challenges in super-resolution. Int J Imaging Syst Technol 14(2):47–57. https://doi.org/10.1002/ima.20007

    Article  Google Scholar 

  19. Freeman WT, Pasztor EC, Carmichael OT (2000) Learning low-level vision. Int J Comput Vis 40(1):25–47. https://doi.org/10.1023/A:1026501619075

    Article  MATH  Google Scholar 

  20. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65. https://doi.org/10.1109/38.988747

    Article  Google Scholar 

  21. Gao X, Wang Q, Li X, Tao D, Zhang K (2011) Zernike-moment-based image super resolution. IEEE Trans Image Process 20 (10):2738–2747. https://doi.org/10.1109/TIP.2011.2134859

    Article  MathSciNet  MATH  Google Scholar 

  22. Gao X, Zhang K, Tao D, Li X (2012) Joint learning for single-image super-resolution via a coupled constraint. IEEE Trans Image Process 21(2):469–480. https://doi.org/10.1109/TIP.2011.2161482

    Article  MathSciNet  MATH  Google Scholar 

  23. Gerchberg R (1974) Super-resolution through error energy reduction. Optica Acta: Int J Opt 21(9):709–720. https://doi.org/10.1080/713818946

    Article  Google Scholar 

  24. Guo K, Yang X, Lin W, Zhang R, Yu S (2012) Learning-based super-resolution method with a combining of both global and local constraints. IET Image Process 6(4):337–344. 10.1049/iet-ipr.2010.0430

    Article  MathSciNet  Google Scholar 

  25. HaCohen Y, Fattal R, Lischinski D (2010) Image upsampling via texture hallucination. In: 2010 IEEE International conference on computational photography, pp 1–8. https://doi.org/10.1109/ICCPHOT.2010.5585097

  26. Jiang J, Ma X, Cai Z, Hu R (2015) Sparse support regression for image super-resolution. IEEE Photon J 7(5):1–11. https://doi.org/10.1109/JPHOT.2015.2484287

    Article  Google Scholar 

  27. Kanan HR, Salkhordeh S (2016) Rotation invariant multi-frame image super resolution reconstruction using pseudo Zernike moments. Signal Process 118:103–114. https://doi.org/10.1016/j.sigpro.2015.05.015

    Article  Google Scholar 

  28. Kim SP, Bose NK, Valenzuela HM (1990) Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Trans Acoust Speech Signal Process 38(6):1013–1027. https://doi.org/10.1109/29.56062

    Article  Google Scholar 

  29. Li X, Orchard MT (2001) New edge-directed interpolation. IEEE Trans Image Process 10(10):1521–1527. https://doi.org/10.1109/83.951537

    Article  Google Scholar 

  30. Lin Z, Shum HY (2004) Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans Pattern Anal Mach Intell 26(1):83–97. https://doi.org/10.1109/TPAMI.2004.1261081

    Article  Google Scholar 

  31. Liu L, Huang W, Wang C (2017) Texture image prior for SAR image super resolution based on total variation regularization using split bregman iteration. Int J Remote Sens 38(20):5673–5687. https://doi.org/10.1080/01431161.2017.1346325

    Article  Google Scholar 

  32. Mishra D, Majhi B, Sa PK (2014) Neighbor embedding based super-resolution using residual luminance. In: 2014 Annual IEEE India conference (INDICON), pp 1–6. https://doi.org/10.1109/INDICON.2014.7030659

  33. Mishra D, Majhi B, Sa PK, Dash R (2016) Development of robust neighbor embedding based super-resolution scheme. Neurocomputing 202:49–66. https://doi.org/10.1016/j.neucom.2016.04.013

    Article  Google Scholar 

  34. Mishra D, Majhi B, Sa PK (2017) Improved feature selection for neighbor embedding super-resolution using Zernike moments. In: Proceedings of international conference on computer vision and image processing, pp 13–24. https://doi.org/10.1007/978-981-10-2107-7_2

  35. Nasrollahi K, Moeslund TB (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468. https://doi.org/10.1007/s00138-014-0623-4

    Article  Google Scholar 

  36. Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36. https://doi.org/10.1109/MSP.2003.1203207

    Article  Google Scholar 

  37. Patanavijit V (2009) Super-resolution reconstruction and its future research direction. AU J Technol 12(3):149–163

    Google Scholar 

  38. Rajan D, Chaudhuri S, Joshi MV (2003) Multi-objective super resolution: concepts and examples. IEEE Signal Process Mag 20(3):49–61. https://doi.org/10.1109/MSP.2003.1203209

    Article  Google Scholar 

  39. Sajjadi MS, Schölkopf B, Hirsch M (2016) Enhancenet: single image super-resolution through automated texture synthesis. arXiv:1612.07919

  40. Sun J, Zheng NN, Tao H, Shum HY (2003) Image hallucination with primal sketch priors. In: 2003 IEEE Computer society conference on computer vision and pattern recognition, vol 2, pp 29–36. https://doi.org/10.1109/CVPR.2003.1211539

  41. Tsai RY, Huang TS (1984) Multiframe image restoration and registration. Adv Comput Vis Image Process 1(2):317–339

    Google Scholar 

  42. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  43. Weber AG (1997) The USC-SIPI image database. http://sipi.usc.edu/database/database.php?volume=textures

  44. Wei LY, Lefebvre S, Kwatra V, Turk G (2009) State of the art in example-based texture synthesis. In: Eurographics 2009, State of the Art Report, EG-STAR

  45. Yang J, Huang T (2010) Image super-resolution: historical overview and future challenges. In: Milanfar P (ed) Super-Resolution imaging, chap. 1. ISBN: 9781439819302. CRC Press, pp 1–24

  46. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873. https://doi.org/10.1109/TIP.2010.2050625

    Article  MathSciNet  MATH  Google Scholar 

  47. Yang S, Wang Z, Zhang L, Wang M (2014) Dual-geometric neighbor embedding for image super resolution with sparse tensor. IEEE Trans Image Process 23 (7):2793–2803. https://doi.org/10.1109/TIP.2014.2319742

    Article  MathSciNet  Google Scholar 

  48. Yoo SB, Choi K, Jeon YW, Ra JB (2016) Texture enhancement for improving single-image super-resolution performance. Signal Process Image Commun 46:29–39. https://doi.org/10.1016/j.image.2016.04.007

    Article  Google Scholar 

  49. Zachevsky I, Zeevi YY (2014) Single-image superresolution of natural stochastic textures based on fractional brownian motion. IEEE Trans Image Process 23 (5):2096–2108. https://doi.org/10.1109/TIP.2014.2312284

    Article  MathSciNet  Google Scholar 

  50. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: Proceedings of the 7th international conference on curves and surfaces, vol 6920, pp 711–730. https://doi.org/10.1007/978-3-642-27413-8_47

  51. Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238. https://doi.org/10.1109/TIP.2006.877407

    Article  Google Scholar 

  52. Zhang K, Gao X, Li X, Tao D (2011) Partially supervised neighbor embedding for example-based image super-resolution. IEEE J Selected Topics Signal Process 5(2):230–239. https://doi.org/10.1109/JSTSP.2010.2048606

    Article  Google Scholar 

  53. Zhang L, Zhang D, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386. https://doi.org/10.1109/TIP.2011.2109730

    Article  MathSciNet  MATH  Google Scholar 

  54. Zhang K, Gao X, Tao D, Li X (2012) Single image super-resolution with non-local means and steering kernel regression. IEEE Trans Image Process 21 (11):4544–4556. https://doi.org/10.1109/TIP.2012.2208977

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This research is partially supported by the following project: Grant No. ETI/359/2014 by Fund for Improvement of S&T Infrastructure in Universities and Higher Educational Institutions (FIST) Program 2016, Department of Science and Technology, Government of India.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sambit Bakshi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mishra, D., Majhi, B., Bakshi, S. et al. Single image super resolution for texture images through neighbor embedding. Multimed Tools Appl 79, 8337–8366 (2020). https://doi.org/10.1007/s11042-017-5367-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5367-5

Keywords

Navigation