Abstract
This article proposes an improved learning based super resolution scheme using manifold learning for texture images. Pseudo Zernike moment (PZM) has been employed to extract features from the texture images. In order to efficiently retrieve similar patches from the training patches, feature similarity index matrix (FSIM) has been used. Subsequently, for reconstruction of the high resolution (HR) patch, a collaborative optimal weight is generated from the least square (LS) and non-negative matrix factorization (NMF) methods. The proposed method is tested on some color texture, gray texture, and some standard images. Results of the proposed method on texture images advocate its superior performance over established state-of-the-art methods.
Similar content being viewed by others
References
Ahn IJ, Nam WH (2016) Texture enhancement via high-resolution style transfer for single-image super-resolution. arXiv:1612.00085
Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183. https://doi.org/10.1109/TPAMI.2002.1033210
Begin I, Ferrie FP (2004) Blind super-resolution using a learning-based approach. In: 2004 International conference on pattern recognition, pp 85–89. https://doi.org/10.1109/ICPR.2004.1334046
Bevilacqua M, Roumy A, Guillemot C, Morel MLA (2013) Super-resolution using neighbor embedding of back-projection residuals. In: 18th International conference on digital signal processing, pp 1–8. https://doi.org/10.1109/ICDSP.2013.6622796
Cao M, Gan Z, Zhu X (2012) Super-resolution algorithm through neighbor embedding with new feature selection and example training. In: 2012 IEEE International conference on signal processing, vol 2, pp 825–828. https://doi.org/10.1109/ICoSP.2012.6491708
Chan TM, Zhang J (2006) An improved super-resolution with manifold learning and histogram matching. In: 2006 International conference on advances in biometrics, pp 756–762. https://doi.org/10.1007/11608288_101
Chan TM, Zhang J, Pu J, Huang H (2009) Neighbor embedding based super-resolution algorithm through edge detection and feature selection. Pattern Recogn Lett 30(5):494–502. https://doi.org/10.1016/j.patrec.2008.11.008
Chang X, Yang Y (2017) Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Transactions on Neural Networks and Learning Systems 28(10):2294–2305. https://doi.org/10.1109/TNNLS.2016.2582746
Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: 2004 IEEE Computer society conference on computer vision and pattern recognition, vol 1, pp 275–28. https://doi.org/10.1109/CVPR.2004.1315043
Chang X, Ma Z, Lin M, Yang Y, Hauptmann A (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 26 (8):3911–3920. https://doi.org/10.1109/TIP.2017.2708506
Chang X, Ma Z, Yang Y, Zeng Z, Hauptmann A G (2017) Bi-level semantic representation analysis for multimedia event detection. IEEE Trans Cybern 47(5):1180–1197. https://doi.org/10.1109/TCYB.2016.2539546
Chang X, Yu YL, Yang Y, Xing EP (2017) Semantic pooling for complex event analysis in untrimmed videos. IEEE Trans Pattern Anal Mach Intell 39 (8):1617–1632. https://doi.org/10.1109/TPAMI.2016.2608901
Chen X, Qi C (2014) Nonlinear neighbor embedding for single image super-resolution via kernel mapping. Signal Process 94:6–22. https://doi.org/10.1016/j.sigpro.2013.06.016
Chong CW, Raveendran P, Mukundan R (2003) The scale invariants of pseudo-Zernike moments. Pattern Anal Appl 6(3):176–184. https://doi.org/10.1007/s10044-002-0183-5
Cruz C, Mehta R, Katkovnik V, Egiazarian K (2017) Single image super-resolution based on Wiener filter in similarity domain. arXiv:1704.04126
Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630. https://doi.org/10.1109/TIP.2012.2235847
Fan W, Yeung DY (2007) Image hallucination using neighbor embedding over visual primitive manifolds. In: 2007 IEEE Conference on computer vision and pattern recognition, pp 1–7. https://doi.org/10.1109/CVPR.2007.383001
Farsiu S, Robinson D, Elad M, Milanfar P (2004) Advances and challenges in super-resolution. Int J Imaging Syst Technol 14(2):47–57. https://doi.org/10.1002/ima.20007
Freeman WT, Pasztor EC, Carmichael OT (2000) Learning low-level vision. Int J Comput Vis 40(1):25–47. https://doi.org/10.1023/A:1026501619075
Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65. https://doi.org/10.1109/38.988747
Gao X, Wang Q, Li X, Tao D, Zhang K (2011) Zernike-moment-based image super resolution. IEEE Trans Image Process 20 (10):2738–2747. https://doi.org/10.1109/TIP.2011.2134859
Gao X, Zhang K, Tao D, Li X (2012) Joint learning for single-image super-resolution via a coupled constraint. IEEE Trans Image Process 21(2):469–480. https://doi.org/10.1109/TIP.2011.2161482
Gerchberg R (1974) Super-resolution through error energy reduction. Optica Acta: Int J Opt 21(9):709–720. https://doi.org/10.1080/713818946
Guo K, Yang X, Lin W, Zhang R, Yu S (2012) Learning-based super-resolution method with a combining of both global and local constraints. IET Image Process 6(4):337–344. 10.1049/iet-ipr.2010.0430
HaCohen Y, Fattal R, Lischinski D (2010) Image upsampling via texture hallucination. In: 2010 IEEE International conference on computational photography, pp 1–8. https://doi.org/10.1109/ICCPHOT.2010.5585097
Jiang J, Ma X, Cai Z, Hu R (2015) Sparse support regression for image super-resolution. IEEE Photon J 7(5):1–11. https://doi.org/10.1109/JPHOT.2015.2484287
Kanan HR, Salkhordeh S (2016) Rotation invariant multi-frame image super resolution reconstruction using pseudo Zernike moments. Signal Process 118:103–114. https://doi.org/10.1016/j.sigpro.2015.05.015
Kim SP, Bose NK, Valenzuela HM (1990) Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Trans Acoust Speech Signal Process 38(6):1013–1027. https://doi.org/10.1109/29.56062
Li X, Orchard MT (2001) New edge-directed interpolation. IEEE Trans Image Process 10(10):1521–1527. https://doi.org/10.1109/83.951537
Lin Z, Shum HY (2004) Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans Pattern Anal Mach Intell 26(1):83–97. https://doi.org/10.1109/TPAMI.2004.1261081
Liu L, Huang W, Wang C (2017) Texture image prior for SAR image super resolution based on total variation regularization using split bregman iteration. Int J Remote Sens 38(20):5673–5687. https://doi.org/10.1080/01431161.2017.1346325
Mishra D, Majhi B, Sa PK (2014) Neighbor embedding based super-resolution using residual luminance. In: 2014 Annual IEEE India conference (INDICON), pp 1–6. https://doi.org/10.1109/INDICON.2014.7030659
Mishra D, Majhi B, Sa PK, Dash R (2016) Development of robust neighbor embedding based super-resolution scheme. Neurocomputing 202:49–66. https://doi.org/10.1016/j.neucom.2016.04.013
Mishra D, Majhi B, Sa PK (2017) Improved feature selection for neighbor embedding super-resolution using Zernike moments. In: Proceedings of international conference on computer vision and image processing, pp 13–24. https://doi.org/10.1007/978-981-10-2107-7_2
Nasrollahi K, Moeslund TB (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468. https://doi.org/10.1007/s00138-014-0623-4
Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36. https://doi.org/10.1109/MSP.2003.1203207
Patanavijit V (2009) Super-resolution reconstruction and its future research direction. AU J Technol 12(3):149–163
Rajan D, Chaudhuri S, Joshi MV (2003) Multi-objective super resolution: concepts and examples. IEEE Signal Process Mag 20(3):49–61. https://doi.org/10.1109/MSP.2003.1203209
Sajjadi MS, Schölkopf B, Hirsch M (2016) Enhancenet: single image super-resolution through automated texture synthesis. arXiv:1612.07919
Sun J, Zheng NN, Tao H, Shum HY (2003) Image hallucination with primal sketch priors. In: 2003 IEEE Computer society conference on computer vision and pattern recognition, vol 2, pp 29–36. https://doi.org/10.1109/CVPR.2003.1211539
Tsai RY, Huang TS (1984) Multiframe image restoration and registration. Adv Comput Vis Image Process 1(2):317–339
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861
Weber AG (1997) The USC-SIPI image database. http://sipi.usc.edu/database/database.php?volume=textures
Wei LY, Lefebvre S, Kwatra V, Turk G (2009) State of the art in example-based texture synthesis. In: Eurographics 2009, State of the Art Report, EG-STAR
Yang J, Huang T (2010) Image super-resolution: historical overview and future challenges. In: Milanfar P (ed) Super-Resolution imaging, chap. 1. ISBN: 9781439819302. CRC Press, pp 1–24
Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873. https://doi.org/10.1109/TIP.2010.2050625
Yang S, Wang Z, Zhang L, Wang M (2014) Dual-geometric neighbor embedding for image super resolution with sparse tensor. IEEE Trans Image Process 23 (7):2793–2803. https://doi.org/10.1109/TIP.2014.2319742
Yoo SB, Choi K, Jeon YW, Ra JB (2016) Texture enhancement for improving single-image super-resolution performance. Signal Process Image Commun 46:29–39. https://doi.org/10.1016/j.image.2016.04.007
Zachevsky I, Zeevi YY (2014) Single-image superresolution of natural stochastic textures based on fractional brownian motion. IEEE Trans Image Process 23 (5):2096–2108. https://doi.org/10.1109/TIP.2014.2312284
Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: Proceedings of the 7th international conference on curves and surfaces, vol 6920, pp 711–730. https://doi.org/10.1007/978-3-642-27413-8_47
Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238. https://doi.org/10.1109/TIP.2006.877407
Zhang K, Gao X, Li X, Tao D (2011) Partially supervised neighbor embedding for example-based image super-resolution. IEEE J Selected Topics Signal Process 5(2):230–239. https://doi.org/10.1109/JSTSP.2010.2048606
Zhang L, Zhang D, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386. https://doi.org/10.1109/TIP.2011.2109730
Zhang K, Gao X, Tao D, Li X (2012) Single image super-resolution with non-local means and steering kernel regression. IEEE Trans Image Process 21 (11):4544–4556. https://doi.org/10.1109/TIP.2012.2208977
Acknowledgments
This research is partially supported by the following project: Grant No. ETI/359/2014 by Fund for Improvement of S&T Infrastructure in Universities and Higher Educational Institutions (FIST) Program 2016, Department of Science and Technology, Government of India.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mishra, D., Majhi, B., Bakshi, S. et al. Single image super resolution for texture images through neighbor embedding. Multimed Tools Appl 79, 8337–8366 (2020). https://doi.org/10.1007/s11042-017-5367-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-5367-5