Gilad Freedman
Abstract
We propose a new high-quality and efficient single-image upscaling technique that extends existing example-based super-resolution frameworks. In our approach we do not rely on an external example database or use the whole input image as a source for example patches. Instead, we follow a local self-similarity assumption on natural images and extract patches from extremely localized regions in the input image. This allows us to reduce considerably the nearest-patch search time without compromising quality in most images. Tests, that we perform and report, show that the local self-similarity assumption holds better for small scaling factors where there are more example patches of greater relevance. We implement these small scalings using dedicated novel nondyadic filter banks, that we derive based on principles that model the upscaling process. Moreover, the new filters are nearly biorthogonal and hence produce high-resolution images that are highly consistent with the input image without solving implicit back-projection equations. The local and explicit nature of our algorithm makes it simple, efficient, and allows a trivial parallel implementation on a GPU. We demonstrate the new method ability to produce high-quality resolution enhancement, its application to video sequences with no algorithmic modification, and its efficiency to perform real-time enhancement of low-resolution video standard into recent high-definition formats.
- Aly, H. and Dubois, E. 2005. Image up-sampling using total-variation regularization with a new observation model. IEEE Trans. Image Process. 14, 10, 1647--1659. Google Scholar
Digital Library
- Barnsley, M. 1988. Fractal modelling of real world images. In The Science of Fractal Images, H.-O. Peitgen and D. Saupe, Eds. Springer, Berlin. Google ScholarDigital Library
- Bhat, P., Zitnick, C. L., Snavely, N., Agarwala, A., Agrawala, M., Curless, B., Cohen, M., and Kang, S. B. 2007. Using photographs to enhance videos of a static scene. In Proceedings of the Conference on Rendering Techniques, J. Kautz and S. Pattanaik, Eds. Eurographics, 327--338. Google ScholarDigital Library
- Ebrahimi, M. and Vrscay, E. R. 2007. Solving the inverse problem of image zooming using self-examples. In Proceedings of the International Conference on Image Analysis and Recognition ICIAR. Lecture Notes in Computer Science, vol. 4633. Springer, 117--130. Google ScholarDigital Library
- Farsiu, S., Robinson, M., Elad, M., and Milanfar, P. 2004. Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13, 10, 1327--1344. Google ScholarDigital Library
- Fattal, R. 2007. Image upsampling via imposed edge statistics. ACM Trans. Graph. 26, 3, 95. Google ScholarDigital Library
- Freeman, W. T., Jones, T. R., and Pasztor, E. C. 2002. Example-based super-resolution. IEEE Comput. Graph. Appl. 22, 2, 56--65. Google ScholarDigital Library
- Freeman, W. T., Pasztor, E. C., and Carmichael, O. T. 2000. Learning low-level vision. Int. J. Comput. Vision 40, 1, 25--47. Google ScholarDigital Library
- Glasner, D., Bagon, S., and Irani, M. 2009. Super-Resolution from a single image. In Proceedings of the IEEE International Conference on Computer Vision (ICCV09). 349--356.Google Scholar
- Li, X. and Orchard, M. T. 2001. New edge-directed interpolation. IEEE Trans. Image Process. 10, 10, 1521--1527. Google ScholarDigital Library
- Lin, Z. and Shum, H.-Y. 2004. Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans. Patt. Anal. Mach. Intell. 26, 1, 83--97. Google ScholarDigital Library
- Mallat, S. 1999. A Wavelet Tour of Signal Processing, 2nd ed. (Wavelet Analysis & Its Applications). Academic Press.Google Scholar
- Polidori, E. and Dugelay, J.-L. 1995. Zooming using iterated function systems. In NATO ASI on Image Coding and Analysis.Google Scholar
- Pollock, S. and Cascio, I. 2007. Non-Dyadic wavelet analysis. In Optimisation, Econometric and Financial Analysis, 167--203.Google Scholar
- Pratt, W. K. 2001. Digital Image Processing: PIKS Inside. John Wiley & Sons, New York. Google ScholarDigital Library
- Reusens, E. 1994. Overlapped adaptive partitioning for image coding based on the theory of iterated functions systems. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. (ICASSP 94). vol. 5.Google ScholarCross Ref
- Robert, M. G.-A., Denardo, R., Tenda, Y., and Huang, T. S. 1997. Resolution enhancement of images using fractal coding. In Proceedings of the Conference on Visual Communications and Image Processing '97. 1089--1100.Google Scholar
- Shan, Q., Li, Z., Jia, J., and Tang, C.-K. 2008. Fast image/video upsampling. ACM Trans. Graph. 27, 5, 1--7. Google ScholarDigital Library
- Su, D. and Willis, P. 2004. Image interpolation by pixel-level data-dependent triangulation. Comput. Graph. Forum 23, 2, 189--202.Google ScholarCross Ref
- Suetake, N., Sakano, M., and Uchino, E. 2008. Image super-resolution based on local self-similarity. J. Optic. Rev. 15, 1 (January), 26--30.Google Scholar
- Sun, J., Ning Zheng, N., Tao, H., and Yeung Shum, H. 2003. Image hallucination with primal sketch priors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 729--736.Google Scholar
- Sun, J., Xu, Z., and Shum, H.-Y. 2008. Image super-resolution using gradient profile prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--8.Google Scholar
- Tappen, M. F., Russell, B. C., and Freeman, W. T. 2004. Efficient graphical models for processing images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 673--680. Google ScholarDigital Library
- Thvenaz, P., Blu, T., and Unser, M. 2000. Image interpolation and resampling. In Handbook of Medical Imaging, Processing and Analysis, I. Bankman, Ed. Academic Press, San Diego CA, 393--420. Google ScholarDigital Library
- Vrscay, E. R. 2002. From Fractal Image Compression to Fractal-Based Methods in Mathematics. The IMA Volumes in Mathematics and Its Applications. Springer, Berlin.Google Scholar
- Xiong, R., Xu, J., and Wu, F. 2006. A lifting-based wavelet transform supporting non-dyadic spatial scalability. In Proceedings of the IEEE International Conference on Image Processing. 1861--1864.Google Scholar
Index Terms
- Image and video upscaling from local self-examples
Recommendations
Super-resolution from a single image based on local self-similarity
Super-resolution from a single image plays an important role in many areas. However, it is still a challenging work, especially in the high-resolution image's quality and the algorithm's efficiency. To obtain high-resolution images, a new single image ...
Super Resolution from a Single Image Based on Self Similarity
ICCIS '11: Proceedings of the 2011 International Conference on Computational and Information SciencesA single image super resolution method based on self-similarity is proposed. It has been proposed that self-similarity occurs in the same image scales as the source image as well as across different image scales. We exploit this novel innovation to ...
Deep Residual Attention Network for Spectral Image Super-Resolution
Computer Vision – ECCV 2018 WorkshopsAbstractSpectral imaging sensors often suffer from low spatial resolution, as there exists an essential tradeoff between the spectral and spatial resolutions that can be simultaneously achieved, especially when the temporal resolution needs to be ...
Reviews
Fernando Santos Osorio
An interesting algorithm for increasing the resolution and size of digital images is presented in this work. The proposed method preserves the image quality and allows one to upscale it, obtaining high-resolution and high-quality images. This method is very useful because efficient upscaling of single images acquired by low-grade sensors to high-quality, high-resolution images is a difficult problem. Most natural images contain both local regularities and discontinuities that can produce undesirable artifacts when scaling up the images. Freedman and Fattal state that their method is better than some other recent well-known algorithms proposed in the literature because it produces more visually appealing images. Another important advantage is that it can be executed in real time, over both images and video sequences, using a parallel implementation on a graphics processing unit (GPU). The authors propose using only local self-examples (extremely localized region samples) of the input image to create filter banks with a gradual upscaling process (small consecutive scaling factor adjustments). The main advantage of this method is that one can apply it in real time, since it uses only the original image as input. The fact that it is based on local self-examples allows one to obtain very efficient implementations in a GPU parallel version. Additionally, according to the authors, the quality of the obtained images is comparable to those obtained using other state-of-the-art algorithms, with the advantage that it outperforms the others. The tests that the authors have conducted demonstrate that the final images usually do not suffer from the side effects of upscaling (such as artifacts, staircasing, or blurring). Interested readers can view these images on the authors' project Web page [1]. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments