Skip to main content
SpringerLink
Log in

Fast multi-scale joint bilateral texture upsampling

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We present a new approach using a multi-scale joint bilateral filter for upsampling the synthesized texture generated by optimization-based methods. Our method is based on the following motivation: if the available exemplar texture is used as a priority to upsample the synthesized texture, a high resolution result that prevents image blurring can be obtained. Our multi-scale joint bilateral upsampling applies a spatial filter on each multi-scale layer of the synthesized texture, and jointly applies a similar range filter on exemplar texture, which guides the interpolation from low to high resolution, by magnifying and combining the upsampled information; the details of the upsampled texture are progressively enhanced, and the image blurring artifacts can be effectively avoided. We offer an accelerated joint bilateral filter, which enables our upsampling method to interactively generate a large texture. In addition, we propose a detail-aware texture optimization approach that incorporates image detail in texture optimization to improve the quality of the synthesized texture, on which the multi-scale joint bilateral filter works to generate a more convincing result. We show results for upsampling image and video textures and compare them to traditional upsampling methods, by this demonstrating that with low computational and memory costs, our method achieves better results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Heeger, D.J., Bergen, J.R.: Pyramid-based texture analysis/syn- thesis. In: ACM SIGGRAPH 95 (August), pp. 229–238 (1995)

  2. Portilla, J., Simoncelli, E.P.: A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vis. 40(1), 49–70 (2000)

    Article  MATH  Google Scholar 

  3. De Bonet, J.S.: Multiresolution sampling procedure for analysis and synthesis of texture images. In: SIGGRAPH 1997, pp. 361–368 (1997)

  4. Efros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: International Conference on Computer Vision 1999, pp. 1033–1038 (1999)

  5. Wei, L.Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: SIGGRAPH 2000, pp. 479–488 (2000)

  6. Praun, E., Finkelstein, A., Hoppe, H.: Lapped textures. In: SIGGRAPH 2000, pp. 465–470 (2000)

  7. Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceedings of SIGGRAPH 2001, Los Angeles, CA, pp. 341–346 (2001)

  8. Liang, L., Liu, C., Xu, Y.Q., Guo, B., Shum, H.Y.: Real-time texture synthesis by patch-based sampling. ACM Trans. Graph. 20(3), 127–150 (2001)

    Article  Google Scholar 

  9. Kwatra, V., Schodl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22(3), 277–286 (2003)

    Article  Google Scholar 

  10. Kwatra, V., Essa, I., Bobick, A., Kwatra, N.: Texture optimization for example-based synthesis. ACM Trans. Graph. 24(3), 795–802 (2005)

    Article  Google Scholar 

  11. Xiao, C., Zheng, W., Miao, Y., Zhao, Y., Peng, Q.: A unified method for appearance and geometry completion of point set surfaces. Vis. Comput. 23(6), 433–443 (2007)

    Article  Google Scholar 

  12. Wexler, Y., Shechtman, E., Irani, M.: Space–time completion of video. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 463–476 (2007)

    Article  Google Scholar 

  13. Kopf, J., Fu, C.W., Cohen-Or, D., Deussen, O., Lischinski, D., Wong, T.T.: Solid texture synthesis from 2d exemplars. In: ACM SIGGRAPH 2007, pp. 21–29 (2007)

  14. Wu, Q., Yu, Y.: Feature matching and deformation for texture synthesis. ACM Trans. Graph. 23(3), 364–367 (2004) (Proc. SIGGRAPH 2004)

    Article  Google Scholar 

  15. Lefebvre, S., Hoppe, H.: Appearance-space texture synthesis. ACM Trans. Graph. 25(3), 541–548 (2006) (Proc. SIGGRAPH 2006)

    Article  Google Scholar 

  16. Zhang, J., Zhou, K., Velho, L., Guo, B., Shum, H.-Y.: Synthesis of progressively-variant textures on arbitrary surfaces. In: ACM SIGGRAPH 2003, pp. 295–302 (2003)

  17. Porikli, F.: Constant time O (1) bilateral filtering. In: CVPR 2008, pp. 1–8 (2008)

  18. Wei, L., Han, J., Zhou, K., Bao, H., Guo, B., Shum, H.-Y.: Inverse texture synthesis. ACM Trans. Graph. 27(3), 52 (2008) (Proc. SIGGRAPH 2008)

    Google Scholar 

  19. Fattal, R.: Image upsampling via imposed edge statistics. ACM Trans. Graph. (TOG) 26(3) (2007)

  20. Su, D., Willis, P.: Image interpolation by pixel-level data-dependent triangulation. Comput. Graph. Forum 23(2), 189–201 (2004)

    Article  Google Scholar 

  21. Aly, H.A., Dubois, E.: Image up-sampling using total-variation regularization with a new observation model. IEEE Trans. Image Process. 14(10), 1647–1659 (2005)

    Article  MathSciNet  Google Scholar 

  22. Thvenaz, P., Blu, T., Unser, M.: Image interpolation and resampling. In: Bankman, I. (ed.) Handbook of Medical Imaging, Processing and Analysis, pp. 393–420. Academic Press, San Diego (2000)

    Chapter  Google Scholar 

  23. Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23(3), 664–672 (2004)

    Article  Google Scholar 

  24. Eisemann, E., Durand, F.: Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. 23(3), 673–678 (2004) (Proc. SIGGRAPH 2004)

    Article  Google Scholar 

  25. Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. ACM Trans. Graph. 26(3), 839–846 (2007)

    Google Scholar 

  26. McLachlan, G.J., Krishnan, T.: The EM Algorithm and Extensions. Wiley, New York (1997)

    MATH  Google Scholar 

  27. Mount, D.M., Arya, S.: Ann: A library for approximate nearest neighbor searching (2006). Available at: http://www.cs.umd.edu/~mount/ANN/

  28. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Sixth International Conference on Computer Vision, 1998, pp. 839–846 (1998)

  29. Fattal, R., Agrawala, M., Rusinkiewicz, S.: Multiscale shape and detail enhancement from multi-light image collections. ACM Trans. Graph. 26(3) (2007)

  30. Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph. 257–266 (2002) (Proc. SIGGRAPH 2002)

  31. Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. 26(3) (2007)

  32. Farbman, Z., Fattal, R., Lischinski, D., Szeliski, R.: Edge-preserving decompositions for multi-scale tone and detail manipulation. In: SIGGRAPH 2008, ACM, New York (2008)

Download references

Author information

Authors and Affiliations

  1. School of Computer, Wuhan University, Wuhan, China

    Chunxia Xiao & Yongwei Nie

  2. State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China

    Wei Hua & Wenting Zheng

Authors
  1. Chunxia Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongwei Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenting Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunxia Xiao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xiao, C., Nie, Y., Hua, W. et al. Fast multi-scale joint bilateral texture upsampling. Vis Comput 26, 263–275 (2010). https://doi.org/10.1007/s00371-009-0409-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-009-0409-2

Search

Navigation