Skip to main content
SpringerLink
Log in

Artistic Style Transfer for Videos

  • Conference paper
  • First Online:
Pattern Recognition (GCPR 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9796))

Included in the following conference series:

Abstract

In the past, manually re-drawing an image in a certain artistic style required a professional artist and a long time. Doing this for a video sequence single-handed was beyond imagination. Nowadays computers provide new possibilities. We present an approach that transfers the style from one image (for example, a painting) to a whole video sequence. We make use of recent advances in style transfer in still images and propose new initializations and loss functions applicable to videos. This allows us to generate consistent and stable stylized video sequences, even in cases with large motion and strong occlusion. We show that the proposed method clearly outperforms simpler baselines both qualitatively and quantitatively.

This work was supported by the ERC Starting Grant VideoLearn.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    GitHub: https://github.com/manuelruder/artistic-videos.

  2. 2.

    GitHub: https://github.com/jcjohnson/neural-style.

References

  1. Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012)

    Google Scholar 

  2. Collobert, R., Kavukcuoglu, K., Farabet, C.: Torch7: A matlab-like environment for machine learning. In: BigLearn, NIPS Workshop (2011)

    Google Scholar 

  3. Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style (2015). CoRR abs/1508.06576. http://arxiv.org/abs/1508.06576

  4. Hays, J., Essa, I.: Image and video based painterly animation. In:Proceedings of the 3rd International Symposium on Non-photorealistic Animation and Rendering, NPAR 2004, pp. 113–120. ACM, New York, NY, USA (2004). http://doi.acm.org/10.1145/987657.987676

  5. Li, C., Wand, M.: Combining Markov random fields and convolutional neural networks for image synthesis (2016). CoRR abs/1601.04589. http://arxiv.org/abs/1601.04589

  6. Litwinowicz, P.: Processing images and video for an impressionist effect. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1997, pp. 407–414. ACMPress/Addison-Wesley Publishing Co., New York, NY, USA (1997). http://dx.doi.org/10.1145/258734.258893

  7. Nikulin, Y., Novak, R.: Exploring the neural algorithm of artisticstyle (2016). CoRR abs/1602.07188. http://arxiv.org/abs/1602.07188

  8. O’Donovan, P., Hertzmann, A.: Anipaint: interactive painterly animation from video. IEEE Trans. Vis. Comput. Graph. 18(3), 475–487 (2012)

    Article  Google Scholar 

  9. Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: EpicFlow: edge-preserving interpolation of correspondences for optical flow. In: CVPR2015 - IEEE Conference on Computer Vision & Pattern Recognition, Boston, United States, June 2015. https://hal.inria.fr/hal-01142656

  10. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). CoRR abs/1409.1556. http://arxiv.org/abs/1409.1556

  11. Sundaram, N., Brox, T., Keutzer, K.: Dense point trajectories by GPU-accelerated large displacement optical flow, September 2010. http://lmb.informatik.uni-freiburg.de//Publications/2010/Bro10e

  12. Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: DeepFlow: Large displacement optical flow with deep matching. In: ICCV 2013 - IEEE International Conference on Computer Vision, pp. 1385–1392. IEEE, Sydney, Australia, December 2013. https://hal.inria.fr/hal-00873592

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Freiburg, Freiburg im Breisgau, Germany

    Manuel Ruder, Alexey Dosovitskiy & Thomas Brox

Authors
  1. Manuel Ruder
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexey Dosovitskiy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Brox
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manuel Ruder .

Editor information

Editors and Affiliations

  1. University of Hannover, Hannover, Germany

    Bodo Rosenhahn

  2. Max Planck Institute for Informatics, Saarbrücken, Germany

    Bjoern Andres

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 5190 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Ruder, M., Dosovitskiy, A., Brox, T. (2016). Artistic Style Transfer for Videos. In: Rosenhahn, B., Andres, B. (eds) Pattern Recognition. GCPR 2016. Lecture Notes in Computer Science(), vol 9796. Springer, Cham. https://doi.org/10.1007/978-3-319-45886-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-45886-1_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-45885-4

  • Online ISBN: 978-3-319-45886-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation