Samet Hicsonmez
ABSTRACT
This paper is motivated from a young boy's capability to recognize an illustrator's style in a totally different context. In the book "We are All Born Free" [1], composed of selected rights from the Universal Declaration of Human Rights interpreted by different illustrators, the boy was surprised to see a picture similar to the ones in the "Winnie the Witch" series drawn by Korky Paul (Figure [1]). The style was noticeable in other characters of the same illustrator in different books as well. The capability of a child to easily spot the style was shown to be valid for other illustrators such as Axel Scheffler and Debi Gliori. The boy's enthusiasm let us to start the journey to explore the capabilities of machines to recognize the style of illustrators.
We collected pages from children's books to construct a new illustrations dataset consisting of about 6500 pages from 24 artists. We exploited deep networks for categorizing illustrators and with around 94% classification performance our method over-performed the traditional methods by more than 10%. Going beyond categorization we explored transferring style. The classification performance on the transferred images has shown the ability of our system to capture the style. Furthermore, we discovered representative illustrations and discriminative stylistic elements.
- 2008. We Are All Born Free: The Universal Declaration of Human Rights in Pictures. Frances Lincoln. (2008). http://www.goodreads.com/book/show/ 3082451-we-are-all-born-freeGoogle Scholar
- 2012. BBC Your Paintings. (2012). Dataset available at http://www.bbc.co.uk/arts/.Google Scholar
- 2014. Caffe Model Zoo: GoogLeNet Model. (2014). Model available at https: //github.com/BVLC/caffe/tree/master/models/bvlc googlenet.Google Scholar
- Yaniv Bar, Noga Levy, and Lior Wolf. 2014. Classification of artistic styles using binarized features derived from a deep neural network. In Workshop at the European Conference on Computer Vision. Springer, 71--84.Google Scholar
- Hongping Cai, Qi Wu, Tadeo Corradi, and Peter Hall. 2015. The Cross-Depiction Problem: Computer Vision Algorithms for Recognising Objects in Artwork and in Photographs. CoRR abs/1505.00110 (2015). http://arxiv.org/abs/1505.00110Google Scholar
- Allan Campbell, Vic Ciesielksi, and AK Qin. 2015. Feature discovery by deep learn- ing for aesthetic analysis of evolved abstract images. In International Conference on Evolutionary and Biologically Inspired Music and Art. Springer International Publishing, 27--38.Google Scholar
- Gustavo Carneiro, Nuno Pinho da Silva, Alessio Del Bue, and João Paulo Costeira. 2012. Artistic image classification: An analysis on the printart database. In European Conference on Computer Vision. Springer, 143--157. Google ScholarDigital Library
- Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2 (2011), 27:1--27:27. Issue 3. Software available at http://www.csie.ntu.edu.tw/?cjlin/libsvm. Google ScholarDigital Library
- Wei-Ta Chu and Yi-Ling Wu. 2016. Deep Correlation Features for Image Style Classification. In Proceedings of the 2016 ACM on Multimedia Conference (MM '16). 402--406. Google ScholarDigital Library
- E. J. Crowley and A. Zisserman. 2014. In Search of Art. In Workshop on Computer Vision for Art Analysis, ECCV.Google Scholar
- E. J. Crowley and A. Zisserman. 2014. the State of the Art: Object Retrieval in Paintings using Discriminative Regions. In British Machine Vision Conference.Google Scholar
- Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Vol. 1. IEEE, 886--893. Google ScholarDigital Library
- Paul DiMaggio. 1987. Classification in art. American sociological review (1987), 440--455.Google Scholar
- Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei A. Efros. 2012. What Makes Paris Look like Paris? ACM Transactions on Graphics (SIGGRAPH) 31, 4 (2012), 101:1--101:9. Google ScholarDigital Library
- Vincent Dumoulin, Jonathon Shlens, and Manjunath Kudlur. 2017. A Learned Representation For Artistic Style. ICLR (2017). https://arxiv.org/abs/1610.07629Google Scholar
- Ahmed Elgammal and Babak Saleh. 2015. Quantifying Creativity in Art Networks. arXiv preprint arXiv:1506.00711 (2015).Google Scholar
- M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2011. the PASCAL Visual Object Classes Challenge 2011 (VOC2011) Results. (2011). http://www.pascal-network.org/challenges/VOC/voc2011/workshop/index.htmlGoogle Scholar
- T. Furuya, S. Kuriyama, and R. Ohbuchi. 2015. An unsupervised approach for comparing styles of illustrations. In 2015 13th International Workshop on Content- Based Multimedia Indexing (CBMI). 1--6.Google Scholar
- Elena Garces, Aseem Agarwala, Diego Gutierrez, and Aaron Hertzmann. 2014. A Similarity Measure for Illustration Style. ACM Trans. Graph. 33, 4, Article 93 (July 2014), 9 pages. Google ScholarDigital Library
- Elena Garces, Aseem Agarwala, Aaron Hertzmann, and Diego Gutierrez. 2016. Style-based exploration of illustration datasets. Multimedia Tools and Applications (2016), 1--20. Google ScholarDigital Library
- Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2015. A Neural Algorithm of Artistic Style. CoRR abs/1508.06576 (2015). http://arxiv.org/abs/1508. 06576Google Scholar
- Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2016. Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2414--2423.Google ScholarCross Ref
- Eren Golge and Pinar Duygulu-Sahin. 2015. FAME: face association through model evolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 43--49.Google ScholarCross Ref
- Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolu- tional Architecture for Fast Feature Embedding. arXiv preprint arXiv:1408.5093 (2014).Google Scholar
- C Richard Johnson, Ella Hendriks, Igor Berezhnoy, Eugene Brevdo, Shannon Hughes, Ingrid Daubechies, Jia Li, Eric Postma, and James Z Wang. 2008. Image Processing for Artist Identification - Computerized Analysis of Vincent van Gogh's Painting Brushstrokes. IEEE Signal Processing Magazine (July 2008).Google Scholar
- Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. arXiv preprint arXiv:1603.08155 (2016).Google Scholar
- Sergey Karayev, Matthew Trentacoste, Helen Han, Aseem Agarwala, Trevor Darrell, Aaron Hertzmann, and Holger Winnemoeller. 2013. Recognizing image style. arXiv preprint arXiv:1311.3715 (2013).Google Scholar
- Fahad Shahbaz Khan, Shida Beigpour, Joost Van de Weijer, and Michael Felsberg. 2014. Painting-91: a large scale database for computational painting categorization. Machine vision and applications 25, 6 (2014), 1385--1397. Google ScholarDigital Library
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classifica- tion with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105. Google ScholarDigital Library
- Jan Eric Kyprianidis, John Collomosse, Tinghuai Wang, and Tobias Isenberg. 2013. State of the Art x201D;: A Taxonomy of Artistic Stylization Techniques for Images and Video. IEEE Transactions on Visualization and Computer Graphics 19, 5 (2013), 866--885. Google ScholarDigital Library
- C. Li and T. Chen. 2009. Aesthetic Visual Quality Assessment of Paintings. IEEE Journal of Selected Topics in Signal Processing 3, 2 (April 2009), 236--252.Google ScholarCross Ref
- Jia Li and J. Z. Wang. 2004. Studying Digital Imagery of Ancient Paintings by Mixtures of Stochastic Models. Trans. Img. Proc. 13, 3 (March 2004), 340--353. Google ScholarDigital Library
- David G Lowe. 2004. Distinctive image features from scale-invariant keypoints. International journal of computer vision 60, 2 (2004), 91--110. Google ScholarDigital Library
- Siwei Lyu, Daniel Rockmore, and Hany Farid. 2004. A digital technique for art authentication. Proceedings of the National Academy of Sciences of the United States of America 101, 49 (2004), 17006--17010.Google ScholarCross Ref
- Shin Matsuo and Keiji Yanai. 2016. CNN-based style vector for style image retrieval. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. ACM, 309--312. Google ScholarDigital Library
- Thomas Mensink and Jan Van Gemert. 2014. The rijksmuseum challenge: Museum-centered visual recognition. In Proceedings of International Conference on Multimedia Retrieval. ACM, 451. Google ScholarDigital Library
- Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexan- der C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) 115, 3 (2015), 211--252. Google ScholarDigital Library
- Babak Saleh, Kanako Abe, Ravneet Singh Arora, and Ahmed Elgammal. 2014. To- ward automated discovery of artistic influence. Multimedia Tools and Applications (2014), 1--27. Google ScholarDigital Library
- Babak Saleh and Ahmed Elgammal. 2015. Large-scale Classification of Fine-Art Paintings: Learning the Right Metric on thee Right Feature. arXiv preprint arXiv:1505.00855 (2015).Google Scholar
- Fadime Sener, Nermin Samet, and Pinar Duygulu Sahin. 2012. Identification of illustrators. In European Conference on Computer Vision. Springer Berlin Heidelberg, 589--597. Google ScholarDigital Library
- K. Simonyan and A. Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs/1409.1556 (2014).Google Scholar
- Josef Sivic, Bryan C Russell, Alexei A Efros, Andrew Zisserman, and William T Freeman. 2005. Discovering object categories in image collections. (2005).Google Scholar
- Emily L. Spratt and Ahmed M. Elgammal. 2014. Computational Beauty: Aesthetic Judgment at the Intersection of Art and Science. In Computer Vision - ECCV 2014 Workshops - Zurich, Switzerland, September 6-7 and 12, 2014, Proceedings, Part I. 35--53.Google Scholar
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1--9.Google ScholarCross Ref
- W. R. Tan, C. S. Chan, H. E. Aguirre, and K. Tanaka. 2016. Ceci n'est pas une pipe: A deep convolutional network for fine-art paintings classification. In 2016 IEEE International Conference on Image Processing (ICIP). 3703--3707.Google Scholar
- Christopher Thomas and Adriana Kovashka. 2015. Who's Behind the Camera? Identifying the Authorship of a Photograph. arXiv preprint arXiv:1508.05038 (2015).Google Scholar
- Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. 2015. Understanding Neural Networks trough Deep Visualization. In Deep Learning Workshop, International Conference on Machine Learning (ICML).Google Scholar
Index Terms
- DRAW: Deep Networks for Recognizing Styles of Artists Who Illustrate Children's Books
Recommendations
Understanding and Creating Art with AI: Review and Outlook
Technologies related to artificial intelligence (AI) have a strong impact on the changes of research and creative practices in visual arts. The growing number of research initiatives and creative applications that emerge in the intersection of AI and art ...
Beauty Is in the Eye of the Beholder: Demographically Oriented Analysis of Aesthetics in Photographs
Special Section on Cross-Media Analysis for Visual Question Answering, Special Section on Big Data, Machine Learning and AI Technologies for Art and Design and Special Section on MMSys/NOSSDAV 2018Aesthetics is a subjective concept that is likely to be perceived differently among people of different ages, genders, and cultural backgrounds. While techniques that directly compute this concept in images has seen increasing attention by the multimedia ...
Comments