Abstract
Relative attributes indicate the strength of a particular attribute between image pairs. We introduce a deep Siamese network with rank SVM loss function, called Deep-RankSVM, that can decide which one of a pair of images has a stronger presence of a specific attribute. The network is trained in an end-to-end fashion to jointly learn the visual features and the ranking function. The trained network for an attribute can predict the relative strength of that attribute in novel images.
We demonstrate the effectiveness of our approach against the state-of-the-art methods on four image benchmark datasets: LFW-10, PubFig, UTZap50K-2 and UTZap50K-lexi datasets. Deep-RankSVM surpasses state-of-art in terms of the average accuracy across attributes, on three of the four image benchmark datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmed, S.A.A., Yanikoglu, B.: Within-network ensemble for face attributes classification. In: Ricci, E., Rota Bulò, S., Snoek, C., Lanz, O., Messelodi, S., Sebe, N. (eds.) ICIAP 2019. LNCS, vol. 11751, pp. 466–476. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30642-7_42
Bansal, A., Sikka, K., Sharma, G., Chellappa, R., Divakaran, A.: Zero-shot object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 384–400 (2018)
Chen, L., Zhang, P., Li, B.: Instructive video retrieval based on hybrid ranking and attribute learning: a case study on surgical skill training. In: Proceedings of the 22nd ACM, pp. 1045–1048 (2014)
Fu, Y., Xiang, T., Jiang, Y.G., Xue, X., Sigal, L., Gong, S.: Recent advances in zero-shot recognition: toward data-efficient understanding of visual content. IEEE Sig. Process. Mag. 35(1), 112–125 (2018)
Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07–49, University of Massachusetts, Amherst, October 2007
Kovashka, A., Grauman, K.: Attributes for image retrieval. Visual Attributes. ACVPR, pp. 89–117. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-50077-5_5
Kovashka, A., Parikh, D., Grauman, K.: WhittleSearch: interactive image search with relative attribute feedback. Int. J. Comput. Vis. 115(2), 185–210 (2015). https://doi.org/10.1007/s11263-015-0814-0
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012)
Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 365–372. IEEE (2009)
Li, S., Shan, S., Chen, X.: Relative forest for attribute prediction. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7724, pp. 316–327. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37331-2_24
Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001). https://doi.org/10.1023/A:1011139631724
Pan, Y., Yao, T., Li, H., Mei, T.: Video captioning with transferred semantic attributes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6504–6512 (2017)
Parikh, D., Grauman, K.: Relative attributes. In: International Conference on Computer Vision, pp. 503–510. IEEE (2011)
Ruff, L., et al.: Deep one-class classification. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 4393–4402. PMLR, Stockholmsmässan, Stockholm, 10–15 July 2018
Sandeep, R.N., Verma, Y., Jawahar, C.: Relative parts: distinctive parts for learning relative attributes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3614–3621 (2014)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Singh, K.K., Lee, Y.J.: End-to-end localization and ranking for relative attributes. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 753–769. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_45
Souri, Y., Noury, E., Adeli, E.: Deep relative attributes. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10115, pp. 118–133. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54193-8_8
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)
Xiao, F., Lee, Y.J.: Discovering the spatial extent of relative attributes. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1458–1466 (2015)
Yang, X., Zhang, T., Xu, C., Yan, S., Hossain, M.S., Ghoneim, A.: Deep relative attributes. IEEE Trans. Multimed. 18(9), 1832–1842 (2016)
Yao, T., Pan, Y., Li, Y., Qiu, Z., Mei, T.: Boosting image captioning with attributes. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4894–4902 (2017)
Yu, A., Grauman, K.: Fine-grained visual comparisons with local learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 192–199 (2014)
Yu, A., Grauman, K.: Just noticeable differences in visual attributes. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2416–2424 (2015)
Yu, A., Grauman, K.: Semantic jitter: dense supervision for visual comparisons via synthetic images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5570–5579 (2017)
Yu, A., Grauman, K.: Thinking outside the pool: active training image creation for relative attributes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 708–718 (2019)
Zhang, Z., Li, Y., Zhang, Z.: Relative attribute learning with deep attentive cross-image representation. In: Asian Conference on Machine Learning, pp. 879–892 (2018)
Zhuang, N., Yan, Y., Chen, S., Wang, H., Shen, C.: Multi-label learning based deep transfer neural network for facial attribute classification. Pattern Recogn. 80, 225–240 (2018)
Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8697–8710 (2018)
Acknowledgment
This work was supported by a grant from The Scientific and Technological Research Council of Turkey (TÜBİTAK) under project number 119E429.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ahmed, S.A.A., Yanikoglu, B. (2021). Relative Attribute Classification with Deep-RankSVM. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12662. Springer, Cham. https://doi.org/10.1007/978-3-030-68790-8_51
Download citation
DOI: https://doi.org/10.1007/978-3-030-68790-8_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68789-2
Online ISBN: 978-3-030-68790-8
eBook Packages: Computer ScienceComputer Science (R0)
