ABSTRACT
Hashing techniques have recently been successfully applied to solve similarity search problems in the information retrieval field because of their significantly reduced storage and high-speed search capabilities. However, the hash codes learned from most recent cross-modal hashing methods lack the ability to comprehensively preserve adequate information, resulting in a less than desirable performance. To solve this limitation, we propose a novel method termed Nonlinear Robust Discrete Hashing (NRDH), for cross-modal retrieval. The main idea behind NRDH is motivated by the success of neural networks, i.e., nonlinear descriptors, in the field of representation learning, and the use of nonlinear descriptors instead of simple linear transformations is more in line with the complex relationships that exist between common latent representation and heterogeneous multimedia data in the real world. In NRDH, we first learn a common latent representation through nonlinear descriptors to encode complementary and consistent information from the features of the heterogeneous multimedia data. Moreover, an asymmetric learning scheme is proposed to correlate the learned hash codes with the common latent representation. Empirically, we demonstrate that NRDH is able to successfully generate a comprehensive common latent representation that significantly improves the quality of the learned hash codes. Then, NRDH adopts a linear learning strategy to fast learn the hash function with the learned hash codes. Extensive experiments performed on two benchmark datasets highlight the superiority of NRDH over several state-of-the-art methods.
Supplemental Material
- Michael M. Bronstein, Alexander M. Bronstein, Fabrice Michel, and Nikos Paragios. 2010. Data fusion through cross-modality metric learning using similarity-sensitive hashing. In CVPR. 3594--3601.Google Scholar
- Micael Carvalho, Rémi Cadène, David Picard, Laure Soulier, Nicolas Thome, and Matthieu Cord. 2018. Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddings. In SIGIR. 35--44.Google Scholar
- Zhen-Duo Chen, Yongxin Wang, Hui-Qiong Li, Xin Luo, Liqiang Nie, and Xin-Shun Xu. 2019. A Two-Step Cross-Modal Hashing by Exploiting Label Correlations and Preserving Similarity in Both Steps. In ACM MM. 1694--1702.Google Scholar
- Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yan-tao Zheng. 2009. NUS-WIDE: a real-world web image database from National University of Singapore. In CIVR.Google Scholar
- Cheng Deng, Zhaojia Chen, Xianglong Liu, Xinbo Gao, and Dacheng Tao. 2018. Triplet-Based Deep Hashing Network for Cross-Modal Retrieval. TIP 27, 8 (2018), 3893--3903.Google ScholarCross Ref
- Cheng Deng, Erkun Yang, Tongliang Liu, Jie Li, Wei Liu, and Dacheng Tao. 2019. Unsupervised Semantic-Preserving Adversarial Hashing for Image Search. TIP 28, 8 (2019), 4032--4044.Google ScholarCross Ref
- Guiguang Ding, Yuchen Guo, and Jile Zhou. 2014. Collective Matrix Factorization Hashing for Multimodal Data. In CVPR. 2083--2090.Google Scholar
- Guiguang Ding, Yuchen Guo, Jile Zhou, and Yue Gao. 2016. Large-Scale Cross-Modality Search via Collective Matrix Factorization Hashing. TIP 25, 11 (2016), 5427--5440.Google ScholarDigital Library
- Kun Ding, Bin Fan, Chunlei Huo, Shiming Xiang, and Chunhong Pan. 2017. Cross-Modal Hashing via Rank-Order Preserving. TMM 19, 3 (2017), 571--585.Google ScholarDigital Library
- Yunchao Gong, Svetlana Lazebnik, Albert Gordo, and Florent Perronnin. 2013. Iterative Quantization: A Procrustean Approach to Learning Binary Codes for Large-Scale Image Retrieval. TPAMI 35, 12 (2013), 2916--2929.Google ScholarDigital Library
- Albert Gordo, Florent Perronnin, Yunchao Gong, and Svetlana Lazebnik. 2014. Asymmetric Distances for Binary Embeddings. TPAMI 36, 1 (2014), 33--47.Google ScholarDigital Library
- Jie Gui, Tongliang Liu, Zhenan Sun, Dacheng Tao, and Tieniu Tan. 2018. Fast Supervised Discrete Hashing. TPAMI 40, 2 (2018), 490--496.Google ScholarDigital Library
- Kaiming He, Fang Wen, and Jian Sun. 2013. K-Means Hashing: An Affinity-Preserving Quantization Method for Learning Binary Compact Codes. In CVPR. 2938--2945.Google Scholar
- Mark J. Huiskes and Michael S. Lew. 2008. The MIR flickr retrieval evaluation. In ACM SIGMM. 39--43.Google Scholar
- Qing-Yuan Jiang and Wu-Jun Li. 2017. Deep Cross-Modal Hashing. In CVPR. 3270--3278.Google Scholar
- Qing-Yuan Jiang and Wu-Jun Li. 2018. Asymmetric Deep Supervised Hashing. In AAAI. 3342--3349.Google Scholar
- Lu Jin, Kai Li, Hao Hu, Guo-Jun Qi, and Jinhui Tang. 2018. Semantic Neighbor Graph Hashing for Multimodal Retrieval. TIP 27, 3 (2018), 1405--1417.Google ScholarCross Ref
- Weihao Kong and Wu-Jun Li. 2012. Isotropic Hashing. In NIPS. 1655--1663.Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In NIPS. 1106--1114.Google ScholarDigital Library
- Brian Kulis and Trevor Darrell. 2009. Learning to Hash with Binary Reconstructive Embeddings. In NIPS. 1042--1050.Google Scholar
- Chuan-Xiang Li, Zhen-Duo Chen, Peng-Fei Zhang, Xin Luo, Liqiang Nie, Wei Zhang, and Xin-Shun Xu. 2018. SCRATCH: A Scalable Discrete Matrix Factorization Hashing for Cross-Modal Retrieval. In ACM MM. 1--9.Google Scholar
- Chao Li, Cheng Deng, Ning Li, Wei Liu, Xinbo Gao, and Dacheng Tao. 2018. Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval. In CVPR. 4242--4251.Google Scholar
- Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. 2016. Feature Learning Based Deep Supervised Hashing with Pairwise Labels. In IJCAI. 1711--1717.Google Scholar
- Zijia Lin, Guiguang Ding, Mingqing Hu, and Jianmin Wang. 2015. Semantics-preserving hashing for cross-view retrieval. In CVPR. 3864--3872.Google Scholar
- Zhouchen Lin, Risheng Liu, and Zhixun Su. 2011. Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation. In NIPS. 612--620.Google Scholar
- Hong Liu, Rongrong Ji, Yongjian Wu, Feiyue Huang, and Baochang Zhang. 2017. Cross-Modality Binary Code Learning via Fusion Similarity Hashing. In CVPR. 6345--6353.Google Scholar
- Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. 2016. Deep Supervised Hashing for Fast Image Retrieval. In CVPR. 2064--2072.Google Scholar
- Li Liu, Mengyang Yu, and Ling Shao. 2015. Multiview Alignment Hashing for Efficient Image Search. TIP 24, 3 (2015), 956--966.Google ScholarDigital Library
- Wei Liu, Cun Mu, Sanjiv Kumar, and Shih-Fu Chang. 2014. Discrete Graph Hashing. In NIPS. 3419--3427.Google Scholar
- Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. 2012. Supervised hashing with kernels. In CVPR. 2074--2081.Google Scholar
- Xingbo Liu, Xiushan Nie, Wenjun Zeng, Chaoran Cui, Lei Zhu, and Yilong Yin. 2018. Fast Discrete Cross-modal Hashing with Regressing from Semantic Labels. In ACM MM. 1662--1669.Google Scholar
- Xuanwu Liu, Guoxian Yu, Carlotta Domeniconi, Jun Wang, Yazhou Ren, and Maozu Guo. 2019. Ranking-Based Deep Cross-Modal Hashing. In AAAI. 4400--4407.Google Scholar
- Xu Lu, Lei Zhu, Zhiyong Cheng, Liqiang Nie, and Huaxiang Zhang. 2019. Online Multi-modal Hashing with Dynamic Query-adaption. In SIGIR. 715--724.Google Scholar
- Xin Luo, Peng-Fei Zhang, Ye Wu, Zhen-Duo Chen, Hua-Junjie Huang, and Xin-Shun Xu. 2018. Asymmetric Discrete Cross-Modal Hashing. In ICMR. 204--212.Google Scholar
- Devraj Mandal, Kunal N. Chaudhury, and Soma Biswas. 2019. Generalized Semantic Preserving Hashing for Cross-Modal Retrieval. TIP 28, 1 (2019), 102--112.Google ScholarDigital Library
- Behnam Neyshabur, Nati Srebro, Ruslan Salakhutdinov, Yury Makarychev, and Payman Yadollahpour. 2013. The Power of Asymmetry in Binary Hashing. In NIPS. 2823--2831.Google Scholar
- Yuxin Peng, Xiaohua Zhai, Yunzhen Zhao, andXinHuang.2016. Semi-Supervised Cross-Media Feature Learning with Unified Patch Graph Regularization. TCSVT 26, 3 (2016), 583--596.Google ScholarDigital Library
- Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. 2015. Supervised Discrete Hashing. In CVPR. 37--45.Google Scholar
- Fumin Shen, Yan Xu, Li Liu, Yang Yang, Zi Huang, and Heng Tao Shen. 2018. Unsupervised Deep Hashing with Similarity-Adaptive and Discrete Optimization. TPAMI 40, 12 (2018), 3034--3044.Google ScholarDigital Library
- Xiaoshuang Shi, Fuyong Xing, Kaidi Xu, Manish Sapkota, and Lin Yang. 2017. Asymmetric Discrete Graph Hashing. In AAAI. 2541--2547.Google Scholar
- Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In ICLR.Google Scholar
- Jingkuan Song, Yang Yang, Yi Yang, Zi Huang, and Heng Tao Shen. 2013. Intermedia hashing for large-scale retrieval from heterogeneous data sources. In SIGMOD. 785--796.Google Scholar
- Jingdong Wang, Ting Zhang, Jingkuan Song, Nicu Sebe, and Heng Tao Shen. 2018. A Survey on Learning to Hash. TPAMI 40, 4 (2018), 769--790.Google ScholarCross Ref
- Yair Weiss, Antonio Torralba, and Robert Fergus. 2008. Spectral Hashing. In NIPS. 1753--1760.Google Scholar
- Xing Xu, Fumin Shen, Yang Yang, Heng Tao Shen, and Xuelong Li. 2017. Learning Discriminative Binary Codes for Large-scale Cross-modal Retrieval. TIP 26, 5 (2017), 2494--2507.Google ScholarDigital Library
- Erkun Yang, Cheng Deng, Wei Liu, Xianglong Liu, Dacheng Tao, and Xinbo Gao. 2017. Pairwise Relationship Guided Deep Hashing for Cross-Modal Retrieval. In AAAI. 1618--1625.Google Scholar
- Dongqing Zhang and Wu-Jun Li. 2014. Large-Scale Supervised Multimodal Hashing with Semantic Correlation Maximization. In AAAI. 2177--2183.Google Scholar
- Dan Zhang, Fei Wang, and Luo Si. 2011. Composite hashing with multiple information sources. In SIGIR. 225--234.Google Scholar
- Haofeng Zhang, Li Liu, Yang Long, and Ling Shao. 2018. Unsupervised Deep Hashing with Pseudo Labels for Scalable Image Retrieval. TIP 27, 4 (2018), 1626--1638.Google ScholarDigital Library
- Jile Zhou, Guiguang Ding, and Yuchen Guo. 2014. Latent semantic sparse hashing for cross-modal similarity search. In SIGIR. 415--424.Google Scholar
Index Terms
- Nonlinear Robust Discrete Hashing for Cross-Modal Retrieval
Recommendations
Semantics-Reconstructing Hashing for Cross-Modal Retrieval
Advances in Knowledge Discovery and Data MiningAbstractRetrieval on Cross-modal data has attracted extensive attention as it enables fast searching across various data sources, such as texts, images and videos. As one of the typical techniques for cross-model searching, hashing methods project ...
Discriminant Cross-modal Hashing
ICMR '16: Proceedings of the 2016 ACM on International Conference on Multimedia RetrievalHashing based methods have attracted considerable attention for efficient cross-modal retrieval on large-scale multimedia data. The core problem of cross-modal hashing is how to effectively integrate heterogeneous features from different modalities to ...
Discrete online cross-modal hashing
Highlights- Different from the majority of related methods, DOCH is a discrete one.
- By ...
AbstractWith the prevalence of multimedia content on the Web which usually continuously comes in a stream fashion, online cross-modal hashing methods have attracted extensive interest in recent years. However, most online hashing methods adopt ...
Comments