Hui Liu
ABSTRACT
Pairwise constraint propagation (PCP) aims to propagate a limited number of initial pairwise constraints (PCs, including must-link and cannot-link constraints) from the constrained data samples to the unconstrained ones to boost subsequent PC-based applications. The existing PCP approaches always suffer from the imbalance characteristic of PCs, which limits their performance significantly. To this end, we propose a novel imbalance-aware PCP method, by comprehensively and theoretically exploring the intrinsic structures of the underlying PCs. Specifically, different from the existing methods that adopt a single representation, we propose to use two separate carriers to represent the two types of links. And the propagation is driven by the structure embedded in data samples and the regularization of the local, global, and complementary structures of the two carries. Our method is elegantly cast as a well-posed constrained optimization model, which can be efficiently solved. Experimental results demonstrate that the proposed PCP method is capable of generating more high-fidelity PCs than the recent PCP algorithms. In addition, the augmented PCs by our method produce higher accuracy than state-of-the-art semi-supervised clustering methods when applied to constrained clustering. To the best of our knowledge, this is the first PCP method taking the imbalance property of PCs into account.
- Edoardo Amaldi and Viggo Kann. 1998. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science , Vol. 209, 1--2 (1998), 237--260.Google Scholar
Digital Library
- Sugato Basu, Ian Davidson, and Kiri Wagstaff. 2008. Constrained clustering: Advances in algorithms, theory, and applications .CRC Press.Google Scholar
- Deng Cai, Xiaofei He, and Jiawei Han. 2005. Document clustering using locality preserving indexing. IEEE Transactions on Knowledge and Data Engineering , Vol. 17, 12 (2005), 1624--1637.Google ScholarDigital Library
- Emmanuel J Candès, Xiaodong Li, Yi Ma, and John Wright. 2011. Robust principal component analysis? J. ACM , Vol. 58, 3 (2011), 11.Google ScholarDigital Library
- Jie Chen, Hua Mao, Yongsheng Sang, and Zhang Yi. 2017. Subspace clustering using a symmetric low-rank representation. Knowledge-Based Systems , Vol. 127 (2017), 46--57.Google ScholarCross Ref
- Mihai Cucuringu, Ioannis Koutis, Sanjay Chawla, Gary Miller, and Richard Peng. 2016. Simple and scalable constrained clustering: a generalized spectral method. In Artificial Intelligence and Statistics. 445--454.Google Scholar
- Sharon Fogel, Hadar Averbuch-Elor, Jacov Goldberger, and Daniel Cohen-Or. 2018. Clustering-driven deep embedding with pairwise constraints. arXiv preprint arXiv:1803.08457 (2018).Google Scholar
- Zhenyong Fu. 2015. Pairwise constraint propagation via low-rank matrix recovery. Computational Visual Media , Vol. 1, 3 (2015), 211--220.Google ScholarCross Ref
- Haibo He and Edwardo A Garcia. 2008. Learning from imbalanced data. IEEE Transactions on Knowledge & Data Engineering 9 (2008), 1263--1284.Google ScholarDigital Library
- Steven CH Hoi, Rong Jin, and Michael R Lyu. 2007. Learning nonparametric kernel matrices from pairwise constraints. In International Conference on Machine Learning. 361--368.Google ScholarDigital Library
- Steven CH Hoi, Wei Liu, Michael R Lyu, and Wei-Ying Ma. 2006. Learning distance metrics with contextual constraints for image retrieval. In IEEE International Conference on Computer Vision and Pattern Recognition , Vol. 2. IEEE, 2072--2078.Google ScholarDigital Library
- Yuheng Jia, Sam Kwong, and Junhui Hou. 2018. Semi-supervised spectral clustering with structured sparsity regularization. IEEE Signal Processing Letters , Vol. 25, 3 (2018), 403--407.Google ScholarCross Ref
- Chengming Jiang, Huiqing Xie, and Zhaojun Bai. 2017. Robust and Efficient Computation of Eigenvectors in a Generalized Spectral Method for Constrained Clustering. In Artificial Intelligence and Statistics . 757--766.Google Scholar
- Sepandar D. Kamvar, Dan Klein, and Christopher D. Manning. 2003. Spectral learning. In International Joint Conference on Artificial Intelligence. 561--566.Google Scholar
- Brian Kulis, Sugato Basu, Inderjit Dhillon, and Raymond Mooney. 2009. Semi-supervised graph clustering: a kernel approach. Machine Learning , Vol. 74, 1 (2009), 1--22.Google ScholarDigital Library
- Zhenguo Li, Jianzhuang Liu, and Xiaoou Tang. 2008. Pairwise constraint propagation by semidefinite programming for semi-supervised classification. In International Conference on Machine Learning . 576--583.Google ScholarDigital Library
- Zhouchen Lin, Minming Chen, and Yi Ma. 2010. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055 (2010).Google Scholar
- Mingxia Liu and Daoqiang Zhang. 2016. Pairwise constraint-guided sparse learning for feature selection. IEEE Transactions on Cybernetics , Vol. 46, 1 (2016), 298--310.Google ScholarCross Ref
- Wenhe Liu, Xiaojun Chang, Ling Chen, and Yi Yang. 2017. Early active learning with pairwise constraint for person re-identification. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases . Springer, 103--118.Google ScholarCross Ref
- Wei Liu, Shiqian Ma, Dacheng Tao, Jianzhuang Liu, and Peng Liu. 2010. Semi-supervised sparse metric learning using alternating linearization optimization. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, 1139--1148.Google ScholarDigital Library
- Zhengdong Lu and Miguel A Carreira-Perpinan. 2008. Constrained spectral clustering through affinity propagation. In IEEE International Conference on Computer Vision and Pattern Recognition. IEEE, 1--8.Google Scholar
- Zhiwu Lu and Horace HS Ip. 2010. Constrained spectral clustering via exhaustive and efficient constraint propagation. In European Conference on Computer Vision. Springer, 1--14.Google ScholarCross Ref
- Zhiwu Lu and Yuxin Peng. 2013. Exhaustive and efficient constraint propagation: A graph-based learning approach and its applications. International Journal of Computer Vision , Vol. 103, 3 (2013), 306--325.Google ScholarCross Ref
- Bac Nguyen and Bernard De Baets. 2019. Kernel distance metric learning using pairwise constraints for person re-identification. IEEE Transactions on Image Processing , Vol. 28, 2 (2019), 589--600.Google ScholarCross Ref
- Nam Nguyen and Rich Caruana. 2008. Improving classification with pairwise constraints: a margin-based approach. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases . Springer, 113--124.Google ScholarDigital Library
- Jianbo Shi and J. Malik. 2000. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 22, 8 (2000), 888--905.Google ScholarDigital Library
- Alexander Strehl and Joydeep Ghosh. 2002. Cluster ensembles--a knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research , Vol. 3, Dec (2002), 583--617.Google ScholarDigital Library
- Ulrike Von Luxburg. 2007. A tutorial on spectral clustering. Statistics and computing , Vol. 17, 4 (2007), 395--416.Google Scholar
- John Wright, Arvind Ganesh, Shankar Rao, Yigang Peng, and Yi Ma. 2009. Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In Advances in Neural Information Processing Systems. 2080--2088.Google Scholar
- Menglin Wu, Qiang Chen, and Quansen Sun. 2014. Medical image retrieval with relevance feedback via pairwise constraint propagation. KSII Transactions on Internet and Information Systems , Vol. 8, 1 (2014), 249--268.Google ScholarCross Ref
- Wenhui Wu, Yuheng Jia, Sam Kwong, and Junhui Hou. 2018. Pairwise Constraint Propagation-Induced Symmetric Nonnegative Matrix Factorization. IEEE Transactions on Neural Networks and Learning Systems 99 (2018), 1--14.Google Scholar
- Rong Yan, Jian Zhang, Jie Yang, and Alexander G Hauptmann. 2006. A discriminative learning framework with pairwise constraints for video object classification. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 28, 4 (2006), 578--593.Google ScholarDigital Library
- Zheng Yang, Yao Hu, Haifeng Liu, Huajun Chen, and Zhaohui Wu. 2014. Matrix Completion for Cross-view Pairwise Constraint Propagation. In ACM International Conference on Multimedia. ACM, 897--900.Google Scholar
- Hong Zeng and Yiu-ming Cheung. 2011. Semi-supervised maximum margin clustering with pairwise constraints. IEEE Transactions on Knowledge and Data Engineering , Vol. 24, 5 (2011), 926--939.Google ScholarDigital Library
- Zhao Zhang, Tommy WS Chow, and Mingbo Zhao. 2012a. Trace ratio optimization-based semi-supervised nonlinear dimensionality reduction for marginal manifold visualization. IEEE Transactions on Knowledge and Data Engineering , Vol. 25, 5 (2012), 1148--1161.Google ScholarDigital Library
- Zhao Zhang, Mingbo Zhao, and Tommy WS Chow. 2012b. Marginal semi-supervised sub-manifold projections with informative constraints for dimensionality reduction and recognition. Neural Networks , Vol. 36 (2012), 97--111.Google ScholarDigital Library
- Dengyong Zhou, Olivier Bousquet, Thomas N Lal, Jason Weston, and Bernhard Schölkopf. 2004. Learning with local and global consistency. In Advances in Neural Information Processing Systems. 321--328.Google Scholar
- Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. 2003. Semi-supervised learning using gaussian fields and harmonic functions. In International Conference on Machine Learning. 912--919.Google Scholar
Index Terms
- Imbalance-aware Pairwise Constraint Propagation
Recommendations
Matrix Completion for Cross-view Pairwise Constraint Propagation
MM '14: Proceedings of the 22nd ACM international conference on MultimediaAs pairwise constraints are usually easier to access than label information, pairwise constraint propagation attracts more and more attention in semi-supervised learning. Most existing pairwise constraint propagation methods are based on canonical graph ...
Bagging Constraint Score for feature selection with pairwise constraints
Constraint Score is a recently proposed method for feature selection by using pairwise constraints which specify whether a pair of instances belongs to the same class or not. It has been shown that the Constraint Score, with only a small amount of ...
Local similarity learning for pairwise constraint propagation
Pairwise constraint propagation studies the problem of propagating the scarce pairwise constraints across the entire dataset. Effective propagation algorithms have previously been designed based on the graph-based semi-supervised learning framework. ...
Comments