Yongshan Zhang
Chuan Zhou
Jian Yang
Skip Abstract Section
Abstract
Unlabeled, multi-view data presents a considerable challenge in many real-world data analysis tasks. These data are worth exploring because they often contain complementary information that improves the quality of the analysis results. Clustering with multi-view data is a particularly challenging problem as revealing the complex data structures between many feature spaces demands discriminative features that are specific to the task and, when too few of these features are present, performance suffers. Extreme learning machines (ELMs) are an emerging form of learning model that have shown an outstanding representation ability and superior performance in a range of different learning tasks. Motivated by the promise of this advancement, we have developed a novel multi-view fusion clustering framework based on an ELM, called MVEC. MVEC learns the embeddings from each view of the data via the ELM network, then constructs a single unified embedding according to the correlations and dependencies between each embedding and automatically weighting the contribution of each. This process exposes the underlying clustering structures embedded within multi-view data with a high degree of accuracy. A simple yet efficient solution is also provided to solve the optimization problem within MVEC. Experiments and comparisons on eight different benchmarks from different domains confirm MVEC’s clustering accuracy.
- Steffen Bickel and Tobias Scheffer. 2004. Multi-view clustering. In Proceedings of the 2004 International Conference on Data Mining, Vol. 4. 19--26.Google Scholar
Cross Ref
- Maria Brbić and Ivica Kopriva. 2018. Multi-view low-rank sparse subspace clustering. Pattern Recognition 73 (2018), 247--258.Google ScholarCross Ref
- Xiao Cai, Feiping Nie, and Heng Huang. 2013. Multi-view K-means clustering on big data. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence. 2598--2604.Google ScholarDigital Library
- Xiao Cai, Feiping Nie, Heng Huang, and Farhad Kamangar. 2011. Heterogeneous image feature integration via multi-modal spectral clustering. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. 1977--1984.Google ScholarDigital Library
- Jiuwen Cao, Zhiping Lin, Guang-Bin Huang, and Nan Liu. 2012. Voting based extreme learning machine. Information Sciences 185, 1 (2012), 66--77.Google ScholarDigital Library
- Kamalika Chaudhuri, Sham M. Kakade, Karen Livescu, and Karthik Sridharan. 2009. Multi-view clustering via canonical correlation analysis. In Proceedings of the 26th International Conference on Machine Learning. 129--136.Google ScholarDigital Library
- Hongchang Gao, Feiping Nie, Xuelong Li, and Heng Huang. 2015. Multi-view subspace clustering. In Proceedings of the 2015 IEEE International Conference on Computer Vision. 4238--4246.Google ScholarDigital Library
- Derek Greene. 2009. A matrix factorization approach for integrating multiple data views. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases. 423--438.Google ScholarCross Ref
- Saima Hassan, Mojtaba Ahmadieh Khanesar, Jafreezal Jaafar, and Abbas Khosravi. 2017. Comparative analysis of three approaches of antecedent part generation for an IT2 TSK FLS. Applied Soft Computing 51 (2017), 130--144.Google ScholarDigital Library
- Lifang He, Chun-Ta Lu, Hao Ding, Shen Wang, Linlin Shen, Philip S. Yu, and Ann B. Ragin. 2017. Multi-way multi-level kernel modeling for neuroimaging classification. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 356--364.Google Scholar
- Qing He, Xin Jin, Changying Du, Fuzhen Zhuang, and Zhongzhi Shi. 2014. Clustering in extreme learning machine feature space. Neurocomputing 128 (2014), 88--95.Google ScholarDigital Library
- Chenping Hou, Feiping Nie, Tao Hong, and Dongyun Yi. 2017. Multi-view unsupervised feature selection with adaptive similarity and view weight. IEEE Transactions on Knowledge and Data Engineering 29, 9 (2017), 1998--2011.Google ScholarDigital Library
- Gao Huang, Tianchi Liu, Yan Yang, Zhiping Lin, Shiji Song, and Cheng Wu. 2015. Discriminative clustering via extreme learning machine. Neural Networks 70 (2015), 1--8.Google ScholarDigital Library
- Gao Huang, Shiji Song, Jatinder ND Gupta, and Cheng Wu. 2014. Semi-supervised and unsupervised extreme learning machines. IEEE Transactions on Cybernetics 44, 12 (2014), 2405--2417.Google ScholarCross Ref
- G.-B. Huang, Hongming Zhou, Xiaojian Ding, and Rui Zhang. 2012. Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics 42, 2 (2012), 513--529.Google ScholarDigital Library
- Liyanaarachchi Lekamalage Chamara Kasun, Yan Yang, Guang-Bin Huang, and Zhengyou Zhang. 2016. Dimension reduction with extreme learning machine. IEEE Transactions on Image Processing 25, 8 (2016), 3906--3918.Google ScholarCross Ref
- Liyanaarachchi Lekamalage Chamara Kasun, Hongming Zhou, Guang-Bin Huang, and Chi Man Vong. 2013. Representational learning with ELMs for big data. IEEE Intelligent Systems 28, 6 (2013), 31--34.Google Scholar
- Young-Min Kim, Massih-Reza Amini, Cyril Goutte, and Patrick Gallinari. 2010. Multi-view clustering of multilingual documents. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 821--822.Google ScholarDigital Library
- Abhishek Kumar, Piyush Rai, and Hal Daume. 2011. Co-regularized multi-view spectral clustering. In Advances in Neural Information Processing Systems. 1413--1421.Google Scholar
- Shao-Yuan Li, Yuan Jiang, and Zhi-Hua Zhou. 2010. Partial multi-view clustering. In Proceedings of the 24th AAAI Conference on Artificial Intelligence. 1968--1974.Google Scholar
- Jialu Liu, Chi Wang, Jing Gao, and Jiawei Han. 2013. Multi-view clustering via joint nonnegative matrix factorization. In Proceedings of the 2013 SIAM International Conference on Data Mining. 252--260.Google ScholarCross Ref
- Tianchi Liu, Chamara Kasun Liyanaarachchi Lekamalage, Guang-Bin Huang, and Zhiping Lin. 2018. An adaptive graph learning method based on dual data representations for clustering. Pattern Recognition 77 (2018), 126--139.Google ScholarDigital Library
- Tianchi Liu, Chamara Kasun Liyanaarachchi Lekamalage, Guang-Bin Huang, and Zhiping Lin. 2018. Extreme learning machine for joint embedding and clustering. Neurocomputing 277 (2018), 78--88.Google ScholarDigital Library
- Feiping Nie, Guohao Cai, Jing Li, and Xuelong Li. 2018. Auto-weighted multi-view learning for image clustering and semi-supervised classification. IEEE Transactions on Image Processing 27, 3 (2018), 1501--1511.Google ScholarCross Ref
- Feiping Nie, Heng Huang, Xiao Cai, and Chris H. Ding. 2010. Efficient and robust feature selection via joint ℓ2,1-norms minimization. In Advances in Neural Information Processing Systems. 1813--1821.Google ScholarDigital Library
- Feiping Nie, Jing Li, and Xuelong Li. 2016. Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification. In Proceedings of the 25th International Joint Conference on Artificial Intelligence. 1881--1887.Google Scholar
- Feiping Nie, Jing Li, and Xuelong Li. 2017. Self-weighted multiview clustering with multiple graphs. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2564--2570.Google ScholarCross Ref
- Feiping Nie, Zinan Zeng, Ivor W. Tsang, Dong Xu, and Changshui Zhang. 2011. Spectral embedded clustering: A framework for in-sample and out-of-sample spectral clustering. IEEE Transactions on Neural Networks 22, 11 (2011), 1796--1808.Google ScholarDigital Library
- Huimin Pei, Kuaini Wang, Qiang Lin, and Ping Zhong. 2018. Robust semi-supervised extreme learning machine. Knowledge-Based Systems 159 (2018), 203--220.Google ScholarCross Ref
- Michael D. Plummer and László Lovász. 1986. Matching Theory. Vol. 29. North-Holland Publishing Company, Amsterdam.Google Scholar
- Lei Shi, Liang Du, and Yi-Dong Shen. 2014. Robust spectral learning for unsupervised feature selection. In Proceedings of the 2014 IEEE International Conference on Data Mining. 977--982.Google ScholarDigital Library
- Yu Su, Shiguang Shan, Xilin Chen, and Wen Gao. 2011. Classifiability-based discriminatory projection pursuit. IEEE Transactions on Neural Networks 22, 12 (2011), 2050--2061.Google ScholarDigital Library
- Chang Tang, Jiajia Chen, Xinwang Liu, Miaomiao Li, Pichao Wang, Minhui Wang, and Peng Lu. 2018. Consensus learning guided multi-view unsupervised feature selection. Knowledge-Based Systems 160 (2018), 49--60.Google ScholarCross Ref
- Chang Tang, Xinzhong Zhu, Xinwang Liu, Miaomiao Li, Pichao Wang, Changqing Zhang, and Lizhe Wang. 2019. Learning a joint affinity graph for multiview subspace clustering. IEEE Transactions on Multimedia 21, 7 (2019), 1724--1736. DOI:https://doi.org/10.1109/TMM.2018.2889560Google ScholarCross Ref
- Chang Tang, Xinzhong Zhu, Xinwang Liu, and Lizhe Wang. 2019. Cross-view local structure preserved diversity and consensus learning for multi-view unsupervised feature selection. In Proceedings of the AAAI Conference on Artificial Intelligence. 595--604.Google ScholarCross Ref
- Jiexiong Tang, Chenwei Deng, and Guang-Bin Huang. 2016. Extreme learning machine for multilayer perceptron. IEEE Transactions on Neural Networks and Learning Systems 27, 4 (2016), 809--821.Google ScholarCross Ref
- Jiliang Tang, Xia Hu, Huiji Gao, and Huan Liu. 2013. Unsupervised feature selection for multi-view data in social media. In Proceedings of the 2013 SIAM International Conference on Data Mining. 270--278.Google ScholarCross Ref
- Wei Tang, Zhengdong Lu, and Inderjit S. Dhillon. 2009. Clustering with multiple graphs. In Proceedings of the 9th IEEE International Conference on Data Mining. 1016--1021.Google Scholar
- Muhammad Uzair and Ajmal Mian. 2017. Blind domain adaptation with augmented extreme learning machine features. IEEE Transactions on Cybernetics 47, 3 (2017), 651--660.Google ScholarCross Ref
- Hao Wang, Yan Yang, Bing Liu, and Hamido Fujita. 2019. A study of graph-based system for multi-view clustering. Knowledge-Based Systems 163 (2019), 1009--1019.Google ScholarCross Ref
- Qiang Wang, Yong Dou, Xinwang Liu, Qi Lv, and Shijie Li. 2016. Multi-view clustering with extreme learning machine. Neurocomputing 214 (2016), 483--494.Google ScholarDigital Library
- Zhelong Wang, Donghui Wu, Raffaele Gravina, Giancarlo Fortino, Yongmei Jiang, and Kai Tang. 2017. Kernel fusion based extreme learning machine for cross-location activity recognition. Information Fusion 37 (2017), 1--9.Google ScholarDigital Library
- Chi Man Wong, Chi Man Vong, Pak Kin Wong, and Jiuwen Cao. 2018. Kernel-based multilayer extreme learning machines for representation learning. IEEE Transactions on Neural Networks and Learning Systems 29, 3 (2018), 757--762.Google ScholarCross Ref
- Jinglin Xu, Junwei Han, and Feiping Nie. 2016. Discriminatively embedded k-means for multi-view clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5356--5364.Google ScholarCross Ref
- Jinglin Xu, Junwei Han, Feiping Nie, and Xuelong Li. 2017. Re-weighted discriminatively embedded -means for multi-view clustering. IEEE Transactions on Image Processing 26, 6 (2017), 3016--3027.Google ScholarDigital Library
- Yimin Yang and Q. M. Jonathan Wu. 2016. Multilayer extreme learning machine with subnetwork nodes for representation learning. IEEE Transactions on Cybernetics 46, 11 (2016), 2570--2583.Google ScholarCross Ref
- Yimin Yang, Q. M. Jonathan Wu, and Yaonan Wang. [n.d.]. Autoencoder with invertible functions for dimension reduction and image reconstruction. IEEE Transactions on Systems, Man, and Cybernetics: Systems 48, 7 ([n.d.]), 1065--1079.Google Scholar
- Changqing Zhang, Qinghua Hu, Huazhu Fu, Pengfei Zhu, and Xiaochun Cao. 2017. Latent multi-view subspace clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 30. 4279--4287.Google ScholarCross Ref
- Yongshan Zhang, Jia Wu, Zhihua Cai, Peng Zhang, and Ling Chen. 2016. Memetic extreme learning machine. Pattern Recognition 58 (2016), 135--148.Google ScholarDigital Library
- Yongshan Zhang, Jia Wu, Chuan Zhou, and Zhihua Cai. 2017. Instance cloned extreme learning machine. Pattern Recognition 68 (2017), 52--65.Google ScholarDigital Library
- Wentao Zhu, Jun Miao, and Laiyun Qing. 2014. Constrained extreme learning machine: A novel highly discriminative random feedforward neural network. In Proceedings of the 2014 International Joint Conference on Neural Networks. 800--807.Google ScholarCross Ref
- Xiaofeng Zhu, Xuelong Li, and Shichao Zhang. 2016. Block-row sparse multiview multilabel learning for image classification. IEEE Transactions on Cybernetics 46, 2 (2016), 450--461.Google ScholarCross Ref
- Fuzhen Zhuang, George Karypis, Xia Ning, Qing He, and Zhongzhi Shi. 2012. Multi-view learning via probabilistic latent semantic analysis. Information Sciences 199 (2012), 20--30.Google ScholarDigital Library
- Wenzhang Zhuge, Feiping Nie, Chenping Hou, and Dongyun Yi. 2017. Unsupervised single and multiple views feature extraction with structured graph. IEEE Transactions on Knowledge and Data Engineering 29, 10 (2017), 2347--2359.Google ScholarDigital Library
Index Terms
- Multi-View Fusion with Extreme Learning Machine for Clustering
Recommendations
Multi-view clustering with extreme learning machine
Nowadays, data always have multiple representations, and a good feature representation usually leads to a good clustering performance. Existing multi-view clustering works generally integrate multiple complementary information to gain better clustering ...
Local kernel alignment based multi-view clustering using extreme learning machine
A similarity or dissimilarity measure, such as the Euclidean distance, is crucial to discriminative clustering algorithms. These measures used to calculate pairwise similarities between samples rely on data representations in a feature space. However, ...
Multi-view clustering via spectral embedding fusion
Multi-view learning, such as multi-view feature learning and multi-view clustering, has a wide range of applications in machine learning and pattern recognition. Most previous studies employ the multiple data information from various views to improve ...
Comments