Zhaohong Deng
Skip Abstract Section
Skip Supplemental Material SectionAbstract
The knowledge-leverage-based Takagi--Sugeno--Kang fuzzy system (KL-TSK-FS) modeling method has shown promising performance for fuzzy modeling tasks where transfer learning is required. However, the knowledge-leverage mechanism of the KL-TSK-FS can be further improved. This is because available training data in the target domain are not utilized for the learning of antecedents and the knowledge transfer mechanism from a source domain to the target domain is still too simple for the learning of consequents when a Takagi--Sugeno--Kang fuzzy system (TSK-FS) model is trained in the target domain. The proposed method, that is, the enhanced KL-TSK-FS (EKL-TSK-FS), has two knowledge-leverage strategies for enhancing the parameter learning of the TSK-FS model for the target domain using available information from the source domain. One strategy is used for the learning of antecedent parameters, while the other is for consequent parameters. It is demonstrated that the proposed EKL-TSK-FS has higher transfer learning abilities than the KL-TSK-FS. In addition, the EKL-TSK-FS has been further extended for the scene of the multisource domain.
Supplemental Material
Available for Download
zip
deng.zip (185.2 KB)
Supplemental movie, appendix, image and software files for, Enhanced Knowledge-Leverage-Based TSK Fuzzy System Modeling for Inductive Transfer Learning
- M. F. Azeem, M. Hanmandlu, and N. Ahmad. 2000. Generalization of adaptive neural-fuzzy inference systems. IEEE Trans. Neural Netw. 11, 6, 1332--1346. Google Scholar
Digital Library
- J. C. Bezdek, J. Keller, and R. Krishnapuram. 1999. Fuzzy Models and Algorithms for Pattern Recognition and Image Processing. San Francisco: Kluwer Academic Publishers. Google ScholarDigital Library
- S. Bickel, M. Brückner, and T. Scheffer. 2007. Discriminative learning for differing training and test distributions. In Proc. 24th Int. Conf. Machine Learning. 81--88. Google ScholarDigital Library
- W. Dai, Q. Yang, G. Xue, and Y. Yu. 2007. Boosting for transfer learning. In Proc. 24th Int. Conf. Machine Learning. 193--200. Google ScholarDigital Library
- W. Dai, Q. Yang, G. Xue, and Y. Yu. 2008. Self-taught clustering. In Proc. 25th Int. Conf. Machine Learning. 200--207. Google ScholarDigital Library
- J. Davis and P. Domingos. 2008. Deep transfer via second-order markov logic. In Proc. Assoc. for the Advancement of Artificial Intelligence (AAAI’08) Workshop Transfer Learning for Complex Tasks.Google Scholar
- Z. H. Deng, K. S. Choi, F. L. Chung, and S. T. Wang. 2010. Enhanced soft subspace clustering integrating within-cluster and between-cluster information. Pattern Recogn. 43, 3, 767--781. Google ScholarDigital Library
- Z. H. Deng, K. S. Choi, F. L. Chung, and S. T. Wang. 2011. Scalable TSK fuzzy modeling for very large datasets using minimal-enclosing-ball approximation. IEEE Trans. Fuzzy Syst. 19, 2, 210--226. Google ScholarDigital Library
- Z. H. Deng, Y. Z. Jiang, F. L. Chung, H. Ishibuchi, and S. T. Wang. 2013a. Knowledge-leverage based fuzzy system and its modeling. IEEE Trans. Fuzzy Syst. 21, 4, 597--609. Google ScholarDigital Library
- Z. H. Deng, Y. Z. Jiang, K. S. Choi, F. L. Chung, and S. T. Wang. 2013b. Knowledge-leverage-based tsk fuzzy system modeling. IEEE Trans. Neural Netw. Learning Syst. 24, 8, 1200--1212.Google ScholarCross Ref
- Z. H. Deng, Y. Z. Jiang, L. B. Cao, and S. T. Wang. 2014. Knowledge-leverage based TSK fuzzy system with improved knowledge transfer. In Proceedings of FUZZ-IEEE 2014.Google Scholar
- L. X. Duan, I. W. Tsang, and D. Xu. 2012a. Domain transfer multiple kernel learning. IEEE Trans. Pattern Anal. Mach. Intell. 34, 3, 465--479. Google ScholarDigital Library
- L. X. Duan, D. Xu, and I. W. Tsang. 2012b. Domain adaptation from multiple sources: A domain-dependent regularization approach. IEEE Trans. Neural Netw. Learning Syst. 23, 3, 504--518.Google ScholarCross Ref
- R. E. Fan, P. H. Chen, and C. J. Lin. 2005. Working set selection using second order information for training support vector machines. J. Mach. Learning Res. 6, 1889--1918. Google ScholarDigital Library
- J. Gao, W. Fan, J. Jiang, and J. Han. 2008. Knowledge transfer via multiple model local structure mapping. In Proc. 14th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining. 283--291. Google ScholarDigital Library
- B. Gong, Y. Shi, F. Sha, and K. Grauman. 2012. Geodesic flow kernel for unsupervised domain adaptation. In Proc. 2012 IEEE Conf. Computer Vision and Pattern Recognition (CVPR’12). IEEE, 2066--2073. Google ScholarDigital Library
- J. Huang, A. Smola, A. Gretton, K. M. Borgwardt, and B. Schölkopf. 2007. Correcting sample selection bias by unlabeled data. In Proc. 19th Ann. Conf. Neural Information Processing Systems. Google ScholarDigital Library
- J. S. R. Jang. 1993. ANFIS: Adaptive-network-based fuzzy inference system. IEEE Trans. Systems, Man and Cybernetics 23, 3, 665--685.Google ScholarCross Ref
- J. S. R. Jang, C. T. Sun, and E. Mizutani. 1997. Neuro-Fuzzy and Soft-Computing. Upper Saddle River, NJ: Prentice-Hall. Google ScholarDigital Library
- W. H. Jiang and F. L. Chung. 2012. Transfer spectral clustering. In Proc. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD’12).Google Scholar
- L. P. Jing, M. K. Ng, and Z. X. Huang. 2007. An entropy weighting k-means algorithm for subspace clustering of high-dimensional sparse data. IEEE Trans. Knowl. Data Eng. 19, 8, 1026--1041. Google ScholarDigital Library
- C. F. Juang, S. H. Chiu, and S. J. Shiu. 2007. Fuzzy system learned through fuzzy clustering and support vector machine for human skin color segmentation. IEEE Trans. Syst. Man Cybernetics 37, 6, 1077--1087. Google ScholarDigital Library
- C. F. Juang and C. D. Hsieh. 2009. TS-fuzzy system-based support vector regression. Fuzzy Sets Syst. 160, 17, 2486--2504. Google ScholarDigital Library
- N. D. Lawrence and J. C. Platt. 2004. Learning to learn with the informative vector machine. In Proc. 21st Int. Conf. Machine Learning. Google ScholarDigital Library
- J. Leski. 2005. TSK-fuzzy modeling based on ϵ-insensitive learning. IEEE Trans. Fuzzy Syst. 13, 2, 181--193. Google ScholarDigital Library
- X. Liao, Y. Xue, and L. Carin. 2005. Logistic regression with an auxiliary data source. In Proc. 21st Int. Conf. Machine Learning. 505--512. Google ScholarDigital Library
- M. Long, J. Wang, G. Ding, S. J. Pan, and P. S. Yu. 2014. Adaptation regularization: A general framework for transfer learning. IEEE Trans. Knowl. Data Eng. 26, 5, 1076--1089. Google ScholarDigital Library
- M. Long, J. Wang, J. Sun, and P. S. Yu. 2015. Domain invariant transfer kernel learning. IEEE Trans. Knowl. Data Eng. 27, 6, 1519--1532.Google ScholarDigital Library
- J. Lu, V. Behbood, P. Hao, H. Zuo, S. Xue, and G. Zhang. 2015. Transfer learning using computational intelligence: A survey. Knowl. Based Syst. 80, 14--23. Google ScholarDigital Library
- E. Lughofer and S. Kindermann. 2010. SparseFIS: Data-driven learning of fuzzy systems with sparsity constraints. IEEE Trans. Fuzzy Syst. 18, 2, 396--411. Google ScholarDigital Library
- E. H. Mamdani. 1977. Application of fuzzy logic to approximate reasoning using linguistic synthesis. IEEE Trans. Comput. C-26, 12, 1182--1191. Google ScholarDigital Library
- J. M. Mendel. 2001. Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions. Upper Saddle River, NJ: Prentice-Hall.Google Scholar
- L. Mihalkova, T. Huynh, and R. J. Mooney. 2007. Mapping and revising Markov logic networks for transfer learning. In Proc. 22nd Assoc. for the Advancement of Artificial Intelligence (AAAI) Conf. Artificial Intelligence. 608--614. Google ScholarDigital Library
- L. Mihalkova and R. J. Mooney. 2008. Transfer learning by mapping with minimal target data. In Proc. Assoc. for the Advancement of Artificial Intelligence (AAAI’08) Workshop Transfer Learning for Complex Tasks.Google Scholar
- M. Oquab, L. Bottou, I. Laptev, and J. Sivic. 2014. Learning and transferring mid-level image representations using convolutional neural networks. In 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). IEEE, 1717--1724. Google ScholarDigital Library
- S. J. Pan and Q. Yang. 2010. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 10, 1345--1359. Google ScholarDigital Library
- S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang. 2011. Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22, 2, 199--210. Google ScholarDigital Library
- W. Pan and Q. Yang. 2013. Transfer learning in heterogeneous collaborative filtering domains. Artific. Intell. 197, 39--55. Google ScholarDigital Library
- N. Patricia and B. Caputo. 2014. Learning to learn, from transfer learning to domain adaptation: A unifying perspective. In 2014 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR’14). IEEE, 1442--1449. Google ScholarDigital Library
- B. Quanz and J. Huan. 2009. Large margin transductive transfer learning. In Proceedings of the 18th ACM Conference on Information and Knowledge Management. 1327--1336. Google ScholarDigital Library
- A. Schwaighofer, V. Tresp, and K. Yu. 2005. Learning Gaussian process kernels via hierarchical Bayes. In Proc. 17th Ann. Conf. Neural Information Processing Systems. 1209--1216. Google ScholarDigital Library
- L. Shao, F. Zhu, and X. Li. 2015. Transfer learning for visual categorization: A survey. IEEE Trans. Neural Netw. Learning Syst. 26, 5, 1019--1034.Google ScholarCross Ref
- M. Sugiyama, S. Nakajima, H. Kashima, P. V. Buenau, and M. Kawanabe. 2008. Direct importance estimation with model selection and its application to covariate shift adaptation. In Proc. 20th Ann. Conf. Neural Information Processing Systems. Google ScholarDigital Library
- T. Takagi and M. Sugeno. 1985. Fuzzy identification of systems and its application to modeling and control. IEEE Trans. Syst. Man Cybernetics 15, 1, 116--132.Google ScholarCross Ref
- J. W. Tao, K. F. L. Chung, and S. T. Wang. 2012. On minimum distribution discrepancy support vector machine for domain adaptation. Pattern Recogn. 45, 11, 3962--3984. Google ScholarDigital Library
- T. Tommasi, F. Orabona, and B. Caputo. 2014. Learning categories from few examples with multi model knowledge transfer. IEEE Trans. Pattern Anal. Mach. Intell. 36, 5, 928--941.Google ScholarCross Ref
- I. W. Tsang, J. T. Kwok, and J. M. Zurada. 2006. Generalized core vector machines. IEEE Trans. Neural Netw. 17, 5, 1126--1140. Google ScholarDigital Library
- Z. Wang, Y. Song, and C. Zhang. 2008. Transferred dimensionality reduction. In Proc. European Conf. Machine Learning and Knowledge Discovery in Databases (ECML/PKDD’08). 550--565.Google Scholar
- P. Yang, Q. Tan, and Y. Ding. 2008. Bayesian task-level transfer learning for non-linear regression. In Proc. Int. Conf. on Computer Science and Software Engineering. 62--65. Google ScholarDigital Library
- C. Yang, Z. Deng, K. S. Choi, Y. Jiang, and S. Wang. 2014. Transductive domain adaptive learning for epileptic electroencephalogram recognition. Artif. Intell. Med. 62, 3, 165--177.Google ScholarCross Ref
- W. Zhang, R. Li, T. Zeng, Q. Sun, S. Kumar, J. Ye, and S. Ji. 2015. Deep model based transfer and multi-task learning for biological image analysis. In Proc. 21th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining. ACM, 1475--1484. Google ScholarDigital Library
- F. Zhuang, P. Luo, C. Du, Q. He, Z. Shi, and H. Xiong. 2014. Triplex transfer learning: Exploiting both shared and distinct concepts for text classification. IEEE Trans. Cybern. 44, 7, 1191--1203.Google ScholarCross Ref
- F. Zhuang, X. Cheng, P. Luo, S. J. Pan, and Q. He. 2015. Supervised representation learning: Transfer learning with deep autoencoders. In Proc. 24th International Conference on Artificial Intelligence. AAAI Press, 4119--4125. Google ScholarDigital Library
Index Terms
- Enhanced Knowledge-Leverage-Based TSK Fuzzy System Modeling for Inductive Transfer Learning
Recommendations
Bayesian zero-order TSK fuzzy system modeling
Display Omitted The proposed B-ZTSK-FS method can make antecedent and consequent parameters become interpretable and simultaneously tunable.The developed iterative sampling algorithm can realize a fast construction of B-ZTSK-FS for a large-scale ...
TSK-fuzzy modeling based on ε-insensitive learning
In this paper, new learning methods tolerant to imprecision are introduced and applied to fuzzy modeling based on the Takagi-Sugeno-Kang fuzzy system. The fuzzy modeling has an intrinsic inconsistency. It may perform thinking tolerant to imprecision, ...
Center-of-gravity fuzzy systems based on normal fuzzy implications
In this paper, the construction and approximation problem of a single input and single output (SISO) fuzzy system with normal implication and center-of-gravity defuzzifier is discussed. First, the method of a non-singleton fuzzifier for the input ...
Comments