ABSTRACT
Since the emergence of extensive multimedia data, feature fusion has been more and more important for image and video retrieval, indexing and annotation. Existing feature fusion techniques simply concatenate a pair of different features or use canonical correlation analysis based methods for joint dimensionality reduction in the feature space. However, how to fuse multiple features in a generalized way is still an open problem. In this paper, we reformulate the multiple feature fusion as a general subspace learning problem. The objective of the framework is to find a general linear subspace in which the cumulative pairwise canonical correlation between every pair of feature sets is maximized after the dimension normalization and subspace projection. The learned subspace couples dimensionality reduction and feature fusion together, which can be applied to both unsupervised and supervised learning cases. In the supervised case, the pairwise canonical correlations of feature sets within the same classes are also counted in the objective function for maximization. To better model the high-order feature structure and overcome the computational difficulty, the features extracted from the same pattern source are represented by a single 2D tensor. The tensor-based dimensionality reduction methods are used to further extract low-dimensional discriminative features from the fused feature ensemble. Extensive experiments on visual data classification demonstrate the effectiveness and robustness of the proposed methods.
- T. Ahonen, A. Hadid, and M. Pietikainen. Face recognition with local binary patterns. In European Conference on Computer Vision, pages 469--481, 2004.Google ScholarCross Ref
- T. Ahonen, A. Hadid, and M. Pietikainen. Face description with local binary patterns: Application to face recognition. IEEE Trans. on PAMI, 28(12):2037--2041, 2006. Google ScholarDigital Library
- M. Barker and W. Rayens. Partial least squares for discrimination. Journal of Chemometrics, 17(3):166--173, 2003.Google ScholarCross Ref
- P. Belhumeur, J. Hespanha, and D. Kriegman. Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. on PAMI, 19(7):711--720, 1997. Google ScholarDigital Library
- A. Bosch, A. Zisserman, and X. Munoz. Representing shape with a spatial pyramid kernel. In Proc. of ACM CIVR, pages 401--408, 2007. Google ScholarDigital Library
- H.-T. Chen, H.-W. Chang, and T.-L. Liu. Local discriminant embedding and its variants. In IEEE Conf. on CVPR, volume 2, pages 846--853, 2005. Google ScholarDigital Library
- N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In IEEE Conf. on CVPR, pages 886--893, 2005. Google ScholarDigital Library
- N. Dalal and B. Triggs. Object detection using histograms of oriented gradients. In European Conference on Computer Vision, Workshop on Pascal VOC'06, 2006.Google Scholar
- Y. Fang, T. Tan, and Y. Wang. Fusion of global and local features for face verification. In IEEE Conf. on ICPR, pages 382--385, 2002.Google Scholar
- Y. Fu and T. S. Huang. Image classification using correlation tensor analysis. IEEE Trans. on Image Processing, 17(2):226--234, 2008. Google ScholarDigital Library
- Y. Fu, M. Liu, and T. Huang. Conformal embedding analysis with local graph modeling on the unit hypersphere. In IEEE Conf. on CVPR, workshop on Component Analysis, 2007.Google ScholarCross Ref
- A. Georghiades, P. Belhumeur, and D. Kriegman. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. on PAMI, 23(6):643--660, 2001. Google ScholarDigital Library
- T.-K. Kim, O. Arandjelovic, and R. Cipolla. Learning over sets using boosted manifold principal angles (bompa). In British Machine Vision Conference, pages 779--788, 2005.Google ScholarCross Ref
- T.-K. Kim, J. Kittler, and R. Cippola. Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans. on PAMI, 29(56):1005--1018, 2007. Google ScholarDigital Library
- T.-K. Kim, S.-F. Wong, and R. Cipolla. Tensor canonical correlation analysis for action classification. In IEEE Conf. on CVPR, 2007.Google ScholarCross Ref
- K.-C. Lee, J. Ho, and D. Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. on PAMI, 27(5):684--698, 2005. Google ScholarDigital Library
- M. Liu, Y. Fu, and T. S. Huang. An audio-visual fusion framework with joint dimensionality reduction. In IEEE Conf. on ICASSP, 2008.Google Scholar
- T. Ojala, M. Pietikainen, and T. Maenpaa. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. on PAMI, 24(7):971--987, 2002. Google ScholarDigital Library
- P. Phillips, P. Flynn, T. Scruggs, K. Bowyer, J. Chang, K. Hoffman, J. Marques, J. Min, and W. Worek. Overview of the face recognition grand challenge. In IEEE Conf. on CVPR, pages 947--954, 2005. Google ScholarDigital Library
- K. S. Rao and A. N. Rajagopalan. A probabilistic fusion methodology for face recognition. EURASIP Journal on Applied Signal Processing, 2005(17):2772--2787, 2005. Google ScholarDigital Library
- M. Sargin, Y. Yemez, E. Erzin, and A. Tekalp. Audiovisual synchronization and fusion using canonical correlation analysis. IEEE Trans. on Multimedia, 9(7):1396--1403, 2007. Google ScholarDigital Library
- T. Sim, S. Baker, and M. Bsat. The cmu pose, illuminlation, and expression database. IEEE Trans. on PAMI, 25(12):1615--1618, 2003. Google ScholarDigital Library
- Q.-S. Sun, Z. Jin, P.-A. Heng, and D.-S. Xia. A novel feature fusion method based on partial least squares regression. Lecture Notes in Computer Science 3686, 3686/2005:268--277, 2005. Google ScholarDigital Library
- Q.-S. Sun, S.-G. Zeng, Y. Liu, P.-A. Heng, and D.-S. Xia. A new method of feature fusion and its application in image recognition. Pattern Recognition, 38(12):2437--2448, 2005. Google ScholarDigital Library
- X. Tan and B. Triggs. Enhanced local texture feature sets for face recognition under difficult lighting conditions. In IEEE Conf. on AMFG, pages 168--182, 2007. Google ScholarDigital Library
- M. Turk and A. Pentland. Face recognition using eigenfaces. In IEEE Conf. on CVPR, pages 586--591, 1991.Google ScholarCross Ref
- P. Viola and M. Jones. Robust real-time face detection. Int'l Journal of Computer Vision, 57(2):137--154, 2004. Google ScholarDigital Library
- X. Wang and X. Tang. Using random subspace to combine multiple features for face recognition. In IEEE Conf. on FGR, pages 284--289, 2004. Google ScholarDigital Library
- L. Wolf and A. Shashua. Learning over sets using kernel principal angles. Journal of Machine Learning Research, 4:913--931, 2003. Google ScholarDigital Library
- S. Yan, D. Xu, Q. Yang, L. Zhang, X. Tang, and H.-J. Zhang. Discriminant analysis with tensor representation. In IEEE Conf. on CVPR, pages 526--532, 2005. Google ScholarDigital Library
- S. Yan, D. Xu, B. Zhang, H.-J. Zhang, Q. Yang, and S. Lin. Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Trans. on PAMI, 29(1):40--51, 2007. Google ScholarDigital Library
- J. Yang, J.-Y. Yang, D. Zhang, and J.-F. Lu. Feature fusion: parallel strategy vs. serial strategy. Pattern Recognition, 36(6):1369--1381, 2003.Google ScholarCross Ref
- J. Zhao, H. Wang, H. Ren, and S.-C. Kee. Lbp discriminant analysis for face verification. In IEEE Conf. on CVPR, pages 167--167, 2005. Google ScholarDigital Library
- X. Zhou and B. Bhanu. Feature fusion of face and gait for human recognition at a distance in video. In IEEE Conf. on ICPR, pages 529--532, Washington, DC, USA, 2006. Google ScholarDigital Library
- Q. Zhu, S. Avidan, M. Yeh, and K. Cheng. Fast human detection using a cascade of histograms of oriented gradients. In IEEE Conf. on CVPR, pages 886--893, 2005. Google ScholarDigital Library
Index Terms
- Multiple feature fusion by subspace learning
Recommendations
Subspace manifold learning with sample weights
Subspace manifold learning represents a popular class of techniques in statistical image analysis and object recognition. Recent research in the field has focused on nonlinear representations; locally linear embedding (LLE) is one such technique that ...
Locality preserving discriminant projections for face and palmprint recognition
A new subspace learning algorithm called locality preserving discriminant projections (LPDP) is proposed by adding the criterion of maximum margin criterion (MMC) into the objective function of locality preserving projections (LPP). LPDP retains the ...
Feature Fusion Using Multiple Component Analysis
Canonical correlation analysis (CCA) and partial least squares (PLS) are always used as fusing two feature sets. How to extend them to fuse multiple features in a generalized way is still an unsolved problem. In this paper, we propose a novel feature ...
Comments