Snehasis Mukherjee
Bandla Vamshi
ABSTRACT
This paper introduces two novel motion based features for recognizing human facial expressions. The proposed motion features are applied for recognizing facial expressions from a video sequence. The proposed bag-of-words based scheme represents each frame of a video sequence as a vector depicting local motion patterns during a facial expression. The local motion patterns are captured by an efficient derivation from optical flow. Motion features are clustered and stored as words in a dictionary. We further generate a reduced dictionary by ranking the words based on some ambiguity measure. We prune out the ambiguous words and continue with key words in the reduced dictionary. The ambiguity measure is given by applying a graph-based technique, where each word is represented as a node in the graph. Ambiguity measures are obtained by modelling the frequency of occurrence of the word during the expression. We form expression descriptors for each expression from the reduced dictionary, by applying an efficient kernel. The training of the expression descriptors are made following an adaptive learning technique. We tested the proposed approach with standard dataset. The proposed approach shows better accuracy compared to the state-of-the-art.
- S. Agarwal and D.P. Mukherjee. Facial Expression Recognition through Adaptive Learning of Local Motion Descriptor, Multimedia Tools and Applications, Springer, pp.- 1--27, 2015Google Scholar
- M.E. Hoque, M. Courgeon, J.C. Martin, B. Mutlu and R.W. Picard. Mach: My automated conversation coach. Proc. of UbiComp, ACM, pp. 697--706, 2013. Google ScholarDigital Library
- G. Zhao and M. Pietikinen. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6), pp.- 915--928, 2007. Google ScholarDigital Library
- P. Ekman, W.V. Friesen and J.C. Hager. Facial Action Coding System: The Manual on CD ROM. A Human Face, Salt Lake City, 2002.Google Scholar
- H. Wang, A. Klaser, C. Schmid and C.-L. Liu. Dense trajectories and motion boundary descriptors for action recognition. International Journal of Computer Vision, Springer, 103(1), pp.- 60--79, 2013.Google Scholar
- S. Mukherjee, S.K. Biswas and D.P. Mukherjee. Recognizing Human Action at a Distance in Video by Key Poses. IEEE Transactions on Circuits and Systems for Video Technology, 21(9), pp.- 1228--1241, 2011.Google ScholarCross Ref
- S. Mukherjee, S.K. Biswas and D.P. Mukherjee. Recognizing Interactions Between Human Performers by 'Dominating Pose Doublet'. Machine Vision and Applications, Springer, 25(4), pp.- 1033--1052, 2014. Google ScholarDigital Library
- T. Wu, S. Fu and G. Yang. Survey of the Facial Expression Recognition Research. Proc. of International Conference on Brain Inspired Cognitive System (BICS), pp.- 392--402, 2012.Google ScholarCross Ref
- S. Jain, C. Hu, J.K. Aggarwal. Facial expression recognition with temporal modeling of shapes. Proc. of IEEE International Conference on Computer Vision (ICCV) Workshops, pp.- 1642--1649, 2011.Google ScholarCross Ref
- F.S. Hsu, W.Y. Lin and T.W. Tsai. Facial expression recognition using bag of distances. Multimedia Tools and Applications, Springer, 73(1), pp.- 309--326, 2014. Google ScholarDigital Library
- Z. Zhang, M.J. Lyons, M. Schuster and S. Akamatsu. Comparison between geometry-based and gabor-wavelets-based facial expression recognition using multi-layer perceptron. Proc. of IEEE International Conference on Automatic Face and Gesture Recognition (AFGR), pp.- 454--459, 1998. Google ScholarDigital Library
- T. Wu, M.S. Bartlett and J.R. Movellan. Facial expression recognition using Gabor motion energy filters. Proc. of IEEE Computer Vision and Pattern Recognition (CVPR) Workshops, pp.- 42--47, 2010.Google ScholarCross Ref
- P. Martins, J. Sampaio and J. Batista. Facial Expression Recognition Using Active Appearance Models. Proc. of IEEE International Conference on Computer Vision Theory and Applications (VISAPP), pp.- 123--129, 2008.Google Scholar
- C. Shan, S. Gong and P.W. McOwan. Facial expression recognition based on Local Binary Patterns: A comprehensive study. Image and Vision Computing, 27(2009), pp.- 803--816, 2009. Google ScholarDigital Library
- A. Vo and N.Q. Ly. Facial Expression Recognition Using Pyramid Local Phase Quantization Descriptor. Proc. of International Conference on Knowledge and Systems Engineering (KSE), pp.- 105--115, 2014.Google Scholar
- B. Jiang, M.F. Valstar and M. Pantic. Action unit detection using sparse appearance descriptors in space-time video volumes. Proc. of IEEE International Conference on Automatic Face and Gesture Recognition (AFGR), pp.- 314--321, 2011.Google ScholarCross Ref
- M. Tang and F. Chen. Facial expression recognition and its application based on curvelet transform and PSO-SVM. Optik - International Journal for Light and Electron Optics, 124(22), pp.- 5401--5406, 2013.Google Scholar
- C.C. Hsieh, M.H. Hsih, M.K. Jiang, Y.M. Cheng and E.H. Liang. Effective semantic features for facial expressions recognition using svm. Multimedia Tools and Applications, Springer, pp.- 1--20, 2015. Google ScholarDigital Library
- Y. Li, S. Wang, Y. Zhao and Q. Ji. Simultaneous facial feature tracking and facial expression recognition. IEEE Transactions on Image Processing, 22(7), pp.- 2559--2573, 2013.Google ScholarCross Ref
- M. Bejani, D. Gharavian and N.M. Charkari. Audiovisual emotion recognition using anova feature selection method and multi-classifier neural networks. Neural Computing and Applications, Springer, 24(2), pp.- 399--412, 2014.Google Scholar
- H. Boughrara, M. Chtourou, C.B. Amar and L. Chen. Facial expression recognition based on a mlp neural network using constructive training algorithm. Multimedia Tools and Applications, Springer, 75(2), pp.- 709--731, 2016. Google ScholarDigital Library
- P. Lucey, J.F. Cohn, T. Kanade and J. Saragih. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. Proc. of IEEE Computer Vision and Pattern Recognition (CVPR), pp.- 94--101, 2010.Google ScholarCross Ref
- B.L. Narayan, C.A. Murthy and S.K. Pal. Maxdiff kd-trees for data condensation. Pattern Recognition Letters, Elsevier, 27(3), pp.- 187--200, 2006. Google ScholarDigital Library
- T.H. Cormen, C.E. Leiserson, R.L. Rivest and C. Stein. Introduction to Algorithms. MIT Press, Cambridge, 2003. Google ScholarDigital Library
- T. Kanade, Y. Tian and J.F. Cohn. Comprehensive database for facial expression analysis. Proc. of IEEE International Conference on Automatic Face and Gesture Recognition (AFGR), pp.- 46--53, 2000. Google ScholarDigital Library
Index Terms
- Recognizing facial expressions using novel motion based features
Recommendations
Recognising facial expressions in video sequences
We introduce a system that processes a sequence of images of a front-facing human face and recognises a set of facial expressions. We use an efficient appearance-based face tracker to locate the face in the image sequence and estimate the deformation of ...
Image ratio features for facial expression recognition application
Special issue on game theoryVideo-based facial expression recognition is a challenging problem in computer vision and human-computer interaction. To target this problem, texture features have been extracted and widely used, because they can capture image intensity changes raised ...
Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion
This paper explores the use of local parametrized models of image motion for recovering and recognizing the non-rigid and articulated motion of human faces. Parametric flow models (for example affine) are popular for estimating motion in rigid scenes. We ...
Comments