Skip to main content
SpringerLink
Log in

Face expression recognition system based on ripplet transform type II and least square SVM

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper discusses the development of an efficient and automated system for the recognition of facial expressions, which is essentially an application augmented with many multimedia computing systems. The proposed scheme works in three stages. In the first stage, ripplet transform type II (ripplet-II) is employed to extract the features from facial images because of its efficiency in representing edges and textures. In the next stage, a principal component analysis (PCA)+linear discriminant analysis (LDA) approach is utilized to obtain a more compact and discriminative feature set. In the final stage, classification is performed using a least squares variant of support vector machine (LS-SVM) with radial basis function (RBF) kernel. The proposed system is validated on two benchmark datasets namely the Extended Cohn-Kanade (CK + ) and Japanese female facial expression (JAFFE). The experimental results demonstrate that the propose system yields superior performance as compared to other state-of-the-art schemes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Abbreviations

FER:

Face expression recognition

Ripplet-II:

Ripplet transform type II

CK+:

Extended Cohn-Kanade

JAFFE:

Japanese female facial expression

PCA:

Principal component analysis

LDA:

Linear discriminant analysis

HOG:

Histograms of Oriented Gradient

DWT:

Discrete wavelet transform

SWT:

Statyionary wavelet transform

RBF:

Radial basis function

SVM:

Support vector machine

LS-SVM:

Least square support vector machine

References

  1. Bartlett M S, Littlewort G, Frank M, Lainscsek C, Fasel I, Movellan J (2006) Fully automatic facial action recognition in spontaneous behavior. In: 7th international conference on automatic face and gesture recognition. IEEE, pp 223–230. https://doi.org/10.1109/FGR.2006.55

  2. Ben-Hur A, Horn D, Siegelmann HT, Vapnik V (2002) Support vector clustering. J Mach Learn Res 2:125–137

    MATH  Google Scholar 

  3. Bettadapura V (2012) Face expression recognition and analysis: the state of the art. arXiv:1203.6722

  4. Bishop CM (2006) Pattern recognition and machine learning. Springer

  5. Chen J, Chen D, Gong Y, Yu M, Zhang K, Wang L (2012) Facial expression recognition using geometric and appearance features. In: Proceedings of the 4th international conference on internet multimedia computing and service. ACM, pp 29–33. https://doi.org/10.1145/2382336.2382345

  6. Chen J, Chen Z, Chi Z, Fu H (2014) Facial expression recognition based on facial components detection and HOG features. In: International workshops on electrical and computer engineering subfields

  7. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297. https://doi.org/10.1007/BF00994018

    MATH  Google Scholar 

  8. Dahmane M, Meunier J (2011) Emotion recognition using dynamic grid-based HoG features. In: International conference on automatic face & gesture recognition and workshops (FG 2011). IEEE, pp 884–888. https://doi.org/10.1109/FG.2011.5771368

  9. Deshmukh S, Patwardhan M, Mahajan A (2016) Survey on real-time facial expression recognition techniques. IET Biom 5(3):155–163. https://doi.org/10.1049/iet-bmt.2014.0104

    Article  Google Scholar 

  10. Dongcheng S, Jieqing J (2010) The method of facial expression recognition based on DWT-PCA/LDA. In: International congress on image and signal processing (CISP), vol 4. IEEE, pp 1970–1974. https://doi.org/10.1109/CISP.2010.5648166

  11. Ekman P, Friesen W V (1971) Constants across cultures in the face and emotion. J Pers Soc Psychol 17(2):124. https://doi.org/10.1037/h0030377

    Article  Google Scholar 

  12. Elaiwat S, Bennamoun M, Boussaid F (2016) A spatio-temporal RBM-based model for facial expression recognition. Pattern Recogn 49:152–161. https://doi.org/10.1016/j.patcog.2015.07.006

    Article  Google Scholar 

  13. Gandhi T, Trivedi MM (2008) Image based estimation of pedestrian orientation for improving path prediction. In: The proceedings of the IEEE intelligent vehicles symposium, pp 506–511. https://doi.org/10.1049/10.1109/IVS.2008.4621257

  14. Gritti T, Shan C, Jeanne V, Braspenning R (2008) Local features based facial expression recognition with face registration errors. In: 8th IEEE international conference on automatic face & gesture recognition, FG’08. IEEE, pp 1–8. https://doi.org/10.1109/AFGR.2008.4813379

  15. Guo M, Hou X, Ma Y, Wu X (2016) Facial expression recognition using ELBP based on covariance matrix transform in KLT. Multimedia Tools and Applications 1–16. https://doi.org/10.1007/s11042-016-3282-9

  16. Happy SL, Routray A (2015) Automatic facial expression recognition using features of salient facial patches. IEEE Trans Affect Comput 6(1):1–12. https://doi.org/10.1109/TAFFC.2014.2386334

    Article  Google Scholar 

  17. Hsieh C-C, Hsih M-H, Jiang M-K, Cheng Y-M, Liang E-H (2015) Effective semantic features for facial expressions recognition using SVM. Multimedia Tools and Applications 1–20. https://doi.org/10.1007/s11042-015-2598-1

  18. Jung H, Lee S, Park S, Lee I, Ahn C, Kim J (2015) Deep temporal appearance-geometry network for facial expression recognition. arXiv:1503.01532

  19. Kar NB, Babu KS, Jena SK (2017) Face expression recognition using histograms of oriented gradients with reduced features. In: Proceedings of international conference on computer vision and image processing. Springer, pp 209–219. https://doi.org/10.1007/978-981-10-2107-7_19

  20. Kazmi SB, Jaffar MA et al (2012) Wavelets-based facial expression recognition using a bank of support vector machines. Soft Comput 16(3):369–379. https://doi.org/10.1007/s00500-011-0721-4

    Article  Google Scholar 

  21. Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The Extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer society conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 94–101. https://doi.org/10.1109/CVPRW.2010.5543262

  22. Lyons M, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with Gabor wavelets. In: Third IEEE international conference on automatic face and gesture recognition. IEEE, pp 200–205. https://doi.org/10.1109/AFGR.1998.670949

  23. Martínez AM, Kak AC (2001) PCA Versus LDA. IEEE Trans Pattern Anal Mach Intell 23(2):228–233. https://doi.org/10.1109/34.908974

    Article  Google Scholar 

  24. Michel P, El Kaliouby R (2003) Real time facial expression recognition in video using support vector machines. In: Proceedings of the 5th international conference on multimodal interfaces. ACM, pp 258–264. https://doi.org/10.1145/958432.958479

  25. Mlakar U, Potočnik B (2015) Automated facial expression recognition based on histograms of oriented gradient feature vector differences. SIViP 1–9. https://doi.org/10.1007/s11760-015-0810-4

  26. Moraes D, Wainer J, Rocha A (2016) Low false positive learning with support vector machines. J Vis Commun Image Represent 38:340–350. https://doi.org/10.1016/j.jvcir.2016.03.007

    Article  Google Scholar 

  27. Nayak DR, Dash R, Majhi B (2016) Brain mr image classification using two-dimensional discrete wavelet transform and adaboost with random forests. Neurocomputing 177:188–197. https://doi.org/10.1016/j.neucom.2015.11.034

    Article  Google Scholar 

  28. Nayak DR, Dash R, Majhi B (2017) Stationary wavelet transform and adaboost with svm based pathological brain detection in mri scanning. CNS Neurol Disord Drug Targets (Formerly Current Drug Targets-CNS & Neurological Disorders) 16(2):137–149. https://doi.org/10.2174/1871527315666161024142036

    Article  Google Scholar 

  29. Qayyum H, Majid M, Anwar S M, Khan B (2017) Facial expression recognition using stationary wavelet transform features. Math Probl Eng 2017. https://doi.org/10.1155/2017/9854050

  30. Saeed A, Al-Hamadi A, Niese R, Elzobi M (2014) Frame-based facial expression recognition using geometrical features. Advances in Human-Computer Interaction 2014:4

    Article  Google Scholar 

  31. Shimizu H, Poggio T (2003) Direction estimation of pedestrian from images. In: AI Memo 2003-020, Massachusetts Institute of Technology, pp 1–11

  32. Siddiqi MH, Lee S (2013) Human facial expression recognition using wavelet transform and hidden markov model. In: International workshop on ambient assisted living. Springer, pp 112–119. https://doi.org/10.1007/978-3-319-03092-0

  33. Siddiqi MH, Ali R, Khan AM, Park Y-T, Lee S (2015) Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields. IEEE Trans Image Process 24(4):1386–1398. https://doi.org/10.1109/TIP.2015.2405346

    Article  MathSciNet  MATH  Google Scholar 

  34. Sun Y, Kamel MS, Wong AKC, Wang Y (2007) Cost-sensitive boosting for classification of imbalanced data. Pattern Recogn 40(12):3358–3378. https://doi.org/10.1016/j.patcog.2007.04.009

    Article  MATH  Google Scholar 

  35. Suwa M, Sugie N, Fujimora K (1978) A preliminary note on pattern recognition of human emotional expression. In: International joint conference on pattern recognition, pp 408–410

  36. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300. https://doi.org/10.1023/A:1018628609742

    Article  Google Scholar 

  37. Theodoridis S, Koutroumbas K (1999) Pattern recognition. Academic Press, New York

    MATH  Google Scholar 

  38. Tsai H-H, Lai Y-S, Zhang Y-C (2010) Using SVM to design facial expression recognition for shape and texture features. In: International conference on machine learning and cybernetics, vol 5. IEEE, pp 2697–2704. https://doi.org/10.1109/ICMLC.2010.5580938

  39. Uçar A, Demir Y, Güzeliş C (2016) A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering. Neural Comput & Applic 27(1):131–142. https://doi.org/10.1007/s00521-014-1569-1

    Article  Google Scholar 

  40. Uddin MZ (2016) A depth video-based facial expression recognition system utilizing generalized local directional deviation-based binary pattern feature discriminant analysis. Multimedia Tools and Applications 75(12):6871–6886. https://doi.org/10.1007/s11042-015-2614-5

    Article  Google Scholar 

  41. Uddin MZ (2016) Facial expression recognition using depth information and spatiotemporal features. In: 18th international conference on advanced communication technology (ICACT), 2016. IEEE, pp 726–731. https://doi.org/10.1109/ICACT.2016.7423536

  42. Uddin MZ, Hassan MM, Almogren A, Alamri A, Alrubaian M, Fortino G (2017) Facial expression recognition utilizing local direction-based robust features and deep belief network. IEEE Access 5:4525–4536. https://doi.org/10.1109/ACCESS.2017.2676238

    Article  Google Scholar 

  43. Valstar MF, Pantic M (2012) Fully automatic recognition of the temporal phases of facial actions. IEEE Trans Syst Man Cybern B Cybern 42(1):28–43. https://doi.org/10.1109/TSMCB.2011.2163710

    Article  Google Scholar 

  44. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, vol 1, pp I–511. https://doi.org/10.1109/CVPR.2001.990517

  45. Wang S-H, Phillips P, Dong Z-C, Zhang Y-D (2017) Intelligent facial emotion recognition based on stationary wavelet entropy and jaya algorithm. Neurocomputing. https://doi.org/10.1016/j.neucom.2017.08.015

  46. Wang X, Jin C, Liu W, Hu M, Xu L, Ren F (2013) Feature fusion of HOG and WLD for facial expression recognition. In: International symposium on system integration (SII). IEEE, pp 227–232. https://doi.org/10.1109/SII.2013.6776664

  47. Wu SH, Lin KP, Chien HH, Chen CM, Chen MS (2013) On generalizable low false-positive learning using asymmetric support vector machines. IEEE Trans Knowl Data Eng 25(5):1083–1096. https://doi.org/10.1109/TKDE.2012.46

    Article  Google Scholar 

  48. Xu J, Wu D (2008) Ripplet transform for feature extraction. In: SPIE defense and security symposium, pages 69700x–69700x. International society for optics and photonics. https://doi.org/10.1117/12.777302

  49. Xu J, Wu D (2012) Ripplet transform type II transform for feature extraction. IET Image Process 6(4):374–385. https://doi.org/10.1049/iet-ipr.2010.0225

    Article  MathSciNet  Google Scholar 

  50. Yu H, Yang J (2001) A direct LDA algorithm for high-dimensional data—with application to face recognition. Pattern Recogn 34(10):2067–2070

    Article  Google Scholar 

  51. Zhang L, Tjondronegoro D (2011) Facial expression recognition using facial movement features. IEEE Trans Affect Comput 2(4):219–229. https://doi.org/10.1109/T-AFFC.2011.13

    Article  Google Scholar 

  52. Zhang Y-D, Yang Z-J, Lu H-M, Zhou X -X, Phillips P, Liu Q-M, Wang S-H (2016) Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine, and stratified cross validation. IEEE Access 4:8375–8385. https://doi.org/10.1109/ACCESS.2016.2628407

    Article  Google Scholar 

Download references

Acknowledgments

This research is partially supported by the following project: Grant No. ETI/359/2014 by Fund for Improvement of S&T Infrastructure in Universities and Higher Educational Institutions (FIST) Program 2016, Department of Science and Technology, Government of India.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology, Rourkela, Odisha, 769 008, India

    Nikunja Bihari Kar, Korra Sathya Babu & Sambit Bakshi

  2. School of Computing Science and Engineering, VIT University, Vellore, Tamil Nadu, 632 014, India

    Arun Kumar Sangaiah

Authors
  1. Nikunja Bihari Kar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Korra Sathya Babu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arun Kumar Sangaiah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sambit Bakshi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikunja Bihari Kar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kar, N.B., Babu, K.S., Sangaiah, A.K. et al. Face expression recognition system based on ripplet transform type II and least square SVM. Multimed Tools Appl 78, 4789–4812 (2019). https://doi.org/10.1007/s11042-017-5485-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5485-0

Keywords

Search

Navigation