Skip to main content
SpringerLink
Log in

Performing content-based retrieval of humans using gait biometrics

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In order to analyse surveillance video, we need to efficiently explore large datasets containing videos of walking humans. Effective analysis of such data relies on retrieval of video data which has been enriched using semantic annotations. A manual annotation process is time-consuming and prone to error due to subject bias however, at surveillance-image resolution, the human walk (their gait) can be analysed automatically. We explore the content-based retrieval of videos containing walking subjects, using semantic queries. We evaluate current research in gait biometrics, unique in its effectiveness at recognising people at a distance. We introduce a set of semantic traits discernible by humans at a distance, outlining their psychological validity. Working under the premise that similarity of the chosen gait signature implies similarity of certain semantic traits we perform a set of semantic retrieval experiments using popular Latent Semantic Analysis techniques. We perform experiments on a dataset of 2000 videos of people walking in laboratory conditions and achieve promising retrieval results for features such as Sex (mAP  =  14% above random), Age (mAP  =  10% above random) and Ethnicity (mAP  =  9% above random).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. 25 frames per second using 352×288 CIF images compressed using MPEG4 (http://www.info4security.com/story.asp?storyCode=3093501).

  2. In practice several r values are attempted to choose an optimal number of concepts for a given dataset.

  3. i.e. only semantic terms, visual terms set to 0

  4. i.e. only visual terms, semantic terms set to 0

  5. chosen to disregard scaling effects Papadimitriou et al. [41]

  6. http://www.statistics.gov.uk/about/Classifications/ns_ethnic_classification.asp Ethnic classification.

References

  1. Aggarwal JK, Cai Q (1999) Human motion analysis: a review. Comput Vis Image Underst 73(3):428–440

    Article  Google Scholar 

  2. Barbujani G (2005) Human races: classifying people vs understanding diversity. Current Genomics 6(12):215–226

    Article  Google Scholar 

  3. BenAbdelkader C, Cutler R, Davis L (2002) Stride and cadence as a biometric in automatic person identification and verification. In: Proc. IEEE FG, pp 372–377

  4. Bennetto J (2006) Big brother Britain 2006: we are waking up to a surveillance society all around us. In: The independent

  5. Berry MW, Dumais ST, O’brien GW, Berry MW (1995) Using linear algebra for intelligent information retrieval. SIAM Rev 37:573–595

    Article  MATH  MathSciNet  Google Scholar 

  6. Bertillon A (1896) Signaletic instructions including the theory and practice of anthropometrical identification. The Werner Company

  7. Bhanu B, Han J (2003) Human recognition on combining kinematic and stationary features. In: Proc. AVBPA, pp 600–608

  8. Bouchrika I, Goffredo M, Carter JN, Nixon MS (2009) Covariate analysis for view-point independent gait recognition. In: Proc. ICB

  9. Chapman GB, Johnson EJ (2002) Incorporating the irrelevant: anchors in judgments of belief and value. In: Heuristics and biases: the psychology of intuitive judgment. Cambridge University Press, Cambridge, pp 120–138

    Google Scholar 

  10. Davies AC, Velastin, SA (2005) A progress review of intelligent CCTV surveillance systems. In: Proc. IEEE IDAACS, pp 417–423

  11. Dawes RM (1977) Suppose we measured height with rating scales instead of rulers. Appl Psychol Meas 1(2):267–273

    Article  Google Scholar 

  12. Deerwester SC, Dumais ST, Landauer TK, Furnas GW, Harshman RA (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41(6):391–407

    Article  Google Scholar 

  13. Dumais SI (1991) Improving the retrieval of information from external sources. Behavior research methods. In: Instruments and computers, pp 229–236

  14. Ellis HD (1984) Practical aspects of facial memory. In: Eyewitness testimony: psychological perspectives, section 2. Cambridge University Press, Cambridge, pp 12–37

    Google Scholar 

  15. Flin RH, Shepherd JW (1986) Tall stories: eyewitnesses’ ability to estimate height and weight characteristics. Hum Learn 5

  16. Goffredo M, Seely RD, Carter JN, Nixon MS (2008) Markerless view independent gait analysis with self-camera calibration. In: Proc. IEEE FG

  17. Gould SJ (1994) The geometer of race. Discover 65–69

  18. Grosky W, Zhao R (2001) Negotiating the semantic gap: from feature maps to semantic landscapes. In: Proc. SOFSEM, pp 33–52

  19. Han J, Bhanu B (2004) Statistical feature fusion for gait-based human recognition. In: Proc. IEEE CVPR, vol 2, pp II–842–II–847

  20. Hare JS, Lewis PH, Enser PGB, Sandom CJ (2006) A linear-algebraic technique with an application in semantic image retrieval. In: Proc. CIVR, pp 31–40

  21. Hare JS, Samangooei S, Lewis PH, Nixon MS (2008) Semantic spaces revisited: investigating the performance of auto-annotation and semantic retrieval using semantic spaces. In: Proc. CIVR, New York, NY, USA. ACM, New York, pp 359–368

  22. Hayfron-Acquah JB, Nixon MS, Carter JN (2003) Automatic gait recognition by symmetry analysis. Pattern Recogn Lett 24(13):2175–2183

    Article  Google Scholar 

  23. Hu W, Tan T, Wang L, Maybank S (2004) A survey on visual surveillance of object motion and behaviors. IEEE Trans SMC(A) 34(3):334–352

    Google Scholar 

  24. Interpol (2008) Disaster victim identification form (yellow). Booklet

  25. Jain AK, Ross A, Prabhakar S (2004) An introduction to biometric recognition. IEEE Trans CSVT 14:4–19

    Google Scholar 

  26. Johansson G (1973) Visual perception of biological motion and a model for its analysis. Percept Phychophys 14(2):201–211

    Google Scholar 

  27. Kale A, Roychowdhury AK, Chellappa R (2004) Fusion of gait and face for human identification. In: Proc. IEEE ICASSP, vol 5, pp 901–904

  28. Li X, Maybank SJ, Yan S, Tao D, Xu D (2008) Gait components and their application to gender recognition. IEEE Trans SMC(C) 38(2):145–155

    MATH  Google Scholar 

  29. Lindsay RCL, Martin R, Webber L (1994) Default values in eyewitness descriptions. Law Hum Behav 18(5):527–541

    Article  Google Scholar 

  30. Little J, Boyd J (1995) Describing motion for recognition. In: Proc. ISCV, p 5A, Motion II

  31. Liu Z, Sarkar S (2004) Simplest representation yet for gait recognition: averaged silhouette. In: Proc. ICPR, vol 4, pp 211–214

  32. Liu Z, Sarkar S (2007) Outdoor recognition at a distance by fusing gait and face. Image Vis Comput 25(6):817–832

    Article  Google Scholar 

  33. MacLeod MD, Frowley JN, Shepherd JW (1994) Whole body information: its relevance to eyewitnesses. In: Adult eyewitness testimony, chapter 6. Cambridge University Press, Cambridge

    Google Scholar 

  34. Macrae CN, Bodenhausen GV (2000) Social cognition: thinking categorically about others. Annu Rev Psychol 51(1):93–120

    Article  Google Scholar 

  35. Monay F, Gatica-Perez D (2003) On image auto-annotation with latent space models. In: Proc. Multimedia, pp 275–278

  36. Murase H, Sakai R (1996) Moving object recognition in eigenspace representation: gait analysis and lip reading. Pattern Recogn Lett 17(2):155–162

    Article  Google Scholar 

  37. Nandakumar K, Dass SC, Jain AK (2004) Soft biometric traits for personal recognition systems. In: Proc. ICBA, pp 731–738

  38. Nixon MS, Carter JN (2006) Automatic recognition by gait. Proc IEEE 94(11):2013–2024

    Article  Google Scholar 

  39. Niyogi SA, Adelson EH (1994) Analyzing and recognizing walking figures in XYT. In: Proc. CVPR, pp 469–474

  40. O’Toole AJ (2004) Psychological and neural perspectives on human face recognition. In: Handbook of face recognition. Springer, New York

    Google Scholar 

  41. Papadimitriou CH, Raghavan P, Tamaki H, Vempala S (1998) Latent semantic indexing: a probabilistic analysis. Comput Syst Sci 61:217–235

    Article  MathSciNet  Google Scholar 

  42. Pecenovic Z (1997) Image retrieval using latent semantic indexing. Master’s thesis, AudioVisual Communications Lab, Ecole Polytechnique, F’ed’erale de Lausanne, Switzerland

  43. Ponterotto JG, Mallinckrodt B (2007) Introduction to the special section on racial and ethnic identity in counseling psychology: conceptual and methodological challenges and proposed solutions. J Couns Psychol 54(3):219–223

    Article  Google Scholar 

  44. Rosse C, Mejino JLV (2003) A reference ontology for biomedical informatics: the foundational model of anatomy. Journal of Biomedical Informatics 36(6):478–500

    Article  Google Scholar 

  45. Samangooei S, Guo B, Nixon MS (2008) The use of semantic human description as a soft biometric. In: Proc. IEEE BTAS

  46. Seely RD, Samangooei S, Middleton L, Carter JN, Nixon MS (2008) The University of Southampton multi-biometric tunnel and introducing a novel 3D gait dataset. In: Proc. IEEE BTAS

  47. Shakhnarovich G, Lee L, Darrell T (2001) Integrated face and gait recognition from multiple views. In: Proc. IEEE CVPR, pp 439–446

  48. Shutler J, Grant M, Nixon MS, Carter JN (2002) On a large sequence-based human gait database. In: Proc. RASC, pp 66–72

  49. Tajfel H (1982) Social psychology of intergroup relations. Annu Rev Psychol 33:1–39

    Article  Google Scholar 

  50. Troje NF, Sadr J, Nakayama K (2006) Axes vs averages: high-level representations of dynamic point-light forms. Vis Cogn 14:119–122

    Google Scholar 

  51. Van Koppen PJ, Lochun SK (1997) Portraying perpetrators; the validity of offender descriptions by witnesses. Law Hum Behav 21(6):662–685

    Google Scholar 

  52. Veres GV, Gordon L, Carter JN, Nixon MS (2004) What image information is important in silhouette-based gait recognition? In: Proc. IEEE CVPR, vol 2, pp II–776–II–782

  53. Vrusias B, Makris D, Renno J-P, Newbold N, Ahmad K, Jones G (2007) A framework for ontology enriched semantic annotation of cctv video. In: Proc. WIAMIS, p 5

  54. Wells GL, Olson EA (2003) Eyewitness testimony. Annu Rev Psychol 54:277–295

    Article  Google Scholar 

  55. Yarmey AD, Yarmey MJ (1997) Eyewitness recall and duration estimates in field settings. J Appl Soc Psychol 27(4):330–344

    Article  Google Scholar 

  56. Zhao R, Grosky W (2002) Bridging the semantic gap in image retrieval. IEEE Trans Multimedia 4:189–200

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Computer Science, Southampton University, Southampton, SO17 1BJ, UK

    Sina Samangooei & Mark S. Nixon

Authors
  1. Sina Samangooei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark S. Nixon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sina Samangooei.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Samangooei, S., Nixon, M.S. Performing content-based retrieval of humans using gait biometrics. Multimed Tools Appl 49, 195–212 (2010). https://doi.org/10.1007/s11042-009-0391-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-009-0391-8

Keywords

Search

Navigation