Skip to main content
SpringerLink
Log in

Security and Surveillance

  • Chapter
Visual Analysis of Humans

Abstract

Human eyes are highly efficient devices for scanning through a large quantity of low-level visual sensory data and delivering selective information to one’s brain for high-level semantic interpretation and gaining situational awareness. Over the last few decades, the computer vision community has endeavoured to bring about similar perceptual capabilities to artificial visual sensors. Substantial efforts have been made towards understanding static images of individual objects and the corresponding processes in the human visual system. This endeavour is intensified further by the need for understanding a massive quantity of video data, with the aim to comprehend multiple entities not only within a single image but also over time across multiple video frames for understanding their spatio-temporal relations. A significant application of video analysis and understanding is intelligent surveillance, which aims to interpret automatically human activity and detect unusual events that could pose a threat to public security and safety.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Frost and Sullivan estimates that the video surveillance software market will reach $670.7 million annually by 2011 [21].

  2. 2.

    The growing interest on video analytics is also evident from various industrial focus conferences such as the IMS Video Content Analysis Conferences (http://www.imsconferences.com).

  3. 3.

    Research conducted by the British Industry Security Association demonstrated that video analytics technologies are deployed by the transport and retail sectors most frequently (http://www.bsia.co.uk/aboutbsia/cctv/O5E926740891).

  4. 4.

    A set of real-world datasets and alarm definitions are released as the Image Library for Intelligent Detection Systems (i-LIDS), a UK government Home Office Scientific Development Branch (HOSDB) benchmark for video analytics systems [63], which has also been adopted by the US National Institute of Standards and Technology (NIST).

  5. 5.

    http://www.brslabs.com/index.php?id=79

  6. 6.

    http://www.honeywellvideo.com/support/library/videos/

  7. 7.

    http://www.marchnetworks.com/Products/Video-and-Data-Analytics/

  8. 8.

    http://mateusa.net/

  9. 9.

    http://www.nice.com/video/analytics

  10. 10.

    http://www.nature.com/news/2011/110527/full/news.2011.323.html

References

  1. Aghajan, H., Cavallaro, A. (eds.): Multi-camera Networks: Principles and Applications. Elsevier, Amsterdam (2009)

    Google Scholar 

  2. Alahia, A., Vandergheynsta, P., Bierlaireb, M., Kunt, M.: Cascade of descriptors to detect and track objects across any network of cameras. Comput. Vis. Image Underst. 114, 624–640 (2010)

    Article  Google Scholar 

  3. Ali, S., Shah, M.: A Lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–6 (2007)

    Chapter  Google Scholar 

  4. Ali, S., Shah, M.: Floor fields for tracking in high density crowd scenes. In: European Conference on Computer Vision, pp. 1–24 (2008)

    Google Scholar 

  5. Bar, M.: Visual objects in context. Nat. Rev. Neurosci. 5, 617–629 (2004)

    Article  Google Scholar 

  6. Bashir, K., Xiang, T., Gong, S.: Gait recognition without subject cooperation. Pattern Recogn. Lett. 31(13), 2052–2060 (2010)

    Article  Google Scholar 

  7. Benezeth, Y., Jodoin, P.-M., Saligrama, V., Rosenberger, C.: Abnormal events detection based on spatio-temporal co-occurrences. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2458–2465 (2009)

    Chapter  Google Scholar 

  8. Benfold, B., Reid, I.: Guiding visual surveillance by tracking human attention. In: British Machine Vision Conference (2009)

    Google Scholar 

  9. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  10. Bosch: Athens International Airport (2001) http://www.boschsecurity.co.uk/

  11. Breitenstein, M.D.: Visual surveillance – dynamic behavior analysis at multiple levels. PhD thesis, ETH Zurich (2009)

    Google Scholar 

  12. Cao, L., Liu, Z., Huang, T.S.: Cross-dataset action detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1998–2005 (2010)

    Google Scholar 

  13. Chen, K.-W., Lai, C.-C., Hung, Y.-P., Chen, C.-S.: An adaptive learning method for target tracking across multiple cameras. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

    Chapter  Google Scholar 

  14. IBM Corporation: Command, Control, Collabo-rate: Public Safety Solutions from IBM. Solution Brief (2009)

    Google Scholar 

  15. Ekman, P.: Facial expressions of emotion: New findings, new questions. Psychol. Sci. 3(1), 34–38 (1992)

    Article  MathSciNet  Google Scholar 

  16. Ekman, P., Friesen, W.V.: Unmasking the Face, 2nd edn. Consulting Psychologists Press, Palo Alto (1984)

    Google Scholar 

  17. Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 757–763 (1997)

    Article  Google Scholar 

  18. Everts, I., Sebe, N., Jones, G.A.: Cooperative object tracking with multiple PTZ cameras. In: International Conference on Image Analysis and Processing, pp. 323–330 (2007)

    Google Scholar 

  19. Farenzena, M., Bazzani, L., Perina, A., Cristani, M., Murino, V.: Person re-identification by symmetry-driven accumulation of local features. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2360–2367 (2010)

    Google Scholar 

  20. Felzenszwalb, P.F., Huttenlocher, D.P.: Pictorial structures for object recognition. Int. J. Comput. Vis. 61(1), 55–79 (2005)

    Article  Google Scholar 

  21. Frost & Sullivan: Video Surveillance Software Emerges as Key Weapon in Fight Against Terrorism. http://www.frost.com/

  22. Frost & Sullivan: Eyes on the Network – Understanding the Shift Toward Network-based Video Surveillance in Asia (2007). http://www.frost.com/prod/servlet/market-insight-top.pag?docid=100416385

  23. Gilbert, A., Bowden, R.: Incremental, scalable tracking of objects inter camera. Comput. Vis. Image Underst. 111(1), 43–58 (2008)

    Article  Google Scholar 

  24. Gill, P.M., Spriggs, A., Allen, J., Hemming, M., Jessiman, P., Kara, D., Kilworth, J., Little, R., Swain, D.: Control room operation: findings from control room observations. Home office online report 14/05, Home Office (2005)

    Google Scholar 

  25. Gouaillier, V., Fleurant, A.-E.: Intelligent video surveillance: Promises and challenges. Technological and commercial intelligence report, CRIM and Technôpole Defence and Security (2009)

    Google Scholar 

  26. Gray, D., Tao, H.: Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features. In: European Conference on Computer Vision, pp. 262–275 (2008)

    Google Scholar 

  27. Hongeng, S., Nevatia, R., Brémond, F.: Video-based event recognition: Activity representation and probabilistic recognition methods. Comput. Vis. Image Underst. 96(2), 129–162 (2004)

    Article  Google Scholar 

  28. Hospedales, T., Gong, S., Xiang, T.: A Markov clustering topic model for mining behaviour in video. In: IEEE International Conference on Computer Vision, pp. 1165–1172 (2009)

    Chapter  Google Scholar 

  29. Hu, Y., Cao, L., Lv, F., Yan, S., Gong, Y., Huang, T.S.: Action detection in complex scenes with spatial and temporal ambiguities. In: IEEE International Conference on Computer Vision (2009)

    Google Scholar 

  30. Javed, O., Shafique, K., Shah, M.: Appearance Modeling for Tracking in Multiple Non-overlapping Cameras. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 26–33 (2005)

    Google Scholar 

  31. Javed, O., Shah, M.: Automated Multi-camera Surveillance: Theory and Practice. Springer, New York (2008)

    Book  Google Scholar 

  32. Junior, J.C.S.J., Musse, S.R., Jung, C.R.: Crowd analysis using computer vision techniques. IEEE Signal Process. Mag. 27, 66–77 (2010)

    Google Scholar 

  33. Ke, Y., Sukthankar, R., Hebert, M.: Event detection in crowded videos. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)

    Chapter  Google Scholar 

  34. Kim, J., Grauman, K.: Observe locally, infer globally: A space–time MRF for detecting abnormal activities with incremental updates. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2928 (2009)

    Google Scholar 

  35. Kratz, L., Nishino, K.: Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1446–1453 (2009)

    Chapter  Google Scholar 

  36. Kratz, L., Nishino, K.: Tracking with local spatio-temporal motion patterns in extremely crowded scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 693–700 (2010)

    Google Scholar 

  37. Kuettel, D., Breitenstein, M.D., Gool, L.V., Ferrari, V.: What’s going on? Discovering spatio-temporal dependencies in dynamic scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1951–1958 (2010)

    Google Scholar 

  38. Lavee, G., Rivlin, E., Rudzsky, M.: Understanding video events: A survey of methods for automatic interpretation of semantic occurrences in video. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 39(5), 489–504 (2009)

    Article  Google Scholar 

  39. Li, J., Gong, S., Xiang, T.: Global behaviour inference using probabilistic latent semantic analysis. In: British Machine Vision Conference, pp. 193–202 (2008)

    Google Scholar 

  40. Li, J., Gong, S., Xiang, T.: Scene segmentation for behaviour correlation. In: European Conference on Computer Vision, pp. 383–395 (2008)

    Google Scholar 

  41. Loy, C.C., Xiang, T., Gong, S.: Stream-based active unusual event detection. In: Asian Conference on Computer Vision (2010)

    Google Scholar 

  42. Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: IEEE Conference on Computer Vision and Pattern Recognition (2010)

    Google Scholar 

  43. Makris, D., Ellis, T., Black, J.: Bridging the Gaps Between Cameras. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 205–210 (2004)

    Google Scholar 

  44. Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behaviour detection using social force model. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 935–942 (2009)

    Chapter  Google Scholar 

  45. Mehran, R., Moore, B.E., Shah, M.: A streakline representation of flow in crowded scenes. In: European Conference on Computer Vision (2010)

    Google Scholar 

  46. Moeslund, T.B., Hilton, A., Krüger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2), 90–126 (2006)

    Article  Google Scholar 

  47. Murphy-Chutorian, E., Trivedi, M.M.: Head pose estimation in computer vision: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 31(4), 607–626 (2009)

    Article  Google Scholar 

  48. Nguyen, H.T., Ji, Q., Smeulders, A.W.M.: Spatio-temporal context for robust multitarget tracking. IEEE Trans. Pattern Anal. Mach. Intell. 29(1), 52–64 (2007)

    Article  Google Scholar 

  49. Nixon, M.S., Tan, T., Chellappa, R.: Human Identification Based on Gait. Springer, New York (2005)

    Google Scholar 

  50. ObjectVideo: Hardening U.S. Borders (2003). http://www.objectvideo.com/

  51. Oliver, N., Rosario, B., Pentland, A.: A Bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)

    Article  Google Scholar 

  52. Orozco, J., Gong, S., Xiang, T.: Head pose classification in crowded scenes. In: British Machine Vision Conference (2009)

    Google Scholar 

  53. Poppe, R.: A survey on vision-based human action recognition. Image Vis. Comput. 28(6), 976–990 (2010)

    Article  Google Scholar 

  54. Prosser, B., Gong, S., Xiang, T.: Multi-camera matching using bi-directional cumulative brightness transfer functions. In: British Machine Vision Conference (2008)

    Google Scholar 

  55. Ran, Y., Zheng, Q., Chellappa, R., Strat, T.M.: Applications of a simple characterization of human gait in surveillance. IEEE Trans. Syst. Man Cybern., Part B, Cybern. 40(4), 1009–1020 (2010)

    Article  Google Scholar 

  56. Rodriguez, M., Ali, S., Kanade, T.: Tracking in unstructured crowded scenes. In: IEEE International Conference on Computer Vision (2009)

    Google Scholar 

  57. Schneiderman, R.: Trends in video surveillance give DSP an apps boost [special reports]. IEEE Signal Process. Mag. 27(6), 6–12 (2010)

    Google Scholar 

  58. Schwartz, O., Hsu, A., Dayan, P.: Space and time in visual context. Nat. Rev., Neurosci. 8, 522–535 (2007)

    Article  Google Scholar 

  59. Settles, B.: Active learning literature survey. Technical report, University of Wisconsin-Madison (2010)

    Google Scholar 

  60. Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: A comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)

    Article  Google Scholar 

  61. Sillito, R.R., Fisher, R.B.: Semi-supervised learning for anomalous trajectory detection. In: British Machine Vision Conference (2008)

    Google Scholar 

  62. Siva, P., Xiang, T.: Action detection in crowd. In: British Machine Vision Conference (2010)

    Google Scholar 

  63. Team, i.: Imagery library for intelligent detection systems (i-LIDS); a standard for testing video based detection systems. In: Annual IEEE International Carnahan Conferences Security Technology, pp. 75–80 (2006)

    Google Scholar 

  64. Teh, Y., Jordan, M., Beal, M., Blei, D.: Hierarchical Dirichlet processes. J. Am. Stat. Assoc. 101(476), 1566–1581 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  65. Loy, C.C., Xiang, T., Gong, S.: Modelling activity global temporal dependencies using time delayed probabilistic graphical model. In: IEEE International Conference on Computer Vision, pp. 120–127 (2009)

    Chapter  Google Scholar 

  66. Loy, C.C., Xiang, T., Gong, S.: Time-delayed correlation analysis for multi-camera activity understanding. Int. J. Comput. Vis. 90(1), 106–129 (2010)

    Article  Google Scholar 

  67. Tian, Y.-L., Kanade, T., Cohn, J.F.: Facial Expression Analysis. Springer, New York (2005). Chap. 11

    Google Scholar 

  68. Tieu, K., Dalley, G., Grimson, W.E.L.: Inference of non-overlapping camera network topology by measuring statistical dependence. In: IEEE International Conference on Computer Vision, pp. 1842–1849 (2005)

    Google Scholar 

  69. Turaga, P., Chellappa, R., Subrahmanian, V.S., Udrea, O.: Machine recognition of human activities – A survey. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1473–1488 (2008)

    Article  Google Scholar 

  70. Wang, X., Ma, X., Grimson, W.E.L.: Unsupervised activity perception in crowded and complicated scenes using hierarchical Bayesian models. IEEE Trans. Pattern Anal. Mach. Intell. 31(3), 539–555 (2009)

    Article  Google Scholar 

  71. Wang, X., Tieu, K., Grimson, W.E.L.: Correspondence-free activity analysis and scene modeling in multiple camera views. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 56–71 (2010)

    Article  Google Scholar 

  72. Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors. Int. J. Comput. Vis. 75(2), 247–266 (2007)

    Article  Google Scholar 

  73. Wu, S., Moore, B.E., Shah, M.: Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2054–2060 (2010)

    Google Scholar 

  74. Yang, M., Wu, Y., Hua, G.: Context-aware visual tracking. IEEE Trans. Pattern Anal. Mach. Intell. 31(7), 1195–1209 (2008)

    Article  Google Scholar 

  75. Yang, W., Wang, Y., Mori, G.: Efficient human action detection using a transferable distance function. In: Asian Conference on Computer Vision (2009)

    Google Scholar 

  76. Yuan, J., Liu, Z., Wu, Y.: Discriminative subvolume search for efficient action detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2442–2449 (2009)

    Chapter  Google Scholar 

  77. Zelniker, E.E., Gong, S., Xiang, T.: Global Abnormal Behaviour Detection Using a Network of CCTV Cameras. In: IEEE International Workshop on Visual Surveillance (2008)

    Google Scholar 

  78. Zheng, W., Gong, S., Xiang, T.: Associating groups of people. In: British Machine Vision Conference (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Queen Mary University of London, London, E1 4NS, UK

    Shaogang Gong, Chen Change Loy & Tao Xiang

Authors
  1. Shaogang Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Change Loy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaogang Gong .

Editor information

Editors and Affiliations

  1. Department of Media Technology, Aalborg University, Niels Jernes Vej 14, Aalborg, 9220, Denmark

    Thomas B. Moeslund

  2. Centre for Vision, Speech & Signal Proc., University of Surrey, Guildford, GU2 7XH, Surrey, United Kingdom

    Adrian Hilton

  3. Copenhagen Institute of Technology, Aalborg University, Lautrupvang 2B, Ballerup, 2750, Denmark

    Volker Krüger

  4. Disney Research, Forbes Avenue 615, Pittsburgh, 15213, Pennsylvania, USA

    Leonid Sigal

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Gong, S., Loy, C.C., Xiang, T. (2011). Security and Surveillance. In: Moeslund, T., Hilton, A., Krüger, V., Sigal, L. (eds) Visual Analysis of Humans. Springer, London. https://doi.org/10.1007/978-0-85729-997-0_23

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-997-0_23

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-996-3

  • Online ISBN: 978-0-85729-997-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation