Skip to main content
Log in

Content and task-based view selection from multiple video streams

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We present a content-aware multi-camera selection technique that uses object- and frame-level features. First objects are detected using a color-based change detector. Next trajectory information for each object is generated using multi-frame graph matching. Finally, multiple features including size and location are used to generate an object score. At frame-level, we consider total activity, event score, number of objects and cumulative object score. These features are used to generate score information using a multivariate Gaussian distribution. The algorithm. The best view is selected using a Dynamic Bayesian Network (DBN), which utilizes camera network information. DBN employs previous view information to select the current view thus increasing resilience to frequent switching. The performance of the proposed approach is demonstrated on three multi-camera setups with semi-overlapping fields of view: a basketball game, an indoor airport surveillance scenario and a synthetic outdoor pedestrian dataset. We compare the proposed view selection approach with a maximum score based camera selection criterion and demonstrate a significant decrease in camera flickering. The performance of the proposed approach is also validated through subjective testing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Batista J, Peixoto P, Araujo H (1998) Real-time active visual surveillance by integrating peripheral motion detection with foveated tracking. In: Proc of IEEE workshop on visual surveillance, pp 18–25

  2. Bimbo AD, Pernici F (2006) Towards on-line saccade planning for high-resolution image sensing. Pattern Recogn Lett 27(15):1826–1834

    Article  Google Scholar 

  3. Chen X, Davis J (2008) An occlusion metric for selecting robust camera configurations. Mach Vis Appl 19(4):217–222

    Article  MathSciNet  Google Scholar 

  4. Costello CJ, Diehl CP, Banerjee A, Fisher H (2004) Scheduling an active camera to observe people. In: Proc of the ACM 2nd int workshop on video surveillance & sensor networks, pp 39–45

  5. Daniyal F, Taj M, Cavallaro A (2008) Content-aware ranking of video segments. In: Proc of ACM/IEEE int conf on distributed smart cameras, pp 1–9

  6. Gilbert A, Bowden R (2006) Tracking objects across cameras by incrementally learning inter-camera colour calibration and patterns of activity. In: Proc of I9th European conference on computer vision, part II, pp 125–136

  7. Goshorn R, Goshorn J, Goshorn D, Aghajan H (2007) Architecture for cluster-based automated surveillance network for detecting and tracking multiple persons. In: Proc of ACM/IEEE int conf on distributed smart cameras

  8. Greiffenhagen M, Ramesh V, Comaniciu D, Niemann H (2000) Statistical modeling and performance characterization of a real-time dual camera surveillance system. In: Proc of IEEE int conf on computer vision and pattern recognition, pp 335–342

  9. Gupta A, Mittal A, Davis LS (2007) Cost: An approach for camera selection and multi-object inference ordering in dynamic scenes. In: Proc of IEEE int conf on computer vision, pp 1–8

  10. Jiang H, Fels S, Little JJ (2008) Optimizing multiple object tracking and best view video synthesis. IEEE Trans Multimedia 10(6):997–1012

    Article  Google Scholar 

  11. Karlsson S, Taj M, Cavallaro A (2008) Detection and tracking of humans and faces. EURASIP J Image Video Process 2008(1):1–9

    Article  Google Scholar 

  12. Khan S, Shah M (2003) Consistent labeling of tracked objects in multiple cameras with overlapping fields of view. IEEE Trans Pattern Anal Mach Intell 25(10):1355–1360

    Article  Google Scholar 

  13. Lien KC, Huang CL (2006) Multi-view-based cooperative tracking of multiple human objects in cluttered scenes. In: International conference on pattern recognition, pp 1123–1126

  14. Murphy K (2002) Dynamic bayesian networks: Representation, inference and learning. PhD thesis, Department of Computer Science, UC Berkeley

  15. Park HS, Lim S, Min JK, Cho SB (2008) Optimal view selection and event retrieval in multi-camera office environment. In: Proc of IEEE int conf on multisensor fusion and integration for intelligent systems, pp 106–110

  16. Prince SJD, Elder JH, Hou Y, Sizinstev M (2005) Pre-attentive face detection for foveated wide-field surveillance. In: Proc of IEEE workshop on application of computer vision, vol 1, pp 439–446

  17. Qureshi FZ, Terzopoulos D (2005) Surveillance camera scheduling: a virtual vision approach. In: Proc of the ACM int workshop on video surveillance & sensor networks, pp 131–140

  18. Qureshi FZ, Terzopoulos D (2007) Surveillance in virtual reality: System design and multi-camera control. In: Proc of IEEE int conf on computer vision and pattern recognition

  19. Rezaeian M (2007) Sensor scheduling for optimal observability using estimation entropy. In: IEEE int workshop on pervasive computing and communications, pp 307–312

  20. Senior A, Hampapur A, Lu M (2005) Acquiring multi-scale images by pan-tilt-zoom control and automatic multi-camera calibration. In: Proc of IEEE workshop on application of computer vision, vol 1, pp 433–438

  21. Shen C, Zhang C, Fels S (2007) A multi-camera surveillance system that estimates quality-of-view measurement. In: Proc of IEEE int conf on image processing, pp 193–196

  22. Smeaton AF, Over P, Kraaij W (2006) Evaluation campaigns and trecvid. In: Proc ACM int workshop on multimedia information retrieval, pp 321–330

  23. Snidaro L, Niu R, Varshney P, Foresti G (2003) Automatic camera selection and fusion for outdoor surveillance under changing weather conditions. In: Proc of IEEE int conf on advanced video and signal based surveillance, pp 364–369

  24. Taj M, Cavallaro A (2008) Object and scene-centric activity detection using state occupancy duration modeling. In: Proc of IEEE int conf on advanced video and signal based surveillance

  25. Taj M, Maggio E, Cavallaro A (2006) Multi-feature graph-based object tracking. In: Proc of classification of events, activities and relationships (CLEAR) workshop, pp 190–199

  26. Taj M, Daniyal F, Cavallaro A (2008) Event analysis on trecvid 2008 london gatwick dataset. In: Online proc of TREC video retrieval workshop

  27. Tarabanis K, Tsai R, Allen P (1995) The MVP sensor planning system for robotic vision tasks. IEEE Trans on Robotics and Automation 11(1):72–85

    Article  Google Scholar 

  28. Taylor G, Chosak A, Brewer P (2007) OVVV: Using virtual worlds to design and evaluate surveillance systems. In: Proc of IEEE int conf on computer vision and pattern recognition

  29. Tessens L, Morbee M, Lee H, Philips W, Aghajan H (2008) Principal view determination for camera selection in distributed smart camera networks. In: Proc of ACM/IEEE int conf on distributed smart cameras, pp 1–10

  30. Viola P, Jones M, Snow D (2003) Detecting pedestrians using patterns of motion and appearance. In: Proc of IEEE int conf on computer vision, pp 734–741

  31. Zhou X, Collins RT, Kanade T, Metes P (2003) A master-slave system to acquire biometric imagery of humans at distance. In: Proc of ACM SIGMM int workshop on video surveillance, pp 113–120

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Fahad Daniyal.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Daniyal, F., Taj, M. & Cavallaro, A. Content and task-based view selection from multiple video streams. Multimed Tools Appl 46, 235–258 (2010). https://doi.org/10.1007/s11042-009-0355-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-009-0355-z

Keywords

Navigation