Skip to main content
SpringerLink
Log in

Multi-cue Pedestrian Detection and Tracking from a Moving Vehicle

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper presents a multi-cue vision system for the real-time detection and tracking of pedestrians from a moving vehicle. The detection component involves a cascade of modules, each utilizing complementary visual criteria to successively narrow down the image search space, balancing robustness and efficiency considerations. Novel is the tight integration of the consecutive modules: (sparse) stereo-based ROI generation, shape-based detection, texture-based classification and (dense) stereo-based verification. For example, shape-based detection activates a weighted combination of texture-based classifiers, each attuned to a particular body pose.

Performance of individual modules and their interaction is analyzed by means of Receiver Operator Characteristics (ROCs). A sequential optimization technique allows the successive combination of individual ROCs, providing optimized system parameter settings in a systematic fashion, avoiding ad-hoc parameter tuning. Application-dependent processing constraints can be incorporated in the optimization procedure.

Results from extensive field tests in difficult urban traffic conditions suggest system performance is at the leading edge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Baumberg, A. and Hogg, D. 1994. Learning flexible models from image sequences. In Proc. of the European Conference on Computer Vision, pp. 299–308.

  • Bertozzi, M., Broggi, A., Grisleri, P., Tibaldi, and Rose, M.D. 2004. A tool for vision based pedestrian detection performance evaluation. In Proc. of the IEEE Intelligent Vehicle Symposium, Parma, Italy, pp. 784–789.

  • Borgefors, G. 1986. Distance transformations in digital images. CVGIP, 34(3):344–371.

    Google Scholar 

  • Bouguet, J.-Y. 2000. Camera calibration toolbox for Matlab. In http://www.vision.caltech.edu/bouguetj/calib_doc/.

  • Boyd, S. and Vandenbergh, L. 2004. Convex Optimization. Cambridge University Press.

  • Broggi, A., Fascioli, A., Fedriga, I., Tibaldi, A., and Rose, M.D. 2003. Stereo-based preprocessing for human shape localization in unstructured environments. In Proc. of the IEEE Intelligent Vehicle Symposium, Ohio, USA, pp. 410–415.

  • Broggi, A., Fascioli, A., Carletti, M., Graf, T., and Meinecke, M. 2004. A multi-resolution approach for infrared vision-based pedestrian detection. In Proc. of the IEEE Intelligent Vehicle Symposium, Parma, Italy, pp. 7–12.

  • Cootes, T., Taylor, C., Cooper, D., and Graham. J. 1995. Active shape models—their training and applications. Computer Vision and Image Understanding, 61(1):38–59.

    Article  Google Scholar 

  • Cutler, R. and Davis, L. 2000. Robust real-time periodic motion detection, analysis and applications. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(8):781–796.

    Article  Google Scholar 

  • Duda, R., Hart, P., and Stork, D. 2001. Pattern Classification, 2nd edition. John Wiley and Sons, New York.

    MATH  Google Scholar 

  • Elzein, H., Lakshmanan, S., and Watta. P. 2003. A motion and shape-based pedestrian detection algorithm. In Proc. of the IEEE Intelligent Vehicle Symposium, Ohio, USA, pp. 500–504.

  • Fang, Y., Yamada, K., Ninomiya, Y., Horn, B., and Masaki, I. 2003. Comparison between infrared-image-based and visible-image-based approaches for pedestrian detection. In Proc. of the IEEE Intelligent Vehicle Symposium, Ohio, USA, pp. 505–510.

  • Franke, U. 2000. Real-time stereo vision for urban traffic scene understanding. In Proc. of the IEEE Intelligent Vehicle Symposium, Detroit, USA.

  • Gavrila, D.M. 2001. Sensor-based pedestrian protection. IEEE Intelligent Systems, 16(6):77–81.

    Article  Google Scholar 

  • Gavrila, D.M., Giebel, J., and Munder, S. 2004. Vision-based pedestrian detection: The PROTECTOR+ system. In Proc. of the IEEE Intelligent Vehicle Symposium, Parma, Italy.

  • Gavrila, D.M. and Philomin, V. 1999. Real-time object detection for “smart” vehicles. In Proc. of the International Conference on Computer Vision, Kerkyra, Greece, pp. 87–93.

  • Giebel, J., Gavrila, D.M., and Schnörr, C. 2004. A Bayesian framework for multi-cue 3d object tracking. In Proc. of the European Conference on Computer Vision, Prague, Czech Republic.

  • Grubb, G., Zelinsky, A., Nilsson, L., and Ribbe, M. 2004. Pedestrian detection for driver assistance systems: Single-frame classification and system level performance. In Proc. of the IEEE Intelligent Vehicle Symposium, Parma, Italy, pp. 19–24.

  • Heinrich, S. 2002. Fast obstacle detection using flow/depth constraint. In Proc. of the IEEE Intelligent Vehicle Symposium, Versailles, France.

  • Heisele, B. and Wöhler, C. 1998. Motion-based recognition of pedestrians. In Proc. of the International Conference on Pattern Recognition.

  • Huo, X. and Chen, J. 2004. Building a cascade detector and applications in automatic target recognition. Applied Optics: Information Processing, 43(2):293–303.

    Google Scholar 

  • Jacobs, R.A., Jordan, M.I., Nowlan, S.J., and Hinton, G.E. 1991. Adaptive mixtures of local experts. Neural Computation, 3(1):79–87.

    Google Scholar 

  • Kuhn, H.W. 1955. The Hungarian method for the assignment problem. Naval Res. Logist. Quart., 2:83–97.

    MathSciNet  Google Scholar 

  • Labayrade, R., Aubert, D., and Tarel, J.-P. 2002. Real time obstacle detection on non flat road geometry through ‘v-disparity’ representation. In Proc. of the IEEE Intelligent Vehicle Symposium, Versailles, France.

  • Liu, X. and Fujimura, K. 2003. Pedestrian detection using stereo night vision. In Proc. of the IEEE International Conference on Intelligent Transportation Systems, Bejing, China, pp. 334–339.

  • Luo, H. 2005. Optimization design of cascaded classifiers. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Conf. pp. 480–485.

  • Mohan, A., Papageorgiou, C., and Poggio, T. 2001. Example-based object detection in images by components. IEEE Trans. on Pattern Analysis and Machine Intelligence, 23(4):349–361.

    Article  Google Scholar 

  • Munder, S. and Gavrila, D.M. 2006. An experimental study on pedestrian classification. Accepted for publication in IEEE Trans. on Pattern Analysis and Machine Intelligence.

  • Papageorgiou, C. and Poggio, T. 2000. A trainable system for object detection. J. of Computer Vision, 38(1):15–33.

    Article  Google Scholar 

  • Philomin, V., Duraiswami, R., and Davis, L. 2000. Quasi-random sampling for condensation. In Proc. of the European Conference on Computer Vision, Dublin, Ireland, pp. 134–149.

  • Polana, R. and Nelson, R. 1994. Low level recognition of human motion. In Proc. of the IEEE Workshop on Motion of Non-Rigid and Articulated Objects, Austin, pp. 77–82.

  • Provost, F. and Fawcett, T. 2001. Robust classification for imprecise environments. Machine Learning, 42(3):203–231.

    Article  Google Scholar 

  • Sarkar, S. and Chavali, S. 2000. Modeling parameter space behavior of vision systems using Bayesian networks. Computer Vision and Image Understanding, 79:185–223.

    Article  Google Scholar 

  • Shashua, A., Gdalyahu, Y., and Hayun, G. 2004. Pedestrian detection for driver assistance systems: Single-frame classification and system level performance. In Proc. of the IEEE Intelligent Vehicle Symposium, Parma, Italy.

  • Shimizu, H. and Poggio, T. 2004. Direction estimation of pedestrian from multiple still images. In Proc. of the IEEE Intelligent Vehicle Symposium, Parma, Italy.

  • Stenger, B., Thayananthan, A., Torr, P., and Cipolla, R. 2003. Filtering using a tree-based estimator. In Proc. of the International Conference on Computer Vision, Nice, France, pp. II:1063–1070.

  • Sun, J., Rehg, J.M., and Bobick, A. 2004. Automatic cascade training with perturbation bias. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition Conf. 2, pp. 276–283.

  • Thompson, W. and Pong, T.-C. 1990. Detecting moving objects. J. of Computer Vision, 4:39–57.

    Article  Google Scholar 

  • Toyama, K. and Blake, A. 2001. Probabilistic tracking in a metric space. In Proc. of the International Conference on Computer Vision, pp. 50–57.

  • Viola, P.A., Jones, M.J., and Snow, D. 2005. Detecting pedestrians using patterns of motion and appearance. J. of Computer Vision, 63(2):153–161.

    Article  Google Scholar 

  • Wöhler, C. and Anlauf, J. 1999. An adaptable time-delay neural-network algorithm for image sequence analysis. IEEE Trans. on Neural Networks, 10(6):1531–1536.

    Article  Google Scholar 

  • Zhao, L. and Thorpe, C. 2000. Stereo- and neural network-based pedestrian detection. IEEE Trans. on Intelligent Transportation Systems, 1(3).

Download references

Author information

Authors and Affiliations

  1. Machine Perception, DaimlerChrysler Research and Development, 89081, Ulm, Germany

    D. M. Gavrila & S. Munder

  2. Intelligent Systems Lab, Faculty of Science, University of Amsterdam, 1098 SJ, Amsterdam, The Netherlands

    D. M. Gavrila

Authors
  1. D. M. Gavrila
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Munder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to D. M. Gavrila.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gavrila, D.M., Munder, S. Multi-cue Pedestrian Detection and Tracking from a Moving Vehicle. Int J Comput Vision 73, 41–59 (2007). https://doi.org/10.1007/s11263-006-9038-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-006-9038-7

Keywords

Search

Navigation