Skip to main content
SpringerLink
Log in

Robust edge-based 3D object tracking with direction-based pose validation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper we propose a robust edge-based approach for 3D textureless object tracking. We first introduce an edge-based pose estimation method, which minimizes the holistic distance between the projected object contour and the query image edges, without explicitly searching for 3D-2D correspondences. This method is accurate with a good initialization; however, it is sensitive to occlusion and fast motion, thus often gets lost in real environments. To improve robustness, we exploit consistency of edge direction for validating the correctness of the estimated 3D pose, and further incorporate the validation scheme for robust estimation, non-local searching and failure recovery. The robust estimation adopts point-wise validation to reduce the effect of outlier, resulting in a direction-based robust estimator. The non-local searching is based on particle filter, with the pose validation for a faithful weighting of particles, which is shown to be better than the distance-based weighting. The failure recovery is based on fast 2D detection, and estimates the recovered pose by searching for 3D-2D point correspondences, with the validation scheme to adaptively determine state transition. The effectiveness of our approach is demonstrated using comparative experiments on real image sequences with occlusions, large motions and background clutters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. PWP3D: http://www.robots.ox.ac.uk/~victor/code.html

  2. GOS: https://github.com/guofengw/GOSTracker

  3. D2CO: https://bitbucket.org/alberto_pretto/d2co.git

  4. For fair comparison, we closed failure recovery for comparisons with PWP3D, GOS, and D2CO.

  5. See the accompany videos.

References

  1. Besl PJ, Mckay ND (1992) Method for registration of 3-D shapes. In: Robotics - DL tentative, pp 239–256

  2. Calonder M, Lepetit V, Strecha C, Fua P (2010) BRIEF: Binary robust independent elementary features. In: European conference on computer vision, pp 778–792

    Chapter  Google Scholar 

  3. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–98

    Article  Google Scholar 

  4. Chen Y, Medioni G (1992) Object modelling by registration of multiple range images. Image Vis Comput 10(3):145–155

    Article  Google Scholar 

  5. Choi C, Christensen HI (2012) 3D textureless object detection and tracking: an edge-based approach. In: International conference on intelligent robotics and system, pp 3877–3884

  6. Choi C, Christensen HI (2012) Robust 3D visual tracking using particle filtering on the special Euclidean group: A combined approach of keypoint and edge features. Int J Robot Res 33(4):498–519

    Article  Google Scholar 

  7. Comport AI, Marchand E, Chaumette F (2003) A real-time tracker for markerless augmented reality. In: IEEE International symposium on mixed and augmented reality, pp 36–45

  8. Felzenszwalb PF, Huttenlocher DP (2004) Distance transforms of sampled functions. Theory of Computing 8(19):415–428

    MathSciNet  MATH  Google Scholar 

  9. Fitzgibbon AW (2003) Robust registration of 2D and 3D point sets. Image Vis Comput 21(13):1145–1153

    Article  Google Scholar 

  10. Gao X, Hou XR, Tang J, Cheng HF (2003) Complete solution classification for the perspective-three-point problem. IEEE Trans Pattern Anal Mach Intell 25 (8):930–943

    Article  Google Scholar 

  11. Harris C, Stennett C (1990) RAPId - a video-rate object tracker. In: British machine vision conference, pp 73–77

  12. Hartley R, Zisserman A (2000) Multiple view geometry in computer vision. Cambridge University Press

  13. Hinterstoisser S, Cagniart C, Ilic S, Sturm P, Navab N, Fua P, Lepetit V (2012) Gradient response maps for real-time detection of textureless objects. IEEE Trans Pattern Anal Mach Intell 34(5):876–888

    Article  Google Scholar 

  14. Hinterstoisser S, Lepetit V, Ilic S, Fua P, Navab N (2010) Dominant orientation templates for real-time detection of texture-less objects. In: IEEE Conference on computer vision and pattern recognition, pp 2257–2264

  15. Hinterstoisser S, Lepetit V, Ilic S, Holzer S, Bradski G, Konolige K, Navab N (2012) Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes. In: Asian conference on computer vision, pp 548–562

  16. Imperoli M, Pretto A (2015) D2CO: Fast and robust registration of 3D textureless objects using the directional chamfer distance. In: International conference on computer vision systems, pp 316–328

  17. Isard M, Blake A (1998) CONDENSATION - Conditional density propagation for visual tracking. In: International journal of computer vision, pp 5–28

  18. Kato H, Billinghurst M (1999) Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: IEEE And ACM international workshop on augmented reality, pp 85–94

  19. Klein G, Murray DW (2006) Full-3D edge tracking with a particle filter. In: British machine vision conference, pp 1119–1128

  20. Lepetit V, Fua P (2005) Monocular model-based 3D tracking of rigid objects: a survey. Found Trends Comput Graph Vis 1(1):1–89

    Article  Google Scholar 

  21. Lepetit V, Moreno-Noguer F (2009) Epnp: an accurate o(n) solution to the pnp problem. Int J Comput Vis 81(2):155–166

    Article  Google Scholar 

  22. Liu MY, Tuzel O, Veeraraghavan A, Chellappa R (2010) Fast directional chamfer matching. In: IEEE Conference on computer vision and pattern recognition, pp 1696–1703

  23. Lourakis M, Zabulis X (2013) Model-based pose estimation for rigid objects. In: International conference on computer vision systems, pp 83–92

    Google Scholar 

  24. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  25. Marchand E, Bouthemy P, Chaumette F (2001) A 2Dc3D model-based approach to real-time visual tracking. Image Vis Comput 19(13):941–955

    Article  Google Scholar 

  26. Marchand E, Uchiyama H, Spindler F (2015) Pose estimation for augmented reality: a hands-on survey. IEEE Trans Vis Comput Graph 22(12):2633–2651

    Article  Google Scholar 

  27. Park Y, Lepetit V, Woo W (2008) Multiple 3D object tracking for augmented reality. In: IEEE International symposium on mixed and augmented reality, pp 117–120

  28. Park Y, Lepetit V, Woo W (2011) Texture-less object tracking with online training using an rgb-d camera. In: IEEE International symposium on mixed and augmented reality, pp 121–126

  29. Prisacariu VA, Reid ID (2012) PWP3D: Real-time segmentation and tracking of 3d objects. Int J Comput Vis 98(3):335–354

    Article  MathSciNet  Google Scholar 

  30. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: An efficient alternative to SIFT or SURF. In: IEEE International conference on computer vision, pp 2564–2571

  31. Seo BK, Park H, Park JI, Hinterstoisser S, Ilic S (2014) Optimal local searching for fast and robust textureless 3D object tracking in highly cluttered backgrounds. IEEE Trans Vis Comput Graph 20(1):99–110

    Article  Google Scholar 

  32. Shotton J, Glocker B, Zach C, Izadi S, Criminisi A, Fitzgibbon A (2013) Scene coordinate regression forests for camera relocalization in rgb-d images. In: IEEE Conference on computer vision and pattern recognition, pp 2930–2937

  33. Vacchetti L, Lepetit V, Fua P (2004) Combining edge and texture information for real-time accurate 3D camera tracking. In: IEEE International symposium on mixed and augmented reality, pp 48–56

  34. Vacchetti L, Lepetit V, Fua P (2004) Stable real-time 3D tracking using online and offline information. IEEE Trans Pattern Anal Mach Intell 26(10):1385–1391

    Article  Google Scholar 

  35. Wang B, Zhong F, Qin X (2017) Pose optimization in edge distance field for textureless 3d object tracking. In: The computer graphics international conference, pp 1–6

  36. Wang G, Wang B, Zhong F, Qin X, Chen B (2015) Global optimal searching for textureless 3D object tracking. Vis Comput 31(6):979–988

    Article  Google Scholar 

  37. Wuest H, Vial F, Strieker D (2005) Adaptive line tracking with multiple hypotheses for augmented reality. In: IEEE International symposium on mixed and augmented reality, pp 62–69

Download references

Acknowledgements

The authors gratefully acknowledge the anonymous reviewers for their comments to help us to improve our paper, and also thank for their enormous help in revising this paper. This work is supported by the National Key Research and Development Program of China (No. 2016YFB1001501).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong University, Qingdao, China

    Bin Wang & Fan Zhong

  2. School of Software, Shandong University, Jinan, China

    Xueying Qin

  3. Engineering Research Center of Digital Media Technology, Ministry of Education of China, Jinan, China

    Bin Wang & Xueying Qin

Authors
  1. Bin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fan Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xueying Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fan Zhong.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 71.1 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, B., Zhong, F. & Qin, X. Robust edge-based 3D object tracking with direction-based pose validation. Multimed Tools Appl 78, 12307–12331 (2019). https://doi.org/10.1007/s11042-018-6727-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6727-5

Keywords

Search

Navigation