Skip to main content
SpringerLink
Log in
Book cover

Robotics Research pp 695–710Cite as

Globally Optimal Object Pose Estimation in Point Clouds with Mixed-Integer Programming

  • Conference paper
  • First Online:

Part of the book series: Springer Proceedings in Advanced Robotics ((SPAR,volume 10))

Abstract

Motivated by the limitations of local object trackers, we present a formulation of the underlying point-cloud object pose estimation problem as a mixed-integer convex program, which we efficiently solve to optimality with an off-the-shelf branch and bound solver. We show that reasoning about object pose estimation in this way allows natural extension to point-to-mesh correspondence, multiple simultaneous object pose estimation, and outlier rejection without losing the ability to obtain a globally optimal solution. We probe the extent to which rich problem-specific formulations typically tackled with unreliable nonlinear optimization can be rigorously treated in a global optimization framework to overcome the limitations of other global pose estimation methods.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Akgül, C.B., Sankur, B., Yemez, Y., Schmitt, F.: 3d model retrieval using probability density-based shape descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1117–1133 (2009)

    Article  Google Scholar 

  2. Arie-Nachimson, M., Kovalsky, S.Z., Kemelmacher-Shlizerman, I., Singer, A., Basri, R.: Global motion estimation from point matches. In 2012 second international conference on 3D imaging, modeling, processing, visualization and transmission (3DIMPVT), pp. 81–88. IEEE (2012)

    Google Scholar 

  3. Besl, P.J., McKay, N.D.: Method for registration of 3-d shapes. In: Robotics-DL tentative, pp. 586–606. International Society for Optics and Photonics (1992)

    Google Scholar 

  4. Chaudhury, K.N., Khoo, Y., Singer, A.: Global registration of multiple point clouds using semidefinite programming. SIAM J. Optim. 25(1), 468–501 (2015)

    Article  MathSciNet  Google Scholar 

  5. Chen, Y., Medioni, G.: Object modelling by registration of multiple range images. Image Vis. Comput. 10(3), 145–155 (1992)

    Article  Google Scholar 

  6. Dai, H., Izatt, G., Tedrake, R.: Global inverse kinematics via mixed-integer convex optimization (2017)

    Google Scholar 

  7. Drost, B., Ulrich, M., Navab, N., Ilic, S.: Model globally, match locally: efficient and robust 3d object recognition. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR), pp. 998–1005. IEEE (2010)

    Google Scholar 

  8. Dunning, I., Huchette, J., Lubin, M.: Jump: a modeling language for mathematical optimization. SIAM Rev. 59(2), 295–320 (2017)

    Article  MathSciNet  Google Scholar 

  9. Eggert, D.W., Lorusso, A., Fisher, R.B.: Estimating 3-d rigid body transformations: a comparison of four major algorithms. Mach. Vis. Appl. 9(5–6), 272–290 (1997)

    Article  Google Scholar 

  10. Enqvist, O., Josephson, K., Kahl, F.: Optimal correspondences from pairwise constraints. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1295–1302. IEEE (2009)

    Google Scholar 

  11. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  12. Gelfand, N., Mitra, N.J., Guibas, L.J., Pottmann, H.: Robust global registration. In: Symposium on Geometry Processing, vol. 2, p. 5 (2005)

    Google Scholar 

  13. G.O. Inc. Gurobi optimizer reference manual (2016)

    Google Scholar 

  14. Hartley, R.I., Kahl, F.: Global optimization through rotation space search. Int. J. Comput. Vis. 82(1), 64–79 (2009)

    Article  Google Scholar 

  15. Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., Navab, N.: Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In: Asian Conference on Computer Vision, pp. 548–562. Springer (2012)

    Google Scholar 

  16. Izatt, G., Mirano, G., Adelson, E., Tedrake, R.: Tracking objects with point clouds from vision and touch. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE (2017)

    Google Scholar 

  17. Johnson, A.E., Hebert, M.: Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 433–449 (1999)

    Article  Google Scholar 

  18. Klingensmith, M., Koval, M.C., Srinivasa, S.S., Pollard, N.S., Kaess, M.: The manifold particle filter for state estimation on high-dimensional implicit manifolds (2016). arXiv:1604.07224

  19. Li, H., Hartley, R.: The 3d-3d registration problem revisited. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, pp. 1–8. IEEE (2007)

    Google Scholar 

  20. Li, S., Lyu, S., Trinkle, J.: State estimation for dynamic systems with intermittent contact. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 3709–3715. IEEE (2015)

    Google Scholar 

  21. Marion, P., Florence, P.R., Manuelli, L., Tedrake, R.: A pipeline for generating ground truth labels for real rgbd data of cluttered scenes. Under Review (2017)

    Google Scholar 

  22. Maron, H., Dym, N., Kezurer, I., Kovalsky, S., Lipman, Y.: Point registration via efficient convex relaxation. ACM Trans. Gr. (TOG) 35(4), 73 (2016)

    Google Scholar 

  23. McCormick, G.P.: Computability of global solutions to factorable nonconvex programs: part i-convex underestimating problems. Math. Program. 10(1), 147–175 (1976)

    Article  Google Scholar 

  24. Mellado, N., Aiger, D., Mitra, N.J.: Super 4pcs fast global pointcloud registration via smart indexing. In: Computer Graphics Forum, vol. 33, pp. 205–215. Wiley Online Library (2014)

    Google Scholar 

  25. Narayanan, V., Likhachev, M.: Deliberative object pose estimation in clutter. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3125–3130. IEEE (2017)

    Google Scholar 

  26. Nemhauser, G.L., Wolsey, L.A., Integer programming and combinatorial optimization. Wiley, Chichester. Nemhauser, G.L., Savelsbergh, M.W.P., Sigismondi, G.S.: Constraint classification for mixed integer programming formulations. COAL Bull. 20(8–12) (1988) (1992)

    Google Scholar 

  27. Olsson, C., Kahl, F., Oskarsson, M.: Branch-and-bound methods for euclidean registration problems. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 783–794 (2009)

    Article  Google Scholar 

  28. Papazov, C., Burschka, D.: An efficient ransac for 3d object recognition in noisy and occluded scenes. Comput. Vis.-ACCV 2010, 135–148 (2011)

    Google Scholar 

  29. Rosen, D.M., Carlone, L., Bandeira, A.S., Leonard, J.J.: Se-sync: a certifiably correct algorithm for synchronization over the special Euclidean group (2016)

    Google Scholar 

  30. Schmidt, T., Hertkorn, K., Newcombe, R., Marton, Z., Suppa, M., Fox, D.: Depth-based tracking with physical constraints for robot manipulation. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 119–126. IEEE (2015)

    Google Scholar 

  31. Schmidt, T., Newcombe, R., Fox, D.: Dart: dense articulated real-time tracking with consumer depth cameras. Auton. Robots 39(3), 239–258 (2015)

    Article  Google Scholar 

  32. Schulman, J., Lee, A., Ho, J., Abbeel, P.: Tracking deformable objects with point clouds. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 1130–1137. IEEE (2013)

    Google Scholar 

  33. Tedrake, R., The Drake Development Team: Drake: A planning, control, and analysis toolbox for nonlinear dynamical systems (2016)

    Google Scholar 

  34. Tombari, F., Salti, S., Di Stefano, L.: Unique signatures of histograms for local surface description. In: European Conference on Computer Vision, pp. 356–369. Springer (2010)

    Google Scholar 

  35. Wohlhart, P., Lepetit, V.: Learning descriptors for object recognition and 3d pose estimation. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

    Google Scholar 

  36. Wong, J.M., Kee, V., Le, T., Wagner, S., Mariottini, G.-L., Schneider, A., Hamilton, L., Chipalkatty, R., Hebert, M., Johnson, D. et al.: Segicp: integrated deep semantic segmentation and pose estimation (2017). arXiv:1703.01661

  37. Yang, J., Li, H., Campbell, D., Jia, Y.: Go-icp: a globally optimal solution to 3d icp point-set registration. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2241–2254 (2016)

    Article  Google Scholar 

  38. Zhou, Q.-Y., Park, J., Koltun, V.: Fast global registration. In: European Conference on Computer Vision, pp. 766–782. Springer (2016)

    Google Scholar 

Download references

Acknowledgements

This material is based upon work supported by NSF Contract IIS-1427050, a National Science Foundation Graduate Research Fellowship under Grant No. 1122374, as well as support from ABB and Draper Laboratory.

Author information

Authors and Affiliations

  1. Computer Science and Artificial Intelligence Lab, MIT, Cambridge, MA, USA

    Gregory Izatt & Russ Tedrake

  2. Toyota Research Institute, Ann Arbor, MI, USA

    Hongkai Dai & Russ Tedrake

Authors
  1. Gregory Izatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongkai Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Russ Tedrake
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gregory Izatt .

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA

    Nancy M. Amato

  2. Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

    Greg Hager

  3. Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA

    Shawna Thomas

  4. Department of Electrical Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile

    Miguel Torres-Torriti

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Izatt, G., Dai, H., Tedrake, R. (2020). Globally Optimal Object Pose Estimation in Point Clouds with Mixed-Integer Programming. In: Amato, N., Hager, G., Thomas, S., Torres-Torriti, M. (eds) Robotics Research. Springer Proceedings in Advanced Robotics, vol 10. Springer, Cham. https://doi.org/10.1007/978-3-030-28619-4_49

Download citation

Publish with us

Policies and ethics

Search

Navigation