Skip to main content
SpringerLink
Log in

Online coverage and inspection planning for 3D modeling

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

In this study, we address an exploration problem when constructing complete 3D models in an unknown environment using a Micro-Aerial Vehicle. Most previous exploration methods were based on the Next-Best-View (NBV) approaches, which iteratively determine the most informative view, that exposes the greatest unknown area from the current partial model. However, these approaches sometimes miss minor unreconstructed regions like holes or sparse surfaces (while these can be important features). Furthermore, because the NBV methods iterate the next-best path from a current partial view, they sometimes produce unnecessarily long trajectories by revisiting known regions. To address these problems, we propose a novel exploration algorithm that integrates coverage and inspection strategies. The suggested algorithm first computes a global plan to cover unexplored regions to complete the target model sequentially. It then plans local inspection paths that comprehensively scans local frontiers. This approach reduces the total exploration time and improves the completeness of the reconstructed models. We evaluate the proposed algorithm in comparison with other state-of-the-art approaches through simulated and real-world experiments. The results show that our algorithm outperforms the other approaches and in particular improves the completeness of surface coverage.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://3dwarehouse.sketchup.com/.

  2. https://www.stereolabs.com/.

References

  • Atkar, P. N., Choset, H., Rizzi, A. A., & Acar, E. U. (2001). Exact cellular decomposition of closed orientable surfaces embedded in/spl rfr//sup 3. In IEEE international conference on robotics and automation (ICRA), 1 (pp. 699–704).

  • Atkar, P. N., Greenfield, A., Conner, D. C., Choset, H., & Rizzi, A. A. (2005). Uniform coverage of automotive surface patches. International Journal of Robotics Research, 24(11), 883–898.

    Article  Google Scholar 

  • Bircher, A., Kamel, M., Alexis, K., Burri, M., Oettershagen, P., Omari, S., et al. (2016). Three-dimensional coverage path planning via viewpoint resampling and tour optimization for aerial robots. Autonomous Robots, 40(6), 1059–1078.

    Article  Google Scholar 

  • Bircher, A., Kamel, M., Alexis, K., Oleynikova, H., & Siegwart, R. (2018). Receding horizon path planning for 3d exploration and surface inspection. Autonomous Robots, 42(2), 291–306.

    Article  Google Scholar 

  • Blaer, P. S., & Allen, P. K. (2009). View planning and automated data acquisition for three-dimensional modeling of complex sites. Journal of Field Robotics, 26(11–12), 865–891.

    Article  Google Scholar 

  • Blochliger, F., Fehr, M., Dymczyk, M., Schneider, T., & Siegwart, R. (2018). Topomap: Topological mapping and navigation based on visual slam maps. In IEEE international conference on robotics and automation (ICRA) (pp. 1–9).

  • Brown, S., & Waslander, S. L. (2016). The constriction decomposition method for coverage path planning. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3233–3238).

  • Charrow, B., Kahn, G., Patil, S., Liu, S., Goldberg, K., Abbeel, P., Michael, N., & Kumar, V. (2015). Information-theoretic planning with trajectory optimization for dense 3d mapping. In Robotics: Science and systems (RSS), vol. 6.

  • Chaves, S. M., Kim, A., Galceran, E., & Eustice, R. M. (2016). Opportunistic sampling-based active visual slam for underwater inspection. Autonomous Robots, 40(7), 1245–1265.

    Article  Google Scholar 

  • Cheng, P., Keller, J., & Kumar, V. (2008). Time-optimal uav trajectory planning for 3d urban structure coverage. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2750–2757).

  • Choset, H. (2000). Coverage of known spaces: The boustrophedon cellular decomposition. Autonomous Robots, 9(3), 247–253.

    Article  Google Scholar 

  • Choset, H., Acar, E., Rizzi, A. A., & Luntz, J. (2000). Exact cellular decompositions in terms of critical points of morse functions. In IEEE international conference on robotics and automation (ICRA), 3 (pp. 2270–2277).

  • Cieslewski, T., Kaufmann, E., & Scaramuzza, D. (2017). Rapid exploration with multi-rotors: A frontier selection method for high speed flight. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2135–2142).

  • Connolly, C. (1985). The determination of next best views. In IEEE international conference on robotics and automation (ICRA), 2 (pp. 432–435).

  • Costante, G., Delmerico, J., Werlberger, M., Valigi, P., & Scaramuzza, D. (2018). Exploiting photometric information for planning under uncertainty. In Robotics research (pp. 107–124). Springer.

  • Das, A., Diu, M., Mathew, N., Scharfenberger, C., Servos, J., Wong, A., et al. (2014). Mapping, planning, and sample detection strategies for autonomous exploration. Journal of Field Robotics, 31(1), 75–106.

    Article  Google Scholar 

  • Durham, J. W., Carli, R., Frasca, P., & Bullo, F. (2011). Discrete partitioning and coverage control for gossiping robots. IEEE Transactions on Robotics, 28(2), 364–378.

    Article  Google Scholar 

  • Emek, Y., & Rosén, A. (2016). Semi-streaming set cover. ACM Transactions on Algorithms, 13(1), 6.

    Article  MathSciNet  MATH  Google Scholar 

  • Englot, B., & Hover, F. S. (2012). Sampling-based coverage path planning for inspection of complex structures. In International conference on automated planning and scheduling (ICAPS).

  • Englot, B., & Hover, F. S. (2013). Three-dimensional coverage planning for an underwater inspection robot. International Journal of Robotics Research, 32(9–10), 1048–1073.

    Article  Google Scholar 

  • Feigenbaum, J., Kannan, S., McGregor, A., Suri, S., & Zhang, J. (2005). On graph problems in a semi-streaming model. Theoretical Computer Science, 348(2–3), 207–216.

    Article  MathSciNet  MATH  Google Scholar 

  • Furrer, F., Burri, M., Achtelik, M., & Siegwart, R. (2016). Rotors—A modular gazebo mav simulator framework. In Robot operating system (ROS) (pp. 595–625). Springer.

  • Gabriely, Y., & Rimon, E. (2002). Spiral-STC: An on-line coverage algorithm of grid environments by a mobile robot. In IEEE international conference on robotics and automation (ICRA), 1 (pp. 954–960).

  • Galceran, E., Campos, R., Palomeras, N., Ribas, D., Carreras, M., & Ridao, P. (2015). Coverage path planning with real-time replanning and surface reconstruction for inspection of three-dimensional underwater structures using autonomous underwater vehicles. Journal of Field Robotics, 32(7), 952–983.

    Article  Google Scholar 

  • Helsgaun, K. (2000). An effective implementation of the Lin–Kernighan traveling salesman heuristic. European Journal of Operational Research, 126(1), 106–130.

    Article  MathSciNet  MATH  Google Scholar 

  • Heng, L., Gotovos, A., Krause, A., & Pollefeys, M. (2015). Efficient visual exploration and coverage with a micro aerial vehicle in unknown environments. In IEEE international conference on robotics and automation (ICRA), 3 (pp. 3–5).

  • Hepp, B., Dey, D., Sinha, S. N., Kapoor, A., Joshi, N., & Hilliges, O. (2018). Learn-to-score: Efficient 3D scene exploration by predicting view utility. In Proceedings of the European conference on computer vision (ECCV) (pp. 437–452).

  • Hess, J., Tipaldi, G. D., & Burgard, W. (2012). Null space optimization for effective coverage of 3d surfaces using redundant manipulators. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1923–1928).

  • Hornung, A., Wurm, K. M., Bennewitz, M., Stachniss, C., & Burgard, W. (2013). Octomap: An efficient probabilistic 3d mapping framework based on octrees. Autonomous Robots, 34(3), 189–206.

    Article  Google Scholar 

  • Jadidi, M. G., Miro, J. V., & Dissanayake, G. (2015). Mutual information-based exploration on continuous occupancy maps. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 6086–6092).

  • Julian, B. J., Karaman, S., & Rus, D. (2014). On mutual information-based control of range sensing robots for mapping applications. International Journal of Robotics Research, 33(10), 1375–1392.

    Article  Google Scholar 

  • Juliá, M., Gil, A., & Reinoso, O. (2012). A comparison of path planning strategies for autonomous exploration and mapping of unknown environments. Autonomous Robots, 33(4), 427–444.

    Article  Google Scholar 

  • Kafka, P., Faigl, J., & Váňa, P. (2016). Random inspection tree algorithm in visual inspection with a realistic sensing model and differential constraints. In IEEE international conference on robotics and automation (ICRA) (pp. 2782–2787).

  • Karaman, S., & Frazzoli, E. (2011). Sampling-based algorithms for optimal motion planning. International Journal of Robotics Research, 30(7), 846–894.

    Article  MATH  Google Scholar 

  • Kuffner, J. J., & LaValle, S. M. (2000). RRT-connect: An efficient approach to single-query path planning. In IEEE international conference on robotics and automation (ICRA), 2 (pp. 995–1001).

  • Li, X., Yu, W., Lin, X., & Iyengar, S. (2012). On optimizing autonomous pipeline inspection. IEEE Transactions on Robotics, 28(1), 223–233.

    Article  Google Scholar 

  • Liu, M., Colas, F., Oth, L., & Siegwart, R. (2015). Incremental topological segmentation for semi-structured environments using discretized GVG. Autonomous Robots, 38(2), 143–160.

    Article  Google Scholar 

  • Moorehead, S. J. (2001). Autonomous surface exploration for mobile robots. Pittsburgh: Carnegie Mellon University.

    Google Scholar 

  • Mur-Artal, R., & Tardós, J. D. (2017). ORB-SLAM2: An open-source slam system for monocular, stereo, and RGB-D cameras. IEEE Transactions on Robotics, 33(5), 1255–1262.

    Article  Google Scholar 

  • Oksanen, T., & Visala, A. (2009). Coverage path planning algorithms for agricultural field machines. Journal of Field Robotics, 26(8), 651–668.

    Article  MATH  Google Scholar 

  • Oßwald, S., Bennewitz, M., Burgard, W., & Stachniss, C. (2016). Speeding-up robot exploration by exploiting background information. IEEE Robotics and Automation Letters, 1(2), 716–723.

    Article  Google Scholar 

  • Papadopoulos, G., Kurniawati, H., & Patrikalakis, N. M. (2013). Asymptotically optimal inspection planning using systems with differential constraints. In IEEE international conference on robotics and automation (ICRA).

  • Papon, J., Abramov, A., Schoeler, M., & Worgotter, F. (2013). Voxel cloud connectivity segmentation-supervoxels for point clouds. In IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2027–2034).

  • Ramanagopal, M. S., Phu-Van Nguyen, A., & Le Ny, J. (2018). A motion planning strategy for the active vision-based mapping of ground-level structures. IEEE Transactions on Automation Science and Engineering, 15(1), 356–368.

    Article  Google Scholar 

  • Roberts, M., Dey, D., Truong, A., Sinha, S., Shah, S., Kapoor, A., Hanrahan, P., & Joshi, N. (2017). Submodular trajectory optimization for aerial 3d scanning. In International conference on computer vision (ICCV).

  • Schönberger, J. L., Zheng, E., Frahm, J. M., & Pollefeys, M. (2016). Pixelwise view selection for unstructured multi-view stereo. In European conference on computer vision (ECCV) (pp. 501–518). Springer.

  • Shade, R., & Newman, P. (2011). Choosing where to go: Complete 3d exploration with stereo. In: IEEE international conference on robotics and automation (ICRA), pp 2806–2811.

  • Shen, S., Michael, N., & Kumar, V. (2012). Stochastic differential equation-based exploration algorithm for autonomous indoor 3d exploration with a micro-aerial vehicle. International Journal of Robotics Research, 31(12), 1431–1444.

    Article  Google Scholar 

  • Shnaps, I., & Rimon, E. (2014). Online coverage by a tethered autonomous mobile robot in planar unknown environments. IEEE Transactions on Robotics, 30(4), 966–974.

    Article  Google Scholar 

  • Song, S., & Jo, S. (2017). Online inspection path planning for autonomous 3d modeling using a micro-aerial vehicle. In IEEE international conference on robotics and automation (ICRA) (pp. 6217–6224).

  • Song, S., & Jo, S. (2018). Surface-based exploration for autonomous 3D modeling. In: IEEE international conference on robotics and automation (ICRA) (pp. 1–8).

  • Song, S., Kim, D., & Jo, S. (2020). Active 3D modeling via online multi-view stereo. In IEEE international conference on robotics and automation (ICRA).

  • Vasquez-Gomez, J. I., Sucar, L. E., & Murrieta-Cid, R. (2014). View planning for 3d object reconstruction with a mobile manipulator robot. In IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 4227–4233).

  • Vasquez-Gomez, J. I., Sucar, L. E., & Murrieta-Cid, R. (2017). View/state planning for three-dimensional object reconstruction under uncertainty. Autonomous Robots, 41(1), 89–109.

    Article  Google Scholar 

  • Vasquez-Gomez, J. I., Sucar, L. E., Murrieta-Cid, R., & Herrera-Lozada, J. C. (2018). Tree-based search of the next best view/state for three-dimensional object reconstruction. International Journal of Advanced Robotic Systems, 15(1), 1729881418754575.

    Article  Google Scholar 

  • Vidal, E., Hernández, J. D., Istenic, K., & Carreras, M. (2017). Online view planning for inspecting unexplored underwater structures. IEEE Robotics and Automation Letters, 2(3), 1436–1443.

    Article  Google Scholar 

  • Wang, Y., James, S., Stathopoulou, E. K., Beltrán-González, C., Konishi, Y., & Del Bue, A. (2019). Autonomous 3-d reconstruction, mapping, and exploration of indoor environments with a robotic arm. IEEE Robotics and Automation Letters, 4(4), 3340–3347.

    Article  Google Scholar 

  • Yamauchi, B. (1997). A frontier-based approach for autonomous exploration. In IEEE international symposium on computational intelligence in robotics and automation (pp. 146–151).

  • Yang, S. X., & Luo, C. (2004). A neural network approach to complete coverage path planning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 34(1), 718–724.

    Article  Google Scholar 

  • Zelinsky, A., Jarvis, R. A., Byrne, J., & Yuta, S. (1993). Planning paths of complete coverage of an unstructured environment by a mobile robot. In International conference on advanced robotics, 13 (pp. 533–538).

  • Zhu, D., Li, T., Ho, D., Wang, C., & Meng, M. Q. H. (2018). Deep reinforcement learning supervised autonomous exploration in office environments. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 7548–7555). IEEE.

Download references

Acknowledgements

This work was supported by Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education (Grant No. NRF-2016R1D1A1B01013573).

Author information

Authors and Affiliations

  1. School of Computing, KAIST, Daejeon, 34141, Yuseong-gu, Republic of Korea

    Soohwan Song, Daekyum Kim & Sungho Jo

Authors
  1. Soohwan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daekyum Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sungho Jo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sungho Jo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 42619 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, S., Kim, D. & Jo, S. Online coverage and inspection planning for 3D modeling. Auton Robot 44, 1431–1450 (2020). https://doi.org/10.1007/s10514-020-09936-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-020-09936-7

Keywords

Search

Navigation