Abstract
This paper is about vision-based autonomous flight of MAVs at night. Despite it being dark almost half of the time, most of the work to date has addressed only daytime operations. Enabling autonomous night-time operation of MAVs with low SWaP on-board sensing capabilities is still an open problem in current robotics research. In this paper, we take a step in this direction and introduce a robust vision-based perception system using thermal-infrared cameras. We present this in the context of safe autonomous landing on rooftop-like structures, and demonstrate the efficacy of our proposed system through extensive real-world flight experiments in outdoor environments at night.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barfoot, T.D., McManus, C., Anderson, S., Dong, H., Beerepoot, E., Tong, C.H., Furgale, P., Gammell, J.D., Enright, J.: Into darkness: visual navigation based on a lidar-intensity-image pipeline. In: Robotics Research, pp. 487–504. Springer (2016)
Borges, P.V.K., Vidas, S.: Practical infrared visual odometry. IEEE Trans. Intell. Transp. Syst. 17(8), 2205–2213 (2016)
Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J.P., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vis. 28(2), 151–167 (2007)
Brunner, C., Peynot, T., Vidal-Calleja, T., Underwood, J.: Selective combination of visual and thermal imaging for resilient localization in adverse conditions: day and night, smoke and fire. J. Field Robot. 30(4), 641–666 (2013)
Burian, F., Kocmanova, P., Zalud, L.: Robot mapping with range camera, CCD cameras and thermal imagers. In: 2014 19th International Conference on Methods and Models in Automation and Robotics (MMAR), pp. 200–205 (2014)
Chen, L., Sun, L., Yang, T., Fan, L., Huang, K., Xuanyuan, Z.: RGB-T SLAM: a flexible slam framework by combining appearance and thermal information. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 5682–5687 (2017)
Daftry, S., Zeng, S., Khan, A., Dey, D., Melik-Barkhudarov, N., Bagnell, J.A., Hebert, M.: Robust monocular flight in cluttered outdoor environments. arXiv preprint arXiv:1604.04779 (2016)
Delmerico, J., Scaramuzza, D.: A benchmark comparison of monocular visual-inertial odometry algorithms for flying robots. Memory 10, 20 (2018)
Dey, D., Shankar, K.S., Zeng, S., Mehta, R., Agcayazi, M.T., Eriksen, C., Daftry, S., Hebert, M., Bagnell, J.A.: Vision and learning for deliberative monocular cluttered flight. In: Proceedings of the International Conference on Field and Service Robotics (FSR) (2015)
Dubbelman, G., van der Mark, W., van den Heuvel, J.C., Groen, F.C.: Obstacle detection during day and night conditions using stereo vision. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2007, pp. 109–116 (2007)
Engel, J., Koltun, V., Cremers, D.: Direct sparse odometry. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 611–625 (2018)
Fankhauser, P., Bloesch, M., Gehring, C., Hutter, M., Siegwart, R.: Robot-centric elevation mapping with uncertainty estimates. In: Mobile Service Robotics, pp. 433–440. World Scientific (2014)
Forster, C., Faessler, M., Fontana, F., Werlberger, M., Scaramuzza, D.: Continuous on-board monocular-vision-based elevation mapping applied to autonomous landing of micro aerial vehicles. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 111–118 (2015)
Gade, R., Moeslund, T.B.: Thermal cameras and applications: a survey. Mach. Vis. Appl. 25(1), 245–262 (2014)
Han, J., Bhanu, B.: Human activity recognition in thermal infrared imagery. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops 2005 CVPR Workshops, p. 17 (2005)
Husain, A., Jones, H., Kannan, B., Wong, U., Pimentel, T., Tang, S., Daftry, S., Huber, S., Whittaker, W.L.: Mapping planetary caves with an autonomous, heterogeneous robot team. In: Aerospace Conference, pp. 1–13. IEEE (2013)
Mascarich, F., Khattak, S., Papachristos, C., Alexis, K.: A multi-modal mapping unit for autonomous exploration and mapping of underground tunnels. In: 2018 IEEE Aerospace Conference, pp. 1–7 (2018)
Milford, M.J., Wyeth, G.F.: SeqSLAM: visual route-based navigation for sunny summer days and stormy winter nights. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 1643–1649. IEEE (2012)
Mouats, T., Aouf, N., Chermak, L., Richardson, M.A.: Thermal stereo odometry for UAVs. IEEE Sens. J. 15(11), 6335–6347 (2015)
Nelson, P., Churchill, W., Posner, I., Newman, P.: From dusk till dawn: localisation at night using artificial light sources. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 5245–5252 (2015)
Papachristos, C., Mascarich, F., Alexis, K.: Thermal-inertial localization for autonomous navigation of aerial robots through obscurants. In: 2018 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 394–399 (2018)
Pizzoli, M., Forster, C., Scaramuzza, D.: Remode: probabilistic, monocular dense reconstruction in real time. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 2609–2616. IEEE (2014)
Qin, T., Li, P., Shen, S.: VINS-Mono: a robust and versatile monocular visual-inertial state estimator. arXiv preprint arXiv:1708.03852 (2017)
Shen, S., Michael, N., Kumar, V.: Autonomous indoor 3D exploration with a micro-aerial vehicle. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 9–15. IEEE (2012)
Vidas, S., Sridharan, S.: Hand-held monocular slam in thermal-infrared. In: 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV), pp. 859–864 (2012)
Wu, C.: Towards linear-time incremental structure from motion. In: 2013 International Conference on 3D Vision-3DV 2013, pp. 127–134. IEEE (2013)
Zach, C., Gallup, D., Frahm, J.M.: Fast gain-adaptive KLT tracking on the GPU. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2008, pp. 1–7. IEEE (2008)
Acknowledgement
The research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Daftry, S. et al. (2020). Robust Vision-Based Autonomous Navigation, Mapping and Landing for MAVs at Night. In: Xiao, J., Kröger, T., Khatib, O. (eds) Proceedings of the 2018 International Symposium on Experimental Robotics. ISER 2018. Springer Proceedings in Advanced Robotics, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-030-33950-0_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-33950-0_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33949-4
Online ISBN: 978-3-030-33950-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)