Charalampos Symeonidis
ABSTRACT
Recent advances in artificial intelligence, control and sensing technologies have facilitated the development of autonomous Unmanned Aerial Vehicles (UAVs, or drones) able to self-navigate in various settings. Although these technologies have already entered a mature stage, ensuring flight safety in crowded areas or performing an emergency landing in case of malfunctions, while adhering to relevant legislation, is generally treated as an afterthought when designing autonomous UAV platforms for unstructured environments. This paper proposes a UAV safe landing navigation pipeline that relies on lightweight computer vision modules, able to be executed on the limited computational resources on-board a typical UAV. Pre-trained Deep Neural Networks (DNNs) are mainly employed as the underlying building blocks, since deep learning has made a major impact on robotic perception by drastically improving the performance of relevant tasks, such as object detection or tracking, semantic image segmentation, etc. Evaluation of the proposed pipeline on a simulated environment indicates highly favorable results.
- Mademlis, I., Mygdalis, V., Nikolaidis, N., & Pitas, I. (2018). Challenges in autonomous UAV cinematography: an overview. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME).Google Scholar
Cross Ref
- Mademlis, I., Nikolaidis, N., Tefas, A., Pitas, I., Wagner, T., & Messina, A. (2018). Autonomous unmanned aerial vehicles filming in dynamic unstructured outdoor environments. IEEE Signal Processing Magazine, vol. 36, pp. 147–153.Google ScholarCross Ref
- Mittal, M., Mohan, R., Burgard, W & Valada, A. (2019). Vision-Based Autonomous UAV Navigation and Landing for Urban Search and Rescue, arXiv preprint arXiv:1906.01304.Google Scholar
- Hinzmann, T., Stastny, T., Lerma, C.C., Siegwart, R. & Gilitschenski, I. (2018). Free LSD: Prior-free visual landing site detection for autonomous planes, IEEE Robotics and Automation Letters, vol. 3, pp. 2545-4552.Google ScholarCross Ref
- Lee, M.-F. R., Aayush, J., Saurav, K. & Anshuman, D.A. (2020). Landing Site Inspection and Autonomous Pose Correction for Unmanned Aerial Vehicles, In Proceedings of the International Conference on Advanced Robotics and Intelligent Systems (ARIS).Google ScholarCross Ref
- Demirhan, M. & Premachandra, C. (2020). Development of an Automated Camera-Based Drone Landing System, IEEE Access.Google ScholarCross Ref
- Yang, T., Li, P., Zhang, H., Li, J. & Li, Z. (2018). Monocular Vision SLAM-Based UAV Autonomous Landing in Emergencies and Unknown Environments, Electronics, vol. 7, pp. 73.Google ScholarCross Ref
- Hornung, A., Wurm, K. M., Bennewitz, M., Stachniss, C., & Burgard, W. (2013). Octomap: An efficient probabilistic 3D mapping framework based on octrees. Autonomous Robots, vol. 34, no. 3, pp. 189–206.Google ScholarDigital Library
- Guo, X., Denman, S., Fookes, C., & Sridharan, S. (2016). A robust UAV landing site detection system using mid-level discriminative patches. In Proceedings of the International Conference on Pattern Recognition (ICPR).Google ScholarCross Ref
- Guo, X., Denman, S., Fookes, C., Mejias, L., & Srid-haran, S. (2014). Automatic UAV forced landing site detection using machine learning. In Proceedings of International Conference on Digital Image Computing: Techniques and Applications (DICTA).Google ScholarCross Ref
- Garg, M., Kumar, A., & P.B., S. (2015). Terrain-based landing site selection and path planning for fixed-wing UAVs. In Proceedings of the IEEE International Conference on Unmanned Aircraft Systems (ICUAS).Google ScholarCross Ref
- Kakaletsis, E., & Nikolaidis, N. (2019). Potential UAV landing sites detection through digital elevation models analysis. In European Signal Processing Conference (EUSIPCO), Satellite Workshops.Google Scholar
- Kakaletsis, E., Tzelepi, M., Kaplanoglou, P. I., Symeonidis, C., Nikolaidis, N., Tefas, A. & Pitas, I. (2019). Semantic map annotation through UAV video analysis using deep learning models in ROS. In proceedings of the International Conference on Multimedia Modeling.Google ScholarCross Ref
- Hosang, J., Omran, M., Benenson, R., & Schiele, B. (2015). Taking a deeper look at pedestrians. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Zhang, S., Benenson, R., Omran, M., Hosang, J., & Schiele, B. (2016). How far are we from solving pedestrian detection? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarDigital Library
- Lan, W., Dang, J., Wang, Y., & Wang, S. (2018). Pedestrian detection based on YOLO network model. In Proceedings of the IEEE International Conference on Mechatronics and Automation (ICMA).Google ScholarDigital Library
- Redmon, J., & Farhadi, A. (2016). YOLO9000: Better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Chang, Y.-C., Chen, H.-T., J.-H., C., & Liao, I.-C. (2018). Pedestrian detection in aerial images using vanishing point transformation and deep learning. In Proceedings of the IEEE International Conference on Image Processing (ICIP).Google ScholarCross Ref
- Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017). Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 318-327.Google ScholarCross Ref
- Lin, T. Y., Dollar, P., Girshick, R., He, K., Hariharan B., & Belongie, S. (2017). Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Symeonidis, C., Mademlis, I., Nikolaidis, N., & Pitas, I. (2019). Improving neural non-maximum suppression for object detection by exploiting interest-point detectors. In Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP).Google ScholarCross Ref
- Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Yu, F., & Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122.Google Scholar
- Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017). Pyramid scene parsing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Yuan, Y., Chen, X., & Wang, J. (2019). Object-contextual representations for semantic segmentation. arXiv preprint arXiv:1909.11065.Google Scholar
- Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., & Sang, N. (2018). Bisenet: Bilateral segmentation network for real-time semantic segmentation. In Proceedings of the European Conference on Computer Vision (ECCV).Google ScholarDigital Library
- Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., . . . Ng, A. (2009). ROS: an open-source Robot Operating System. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) Workshop on Open Source Robotics.Google Scholar
- Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017). Scene parsing through ADE20K dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Duchoň, F., Babinec, A., Kajan, M., Beňo, P., Florek, M., Fico, T., & Jurišica, L. (2014). Path planning with modified a-star algorithm for a mobile robot. Procedia Engineering, vol. 96, pp. 59–69.Google ScholarCross Ref
- Shah, S., Dey, D., Lovett, C., Kapoor, A., & Burgard, W. (2017). Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Proceedings of the Field and Service Robotics Conference.Google Scholar
- Stöcker, C., Bennett, R., Nex, F., Gerke, M., & Zevenbergen, J. (2017). Review of the current state of UAV regulations. Remote Sensing, vol. 9, pp. 459.Google ScholarCross Ref
- Ren, S., He, K., Gurshick, R. & Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, Advances in Neural Information Processing Systems (NIPS), vol 28, pp 91-99.Google Scholar
Recommendations
Computer Vision for Autonomous UAV Flight Safety: An Overview and a Vision-based Safe Landing Pipeline Example
Recent years have seen an unprecedented spread of Unmanned Aerial Vehicles (UAVs, or “drones”), which are highly useful for both civilian and military applications. Flight safety is a crucial issue in UAV navigation, having to ensure accurate compliance ...
Vision-Based Safe Landing of UAV using Tiny-SURF Algorithm
2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC)A general rule of flying is applicable when developing an aircraft system and the same rule is applicable on Unmanned Aerial Vehicles (UAVs) as well when developing an intelligent autonomous flying machine. The rule states, "Take-off is optional but ...
A Vision-Based Guidance System for UAV Navigation and Safe Landing using Natural Landmarks
In this paper a vision-based approach for guidance and safe landing of an Unmanned Aerial Vehicle (UAV) is proposed. The UAV is required to navigate from an initial to a final position in a partially known environment. The guidance system allows a ...
Comments