Skip to main content
SpringerLink
Log in

Mobile robot monocular vision-based obstacle avoidance algorithm using a deep neural network

  • Special Issue
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

In recent decades, obstacle avoidance has been one of the main challenges in autonomous vehicle navigation, which has been used in a wide range of different autonomous devices and it is also a fundamental requirement for mobile robots. In this paper, a novel method is proposed and evaluated for mobile robot navigation in an unknown environment while avoiding collision with unexpected obstacles. This mobile robot has been equipped with a mono-vision sensor, which communicates with the environment. Therefore, the decision-making unit is entirely vision-based and no other sensor is used to communicate with the environment. The deep neural network method has been used to inform the robot about the suitable direction of movement but central decision-making unit will do some preprocessing on the output command before sending it to the operation unit. In this research, various techniques are investigated to train the convolutional neural network in a better way to generate more accurate commands for the mobile robot’s movement towards the safe area without colliding into any obstacles. Besides, lots of various methods have been used to increase the performance of the neural network. The most important one is using an adjustable learning rate. The results show the vital role of learning rate in network performance. The designed network attained a 96.88% accuracy and a 0.14 loss, which indicates that this approach could successfully improve the neural network performance and reasonably solve the obstacle avoidance problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Strömgren O (2018) Deep learning for autonomous collision avoidance. Dissertation, Linkoping University

  2. Kakoty NM, Mazumdar M, Sonowal D (2019) Mobile robot navigation in unknown dynamic environment inspired by human pedestrian behavior. Prog Adv Comput Intell Eng Adv Intell Syst Comput. https://doi.org/10.1007/978-981-13-0224-4_40

    Article  Google Scholar 

  3. Quan H, Li Y, Zhang Y (2020) A novel mobile robot navigation method based on deep reinforcement learning. Int J Adv Rob Syst. https://doi.org/10.1177/1729881420921672

    Article  Google Scholar 

  4. Lee HY, Ho HW, Zhou Y (2020) Deep learning-based monocular obstacle avoidance for unmanned aerial vehicle navigation in tree plantations. J Intell Rob Syst. https://doi.org/10.1007/s10846-020-01284-z

    Article  Google Scholar 

  5. Azizi A, Entesari F, Ghaemi K, Cheragh M (2013) Intelligent mobile robot navigation in an uncertain dynamic environment. Appl Mech Mater. https://doi.org/10.4028/www.scientific.net/AMM.367.388

    Article  Google Scholar 

  6. Shitsukane A, Cheriuyot W, Otieno C, Mgala M (2018) A survey on obstacle avoidance mobile robot in static unknown environment. Int J Comput (IJC) 28:160–173

    Google Scholar 

  7. Tai L, Li SH, Liu M (2016) A deep-network solution towards model-less obstacle avoidance. In: (IROS 2016) IEEE/RSJ international conference on intelligent robots and systems. https://doi.org/10.1109/IROS.2016.7759428

  8. Rashidnejhad S, Asfia A, Ghaemi K, Meghdari A, Azizi A (2013) Optimal trajectory planning for parallel robots considering time-jerk. Appl Mech Mater. https://doi.org/10.4028/www.scientific.net/AMM.390.471

    Article  Google Scholar 

  9. Lenz I, Lee H, Saxena A (2015) Deep learning for detecting robotic grasps. Int J Robot Res. https://doi.org/10.1177/0278364914549607

    Article  Google Scholar 

  10. Maturana D, Scherer S (2015) 3d convolutional neural networks for landing zone detection from lidar. In: IEEE international conference on robotics and automation (ICRA). https://doi.org/10.1109/ICRA.2015.7139679

  11. Redmon J, Angelova A (2015) Real-time grasp detection using convolutional neural networks. In: 2015 IEEE international conference on robotics and automation (ICRA). https://doi.org/10.1109/ICRA.2015.7139361

  12. Lei T, Ming L (2016) A robot exploration strategy based on Q-learning network. In: (RCAR 2016) IEEE international conference on real-time computing and robotics. https://doi.org/10.1109/RCAR.2016.7784001

  13. Chihyun P, Jihwan H, Sanghyun P (2020) Prediction of Alzheimer’s disease based on deep neural network by integrating gene expression and DNA methylation dataset. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2019.112873

    Article  Google Scholar 

  14. Wei B, Han Z, He X, Yin Y (2017) Deep learning model based breast cancer histopathological image classification. In: 2017 IEEE 2nd international conference on cloud computing and big data analysis (ICCCBDA). https://doi.org/10.1109/ICCCBDA.2017.7951937

  15. Szegedy CH, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2015.7298594

  16. Azizi A, Entessari F, Ghaemi K, Rezaei AR (2014) Introducing neural networks as a computational intelligent technique. Appl Mech Mater. https://doi.org/10.4028/www.scientific.net/AMM.464.369

    Article  Google Scholar 

  17. Azizi A (2020) Applications of artificial intelligence techniques to enhance sustainability of industry 4.0: design of an artificial neural network model as dynamic behavior optimizer of robotic arms. Hindawi. https://doi.org/10.1155/2020/8564140

    Article  Google Scholar 

  18. Wang S, Clark R, Wen H, Trigoni N (2017) Deepvo: towards end-to-end visual odometry with deep recurrent convolutional neural networks. In: 2017 IEEE international conference on robotics and automation (ICRA). https://doi.org/10.1109/ICRA.2017.7989236

  19. Clark R, Wang S, Trigoni N, Markham A, Wen H (2017) Vidloc: a deep spatio-temporal model for 6-dof video-clip relocalization. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2017.284

  20. Clark R, Wang S, Wen H, Markham A, Trigoni N (2017) Vinet: visual-inertial odometry as a sequence-to-sequence learning problem. In: Association for the advancement of artificial intelligence (AAAI)

  21. Tai L, Paolo G, Liu M (2017) Virtual-to-real deep reinforcement learning: continuous control of mobile robots for mapless navigation. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS). https://doi.org/10.1109/IROS.2017.8202134

  22. Zhu Y, Mottaghi R, Kolve E, Lim J, Gupta A, Fei-Fei L, Farhadi A (2017) Target-driven visual navigation in indoor scenes using deep reinforcement learning. In: 2017 IEEE International conference on robotics and automation (ICRA). https://doi.org/10.1109/ICRA.2017.7989381

  23. Richter CH, Roy N (2017) Safe visual navigation via deep learning and novelty detection. Robot Sci Syst XIII. https://doi.org/10.15607/RSS.2017.XIII.064

    Article  Google Scholar 

  24. Zhang J, Springenberg JT, Boedecker J, Burgard W (2017) Deep reinforcement learning with successor features for navigation across similar environments. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS). https://doi.org/10.1109/IROS.2017.8206049

  25. Cimurs R, Lee JH, Suh H (2020) Goal-oriented obstacle avoidance with deep reinforcement learning in continuous action space. Electronics 9(3):411. https://doi.org/10.3390/electronics9030411

    Article  Google Scholar 

  26. Gaya JO, Gonçalves LT, Duarte AC, Zanchetta B, Drews P, Botelho SSC (2016) Vision-based obstacle avoidance using deep learning. In: 2016 XIII Latin American robotics symposium and IV Brazilian robotics symposium (LARS/SBR). https://doi.org/10.1109/LARS-SBR.2016.9

  27. Hsieh Y, Lin W, Li D, Chuang J (2019) Deep learning-based obstacle detection and depth estimation. In: 2019 IEEE international conference on image processing (ICIP). https://doi.org/10.1109/ICIP.2019.8803100

  28. Takiguchi T, Hoon J, Okamoto SH (2018) Collision avoidance algorithm using deep learning type artificial intelligence for a mobile robot. In: Proceedings of the international multiconference of engineers and computer scientists 2018 (IMECS 2018)

  29. Thomas A, Hedley J (2019) FumeBot: a deep convolutional neural network controlled robot. Robotics. https://doi.org/10.3390/robotics8030062

    Article  Google Scholar 

  30. Liu C, Zheng B, Wang CH, Zhao Y, Fu SH, Li H (2017) CNN-based Vision model for obstacle avoidance of mobile robot. In: MATEC web of conferences. https://doi.org/10.1051/matecconf/201713900007

  31. Wu H, Lv D, Cui T et al (2021) SDLV: verification of steering angle safety for self-driving cars. Form Asp Comput. https://doi.org/10.1007/s00165-021-00539-2

    Article  MathSciNet  Google Scholar 

  32. Tai L, Liu M (2016) Mobile robots exploration through CNN-based reinforcement learning. Robot Biomim. https://doi.org/10.1186/s40638-016-0055-x

    Article  Google Scholar 

  33. Chakravarty P, Kelchtermans K, Roussel T, Wellens S, Tuytelaars T, Van Eycken L (2017) CNN-based single image obstacle avoidance on a quadrotor. In: 2017 IEEE international conference on robotics and automation (ICRA). https://doi.org/10.1109/ICRA.2017.7989752

  34. Wen L, Goa L, Li X, Zeng B (2021) Convolutional neural network with automatic learning rate scheduler for fault classification. IEEE Trans Instrum Meas. https://doi.org/10.1109/TIM.2020.3048792

    Article  Google Scholar 

  35. Macias-Garcia E, Galeana-Perez D, Bayro-Corrochano E (2020) CNN based perception system for collision avoidance in mobile robots using stereo vision. In: 2020 International joint conference on neural networks (IJCNN). https://ieeexplore.ieee.org/document/9206747

  36. Park B, Oh H (2018) Obstacle avoidance for UAVs via imitation learning from human data with 3D-convolutional neural networks. In: 10th international micro-air vehicles conference (IMIV 2018)

  37. Ramachandran P, Zoph B, Le QV (2017) searching for activation functions. Google Brain Residency program

  38. Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS (2015) Deep learning for visual understanding: a review. Neurocomputing. https://doi.org/10.1016/j.neucom.2015.09.116

    Article  Google Scholar 

  39. Krizhevsky A, Sutskever I, Hinton GE (2017) imagenet classification with deep convolutional neural networks. Commun ACM. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  40. Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw. https://doi.org/10.1016/j.neunet.2014.09.003

    Article  Google Scholar 

  41. Chollet F (2018) Deep learning with Python and Keras: the practical guide from the developer of the Keras library. Manning Shlter Island, Shelter Island

    Google Scholar 

  42. Siegwart R, Nourbakhsh IR, Scaramuzza D (2011) Introduction to autonomous mobile robots. The MIT Press, Cambridge

    Google Scholar 

  43. Mohanty PK, Parhi D (2013) Controlling the motion of an autonomous mobile robot using various techniques: a review. J Adv Mech Eng 1:24–39. https://doi.org/10.7726/JAME.2013.1003

    Article  Google Scholar 

  44. Lee K, Chung W, Chang H, Yoon P (2007) Odometry calibration of a car-like mobile robot. In: 2007 IEEE international conference on control, automation and systems. https://doi.org/10.1109/ICCAS.2007.4406986

  45. Jha A, Kumar M (2015) Two wheels differential type odometry for mobile robots. In: Proceedings of 3rd international conference on reliability, infocom technologies and optimization. https://doi.org/10.1109/ICRITO.2014.7014709

  46. An W, Wang H, Zhang Y, Dai Q (2017) Exponential decay sine wave learning rate for fast deep neural network training. In: 2017 IEEE visual communications and image processing. https://doi.org/10.1109/VCIP.2017.8305126

  47. Alkurdi DA, Ilyas M, Jamil A (2021) Cancer detection using deep learning techniques. Evolut Intell. https://doi.org/10.1007/s12065-021-00635-5

    Article  Google Scholar 

Download references

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. The Department of Electrical Engineering, Iran University of Science and Technology (IUST), Tehran, Iran

    Niloofar Rezaei & Samira Darabi

Authors
  1. Niloofar Rezaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samira Darabi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Niloofar Rezaei.

Ethics declarations

Conflict of intrest

The authors declare that they have no conflicting of interests.

Human or animal rights

The Research does not involve Human/Animal Participation.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rezaei, N., Darabi, S. Mobile robot monocular vision-based obstacle avoidance algorithm using a deep neural network. Evol. Intel. 16, 1999–2014 (2023). https://doi.org/10.1007/s12065-023-00829-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-023-00829-z

Keywords

Search

Navigation