Skip to main content
SpringerLink
Log in

Scalable Markov chain approximation for a safe intercept navigation in the presence of multiple vehicles

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

This paper studies a safe intercept navigation which accounts for the uncertainty of other vehicles’ trajectories, avoids collisions and any other positions in which vehicle safety is compromised. Since the number of vehicles can vary with time, it is important that the navigation strategy can quickly adjust to the current number of vehicles, i.e, that it scales well with the number of vehicles. The scalable strategy is based on a stochastic optimal control problem formulation of safe navigation in the presence of a single vehicle, denoted as the one-on-one vehicle problem. It is shown that safe navigation in the presence of multiple vehicles can be solved exactly as an auxiliary Markov decision problem. This allows us to approximate the solution based on the one-on-one vehicle optimal control solution and achieve scalable navigation. Our work is illustrated by a numerical example of safely navigating a vehicle in the presence of four other vehicles and by a robot experiment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Aigner, M., & Fromme, M. (1984). A game of cops and robbers. Discrete Applied Mathematics, 8(1), 1–12.

    Article  MathSciNet  MATH  Google Scholar 

  • Alonso-Mora, J., Breitenmoser, A., Rufli, M., Beardsley, P., & Siegwart, R. (2013). Optimal reciprocal collision avoidance for multiple non-holonomic robots (pp. 203–216). Berlin, Heidelberg: Springer.

    Google Scholar 

  • Anderson, R., & Milutinović, D. (2011). A stochastic approach to dubins feedback control for target tracking. In 2011 IEEE/RSJ international conference on intelligent robots and systems (pp. 3917–3922). https://doi.org/10.1109/IROS.2011.6094760.

  • Anderson, R. P., & Milutinović, D. (2014). A stochastic approach to dubins vehicle tracking problems. IEEE Transactions on Automatic Control, 59(10), 2801–2806. https://doi.org/10.1109/TAC.2014.2314224.

    Article  MathSciNet  MATH  Google Scholar 

  • Ardema, M. D., Heymann, M., & Rajan, N. (1985). Combat games. Journal of Optimization Theory and Applications, 46(4), 391–398.

    Article  MathSciNet  MATH  Google Scholar 

  • Eklund, J., Sprinkle, J., Kim, H., & Sastry, S. (2005). Implementing and testing a nonlinear model predictive tracking controller for aerial pursuit/evasion games on a fixed wing aircraft. In 2005 American control conference (ACC) (Vol. 3, pp. 1509–1514).

  • Festa, A., & Vinter, R. B. (2016). Decomposition of differential games with multiple targets. Journal of Optimization Theory and Applications, 169, 849–875.

    Article  MathSciNet  MATH  Google Scholar 

  • Fleming, W. H., & Rishel, R. W. (1975). Deterministic and stochastic optimal control. New York: Springer.

    Book  MATH  Google Scholar 

  • Gardiner, C. (2009). Stochastic methods: A handbook for the natural and social sciences. Berlin, Heidelberg: Springer.

    MATH  Google Scholar 

  • Getz, W. M., & Leitmann, G. (1979). Qualitative differential games with two targets. Journal of Mathematical Analysis and Applications, 68, 421–430.

    Article  MathSciNet  MATH  Google Scholar 

  • Getz, W. M., & Pachter, M. (1981). Capturability in a two-target “game of two cars”. Journal of Guidance and Control, 4(1), 15–22.

    Article  MATH  Google Scholar 

  • Grimm, W., & Well, K. H. (1991). Modelling air combat as differential game recent approaches and future requirements. In R. P. Hämäläinen, & H. K. Ehtamo (Eds.), Differential games—Developments in modelling and computation. Lecture notes in control and information sciences (Vol. 156). Berlin, Heidelberg: Springer.

  • Hashemi, A., Casbeer, D. W., & Milutinović, D. (2016). Scalable value approximation for multiple target tail-chase with collision avoidance. In 2016 IEEE 55th conference on decision and control (CDC) (pp. 2543–2548). https://doi.org/10.1109/CDC.2016.7798645.

  • Hoy, M., Matveev, A., & Savkin, A. (2015). Algorithms for collision-free navigation of mobile robots in complex cluttered environments: A survey. Robotica, 33(3), 463–497.

    Article  Google Scholar 

  • Huang, H., Ding, J., Zhang, W., & Tomlin, C. J. (2015). Automation-assisted capture-the-flag: A differential game approach. IEEE Transactions on Control Systems Technology, 23(3), 1014–1028.

    Article  Google Scholar 

  • Isaacs, R. (1965). Differential games. New York, NY: Wiley.

    MATH  Google Scholar 

  • Israelsen, B. W., Ahmed, N., Center, K., Green, R., & Bennett Jr., W. (2017). Adaptive simulation-based training of ai decision-makers using bayesian optimization. arxiv:1703.09310.

  • Kushner, H. J., & Dupuis, P. (2001). Numerical methods for stochastic control problems in continuous time, stochastic modelling and applied probability (Vol. 24). New York, NY: Springer.

    Book  MATH  Google Scholar 

  • Li, D., Cruz, J. B., & Schumacher, C. J. (2008). Stochastic multi-player pursuit-evasion differential games. International Journal of Robust and Nonlinear Control, 18(6), 218–247.

    Article  MathSciNet  MATH  Google Scholar 

  • McGrew, J. S., How, J. P., Williams, B., & Roy, N. (2010). Air-combat strategy using approximate dynamic programming. Journal of Guidance, Control, and Dynamics, 33(5), 1509–1514.

    Article  Google Scholar 

  • Milutinović, D., Casbeer, D. W., Kingston, D., & Rasmussen, S. A. (2017). Stochastic approach to small uav feedback control for target tracking and blind spot avoidance. In Proceedings of the 1st IEEE conference on control technology and applications.

  • Munishkin, A. A., Milutinović, D., & Casbeer, D. W. (2016). Stochastic optimal control navigation with the avoidance of unsafe configurations. In 2016 international conference on unmanned aircraft systems (ICUAS) (pp. 211–218). https://doi.org/10.1109/ICUAS.2016.7502568.

  • Panagou, D., Stipanović, D. M., & Voulgaris, P. G. (2016). Distributed coordination control for multi-robot networks using Lyapunov-like barrier functions. IEEE Transactions on Automatic Control, 61(3), 617–632.

    Article  MathSciNet  MATH  Google Scholar 

  • Powell, W. B. (2009). What you should know about approximate dynamic programming. Naval Research Logistics (NRL), 56(3), 239–249.

    Article  MathSciNet  MATH  Google Scholar 

  • Song, Q., & Yin, G. G. (2010). Convergence rates of Markov chain approximation methods for controlled diffusions with stopping. Journal of Systems Science and Complexity, 23(3), 600–621.

    Article  MathSciNet  MATH  Google Scholar 

  • Vidal, R., Shakernia, O., Kim, H. J., Shim, D. H., & Sastry, S. (2002). Probabilistic pursuit-evasion games: Theory, implementation, and experimental evaluation. IEEE Transactions on Robotics and Automation, 18(5), 662–669.

    Article  Google Scholar 

  • Vieira, M. A. M., Govindan, R., & Sukhatme, G. S. (2009). Scalable and practical pursuit-evasion with networked robots. Intelligent Service Robotics, 2(4), 247.

    Article  Google Scholar 

  • Virtanen, K., Karelahti, J., & Raivio, T. (2006). Modeling air combat by a moving horizon influence diagram game. Journal of Guidance, Control, and Dynamics, 29(5), 1509–1514.

    Article  Google Scholar 

  • Wang, L., Ames, A. D., & Egerstedt, M. (2017). Safety barrier certificates for collisions-free multirobot systems. IEEE Transactions on Robotics, 33(3), 661–674. https://doi.org/10.1109/TRO.2017.2659727.

    Article  Google Scholar 

  • Yavin, Y. (1988). Stochastic two-target pursuit-evasion differential games in the plane. Journal of Optimization Theory and Applications, 56(3), 325–343.

    Article  MathSciNet  MATH  Google Scholar 

  • Yavin, Y., & Villers, R. D. (1988). Stochastic pursuit-evasion differential games in 3D. Journal of Optimization Theory and Applications, 56(3), 345–357.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

Funding was provided by U.S. Department of Defense (Grant No. FA8650-15-D-2516).

Author information

Authors and Affiliations

  1. Computer Engineering, University of California, Santa Cruz, CA, 95064, USA

    Alexey A. Munishkin & Dejan Milutinović

  2. Control Science Center of Excellence, Air Force Research Laboratory, Wright-Patterson AFB, Dayton, OH, 45433, USA

    Araz Hashemi & David W. Casbeer

Authors
  1. Alexey A. Munishkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Araz Hashemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David W. Casbeer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dejan Milutinović
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dejan Milutinović.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 1917 KB)

Supplementary material 2 (mp4 6697 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Munishkin, A.A., Hashemi, A., Casbeer, D.W. et al. Scalable Markov chain approximation for a safe intercept navigation in the presence of multiple vehicles. Auton Robot 43, 575–588 (2019). https://doi.org/10.1007/s10514-018-9739-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-018-9739-0

Keywords

Search

Navigation