Skip to main content
SpringerLink
Log in

On the Hamilton-Jacobi-Bellman equations

  • Published:
Acta Applicandae Mathematica Aims and scope Submit manuscript

Abstract

We consider general problems of optimal stochastic control and the associated Hamilton-Jacobi-Bellman equations. We recall first the usual derivation of the Hamilton-Jacobi-Bellman equations from the Dynamic Programming Principle. We then show and explain various results, including (i) continuity results for the optimal cost function, (ii) characterizations of the optimal cost function as the maximum subsolution, (iii) regularity results, and (iv) uniqueness results. We also develop the recent notion of viscosity solutions of Hamilton-Jacobi-Bellman equations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alexandrov, A. D.:Ucen. Zap. Leningard Gos. Univ. Ser. Fiz. Nauk 37 (1939), 3–35 (in Russian).

    Google Scholar 

  2. Belbas, S. and Lenhart, S.: A system of nonlinear partial differential equations arising in the optimal control of stochastic systems with switching costs', preprint.

  3. Bellman, R.:Dynamic Programming, Princeton Univ. Press, Princeton, N.J., 1957.

    Google Scholar 

  4. Bensoussan, A. and Lions, J. L.:Applications des inéquations variationnelles en contrôle stochastique, Dunod, Paris, 1978.

    MATH  Google Scholar 

  5. Brèzis, H. and Evans, L. C.:Arch. Rat. Mech. Anal. 41 (1979), 1–13.

    Article  Google Scholar 

  6. Busemann, H.:Convex Surfaces, Interscience, New York, 1958.

    MATH  Google Scholar 

  7. Coron, J. M. and Lions, P. L.: To appear.

  8. Crandall, M. G., Evans, L. C., and Lions, P. L.: ‘Some properties of viscosity solutions of Hamilton-Jacobi equations’, to appear inTrans. Amer. Math. Soc. (1983).

  9. Crandall, M. G. and Lions, P. L.:Comptes-Rendus Acad. Sc. Paris Sér. A,252 (1981), 183–186.

    MathSciNet  Google Scholar 

  10. Crandall, M. G. and Lions, P. L.: ‘Viscosity solutions of Hamilton-Jacobi equations’, to appear inTrans. Amer. Math. Soc. (1983).

  11. Crandall, M. G. and Lions, P. L.: ‘Two approximations of solutions of Hamilton-Jacobi equations’, preprint.

  12. Evans, L. C.:Comm. Pure Appl. Math. (1982).

  13. Evans, L. C.: ‘Classical solutions of the Hamilton-Jacobi-Bellman equation for uniformly elliptic operators’, to appear inTrans. Amer. Math. Soc. (1983).

  14. Evans, L. C. and Friedman, A.:Trans. Amer. Math. Soc. 253 (1979), 365–389.

    Article  MATH  MathSciNet  Google Scholar 

  15. Evans, L. C. and Lions, P. L.:Comptes-Rendus Acad. Sc. Paris Sér. A,290 (1980), 1049–1052.

    MATH  MathSciNet  Google Scholar 

  16. Fleming, W. H. and Rishel, R.:Deterministic and Stochastic Optimal Control, Springer, Berlin, 1975.

    MATH  Google Scholar 

  17. Genis, T. L. and Krylov, N. V.:Th. Proba. Appl. 21 (1976), 148–152.

    Article  MATH  MathSciNet  Google Scholar 

  18. Itô, K.: to appear.

  19. Jensen, R. and Lions, P. L.: ‘Some asymptotic problems in fully nonlinear equations and optimal stochastic control’, preprint (to appear inAnn. Sc. Norm. Sup. Pisa).

  20. Krylov, N. V.:Th. Proba. Appl. 17 (1972), 114–131.

    Article  MATH  Google Scholar 

  21. Krylov, N. V.:Controlled Diffusion Processes, Springer, Berlin, 1980.

    MATH  Google Scholar 

  22. Krylov, N. V.:Math. U.S.S.R. Izv. 7 (1973), 691–708.

    Article  MATH  MathSciNet  Google Scholar 

  23. Krylov, N. V.:Math. U.S.S.R. Izv. 6 (1972), 249–262.

    Article  MATH  Google Scholar 

  24. Krylov, N. V.: ‘Control of the diffusion-type processes’,Proc. Internat. Congress of Mathematicians, Helsinki, 1978.

  25. Krylov, N. V.:Math. U.S.S.R. Sbornik,32 (1980), 133–149.

    Article  Google Scholar 

  26. Krylov, N. V.:Th. Proba. Appl. 18 (1973), 54–63.

    Article  MATH  Google Scholar 

  27. Krylov, N. V.:Math. U.S.S.R. Izv. 8 (1974), 233–254.

    Article  Google Scholar 

  28. Krylov, N. V.:Sibirsk Mat. 17 (1976), 226–236.

    Article  MATH  MathSciNet  Google Scholar 

  29. Krylov, N. V. and Pragarauskas, G.:Liet. Mat. Rink 21 (1981), 101–110 (in Russian).

    MATH  MathSciNet  Google Scholar 

  30. Lasry, J. M. and Lions, P. L.: To appear.

  31. Lenhart, S.: ‘Partial differential equations from dynamic programming equations’, PhD Thesis, Univ. of Kentucky-Lexington, 1981.

  32. Lions, P. L.: ‘Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part 1: The dynamic programming principle and applications’, to appear inComm. P.D.E. (1983).

  33. Lions, P. L.:Comm. Pure Appl. Math. 34 (1981), 121–147.

    MATH  MathSciNet  Google Scholar 

  34. Lions, P. L.: ‘Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part 2: Viscosity solutions and uniqueness’, to appear inComm. P.D.E. (1983).

  35. Lions, P. L.: ‘Optimal stochastic control of diffusion type processes and Hamilton-Jacobi-Bellman equations’, in W. H. Fleming and L. Gorostiza (eds.),Advances in Filtering and Optimal Stochastic Control, Springer, Berlin, 1982.

    Google Scholar 

  36. Lions, P. L.: ‘Fully nonlinear elliptic equations and applications’,Nonlinear Analysis, Function Spaces and Applications, Teubner, Leipzig, 1982.

    Google Scholar 

  37. Lions, P. L.: ‘Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part 3’.Nonlinear partial differential equations and their applications, Collége de France Seminar, Vol. 5, Pitman, London, 1983.

    Google Scholar 

  38. Lions, P. L.: ‘Contrôle optimal stochastique et equations de Hamilton-Jacobi-Bellman’,Comptes-Rendus Acad. Sc. Paris Sér. A 295 (1982).

  39. Lions, P. L.: ‘A remark on Bony maximum principle, to appear inProc. A.M.S. (1983).

  40. Lions, P. L.:Comptes-Rendus Paris,289 (1979), 329–332.

    MATH  Google Scholar 

  41. Lions, P. L.: ‘Equations de Hamilton-Jacobi-Bellman’,Seminaire Goulaouic Schwartz 1979–1980, Ecole Polytechnique, Palaiseau.

    Google Scholar 

  42. Lions, P. L.: ‘Optimal stochastic control and Hamilton-Jacobi-Bellman equations’,Mathematical Control Theory, Banach Centre Publications, Warsaw, to appear.

  43. Lions, P. L.:Comm. PDE 5 (1980), 753–771.

    Article  MATH  Google Scholar 

  44. Lions, P. L.:Comptes-Rendus Paris 288 (1979), 339–342.

    MATH  Google Scholar 

  45. Lions, P. L.:Comptes-Rendus Paris 287 (1978), 747–750.

    MATH  Google Scholar 

  46. Lions, P. L.:Acta Math. 146 (1981), 121–147.

    Article  Google Scholar 

  47. Lions, P. L.: ‘Bifurcation and optimal stochastic control’, to appear inNonlinear Anal. TMA.

  48. Lions, P. L.: to appear.

  49. Lions, P. L.:Generated Solutions of Hamilton-Jacobi Equations, Pitman, London, 1982.

    Google Scholar 

  50. Lions, P. L.: to appear.

  51. Lions, P. L.: to appear.

  52. Lions, P. L. and Menaldi, J. L.:SIAM J. Control Optim. 20 (1982), 58–81.

    Article  MATH  MathSciNet  Google Scholar 

  53. Lions, P. L. and Menaldi, J. L.:SIAM J. Control Optim. 20 (1982), 82–95.

    Article  MATH  MathSciNet  Google Scholar 

  54. Lions, P. L. and Menaldi, J. L.:Comptes-Rendus Paris,287 (1978), 503–506.

    MATH  MathSciNet  Google Scholar 

  55. Lions, P. L. and Mercier, B.:RAIRO 14 (1980), 369–393.

    MATH  MathSciNet  Google Scholar 

  56. Lions, P. L. and Nisio, M.: ‘A uniqueness result for the semigroup associated with the Hamilton-Jacobi-Bellman operator’,Proc. Japan Acad.

  57. Lions, P. L., Papanicolaou, G. and Varadhan, S. R. S.: to appear.

  58. Nisio, M.: ‘Some remarks on stochastic optimal controls’,Proc. 3rd. U.S.S.R.-Japan Symp. Proba. Theory, Lecture Notes in Maths 550, Springer, Berlin, 1976.

    Google Scholar 

  59. Nisio, M.:Lectures on Stochastic Control Theory, ISI Lecture Notes 9. Macmillan India Bombay.

  60. Nisio, M.:Publ. RIMS Kyoto Univ. 12 (1976), 513–537.

    Article  MATH  MathSciNet  Google Scholar 

  61. Nisio, M.: ‘On stochastic optimal controls and envelope of Markovian semigroups’,Proc. Int. Symp. SDE, Kyoto 1976, Kinskuniga, Tokyo, 1978.

  62. Quadrat, J. P.: ‘Analyse numèrique de l'équation de Bellman stochastique’, Rapport Laboria 140 (1974), INRIA, Rocquencourt.

    Google Scholar 

  63. Safonov, M. V.:Math. U.S.S.R. Sbornik 31 (1977), 231–248.

    Article  MATH  MathSciNet  Google Scholar 

  64. Safonov, M. V.:Math. U.S.S.R. Sbornik 34 (1978), 521–526.

    Article  MATH  MathSciNet  Google Scholar 

  65. Stroock, D. W. and Varadhan, S. R. S.:Multidimensional Diffusion Processes, Springer, Berlin, 1979.

    MATH  Google Scholar 

  66. Trudinger, N. S.: ‘Elliptic equations in nondivergence form’,Proc. Miniconference on Partial Differential Equations, Canberra, 1981.

  67. Trudinger, N. S.: ‘Fully nonlinear uniformly elliptic equations under natural structure conditions’, to appear inTrans. Amer. Math. Soc. (1983).

Download references

Author information

Authors and Affiliations

  1. Ceremade, Universitè Paris IX-Dauphine, Place de Lattre de Tassigny, 75775, Paris Cedex 16, France

    P. L. Lions

Authors
  1. P. L. Lions
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lions, P.L. On the Hamilton-Jacobi-Bellman equations. Acta Appl Math 1, 17–41 (1983). https://doi.org/10.1007/BF02433840

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02433840

AMS (MOS) subject classification (1980)

Key words

Search

Navigation