Skip to main content
SpringerLink
Log in

A first-order approximation to stochastic optimal control of reservoirs

  • Originals
  • Published:
Stochastic Hydrology and Hydraulics Aims and scope Submit manuscript

Abstract

A new approximate method of solution for stochastic optimal control problems with many state and control variables is introduced. The method is based on the expansion of the optimal control into the deterministic feedback control plus a caution term. The analytic, small-perturbation calculation of the caution term is at the heart of the new method. The developed approximation depends only on the first two statistical moments of the random inputs and up to the third derivatives of the cost functions. Its computational requirements do not exhibit the exponential growth exhibited by discrete stochastic DP and can be used as a suboptimal solution to problems for which application of stochastic DP is not feasible. The method is accurate when the cost-to-go functions are approximately cubic in a neighbourhood around the deterministic trajectory whose size depends on forecasting uncertainty. Furthermore, the method elucidates the stochastic optimization problem yielding insights which cannot be easily obtained from the numerical application of discrete DP.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alarcon, L.F.; Marks, D.H. 1979: A stochastic dynamic programming model for the operation of the High Aswan Dam. R.M. Parsons Laboratory, Massachusetts Institute of Technology. Technical Report No. 246, 109

  • Athans, M.; Schweppe, F.C. 1965: Gradient matrices and matrix calculations. Lincoln Laboratory, Technical Note 1965–53, Massachusetts Institute of Technology, Lexington, Massachusetts

    Google Scholar 

  • Bellman, R. 1957: Dynamic programming. Princeton University Press, Princeton, N.J.

    Google Scholar 

  • Bellman, R. 1960: Introduction to matrix analysis. McGraw-Hill Book Co., New York

    Google Scholar 

  • Bellman, R.; Dreyfus, S.E. 1962: Applied dynamic programming. Princeton University Press, Princeton, N.J.

    Google Scholar 

  • Bodewing, E. 1956: Matrix calculations. North Holland Publishing Co., New York

    Google Scholar 

  • Bogle, M.G.V.; O'Sullivan, M.J. 1979: Stochastic optimization of a water supply system. Water Resour. Res. 15(4), 778–786

    Google Scholar 

  • Bras, R.L.; Buchanan, R.B.; Curry, K.C. 1983: Real-time adaptive closed-loop control of reservoirs with the High Aswan Dam as a case study. Water Resour. Res. 19(1), 33–52

    Google Scholar 

  • Buras, N. 1963: Conjunctive operation of dams and aquifers. J. Hydraulic Div. ASCE 89(HY6), 111–131

    Google Scholar 

  • Butcher, W.S. 1971: Stochastic dynamic programming for optimal reservoir operation. Water Resour. Bull. 7(1), 115–123

    Google Scholar 

  • Collado, J. 1984: parameter and forecasting uncertainties in the optimization of reservoir operation. Ph.D. Thesis, Civil and Environmental Engineering, University of Iowa, Iowa City, Iowa

    Google Scholar 

  • Dudley, N.J.; Burt, O.R. 1973: Stochastic reservoir management and system design for irrigation. Water Resour. Res. 9(3), 507–522

    Google Scholar 

  • Esmail-Beik, S.; Yu, Y.S. 1984: Optimal operation of multipurpose pool of Elk-City Lake. J. Water Resour. Plann. Manage. Div., ASCE 110(WR1), 1–4

    Google Scholar 

  • Gablinger, M.; Loucks, D.P. 1970: Markov models for flow regulation. J. Hydraulic Div., ASCE 96(HY1), 165–181

    Google Scholar 

  • Gal, S. 1979: Optimal management of a multireservoir water system. Water Resour. Res. 15(4), 737–749

    Google Scholar 

  • Georgakakos, A.P.; Marks, D.K. 1985: Real-time control of reservoir system. Tech. Rep. 301, R.M. Partons Laboratory for Hydrology and Water Resources, MIT

  • Holtzman, T.M.; Halkin, H. 1966: Convexity and the maximum principle for discrete systems. J. SIAM on Control. 4(2), 263–275

    Google Scholar 

  • Jacobson, D.H.; Mayne, D.Q. 1970: Differential dynamic programming. Elsevier, New York

    Google Scholar 

  • Kitanidis, P.K.; Bras, R.L. 1980: Real-time forecasting with a conceptual hydrologic model. Water Resour. Res. 16(6), 1025–1044

    Google Scholar 

  • Kitanidis, P.K. 1983: Real-time forecasting and stochastic optimal operation of multireservoir systems. IIHR Report No. 258, Institute of Hydraulic Research, The University of Iowa, Iowa City

    Google Scholar 

  • Kitanidis, P.K.; Foufoula-Georgiou, E. 1987: Error analysis of conventional discrete and gradient dynamic programming. Water Resour. Res. 23(5), 845–858

    Google Scholar 

  • Kitanidis, P.K.; Andricevic, R. 1987: Accuracy of the first-order approximation to the stochastic optimal control of reservoirs (in appear) in Dynamic Programming for Optimal Water Resources Systems Management. Elsevier Science Publishers

  • Klemes, V. 1977: Value of information in reservoir optimization. Water Resour. Res. 13(5), 837–850

    Google Scholar 

  • Little, J.D.C. 1955: The use of storage water in a hydroelectric system. J. Oper. Res. Soc. Am. 3, 187–197

    Google Scholar 

  • Loaiciga, H.A.; Marino, M.A. 1985: An approach to parameter estimation and stochastic control in water resources with an application to reservoir operation. Water Resour. Res. 21(11), 1575–1585

    Google Scholar 

  • Loucks, D.P.; Falkson, L.M. 1970: A comparison of some dynamic, linear, and policy iteration methods for reservoir operation. Water Resour. Bull. 6(3), 384–400

    Google Scholar 

  • Maidment, D.R.; Chow, V.T. 1981: Stochastic state variable dynamic programming for reservoir system analysis. Water Resour. Res. 17(6), 1578–1584

    Google Scholar 

  • Masse, P. 1946: Les reserves et la regulation de L'Avenir, Hermann, Paris

    Google Scholar 

  • Murray, D.M.; Yakowitz, S.J. 1979: Constrained differential dynamic programming and its application to reservoir control. Water Resour. Res. 15(5), 1017–1027

    Google Scholar 

  • Nayfeh, A.H. 1973: Perturbation methods. J. Wiley-Interscience Series on Pure and Applied Mathematics, 425

  • Oven-Thompson, K.; Alarcon, L.; Marks, D.H. 1982: Agricultural versus hydropower trade-offs in the operation of the High Aswan Dam. Water Resour. Res. 18(6), 1605–1614

    Google Scholar 

  • ReVelle, C.; Joeres, E.; Kirby, W. 1969: The linear decision rule in reservoir management and design, I. Development of the stochastic model, Water Resour. Res. 5(4)

  • Rhenals, A.E.; Bras, R.L. 1981: The irrigation scheduling problem and evapotranspiration uncertainty. Water Resour. Res. 17(5), 1328–1338

    Google Scholar 

  • Roefs, T.G.; Guitron, R.A. 1975: Stochastic reservoir models: Relative computational effort. Water Resour. Res. 11(6), 801–804

    Google Scholar 

  • Schweig, Z.; Cole, J.A. 1968: Optimal control of linked reservoirs, Water Resour. Res. 4(3), 479–497

    Google Scholar 

  • Schweppe, F.C. 1973: Uncertain dynamic systems. Prentice-hall Englewood Cliffs, New Jersey

    Google Scholar 

  • Stedinger, J.R.; Sale, B.F.; Loucks, D.P. 1984: Stochastic dynamic programming models for reservoir operation optimization. Water Resour. Res. 20(11), 1499–1505

    Google Scholar 

  • Su, S.Y.; Deininger, R.A. 1974: Modeling the regulation of Lake Superior under uncertainty of future water supplies. Water Resour. Res. 10(1), 11–25

    Google Scholar 

  • Takeuchi, K.; Moreau, D.H. 1974: Optimal control of multiunit interbasin water resource systems. Water Resour. Res. 10(3), 407–414

    Google Scholar 

  • Torabi, M.; Mobasheri, F. 1973: A stochastic dynamic programming model for the optimum operation of a multi-purpose reservoir. Water Resour. Bull. 9(6), 1089–1099

    Google Scholar 

  • Turgeon, A. 1980: Optimal operation of multireservoir power systems with stochastic inflows. Water Resour. Res. 16(2), 275–283

    Google Scholar 

  • Turgeon, A. 1981: Optimal short-term hydro scheduling from the principle of progressive optimality. Water Resour. Res. 17(3), 481–486

    Google Scholar 

  • Wasimi, S.A.; Kitanidis, P.K. 1983: Real-time forecasting and daily operation of multireservoir system during floods by linear quadratic Gaussian control. Water Resour. Res. 19(6), 1511–1522

    Google Scholar 

  • Yakowitz, S. 1982: Dynamic programming applications in water resources. Water Resour. Res. 18(4), 673–696

    Google Scholar 

  • Young, G.K. 1966: Techniques for finding reservoir operating rules. Ph.D. Thesis, Harvard Water Program, Harvard Univ., Cambridge, Massachusetts

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Civil Engineering, Stanford University, 94305, Stanford, CA, USA

    P. K. Kitanidis

Authors
  1. P. K. Kitanidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kitanidis, P.K. A first-order approximation to stochastic optimal control of reservoirs. Stochastic Hydrol Hydraul 1, 169–184 (1987). https://doi.org/10.1007/BF01543889

Download citation

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01543889

Key words

Search

Navigation