Abstract
Conditions are presented for the existence of increasing and Lipschitz continuous maximizers in a general one-stage optimization problem. This property results in substantial numerical savings in case of a discrete parameter space. The one-stage result and properties of concave functions lead to simple conditions for the existence of optimal policies, composed of increasing and Lipschitz continuous decision rules, for several dynamic programs with discrete state and action space, in which case discrete concavity plays a dominant role. One of the examples, a general multi-stage allocation problem, is considered in detail. Finally, some known results in the case of a continuous state and action space are generalized.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
S. Arunkumar, Characterization of optimal operating policies for finite dams, J. Math. Anal. Appl. 49 (1975) 267–274.
S. Arunkumar, Optimal regulation policies for a multi-purpose reservoir with seasonal input and return function, J. Optim. Theory Appl. 21 (1977) 319–328.
H. Benzing and M. Kolonko, Monotone optimal decision rules and their computation, J. Optim. Theory Appl. 49 (1986) 489–492.
J. Gessford and S. Karlin, Optimal policy for hydroelectric operations, in:Studies in the Mathematical Theory of Inventory and Production, eds. K. Arrow, S. Karlin and H. Scarf (Stanford University Press, Stanford, 1958) pp. 179–200.
R. Hassin, A dichotomous search for a geometric random variable, Oper. Res. 32 (1984) 423–439.
E. Hewitt and K. Stromberg,Real and Abstract Analysis (Springer, New York, 1965).
W. Hildenbrand,Core and Equilibria of a Large Economy (Princeton University Press, Princeton, 1974).
K. Hinderer,Foundations of Non-Stationary Dynamic Programming with Discrete Time Parameter (Springer, Berlin, 1970).
K. Hinderer, On the structure of solutions of stochastic dynamic programs, in:Proc. 7th Conf. on Probability, Brasov (1984) pp. 173–182.
K. Hinderer, On dichotomous search with direction-dependent costs for a uniformly hidden object, Optimization 21 (1990) 215–229.
D. Kalin, A note on “Monotone optimal policies for markov decision processes”, Math. Progr. 15 (1978) 220–222.
M. Kolonko and H. Benzing, On monotone optimal decision rules and the stay-on-a-winner rule for the two-armed bandit, Metrika 32 (1985) 395–407.
R. Mendelssohn and M.J. Sobel, Capital accumulation and the optimization of renewable resource models, J. Econ. Theory 23 (1980) 243–260.
J. Murakami, A dichotomous search, J. Oper. Res. Soc. Japan 14 (1971) 127–142.
A.W. Roberts and D.E. Varberg,Convex Functions (Academic Press, New York, 1973).
D.R. Robinson, A computational refinement for discrete-valued dynamic programs with convex functions, Man. Sci. 23 (1977) 1251–1255.
R.F. Serfozo, Monotone optimal policies for Markov decision processes, Math. Progr. 6 (1976) 202–215.
M.J. Sobel, Reservoir management models, Water Resources Res. 2 (1975), 767–776.
W. Walter,Analysis I (Springer, Berlin, 1985).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hinderer, K. Increasing Lipschitz continuous maximizers of some dynamic programs. Ann Oper Res 29, 565–585 (1991). https://doi.org/10.1007/BF02283614
Issue Date:
DOI: https://doi.org/10.1007/BF02283614