Abstract
This paper deals with approximation techniques for the optimal stopping of a piecewise-deterministic Markov process (P.D.P.). Such processes consist of a mixture of deterministic motion and random jumps. In the first part of the paper (Section 3) we study the optimal stopping problem with lower semianalytic gain function; our main result is the construction of ε-optimal stopping times. In the second part (Section 4) we consider a P.D.P. satisfying some smoothness conditions, and forN integer we construct a discretized P.D.P. which retains the main characteristics of the original process. By iterations of the single jump operator from ℝN to ℝN, each iteration consisting ofN one-dimensional minimizations, we can calculate the payoff function of the discretized process. We demonstrate the convergence of the payoff functions, and for the case when the state space is compact we construct ε-optimal stopping times for the original problem using the payoff function of the discretized problem. A numerical example is presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
D. P. Bertsekas, Convergence of discretization procedures in dynamic programming,IEEE Trans. Automat. Control,20 (1975), 415–419.
D. Bertsekas and S. E. Shreve,Stochastic Optimal Control: The Discrete-Time Case, Academic Press, New York, 1978.
P. Billingsley,Probability and Measure, Wiley, New York, 1979.
J. M. Bismut and B. Skalli, Temps d’arrêt optimal, théorie générale des processus et processus de Markov,Z. Wahrsch. Verw. Gebiete,39 (1977), 301–313.
O. L. V. Costa and M. H. A. Davis, Impulse control of piecewise-deterministic processes,Math. Control Signals Systems (submitted).
M. H. A. Davis, The representation of martingales of jump processes,SIAM J. Control Optim.,14 (1976), 623–638.
M. H. A. Davis, Piecewise-deterministic Markov processes: a general class of non-diffusion stochastic models.J. Roy. Statist. Soc. Ser. B,46 (1984), 353–388.
M. H. A. Davis, Control of piecewise-deterministic processes via discrete time dynamic programming,Proceedings of the 3rd Bad Honnef Conference on Stochastic Differential Systems, June 3–7, 1985, 140–150, Lecture Notes in Control and Information Sciences, Vol 16, Springer-Verlag, Berlin, 1986.
M. H. A. Davis, M. A. H. Dempster, S. P. Sethi, and D. Vermes, Optimal capacity expansion under uncertainty,Adv. in Appl. Probab.,19 (1987), 156–176.
C. Dellacherie and P. A. Meyer,Probabilities and Potential, Mathematics Studies, Vol. 29, North-Holland, Amsterdam, 1978.
R. K. Getoor,Markov Processes: Ray Processes and Right Processes, Lectures Notes in Mathematics, Vol. 440, Springer-Verlag, Berlin, 1981.
U. S. Gugerli, Optimal stopping of a piecewise-deterministic Markov process,Stochastics,19 (1986), 221–236.
A. Haurie and P. L’Ecuyer, Approximation and bounds in discrete event dynamic programming,IEEE Trans. Automat. Control,31 (1986), 227–235.
A. Hordijk and F. A. van der Duyn Schouten, Discretization and weak convergence in Markov decision drift processes,Math. Oper. Res.,9 (1984), 112–141.
H. J. Kushner, Approximations and computational methods for optimal stopping and stochastic impulsive control problems,Appl. Math. Optim.,3 (1977), 81–100.
H. J. Kushner,Probability Methods for Approximations in Stochastic Control and for Elliptic Equations, Academic Press, New York, 1977.
S. M. Lenhart and Y. C. Liao, Integro-differential equations associated with optimal stopping of a piecewise deterministic process,Stochastics,15 (1985), 183–208.
J. P. Lepeltier and B. Marchal, Théorie générale du contrôle impulsionnel Markovien,SIAM J. Control Optim.,22 (1984), 645–665.
J. F. Mertens, Strongly supermedian functions and optimal stopping,Z. Wahrsch. Verw. Gebiete,26 (1973), 119–139.
M. E. Munroe,Introductory Real Analysis, Addison-Wesley, Reading, MA, 1964.
A. N. Shirvayev,Optimal Stopping Rules, Springer-Verlag, New York, 1978.
M. Soner, Optimal control with state space constraint,SIAM J. Control Optim.,24 (1986), 1110–1122.
D. Vermes, Optimal control of piecewise deterministic Markov processes,Stochastics,14 (1985), 165–208.
A. A. Yushkevich, Continuous-time Markov decision processes with interventions,Stochastics,9 (1983), 235–274.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Costa, O.L.V., Davis, M.H.A. Approximations for optimal stopping of a piecewise-deterministic process. Math. Control Signal Systems 1, 123–146 (1988). https://doi.org/10.1007/BF02551405
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02551405