Jan Robin Rohlicek
Abstract
A new algorithm for the hierarchical aggregation of singularly perturbed finite-state Markov processes is derived. The approach taken bridges the gap between conceptually simple results for a relatively restricted class of processes and the significantly more complex results for the general case. The critical role played by (almost) transient states is exposed, resulting in a straightforward algorithm for the construction of a sequence of aggregate generators associated with various time scales. These generators together provide a uniform asymptotic approximation of the original probability transition function.
- 1 ANDO, A., AND FISHER, F.M. Near-decomposability, partition and aggregation. In Essays on the Structure of Social Science Models, A. Ando, and F. H. Fisher, Eds. MIT Press, Cambridge, Mass., 1963, pp. 92-106.Google Scholar
- 2 BAHL, L. R., JELINEK, F., AND MERCER, R.L. A maximum likelihood approach to continuous speech recognition. IEEE Trans. Pattern Anal. Mach. Intell. 5, 2 (1983), 179-190.Google Scholar
- 3 BOBBIO, A., AND TRIVEDI, K.S. An aggregation technique for the analysis of stiff Markov chains. IEEE Trans. Comput., to appear. Google Scholar
- 4 CAO, W.-L., AND STEWART, W. J. Iterative aggregation/disaggregation techniques for nearly uncoupled Markov chains. J. ACM 32, 3 (July 1985), 702-719. Google Scholar
- 5 CODERCH, M., WILLSKY, A. S., SASTRY, S. S., AND CASTANON, D.A. Hierarchical aggregation of singularly perturbed finite state Markov processes. Stochastics 8 (1983), 259-289.Google Scholar
- 6 CODERCH, M., WILLSKY, m. S., SASTRY, S. S., AND CASTANON, D.A. Hierarchical aggregation of linear systems with multiple time scales. IEEE Trans. Autom. Contr. AC-28, 11 (1983), 1017-1030.Google Scholar
- 7 COURTOIS, P.J. Decomposability: Queuing and Computer System Applications. Academic Press, Orlando, Fla., 1977.Google Scholar
- 8 COURTOIS, P. J., AND SEMAL, P. Error bounds for the analysis by decomposition of non-negative matrices. In Mathematical Computer Performance and Reliability, G. Iazeolla, P. J. Courtois, and A. Hordijk, Eds. North-Holland, Amsterdam, 1984, pp. 209-224. Google Scholar
- 9 DELEBECQUE, F. A reduction process for perturbed Markov chains. SIAM J. Appl. Math. 43, 2 (1983), pp. 325-330.Google Scholar
- 10 DELEBECQUE, F., AND QUADRAT, J.-P. Optimal Control of Markov chains admitting strong and weak interactions. Automatica 17 ( 1981), 281-296.Google Scholar
- 11 DOERSCHUK, P.C. A Markov chain approach to electrocardiogram modeling and analysis. Ph.D. dissertation. Massachusetts Institute of Technology, Cambridge, Mass., 1985.Google Scholar
- 12 KATO, T. Perturbation Theory for Linear Operators. Springer-Verlag, Berlin, 1966.Google Scholar
- 13 KOKOTOVlC, P.V. Singular perturbations and iterative separation of timescales. Automatica 16 (1980), 23-24.Google Scholar
- 13a KOROLYUK, V. S., AND TURBIN, A.F. On the asymptotic behavior of the occupancy time of a semi-Markov process in a reducible subset of states. Theor. Probl. Math. Stat. 2 (1974), 133-143.Google Scholar
- 14 KOROLYUK, V. S., AND TURBIN, A. F. Limit theorems for Markov random evolution in the scheme of asymptotic state lumping, in Lecture Notes in Mathematics, vol. 1021. Springer-Verlag, Berlin, 1983.Google Scholar
- 15 LOU, X.-C., ROHLICEK, J. R., COXSON, P. G., VERGHESE, G. C., AND WILLSKY, A.S. Time scale decomposition: The role of scaling in linear systems and transient states in finite-state Markov processes. In Proceedings of the 1985 American Control Conference (June). American Automatic Control Council, Green Valley, Ariz., 1985, pp. 1408-1413.Google Scholar
- 16 Lou, X.-C., VERGHESE, G., WILLSKY, A. S., AND VIDYASAGAR, M. An algebraic approach to analysis and control of timescales. In Proceedings of the 1984 American Control Conference (June). American Automatic Control Council, Green Valley, Ariz., 1984, pp. 1365-1372.Google Scholar
- 17 ROHLICEK, J. R. Aggregation and time scale analysis of perturbed Markov systems. Ph.D. dissertation. Massachusetts Institute of Technology, Cambridge, Mass., 1987.Google Scholar
- 18 ROHLICEK, J. R., AND WILLSKY, A. S. The reduction of perturbed Markov generators: An algorithm exposing the role of transient states. M.I.T. Report LIDS-P- 1492. Massachusetts Institute of Technology, Cambridge, Mass., Sept. 1985.Google Scholar
- 19 SILJAK, D.D. Large-Scale Dynamic Systems: Stability and Structure. North-Holland, Amsterdam, 1978.Google Scholar
- 20 SIMON, H., AND ANDO, A. Aggregation of variables in dynamic systems. Econometrica 29 (I963), 111-139.Google Scholar
- 21 WALKER, B.K. A semi-Markov approach to quantifying fault-tolerant system performance. Ph.D. dissertation. Massachusetts Institute of Technology, Cambridge, Mass., 1980.Google Scholar
Index Terms
- The reduction of perturbed Markov generators: an algorithm exposing the role of transient states
Recommendations
Singularly Perturbed Markov Chains
Asymptotic properties of singularly perturbed Markov chains having measurable and/or continuous generators are developed in this work. The Markov chain under consideration has a finite-state space and is allowed to be nonstationary. Its generator ...
Bit-Wise Behavior of Random Number Generators
In 1985, G. Marsaglia proposed the m-tuple test, a runs test on bits, as a test of nonrandomness of a sequence of pseudorandom integers. We try this test on the outputs from a large set of pseudorandom number generators and discuss the behavior of the ...
Paradigms for Unconditional Pseudorandom Generators
This is a survey of unconditional pseudorandom generators (PRGs). A PRG uses a short, truly random seed to generate a long, "pseudorandom" sequence of bits. To be more specific, for each restricted model of computation (e.g., bounded-depth circuits or ...
Reviews
Robert James Plemmons
This paper presents results concerning the decomposition of a general class of perturbed Markov processes. Specifically, an algorithm for the hierarchical aggregation of singularly perturbed finite-state Markov processes is derived. The algorithm was originally outlined by Lou et al. [1]. The approach taken in the present paper bridges the gap between conceptually simple results for a relatively restricted class of processes and the significantly more complex results for the general case. The algorithm is illustrated using the example of a four-state perturbed Markov process.
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments