Abstract
We present an on-the-fly abstraction technique for infinite-state continuous -time Markov chains. We consider Markov chains that are specified by a finite set of transition classes. Such models naturally represent biochemical reactions and therefore play an important role in the stochastic modeling of biological systems. We approximate the transient probability distributions at various time instances by solving a sequence of dynamically constructed abstract models, each depending on the previous one. Each abstract model is a finite Markov chain that represents the behavior of the original, infinite chain during a specific time interval. Our approach provides complete information about probability distributions, not just about individual parameters like the mean. The error of each abstraction can be computed, and the precision of the abstraction refined when desired. We implemented the algorithm and demonstrate its usefulness and efficiency on several case studies from systems biology.
The research has been partially funded by the Swiss National Science Foundation under grant 205321-111840.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Abdulla, P., Baier, C., Iyer, S., Jonsson, B.: Reasoning about probabilistic lossy channel systems. In: Palamidessi, C. (ed.) CONCUR 2000. LNCS, vol. 1877, pp. 320–333. Springer, Heidelberg (2000)
Abdulla, P., Bertrand, N., Rabinovich, A., Schnoebelen, P.: Verification of probabilistic systems with faulty communication. Inf. Comput. 202(2), 141–165 (2005)
Abdulla, P., Henda, N.B., Mayr, R.: Verifying infinite Markov chains with a finite attractor or the global coarseness property. In: Proc. LICS 2005, pp. 127–136. IEEE Computer Society, Los Alamitos (2005)
Arkin, A., Ross, J., McAdams, H.H.: Stochastic kinetic analysis of developmental pathway bifurcation in phage λ-infected escherichia coli cells. Genetics 149, 1633–1648 (1998)
Baker, G.A.: The essentials of Padé approximants. Academic Press, New York (1975)
De Boer, R.: Theoretical Fysiology. Online Lecture Notes (2006)
Burrage, K., Hegland, M., Macnamara, F., Sidje, B.: A Krylov-based finite state projection algorithm for solving the chemical master equation arising in the discrete modelling of biological systems. In: Proc. of the Markov 150th Anniversary Conference, pp. 21–38. Boson Books (2006)
Busch, H., Sandmann, W., Wolf, V.: A numerical aggregation algorithm for the enzyme-catalyzed substrate conversion. In: Priami, C. (ed.) CMSB 2006. LNCS (LNBI), vol. 4210, pp. 298–311. Springer, Heidelberg (2006)
Çinlar, E.: Introduction to Stochastic Processes. Prentice-Hall, Englewood Cliffs (1975)
D’Argenio, P., Jeannet, B., Jensen, H., Larsen, K.: Reachability analysis of probabilistic systems by successive refinements. In: de Luca, L., Gilmore, S. (eds.) PROBMIV 2001, PAPM-PROBMIV 2001, and PAPM 2001. LNCS, vol. 2165, pp. 39–56. Springer, Heidelberg (2001)
de Alfaro, L., Pritam, R.: Magnifying-lens abstraction for Markov decision processes. In: Damm, W., Hermanns, H. (eds.) CAV 2007. LNCS, vol. 4590, pp. 325–338. Springer, Heidelberg (2007)
Didier, F., Henzinger, T.A., Mateescu, M., Wolf, V.: Approximation of event probabilities in noisy cellular processes. Technical Report MTC-REPORT-2009-002, EPF Lausanne, Switzerland (2009), http://infoscience.epfl.ch/record/135535
Esparza, J., Etessami, K.: Verifying probabilistic procedural programs. In: Lodaya, K., Mahajan, M. (eds.) FSTTCS 2004. LNCS, vol. 3328, pp. 16–31. Springer, Heidelberg (2004)
Esparza, J., Kucera, A., Mayr, R.: Model checking probabilistic pushdown automata. In: Proc. LICS 2004, pp. 12–21. IEEE Computer Society, Los Alamitos (2004)
Etessami, K., Yannakakis, M.: Algorithmic verification of recursive probabilistic state machines. In: Halbwachs, N., Zuck, L.D. (eds.) TACAS 2005. LNCS, vol. 3440, pp. 253–270. Springer, Heidelberg (2005)
Gillespie, D.T.: Markov Processes. Academic Press, New York (1992)
Gross, D., Miller, D.: The randomization technique as a modeling tool and solution procedure for transient Markov processes. Operations Research 32(2), 926–944 (1984)
Hairer, E., Norsett, S., Wanner, G.: Solving Ordinary Differential Equations I: Nonstiff Problems. Springer, Heidelberg (2008)
Hairer, E., Wanner, G.: Solving Ordinary Differential Equations II. Stiff and Differential-Algebraic Problems. Springer, Heidelberg (2004)
Haseltine, E., Rawlings, J.: Approximate simulation of coupled fast and slow reactions for stochastic chemical kinetics. The Journal of Chemical Physics 117(15), 6959–6969 (2002)
Hasty, J., Pradines, J., Dolnik, M., Collins, J.J.: Noise-based switches and amplifiers for gene expression. PNASÂ 97, 2075 (2000)
Henderson, D.A., Boys, R.J., Proctor, C.J., Wilkinson, D.J.: Linking systems biology models to data: a stochastic kinetic model of p53 oscillations. In: O’Hagan, A., West, M. (eds.) Handbook of Applied Bayesian Analysis, Oxford University Press, Oxford (2009)
Henzinger, T.A., Mateescu, M., Wolf, V.: Sliding window abstraction for infinite Markov chains. Technical Report MTC-REPORT-2009-003, EPF Lausanne, Switzerland (2009), http://infoscience.epfl.ch/record/135537
Jensen, A.: Markoff chains as an aid in the study of Markoff processes. Skandinavisk Aktuarietidskrift 36, 87–91 (1953)
van Kampen, N.G.: Stochastic Processes in Physics and Chemistry, 3rd edn. Elsevier, Amsterdam (2007)
Katoen, J.-P., Klink, D., Leucker, M., Wolf, V.: Three-valued abstraction for continuous-time Markov chains. In: Damm, W., Hermanns, H. (eds.) CAV 2007. LNCS, vol. 4590, pp. 316–329. Springer, Heidelberg (2007)
Kucera, A.: Methods for quantitative analysis of probabilistic pushdown automata. Electr. Notes Theor. Comput. Sci. 149(1), 3–15 (2006)
Kwiatkowska, M., Norman, G., Parker, D.: Game-based abstraction for Markov decision processes. In: QEST, pp. 157–166. IEEE CS Press, Los Alamitos (2006)
Marie, R.A., Reibman, A.L., Trivedi, K.S.: Transient analysis of acyclic Markov chains. Perform. Eval. 7(3), 175–194 (1987)
McAdams, H.H., Arkin, A.: Stochastic mechanisms in gene expression. Proceedings of the National Academy of Science 94, 814–819 (1997)
Moler, C., Van Loan, C.: Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Review 45(1), 3–49 (2003)
Munsky, B., Khammash, M.: The finite state projection algorithm for the solution of the chemical master equation. J. Chem. Phys. 124, 44144 (2006)
Norris, J.: Markov Chains, 1st edn. Cambridge University Press, Cambridge (1999)
Patel, P., Arcangioli, B., Baker, S., Bensimon, A., Rhind, N.: DNA replication origins fire stochastically in fission yeast. Mol. Biol. Cell 17, 308–316 (2006)
Paulsson, J.: Summing up the noise in gene networks. Nature 427(6973), 415–418 (2004)
Peles, S., Munsky, B., Khammash, M.: Reduction and solution of the chemical master equation using time scale separation and finite state projection. J. Chem. Phys. 125, 204104 (2006)
Philippe, B., Sidje, R.: Transient solutions of Markov processes by Krylov subspaces. In: Proc. International Workshop on the Numerical Solution of Markov Chains, pp. 95–119. Kluwer Academic Publishers, Dordrecht (1995)
Rabinovich, A.: Quantitative analysis of probabilistic lossy channel systems. Inf. Comput. 204(5), 713–740 (2006)
Rao, C., Wolf, D., Arkin, A.: Control, exploitation and tolerance of intracellular noise. Nature 420(6912), 231–237 (2002)
Remke, A.: Model Checking Structured Infinite Markov Chains. PhD thesis (2008)
Sandmann, W., Wolf, V.: A computational stochastic modeling formalism for biological networks. Enformatika Transactions on Engineering, Computing and Technology 14, 132–137 (2006)
Stewart, W.J.: Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton (1995)
Strelen, C.: Approximate disaggregation-aggregation solutions for general queueing networks. In: Society for Computer Simulation, pp. 773–778 (1997)
Swain, P.S., Elowitz, M.B., Siggia, E.D.: Intrinsic and extrinsic contributions to stochasticity in gene expression. Proceedings of the National Academy of Science, USA 99(20), 12795–12800 (2002)
Thattai, M., van Oudenaarden, A.: Intrinsic noise in gene regulatory networks. PNAS 98(15), 8614–8619 (2001)
Warmflash, A., Dinner, A.: Signatures of combinatorial regulation in intrinsic biological noise. PNAS 105(45), 17262–17267 (2008)
Zhang, L., Hermanns, H., Moritz Hahn, E., Wachter, B.: Time-bounded model checking of infinite-state continuous-time Markov chains. In: ACSD, China (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Henzinger, T.A., Mateescu, M., Wolf, V. (2009). Sliding Window Abstraction for Infinite Markov Chains. In: Bouajjani, A., Maler, O. (eds) Computer Aided Verification. CAV 2009. Lecture Notes in Computer Science, vol 5643. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02658-4_27
Download citation
DOI: https://doi.org/10.1007/978-3-642-02658-4_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02657-7
Online ISBN: 978-3-642-02658-4
eBook Packages: Computer ScienceComputer Science (R0)