Krishnendu Chatterjee
ABSTRACT
Termination is one of the basic liveness properties, and we study the termination problem for probabilistic programs with real-valued variables. Previous works focused on the qualitative problem that asks whether an input program terminates with probability 1 (almost-sure termination). A powerful approach for this qualitative problem is the notion of ranking supermartingales with respect to a given set of invariants. The quantitative problem (probabilistic termination) asks for bounds on the termination probability, and this problem has not been addressed yet. A fundamental and conceptual drawback of the existing approaches to address probabilistic termination is that even though the supermartingales consider the probabilistic behaviour of the programs, the invariants are obtained completely ignoring the probabilistic aspect (i.e., the invariants are obtained considering all behaviours with no information about the probability).
In this work we address the probabilistic termination problem for linear-arithmetic probabilistic programs with nondeterminism. We formally define the notion of stochastic invariants, which are constraints along with a probability bound that the constraints hold. We introduce a concept of repulsing supermartingales. First, we show that repulsing supermartingales can be used to obtain bounds on the probability of the stochastic invariants. Second, we show the effectiveness of repulsing supermartingales in the following three ways: (1) With a combination of ranking and repulsing supermartingales we can compute lower bounds on the probability of termination; (2) repulsing supermartingales provide witnesses for refutation of almost-sure termination; and (3) with a combination of ranking and repulsing supermartingales we can establish persistence properties of probabilistic programs.
Along with our conceptual contributions, we establish the following computational results: First, the synthesis of a stochastic invariant which supports some ranking supermartingale and at the same time admits a repulsing supermartingale can be achieved via reduction to the existential first-order theory of reals, which generalizes existing results from the non-probabilistic setting. Second, given a program with "strict invariants" (e.g., obtained via abstract interpretation) and a stochastic invariant, we can check in polynomial time whether there exists a linear repulsing supermartingale w.r.t. the stochastic invariant (via reduction to LP). We also present experimental evaluation of our approach on academic examples.
- IBM ILOG CPLEX Optimizer. http://www-01.ibm.com/software/integration/optimization/cplexoptimizer/, 2010.Google Scholar
- A. Abate, J.-P. Katoen, J. Lygeros, and M. Prandini. Approximate Model Checking of Stochastic Hybrid Systems. European Journal of Control, 16(6):624–641, 2010. ISSN 0947-3580.Google ScholarCross Ref
- M. F. Atig, A. Bouajjani, M. Emmi, and A. Lal. Detecting Fair Nontermination in Multithreaded Programs, pages 210–226. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012. ISBN 978-3-642-31424- 7. Google ScholarDigital Library
- K. Azuma. Weighted sums of certain dependent random variables. Tohoku Mathematical Journal, Second Series, 19(3):357–367, 1967.Google Scholar
- C. Baier and J.-P. Katoen. Principles of model checking. MIT Press, 2008. ISBN 978-0-262-02649-9. Google ScholarDigital Library
- G. Barthe, T. Espitau, L. M. F. Fioriti, and J. Hsu. Synthesizing Probabilistic Invariants via Doob’s Decomposition. In Computer Aided Verification - 28th International Conference, CAV 2016, Toronto, ON, Canada, July 17-23, 2016, Proceedings, Part I, pages 43–61, 2016.Google Scholar
- M. Berkelaar, K. Eikland, and P. Notebaert. lp_solve: Open source (Mixed-Integer) Linear Programming system.Google Scholar
- P. Billingsley. Probability and Measure. 1995.Google Scholar
- O. Bournez and F. Garnier. Proving Positive Almost-Sure Termination. In RTA, pages 323–337, 2005. Google ScholarDigital Library
- A. R. Bradley, Z. Manna, and H. B. Sipma. The Polyranking Principle. In ICALP, pages 1349–1361, 2005. Google ScholarDigital Library
- A. R. Bradley, Z. Manna, and H. B. Sipma. Linear Ranking with Reachability. In K. Etessami and S. K. Rajamani, editors, Computer Aided Verification, 17th International Conference, CAV 2005, Edinburgh, Scotland, UK, July 6-10, 2005, Proceedings, volume 3576 of Lecture Notes in Computer Science, pages 491–504. Springer, 2005. Google ScholarDigital Library
- ISBN 3-540-27231-3.Google Scholar
- T. Brázdil, V. Brožek, K. Etessami, and A. Kuˇcera. Approximating the termination value of one-counter MDPs and stochastic games. Inf. Comput., 222:121–138, 2013. Google ScholarDigital Library
- T. Brázdil, S. Kiefer, and A. Kuˇcera. Efficient Analysis of Probabilistic Programs with an Unbounded Counter. J. ACM, 61(6):41:1–41:35, Dec. 2014. ISSN 0004-5411. Google ScholarDigital Library
- T. Brázdil, S. Kiefer, A. Kuˇcera, P. Novotný, and J.-P. Katoen. Zero-Reachability in Probabilistic Multi-Counter Automata. In Proceedings of LICS 2014, 2014. Google ScholarDigital Library
- A. Chakarov and S. Sankaranarayanan. Probabilistic Program Analysis with Martingales. In N. Sharygina and H. Veith, editors, Computer Aided Verification - 25th International Conference, CAV 2013, Saint Petersburg, Russia, July 13-19, 2013. Proceedings, volume 8044 of Lecture Notes in Computer Science, pages 511–526. Springer, 2013. ISBN 978-3-642-39798-1.Google Scholar
- A. Chakarov and S. Sankaranarayanan. Expectation Invariants for Probabilistic Program Loops as Fixed Points, pages 85–100. Springer International Publishing, 2014. ISBN 978-3-319-10936-7.Google Scholar
- A. Chakarov, Y.-L. Voronin, and S. Sankaranarayanan. Deductive Proofs of Almost Sure Persistence and Recurrence Properties, pages 260–279. Springer Berlin Heidelberg, Berlin, Heidelberg, 2016. ISBN 978-3-662-49674-9. Google ScholarDigital Library
- K. Chatterjee, H. Fu, and A. K. Goharshady. Termination Analysis of Probabilistic Programs through Positivstellensatz’s. CoRR, abs/1604.07169, 2016.Google Scholar
- K. Chatterjee, H. Fu, P. Novotný, and R. Hasheminezhad. Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs. In R. Bodík and R. Majumdar, editors, Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2016, St. Petersburg, FL, USA, January 20 - 22, 2016, pages 327–342. ACM, 2016. ISBN 978-1-4503-3549-2. Google ScholarDigital Library
- K. Chatterjee, P. Novotný, and Ð. Žikeli´c. Stochastic Invariants for Probabilistic Termination. CoRR, abs/1611.01063, 2016. Google ScholarDigital Library
- H.-Y. Chen, B. Cook, C. Fuhs, K. Nimkar, and P. O’Hearn. Proving Nontermination via Safety, pages 156–171. Springer Berlin Heidelberg, Berlin, Heidelberg, 2014. ISBN 978-3-642-54862-8.Google Scholar
- M. Colón and H. Sipma. Synthesis of Linear Ranking Functions. In T. Margaria and W. Yi, editors, Tools and Algorithms for the Construction and Analysis of Systems, 7th International Conference, TACAS 2001 Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2001 Genova, Italy, April 2-6, 2001, Proceedings, volume 2031 of Lecture Notes in Computer Science, pages 67–81. Springer, 2001. ISBN 3-540-41865-2. Google ScholarDigital Library
- M. A. Colón and H. B. Sipma. Practical Methods for Proving Program Termination, pages 442–454. Springer Berlin Heidelberg, Berlin, Heidelberg, 2002. ISBN 978-3-540-45657-5.Google Scholar
- M. A. Colón, S. Sankaranarayanan, and H. B. Sipma. Linear invariant generation using non-linear constraint solving. In International Conference on Computer Aided Verification, pages 420–432. Springer, 2003.Google Scholar
- B. Cook, A. See, and F. Zuleger. Ramsey vs. Lexicographic Termination Proving. In TACAS, pages 47–61, 2013. Google ScholarDigital Library
- P. Cousot and R. Cousot. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In R. M. Graham, M. A. Harrison, and R. Sethi, editors, Conference Record of the Fourth ACM Symposium on Principles of Programming Languages, Los Angeles, California, USA, January 1977, pages 238–252. ACM, 1977. Google ScholarDigital Library
- D. Dubhashi and A. Panconesi. Concentration of Measure for the Analysis of Randomized Algorithms. 2009. Google ScholarCross Ref
- D. Dubhashi and A. Panconesi. Concentration of Measure for the Analysis of Randomized Algorithms. Cambridge University Press, New York, NY, USA, 1st edition, 2009. ISBN 0521884276, 9780521884273. Google ScholarCross Ref
- R. Durrett. Probability: Theory and Examples (Second Edition). Duxbury Press, 1996.Google Scholar
- J. Esparza, A. Gaiser, and S. Kiefer. Proving Termination of Probabilistic Programs Using Patterns. In CAV, pages 123–138, 2012. Google ScholarDigital Library
- J. Farkas. A Fourier-féle mechanikai elv alkalmazásai (Hungarian). Mathematikaiés Természettudományi Értesitö, 12:457–472, 1894.Google Scholar
- Y. A. Feldman. A decidable propositional dynamic logic with explicit probabilities. Information and Control, 63(1):11–38, 1984. ISSN 0019-9958. Google ScholarDigital Library
- Y. A. Feldman and D. Harel. A probabilistic dynamic logic. In Proceedings of the fourteenth annual ACM Symposium on Theory of computing, pages 181–195. ACM, 1982. Google ScholarDigital Library
- L. M. F. Fioriti and H. Hermanns. Probabilistic Termination: Soundness, Completeness, and Compositionality. In S. K. Rajamani and D. Walker, editors, Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2015, Mumbai, India, January 15-17, 2015, pages 489–501. ACM, 2015. ISBN 978-1-4503-3300-9. Google ScholarDigital Library
- R. W. Floyd. Assigning meanings to programs. Mathematical Aspects of Computer Science, 19:19–33, 1967.Google ScholarCross Ref
- F. G. Foster. On the Stochastic Matrices Associated with Certain Queuing Processes. The Annals of Mathematical Statistics, 24(3):pp. 355–360, 1953.Google ScholarCross Ref
- A. D. Gordon, T. A. Henzinger, A. V. Nori, and S. K. Rajamani. Probabilistic programming. In Proceedings of the on Future of Software Engineering, pages 167–181. ACM, 2014. Google ScholarDigital Library
- F. Gretz, J.-P. Katoen, and A. McIver. Prinsys - On a Quest for Probabilistic Loop Invariants. In Quantitative Evaluation of Systems - 10th International Conference, QEST 2013, Buenos Aires, Argentina, August 27-30, 2013. Proceedings, pages 193–208, 2013. Google ScholarDigital Library
- A. Gupta, T. A. Henzinger, R. Majumdar, A. Rybalchenko, and R.-G. Xu. Proving Non-termination. SIGPLAN Not., 43(1):147–158, Jan. 2008. ISSN 0362-1340. Google ScholarDigital Library
- S. Hart and M. Sharir. Concurrent Probabilistic Programs, Or: How to Schedule if You Must. SIAM J. Comput., 14(4):991–1012, 1985.Google ScholarDigital Library
- H. Hermanns, B. Wachter, and L. Zhang. Probabilistic CEGAR. In CAV, LNCS 5123, pages 162–175. Springer, 2008. Google ScholarDigital Library
- H. Howard. Dynamic Programming and Markov Processes. MIT Press, 1960.Google Scholar
- L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237– 285, 1996. Google ScholarDigital Library
- L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1):99–134, 1998. Google ScholarCross Ref
- B. L. Kaminski, J.-P. Katoen, C. Matheja, and F. Olmedo. Weakest Precondition Reasoning for Expected Run–Times of Probabilistic Programs, pages 364–389. Springer Berlin Heidelberg, Berlin, Heidelberg, 2016. ISBN 978-3-662-49498-1.Google Scholar
- J.-P. Katoen, A. McIver, L. Meinicke, and C. C. Morgan. Linear-Invariant Generation for Probabilistic Programs: - Automated Support for Proof-Based Methods. In SAS, volume LNCS 6337, Springer, pages 390–406, 2010. Google ScholarDigital Library
- M. Kattenbelt, M. Kwiatkowska, G. Norman, and D. Parker. Abstraction refinement for probabilistic software. In International Workshop on Verification, Model Checking, and Abstract Interpretation, pages 182–197. Springer, 2009. Google ScholarDigital Library
- J. Kemeny, J. Snell, and A. Knapp. Denumerable Markov Chains. D. Van Nostrand Company, 1966.Google Scholar
- D. Kozen. Semantics of Probabilistic Programs. Journal of Computer and System Sciences, 22(3):328–350, 1981. ISSN 0022-0000.Google ScholarCross Ref
- D. Kozen. A Probabilistic PDL. In Proceedings of the Fifteenth Annual ACM Symposium on Theory of Computing, STOC ’83, pages 291–297, New York, NY, USA, 1983. ACM. ISBN 0-89791-099-0. Google ScholarDigital Library
- H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas. Temporal-Logic-Based Reactive Mission and Motion Planning. IEEE Transactions on Robotics, 25(6):1370–1381, 2009. Google ScholarDigital Library
- M. Z. Kwiatkowska, G. Norman, and D. Parker. PRISM 4.0: Verification of Probabilistic Real-Time Systems. In CAV, LNCS 6806, pages 585–591, 2011. Google ScholarDigital Library
- D. Larraz, K. Nimkar, A. Oliveras, E. Rodríguez-Carbonell, and A. Rubio. Proving Non-termination Using Max-SMT, pages 779–796. Springer International Publishing, Cham, 2014. ISBN 978-3-319- 08867-9. Google ScholarDigital Library
- C. S. Lee, N. D. Jones, and A. M. Ben-Amram. The size-change principle for program termination. In POPL, pages 81–92, 2001. Google ScholarDigital Library
- A. McIver and C. Morgan. Developing and Reasoning About Probabilistic Programs in pGCL. In PSSE, pages 123–155, 2004.Google Scholar
- A. McIver and C. Morgan. Abstraction, Refinement and Proof for Probabilistic Systems. Monographs in Computer Science. Springer, 2005. Google ScholarDigital Library
- D. Monniaux. An Abstract Analysis of the Probabilistic Termination of Programs. In P. Cousot, editor, Static Analysis, 8th International Symposium, SAS 2001, Paris, France, July 16-18, 2001, Proceedings, volume 2126 of Lecture Notes in Computer Science, pages 111–126. Springer, 2001. ISBN 3-540-42314-1. Google ScholarDigital Library
- R. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, New York, NY, USA, 1995. ISBN 0-521-47465-5, 9780521474658. Google ScholarCross Ref
- A. Paz. Introduction to probabilistic automata (Computer science and applied mathematics). Academic Press, 1971. Google ScholarDigital Library
- A. Podelski and A. Rybalchenko. A Complete Method for the Synthesis of Linear Ranking Functions. In B. Steffen and G. Levi, editors, Verification, Model Checking, and Abstract Interpretation, 5th International Conference, VMCAI 2004, Venice, January 11-13, 2004, Proceedings, volume 2937 of Lecture Notes in Computer Science, pages 239–251. Springer, 2004. ISBN 3-540-20803-8.Google Scholar
- V. R. Pratt. Semantical consideration on floyo-hoare logic. In Foundations of Computer Science, 1976., 17th Annual Symposium on, pages 109–121, Oct 1976. Google ScholarDigital Library
- M. Rabin. Probabilistic automata. Information and Control, 6:230–245, 1963.Google ScholarCross Ref
- J. H. Reif. Logics for Probabilistic Programming (Extended Abstract). In Proceedings of the Twelfth Annual ACM Symposium on Theory of Computing, STOC ’80, pages 8–13, New York, NY, USA, 1980. ACM. Google ScholarDigital Library
- ISBN 0-89791-017-6.Google Scholar
- A. Sampson, P. Panchekha, T. Mytkowicz, K. S. McKinley, D. Grossman, and L. Ceze. Expressing and verifying probabilistic assertions. In M. F. P. O’Boyle and K. Pingali, editors, ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’14, Edinburgh, United Kingdom - June 09 - 11, 2014, page 14. ACM, 2014. ISBN 978-1-4503-2784-8. Google ScholarDigital Library
- S. Sankaranarayanan, A. Chakarov, and S. Gulwani. Static Analysis for Probabilistic Programs: Inferring Whole Program Properties from Finitely Many Paths. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’13, pages 447–458, New York, NY, USA, 2013. ACM. ISBN 978-1-4503-2014-6. Google ScholarDigital Library
- S. Sankaranarayanan, A. Chakarov, and S. Gulwani. Static analysis for probabilistic programs: inferring whole program properties from finitely many paths. In PLDI, pages 447–458, 2013. Google ScholarDigital Library
- M. Sharir, A. Pnueli, and S. Hart. Verification of Probabilistic Programs. SIAM J. Comput., 13(2):292–314, 1984. Google ScholarDigital Library
- K. Sohn and A. V. Gelder. Termination Detection in Logic Programs using Argument Sizes. In D. J. Rosenkrantz, editor, Proceedings of the Tenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, May 29-31, 1991, Denver, Colorado, USA, pages 216–226. ACM Press, 1991. ISBN 0-89791-430-9. Google ScholarDigital Library
- H. Velroyen and P. Rümmer. Non-termination Checking for Imperative Programs, pages 154–170. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008. ISBN 978-3-540-79124-9. Google ScholarDigital Library
- D. Williams. Probability with Martingales. 1991.Google Scholar
Index Terms
- Stochastic invariants for probabilistic termination
Recommendations
Modular verification for almost-sure termination of probabilistic programs
In this work, we consider the almost-sure termination problem for probabilistic programs that asks whether a given probabilistic program terminates with probability 1. Scalable approaches for program analysis often rely on modularity as their theoretical ...
Lexicographic ranking supermartingales: an efficient approach to termination of probabilistic programs
Probabilistic programs extend classical imperative programs with real-valued random variables and random branching. The most basic liveness property for such programs is the termination property. The qualitative (aka almost-sure) termination problem asks ...
Stochastic invariants for probabilistic termination
POPL '17Termination is one of the basic liveness properties, and we study the termination problem for probabilistic programs with real-valued variables. Previous works focused on the qualitative problem that asks whether an input program terminates with ...
Comments