Abstract
We consider Markov decision processes (MDPs) which are a standard model for probabilistic systems. We focus on qualitative properties for MDPs that can express that desired behaviors of the system arise almost-surely (with probability 1) or with positive probability. We introduce a new simulation relation to capture the refinement relation of MDPs with respect to qualitative properties, and present discrete graph algorithms with quadratic complexity to compute the simulation relation. We present an automated technique for assume-guarantee style reasoning for compositional analysis of two-player games by giving a counterexample guided abstraction-refinement approach to compute our new simulation relation. We show a tight link between two-player games and MDPs, and as a consequence the results for games are lifted to MDPs with qualitative properties. We have implemented our algorithms and show that the compositional analysis leads to significant improvements.
Similar content being viewed by others
References
Alur R, Henzinger T, Kupferman O, Vardi M (1998) Alternating refinement relations. In: CONCUR, LNCS 1466. Springer, Heidelberg, pp 163–178
Alur R, Henzinger TA (2004) Computer-aided verification (unpublished)
Alur R, Henzinger TA, Kupferman O (2002) Alternating-time temporal logic. J ACM 49(5):672–713
Aziz A, Singhal V, Balarin F, Brayton R, Sangiovanni-Vincentelli A (1995) It usually works: the temporal logic of stochastic systems. In: CAV, LNCS 939. Springer, Heidelberg, pp 155–165
Baier C, Bertrand N, Bouyer P, Brihaye T, Größer M (2008) Almost-sure model checking of infinite paths in one-clock timed automata. In: LICS, pp 217–226
Baier C, Bertrand N, Größer M (2008) On decision problems for probabilistic Büchi automata. In: FoSSaCS, LNCS 4962. Springer, Heidelberg, pp 287–301
Baier C, Katoen J-P (2008) Principles of model checking. MIT Press, Cambridge
Beeri C (1980) On the membership problem for functional and multivalued dependencies in relational databases. ACM Trans Database Syst 5:241–259
Bertrand N, Genest B, Gimbert H (2009) Qualitative determinacy and decidability of stochastic games with signals. In: Proceedings of LICS. IEEE Computer Society, Los Alamitos, pp 319–328
Bianco A, de Alfaro L (1995) Model checking of probabilistic and nondeterministic systems. In: FSTTCS, LNCS 1026. Springer, Heidelberg, pp 499–513
Bouyer P, Brihaye T, Jurdzinski M, Menet Q (2012) Almost-sure model-checking of reactive timed automata. In: QEST, pp 138–147
Cerný P, Chmelik M, Henzinger TA, Radhakrishna A (2012) Interface simulation distances. In: GandALF, EPTCS 96, pp 29–42
Chadha R, Viswanathan M (2010) A counterexample-guided abstraction-refinement framework for Markov decision processes. ACM Trans Comput Log 12(1):1–49
Chaki S, Clarke EM, Sinha N, Thati P (2005) Automated assume-guarantee reasoning for simulation conformance. In: CAV, LNCS 3576. Springer, Heidelberg, pp 534–547
Chatterjee K (2007) Stochastic \(\omega \)-regular games. PhD thesis, UC Berkeley
Chatterjee K (2012) The complexity of stochastic müller games. Inf Comput 211:29–48
Chatterjee K (2014) Qualitative concurrent parity games: bounded rationality. In: CONCUR, pp 544–559
Chatterjee K, Chaubal S, Kamath P (2012) Faster algorithms for alternating refinement relations. In: CSL, LIPIcs 16. Schloss Dagstuhl, Wadern, pp 167–182
Chatterjee K, Chmelik M (2015) POMDPs under probabilistic semantics. Artif Intell 221:46–72
Chatterjee K, Chmelik M, Tracol M (2013) What is decidable about partially observable Markov decision processes with omega-regular objectives. In: Proceedings of computer science logic (CSL 2013)
Chatterjee K, de Alfaro L, Faella M, Legay A (2009) Qualitative logics and equivalences for probabilistic systems. Log Methods Comput Sci 5(2)
Chatterjee K, de Alfaro L, Faella M, Majumdar R, Raman V (2013) Code-aware resource management. Form Methods Syst Des 42(2):146–174
Chatterjee K, de Alfaro L, Henzinger TA (2005) The complexity of stochastic Rabin and Streett games. In: ICALP, LNCS 3580. Springer, Heidelberg, pp 878–890
Chatterjee K, de Alfaro L, Henzinger TA (2006) The complexity of quantitative concurrent parity games. In: SODA. ACM-SIAM, pp 678–687
Chatterjee K, de Alfaro L, Henzinger TA (2006) Strategy improvement in concurrent reachability games. In: QEST. IEEE, New York, pp 291–300
Chatterjee K, de Alfaro L, Henzinger TA (2011) Qualitative concurrent parity games. ACM Trans Comput Log 12(4):28
Chatterjee K, Doyen L (2012) Partial-observation stochastic games: how to win when belief fails. In: Proceedings of LICS 2012: Logic in Computer Science. IEEE Computer Society Press, Washington, DC, pp 175–184
Chatterjee K, Doyen L (2014) Games with a weak adversary. In: ICALP, pp 110–121
Chatterjee K, Doyen L, Gimbert H, Henzinger TA (2010) Randomness for free. In: MFCS, pp 246–257
Chatterjee K, Doyen L, Henzinger TA (2010) Qualitative analysis of partially-observable Markov decision processes. In: MFCS, LNCS 6281. Springer, Heidelberg, pp 258–269
Chatterjee K, Doyen L, Henzinger TA (2013) A survey of partial-observation stochastic parity games. Form Methods Syst Des 43(2):268–284
Chatterjee K, Doyen L, Henzinger TA, Raskin J (2006) Algorithms for omega-regular games with imperfect information. In: CSL’06. LNCS 4207. Springer, Heidelberg, pp 287–302
Chatterjee K, Doyen L, Nain S, Vardi MY (2014) The complexity of partial-observation stochastic parity games with finite-memory strategies. In: FoSSaCS, pp 242–257
Chatterjee K, Henzinger M (2011) Faster and dynamic algorithms for maximal end-component decomposition and related graph problems in probabilistic verification. In: SODA, pp 1318–1336
Chatterjee K, Henzinger M ( 2012) An \(O(n^{\text{2 }})\) time algorithm for alternating Büchi games. In: SODA, pp 1386–1399
Chatterjee K, Henzinger M (2014) Efficient and dynamic algorithms for alternating Büchi games and maximal end-component decomposition. JACM 61(3):15
Chatterjee K, Henzinger M, Joglekar M, Shah N (2013) Symbolic algorithms for qualitative analysis of Markov decision processes with Büchi objectives. Form Methods Syst Des 42(3):301–327
Chatterjee K, Ibsen-Jensen R (2015) Qualitative analysis of concurrent mean-payoff games. Inf Comput
Chatterjee K, Ibsen-Jensen R (2015) The value 1 problem under finite-memory strategies for concurrent mean-payoff games. In: SODA, pp 1018–1029
Chatterjee K, Jurdziński M, Henzinger TA (2003) Simple stochastic parity games. In: CSL’03, volume 2803 of LNCS. Springer, Heidelberg, pp 100–113
Chatterjee K, Jurdziński M, Henzinger TA (2004) Quantitative stochastic parity games. In: SODA. SIAM, pp 121–130
Chatterjee K, Lacki J (2013) Faster algorithms for markov decision processes with low treewidth. In: CAV, pp 543–558
Chatterjee K, Tracol M (2012) Decidable problems for probabilistic automata on infinite words. In: LICS, pp 185–194
Clarke E, Grumberg O, Peled D (1999) Model checking. MIT Press, Cambridge
Clarke EM, Grumberg O, Jha S, Lu Y, Veith H (2000) Counterexample-guided abstraction refinement. In: CAV, LNCS 1855, pp 154–169
Cleaveland R, Steffen B (1991) Computing behavioural relations, logically. In: ICALP, LNCS 510. Springer, Heidelberg, pp 127–138
Courcoubetis C, Yannakakis M (1995) The complexity of probabilistic verification. J ACM 42(4):857–907
D’Argenio PR, Jeannet B, Jensen HE, Larsen KG (2001) Reachability analysis of probabilistic systems by successive refinements. In: PAPM-PROBMIV, LNCS 2165. Springer, Heidelberg, pp 39–56
D’Argenio PR, Jeannet B, Jensen HE, Larsen KG (2002) Reduction and refinement strategies for probabilistic analysis. In: PAPM-PROBMIV, LNCS 2399. Springer, Heidelberg, pp 57–76
de Alfaro L, Henzinger TA, Jhala R (2001) Compositional methods for probabilistic systems. In: CONCUR, LNCS 2154. Springer, Heidelberg, pp 351–365
de Alfaro L, Henzinger TA, Kupferman O (1998) Concurrent reachability games. In: FOCS, pp 564–575
Etessami K, Kwiatkowska MZ, Vardi MY, Yannakakis M (2008) Multi-objective model checking of Markov decision processes. Log Methods Comput Sci 4(4):1–21
Feng L, Kwiatkowska MZ, Parker D (2011) Automated learning of probabilistic assumptions for compositional reasoning. In: FASE, LNCS 6603. Springer, Heidelberg, pp 2–17
Filar J, Vrieze K (1997) Competitive Markov decision processes. Springer, Berlin
Grädel E, Thomas W, Wilke T (2002) Automata, logics, and infinite games: a guide to current research. LNCS 2500. Springer, Heidelberg
Hansson H, Jonsson B (1994) A logic for reasoning about time and reliability. Form Asp Comput 6(5):512–535
Henzinger MR, Henzinger TA, Kopke PW (1995) Computing simulations on finite and infinite graphs. In: FOCS, pp 453–462
Henzinger TA, Jhala R, Majumdar R (2003) Counterexample-guided control. In: ICALP, LNCS 2719. Springer, Heidelberg, pp 886–902
Henzinger TA, Jhala R, Majumdar R, Qadeer S (2003) Thread-modular abstraction refinement. In: CAV, LNCS 2725. Springer, Heidelberg, pp 262–274
Hermanns H, Wachter B, Zhang L (2008) Probabilistic CEGAR. In: CAV, LNCS 5123. Springer, Heidelberg, pp 162–175
Howard RA (1960) Dynamic programming and Markov processes. MIT Press, Cambridge
Immerman N (1981) Number of quantifiers is better than number of tape cells. J Comput Syst Sci 22:384–406
Itai A, Rodeh M (1990) Symmetry breaking in distributed networks. Inf Comput 88(1):60–87
Jeannet B, dArgenio P, Larsen K (2002) Rapture: a tool for verifying Markov decision processes. Tools Day 2:149
Komuravelli A, Pasareanu CS, Clarke EM (2012) Assume-guarantee abstraction refinement for probabilistic systems. In: CAV, LNCS 7358. Springer, Heidelberg, pp 310–326
Kwiatkowska MZ, Norman G, Parker D (2006) Game-based abstraction for Markov decision processes. In: QEST, pp 157–166
Kwiatkowska MZ, Norman G, Parker D (2011) Prism 4.0: verification of probabilistic real-time systems. In: CAV, LNCS 6806, pp 585–591
Kwiatkowska MZ, Norman G, Parker D, Qu H (2010) Assume-guarantee verification for probabilistic systems. In: TACAS, LNCS 6015. Springer, Heidelberg, pp 23–37
Milner R (1971) An algebraic definition of simulation between programs. In: IJCAI, pp 481–489
Nain S, Vardi MY (2013) Solving partial-information stochastic parity games. In: LICS, pp 341–348
Pasareanu CS, Giannakopoulou D, Bobaru MG, Cobleigh JM, Barringer H (2008) Learning to divide and conquer: applying the L* algorithm to automate assume-guarantee reasoning. Form Methods Syst Des 32(3):175–205
Peterson GL (1981) Myths about the mutual exclusion problem. Inf Process Lett 12(3):115–116
Pnueli A (1985) In transition from global to modular temporal reasoning about programs. In: Logics and models of concurrent systems, NATO Advanced Summer Institutes F-13. Springer, Heidelberg, pp 123–144
Pogosyants A, Segala R, Lynch N (2000) Verification of the randomized consensus algorithm of Aspnes and Herlihy: a case study. Distrib Comput 13(3):155–186
Schewe S (2009) Tighter bounds for the determinisation of büchi automata. In: FoSSaCS, pp 167–181
Segala R (1995) Modeling and verification of randomized distributed real-time systems. PhD thesis, MIT Press. Technical Report MIT/LCS/TR-676
Segala R, Lynch NA (1995) Probabilistic simulations for probabilistic processes. Nord J Comput 2(2):250–273
Stoelinga M (2002) Fun with FireWire: experiments with verifying the IEEE1394 root contention protocol. In: Formal aspects of computing
Szymanski BK (1988) A simple solution to Lamport’s concurrent programming problem with linear wait. In: ICS, pp 621–626
Vardi MY (1985) Automatic verification of probabilistic concurrent finite-state programs. In: FOCS, pp 327–338
Zielonka W (1998) Infinite games on finitely coloured graphs with applications to automata on infinite trees. Theor Comput Sci 200(1–2):135–183
Acknowledgments
We thank Anvesh Komuravelli for sharing his implementation with us. The research was partly supported by Austrian Science Fund (FWF) Grant No. P23499- N23, FWF NFN Grant No. S11407-N23, FWF Grant S11403-N23 (RiSE), and FWF Grant Z211-N23 (Wittgenstein Award), ERC Start Grant (279307: Graph Games), Microsoft faculty fellows award, the ERC Advanced Grant QUAREM (Quantitative Reactive Modeling).
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
We start with an example that shows that also for alternating games combined simulation is finer that the intersection of simulation and alternating-simulation relation.
Example 7
Figure 7 shows two alternating games \(G, G'\), where the circular states belong to Player 1 and the rectangular states belong to Player 2, white nodes are labeled by proposition p and gray nodes by proposition q. The largest simulation and alternating-simulation relations between \(G\) and \(G'\) are: \({\mathcal {S}}_{\max }=\{(s_0, t_0),(s_1, t_1),(s_2, t_2), (s_3, t_1)\}, {\mathcal {A}}_{\max }=\{(s_0, t_0),(s_0, t_4),(s_2, t_2), (s_3, t_3), (s_1, t_3), (s_1, t_1)\}\). Formula \(\langle \!\langle 1 \rangle \!\rangle (\Box (p \wedge \langle \!\langle 1,2 \rangle \!\rangle ({\mathsf {true}}\, \, {\mathcal {U}}\, q)))\) is satisfied in state \(s_0\), but not in state \(t_0\), hence \((s_0, t_0)\not \in {\mathcal {C}}_{\max }\).\(\square \)
We now present detailed proofs of Lemma 1 and Theorem 2 in the context of alternating games.
Lemma 8
Given two alternating games \(G\) and \(G'\), let \({\mathcal {C}}_{\max }\) be the combined simulation. For all \((s,s') \in {\mathcal {C}}_{\max }\) the following assertions hold:
-
1.
For all Player 1 strategies \(\sigma \) in \(G\), there exists a Player 1 strategy \(\sigma '\) in \(G'\) such that for every play \(\omega ' \in {\mathsf {Plays}}(s',\sigma ')\) there exists a play \(\omega \in {\mathsf {Plays}}(s,\sigma )\) such that \(\omega \leqslant _{\mathcal {C}}\omega '\).
-
2.
For all pairs of strategies \(\sigma \) and \(\theta \) in \(G\), there exists a pair of strategies \(\sigma '\) and \(\theta '\) in \(G'\) such that \({\mathsf {Play}}(s,\sigma ,\theta ) \leqslant _{\mathcal {C}}{\mathsf {Play}}(s',\sigma ',\theta ')\),
Proof
Assertion 1 As the states of Player 1 and Player 2 are distinguished by the \({\mathsf {turn}}\) atomic proposition, it follows from the fact that \((s,s') \in {\mathcal {C}}_{\max }\), that either (i) \(s \in S_1\) and \(s' \in S'_1\) or (ii) \(s \in S_2\) and \(s' \in S'_2\).
For the first case (i) we consider a winning strategy \(\sigma ^{{\mathcal {C}}}\) in \(G^{\mathcal {C}}\) such that for all \((s,s') \in {\mathcal {C}}_{\max }\) and against all strategies \(\theta ^{\mathcal {C}}\) we have \({\mathsf {Play}}((s,s'),\sigma ^{\mathcal {C}},\theta ^{\mathcal {C}}) \in \llbracket \Box (\lnot p) \rrbracket _{G^{\mathcal {C}}}\). Given the Player 1 strategy \(\sigma \) in \(G\) we construct \(\sigma '\) in \(G'\) using the strategy \(\sigma ^{\mathcal {C}}\). Let h be an arbitrary history in \(G^{\mathcal {C}}\) that visits only states of type \((S\times S')\) that are in \({\mathcal {C}}_{\max }\) and ends in \((s,s')\). Consider a history \(w \cdot s\) in \(G\) and \(w'\cdot s'\) in \(G'\). Let \(\sigma (w \cdot s) = a\), we define \(\sigma '(w' \cdot s')\) as action \(a' = \sigma ^{{\mathcal {C}}}(h \cdot ((s,s'),{\mathsf {Alt}},2) \cdot ((s,s'),{\mathsf {Alt}},a,2))\), i.e., action \(a'\) corresponds to the choice of the proponents winning strategy \(\sigma ^{\mathcal {C}}\) in response to the adversarial choice of checking step-wise alternating-simulation followed by action a in \(G\). As both s and \(s'\) are Player-1 states we have that \(\vert \delta (s,a) \vert =1\) and \(\vert \delta '(s',a') \vert =1\). Let \((t,t')\) be the unique state reached in 2 steps from \(((s,s'),{\mathsf {Alt}},a,a',2)\) in \(G^{\mathcal {C}}\). Assume towards contradiction that \({\mathcal {L}}^{\mathcal {C}}((t,t')) = \{ p \}\), then there exists a strategy for adversary that reaches a loosing state while the proponent plays a winning strategy \(\sigma ^{\mathcal {C}}\) and the contradiction follows. For the second case (ii) we have that states s and \(s'\) belong to Player 2, and there is a single action available for \(\sigma '\).
Assertion 2 The proof is similar to the first assertion, and instead of using the step-wise alternating-simulation gadget for strategy construction (of the first item) we use the step-wise simulation gadget from \(G^{\mathcal {C}}\) to construct the strategy pairs. \(\square \)
Theorem 7
For all alternating games \(G\) and \(G'\) we have \({\mathcal {C}}_{\max } = \preccurlyeq _{C}^*= \preccurlyeq _{C}\).
Proof
First implication: We first prove the implication \({\mathcal {C}}_{\max }\subseteq \preccurlyeq _{C}^*\). We will show the following assertions:
-
For all states s and \(s'\) such that \((s,s') \in {\mathcal {C}}_{\max }\), we have that every \({\text {C-ATL}}^*\) state formula satisfied in s is also satisfied in \(s'\).
-
For all plays \(\omega \) and \(\omega '\) such that \(\omega \leqslant _{\mathcal {C}}\omega '\), we have that every \({\text {C-ATL}}^*\) path formula satisfied in \(\omega \) is also satisfied in \(\omega '\).
We will prove the theorem by induction on the structure of the formulas. The interesting cases for the induction step are formulas \(\langle \!\langle 1 \rangle \!\rangle (\varphi )\) and \(\langle \!\langle 1,2 \rangle \!\rangle (\varphi )\), where \(\varphi \) are path formulas.
-
Assume \(s \models \langle \!\langle 1 \rangle \!\rangle (\varphi )\) and \((s,s') \in {\mathcal {C}}_{\max }\). It follows that there exists a strategy \(\sigma \in \varSigma \) that ensures the path formula \(\varphi \) from state s against any strategy \(\theta \in \varTheta \). We want to show that \(s' \models \langle \!\langle 1 \rangle \!\rangle (\varphi )\). By Lemma 8 (item 1) we have that there exists a strategy \(\sigma '\) for Player 1 from \(s'\) such that for every play \(\omega ' \in {\mathsf {Plays}}(s',\sigma ')\) there exists a play \(\omega \in {\mathsf {Plays}}(s,\sigma )\) such that \(\omega \leqslant _{\mathcal {C}}\omega '\). By inductive hypothesis we have that \(s' \models \langle \!\langle 1 \rangle \!\rangle (\varphi )\).
-
Assume \(s \models \langle \!\langle 1,2 \rangle \!\rangle (\varphi )\) and \((s,s') \in {\mathcal {C}}_{\max }\). It follows that there exist strategies \(\sigma \in \varSigma , \theta \in \varTheta \) that ensure the path formula \(\varphi \) from state s. By Lemma 8 (item 2) we have that there exist strategies \(\sigma '\) and \(\theta '\) such that the two plays \(\omega ' = {\mathsf {Play}}(s',\sigma ',\theta ')\) and \(\omega ={\mathsf {Play}}(s,\sigma ,\theta )\) satisfy \(\omega \leqslant _{\mathcal {C}}\omega '\). By inductive hypothesis we have that \(s' \models \langle \!\langle 1,2 \rangle \!\rangle (\varphi )\).
-
Consider a path formula \(\varphi \). If \(\omega \leqslant _{\mathcal {C}}\omega '\), then by inductive hypothesis for every sub-formula \(\varphi '\) of \(\varphi \) we have that if \(\omega \models \varphi '\) then \(\omega '\models \varphi '\). It follows that if \(\omega \models \varphi \) then \(\omega '\models \varphi \).
Second implication: It remains to prove the second implication \(\preccurlyeq _{C}^* \subseteq \preccurlyeq _{C}\subseteq {\mathcal {C}}_{\max }\). We prove that from the assumption that \((s,s') \not \in {\mathcal {C}}_{\max }\) we can construct a \({\text {C-ATL}}\) formula \(\varphi \) such that \(s \models \varphi \) and \(s' \not \models \varphi \). We refer to the formula \(\varphi \) as a distinguishing formula. Assume that given states s and \(s'\) we have that \((s,s') \not \in {\mathcal {C}}_{\max }\), then there exists a winning strategy in the corresponding combined-simulation game for the adversary from state \((s,s')\), i.e., there exists a strategy \(\theta ^{\mathcal {C}}\) such that against all strategies \(\sigma ^{\mathcal {C}}\) we have \({\mathsf {Play}}((s,s'),\sigma ^{\mathcal {C}},\theta ^{\mathcal {C}})\) reaches a state labeled by p. As memoryless strategies are sufficient for both players in \(G^{\mathcal {C}}\) [55], there also exists a bound \(i \in {\mathbb {N}}\), such that the proponent fails to match the choice of the adversary in at most i turns. We construct the \({\text {C-ATL}}\) formula \(\varphi \) inductively:
- Base case::
-
Assume \((s,s') \not \in {\mathcal {C}}_{\max }\) and let 0 be the number of turns the adversary needs to play in order to win. It follows that \((s,s')\) is a winning state for the adversary, i.e., \({\mathcal {L}}^{{\mathcal {C}}}((s,s')) = \{p\}\). It follows that \({\mathcal {L}}(s) \ne {\mathcal {L}}'(s')\). There are two options: (i) there exists an atomic proposition \(q \in {\mathsf {AP}}\) that is true in s and not true in \(s'\) and distinguishes the two states, or (ii) there exists an atomic proposition \(q \in {\mathsf {AP}}\) that is not true in s and true in \(s'\), in that case the formula \(\lnot q\) distinguishes the two states.
- Induction step::
-
Assume \((s,s') \not \in {\mathcal {C}}_{\max }\) and let \(n+1\) be the number of turns the adversary needs to play in order to win. As the states of Player 1 and Player 2 are distinguished by the \({\mathsf {turn}}\) atomic proposition, it follows that either (i) \(s \in S_1\) and \(s' \in S'_1\) or (ii) \(s \in S_2\) and \(s' \in S'_2\). Otherwise the adversary could win in 0 turns from \((s,s')\).
We first consider case (i), i.e., \((s,s') \in S_1 \times S'_1\). The adversary can choose whether to verify (1) step-wise alternating-simulation (\({\mathsf {Alt}}\)) or (2) step-wise simulation (\({\mathsf {Sim}}\)). After that he chooses an action a to be played according the adversarial strategy \(\theta ^{\mathcal {C}}\) in state \((s,s')\), such that no matter what the proponent plays, the adversary will win in n turns. We consider two cases: (1) the adversary checks for step-wise alternating-simulation relation (\({\mathsf {Alt}}\)), or (2) the adversary checks for step-wise simulation relation (\({\mathsf {Sim}}\)). For case (1) we have that there exists an action a for the adversary such that for all actions \(a'\) of the proponent the adversary can win in n turns from the unique successor \((t,t')\) of \((s,s')\) given \({\mathsf {Alt}}\) and a was played by the adversary and \(a'\) by the proponent. From the induction hypothesis there exists a \({\text {C-ATL}}\) formula \(\varphi _n\) such that \(t \models \varphi _n\) and \(t' \not \models \varphi _n\). We define the formula \(\varphi _{n+1}\) that distinguishes states s and \(s'\) as \(\langle \!\langle 1 \rangle \!\rangle (\bigcirc \varphi _n)\). For case (2), where the adversary plays \({\mathsf {Sim}}\) the proof is exactly the same, as step-wise simulation turn from Player 1 states coincides with step-wise alternating-simulation turn.
Next we first consider case (ii), i.e., \((s,s') \in S_2 \times S'_2\). The adversary can choose whether to verify (1) step-wise alternating-simulation (\({\mathsf {Alt}}\)) or(2) step-wise simulation (\({\mathsf {Sim}}\)). We start with first case (1): there is a unique action a available to the adversary from state \(((s,s'),{\mathsf {Alt}},2)\) and similarly a unique action \(a'\) for the proponent from \(((s,s'),a,{\mathsf {Alt}},1)\). The adversary chooses an action \(t'\) from the \(((s,s'),a,a',{\mathsf {Alt}},2)\) according to the winning strategy and the proponent chooses some action \(t_i\) from a set of available successor \((t_1,t_2, \ldots , t_m)\). As the adversary follows a winning strategy \(\theta ^{\mathcal {C}}\) we have that it wins from all states \((t_i,t')\) for \(1 \le i \le m\) in at most n turns. From the induction hypothesis there exist \({\text {C-ATL}}\) formulas \(\varphi ^i_n\) such that \(t_i \models \varphi ^i_n\) and \(t' \not \models \varphi ^i_n\). We define the formula \(\varphi _{n+1}\) that distinguishes states s and \(s'\) as \(\langle \!\langle 1 \rangle \!\rangle (\bigcirc (\bigvee _{1 \le i \le m} \varphi ^i_n)\). For case (2) where the adversary verifies the step-wise simulation step, the proof is analogous. The formula that distinguishes states s and \(s'\) is \(\langle \!\langle 1,2 \rangle \!\rangle ((\bigcirc \bigvee _{1 \le i \le m} \varphi ^i_n))\).
The desired result follows. \(\square \)
Rights and permissions
About this article
Cite this article
Chatterjee, K., Chmelík, M. & Daca, P. CEGAR for compositional analysis of qualitative properties in Markov decision processes. Form Methods Syst Des 47, 230–264 (2015). https://doi.org/10.1007/s10703-015-0235-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10703-015-0235-2