Skip to main content
SpringerLink
Log in

Solving subclasses of multi-player stochastic games via linear complementarity problem formulations—a survey and some new results

  • Published:
Optimization and Engineering Aims and scope Submit manuscript

Abstract

We briefly survey some results on the orderfield property of 2-player and multi-player stochastic games and their mixtures. Some of these classes of stochastic games can be solved by formulating them as a Linear Complementarity Problem (LCP) or (Generalized) Vertical Linear Complementarity Problem (VLCP). We discuss some of these results and prove that certain new subclasses and mixtures of multi-player (or n-person) stochastic games can be solved via LCP formulations.

Mohan, Neogy and Parthasarathy (in Proceedings of the International Conference on Complementarity Problems, 1997) proposed an LCP formulation of β-discounted (multi-player) polystochastic games where the transitions are controlled by one player, and proved that this LCP is processible by Lemke’s algorithm. Using this formulation repeatedly, we prove that we can solve a subclass of β-discounted switching control polystochastic games. As our proof is constructive, we have an algorithm for solving this subclass. This algorithm only involves iteratively solving different LCPs and hence, it follows that this subclass has the orderfield property, a question left open in the paper on orderfield property of mixtures of stochastic games by Krishnamurthy et al., (Indian J. Stat. 72:246–275, 2010). Furthermore, we use results from Krishnamurthy et al., Indian J. Stat. 72:246–275, (2010) to solve some mixture classes using LCP (or VLCP) formulations.

We also propose two different VLCP formulations for β-discounted zero-sum perfect information stochastic games, the underlying matrices of both formulations being R 0. As a result, we also have an alternative proof of the orderfield property of such games.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Condon A (1992) The complexity of stochastic games. Inf Comput 96:203–224

    Article  MathSciNet  MATH  Google Scholar 

  • Cottle RW, Dantzig G (1968) Complementary pivot theory of mathematical programming. Linear Algebra Appl 1:103–125

    Article  MathSciNet  MATH  Google Scholar 

  • Cottle RW, Pang JS, Stone RE (1992) The linear complementarity problem. Academic Press, New York

    MATH  Google Scholar 

  • Eaves BC (1971) The linear complementarity problem. Manag Sci 17:612–634

    Article  MathSciNet  MATH  Google Scholar 

  • Filar JA (1981) Orderfield property of stochastic games when the player who controls the transition changes from state to state. J Optim Theory Appl 34:505–517

    Article  MathSciNet  Google Scholar 

  • Filar JA, Schultz T, Thuijsman F, Vrieze OJ (1991) Nonlinear programming and stationary equilibria in stochastic games. Math Program 50:227–237

    Article  MathSciNet  MATH  Google Scholar 

  • Filar JA, Vrieze OJ (1997) Competitive Markov decision processes. Springer, Berlin

    MATH  Google Scholar 

  • Garcia CB (1973) Some classes of matrices in linear complementarity theory. Math Program 5:299–310

    Article  MATH  Google Scholar 

  • Gartner B, Rust L (2005) Simple stochastic games and P-matrix generalized linear complementarity problems. In: 15th international symposium on fundamentals of computation theory (FCT). Lecture Notes in Computer Science, vol 3623. Springer, Berlin, pp 209–220

    Chapter  Google Scholar 

  • Gillette D (1953) Representable infinite games. PhD Thesis. University of California, Berkeley

  • Gillette D (1957) Stochastic games with zero-stop probabilities. In: Dresher M, Tucker AW, Wolfe P (eds) Contributions to the theory of games, vol III. Ann Math Studies, vol 39. Princeton University Press, Princeton, pp 179–188

    Google Scholar 

  • Howson JT Jr (1972) Equilibria of polymatrix games. Manag Sci 18:312–318

    Article  MathSciNet  MATH  Google Scholar 

  • Janovskaya EB (1968) Equilibrium points of polymatrix games. Latvian mathematical collection (in Russian)

    Google Scholar 

  • Kostreva MM (1976) Direct algorithms for complementarity problems. PhD Thesis, Rensselaer Polytechnic Institute, Troy, New York

  • Krishnamurthy N, Parthasarathy T, Ravindran G (2009–2010) Orderfield property and algorithms for stochastic games via dependency graphs. In: Proceedings of the international conference on frontiers of interface between statistics and sciences (in Honor of Prof CR Rao on Occasion of his 90th Birthday). Hyderabad, India, pp 286–297

    Google Scholar 

  • Krishnamurthy N, Parthasarathy T, Ravindran G (2010) Orderfield property of mixtures of stochastic games. Sankhya: Indian J Stat A 72:246–275 (also available online at sankhya.isical.ac.in)

    MathSciNet  MATH  Google Scholar 

  • Lemke C (1965) Bimatrix equilibrium points and mathematical programming. Manag Sci 11:681–689

    Article  MathSciNet  MATH  Google Scholar 

  • Mertens JF (2002) Stochastic games. In: Aumann R, Hart S (eds) Handbook of game theory with economic applications, vol 3. Elsevier, Amsterdam, pp 1809–1832. Chapter 47

    Google Scholar 

  • Mertens JF, Neyman A (1981) Stochastic games. Int J Game Theory 10:53–66

    Article  MathSciNet  MATH  Google Scholar 

  • Mohan SR, Neogy SK, Parthasarathy T (1997) Linear complementarity and discounted polystochastic game when one player controls transitions. In: Ferris MC, Pang J-S (eds) Proceedings of the international conference on complementarity problems. SIAM, Philadelphia, pp 284–294

    Google Scholar 

  • Mohan SR, Neogy SK, Parthasarathy T (2001) Pivoting algorithms for some classes of stochastic games: A survey. Int Game Theory Rev 3(2–3):253–281

    Article  MathSciNet  MATH  Google Scholar 

  • Mohan SR, Neogy SK, Parthasarathy T, Sinha S (1999) Vertical linear complementarity and discounted zero-sum stochastic games with ARAT structure. Math Program, Ser A, 86(3):637–648

    Article  MathSciNet  MATH  Google Scholar 

  • Mohan SR, Neogy SK, Sridhar R (1996) The general linear complementarity problem revisited. Math Program 74:197–218

    MathSciNet  MATH  Google Scholar 

  • Murty KG (1997) In: Yu F-T (ed) Linear complementarity, linear and nonlinear programming. (First published in 1988 by Heldermann Verlag, Berlin, and out of print since 1994)

    Google Scholar 

  • Nash JF (1951) Non-cooperative games. Ann Math 54:286–295

    Article  MathSciNet  MATH  Google Scholar 

  • Neogy SK, Das AK, Sinha S, Gupta A (2008) On a mixture class of stochastic game with ordered field property. In: Neogy SK, Bapat RB, Das AK, Parthasarathy T (eds) Mathematical programming and game theory for decision making. ISI Platinum Jubilee Series on Statistical Science and Interdisciplinary Research, vol 1. World Scientific, Singapore, pp 451–477

    Chapter  Google Scholar 

  • Nowak AS, Raghavan TES (1993) A finite step algorithm via a bimatrix game to a single controller non-zero sum stochastic game. Math Program 59:249–259

    Article  MathSciNet  MATH  Google Scholar 

  • Parthasarathy T, Raghavan TES (1981) An orderfield property for stochastic games when one player controls transition probabilities. J Optim Theory Appl 33(3):375–392

    Article  MathSciNet  MATH  Google Scholar 

  • Parthasarathy T, Tijs SH, Vrieze OJ (1984) Stochastic games with state independent transitions and separable rewards. Lect Notes Econ Math Syst, 226:262–271

    Article  MathSciNet  Google Scholar 

  • Raghavan TES (2003) Finite-step algorithms for single controller and perfect information stochastic games. In: Neyman A, Sorin S (eds) NATO Science series: C, Stochastic games and applications. Kluwer Academic, Norwell. Chapter 15

    Google Scholar 

  • Raghavan TES, Filar JA (1991) Algorithms for stochastic games—A survey. Z Oper-Res, 35:437–472

    MathSciNet  MATH  Google Scholar 

  • Raghavan TES, Syed Z (2002) Computing stationary Nash equilibria of undiscounted single-controller stochastic games. Math Oper Res 27(2):384–400

    Article  MathSciNet  MATH  Google Scholar 

  • Raghavan TES, Syed Z (2003) A policy-improvement type algorithm for solving zero-sum two-person stochastic games of perfect information. Math Program A 95:513–532

    Article  MathSciNet  MATH  Google Scholar 

  • Raghavan TES, Tijs SJ, Vrieze OJ (1986) Stochastic games with additive rewards and additive transitions. J Optim Theory Appl 47:451–464

    Article  MathSciNet  Google Scholar 

  • Schultz TA (1992) Linear complementarity and discounted switching controller stochastic games. J Optim Theory Appl 73(1):89–99

    Article  MathSciNet  MATH  Google Scholar 

  • Shapley L (1953) Stochastic games. Proc Natl Acad Sci USA 39:1095–1100

    Article  MathSciNet  MATH  Google Scholar 

  • Sinha S (1989) A contribution to the theory of stochastic games. PhD thesis, Indian Statistical Institute, New Delhi, India

  • Sobel MJ (1981) Myopic solutions of Markov decision processes and stochastic games. Oper Res 29:995–1009

    Article  MathSciNet  MATH  Google Scholar 

  • Thuijsman F, Raghavan TES (1997) Perfect information stochastic games and related classes. Int J Game Theory 26:403–408

    Article  MathSciNet  MATH  Google Scholar 

  • Vrieze OJ (1981) Linear programming and undiscounted stochastic game in which one player controls transitions. OR Spektrum 3:29–35

    Article  MATH  Google Scholar 

  • Vrieze OJ, Tijs SH, Raghavan TES, Filar JA (1983) A finite algorithm for the switching control stochastic game. OR Spektrum 5(1):15–24

    Article  MATH  Google Scholar 

  • Weyl H (1950) Elementary proof of a minimax theorem due to von Neumann. In: Kuhn HW, Tucker AW (eds) Contributions to the theory of games, vol I. Ann Math Studies, vol 24. Princeton University Press, Princeton, pp 19–25

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Chennai Mathematical Institute, Siruseri, India

    Nagarajan Krishnamurthy

  2. Indian Statistical Institute, Chennai, India

    Thiruvenkatachari Parthasarathy & Gomatam Ravindran

Authors
  1. Nagarajan Krishnamurthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thiruvenkatachari Parthasarathy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gomatam Ravindran
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nagarajan Krishnamurthy.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Krishnamurthy, N., Parthasarathy, T. & Ravindran, G. Solving subclasses of multi-player stochastic games via linear complementarity problem formulations—a survey and some new results. Optim Eng 13, 435–457 (2012). https://doi.org/10.1007/s11081-011-9163-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11081-011-9163-1

Keywords

Search

Navigation