Skip to main content
SpringerLink
Log in

Learning discrete decomposable graphical models via constraint optimization

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

Statistical model learning problems are traditionally solved using either heuristic greedy optimization or stochastic simulation, such as Markov chain Monte Carlo or simulated annealing. Recently, there has been an increasing interest in the use of combinatorial search methods, including those based on computational logic. Some of these methods are particularly attractive since they can also be successful in proving the global optimality of solutions, in contrast to stochastic algorithms that only guarantee optimality at the limit. Here we improve and generalize a recently introduced constraint-based method for learning undirected graphical models. The new method combines perfect elimination orderings with various strategies for solution pruning and offers a dramatic improvement both in terms of time and memory complexity. We also show that the method is capable of efficiently handling a more general class of models, called stratified/labeled graphical models, which have an astronomically larger model space.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://research.ics.aalto.fi/software/asp/encodings/.

References

  • Bartlett, M., Cussens, J.: Advances in Bayesian network learning using integer programming. In: Proceedings of the 29th International Conference on Uncertainty in Artificial Intelligence, pp. 182–191. AUAI Press (2013)

  • Berg, J., Järvisalo, M., Malone, B.: Learning optimal bounded treewidth Bayesian networks via maximum satisfiability. In: Proceedings of the 17th International Conference on Artificial Intelligence and Statistics, pp. 86–95. JMLR.org (2014)

  • Boutilier, C., Friedman, N., Goldszmidt, M., Koller, D.: Context-specific independence in Bayesian networks. In: Proceedings of the 12th International Conference on Uncertainty in Artificial Intelligence, pp. 115–123. Morgan Kaufmann (1996)

  • Brewka, G., Eiter, T., Truszczyński, M.: Answer set programming at a glance. Commun. ACM 54(12), 92–103 (2011)

    Article  Google Scholar 

  • Chandran, L.S., Ibarra, L., Ruskey, F., Sawada, J.: Generating and characterizing the perfect elimination orderings of a chordal graph. Theor. Comput. Sci. 307(2), 303–317 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Cook, S.A.: The complexity of theorem-proving procedures. In: Proceedings of the 3rd Annual ACM Symposium on Theory of Computing, pp. 151–158. ACM Press (1971)

  • Corander, J.: Labelled graphical models. Scand. J. Stat. 30(3), 493–508 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Corander, J., Ekdahl, M., Koski, T.: Parallel interacting MCMC for learning of topologies of graphical models. Data Min. Knowl. Discov. 17(3), 431–456 (2008)

    Article  MathSciNet  Google Scholar 

  • Corander, J., Janhunen, T., Rintanen, J., Nyman, H., Pensar, J.: Learning chordal Markov networks by constraint satisfaction. In: Proceedings of the 27th Annual Conference on Neural Information Processing Systems, pp. 1349–1357. NIPS Foundation (2013)

  • Cussens, J.: Bayesian network learning by compiling to weighted MAX-SAT. In: Proceedings of the 24th International Conference on Uncertainty in Artificial Intelligence, pp. 105–112. AUAI Press (2008)

  • Dawid, A.P., Lauritzen, S.L.: Hyper-Markov laws in the statistical analysis of decomposable graphical models. Ann. Stat. 21(3), 1272–1317 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  • Dellaportas, P., Forster, J.J.: Markov chain Monte Carlo model determination for hierarchical and graphical log-linear models. Biometrika 86(3), 615–633 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Dirac, G.A.: On rigid circuit graphs. Abhandlungen aus dem Mathematischen Seminar der Universität Hamburg 25(1–2), 71–76 (1961)

    Article  MathSciNet  MATH  Google Scholar 

  • Eriksen, P.S.: Context specific interaction models. Technical Report, Department of Mathematical Sciences, Aalborg University (1999)

  • Eriksen, P.S.: Decomposable log-linear models. Technical Report, Department of Mathematical Sciences, Aalborg University (2005)

  • Friedman, N., Goldszmidt, M.: Learning Bayesian networks with local structure. In: Proceedings of the 12th International Conference on Uncertainty in Artificial Intelligence, pp. 252–262. Morgan Kaufmann (1996)

  • Galinier, P., Habib, M., Paul, C.: Chordal graphs and their clique graphs. In: Proceedings of the 21st International Workshop on Graph-Theoretic Concepts in Computer Science, pp. 358–371. Springer (1995)

  • Gebser, M., Janhunen, T., Rintanen, J.: Answer set programming as SAT modulo acyclicity. In: Proceedings of the 21st European Conference on Artificial Intelligence, pp. 351–356. IOS Press (2014)

  • Gebser, M., Kaufmann, B., Schaub, T.: Conflict-driven answer set solving: From theory to practice. Artificial Intelligence 187–188, 52–89 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Giudici, P., Castello, R.: Improving Markov chain Monte Carlo model search for data mining. Mach. Learn. 50(1–2), 127–158 (2003)

    Article  MATH  Google Scholar 

  • Giudici, P., Green, P.J.: Decomposable graphical Gaussian model determination. Biometrika 86(4), 785–801 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  • Golumbic, M.C.: Algorithmic Graph Theory and Perfect Graphs. Academic Press, New York (1980)

    MATH  Google Scholar 

  • Gomory, R.E.: Outline of an algorithm for integer solutions to linear programs. Bull. Am. Math. Soc. 64(5), 275–278 (1958)

    Article  MathSciNet  MATH  Google Scholar 

  • Højsgaard, S.: Split models for contingency tables. Comput. Stat. Data Anal. 42(4), 621–645 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Højsgaard, S.: Statistical inference in context specific interaction models for contingency tables. Scand. J. Stat. 31(1), 143–158 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • IBM Corporation: IBM ILOG CPLEX Optimization Studio CP Optimizer User’s Manual, version 12 release 6.0 edn. (2013)

  • Johnson, D.S.: Approximation algorithms for combinatorial problems. J. Comput. Syst. Sci. 9(3), 256–278 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  • Kangas, K., Koivisto, M., Niinimäki, T.: Learning chordal Markov networks by dynamic programming. In: Proceedings of the 28th Annual Conference on Neural Information Processing Systems, pp. 2357–2365. NIPS Foundation (2014)

  • Kass, R., Raftery, A.: Bayes factors. J. Am. Stat. Assoc. 90(430), 773–795 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  • Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. The MIT Press, Cambridge (2009)

    MATH  Google Scholar 

  • Lauritzen, S.L.: Graphical Models. Oxford University Press, Oxford (1996)

    MATH  Google Scholar 

  • Le Berre, D., Parrain, A.: The Sat4j library, release 2.2 system description. J. Satisf. Boolean Model. Comput. 7, 59–64 (2010)

    Google Scholar 

  • Lifschitz, V.: Answer set programming and plan generation. Artif. Intell. 138(1–2), 39–54 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  • Madigan, D., Raftery, A.E.: Model selection and accounting for model uncertainty in graphical models using Occam’s window. J. Am. Stat. Assoc. 89(428), 1535–1546 (1994)

    Article  MATH  Google Scholar 

  • Manquinho, V., Marques-Silva, J., Planes, J.: Algorithms for weighted Boolean optimization. In: Proceedings of the 12th International Conference on Theory and Applications of Satisfiability Testing, pp. 495–508. Springer (2009)

  • Marek, V., Truszczyński, M.: Stable models and an alternative logic programming paradigm. In: The Logic Programming Paradigm: A 25-Year Perspective, pp. 375–398. Springer (1999)

  • Martins, R., Manquinho, V., Lynce, I.: Parallel search for maximum satisfiability. AI Commun. 25(2), 75–95 (2012)

    MathSciNet  MATH  Google Scholar 

  • Niemelä, I.: Logic programming with stable model semantics as a constraint programming paradigm. Ann. Math. Artif. Intell. 25(3–4), 241–273 (1999)

  • Nyman, H., Pensar, J., Koski, T., Corander, J.: Stratified graphical models-context-specific independence in graphical models. Bayesian Anal. 9(4), 883–908 (2014)

  • Parviainen, P., Farahani, H.S., Lagergren, J.: Learning bounded tree-width Bayesian networks using integer linear programming. In: Proceedings of the 17th International Conference on Artificial Intelligence and Statistics, pp. 751–759. JMLR.org (2014)

  • Pensar, J., Nyman, H., Koski, T., Corander, J.: Labeled directed acyclic graphs: a generalization of context-specific independence in directed graphical models. Data Min. Knowl. Discov. 29(2), 503–533 (2015)

    Article  MathSciNet  Google Scholar 

  • Rose, D.J.: Triangulated graphs and the elimination process. J. Math. Anal. Appl. 32(3), 597–609 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  • Sebastiani, R., Tomasi, S.: Optimization in SMT with LA(Q) cost functions. In: Gramlich, B., Miller, D., Sattler, U. (eds.) Automated Reasoning, pp. 484–498. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  • Silander, T., Kontkanen, P., Myllymäki, P.: On sensitivity of the MAP Bayesian network structure to the equivalent sample size parameter. In: Proceedings of the The 23rd Conference on Uncertainty in Artificial Intelligence (UAI-2007), pp. 360–367. AUAI Press (2007)

  • Silander, T., Roos, T., Kontkanen, P., Myllymäki, P.: Factorized NML criterion for learning Bayesian network structures. In: Proceedings 4th European Workshop on Probabilistic Graphical Models (PGM-2008) (2008)

  • Silander, T., Roos, T., Myllymäki, P.: Learning locally minimax optimal Bayesian networks. Int. J. Approx. Reason. 51(5), 544–557 (2010)

    Article  MathSciNet  Google Scholar 

  • Sinz, C.: Towards an optimal CNF encoding of Boolean cardinality constraints. In: Proceedings of the 11th International Conference on Principles and Practice of Constraint Programming, pp. 827–831. Springer (2005)

  • Whittaker, J.: Graphical Models in Applied Multivariate Statistics. Wiley, New York (1990)

    MATH  Google Scholar 

Download references

Acknowledgments

This work was supported by the Academy of Finland (Finnish Centre of Excellence in Computational Inference Research COIN, 251170).

Author information

Authors and Affiliations

  1. Department of Computer Science, Aalto University, Espoo, Finland

    Tomi Janhunen, Martin Gebser & Jussi Rintanen

  2. University of Potsdam, Potsdam, Germany

    Martin Gebser

  3. Griffith University, Brisbane, Australia

    Jussi Rintanen

  4. Department of Mathematics and Statistics, Åbo Akademi University, Åbo, Finland

    Henrik Nyman & Johan Pensar

  5. Department of Mathematics and Statistics, University of Helsinki, Helsinki, Finland

    Jukka Corander

Authors
  1. Tomi Janhunen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Gebser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jussi Rintanen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Henrik Nyman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Johan Pensar
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jukka Corander
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jukka Corander.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Janhunen, T., Gebser, M., Rintanen, J. et al. Learning discrete decomposable graphical models via constraint optimization. Stat Comput 27, 115–130 (2017). https://doi.org/10.1007/s11222-015-9611-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-015-9611-4

Keywords

Search

Navigation