Skip to main content
SpringerLink
Log in

Convergence assessment techniques for Markov chain Monte Carlo

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

MCMC methods have effectively revolutionised the field of Bayesian statistics over the past few years. Such methods provide invaluable tools to overcome problems with analytic intractability inherent in adopting the Bayesian approach to statistical modelling.

However, any inference based upon MCMC output relies critically upon the assumption that the Markov chain being simulated has achieved a steady state or “converged”. Many techniques have been developed for trying to determine whether or not a particular Markov chain has converged, and this paper aims to review these methods with an emphasis on the mathematics underpinning these techniques, in an attempt to summarise the current “state-of-play” for convergence assessment techniques and to motivate directions for future research in this area.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ahmad, R. (1976) On the Multivariate K-Sample Problem and the Generalization of the Kolmogorov-Smirnov Test. Annals of the Institute of Statistical Mathematics 28, 259–265.

    Google Scholar 

  • Amit, Y. (1991) On Rates of Convergence of Stochastic Relaxation for Gaussian and Non-Gaussian Distributions. Journal of Multivariate Analysis 38, 82–99.

    Google Scholar 

  • Amit, Y. and Grenander, U. (1991) Comparing Sweep Strategies for Stochastic Relaxation. Journal of Multivariate Analysis 37, 197–222.

    Google Scholar 

  • Applegate, D., Kannan, R. and Polson, N. G. (1990) Random Polynomial Time Algorithms for Sampling from Joint Distributions. Technical report, Carnegie-Mellon University, Tech. Rep. No 500.

  • Asmussen, S., Glynn, P. W. and Thorisson, H. (1992) Stationarity Detection in the Initial Transient Problem. ACM Transactions on Modelling and Computer Simulation 2, 130–157.

    Google Scholar 

  • Brittain, E. H. (1987) P-Values for the Multi-Sample Kolmogorov-Smirnov Test using the Expanded Bonferroni Approximation. Communications in Statistics — Theory and Methods 16(3), 821–835.

    Google Scholar 

  • Brooks, S. P. (1997) Discussion to Richardson and Green (1997) Journal of the Royal Statistical Society, Series B 59, 774–775

    Google Scholar 

  • Brooks, S. P. (1998a), Markov Chain Monte Carlo Method and its Application. The Statistician 47, 69–100.

    Google Scholar 

  • Brooks, S. P. (1998b) MCMC Convergence Diagnosis via Multivariate Bounds on Log-Concave Densities. The Annals of Statistics 26, 398–433.

    Google Scholar 

  • Brooks, S. P. (1998c) Quantitative Convergence Diagnosis for MCMC via CUSUMS. Statistics and Computing 8, 267–274.

    Google Scholar 

  • Brooks, S. P., Dellaportas, P. and Roberts, G. O. (1997) A Total Variation Method for Diagnosing Convergence of MCMC Algorithms. Journal of Computational and Graphical Statistics 6, 251–265.

    Google Scholar 

  • Brooks, S. P. and Gelman, A. (1998) Alternative Methods for Monitoring Convergence of Iterative Simulations. Journal of Computational and Graphical Statistics 7, 434–455.

    Google Scholar 

  • Brooks, S. P. and Roberts, G. O. (1997) On Quantile Estimation and MCMC Convergence. Biometrika In press.

  • Chan, K. S. (1993) Asymptotic Behaviour of the Gibbs Sampler. Journal of the American Statistical Association 88, 320–328.

    Google Scholar 

  • Chorneyko, I. Z. and Zing, L. L. K. (1982) K-Sample Analogues of Two-Sample Kolmogorov-Smirnov Statistics. Selecta Statistica Canadiana 6, 37–62.

    Google Scholar 

  • Conover, W. J. (1965) Several K-Sample Kolmogorov-Smirnov Tests. Annals of Mathematical Statistics 36, 1019–1026.

    Google Scholar 

  • Conover, W. J. (1967) A k-Sample Extension of the One-Sided Two-Sample Smirnov Test Statistic. Annals of Mathematical Statistics 38, 1726–1730.

    Google Scholar 

  • Cowles, M. K. and Carlin, B. P. (1996) Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review. Journal of the American Statistical Association 91, 883–904.

    Google Scholar 

  • Cowles, M. K. and Rosenthal, J. S. (1998) A Simulation Approach to Convergence Rates for Markov Chain Monte Carlo. Technical report, Harvard School of Public Health.

  • Diaconis, P. (1988) Group Representations in Probability and Statistics, vol. 11 of Lecture Notes — Monograph Series. Institute of Mathematical Statistics.

  • Diaconis, P. and Stroock, D. (1991) Geometric Bounds for Eigenvalues of Markov Chains. Annals of Applied Probability 1, 36–61.

    Google Scholar 

  • Fasano, G. and Franceschini, A. (1987) A Multidimensional Version of the Kolmogorov-Smirnov Test. Monthly Notices of the Royal Astronomical Society 225, 155–170.

    Google Scholar 

  • Garren, S. and Smith, R. L. (1995) Estimating the Second Largest Eigenvalue of a Markov Transition Matrix. Technical report, University of Cambridge.

  • Gelfand, A. E. (1992) Discussion of Gelman and Rubin (1992) Statistical Science 7, 486–487.

    Google Scholar 

  • Gelfand, A. E., Hills, S. E., Racine-Poon, A. and Smith, A. F. M. (1990) Illustration of Bayesian Inference in Normal data Models using Gibbs Sampling. Journal of the American Statistical Association 85, 972–985.

    Google Scholar 

  • Gelfand, A. E. and Smith, A. F. M. (1990) Sampling Based Approaches to Calculating Marginal Densities. Journal of the American Statistical Association 85, 398–409.

    Google Scholar 

  • Gelman, A. and Rubin, D. B. (1992) Inference from Iterative Simulation using Multiple Sequences. Statistical Science 7, 457–511.

    Google Scholar 

  • Geman, S. and Geman, D. (1984) Stochastic Relaxation, Gibbs Distributions and the Bayesian Restoration of Images. IEEE Transactions on pattern analysis and machine intelligence 6, 721–741.

    Google Scholar 

  • Geweke, J. (1989) Bayesian Inference in Econometric Models using Monte-Carlo Integration. Econometrica 57, 1317–1339.

    Google Scholar 

  • Geweke, J. (1992) Evaluating the Accuracy of Sampling-Based Approaches to the Calculation of Posterior Moments. In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 4, pp. 169–193, New York: Oxford University Press.

    Google Scholar 

  • Geyer, C. J. and Thompson, E. A. (1995) Annealing Markov Chain Monte Carlo with Applications to Ancestral Inference. Journal of the American Statistical Association 90, 909–920.

    Google Scholar 

  • Gilks, W. R., Roberts, G. O. and Sahu, S. K. (1996) Adaptive Markov Chain Monte Carlo. Technical report, MRC Biostatistics Unit, Cambridge.

    Google Scholar 

  • Green, P. J. (1995) Reversible Jump MCMC Computation and Bayesian Model determination. Biometrika 82, 711–732.

    Google Scholar 

  • Hastings, W. K. (1970) Monte Carlo Sampling Methods using Markov Chains and their Applications. Biometrika 57, 97–109.

    Google Scholar 

  • Heidelberger, P. and Welch, P. D. (1981a) Adaptive Spectral Methods for Simulation Output Analysis. IBM Journal of Research and Development 25, 860–876.

    Google Scholar 

  • Heidelberger, P. and Welch, P. D. (1981b) A Spectral Method for Confidence Interval Generation and Run Length Control in Simulations. Communications of the Association Computing Machinery 24, 233–245.

    Google Scholar 

  • Heidelberger, P. and Welch, P. D. (1983) Simulation Run Length Control in the Presence of an Initial Transient. Operations Research 31, 1109–1144.

    Google Scholar 

  • Jennison, C. (1993) Discussion to Smith and Roberts (1993) Journal of the Royal Statistical Society, Series B 55, 54–56.

    Google Scholar 

  • Johnson, V. E. (1996) Studying Convergence of Markov chain Monte Carlo Algorithms using Coupled Sample Paths. Journal of the American Statistical Association 91, 154–166.

    Google Scholar 

  • Kendall, W. S. (1997) Perfect simulation for the area interaction process. In C. C. Heyde and L. Acardi (eds.), Probability Perspective, World Scientific Press.

  • Kolmogorov, A. N. (1933) Sulla Determiazione Empirica delle Leggi di Probabilita. Giornal Istitutto Italia Attuari 4, 1–11.

    Google Scholar 

  • Lawler, G. F. and Sokal, A. D. (1988) Bounds on the L 2 spectrum for Markov Chains and their Applications. Transactions of the American Mathematical Society 309, 557–580.

    Google Scholar 

  • Lin, Z. Y. (1992) On the Increments of Partial Sums of a ø-mixing Sequence. Theory of Probability and its Applications 36, 316–326.

    Google Scholar 

  • Lindvall, T. (1992) Lectures on the Coupling Method. Wiley.

  • Liu, C., J. Liu and Rubin, D. B. (1993) A Control Variable for Assessment the Convergence of the Gibbs Sampler. In Proceedings of the Statistical Computing Section of the American Statistical Association, pp. 74–78.

  • Liu, J. S. (1994) Metropolized Independent Sampling with Comparisons to Rejection Sampling and Importance Sampling. Technical report, Harvard University.

  • Müller, P. (1991) A Generic Approach to Posterior Integration and Gibbs Sampling. Technical report, Dept. of Statistics, Purdue University.

  • Mengersen, K. L. and Tweedie, R. L. (1995) Rates of Convergence of the Hastings and Metropolis Algorithms. Annals of Statistics 24, 101–121.

    Google Scholar 

  • Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953) Equations of State Calculations by Fast Computing Machines. Journal of Chemical Physics 21, 1087–1091.

    Google Scholar 

  • Meyn, S. P. and Tweedie, R. L. (1993) Markov Chains and Stochastic Stability. Springer-Verlag.

  • Meyn, S. P. and Tweedie, R. L. (1994) Computable Bounds for geometric Convergence Rates of Markov Chains. Annals of Applied Probability 4, 981–1011.

    Google Scholar 

  • Mises von, R. (1931) Wahrscheinlichkeitschrechnung. Deutiche, Vienna.

    Google Scholar 

  • Møller, J. (1997) Markov chain Monte Carlo and spatial point processes. In W. S. Kendall and M. N. M. vanLieshout (eds.), Stochastic geometry, likelihood and computation, Chapman and Hall.

  • Murdoch, D. J. and Green, P. J. (1997) Exact Sampling from a Continuous State Space. Technical report, Queen's University, Ontario.

    Google Scholar 

  • Mykland, P., Tierney, L., and Yu, B. (1995) Regeneration in Markov Chain Samplers. Journal of the American Statistical Association 90, 233–241.

    Google Scholar 

  • Nummelin, E. (1984) General Irreducible Markov Chains and Non-Negative Operators. Cambridge University Press.

  • Peskun, P. H. (1973) Optimum Monte Carlo Sampling using Markov Chains. Biometrika 60, 607–612.

    Google Scholar 

  • Polson, N. G. (1996) Convergence of Markov Chain Monte Carlo Algorithms. In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 5, Oxford University Press.

  • Propp, J. G. and Wilson, D. B. (1996) Exact Sampling with Coupled Markov Chains and Applications to Statistical Mechanics. Random Structures and Algorithms 9, 223–252.

    Google Scholar 

  • Raftery, A. E. and Lewis, S. M. (1992) How Many Iterations in the Gibbs Sampler? In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 4, Oxford University Press.

  • Reutter, A. and Johnson, V. (1995) General Strategies for Assessing Convergence of MCMC Algorithms Using Coupled Sample Paths. Technical report, Duke University.

  • Ripley, B. D. (1987) Stochastic Simulation. John Wiley and Sons.

  • Ritter, C. and Tanner, M. A. (1992) Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler. Journal of the American Statistical Association 87, 861–868.

    Google Scholar 

  • Robert, C. P. (1996) Convergence Assessments for Markov Chain Monte Carlo Methods. Statistical Science 10, 231–253.

    Google Scholar 

  • Roberts, G. O. (1992) Convergence Diagnostics of the Gibbs sampler. In J. M. Bernardo, A. F. M. Smith, A. P. Dawid and J. O. Berger (eds.), Bayesian Statistics 4, Oxford University Press.

  • Roberts, G. O. (1994) Methods for Estimating L 2 Convergence of Markov Chain Monte Carlo. In D. Berry, K. Chaloner and J. Geweke (eds.), Bayesian Statistics and Econometrics: Essays in Honor of Arnold Zellner, North Holland: Amsterdam.

    Google Scholar 

  • Roberts, G. O. and Polson, N. G. (1994) On the Geometric Convergence of the Gibbs Sampler. Journal of the Royal Statistical Society, Series B 56, 377–384.

    Google Scholar 

  • Roberts, G. O. and Rosenthal, J. S. (1998) Convergence of Slice Sampler Markov Chains. Journal of the Royal Statistical Society, Series B In press.

  • Roberts, G. O. and Sahu, S. K. (1996) Rate of Convergence of the Gibbs Sampler by Gaussian Approximation. Technical report, University of Cambridge.

  • Roberts, G. O. and Sahu, S. K. (1997) Updating Schemes, Covariance Structure, Blocking and Parameterisation for the Gibbs Sampler. Journal of the Royal Statistical Society, Series B 59, 291–318.

    Google Scholar 

  • Roberts, G. O. and Smith, A. F. M. (1994) Simple Conditions for the Convergence of the Gibbs Sampler and Metropolis Hastings Algorithms. Stochastic Processes and Applications 49, 207–216.

    Google Scholar 

  • Roberts, G. O. and Tweedie, R. L. (1996a) Exponential Convergence of Langevin Diffusions and their Discrete Approximations. Bernoulli 2, 341–363.

    Google Scholar 

  • Roberts, G. O. and Tweedie, R. L. (1996b) Geometric Convergence and Central Limit Theorems, for Multidimensional Hastings and Metropolis Algorithms. Biometrika 83, 95–110.

    Google Scholar 

  • Roberts, G. O. and Tweedie, R. L. (1998) Bounds on Regeneration Times and Convergence Rates for Markov Chains. Technical report, University of Cambridge.

  • Rosenthal, J. (1995a) Rates of Convergence for Gibbs Sampling for Variance Component Models. Annals of Statistics 23, 740–761.

    Google Scholar 

  • Rosenthal, J. S. (1995b) Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo. Journal of the American Statistical Association 90, 558–566.

    Google Scholar 

  • Rosenthal, J. S. (1995c) Rates of Convergence for Data Augmentation on Finite Sample Spaces. Annals of Applied Probability 3, 319–339.

    Google Scholar 

  • Schervish, M. J. and Carlin, B. P. (1992) On the Convergence of Successive Substitution Sampling. Journal of Computational and Graphical statistics 1, 111–127.

    Google Scholar 

  • Schruben, L., Singh, H. and Tierney, L. (1983) Optimal Tests for Initialization Bias in Simulation Output. Operations Research 31, 1167–1178.

    Google Scholar 

  • Schruben, L. W. (1982) Detecting Initialization Bias in Simulation Output. Operations Research 30, 569–590.

    Google Scholar 

  • Silverman, B. W. (1986) Density Estimation for Statistics and Data Analysis. London: Chapman and Hall.

    Google Scholar 

  • Sinclair, A. J. and Jerrum, M. R. (1988) Conductance and the Rapid Mixing Property for Markov Chains: The Approximation of the Permanent Resolved. In Proceedings of the 20th annual ACM symposium on the Theory of Computing.

  • Smith, A. F. M. and Gelfand, A. E. (1992) Bayesian Statistics Without Tears: A Sampling-Resampling Perspective. The American Statistician 46, 84–88.

    Google Scholar 

  • Smith, R. L. and Tierney, L. (1996) Exact Transition Probabilities for the Independence Metropolis Sampler. Technical report, Statistical Laboratory, Cambridge.

    Google Scholar 

  • Tierney, L. (1994) Markov Chains for Exploring Posterior Distributions. Annals of Statistics 22, 1701–1762.

    Google Scholar 

  • Yu, B. (1995a) Discussion to Besag et al. (1995) Statistical Science 10, 3–66.

    Google Scholar 

  • Yu, B. (1995b) Estimating L 1 Error of Kernel Estimator: Monitoring Convergence of Markov Samplers. Technical report, Dept. of Statistics, University of California, Berkeley.

    Google Scholar 

  • Yu, B. and Mykland, P. (1998) Looking at Markov Sampler through Cusum Path Plots: A Simple Diagnostic Idea. Statistics and Computing 8, 275–286.

    Google Scholar 

  • Zellner, A. and Min, C. (1995) Gibbs Sampler Convergence Criteria. Journal of the American Statistical Association 90, 921–927.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics, University of Bristol, University Walk, Bristol, BS8 1TW

    STEPHEN P. BROOKS & GARETH O. ROBERTS

  2. Department of Maths and Statistics, University of Leicester, LA1 4YF

    STEPHEN P. BROOKS & GARETH O. ROBERTS

Authors
  1. STEPHEN P. BROOKS
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. GARETH O. ROBERTS
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

BROOKS, S.P., ROBERTS, G.O. Convergence assessment techniques for Markov chain Monte Carlo. Statistics and Computing 8, 319–335 (1998). https://doi.org/10.1023/A:1008820505350

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008820505350

Search

Navigation