Minimum message length analysis of multiple short time series

Abstract

This paper applies the Bayesian minimum message length principle to the multiple short time series problem, yielding satisfactory estimates for all model parameters as well as a test for autocorrelation. Connections with the method of conditional likelihood are also discussed.

Introduction

Consider data $Y={(y_1,\dots ,y_m)}^{\prime }\in {\mathbb{R}}^{m\times n}$ comprised of $m$ sequences $y_i={(y_{i,1},\dots ,y_{i,n})}^{\prime }\in {\mathbb{R}}^n$ generated by the following stationary first order Gaussian autoregressive model:

$$y_{i,j}={\mu }_i+{\epsilon }_{i,j}\text{,}$$$${\epsilon }_{i,j}=\rho {\epsilon }_{i,j-1}+v_{i,j}\text{,}$$

where $(i=1,\dots ,m;j=1,\dots ,n)$, $\mu ={({\mu }_1,\dots ,{\mu }_m)}^{\prime }\in {\mathbb{R}}^m$ are the sequence means, $\rho \in (-1,1)$ is a common autoregressive parameter and $v_{i,j}$ denotes the innovations which are independently and identically distributed as $N(0,\tau )$. The starting point of this paper is to make inferences about the parameters $\theta ={(\mu ,\rho ,\tau )}^{\prime }\in {\mathbb{R}}^{m+2}$ given data sampled from the model  (1)–(2). The sequences are considered exchangeable, in the sense that inferences made about the model parameters should be invariant under the interchange of any pair of sequences in the matrix $Y$. This model appears frequently in epidemiological and medical studies in which several measurements have been made over time on a large number of people. In this case, the autocorrelation parameter $\rho$ is of particular interest, as it represents how well the physical quantity “tracks” over time.

Making inferences about $\rho$ in this setting is complicated by the fact that the number of parameters grows with the number of sequences $m$ and a straightforward application of the maximum likelihood principle leads to inconsistent estimates of both $\tau$ and $\rho$. A likelihood-based solution to the problem of estimating $\rho$ in the model  (1)–(2) using the method of approximate conditional likelihood was presented in  Cruddas et al. (1989) and shown to yield significant improvements over the standard maximum likelihood estimates. Two frequentist test procedures for the presence of autocorrelation are also discussed in  Cox and Solomon (1988).

A solution within the Bayesian framework of inference would be of great value. Unfortunately, with the choice of sensible priors that reflect the invariance properties required of the problem, the usual method of analysing the posterior distribution formed from the product of the prior distribution and likelihood is unsatisfactory. The posterior distribution does not concentrate probability mass around the true parameter values even as the number of sequences $m\to \infty$, and parameter estimation based on this posterior is subsequently inconsistent. This paper demonstrates that estimation based on the alternative information-theoretic Bayesian principle of minimum message length (Wallace, 2005) leads to satisfactory estimates of all parameters $\theta$ as well as providing a simple basis for testing for autocorrelation.

This paper has three aims: (1) to produce satisfactory point estimates for all parameters of the first order Gaussian autoregressive model, (2) to produce a suitable test for autocorrelation, and (3) demonstrate the resolution of a difficult estimation problem using the minimum message length principle.