Computing inclusion probabilities for order sampling

Abstract

Rosèn [1997. J. Statist. Plann. Inference 62, 159–191] introduced order sampling schemes of fixed shape which have inclusion probabilities roughly proportional to given size measures ($\pi$ps schemes). Three particular cases where the fixed shape distributions are Pareto, exponential and uniform, respectively, are specially treated. In this paper, we give general algorithms for computing the first- and second-order inclusion probabilities for a general fixed shape order sampling scheme and explicit formulae for the three special cases. Identities are given that can be used to check the accuracy of the numerical results. Examples are included as well as some comments on improving the computational efficiency and accuracy of the algorithms.

Introduction

Rosén (1997a) introduced a new class of fixed-size sampling designs called order sampling, where with each unit i in a population $U=\{1,\dots ,N\}$ is associated a random variable, called an ordering variable. Independent realizations of the ordering variables are taken. The units with the n smallest realized values constitute the sample. By requiring all the ordering variables to be proportional to a common random variable, Rosén introduced the idea of order sampling with fixed distribution shape. Three distribution shapes were specifically investigated; namely, the uniform, exponential and Pareto distributions.

Let $y_i$ denote the variable of interest. The central inference task is usually the estimation of its population total $\tau ={\sum }_{i=1}^N{y}_i$. Suppose auxiliary information is available on all the units in the population in the form of size measures $x_i$; that is, $x_i$ is roughly proportional to $y_i$. There are several ways of utilizing this auxiliary information. One of them is to use a sampling scheme where the inclusion probabilities are proportional to the size measures. Such a scheme is called a $\pi$ps scheme. Let ${\lambda }_i={\mathit{nx}}_i/{\sum }_{j=1}^N{x}_j$ denote the target inclusion probability. Rosén, 1997b, Rosén, 2000 showed that, in order sampling with fixed shape, the constants of proportionality can be chosen so that the resultant sampling scheme has inclusion probability approximately equal to ${\lambda }_i$. As $x_i$ is only approximately proportional to $y_i$, such a scheme is satisfactory and can also be classified as a $\pi$ps scheme. The three cases where the underlying shape is uniform, exponential and Pareto are called, respectively, uniform OS$\pi$ps, exponential OS$\pi$ps and Pareto OS$\pi$ps schemes.

Rosén (1997a) showed that uniform OS$\pi$ps was the same as sequential Poisson sampling presented by Ohlsson, 1990, Ohlsson, 1995 and exponential OS$\pi$ps coincided with successive sampling studied by Rosén (1972) and Hájek (1981). Pareto sampling was new and was shown to have an optimal property explained below. Pareto sampling was independently introduced by Rosén (1997a) and Saavedra (1995).

For estimation of $\tau$, Rosén suggested the estimator

$${\widehat{\tau }}_{\mathrm{R}}=\sum _{i=1}^N\frac{y_i}{{\lambda }_i}{I}_i\text{,}$$

where $I_i$ is the sample inclusion indicator ($I_i=1$ if unit i is in the sample and $=0$ otherwise). This avoids the computation of exact inclusion probabilities. Rosén gave the asymptotic variance of ${\widehat{\tau }}_{\mathrm{R}}$ and a variance estimator. He showed that, among all $\pi$ps schemes of fixed shape, the Pareto scheme minimizes the asymptotic variance. On the other hand, if the inclusion probabilities are known, the estimation of $\tau$ can be based on the Horvitz–Thompson (HT) estimator

$${\widehat{\tau }}_{\mathrm{HT}}=\sum _{i=1}^N\frac{y_i}{{\pi }_i}{I}_i\text{,}$$

where ${\pi }_i$ is the actual first-order inclusion probability: ${\pi }_i=\mathrm{Pr}$(unit i is in the sample). This is well known to have minimum variance among the class of homogeneous linear unbiased estimators of $\tau$. The Sen–Yates–Grundy variance estimator for ${\widehat{\tau }}_{\mathrm{HT}}$ is

$$\widehat{V}({\widehat{\tau }}_{\mathrm{HT}})=\frac{1}{2}\sum _{i=1}^N\sum _{j=1}^N{\left(\frac{y_i}{{\pi }_i}-\frac{y_j}{{\pi }_j}\right)}^2\left(\frac{{\pi }_i{\pi }_j}{{\pi }_{\mathit{ij}}}-1\right)I_i{I}_j\text{,}$$

where ${\pi }_{\mathit{ij}}=\mathrm{Pr}$(units i and j are in the sample) are the second-order inclusion probabilities.

Rosén (2000) considered the problem of computing the first-order inclusion probabilities. He obtained explicit formulae for the three fixed shape $\pi$ps schemes for the particular case where all units in a population have the same size measure except for one odd unit. He also obtained a formula for the Pareto $\pi$ps scheme when taking a sample of size 1 from a population with all units having distinct size measures. These are very special and limited cases and Rosèn's overall conclusion is that the computation of inclusion probabilities for $\pi$ps order sampling is exceedingly hard. This gives weight to the paper's main result which is that ${\pi }_i\approx {\lambda }_i$ for large n and N. More precisely, $\lim {\pi }_i/{\lambda }_i\to 1$ as n and N tend to infinity. This justifies using the estimator ${\widehat{\tau }}_{\mathrm{R}}$ as a substitute for ${\widehat{\tau }}_{\mathrm{HT}}$. Rosén (2000) showed that the approximation was reasonable if $\min (n,N-n)⩾5$.

It is desirable to have methods to calculate exact inclusion probabilities. It facilitates the theoretical investigation of the method and the approximations. In some practical applications, some subsamples may be quite small and one may wish to compute the exact inclusion probabilities, as the computation is not time-intensive for small sizes. Furthermore, the variance estimator given by Rosén is also based on an asymptotic result. There is as yet no study on how large n needs to be for the variance estimator to be reliable. A method for computing exact first- and second-order inclusion probabilities will facilitate such a study.

Aires (1999) considered the computation of first- and second-order inclusion probabilities for Pareto $\pi$ps sampling schemes. We propose a different algorithm. It is useful to have different approaches to computing probabilities as they give different insights to the problem. Moreover, one approach may be easier to extend to a different situation than another, such as the selection of coordinated samples. In Section 8, we discuss the mathematical and computational differences between the two methods. Our approach is combinatoric, and, in our view, computes more fundamental quantities which are used with combinatoric arguments to build up the required probabilities. The combinatoric arguments will be easier to adapt to a changed situation.

Even though Rosén (1997a) has shown that the Pareto $\pi$ps scheme is optimal in the sense of minimizing the asymptotic variance of ${\widehat{\tau }}_{\mathrm{R}}$, we have included the computation of first- and second-order inclusion probabilities of all three schemes here for completeness. The exponential and uniform $\pi$ps schemes are already in the literature (Rosén, 1972, Hájek, 1981; Ohlsson, 1990, Ohlsson, 1995). They may have other desirable properties upon further investigation.

In Section 2, we describe the method for computing the first- and second-order inclusion probabilities for a general $\pi$ps order sampling scheme. Sections 3–5 consider the special cases of Pareto, exponential and uniform OS$\pi$ps schemes, respectively. Section 6 gives three identities that are satisfied by the inclusion probabilities and some intermediate results of the computation. These can be used to check the plausibility and accuracy of the numerical results. Section 7 gives some examples. In Section 8, we give some comments on some computational aspects of the formulae for the Pareto $\pi$ps sampling scheme, and the differences between our algorithm and Aires’.