Upward and downward bias when measuring inequality of opportunity

Abstract

Estimates of the level of inequality of opportunity have traditionally been proposed as lower bounds due to the downward bias resulting from the partial observability of circumstances that affect individual outcome. We show that such estimates may also suffer from upward bias as a consequence of sampling variance. The magnitude of the latter distortion depends on both the empirical strategy used and the observed sample. We suggest that, although neglected in empirical contributions, the upward bias may be significant and challenge the interpretation of inequality of opportunity estimates as lower bounds. We propose a simple criterion to select the best specification that balances the two sources of bias. Our method is based on cross-validation and can easily be implemented with survey data. To show how this method can improve the reliability of inequality of opportunity measurement, we provide an empirical illustration based on income data from 31 European countries. Our evidence shows that estimates of inequality of opportunity are sensitive to model selection. Alternative specifications lead to significant differences in the absolute level of inequality of opportunity and to the re-ranking of a number of countries, which confirms the need for an objective criterion to select the best econometric model when measuring inequality of opportunity.

Appendices

A Upward bias when estimating IOp with survey data

Chakravarty and Eichhorn (1994) distinguish between the true distribution of income, y, and the observed distribution, $\tilde{y}$, where $\tilde{y}=$$y+e$ and e is commonly defined as the measurement error such that $e\sim iid(0,\sigma ^{2})$. By considering a strictly concave von Neumann–Morgenstern utility function, U, they prove by analogy that if we measure inequality $I(\tilde{y})$ with an inequality index I that satisfies symmetry and the Pigou-Dalton transfer principle, then the inequality of the true y distribution is smaller than inequality in the observed distribution.

Without loss of generality, we apply their result to the case of non-parametric IOp measurement (Eq. 2).

Proposition

Let $\tilde{Y}$ be the counterfactual distribution estimated with Eq. 2. Assume that $\tilde{Y}$ is estimated by observing the full set of circumstances and the entire population. Let $\hat{\tilde{Y}}$ be the same counterfactual distribution estimated by observing the full set of circumstances but considering only a proper subsample of the entire population. Let IOp and $\hat{IOp}$ be any measure of inequality that satisfies symmetry and the Pigou-Dalton transfer principle applied to $\tilde{Y}$ and $\hat{\tilde{Y}}$ respectively. Then, $E(\hat{IOp})>IOp$.

Proof

Let $\mathbf {M} =\mu _1, \ldots , \mu _T$ be the vector of types’ mean outcomes in the population. Let $\hat{\mathbf {M}} =\hat{\mu _1}, \ldots , \hat{\mu _T}$ be the estimates of types’ means based on a proper subsample of the population. Then, for each $t=1,\ldots ,n$, $\hat{\mu }_{t}=$$\mu _{t}+\eta$, where $\eta = \frac{\sigma }{\sqrt{N_{t}}}\sim (0, \chi ^2)$ is the standard error of $\hat{ \mu }_{t}$.

Following Chakravarty and Eichhorn (1994), we assume that U is a strictly concave function. By Jensen’s inequality, we have

$$\begin{aligned} E\left( U\left( \hat{\mathbf {M}}| \mathbf {M}\right) \right) <U\left( E\left( \hat{\mathbf {M}}| \mathbf {M} \right) \right) . \end{aligned}$$

(5)

Note that $E\left( \hat{\mathbf {M}}| \mathbf {M} \right) = \mathbf {M}$, so:

$$\begin{aligned} E\left( U\left( \hat{\mathbf {M}}|\mathbf {M} \right) \right) <U\left( \mathbf { M} \right) . \end{aligned}$$

(6)

By taking expectations with respect to $\mathbf {M}$ on both sides, (4) becomes:

$$\begin{aligned} E \left( U\left( \hat{\mathbf {M} } \right) \right) <U\left( E(\mathbf {M} ) \right) . \end{aligned}$$

(7)

Because $E(\eta )=0$, the two distributions have the same mean. If U is a strictly concave function, then (5) is equivalent to saying that the distribution of $\mathbf {M}$ Lorenz dominates the distribution of $\hat{ \mathbf {M}}$, which implies that $E(\hat{IOp})>IOp$. $\square$

Corollary

When one or more of the relevant circumstances is not used to partition the population into types (partial observability) and $\tilde{Y}$ is estimated on a proper subsample of the population, $\hat{{IOp}}$ cannot be interpreted as a lower bound of IOp.

B A simulation to assess the magnitude of the upward bias

The reader may wonder whether the upward bias discussed in this paper actually represents a non-negligible issue in empirical implementations. To provide an idea of the possible magnitude of the bias, we perform a simulation. When estimating inequality of opportunity, the data generating process is typically unknown. We therefore prefer to base the simulation on the entire EU-SILC dataset instead of creating an ad hoc dataset.

Assume that the entire EU-SILC dataset is our population of interest. A population composed of 202,843 individuals aged between 26 and 60 years (more than the same age population in Iceland and approximately the same population in Luxembourg). Additionally, assume that a few observable circumstances are the only circumstances that determine inequality of opportunity. Individual outcome is assumed to be the result of the interactions of three circumstances: parental education, parental occupation, and origin. Individuals in the same type share the same highest parental education (five values), same immigration history (a dummy that takes the value of one if the respondent is a first- or second-generation immigrant), and the same highest parental occupation (ISCO 1 digit).

Under our assumptions, we can observe the real partition of the population into types. The observed between-type inequality is then the real IOp in the population. The residual inequality is assumed to be due to effort. Measured by MLD, IOp in the entire sample is 0.0314, approximately 7% of the total variability.

Our aim is then to understand the circumstances under which an estimate of inequality of opportunity based on a random subsample of this population results in upward bias. To this end, we estimate IOp using samples of increasing size. We start with 500, which is approximately the sample size of the smallest country in EU-SILC (Sweden). We then add 500 observations in each step until we have a sample of 20,000 observations (not far from Italy’s sample size, the largest country in EU-SILC). Each sample is randomly drawn 500 times to obtain normalized bootstrap confidence intervals around the point estimate.

Figure 3 shows the IOp estimates for samples of increasing size. In grey, we provide a histogram showing the frequency of countries’ sample size (reported on the right y-axis) in EU-SILC 2011.¹⁵

The estimates show a marked upward bias for the smallest samples. The average IOp based on the samples is more than 1.2 times higher than the IOp in the population for samples smaller than 4000. These are not unrealistically small samples: six of the 31 countries have smaller sample sizes. Interestingly, the confidence intervals of the estimates do not contain the population’s estimate for all samples smaller than 3000 (Sweden, Iceland, Denmark, and Norway have smaller sample sizes). Moreover, the upward bias is less than 10% only for sample sizes larger than 9000. Only France, Germany, Hungary, Poland, Spain, and Italy have larger sample sizes.

Estimates based on the samples approach the IOp in the population rather slowly; at the extreme right of the graph, the bias is approximately 4%. This may be considered a negligible distortion. Interestingly, the reader may recall that in Fig. 1 of Sect. 4, we found a relatively small difference between the IOp estimated with the two extreme specifications for countries with sample sizes larger than 10,000. However, in our simulation, a sample size of 20,000 observations is extremely large as it represents slightly less than 10% of the population.

Fig. 3

Source: EU-SILC, 2011

IOp estimated on samples of increasing size

C Additional tables and figures

See Table 3 and Fig. 4.

Table 3 Model specifications

Fig. 4

Source: EU-SILC, 2011

IOp estimates for 24 European countries from different studies. The figure shows the rank correlation of countries in terms of IOp. The ranking proposed by Suárez and Menéndez (2017) is compared with the ranking proposed by Brzenziński (2015)