The query complexity of estimating weighted averages

Abstract

The query complexity of estimating the mean of some [0, 1] variables is understood. Inspired by some work by Carterette et al. on evaluating retrieval systems, and by Moffat and Zobel’s new proposal for such evaluation, we examine the query complexity of weighted average calculation. In general, determining an answer within accuracy ${\varepsilon}$, with high probability, requires ${\Omega(\varepsilon^{-2})}$ queries, as the mean is a special case. There is a matching upper bound for the weighted mean. If the weights are a normalized prefix of a divergent series, the same result holds. However, if the weights follow a geometric sequence, a sample of size ${\Omega(\log (1/\varepsilon))}$ suffices. Our principal contribution is the investigation of power-law sequences of weights. We show that if the ith largest weight is proportional to i ^−p, for p > 1, then the query complexity is in ${\Omega(\varepsilon^{2/(1-2p)})}$.