Matthias Petri
Alistair Moffat
ABSTRACT
Processing top-k bag-of-words queries is critical to many information retrieval applications, including web-scale search. In this work, we consider algorithmic properties associated with dynamic pruning mechanisms. Such algorithms maintain a score threshold (the k th highest similarity score identified so far) so that low-scoring documents can be bypassed, allowing fast top-k retrieval with no loss in effectiveness. In standard pruning algorithms the score threshold is initialized to the lowest possible value. To accelerate processing, we make use of term- and query-dependent features to predict the final value of that threshold, and then employ the predicted value right from the commencement of processing. Because of the asymmetry associated with prediction errors (if the estimated threshold is too high the query will need to be re-executed in order to assure the correct answer), the prediction process must be risk-sensitive. We explore techniques for balancing those factors, and provide detailed experimental results that show the practical usefulness of the new approach.
- D. Beck, L. Specia, and T. Cohn. Exploring prediction uncertainty in machine translation quality estimation. In Proc. CoNLL, pages 208--218, 2016.Google Scholar
Cross Ref
- A. Z. Broder, D. Carmel, M. Herscovici, A. Soffer, and J. Zien. Efficient query evaluation using a two-level retrieval process. In Proc. CIKM, pages 426--434, 2003. Google ScholarDigital Library
- C. Burges. From RankNet to LambdaRank to LambdaMart: An overview. Learning, 11 (23--581): 81, 2010.Google Scholar
- B. B. Cambazoglu, H. Zaragoza, O. Chapelle, J. Chen, C. Liao, Z. Zheng, and J. Degenhardt. Early exit optimizations for additive machine learned ranking systems. In Proc. WSDM, pages 411--420, 2010. Google ScholarDigital Library
- R.-C. Chen, L. Gallagher, R. Blanco, and J. S. Culpepper. Efficient cost-aware cascade ranking in multi-stage retrieval. In Proc. SIGIR, pages 445--454, 2017. Google ScholarDigital Library
- P. F. Christoffersen and F. X. Diebold. Optimal prediction under asymmetric loss. Econometric Theory, 13 (06): 808--817, 1997.Google ScholarCross Ref
- C. L. A. Clarke, J. S. Culpepper, and A. Moffat. Assessing efficiency-effectiveness tradeoffs in multi-stage retrieval systems without using relevance judgments. Inf. Retr., 19 (4): 351--377, 2016. Google ScholarDigital Library
- N. Craswell, R. Jones, G. Dupret, and E. Viegas, editors. Proc. 2009 Workshop on Web Search Click Data: WSCD@WSDM. ACM, 2009. Google ScholarCross Ref
- J. S. Culpepper, C. L. A. Clarke, and J. Lin. Dynamic cutoff prediction in multi-stage retrieval systems. In Proc. Aust. Doc. Comp. Symp., pages 17--24, 2016. Google ScholarDigital Library
- W. Dabney, M. Rowland, M. G. Bellemare, and R. Munos. Distributional reinforcement learning with quantile regression. In Proc. AAAI, pages 2892--2901, 2018.Google Scholar
- C. M. Daoud, E. S. de Moura, D. Fernandes, A. S. da Silva, C. Rossi, and A. Carvalho. Waves: A fast multi-tier top-k query processing algorithm. Inf. Retr., 20 (3): 292--316, 2017. Google ScholarDigital Library
- D. Dato, C. Lucchese, F. M. Nardini, S. Orlando, R. Perego, N. Tonellotto, and R. Venturini. Fast ranking with additive ensembles of oblivious and non-oblivious regression trees. ACM Trans. Inf. Sys., 35 (2): 15.1--15.31, 2016. Google ScholarDigital Library
- L. L. S. de Carvalho, E. S. de Moura, C. M. Daoud, and A. S. da Silva. Heuristics to improve the BMW method and its variants. J. Data Inf. Qual., 6: 178--191, 2015.Google Scholar
- L. Dhulipala, I. Kabiljo, B. Karrer, G. Ottaviano, S. Pupyrev, and A. Shalita. Compressing graphs and indexes with recursive graph bisection. In Proc. KDD, pages 1535--1544, 2016. Google ScholarDigital Library
- C. Dimopoulos, S. Nepomnyachiy, and T. Suel. Optimizing top-k document retrieval strategies for block-max indexes. In Proc. WSDM, pages 113--122, 2013. Google ScholarDigital Library
- S. Ding and T. Suel. Faster top-k document retrieval using block-max indexes. In Proc. SIGIR, pages 993--1002, 2011. Google ScholarDigital Library
- M. Fontoura, V. Josifovski, J. Liu, S. Venkatesan, X. Zhu, and J. Zien. Evaluation strategies for top-k queries over memory-resident inverted indexes. Proc. VLDB, 4 (12): 1213--1224, 2011.Google ScholarDigital Library
- J. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29 (5): 1189--1232, 2001.Google ScholarCross Ref
- Y. Gal and Z. Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proc. ICML, pages 1050--1059, 2016. Google ScholarDigital Library
- Y. He, J. Tang, H. Ouyang, C. Kang, D. Yin, and Y. Chang. Learning to rewrite queries. In Proc. CIKM, pages 1443--1452, 2016. Google ScholarDigital Library
- J. Hensman, N. Fusi, and N. D. Lawrence. Gaussian processes for big data. In Proc. UAI, pages 282--290, 2013. Google ScholarDigital Library
- P. J. Huber. Robust estimation of a location parameter. Ann. Math. Statist., 35 (1): 73--101, 1964.Google ScholarCross Ref
- S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proc. ICML, pages 448--456, 2015. Google ScholarDigital Library
- X. Jin, T. Yang, and X. Tang. A comparison of cache blocking methods for fast execution of ensemble-based score computation. In Proc. SIGIR, pages 629--638, 2016. Google ScholarDigital Library
- A. Kane and F. W. Tompa. Split-lists and initial thresholds for WAND-based search. In Proc. SIGIR, pages 877--880, 2018. Google ScholarDigital Library
- Y. Kim, J. Callan, J. S. Culpepper, and A. Moffat. Does selective search benefit from WAND optimization? In Proc. ECIR, pages 145--158, 2016.Google ScholarCross Ref
- D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proc. ICLR, pages 1--15, 2015.Google Scholar
- T. Kraska, A. Beutel, E. H. Chi, J. Dean, and N. Polyzotis. The case for learned index structures. In Proc. SIGMOD, pages 489--504, 2018. Google ScholarDigital Library
- C. Louizos, K. Ullrich, and M. Welling. Bayesian compression for deep learning. In Proc. NeurIPS, pages 3288--3298, 2017. Google ScholarDigital Library
- C. Lucchese, F. M. Nardini, S. Orlando, R. Perego, N. Tonellotto, and R. Venturini. Quickscorer: A fast algorithm to rank documents with additive ensembles of regression trees. In Proc. SIGIR, pages 73--82, 2015. Google ScholarDigital Library
- C. Lucchese, F. M. Nardini, S. Orlando, R. Perego, N. Tonellotto, and R. Venturini. Exploiting CPU SIMD extensions to speed-up document scoring with tree ensembles. In Proc. SIGIR, pages 833--836, 2016. Google ScholarDigital Library
- C. Lucchese, F. M. Nardini, S. Orlando, R. Perego, and S. Trani. X-DART: Blending dropout and pruning for efficient learning to rank. In Proc. SIGIR, pages 1077--1080, 2017. Google ScholarDigital Library
- C. Macdonald, N. Tonellotto, and I. Ounis. Efficient and effective selective query rewriting with efficiency predictions. In Proc. SIGIR, pages 495--504, 2017. Google ScholarDigital Library
- D. J. C. MacKay. Bayesian interpolation. Neural Computation, 4 (3): 415--447, 1992. Google ScholarDigital Library
- J. Mackenzie, J. S. Culpepper, R. Blanco, M. Crane, C. L. A. Clarke, and J. Lin. Query driven algorithm selection in early stage retrieval. In Proc. SIGIR, pages 396--404, 2018. Google ScholarDigital Library
- J. Mackenzie, A. Mallia, M. Petri, J. S. Culpepper, and T. Suel. Compressing inverted indexes with recursive graph bisection: A reproducibility study. In Proc. ECIR, pages 339--352, 2019.Google ScholarDigital Library
- A. Mallia and E. Porciani. Faster BlockMax WAND with longer skipping. In Proc. ECIR, pages 771--778, 2019.Google ScholarDigital Library
- A. Mallia, G. Ottaviano, E. Porciani, N. Tonellotto, and R. Venturini. Faster BlockMax WAND with variable-sized blocks. In Proc. SIGIR, pages 625--634, 2017. Google ScholarDigital Library
- A. Mallia, M. Siedlaczek, and T. Suel. An experimental study of index compression and DAAT query processing methods. In Proc. ECIR, pages 353--368, 2019.Google ScholarDigital Library
- N. Mamoulis, M. L. Yiu, K. H. Cheng, and D. W. Cheung. Efficient top-k aggregation of ranked inputs. ACM Trans. Data. Sys., 32 (3): 19, 2007. Google ScholarDigital Library
- B. Mitra and N. Craswell. An introduction to neural information retrieval. Found. Trnd. Inf. Retr., 13 (1): 1--126, 2018.Google ScholarCross Ref
- K. P. Murphy. Machine Learning: A Probabilistic Perspective. MIT Press, 2012. Google ScholarDigital Library
- G. Ottaviano and R. Venturini. Partitioned Elias-Fano indexes. In Proc. SIGIR, pages 273--282, 2014. Google ScholarDigital Library
- S. Peter, F. Diego, F. A. Hamprecht, and B. Nadler. Cost efficient gradient boosting. In Proc. NeurIPS, pages 1550--1560, 2017. Google ScholarDigital Library
- M. Petri, J. S. Culpepper, and A. Moffat. Exploring the magic of WAND. In Proc. Aust. Doc. Comp. Symp., pages 58--65, 2013. Google ScholarDigital Library
- C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006. Google ScholarDigital Library
- N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15: 1929--1958, 2014. Google ScholarDigital Library
- T. Strohman, H. R. Turtle, and W. B. Croft. Optimization strategies for complex queries. In Proc. SIGIR, pages 219--225, 2005. Google ScholarDigital Library
- M. Theobald, G. Weikum, and R. Schenkel. Top-k query evaluation with probabilistic guarantees. In Proc. VLDB, pages 648--659, 2004. Google ScholarDigital Library
- N. Tonellotto, C. Macdonald, and I. Ounis. Efficient and effective retrieval using selective pruning. In Proc. WSDM, pages 63--72, 2013. Google ScholarDigital Library
- H. R. Turtle and J. Flood. Query evaluation: Strategies and optimizations. Inf. Proc. & Man., 31 (6): 831--850, 1995. Google ScholarDigital Library
- H. Varian. A Bayesian approach to real estate assessment. In S. E. Fienberg and A. Zellner, editors, Studies in Bayesian Econometrics and Statistics in Honor of Leonard J. Savage, pages 195--208. 1975.Google Scholar
- L. Wang, J. Lin, and D. Metzler. A cascade ranking model for efficient ranked retrieval. In Proc. SIGIR, pages 105--114, 2011. Google ScholarDigital Library
- H. Wu and H. Fang. Document prioritization for scalable query processing. In Proc. CIKM, pages 1609--1618, 2014. Google ScholarDigital Library
- Z. Xu, M. J. Kusner, K. Q. Weinberger, M. Chen, and O. Chapelle. Classifier cascades and trees for minimizing feature evaluation cost. J. Mach. Learn. Res., 15: 2113--2144, 2014. Google ScholarDigital Library
- D. Yin, Y. Hu, J. Tang, T. Daly, M. Zhou, H. Ouyang, J. Chen, C. Kang, H. Deng, C. Nobata, J.-M. Langlois, and Y. Chang. Ranking relevance in Yahoo search. In Proc. KDD, pages 323--332, 2016. Google ScholarDigital Library
- H. Zamani, M. Dehghani, W. B. Croft, E. Learned-Miller, and J. Kamps. From neural re-ranking to neural ranking: Learning a sparse representation for inverted indexing. In Proc. CIKM, pages 497-506, 2018. Google ScholarDigital Library
Index Terms
- Accelerated Query Processing Via Similarity Score Prediction
Recommendations
Using an Inverted Index Synopsis for Query Latency and Performance Prediction
Predicting the query latency by a search engine has important benefits, for instance, in allowing the search engine to adjust its configuration to address long-running queries without unnecessarily sacrificing its effectiveness. However, for the dynamic ...
Optimizing Scoring and Sorting Operations for Faster WAND Processing
Advanced Data Mining and ApplicationsAbstractRecent years, a lot of research has focused on how to improve query processing efficiency of large-scale search engines. In this paper, we focus on top-k query processing on document-sorted indexes and the well-known rank-safe dynamic pruning ...
Query efficiency prediction for dynamic pruning
LSDS-IR '11: Proceedings of the 9th workshop on Large-scale and distributed informational retrievalDynamic pruning strategies are effective yet permit efficient retrieval by pruning - i.e. not fully scoring all postings of all documents matching a given query. However, the amount of pruning possible for a query can vary, resulting in queries with ...
Comments