Hang Cui
ABSTRACT
State-of-the-art question answering (QA) systems employ term-density ranking to retrieve answer passages. Such methods often retrieve incorrect passages as relationships among question terms are not considered. Previous studies attempted to address this problem by matching dependency relations between questions and answers. They used strict matching, which fails when semantically equivalent relationships are phrased differently. We propose fuzzy relation matching based on statistical models. We present two methods for learning relation mapping scores from past QA pairs: one based on mutual information and the other on expectation maximization. Experimental results show that our method significantly outperforms state-of-the-art density-based passage retrieval methods by up to 78% in mean reciprocal rank. Relation matching also brings about a 50% improvement in a system enhanced by query expansion.
- Y. Al-Onaizan, J. Curin, M. Jahr, K. Knight, J. Lafferty, D. Melamed, F. Och, D. Purdy, N. Smith, and D. Yarowsky, Statistical machine translation, Final Report, JHU Summer Workshop, 1999.Google Scholar
- G. Attardi, A. Cisternino, F. Formica, M. Simi and A. Tommasi, PiQASso: Pisa Question Answering System, Proc. of TREC-2001, 2001, pp. 599--607.Google Scholar
- A. Berger and J. Lafferty, Information retrieval as statistical translation, Proc. of SIGIR '99, 1999, pp. 222--229. Google ScholarDigital Library
- P. Brown, S. Della, V. Della Pietra and R. Mercer, The mathematics of statistical machine translation: Parameter estimation, Computational Linguistics, 19(2), 1993, pp. 263--311. Google ScholarDigital Library
- H. Cui, K. Li, R. Sun, T.-S. Chua and M.-Y. Kan, National University of Singapore at the TREC-13 Question Answering Main Task, Proc. of TREC-13, 2004.Google Scholar
- A. Echihabi and D. Marcu, A noisy-channel approach for question answering, Proc. of ACL '03, 2003. Google ScholarDigital Library
- J. Gao, J.-Y. Nie, G. Wu and G. Cao, Dependency language model for information retrieval, Proc. of SIGIR '04, Sheffield, UK, 2004, pp. 170--177. Google ScholarDigital Library
- S. Harabagiu, D. Moldovan, C. Clark, M. Bowden, J. Williams and J. Bensley, Answer Mining by Combining Extraction Techniques with Abductive Reasoning, Proc. of TREC-12, 2003, pp. 375--382.Google Scholar
- D. Hull, Using statistical testing in the evaluation of retrieval experiments, Proc. of SIGIR '93, 1993. Google ScholarDigital Library
- A. Ittycheriah, M. Franz, and S. Roukos, IBM's statistical question answering system - TREC-10, Proc. of TREC-10, 2001.Google Scholar
- M. Kaszkeil and J. Zobel, Passage retrieval revisited, Proc. of SIGIR '97, Philadelphia, PA, USA, 1997, pp. 178--185. Google ScholarDigital Library
- B. Katz and J. Lin, Selectively Using Relations to Improve Precision in Question Answering, Proc. of the EACL-2003 Workshop on Natural Language Processing for Question Answering, April 2003.Google Scholar
- G. G. Lee, J. Seo, S. Lee, H. Jung, B.-H. Cho, C. Lee, B.-K. Kwak, J. Cha, D. Kim, J. An, H. Kim, and K. Kim, SiteQ: Engineering high performance QA system using lexico-semantic pattern matching and shallow NLP, Proc. of TREC-10, 2001, pp. 442--451.Google Scholar
- M. Light, G. S. Mann, E. Riloff, and E. Breck, Analyses for elucidating current question answering technology, Journal of Natural Language Engineering, Special Issue on Question Answering, Fall--Winter, 2001. Google ScholarDigital Library
- D. Lin, Dependency-based Evaluation of MINIPAR, Proc. of Workshop on the Evaluation of Parsing Systems, Granada, Spain, May, 1998.Google Scholar
- J. Lin, D. Quan, V. Sinha, K. Bakshi, D. Huynh, B. Katz and D. R. Karger, What makes a good answer? The role of context in question answering, Proc. of the ninth IFIP TC13 International Conference on Human-Computer Interaction, 2003.Google Scholar
- D. Lin and P. Pantel, Discovery of Inference Rules for Question Answering, Natural Language Engineering, 2001, 7(4): pp. 343--360. Google ScholarDigital Library
- F. Song and B. Croft, A general language model for information retrieval, Proc. of CIKM'99, 1999, pp. 316--321. Google ScholarDigital Library
- S. Tellex, B. Katz, J. Lin, A.Fernandes and G.Marton, Quantitative evaluation of passage retrieval algorithms for question answering, Proc. of SIGIR '03, 2003, Toronto, Canada, pp. 41--47. Google ScholarDigital Library
- E.M. Voorhees, Overview of the TREC 2003 Question Answering Track, Proc. of TREC-12, pp. 54--68.Google Scholar
Index Terms
- Question answering passage retrieval using dependency relations
Recommendations
Quantitative evaluation of passage retrieval algorithms for question answering
SIGIR '03: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrievalPassage retrieval is an important component common to many question answering systems. Because most evaluations of question answering systems focus on end-to-end performance, comparison of common components becomes difficult. To address this shortcoming,...
Passage Reranking for Question Answering Using Syntactic Structures and Answer Types
ECIR 2011: Proceedings of the 33rd European Conference on Advances in Information Retrieval - Volume 6611Passage Retrieval is a crucial step in question answering systems, one that has been well researched in the past. Due to the vocabulary mismatch problem and independence assumption of bag-of-words retrieval models, correct passages are often ranked ...
Mining dependency relations for query expansion in passage retrieval
SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrievalClassical query expansion techniques such as the local context analysis (LCA) make use of term co-occurrence statistics to incorporate additional contextual terms for enhancing passage retrieval. However, relevant contextual terms do not always co-occur ...
Comments