Skip to main content
SpringerLink
Log in

An efficient concept-based retrieval model for enhancing text retrieval quality

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Most of the common techniques in text retrieval are based on the statistical analysis terms (words or phrases). Statistical analysis of term frequency captures the importance of the term within a document only. Thus, to achieve a more accurate analysis, the underlying model should indicate terms that capture the semantics of text. In this case, the model can capture terms that represent the concepts of the sentence, which leads to discovering the topic of the document. In this paper, a new concept-based retrieval model is introduced. The proposed concept-based retrieval model consists of conceptual ontological graph (COG) representation and concept-based weighting scheme. The COG representation captures the semantic structure of each term within a sentence. Then, all the terms are placed in the COG representation according to their contribution to the meaning of the sentence. The concept-based weighting analyzes terms at the sentence and document levels. This is different from the classical approach of analyzing terms at the document level only. The weighted terms are then ranked, and the top concepts are used to build a concept-based document index for text retrieval. The concept-based retrieval model can effectively discriminate between unimportant terms with respect to sentence semantics and terms which represent the concepts that capture the sentence meaning. Experiments using the proposed concept-based retrieval model on different data sets in text retrieval are conducted. The experiments provide comparison between traditional approaches and the concept-based retrieval model obtained by the combined approach of the conceptual ontological graph and the concept-based weighting scheme. The evaluation of results is performed using three quality measures, the preference measure (bpref), precision at 10 documents retrieved (P(10)) and the mean uninterpolated average precision (MAP). All of these quality measures are improved when the newly developed concept-based retrieval model is used, confirming that such model enhances the quality of text retrieval.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aas K, Eikvil L (1999) Text categorisation: a survey. Technical report 941, Norwegian Computing Center

  2. Amati G, van Rijsbergen CJ (2002) Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans Inf Syst 20(4): 357–389

    Article  Google Scholar 

  3. Apache jakarta lucene search engine (version 1.3). http://lucene.apache.org/

  4. Apostolakis J (2009) An introduction to data mining. In: Data mining in crystallography. Springer, pp 1–35

  5. Baccini A, Déjean S, Lafage L, Mothe J (2011) How many performance measures to evaluate information retrieval systems? Knowl Inf Syst 1–21

  6. Belkin N, Croft W (1987) Retrieval techniques. Annu Rev Inf Sci Technol 22: 109–145

    Google Scholar 

  7. Buckley C, Voorhees EM (2004) Retrieval evaluation with incomplete information. In: SIGIR’04: proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval. ACM Press, New York, pp 25–32

  8. Cole R (1998) Survey of the state of the art in human language technology (studies in natural language processing). Cambridge University Press, New York

    Google Scholar 

  9. Collins M (1999) Head-driven statistical model for natural language parsing. PhD thesis, University of Pennsylvania

  10. Cucerzan S (2010) A case study of using web search statistics: case restoration. In: Computational linguistics and intelligent text processing. Springer, pp 199–211

  11. Fagan J (1989) The effectiveness of a nonsyntactic approach to automatic phrase indexing for document retrieval. J Am Soc Inf Sci 40(2): 115–132

    Article  Google Scholar 

  12. Fillmore C (1968) The case for case In Universals in linguistic theory. Holt, Rinehart and Winston Inc., , New York

    Google Scholar 

  13. Géry M, Jurafsky D (2002) Automatic labeling of semantic roles. Comput Linguist 28(3): 245–288

    Article  Google Scholar 

  14. Gildea D, Largeron C (2011) Bm25t: a bm25 extension for focused information retrieval. Knowl Inf Syst 1–25

  15. Hull D (1993) Using statistical testing in the evaluation of retrieval experiments. In: Proceedings of special interest group on information retrieval (ACM SIGIR)

  16. Jurafsky D, Martin JH (2000) Speech and language processing. Prentice Hall Inc., Upper Saddle River

    Google Scholar 

  17. Kalogeratos A, Likas A (2011) Text document clustering using global term context vectors. Knowl Inf Syst 1–20

  18. Kingsbury P, Palmer M (2003) Propbank: the next level of treebank. In: Proceedings of treebanks and lexical theories

  19. Liu X, Webster J, Kit C (2009) An extractive text summarizer based on significant words. In: Computer processing of oriental languages. Language Technology for the Knowledge-based Economy. Springer, pp 168–178

  20. Ounis I, Amati G, Plachouras V, He B, Macdonald C, Lioma C (2006) Terrier: a high performance and scalable information retrieval platform. In: Proceedings of ACM SIGIR’06 workshop on open source information retrieval (OSIR 2006)

  21. Park J, Lee S (2011) Keyword search in relational databases. Knowl Inf Syst 26(2): 175–193

    Article  Google Scholar 

  22. Porter MF (1980) An algorithm for suffix stripping. Program 14(3): 130–137

    Article  Google Scholar 

  23. Pradhan S, Hacioglu K, Ward W, Martin JH, Jurafsky D (2003) Semantic role parsing: Adding semantic structure to unstructured text. In: Proceedings of the 3th IEEE international conference on data mining (ICDM), pp 629–632

  24. Pradhan S, Ward W, Hacioglu K, Martin J, Jurafsky D (2004) Shallow semantic parsing using support vector machines. In: Proceedings of the Human Language Technology/North American Association for Computational Linguistics (HLT/NAACL)

  25. Pradhan S, Hacioglu K, Krugler V, Ward W, Martin JH, Jurafsky D (2005) Support vector learning for semantic argument classification. Mach Learn 60(1–3): 11–39

    Article  Google Scholar 

  26. Salton G, McGill MJ (1983) Introduction to modern information retrieval. McGraw-Hill, New York

    MATH  Google Scholar 

  27. Salton G, Wong A, Yang CS (1975) A vector space model for automatic indexing. Commun ACM 18(11): 112–117

    Article  Google Scholar 

  28. Shehata S, Karray F, Kamel M (2006) Enhancing text retrieval performance using conceptual ontological graph. In: ICDM workshops, pp 39–44

  29. Shehata S, Karray F, Kamel M (2007) Enhancing search engine quality using concept-based text retrieval. In: International conference on web intelligence (WI), USA

  30. Shehata S, Karray F, Kamel M (2007) A concept-based model for enhancing text categorization. In: Knowledge discovery and data mining (KDD), USA

  31. Tombros A, Rijsbergen CJ (2004) Query-sensitive similarity measures for information retrieval. Knowl Inf Syst 617–642

Download references

Author information

Authors and Affiliations

  1. Desire2Learn Incorporated, Kitchener, Ontario, Canada

    Shady Shehata

  2. Pattern Analysis and Machine Intelligence (PAMI) Research Group, University of Waterloo, Waterloo, ON, N2L 3G1, Canada

    Fakhri Karray & Mohamed S. Kamel

Authors
  1. Shady Shehata
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fakhri Karray
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed S. Kamel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shady Shehata.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shehata, S., Karray, F. & Kamel, M.S. An efficient concept-based retrieval model for enhancing text retrieval quality. Knowl Inf Syst 35, 411–434 (2013). https://doi.org/10.1007/s10115-012-0504-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-012-0504-y

Keywords

Search

Navigation