Summary
The purpose of this paper is to describe various quality measures for search engines and to ask whether these are suitable. We especially focus on user needs and their use of Web search engines. The paper presents an extensive literature review and a first quality measurement model, as well. Findings include that Web search engine quality can not be measured by just retrieval effectiveness (the quality of the results), but should also consider index quality, the quality of the search features and Web search engine usability. For each of these sections, empirical results from studies conducted in the past, as well as from our own research are presented. These results have implications for the evaluation of Web search engines and for the development of better search systems that give the user the best possible search experience.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Acharya A, Cutts M, Dean J, Haahr P, Henzinger M, Hoelzle U, et al. (2005) Information retrieval based on historical data. USA
Beitzel S, Jensen C, Chowdhury A, Grossman D, Frieder O (2004) Hourly analysis of a very large topically categorized web query log (pp. 321–328). Paper presented at the ACM SIGIR Conference on Research and Development in Information Retrieval, Sheffield, UK. ACM Press
Bergman MK (2001) The deep web: surfacing hidden value. Journal of Electronic Publishing 7
Bharat K, Broder A (1998) A technique for measuring the relative size and overlap of public web search engines. Computer Networks and ISDN Systems 30: 379–388
Brin S, Page L (1998) The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems 30: 107–117
Broder A (2002) A taxonomy of web search. SIGIR Forum 36: 3–10
Broder A, Kumar R, Maghoul F, Raghavan P, Rajagopalan S, Stata R, et al. (2000) Graph structure in the web. Retrieved 15.4.2006, from http://www.almaden.ibm.com/webfountain/resources/GraphStructureintheWeb.pdf
Cacheda F, Viña Á (2001) Understanding how people use search engines: a statistical analysis for e-business. 1: 319–325. Paper presented at the e-2001 E-Business and E-Work Conference and Exhibition
Chignell MH, Gwizdka J, & Bodner RC (1999) Discriminating meta-search: a framework for evaluation. Information Processing and Management, 35(3), 337–362
Ding W, Marchionini G (1996) A comparative study of web search service performance. Proceedings of the 59th American Society for Information Science Annual Meeting (pp. 136–142): Learned Information
Fauldrath J, & Kunisch A (2005) Kooperative Evaluation der Usability von Suchmaschineninter-faces. Information: Wissenschaft und Praxis, 56(1), 21–28
Ford N, Miller D, Moss N (2002) Web search strategies and retrieval effectiveness: an empirical study. Journal of Documentation 58: 30–48
Geoghegan T (2004) Search wars: which is best? from news.bbc.co.uk/2/hi/uk_news/magazine/4003193.stm
Gordon M, & Pathak P (1999) Finding information on the World Wide Web: the retrieval effectiveness of search engines. Information Processing & Management, 35(2), 141–180
Greisdorf H, Spink A (2001) Median measure: an approach to IR systems evaluation. Information Processing and Management 37: 843–857
Griesbaum J (2004) Evaluation of three German search engines: Altavista.de, Google.de and Lycos.de. Information Research 9
Griesbaum J, Rittberger M, Bekavac B (2002) In: R. Hammwöhner, C. Wolff and C. Womser-Hacker (Eds.), Deutsche Suchmaschinen im Vergleich: AltaVista.de, Fireball.de, Google.de und Lycos.de (pp. 201–223). Paper presented at the Information und Mobilität. Optimierung und Vermeidung von Mobilität durch Information. 8. Internationales Symposium für Informationswissenschaft. UVK
Gulli A, Signorini A (2005) The indexable web is more than 11.5 billion pages (pp. 902–903). Paper presented at the Special Interest Tracks and Posters of the 14th International Conference on World Wide Web, Chiba, Japan
Hock R (2004) The latest field trip: An update on field searching in web search engines. Online (Wilton, Connecticut), 28(5), 15–21
Hoelscher C, Strube G (2000) Web search behavior of Internet experts and newbies (pp. 337–346). Paper presented at the 9th International World Wide Web Conference
Hotchkiss G, Garrison M, & Jensen S (2004) Search Engine Usage in North America, A Research Initiative by Enquiro. Retrieved 16.3.2006, from www.enquiro.com
Ingwersen P, Järvelin K (2005). The turn: integration of information seeking and retrieval in context. Springer, Dordrecht
Jansen BJ (2000) An investigation into the use of simple queries on web IR systems. Information Research 6
Jansen BJ, Spink A (2003) An analysis of web documents retrieved and viewed (pp. 64–69). Paper presented at the 4th International Conference on Internet Computing
Jansen BJ, Spink A (2006) How we are searching the world wide web? a comparison of nine search engine transaction logs. Information Processing and Management 42: 258–263
Ke Y, Deng L, Ng W, Lee DL (2006) Web dynamics and their ramifications for the development of web search engines. Computer Networks 50: 1430–1447
Kleinberg JM (1999) Authoritative sources in a hyperlinked environment. Journal of the ACM 46: 604–632
Korfhage RR (1997) Information storage and retrieval. Wiley, New York
Lawrence S, Giles CL (1998) Searching the world wide web. Science 280: 98–100
Lawrence S, Giles CL (1999) Accessibility of information on the web. Nature 400: 107–109
Leighton HV, Srivastava J (1999) First 20 precision among world wide web search services (search engines). Journal of the American Society for Information Science 50: 870–881
Lewandowski D (2004a) Abfragesprachen und erweiterte auchfunktionen von WWW-auchmaschinen. Information Wissenschaft und Praxis 55: 97–102
Lewandowski D (2004b) Bewertung von linktopologischen Verfahren als bestimmender Ranking-Faktor bei WWW-Suchmaschinen, Wissensorganisation und gesellschaftliche Verantwortung. 9. Tagung der Deutschen ISKO (Wissensorganisation 2004). Duisburg, Germany
Lewandowski D (2004c) Date-restricted queries in web search engines. Online Information Review 28: 420–427
Lewandowski D (2005a) Web searching, search engines and information retrieval. Information Services and Use 18: 137–147
Lewandowski D (2005b) Yahoo–Zweifel an den Angaben zur Indexgröße, Suche in mehreren Sprachen. Password 20: 21–22
Lewandowski D (2006a) Aktualität als erfolgskritischer Faktor bei Suchmaschinen. Information Wissenschaft und Praxis 57: 141–148
Lewandowski D (2006b) Suchmaschinen als Konkurrenten der Bibliothekskataloge: Wie Bibliotheken ihre Angebote durch Suchmaschinentechnologie attraktiver und durch Öffnung für die allgemeinen Suchmaschinen populärer machen können. Zeitschrift für Bibliothekswesen und Bibliographie 53: 71–78
Lewandowski D (2006c) Zur Bewertung der Qualität von Suchmaschinen. In: J. Eberspächer and S. Holtel (Eds.), Suchen und Finden im Internet (pp. 195–199). Heidelberg: Springer
Lewandowski D, Mayr P (2006) Exploring the academic invisible web. Library Hi Tech 24: 529–539
Lewandowski D, Wahlig H, Meyer-Bautor G (2006) The freshness of web search engine databases. Journal of Information Science 32: 133–150
MacCall SL, Cleveland AD (1999) A relevance-based quantitative measure for Internet information retrieval evaluation (pp. 763–768). Paper presented at the Proceedings of the American Society for Information Science Annual Meeting
Machill M, Neuberger C, Schweiger W, Wirth W (2003) Wegweiser im netz: qualität und nutzung von suchmaschinen. In: M. Machill and C. Welp (Eds.), Wegweiser im Netz. Gütersloh: Bertelsmann Stiftung
Machill M, Neuberger C, Schweiger W, Wirth W (2004) Navigating the Internet: a study of German-language search engines. European Journal of Communication 19: 321–347
Nicholson S (2000) Raising reliability of Web search tool research through replication and chaos theory. Journal of the American Society for Information Science, 51(8), 724–729
Notess GR (2006, 22.5.2006) Search Engine Features Chart. Retrieved 16.3.2006, from http://www.searchengineshowdown.com/features/
Notess GR (2003) Search engine statistics: freshness showdown. Retrieved 4.1.2005, from http://www.searchengineshowdown.com/stats/freshness.shtml
Ntoulas A, Cho J, Olston C (2004) What’s new on the web? the evolution of the web from a search engine perspective. Paper presented at the Thirteenth WWW Conference, New York, USA
Ojala M (2002) Web search engines: Search syntax and features. Online (Wilton, Connecticut), 26(5), 28
Ozmutlu H, Spink A, Ozmutlu S (2003) A study of multitasking web search (pp. 145–148). Paper presented at the International Conference on Information Technology: Computers and Communications
Page L, Brin S, Motwani R, Winograd T (1998) The pagerank citation ranking : bringing order to the web. Retrieved 24.7.2006, from http://dbpubs.stanford.edu:8090/pub/1999–66
Parasuraman A, Zeithaml VA, Berry LL (1988). SERVQUAL: a multiple-item scale for measuring consumer perceptions of service quality. Journal of Retailing 64: 12–40
Risvik KM, Michelsen R (2002) Search engines and web dynamics. Computer Networks 39: 289–302
Saracevic T (1995) Evaluation of evaluation in information retrieval (pp. 138–146). Paper presented at the SIGIRS95, Seattle, CA. ACM Press
Schmidt-Maenz (2007) Untersuchung des suchverhaltens im web–Interaktion von Internetnutzern mit Suchmaschinen, Dr. Kovac Verlag, Hamburg
Schmidt-Maenz N, Bomhardt C (2005) Wie suchen onliner im Internet? Science Factory/Absatzwirtschaftb 2: 5–8
Schmidt-Maenz N, Gaul W (2005). Web mining and online visibility. In: C. Weihs and W. Gaul (Eds.), Classification–the ubiquitous challenge (pp. 418–425): Springer
Schmidt-Maenz N, & Koch M (2005) Patterns in Search Queries. In D. Baier, R. Decker & L. Schmidt-Thieme (Eds.), Data Analysis and Decision Support (pp. 122–129). Heidelberg: Springer
Schmidt-Maenz N, Koch M (2006) A general classification of (search) queries and terms (pp. 375–381). Paper presented at the 3rd International Conference on Information Technologies: Next Generations, Las Vegas, Nevada, USA
Schwartz C (1998) Web search engines. Journal of the American Society for Information Science and Technology, 49(11), 973–982
Sherman C, Price G (2001) The invisible web: uncovering information sources search engines can’t see. Information Today, Medford, NJ
Silverstein C, Henzinger M, Marais H, Moricz M (1999) Analysis of a very large web search engine query log. ACM SIGIR Forum 33: 6–12
Singhal A, Kaszkiel M (2001) A case study in web search using TREC algorithms (pp. 708–716). Paper presented at the 10th International Conference on World Wide Web, Hong Kong
Spink A, Jansen BJ (2004) Web search: public searching of the web (Vol. 6). Dordrecht, Boston, London: Kluwer Academic
Spink A, Jansen B, Ozmutlu H (2000) Use of query reformulation and relevance feedback by excite users. Internet Research: Electronic Networking Applications and Policy 19: 317–328
Spink A, Ozmutlu S, Ozmutlu H, Jansen B (2002) U.S. versus european web searching processes. Journal of the American Society for Information Science and Technology 53: 639–652
Spink A, Wolfram D, Jansen B, Saracevic T (2001) Searching the web: the public and their queries. Journal of the American Society for Information Science and Technology 52: 226–234
Spink A, Jansen BJ, Blakely C, & Koshman S. (2006). A study of results overlap and uniqueness among major Web search engines. Information Processing & Management, 42(5), 1379–1391
Su LT (1998) Value of search results as a whole as the best single measure of information retrieval performance. Information Processing and Management 34: 557–579
Sullivan D (2005) Search engine sizes. Retrieved 24.7.2006, from http://searchenginewatch.com/showPage.html?page=2156481x
Vaughan L (2004) New measurements for search engine evaluation: proposed and tested. Information Processing and Management 40: 677–691
Vaughan L, Thelwall M (2004) Search engine coverage bias: evidence and possible causes. Information Processing and Management 40: 693–707
Véronis J (2006) A comparative study of six search engines. Retrieved 15.3.2006, from http://www.up.univ-mrs.fr/veronis/pdf/2006-comparative-study.pdf
Wang H, Xie M, Goh TN (1999) Service quality of Internet search engines. Journal of Information Science 25: 499–507
Williams ME (2005) The state of databases today: 2005. In: Gale Directory of Databases (Vol. 2, pp. XV-XXV). Detroit, Mich.: Gale Group
Wolff, C. (2000) Vergleichende evaluierung von such- und metasuchmaschinen, 7. Internationales Symposium für Informationswissenschaft (ISI 2000) (pp. 31–38). Universitätsverlag Konstanz, Darmstadt, Germany
Xie M, Wang H, Goh TN (1998) Quality dimensions of Internet search engines. Journal of Information Science 24: 365–372
Zien J, Meyer J, Tomlin J, Liu J (2000) Web query characteristics and their implications on search engines: Almaden Research Center
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Lewandowski, D., Höchstötter, N. (2008). Web Searching: A Quality Measurement Perspective. In: Spink, A., Zimmer, M. (eds) Web Search. Information Science and Knowledge Management, vol 14. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75829-7_16
Download citation
DOI: https://doi.org/10.1007/978-3-540-75829-7_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75828-0
Online ISBN: 978-3-540-75829-7
eBook Packages: Computer ScienceComputer Science (R0)