Abstract
Google Scholar, the academic bibliographic database provided free-of-charge by the search engine giant Google, has been suggested as an alternative or complementary resource to the commercial citation databases like Web of Knowledge (ISI/Thomson) or Scopus (Elsevier). In order to check the usefulness of this database for bibliometric analysis, and especially research evaluation, a novel approach is introduced. Instead of names of authors or institutions, a webometric analysis of academic web domains is performed. The bibliographic records for 225 top level web domains (TLD), 19,240 university and 6,380 research centres institutional web domains have been collected from the Google Scholar database. About 63.8% of the records are hosted in generic domains like .com or .org, confirming that most of the Scholar data come from large commercial or non-profit sources. Considering only institutions with at least one record, one-third of the other items (10.6% from the global) are hosted by the 10,442 universities, while 3,901 research centres amount for an additional 7.9% from the total. The individual analysis show that universities from China, Brazil, Spain, Taiwan or Indonesia are far better ranked than expected. In some cases, large international or national databases, or repositories are responsible for the high numbers found. However, in many others, the local contents, including papers in low impact journals, popular scientific literature, and unpublished reports or teaching supporting materials are clearly overrepresented. Google Scholar lacks the quality control needed for its use as a bibliometric tool; the larger coverage it provides consists in some cases of items not comparable with those provided by other similar databases.
Similar content being viewed by others
References
Aguillo, I. (2009). Measuring the institution’s footprint in the web. Library Hi Tech, 27(4), 540–556.
Aguillo, I. (2011). Is Google Scholar useful for bibliometrics? A webometric analysis. In E. Noyons, P. Ngulube & J. Leta (Eds.), Proceedings of ISSI 2011—The 13th International Conference on Scientometrics and Informetrics (pp. 19–25), Durban, 4–7 July 2011.
Bar-Ilan, J. (2007). Which h-index? A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2), 257–271.
Bar-Ilan, J. (2009). A Closer Look at the Sources of Informetric Research. Cybermetrics, 13: Paper 4. http://www.cindoc.csic.es/cybermetrics/articles/v13i1p4.pdf.
Bar-Ilan, J. (2010). Citations to the “Introduction to informetrics” indexed by WOS, Scopus and Google Scholar. Scientometrics, 82(3), 495–506.
Beel, J. & Gipp, B. (2010). Academic search engine spam and Google Scholar’s resilience against it. Journal of Electronic Publishing, 13 (3). http://quod.lib.umich.edu/j/jep/3336451.0013.305?rgn=main;view=fulltext. doi:10.3998/3336451.0013.305.
Butler, D. (2011). Computing giants launch free science metrics. Nature, 476(7358), 18.
García-Pérez, M. A. (2010). Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google scholar: A case study for the computation of h indices in psychology. Journal of the American Society for Information Science and Technology, 61(10), 2070–2085.
Harzing, A., & van der Wal, R. (2008a). Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, 8(1), 61–73.
Harzing, A., & van der Wal, R. (2008b). A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science, 60(1), 41–46.
Jacsó, P. (2008). Google Scholar revisited. Online Information Review, 32(1), 102–114.
Jacsó, P. (2010). Savvy searching pragmatic issues in calculating and comparing the quantity and quality of research through rating and ranking of researchers based on peer reviews and bibliometric indicators from Web of Science, Scopus and Google Scholar. Online Information Review, 34(6), 972–982.
Kousha, K., & Thelwall, M. (2008). Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines. Scientometrics, 74(2), 273–294.
Li, J., Burnham, J. F., Lemley, T., & Britton, R. M. (2010). Citation analysis: Comparison of Web of Science, Scopus, Scifinder, and Google Scholar. Journal of Electronic Resources in Medical Libraries, 7(3), 196–217.
Mayr, P., & Walter, A.-K. (2007). An exploratory study of Google Scholar. Online Information Review, 31(6), 814–830.
Meho, L., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science vs Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58, 2105–2125.
Mikki, S. (2010). Comparing Google Scholar and ISI Web of Science for earth sciences. Scientometrics, 82(2), 321–331.
Torres-Salinas, D., Ruiz-Pérez, R., & Delgado-López-Cózar, E. (2008). Google Scholar como herramienta para la evaluación científica. El profesional de la información, 18(5), 501–510.
White, B. (2006). Examining the claims of Google Scholar as a serious Information Source. New Zealand Library & Information Management Journal, 50(1), 11–24.
Acknowledgments
An extended version of a paper presented at the 13th International Conference on Scientometrics and Informetrics, Durban (South Africa), 4–7 July 2011 (Aguillo 2011). This paper was funded by the EU project ACUMEN—Academic Careers Understood through Measurement and Norms (FP7-SCIENCE-IN-SOCIETY-2010-1. RI-266632).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Aguillo, I.F. Is Google Scholar useful for bibliometrics? A webometric analysis. Scientometrics 91, 343–351 (2012). https://doi.org/10.1007/s11192-011-0582-8
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-011-0582-8