Are Bibliographic Management Software Search Interfaces Reliable?: A Comparison between Search Results Obtained Using Database Interfaces and the EndNote Online Search Function

https://doi.org/10.1016/j.acalib.2010.01.005Get rights and content

Abstract

The use of bibliographic management software and its internal search interfaces is now pervasive among researchers. This study compares the results between searches conducted in academic databases' search interfaces versus the EndNote search interface. The results show mixed search reliability, depending on the database and type of search performed.

Introduction

The use of bibliographic management software packages, such as EndNote, RefWorks, BibTeX, and Zotero, is now firmly established among researchers and students as a time saving tool for writing academic papers. EndNote, in particular, is sold in most university bookstores in North America, as well as through independent distributors worldwide. As an illustrative sample, twelve of the top twenty-five universities in the United States1 and four of the top fifteen Canadian universities2 provide EndNote site licenses to their clients as a standard part of their services, and eight of these universities' library Web sites offer EndNote for purchase, while supporting other bibliographic management programs. This widespread access at major academic institutions and adoption by sophisticated users lends tacit approval to the effectiveness of the software as a bibliography management tool.

Citations management programs are continuously increasing in sophistication and have begun to add and promote new features like remote search capability. EndNote version X2 has recently emphasized its online search interface by moving it to the main page of the users' “library,” apparently to increase its importance among the features available to the user.

Some librarians are uncomfortable with unvalidated engines for searching databases such as that of EndNote, particularly when used by graduate level students and faculty, and tend to promote caution. Despite such warnings, the proximity and apparent ease-of-use of such search features are proving to be very provocative to both researchers and students. EndNote is increasingly being cited as the only search tool used for conducting literature searches including for systematic reviews.

The EndNote discussion forums3 provided by Thomson Reuters provide many examples of academic users attempting to employ the online search function of EndNote and encountering difficulty with their searches. It should be noted that this source is not representative in itself as a metric of EndNote's effectiveness, as few if any would post about successfully retrieving a desired result. Of those who experienced difficulty, an illustrative example is found in a message posted on September 23, 2008 with the subject line “Search failure: multi-word terms on PubMed.”4 The user, through EndNote search, received a message saying that no hits were retrieved for the two-word phrase “adaptation index” in the title field using the PubMed connection file, even though the PubMed search interface hosted by the National Library of Medicine retrieved several items for the same search string. In this case, EndNote does not support the exact phrase searching technique (i.e., using quotation marks) that the database itself employs.5

In another sample message,6 an EndNote user described difficulties searching for a compound Library of Congress subject heading in a remote search of the Library of Congress catalog. One knowledgeable forum participant responded with instructions for modifying the connection file to retrieve the desired results.

Other messages illustrate that users often do not understand the mechanics of Endnote's online search function. Some exhibit confusion when they are prompted for a password before connecting to an online search, not realizing that indirectly through EndNote they are attempting to search proprietary subscription-based databases.7 Other users understand this, but still experience problems connecting because their institutions' subscriptions to the databases are not configured to permit authentication through EndNote.8

Section snippets

Literature Review

EndNote is often praised for the convenience of its online search function.9 The common retrieval interface is attractive to researchers because it conflates the steps of searching for materials and documenting citations in a single program. Moreover, it ostensibly eliminates the need for searchers to learn to use disparate databases, as many different resources can be searched from the EndNote interface.10

Indeed, some discussions of bibliographic software promote using remote search functions

Purpose

The purpose of this study is to assess the accuracy, reliability, and efficiency of EndNote's in-software search interface versus directly accessing the search interfaces of databases themselves. A sample of core databases used at North American academic institutions was selected for comparison, to be searched both from within the database's own search interface and with the EndNote online search interface using the appropriate connection files (i.e., the customized files that map EndNote's

Methodology

The bibliographic management software EndNote version X2.0.1 for Windows was used for all searches in this study except for searches in the database Academic Search Premier. For this particular search, EndNote version X1 was used due to software availability issues.

Four test questions were created and used to test the interfaces of EndNote and each database:

  • 1.

    Does zinc help to prevent type 2 diabetes?

  • 2.

    What is the impact of low economic status on physical health in children?

  • 3.

    How are recent

Web of Science

All four test questions were used to assess the Web of Science (ISI) connection file provided by EndNote (updated to November 2008). Search terms were entered in the “title/keywords/abstract” field, using the AND operator from the drop-down list. In the Web of Science database, the “topic” field was searched, again with the AND operator from the drop-down list to combine terms. The two interfaces demonstrated the same phrase searching behavior; double quotation marks were required to retrieve

Discussion

Generally, the EndNote search interface did an excellent job searching databases that use keyword searching. There are, however, a variety of ways in which databases use the words “text search,” “keyword search,” “smart search,” “all fields,” etc., interchangeably. Understanding of which search term is being used by EndNote, and how that search term equates to the original database, will directly affect the results.

The database PsycINFO is a good example of this. In the Ovid interface for the

Assessment and Future Plans

The users' manual that is distributed with an EndNote X2.0.1 installation (a PDF file) provides instructions for conducting Z39.50 command-line syntax searches. This method allows users to “enter a Z39.50 search which EndNote passes directly to the [database's] server without any translation,”15 which obviously requires knowledge of the database's Z39.50 attributes. This method of searching is useful for librarians and advanced EndNote users, particularly creators of connection files, but for

Conclusion

The EndNote program, with its own search interface for querying bibliographic databases, has widespread use among librarians and researchers. But as this study illustrates, the results retrieved via EndNote versus direct searching of a database vary depending on both the database searched and the technique used to search it. Librarians and others who promote and provide technical support for citation management programs should be aware of the strengths and potential problems that users will

Notes and References (15)

  • “Best Colleges 2009: National Universities Rankings,” U.S. News & World Report. Available:...
  • Mary Dwyer

    Our 17th Annual Rankings

    Maclean's

    (2007)
  • Thomson Reuters Forums: EndNote. Available: http://forums.thomsonscientific.com/ (accessed August 18,...
  • Achristopherr, “Search Failure: Multi-Word Terms on PubMed.” Thomson Reuters Forums: EndNote: How To, posted September...
  • As of August 19, 2009, the PubMed database (http://www.ncbi.nlm.nih.gov/sites/entrez) retrieves eight items for the...
  • MrBill, “Searching Library of Congress with LC Subject Headings (LCSH).” Thomson Reuters Forums: EndNote: How To,...
  • E.g., blubb, “Help with EndNote and Online Search PsycInfo,” Thomson Reuters Forums: EndNote: How To, posted June 5,...
There are more references available in the full text version of this article.

Cited by (22)

  • Graduate Student Use and Non-use of Reference and PDF Management Software: An Exploratory Study

    2018, Journal of Academic Librarianship
    Citation Excerpt :

    Tramullas et al. (2015) noted that papers that included PDF management features in their comparisons started appearing around 2010. Several studies have also focused on quality of citations generated by reference management software (Brahmi & Gall, 2006; Fitzgibbons & Meert, 2010; Gilmour & Cobus-Kuo, 2011; Homol, 2014; Kratochvíl, 2017) but there are fewer studies on the use of reference management software by particular user groups. Some earlier studies that included use of reference management software (RMS) by graduate students were part of a larger study looking at information literacy skills or information seeking behaviors of students or faculty and asked whether or not the researchers used a reference manager.

  • Use of reference management software among postgraduate students in Greece

    2022, Journal of Librarianship and Information Science
View all citing articles on Scopus
View full text