skip to main content
10.1145/989863.989880acmconferencesArticle/Chapter ViewAbstractPublication PagesaviConference Proceedingsconference-collections
Article

The challenge of information visualization evaluation

Published:25 May 2004Publication History

ABSTRACT

As the field of information visualization matures, the tools and ideas described in our research publications are reaching users. The reports of usability studies and controlled experiments are helpful to understand the potential and limitations of our tools, but we need to consider other evaluation approaches that take into account the long exploratory nature of users tasks, the value of potential discoveries or the benefits of overall awareness. We need better metrics and benchmark repositories to compare tools, and we should also seek reports of successful adoption and demonstrated utility.

References

  1. Borzo, Janet, Get the picture, Wall Street Journal, January 12 (2004).Google ScholarGoogle Scholar
  2. National Cancer Institute Cancer Mortality Maps and Graphs www3.cancer.gov/atlasplus.Google ScholarGoogle Scholar
  3. Sytadin. www.sytadin.tm.fr.Google ScholarGoogle Scholar
  4. Komlodi, A., Sears, A., Stanziola, E., Information Visualization Evaluation Review, ISRC Tech. Report, Dept. of Information Systems, UMBC. UMBC-ISRC-2004-1 http://www.research.umbc.edu/~komlodi/IV_eval (2004).Google ScholarGoogle Scholar
  5. Ahlberg, C., Shneiderman, B. The alphaslider: a compact and rapid selector, Proc. of the SIGCHI conference on Human factors in computing systems (1994) 365--371. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Irani, P., Ware, C., Diagramming information structures using 3D perceptual primitives, ACM Transactions on Computer-Human Interaction, 10, 1 (2003), 1--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Sutcliffe, A. G., Ennis, M., Hu, J., Evaluating the effectiveness of visual user interfaces for information retrieval. International Journal of Human Computer Studies, 53, 5 (2000), 741--763. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Byrd, D. (1999). A scrollbar-based visualization for document navigation. Proc. of the fourth ACM Conference on Digital Libraries (1999) 122--129. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Plaisant, C., Grosjean, J., and Bederson, B. B. SpaceTree: Supporting exploration in large node-link tree: design evolution and empirical evaluation IEEE Symposium on Information Visualization (2002), 57--64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Trafton, J., Tsui, T., Miyamoto, R.; Ballas, J., Raymond, P., Turning pictures into numbers: extracting and generating information from complex visualizations. International Journal of Human Computer Studies, 53, 5 (2000), 827--850. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Moore, G., Crossing the Chasm, HarpersCollins Publishers, New York, NY (1991).Google ScholarGoogle Scholar
  12. Moore, G., Inside the Tornado, HarpersCollins Publishers, New York, NY (1995).Google ScholarGoogle Scholar
  13. Chen, C., Czerwinski, M. (Eds.) Introduction to the Special Issue on Empirical evaluation of information visualizations, International Journal of Human-Computer Studies, 53, 5, (2000), 631--635. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Plaisant, C., Information visualization and the challenge of universal access, in Exploring Geovisualization, Dykes, J., MacEachren, A. and Kraak, M.-J. (Eds.), Elsevier, Oxford (2004) in press.Google ScholarGoogle Scholar
  15. Zhao, H., Shneiderman, B., Image-based highly interactive web mapping for geo-referenced data publishing, Tech Report CS-TR-4431. (2003) www.cs.umd.edu/hcil.Google ScholarGoogle Scholar
  16. Brewster, S, A., Non-speech auditory output. In Jacko, J. and Sears, A. (Eds.), Human Computer Interaction Handbook, Lawrence Erlbaum Associates, Mahwah, NJ (2002), 220--239. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Zhao, H., Plaisant, C., Shneiderman, B., Duraiswani, R., Sonification of geo-referenced data for auditory information seeking: Design principle and pilot study, University of Maryland Tech. Report HCIL 2004-03 (2003) available at www.cs.umd.edu/hcil.Google ScholarGoogle Scholar
  18. Brewer, Cynthia, ColorBrewer, www.personal.psu.edu/faculty/c/a/cab38/ColorBrewerBeta.html.Google ScholarGoogle Scholar
  19. Dougherty, R., Wade, A. VisCheck: www.vischeck.comGoogle ScholarGoogle Scholar
  20. Shneiderman, B., Promoting universal usability with multilayer interface design, Proc. of ACM Conf. on Universal Usability (2003) 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Fekete, J-D and Plaisant, C., InfoVis 2003 Contest, www.cs.umd.edu/hcil/iv03contest (2003)Google ScholarGoogle Scholar
  22. Fekete, J.-D., Greinstein, G. and Plaisant, C., InfoVis 2004 Contest, www.cs.umd.edu/hcil/iv04contestGoogle ScholarGoogle Scholar
  23. Fekete, J-D and Plaisant, C., Information Visualization Benchmark Repository www.cs.umd.edu/hcil/InfovisRepositoryGoogle ScholarGoogle Scholar
  24. Munzner, T., Guimbretière, F., Tasiran, S., Zhang, L. and Zhou, Y., TreeJuxtaposer: Scalable tree comparison using Focus+Context with guaranteed visibility. ACM Transactions on Graphics, SIGGRAPH 03 (2003) 453--462. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Hong, J. Y., D'Andries, J., Richman, M., Westfall, M., Zoomology: Comparing Two Large Hierarchical Trees, in Poster Compendium of IEEE Information Visualization (2003) (also available at www.cs.umd.edu/hcil/iv03contest)Google ScholarGoogle Scholar
  26. Spenke, M., Beilken, C., InfoZoom - Analysing Formula One racing results with an interactive data mining and visualization tool, Ebecken, N. Data mining II, (2000), 455--464.Google ScholarGoogle Scholar
  27. Auber, D., Delest, M., Domenger, J-P., Ferraro, P., Strandh, R., EVAT - Environment for Visualization and Analysis of Trees, in Poster Compendium of IEEE Information Visualization (2003) (also available at www.cs.umd.edu/hcil/iv03contest).Google ScholarGoogle Scholar
  28. Morse, D. R., Ytow, N., Roberts, D. McL., Sato, A., Comparison of Multiple Taxonomic Hierarchies Using TaxoNote, in Poster Compendium of IEEE Information Visualization (2003) (also available at www.cs.umd.edu/hcil/iv03contest).Google ScholarGoogle Scholar
  29. Sheth, N., Börner, K., Baumgartner, J., Mane, K., Wernert, E., Treemap, Radial Tree, and 3D Tree Visualizations, in Poster Compendium of IEEE Information Visualization (2003) (also available at www.cs.umd.edu/hcil/iv03contest)Google ScholarGoogle Scholar
  30. Alexander, Dey, Return on investment resources, http://deyalexander.com/resources/roi.html.Google ScholarGoogle Scholar
  31. Dürsteler, Juan Carlos, www.infovis.net.Google ScholarGoogle Scholar
  32. Ahlberg, Christopher and Shneiderman, Ben, Visual information seeking: Tight coupling of dynamic query filters with starfield displays, Proc. CHI '94 Conference: Human Factors in Computing Systems, ACM, New York (1994), 313--321 and color plates. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Shneiderman, B. Dynamic queries, starfield displays, and the path to Spotfire, www.cs.umd.edu/hcil/spotfire.Google ScholarGoogle Scholar
  34. Shneiderman, B., Treemaps for space-constrained visualization of hierarchies, www.cs.umd.edu/hcil/treemapsGoogle ScholarGoogle Scholar
  35. Shneiderman, Ben, Tree visualization with treemaps: a 2-d space-filling approach, ACM Transactions on Graphics, 11, 1 (1992) 92--99. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Bederson, B. B., Shneiderman, B., and Wattenberg, M. Ordered and Quantum Treemaps: Making Effective Use of 2D Space to Display Hierarchies, ACM Transactions on Graphics, 21, 4 (2002) 833--854. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Plaisant, C., Facilitating Data Exploration: Dynamic Queries on a Health Statistics Map. Proc. of the Annual Meeting of the American Statistical Association (Government Statistics Section), ASA, Alexandria, VA (1993) 18--23.Google ScholarGoogle Scholar
  38. Li, Q., North, C., Empirical comparison of dynamic queries sliders and brushing histograms, Proc. IEEE Information Visualization (2003), 147--15. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. The challenge of information visualization evaluation

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      AVI '04: Proceedings of the working conference on Advanced visual interfaces
      May 2004
      425 pages
      ISBN:1581138679
      DOI:10.1145/989863

      Copyright © 2004 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 25 May 2004

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • Article

      Acceptance Rates

      Overall Acceptance Rate107of408submissions,26%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader