David Ledo
Steven Houben
Jo Vermeulen
Nicolai Marquardt
Saul Greenberg
ABSTRACT
Toolkit research plays an important role in the field of HCI, as it can heavily influence both the design and implementation of interactive systems. For publication, the HCI community typically expects toolkit research to include an evaluation component. The problem is that toolkit evaluation is challenging, as it is often unclear what 'evaluating' a toolkit means and what methods are appropriate. To address this problem, we analyzed 68 published toolkit papers. From our analysis, we provide an overview of, reflection on, and discussion of evaluation methods for toolkit contributions. We identify and discuss the value of four toolkit evaluation strategies, including the associated techniques that each employs. We offer a categorization of evaluation strategies for toolkit researchers, along with a discussion of the value, potential limitations, and trade-offs associated with each strategy.
Supplemental Material
- Georg Apitz and François Guimbretière. 2004. CrossY: a crossing-based drawing application. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST'04). ACM, New York, NY, USA, 3--12. Google Scholar
Digital Library
- Caroline Appert and Michel Beaudouin-Lafon. 2006. SwingStates: adding state machines to the swing toolkit. In Proceedings of the 19th annual ACM symposium on User interface software and technology (UIST'06). ACM, New York, NY, USA, 319--322. Google ScholarDigital Library
- Sriram Karthik Badam and Niklas Elmqvist. 2014. PolyChrome: A Cross-Device Framework for Collaborative Web Visualization. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (ITS'14). ACM, New York, NY, USA, 109--118. Google ScholarDigital Library
- Rafael Ballagas, Meredith Ringel, Maureen Stone, and Jan Borchers. 2003. iStuff: a physical user interface toolkit for ubiquitous computing environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'03). ACM, New York, NY, USA, 537--544. Google ScholarDigital Library
- Till Ballendat, Nicolai Marquardt, and Saul Greenberg. 2010. Proxemic interaction: designing for a proximity and orientation-aware environment. In ACM International Conference on Interactive Tabletops and Surfaces (ITS'10). ACM, New York, NY, USA, 121--130. Google ScholarDigital Library
- Ben Bederson, Jesse Grosjean and Jon Meyer. 2004. Toolkit design for interactive structured graphics. IEEE Transactions on Software Engineering, 30(8). IEEE 535--546. Google ScholarDigital Library
- Victoria Bellotti and Keith Edwards. 2001. Intelligibility and accountability: human considerations in context-aware systems. Hum.-Comput. Interact. 16, 2 (December 2001), 193--212. Google ScholarDigital Library
- Michael S. Bernstein, Mark S. Ackerman, Ed H. Chi, and Robert C. Miller. 2011. The trouble with social computing systems research. In CHI'11 Extended Abstracts on Human Factors in Computing Systems (CHI EA'11). ACM, New York, NY, USA, 389--398. Google ScholarDigital Library
- Stephen M. Blackburn, Robin Garner, Chris Hoffmann, Asjad M. Khang, Kathryn S. McKinley, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z. Guyer, Martin Hirzel, Antony Hosking, Maria Jump, Han Lee, J. Eliot B. Moss, Aashish Phansalkar, Darko Stefanovi?, Thomas VanDrunen, Daniel von Dincklage, and Ben Wiedermann. 2006. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications (OOPSLA'06). ACM, New York, NY, USA, 169--190. Google ScholarDigital Library
- Alan Blackwell, Carol Britton, Blackwell, A. Cox, Thomas Green, Corin Gurr, Gada Kadoda, M.S. Kutar et al. 2001. Cognitive dimensions of notations: Design tools for cognitive technology. In Cognitive technology: instruments of mind Springer, Berlin, Heidelberg. 325--341. Google ScholarDigital Library
- Alan Blackwell and Thomas Green. 2000. A Cognitive Dimensions questionnaire optimised for users. In proceedings of the twelfth annual meeting of the psychology of programming interest group. 137--152.Google Scholar
- Florian Block, Michael Haller, Hans Gellersen, Carl Gutwin, and Mark Billinghurst. 2008. VoodooSketch: extending interactive surfaces with adaptable interface palettes. In Proceedings of the 2nd international conference on Tangible and embedded interaction (TEI'08). ACM, New York, NY, USA, 55--58. Google ScholarDigital Library
- Michael Bostock and Jeffrey Heer. 2009. Protovis: A Graphical Toolkit for Visualization. In IEEE Transactions on Visualization and Computer Graphics, vol. 15, no. 6. IEEE. 1121--1128. Google ScholarDigital Library
- Michael Bostock, Vadim Ogievetsky and Jeffrey Heer. 2011. D³ Data-Driven Documents. In IEEE Transactions on Visualization and Computer Graphics, vol. 17, no. 12. IEEE. 2301--2309. Google ScholarDigital Library
- Jorge Cardoso and Rui José. 2012. PuReWidgets: a programming toolkit for interactive public display applications. In Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems (EICS'12). ACM, New York, NY, USA, 51--60. Google ScholarDigital Library
- Kathy Charmaz. 2014. Constructing grounded theory. Sage.Google Scholar
- Pei-Yu (Peggy) Chi and Yang Li. 2015. Weave: Scripting Cross-Device Wearable Interaction. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI'15). ACM, New York, NY, USA, 3923--3932. Google ScholarDigital Library
- Alan Cooper. 2004. The inmates are running the asylum: Why high-tech products drive us crazy and how to restore the sanity. Sams Indianapolis. Google ScholarDigital Library
- Anind K. Dey and Alan Newberger. 2009. Support for context-aware intelligibility and control. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'09). ACM, New York, NY, USA, 859--868. Google ScholarDigital Library
- Morgan Dixon and James Fogarty. 2010. Prefab: implementing advanced behaviors using pixel-based reverse engineering of interface structure. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'10). ACM, New York, NY, USA, 1525--1534. Google ScholarDigital Library
- Morgan Dixon, Alexander Nied, and James Fogarty. 2014. Prefab layers and prefab annotations: extensible pixel-based interpretation of graphical interfaces. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST'14). ACM, New York, NY, USA, 221--230. Google ScholarDigital Library
- Pierre Dragicevic and Jean-Daniel Fekete. 2004. Support for input adaptability in the ICON toolkit. In Proceedings of the 6th international conference on Multimodal interfaces (ICMI'04). ACM, New York, NY, USA, 212--219. Google ScholarDigital Library
- Brian de Alwis, Carl Gutwin, and Saul Greenberg. 2009. GT/SD: performance and simplicity in a groupware toolkit. In Proceedings of the 1st ACM SIGCHI symposium on Engineering interactive computing systems (EICS'09). ACM, New York, NY, USA, 265274. Google ScholarDigital Library
- Joseph Dumas and Janice Redish. 1999. A practical guide to usability testing. Intellect books. Google ScholarDigital Library
- W. Keith Edwards, Mark W. Newman, and Erika Shehan Poole. 2010. The infrastructure problem in HCI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'10). ACM, New York, NY, USA, 423--432. Google ScholarDigital Library
- Douglas C Engelbart (1968). The mother of all demos. Presented at ACM/IEEE - Computer Society's Fall Joint Computer Conference. San FranciscoGoogle Scholar
- James Fogarty. Code and Contribution in Interactive Systems Research. In workshop on HCI.Tools at CHI'2017.Google Scholar
- Aaron M. Genest, Carl Gutwin, Anthony Tang, Michael Kalyn, and Zenja Ivkovic. 2013. KinectArms: a toolkit for capturing and displaying arm embodiments in distributed tabletop groupware. In Proceedings of the 2013 conference on Computer supported cooperative work (CSCW'13). ACM, New York, NY, USA, 157--166. Google ScholarDigital Library
- Saul Greenberg. 2001. Context as a dynamic construct. Hum.-Comput. Interact. 16, 2 (December 2001), 257--268. Google ScholarDigital Library
- Saul Greenberg. 2007. Toolkits and interface creativity. Multimedia Tools and Applications, 32(2), Springer, 139--159. Google ScholarDigital Library
- Saul Greenberg and Bill Buxton. 2008. Usability evaluation considered harmful (some of the time). In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'08). ACM, New York, NY, USA, 111--120. Google ScholarDigital Library
- Saul Greenberg and Chester Fitchett. 2001. Phidgets: easy development of physical interfaces through physical widgets. In Proceedings of the 14th annual ACM symposium on User interface software and technology (UIST'01). ACM, New York, NY, USA, 209--218. Google ScholarDigital Library
- Tobias Grosse-Puppendahl, Yannick Berghoefer, Andreas Braun, Raphael Wimmer and Arjan Kuijper. 2013. OpenCapSense: A rapid prototyping toolkit for pervasive interaction using capacitive sensing. In Proc. IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE. San Diego, CA, USA, 152--159.Google ScholarCross Ref
- Tovi Grossman and Ravin Balakrishnan. 2005. The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'05). ACM, New York, NY, USA, 281--290. Google ScholarDigital Library
- Thomas E. Hansen, Juan Pablo Hourcade, Mathieu Virbel, Sharath Patali, and Tiago Serra. 2009. PyMT: a post-WIMP multi-touch user interface toolkit. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS'09). ACM, New York, NY, USA, 17--24. Google ScholarDigital Library
- Björn Hartmann, Leith Abdulla, Manas Mittal, and Scott R. Klemmer. 2007. Authoring sensor-based interactions by demonstration with direct manipulation and pattern recognition. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'07). ACM, New York, NY, USA, 145--154. Google ScholarDigital Library
- Björn Hartmann, Scott R. Klemmer, Michael Bernstein, Leith Abdulla, Brandon Burr, Avi RobinsonMosher, and Jennifer Gee. 2006. Reflective physical prototyping through integrated design, test, and analysis. In Proceedings of the 19th annual ACM symposium on User interface software and technology (UIST'06). ACM, New York, NY, USA, 299--308. Google ScholarDigital Library
- Jeffrey Heer, Stuart K. Card, and James A. Landay. 2005. prefuse: a toolkit for interactive information visualization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'05). ACM, New York, NY, USA, 421--430. Google ScholarDigital Library
- Alan R. Hevner, Salvatore T. March, Jinsoo Park, and Sudha Ram. 2004. Design science in information systems research. MIS Q. 28, 1 (March 2004), 75--105. Google ScholarDigital Library
- Jason Hill and Carl Gutwin. 2004. The MAUI toolkit: Groupware widgets for group awareness. Computing Supported Cooperative Work, 13, Springer 539--571. Google ScholarDigital Library
- Jason I. Hong and James A. Landay. 2000. SATIN: a toolkit for informal ink-based applications. In Proceedings of the 13th annual ACM symposium on User interface software and technology (UIST'00). ACM, New York, NY, USA, 63--72. Google ScholarDigital Library
- Steven Houben, Connie Golsteijn, Sarah Gallacher, Rose Johnson, Saskia Bakker, Nicolai Marquardt, Licia Capra, and Yvonne Rogers. 2016. Physikit: Data Engagement Through Physical Ambient Visualizations in the Home. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI'16). ACM, New York, NY, USA, 1608--1619. Google ScholarDigital Library
- Steven Houben and Nicolai Marquardt. 2015. WatchConnect: A Toolkit for Prototyping Smartwatch-Centric Cross-Device Applications. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI'15). ACM, New York, NY, USA, 1247--1256. Google ScholarDigital Library
- Steven Houben, Nicolai Marquardt, Jo Vermeulen, Clemens Klokmose, Johannes Schöning, Harald Reiterer, and Christian Holz. 2017. Opportunities and challenges for cross-device interactions in the wild. interactions 24, 5 (August 2017), 58--63. Google ScholarDigital Library
- Scott E. Hudson, and Jennifer Mankoff. 2014. Concepts, Values, and Methods for Technical Human-- Computer Interaction Research. In Ways of Knowing in HCI, Springer New York, NY, USA, 69--93.Google Scholar
- Scott E. Hudson, Jennifer Mankoff, and Ian Smith. 2005. Extensible input handling in the subArctic toolkit. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'05). ACM, New York, NY, USA, 381--390. Google ScholarDigital Library
- Stéphane Huot, Cédric Dumas, Pierre Dragicevic, JeanDaniel Fekete, and Gérard Hégron. 2004. The MaggLite post-WIMP toolkit: draw it, connect it and run it. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST'04). ACM, New York, NY, USA, 257--266. Google ScholarDigital Library
- Martin Kaltenbrunner and Ross Bencina. 2007. reacTIVision: a computer-vision framework for table-based tangible interaction. In Proceedings of the 1st international conference on Tangible and embedded interaction (TEI'07). ACM, New York, NY, USA, 69--74. Google ScholarDigital Library
- Jun Kato, Daisuke Sakamoto, and Takeo Igarashi. 2012. Phybots: a toolkit for making robotic things. In Proceedings of the Designing Interactive Systems Conference (DIS'12). ACM, New York, NY, USA, 248--257. Google ScholarDigital Library
- Bonifaz Kaufmann and Leah Buechley. 2010. Amarino: a toolkit for the rapid prototyping of mobile ubiquitous computing. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services (MobileHCI'10). ACM, New York, NY, USA, 291--298. Google ScholarDigital Library
- Travis Kirton, Sebastien Boring, Dominikus Baur, Lindsay MacDonald, and Sheelagh Carpendale. 2013. C4: a creative-coding API for media, interaction and animation. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (TEI'13). ACM, New York, NY, USA, 279286. Google ScholarDigital Library
- Scott R. Klemmer, Jack Li, James Lin, and James A. Landay. 2004. Papier-Mâché: toolkit support for tangible input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'04). ACM, New York, NY, USA, 399--406. Google ScholarDigital Library
- Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research methods in human-computer interaction. Morgan Kaufmann. Google Scholar
- David Ledo, Fraser Anderson, Ryan Schmidt, Lora Oehlberg, Saul Greenberg, and Tovi Grossman. 2017. Pineal: Bringing Passive Objects to Life with Embedded Mobile Devices. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI'17). ACM, New York, NY, USA, 25832593. Google ScholarDigital Library
- David Ledo, Miguel A. Nacenta, Nicolai Marquardt, Sebastian Boring, and Saul Greenberg. 2012. The HapticTouch toolkit: enabling exploration of haptic interactions. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI'12), Stephen N. Spencer (Ed.). ACM, New York, NY, USA, 115--122. Google ScholarDigital Library
- David Ledo, Lora Oehlberg and Saul Greenberg. 2017. The Toolkit-Audience Challenge. In Workshop on HCI.Tools at CHI 2017.Google Scholar
- Johnny C. Lee, Daniel Avrahami, Scott E. Hudson, Jodi Forlizzi, Paul H. Dietz, and Darren Leigh. 2004. The calder toolkit: wired and wireless components for rapidly prototyping interactive devices. In Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques (DIS'04). ACM, New York, NY, USA, 167--175. Google ScholarDigital Library
- Brian Y. Lim and Anind K. Dey. 2010. Toolkit to support intelligibility in context-aware applications. In Proceedings of the 12th ACM international conference on Ubiquitous computing (UbiComp'10). ACM, New York, NY, USA, 13--22. Google ScholarDigital Library
- James Lin and James A. Landay. 2008. Employing patterns and layers for early-stage design and prototyping of cross-device user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'08). ACM, New York, NY, USA, 13131322. Google ScholarDigital Library
- James Lin, Mark W. Newman, Jason I. Hong, and James A. Landay. 2000. DENIM: finding a tighter fit between tools and practice for Web site design. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI'00). ACM, New York, NY, USA, 510--517. Google ScholarDigital Library
- Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. 2004. DART: a toolkit for rapid design exploration of augmented reality experiences. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST'04). ACM, New York, NY, USA, 197--206. Google ScholarDigital Library
- Jennifer Mankoff, Scott E. Hudson, and Gregory D. Abowd. 2000. Providing integrated toolkit-level support for ambiguity in recognition-based interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI'00). ACM, New York, NY, USA, 368--375. Google ScholarDigital Library
- Javier Marco, Eva Cerezo, and Sandra Baldassarri. 2012. ToyVision: a toolkit for prototyping tabletop tangible games. In Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems (EICS'12). ACM, New York, NY, USA, 71--80. Google ScholarDigital Library
- Nicolai Marquardt, Robert Diaz-Marino, Sebastian Boring, and Saul Greenberg. 2011. The proximity toolkit: prototyping proxemic interactions in ubiquitous computing ecologies. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST'11). ACM, New York, NY, USA, 315--326. Google ScholarDigital Library
- Nicolai Marquardt and Saul Greenberg. 2007. Distributed physical interfaces with shared phidgets. In Proceedings of the 1st international conference on Tangible and embedded interaction (TEI'07). ACM, New York, NY, USA, 13--20. Google ScholarDigital Library
- Nicolai Marquardt, Steven Houben, Michel BeaudouinLafon, and Andrew D. Wilson. 2017. HCITools: Strategies and Best Practices for Designing, Evaluating and Sharing Technical HCI Toolkits. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA'17). ACM, New York, NY, USA, 624--627. Google ScholarDigital Library
- Nicolai Marquardt, Johannes Kiemer, David Ledo, Sebastian Boring, and Saul Greenberg. 2011. Designing user-, hand-, and handpart-aware tabletop interactions with the TouchID toolkit. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS'11). ACM, New York, NY, USA, 21--30. Google ScholarDigital Library
- Tara Matthews, Anind K. Dey, Jennifer Mankoff, Scott Carter, and Tye Rattenbury. 2004. A toolkit for managing user attention in peripheral displays. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST'04). ACM, New York, NY, USA, 247--256. Google ScholarDigital Library
- Joseph McGrath. 1995. Methodology matters: Doing research in the behavioral and social sciences. In Readings in Human-Computer Interaction: Toward the Year 2000 (2nd ed). 152--169. Google ScholarDigital Library
- Jan Meskens, Kris Luyten, and Karin Coninx. 2010. DMacs: building multi-device user interfaces by demonstrating, sharing and replaying design actions. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST'10). ACM, New York, NY, USA, 129--138. Google ScholarDigital Library
- Jan Meskens, Jo Vermeulen, Kris Luyten, and Karin Coninx. 2008. Gummy for multi-platform user interface designs: shape me, multiply me, fix me, use me. In Proceedings of the working conference on Advanced visual interfaces (AVI'08). ACM, New York, NY, USA, 233--240. Google ScholarDigital Library
- Rolf Molich and Jakob Nielsen. 1990. Improving a human-computer dialogue. Commun. ACM 33, 3 (March 1990), 338--348. Google ScholarDigital Library
- Brad Myers, Scott E. Hudson, and Randy Pausch. 2000. Past, present, and future of user interface software tools. ACM Trans. Comput.-Hum. Interact. 7, 1 (March 2000), 3--28. Google ScholarDigital Library
- Michael Nebeling, Elena Teunissen, Maria Husmann, and Moira C. Norrie. 2014. XDKinect: development framework for cross-device interaction using kinect. In Proceedings of the 2014 ACM SIGCHI symposium on Engineering interactive computing systems (EICS'14). ACM, New York, NY, USA, 65--74. Google ScholarDigital Library
- Michael Nebeling, Theano Mintsi, Maria Husmann, and Moira Norrie. 2014. Interactive development of cross-device user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'14). ACM, New York, NY, USA, 27932802. Google ScholarDigital Library
- Michael Nebeling and Moira Norrie. 2012. jQMultiTouch: lightweight toolkit and development framework for multi-touch/multi-device web interfaces. In Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems (EICS'12). ACM, New York, NY, USA, 61--70. Google ScholarDigital Library
- Michael Nebeling. Playing the Tricky Game of Toolkits Research. In workshop on HCI.Tools at CHI'2017.Google Scholar
- Carman Neustaedter and Phoebe Sengers. 2012. Autobiographical design in HCI research: designing and learning through use-it-yourself. In Proceedings of the Designing Interactive Systems Conference (DIS'12). ACM, New York, NY, USA, 514--523. Google ScholarDigital Library
- Richard A. Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J. Davison, Pushmeet Kohli, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. 2011. KinectFusion: Real-time dense surface mapping and tracking. In Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR'11). IEEE Computer Society, Washington, DC, USA, 127--136. Google ScholarDigital Library
- Mark W. Newman, Shahram Izadi, W. Keith Edwards, Jana Z. Sedivy, and Trevor F. Smith. 2002. User interfaces when and where they are needed: an infrastructure for recombinant computing. In Proceedings of the 15th annual ACM symposium on User interface software and technology (UIST'02). ACM, New York, NY, USA, 171--180. Google ScholarDigital Library
- Jakob Nielsen. 1994. Usability engineering. Elsevier.Google ScholarDigital Library
- Dan R. Olsen, Jr.. 2007. Evaluating user interface systems research. In Proceedings of the 20th annual ACM symposium on User interface software and technology (UIST'07). ACM, New York, NY, USA, 251--258. Google ScholarDigital Library
- Antti Oulasvirta and Kasper Hornbæk. 2016. HCI Research as Problem-Solving. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI'16). ACM, New York, NY, USA, 49564967. Google ScholarDigital Library
- Max Pfeiffer, Tim Duente, and Michael Rohs. 2016. Let your body move: a prototyping toolkit for wearable force feedback with electrical muscle stimulation. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI'16). ACM, New York, NY, USA, 418--427. Google ScholarDigital Library
- Peter G. Polson, Clayton Lewis, John Rieman, and Cathleen Wharton. 1992. Cognitive walkthroughs: a method for theory-based evaluation of user interfaces. Int. J. Man-Mach. Stud.36, 5 (May 1992), 741773. Google ScholarDigital Library
- Jenny Preece and H. Dieter Rombach. 1994. A taxonomy for combining software engineering and humancomputer interaction measurement approaches: towards a common framework. Int. J. Hum.-Comput. Stud. 41, 4 (October 1994), 553--583. Google ScholarDigital Library
- Raf Ramakers, Fraser Anderson, Tovi Grossman, and George Fitzmaurice. 2016. RetroFab: A Design Tool for Retrofitting Physical Interfaces using Actuators, Sensors and 3D Printing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI'16). ACM, New York, NY, USA, 409--419. Google ScholarDigital Library
- Raf Ramakers, Kashyap Todi, and Kris Luyten. 2015. PaperPulse: An Integrated Approach for Embedding Electronics in Paper Designs. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI'15). ACM, New York, NY, USA, 2457--2466. Google ScholarDigital Library
- Yvonne Rogers, and Paul Marshall. (2017). Research in the Wild. Synthesis Lectures on Human-Centered Informatics, 10(3). Google ScholarDigital Library
- Mark Roseman and Saul Greenberg. 1996. Building real-time groupware with GroupKit, a groupware toolkit. ACM Trans. Comput.-Hum. Interact. 3, 1 (March 1996), 66--106. Google ScholarDigital Library
- Daniel Salber, Anind K. Dey, and Gregory D. Abowd. 1999. The context toolkit: aiding the development of context-enabled applications. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI'99). ACM, New York, NY, USA, 434441. Google ScholarDigital Library
- Antti Salovaara, Antti Oulasvirta, and Giulio Jacucci. 2017. Evaluation of Prototypes and the Problem of Possible Futures. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI'17). ACM, New York, NY, USA, 2064--2077. Google ScholarDigital Library
- Valkyrie Savage, Colin Chang, and Björn Hartmann. 2013. Sauron: embedded single-camera sensing of printed physical user interfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology (UIST'13). ACM, New York, NY, USA, 447--456. Google ScholarDigital Library
- Valkyrie Savage, Sean Follmer, Jingyi Li, and Björn Hartmann. 2015. Makers' Marks: Physical Markup for Designing and Fabricating Functional Objects. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST'15). ACM, New York, NY, USA, 103--108. Google ScholarDigital Library
- Valkyrie Savage, Xiaohan Zhang, and Björn Hartmann. 2012. Midas: fabricating custom capacitive touch sensors to prototype interactive objects. In Proceedings of the 25th annual ACM symposium on User interface software and technology (UIST'12). ACM, New York, NY, USA, 579--588. Google ScholarDigital Library
- Teddy Seyed, Alaa Azazi, Edwin Chan, Yuxi Wang, and Frank Maurer. 2015. SoD-Toolkit: A Toolkit for Interactively Prototyping and Developing Multi-Sensor, Multi-Device Environments. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS'15). ACM, New York, NY, USA, 171--180. Google ScholarDigital Library
- B. Schilit, N. Adams, and R. Want. 1994. ContextAware Computing Applications. In Proceedings of the 1994 First Workshop on Mobile Computing Systems and Applications (WMCSA'94). IEEE Computer Society, Washington, DC, USA, 85--90. Google ScholarDigital Library
- Chia Shen, Frédéric D. Vernier, Clifton Forlines, and Meredith Ringel. 2004. DiamondSpin: an extensible toolkit for around-the-table interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'04). ACM, New York, NY, USA, 167--174. Google ScholarDigital Library
- Jeffrey Stylos and Brad A. Myers. 2008. The implications of method placement on API learnability. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering (SIGSOFT'08/FSE-16). ACM, New York, NY, USA, 105--112. Google ScholarDigital Library
- Ivan Edward Sutherland. 1980. Sketchpad: A Man-Machine Graphical Communication System. Garland Publishing, Inc., New York, NY, USA. Google ScholarDigital Library
- Nicolas Villar, Kiel Mark Gilleade, Devina Ramdunyellis and Hans Gellersen. 2007. The VoodooIO gaming kit: a real-time adaptable gaming controller. Comput. Entertain. 5, 3, pages. Google ScholarDigital Library
- Nicolas Villar, James Scott, Steve Hodges, Kerry Hammil, and Colin Miller. (2012) .NET Gadgeteer: A Platform for Custom Devices. In Pervasive Computing. Pervasive 2012. Lecture Notes in Computer Science, vol 7319. Springer, Berlin, Heidelberg. 216--233 Google ScholarDigital Library
- Akira Wakita and Yuki Anezaki. 2010. Intuino: an authoring tool for supporting the prototyping of organic interfaces. In Proceedings of the 8th ACM Conference on Designing Interactive Systems (DIS'10). ACM, New York, NY, USA, 179--188. Google ScholarDigital Library
- Chiuan Wang, Hsuan-Ming Yeh, Bryan Wang, Te-Yen Wu, Hsin-Ruey Tsai, Rong-Hao Liang, Yi-Ping Hung, and Mike Y. Chen. 2016. CircuitStack: Supporting Rapid Prototyping and Evolution of Electronic Circuits. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST'16). ACM, New York, NY, USA, 687--695. Google ScholarDigital Library
- Mark Weiser, (1991). The Computer for the 21st Century. Scientific American, 265(3), 94--105.Google ScholarCross Ref
- Mikael Wiberg and Erik Stolterman. 2014. What makes a prototype novel?: a knowledge contribution concern for interaction design research. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI'14). ACM, New York, NY, USA, 531--540. Google ScholarDigital Library
- Alexander Wiethoff, Hanna Schneider, Julia Küfner, Michael Rohs, Andreas Butz, and Saul Greenberg. 2013. Paperbox: a toolkit for exploring tangible interaction on interactive surfaces. In Proceedings of the 9th ACM Conference on Creativity & Cognition (C&C'13), Ellen Yi-Luen Do, Steven Dow, Jack Ox, Steve Smith, Kazushi Nishimoto, and Chek Tien Tan (Eds.). ACM, New York, NY, USA, 64--73. Google ScholarDigital Library
- Daniel Wigdor, Hrvoje Benko, John Pella, Jarrod Lombardo, and Sarah Williams. 2011. Rock & rails: extending multi-touch interactions with shape gestures to enable precise spatial manipulations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI'11). ACM, New York, NY, USA, 15811590. Google ScholarDigital Library
- Raphael Wimmer, Matthias Kranz, Sebastian Boring and Albrecht Schmidt. 2007. A Capacitive Sensing Toolkit for Pervasive Activity Detection and Recognition. In Proc. International Conference on Pervasive Computing and Communications (PerCom'07), IEEE, White Plains, NY., 171--180. Google ScholarDigital Library
- Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In Proceedings of the 20th annual ACM symposium on User interface software and technology (UIST'07). ACM, New York, NY, USA, 159--168. Google ScholarDigital Library
- Jacob O. Wobbrock and Julie A. Kientz. 2016. Research contributions in human-computer interaction. interactions 23, 3 (April 2016), 38--44. Google ScholarDigital Library
- Chi-Jui Wu, Steven Houben, and Nicolai Marquardt. 2017. EagleSense: Tracking People and Devices in Interactive Spaces using Real-Time Top-View DepthSensing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI'17). ACM, New York, NY, USA, 3929--3942. Google ScholarDigital Library
- Robert Xiao, Chris Harrison, and Scott E. Hudson. 2013. WorldKit: rapid and easy creation of ad-hoc interactive applications on everyday surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'13). ACM, New York, NY, USA, 879--888. Google ScholarDigital Library
- Jishuo Yang and Daniel Wigdor. 2014. Panelrama: enabling easy specification of cross-device web applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'14). ACM, New York, NY, USA, 2783--2792. Google ScholarDigital Library
- Tom Yeh, Tsung-Hsiang Chang, and Robert C. Miller. 2009. Sikuli: using GUI screenshots for search and automation. In Proceedings of the 22nd annual ACM symposium on User interface software and technology (UIST'09). ACM, New York, NY, USA, 183--192. Google ScholarDigital Library
Index Terms
- Evaluation Strategies for HCI Toolkit Research
Recommendations
Iterative design and evaluation of an event architecture for pen-and-paper interfaces
UIST '08: Proceedings of the 21st annual ACM symposium on User interface software and technologyThis paper explores architectural support for interfaces combining pen, paper, and PC. We show how the event-based approach common to GUIs can apply to augmented paper, and describe additions to address paper's distinguishing characteristics. To ...
The context toolkit: aiding the development of context-enabled applications
CHI '99: Proceedings of the SIGCHI conference on Human Factors in Computing SystemsContext-enabled applications are just emerging and promise richer interaction by taking environmental context into account. However, they are difficult to build due to their distributed nature and the use of unconventional sensors. The concepts of ...
Thinking outside the (tool) box: empowering people with toolkits
NordiCHI '18: Proceedings of the 10th Nordic Conference on Human-Computer InteractionToolkits enable individuals to create and design their own technologies. With more toolkits being commercially available, movements such as the Maker Movements has led to a hype around Do-It-Yourself and Making practices. This development aligns with ...
Comments