Hyewon Suh
Julie A. Kientz
ABSTRACT
Computing systems that place a high level of burden on their users can have a negative affect on initial adoption, retention, and overall user experience. Through an iterative process, we have developed a model for user burden that consists of six constructs: 1) difficulty of use, 2) physical, 3) time and social, 4) mental and emotional, 5) privacy, and 6) financial. If researchers and practitioners can have an understanding of the overall level of burden systems may be having on the user, they can have a better sense of whether and where to target future design efforts that can reduce those burdens. To help assist with understanding and measuring user burden, we have also developed and validated a measure of user burden in computing systems called the User Burden Scale (UBS), which is a 20-item scale with 6 individual sub-scales representing each construct. This paper presents the process we followed to develop and validate this scale for use in evaluating user burden in computing systems. Results indicate that the User Burden Scale has good overall inter-item reliability, convergent validity with similar scales, and concurrent validity when compared to systems abandoned vs. those still in use.
- Sajay Arthanat, Stephen M. Bauer, James A. Lenker, Susan M. Nochajski, and Yow Wu B. Wu. 2007. Conceptualization and measurement of assistive technology usability. Disability & Rehabilitation: Assistive Technology 2, 4: 235--248.Google Scholar
Cross Ref
- Ilhan Aslan, Martin Murer, Verena Fuchsberger, Andrew Fugard, and Manfred Tscheligi. 2013. Workload on your fingertips: the influence of workload on touchbased drag and drop. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces (ITS '13), 417--420. Google ScholarDigital Library
- Sasikanth Avancha, Amit Baxi, and David Kotz. 2012. Privacy in mobile technology for personal healthcare. ACM Computing Surveys (CSUR) 45, 1: 3. Google ScholarDigital Library
- James E. Bailey and Sammy W. Pearson. 1983. Development of a tool for measuring and analyzing computer user satisfaction. Management science 29, 5: 530--545. Google ScholarDigital Library
- Louise Barkhuus and Anind K. Dey. 2003. LocationBased Services for Mobile Telephony: a Study of Users' Privacy Concerns. In INTERACT, 3: 702--712.Google Scholar
- Thomas Beauvisage. 2009. Computer usage in daily life. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '09), 575--584. Google ScholarDigital Library
- John L. Bennett 1984. Managing to meet usability requirements: establishing and meeting software development goals. Visual display terminals, 161--84.Google Scholar
- Mark Bilandzic. 2010. The embodied hybrid space: designing ubiquitous computing towards an amplification of situated real world experiences. In Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction (OzCHI '10), 422--427. Google ScholarDigital Library
- Richard E. Boyatzis. 1998. Transforming qualitative information: Thematic analysis and code development. Sage.Google Scholar
- John Brooke. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194: 4--7.Google Scholar
- Michael Buhrmester, Tracy Kwang, and Samuel D. Gosling. 2011. Amazon's Mechanical Turk a new source of inexpensive, yet high-quality, data?. Perspectives on psychological science 6, 1: 3--5.Google Scholar
- Rudy Den Buurman. 1997. User-centred design of smart products. Ergonomics 40, 10: 1159--1169.Google ScholarCross Ref
- Fang Chen, Natalie Ruiz, Eric Choi, Julien Epps, M. Asif Khawaja, Ronnie Taib, Bo Yin, and Yang Wang. Multimodal behavior and interaction as indicators of cognitive load. ACM Transactions on Interactive Intelligent Systems (TiiS '12) 2, 4: 22. Google ScholarDigital Library
- John P. Chin, Virginia A. Diehl, and Kent L. Norman. 1988. Development of an instrument measuring user satisfaction of the human-computer interface. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '88), 213--218. Google ScholarDigital Library
- Lee J. Cronbach and R. L. Thorndike. 1971. Educational measurement. Test validation, 443--507.Google Scholar
- Fred D. Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 319--340. Google ScholarDigital Library
- Fred D. Davis. 1993. User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International journal of man machine studies 38, 3: 475--487. Google ScholarDigital Library
- Louise Demers, Rhoda Weiss-Lambrou and Bernadette Ska. 2002. The Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST 2.0): an overview and recent progress. Technology and Disability 14, 3: 101--105.Google ScholarCross Ref
- David De Vaus. 2002. Analyzing social science data: 50 key problems in data analysis. Sage.Google Scholar
- Judith Donath. 2014. How social media design shapes society. In Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems (CHI '14), 1057--1058. Google ScholarDigital Library
- Joseph S. Dumas and Janice Redish. 1999. A practical guide to usability testing. Intellect Books. Google ScholarDigital Library
- Günther Gediga, Kai-Christoph Hamborg, and Ivo Düntsch. 1999. The IsoMetrics usability inventory: an operationalization of ISO 9241--10 supporting summative and formative evaluation of software systems. Behaviour & Information Technology 18, 3: 151--164.Google ScholarCross Ref
- Joseph A. Gliem and Rosemary R. Gliem. 2003. Calculating, interpreting, and reporting Cronbach's alpha reliability coefficient for Likert-type scales. Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education.Google Scholar
- John D. Gould and Clayton Lewis. 1985. Designing for usability: key principles and what designers think. Communications of the ACM 28, 3: 300--311. Google ScholarDigital Library
- Jeroen B. Guinée. 2002. Handbook on life cycle assessment operational guide to the ISO standards." The international journal of life cycle assessment 7, 5: 311--313.Google Scholar
- Robert Harmon, David Raffo, and Stuart Faulk. 2003. Incorporating price sensitivity measurement into the software engineering process. In Management of Engineering and Technology (PICMET'03), Technology Management for Reshaping the World. Portland International Conference on, 316--323.Google Scholar
- Sandra G. Hart and Lowell E. Staveland. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology, 52: 139--183..Google Scholar
- Leanne M. Hirshfield, Erin Treacy Solovey, Audrey Girouard, James Kebinger, Robert JK Jacob, Angelo Sassaroli, and Sergio Fantini. 2009. Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 2185--2194. Google ScholarDigital Library
- Jason I. Hong, Jennifer D. Ng, Scott Lederer, and James A. Landay. 2004. Privacy risk models for designing privacy-sensitive ubiquitous computing systems. In Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques (DIS '04), 91--100. Google ScholarDigital Library
- Salar Jahedi and Fabio Méndez. 2014. On the advantages and disadvantages of subjective measures. Journal of Economic Behavior & Organization 98: 97114.Google ScholarCross Ref
- Beom Suk Jin and Yong Gu Ji. 2010. Usability risk level evaluation for physical user interface of mobile phone. Computers in Industry 61, 4: 350--363. Google ScholarDigital Library
- Jurek Kirakowski and A. Dillon. 1988. The computer user satisfaction inventory (CUSI): Manual and scoring key. Cork, Ireland: Human Factors Research Group, University College of Cork.Google Scholar
- Jurek Kirakowski and Mary Corbett. 1993. SUMI: The software usability measurement inventory. British journal of educational technology 24, 3: 210--212.Google Scholar
- Jonathan Klein, Youngme Moon, and Rosalind W. Picard. 2002. This computer responds to user frustration:: Theory, design, and results. Interacting with computers 14, no. 2 (2002): 119--140.Google Scholar
- Shelah Leader, Phillip Jacobson, James Marcin, Ralph Vardis, Mark Sorrentino, and Dennis Murray. 2002. A method for identifying the financial burden of hospitalized infants on families. Value in Health 5, 1: 55--59.Google ScholarCross Ref
- Han X. Lin, Yee-Yin Choong, and Gavriel Salvendy. 1997. A proposed index of usability: a method for comparing the relative usability of different software systems. Behaviour & information technology 16, 4--5: 267277.Google Scholar
- Jakob Nielsen. 1991. Usability metrics and methodologies. ACM SIGCHI Bulletin 23, 2: 37--39. Google ScholarDigital Library
- Jum C. Nunnally, Ira H. Bernstein, and Jos MF ten Berge. 1967. Psychometric theory. Vol. 226. NY: McGraw-Hill.Google Scholar
- Jane Ogden and Jessica Lo. 2012. How meaningful are data from Likert scales? An evaluation of how ratings are made and the role of the response shift in the socially disadvantaged. Journal of health psychology 17, 3: 350--361.Google ScholarCross Ref
- Helen Petrie and Nigel Bevan. 2009. The evaluation of accessibility, usability and user experience. The universal access handbook: 10--20.Google Scholar
- Ahmad Rahmati, Chad Tossell, Clayton Shepard, Philip Kortum, and Lin Zhong. 2012. Exploring iPhone usage: the influence of socioeconomic differences on smartphone adoption, usage and usability. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services (Mobile HCI '12), 11--20. Google ScholarDigital Library
- Jeffrey Rubin and Dana Chisnell. 2008. Handbook of usability testing: how to plan, design and conduct effective tests. John Wiley & Sons. Google ScholarDigital Library
- Jocelyn Scheirer, Raul Fernandez, Jonathan Klein, and Rosalind W. Picard. 2002. Frustrating the user on purpose: a step toward building an affective computer. Interacting with computers 14, 2: 93--118.Google Scholar
- Brian Shackel. 1984. The concept of usability. Visual display terminals: Usability issues and health concerns: 45--87.Google Scholar
- Ben Shneiderman. 1992. Designing the user interface: strategies for effective human-computer interaction. Vol. 3. Reading, MA: Addison-Wesley. Google ScholarDigital Library
- Deb Sledgianowski and Songpol Kulviwat. 2008. Social network sites: antecedents of user adoption and usage. Americas Conference on Information Systems (AMCIS) Proceedings: 83.Google Scholar
- Godwin J. Udo. 2001. Privacy and security concerns as major barriers for e-commerce: a survey study. Information Management & Computer Security 9, 4: 165--174.Google ScholarCross Ref
- Robert S. Weiss. 1995. Learning from strangers: The art and method of qualitative interview studies. Simon and Schuster.Google Scholar
- Anna M. Wichansky. 2000. Usability testing in 2000 and beyond. Ergonomics 43, 7: 998--1006.Google ScholarCross Ref
- David Wright. 2011. Should privacy impact assessments be mandatory?. Communications of the ACM 54, 8: 121131. Google ScholarDigital Library
- Svetlana Yarosh, Panos Markopoulos, and Gregory D. Abowd. 2014. Towards a questionnaire for measuring affective benefits and costs of communication technologies. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14), 84--96. Google ScholarDigital Library
Index Terms
- Developing and Validating the User Burden Scale: A Tool for Assessing User Burden in Computing Systems
Recommendations
User Burden of Microinteractions: An In-lab Experiment Examining User Performance and Perceived Burden Related to In-situ Self-reporting
MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer InteractionIn-situ self-reporting on smartwatches has been widely-used to collect data about participants’ experience. Yet it places a burden on participants by requiring immediate response in context. Such user burden can be studied through users’ performance and ...
Developing practical tools for user experience evaluation: a case from mobile news journalism
ECCE '09: European Conference on Cognitive Ergonomics: Designing beyond the Product --- Understanding Activity and User Experience in Ubiquitous EnvironmentsWe present a questionnaire called Attrak-Work to support the evaluation of user experience of mobile systems in the context of mobile news journalism. We discuss theoretical background of the questionnaire and describe the development process including ...
Actionable Inexpensive Games User Research
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing SystemsThis course will allow people to understand the intricacies of rapid games user research methods. For this we will weave together playtesting exercises and help participants turn player feedback into actionable design recommendations. The course is ...
Comments