Sebastian S. Feger
Sünje Dallmeier-Tiessen
Paweł W. Woźniak
Albrecht Schmidt
ABSTRACT
The need for data preservation and reproducible research is widely recognized in the scientific community. Yet, researchers often struggle to find the motivation to contribute to data repositories and to use tools that foster reproducibility. In this paper, we explore possible uses of gamification to support reproducible practices in High Energy Physics. To understand how gamification can be effective in research tools, we participated in a workshop and performed interviews with data analysts. We then designed two interactive prototypes of a research preservation service that use contrasting gamification strategies. The evaluation of the prototypes showed that gamification needs to address core scientific challenges, in particular the fair reflection of quality and individual contribution. Through thematic analysis, we identified four themes which describe perceptions and requirements of gamification in research: Contribution, Metrics, Applications and Scientific practice. Based on these, we discuss design implications for gamification in science.
Supplemental Material
Available for Download
zip
pn3612.zip (9.6 MB)
In order to foster the reproducibility of our work and to provide a base for future research, we make available several of the study?s resources. Those include a code group report from the qualitative data analysis, the questionnaires, the quantitative responses and the prototypes. A README file is available at the root of the archive, providing extensive information about the resources.
- ACM. 2018. Artifact Review and Badging. Website. (April 2018). https: //www.acm.org/publications/policies/artifact-review-badging Retrieved September 10, 2018.Google Scholar
- Monya Baker. 2016. 1,500 scientists lift the lid on reproducibility. Nature 533, 7604 (2016), 452--454.Google ScholarCross Ref
- Sean Bechhofer, Iain Buchan, David De Roure, Paolo Missier, John Ainsworth, Jiten Bhagat, Philip Couch, Don Cruickshank, Mark Delderfield, Ian Dunlop, Matthew Gamble, Danius Michaelides, Stuart Owen, David Newman, Shoaib Sufi, and Carole Goble. 2013. Why linked data is not enough for scientists. Future Generation Computer Systems 29, 2 (2013), 599--611. Google ScholarDigital Library
- C. Glenn Begley and Lee M. Ellis. 2012. Drug development: Raise standards for preclinical cancer research. Nature 483, 7391 (2012), 531-- 3. arXiv:arXiv:cond-mat/9907372v1Google Scholar
- Khalid Belhajjame, Jun Zhao, Daniel Garijo, Kristina Hettne, Raul Palma, Óscar Corcho, José-Manuel Gómez-Pérez, Sean Bechhofer, Graham Klyne, and Carole Goble. 2014. The Research Object Suite of Ontologies: Sharing and Exchanging Research Data and Methods on the Open Web. arXiv preprint arXiv: 1401.4307 February 2014 (2014), 20. arXiv:1401.4307 http://arxiv.org/abs/1401.4307Google Scholar
- T. Berners-Lee, R. Cailliau, J .F. Groff, and B. Pollermann. 1992. Worldwide web: The information universe., 52--58 pages.Google Scholar
- Ann Blandford, Dominic Furniss, and Stephann Makri. 2016. Qualitative HCI Research: Going Behind the Scenes. Morgan & Claypool Publishers, 51--60. Google Scholar
- Ronald F Boisvert. 2016. Incentivizing reproducibility. Commun. ACM 59, 10 (2016), 5--5. Google ScholarDigital Library
- Christine L Borgman. 2007. Scholarship in the digital age: information, infrastructure, and the internet. MIT Press, Cambridge, MA. Google ScholarDigital Library
- A Bowser, D Hansen, J Preece, Y He, C Boston, and J Hammock. 2014. Gamifying citizen science: A study of two user groups. In 17th ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW 2014. 137--140. Google ScholarDigital Library
- Jailson Brito, Vaninha Vieira, and Adolfo Duran. 2015. Towards a Framework for Gamification Design on Crowdsourcing Systems: The G.A.M.E. Approach. 2015 12th International Conference on Information Technology - New Generations (2015), 445--450. Google ScholarDigital Library
- Huseyin Cavusoglu, Zhuolun Li, and Ke-Wei Huang. 2015. Can Gamification Motivate Voluntary Contributions? Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing - CSCW'15 Companion (2015), 171--174. Google ScholarDigital Library
- CERN. 2013. The birth of the web. Website. (Dec 2013). http: //cds.cern.ch/record/1998446 Retrieved March 15, 2018.Google Scholar
- CERN. 2017. CERN Annual Personnel Statistics 2017. (2017). https: //cds.cern.ch/record/2317058 Personnel Statistics 2017.Google Scholar
- Karin Knorr Cetina. 2009. Epistemic cultures: How the sciences make knowledge. Harvard University Press.Google Scholar
- Xiaoli Chen, Sünje Dallmeier-Tiessen, Anxhela Dani, Robin Dasler, Javier Delgado Fernández, Pamfilos Fokianos, Patricia Herterich, and Tibor Simko. 2016. CERN Analysis Preservation: A Novel Digital Library Service to Enable Reusable and Reproducible Research. In International Conference on Theory and Practice of Digital Libraries. Springer, 347--356.Google Scholar
- Open Science Collaboration. 2012. An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science. Perspectives on Psychological Science 7, 6 (2012), 657--660.Google ScholarCross Ref
- J Cowton, S Dallmeier-Tiessen, P Fokianos, L Rueda, P Herterich, J Kuncar, T Simko, and T Smith. 2015. Open Data and Data Analysis Preservation Services for LHC Experiments. 21st International Conference on Computing in High Energy and Nuclear Physics 032030 (2015).Google Scholar
- Steve Dale. 2014. Gamification: Making work fun, or making fun of work? Business Information Review 31, 2 (2014), 82--90.Google ScholarCross Ref
- Sebastian Deterding, Alessandro Canossa, Casper Harteveld, Seth Cooper, Lennart E Nacke, and Jennifer R Whitson. 2015. Gamifying research: Strategies, opportunities, challenges, ethics. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 2421--2424. Google ScholarDigital Library
- Sebastian Deterding, Rilla Khaled, Lennart Nacke, and Dan Dixon. 2011. Gamification: toward a definition. Chi 2011 (2011), 12--15. https: //doi.org/978--1--4503-0268--5/11/0 arXiv:9781450302685Google Scholar
- Alexandra Eveleigh, Charlene Jennett, Stuart Lynn, and Anna L. Cox. 2013. "I want to be a captain! I want to be a captain!": gamification in the old weather citizen science project. Proceedings of the First International Conference on Gameful Design, Research, and Applications - Gamification '13 (2013), 79--82. Google ScholarDigital Library
- Sebastian Feger, Sünje Dallmeier-Tiessen, Pawel Wozniak, and Albrecht Schmidt. 2018. Just Not The Usual Workplace: Meaningful Gamification in Science. Mensch und Computer 2018-Workshopband (2018).Google Scholar
- Mathias Fuchs, Sonia Fizek, Paolo Ruffino, Niklas Schrape, et al. 2014. Rethinking gamification. meson press.Google Scholar
- Mélissa Gaillard and Stefania Pandolfi. 2017. CERN Data Centre passes the 200-petabyte milestone. (Jul 2017). http://cds.cern.ch/record/Google Scholar
- Juho Hamari, Jonna Koivisto, and Harri Sarsa. 2014. Does gamification work?--a literature review of empirical studies on gamification. In 2014 47th Hawaii international conference on system sciences (HICSS). IEEE, 3025--3034. Google ScholarDigital Library
- Maria-Blanca Ibanez, Angela Di-Serio, and Carlos Delgado-Kloos. 2014. Gamification for Engaging Computer Science Students in Learning Activities: A Case Study. IEEE Transactions on Learning Technologies 7, 3 (2014), 291--301.Google ScholarCross Ref
- Mallory C Kidwell, Ljiljana B Lazarevic, Erica Baranski, Tom E Hardwicke, Sarah Piechowski, Lina-Sophia Falkenberg, Curtis Kennett, Agnieszka Slowik, Carina Sonnleitner, Chelsey Hess-Holden, et al. 2016. Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency. PLoS biology 14, 5 (2016), e1002456.Google Scholar
- Janaki Kumar. 2013. Gamification at work: Designing engaging business software. In International conference of design, user experience, and usability. Springer, 528--537. Google ScholarDigital Library
- Zeeya Merali. 2010. Physics: The large human collider., 482--484 pages.Google Scholar
- Lennart E. Nacke and Sebastian Deterding. 2017. The maturing of gamification research. Computers in Human Behavior (2017).Google Scholar
- Scott Nicholson. 2015. A recipe for meaningful gamification. In Gamification in education and business. Springer, 1--20.Google Scholar
- Jasmin Niess and Pawel W Wozniak. 2018. Supporting Meaningful Personal Fitness: the Tracker Goal Evolution Model. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 171. Google ScholarDigital Library
- Rita Orji, Gustavo F Tondello, and Lennart E Nacke. 2018. Personalizing persuasive strategies in gameful systems to gamification user types. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 435. Google ScholarDigital Library
- Anisa Rowhani-Farid, Michelle Allen, and Adrian G. Barnett. 2017. What incentives increase data sharing in health and medical research? A systematic review. Research Integrity and Peer Review 2, 1 (2017), 4.Google ScholarCross Ref
- Jonathan F Russell. 2013. If a job is worth doing, it is worth doing twice: researchers and funding agencies need to put a premium on ensuring that results are reproducible. Nature 496, 7443 (2013), 7--8.Google ScholarCross Ref
- Richard M Ryan and Edward L Deci. 2000. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American psychologist 55, 1 (2000), 68.Google Scholar
- Michael Sailer, Jan Ulrich Hense, Sarah Katharina Mayr, and Heinz Mandl. 2017. How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Computers in Human Behavior 69 (2017), 371--380. Google ScholarDigital Library
- Victoria Stodden and Sheila Miguez. 2014. Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research. Journal of Open Research Software 2, 1 (2014), 21.Google ScholarCross Ref
- Jakub Swacha and Karolina Muszy'ska. 2016. Design patterns for gamification of work. Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality - TEEM '16 (2016), 763--769. Google ScholarDigital Library
- Gustavo F. Tondello, Alberto Mora, and Lennart E. Nacke. 2017. Elements of Gameful Design Emerging from User Preferences. Proceedings of the Annual Symposium on Computer-Human Interaction in Play - CHI PLAY '17 (2017), 129--142. Google ScholarDigital Library
- K. Werbach and D. Hunter. 2012. For the Win: How Game Thinking Can Revolutionize Your Business. Wharton Digital Press. https://books. google.ch/books?id=abg0SnK3XdMCGoogle Scholar
- Mark D. Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, JanWillem Boiten, Luiz Bonino da Silva Santos, Philip E. Bourne, Jildau Bouwman, Anthony J. Brookes, Tim Clark, Mercè Crosas, Ingrid Dillo, Olivier Dumon, Scott Edmunds, Chris T. Evelo, Richard Finkers, Alejandra Gonzalez-Beltran, Alasdair J.G. Gray, Paul Groth, Carole Goble, Jeffrey S. Grethe, Jaap Heringa, Peter A.C 't Hoen, Rob Hooft, Tobias Kuhn, Ruben Kok, Joost Kok, Scott J. Lusher, Maryann E. Martone, Albert Mons, Abel L. Packer, Bengt Persson, Philippe Rocca-Serra, Marco Roos, Rene van Schaik, Susanna-Assunta Sansone, Erik Schultes, Thierry Sengstag, Ted Slater, George Strawn, Morris a. Swertz, Mark Thompson, Johan van der Lei, Erik van Mulligen, Jan Velterop, Andra Waagmeester, Peter Wittenburg, Katherine Wolstencroft, Jun Zhao, and Barend Mons. 2016. The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data 3 (2016), 160018.Google ScholarCross Ref
Index Terms
- Gamification in Science: A Study of Requirements in the Context of Reproducible Research
Recommendations
Gamification Science, Its History and Future
Background. Definitions of gamification tend to vary by person, both in industry and within academia. One particularly popular lay interpretation, introduced and popularized by Ian Bogost, and reiterated by Jan Klabbers, is that gamification is ?...
Gamification usage ecology
SIGDOC '17: Proceedings of the 35th ACM International Conference on the Design of CommunicationGamification is applied in academia and industry and is being studied in the last years. Gamification is nowadays applied to several areas, such as learning, computer science, business, marketing, tourism, among others. However, we need to understand to ...
How gamification motivates
The main aim of gamification, i.e. the implementation of game design elements in real-world contexts for non-gaming purposes, is to foster human motivation and performance in regard to a given activity. Previous research, although not entirely ...
Comments