Skip to main content
SpringerLink
Log in
Book cover

Macrotask Crowdsourcing pp 127–145Cite as

Addressing Cooperation Issues in Situated Crowdsourcing

  • Chapter
  • First Online:

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

Situated crowdsourcing has been growing in popularity as an alternative way to collect complex and often creative crowd work. However, previous situated crowdsourcing deployments have not successfully leveraged cooperation possibilities with their audiences, which can improve the data quality of deployed macrotasks. In this chapter, we present three situated crowdsourcing case studies that used different situated technologies and identify the reasons behind their missteps regarding promoting cooperation between workers. Then, based on the identified issues, we propose the design of a novel situated crowdsourcing platform that aims to effectively support cooperation without alienating solo workers. In order to gather insights on our proposed design, we built a prototype platform and evaluated it using a laboratory study with 24 participants. In general, participants were positive about the idea as it provided an easy way to cooperate with friends when completing tasks, while also allowing them to adjust the working environment to their liking. Finally, we conclude by offering insights towards improving cooperation in future situated crowdsourcing deployments and how this can assist in completing macrotasks.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

    Article  Google Scholar 

  • Brignull, H., & Rogers, Y. (2003). Enticing people to interact with large public displays in public spaces. In M. Rauterberg, M. Menozzi, & J. Wesson (Eds.), Proceedings of 9th IFIP TC13 International Conference on Human-Computer Interaction, INTERACT ‘03, Zurich, Switzerland, 1–5 September 2003 (pp. 17–24). Amsterdam: IOS Press.

    Google Scholar 

  • Cheng, J., Teevan, J., Iqbal, S. T., & Bernstein, M. S. (2015). Break it down: A comparison of macro- and microtasks. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ‘15) (pp. 4061–4064). ACM Press.

    Google Scholar 

  • Felstiner, A. (2012). The weakness of crowds. Crowds and Clouds. Retrieved March 15, 2017, from http://limn.it/the-weakness-of-crowds/.

  • Goncalves, J., Ferreira, D., Hosio, S., Liu, Y., Rogstadius, J., Kukka, H., et al. (2013). Crowdsourcing on the spot: Altruistic use of public displays, feasibility, performance, and behaviours. In J. F. Canny, M. Langheinrich, & J. Rekimoto (Eds.), Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ‘13, Zurich, Switzerland, 8–12 September 2013 (pp. 753–762). New York: ACM Press.

    Google Scholar 

  • Goncalves, J., Pandab, P., Ferreira, D., Ghahramani, M., Zhao, G., & Kostakos, V. (2014a). Projective testing of diurnal collective emotion. In J. Kientz, J. Scott, & J. Song (Eds.), Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ‘14, Seattle, USA, 13–17 September 2014 (pp. 487–497). New York: ACM Press.

    Google Scholar 

  • Goncalves, J., Hosio, S., Ferreira, D., & Kostakos, V. (2014b). Game of words: Tagging places through crowdsourcing on public displays. In C. Neustaedter, S. Bardzell, & E. Paulos (Eds.), Proceedings of the 2014 Conference on Designing Interactive Systems, DIS ‘14, Vancouver, Canada, 21–25 June 2014 (pp. 705–714). New York: ACM Press.

    Google Scholar 

  • Goncalves, J., Kukka, H., Sanchez, I., & Kostakos, V. (2016). Crowdsourcing queue estimations in situ. In P. Bjørn & J. Konstan (Eds.), Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW ‘16, San Francisco, USA, 27 February–2 March 2016 (pp. 1040–1051). New York: ACM Press.

    Google Scholar 

  • Goncalves, J., Hosio, S., & Kostakos, V. (2017). Eliciting structured knowledge from situated crowd markets. ACM Transactions on Internet Technology, 17(2), Article 14.

    Google Scholar 

  • Gray, M. L., Suri, S., Ali, S. S., & Kulkarni, D. (2016). The crowd is a collaborative network. In P. Bjørn & J. Konstan (Eds.), Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW ‘16, San Francisco, USA, 27 February–2 March 2016 (pp. 134–147). New York: ACM Press.

    Google Scholar 

  • Gupta, N., Martin, D., Hanrahan, B. V., & O’Neill, J. (2014). Turk-life in India. In D. W. McDonald & P. Bjørn (Eds.), Proceedings of the 18th International Conference on Supporting Group Work, GROUP ‘14, Sanibel Island, USA, 9–12 November 2014 (pp. 1–11). New York: ACM Press.

    Google Scholar 

  • Haas, D., Ansel, J., Gu, L., & Marcus, A. (2015, August). Argonaut: Macrotask crowdsourcing for complex data processing. In Proceedings of the VLDB Endow (Vol. 8, no. 12, pp. 1642–1653). http://dx.doi.org/10.14778/2824032.2824062.

  • Heikkinen, T., Goncalves, J., Kostakos, V., Elhart, I., & Ojala, T. (2014). Tandem browsing toolkit: Distributed multi-display interfaces with web technologies. In A. Quigley (Ed.), Proceedings of the International Symposium on Pervasive Displays, PerDis ‘14, Copenhagen, Denmark, 3–4 June 2014 (pp. 142–147). New York: ACM Press.

    Google Scholar 

  • Heimerl, K., Gawalt, B., Chen, K., Parikh, T., & Hartmann, B. (2012). Community sourcing: Engaging local crowds to perform expert work via physical kiosks. In H. Chi & K. Höök (Eds.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ‘12, Austin, USA, 5–10 May 2012 (pp. 1539–1548). New York: ACM Press.

    Google Scholar 

  • Hosio, S., Goncalves, J., & Kostakos, V. (2013). Application discoverability on multipurpose public displays: Popularity comes at a price. In Proceedings of International Symposium on Pervasive Displays 2013 (PerDis’13) (pp. 31–36).

    Google Scholar 

  • Hosio, S., Goncalves, J., Kostakos, V., & Riekki, J. (2015a). Crowdsourcing public opinion using urban pervasive technologies: Lessons from real-life experiments in Oulu. Policy & Internet, 7(2), 203–222.

    Article  Google Scholar 

  • Hosio, S., Goncalves, J., Lehdonvirta, V., Ferreira, D., & Kostakos, V. (2014). Situated crowdsourcing using a market model. In M. Dontcheva & D. Wigdor (Eds.), Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ‘14, Honolulu, USA, 5–8 October 2014 (pp. 55–64). New York: ACM Press.

    Google Scholar 

  • Hosio, S., Harper, R., O’Hara, K., Goncalves, J., & Kostakos, V. (2015). Life through the lens: A qualitative investigation of human behaviour with an urban photography service. In P. Olivier & D. Foster (Eds.), Proceedings of the 2015 British HCI Conference, British HCI ‘15, Lincoln, United Kingdom, 13–17 July 2015 (pp. 157–164). New York: ACM Press.

    Google Scholar 

  • Hosio, S., Kukka, H., Goncalves, J., Kostakos, V., & Ojala, T. (2016). Toward meaningful engagement with pervasive displays. IEEE Pervasive Computing, 15(3), 24–31.

    Article  Google Scholar 

  • Huan, Y., Shema, A., & Xia, H. (2017). A proposed genome of mobile and situated crowdsourcing and its design implications for encouraging contributions. International Journal of Human-Computer Studies, 102, 69–80.

    Article  Google Scholar 

  • Huang, Y.-C. (2015). Designing a micro-volunteering platform for situated crowdsourcing. In L. Ciolfi & D. McDonald (Eds.), Proceedings of the 19th ACM Conference Companion on Computer-Supported Cooperative Work and Social Computing, CSCW ‘15, Vancouver, Canada, 14–18 March 2015 (pp. 73–76). New York: ACM Press.

    Google Scholar 

  • Ipeirotis, P. G., & Gabrilovich, E. (2014). Quizz: Targeted crowdsourcing with a billion (potential) users. In A. Broder, K. Shim, & T. Suel (Eds.), Proceedings of the 23rd International Conference on World Wide Web, WWW ‘14, Seoul, South Korea, 7–11 April 2014 (pp. 143–154). New York: ACM Press.

    Google Scholar 

  • Jurmu, M., Goncalves, J., Riekki, J., & Ojala, T. (2014). Exploring use and appropriation of a non-moderated community display. In S. W. Loke, L. Kulik, & E. Pitoura (Eds.), Proceedings of the 13th International Conference on Mobile and Ubiquitous Multimedia, MUM ‘14, Melbourne, Australia, 25–28 November 2014 (pp. 107–115). New York: ACM Press.

    Google Scholar 

  • Kittur, A. (2010). Crowdsourcing, collaboration and creativity. XRDS: Crossroads, The ACM Magazine for Students, 17(2), 22–26.

    Article  Google Scholar 

  • Kittur, A., Nickerson, J. V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., et al. (2013). The future of crowd work. In C. Lampe & L. Terveen (Eds.), Proceedings of the 2013 Conference on Computer-Supported Cooperative Work and Social Computing, CSCW ‘13, San Antonio, USA, 23–27 February 2013 (pp. 1301–1318). New York: ACM Press.

    Google Scholar 

  • Kuikkaniemi, K., Jacucci, G., Turpeinen, M., Hoggan, E., & Müller, J. (2011). From space to stage: How interactive screens will change urban life. Computer, 44(6), 40–47.

    Article  Google Scholar 

  • Ludwig, T., Kotthaus, C., Reuter, C., van Dongen, S., & Pipek, V. (2017). Situated crowdsourcing during disasters: Managing the tasks of spontaneous volunteers through public displays. International Journal of Human-Computer Studies, 102, 103–121.

    Article  Google Scholar 

  • Malone, T. W., Laubacher, R., & Dellarocas, C. (2010). The collective intelligence genome. MIT Sloan Management Review, 51(3), 21–31.

    Google Scholar 

  • Martin, D., Hanrahan, B. V., O’Neill, J., & Gupta, N. (2014). Being a turker. In M. R. Morris, & M. Reddy (Eds.), Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW ‘14, Baltimore, USA, 15–19 February 2013 (pp. 224–235). New York: ACM Press.

    Google Scholar 

  • Martin, D., O’Neill, J., Gupta, N., & Hanrahan, B. V. (2016). Turking in a global labour market. Computer Supported Cooperative Work, 25(1), 39–77.

    Article  Google Scholar 

  • Müller, J., Alt, F., Michelis, D., & Schmidt, A. (2010). Requirements and design space for interactive public displays. In Smeulders, A. (Ed.), Proceedings of the 18th ACM International Conference on Multimedia, MM ‘10, Firenze, Italy, 25–29 October 2010 (pp. 1285–1294). New York: ACM Press.

    Google Scholar 

  • Müller, J., Wilmsmann, D., Exeler, J., Buzeck, M., Schmidt, A., Jay, T., et al. (2009). Display blindness: The effect of expectations on attention towards digital signage. In H. Tokuda, M. Beigl, A. Friday, A. J. Brush, & Y. Tobe (Eds.), Proceedings of the International Conference on Pervasive Computing, Pervasive ‘09, Nara, Japan, 17–20 May 2010 (pp. 1–8). Berlin, Heidelberg: Springer.

    Google Scholar 

  • Peltonen, P., Kurvinen, E., Salovaara, A., Jacucci, G., Ilmonen, T., Evans, J., et al. (2008). It’s mine, don’t touch!: Interactions at a large multi-touch display. In D. Tan (Ed.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ‘08, Florence, Italy, 5–10 April 2008 (pp. 1285–1294). New York: ACM Press.

    Google Scholar 

  • Retelny, D., Robaszkiewicz, S., To, A., Lasecki, W. S., Patel, J.. Rahmati, N., et al. (2014). Expert crowdsourcing with flash teams. In M. Dontcheva & D. Wigdor (Eds.), Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ‘14, Honolulu, USA, 5–8 October 2014 (pp. 75–85). New York: ACM Press.

    Google Scholar 

  • Salehi, N., McCabe, A., Valentine, M., & Bernstein, M. (2017). Huddler: Convening stable and familiar crowd teams despite unpredictable availability. In L. Barkhuus, M. Borges, & W. Kellogg (Eds.), Proceedings of the 2017 ACM Conference on Computer-Supported Cooperative Work and Social Computing, CSCW ‘17, San Francisco, USA, 25 February–1 March 2017 (pp. 1700–1713). New York: ACM Press.

    Google Scholar 

  • Saxton, G. D., Onook, O., & Kishore, R. (2013). Rules of crowdsourcing: Models, issues, and systems of control. Information Systems Management, 30(1), 2–20.

    Article  Google Scholar 

  • Schmitz, H., & Lykourentzou, I. (2018, January). Online sequencing of non-decomposable macrotasks in expert crowdsourcing. ACM Transactions on Social Computing, 1, 1, Article 1, 33 pp. https://doi.org/10.1145/3140459.

    Article  Google Scholar 

  • Shen, C., Everitt, K., & Ryall, K. (2003). UbiTable: Impromptu face-to-face collaboration on horizontal interactive surfaces. In A. K. Dey, A. Schimdt, & J. F. McCarthy (Eds.), Proceedings of the International Conference on Ubiquitous Computing, UbiComp ‘03, Seattle, USA, 12–15 October 2003 (pp. 281–288). Berlin, Heidelberg: Springer.

    Google Scholar 

  • Shirky, C. (2010). Cognitive surplus: How technology makes consumers into collaborators. Penguin.

    Google Scholar 

  • Terrenghi, L., Quigley, A., & Dix, A. (2009). A taxonomy for and analysis of multi-person-display ecosystems. Personal and Ubiquitous Computing, 13(8), 583–598.

    Article  Google Scholar 

  • Weißker, T., Berst, A., Hartmann, J., & Echtler, F. (2016). The massive mobile multiuser framework: Enabling ad-hoc realtime interaction on public displays with mobile devices. In J. Muller & N. Memarovic (Eds.), Proceedings of the 5th ACM International Symposium on Pervasive Displays, PerDis ‘16, Oulu, Finland, 20–22 June 2016 (pp. 168–174). New York: ACM Press.

    Google Scholar 

Download references

Acknowledgements

This work is partially funded by the Academy of Finland (286386-CPDSS, 285459-iSCIENCE), the European Commission (Grant 6AIKA-A71143-AKAI), and Marie Skłodowska-Curie Actions (645706-GRAGE).

Author information

Authors and Affiliations

  1. School of Computing and Information Systems, The University of Melbourne, Parkville, VIC, 3010, Australia

    Jorge Goncalves & Niels van Berkel

  2. Center for Ubiquitous Computing, University of Oulu, Pentti Kaiteran Katu 1, P.O. Box 4500, 90014, Oulu, Finland

    Simo Hosio & Simon Klakegg

Authors
  1. Jorge Goncalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simo Hosio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Niels van Berkel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Simon Klakegg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jorge Goncalves .

Editor information

Editors and Affiliations

  1. Eindhoven University of Technology, Eindhoven, The Netherlands

    Vassillis-Javed Khan

  2. Xi’an Jiaotong-Liverpool University, Suzhou, China

    Konstantinos Papangelis

  3. Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands

    Ioanna Lykourentzou

  4. Department of Industrial Design, Eindhoven University of Technology, Eindhoven, The Netherlands

    Panos Markopoulos

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Goncalves, J., Hosio, S., van Berkel, N., Klakegg, S. (2019). Addressing Cooperation Issues in Situated Crowdsourcing. In: Khan, VJ., Papangelis, K., Lykourentzou, I., Markopoulos, P. (eds) Macrotask Crowdsourcing. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-030-12334-5_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-12334-5_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-12333-8

  • Online ISBN: 978-3-030-12334-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation