Simo Hosio
Denzil Ferreira
Vassilis Kostakos
ABSTRACT
Research is increasingly highlighting the potential for situated crowdsourcing to overcome some crucial limitations of online crowdsourcing. However, it remains unclear whether a situated crowdsourcing market can be sustained, and whether worker supply responds to price-setting in such a market. Our work is the first to systematically investigate workers' behaviour and response to economic incentives in a situated crowdsourcing market. We show that the market-based model is a sustainable approach to recruiting workers and obtaining situated crowdsourcing contributions. We also show that the price mechanism is a very effective tool for adjusting the supply of labour in a situated crowdsourcing market. Our work advances the body of work investigating situated crowdsourcing.
- Alt, F., Shirazi, A., Schmidt, A., Kramer, U. and Nawaz, Z. Location-based crowdsourcing: extending crowdsourcing to the real world. Proc. NordiCHI 2010, ACM (2010), 13--22. Google Scholar
Digital Library
- Ariely, D. Predictably Irrational: The Hidden Forces That Shape Our Decisions. London: HarperCollins (2009).Google Scholar
- Arslan, M.Y., Singh, I., Singh, S., Madhyastha, H.V., Sundaresan, K. and Krishnamurthy, S.V. Computing while charging: building a distributed computing infrastructure using smartphones. Proc. CoNEXT 2012, ACM (2012), 193--204. Google ScholarDigital Library
- Brignull, H. and Rogers, Y. Enticing people to interact with large public displays in public spaces. Proc. INTERACT, Vol. 3 (2003), 17--24.Google Scholar
- Burke, J.A., Estrin, D., Hansen, M., Parker, A., Ramanathan, N., Reddy, S. and Srivastava, M.B. Participatory sensing. In Workshop on World-SensorWeb: Mobile Device Centric Sensory Networks and Applications at Sensys (2006).Google Scholar
- Ekman, P. and Friesen, W.V. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology 17, 2 (1971), 124--129.Google ScholarCross Ref
- Goncalves, J., Ferreira, D., Hosio, S., Liu, Y., Rogstadius, J., Kukka, H. and Kostakos, V. Crowdsourcing on the Spot: Altruistic use of Public Displays, Feasibility, Performance, and Behaviours. Proc. UbiComp 2013, ACM (2013), 753--762. Google ScholarDigital Library
- Goncalves, J., Hosio, S., Ferreira, D. and Kostakos, V. Game of Words: Tagging Places through Crowdsourcing on Public Displays. Proc. DIS 2014, ACM (2014), 705--714. Google ScholarDigital Library
- Goncalves, J., Pandab, P., Ferreira, D., Ghahramani, M., Zhao, G., Kostakos, V. Projective Testing of Diurnal Collective Emotion, Proc. Ubicomp 2014, ACM (2014). Google ScholarDigital Library
- Gupta, A., Thies, W., Cutrell, E. and Balakrishnan, R. mClerk: enabling mobile crowdsourcing in developing regions. Proc. CHI 2012, ACM (2012), 1843--1852. Google ScholarDigital Library
- Heimerl, K., Gawalt, B., Chen, K., Parikh, T. and Hartmann, B. CommunitySourcing: engaging local crowds to perform expert work via physical kiosks. Proc. CHI 2012, ACM (2012), 1539--1548. Google ScholarDigital Library
- Horton, J. J., and Chilton, L. The Labor Economics of Paid Crowdsourcing. Proc. ACM-EC 2010, ACM (2010), 209--218. Google ScholarDigital Library
- Hosio, S., Goncalves, J., Kostakos, V., Riekki, J. Exploring Civic Engagement on Public Displays. In S. Saeed (Ed.), User-Centric Technology Design for Nonprofit and Civic Engagements. Springer International Publishing (2014), 91--111.Google Scholar
- Hosio, S., Goncalves, J., Kukka, H., Chamberlain, A., Malizia, A. What's in it for me: Exploring the RealWorld Value Proposition of Pervasive Displays. Proc. PerDis'14, ACM (2014), 174--179. Google ScholarDigital Library
- Kittur, A., Nickerson, J.V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M. and Horton, J. The future of crowd work. Proc. CSCW '13, ACM (2013), 1301--1318. Google ScholarDigital Library
- Kochhar, S., Mazzocchi, S. and Paritosh, P. The anatomy of a large-scale human computation engine. Proc. HCOMP 2010, ACM (2010), 10--17. Google ScholarDigital Library
- Lampe, C., Zube, P., Lee, J., Park, C.H. and Johnston, E. Crowdsourcing civility: A natural experiment examining the effects of distributed moderation in online forums. Government Information Quarterly 31, 2 (2014), 317--326.Google ScholarCross Ref
- Lehdonvirta, V. and Castronova, E. Virtual Economies: Design and Analysis. Cambridge, MA: MIT Press (2014). Google ScholarDigital Library
- Liu, Y., Lehdonvirta, V., Alexandrova, T. & Nakajima, T. (2012) Drawing on Mobile Crowds via Social Media. Case UbiAsk: Image Based Mobile Social Search Across Languages. Proc. Multimedia Systems 18, 1 (2012), 53--56.Google ScholarCross Ref
- LocalMind, URL: http://www.localmind.com/, cited 04/04/2014Google Scholar
- Martin, D., Hanrahan, B.V., O'Neill, J. and Gupta, N. Being a turker. Proc. CSCW '14, ACM (2014), 224--235. Google ScholarDigital Library
- Müller, J., Alt, F., Michelis, D. and Schmidt, A. Requirements and design space for interactive public displays. Proc. Multimedia 2014, ACM (2010), 12851294. Google ScholarDigital Library
- Paulos, E., Honicky, R.J. and Hooker, B. Citizen science: Enabling participatory urbanism. In M. Foth (ed), Handbook of Research on Urban Informatics: The Practice and Promise of the Real-Time City. IGI Global (2008), 414--436.Google Scholar
- Quinn, A.J. and Bederson, B.B. Human computation: a survey and taxonomy of a growing field. Proc. CHI 2011, ACM (2011), 1403--1412. Google ScholarDigital Library
- Rashtchian, C., Young, P., Hodosh, M. and Hockenmaier, J. Collecting image annotations using Amazon's Mechanical Turk. Proc. NAACL HLT 2010 Workshop CSLDAMT '10, ACL (2010), 139--147. Google ScholarDigital Library
- Rogstadius, J., Kostakos, V., Kittur, A., Smus, B., Laredo, J. and Vukovic, M. An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets. Proc. ICWSM 2011, AAAI (2011), 321--328.Google Scholar
- Sauro, J. A Practical Guide to the System Usability Scale (SUS): Background, Benchmarks & Best Practices. Measuring Usability LLC, Denver, (2011)Google Scholar
- Sauro, J. http://www.measuringusability.com/sus.php, cited: 08/04/2014Google Scholar
- SureLock | Mobile Device Lockdown. http://www.42gears.com/surelock/, cited 04/04/2014Google Scholar
- Takayama, C. & Lehdonvirta, V. EcoIsland: A System For Persuading Users To Reduce CO2 Emissions. In Pervasive Persuasive Technology and Environmental Sustainability Workshop, Proc. Pervasive 2008 Workshop Proceedings, Springer (2008), 113--119.Google Scholar
- Väätäjä, H., Vainio, T., Sirkkunen, E. and Salo, K. Crowdsourced news reporting: supporting news content creation with mobile phones. Proc. MobileHCI 2011, ACM (2011), 435--444. Google ScholarDigital Library
- Vukovic, M. and Kumara, S. Second international workshop on ubiquitous crowdsourcing: towards a platform for crowd computing. Proc. UbiComp 2011, ACM (2011), 617--618. Google ScholarDigital Library
- Wang, Y. and Mainwaring, S. Human-Currency Interaction: Learning from Virtual Currency Use in China. Proc. CHI 2008, ACM (2008), 25--28 Google ScholarDigital Library
- Zaidan, O.F. and Callison-Burch, C. Crowdsourcing translation: Professional quality from non-professionals. Proc. Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, ACL (2011) 1220--1229. Google ScholarDigital Library
- Zelizer, V. A. The Social Meaning of Money, 2nd ed. Princeton: Princeton University Press (1994).Google Scholar
Index Terms
- Situated crowdsourcing using a market model
Recommendations
Bazaar: a situated crowdsourcing market
UbiComp/ISWC'15 Adjunct: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable ComputersIn this paper we present a situated crowdsourcing market named Bazaar. Our platform enables researchers to deploy crowdsourcing tasks on simple crowdsourcing kiosks, which can then be placed in different locations. We describe the different components ...
Turking in a Global Labour Market
This paper examines how working in the global labour market of Amazon Mechanical Turk (AMT) impacts upon and is understood by two different groups of workers. To do this we report on two qualitative studies; one of US and another of Indian crowdworkers (...
A Model of Economy Operation Under Currency Market Rate
AbstractA model of functioning of an open economy with a currency market is presented. It allows determining macroeconomic indicators in the medium term with the Central Bank monetary regulation of the amount of money in circulation, interest rates, and ...
Comments