1 Introduction

Crowdsourcing is a technique for completing work by distributing parallelisable tasks to a pool of individuals (often referred to as ‘the crowd’ or ‘workers’) [11, 33]. While this traditionally takes place online, more recently public displays (PD) have been adopted for a variety of different crowdsourcing use cases. A key characteristic making displays an interesting approach for eliciting data is their inherent capacity to tap into people’s idle time. In other words, people stumble upon the displays and start working, often out of curiosity [28]. The users could be people with knowledge about the local environment, untapped human potential, or specific individuals with specific skill sets [15, 28]. Central to many crowdsourcing applications is the acquisition, storage, and processing of data from research participants. One interesting research direction where data generally perceived as sensitive is of paramount importance is digital health and, more specifically, automated approaches to detect or monitor health conditions using media (images, video). The dearth of public datasets containing suitable tagged media for this purpose is a key bottleneck slowing down research in the intersection of automation and health [55].

In this paper, we investigate the potential of interactive public displays in collecting sensitive data, in the form of tagged media, from passersby. Related to the challenge described above, computer vision depends on such high-quality training data and metadata, and public displays may well offer an avenue worth exploring for the collection of ecologically valid and accurately tagged media. At the same time, the approach acts as an excellent catalyst for ethics, consent, and privacy-related discussions and for contemplating futuristic application scenarios.

We created a public display deployment for collecting ‘selfie videos’ from passersby, initially presented in [23]. Our application asks users to donate a 15-s video selfie. Then, the users are encouraged to augment the video with additional metadata by answering a few questions. There exist interesting scenarios that could be enabled by this type of data collection, such as training algorithms to detect and monitor health conditions—according to latest research, up to 30 different symptoms and medical conditions can be detected from observing human faces with a camera [52]. The key contributions of our work are as follows:

  • A contribution to crowdsourcing literature, by investigating the use of public display interactive technologies to collect potentially sensitive data.

  • A dynamic, easy-to install setup to collect media files that are tagged with biometric metadata.

  • A feasibility study that analyses the collected material and highlights important contextual aspects that must be considered in future deployments.

  • A narrative transportation study to further understand aspects of the data collection concept that need to be considered in future digital research ecosystems that exploit situated technologies as citizen-facing data sources.

To obtain a rich set of findings, we employed two methodologically complementary studies: a field study [4] on a university campus with kiosk-sized displays and a subsequent online study to complement the findings using narrative transportation [35]. In the first study (presented in [23]), our display setup was deployed in the wild for a period of 61 days where we collected 199 15-s video selfies, 78 of which had voluntarily labelled metadata attached to them by participants. The validity of the collected videos for machine vision research was investigated in two ways: algorithmically using frame analysis and by manual inspection by machine vision experts. In the second study, we conducted an in-depth online narrative transportation with 53 participants using a crowdsourcing marketplace, Prolific [48]. This study was conducted to understand key issues raised in the previous study and to provide supplementary data. The combined results attest to the validity of the chosen approach, while highlighting a number of challenges and opportunities that can help design and discuss pervasive data collection infrastructures.

2 Related work

2.1 Data collection on public displays

Public displays have in recent years become popular means of conducting research, due to their low barrier of entry especially in situations where the tasks are simple and require little effort to complete. Their visibility makes them a viable choice for creating awareness as well as inviting passersby for voluntary contributions [26, 42, 46]. An attribute making public displays particularly suitable for data collection for a variety of use cases is their serendipitous nature of attracting passersby and their attention [16, 26, 34, 42]. This makes them an excellent tool for collecting data in an ad hoc manner from passersby who are willing to donate their time and effort. Public spaces where PDs are often deployed are known for attracting various audiences who differ in interests, age, and experience with technology and who will engage in spontaneous and often unpredictable activities, both individually and in groups [22].

To optimise the use of public displays, Alt et al. [3] suggest that public displays should be designed to fit both the social and the cultural context of the community where it will be deployed. Doing so ensures data quality and optimises for accuracy. Several types of data can be collected using public displays. While it is common to use touch screens for input, also more novel interaction modalities such as gaze detection can be employed [32]. Public displays have been used to collect large-scale civic polling data through open-ended long format answers [27] or even classified ads directly using the screen [2]. Playfulness plays a crucial role in public display interactions and can be used in designing engaging applications [54]. For example, collecting situated snapshots by using the displays as camera devices have been a class of well-received applications [38]. The mirror metaphor, which entails participants seeing themselves or some parts of the screen responding to their body movement, seems to be a compelling playful design element [43].

2.2 Situated crowdsourcing

Originally coined as a way to distribute large tasks as smaller chunks to online crowds, crowdsourcing (CS) is now the primary means to collect high-quality data from human subjects at scale [47]. The online labour force is, however, highly self-selected and the markets where on-demand labour can be purchased are typically by design limited to specific types of work [48]. A complement to online crowdsourcing is situated crowdsourcing, referring to purpose-built deployments and methods to elicit batches of contributions for highly specific tasks in typically geo-fenced contexts [13, 14]. Public interactive displays excel in this type of work, due to their inherent characteristics of being location-bound and capable of self-advertising their contents to any potential curious passersby (e.g. [17, 21, 25, 26, 42]).

Over the past decade, numerous crowdsourcing deployments have exploited public interactive screens. Just to provide one notable example, Umati is an augmented vending machine that was deployed on a university campus to crowdsource contributions from the student community [21]. The non-financial rewards (snacks from the vending machine itself) were found as sufficient to elicit high-quality contributions in expert-level work of grading exams. Further, entire market-like situated deployments have been developed to elicit work from visitors and dwellers of university campuses to a dynamic array of momentarily available and diverse tasks (e.g. Bazaar [28]).

2.3 Data management and ethics issues

As briefly mentioned earlier, one visionary application domain of computer vision is health care and health monitoring applications in general. Much progress has already been made in detecting health conditions based on video material or image frames [52], and more is expected to rapidly follow. Such future applications, however, do not necessarily need to only rely on already available training data but can increasingly tap into real-time, ubiquitous data collection methods. These digital data collection ecosystems are, as an unfortunate side effect, rapidly antiquating the concept of informed consent [9, 45]—the very ethical cornerstone of any research dealing with human contributions. As data collection is increasingly automated, human subjects may not be aware of data collection, how the data is processed, used as training data, or even monetised [9]. Concerning medical ethics in particular, participants may feel obliged to donate data under the current circumstances (e.g. enrolled in a study as a patient-participant) but reconsider afterwards [8]. How can one even give a truly informed consent, when algorithms mine data for presently unknown anomalies? All this calls for a change in data management to a more human-centric viewpoint [51] as well as thoroughly debating ethical issues with the data subjects themselves. Our research setup allows for post hoc exploration of these issues with people who already donated data.

2.4 Narrative transportation

Narrative transportation enables participants to engage in a story, which changes their attitudes and intentions to reflect the circumstances of the given story narrative [35]. The effect of narrative transportation has been employed in a number of different domains, primarily in the area of Communication and Psychology (e.g. [40]). For instance, an increasing number of research papers suggest that smoking displayed in movies increases smoking behaviour among adolescents [41]. Researchers have also actively applied narrative transportation as a method of scientific inquiry. In a study investigating prejudices, Johnson et al. presented participants with a narrative text describing Arab-Muslim culture containing a number of counter-stereotypical exemplars [31]. Following this, participants’ prejudice against Arab-Muslims was measured using an Implicit Association Test (IAT) [18]. The results show that both implicit and explicit prejudice was decreased among participants which read the full narrative as compared with those presented with a condensed narrative or no narrative at all. Furthermore, participants report increased empathy for Arab-Muslims. Similar results were reported by Johnson et al. on the perception of different races [30].

A study by Grizzard et al. revealed that playing a game in a guilt-inducing condition results in increased moral sensitivity [19]. Participants played the game either in the role of terrorist (guilt inducing) or the role of UN soldier (control condition). Participants’ level of guilt increased significantly when engaging in unjustified violence in the video game. The aforementioned examples indicate that it is possible for people’s moral standards and convictions to change in a short period of time following a transportative experience. The prior work with narrative transportation suggests that people who are sufficiently immersed in the scenario or story alter their feelings and even decision-making, based on the narrative. For the same reasons, we find it fascinating to explore and demonstrate how, methodologically speaking, the concept of narrative transportation may apply to the field of Human-Computer Interaction.

3 System design

In this section, we present the physical apparatus hosting VideoSourcing. In Study 1, the deployment was used in a field study, and the narrative in Study 2 presented a single encounter with this system.

3.1 The physical setup

For the studies we manufactured a desk with adjustable height and a circular wooden tabletop (60 cm in diameter) that hosts three Android tablet mounts. The tablet devices are enclosed in a metal casing (see Fig. 1) and positioned at 120∘ from each other, facing outward from the table. This setup makes it possible for 1–3 people to use the desk at the same time. Users cannot, however, easily see the screens of other users without consciously making an effort to peek by moving aside. We purchased a prepaid SIM card with unlimited data plan and used our own router, so that the deployment depended only on access to power and would not suffer from WiFi outages or poor connection quality. Similar setups have been used successfully for several situated studies, such as in the case of UbiTable to support easy access to extemporised face-to-face collaborations for small groups of people [49] or TeamSourcer in exploring team dynamics in a collocated crowdsourcing setup [25].

Fig. 1
figure 1

The VideoSourcing desk deployed at the corridor of our campus

Full size image

While there is no critical reason to feature three separate tablets in our studies, and it would have been trivial to not include more than one tablet on the desk for the study, we opted to use three tablets for two reasons. First, our goal was to study feasibility from the viewpoint of media quality and not just to see whether people would be willing to donate their data. To this end, we hypothesised that having the displays—the front cameras of the mounted Android tablets—facing in different directions would help us explore the potential effect of different backgrounds that end up in the collected videos. Second, a desk with three tablets is a more disruptive element on the corridor, potentially evoking curiosity—typically attracting higher levels of engagement [29]. In a way, this can be thought of as introducing a novelty effect on purpose, as our goal was to make people notice the deployment and our call to action.

3.2 VideoSourcing application

We designed and implemented an Android application VideoSourcing to facilitate the data collection. VideoSourcing is designed to be run on tablet devices that would later act as our public kiosk-sized displays (see Fig. 1).

3.2.1 Capturing video selfies

The user interface (UI) of VideoSourcing is straightforward, consisting of three stages, or activities, as depicted in Fig. 2.

Fig. 2
figure 2

The user interface of our data collection application. From left: the front-facing splash screen with a call to action and a mandatory consent request, a video capture window with a countdown to zero, and a metadata entry screen to tag the video with additional information

Full size image

First, the welcome screen features a large call-to-action as the heading with a smaller subheading which simply invites users to ‘participate to win’ and help us in our research. Second, the screen contains two text areas that can be dynamically configured to provide different task instructions at different times. For instance, based on identified gaps in a dataset, researchers can adjust the data that is being requested from participants. However, for this study, the task was simply to ‘shoot a 15-s video selfie’ and ‘just be yourself,’ as can be seen in Fig. 2.

Most importantly, the welcome screen features a mandatory consent check box: any participant wishing to proceed from this screen has to toggle the checkbox (unchecked by default), to give their consent for donating the experiment data for scientific purposes. A link to a more nuanced data and privacy disclaimer (pop-up) as well as the e-mail address of the responsible researcher was included in the footer of the screen for participants to contact for questions or data removal requests.

After tapping the ‘start task’ button in the welcome screen, a 3-s countdown timer is started. After the countdown is complete, the video recording begins. The user performs whatever task was assigned on the previous screen here (in this case, just shooting a 15-s selfie with instructions to ‘be yourself’). A new timer is then displayed at the top left corner of the screen informing the user of the time spent on performing the assigned task. After the duration set by the researcher has run out, the video recording stops (automatically) and the screen transitions to the final stage, which entails the collection of metadata. The video is first saved locally and then immediately uploaded to an Amazon S3 bucket, while the user is still in the metadata entry stage.

In the metadata entry screen, users are asked to provide basic demographic data (age, gender) and, for this study, we configured the screen to ask height, weight, and a self-reported truthfulness value about the reported height and weight (using a scale from 1 to 5). We are aware that anyone can simply enter erroneous or mock data to all of this, but we were simply curious what would happen if we allow people to enter their personal data and then be conscious about its truthfulness, given how people tend to provide false data on their own height and weight online [44]. Further, using an open-ended text, we asked the users to describe shortly, in one or two words, their current mood. Again such mood information and, to a degree, also the truthfulness information are interesting from the point of potentially teaching computers to detect these characteristics from faces in videos.

The customisability of VideoSourcing makes it a unique tool for conducting studies by dynamically changing labels and texts within the application. This decreases the need for manual maintenance or installation of new software, which can be surprisingly laborious with situated technologies. Furthermore, it provides a means to study different phenomena at different times, or based on user interaction, depending on the researcher’s interests.

3.2.2 Third party components

VideoSourcing (initially presented in [23]) integrates the Amazon Web Services (AWS) SDK for Android to facilitate a robust, secure, and persistent storage by using a write-only API key. The metadata that links to each of the uploaded video files is stored in a MongoDB database. The linking is based on a unique identifier, generated by VideoSourcing application, that is passed in the metadata and forms a part of the filenames of the videos. To facilitate remote configuration (dynamic features, as discussed next), the application also integrates the OneSignalFootnote 1 SDK.

3.2.3 Dynamic features

VideoSourcing was designed to be usable in different types of video collection studies far beyond this initial feasibility study. To this end, several parts of the UI are remotely configurable via push messages triggered through the OneSignal web dashboard or programmatically using the OneSignal API. The different configurable text fields are injected in the notifications as additional payload, and the moment that the device where VideoSourcing is installed receives the notification, i.e. the user does not have to even open the notification, VideoSourcing is configured with the new dynamic values. First, the task description (title + instructions) in the welcome screen can be configured to encourage the user to perform a specific action or task while taking the video. Second, in the metadata entry screen, both the slider and the open-ended items can be configured remotely. This way, the user can be asked one numerical value question (from 1–5) and one open-ended question. For instance, in this study, we asked about the truthfulness of self-reported height and weight as well as a short open-ended description of the participant’s current mood.

4 Study 1: A field study with videosourcing

4.1 A pilot experiment

Before commencing a longer deployment, we validated the technical solution and feasibility of data collection in two short studies: (1) as an example of a technology probe during a lecture on Human-Computer Interaction, and (2) as a standalone installation in an open laboratory space within our own research premises. We used the deployment at this stage to undergo the local ethical review process, as we were not planning to change any major aspects of the study design itself. As a result, we included more comprehensive disclaimers to the application. During these studies, we discovered technical problems such as loss of data due to the devices not staying connected to the public WiFi network and minor usability problems with the screen flow of the application itself (e.g. ambiguous wording in the final submit button).

In these pre-studies, we collected 11 videos and most likely lost several more due to the aforementioned technical challenges. The 11 participants (10 male, 1 female) also responded to a brief online questionnaire about the deployment. In the invitation, we clarified that the responses were not going to be connected with any of the data they donated earlier in the app itself and that the purpose of the study was to inform our longer future field study. From these results, we note two interesting findings: (1) the considered monetary value of their data (video and metadata combined) to range between 0 and 5 euros and (2) at least one of the respondents was willing to donate all of their data, including information on matters considered highly sensitive such as STDs and even serious health conditions. Insights like these were useful in designing the final questionnaire for the field deployment as well as in ensuring that the concept is feasible in general.

4.2 The longitudinal deployment

Shortly after the pilot study and making small modifications based on the early findings to the setup itself and to the online questionnaire, we launched a field study on our University campus. Field studies with public displays are particularly important, since only with field studies can researchers really delve into matters that manifest only in natural settings such as privacy considerations of authentic responses to, e.g. questionnaires that are parts of the deployment [4]. To encourage passersby to interact with the setup, we provided printed A4-posters about a raffle that would take place after the study is over. We did not specify any guaranteed reward for everyone nor did we specify exactly how many lunch vouchers would we raffle. The desk was positioned at a somewhat quiet spot along a corridor so that it would not disturb the passersby too much by, e.g. blocking a passageway. To ensure that the passersby could not tamper with the system and exit the VideoSourcing application, we used the SureLockFootnote 2 kiosk software for Android.

4.2.1 Additional data collection

For the participants who provided their e-mail addresses in the metadata entry screen, we e-mailed a separate invite for the final online questionnaire. The invite was automatically sent via a dedicated Gmail account for this study and by using a ZapierFootnote 3 automation flow (when a new document was added to the MongoDB, Zapier would extract it and trigger an outgoing e-mail in Gmail). The questionnaire was hosted in Google Forms where users could contribute fully anonymously to avoid any ethical issues and contained items as follows.

  • Basic demographic details (gender, age).

  • Perceived monetary value of the contributed data, separated by purpose of use: non-profit academic use or any use, including commercial purposes.

  • Choosing among a list of different types of data types that the participant would feel comfortable to disclose: age, gender, origin country, height, weight, sexual orientation, mood (happy, sad, neutral, etc.), non-serious medical conditions (flu symptoms, fever, etc.), serious health conditions (cancer, severe depression, diabetes, etc.), family details (number of children, married status, etc.), nothing or none of the aforementioned, and an open-ended option (‘Other...’).

  • Elaborating on the previous item: why and what?

  • Open-ended text fields specifically asking the participant’s opinions on any potential ethical issues of the setup as well as any perceived dangers or opportunities afforded by the data collection.

Further, we probed for which purposes would the user be willing to donate data with a similar setup in the future.

  • Strictly academic, non-profit research.

  • For commercial research (companies train algorithms that make money).

  • As part of a public free online dataset that contains everything—the video itself and all the data you provided us (age, gender, height, weight, ...).

  • Basically for anything, but I want to be be compensated somehow (e.g. entering a prize draw, money, other rewards).

  • Basically for anything, no compensation needed.

  • Nothing, or none of the above.

  • Other...

5 Study 2: Online narrative transportation

5.1 Rationale and setup

We conducted a subsequent online narrative transportation study to elicit further insights on specific issues that emerged in Study 1 and, in our judgement, warranted further input due to the relatively low initial amount of input from the participants. Additionally, in Study 1, we were only able to obtain input from people who already used the deployment. We wanted also to reach out to people who would, under no circumstances, use the deployment. To this end, we administered a questionnaire again using Google Forms to users of Prolific, a crowdsourcing marketplace designed specifically for behavioural experiments by researchers [48]. First, we presented the setup in its entirety in the first section of the form along with a narrative of the scenario that participants in the first study experienced. The narrative described how the user first encounters the display (including a picture, Fig. 1 in this paper, introduced the application screen briefly, with the screenshots from Fig. 2. Then, the participants clicked proceed, to start the actual questionnaire. The narrative was articulated as follows.

Imagine the following scenario:

You are walking by yourself on a public campus, e.g. a university campus, and encounter a public technology deployment, as depicted in the photo below.

You are not in a hurry, and as you have time to spare, decide to approach the device. There is a touch-screen tablet mounted to the device, showing an application which asks you to take a 15-second selfie video.

The application explains that by taking this video you would help local research and researchers.

– The image of the desk, Fig. 1 in this paper, was placed here –

The application you find running on one of the tablets consists of three screens (as shown from left to right in the picture below):

1) An introduction screen, which invites people to participate, offering a raffle (prize draw) as an incentive for participation. Most importantly, the application requires you to provide consent for all of the data you’re about to donate to be used for research.

2) A selfie screen. Once you click “start” in the first screen, the application starts capturing the 15-second video selfie and simply shows the view from the front camera. You are therefore essentially looking at your own face on the display.

3) A final questionnaire which asks you to input certain personal data. The application asks you to input data on your height, weight, the truthfulness of your answers to these two previous questions, and a short description of how you are currently feeling.

– The screenshots collage, Fig. 2 in this paper, was placed here –

This is the end of the deployment description. Click to continue and take the questionnaire.

5.2 Data collection

The first section of the questionnaire contained a single 7-point Likert question concerning immersion (how well could the participant relate to the narrative). After this, the second section of the questionnaire contained another single item, a text field on ‘first thoughts,’ to elicit open-ended ideas, commentary, and questions about the use of such situated tech for collecting data. After these initial thoughts were recorded, the subsequent section of the questionnaire contained two open-ended items: (1) what type of data would the participants be happy to donate with the deployment, and (2) what kind of control would the participants want to exercise, as articulated by themselves. In the same section, we inquired about the willingness of the participants to donate different types of data in public with the described setup by using a 5-point Likert scale that required the participants to rate their willingness to donate data for the following data types:

  1. A.

    E-mail

  2. B.

    Current location

  3. C.

    Surname

  4. D.

    Racial information

  5. E.

    Political opinion

  6. F.

    Religion

  7. G.

    Health condition—non serious diseases (e.g. flu)

  8. H.

    Health condition—serious diseases (e.g. cancer, HIV)

These data types were partially informed by the European Union’s General Data Protection Regulation (GDPR), which distinguishes between ‘Personal Data’ (A–C) and ‘Sensitive Personal Data’ (D–F) [10].

Finally, in the last section, we specifically elicited open-ended ideas on five of the issues that we deemed emerging as the most interesting ideas from Study 1 and that in our opinion are key to unlocking the future potential of using situated technologies for collecting potentially sensitive data:

  • Privacy and data control

  • Compensation

  • Institutional trust

  • Role of surrounding context in data collection

  • Thoughts on pervasive data collection infrastructures

6 Results

6.1 Application use data

During Study 1, over the course of 61 days (4 days of pilot study + 57-day field study), we collected a total of 199 selfie videos, corresponding to 3 videos per day. This slightly exceeded our expectations, but at the same time, we must note that our setup does not allow for calculating things like conversion ratio (of passersby or even from people starting to interact with the application) and we will leave this as future work. Further, we received 78 metadata submissions to supplement the videos. 63 of those left their e-mail addresses, and of those 22 proceeded to provide online questionnaire responses (a 35% conversion ratio).

6.2 Video metadata analysis

Concerning the 78 metadata entries (48 male, 20 female, 5 other, 2 not disclosed; mean age 28.6, SD 12.2), we note that 62 people in total left diverse and rich open-ended mood information, such as ‘super hungry,’ ‘normal, little tired,’ ‘fine, a bit confused,’ ‘stressful because of the thesis,’ or ‘A little stressed, assignment due at midnight. slightly amused by the weight truth question :-).’ Seventy-two people provided information on their height and 71 on their weight. The average self-reported truthfulness, with no observed statistically significant differences between genders or age groups, was 5.4 (SD, 0.8).

6.3 Media analysis

In order to statistically assess the quality of the faces collected in the videos and their usability for face biometrics and computer vision in general, we analysed them using a state-of-the-art face detector, based on the SSD-framework [36] and ResNet [20] as implemented in OpenCV [6]. The results of the automatic analysis show that:

  • 179 videos (90% of the total) show a detected face during at least one full second and are thus considered useful for several machine learning tasks as training data

  • 113 videos contain a detected face during 100% of the duration of the video, 145 over 90% and 155 over 80%

  • 20 videos do not contain a single detected face and could be discarded from a possible face database build from our results

While these can be considered remarkably good results and speak for the feasibility of the overall approach, an issue with some of the videos is that users not always show the whole face for the camera. Sometimes the users were short and either did not realise that the tablet stand can be tilted or simply did not bother to tilt it so that the full face would be visible. To assess the impact of this issue, we computed the detected facial sizes and their locations in the screen. In this context, we considered faces to be too small and prone to limited usefulness if their size is below 130 × 130 pixels, and too big and prone to have occlusions or bad camera angles if their size is over 280 × 280 pixels. We also flagged the videos where the detected face is too close to the border since they are very likely to present occlusions or not full faces. This analysis shows that:

  • 32 videos are perfect in terms of face size, screen location, and detection percentage.

  • 39 additional videos have faces with perfect face size and location over 70% of the time.

  • 56 additional videos have perfect segments during at least 5 s.

  • 57 videos have faces positioned too close to the bottom, 22 faces too close to the top, 15 have faces too small (11 of them with good location), and 68 have faces too big (22 with good location).

  • Up to 100 videos could have benefited from better positioning of the user in respect to the camera.

By visually inspecting the videos, we noticed that sub-optimal illumination was a problem in up to 60 of the videos. Of them, 43 videos show different types of spotlights close to the facial area and thus have a limited dynamic range, while 17 additional videos present different problems such as back-light illumination or being captured in the dark. Also, hand movements around face and head caused issues (partially hidden facial features during such movements).

Finally, our design allows the same user to record several videos even in a short time span. We manually identified 47 repeated users in the total 179 videos (132 unique users, 73.7%).

6.4 Study 1 post-questionnaire

6.4.1 Descriptive data

First, it was clear that most people answering the items about monetary value of the data had no real credible insights to offer, as the answers ranged anywhere from zero to hundreds of thousands of dollars, with no convergence points. As for what type of information people would feel comfortable disclosing, we noticed significant differences. Of the 22 respondents, 21 would be willing to reveal their gender and 19 their age. This can be contrasted to e.g. height and weight (15 and 12, respectively), or to sexual orientation and non-serious health conditions (seven respondents for both). Five participants indicated to be willing to disclose serious health conditions and family details. Further, 19 participants were willing to donate data for strictly academic, non-profit research, whereas only five (5) were happy to donate data for commercial purposes. Zero participants chose to agree to ‘for practically anything, for no compensation whatsoever,’ indicating that people seem to perceive at least some value to their data, regardless of their inability to clearly articulate such value in numbers.

6.4.2 Qualitative analysis

First, the respondents provided interesting insights when elaborating on why they would donate or not donate data, and how they could be persuaded to disclose more about themselves. Naturally, many participants hesitated donating data if they do not fully trust the organisation managing the collection or for concerns about the future use of the data, as exemplified by comments such as, ‘I would tell more about myself if I knew exactly how that info will be used.,’ ‘If I trusted the researcher and the organisation that is dealing with my data, I would be willing to tell more.,’ or ‘I am just scared information will go too public and will be bullied.’

Some participants also took a far more nuanced approach in their responses: ‘Age, gender and mood are very general. Also my height and weight aren’t that secure info for me. To make me give more info the use of it has to actually help someone in non monetary way,’ or ‘I would be willing to disclose pretty much anything that doesn’t get me into trouble (e.g. drug use, or other criminal activities that I might be involved in). I wouldn’t want to disclose information that might be used to develop better mass-manipulation method; but if the motives are pure, e.g. to better understand the human condition, then why not. I could be further persuaded by informing me that my disclosures wouldn’t come back to bite me’ and ‘Yes [I do see a problem]. They might get lost or stolen and misused. Also the laws can change in 20 years and my data might be used in ways I didn’t agree. I can also become a public person and this data could be used against me.’

Overall, helping research was seen as a good motive for donating data, however: ‘If it’s used for non-profit research it’s fine, as long as people whose data is gathered are fine with it. If it’s for commercial purpose, I think some kind of compensation for those people is necessary (they helped you to make money after all)’ and ‘If I was told that it was safe to do so and that I wouldn’t be identified I would share pretty much anything for research purposes. For other purposes I would be more careful and would demand more money.’ Further, one participant came up with a suggestion to not only do research but exploit the deployment as a civic feedback medium: ‘This could be a great way for people to express their ideas about issues and not just be about personal data collection.’

Finally, only six of the 22 participants found potential issues or raised related additional questions concerning ethical aspects, such as, ‘However, am I allowed to have access to my data at any point in time should I request it?,’ ‘People might end up sharing more than they are actually willing to and the information might end up somewhere they don’t want to,’ or ‘I would be willing to disclose my age and gender in a non profit or industrial research but I won’t like to disclose my nationality because of the racial reasons where it drags down to deal the data differently.’

6.5 Study 2: Online narrative transport

6.5.1 Participants

We recruited a total of 53 (female 33, male 18, 2 undisclosed) participants from Prolific Academic. The mean age of the participants was 40.2 (SD 13.3). Most of the participants were from The UK, and we used the platform’s exclusion criteria to obtain only participants who spoke English as their first language and who had successfully completed a minimum of 100 assignments on the platform to ensure high-quality input. When asked about their perceived immersion to the story (i.e. their ability to imagine participating in the described scenario), the average rating was 5.3 (SD 1.7), suggesting that the participants were able to immerse themselves into the scenario.

6.5.2 Willingness to donate data

A key element in our questionnaire was a Likert Scale on what type of data the online participants would be willing to donate using our system. To this end, Fig. 3 reveals fairly stark differences in the types. Types such as surname (akin to identity) and serious health conditions are types that participants were not happy to reveal (with only 21% and 32% of the respondents answering either 4 or 5, respectively), while—perhaps surprisingly—they had no problems revealing their religion or racial identity (74% and 72% answering 4 or 5, respectively). A Kruskal-Wallis test also showed a significant difference between the willingness to donate different data types (p< 0.01). Subsequent analyses with Wilcoxon Signed Rank tests revealed that religion and racial identity were both more likely being donated than email, location, serious health conditions, or one’s surname (p< 0.01). Additionally, religion was more likely being donated than non-serious health conditions (p< 0.01); political opinions were more likely being donated than surname or serious health conditions (p< 0.01); non-serious health conditions more likely than surname (p< 0.01); and email more likely than surname (p< 0.01).

Fig. 3
figure 3

Responses to a Likert Scale on the types of data the participants would be willing to donate (1 = not at all willing, 5 = extremely willing)

Full size image

6.5.3 Qualitative analysis methodology

The methodology for analysing the open-ended items in the questionnaire followed key principles of grounded theory and, more specifically, thematic analysis [7]. Our analysis was carried out in two phases. The first phase involved two of the authors working together to identify higher-level themes of the responses that were then confirmed by a third analyst to ensure that we capture the most prevalent themes for each of the items. However, we note that at times the responses clearly answered to more than one of the questionnaire items and contained multiple ideas at the same time, and we therefore chose to not tag each response with a single theme only. Therefore, the themes are not exclusive. In the following, we include 2–3 major emerging themes for each of the analysed items.

6.5.4 Participants’ initial reactions to the narrative

The narrative was able to elicit a wide range of responses from participants, most of which were rich in detail—further indicating that the participants were able to imagine themselves in the described scenario. A large majority of participants had immediately decided for themselves whether or not they would participate in the research. A significant portion of participants expressed an interest in additional information needs before they would be willing to participate. This was evident by participants who expressed excitement over the setup; ‘I would be intrigued, I would definitely investigate further’ or ‘It sounds interesting - I am wondering who the local researchers are, and what they are trying to discover.’ Others straight out rejected the opportunity to participate, citing reasons such as privacy or fear of public humiliation when recording a video of oneself in public. As summarised by one participant; ‘I would not participate for two main reasons. First, I cannot abide being videoed, and second, I would be worried about how my data was going to be used.’

6.5.5 Privacy and control over data

When asked about their expectations in relation to the control and privacy of their data following donation, participant expectations varied considerably. A number of our participants expressed significant concerns on how their data would be handled; ‘Explicit information about who would use it, how it would be anonymised. I’d want the right to access any data retained and have it removed / deleted / destroyed if I so wished.’ On the other hand, an equal portion of participants expressed indifference as to how their data would be handled; ‘I don’t care what the researcher does with my data,’ or expressing a loss of ownership over the data following donation; ‘Once I have given the data it is not mine to control.’

Most participants agreed that data should be used solely by the organisation(s) advertised on the donation screen for a clearly defined and pre-determined (research) purpose. Such clarity could potentially increase the number of people willing to volunteer their time to donate data ‘I would be more willing to donate my data if I had cast iron guarantees that it would only be used for specific research purposes.’ Interestingly enough, the idea of removing one’s data after donation was raised by a number of participants. ‘I always like to have the opportunity to fully withdraw my data if I were to change my mind for some reason. I personally like the idea of contributing to research for advancements in education purposes so I couldn’t see ever requesting my data be returned but I like to have the option.’ This functionality may however be difficult to achieve when data is recorded and stored in a completely anonymous manner.

Concerns around data sharing were focused on the (mis)use of the provided data in the future, potential for personal negative triggers, and the possibility to be identified. Although we did not specifically probe participants on the misuse of data, the issue was raised repeatedly by participants, especially in conjunction with specifying the data type in question in more detail, as discussed next.

6.5.6 Willingness to donate different types of data

Determining which types of (sensitive) data to include and which data types to exclude is a critical decision for any crowdsourcing deployment. We asked participants to describe which data they would (not) donate when faced with a similar deployment as described in the narrative. Although a few participants told us that they would not be willing to donate any data, the majority of participants were open to donating (some of) their data. A few of our participants indicated that all data types were open for data collection; ‘I’d disclose whatever the researcher asked. I love research and I think it’s important. It doesn’t matter that’s it’s in a public setting as people can’t see what I’m writing down.’

Identifiable information was clearly a key source of concern: ‘I would be willing to provide information that would not in the future increment me or cause me to get identified easily. Feelings are very ephemeral, so I would be glad to share that. My date of birth though is not and therefore I would not be glad to share that.’ Although not directly related to participant privacy, questions surrounding participant health may unexpectedly result in personal pain; ‘I wouldn’t for example be happy to disclose a mental health condition or health condition or treatment as it might be a potential trigger.’

Participants largely agree that the sharing of data which is already ‘publicly’ observable and cannot be used to identify an individual is acceptable to collect; ‘I would be happy disclosing ‘personal’ data such as age, height, weight etc, because all those things are publicly available anyway - you only have to look at me to work out more or less what those things are.’ The inclusion of video footage (as in our described scenario) limited some of the data types which participants were willing to ‘link’ to their identifiable video; ‘If it was anonymous I would have no issues, but as I can be identified I think my limit would be age, height, gender, relationship status, employment status, etc.’

6.5.7 Participants’ response to specific targeted inquiry

Sharing data after donating it

Consent in the age of digital data collection is no longer a simple matter [9]. When a participant gives consent to use data for research, what does it actually mean? Digital data is easy to replicate, copy, share, and increasingly use as training data for algorithms that do not necessarily disclose what data were used in creating them. When asked if the researchers overseeing this deployment would be allowed to share the data with other organisations, behind the curtains, the verdict was rather clear. Sharing should not be allowed: ‘Yes it matters- consent should be given in advance to share across countries (any distinguishing data anyways).’ A clear majority of the 53 participants stated rather unambiguously that they should not share, at least not without consent. This is interesting, since nothing would actually prevent a researcher from doing so, given the type of consent our system collects. This is what the consent form on the beginning actually allows: a blanket access for the data to be used ‘for research.’ Indeed, more nuanced consent methods are called for in cases like this: ‘No, unless the person sharing data gives explicit consent and is aware where their data is going.’ Clearly, this works both ways: if consent is given, participants did realise that sharing of data might happen: ‘With your consent, and having made the breadth and limits of their use completely clear, then yes, that sounds OK.’ Two participants also explicitly mentioned that the reputation of the parties that the data are shared with matter a lot, as discussed next.

Equality of institutions

Institutions, even inside academia, are not equal. Once again, these are things that the current consent models typically used in research fail to accommodate. More specifically, there seems to be a dichotomy between private and public institutions, which is something many of the participants paid attention to: ‘I would be more willing to support public sector university research as I have a higher level of trust that proper policies would be in place’ or ‘I would much prefer my data went to public sector researchers.’ But, on the other hand, it is not quite that straightforward and institutions have to safeguard their reputation when it comes to data management: ‘No, all institutions are not equal. Not due to whether they are private or public, but due to what their intentions are regarding using the data, ultimate goals, social responsibility, and ability to safeguard its donors.’

Role of context in donating data

Context significantly impacts decision-making by affecting the evaluative system the decision-maker will follow. Depending on the context in which one finds himself, (be it situational, environmental, social, etc.), one’s decision may differ [53]. Most participants agreed with this line of thought, providing examples of situations or moods that may affect their willingness to donate data. Time pressure and mood (whether happy, angry, or sad) were some of the frequently mentioned reasons why one may donate data; ‘If I was in a bad mood I wouldn’t really be bothered about helping with research.’ Others were uncompromising in their beliefs, ‘No effect. My beliefs about data privacy, data sharing, and exploitation are fixed regardless of the environment or well being.’ Interestingly, according to one participant, the issue of self-esteem is critical to making a decision on the use of such a deployment, ‘I think maybe if you feel down that day, or would have liked to look better, perhaps put makeup on and washed your hair just for your own self-esteem.’

Compensation

Majority of the responses received from participants regarding compensation contained conditions that dictated the need for payments. The purpose of the data collection, the length of the data collection process, and the type of sensitive data collected were the most often mentioned concerns that would impact their preference for compensation: ‘There is no amount of compensation that would persuade me to participate in the setting described initially.’ A lot of emphasis was laid on the purpose of the research (data collection) and its subsequent contribution to society. Many participants expressed their willingness to donate their data without compensation as long as the outcome of the research would contribute positively to society, ‘Contributing to research is enough reward’ while a few others were of the opinion that even for the purposes of research, participants ought to be compensated, ‘I think people should be compensated for research contributions.’ Participants suggested money, free tea/coffee vouchers, and shopping vouchers as means of showing appreciation to donors for their time.

Digital data collection ecosystems

A significant number of participants expressed concern about how their privacy is being invaded and the unease they feel knowing someone somewhere is possibly watching or collecting some data about them. ‘It is unsettling, but it cannot be avoided unless you refuse to use all technology, which is difficult nowadays. You just have to be very careful what you disclose. Data shouldn’t be handed out without care.’ ‘It does freak me out a bit. I used to keep a piece of tape over my laptop camera. I don’t doubt that any information that’s available is being collected, but the illusion of privacy is important to me.’ However, the need to have access to and use modern-day technologies while potentially exposing oneself to possible data exploitation creates a bittersweet experience for some, ‘I don’t like it, but am aware of it. I would prefer it wasn’t the case, but it is an inevitable trade-off at present, in order to have access to technology that I want.’ Some others find no fault with such data collection ecosystems considering the overall good they bring to society. ‘Its just part and parcel of everyday life now. I have no problem with this as the streets, public transport, cars, and driving feel safer now knowing that you can be recorded in public. Its a very useful thing when people go missing or have been assaulted.’ ‘It does not provoke any concern as this is just natural in our current day and age.’ In a way, these sentiments speak for people slowly succumbing to the idea of surveillance. The media is sharing stories of such monitoring of people in public practically on a daily basis, and the younger generations might very well grow up to accept this style of personal data being collected in a truly pervasive fashion.

7 Discussion

With the technical challenges regarding display technologies and input devices gradually diminishing, public displays are currently moving out of research laboratories into the public domain and are commonly used for interactive advertising, directory services, and presenting information and enhancing experiences in a more visual and interactive way. While this transformation can be attributed to the rather rapid advances in the design and development of display and projection technologies as well as input methods that support differing interaction mechanisms, a lot has to do with the expanding knowledge and understanding of the potential that exists for interaction forms and scenarios [15, 22]. Our work is, in a way, a provocation aimed at probing a new use case: collecting sensitive data for research use through public interactive displays.

Overall, our results speak favourably for the feasibility of the described data collection concept. More specifically, the subsequent exploitation of the collected video and metadata in e.g. computer vision research seems to be possible while overall there were certainly problems voiced by our participants. However, the setup is technically robust and people do not hesitate to donate data with it. Further, people are willing to contribute—even when the rewards are not guaranteed: we employed the prize draw model to encourage participation without specifying how many prizes were available. At the same time, we note that systems like this are by definition in the exploration stage and strictly complementary to the tried and tested traditional methods, such as collecting bespoke databases in laboratories or through online systems.

7.1 Broader context

A common criticism towards the legal disclaimers protecting practically all of the contemporary data-intensive platforms online is their poor readability: users do not read them nor do they fully understand them. Who is then to blame when things go wrong with personal data? Prior research has identified a clear division in the sense of responsibility when things backfire: people blame themselves for trusting any of those platforms in the first place and the platforms for poor conduct [12].

Our first study was deployed on a university campus and most likely benefited from the prevailing sense of trust in the academic context. Further, the users of our deployment could not even proceed to the video capture stage before agreeing to our simple but, as far as research goes, all-encompassing disclaimer that we may use all of their data for our own non-profit purposes. An interesting aspect of this is the number of respondents who expressed their willingness to provide much more detailed and sensitive information about themselves as long as they could trust the organisation behind the data collection, the intention of data use, and the security of the platform. This was further highlighted in the findings from Study 2, where several participants indicated that the reputation of the institution matters a lot and that institutions are not equal. Therefore, more granular data management mechanisms must most likely exist in the future for guaranteeing that people can dictate who gets to benefit from their donated data. Nevertheless, we must remember that there is a mismatch between what people say and what they do in relation to data privacy [1].

Further studies (preferably outside the university environment) need to be conducted to ascertain whether our experiences obtained within the academic community apply outside the campus as well. The required human-centric viewpoint in sensitive data collection [5, 51] has yet to be exhaustively discussed within the scope of public displays. We hope that our study is a call to action in itself concerning these aspects; How can we start leveraging public displays ethically for what they really excel in—collecting data serendipitously and largely on autopilot—for purposes beyond one single application at a time? Indeed, the whole prospect of data collection with ubiquitous technologies for common good via open science is an interesting proposition. The big corporations have all the data, and such data can be turned into profit. Is the academic line of work falling behind the curve? Is there a way to collaborate?

7.2 Design implications

The concerns raised by our participants regarding our intentions with the data as well as ethics concerns in general alert us to rethink the user-facing introduction to the concept, the invitation to donate data. Indeed, ‘why’ is more important than ‘what’ [12] in explaining the intricacies of data collection. While a short disclaimer such as ours is perhaps technically all-encompassing, such deployments should in the very beginning of the user interaction flow include a mandatory disclaimer that proactively addresses the why-question. Why exactly is the data collected? And why is all this useful for research? This would in all likelihood increase participation, reduce concerns about data use, and make the collection procedure more ethical and less suspicious in general.

Second, the design itself needs to optimise for the quality of the captured media. To encourage users to take videos where the head is positioned in the middle of the frame and correctly sized, a ‘target’ icon or simply drawing a square on the screen as an overlay would most likely nudge [39] participants to place the face in the centre. The square with a textual hint would help users adjust their distance from the camera and optimise the size of the face visible in the video.

Finally, while we explicitly set out to place our deployment in a well-lit corridor, poor illumination was still a problem in some instances. More specifically, the ceiling lights were often visible in the background, which may degrade the performance of computer vision algorithms. Therefore, similar deployments should be placed in physical contexts where there is ample ambient light available and that would encourage users to stare horizontally at the camera to avoid sub-optimal camera angles.

7.3 Challenges of the setup

Passersby were required to perform the selfie video task and its associated post-task questionnaire with some physical constraints such as mobility, time, and location.

Mobility constraint

Participants were constrained to the physical location of the setup. There was little room for movement as this would affect their visibility in the video. Irrespective of the background lighting or view, users were left with two choices: either participate or not. The weight of the setup also played a critical role in this regard, since it was difficult for one person to easily move it around. This also ensured that the setup could not be moved about by disgruntled users. The structure of the setup also forced participants to stand throughout the entire session. This likely posed a challenge for the physically challenged who are constrained to their wheel chairs and thus unknowingly excludes them from the study.

Time constraint

Time availability on the part of participants is critical to their participation in a deployment such as ours. Considering the location of our setup, passersby encounter it when entering or exiting the university campus. In a rush not to be late for a class or to catch a bus (when exiting) to get home. Such a moment of rush would make it difficult for them to approach the setup even if they wanted to. Passersby with some idle time are more likely to approach and interact with the setup, and participate if interested. Participants of our second study highlighted this challenge.

Location constraint

The setup was located in front of a campus restaurant at one of the entrances to the university campus. This made the setup easy to spot. We discovered however that most of the video submissions were done after campus hours between the hours of 5pm to midnight. Although various reasons could be attested to this, we believe the nature of the task at hand (taking a selfie video) was a contributing factor with participants waiting until they were possibly alone at the location to participate.

7.4 Limitations of the study

Similar to most situated studies, the generalisability of our results is subject to limitations. First, our results are restricted to the context and cultures of the locations where the data was collected. For Study 1 and Study 2, these locations were respectively Finland and the UK. Second, our results of Study 1 are inevitably biased towards those who are willing to interact with a public display. However, results from this part of the population are highly valuable to other researchers as it is exactly this subset of the population on which researchers will rely when deploying in situ crowdsourcing applications.

7.5 Future work

We plan to extend the deployment first by ensuring granular data use rights, by letting the users indicate their preferences prior to taking a video and donating metadata. We will improve passersby tracking (to study conversion rates) and also crowdsource additional labels by automating the labelling process via Amazon’s Mechanical Turk. We envision a longitudinal and distributed deployment across several campuses which provisions an open online database for researchers to use. This will also allow us to explore the differences between situated data collection and traditional ways, such as using social media and online applications.

Another exciting opportunity for research is gauging the actual monetary value people assign to their media and metadata. Given how our participants in Study 1 had difficulties in formulating an estimate about the value, this could be studied, e.g. by using reverse second-prize auctions [37] that are ideal for extracting the real valuation in this type of scenarios of auctioning intangible goods [24, 50].

Finally, a thorough cross-validation of results obtained through narrative transportation and actual field deployments conducted in different setups will be needed in validating the method as a viable tool for HCI researchers.

8 Conclusion

We investigated exploiting interactive kiosk-sized displays for collecting videos, in this case short selfie videos, with metadata. We conducted two complementary studies that included a field study and an online counterpart to elicit further insights to key identified issues from a secondary sample. First, we conclude the approach as feasible since passersby started donating videos, which is essentially quantifiable evidence of the approach producing data. In the second study, we showed how willing people are to donate different data types and present vast amount of qualitative insights around different aspects of such public data collection systems, such as the perceptions of private vs. public organisations as data collectors, role of context, and reuse of the donated data. All this presents provocative ways of looking at pervasive data collection: on the one hand, people are willing to donate data, but on the other hand, the systems should be designed to collect granular data and have clear data management policies attached to them. Together, the findings in the article contribute not only a novel application concept but also a great deal of considerations for the future design of public data collection tools.