The Effects of Natural Sounds and Proxemic Distances on the Perception of a Noisy Domestic Flying Robot

2.1 Existing Solutions to Drone Noise

The noise of interactive domestic drones, which is a major focus of our research, is a consequential sound (see Section 2.2) that is undesirable and considered an obvious disadvantage [11, 14]. Many methods have been tried to reduce the impact of drone noise on humans. Miljković [43] systematically reviewed these methods and proposed multiple concepts for attenuating drone noise, including both passive and active solutions. In the passive noise reduction method, optimizing the shape of propellers to reduce drone noise has been successfully commercialized. A kind of low-noise propeller is currently being sold by DJI as a modified part for their drones [65]. However, this method only reduces the noise level by a few dB [43]. In user comments on the online shopping platform Amazon, many customers have pointed out that the noise reduction effect from the low-noise propeller was not obvious [16]. The active noise canceling (ANC) solution, which emits a same-amplitude sound wave with an inverted phase, could achieve a significant noise canceling effect [47]. However, due to the complexity of the ANC system for drones, it is still in the laboratory research stage and far from being introduced in practice. Also, the need for an additional energy source to power an ANC system limits the scope of this solution to large drones [43]. For small domestic drones, which is the scope of this study, La Delfa et al. commented that these drones’ small size already minimizes propeller noise and airflow, thereby tackling the main pain points in close-range human-drone interaction [34]. However, this is debatable—although smaller drones do emit fewer consequential sounds, making them relatively inoffensive, small drones are by no means silent, and their consequential sound is still noisy and problematic when they are flying indoors, especially in close proximity to humans (by analogy, similar to an insect like a mosquito or a bee flying next to your ear). Therefore, while we agree that smaller flying robots are more suitable for close-range human-drone interaction, their consequential sound still needs masking. Finally, another recent solution is to completely remove propellers. Yamada et al. present an indoor helium blimp drone that uses piezo elements for propulsion [76]. While using a balloon makes the drone move with a low noise level, its poor mobility makes it impractical for domestic applications. In summary, currently existing solutions to the problem of noise in flying robots are insufficient.

2.2 Consequential Sound in HRI: Robot Noise and Altering Sound Profile

As one core modality robots can use to communicate with humans, sound may be used to create rich and engaging human-robot interactions (HRI) [55]. Thus, a wide variety of different types of robot sounds have been considered in HRI [52, 55]. While in the past, sound-related research in the HRI field mainly focused on speech and semantic-free utterances, implicit communication in the form of robots’ non-verbal sounds, including consequential sounds, is currently attracting growing attention [56, 57]. A review of consequential sound in HRI can be found in Robinson et al. [57]. Consequential sounds are sounds radiated by machines during their motions as a consequence of their operation and construction [35, 78] that are perceived by people as noise [35]. Negative effects of uncontrolled consequential sound on users’ perceptions of interacting with robots have already been reported [30, 46, 66]. Although past researchers encountered considerable challenges connecting objective auditory characteristics to the subjective perception of consequential robot sounds, past results demonstrate that altering the sound profile of robots might improve user experiences [78]. For instance, masking robots’ undesirable consequential sound with musical sounds has been proposed and validated as effective for mitigating unpleasant feelings [23, 67, 79]; Zhang et al. explored consequential robot sound and confirmed that quieter robots were perceived as less discomforting, while higher-pitched sounds were preferred for a UR5e robot arm [78]; Robinson et al. found that overlaying movement sound designs to the same video of a robot both increased and decreased perceived movement quality compared to the silent control condition [57]; and so forth (e.g., References [45, 46, 64]). Nevertheless, we believe an obvious limitation of the aforementioned past work is that most tested only video or audio recordings with participants rather than actual robots, thereby neglecting the impact of real-world dependent factors (e.g., multisensory inputs and realistic experience—see further discussion in Section 2.5) and potentially leading to poor ecological validity. In addition, consequential sounds of robots are underexplored, and little is known about how they could influence human-robot interaction [37]. Therefore, more research is needed to explore consequential sounds in HRI, especially research using real robots in a real-world HRI environment.

2.3 Changing the Perception of a Sound Event by Adding Natural Sounds

Mitigating the problematic consequential sounds of domestic drones by incorporating natural sounds is an area where we see potential benefits. Recent work has shown the benefits of exposure to nature in terms of behavioral and psychophysiological responses, decreasing negative affect and increasing positive affect and subjective well-being [8]. A systematic literature review and meta-analysis indicated aggregate evidence for the health benefits of natural sounds [10]. Adding natural sounds to mask noises or soothe users has been proposed in both research and commercial applications: (i) De Coensel et al. conducted a listening experiment on sound quality combining road traffic noise with either bird or fountain sounds, concluding that both of these natural sounds had a positive impact on auditory perception, but that bird sound had a significantly better impact [13]. (ii) The sound blocker “otohime,” which adds a water sound to mask embarrassing noise in toilets, has been commonly used in Japan since the 1980s [62, 75]. (iii) According to the developers of a sound box called “Zwitscher Box,” which generates the sound of birds chirping when people pass by, the sound “eases our minds and softens our gaze, we relax intuitively” [53]. De Coensel et al. also mentioned that the soothing effect of adding natural sound is stronger when the noise has low temporal variability [13]. Compared with traffic noise, flying robot noise has smaller fluctuations in sound level and frequency. This implies that we might achieve good results by adding natural sounds to flying robot noise. A white noise like rain always has a masking effect that may reduce noise [40] and soothe people who are exposed to it [58]; thus, it stands to reason to add a rain sound to drones’ consequential sound. Similarly, it is intuitive to associate a flying robot with a flying animal, leading us to expect a positive effect of adding the sound of birdsong to the drone.

2.4 Proxemics and HRI

Although adding natural sounds seems like a promising approach to masking the sounds of drones, as outlined above, it is vital to consider that human experiences of interactions also depend on the distance involved, usually referred to as proxemics. The term “proxemics” was coined by Edward Hall in 1966 and “identifies the culturally dependent ways in which people use interpersonal distance to understand and mediate their interactions with other people” [26]. He defined four proxemic zones for how people interpret interpersonal distance: intimate (less than 45 cm), personal (about 45 cm to 122 cm), social (about 122 cm to 366 cm), and public (about 366 cm to 762 cm). People adjust these distances to solidify their defense mechanisms when others invade these zones. Proxemics interactions have been investigated from several perspectives in the area of HRI, e.g., in industry settings [36], in experimental settings [50], in shape-shifting display settings [63], and in domestic settings [70]. The definitions of proxemic zones are considered the most relevant aspect for HRI [11, 18]. A number of studies have focused on investigating the relationship between sounds and proxemics for HRI (excluding flying robots). For instance, Walters et al. used a mechanical-looking robot with different synthesized voices and found that unhuman-like voices require a larger approach distance between participants and the robot [71]. Trovato et al. investigated the influence of robot noise on proxemics and found an increased effectiveness of a mask for eliminating the bad effects of noise, compared to no mask [67]. In addition, there have been some studies focusing on proxemics and flying robots, as small domestic drones with aerial capabilities can assist users in collaborative tasks, which necessitate interactions with people in close range [61]. For instance, Duncan and Murphy found no conclusive difference in comfort for a small drone approaching a human at different heights [19]. Yeh et al. designed a social-look drone with a welcoming voice and pet-like face, which decreased the minimum acceptable distance significantly [77]. Nevertheless, we found a lack of literature investigating the relationship between sounds and proxemics specifically with respect to flying robots.

2.5 The Importance of Realism for Sensory Experiences

In the HRI field, non-verbal sound does affect human perceptions, but it is not the sole factor in the multi-modal affective system [5]. Therefore, sound design for products and interactions must take the input of other senses into account [7]. Human perceptions in physical interactions are multisensory. Usually one or more of the senses: sight (vision), hearing (audition), touch (somatosensation), smell (olfaction), and taste (gustation), intervene in the same interaction [6, 51]. It has been demonstrated that one modality of sensory information can change how a person perceives another [4]. For some people, noise and airflow combined could even make the flying robot “threatening” [12]. Due to the multisensory and exquisite repertoire of human nature, offering a full sensory experience with high realism is particularly important in research on close-range human-drone interaction [73]. Moreover, even when it comes to a single sense, experiencing something through a recording is different than sensing the real world. For example, regarding vision, our two eyes with overlapping and slightly different viewpoints are delicate enough to determine fine differences in depth [31], such that seeing a photo or watching a video does not provide the same level of realism as seeing things in reality. As already mentioned in Section 2.2, we consider the use of video or audio recordings in HRI studies as an obvious limitation to their ecological validity. Instead, it would be arguably beneficial to use real robots interacting with participants in a real-world HRI environment.

Our work addresses the lack of knowledge related to human perceptions of noisy domestic flying robots by combining research on how to use natural sounds and proxemic distances. Specifically, our work will address the following needs and objectives: (i) A satisfying solution to the small domestic drone noise problem in the field of human-robot interaction is absent; the current method of suppressing noise is not suitable for small drones [43]. (ii) The validity of adding masking sounds to robots and adding natural sound to noisy traffic settings to improve people’s perceptions has been confirmed [13, 23, 67]. To the best of our knowledge, there is no existing literature or similar proposals to study user perceptions when adding natural sound to robots or drones that take proxemics into account. (iii) Previous studies mainly used only audio or video recordings to investigate how users’ perceptions changed when adding sound to a robot or noisy traffic scenario. However, the lack of other sensory stimuli such as somatosensation may lead to imperfect conclusions. Here, we set up a real scene that offers a full sensory experience with high realism. (iv) Furthermore, we extend our perspective by investigating the individual experiential dimension, building on a combination of quantitative and qualitative methods, namely, a mixed-methods approach [17], which we consider valuable, since most of the aforementioned related work took either a quantitative or a qualitative approach exclusively.

3.1 Ideation and Hypothesis

Adding bird or rain sounds to a drone while it flies may positively affect people’s perception of its presence, especially for close-range interactions. Proxemic distances may impact how people perceive the flying robot within certain sound conditions.

We designed and prototyped a small flying robot with an on-board loudspeaker to play chosen natural sounds (either birdsong or a rain sound; see details in Section 3.3) while flying at different proxemic distances (see Section 3.4). We hypothesized that adding natural sound would improve people’s perceptions of the drone. We also hypothesized that closer distance has a negative effect on perception due to the louder noise and higher risk of collisions at a closer distance.

3.2 Experimental Design

Our experimental design has two factors, while each factor has three levels, namely: 3 (sound conditions: bird, rain, none) \(\times\) 3 (distance conditions: near, middle, far). The setup was a randomized within-subjects approach, where each participant experienced all nine conditions, which were presented in different orders. For each factor with three levels A, B, C, there are six possible sequences (3! = 6), namely: ABC, ACB, BAC, BCA, CBA, CAB. We listed all six possible permutations of conditions for each factor (namely, either sound or distance) accordingly. For practical reasons, we first stipulated the order of the three distance conditions via complete counterbalancing, so each participant received a prearranged sequence of distance conditions; then, at each distance, the order of the three sound conditions was randomly determined by letting each participant roll a six-sided dice.

3.3 Natural Sound Design and Choices

Based on past studies and the particular rationales articulated in Section 2.3, we set up three different sound conditions for our experiment, namely: the original drone noise condition with no additional sound (control condition), the added birdsong condition, and added rain sound condition. The three sound conditions will be subsequently referred to as the none condition, the bird condition and the rain condition, respectively, for short.

3.4 Proxemic Distance Choices

Three takeoff locations were chosen according to the theory of proxemics [26, 73] and the size constraints of the experimental setting. The three takeoff distances were designated to be approximately 45 cm, 115 cm, and 185 cm away from participants, i.e., in the range of intimate space, personal space, and social space [18], respectively. We tried to understand these distances in the context of human-robot interaction. These three different locations will be subsequently referred to as the near, middle, and far locations. The three sound conditions were randomly played at each takeoff location.

3.5 Choice of Drone and Engineering the Flying Robot

Crazyflie is a small programmable quadcopter that is designed for indoor flying [15]. Crazyflie provides a wide range of open source Python programs, decks, and components to meet different research demands. In previous work, a mini flying robot with a smooth, stable, and high-precision flight trajectory was considered more acceptable by participants [34]. To achieve stable and precise flying trajectories, we chose the lighthouse positioning system to assist our experiment. With two lighthouse base stations and a lighthouse positioning deck on top of the quadcopter, the flying robots were able to fly with precision under our program’s control. To play natural sounds, we mounted a 28 mm round metal loudspeaker to the bottom side of the drone. The loudspeaker was powered by a 5 W Bluetooth amplifier with an extra Li-on battery through wires. The wires allowed the Bluetooth board and the battery to be installed in a small box and hidden under the desk during the test, which is a temporary solution to save on-board battery by lowering the takeoff weight. The small drone and accessories used in the experiment are shown in Figure 1. Following a Research through Design (RtD) approach [80], we decided on the following drone trajectory: The robot would first take off and fly vertically to a height of 40 cm above the table, then stay hovering for 10 seconds, afterward vertically land on the table. The selected natural sounds adapted to the drone flight durations and were played through the on-board loudspeaker.

Fig. 1.

View Figure

3.6 Experimental Setup

The experiment was conducted in a soundproof chamber to avoid interference from outside noise. To make the experimental environment closely approximate the household setting where domestic flying robots would be expected, we placed several pieces of furniture as shown in Figure 2. Participants sat in front of the long desk with two lighthouse positioning base stations set behind them out of sight. The desk and chair were pre-located and marked to keep all participants at a similar distance from the flying robot. A long blanket with three position marks was placed in the middle of the desk. This blanket was used as an absorber to decrease the reflected sound wave from the desk, and the marks on the blanket were used to show the three different takeoff settings.

Fig. 2.

View Figure

3.7 Preliminary Study and Engineering Evaluation

During a preliminary study, we recorded the three chosen sound conditions and measured their sound pressure levels. The total A-weighted SPL of the bird condition and the rain condition were calibrated to a similar level during the test (i.e., 71 dBA, 66 dBA, 61 dBA from the near to the far locations, respectively). The frequency spectra of the three sound conditions are shown in Figure 3, which matched our expectations well. The spectrum of flying robot noise is a wide-bandwidth noise mainly concentrated below 1 KHz. The rain sound is more constant and close to white noise, which is known to achieve a decent sound-masking effect when added to a wide-bandwidth noise, as some noise features may be hidden. However, bird sound is usually high-frequency and narrow-bandwidth. For this reason, purely from a spectrum engineering perspective, adding bird sound to a wide-bandwidth noise would have only a very limited sound-masking effect. Nonetheless, previous literature shows that adding bird sound to a similar noise can work very well [13, 27], even better than water sound [13]. This obviously shows that how humans perceive sounds is not purely dependent on the features of mechanical waves, in accordance with the biological principles mentioned later on in the discussion (see Section 6.2).

Fig. 3.

View Figure

3.8 Participants and Study Procedures

We recruited participants through multiple ways, including social media, flyers on campus and at student residences, and sending invitations to friends and colleagues (snowball sampling). Each session involved one individual participant, and each participant received a cinema ticket after the test as compensation. As all experiments were carried out entirely in Sweden, we carefully followed the Swedish Ethics Review Authority’s guidelines [21] and ensured that the national Ethics Review Act [60] and relevant regulatory requirements were complied with.

3.9 Measurements of Reported Perception

Participants were asked to evaluate each performance after its end with respect to six measurements describing the perceived characteristics of the flying robot: “loud,” “sharp,” “pleasant,” “safe,” “relaxing,” and “attractive,” on a scale of 0–10, with 0 representing “not at all” and 10 “extremely.” These characteristics were selected based on both existing literature and the focus of this study, as indicated below. After all the performances finished, every participant was also asked to rank their preference for the three sound conditions played at each distance by giving 0 points to the least liked, 1 point to the medium favorite, and 2 points to the most favored.

The measurements of perceived loudness and perceived sharpness were intended to examine how participants would feel about adding natural sounds to the drone noise soundscape. For the rest of the four measurements, participants were explicitly asked to consider their full sensory experience with the demonstrated flying robot performances. Pleasantness and attractiveness are the most commonly used perceptual assessment criteria in previous studies in both user experience [38] and soundscape quality [3, 22, 24]. Safety and causes of stress are further critical parameters for user acceptance of drones used in close proximity; thus, we wanted to examine both perceived safety and perceived absence of stress.

3.10 Post-experiment Interview Questions

The first and second authors conducted all interviews together, with detailed interview notes taken by each author separately. The interviews were primarily conducted in English. However, a number of participants were international students from China newly arrived in Sweden and felt more comfortable communicating in their native language. As both the first and second authors were native Mandarin Chinese speakers, these interviews were conducted in Chinese. Participants from other countries did not indicate a need to switch to another language. The full interviews were audio-recorded.

We used a semi-structured interview guide to elicit information about participants’ experiences and perceptions of the different noise conditions. Our questions addressed the participants’ preferences among the performances (and the reasons for these preferences), their impressions of the tested sounds, their personal background, and their impressions of the study setup. Finally, the interviewers asked follow-up questions when appropriate, and the participants had the opportunity to add information they considered important. The specific questions used are listed in Table 1.

4.1 Overall Description of Statistical Methods

Statistical analysis was done using IBM SPSS Statistics (version 28.0.0.0) [29]. For our within-subjects factorial design, we conducted two-way repeated measures ANOVAs on reported perception for each of the six measurements. For each measurement, we checked the significance of the main effects of each of the two factors (sound and distance) and the interaction effect between them. In cases where one factor’s main effect was significant, we carried out a post hoc analysis through multiple comparisons with Fisher’s Least Significant Difference (LSD) test to examine the relationships between the corresponding individual levels. Regardless of whether there was a significant interaction effect, we conducted simple effects tests to compare all pairs of three levels of one factor for each of the three levels of the other factor. The simple effects tests were done with one-way repeated measures ANOVAs followed by multiple comparisons with the LSD tests. We checked the normality of residuals via the Shapiro-Wilk test and the homogeneity of variance via Levene’s Test. We decide to report partial eta squared as the estimate of effect size, denoted as \(\eta ^2_p\), as it offers a more comparable estimate for factorial designs with multiple independent variables [39]. Table 2 presents a summary of the statistical analysis results.

The preference rating is different from the six parameters mentioned above. Each participant ranked their preferences for the three performances played at each distance by giving 2 points for their most preferred, 1 point for the next preferred, and 0 points for their least favorite. We conducted a one-way repeated measures ANOVA at each distance to compare the effects of the three sound conditions on the participants’ preferences. Where the ANOVA revealed a significant difference, we used the LSD test to see the relationships among the three sound conditions at the specific distances.

For the six measurements of reported perceptions, the data were plotted as box-whisker plots with asterisks highlighting the significance level, where * indicates p < .05, ** indicates p < .01, and *** indicates p < .001. The preference data were plotted as a stacked bar chart. See related figures in the following sections.

4.2 Perceived Loudness

Figure 4 shows the ratings of perceived loudness at the three locations with three sound conditions. The main effect of the sounds (F(2,94) = 18.44, p < .001, \(\eta ^2_p\) = 0.282) and the distances (F(2,94) = 24.87, p < .001, \(\eta ^2_p\) = 0.346) on perceived loudness were both significant. The interaction between sounds and distances was also significant (F(4,188) = 3.58, p = .008, \(\eta ^2_p\) = 0.071).

Fig. 4.

View Figure

The simple effects analyses indicated for the three distances: (i) At the near location, the mean perceived loudness rating for the bird condition (M = 7.38, SD = 1.63) was significantly higher than both the none (M = 6.06, SD = 1.92) and the rain conditions (M = 6.46, SD = 1.66), and the rain condition was also rated significantly louder than the none condition. The effect size was \(\eta ^2_p\) = 0.388. (ii) At the middle location, the mean perceived loudness rating for the bird condition (M = 6.52, SD = 1.82) was significantly higher than both the none (M = 5.65, SD = 1.39) and the rain conditions (M = 6.02, SD = 1.58). The effect size was \(\eta ^2_p\) = 0.273. (iii) At the far location, only the mean perceived loudness rating for the bird condition (M = 5.73, SD = 1.90) was significantly higher than the none condition (M = 4.96, SD = 1.96), with an effect size of \(\eta ^2_p\) = 0.140.

For the three sound conditions: (i) For the none condition, the mean perceived loudness ratings at the near (M = 6.06, SD = 1.92) and middle locations (M = 5.65, SD = 1.39) were both significantly higher than the far location (M = 4.96, SD = 1.96). The effect size was \(\eta ^2_p\) = 0.213. (ii) For the bird condition, the mean perceived loudness rating at the near location (M = 7.38, SD = 1.63) was significantly higher than both the middle (M = 6.52, SD = 1.82) and the far locations (M = 5.73, SD = 1.90). The difference between the middle and far locations was also significant, with an effect size of \(\eta ^2_p\) = 0.487. (iii) For the rain condition, the mean perceived loudness ratings at the near (M = 6.46, SD = 1.66) and middle locations (M = 6.02, SD = 1.58) were both significantly higher than the far location (M = 5.48, SD = 2.06). The effect size was \(\eta ^2_p\) = 0.209.

4.3 Perceived Sharpness

Figure 5 shows the ratings of perceived sharpness at the three locations with three sound conditions. The main effects of the sounds (F(2,94) = 27.47, p < .001, \(\eta ^2_p\) = 0.369) and the distances (F(2,94) = 13.60, p < .001, \(\eta ^2_p\) = 0.224) on perceived sharpness were both significant. The interaction between sounds and distances was not significant (F(4,188) = 1.82, p = 0.13, \(\eta ^2_p\) = 0.037), but the simple effects analyses nevertheless indicated a possible interaction.

Fig. 5.

View Figure

The simple effects analyses indicated for the three distances: (i) The mean perceived sharpness rating of the bird condition (near: M = 7.65, SD = 1.72; middle: M = 6.73, SD = 2.14; far: M = 6.29, SD = 2.14) was significantly higher than the other two sound conditions. (ii) The mean perceived sharpness ratings for the none (near: M = 5.73, SD = 2.10; middle: M = 5.33, SD = 1.83; far: M = 4.90, SD = 2.03) and the rain conditions (near: M = 5.52, SD = 1.88; middle: M = 4.81, SD = 1.79; far: M = 4.85, SD = 1.99) were less decisive. The effect sizes for the three locations were \(\eta ^2_p\) = 0.404 (near), \(\eta ^2_p\) = 0.267 (middle), \(\eta ^2_p\) = 0.202 (far).

For the three sound conditions: (i) For the none condition, the mean perceived sharpness rating of the near location (M = 5.73, SD = 2.10) was significantly higher than the far location (M = 4.90, SD = 2.03). The effect size was \(\eta ^2_p\) = 0.093. (ii) For the bird condition, the mean perceived sharpness rating at the near location (M = 7.65, SD = 1.72) was significantly higher than both the middle (M = 6.73, SD = 2.14) and the far locations (M = 6.30, SD = 2.14). The difference between the middle and the far locations was also significant. The effect size was \(\eta ^2_p\) = 0.294. (iii) For the rain condition, the perceived sharpness rating at the near location (M = 5.52, SD = 1.88) was significantly higher than both the middle (M = 4.81, SD = 1.79) and the far locations (M = 4.85, SD = 1.99). There was no significant difference between the middle and the far locations. The effect size was \(\eta ^2_p\) = 0.294.

4.4 Perceived Pleasantness

Figure 6 shows the ratings of perceived pleasantness at the three locations with three sound conditions. The main effects of sounds (F(2,94) = 6.01, p = .004, \(\eta ^2_p\) = 0.113) and distances on perceived pleasantness (F(2,94) = 13.08, p < .001) were both significant, as was the interaction between sounds and distances (F(4,188) = 3.40, p = .010, \(\eta ^2_p\) = 0.068).

Fig. 6.

View Figure

The simple effects analyses indicated for the three locations: (i) The mean perceived pleasantness ratings for the sound conditions did not significantly differ at the near location. (ii) At the middle location, the mean perceived pleasantness ratings for both the bird (M = 5.48, SD = 2.52) and the rain conditions (M = 5.37, SD = 1.96) were significantly higher than the none condition (M = 4.65, SD = 1.72). There was no significant difference between the bird and the rain conditions. The effect size was \(\eta ^2_p\) = 0.066. (iii) At the far location, the mean perceived pleasantness rating for the bird condition (M = 6.21, SD = 2.31) was significantly higher than both the none (M = 4.71, SD = 2.09) and the rain conditions (M = 5.08, SD = 2.14), with no significant difference between the none and the rain conditions. The effect size was \(\eta ^2_p\) = 0.186. It seems that the sound conditions played an important role in the perception of pleasantness at the far and middle distances, but not at the near location.

For the three sound conditions: (i) For the none condition, the mean perceived pleasantness ratings of both the middle (M = 4.65, SD = 1.72) and the far locations (M = 4.71, SD = 2.09) were significantly higher than the near location (M = 3.90, SD = 2.16), with no significant difference between the middle and the far. The effect size was \(\eta ^2_p\) = 0.130. (ii) For the bird condition, the mean perceived pleasantness rating at the far location (M = 6.21, SD = 2.31) was significantly higher than both the near (M = 4.56, SD = 2.53) and the middle locations (M = 5.48, SD = 2.52). The difference between the near and the middle locations was also significant. The effect size was \(\eta ^2_p\) = 0.246. (iii) For the rain condition, only the mean perceived pleasantness rating of the middle location (M = 5.38, SD = 1.96) was significantly higher than the near location (M = 4.54, SD = 2.27). The effect size was \(\eta ^2_p\) = 0.068. The distances played an important role in the perception of pleasantness for all three sound conditions.

4.5 Perceived Safety

Figure 7 shows the safety ratings at the three locations with three sound conditions. The main effects of sounds on perceived safety (F(2,94) = 1.49, p = .23, \(\eta ^2_p\) = 0.031) were not significant. However, the main effect of distances on perceived safety (F(2,94) = 29.68, p < .001) was significant. The sound conditions very likely had no effect on the perception of safety, but the distances did.

Fig. 7.

View Figure

The simple effects analyses indicated for the three sound conditions: (i) For both the none and the bird conditions, the mean perceived safety rating at the middle (none: M = 6.79, SD = 2.23; bird: M = 7.02, SD = 2.48) and the far locations (none: M = 7.29, SD = 2.46; bird: M = 7.73, SD = 2.39) was significantly higher than the near location (none: M = 5.13, SD = 2.74; bird: M = 5.35, SD = 2.74), and the mean perceived safety rating at the far location was also significantly higher than at the middle location. The effect size was \(\eta ^2_p\) = 0.348 for the none condition and \(\eta ^2_p\) = 0.315 for the bird condition. (ii) For the rain condition, the mean perceived safety rating for both the middle (M = 6.96, SD = 2.12) and the far conditions (M = 7.30, SD = 2.49) was significantly higher than the near (M = 5.35, SD = 2.62) condition. The effect size was \(\eta ^2_p\) = 0.313. The distances played an important role in the perception of safety for all three sound conditions.

4.6 Perceived Relaxedness

Figure 8 shows the ratings of relaxedness at the three distances with three sound conditions. The main effect of sounds (F(2,94) = 5.33, p = .006, \(\eta ^2_p\) = 0.102) and distances on perceived relaxedness (F(2,94) = 27.77, p < .001, \(\eta ^2_p\) = 0.371) were both significant. The interaction between sounds and distances was not significant, (F(4,188) = 2.05, p = 0.09, \(\eta ^2_p\) = 0.042), but simple effects analyses indicated the possibility of an interaction.

Fig. 8.

View Figure

The simple effects analyses indicated for the three distances: (i) The mean perceived relaxedness ratings between sound conditions did not significantly differ at the near location. (ii) At the middle location, the mean perceived relaxedness ratings in both the bird (M = 5.56, SD = 2.68) and the rain (M = 5.52, SD = 2.26) conditions were significantly higher than the none condition (M = 4.63, SD = 2.10), with no significant difference between the bird and the rain conditions. The effect size was \(\eta ^2_p\) = 0.075. (iii) At the far location, the mean perceived relaxedness rating for the bird condition (M = 6.10, SD = 2.35) was significantly higher than both the none (M = 4.71, SD = 2.19) and the rain conditions (M = 5.27, SD = 2.52), with no significant difference between the none and the rain conditions. The effect size was \(\eta ^2_p\) = 0.251.

For the three sound conditions, the none, the bird, and the rain conditions: The mean perceived relaxedness ratings at the middle (none: M = 4.62, SD = 2.01; bird: M = 5.56, SD = 2.68; rain: M = 5.52, SD = 2.26) and the far (none: M = 4.70, SD = 2.19; bird: M = 6.10, SD = 2.35; rain: M = 5.27, SD = 2.52) locations were significantly higher than at the near location (none: M = 3.75, SD = 2.12; bird: M = 4.33, SD = 2.31; rain: M = 4.19, SD = 2.27), with no significant difference between the far and the middle locations. The effect size was \(\eta ^2_p\) = 0.145 for the none condition, \(\eta ^2_p\) = 0.270 for the bird condition, and \(\eta ^2_p\) = 0.226 for the rain condition.

4.7 Perceived Attractiveness

Figure 9 shows the ratings of perceived attractiveness at the three distances with three sound conditions. The main effects of sounds (F(2,94) = 10.30, p < .001, \(\eta ^2_p\) = 0.180) and distances on perceived attractiveness (F(2,94) = 11.38, p < .001, \(\eta ^2_p\) = 0.195) were both significant. The interaction between sounds and distances was also significant (F(4,188) = 3.32, p = .012, \(\eta ^2_p\) = 0.066).

Fig. 9.

View Figure

The simple effects analyses indicated for the three distances: (i) At the near location, only the mean perceived attractiveness rating of the bird condition (M = 4.79, SD = 2.77) was significantly higher than the none condition (M = 3.98, SD = 2.61), with no significant difference between the other conditions. The effect size was \(\eta ^2_p\) = 0.054. (ii) At the middle location, the mean ratings of perceived attractiveness in both the bird (M = 5.79, SD = 2.43) and the rain conditions (M = 5.35, SD = 2.23) were significantly higher than the none condition (M = 4.43, SD = 2.11), with no significant difference between the bird and rain conditions. The effect size was \(\eta ^2_p\) = 0.167. (iii) At the far location, the mean perceived attractiveness rating for the bird condition (M = 6.21, SD = 2.44) was significantly higher than both the none (M = 4.48, SD = 2.60) and the rain conditions (M = 5.04, SD = 2.34), with no significant difference between the none and the rain conditions. The effect size was \(\eta ^2_p\) = 0.351.

For the three sound conditions: (i) For the none condition, only the mean perceived attractiveness rating at the far location (M = 4.48, SD = 2.60) was significantly higher than at the near location (M = 3.98, SD = 2.61), with no significant difference between other locations. The effect size was \(\eta ^2_p\) = 0.047. (ii) For the bird condition, the mean perceived attractiveness rating at the far location (M = 6.21, SD = 2.44) was significantly higher than at the middle location (M = 5.79, SD = 2.43), and the middle condition was significantly higher than the near condition (M = 4.79, SD = 2.77). The effect size was \(\eta ^2_p\) = 0.237. (iii) For the rain condition, the mean perceived attractiveness ratings at both the middle (M = 5.35, SD = 2.23) and the far locations (M = 5.04, SD = 2.34) were significantly higher than at the near location (M = 4.48, SD = 2.63), with no significant difference between the far and the middle locations. The effect size was \(\eta ^2_p\) = 0.106.

4.8 Preference

Figure 10 shows the participants’ mean preference ratings among the three sound conditions at each distance. Each participant ranked their preference for the three performances at each distance by giving 2 points to their top favorite, 1 point for the medium favorite, and 0 points for their least favorite.

Fig. 10.

View Figure

At both the near and the middle locations, the rain sound was most preferred, followed by the bird sound, and the none condition was the least preferred. However, the gap between the rain and the bird sound at the middle distance was smaller than at the near location. The mean preference rating did not significantly differ between sound conditions at both the near and the middle locations.

By contrast, at the far location, the bird condition was ranked the highest, the rain condition second, and the none condition remained last. Remarkably, the bird scored almost double that of the rain and triple that of the none condition. The main effect of sound on preference was significant at the far location (F(2,94) = 19.39, p < .001, \(\eta ^2_p\) = 0.292), but not significant at the near (F(2,94) = 1.02, p = .364, \(\eta ^2_p\) = 0.021) and middle (F(2,94) = 1.02, p = .364, \(\eta ^2_p\) = 0.043) locations.

For all three locations, adding natural sounds (bird or rain) seemed to have a positive effect on participants’ overall preferences; the distance also had an obvious influence on the preference ratings. For instance, ratings of the bird condition improved markedly from the near to the far location, indicating that participants particularly preferred the bird sound at the far, but not so much at the middle or near location.

5.1 Themes Identified for the Bird Sounds

Regarding the theme of familiarity, the majority of participants mentioned that the selected birdsong was sharp and implied they had a common understanding of the objective features of the birdsong as a high-frequency sound. However, the theme of familiarity had a strong impact on the participants’ attitudes toward the added bird sound, which was closely related to the theme of personal experiences. Since the bird species (great tit) we chose is widespread in Europe, the attitudes of participants who had lived in Europe for at least some years generally tended to be more positive, for they mostly associated the sound with a known bird and nature. P25 stated “I feel relaxed when I hear it. It is a nice melody.” P33 said “It sounds very pleasant. The bird reminds me of going to the zoo, like the jungle area in Universeum (a public science center and museum in Gothenburg).” P31 could even identify the exact bird species from the birdsong. However, among participants newly arrived in Europe (mainly international students), most reported that the sound did not sound like a bird, but rather like an alarm, and for this reason their response tended to be negative. P12 mentioned the sound was sharp and annoying and reminded her of a fire alarm beeping. P18 emphasized “It is a threatening alarm. Feels like it’s coming to attack me!” The attitudes of participants also varied depending on whether they were more outdoors or indoors people, as outdoorsy people were more likely to prefer this birdsong. P32, who identified as an outdoorsy person, stated, “I find it very familiar. I heard this specific bird in spring at our summer house so I recognized it very well! It is really good and comforting.” P34, who mentioned that he was an indoors person, said, “The bird gave me a headache. It was way too loud and high-pitched for me.”

Fig. 11.

View Figure

The proxemic distances had a strong impact on the participants’ perceptions of the bird sound, mainly associated with the social dimension of proxemics. Among the nine performances, most participants liked the bird condition the most when the drone was at the far location, while they disliked the same sound the most at the near location—this matches the quantitative data on preference (see Figure 10). Participants reported that at the far location, they perceived the bird as pleasant, attractive, and comforting, but at the near location, they felt it was annoying, stressful, dangerous, and uncomfortable. P04 emphasized “the distances obviously made the bird sound very different.” P52 mentioned that the bird’s song had a different effect according to the three distances, “when it was far, the bird made me feel comfortable and it didn’t sound so sharp, I liked it the most. However, when the same sound was played at middle switched from far, I felt it became sharper and had an opposite effect which made me feel uncomfortable. I didn’t expect that...When it was near, it made me even more uncomfortable.” Participants said it was weird to have a bird so close in a real situation. This finding was closely related to the theme of safety. Some participants mentioned feeling that they were being watched by the drone (see details in the Section 5.3.2), and many felt they might even be attacked by the “bird”: P18: “feels like it is coming to attack me.” In their opinion, it is more common to experience a distant bird rather than a near one, and that might be the reason why they preferred the drone to play the birdsong far away, but not when it is near.

Finally, the theme of personal experiences and preferences explained the presence of somewhat extreme opposing attitudes towards the same bird condition, especially when the drone was at the near location. P23, P34, and P50 seemed to detest and dread the near-bird condition. P34 stated: “The bird gave me a headache, especially when it was so close to me, I just wanted it to go away! Oh my god! Please! I wanted to kill the bird, almost like ‘get me a rifle.”’ P50 said: “when it’s coming so close, I would definitely want to grab it, break the wings and hide it...” In contrast, P31, P32, and P45 were true bird lovers. P31 preferred the near-bird condition the most, commenting “I like birds a lot. I’ve seen this bird in real life. It’s a common bird in Europe. I know its name in Swedish, it’s ‘Talgoxe.’” Besides correctly identifying the birdsong as belonging to the talgoxe (great tit), P31 talked about the sound features of the great tit and the change from three syllables to two due to increasing urban noise pollution, which perfectly matches the literature we found (see Section 3.3.2). P32: “I want it (the drone with the birdsong) to come onto my hand,” and P45 stated: “I love it. It made me recall I gave food to wild birds and their babies—they came to the balcony singing.”

5.2 Themes Identified for the Rain Sounds

Regarding the theme of familiarity, participants’ interpretations of the added rain sound fell into two groups. The majority of participants belonged to the first group and claimed the sound was like rain. The second group of around one-third of the participants claimed that it was not like rain, but could be further split into two subgroups. The first subgroup, consisting of P1, P12, P20, P23, P27, P29, P42, P44, P53, associated it with water other than rain, e.g., water leakage, a water splash, or a waterfall. The other subgroup mentioned that it felt artificial due to the fact that it was together with the drone noise. Participants seemed to have more negative feelings if they associated the sound with something artificial or something wrong (e.g., water leaking). P38 stated “It irritates me. It sounds very artificial, mechanic, and annoying. It is not natural at all.” However, participants tended to feel more positively if they correctly associated the sound with rain. In the case of rain sounds, the theme of familiarity was also closely associated with the theme of personal experiences, as participants from South Asian cultures tended to identify the sound correctly as heavy rain.

The proxemic distances of the rain sounds impacted participants’ perceptions, but in sharp contrast to the bird sound, those perceptions were unaffected by the theme of the social dimension of proxemics. In the case of rain, the impact was related to the blending of the rain sound with the drone noise. Many participants reported that under the far-rain condition, they could not distinguish the added rain sound, as it blended into the drone noise, but they could distinguish it at the near or middle location. P01 stated, “I couldn’t distinguish the rain sound when at far, but at near it was ok.” P52 mentioned “when far, it didn’t sound like rain when it was together with the machine noise—it was not clear. But the closer, the clearer, the more it sounded like rain.” The social dimension of proxemics entirely disappeared in the rain setting, as no sounds from living or artificial social beings were involved.

In the case of rain sounds, the theme of personal experience was mainly associated with cultural factors. We noticed that the passionate rain lovers P45, P47, and P48, who all came from the monsoon region in South Asia, highly rated the rain condition. They were able to correctly identify the added sound as a heavy rain even though it was together with drone noise, as P47 said: “The rain sound was suppressed by the machine sound, but I still heard the sound of water, the rain was quite heavy.” And P47 continued: “Especially when at far, this rain sound felt special—it reminds me of my home where it has a lot of rain! My school usually reopened during the rainy season, this is exactly the same sound I used to hear when I was a kid sitting in a classroom with heavy rain outside. I could relate to it. It’s a nostalgic feeling.” P45: “This sound is like heavy rain, it reminds me of the rainy season in my home country. I love rain. I have a name that means ‘rain’ in my native language...I recorded the sounds of rain myself...I took shower in the rain—in my country, the raindrops are very big so you can take a shower with them.” P45, P48 associated the rain sound with other sensory experiences as well. P48: “I like rain in my country, it is warm rain [tactile] with the smell of soil [olfactory].” P45: “After raining, it became green [vision], fresh [olfactory], and cool [tactile].”

We did not identify any associations relating to the theme of safety in the case of the rain condition, indicating that the sound might be interpreted as a more neutral alternative.

5.3 Themes Independent of Sound Conditions

Several comments by the participants covered the identified themes but did not apply to any particular sound condition. Several participants commented on personal experiences and perceptions independent of particular sound conditions, such as the purpose of the drone, individual suggestions for alternative sounds, and the feeling of the wind. Furthermore, the drone was sometimes experienced as an invasion of privacy, associated with the social dimension of proxemics. The most pronounced theme, however, was safety. Several comments about safety ranged across the different sound conditions, indicating that this is a general concern when interacting with drones.

6.1 Quantitative Analysis Findings

The statistical analysis of the quantitative data demonstrated a collective pattern among participants regarding their experience encountering a noisy domestic drone with the three sound conditions (bird, rain, none) at three proxemic distances (far, middle, near). The two measurements of the perceived sound characteristics, namely, loudness and sharpness, met our expectations—natural sound conditions were perceived as louder, and the bird in particular was recognized as sharper; the closer the distance, the louder and sharper the sound were perceived. We had hypothesized that it would be beneficial to add natural sounds in terms of people’s perceptions of the drone, and we did find support for this. The natural sounds we added, namely, the bird and the rain, both significantly increased the participants’ ratings for the measurements of pleasantness, relaxedness, and attractiveness, but had no effect on perceived safety. Meanwhile, the proxemic distances had significant effects on all measurements, which was in line with our hypothesis that the closer distance would have a more negative effect on the perceptions. Although we had not explicitly considered an assumption for the interaction effects between both factors, in fact, the interaction effects between sound and distance were significant (p < .05) for loudness, pleasantness, and attractiveness. The participants significantly preferred the bird at the far location, but not at themiddle or near location, implying that they started to dislike the bird when the distance got close.

6.2 Qualitative Analysis Findings

The qualitative data enabled us to look further into each individual participant’s reasons for their feelings and thoughts about the experiment. The findings generalized into four themes: (1) familiarity with the sounds, (2) personal experiences and preferences with the sounds, (3) the social dimension of proxemics, and (4) safety associations. Regarding the bird and rain sound conditions, we found that participants’ sensations were similar with regard to the objective features of the sounds—e.g., no matter whether they liked or disliked the bird, they shared the same view that the sound was relatively sharp; regardless of whether they considered the rain to be natural or not, they recognized it as somewhat white noise with a water sound, which blended into the drone noise at the far location but was more distinct at the near location. However, this common understanding of objective features funneled into various subjective associations and interpretations—e.g., some participants claimed the bird sounded not like a bird but like an alarm, while others experienced the opposite. This finding emphasized the role that personal experiences and preferences play in the interpretation of sound. Even participants who associated the sound with a bird’s song still ranged from interpreting it as a dangerous “bird” that might attack them (especially in the near condition, associated with the social dimension of proxemics) to seeing it as a cute and friendly animal. The participants’ associations and interpretations of the sound affected their attitude and experience of the flying robot and were dependent on their previous personal experiences, perceptions, and cultural backgrounds. This empirical finding is supported by neurobiology—the number of neurons in the primary auditory cortex, namely, those dedicated to figuring out what sound information means, greatly outnumber those that transform sound into electrical neural signals, resulting in what humans expect to hear plays a great role in what they indeed do hear [41]. In other words, how humans perceive sounds is not purely dependent on actual sonifications, but also and more importantly on their previous experience.

We found further support for this interpretation in the findings for themes across all sound conditions. For instance, participants acknowledged the presence of the drone airflow and reached a consensus that the closer the drone, the more obvious the airflow felt. Then, some associated the airflow with natural weather patterns, while others associated it as an amplification of the perceived presence of the robot. When associated with natural weather, it could be further interpreted as either a comfortable cool breeze or an uncomfortable cold wind. Similarly, the association with robot presence amplification could be interpreted as negative feelings of danger or positive feelings of being more connected to the drone. These personal interpretations shaped participants judgments’ of the airflow and by extension, the flying robot and the whole experiment. Across all sound conditions, we found support for the vital role that safety perceptions play in HRI, as numerous statements were categorized into the theme of safety.

The thematic analysis of the interview data provided additional information of the participants’ qualitative experience beyond the quantitative measurements, which allowed us to extract patterns within participants’ perceptions of a noisy flying robot with natural sounds. From the aforementioned themes, we found a generalized pattern within participants’ various qualitative experiences of their encounters with the flying robot with added natural sounds in the experiment. This pattern has been consolidated into a visual summary that setting multiple sequential steps in relation with one another (see Figure 12). This visual summary illustrates why an identical stimulus might lead to diverse and even opposite individual judgments. Our visual summary resonates with models from the existing literature in the fields of user experience [28, 33] and neurobiology [41]. Figure 12 shows a sketch of how sounds and other types of stimuli in our experiment are perceived by the participants through a process of sensation and perception, illustrating that perception is a phenomenon of sensemaking. A given stimulus may be first perceived as a physical sensation with certain physical characteristics [54]. This step depends on the functioning of participants’ sensory apparatus and usually converges into some common understanding of the objective features of the stimulus (except among people with sensory impairments). This sensation may trigger participants to associate the stimulus with something that they are already familiar with. Then, the association develops into an emotional interpretation. Both of these steps, association and interpretation, highly depend on individuals’ previous experience [59], which further determines personal preferences—participants diverge into viewing the same stimulus either favorably or less favorably.

Fig. 12.

View Figure

7.1 Consider Proxemics for Sound Design in HRI

Our study found support for the inclusion of proxemics in the investigation of sound design in HRI. The quantitative results demonstrated statistically significant interaction effects between the studied sound and distance conditions on reported perceptions. Regarding the qualitative results, several participants mentioned that they perceived the drone as an invasion of privacy that was surveilling them, especially when it was close (despite knowing that there was no camera). These perceptions of a drone as a potential threat were even more pronounced when bird sounds were added. Although participants generally accepted bird sounds when the drone was far away, their perceptions were more diverse when the drone was near. Consistent with this interpretation, the rain sounds were not associated with an active social being approaching the participant. It is therefore imperative to consider proxemics for sound design in domestic flying robots. This design recommendation could conceivably apply to sound design in close-range HRI in general.

7.2 Allow Customized Sound

When examining the participants’ feelings on the sounds, it is quite interesting to find that the description of natural sound can be contradictory. Participants perceived it as, variously, “annoying” or “soothing,” “artificial” or “natural,” “noisy” or “relaxing.” In general, the added natural sound improved people’s feelings about the drone noise, but also introduced the topic of the listener’s personal taste [57]. A variety of responses from participants about their own preferred added sounds also revealed individual diversity. We believe that allowing customers to choose their own added sound would be a good design improvement, since it can take personal preference into consideration rather than trying to satisfy everyone with a single common denominator.

7.3 Consider Functionality and Use Cases

Some participants mentioned during interviews that they would be keen to know the intended function or purpose of the small drone, which was not specified in this study. Knowing the intended function of the drone may significantly affect users’ perceptions and experience. Other participants informally asked about this point after the interviews. The empirical data thus indicated the necessity of clarifying the purpose of usage. Functionality and use cases need to be considered when designing domestic flying robot interactions.

7.4 Explore the Airflow

The presence of airflow was reported as particularly salient when the drone was flying close to participants, and airflow evoked both positive and negative feelings in participants. Thus, airflow is influential for user acceptance of domestic flying robots, especially in terms of close-range interactions. Coupling the airflow with certain conditions (e.g., sounds, ambient temperatures, vision, smell) may trigger certain effects, and a drone’s airflow could be applied to perform some useful functions—for instance, bringing cool breeze on a hot summer day. In general, airflow is a design space that needs exploring.

7.5 Keep Safe Distance

Although the wires under the drone were only used as a temporary solution to attach the loudspeaker and lower the takeoff weight, some participants believed that they worked like a leash to restrict the drone’s flying zone, which improved perceived safety. However, in practice, the electric wires may easily become entwined with the propellers and result in danger [72]. Nevertheless, the demand to restrict domestic drones’ room to maneuver remains considerable. Inspired by wall barriers for robot vacuums [69], an auxiliary device with an electric fence function that can keep domestic drones a safe distance away from people may be useful, or the flying robot should be capable of autonomously avoiding collisions with humans.

7.6 Add Propeller Guard

Adding a propeller guard or protector was suggested by several participants as an additional safety measure, since the drone flew very close to them during the test. Safety considerations were a frequent theme in the qualitative analysis across all sound conditions. We also found that the propellers were the most vulnerable parts in a few crashes during commissioning. Thus, adding a propeller guard to a domestic flying robot would better protect both the human user and the robot itself.