1 Introduction

There is currently a strong desire by manufacturers to introduce automated vehicles (AVs) to the market (see the roadmap of the European Road Transport Research Advisory Council; ERTRAC 2017). These vehicles allow the driver to hand over the complete dynamic driving task to the automation of the vehicle (SAE 3 and higher; SAE International 2016). Examples of this trend are the activities of all main vehicle manufacturers to provide automation functionalities for their passenger vehicles. A slightly different solution for new forms of mobility are Automated Driving System (ADS)-dedicated vehicles (see SAE International 2016) or Automated Road Transport System (ARTS) that provide bus-shuttle or robotaxi-like services, operated completely driverless, especially for the first and last mile of transport. In Europe, a number of active projects are currently demonstrating and testing such systems, including the CityMobil2 project funded by the European Commission (http://www.citymobil2.eu; Alessandrini et al. 2014), the GateWay project in the UK (https://gateway-project.org.uk), the WEpod project in the Netherlands (http://wepods.com) and first applications in Germany, such as the test operation of automated shuttle busses from Deutsche Bahn (Saunders 2017).

Both privately owned and public forms of AVs are likely to be deployed in mixed traffic environments, and, therefore, need to interact safely and efficiently with other traffic participants, including conventional, manually driven vehicles, cyclists and pedestrians. As traffic participants, humans use multiple implicit cues to anticipate the behaviour of vehicles on the road, such as the approaching speed of an oncoming vehicle. Other cues involve explicit means of communication, such as eye contact and gestures, as well as vehicle-based signals, such as brake lights or indicators. All these means of communication allow effective understanding and coordination of future motion plans between different traffic participants. However, current AVs are thoroughly lacking such communication and coordination capabilities, and their interaction with other traffic participants is often limited to, and mostly dominated by, the rational principle of collision avoidance. This results in non-human-like behaviour of the AV, such as sudden stopping, unexpected lane changes or long standstill periods (e.g. standstill until a blocking obstacle moves away). These actions are not well predictable, and can be quite frustrating for other traffic participants, leading to reductions in safety and efficient traffic flow (Brown and Laurier 2017). Instead, an AV’s behaviour and future path of movement should be well understood by other traffic participants. This allows the surrounding traffic participants to correctly interpret the intentions of the AV and coordinate their planned actions accordingly. To safely integrate AVs in mixed traffic environments, we need to ensure that the common human–human interaction between the on-board driver and other traffic participants today is replaced by an artificial interaction of the AV. This opens a completely new field for human factors researchers and human–machine interface (HMI) designers of AVs (Cacciabue et al. 2014).

In the future, the situation will change from a dyad of interaction today (on-board driver and other traffic participants) to a triad of interaction (on-board user, AV and other traffic participants) in which the direct interaction between the AV and the surrounding traffic participants will play a significant role (Fig. 1). The interaction between the AV and surrounding traffic participants, replacing some parts of the common human–human interaction, needs to be newly developed as this will become most prominent. In the new triad of interaction, the information needs of the on-board user of an AV should be also considered to avoid confusion and unsafe situations (e.g. the on-board user gives additional signs to traffic participants or that other traffic participants interpret the body language or gestures of users on board in a way they are used to, even though the user on-board is not in charge of controlling the vehicle).

Fig. 1
figure 1

(source: EU project interACT)

From today’s situation of human–human interaction in mixed traffic environments towards a future situation with a triad of interaction between automated vehicles, other traffic participants and the on-board users

Full size image

This paper focuses on how to design the interaction of the AV with other traffic participants. It presents some design considerations for designing the interactions of AVs with other traffic participants, based on current human–human interaction in mixed traffic environments. The paper is structured in the following way: In the first part, we describe our analysis of the common features in human–human communication between traffic participants. Based on this analysis we derived four categories of information that are presented in the second part of the paper as design considerations. These are (1) information about vehicle driving mode, (2) information about next manoeuvres, (3) information about perception of the environment and (4) information about cooperation capabilities. In the third part, we use the four categories of information to analyse available studies and data from face-to-face interviews, onsite questionnaires and two focus group studies, which we conducted as part of the CityMobil2 project. These give a first insight into the needs and expectations of traffic participants interacting with AVs and the relevance of each of the four categories for the AV design. In the fourth and final part, we present an overview on existing AV design concepts from vehicle manufacturers, suppliers and research institutions and describe several methods on how relevant information can be transferred to surrounding traffic participants and we discuss their advantages and disadvantages.

The overall aim of the design considerations was to provide a structuring aid for designers, such as those developing external HMI (defined as the HMI of an AV that is meant to send information to surrounding traffic participants) and interaction concepts for AVs, to reduce the complexity of the design space and to support designers in their decisions so that traffic participants receive the most pertinent information from the AV, using the most appropriate communication channels.

2 Analysis of common interactions between traffic participants

When designing the behaviour of the AV in a mixed traffic environment, we assume that it is highly relevant to understand how human traffic participants currently interact with each other. This is in line with the work of other authors who analyse the current human–human behaviour to understand how an AV needs to be designed for safe implementation in mixed traffic scenarios (see Beggiato et al. 2017; Färber 2016; Imbsweiler et al. 2017a, b; Parkin et al. 2017; Portouli et al. 2014; Rasouli et al. 2017; Schneemann and Gohl 2016; Vissers et al. 2016).

According to Visser et al. (2016), day-to-day interaction between traffic participants is thought to be guided by five main factors: (a) traffic rules, (b) expectations, (c) individual differences, (d) behavioural adaptation and (e) informal rules and non-verbal communication. While traffic rules, individual differences and behavioural adaptations are factors that lie outside of what can be directly influenced by the AV designers, this paper focuses on documenting design considerations for the design of AVs that address expectations of traffic participants about the behaviour and reactions of AVs, as well as the informal rules and non-verbal communications currently in place for such interactions.

It is well known that the expectations of traffic participants have large influences on their decisions and actions in traffic (Houtenbos et al. 2004; Shor 1964). Björklund and Åberg (2005) report that traffic participants use traffic rules and design features of the road as well as the behaviour of other traffic participants as a basis for their expectations on how a traffic scene evolves. The behaviour of others consists of multiple implicit cues such as the vehicle’s approach speed, gap size or lane deviation and these are used to anticipate the intentions of others (Demiroz et al. 2015; Kitazaki and Myhre 2015; Sucha et al. 2017; Várhelyi 1998; Zito et al. 2015). A good example is the approaching behaviour of a vehicle at a pedestrian crossing. If the vehicle slows down this could be interpreted as the beginning of a stopping manoeuvre and a signal that the vehicle will stop for the pedestrians to cross (Rasouli et al. 2017; Schneemann and Gohl 2016). Additionally, the vehicle type and appearance provide another source of information that forms expectations (Klatt et al. 2016). Due to its size, a truck is associated with being slower than a sports car, but also likely more dangerous to collide with (Färber 2016). Thus, the vehicle type helps traffic participants to determine the general motion characteristics of a vehicle and its driving style.

In addition, informal rules and non-verbal communication play an important role in situations where the traffic rules lead to ambiguous situations, or where formal rules are replaced by informal ones. Informal rules are rules that are often not in line with the official traffic rules, but that are established in the interactions between traffic participants, such as yielding also for a cyclist at a pedestrian crossing (Bjørnskau 2017) or an informal first-come-first-served approach at some junctions (Ceunynck et al. 2013). In current traffic with conventional, manually driven vehicles, other traffic participants use non-verbal communication, such as implicit cues such as motion patterns and vehicle behaviour and explicit vehicle-based signals (e.g. flashing lights or turn signals) to predict the next manoeuvres (Dey and Terken 2017; Ba et al. 2015). Also, body language, such as head orientation, nodding, hand gestures and eye-contact are taken into account to coordinate common actions of traffic participants in specific situations (Guéguen et al. 2015; Kitazaki and Myhre 2015; Schneemann and Gohl 2016; Sucha et al. 2017; Ren et al. 2016). These signals are especially important in complex situations or in situations that need some form of cooperation such as urban environments. For example, Sucha et al. (2017) and Schneemann and Gohl (2016) reported that pedestrians use a driver’s non-verbal gestures and eye contact with the driver as indicators beside the deceleration of a vehicle that a driver is giving them priority. Another study explored the interaction of German drivers at bottleneck roads (Imbsweiler et al. 2017b). The authors found that the drivers most often used the driving manoeuvre itself as well as the headlight flashes as explicit signals for the coordination of their actions.

In all cases, the context of such information is important when interpreting its meaning, and the intention conveyed by the same signal may vary, depending on the traffic situation and the behaviour of others (Savigny 1995). For example, a flashing headlight may mean that a vehicle will wait for others to pass, e.g. at a bottleneck road, or that it is giving a warning to other drivers. Thus, humans need to constantly interpret the communication signals in light of the current driving context, in light of cultural norms (Färber 2016; Nordfjærn et al. 2014) and on past experiences made in comparable traffic situations (Herslund and Joergsen 2003).

As described in this section, specific means of communication are used between humans in today’s traffic to coordinate the interaction of different traffic participants. When introducing AVs in a mixed traffic environment on today’s roads, these automated vehicles need to provide some forms of communication to other traffic participants, to allow for safe, smooth and intuitive interactions.

3 Deriving design considerations for the design of the AVs’ interaction from human–human interaction

With the introduction of AVs in mixed traffic environments, designers are facing the challenge to design an appropriate interaction strategy between the AV and other traffic participants. It seems likely that central elements of the common human–human interaction need to be replaced by technical means for the AV-traffic participant interaction. As human–human interaction is quite complex we extracted four categories of information from the reported studies above. For this, we analysed which information human traffic participants use to build their expectations of future vehicle behaviour and their decisions and what kind of information they use to coordinate their actions. The categories are meant as a structuring aid for HMI designers in this complex design space and are intended to help designers to understand which information is needed by other traffic participants to successfully interact with an AV. The categories are defined as:

Category A—information about vehicle driving mode Based on the reported studies of Klatt et al. (2016) and Färber (2016) on vehicle type and vehicle size (Mathieu et al. 2017) we define a category named vehicle driving mode which includes all information that activates specific schemata about AV characteristics. Information of this category supports other traffic participants to develop the right expectations about the AV behaviour by informing others about the vehicle driving mode, e.g. if the vehicle is driverless or driven in automated or manual driving mode.

Category B—information about the vehicle’s next manoeuvres A central element of conventional human–human communication is the vehicle movement, for example changes in a vehicle’s trajectory, deceleration, and acceleration and resulting changes in gap size (Björklund and Åberg 2005; Demiroz et al. 2015; Dey and Terken 2017; Kitazaki and Myhre 2015; Sucha et al. 2017; Várhelyi 1998; Zito et al. 2015). Also vehicle-based signals are taken into account by others to predict the next manoeuvres (Ba et al. 2015). Based on these results we conclude that also an AV needs to provide information about its intentions that means about its next manoeuvres to support other traffic participants to anticipate the current and future behaviour of the AV. This information is summarized in the category vehicle’s next manoeuvres.

Category C—information about perception of environment In several situations vehicle movements (such as vehicle deceleration) are interpreted as an indication that a driver has detected surrounding road users and is willing to react to those. This expectation is becoming stronger even more when more explicit signals such as eye contact between traffic participants and/or head orientation of the driver are taken into account in low speed environments as indication that the driver has detected the traffic participant (Guéguen et al. 2015; Kitazaki and Myhre 2015; Schneemann and Gohl 2016; Sucha et al. 2017; Ren et al. 2016). Therefore, we define the category perception of environment for the AV design that covers all information that helps others to understand that they were detected by the AV. As common human–human communication is no longer available for AVs this interaction needs to be replaced by technical means.

Category D—information about cooperation capabilities According to Vanderhaegen et al. (2006) cooperation can be defined as managing the interference that can occur when two human agents each achieve goals that can interfere with those of the other. In common human–human interactions in mixed traffic, establishing explicit cooperation is often indicated by gestures, eye contact, or the use of headlight flashes in addition to the adaptation of vehicle movements (Sucha et al. 2017; Imbsweiler et al. 2017a, b). Therefore, we conclude that there is also a need for AVs to inform other traffic participants about their capabilities to start direct bilateral coordination of actions and to give advice to others regarding their next actions (e.g. give right-of-way to other traffic participants; ask them to stop; advice pedestrians to cross etc.). This information is summarized in the category cooperation capabilities.

Figure 2 summarizes the proposed categories of information for our design considerations.

Fig. 2
figure 2

The four main categories of information in the proposed design considerations for AV interaction with other traffic participants

Full size image

Based on the four extracted categories we analysed currently available research studies on communication needs of traffic participants interacting with AVs as well as the results of the Citymobil2 studies to find out more about the relevance of the categories for AV design. The results are documented in Sect. 4.

4 Communication needs of traffic participants interacting with AVs

The communication needs of other traffic participants with regards to AVs are currently a very new and unexplored field of research, where only results from a few relevant studies are available. This is partly based on the absence of AVs on today’s roads. However, the successful uptake, acceptance and use of such vehicles in the future depend on their ability to interact and cooperate successfully with other traffic participants. This section provides the first insight into the communication needs of other traffic participants during their interaction with AVs in a mixed traffic environment, based on a small number of completed studies in this area, including data from the CityMobil2 project. The four categories defined in Sect. 3 are used to structure the analysis of results.

4.1 Literature review

Category A—information about vehicle driving mode Existing studies reveal that other traffic participants expect to react and feel different in the presence of AVs compared to manually driven vehicles. Hagenzieker et al. (2016) conducted an interview study with 29 cyclists, using photographs as stimulus material. The participants were asked about their expectations and self-reported behaviours when interacting with conventional, manually driven cars, compared to AVs. Results showed that the participants were quite conservative, expecting to be detected more often when interacting with a manually driven vehicle than with an AV. Different expectations and reactions of pedestrians were also reported by the AVIP project studies (Lagström and Malmsten Lundgren 2015; Malmsten Lundgren et al. 2016). The authors explored whether pedestrians were comfortable to cross in front of an AV (passenger vehicle) during a field study with a ‘fake’ driver, and a questionnaire study using photographs. Here, the results revealed that pedestrians were less willing to cross in cases where the “fake human driver” on board of the AV showed uncommon behaviour, such as reading a newspaper or sleeping. This effect was even stronger when other traffic participants could not actually see any driver in the vehicle because he was hidden using a specially adapted vehicle (Habibovic et al. 2016). Thus, these authors conclude that there is a need for an AV to indicate that it is in automated mode to explain uncommon behaviour of the human user on board. From the findings of both studies it can be concluded that it is important to give surrounding traffic participants an indication that they share the road with an AV to induce appropriate expectations and reactions. First results from real environment are reported by Brown and Laurier (2017) who analysed several publicly available user videos of automated driving (up to SAE level 2). They found several situations where other drivers were confused by the reactions of the automated vehicle and conclude that an indication of the vehicle driving mode would help others to build appropriate expectations on the AV behaviour.

However, there are contrary findings on the question if the vehicle driving mode needs (AV vs. manually driven vehicle) to be displayed explicitly. Rothenbücher et al. (2016) report the results of a field experiment with a Wizard-of-Oz passenger vehicle that looked driver-less to other traffic participants. In contrast to the AVIP project studies, the authors did not find any safety concerns from the vulnerable road users (VRUs). They documented normal crossing behaviour, based on a series of video recordings of these interactions, with hesitant behaviour only observed when the vehicle misbehaved in some way. Another study also did not find any differences when exploring the behaviour of drivers of conventional vehicles interacting with an AV (GATEway Project 2017). In a simulator study of the GATEWAY project, the authors tested if drivers show different driving behaviours in an intersection scenario and an urban carriageway scenario when the recognisability of the AVs was varied (high visibility—sensors installed on a rack of the AV’s roof; low visibility—only sensors/no rack installed on the AV’s roof) and compared this to conventional, manually driven vehicles (no sensors on the vehicle’s roof). The results showed no differences in driving behaviour for the two scenarios, and subjective reports revealed that drivers’ decisions were mainly based on vehicle behaviour-related characteristics, such as the gap size and assessment of safety, but not influenced by whether an AV or manually driven vehicle was involved.

In addition, the implication of this information on the behaviour of other traffic participants is currently not fully understood. Millard-Ball (Millard-Ball 2017) forecasts in a simulation that such indication of the vehicle driving mode might lead to a negative behaviour adaptation of other traffic participants “playing” with the AVs, taking more risky behaviour or stopping them intentionally.

Category B—information about next manoeuvres Lagström and Malmsten Lundgren (2015), Malmsten Lundgren et al. (2016) reported in their above mentioned study that pedestrians took the vehicle movements (such as approaching speed of the AV and standstill position) as important indication for their decision to cross in front of the AV. Thus, they came to the conclusion that other traffic participants would profit from information on current and future manoeuvres of an AV. This is also supported by the studies of Rothenbücher et al. (2016) and the GATEway Project 2017 (reported above) that did not find any effect of an explicit information of vehicle driving mode but showed that surrounding traffic participants based their decisions on assumptions of next manoeuvres of AVs. This is in line with another study that showed a main effect of gap size but did not find any effects of additional display information for AVs (Clamann et al. 2016). Here, two different versions of a visual display providing advice about crossing or information on vehicle manoeuvres (in this case continuous speed information) were installed on an AV during a test track study. Results showed no differences in behaviour with the “no display” condition, with participants basing their decision to cross mainly on gap size, followed by vehicle speed. However, around 50% of the participants reported in a subjective interview that they would profit from an additional display for AVs even though this did not influence their decision to cross in this study. The need for explicit information was also reported in a study of Rodriguez (2017). She reports results from an interview study, an online survey and a focus group, conducted as part of the WEpod project. Among other results, participants in this study expressed a need for information about the AV’s next manoeuvres, especially regarding its turning and stopping behaviour of the complete driverless AV. That pedestrians benefit from additional information of current and next vehicle AV manoeuvres was also shown in a study of Böckle et al. (2017). The authors reported increased levels of self-reported safety and comfort when the AV indicated its manoeuvre intention using light and acoustic signals.

Category C—information about perception of environment In the studies of the AVIP project (Lagström and Malmsten Lundgren 2015; Malmsten Lundgren et al. 2016) the authors report that pedestrians based their decision to cross the road on driver-related cues—mainly on eye contact. In all conditions in which the fake driver of the AV was distracted, sleeping or absent, participants reported that they decided not to cross the road because they did not receive any cues from the driver that they were noticed and, therefore, felt unsafe. Participants also reported that they may expect to get confirmation from the person on the driver seat even if the vehicle is in an automated driving mode. The authors conclude that common gestures and eye contact might need to be replaced by other forms of communication, such that other traffic participants can confirm they have been detected by the AV.

Category D—information about cooperation capabilities In several situations there is no need and often no opportunity for signals such as eye contact or the use of gestures in the human–human communication, e.g. at night time. However, some complex traffic situations exist that require the coordination of actions between traffic participants and thus, further information from the AV on its cooperation capabilities might be needed. Imbsweiler et al. (2017b) conclude from their study results on human–human interaction that the combination of implicit signals such as vehicle manoeuvres and explicit signals to inform others about the willingness to cooperate would be most promising. This is in line with the results of Schneemann and Gohl (2016) who reported that ambiguous traffic situations between drivers and pedestrians in low-speed environments are often solved by using gestures and eye-contact. There is only one study known by the authors so far that tested a display of the category cooperation capabilities for an AV. In their study mentioned above, Clamann et al. (2016) tested a display variant giving advice that it is safe/not safe to cross for a pedestrian. It was tested against a display variant with continuous speed information of the AV and no additional display information. The authors did not find any effects of additional display information for AVs as participants based their decision to cross on gap size. This could be an effect of the selected scenario where the most relevant information for the crossing decision was gap size and no further information about cooperation capabilities was necessary.

4.2 Overview of the CityMobil2 studies

The EU Project CityMobil2 was a pioneer project for exploring the live interaction of AVs and other traffic participants in shared space (Alessandrini et al. 2014). The main focus of CityMobil2 was to realise a number of showcases and demonstrations of ARTS in different European cities [Lausanne (Switzerland), La Rochelle (France), Trikala (Greece), Oristano (Italy) and Vantaa (Finland)]. These AVs (ADS-dedicated vehicles according to SAE International 2016) were developed, tested and implemented in the respective cities, where they offered an additional service to the local public transport system. The AVs drove without a driver, on fixed routes through the city, interacting in mixed settings for some demonstrations (Italy, Greece, Switzerland and France). For legal reasons, an operator was present at all times. The AVs were electric vehicles and looked like small mini-busses (see Fig. 3).

Fig. 3
figure 3

ADS-dedicated vehicles used for the CityMobil2 showcase in La Rochelle

Full size image

To investigate the users’ attitudes towards these vehicles and their interaction needs as pedestrians and cyclists, a number of studies were conducted as part of the project. This began with a face-to-face interview study in Braunschweig (Germany) and Leeds (UK), followed by an on-site questionnaire study administered during demonstrations in La Rochelle, Lausanne, and Trikala. Finally, a focus group study was conducted with users of the AVs in La Rochelle. For the validation of the four categories of our design considerations, we analysed the results of all three CityMobil2 studies to find out more about the specific needs of traffic participants interacting with AVs. Results of the three studies are summarised below:

4.2.1 CityMobil2—interview study

Main objective Two interview studies were conducted in 2014 at the German Aerospace Center (DLR) and the Institute for Transport Studies at the University of Leeds (ITS Leeds). Participants of the interview study had no previous experiences with AVs and the main objectives of the interviews were to gain knowledge on where and for whom AVs might be usefully implemented, which kind of interaction with the AV participants would expect and what kind of design they would prefer.

Method Two semi-structured interviews were conducted, one in each location. A total of 26 participants took part in the explorative face-to-face interviews (see details in Table 1).

Table 1 Number, gender and age range of participants in the interview studies at DLR and ITS Leeds
Full size table

All participants were in possession of a valid driving licence. Participants were paid for their participation (interview duration of 40–60 min, 10€ per hour in Braunschweig, 15£ in total for participation in Leeds). Both interviews were similar in their structure and started with an introduction of AVs and their capabilities (using photo and video material). The interview consisted of three main sections: the first section dealt with users’ expectations regarding AVs. Here, we sought participants’ views regarding where and for whom AVs might be usefully implemented. In the second section, participants were shown pictures of different traffic scenarios, and they were asked to imagine themselves in these situations, as either a cyclist, pedestrian or car driver. They were then asked to suggest the expected behaviour of the AV, when interacting with them in these situations, and also how they might interact with the AV. The scenarios were chosen to illustrate settings where an interaction between different actors was required and included: (a) a pedestrian zone, (b) intersection with traffic lights, (c) zebra crossing, (d) intersection without traffic lights, (e) bicycle track and (f) merging from a minor to a major road. Participants were also asked what kind of additional information they would like to receive from the AV in these specific traffic scenarios, in the absence of a human driver. In the third section of the interview, participants were asked about their personal preferences regarding the design of interaction-related information from the AV (e.g. what kind of communication channel or location for information would be preferred by the participants?). As the focus of this paper was on AV design, only the results of interview sections two and three are presented here.

Results

Category A—vehicle driving mode Participants pointed out that they would like to be able to identify an AV by its appearance [example statement: An AV should be identified as an AV on first glance]. Furthermore, a distinction between different types of AVs and their abilities regarding maximum speed was mentioned as helpful to build up appropriate expectations [example statement: What is the maximum speed? Is it an AV for the city or for the motorway].

Category B—vehicle manoeuvres The majority of participants at both interview sites pointed out that in clear situations (regulated by traffic law and road infrastructure), they would expect the AVs to respect and obey the traffic rules (scenario b, c and e). For example, While crossing the street as a pedestrian at a traffic light (scenario b), all 26 participants would expect the AV to stop and participants did not see the need for additional information presented by the AV that goes beyond the conventional, established communication behaviour. However, in less structured environments, such as a zebra crossing or driving on a bicycle track, participants seem to feel less safe and the majority would prefer additional information about the AV’s behaviour [example statements: Maybe a light signal, so that I know it [the AV] will stop soon; a sign on the road that I know it [the AV] will drive until there and not further.]. In addition to these scenarios, participants mentioned that they would appreciate to have additional information regarding the future behaviour of the AV while crossing the street without traffic lights (scenario d) [example statement: If [the AV] detected me, and will stop or decelerate. I expect more from an AV than from a normal car].

Category C—perception of environment In multiple scenarios participants pointed out that they would like to have information if they were detected by the AV. This was true for scenarios where the pedestrians had the right of way [zebra crossing (a), traffic light (b)] [example statements: Person detected; I [the AV] have detected you] and in scenarios where the AV had the right of way [intersection without traffic lights (d)] [example statement: Information that it [the AV] detected a human or an obstacle]. Especially in scenario e (bicycle track) where the participants should take the view of the cyclist they asked for an indication that they were detected by the AV [example statement: It would be perfect if there would be a picture or symbol for “cyclist detected” so I would know that “I have been detected” and I would just ride on].

Category D—cooperation capabilities In scenarios where the pedestrian had the right of way [zebra crossing (a), traffic light (b)], participants mentioned that the AV could signalise when it is safe to cross [example statement: [..] At a specific distance a light on the AV will light up and it is safe to cross and if the light goes off you cannot cross the zebra crossing [..]]. Regarding the scenario where the pedestrian had not the right of way but was instructed to imagine he/she would pass the street in front of the AV [intersection without traffic lights (d)] participants would even expect some kind of reprehension by the AV to indicate that the AV is not willing to cooperate [example statements: I ran over the street and the AV honks at me [..]; maybe an acoustic signal like “you know that I [the AV] am here, watch out, if you want to pass than hurry up.”].

4.2.2 CityMobil 2—on-site questionnaire study

Main objectives While the interviews were conducted with participants who had little or no experience with AVs, the on-site questionnaire recruited participants who had actually experienced interaction with the AVs during the demonstrations of the vehicles provided by the EU Project CityMobil2. The questionnaire study was developed to gain further insight into the factors important to road users in the interaction with AVs.

Methodology The questionnaire was completed by 664 (386 male, 278 female) participants across the three demonstration sites: La Rochelle, Lausanne, and Trikala (see Madigan et al. 2017, 2016 for further details). An important criterion for the participant selection was their knowledge regarding interactions with the AV. Only participants who had interacted with the AV at least once were asked to complete the questionnaire. Participants were recruited near the demonstration sites on a voluntary basis. The questionnaire consisted of two parts. The second part (35 items), which is described in this paper, focused on understanding the interaction of traffic participants with AVs. Participants were asked if they felt safe interacting with the AVs and how this feeling of safety was influenced by different environmental factors (shared spaces vs. dedicated lanes). Participants were also asked what kind of information they would wish to receive from the AV, regarding its current and future behaviour.

Results As the results of the study are presented in Merat et al. (2018), we provide a brief overview of these for the purpose of this paper.

Category A—vehicle driving mode There are no results on this category as the CityMobil2 vehicles were easily recognizable as driverless, ADS-dedicated vehicles with no driver on board. Thus, this category was not assessed in the questionnaire study.

Category B—vehicle manoeuvres Overall, the main findings from this study showed a significant difference in traffic participants’ needs and expectations with regards to the vehicle manoeuvres when the AVs were demonstrated on a dedicated track vs. a shared space. Results showed that in the presence of road markings, participants believed that the AV had priority, but that when there were no road markings they, as pedestrians, had priority and the right of way. Regarding information about the manoeuvre of the AV, participants reported a greater need for information about the behaviour of the AV (whether it is stopping, turning, going to start moving) in the absence of road markings or dedicated tracks for the AV. Participants ranked information about whether the AV is stopping, turning and starting as more important than information about the AV’s current travel speed. This is possibly because (due to safety reasons in a mixed traffic environment) the vehicles travelled at quite a slow speed (around 10 km/h), but also because this information is perhaps easier to observe than the intentions of the AV.

Category C—perception of environment Participants from all three locations rated information about the perception of the environment, e.g. confirming they had been detected as important information from the AV, again highlighting the importance of feeling safe when interacting with these vehicles.

Category D—cooperation capabilities There are no results on this category as this category was not assessed in the questionnaire study.

4.2.3 CityMobil 2—focus group study

Main objectives To gain further insight into people’s attitudes towards AVs two focus group discussions were conducted in La Rochelle in 2015, approximately a month after the end of the demonstrations. The main objective of the two focus groups was to understand and collect further insights regarding the experience of participants during their daily interaction with the AV.

Method Nine male and eleven female participants took part in the focus group discussions, which were conducted separately, on the same day, by two French facilitators. The groups were divided by gender to avoid any gender-based effects of mixed group discussions. All participants were residents of La Rochelle and confirmed that they had interacted with the AV at least once during the demonstration period. The experience of participants with the AV was discussed in the focus group setting. The focus groups were both audio and video recorded, and the following transcription and translation to English was done by a qualified transcriber and translator. Analysis involved categorising participants’ statements (Braun and Clarke 2006) into seven different categories, with regards to (a) the AV in general, (b) the manoeuvre communication of an AV, (c) the environment perception capabilities of the AV, (d) the cooperation and interaction strategies of an AV, (e) the safety and traffic rules for an AV, (f) the influence of infrastructure (e.g. dedicated roads, lane markings) for an AV and (g) the design (e.g. appearance, size) of an AV. Only the results for a, b, c, d and g are reported in the following as they are of highest relevance for the categories of the design considerations.

Results

Category A—vehicle driving mode Participants pointed out that it is important for the design of the AV (g) to look unlike normal public transport systems to be clearly identified as an AV [example statements: “For me, the fact that it looks like a bus is bad.”, “It must break with the typical model for buses.”]. Participant expressed the importance of the AV design to build up the right expectations on the behaviour and next manoeuvres of the AV. Participants pointed out that if an AV is too symmetric in its appearance/format it will be hard to distinguish between the front and the back of the AV, which make a prediction of the moving direction harder [example statements: “[…] when the vehicle is at a stop, you don’t really know in which direction it’s about to set off. And when you’re a pedestrian or a cyclist or you have a pram, even if it’s not going fast, you don’t really know in which direction.”, “[…] outside of the reserved lanes, it’s true that the direction isn’t clear.”]. Furthermore, a futuristic design of an AV could also attract people to use an AV [example statements: “If we want people to ride in it, it must be cool, attractive, sexy. Nobody wants to ride in it now, because it’s not attractive.”].

Category B—vehicle manoeuvres When participants were asked if they required any specific information about the AV in general (a) and about the AV’s manoeuvres (b), they reported that they expected AVs to have at least the same signals as common vehicles for indicating the next vehicle manoeuvre (e.g. indicator, brake lights, horns) [example statement: “Like for a car. I think it [the vehicle] should have the same signals as on an ordinary vehicle.”]. In case that the AV would be an electric vehicle such as those implemented within CityMobil2, participants expressed their need to have additional signals so that the approaching vehicle could be recognised more easily than it was the case in the CityMobil2 demonstrations [example statements: “It’s like people on bicycles, because when you’re a pedestrian you don’t hear them coming, and I’m often startled by cyclists on the pavement, and it’s the same here with the vehicle (AV).”, “I’m for anything that can help with signage. The fact that there’s no noise is a big plus, it should be retained. So I think that other signalling methods should be found”]. Participants also mentioned that priorities and traffic rules for AVs (e) are unknown by most of the traffic participants making it hard for them to predict next manoeuvres [example statements: “Which is why there should be rules. […] In this case, the vehicle is new, and in the middle of traffic, it’s motorised, and no one knows the rule to follow.”]. Regarding the influence of infrastructure (f), participants reported that they would feel safer if the AV was driven on a clearly marked and predefined track, so that traffic participants could anticipate where the AV will go next [example statement: “I also noticed that there are no route markings, and that you don’t really know where it’s going. It is a bit of a hassle, especially when you’re on a big square where everyone is walking;”].

Category C—perception of environment Regarding information about environment perception of the AV (c), interviewees pointed out that they would like to know if they were detected. This might be caused by a general lack of knowledge of how the technical system works [example statements: “Isn’t there a system that allows it to detect that others are present?”]. Participants did not express a general need for signals for all obstacles detected, but discussed that it would be helpful if the AV could warn pedestrians when they are moving in its way [example statement: “[..] like the buses—when a pedestrian is in the way, they have a kind of sound signal that warns the pedestrian that the bus is coming.”].

Category D—cooperation capabilities Regarding the cooperation and interaction strategies of an AV (d), participants mentioned that they would benefit from additional information for interaction with the AV that could help to coordinate their actions [example statements: “Often, this happens with eye contact. You work out whether you should give way or not, and in this case, you don’t know, so there is doubt about whether to yield or not.”, “Colour codes that indicates: Now I have right of way, now I don’t, as if you were interacting with a kind of driver.”].

4.3 Summary of results on communication needs of other traffic participants interacting with AVs

In the following, the results of the literature review and the CityMobil2 studies are summarized for each of the four categories of information.

Category A: information about vehicle driving mode The few research studies available so far show mixed results with regards to the need of traffic participants for explicit information about the vehicle type (see Sect. 3). In the research study of Lagström and Malmsten Lundgren (2015), Brown and Laurier (2017) and in the CityMobil2 studies participants expressed their need to understand that the vehicle is driven in an automated mode while the GATEway study (2017) did not find any effect of such indication on the driving behaviour of surrounding drivers. In situations where the vehicle was driven completely driverless (CityMobil2 studies), the user on board behaved unexpectedly (e.g. reading the newspaper; see Lagström and Malmsten Lundgren 2015) or the vehicle showed unexpected behaviour (Brown and Laurier 2017) there was a need for additional information about the vehicle driving mode. However, the implication of this information on the behaviour of other traffic participants is currently not fully understood (Millard-Ball 2017). Thus, there is little knowledge so far if the category of information on vehicle driving mode provides advantages or disadvantages when implementing AVs into mixed traffic environments. It is also not known yet if other traffic participants need to distinguish between lower (SAE level 2–3) and higher levels of automation (SAE 4–5) and driverless, ADS-dedicated vehicles. Further research on how an indication of the vehicle driving mode influences the behaviour of surrounding traffic participants is needed. Designers should consider this category carefully in their design and decide depending on the envisioned application of their AV if information on the vehicle driving mode is needed.

Category B: information about AVs’ next manoeuvres This category includes all information that supports other traffic participants to anticipate the current and future vehicle manoeuvres. Applied to AVs, all study results summarised in Sect. 3 show that these conventional communication means, and the vehicle behaviour itself, are of significant importance for providing other traffic participants with information about future AV manoeuvres and supporting their decision making process. This could be also supported by explicit indication of next manoeuvres by new external HMI (Böckle et al. 2017; CityMobil2 studies). Also, dedicated environments for AVs help others to better predict next manoeuvres as the AVs are running on dedicated tracks. Thus, information about an AV’s next manoeuvres is of central importance for the safe interaction with AVs in mixed traffic environments and should be considered by designers in any case.

Category C: information about AVs’ perception of the environment All information that helps other traffic participants predicts if the AV has detected them is summarised in this category. For AVs, the communication that is based on the signals of the driver on-board is no longer available and needs to be replaced by technical means. In the research study of Lagström and Malmsten Lundgren (2015) and in the CityMobil2 studies (see Sect. 3.2), participants stressed their needs to understand if the AV has detected them or not. In contradiction to that Rothenbücher et al. (2016) found normal crossing behaviours of pedestrians when interacting with a driverless vehicle and conclude that designers can avoid signals to replace, e.g. the conventional eye-contact between humans, and only need to provide some explicit design for the minority who are irritated by the fact that they cannot see a driver on board. Thus, based on what we know from research so far, designers need to decide from case to case, and for their specific AV application if, and when, information on the AV’s perception of the environment is required. It also seems very important that the information given by the AV is not contradicted by the on-board user of the AV. This could be particularly problematic if other traffic participants use the body language or gestures of the user on board for deciding their next actions, especially if this information is in conflict with the AV’s intention. Anecdotal evidence, observed in the CityMobil2 trials, illustrated frustrations and near-miss scenarios in such circumstances.

Category D: information about AVs’ cooperation capabilities. While category C covers only the status information on the AV’s perception of the environment, category D goes one step further. It includes all information that an AV can send to other traffic participants to start a direct bilateral coordination of actions and to give advice to them regarding their next actions. Participants in the CityMobil2 studies explained that they would benefit from any information regarding whether they are allowed to cross or not, whereas participants of the study conducted by Clamann et al. (2016) did not benefit from any advisory information on an external display. In general, there is currently no consensus as to whether AVs should be allowed to establish explicit cooperation by giving explicit advice to others about their next actions as humans do today, or should stick to pure intention based information about its next manoeuvres (Andersson et al. 2017). There is also a risk that the AV will provide false or at least risky advice to other traffic participants, and it is also challenging to know which traffic participant is the target of such information. Therefore, an understanding of how to provide context and individually specific advice may be required for such scenarios.

Summing up, the relevance of each of the four categories can vary between different traffic situations and vehicle applications—some categories of information are highly relevant, while others are either not relevant or are less relevant for the safe interaction in specific traffic situations. Thus, the intention of the authors is not to take the four categories of information as strict requirements that need to be addressed in every case, but rather as an important design guideline for inspiring the design process. The designers of AVs should think carefully about which categories of information are essential, which are not relevant for specific traffic situations and which design options are appropriate to transfer the relevant information. Future research will also provide us with further insight into what information is needed in specific traffic situations, allowing a continuous update of the communication needs of other traffic participants with AVs.

5 Design options for the transfer of information

In this final section of the paper, we provide an overview of the different design options that AV developers and designers will need to consider, in order to relay the information outlined by our design considerations. A limited number of design concepts and patents for AVs have already been investigated and published by industry and the research community. Here, we consider some examples (as of mid-2017) on the basis of our proposed four design options: (1) the design of the infrastructure, (2) the design of the vehicle shape, (3) the design of vehicle behaviour and (4) the design of external HMI elements for visual and acoustic communication. Table 2 gives an overview of the chosen design options, per design concept or patent. As illustrated below, most design concepts relate to vehicle-based design elements. However, especially for driverless, ADS-dedicated vehicles the design of the infrastructure is used to provide other traffic participants with the relevant information about an AV. The design options listed below come along with different advantages and disadvantages (see Sect. 5.1 for a detailed discussion) and should be considered carefully by the design team.

Table 2 Categories of information and suitable design options to transfer information (marked with “+”)
Full size table
  1. 1.

    Design of infrastructure—designing the infrastructure (such as marked lanes, specific road signs, or physical separation of tracks) could support other traffic participants to understand the driving mode of the vehicle (Category A of the design considerations) and what kind of manoeuvres are possible by the AV (Category B). Separated and/or marked lanes for AVs make it easier for other traffic participants to predict the path of the AV. Systems such as CityMobil2 applications on demonstration sites (see CityMobil 2, Alessandrini 2016), the Rivium Group Rapid Transit (2getthere 2017b) and the Personal Rapid Transit in Masdar City (2getthere 2017a) use especially prepared infrastructure to allow for safe operation of fully automated vehicles. This is in line with the results of the CityMobil2 studies, where participants mentioned that dedicated tracks and corresponding road markings would help them identify the path of the AV, increasing their feeling of personal safety, because they were able to predict the next manoeuvres of the AV.

  2. 2.

    Design of the vehicle shape—designing the vehicle shape can support comprehension of the vehicle’s driving mode (Category A), and an ability to identify the AV in the external environment. One of the most well-known examples of an AV that does not look like a passenger vehicle is the Google car, a vehicle that has a very characteristic “bubble” shape, which allows it to be easily identified as an AV (Waymo 2017). The same is true for AV shuttles that can be easily detected as driverless, ADS-dedicated vehicles since they do not have a conventional driver seat on board and look different to a conventional passenger vehicle or bus (Alessandrini et al. 2014; 2getthere 2017a, b). The design of the vehicle’s shape can also support or hinder the prediction of manoeuvres (Category B). For example, from CityMobil2 research we know that symmetric vehicle shapes make it much harder for other traffic participants to predict the direction of travel, since the forward and rear view look identical, allowing travel in both directions.

  3. 3.

    Design of vehicle behaviour—the adequate design of vehicle behaviour, such as an indication of its intention to accelerate, decelerate or change direction (Category B), provides other traffic participants with an understanding of its manoeuvres, which is especially critical in shared space. The appropriate design of vehicle manoeuvres also influences the behaviour of surrounding traffic participants and increase traffic efficiency and cooperation (Sadigh et al. 2016). Vehicle movements might also be used by other traffic participants to interpret if they have been detected (Category C). For example, if the AV slows down when approaching a zebra crossing, other traffic participants might assume that the AV will stop and that this is because they were detected. Some authors suggest that AVs should mainly be based on transferring information to other traffic participants by designing appropriate vehicle manoeuvres and not putting too much focus on extra HMI (Risto et al. 2017).

  4. 4.

    Design of HMI elements for the visual and acoustic communication—the design of HMI elements can help transfer relevant information, using the visual or acoustic communication channels. Visual information can either be sent in the form of text messages, icons or light patterns displayed on the AV or projected on the road. While visual signs in the form of lights might be used to inform other traffic participants about the vehicle driving mode and vehicle manoeuvres (Category A and B), visual text messages or icons might be chosen to address only one specific recipient and provide advice on ensuing actions (Category C and D). The same is true for audio signals, where generic audio signals might be used to address all other traffic participants in the surrounding area. Compared to that, spoken words can be chosen to give some kind of specific information or advice such as “Walk; AV is stopping” to inform other traffic participants about the AVs’ intentions (Category B) and cooperation capabilities (Category D).

With regard to concepts and prototypes being considered by vehicle manufactures, almost all make use of additional visual and acoustic communication. Specific HMI messages such as text, signs or light patterns are applied to help others identify the vehicle driving mode (see AVIP—Lagström and Malmsten Lundgren 2015; Concept-I from Toyota—Fink 2017; BMW Vision Next 100 2016; FORD 2017). In addition to conventional communication devices such as indicators, brake lights, rear lights and sounds, AV design concepts often make use of additional HMI devices to inform others about the AVs’ next manoeuvres. These are displayed at the front and rear of the car (Mercedes Benz F015 2017; Nissan Concept 2017; AVIP—Lagström and Malmsten Lundgren 2015; Duke University—Clamann et al. 2016; Semcon Smiling Car—Semcon 2017; Google US9196164B1 patent—Urmson et al. 2015; Concept-I Toyota—Fink 2017; FORD 2017), or devices that project information on the road surface (Mitsubishi road-illuminating directional indicators—Mitsubishi Electric 2015; Mercedes Benz F015 2017) to inform others about the vehicle’s next manoeuvres, the perception of the environment or cooperation capabilities and advisory information. For some concepts, acoustic information is also used to provide sound signals or spoken words (Mercedes Benz F015 2017; Google patent—Urmson et al. 2015).

Table 2 provides an overview which design options are suitable for the transfer of information.

As mentioned above, some design concepts and patents for AV already provide first insight by which technical means other traffic participants could gather information from the AV. Table 3 summarizes the addressed categories of information and the chosen design options per existing AV design concept.

Table 3 Overview on addressed category of information and chosen design options (marked with asterisk) of some existing AV design concepts (as of mid-2017)
Full size table

5.1 Assessment of different design options

The design options described in Sect. 5 above are meant to be used as guidelines for designers to work on an appropriate design for AVs. Depending on the specific purpose of the AV, and the traffic environment in which the AVs will be implemented, some design options might be favoured over others. Table 4 gives an overview of some of the benefits and disadvantages that each design option is likely to provide. In addition, the technical feasibility of the design options needs to be considered. Some of the human factors related disadvantages, such as strong limitations for visually impaired people of visual signals, can be overcome by including multi-modal solutions, which are likely to improve the interaction and cooperation between the AV and other traffic participants. This is of utmost importance as the AV design should be inclusive and consider various groups of traffic participants, e.g. disabled people, children with no reading skills or elderly people. However, we assume that it is not necessary to display all relevant information on the visual or acoustic communication channel, but to only use these channels to support the interaction in specific traffic situations. To support these messages, essential information can also be relayed via specific infrastructure design, the vehicle shape and the appropriate design of vehicle behaviour. Vehicle behaviour is already used as one of the main sources to allow for safe interaction between traffic participants. In any case, for a multi-modal approach of external HMI and vehicle behaviour, the chosen design should be well aligned and appropriately synchronised. This means that the vehicle behaviour should be in line and timely synchronized with the information sent by the external HMI elements, and not contradicted by any on-board users/operators. This will require an understanding of the timing for the presentation of HMI messages and the control of the AV manoeuvres. In addition, from a Human Factors perspective, it will be important for vehicle manufacturers to agree on the same standards especially for the design of new external HMI (Emmenegger et al. 2016).

Table 4 Overview of advantages and disadvantages of the different design options
Full size table

6 Summary and conclusion

While developments in AV technology are occurring very rapidly, one of the great challenges facing engineers, designers and human factors researchers in this area is an understanding of how we can integrate AVs in mixed traffic environments, in a safe and intuitive manner. To achieve a better understanding of the needs of other traffic participants, we analysed existing research knowledge on the common human–human interactions in mixed traffic settings, as well as we considered some preliminary research results on the needs from other traffic participants interacting with AVs, obtained primarily from the CityMobil2 project. On this basis, we describe the design considerations covering four categories of information that designers can use for their design work in this context. The considerations cover information on vehicle driving mode, manoeuvres, perception of the environment and cooperation capabilities. These categories reflect the expressed needs of other traffic participants in the vicinity of the AV, supporting the safe interaction of all actors. The design considerations are intended to help designers of AVs to structure the complexity of the design space and provide support for finding appropriate design solutions to assist such interactions. Our preliminary understanding is that there is currently no single solution that applies to all traffic situations and vehicle types, as the requirements for the interaction design are high, and the implementation scenarios for AVs are still highly versatile. Designers must, therefore, consider the relevant information, based on the deployment setting of the AV. As AVs become more commonplace, and are deployed more widely, such considerations may also help provide guidance for manufacturers and designers who wish to work on standardising some of the above factors. Such a standardisation is likely to help ensure appropriate use, trust and uptake of these vehicles and ultimately safeguard improvements in road safety.