Virtual and Augmented Reality in the Disaster Management Technology: A Literature Review of the Past 11 years

1 Introduction

Whether they are human-caused, natural, or hybrid (Shaluf, 2007), disasters are significantly responsible for the massive destruction of both life and property. Appropriate preparedness for disaster can make significant difference on society. Disaster management is “the body of policy and administrative decisions, the operational activities, the actors and technologies that pertain to various stages of disasters at all levels” (Lettieri et al., 2009). Although technology is a single aspect of disaster management, it becomes apparent in the discussion that its role is intricately coupled with the rest of the aspects of disaster management. Within the disaster management field, VR and AR technology has discovered some unique opportunities in the areas of information communication, rapid damage assessment, reliable Building Information Modeling (BIM), and so on.

Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR), collectively called Xtended Reality (XR), are the application areas of computer graphics that allow the user to interact with the computer-generated imagery in both real and virtual environment scenarios. VR scenes consist of a fully virtual environment with virtual elements that obscure the physical environment, and simulate the physical objects whereas an AR scene consists of a real environment with integrated but non - dominating virtual objects that can react to the user and/or the scene (Anderson et al., 2021). In addition, we consider an umbrella term MR to be synonymous to Augmented Virtuality where the virtual elements fuse with real elements to provide a perception of a single element to the user (Girau et al., 2019). VR allows rapid prototyping at a low cost and early human-in-the-loop design (Anderson et al., 2021). This allows the integration of physical systems in the simulation cutting down costs in the modeling and testing of the models. These features make these tools extremely useful for Disaster Management Applications. Various governmental agencies like New York Office of Emergency Management, Los Angeles Police Department (LAPD), DHS Federal Emergency Management Agency, etc. and academic institutions such as The University of Illinois at Chicago School of Public Health, the University of Minnesota Public Health Preparedness Center, the Johns Hopkins Office of Critical Event Preparedness and Response etc. are utilizing and exploring the applicability of the XR technology in Training activities (Hsu et al., 2013).

Nevertheless, as developing technology, disaster management AR/VR systems, have some barriers to overcome such as efficient processing in low-cost XR, smart rendering, and minimizing motion sickness (Kawai et al., 2016; Lovreglio et al., 2018). Similarly, every disaster simulation designer needs to balance ecological validity and the potential psychological effect of a scenario.

This systematic review (Borrego et al., 2014) seeks to add to the theoretical framework in the major trends of XR usage in disaster management by exploring the research questions below:

Q1. What are the trends of XR in Disaster Management Technology?

Q2. Why is XR preferred as a Disaster Management technology and what is its boundary?

Q3. What are the common research areas on the application of XR in disaster management technology?

Q4. What are the challenges and gaps in the literature in the application of XR in disaster management technology?

The paper starts with the reviewed papers’ demographics (Q1), search tactics, selection criteria, and classification based on their source. After that, it discusses the analysis of the reviewed papers based on their application focus (Q3) with an embedded report of the peculiarity and boundary of the XR (Q2). Then, it discusses the limitations of the research and suggests possible future research opportunities (Q4). This paper maps the recent and current application areas of XR technology in disaster response on the civil infrastructure. The paper performs a critical analysis of the conference and journal papers published from 2009 to 2019 and describes their focus, results, and problems. Finally, this paper provides directions for what the current literature does not address. The paper will be specifically useful to scientists and engineers with a research interest in advanced visualization and modeling applications in the area of emergency and disaster risk management.

2 Related Work

The main objective of the Literature Review is to investigate the major research directions for the application of XR technology in disaster management. A significant amount of research papers were tracked in the review but no Literature Review papers in this interdisciplinary field were found.

3 Methods

The literature for this paper was tracked and synthesized using a systematic review protocol. The scope of this paper spans two large fields: disaster management and XR technology. After a rigorous examination of systematic review protocols, Borrego et al.’s paper (Borrego et al., 2014) was found to be designed to be relevant for the engineering and interdisciplinary field of disaster management. The paper was implemented to conduct the review and report the findings which involved: 1) scope identification, 2) resource tracking, 3) critique and 4) synthesis.

3.1 Identification of Scope and Research Questions

The population, intervention, comparison, and outcome (PICO) are constructed using the PICO framework (Table 1) to identify the scope of the research questions. To investigate the boundaries of the effectiveness of the XR technology, the XR applications research papers are compared with the traditional disaster management methods. Because the potential outcome of the research demands both quantitative and qualitative research, the overall research was designed using multiple approaches under a mixed design approach (Borrego et al., 2009). The qualitative outcome (i.e., scope) was a major priority than the quantitative outcome (i.e., effectiveness), the embedded research design (Borrego et al., 2009) was used with a focus to map out application areas with a higher priority than the effectiveness and boundaries. The findings were also reported accordingly.

TABLE 1. Summary of PICO.

3.2 Inclusion Criteria

The major objective of the research review was to investigate a recent research trend in the application development of XR technology to support emergency and disaster risk management. The papers published from 2009 to 2019 in both the journal, conferences, and book chapters were considered for review. Similarly, only the papers published in the English language were selected for the review. The papers were searched using specific keywords containing the words related to “disaster” and “XR technology”.

3.3 Finding and Cataloging Sources

The tracking source had to be chosen to identify the progression research papers mostly found in conferences and seminal research papers in journals. For this purpose, two tracking sources: Google Scholar (2020) and Web of Science (2020) were used to search for relevant papers. The catalogs were searched under the date range setting mentioned in the inclusion criteria using the respective keywords documented in Table 2. Google Scholar returned a large number of documents, but most of them were relevant to only one domain. Although the authors had to experiment with a significant range of keywords, the database returned papers from diverse venues which has a strong contribution in tracing unique research works happening in this interdisciplinary field.

TABLE 2. List of keywords used in respective tracking source.

3.4 Critique and Appraisal

The retrieved papers were processed using the systematic protocol presented in the Table 3. The tracking source settings were maintained according to the predetermined eligibility criteria. After that, the papers were searched using the outlined search strategy. Then, the individual papers were manually screened using the displayed title on the results of the tracking source. Because of the interdisciplinary nature of the review, screening was performed by looking at the title of the papers and making sure they had both XR related terms like but not limited to “Augmented Reality”, “Virtual Reality” and “Mixed Reality” and disaster management related terms like but not limited to “flood”, “response”, and “emergency”. Some titles had keywords like “wearables”, “simulators”, “systems” along with the keywords related to the disaster management. Authors went through their abstract to understand their relevance in the screening process. Google Scholar output numerous research applications related to disaster management but they were developed using 3D computer graphics only. This step was really important in screening a large number of papers in google scholar. Thereafter, only the relevant papers were used for the further study of their abstract and introduction. This phase was significant in excluding patent documents. Finally, selected papers were explored in-depth and the useful information was synthesized and recorded based on the headings Keywords, Title of Paper, Year Published, Tracking Source, Description of Work, Disaster Focus, Findings and Problems associated with the research. During this phase, it was found that some papers that used XR in disaster management didn’t necessarily develop an XR system but only used the system to conduct behavioral studies. Such papers were manually searched and excluded from the review.

TABLE 3. Criteria for the PRISMA flowchart.

4 Analysis

4.1 Demographics

After the relevant papers were collected, demographic analysis of the papers was performed based on the distribution of papers based on the published year, publication venue, and disaster focus.

Figure 1 shows the distribution of papers based on the year published which shows that the trend of research in this area is rapidly growing. The reason for this trend could be attributed to the acceptance of the XR technology by the disaster management community which could be a reaction of the recent commercialization of the XR products in video games.

FIGURE 1. Distribution of papers based on publication year.

Figure 2 provides a distribution of the papers based on their focus on disaster events. The chart shows that most of the papers focused on disaster preparedness, rescue, and reconstruction. These papers did not focus on a particular disaster scenario, but assumed a disaster scenario and built a system that could be used to achieve their disaster management objectives for multiple disaster situations.

FIGURE 2. Distribution of papers based on nature of disaster.

Similarly, Table 4 represents the distribution of papers based on the venue of publication. The selected papers came from an extremely diverse publication sources with no more than three papers from a single publication source. This was true for well-known XR venues such as IEEE, ISMAR and ISCRAM. We, the authors, attribute this finding to the use of unique tracking sources to search papers. However, the authors are also concerned that this result could be because of the lack of a publishing venue for this interdisciplinary field. It was identified that this area is a relatively new field that not only needs a significant research framework, but also the venues that publish such work.

TABLE 4. Distribution of selected papers based on publication source.

4.2 Application Domain

The review of 84 papers that contributed to the development of XR technology in the domain of Disaster Management found that the major application areas of XR in Disaster Management are: 1) computer simulation modeling, 2) interaction techniques, 3) training, 4) infrastructure assessment and reconnaissance and 5) public awareness.

The first two applications: computer simulation modeling and interaction techniques are strictly research application areas that play a crucial role in serving the remaining three service/product-type applications. As it will be apparent in the discussion, the majority of the papers focused on modeling and simulation and interaction techniques. Every new XR system has to go through the Modeling and Interaction design. This finding also provides evidence that XR research for disaster management is a relatively new field.

The categories the authors based on the literature search are not mutually exclusive. All of these applications require modeling and interaction design. However, because of the nature of these applications and the impact of the XR technology in revolutionizing these applications, there is a convincing argument that each of these applications and their sub-categories has the potential to evolve as a separate area for research activities.

5 List of Category of Paper

The definitions for each of the mapped categories along with their demographics are provided below with a basic purview of each of the categories. As shown by Table 5, the categories are mutually non-exclusive. Any single research work could feature multiple aspects of the categories. A summary of the taxonomy of these applications along with the papers used to construct the synthesis is provided in Table 6. The papers referenced in the title of the application area (e.g., Computer simulation modeling, interaction techniques, etc.) are used to construct introduction and limitation sections whereas the ones cited in the subcategories (e.g., Surveillance and Monitoring Network, Command systems, etc.) are used to construct their respective paragraphs. These categories are discussed in-depth in the next section.

TABLE 5. List of papers used for synthesis.

TABLE 6. Organization of the review papers.

5.1 Computer Simulation Modeling

Modeling is the process of creating a computer-based representation of the natural process or phenomena that produces a close input-output relationship as the original system (de Jong and van Joolingen, 2008). The research trend in this area is directed in the development of an immersive system that can simulate a specific disaster scenario to the user. While any application area provided in Table 5 needs an immersive environment, this application area is strictly focused on tracing out domains on the computer generated modeling.

5.1.1 System Development

The system development is a combination of phases involved in the validated construction of the engineering design of the system of interest (Wasson, 2016). The architectural development works focused on the development of cyber-physical system designs that allows the users to be immersed in a unique and intelligent simulation. Subcategories of the system development research areas are as follows: Surveillance and Monitoring Network, Command System, Hardware Accelerated Systems, Serious Games (SG) and Distributed Training Architecture.

5.1.2 Simulation Development

Simulation is the implementation of a model to evaluate the result (Ingels, 1985). Simulation development research focused on the development of the 3D immersive environment and/or scene for a disaster scenario. The sub-categories under this heading were Natural Disasters, Humanoid Simulation, and Simulation for Skilled Oriented Applications.

5.2 Interaction Techniques

Interaction design is the research area of the computer science that deals with designing the products to support the way users interact and communicate in the XR environment (Preece et al., 2015). The interaction techniques application area encompasses the research which focused on the development and improvement of the components for an XR system to better appeal to our sense of reality through emotional, cognitive and social interaction aspects (Kha, 2022).

5.2.1 Comparative Studies

This area provides an audit of research works that involves the studies that compare the effectiveness of XR systems versus those of traditional methods. The information collected in this section plays a crucial role in identifying the boundary of the application of the XR in disaster management.

5.2.2 Developmental Studies

This study provides a brief summary of the important research approaches that involved the creation of unique and innovative interaction designs to aid the immersiveness of the XR environment. The subcategories under this heading are On-Site XR, Special Algorithms for XR interaction, Teleoperation Systems, Intelligent Interaction, Interaction using Big Data, Interaction using Haptics and Interaction for Targeted Population.

5.3 Training

Training is a systematic process of transmitting the knowledge and lessons learned in a disaster response situation so that the trainees are current about the trends in disaster management (Phillips et al., 2012). The training application includes research focused on the development of the XR system to educate the responders or group of general people in a disaster-related scenario. While this research could be categorized under modeling simulation, the discussion under this category have specifically involved the evaluation of systems to serve the need and understanding of the stakeholders like evacuees and first responders. The following subcategories were discovered: Direct Disaster Response Training, Emergency Room Training, Evacuation Training, and Law Enforcement Training.

5.4 Infrastructure Assessment and Reconnaissance

Infrastructure assessment and reconnaissance refers to the application area that is focused on the development of XR tools and techniques to visually inspect and assess the structural integrity of infrastructure. Pragmatic approaches follow the ATC-20 post-earthquake safety evaluation field manual (Rojahn, 2005) which could potentially be a lengthy evaluation and also be subjected to misinterpretation (Structural Engineers Association of Hawaii, 2006).

This research area aims to provide an efficient and rapid assessment of the damage after a disaster event and explore ways to closely reconstruct the virtual model of the important infrastructures that are damaged by a disaster. The research applications under this branch include Building Information Modeling, Rapid Infrastructure Assessment, and Virtual Reconstruction.

5.5 Public Awareness

Finally, the public awareness category focuses on the research aimed toward the utilization of XR technology to educate the general public about a particular disaster scenario and how to properly act in a disaster situation. However, it is not the same as the awareness of the user immersed in the simulation. This application is distinct from the training category in that training-related XR is aimed at a specific group of interest whereas awareness-related XR is designed for the general public (see Table 7). No further applications were under this branch were discovered.

TABLE 7. Concept of disaster management [Albayrak (2006)].

6 Discussion

Effective disaster management involves timely and strategic execution of various tasks before, during and after the disaster events. Table 7 provides a high overview of tasks that need to be completed to achieve an optimal result in terms of minimal damage and maximal preparedness. This section provides an aggregative synthesis of the selected research papers categorized based on the featured application area by explaining how these applications can be a companion in different disaster management tasks.

6.1 Computer Simulation Modeling

Two areas of computer simulation modeling were discovered during the review: system development and simulation development.

6.1.1 System Development

The Systems Development phase involves the development of complex system-level software that combines multiple feature threads to form an overall infrastructure that can perform a predefined task to support disaster management. Important disaster response system architectures are described below.

6.1.1.1 Surveillance and Monitoring Network

A surveillance and monitoring network is designed to inspect critical regions by using an array of sensors. The system is designed to synthesize the continuously sampled sensor data and display meaningful information to the user. Not only that, with the adequate addition of features, it is possible for the users to communicate using diverse notifications which are triggered when certain conditions are met. Some systems can monitor wildfires (Stipani et al., 2010), identify the geolocation of disasters (Luchetti et al., 2017), and detect criminal activities (Bang et al., 2019). The major contribution of XR technology in these scenarios is that the XR technology can visualize the information in a manner that is easy for the user to process. For example, the wildfire surveillance system uses geo-coordinate information to pinpoint the location of the fire and a smoke simulation to visualize the fire in the AR platform (Stipani et al., 2010). In case of a wildfire, this information is sufficient for the operator to dispatch the responders to the location in a minimum amount of time. Hence, the usage of XR technology has made the surveillance and monitoring network more efficient and effective in the past few years by offering a common operational picture.

6.1.1.2 Command System

During a disaster response scenario, it is crucial for the response team to adequately communicate with each other. Similarly, the commander should have access to the real-time disaster scene data and also the means to contact the disaster response facilities like ambulance, fire services, etc. Panacea’s CloudTM was found to excel in the area of Incident Command response by developing an AR-based system that supported the coordination between Incident Commanders and the responders. Their system implemented Internet of Things (IoT) devices, virtual beacons, and sensor network nodes (Vassell et al., 2016) that could offer real-time communication between all the levels of the disaster response staff. The system, evaluated on a simulation, got extremely positive results on ease-to-set-up-and-configure based on the staff, incident, and facilities template rating (Demir et al., 2017). Similarly, Molka-Danielsen et al. (2019) designed a VR-based simulator called “VAcademia” that allowed tactical and operational communication between respective leaders to facilitate dynamic decision-making in case of a fire emergency. The results of the evaluation suggested that the simulator training works better when blended with an introductory training of the system before the actual training (Molka-Danielsen et al., 2019). In addition, Wani et al. (2013) also presented a system that helps to build record of fire incidents without interfering the work of emergency team by blending workflow management, augmented reality, and co-ordination. They concluded that the “mobile equipment integrated with adaptive context aware work environments” (Wani et al., 2013) are valuable in the emergency situations. The incorporation of available technologies in XR has made the incident command system more reliable for disaster response.

6.1.1.3 Hardware Accelerated Systems

An XR-based simulation allows the users to interact with a complex simulation, e.g., 4D data-based visualization (Berberich et al., 2009). Unfortunately, the computational requirement puts a strong load on the processors and Random Access Memory on nominal computers. However, the incorporation of special computing hardware like Graphical Processing Unit (GPU) allows the computers to off-load the processing for efficient rendering. The usage of techniques like Direct Volume Rendering offers superior image quality and higher frame rates that allow realistic interaction with the simulation (Berberich et al., 2009). In conclusion, the usage of hardware accelerated XR system enhances the realism of any complicated disaster simulation.

6.1.1.4 Serious Games

A comprehensive definition of serious games is, “games which aim at providing an engaging and self-reinforcing context in which to motivate and educate the players” (Kankaanranta and Neittaanmäki, 2008). The major components of serious games involve story, characters, game host, and game play (Iuppa and Borst, 2006). Disaster-related simulations consist of stories related to the earthquake scenario inside a closed space (Lovreglio et al., 2018), radioactive catastrophe (Fischer et al., 2012), emergency evacuation (Iguchi et al., 2017; Ilmi and Hendradjaya, 2018), anesthesia crisis (Shewaga et al., 2018), underground mines (Liang et al., 2019), flood scene (Luk et al., 2018; Sermet and Demir, 2019a) and fire scene (Caroca et al., 2016; Alharthi et al., 2018b). The story needs to be designed carefully because it affects the usability of the game (Lovreglio et al., 2018). Similarly, there are usually two types of characters: Player and Non-player characters (Fischer et al., 2012; Iguchi et al., 2017; Lovreglio et al., 2018). The player is the user of the simulation and the non-player characters are the autonomous characters that the user cannot control (Fischer et al., 2012; Iguchi et al., 2017; Lovreglio et al., 2018). Also, game hosts could be a human or a computer interface (Iuppa and Borst, 2006) but AI has not been able to become a reliable leader hosts (Ilmi and Hendradjaya, 2018). The major responsibility of the game host is to assist the user in the navigation of the serious game. Finally, gameplay involves the interaction design aspect for the player. The usage of XR technology enhances the interaction for the player by making the game appealing and memorable (Liang et al., 2019), learner-centric (Shewaga et al., 2018), and an overall effective (Iguchi et al., 2017) design for disaster management. Thus, serious game design framework is a valuable design approach in building the disaster management systems.

6.1.1.5 Distributed Training Architecture

Disaster-specific response knowledge is not sufficient for disaster management personnel. The disaster-field-issues of responders such as cognitive bias, teamwork errors, and communication issues (Lusk et al., 1993) also need to be properly educated. As such, a comprehensive training framework comprising of teaching strategy, training environment, and the training architecture (Tena-Chollet et al., 2017) is desired. Longo et al. (2019) designed a training system that has a cooperative, experiential, and differentiated learning-based strategy and uses a VR-based training simulator along with Industry 4.0 products. Their system could provide not only the feedback to support disaster response systems design but greater flexibility towards the change in the security norms of a nation (Longo et al., 2019). Similarly, Chen (2014) designed Online Interactive Virtual Environment (OLIVE), a collaborative training environment which allowed the trainees to work together in the case of pre-disaster scene and disaster event scene. The validation of the system with 11 trainees concluded that the virtual platform is desirable and has the potential to become better to run emergency response exercise (Chen, 2014) than the traditional approaches. Finally, Passos et al. (2016) designed a training simulator for enhancing collaborative communication that tasks the trainees to communicate with each other about a possible suspect in a crowded situation of a soccer game using oral signals and gestures. Their simulation is successful in training security officers’ competency in collaboration during the emergency situation (Passos et al., 2016).

6.1.2 Simulation Development

This section focuses on the research areas that are involved in the visual and graphical aspects of the XR models. Major categories in this area are provided below:

6.1.2.1 Natural Disasters

A disaster simulation requires a perceptible experience that could be relatable to users. For example, in an earthquake simulation, a sense of shaking; for a flood simulation, it could be a water level rising; and for a tornado, it could be a rolling column of air at a distance. For simulations designed to appeal through visuals like a flood; tsunami; and hurricanes unique virtual objects are required. The literature search resulted in the research works that focused on the development and evaluation of disaster-related dynamic graphics for smoke (Stipani et al., 2010), fire environment (Cimellaro et al., 2019), a tsunami (Jaiswal et al., 2009), radioactive catastrophe (Fischer et al., 2012), debris flow (Zhang et al., 2019), flood (Massaâbi et al., 2018; Wang et al., 2019), earthquake (Sinha et al., 2012) and power failure situations (Chengyun et al., 2018). Underlying dynamics were found to be designed using numerical modeling (Jaiswal et al., 2009; Cimellaro et al., 2019), nonlinear dynamics analysis (Sinha et al., 2012), multilevel visualization (Zhang et al., 2019), and predictive modeling (Wang et al., 2019). As such, XR provides an unique opportunity to develop realistic disaster simulations.

6.1.2.2 Humanoid Simulation

Dealing with human casualties is one of the common tasks of disaster responders. It is desirable to have a realistic humanoid simulation, because the users feel more immersed and present in the simulation. Girau et al. (2019) developed a MR training system for first-aid scenarios by mapping a real mannequin into its virtual representation. The simulation was able to achieve good results in the tests for realism, presence, and involvement (Girau et al., 2019). Therefore, XR humanoid simulation is a powerful tool as it allows gaining direct experience in the controlled and repeatable way.

6.1.2.3 Simulation for Skilled Oriented Applications

Finally, two unique research papers were found that focused their application on training for non-technical skills and risk management in a disaster scenario. Lourdeaux et al. (2019) provided a brief description of their system: VICTEAMS which provides immersive training situations for non-technical skills that include a strong collective decision for mass casualty management by training responders about detection and correction of their teammates’ errors (Lourdeaux et al., 2019). Lutz (2018) proposed a unique “Safe AR” application that utilizes risk identification, risk assessment, and risk mitigation, and applied the concepts to the “AR Left-Turn Assist” application. The results showed that the user-involved AR failure modes have better coverage than the current approaches (Lutz, 2018). In conclusion, both applications have demonstrated the effectiveness in the emergency training in terms of non-technical applications.

6.2 Interaction Techniques

Two branches in the interaction techniques studies were seen during the review between comparative and developmental studies.

6.2.1 Comparative Studies

The traditional approaches to the visualization techniques are PowerPoint slides (Koutitas et al., 2019; Leder et al., 2019), didactic training (Rossler et al., 2019), video media (Liang et al., 2019), and a walkthrough of the system (Koutitas et al., 2019). In this section, case studies are presented on what ways XR outperformed its traditional counterparts. Shi et al. (2019) conducted a study that investigated the impact of information format and how it is perceived by the first responders via a human-subject experiment. It was found that the 2D format is completely outperformed by the 3D and VR visual contexts in spatial working memory, partially due to differences in attention patterns driven by different visual information formats (Shi et al., 2019). Similarly, Rossler et al. (2019) performed a study to evaluate the effectiveness of a VR-based Virtual Electrosurgery Skill Trainer (VEST) on operation room fire training with nursing students. The findings showed a greater increase in knowledge post-test versus pre-test in the group with the VEST training than with the traditional programmatic education (Rossler et al., 2019). Koutitas et al. (2019) evaluated the effectiveness of the ‘Ambus’ application that simulates actual Ambus vehicles designed for large-scale emergencies. The design consisted of an AR and a VR system for the Ambus. The evaluation of these systems suggested that the AR and VR outperformed the traditional training both time on tasks and the number of errors (Koutitas et al., 2019).

Similarly, a study for the effectiveness within the XR technologies was also found. Ronchi et al. (2019) compared the results of tunnel evacuation experiments aimed at investigating the design of flashing lights on emergency exit portals using Cave Automatic Virtual Environment (CAVE) and mobile-powered Head Mounted Display (HMD) methods and found consistency between the two methods for the sensory, cognitive, and functional affordance. The study found that, with the case of a relatively limited level of complexity, the use of low-cost mobile-powered HMD tools is more reliable in evacuation scenarios (Ronchi et al., 2019). The importance of this finding is that the usage of a complicated and more expensive XR platform is not essential for effective disaster management applications especially for training and command response activities.

Finally, Leder et al. (2019) conducted a study to compare the effectiveness of VR and PowerPoint in delivering safety training by using CAVE VR and PowerPoint slides for the safety training on the usage of pillar drills. The study found that although immersiveness is more apparent in VR training, content is more important than the method of information. The novelty of the last research is that it sets a boundary for the usefulness of the XR technology in conveying the information. This work suggests that content and visualization should be considered sequentially.

6.2.2 Developmental Studies

6.2.2.1 On-Site XR

The term “On-Site XR” is inspired by the research papers of Veas et al. (2013) and Pierdicca et al. (2016). This technology involves the use of a data layer consisting of past and real-time data in the XR environment on a mobile platform to perform on-site investigations (Pierdicca et al., 2016). The discovered applications involved inspection of buffer strips (Pierdicca et al., 2016), navigation for evacuation (Caballero et al., 2019), photogrammetric measurements (Dai et al., 2011), flood risk management application (Haynes et al., 2018) and damage assessment (Kim et al., 2016). Some important sensing parameters in these systems are georeferenced sensors (Veas et al., 2013; Pierdicca et al., 2016) and microphone sensors for communication (Veas et al., 2013). Similarly, usage of the cloud-based database system to access the live sensor reading makes the system useful for emergency services (Haynes et al., 2018). Hence, the “On-site XR” provides a portable tool that favors low cost information assimilation which enhances faster decision making in the disaster related scenario.

6.2.2.2 Special Algorithms for XR Interaction

The two major components of the brain of XR are data and algorithms. As such, algorithms are a crucial part of interaction design. During the research, works on computer vision for surveying (Dai et al., 2011), safety (Oh et al., 2019) algorithm, human-robot cooperation methods (Matsas and Vosniakos, 2017), and trajectory for optimal rescue operation (Nguyen et al., 2019) were found. A representative example of the algorithm for interaction is the work of Matsas and Vosniakos (2017). They introduced a highly interactive training system named “beWare of the robot”, where they designed interaction techniques for the simulation of cooperation between an industrial robot and an individual by executing a manufacturing task. They used ray casting for selection and the direct virtual hand technique for manipulation and activation of controls. They performed system control by using the tracking-driven virtual hand that interacts with specific objects, and that triggers an event or a change in the system state/interaction mode. They also implemented collision detection and triggering by using sphere, box, and cylinder covering techniques (Matsas and Vosniakos, 2017). XR technology has been backed up by unique computer algorithms that support the unique interaction for the disaster management application. Consequently, innovation in algorithms will introduce unique interaction features of XR system which will revolutionize its applicability in disaster management technology.

6.2.2.3 Teleoperation Systems

Teleoperation systems consist of two subsystems: one for the expert and another for the operator. The usage of the AR allows the projection of the expert’s hand gesture on the operator’s field of view which allows the operator to accomplish a complicated task (Oyama et al., 2016). The Behavioral Navigation System designed by Oyama et al. (2016) outperformed voice communication instructions on time taken to complete the task. The superiority of XR over any other methods is that the expert can virtually show how to complete the task but the worker can experience the information as if the expert is physically present in the room. Hence, teleportation with the XR systems can revolutionize information communication in the disaster scenario by allowing the operator to mimic how the expert would perform a task.

6.2.2.4 Intelligent Interaction

This section provides brief case studies of the research works that focus on the development of intelligent methods to enhance interaction in the XR environment. Danial et al. (2019) proposed a sequential route learning methodology, Generalized Stochastic Petri-Net-based Route Learning (GSPNRL), for agents that resemble the way people learn routes by successive exposure. Similarly, Subhedar et al. (2019) designed an intelligent system that uses drones to scan the RFID signals of wearable devices representing casualties and uses an intelligent algorithm to prioritize them based on age, gender, heart rate, and current situation of the trapped object and augments the information on rescuer’s mobile or AR glasses to navigate. Also, Ilmi and Hendradjaya (2018) designed a SG, where players must evacuate the agents to get out of the building as soon as possible. Their scripts were capable of changing the agents’ speed based on the situation, finding the position of the behind leaders and a choice for the agents on whether to exit or follow the leader. These intelligent methods have a strong prospect of being used as a tool for the future disaster management applications in aspects like virtual human (Danial et al., 2019), intelligent rescue, smart simulation, and so on. Hence, soft computing algorithms based systems can provide unique interactions to the user for disaster management scenarios, most of which could be computationally expensive with hard computing algorithms.

6.2.2.5 Interaction Using Big Data

Although a relatively new area, the impact, and promise of big data in disaster management is significant. Big data allows the user to rapidly assess the situation and quickly take action (Yu et al., 2018). Similarly, big data also helps in the forecasting of natural hazards (Yu et al., 2018). As it was discussed in the System Development sub-sections (e.g., Command System), many systems have a specific data layer to synthesize big data and interpret it to the XR user. Representative examples of the usage of big data in the development of interaction in the area of disaster management are: disaster identification using Twitter’s API (Luchetti et al., 2017), the core of Incident Command System in Panaceas CloudTM (Vassell et al., 2016) and Panacea’s GlassTM (Vassell et al., 2016), and finding escape routes in escape assisting software (Tsai and Yau, 2013), autonomous virtual agents capable of making mistakes (Lourdeaux et al., 2019), subway evacuation scenario (Sharma et al., 2014) and Choi et al. (2019)s’ smart safety and disaster response framework. In addition, a futuristic big data-based interaction system was proposed by Bang et al. (2019). They proposed a system for sensing different activities like emergency call responding, patrolling, investigation, arrests, and also for detecting criminal activities by processing a distributed array of sensors. After the criminal arrest, the VR crime recording room can help authorities understand the crime (Bang et al., 2019). Also, the field officers can get help with information and decision support from the AR application (Bang et al., 2019). Hence, big data has a potential to revolutionize the important application areas of disaster management like XR training simulations, command response data synthesis and navigation systems.

6.2.2.6 Interaction Using Haptics

Haptics are the devices that allow bidirectional transmission of senses like pressure, shear, forces, motion, etc. (Rodríguez et al., 2019). The haptics sensors reinforce the feedback provided to the users (Brewster, 2003; MacLean, 2008) which makes them applicable in the disaster scenario. Two types of interaction are possible using haptics: active haptic interaction and passive haptic interaction (Rodríguez et al., 2019). In active haptic systems, the user is in control of haptic interaction whereas in passive haptic systems, the haptic actuators automate the interaction for a passive user (Rodríguez et al., 2019). Suhail et al. (2019) designed a simulator that uses passive haptics to create a training simulation for rotation, squeeze, pulling/pushing, and pressing to configure a pump panel to extinguish a burning training vehicle using the correct foam class by configuring the pump and releasing the foam (Suhail et al., 2019). In conclusion, the haptics have a strong potential to be used in the disaster training scenarios to enhance the realism of the simulator.

6.2.2.7 Interaction for Targeted Population

Under the social inclusion principle of the United Nations (UN), the Convention on the Rights of Persons with Disabilities (CRPD) demands all the countries under the UN to conduct disaster awareness, preparedness, and response which are inclusive of and accessible to people with disabilities (Inclusion Made Easy, 2012). As such, XR systems can become an effective medium to teach and train the disabled population on disaster safety. Caballero et al. (2019) investigated the effectiveness of Virtual Reality training simulations to train hearing-impaired people. Their system consisted of a regular simulation but the instructions were delivered using a guidelines-based approach, and infographics approach where sign language avatars communicated the instructions to the trainees. They designed the training simulations for earthquake, fire, typhoon, and tsunami. The usability study confirmed that the VR based system is effective for training hearing impaired population along with the important results favoring infographic-based study for performance and effectiveness in terms of learning (Caballero and Niguidula, 2018). Hence, there is a strong opportunity and potential for the development of the XR simulators for the targeted population.

6.3 Training

Training is part of the “preparedness” phase of disaster management where the past lessons are converted into pedagogical entities for the students to study. The XR systems have proven to be effective in knowledge transfer, capture and discovery (Fernández del Amo et al., 2018) which makes them highly applicable for training. The novelty of XR based training systems arise especially in situations when travel is restricted or only limited funds are available to develop the training environment. As effectiveness of a simulation is not related to the fidelity of the simulation (Hays and Singer, 2012), the low-fidelity simulation requiring fewer resources could be promising (Alharthi et al., 2018a). Similarly, mixed reality also shows a promising future as a disaster training technology, as new training simulations related to human factors can be designed in the mixed reality environment (Alharthi et al., 2018a). The major application areas of XR in disaster management training were found in emergency room training, law enforcement training, evacuation training, and direct disaster response training.

6.3.1 Emergency Room Training

The training systems related to the emergency room environments were the evacuation of newborn babies (Farra et al., 2019), intensive care unit equipment usage (Dhanasree et al., 2019), anesthesia crisis (Shewaga et al., 2018), and operating room fire (Dorozhkin et al., 2017). The XR-based training simulator excelled over the traditional methods in time to complete the task (Farra et al., 2019), enhancing learner-centric approach (Shewaga et al., 2018), and making the training medium desirable to the students (Dorozhkin et al., 2017). As a representative example, Farra et al. (2019) examined the effectiveness of VR, training the newborn intensive care unit with workers over web-based updates on knowledge gained, confidence with evacuation, and performance in a live evacuation exercise. They found that although tests on self-efficacy and cognitive assessments didn’t show significant differences, the VR group performed a lot better in the live evacuation drills in terms of time taken to evacuate babies (Farra et al., 2019). Therefore, XR training simulators efficiently convey the skills to the users.

6.3.2 Law Enforcement Training

The law enforcement officers’ experience stress and anxiety on the job which directly affects their performance and ability to take appropriate decisions in the field. There is a need to develop a simulation that encapsulates the psychological parameters of their job and train them properly. Similarly, an important theoretical pedagogical framework to train for such a situation is called situated cognition model which is based on the idea that situations are responsible for structuring cognition (Brown et al., 1988). The two major implications of the situated cognition model are: knowledge is obtained by living practice and learning is achieved by participating in the communities of practice (Brown et al., 1988). The XR training system checks both of the implications to offer a unique solution to train the law enforcement and emergency response officers in a modular and extensive manner (Carlson and Caporusso, 2019). Carlson and Caporusso (2019) introduced a modular and extensible training simulator. The system allows the operator to experience different tactics, dynamics, locations, levels of threat and outcomes, and also run the simulation multiple times by implementing a situated cognition model intended for training law enforcement and emergency personnel (Carlson and Caporusso, 2019). In summary, the XR training simulators can be effectively implemented with reference to a philosophical learning objective for critical fields like law enforcement.

6.3.3 Evacuation Training

Unique evacuation training scenarios were discovered during the research like active evacuation strategy search (Iguchi et al., 2017), active decision making for time-constrained situation (Kawai et al., 2016) and indoor crowd situation (Cao et al., 2019). The AR based systems can actively blend the virtual elements through similarity calculation and superposition and synchronization (Kawai et al., 2016) with respect to the physical scene. Also, there are two major categories for spatial exploration modes known for an evacuation scene: active exploration and passive exploration (Chrastil and Warren, 2012). In active exploration, the trainees are allowed to freely evacuate their environment, whereas in passive exploration, they follow a predetermined path in the environment (Chrastil and Warren, 2013). The passive exploration mode is successful in the evacuation for the indoor scenario where the predetermined routes are more rigidly applicable (Cao et al., 2019). Hence, the XR based training simulators can become highly accepted (Iguchi et al., 2017), motivating (Kawai et al., 2016) and effective feedback appliances (Liang et al., 2018) for the evacuation drills. Such training will effectively prevent secondary disasters like a stampede by educating the evacuees to effectively evacuate the disaster area.

On the other hand, XR systems provide modern means for the evaluation of the trainees data which helps in minimization of biases that can arise in the pragmatic means of evaluation like post-training survey. Tian et al. (2019) designed an effective system to analyze the indoor fire escape behavior. The system collects the information like movement, 3D position gaze information from eyes and also head position simultaneously in the virtual environment which was then analyzed to get information about the participants’ spatial cognitive process. This system was able to support advanced depth behavioral analysis for qualitative and quantitative inspection of visual attention. Hence, XR based training simulators also provide reliable tools to gather cognitive information about the participants. This can help during the interpretation of the results obtained after the analysis of training data.

6.3.4 Direct Disaster Response Training

Two papers in this application area were found: railway accident training (Xu et al., 2018), mining accident training (Liang et al., 2019). Xu et al. (2018) designed a single user railway crane training system that uses multibody dynamics to get feedback between rigid bodies, a PhysX engine to simulate a train crash accident at any velocity, automatic collision detection of the load with surrounding objects and crane in the lifting process, and got a applicable conclusion from the evaluation of the VR training to traditional onsite training. Similarly, Liang et al. (2019) designed a SG application for safety training in rock-related hazards in underground mines. The application has two modes which are scaling and perception training modes. In scaling mode, the players identify and remove loose rocks. In the perception training mode, the players use their advanced hazard perception skills to identify random loose rocks and signs of unstable ground. The validation of the system compared with a training video resulted in the conclusion that participants who receive training with VR media perform better than those who receive training with video media (Liang et al., 2019). Therefore, the XR training system can exercise coordination and hazard perception which helps in the development of competency (Caballero and Niguidula, 2018) during the disaster response.

6.4 Infrastructure Assessment and Reconnaissance

The two major technologies that support infrastructure assessment and reconnaissance are based on computer vision techniques. The idea is to collect the infrastructure data pre-disaster and then use that to compare with the data post-disaster to analyze the damage of the infrastructure (Ono et al., 2008; Dong et al., 2013; Kim et al., 2016; Li et al., 2019; Xiong et al., 2020). The role of the XR is to visualize the synthesized information in a manner that facilitates decision making.

6.4.1 Building Information Modeling

Building Information Modeling (BIM) deals with the development of a database that contains the information regarding geometry, geographic information and other building construction and build details of the infrastructure (Azhar, 2011). Park et al. (2018) proposed and simulated an AR-based disaster management system for multi-story buildings that provide intelligent service, safety guidelines, and an IoT-based real-time remote system. As a subsystem, an AR-based system could scan and generate a floor map to provide a safe evacuation guideline service to the occupants and rescuers (Park et al., 2018; Lochhead and Hedley, 2019). The subsystems use smart pads that can be scanned to obtain the entire building structure in case of a fire (Park et al., 2018). As a consequence, BIM in XR facilitates fast decision making in an event of disaster which makes it an important application area for the disaster management technology.

6.4.2 Rapid Infrastructure Assessment

The infrastructure assessment approaches involved: assessment for safety (Nilsson et al., 2018) and assessment for damage (Phillips et al., 2012; Behzadan et al., 2015). Similarly, assessment for damage involved assessment using drones (Kim et al., 2016; Li et al., 2019) and assessment using handheld devices.

For the safety research, XR technology allows a unique way to optimize our infrastructures in terms of user-centered design at a minimal cost and time. A notable application developed by Nilsson et al. (2018) utilizes VR to evaluate designs for different portal exits. This allowed rapid and effective testing of various designs without a significant budget burden.

After a disaster hits a community, the first step to take in the recovery process is to assess the damage on the infrastructure (Phillips et al., 2012). This step will tell which buildings are safe to stay in and which are damaged from the disaster. A rapid damage assessment can be done by deploying a scanner drone equipped with the image processing algorithm that can compare the pre and post disaster scenes for the damage assessment (Dong et al., 2013; Li et al., 2019; Xiong et al., 2020). Usage of handheld AR can provide a portable environment to access the damage more closely for the parameters like interstory drift ratio (Dong et al., 2013), inaccuracy and uncertainty of the damage analysis (Kim et al., 2016). The intelligent algorithm using VGGNET Convolutional Neural Network is able to assess the damage of the buildings with an accuracy of 89.39% (Xiong et al., 2020). A point-cloud based algorithm along with the usage of slave drones is also successful in reducing the time complexity of the damage assessment task (Li et al., 2019).

Finally, another approach to access safety using XR was found to be a Mobile XR system. This reconnaissance approach involved the utilization of the XR technology to implement 4D modeling of the system of interest, such as a disaster scenario, through the superimposition of virtual models (such as BIM) over time-lapsed real graphics (Behzadan et al., 2015). In conclusion, XR based systems can facilitate quicker decision making for the safe infrastructures to facilitate the disaster recovery process consuming minimal funds and time.

6.4.3 Virtual Museums

One of the dimensions of a disaster recovery also involves the recovery of historical and cultural heritage (Phillips, 2009). After the disaster, a country or an organization may not have enough funds to rebuild the original infrastructure in its original state. In such a situation, virtual reconstruction can be the next best alternative. The Citadel of Bam which is situated in Kerman province of Iran is a UNESCO world heritage site that was destroyed in a 2003 earthquake (Ono et al., 2008). A team of researchers used heterogeneous knowledge of the relics and used a metadata-based layer naming process to link the available components for the generation of 3D models (Ono et al., 2008). They also developed a VR-based system to be able to provide virtual tours of the site (Ono et al., 2008). The usage of VR allows the visitors to get a similar experience as of visiting the actual site. Also, the VR reconstruction is more cost-effective and less prone to damage than a physical reconstruction alternative.

6.5 Public Awareness

The public awareness plays an important role in the effective mitigation of a disaster by arousing community interest and educating the society to reduce risk from potential threats (Davis et al., 2003). Bernhardt et al. (2019) tested the effectiveness of VR in making people aware of the landfall of hurricanes over traditional media outlets. The evaluation concluded that viewing a VR simulation of a category 3 Time of Concentration (TC) landfall, in comparison to more traditional products, encourages survey respondents to take warnings more seriously when placed in a hypothetical scenario (Bernhardt et al., 2019). Wendler and Shuttleworth (2019) created a street game named “Downpour!” which allows players to independently interrogate both scientific and political dimensions of flood management. The immersive nature of a street game is intended to potentially trigger active involvement in flood-related efforts by creating an emotional connection with the issues (Wendler and Shuttleworth, 2019). Similarly, Sermet and Demir (2019b) presented a game-based VR framework that uses geographical and weather data to generate simulation and capture the behavioral and psychological data of the users in both the single and multi-user player mode. The validation of the system at a conference concluded that the system was easy to use and navigate (Sermet and Demir, 2019b). Finally, Luk et al. (2018) presented a VR system that simulates the experience of how the nuclear power plants operate and accidents can occur in the virtual world. The validation of the technology in education aspect concluded that the immersive VR system was very successful in helping visitors understand nuclear power (Luk et al., 2018). XR systems can provide interesting and engaging solutions to teach disaster related information to the general public.

XR provides a unique solution to create a similar experience to real life. Its simulation is robust as it allows gaining direct experience in a controlled and repeated way compared to poorly offered traditions. The XR systems teleportation has revolutionized information communication by enabling operators to mimic how experts perform tasks. It also revolutionized the interaction features using haptics to enhance realism in the simulator. The XR systems can also help make quicker infrastructure decisions to facilitate disaster recovery, consuming minimal funds and time. A VR-trained group performed a lot better in live evacuation drills when compared to the traditional group showing how the XR training systems convey the skills to the users. However one of the important challenge that still exist is while working in the real-live disaster zone requires a significant infrastructure investment but the researchers need to evaluate overall effectiveness and its impact. For example, to conduct a fire drill, the local fire department has to create a scenario of a fire accident, and the amount of capital required for it is very high. Once the training is done, it is difficult to replicate the scenario. The first responders might get injured in the process of training or even have fatal injuries. The XR technology can be used to develop a virtual environment to train interested users in a risk-free environment and also with less expense. Finally, XR systems provide efficient solutions for disaster training in both technical and non-technical applications related to disaster response.

7 Limitations and Future Directions

7.1 Limitations of the Review

The innovation of this paper is in mapping out non-mutually exclusive research application areas in disaster management. We analyzed a set of 84 papers that represent a subset of a much larger document corpus. We believe that the assessments we made (Section 6) and limitations we identified (Section 7) can be useful in future literature reviews and identifying future research. The limitation of this review is that the review focuses on the breadth-wise exploration of the applications through the synthesis of research papers. An important future direction is to separately conduct an in-depth Systematic Literature Review on each of the application areas to support the development of theory on this evolving field. In addition, studying XR based disaster management system patents and state of art systems used in practice could also provide new opportunities for research. Also, useful insights can emerge by studying the cybersickness issues of using XR systems. This can inspire the research community to develop XR systems that have minimum health-related side effects to the users.

Similarly, we have chosen diverse keywords to search papers from two tracking sources (Google Scholar and Web of Science). Previous research work [Taster (2022)] has shown that the use of these two tracking sources accommodates nearly all of the existing research scholar articles. We understand that a few research articles might be missing, but the research work included in this review article summarizes the existing work and provides guidance to interested readers with the current state-of-art in the field of disaster management. Furthermore, one of the limitations might be the keyword asymmetry while using the tracking sources. While using the seventeen different keywords in Google Scholar, it provides the complete list of research papers that were verified by manually searching the search results, and the two keywords used in the Web of Science show the remaining list of search results that were not available in Google Scholar. However, with the rigorous review process using 84 papers, we believe that the review paper provides a framework for looking at many more papers. The limitations and future directions we’ve found include information about the use of virtual and augmented reality in the area of disaster management technology, which might be beneficial to the research community. Finally, another limitation of this review is the exclusion of the study of human behavior in disaster scenarios using XR. Although the studies that focused on the development of XR technology for behavioral studies were included, the studies that focused on situational awareness, stress, bias, and other human factors in hazardous emergencies in the XR environment were not reviewed.

7.2 Future Directions

7.2.1 Computer Simulation Modeling

Generating a simulation requires substantial computational power. It is a good idea to use robust hardware to develop and run simulations. Lack of this can cause buffering and pausing in frame transition and also the position tracking in the VR, AR and MR system. The usage of advanced hardware programmed to accelerate computation using GPUs provides a realistic visualization that can enhance the simulation’s ecological validity at a basic level. Park et al. (2018) recommended that the incorporation of location-based technologies such as Building Information Modeling (BIM), Geographic information system (GIS), and Radio-Frequency Identification (RFID) can significantly improve the disaster management systems (Park et al., 2018).

After the review, it was seen that a significant amount of the research papers related to the development of unique simulations lacked adequate testing and validation. This was another indication that XR application in disaster management is still in the beginning phase and could use more research.

Multiple Incident Command Response systems were discovered during the review. The major system integration components involved XR for Common Operational Picture, cloud computing for the effective communication and information storage, geo-spatial network for positioning and deployment of an array of sensors for the information collection of the disaster environment. Multiple drone related projects were discovered during the review. However, no command systems used Drones for search and rescue applications. Incident command systems could incorporate drones in their systems to enhance resiliency (LaLone et al., 2019) in their search and rescue tasks.

In the area of 3D reconstruction, the uncalibrated images caused stretching and pinching issues in the 3D model. More research is needed to figure out a way to normalize the images to appropriately align with a 3D geometry so that the dependence of the quality of simulation on the quality of the image captured could be eliminated (Macchione et al., 2019).

Similarly, it is a desirable feature that the simulation is designed to be as realistic as possible. However, an extremely immersive simulation for a disaster scenario can induce psychological trauma to the users. A tradeoff of realism with mental safety for a disaster-related simulation could be a multidisciplinary interest of research as a study of XR and safety science (Markwart et al., 2019).

Finally, the overall emergencies in disaster management field are categorized as Routine Emergencies and Crisis Emergencies (Leonard and Howitt, 2009). A comprehensive theory of each of the emergencies are available in disaster management texts along with the key elements that result in an effective response. Development of these simulators using ideas from these literature can provide trainees skills to become adaptive in unforeseen disaster scenarios.

7.2.2 Interaction Techniques

With the advent of new interaction tools and techniques, it has helped researchers and scientists interact with diverse scenarios in an immersive virtual environment. Shi et al. (2019) identified that with eye tracking we can identify and investigate more complex and dynamic building scenarios. The adoption of newer gadgets like haptics can aid interaction which can enhance immersiveness of an XR system.

Similarly, the AR technology is desired to be adaptable to all environmental conditions. Veas et al. (2013) suggested that performance adaption to all weather conditions could use future research work, which will directly improve the technology’s usability. A research on improved computer vision for AR could be promising. The possibility of Remote Sensing for XR can enhance such applications. Additionally, AR visualizations need a practical solution on optimizing the adaptive latency compensation algorithm with image processing techniques. Otherwise, the interactivity aspect gets directly impacted by slacking objects (Veas et al., 2013).

Furthermore, maintenance applications need a routine assessment and evaluation. It would be more efficient to automate the inspection process. Pierdicca et al. (2016) recommended that to control automatic land use classification, head mounted display could be used and to automate the inspection process the unmanned robot vehicles could be used.

Finally, the design of evacuation assisting tools involves a detailed study of the human behavior of wayfinding. Cao et al. (2019) recommended a study of how complex cognitive processes affect spatial exploration mode in fire emergency. They found that it could be used to develop better wayfinding behavior interference methods and tools to support efficient evacuation during building fire emergencies.

7.2.3 Training

A realistic simulation geared with the hardware-accelerated code and an adequate amount of sensing make an effective training simulation. It is the author’s opinion that the training using marker-based AR should either be avoided or the markers that camouflage with the simulation should be used in training simulators. This is because markers are used to place the virtual objects in the real world, and the trainees will see a paper attached to a dead body or a moving storm which will strongly impact the quality of the simulation. If the marker-based AR must be used, creative markers such as pictures of blood as a marker to place a virtual human dead body are highly recommended. As a better alternative, smart algorithms could be used to identify the objects in the provided image to figure out where the virtual object needs to be overlaid.

Similarly, the training simulators designed under User-Centered design approaches have a strong scope in the disaster preparedness-and-response training simulators. The usage of user-centered models can produce a highly usable product within a short timeframe (Bailie et al., 2016), improved preception and experience of the virtual environment (Jeon et al., 2019). Not only that, the Situated Cognition framework also has a strong pedagogical effectiveness in the virtual world (Caroca et al., 2016). One of the proposed system design procedures is: requirement analysis, prototyping interaction, prototyping comprehensive system, and prototyping cross-functional system (Grandi et al., 2019).

Also, as the XR technology is rapidly accelerating to other fields, pedagogical investigations on how to design XR to aid student learning is a growing trend of interest to the researchers. (Sampaio et al., 2010) presented VR didactic models for the cantilever method and an incremental launching method of bridge deck construction. These models could be utilized to design effective training simulators that would aid construction workers’ training process, which would enhance workplace safety in construction sites.

Multi-person simulation is also an important area that requires more research. Like train accident response, some rescue operations require multiple crane operators synchronizing to lift a heavy load that could contain life and property. Thus, this aspect could use some research and development to aid the coordinated disaster response (Xu et al., 2018). Finally, motion sickness is another health issue associated with the current limitation of XR technology. Some of the trainees in their simulation faced 3D motion sickness and difficulty in moving (Kawai et al., 2016; Lovreglio et al., 2018). This issue, in not resolved, can hamper the growth of XR technology thus needs a serious attention.

7.3 Infrastructure Assessment and Reconnaissance

The reconnaissance process mostly involves going to the site and providing the visual information to the reconnaissance software. It is desired that the hardware to run the software be portable. Mobile AR with cloud processing is a popular option as it provides the portability of a handheld device while giving the processing power from the cloud. However, there is a tradeoff with this feature. Kim et al. (2016) found that the reliability of networks creates uncertainty in this kind of system, and there is a cost-benefit tradeoff of data accuracy and hardware requirements (Kim et al., 2016) (Dong et al., 2013) also identified that their automatic algorithm for measuring the Interstory Drift Ratio (IDR) could not do better than the manual selection due to uncertainties and errors associated with the instruments (Dong et al., 2013).

An important connection was found between two papers that have a composite application (Li et al., 2019). presented a rapid method for data collection and 3D reconstruction. Dai et al. (2011) provided a mobile, non-contact method of calculating the IDR. These two systems could be combined to form a system that takes an initial map of a city prior to a disaster scenario and can rapidly be deployed to obtain real-time reconnaissance of the whole city. This not only provides important technical information but also provides the visual information of the damage for the non-technical audience.

7.4 Public Awareness

Only two papers were found that specifically used XR technology to alert the general public about disaster situations. More research could be done to develop tools and techniques that could be used to inform the general population about the risks of disaster. Wendler and Shuttleworth (2019) noted that their SG application could be tailored to environmental issues specific to the place of interest by creating a unique sequence of interactions and challenges, which is currently designed for Manchester and North West United Kingdom.

Mass awareness is a unique field of study and should not be mixed with training. (Bernhardt et al., 2019) showed concern over the effectiveness of VR that it may mislead populations less vulnerable to a particular storm and convince them to evacuate when they should not. Technology validation is essential prior to its implementation to prevent mass chaos (Bernhardt et al., 2019).

8 Conclusion

In this literature review, a recent 11-year trend of the development of application of XR technology in the area of disaster management with the focus on the XR systems was explored. A set of 84 papers were tracked from a large document corpus, and critically synthesized to provide assessments of current trend and future recommendations. During the review, it was found that, the application of XR is highly trending as a disaster management technology in the research community. Although bounded by content (Leder et al., 2019), the medium of information plays a crucial role to ensure that the information is conveyed properly. The XR technology-based systems excel in retainment of the spatial working memory (Shi et al., 2019), increase in knowledge (Rossler et al., 2019), and improvement in the time on tasks and the number of errors (Koutitas et al., 2019) when compared with the traditional and pragmatic methods. Similarly, it was also found that XR systems were used extensively in the computer simulation modeling, interaction techniques, training, infrastructure assessment and reconnaissance and public awareness application areas of disaster management. A brief examination of each research focus, achievements, and limitations was done. More research in the testing and validation of systems, integration of computing and sensing hardware, automation in sensing and mapping, and health issues related to VR could be done to better serve disaster management science with the XR technology. The finding of this research will be specifically useful to the research and industrial agencies that seek to design effective and cost-efficient XR based systems to tackle disaster management issues.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

This research project was funded by Higher Education Research Council - Idaho Global Entrepreneurial Mission (HERC-IGEM).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank Nirajan Bhattarai for helping us to prepare the paper for submission. We would also like to thank Rachel Brownell, Jose Duran, and Jodi Quigley providing feedback and reviewing the paper.

References