Integration of augmented reality and GIS: A new approach to realistic landscape visualisation

https://doi.org/10.1016/j.landurbplan.2008.03.004Get rights and content

Abstract

This paper describes the development of a photo-realistic visualisation method which uses the combination of a geographic information system (GIS) with off-line augmented reality (AR) techniques. The technique makes a linkage between GIS-based modelling and realistic panoramic video frames to dynamically augment a landscape view with modelled temporal changes. By using real object textures, computer-generated objects can be well matched to background frames and become hardly detectable as computer objects. The technique has been tested with data from an area in northeast Victoria, Australia and has successfully represented the dynamic spread of weeds (blackberries) and their effects on landscape over a period of 14 years. The proposed approach has the potential to improve the communication between policy makers and non-experts and to improve decision-making process. Extensions to other model types and to real-time usage are envisaged.

Introduction

GIS has become a key tool for environmental scientists. Managing problems of the environment has been one of the most successful application areas for these systems. The ability of GIS to integrate spatial data from different sources, with different formats, structures, projections or levels of resolution is a powerful aid to spatially distributed modelling, particularly when working with models that integrate more than one process. Often, however, results are presented in the form of a series of data files, images or even graphs that are not directly useful or even understandable for non-expert users. A number of visualisation techniques have been devised to provide an interface between the modelling outputs and users. As has been long recognised, these techniques translate the results of modelling to a more comprehensible and understandable format for different levels of user (Bishop, 1994, Lange, 1994, Orland, 1994).

Using computer graphics in landscape visualisation began in 1970s but was static and highly abstract, e.g., (Myklestad and Wager, 1977). Photomontage was perhaps the first computer-based technique in representation of landscape changes that could also achieve relatively high level of geometric accuracy (Lange, 1990). Since that time, landscape visualisation technologies have grown up significantly and many different methods and techniques have developed as reviewed in Ervin and Hasbrouck (2001) and Bishop and Lange (2005).

Landscape visualisation is often used for communicating complex information about the state of a landscape and how it might change, and can be particularly effective when communicating to community groups and policymakers (Sheppard, 2001). Representation of existing real world and potential alternatives is a powerful tool for public understanding (Honjo and Lim, 2001) and an essential means for landscape designers and planners to express and communicate their thoughts. In general, landscape visualisation systems, which try to imitate the features of the real environment, should be able to provide accurate, detailed, diverse and spatially explicit information for users (Bryan, 2003).

Appleton et al. (2001) discuss three broad categories in current landscape visualisation; image draping, photorealistic rendering and virtual worlds. In the image draping method, a single image or combination of image layers is draped over a 3D representation of regional terrain; that is Digital Elevation Model (DEM). Because of the lack of complex information, this method supports high-speed rendering and easy navigation. While, it could be convenient for overview of a large area, it suffers from the low resolution and large pixel sizes that reduce the realism of the representation for view-points close to the ground (Fig. 1). Many systems including GIS provide this function.

In photorealistic rendering method, vegetation and other features of landscape come together for better and more realistic simulation of area. The more detail with which objects are modelled, the more real they appear. This detailed modelling is supported by rendering algorithms which are precise and realistic rather than being optimised for rapid performance. A high degree of realism is, therefore, the main benefit of this approach, while the output is not interactive and would be still images or animation of a series of images (Fig. 2). Products such Visual Nature Studio (www.3dnature.com) are prominent in this space with a focus on landscape simulation.

In the virtual world technique, the user is allowed to interactively explore the environment and the area might have links to supplementary information as well. Such systems are fully interactive and enable the user to fly or walk through the region, find different viewpoints. This virtual reality (VR) approach is similar to real life experience and so provides the chance for a user to discover the place rather than just see the space (Bishop, 2008). However, commonly, to reduce the required time for the rendering process, objects are simplified, realism is reduced and orientation may be more difficult (Fig. 3A). However, increasing computer power, algorithms for level of detail management and efficient rendering are reducing the gap between animation and real-time systems (Paar and Rekittke, 2005) as illustrated in Fig. 3B. These advances are being driven also by the games industry in their quest for realism in the interactive game experience. Games engines are now also being used in landscape simulation (Stock et al., 2007).

In recent years, this desire for a link between GIS and visualisation/virtual reality has become popular and many research projects have explored different aspects of a merged system. Verbree et al. (1999) propose merging a 2D GIS with a virtual reality interface. They suggest a multi view approach based on different types of visualisation to support 3D GIS interaction within virtual reality environments. This system is operational in a range of visualisation modes from PC monitor to Virtual Workbench (looking down on the model as if it is a 3D scale model) to the multi-projector CAVE (Cruz-Neira et al., 1992) and allows interaction with GIS data in a range of virtual views: plan view, model view and world view. These approaches have not, however, generally supported high quality graphics.

Stock and Bishop (2002) and Stock et al. (2003) have developed a fully interactive game-based 3D visualisation system merged with GIS such that any alternatives in GIS environment will automatically reflect on the 3D environment. Chen et al. (2006) report a live link between a GIS and a game engine to deliver high performance graphics and editing in 3D with layer control and modelling in 2D. They also foreshadowed use of this integrated GIS and game environment in an augmented reality (AR) interface for in-field application.

AR has been defined as a combination of the real scene viewed and virtual (computer-generated) images (Azuma, 1997). By various means virtual images are superimposed over the real scene and look like a part of the real world. Augmented reality techniques allow us to mix computer-generated images with real-world views. While in the longer term a real-time approach using head-mounted displays may be used in the landscape, this possibility is still limited by communications infrastructure and the time required to run the environmental models whose outputs need to be displayed. Therefore, it remains useful to consider alternatives to real-time application. Video sequences and monitor-based AR (Fig. 4) form perhaps the simplest approach to augmentation. In this method, video footage is recorded and augmented with computer-generated objects and then displayed through a computer monitor or even large screens as discussed by Raskar et al. (1998). In addition, Gibson et al. (2002) argue that because of the algorithm used in the registration process, real-time techniques have limited functionality when high accuracy is required while off-line a precise augmentation could be achieved (Nakamae et al., 2005). This research was, therefore, based on off-line video-based augmented reality and its applications in environmental visualisation.

The major challenge in AR is the combination of real-world and computer-generated objects into a single augmented environment such that the user cannot distinguish the difference between them. In a video-based technique, chroma-keying is a simple but efficient approach. In this method, used in video special effects for a long time, the background of the computer-generated images is set to a particular colour which none of the virtual objects use, for example blue or green. Then, this colour is replaced with corresponding parts of the video footage such that it looks as if virtual objects are superimposed over the real-world elements.

Augmented reality has been used for environmental assessment and pre-evaluation of the visual impact of large-scale constructions in landscape (Rokita, 1998), or observing the harmony between a proposed project and landscape around it from different point of views (Nakamae et al., 2001, Qin et al., 2002). Photorealistic AR techniques are able to represent alternative landscape changes realistically so that non-expert audiences can interpret the imagery as easily as they interpret a photograph. Since the background consists of real-world elements, in comparison with even the best fully synthetic virtual environments, non-expert viewers will have a greater sense of place and familiarity with the landscape; especially if the presentation runs within an immersion situation.

Section snippets

Conceptualisation of the visualisation technique

Achievement of an AR expression of the output of a GIS model requires: (i) Modelling inside GIS to provide environmental change data under different starting conditions or assumptions; (ii) creation of a realistic panoramic backdrop; (iii) creating realistic computer-generated images from the outcomes of modelling inside the GIS; (iv) application of an off-line video-based augmented reality technique to superimpose the computer images on the panoramic backdrop to realistically represent the

Technical detail and case study

The Cudgewa Valley in rural Victoria, Australia (approx Lat: 36°, 12′S and Long: 147°, 45′E) was selected as the case study area and a detailed 3D virtual environment of the region was created using GIS data (DTM, Land cover, Roads, Rivers, etc.) and aerial ortho-photo imagery. This virtual environment was developed as part of a wider study of the role of visualisation in envisaging alternative futures (Stock et al., 2007). Weed problems are a key issue in the region and so the site was very

Results and discussion

Visible weed outbreak locations, from the GIS-based weed spread model, are shown in Fig. 8. This model has not been calibrated or tested and the results, therefore, are simply a plausible outcome of blackberry spread across a valley. In Fig. 9, the corresponding outputs are shown as a single frame within an augmented reality sequence (either the pan across the landscape or the dynamic spread simulation) and a sample GIF animation is available at (www.geom.unimelb.edu.au/cgism/AR/weeds.gif).

The

Conclusion

A GIS-based photorealistic visualisation method, which uses an off-line video-based Augmented Reality technique to represent GIS-model-based landscape changes in an immersive environment, was introduced. In this method, the outputs of modelling inside GIS are realistically rendered and superimposed onto corresponding video frames of the area, in correct spatial position.

The proposed method is not real-time, but is dynamic in supporting both panning motion of the image across the landscape and

References (35)

  • E. Nakamae et al.

    Rendering of landscapes for environmental assessment

    Landscape and Urban Planning

    (2001)
  • P. Rokita

    Compositing computer graphics and real world video sequences

    Computer Networks and ISDN Systems

    (1998)
  • S.R.J. Sheppard

    Guidance for crystal ball gazers: developing a code of ethics for landscape visualization

    Landscape and Urban Planning

    (2001)
  • C. Stock et al.

    Exploring landscape changes using an envisioning system in rural community workshops

    Landscape and Urban Planning

    (2007)
  • B. Tress et al.

    Scenario visualisation for participatory landscape planning—a study from Denmark

    Landscape and Urban Planning

    (2003)
  • K. Appleton et al.

    Rural Landscape Visualisation from GIS Databases: A Comparison of Approaches, Possibilities and Problems

    (2001)
  • R.T. Azuma

    A survey of augmented reality

    Presence-Teleoperators and Virtual Environments

    (1997)
  • Cited by (0)

    1

    Tel.: +61 3 8344 7500; Fax: +61 3 9347 2916.

    View full text