Elsevier

CIRP Annals

Volume 61, Issue 2, 2012, Pages 657-679
CIRP Annals

Augmented reality applications in design and manufacturing

https://doi.org/10.1016/j.cirp.2012.05.010Get rights and content

Abstract

This paper reviews the research and development of augmented reality (AR) applications in design and manufacturing. It consists of seven main sections. The first section introduces the background of manufacturing simulation applications and the initial AR developments. The second section describes the current hardware and software tools associated with AR. The third section reports on the various studies of design and manufacturing activities, such as AR collaborative design, robot path planning, plant layout, maintenance, CNC simulation, and assembly using AR tools and techniques. The fourth section outlines the technology challenges in AR. Section 5 looks at some of the industrial applications. Section 6 addresses the human factors and interactions in AR systems. Section 7 looks into some future trends and developments, followed by conclusion in the last section.

Introduction

Manufacturing is the process of transforming raw materials and information into finished commodities with good value-added for the satisfaction of human needs. It has been a key contribution to a nation's economic growth for the last few centuries and will continue to do so in the future. In the current highly competitive and dynamic business environment, the manufacturing industry is facing new challenges, which require a holistic perspective on the four main classes of manufacturing attributes, i.e., cost, time, quality and flexibility [30]. Manufacturing companies should be producing innovative products at low cost and reduced time-to-market. High product-mix with low volume, customisation to meet the individual demands of the customers, increasing legislation of environmental and other issues have further made manufacturing processes more complex and demanding. In addition, the increasing trend of globalized manufacturing environments requires real-time information exchanges between the various functional units in a product development life cycle, e.g., design, setup planning, production scheduling, machining, assembly, etc., as well as seamless task of collaboration among them. Manufacturing processes have to be more responsive and systematic in order to be efficient and economically competitive. On top of that, the increasing demand for goods results in an increasing demand for natural resources and energy. However, since resources and energy are finite, new ways of producing more with less ought to be found [33].

Due to the recent advancements in information technology (IT), manufacturing research has entered a new level of product planning, analysis and trouble shooting. Digital manufacturing has been considered, over the last decade, as a highly promising set of technologies for reducing product development times and cost as well as for addressing the need for customization, increased product quality, and faster response to the market [31]. Digital manufacturing has become a common trend worldwide as computer-integrated manufacturing systems have eliminated data handling errors and enhanced decision making. Computer simulation using CAD modelling tools and finite element analysis has assisted manufacturing engineers to reach decisions faster and free from errors. In just over a decade, augmented reality (AR) technology has matured and proven to be an innovative and effective solution to help solve some of the critical problems to simulate, assist and improve manufacturing processes before they are to be carried out. This would ensure that activities, such as design, planning, machining, etc., are done right-the-first-time without the need for subsequent re-work and modifications. AR is a novel human–computer interaction tool that overlays computer-generated information on the real world environment. The information display and image overlay are context-sensitive, which means that they depend on the observed objects [8], [9]. This novel technique can be combined with human abilities to provide efficient and complementary tools to assist manufacturing tasks. AR has already been demonstrated to be a solution in many applications, both in manufacturing and other fields. Several successful demonstrations have been made in the medical domain, military training, tele-robotics, entertainment, maintenance, and manufacturing [107], [177].

Research in the manufacturing applications using AR technology is a strong and growing area. The challenge is to design and implement integrated AR-assisted manufacturing systems that could enhance the manufacturing processes, as well as product and process development, leading to shorter lead-time, reduced cost and improved quality.

Section snippets

Hardware and software systems in AR

Developing successful AR applications is a challenging task. Despite recent advances in the AR technology in the last ten years, most of the AR systems developed so far are laboratory-based implementations. Extensive research has been carried out worldwide in addressing some of the critical issues in AR technology. Real-time tracking and computation are crucial since synchronization between the real and the virtual worlds must be achieved in the shortest possible time interval.

AR applications

Major AR research in design and manufacturing

Manufacturing is one of the most promising fields where AR can be used to improve the current techniques and provide solutions in the future [107]. With new advances in computer and manufacturing technologies, there is a growing trend of allowing users to interact directly with the manufacturing information associated with the manufacturing processes. AR has the ability to integrate these modalities in real time into the real working environment, which is useful for manufacturing activities,

Accuracy

Unlike applications in advertising, gaming, fashion, etc., AR applications in manufacturing and design requires a high level of accuracy in tracking and superimposition of augmented information. Outdoor AR systems use GPS and inertial tracking techniques with a combination of gyroscopes, electronic compass, accelerometers, and other types of sensors, together with CV tracking techniques. Precision and accuracy are generally lacking in outdoor applications but very often, a high level of

Industrial applications of AR

The majority of the AR research appears to have originated from the academia over the last two decades. Industrial AR applications are far less reported in comparison.

Regenbrecht et al. [125] reported a number of industrial applications of AR. The Boeing wire harnessing project in the early 1990s was amongst one of the earliest [96], [97] case studies.

In servicing and maintenance of complex equipment, and even a car's electrical circuits would call for a database system and advanced computer

Human factors and interaction in AR systems

Although AR has found a good number of applications in design and manufacturing, there are few in-depth studies that assess and evaluate human factors and interaction in AR systems. The limited understanding of human factor issues is likely to hinder widespread adaptation of AR systems beyond laboratory prototypes.

Trevisan et al. [163] mentioned that one of the central design aspects in HCI in AR is concerned with how real and virtual objects are combined into a real environment where a user

Future trends and directions

Although much progress in AR has been made in the recent two decades, potential AR manufacturing applications are still in exploratory and prototyping stages. This is unlike in games, education and entertainment, where accuracy and robustness are of a lesser concern. With significant improvements in tracking algorithms and faster response time of hardware, many manufacturing operations can be simulated effectively in near real-time where users will no longer perceive a time lapse, jittering of

Conclusions

Augmented reality has taken the world by storm as it has found applications in every area from sports, gaming, sales, advertising, learning, touring, to medical and manufacturing applications. New applications are being developed almost daily. Its ability to provide high user intuition and the relative ease of implementation has outperformed VR, which was one of the most notable impacts of the late 1990s. AR on handheld devices has found a proliferation of applications with the advent of smart

Acknowledgements

The authors would like to thank the following colleagues for providing insightful and useful inputs to substantiate this keynote paper: R. Starker, Stephen Lu, Chris Zhang, and many other colleagues through verbal discussions. The authors would also like to thank all their graduate students for their contributions, without which this keynote paper would not have been possible.

References (187)

  • H. Kaufmann et al.

    Mathematics and Geometry Education with Collaborative Augmented Reality

    Computers and Graphics

    (2003)
  • S. Kim et al.

    AR Interfacing with Prototype 3D Applications Based on User-centered Interactivity

    Computer Aided Design

    (2010)
  • S. Lee et al.

    Augmented Reality-based Computational Fieldwork Support for Equipment Operations and Maintenance

    Automation in Construction

    (2011)
  • Y. Lin et al.

    Towards a Novel Interface Design Framework: Function–Behavior-State Paradigm

    International Journal of Human Computer Studies

    (2004)
  • Y. Lin et al.

    Using Eye Movement Parameters for Evaluating Human–Machine Interface Frameworks under Normal Control Operation and Fault Detection Situations

    International Journal of Human Computer Studies

    (2003)
  • Y. Lin et al.

    An Approach to Computing Human Attention Allocation Using Eyegaze Tracking Technique

    International Journal of Human Computer Studies

    (2009)
  • Z. Liu et al.

    Motion Navigation for Arc Welding Robots Based on Feature Mapping in a Simulation Environment

    Robotics and Computer-Integrated Manufacturing

    (2010)
  • S.C.-Y. Lu et al.

    Virtual and Augmented Reality Technologies for Product Realization

    Annals of the CIRP

    (1999)
  • K.H. Ahlers et al.

    Distributed Augmented Reality for Collaborative Design Applications

  • K. Alexopoulos et al.

    A Web-based Integrated Framework for Collaborative Product Design

  • AndAR, http://code.google.com/p/andar/ (last accessed on...
  • B. Arendarski et al.

    Maintenance of Complex Machines in Electric Power Systems using Virtual Reality Techniques

  • ARMES, http://www.armes-tech.com/(last accessed...
  • ARTESAS – Advanced Augmented Reality Technologies for Industrial Service Applications, retrieved from,...
  • ARToolKit 2.11, http://www.hitl.washington.edu/research/shared_space/download (last accessed...
  • R. Azuma

    A Survey of Augmented Reality

    Presence: Teleoperators and Virtual Environments

    (1997)
  • R. Azuma et al.

    Recent Advances in Augmented Reality

    IEEE Computer Graphics and Applications

    (2001)
  • G. Baratoff et al.

    Developing and applying AR Technology in design, production, service and training

  • M. Billinghurst et al.

    Out and about: Real World Teleconferencing

    British Telecom Technical Journal

    (2000)
  • M. Billinghurst et al.

    The MagicBook-Moving Seamlessly between Reality and Virtuality

    IEEE Computer Graphics and Applications

    (2001)
  • R. Bischoff et al.

    The MORPHA Style Guide for Icon-based Programming

  • R. Bischoff et al.

    Perspectives on Augmented Reality Based Human–Robot Interaction with Industrial Robots

  • R. Bischoff et al.

    Concepts, Tools and Devices for Facilitating Human–Robot Interaction with Industrial Robots through Augmented Reality

  • W. Broll et al.

    The Virtual Round Table – a Collaborative Augmented Multi-User Environment

  • G.C. Burdea

    Virtual Reality and Robotics in Medicine

  • CAd Shroer MPDS4 Factory Layout. http://www.cadschroer.com/index.php?screen=1&ziel...
  • C. Calderon et al.

    A New Approach to Virtual Design for Spatial Configuration Problems

  • J. Carmigniani et al.

    Augmented Reality Technologies, Systems and Applications

    Multimedia Tools and Applications

    (2011)
  • G. Caruso et al.

    Interactive Augmented Reality System for Product Design Review

  • R. Castle et al.

    Video-rate Localization in Multiple Maps for Wearable Augmented Reality

  • S. Cawood et al.

    Augmented Reality: A Practical Guide

    (2008)
  • C.J. Chen et al.

    Haptic-based Interactive Path Planning for a Virtual Robot Arm

    International Journal of Interactive Design and Manufacturing

    (2010)
  • D. Cheng et al.

    Design of an Optical See-through Head-mounted Display with a Low f-Number and Large Field of View using a Freeform Prism

    Applied Optics

    (2009)
  • K. Chintamani et al.

    Improved Tele-Manipulator Navigation During Display–Control Misalignments using Augmented Reality cues

    IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans

    (2010)
  • G. Chryssolouris

    Manufacturing Systems – Theory and Practice

    (2006)
  • G. Chryssolouris et al.

    Digital Manufacturing: History, Perspectives, and Outlook

    Proceedings of the IMECHE Part B: Journal of Engineering Manufacture

    (2008)
  • G. Chryssolouris et al.

    A Web and Virtual Reality-based Platform for Collaborative Product Review and Customisation

  • Delmia 2009, DELMIA Virtual NC, http://www.delmia.com (last accessed...
  • J.Y. Didier et al.

    AMRA: Augmented Reality Assistance in Train Maintenance Tasks

  • F. Doil et al.

    Augmented Reality for Manufacturing Planning

  • Cited by (0)

    View full text