Elsevier

CIRP Annals

Volume 58, Issue 1, 2009, Pages 139-142
CIRP Annals

A mixed reality environment for collaborative product design and development

https://doi.org/10.1016/j.cirp.2009.03.020Get rights and content

Abstract

This paper presents a mixed reality (MR) environment for collaborative product design and development. This MR collaborative environment is based on a client/server architecture where multiple users can create and modify product features in a 3D physical space simultaneously. A tri-layer model representation scheme is designed to facilitate product creation and visualization. Intuitive feature manipulation methods and grid-and-snap modes have been designed to support solid modelling in the MR environment. Bi-directional communication between the MR environment and the CAD system ensures any modifications made by one user are propagated to the views of other users for maintaining design data consistency.

Introduction

Collaborative design is a “human-centric” technical activity [1], where at least two designers interact with each other during the design process. Web-based [2], [3] and agent-based [4], [5] collaborative design systems have been developed to support distributed collaborative design. However, these systems are mainly computer-centric where the modelling process and users are constrained to 2D model visualization and interfaces [6], [7]. Virtual reality (VR) technology has been explored to provide visualization and interaction of 3D models [8], [9].

Mixed reality (MR) is the superimposition of computer graphics over real objects/scenes. With virtual information augmented onto a real scene, MR can improve a user's perception of the real world and facilitate user-to-computer and user-to-user interactions. A semi-immersive MR design environment allows the users to see the real world and make use of real entities in the real world to support and perform feature modelling with virtual products, providing more realism to these users [10]. An MR environment is easy and inexpensive to set up without the need to model the entire background environment entities [11]. Users can visualize and interact with virtual products in a 3D physical space using intuitive virtual interfaces, and take full advantage of the real objects as the communication/interaction tools. Hence, MR allows product design and modelling in a real environment using intuitive interfaces, leading to a shorter time-to-market [12] and without the need for expensive modelling.

Collaborative MR design systems are either visualization-based or co-design systems in which designers can create and modify a 3D model collaboratively in a 3D space. There are few co-design MR systems due to the limitations in the data exchange, interaction tools and tracking accuracy. Most existing MR collaborative systems are visualization-based. In these systems, the users cannot directly create or modify the 3D models. They can only change the position, orientation and scale of the models, e.g., Construct3D [10].

This paper presents a MR collaborative product design development, MR-CD, where users can visualize and modify product models using intuitive interfaces, and view and interact with the virtual products in relation real objects to the real environment. When the users are co-located, gestures, gaze and face expressions enhance the communications and facilitate the discussion among the users. When the users are geographically distributed, MR-CD provides more intuitive information to enhance the users’ awareness of the existence and actions of the other collaborators. Fig. 1 shows two users in the same room interacting with a virtual model in the MR-CD environment, using a pen as an interaction and communication tool.

Section snippets

MR-CD system architecture

MR-CD is developed based on a client/server architecture where multiple users can create and modify product features in a 3D physical space simultaneously. The system architecture consists of the following main components (Fig. 2):

  • Clients representing views of experts in a product life cycle;

  • A server for product modelling, and collaboration and constraints management;

  • Interaction techniques for product and features modelling in the clients’ views; and

  • Embedded modelling kernel for solid

Features creation and locating

The modelling process in MR-CD is shown in Fig. 6. Based on the properties of the features, three sets of rules have been devised for features modelling, as shown in Fig. 7. An intuitive drag-and-drop method based on shape control points (SCP) has been implemented for feature modelling.

The SCPs of a displayed feature are the points that completely define the shape of the feature in the spatial 3D space. The SCPs are devised to be used with the drag-and-drop method in the MR 3D space. In MR-CD,

Case studies and user studies

Case studies have been performed to validate the MR-CD system. It was observed that the modelling and geometrical information extraction processes are computationally intensive, and consumed more than 50% of the CPU time. However, they do not affect the rendering of the virtual objects as they are performed on the server side.

User studies have been performed to evaluate the system and the interfaces. It was found that the system is user-friendly and easy to use, e.g., manipulating virtual

Conclusion

This paper presents an architecture and the mechanisms supporting product design in multiple view collaboration. Based on a server/client framework, this system can be used by multiple experts to modify and discuss a product or introduce a new product. The co-modelling mechanism facilitates distributed design modelling, and a tri-layer model representation scheme supports product visualization and collaborative design.

Direct design visualization, exploration and modification in MR-CD provide

References (12)

There are more references available in the full text version of this article.

Cited by (44)

  • Development of an automatic modeling method for patient-specific aortic graft reconstruction guide in thoracoabdominal aortic repair

    2022, Computer Methods and Programs in Biomedicine
    Citation Excerpt :

    An efficient computer-aided design (CAD) system for modeling a patient-specific surgical guide, a simulator, an education phantom, and biology applications is necessary in the medical field [1–8]. These designs have been implemented by combining the various manufacturing and visualizing processes, including additive and subtractive manufacturing, augment reality, virtual reality, mixed reality, and medical robotics [9–16]. The benefits of efficient CAD are less lead exposure time and increased productivity and quality.

  • Closed-loop augmented reality towards accurate human-robot collaboration

    2020, CIRP Annals
    Citation Excerpt :

    Interactive drag-and-drop functions were realised for robot programming [4]. Similarly, a Mix Reality (MR) approach was proposed for collaborative design [5,6]. In this approach, the virtual prototype can be manipulated and its process sequence can be decided.

View all citing articles on Scopus
View full text