Elsevier

Computers & Graphics

Volume 24, Issue 6, December 2000, Pages 851-867
Computers & Graphics

Calligraphic Interfaces
A multi-layered architecture for sketch-based interaction within virtual environments

https://doi.org/10.1016/S0097-8493(00)00088-1Get rights and content

Abstract

In this article, we describe a multi-layered architecture for sketch-based interaction within virtual environments. Our architecture consists of eight hierarchically arranged layers that are described by giving examples of how they are implemented and how they interact. Focusing on table-like projection systems (such as Virtual Tables or Responsive Workbenches) as human-centered output-devices, we show examples of how to integrate parts or all of the architecture into existing domain-specific applications — rather than realizing new general sketch applications — to make sketching an integral part of the next-generation human–computer interface.

Introduction

Leonardo da Vinci's drawings of machines and other objects illustrate one of the most fundamental purposes of sketches: the ability to communicate design and functionality to others. Nowadays, it is widely accepted that sketching is a form of critical, reflective dialog that handles communication on one or more different levels of abstraction simultaneously [1]. Various approaches have been taken to support this kind of dialog between humans and computers, and to build human–computer interfaces that are able to interpret such freehand sketches for different purposes.

In this context, the creation or reconstruction of 3D objects from 2D sketches is of major concern in many application areas. This so-called “pencil-and-paper” approach is used for rapidly designing approximate three-dimensional scenes. While some systems analyze the orthographic or perspective projections to reconstruct 3D shapes that, based on psychological assumptions, are most plausible to the human observer, others interpret 2D gestures while the objects are sketched.

Within the last decade, the conceptual design phase has been increasingly supported by sketch systems that allow the expression of ideas on a computer-aided, but still human-centered basis. However, putting an emphasis on sketching, most of these systems are sealed off from real-world applications rather than being generally applicable as components.

To prevent a separation between sketch systems and real-world applications, we propose the integration of current results into an architectural pattern that offers existing applications an individual utilization of sketching within their user interface. Multi-layered architectural patterns are widely employed in many areas of soft- and hardware engineering. They offer multiple levels of abstraction, component reuse, exchangeability and encapsulated reengineering of single components, the individual combination of components, and extendibility through pre-defined interfaces.

With this article, we want to introduce a multi-layered architecture for sketch-based interaction within virtual environments to benefit from the inherent advantages mentioned above. We describe each layer of the architecture using examples illustrating its implementation. Furthermore, we present a variety of domain-specific applications of sketching within virtual environments based on our architecture instead of implementing yet another application for sketching.

Section snippets

Previous work on sketching

Brown University's Sketch system [2] is an example of an early development that processes 2D strokes while they are sketched on the film plane to create predefined 3D primitives.

Since Sketch supports the creation of simple CSG-like primitives, its concept has been extended towards freeform modeling. Teddy [3], for instance, is another desktop-based system that allows the creation of 3D polygonal freeform surfaces from sketched 2D silhouettes.

STILTON [4] is yet another desktop-system that allows

The architecture

Our architecture (cf. Fig. 1) consists of eight hierarchically arranged layers, which are described below by giving examples of how they are implemented. Each layer can interact with its direct upper or lower neighbor, whereby every layer can be deactivated, making the next activated layer a direct neighbor. This modularization concept offers applications the opportunity for individual utilization of the required functionality.

The core layers (emphasized in Fig. 1) contain the intelligent parts

Conclusion and discussion

In this article, we have described a multi-layered architecture for sketch-based interaction within three-dimensional virtual environments. We have demonstrated that a broad palette of high-level interaction techniques, such as object creation, object interaction, freeform modeling, text input, and environment control can be realized by offering sketching. Rather than developing general sketching applications, these techniques were integrated into existing domain-specific applications to extend

References (31)

  • N. Cross

    Natural intelligence in design

    Elsevier Design Studies — The International Journal for Design Research in Engineering, Architecture, Products and Systems

    (1999)
  • T. Sanger

    Optimal unsupervised learning on a single-layer feed-forward neural network

    Neural Networks

    (1989)
  • R.C. Zeleznik et al.

    Sketch: an interface for sketching 3D scenes

    Computer Graphics (Proceedings of SIGGRAPH’96, Annual Conference Series)

    (1996)
  • T. Igarashi et al.

    Teddy: a sketching interface for 3D freeform design

    Computer Graphics (Proceedings, Annual Conference Series, ACM SIGGRAPH)

    (1999)
  • Turner A, Chapman D, Penn A. Sketching a virtual environment: modeling using line-drawing interpretation. Proceedings...
  • E. Sachs et al.

    3-draw: a tool for designing 3D shapes

    IEEE Computer Graphics and Applications

    (1991)
  • Forsberg AS, LaViola JJ, Zeleznik RC. ErgoDesk: a framework for two- and three-dimensional interaction at the...
  • Bier E, Stone M, Pier K, Buxton W, DeRose T. Toolglasses and magic lenses: the see-through interface. Proceedings of...
  • Schkolne S, Schroeder P. Surface drawing. Technical report CS-TR-99-03, Caltech Department of Computer...
  • A. Forsberg et al.

    Seamless interaction in virtual reality

    IEEE Computer Graphics & Applications

    (1997)
  • Barco, Inc., BARON, URL: http://www.barco.com/projecti/products/bsp/baron.htm,...
  • Schmalstieg D, Encarnação LM, Szalavári ZS. Using transparent props for interacting with the virtual table. Proceedings...
  • Coquillart S, Wesche G. The virtual palette and the virtual remote control panel: a device and an interaction paradigm...
  • L.M. Encarnação et al.

    A translucent sketch-pad for the virtual table exploring motion-based gesture recognition

    Computer and Graphics Forum (Proceedings of EUROGRAPHICS’99)

    (1999)
  • Poupyrev I, Tomokuza N, Weghorst S. Virtual notepad: handwriting in immersive VR. Proceedings of IEEE VRAIS’98, 1998....
  • Cited by (52)

    • A procedural method to exchange editable 3D data from a free-hand 2D sketch modeling system into 3D mechanical CAD systems

      2012, CAD Computer Aided Design
      Citation Excerpt :

      Interpretation is a process of inferring a modeling command corresponding to a sequence of the recognized strokes. In this study, existing methods are used for stroke recognition and stroke interpretation for gestural modeling [6,29]. A prototype system implemented in this study recognizes seven types of operational strokes or geometry strokes, as shown in Fig. 5.

    • A survey of sketch-based 3-D modeling techniques

      2009, Interacting with Computers
      Citation Excerpt :

      Some projects have taken this approach quite literally. Bimber et al. (2000) for example describe an immersive modeling system where in the user stands above a 3-D workbench display wearing special glasses. The user is provided both a stylus, as well as a translucent plexiglass sketchpad, both of which are tracked in 3-D.

    • Virtual environment applications for front-end design and manufacturing planning applications

      2021, Manufacturing In The Era Of 4th Industrial Revolution: A World Scientific Reference (In 3 Volumes)
    • Analysis and recognition of hand-drawn images with effective data handling

      2019, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    View all citing articles on Scopus
    View full text