Calligraphic InterfacesA multi-layered architecture for sketch-based interaction within virtual environments
Introduction
Leonardo da Vinci's drawings of machines and other objects illustrate one of the most fundamental purposes of sketches: the ability to communicate design and functionality to others. Nowadays, it is widely accepted that sketching is a form of critical, reflective dialog that handles communication on one or more different levels of abstraction simultaneously [1]. Various approaches have been taken to support this kind of dialog between humans and computers, and to build human–computer interfaces that are able to interpret such freehand sketches for different purposes.
In this context, the creation or reconstruction of 3D objects from 2D sketches is of major concern in many application areas. This so-called “pencil-and-paper” approach is used for rapidly designing approximate three-dimensional scenes. While some systems analyze the orthographic or perspective projections to reconstruct 3D shapes that, based on psychological assumptions, are most plausible to the human observer, others interpret 2D gestures while the objects are sketched.
Within the last decade, the conceptual design phase has been increasingly supported by sketch systems that allow the expression of ideas on a computer-aided, but still human-centered basis. However, putting an emphasis on sketching, most of these systems are sealed off from real-world applications rather than being generally applicable as components.
To prevent a separation between sketch systems and real-world applications, we propose the integration of current results into an architectural pattern that offers existing applications an individual utilization of sketching within their user interface. Multi-layered architectural patterns are widely employed in many areas of soft- and hardware engineering. They offer multiple levels of abstraction, component reuse, exchangeability and encapsulated reengineering of single components, the individual combination of components, and extendibility through pre-defined interfaces.
With this article, we want to introduce a multi-layered architecture for sketch-based interaction within virtual environments to benefit from the inherent advantages mentioned above. We describe each layer of the architecture using examples illustrating its implementation. Furthermore, we present a variety of domain-specific applications of sketching within virtual environments based on our architecture instead of implementing yet another application for sketching.
Section snippets
Previous work on sketching
Brown University's Sketch system [2] is an example of an early development that processes 2D strokes while they are sketched on the film plane to create predefined 3D primitives.
Since Sketch supports the creation of simple CSG-like primitives, its concept has been extended towards freeform modeling. Teddy [3], for instance, is another desktop-based system that allows the creation of 3D polygonal freeform surfaces from sketched 2D silhouettes.
STILTON [4] is yet another desktop-system that allows
The architecture
Our architecture (cf. Fig. 1) consists of eight hierarchically arranged layers, which are described below by giving examples of how they are implemented. Each layer can interact with its direct upper or lower neighbor, whereby every layer can be deactivated, making the next activated layer a direct neighbor. This modularization concept offers applications the opportunity for individual utilization of the required functionality.
The core layers (emphasized in Fig. 1) contain the intelligent parts
Conclusion and discussion
In this article, we have described a multi-layered architecture for sketch-based interaction within three-dimensional virtual environments. We have demonstrated that a broad palette of high-level interaction techniques, such as object creation, object interaction, freeform modeling, text input, and environment control can be realized by offering sketching. Rather than developing general sketching applications, these techniques were integrated into existing domain-specific applications to extend
References (31)
Natural intelligence in design
Elsevier Design Studies — The International Journal for Design Research in Engineering, Architecture, Products and Systems
(1999)Optimal unsupervised learning on a single-layer feed-forward neural network
Neural Networks
(1989)- et al.
Sketch: an interface for sketching 3D scenes
Computer Graphics (Proceedings of SIGGRAPH’96, Annual Conference Series)
(1996) - et al.
Teddy: a sketching interface for 3D freeform design
Computer Graphics (Proceedings, Annual Conference Series, ACM SIGGRAPH)
(1999) - Turner A, Chapman D, Penn A. Sketching a virtual environment: modeling using line-drawing interpretation. Proceedings...
- et al.
3-draw: a tool for designing 3D shapes
IEEE Computer Graphics and Applications
(1991) - Forsberg AS, LaViola JJ, Zeleznik RC. ErgoDesk: a framework for two- and three-dimensional interaction at the...
- Bier E, Stone M, Pier K, Buxton W, DeRose T. Toolglasses and magic lenses: the see-through interface. Proceedings of...
- Schkolne S, Schroeder P. Surface drawing. Technical report CS-TR-99-03, Caltech Department of Computer...
- et al.
Seamless interaction in virtual reality
IEEE Computer Graphics & Applications
(1997)
A translucent sketch-pad for the virtual table exploring motion-based gesture recognition
Computer and Graphics Forum (Proceedings of EUROGRAPHICS’99)
Cited by (52)
A procedural method to exchange editable 3D data from a free-hand 2D sketch modeling system into 3D mechanical CAD systems
2012, CAD Computer Aided DesignCitation Excerpt :Interpretation is a process of inferring a modeling command corresponding to a sequence of the recognized strokes. In this study, existing methods are used for stroke recognition and stroke interpretation for gestural modeling [6,29]. A prototype system implemented in this study recognizes seven types of operational strokes or geometry strokes, as shown in Fig. 5.
A survey of sketch-based 3-D modeling techniques
2009, Interacting with ComputersCitation Excerpt :Some projects have taken this approach quite literally. Bimber et al. (2000) for example describe an immersive modeling system where in the user stands above a 3-D workbench display wearing special glasses. The user is provided both a stylus, as well as a translucent plexiglass sketchpad, both of which are tracked in 3-D.
Regularity selection for effective 3D object reconstruction from a single line drawing
2008, Pattern Recognition LettersVirtual environment applications for front-end design and manufacturing planning applications
2021, Manufacturing In The Era Of 4th Industrial Revolution: A World Scientific Reference (In 3 Volumes)Analysis and recognition of hand-drawn images with effective data handling
2019, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)