Skip to main content
SpringerLink
Log in

Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

In the environment of mixed reality (MR) or augmented reality (AR), there have been several previous works dealing with user interfaces for manipulating and interacting with virtual objects aimed at improving immersive feeling and natural interaction. However, it is still considered that there must be much progress in supporting human behavior-like interactions for providing control efficiency and natural feeling in MR/AR environments. This paper proposes a tangible interaction method by combining the advantages of soft interactions such as hand gesture and MR and hard interactions such as vibro-tactile feedback. One of the main goals is to provide more natural interaction interfaces similar to the manipulation task in the real world by utilizing hand gesture-based tangible interactions. It also provides multimodal interfaces by adopting the vibro-tactile feedback and tangible interaction for the virtual object manipulation. Thus, it can make users get more immersive and natural feeling in the manipulation and interaction with virtual objects. Furthermore, it provides an alternative instruction guideline based on the analysis of the previous interaction while manipulating virtual objects, which makes it possible for the user to minimize manipulation errors during the interaction phase and the learning process, which guides the user to the right direction. We will show the effectiveness and advantage of the proposed approach by demonstrating several implementation results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Weiser M (1999) The computer of the 21st century. ACM SIGMOBILE Mobile Computing and Communications Review 3:3–11

  2. Lee JY, Rhee GW, Kim H, Lee K-W, Suh Y-H, Kim K (2006) Convergence of context-awareness and augmented reality for ubiquitous services and immersive interactions. LNCS 3983:466–474

    Google Scholar 

  3. Schmalstieg D, Reitmayr G (2005) The world as a user interface: augmented reality for ubiquitous computing. Proc. of Central European Multimedia and Virtual Reality Conf., Prague, Czech Republic

    Google Scholar 

  4. Piekarski W (2004) Interactive 3D modeling in outdoor augmented reality worlds. PhD Thesis, The University of South Australia

  5. Ong SK, Yuan ML, Nee AYC (2008) Augmented reality applications in manufacturing: a survey. Int J Prod Res 46:2707–2742

    Article  MATH  Google Scholar 

  6. Seichter H (2007) Augmented reality and tangible user interfaces in collaborative urban design. Proc. of the 12th International CAAD Futures Conf., Sydney, Australia

  7. Sidharta R, Oliver J, Sannier A (2006) Augmented reality tangible interface for distributed design review. Proc. of the International Conference on Computer Graphics, Imaging and Visualization 464–470, Sydney, Australia

  8. Lee JY, Rhee GW (2008) Context aware adaptable ubiquitous car services using augmented reality. Int Jo Advan Manuf Technol 37:431–442

    Article  Google Scholar 

  9. Lee JY, Seo DW, Rhee GW (2008) Visualization and interaction of pervasive services using context-aware augmented reality. Expert Syst Appl 35:1873–1882

    Article  Google Scholar 

  10. Kato H, Billinghurst M, Poupyrev I, Imamoto K, Tachibana K (2000) Virtual object manipulation on a table-top AR environment. Proc. of ISAR 2000 111-119, Munich, Germany

  11. Park H, Moon H-C, Lee JY (2009) Tangible augmented prototyping of digital handheld products. Comput Ind 60:114–125

    Article  Google Scholar 

  12. Lee JY, Rhee G, Park H (2009) AR/RP-based tangible interactions for collaborative design evaluation of digital products. Int J AdvanManuf Technol 45:649–665

    Article  Google Scholar 

  13. Terry M, Cheung J, Lee J, Park T, Williams N (2007) Jump: a system for interactive, tangible queries of paper. Proc. of Graphics Interface 2007 127–134, Montreal, Canada

  14. Hong D, Looser J, Seichter H, Billinghurst M, Woo W (2008) A sensor-based interaction for ubiquitous virtual reality systems. Proc. of International Symp. on Ubiquitous Virtual Reality 2008 10–13, Gwangju, South Korea

  15. Buchmann V, Violich S, Billinghurst M, Cockburn A (2004) FingARtips: gesture based direct manipulation in augmented reality. Proc. of the 2nd International Conf. on Computer Graphics and Interactive Techniques in Australasia and South East Asia 212–221, Singapore

  16. Ott R, Thalmann D, Vexo F (2007) Haptic feedback in mixed-reality environment. Vis Comput 23:843–849

    Article  Google Scholar 

  17. White M, Petridis P, Liarokapis F, Plecinckx D (2007) Multimodal mixed reality interfaces for visualizing digital heritage. Int J Architect Comput 5:322–337

    Article  Google Scholar 

  18. Thomas BH, Piekarski W (2002) Glove based user interaction techniques for augmented reality in an outdoor environment. Virtual Real 6:167–180

    Article  Google Scholar 

  19. Robert W, Linderman JN, Sibert JL, Cutler JR (2002) Handling of virtual contact in immersive virtual environments: beyond visuals. Virtual Real 6:130–139

    Article  Google Scholar 

  20. Veigl S, Kaltenbach A, Ledermann F, Reitmayr G, Schmalstieg D (2002) Two-handed direct interaction with ARToolkit. Proc. of IEEE First International Augmented Reality Toolkit Workshop, Darmstadt, Germany

  21. Yi X, Qin S, Kang J (2009) Generating 3D architectural models based on hand motion and gesture. Comput Ind 60:677–685

    Article  Google Scholar 

  22. ARToolkit (2005) http://artoolkit.sourceforge.net

  23. Bowman DA, Wingrave CA (2001) Design and evaluation of menu systems for immersive virtual environments. Proc. of IEEE Virtual Reality 2001 149-156, Yokohama, Japan

  24. David H (1987) Statecharts: a visual formalism for complex systems. Sci Comput Programming archive 8:231–274

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, Chonnam National University, Yongbong-Dong, Buk-gu, Gwangju, 500-757, South Korea

    Jae Yeol Lee & Dong Woo Seo

  2. Asiana IDT, Gwangju, South Korea

    Gue Won Rhee

Authors
  1. Jae Yeol Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gue Won Rhee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong Woo Seo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jae Yeol Lee.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lee, J.Y., Rhee, G.W. & Seo, D.W. Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment. Int J Adv Manuf Technol 51, 1069–1082 (2010). https://doi.org/10.1007/s00170-010-2671-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-010-2671-x

Keywords

Search

Navigation