Skip to main content
SpringerLink
Log in

Virtual reality training for assembly of hybrid medical devices

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Skill training in the medical device manufacturing industry is essential to optimize and expedite the efficiency level of new workers. This process, however, gives rise to many underlying issues such as contamination and safety risks, long training period, high skill and experience requirements of operators, and greater training costs. In this paper, we proposed and evaluated a novel virtual reality (VR) training system for the assembly of hybrid medical devices. The proposed system, which is an integration of Artificial Intelligence (AI), VR and gaming concepts, is self-adaptive and autonomous. This enables the training to take place in a virtual workcell environment without the supervision of a physical trainer. In this system, a sequential framework is proposed and utilized to enhance the training through its various “game” levels of familiarity-building processes. A type of hybrid medical device: carbon nanotubes-polydimethylsiloxane (CNT-PDMS) based artificial trachea prosthesis is used as a case study in this paper to demonstrate the effectiveness of the proposed system. Evaluation results with quantitative and qualitative comparisons demonstrated that our proposed training method has significant advantages over common VR training and conventional training methods. The proposed system has addressed the underlying training issues for hybrid medical device assembly by providing trainees with effective, efficient, risk-free and low cost training.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  1. Alamri A, Eid M, Iglesias R, Shirmohammadi S, Saddik AE (2008) Haptic virtual rehabilitation exercises for post-stroke diagnosis. IEEE Trans Instrum Meas 57(9):1876–1884

    Google Scholar 

  2. Albert DE (2015) Methods for verifying medical device cleanliness. In: Developments in surface contamination and cleaning, volume 7: cleanliness validation and verification. Elsevier, Amsterdam, pp 109–128

    Google Scholar 

  3. Boian RF, Sharma AK, Han C, Merians AN, Burdea GC, Adamovich S, Recce ML, Tremaine M, Poizner H (2002) Virtual reality-based post-stroke hand rehabilitation. Stud Health Technol Inform 85(1):64–70

    Google Scholar 

  4. Borst CW, Indugula AP (2006) A spring model for whole-hand virtual grasping. Presence-Teleop VIRT 15(1):47–61

    Google Scholar 

  5. Brough JE, Schwartz M, Gupta SK, Anand DK, Kavetsky R, Pettersen R (2007) Towards the development of a virtual environment-based training system for mechanical assembly operations. Virtual Reality 11(4):189–206

    Google Scholar 

  6. Chua M, Chui CK, Chng CB, Lau D (2013) Carbon nanotube-based artificial tracheal prosthesis: carbon nanocomposite implants for patient-specific ENT care. IEEE Nanotechnol Mag 7(4):27–31

    Google Scholar 

  7. Chua M, Chui CK, Chng CB, Lau D (2013) Experiments of carbon nanocomposite implants for patient specific ENT care. In: The 7th IEEE international conference on nano/molecular medicine and engineering (NANOMED), Phuket

  8. Chua M, Chui CK, Teo C, Lau D (2015) Patient-specific carbon nanocomposite tracheal prosthesis. Int J Artif Organs 38(1):31–38

    Google Scholar 

  9. Ciurana J (2014) Designing, prototyping and manufacturing medical devices: an overview. Int J Comput Integr Manuf 27(10):901–918

    Google Scholar 

  10. Ciuti G, Ricotti L, Menciassi A, Dario P (2015) MEMS sensor technologies for human centred applications in healthcare, physical activities, safety and environmental sensing: a review on research activities in italy. Sensors (Basel) 15(3):6441–6468

    Google Scholar 

  11. Connolly J, Condell J, O’Flynn B, Sanchez JT, Gardiner P (2018) IMU sensor-based electronic goniometric glove for clinical finger movement analysis. IEEE Sensors J 18(3):1273–1281

    Google Scholar 

  12. Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low- and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst Hum 43(4):996–1002

    Google Scholar 

  13. Ding M, Fan G (2016) Articulated and generalized gaussian kernel correlation for human pose estimation. IEEE Trans Image Process 25(2):776–789

    MathSciNet  MATH  Google Scholar 

  14. Gallagher AG, Satava RM (2002) Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Surg Endosc 16(12):1746–1752

    Google Scholar 

  15. Gislén L (2016) Alternative design of robot cell concepts for flexible production. (Master’s thesis, University West), Trollhättan, Sweden

  16. Grantcharov TP, Kristiansen VB, Bendi J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91(2):146–150

    Google Scholar 

  17. He Y, Zhang Z, Nan X, Zhang N, Guo F, Rosales E, Guan L (2017) vConnect: perceive and interact with real world from CAVE. Multimed Tools Appl 76(1):1479–1508

    Google Scholar 

  18. Ho N, Ang W, Chui CK (2017) Workcell for hybrid medical device fabrication. In: 3rd CIRP conference on biomanufacturing, Chicago

  19. Hoedt S, Claeys A, Landeghem HV, Cottyn J (2016) Evaluation framework for virtual training within mixed-model manual assembly. IFAC-PapersOnLine 49(12):261–266

    Google Scholar 

  20. Hořejší P (2015) Augmented reality system for virtual training of parts assembly. In: 25th DAAAM international symposium on intelligent manufacturing & automation, Vienna

  21. Hyppölä J, Martínez H, Laukkanen S (2014) Experiential learning theory and virtual and augmented reality applications. In: 4th global conference on experiential learning in virtual worlds, Prague, Czech Republic

  22. Jayaram S, Jayaram U, Kim YJ, DeChenne C, Lyons KW, Palmer C, Mitsui T (2007) Industry case studies in the use of immersive virtual assembly. Virtual Reality 11(4):217–228

    Google Scholar 

  23. Kılıboz NÇ, Güdükbay U (2015) A hand gesture recognition technique for human–computer interaction. J Vis Commun Image Represent 28(C):97–104

    Google Scholar 

  24. Kizony R, Raz L, Katz N, Weingarden H, Weiss PLT (2005) Video-capture virtual reality system for patients with paraplegic spinal cord injury. J Rehabil Res Dev 42(5):595–608

    Google Scholar 

  25. Konig J, Koskela E (2013) The role of profit sharing in dual labour markets with flexible outsourcing. Labour 27(4):357–370

    Google Scholar 

  26. Kusuda Y (2010) IDEC's robot-based cellular production system: a challenge to automate high-mix low-volume production. Assem Autom 30(4):306–312

    Google Scholar 

  27. Langley A, Lawson G, Hermawati S, D'Cruz M, Apold J, Arlt F, Mura K (2016) Establishing the usability of a virtual training system for assembly operations within the automotive industry. Hum Factors Ergon Manuf Serv Ind 26(6):667–679

    Google Scholar 

  28. Lawson MV (2003) Finite automata. Chapman and Hall/CRC, Boca Raton

    Google Scholar 

  29. Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181(1):108–115

    Google Scholar 

  30. Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2016) Action2Activity: recognizing complex activities from sensor data. In: Proceedings of the twenty-fourth international joint conference on artificial intelligence, Buenos Aires

  31. Lucke L, Anderson D, Smith D (2009) Harnessing experience for efficient medical device product development. J Med Devices 3(2):027551

    Google Scholar 

  32. Manjiyani ZAA, Jacob RT, R KK, Varghese B (2014) Development of MEMS based 3-axis accelerometer for hand movement monitoring. International Journal of Scientific and Research Publ 4(2):1–4

    Google Scholar 

  33. Müller BC, Reisea C, Duc BM, Seliger G (2016) Simulation-games for learning conducive workplaces: a case study for manual assembly. In: 13th global conference on sustainable manufacturing - decoupling growth from resource use, Ho Chi Minh

  34. Nee AYC, Ong SK (2013) Virtual and augmented reality applications in manufacturing. In: 7th IFAC conference on manufacturing modelling, management, and control, Saint Petersburg, Russia

  35. Nerem RM, Sambanis A (2007) Tissue engineering: from biology to biological substitutes. Tissue Eng 1(1):3–13

    Google Scholar 

  36. Neto P, Pires JN, Moreira AP (2013) 3-D position estimation from inertial sensing: minimizing the error from the process of double integration of accelerations. In: IECON 2013 - 39th annual conference of the IEEE industrial electronics society, Vienna

  37. Pilia A, Brun C, Doceul L, Gargiulo L, Hatchressian JC, Keller D, Le R, Poli S, Zago B (2015) Application of virtual reality tools for assembly of WEST components: comparison between simulations and physical mockups. Fusion Eng Des 98–99:1589–1592

    Google Scholar 

  38. Premkumar T, Geckeler KE (2012) Graphene–DNA hybrid materials: assembly, applications, and prospects. Prog Polym Sci 37(4):515–529

    Google Scholar 

  39. Rahni AAA, Yahya I (2008) Obtaining translation from a 6-DOF MEMS IMU — an overview. In: 2007 Asia-Pacific conference on applied electromagnetics, Melaka

  40. Rodriguez L, Quint F, Gorecky D, Romero D, Siller HR (2015) Developing a mixed reality assistance system based on projection mapping technology for manual operations at assembly workstations. In: International conference on virtual and augmented reality in education, Monterrey

  41. Rose P (2014) Whitepaper: contamination control of medical device cleanroom. Maetrics, Indianapolis

    Google Scholar 

  42. Sahithia K, Swethaa M, Ramasamya K, Srinivasanb N, Selvamurugan N (2010) Polymeric composites containing carbon nanotubes for bone tissue engineering. Int J Biol Macromol 46(3):281–283

    Google Scholar 

  43. Saposnik G, Levin M (2011) Virtual reality in stroke rehabilitation: a meta-analysis and implications for clinicians. Stroke 42(5):1380–1386

    Google Scholar 

  44. Schwaitzberg SD, Dorozhkin D, Sankaranarayanan G, Matthes K, Jones DB, De S (2016) Natural orifice translumenal endoscopic surgery (NOTES): emerging trends and specifications for a virtual simulator. Surg Endosc 30(1):190–198

    Google Scholar 

  45. Seth A, Vance JM, Oliver JH (2011) Virtual reality for assembly methods prototyping: a review. Virtual Reality 15(1):5–20

    Google Scholar 

  46. Shirke S, Chilaveri PG (2013) Traffic light control using accelerometer sensor on ARM platform. Int J Eng Res Technol 2(9):3124–3126

    Google Scholar 

  47. Stančin S, Tomažič S (2011) Angle estimation of simultaneous orthogonal rotations from 3D gyroscope measurements. Sensors (Basel) 11(9):8536–8549

    Google Scholar 

  48. Stratos A, Loukas R, Dimitris M, Konstantinos G, Dimitris M, George C (2016) A virtual reality application to attract young talents to manufacturing. In: 49th CIRP Conference on MANUFACTURING SYSTEMS - CIRP CMS 2016, Stuttgart, Germany

  49. Sturman DJ, Zeltzer D (1994) A survey of glove-based input. IEEE Comput Graph Appl 14(1):30–39

    Google Scholar 

  50. Suzuki N, Hattori A, Ieiri S, Tomikawa M, Kenmotsu H, Hashizume M (2012) VR training system for endoscopic surgery robot. In: Augmented environments for computer-assisted interventions. AE-CAI 2011. Lecture notes in computer science, vol 7264. Springer, Berlin, pp 117–129

    Google Scholar 

  51. Tang Z, Sekine M, Tamura T, Tanaka N, Yoshida M, Chen W (2015) Measurement and estimation of 3D orientation using magnetic and inertial sensors. Adv Biomed Eng 4(1):135–143

    Google Scholar 

  52. Vaughan N, Gabrys B, Dubey VN (2016) An overview of self-adaptive technologies within virtual reality training. Comput Sci Rev 22:65–87

    MathSciNet  Google Scholar 

  53. Wang Y, Guo S, Li Y, Tamiya T, Song Y (2018) Design and evaluation of safety operation VR training system for robotic catheter surgery. Med Biol Eng Comput 56(1):25–35

    Google Scholar 

  54. Wasfy A, Wasfy T, Noor A (2004) Intelligent virtual environment for process training. Adv Eng Softw 35(6):337–355

    Google Scholar 

  55. Xia P, Lopes AM, Restivo MT, Yao Y (2012) A new type haptics-based virtual environment system for assembly training of complex products. Int J Adv Manuf Technol 58(1–4):379–396

    Google Scholar 

  56. Yang Q, Wu DL, Zhu HM, Bao JS, Wei ZH (2013) Assembly operation process planning by mapping a virtual assembly simulation to real operation. Comput Ind 64(7):869–879

    Google Scholar 

  57. Yu Y, Duan M, Sun C, Zhong Z, Liu H (2017) A virtual reality simulation for coordination and interaction based on dynamics calculation. Ships Offshore Struc 12(6):873–884

    Google Scholar 

  58. Yuviler-Gavish N, Krupenia S, Gopher D (2013) Task analysis for developing maintenance and assembly VR training simulators. Ergon Des 21(1):12–19

    Google Scholar 

  59. Zhou J, Malric F, Shirmohammadi S (2010) A new hand-measurement method to simplify calibration in cyber glove-based virtual rehabilitation. IEEE Trans Instrum Meas 59(10):2496–2504

    Google Scholar 

Download references

Acknowledgements

We wish to acknowledge the contributions from Mr. Rahul Singh Chauhan, Mr. Jayendra Laxman Zambre and Mr. Ritesh Kumar Agrahari for their assistance in the development of the VR systems. This project is supported in parts by a MOE FRC Tier 1 Grant from National University of Singapore (WBS: R-265-000-614-114).

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, National University of Singapore, 9 Engineering Drive 1, Singapore, 117576, Singapore

    Nicholas Ho, Pooi-Mun Wong & Chee-Kong Chui

  2. Institute of Systems Science, National University of Singapore, 25 Heng Mui Keng Terrace, Singapore, 119615, Singapore

    Matthew Chua

Authors
  1. Nicholas Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pooi-Mun Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew Chua
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chee-Kong Chui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicholas Ho.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ho, N., Wong, PM., Chua, M. et al. Virtual reality training for assembly of hybrid medical devices. Multimed Tools Appl 77, 30651–30682 (2018). https://doi.org/10.1007/s11042-018-6216-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6216-x

Keywords

Search

Navigation