Abstract
In surgery, where mistakes have the potential for dire consequences, proper training plays a crucial role. Surgical training has traditionally relied upon experienced surgeons mentoring trainees through cadaveric dissection and operating theatre practice. However, with the growing demand for more surgeons and more efficient training programs, it has become necessary to employ supplementary forms of training such as virtual reality simulation. However, the use of such simulations as autonomous training platforms is limited by the extent to which they can provide automated performance feedback. Recent work has focused on overcoming this issue by developing algorithms to provide feedback that emulates the advice of human experts. These algorithms can mainly be categorized into rule-based and machine learning based methods, and they have typically been validated through user studies against controls that received no feedback. To our knowledge, no investigations into the performance of the two types of feedback generation methods in comparison to each other have so far been conducted. To this end, we introduce a rule-based method of providing technical feedback in virtual reality simulation-based temporal bone surgery, implement a machine learning based method that has been proven to outperform other similar methods, and compare their performance in teaching surgical skills in practice through a user study. We show that simpler rule-based methods can be equally or more effective in teaching surgical skills when compared to more complex methods of feedback generation.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Andersen, S.A.W., Foghsgaard, S., Konge, L., Cayé-Thomasen, P., Sørensen, M.S.: The effect of self-directed virtual reality simulation on dissection training performance in mastoidectomy. Laryngoscope 126(8), 1883–1888 (2016)
Baddeley, A.: Working memory: looking back and looking forward. Nat. Rev. Neurosci. 4(10), 829 (2003)
Burke, J.L., Prewett, M.S., Gray, A.A., Yang, L., Stilson, F.R., Coovert, M.D., Elliot, L.R., Redden, E.: Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis. In: ICMI, pp. 108–117 (2006)
Copson, B., Wijewickrema, S., Zhou, Y., Piromchai, P., Briggs, R., Bailey, J., Kennedy, G., O’Leary, S.: Supporting skill acquisition in cochlear implant surgery through virtual reality simulation. Cochlear Implants Int. 18(2), 89–96 (2017)
Crochet, P., Aggarwal, R., Dubb, S.S., Ziprin, P., Rajaretnam, N., Grantcharov, T., Ericsson, K.A., Darzi, A.: Deliberate practice on a virtual reality laparoscopic simulator enhances the quality of surgical technical skills. Ann. Surg. 253(6), 1216–1222 (2011)
Crossan, A., Brewster, S., Reid, S., Mellor, D.: Multimodal feedback cues to aid veterinary training simulations (2000)
Cui, Z., Chen, W., He, Y., Chen, Y.: Optimal action extraction for random forests and boosted trees. In: KDD, pp. 179–188 (2015)
Ericsson, K.A.: Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad. Med. 79(10), S70–S81 (2004)
Francis, H.W., Malik, M.U., Diaz Voss Varela, D.A., Barffour, M.A., Chien, W.W., Carey, J.P., Niparko, J.K., Bhatti, N.I.: Technical skills improve after practice on virtual-reality temporal bone simulator. Laryngoscope 122(6), 1385–1391 (2012)
Fried, M.P., Satava, R., Weghorst, S., Gallagher, A., Sasaki, C., Ross, D., Sinanan, M., Uribe, J., Zeltsan, M., Arora, H., et al.: Identifying and reducing errors with surgical simulation. Qual. Saf. Health Care 13(suppl 1), i19–i26 (2004)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: ICLR (2015)
Hall, R., Rathod, H., Maiorca, M., Ioannou, I., Kazmierczak, E., O’Leary, S., Harris, P.: Towards haptic performance analysis using k-metrics. In: Pirhonen, A., Brewster, S. (eds.) HAID 2008. LNCS, vol. 5270, pp. 50–59. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87883-4_6
Hattie, J., Timperley, H.: The power of feedback. Rev. Educ. Res. 77(1), 81–112 (2007)
Kerwin, T., Wiet, G., Stredney, D., Shen, H.W.: Automatic scoring of virtual mastoidectomies using expert examples. Int. J. Comput. Assist. Radiol. Surg. 7(1), 1–11 (2012)
Laeeq, K., Bhatti, N.I., Carey, J.P., Della Santina, C.C., Limb, C.J., Niparko, J.K., Minor, L.B., Francis, H.W.: Pilot testing of an assessment tool for competency in mastoidectomy. Laryngoscope 119(12), 2402–2410 (2009)
Lamata, P., Gomez, E.J., Bello, F., Kneebone, R.L., Aggarwal, R., Lamata, F.: Conceptual framework for laparoscopic VR simulators. IEEE Comput. Graph. Appl. 26(6), 69–79 (2006)
Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation forest. In: ICDM, pp. 413–422 (2008)
Ma, X., Bailey, J., Wijewickrema, S., Zhou, S., Mhammedi, Z., Zhou, Y., OLeary, S.: Adversarial generation of real-time feedback with neural networks for simulation-based training. In: IJCAI, pp. 3763–3769 (2017)
Ma, X., Wijewickrema, S., Zhou, Y., Zhou, S., O’Leary, S., Bailey, J.: Providing effective real-time feedback in simulation-based surgical training. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 566–574. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_64
Mackel, T., Rosen, J., Pugh, C.: Data mining of the e-pelvis simulator database: a quest for a generalized algorithm for objectively assessing medical skill. Stud. Health Technol. Inf. 119, 355–360 (2006)
Martin, J., Regehr, G., Reznick, R., Macrae, H., Murnaghan, J., Hutchison, C., Brown, M.: Objective structured assessment of technical skill (OSATS) for surgical residents. Br. J. Surg. 84(2), 273–278 (1997)
McGaghie, W.C., Issenberg, S.B., Petrusa, E.R., Scalese, R.J.: A critical review of simulation-based medical education research: 2003–2009. Med. Educ. 44(1), 50–63 (2010)
Oviatt, S.: Human-centered design meets cognitive load theory: designing interfaces that help people think. In: ACMMM, pp. 871–880 (2006)
Rhienmora, P., Haddawy, P., Suebnukarn, S., Dailey, M.N.: Intelligent dental training simulator with objective skill assessment and feedback. Artif. Intell. Med. 52(2), 115–121 (2011)
Rosenthal, R., DiMatteo, M.R.: Meta-analysis: recent developments in quantitative methods for literature reviews. Annu. Rev. Psychol. 52(1), 59–82 (2001)
Schmidt, R.A., Lee, T.D., et al.: Motor control and learning: a behavioral emphasis, vol. 4 (2005)
Schmidt, R.A., Young, D.E., Swinnen, S., Shapiro, D.C.: Summary knowledge of results for skill acquisition: support for the guidance hypothesis. J. Exp. Psychol. Learn. Mem. Cogn. 15(2), 352 (1989)
Sewell, C., Morris, D., Blevins, N.H., Dutta, S., Agrawal, S., Barbagli, F., Salisbury, K.: Providing metrics and performance feedback in a surgical simulator. Comput. Aided Surg. 13(2), 63–81 (2008)
Sigrist, R., Rauter, G., Riener, R., Wolf, P.: Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon. Bull. Rev. 20(1), 21–53 (2013)
Stefanidis, D.: Optimal acquisition and assessment of proficiency on simulators in surgery. Surg. Clin. North Am. 90(3), 475–489 (2010)
Stefanidis, D., Heniford, B.T.: The formula for a successful laparoscopic skills curriculum. Arch. Surg. 144(1), 77–82 (2009)
Wijewickrema, S., Ioannou, I., Zhou, Y., Piromchai, P., Bailey, J., Kennedy, G., OLeary, S.: Region-specific automated feedback in temporal bone surgery simulation. In: CBMS, pp. 310–315 (2015)
Wijewickrema, S., Ioannou, I., Zhou, Y., Promchai, P., Bailey, J., Kennedy, G., OLeary, S.: A temporal bone surgery simulator with real-time feedback for surgical training. In: NextMed/MMVR21, pp. 462–468 (2014)
Wijewickrema, S., Zhou, Y., Bailey, J., Kennedy, G., O’Leary, S.: Provision of automated step-by-step procedural guidance in virtual reality surgery simulation. In: VRST, pp. 69–72 (2016)
Zhou, Y., Bailey, J., Ioannou, I., Wijewickrema, S., Kennedy, G., O’Leary, S.: Constructive real time feedback for a temporal bone simulator. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8151, pp. 315–322. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40760-4_40
Zhou, Y., Bailey, J., Ioannou, I., Wijewickrema, S., O’Leary, S., Kennedy, G.: Pattern-based real-time feedback for a temporal bone simulator. In: VRST, pp. 7–16 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Wijewickrema, S. et al. (2018). Providing Automated Real-Time Technical Feedback for Virtual Reality Based Surgical Training: Is the Simpler the Better?. In: Penstein Rosé, C., et al. Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science(), vol 10947. Springer, Cham. https://doi.org/10.1007/978-3-319-93843-1_43
Download citation
DOI: https://doi.org/10.1007/978-3-319-93843-1_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93842-4
Online ISBN: 978-3-319-93843-1
eBook Packages: Computer ScienceComputer Science (R0)