Elsevier

Surgery

Volume 159, Issue 6, June 2016, Pages 1646-1653
Surgery

Education
Medical telementoring using an augmented reality transparent display

https://doi.org/10.1016/j.surg.2015.12.016Get rights and content

Background

The goal of this study was to design and implement a novel surgical telementoring system called the System for Telementoring with Augmented Reality (STAR) that uses a virtual transparent display to convey precise locations in the operating field to a trainee surgeon. This system was compared with a conventional system based on a telestrator for surgical instruction.

Methods

A telementoring system was developed and evaluated in a study which used a 1 × 2 between-subjects design with telementoring system, that is, STAR or conventional, as the independent variable. The participants in the study were 20 premedical or medical students who had no prior experience with telementoring. Each participant completed a task of port placement and a task of abdominal incision under telementoring using either the STAR or the conventional system. The metrics used to test performance when using the system were placement error, number of focus shifts, and time to task completion.

Results

When compared with the conventional system, participants using STAR completed the 2 tasks with less placement error (45% and 68%) and with fewer focus shifts (86% and 44%), but more slowly (19% for each task).

Conclusions

Using STAR resulted in decreased annotation placement error, fewer focus shifts, but greater times to task completion. STAR placed virtual annotations directly onto the trainee surgeon's field of view of the operating field by conveying location with great accuracy; this technology helped to avoid shifts in focus, decreased depth perception, and enabled fine-tuning execution of the task to match telementored instruction, but led to greater times to task completion.

Section snippets

System description

STAR provides the trainee with an augmented “window” through which the trainee views the operating field while completing the operation. The following discussion describes the design and architecture of STAR and the subsystems (the trainee subsystem and the mentor subsystem) through which the trainee and mentor interact to complete a telementored operation.

Fig 1 shows the trainee subsystem. The tablet captures and displays live video of the operating field using its camera. When the trainee

Methods

Our study was conducted with 17 premedical and 3 medical students at Purdue University. Each participant completed 2 simulated tasks: placement of an incision for port placement and an abdominal incision. In each task, participants acted as a trainee in a telementoring scenario with a simulated mentor providing incremental graphic annotations as the participant completed each stage of the task. Ten students used the STAR surgical telementoring system, and the other 10 students used a

Placement error

For the task of port placement, placement error for participants using STAR (median, 23.73; IQR, 13.28 px) and the conventional system (median, 57.55; IQR, 32.80 px) was different (P < .001) with a median improvement of 33.8 px, representing a 59% decrease. Likewise, there was a 68% improvement (P < .001) decreasing 49.5 px in average the placement error for the abdominal incision task, from the conventional system (μ, 72.6; σx, 16.9 px) to STAR (μ, 23.1; σx, 8.4 px).

Focus shifts

For the port placement

Placement error

In each task, participants using STAR completed the task with significantly less placement error than participants using the conventional system. One likely cause for the decrease in placement error is the direct overlay of annotations onto the operating field. When using STAR, trainees avoided the cognitive load involved in looking at a separate image of the operating field, interpreting it, and mapping the separate instructions onto the actual operating field.

Operations rely on a sequence of

References (14)

There are more references available in the full text version of this article.

Cited by (65)

  • Privacy-protection system using a unidirectional emission waveguide liquid crystal display and an asymmetrical transmission window system

    2022, Optics and Lasers in Engineering
    Citation Excerpt :

    Transparent displays, also known as see-through displays, have been widely developed in recent decades. People can view scenes behind transparent displays, and thus, they are extensively applied to several fields, such as augmented reality, medical uses, and so on [1–6]. A transparent organic light-emitting diode (T-OLED) display is one of the potential candidates to realize transparent displays [7–11].

  • AR/MR Remote Collaboration on Physical Tasks: A Review

    2021, Robotics and Computer-Integrated Manufacturing
View all citing articles on Scopus

Sources of Funding: Supported by the Office of the Assistant Secretary of Defense for Health Affairs under Award No. W81XWH-14-1-0042.

View full text