Skip to main content

Advertisement

SpringerLink
Log in

Projection-based visual guidance for robot-aided RF needle insertion

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

The use of projector-based augmented reality (AR) in surgery may enable surgeons to directly view anatomical models and surgical data from the patient’s surface (skin). It has the advantages of a consistent viewing focus on the patient, an extended field of view and augmented interaction. This paper presents an AR guidance mechanism with a projector-camera system to provide the surgeon with direct visual feedback for supervision of robotic needle insertion in radiofrequency (RF) ablation treatment.

Methods

The registration of target organ models to specific positions on the patient body is performed using a surface-matching algorithm and point-based registration. An algorithm based on the extended Kalman filter and spatial transformation is used to intraoperatively compute the virtual needle’s depth in the patient’s body for AR display.

Results

Experiments of this AR system on a mannequin were conducted to evaluate AR visualization and accuracy of virtual RF needle insertion. The average accuracy of 1.86 mm for virtual needle insertion met the clinical requirement of 2 mm or better. The feasibility of augmented interaction with a surgical robot using the proposed open AR interface with active visual feedback was demonstrated.

Conclusions

The experimental results demonstrate that this guidance system is effective in assisting a surgeon to perform a robot-assisted radiofrequency ablation procedure. The novelty of the work lies in establishing a navigational procedure for percutaneous surgical augmented intervention integrating a projection-based AR guidance and robotic implementation for surgical needle insertion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Yamakado K, Hase S, Matsuoka T, Tanigawa N, Nakatsuka A, Takaki H, Takao M, Inoue Y, Kanazawa S, Inoue Y, Sawada S, Kusunoki M, Takeda K (2007) Radiofrequency ablation for the treatment of unresectable lung metastases in patients with colorectal cancer: a multicenter study in Japan. J Vasc Interv Radiol 18(3):393–398

    Article  PubMed  Google Scholar 

  2. Benoist S, Nordlinger B (2004) Radiofrequency ablation in liver tumours. Ann Oncol 15(Suppl. 4):iv313–iv317

    PubMed  Google Scholar 

  3. Park BK, Kim CK, Choi HY, Lee HM, Jeon SS, Seo S, Han DH (2010) Limitation for performing ultrasound-guided radiofrequency ablation of small renal masses. Eur J Radiol 75(2):248–252

    Article  PubMed  Google Scholar 

  4. Cura J, Zabala R, Iriarte J, Unda M (2010) Treatment of renal tumors by percutaneous ultrasound-guided radiofrequency ablation using a multitined electrode: effectiveness and complications. Eur Urol 57:459–465

    Article  PubMed  Google Scholar 

  5. Gandhi NS, Dupuy DE (2005) Image-guided radiofrequency ablation as a new treatment option for patients with lung cancer. Semin Roentgenol 40(2):171–181

    Article  PubMed  Google Scholar 

  6. Bosch M, Daniel B, Rieke V, Butts-Pauly K, Kermit E, Jeffrey S (2008) MRI-guided radiofrequency ablation of breast cancer: preliminary clinical experience. J Magn Reson Imaging 27(1):204–208

    Article  PubMed  Google Scholar 

  7. Wacker FK, Nour SG, Eisenberg R, Duerk JL, Lewin JS (2004) MRI-guided radiofrequency thermal ablation of normal lung tissue: in vivo study in a rabbit model. AJR Am J Roentgenol 183(3):599–603

    Article  PubMed  Google Scholar 

  8. Wang X, Zhang Q, Han Q, Yang R, Carswell M, Seales B, Sutton E (2010) Endoscopic video texture mapping on pre-built 3-d anatomical objects without camera tracking. IEEE Trans Medical Imaging 29(6):1213–1223

    Article  Google Scholar 

  9. Traub J, Sielhorst T, Heining S-M, Navab N (2008) Advanced display and visualization concepts for image guided surgery. J Disp Technol 4(4):483–490

    Article  Google Scholar 

  10. Kellner F, Bolte B, Bruder G, Rautenberg U, Steinicke F, Lappe M, Koch R (2012) Geometric calibration of head-mounted displays and its effects on distance estimation. IEEE Trans Vis Comput Graph 18(4):589–596

    Article  PubMed  Google Scholar 

  11. Ferrari V, Megali G, Troia E, Pietrabissa A, Mosca F (2009) A 3-d mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans Biomed Eng 56(11):2627–2633

    Article  PubMed  Google Scholar 

  12. Fichtinger G, Deguet A, Masamune K, Balogh E, Fischer GS, Mathieu H, Taylor RH, Zinreich SJ, Fayad LM (2005) Image overlay guidance for needle insertion in CT scanner. IEEE Trans Biomed Eng 52(8):1415–1424

    Article  PubMed  Google Scholar 

  13. Gavaghan KA, Peterhans M, Santos TO, Weber S (2011) A portable image overlay projection device for computer-aided open liver surgery. IEEE Trans Biomed Eng 58(6):1855–1864

    Article  PubMed  Google Scholar 

  14. Krempien R, Hoppe H, Kahrs L, Daeuber S, Schorr O, Eggers G, Bischof M, Munter MW, Debus J, Harms W (2008) Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3d interstitial brachytherapy. Int J Radiat Oncol Biol Phys 70(3):944–952

    Article  PubMed  Google Scholar 

  15. Wen R, Chui C-K, Lim K-B (2010) Intraoperative visual guidance and control interface for augmented reality robotic surgery. In Augmented reality-some emerging application areas. InTech, pp 191–208

  16. Juang R, Majumder A (2007) Photometric self-calibration of a projector-camera system. In: Proceedings of IEEE conference computer vision and pattern recognition (CVPR), California, pp 1–8

  17. Fujii K, Grossberg MD, Nayar SK (2005) A projector-camera system with real-time photometric adaptation for dynamic environments. In: Proceedings of IEEE conference computer vision and pattern recognition (CVPR), Yokosuka, pp 20–25

  18. Griesser A, Gool LV (2006) Automatic interactive calibration of multi-projector-camera systems. In: Proceedings of IEEE conference computer vision and pattern recognition workshop (CVPRW), Zurich, pp 8–8

  19. Taylor RH, Stoianovici D (2003) Medical robotics in computer-integrated surgery. IEEE Trans Robotics Autom 5:765–781

    Article  Google Scholar 

  20. Yang L, Chui C-K, Chang S (2009) Design and development of an augmented reality robotic system for large tumor ablation. Int J Virtual Real 8(1):27–35

    CAS  Google Scholar 

  21. Yang L, Wen R, Qin J, Chui C-K, Lim K-B, Chang SK-Y (2010) A robotic system for overlapping radiofrequency ablation in large tumor treatment. IEEE/ASME Trans Mechatro 15(6):887–897

    Google Scholar 

  22. Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27(3):355–369

    Google Scholar 

  23. Rieder C, Kroeger T, Schumann C, Hahn HK (2011) GPU-based real-time approximation of the ablation zone for radiofrequency ablation. IEEE Trans Vis Comput Graph 17(12):1812–1821

    Google Scholar 

  24. Wen R, Yang L, Chui C-K, Lim K-B, Chang S (2010) Intraoperative visual guidance and control interface for augmented reality robotic surgery. In: Proceedings of IEEE international conference control and automation (ICCA), Singapore, pp 947–952

  25. Bouguet JY (2010) Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/index.html

  26. Frankowski G, Hainich R (2009) DLP-based 3D metrology by structured light or projected fringe technology for life sciences and industrial metrology. In: Proceedings of SPIE, California, vol 7210. pp 72100C1–72100C12

  27. Gao J, Kosaka A, Kak AC (2005) A multi-Kalman filtering approach for video tracking of human-delineated objects in cluttered environments. Comput Vis Image Underst 99(1):1–57

    Article  Google Scholar 

  28. Garstka J, Peters G (2011) View-Dependent 3D Projection Using Depth-Image-based Head Tracking. In: Proceedings of IEEE interenational conference workshop. Projector-Camera Systems (PROCAMS), Colorado, Available: http://www.cs.ubc.ca/labs/imager/PROCAMS2011/program4.php

  29. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge, pp 24–38

  30. Bar-Shaloom Y, Fortmann TE (1988) Tracking and data association. Academic, New York

    Google Scholar 

  31. Hostettler A, Nicolau SA, Rémond Y, Marescaux J, Soler L (2010) A real-time predictive simulation of abdominal viscera positions during quiet free breathing. Prog Biophys Mol Biol 103(2–3):169–184

    Article  PubMed  CAS  Google Scholar 

  32. Tungjitkusolmun S, Staelin ST, Haemmerich D, Tsai J-Z, Cao H, Webster JG, Lee FT Jr, Mahvi DM, Vorperian VR (2002) Three-dimensional finite-element analyses for radio-frequency hepatic tumor ablation. IEEE Trans Biomed Eng 49(1):3–9

    Article  PubMed  Google Scholar 

  33. Wen R, Chng CB, Chui CK, Lim KB, Ong SH, Chang SK (2012) Robot-assisted RF ablation with interactive planning and mixed reality guidance. IEEE/SICE international symposium on system integration (SII), Fukuoka, Japan, pp 31–36

Download references

Acknowledgments

This work was supported in part by the National University of Singapore, under Grants R-265-000-270-112 and R-265-000-270-127. The authors would like to thank Chng Chin Boon for his expertise and support in the surgical robot setup.

Conflict of Interest

None.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, National University of Singapore, Singapore, 117576, Singapore

    Rong Wen, Chee-Kong Chui & Kah-Bin Lim

  2. Department of Electrical and Computer Engineering, National University of Singapore, Singapore, 117576, Singapore

    Sim-Heng Ong

  3. Department of Bioengineering, National University of Singapore, Singapore, 117576, Singapore

    Sim-Heng Ong

  4. Department of Surgery, National University Hospital, Singapore, 119077, Singapore

    Stephen Kin-Yong Chang

Authors
  1. Rong Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chee-Kong Chui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sim-Heng Ong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kah-Bin Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stephen Kin-Yong Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rong Wen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wen, R., Chui, CK., Ong, SH. et al. Projection-based visual guidance for robot-aided RF needle insertion. Int J CARS 8, 1015–1025 (2013). https://doi.org/10.1007/s11548-013-0897-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-013-0897-4

Keywords

Search

Navigation