Abstract
We aim to track the endoscope location inside the surgical scene and provide 3D reconstruction, in real-time, from the sole input of the image sequence captured by the monocular endoscope. This information offers new possibilities for developing surgical navigation and augmented reality applications. The main benefit of this approach is the lack of extra tracking elements which can disturb the surgeon performance in the clinical routine. It is our first contribution to exploit ORBSLAM, one of the best performing monocular SLAM algorithms, to estimate both of the endoscope location, and 3D structure of the surgical scene. However, the reconstructed 3D map poorly describe textureless soft organ surfaces such as liver. It is our second contribution to extend ORBSLAM to be able to reconstruct a semi-dense map of soft organs. Experimental results on in-vivo pigs, shows a robust endoscope tracking even with organs deformations and partial instrument occlusions. It also shows the reconstruction density, and accuracy against ground truth surface obtained from CT.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Stoyanov, D., Scarzanella, M.V., Pratt, P., Yang, G.-Z.: Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6361, pp. 275–282. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15705-9_34
Lin, B., Johnson, A., Qian, X., Sanchez, J., Sun, Y.: Simultaneous tracking, 3D reconstruction and deforming point detection for stereoscope guided surgery. In: Liao, H., Linte, C.A., Masamune, K., Peters, T.M., Zheng, G. (eds.) AE-CAI/MIAR -2013. LNCS, vol. 8090, pp. 35–44. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40843-4_5
Wu, C.-H., Sun, Y.-N., Chang, C.-C.: Three-dimensional modeling from endoscopic video using geometric constraints via feature positioning. IEEE Trans. Biomed. Eng. 54(7), 1199–1211 (2007)
Sun, D., Liu, J., Linte, C.A., Duan, H., Robb, R.A.: Surface reconstruction from tracked endoscopic video using the structure from motion approach. In: Liao, H., Linte, C.A., Masamune, K., Peters, T.M., Zheng, G. (eds.) AE-CAI/MIAR -2013. LNCS, vol. 8090, pp. 127–135. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40843-4_14
Mountney, P., Stoyanov, D., Davison, A., Yang, G.-Z.: Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4190, pp. 347–354. Springer, Heidelberg (2006). doi:10.1007/11866565_43
Davison, A.J.: Real-time simultaneous localisation and mapping with a single camera. In: ICCV, pp. 1403–1410 (2003)
Mountney, P., Yang, G.-Z.: Motion compensated SLAM for image guided surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6362, pp. 496–504. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15745-5_61
Klein, G., Murray, D.W.: Parallel tracking and mapping for smallar workspaces. In: ISMAR, pp. 225–234 (2007)
Grasa, O.G., Civera, J., Montiel, J.M.M.: EKF monocular SLAM 3D modeling, measuring and augmented reality from endoscope image sequences. In: MICCAI 2009, vol. 2 (2009)
Grasa, O.G., Bernal, E., Casado, S., Gil, I., Montiel, J.M.M.: Visual SLAM for handheld monocular endoscope. IEEE Trans. Med. Imaging 33(1), 135–146 (2014)
Mur-Artal, R., Montiel, J.M.M., Tardós, J.D.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)
Strasdat, H., Davison, A.J., Montiel, J.M.M., Konolige, K.: Double window optimization for constant time visual SLAM. In: IEEE International Conference on Computer Vision (ICCV), pp. 2352–2359 (2011)
Galvez-López, D., Tardós, J.D.: Bags of binary words for fast place recognition in image sequences. IEEE Trans. Robot. 28(5), 1188–1197 (2012)
Strasdat, H., Montiel, J.M.M., Davison, A.J.: Scale drift-aware large scale monocular SLAM. In: Robotics: Science and Systems VI (2010)
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: IEEE International Conference on Computer Visition (ICCV), pp. 2564–2571 (2011)
Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)
Youtube. ORBSLAM-based Endoscope Tracking and 3D Reconstruction [Video file], 1 August 2016. https://www.youtube.com/watch?v=UzPjHQX5-9A. Accessed
Bernhardt, S., Nicolau, S.A., Agnus, V., Soler, L., doignon, C., Marescaux, J.: Automatic detection of endoscope in intraoperative CT image: application to AR guidance in laparoscopic surgery. IEEE 11th International Symposium on Biomedical Imaging (ISBI), pp. 563–567 (2014)
Acknowledgments
This work is supported by the Direccíon General de Investigacíon Centífica y Técnica of Spain under Project RT-SLAM DPI2015-67275-P
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Mahmoud, N. et al. (2017). ORBSLAM-Based Endoscope Tracking and 3D Reconstruction. In: Peters, T., et al. Computer-Assisted and Robotic Endoscopy. CARE 2016. Lecture Notes in Computer Science(), vol 10170. Springer, Cham. https://doi.org/10.1007/978-3-319-54057-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-54057-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54056-6
Online ISBN: 978-3-319-54057-3
eBook Packages: Computer ScienceComputer Science (R0)