Skip to main content
SpringerLink
Log in

Deep learning in medical image registration: a survey

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

A Publisher Correction to this article was published on 27 February 2020

This article has been updated

Abstract

The establishment of image correspondence through robust image registration is critical to many clinical tasks such as image fusion, organ atlas creation, and tumor growth monitoring and is a very challenging problem. Since the beginning of the recent deep learning renaissance, the medical imaging research community has developed deep learning-based approaches and achieved the state-of-the-art in many applications, including image registration. The rapid adoption of deep learning for image registration applications over the past few years necessitates a comprehensive summary and outlook, which is the main scope of this survey. This requires placing a focus on the different research areas as well as highlighting challenges that practitioners face. This survey, therefore, outlines the evolution of deep learning-based medical image registration in the context of both research challenges and relevant innovations in the past few years. Further, this survey highlights future research directions to show how this field may be possibly moved forward to the next level.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Change history

  • 27 February 2020

    The articles listed below were published in Issue January 2020, Issue 1, instead of Issue February 2020, Issues 1–2.

References

  1. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al.: Tensorflow: a system for large-scale machine learning. In: OSDI, vol. 16, pp. 265–283 (2016)

  2. Ali, S., Rittscher, J.: Conv2Warp: an unsupervised deformable image registration with continuous convolution and warping. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) Machine Learning in Medical Imaging, pp. 489–497. Springer International Publishing, Cham (2019)

    Google Scholar 

  3. Alom, M.Z., Taha, T.M., Yakopcic, C., Westberg, S., Hasan, M., Van Esesn, B.C., Awwal, A.A.S., Asari, V.K.: The history began from alexnet: a comprehensive survey on deep learning approaches. Preprint (2018). arXiv:1803.01164

  4. Ambinder, E.P.: A history of the shift toward full computerization of medicine. J. Oncol. Pract. 1(2), 54–56 (2005)

    Google Scholar 

  5. Arganda-Carreras, I., Sorzano, C.O., Marabini, R., Carazo, J.M., Ortiz-de Solorzano, C., Kybic, J.: Consistent and elastic registration of histological sections using vector-spline regularization. In: International Workshop on Computer Vision Approaches to Medical Image Analysis, pp. 85–95. Springer, Berlin (2006)

  6. Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C.: Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med. Image Anal. 12(1), 26–41 (2008)

    Google Scholar 

  7. Avants, B.B., Tustison, N.J., Song, G., Cook, P.A., Klein, A., Gee, J.C.: A reproducible evaluation of ants similarity metric performance in brain image registration. Neuroimage 54(3), 2033–2044 (2011)

    Google Scholar 

  8. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9252–9260 (2018)

  9. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: Voxelmorph: A learning framework for deformable medical image registration. Preprint (2018). arXiv:1809.05231

  10. Blendowski, M., Heinrich, M.P.: Combining MRF-based deformable registration and deep binary 3D-CNN descriptors for large lung motion estimation in COPD patients. Int. J. Comput. Assisted Radiol. Surg. 14, 1–10 (2018)

    Google Scholar 

  11. Cao, T., Singh, N., Jojic, V., Niethammer, M.: Semi-coupled dictionary learning for deformation prediction. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 691–694. IEEE, New York (2015)

  12. Cao, X., Yang, J., Wang, L., Xue, Z., Wang, Q., Shen, D.: Deep learning based inter-modality image registration supervised by intra-modality similarity. Preprint (2018). arXiv:1804.10735

  13. Cao, X., Yang, J., Zhang, J., Nie, D., Kim, M., Wang, Q., Shen, D.: Deformable image registration based on similarity-steered CNN regression. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 300–308. Springer, New York (2017)

  14. Chan, T.-H., Jia, K., Gao, S., Lu, J., Zeng, Z., Ma, Y.: Pcanet: A simple deep learning baseline for image classification? IEEE Trans. Image Process. 24(12), 5017–5032 (2015)

    MathSciNet  MATH  Google Scholar 

  15. Chee, E., Wu, J.: Airnet: self-supervised affine registration for 3D medical images using neural networks. Preprint (2018). arXiv:1810.02583

  16. Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., Zhang, Z.: Mxnet: a flexible and efficient machine learning library for heterogeneous distributed systems. Preprint (2015). arXiv:1512.01274

  17. Cheng, X., Zhang, L., Zheng, Y.: Deep similarity learning for multimodal medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2016)

  18. Cheng, X., Zhang, L., Zheng, Y.: Deep similarity learning for multimodal medical images. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 6(3), 248–252 (2018)

    Google Scholar 

  19. Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., Choo, J.: Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797 (2018)

  20. Chollet, F., et al.: Keras (2015). https://keras.io

  21. Dalca, A.V., Balakrishnan, G., Guttag, J., Sabuncu, M.R.: Unsupervised learning for fast probabilistic diffeomorphic registration. Preprint (2018). arXiv:1805.04605

  22. De Silva, T., Uneri, A., Ketcha, M., Reaungamornrat, S., Kleinszig, G., Vogt, S., Aygun, N., Lo, S., Wolinsky, J., Siewerdsen, J.: 3D–2D image registration for target localization in spine surgery: investigation of similarity metrics providing robustness to content mismatch. Phys. Med. Biol. 61(8), 3009 (2016)

    Google Scholar 

  23. de Vos, B.D., Berendsen, F.F., Viergever, M.A., Sokooti, H., Staring, M., Išgum, I.: A deep learning framework for unsupervised affine and deformable image registration. Med. Image Anal. 52, 128–143 (2018)

    Google Scholar 

  24. de Vos, B.D., Berendsen, F.F., Viergever, M.A., Staring, M., Išgum, I.: End-to-end unsupervised deformable image registration with a convolutional neural network. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 204–212. Springer, Berlin (2017)

  25. Doersch, C.: Tutorial on variational autoencoders. Preprint (2016). arXiv:1606.05908

  26. Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Van Der Smagt, P., Cremers, D., Brox, T.: Flownet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2758–2766 (2015)

  27. Ehrhardt, J., Schmidt-Richberg, A., Werner, R., Handels, H.: Variational registration. In: Bildverarbeitung für die Medizin 2015, pp. 209–214. Springer, Berlin (2015)

  28. Eppenhof, K.A., Pluim, J.P.: Pulmonary CT registration through supervised learning with convolutional neural networks. IEEE Trans. Med. Imaging 38, 1097–1105 (2018)

    Google Scholar 

  29. Eppenhof, K.A.J., Pluim, J.P.: Error estimation of deformable image registration of pulmonary CT scans using convolutional neural networks. J. Med. Imaging 5(2), 024003 (2018b)

    Google Scholar 

  30. Fan, J., Cao, X., Xue, Z., Yap, P.-T., Shen, D.: Adversarial similarity network for evaluating image alignment in deep learning based registration. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 739–746. Springer, Berlin (2018)

  31. Fan, J., Cao, X., Yap, P.-T., Shen, D.: Birnet: brain image registration using dual-supervised fully convolutional networks. Preprint (2018). arXiv:1802.04692

  32. Ferrante, E., Oktay, O., Glocker, B., Milone, D.H.: On the adaptability of unsupervised CNN-based deformable image registration to unseen image domains. In: International Workshop on Machine Learning in Medical Imaging, pp. 294–302. Springer, Berlin (2018)

  33. Ghosal, S., Ray, N.: Deep deformable registration: enhancing accuracy by fully convolutional neural net. Pattern Recogn. Lett. 94, 81–86 (2017)

    Google Scholar 

  34. Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep learning, vol. 1. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  35. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 2672–2680. Curran Associates, Inc. (2014). http://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf

  36. Haskins, G., Kruecker, J., Kruger, U., Xu, S., Pinto, P.A., Wood, B.J., Yan, P.: Learning deep similarity metric for 3D MR–TRUS image registration. Int. J. Comput. Assist. Radiol. Surg. 14, 417–425 (2019)

    Google Scholar 

  37. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  38. Heinrich, M.P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F.V., Brady, M., Schnabel, J.A.: Mind: modality independent neighbourhood descriptor for multi-modal deformable registration. Med. Image Anal. 16(7), 1423–1435 (2012)

    Google Scholar 

  39. Heinrich, M.P., Jenkinson, M., Papież, B. W., Brady, M., Schnabel, J.A.: Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 187–194. Springer, Berlin (2013)

  40. Hering, A., Kuckertz, S., Heldmann, S., Heinrich, M.: Enhancing label-driven deep deformable image registration with local distance metrics for state-of-the-art cardiac motion tracking. Preprint (2018). arXiv:1812.01859

  41. Hill, D.L., Batchelor, P.G., Holden, M., Hawkes, D.J.: Medical image registration. Phys. Med. Biol. 46(3), R1–R45 (2001)

    Google Scholar 

  42. Hu, Y., Gibson, E., Ghavami, N., Bonmati, E., Moore, C.M., Emberton, M., Vercauteren, T., Noble, J.A., Barratt, D.C.: Adversarial deformation regularization for training image registration neural networks. Preprint (2018). arXiv:1805.10665

  43. Hu, Y., Modat, M., Gibson, E., Ghavami, N., Bonmati, E., Moore, C.M., Emberton, M., Noble, J.A., Barratt, D.C., Vercauteren, T.: Label-driven weakly-supervised learning for multimodal deformarle image registration. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 1070–1074. IEEE, New York (2018)

  44. Hu, Y., Modat, M., Gibson, E., Li, W., Ghavami, N., Bonmati, E., Wang, G., Bandula, S., Moore, C.M., Emberton, M., et al.: Weakly-supervised convolutional neural networks for multimodal image registration. Med. Image Anal. 49, 1–13 (2018c)

    Google Scholar 

  45. Ikeda, K., Ino, F., Hagihara, K.: Efficient acceleration of mutual information computation for nonrigid registration using CUDA. IEEE J. Biomed. Health Inf. 18(3), 956–968 (2014)

    Google Scholar 

  46. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. arXiv Preprint (2017)

  47. Ito, M., Ino, F.: An automated method for generating training sets for deep learning based image registration. In: The 11th International Joint Conference on Biomedical Engineering Systems and Technologies—Volume 2: BIOIMAGING, pp. 140–147. INSTICC SciTePress (2018)

  48. Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K: Spatial transformer networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, pp. 2017–2025. Curran Associates, Inc. (2015). http://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf

  49. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM, New York (2014)

  50. Jiang, P., Shackleford, J.A.: CNN driven sparse multi-level b-spline image registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9281–9289 (2018)

  51. Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)

    Google Scholar 

  52. Kazeminia, S., Baur, C., Kuijper, A., van Ginneken, B., Navab, N., Albarqouni, S., Mukhopadhyay, A.: Gans for medical image analysis. Preprint (2018). arXiv:1809.06222

  53. Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.: Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging 29(1), 196–205 (2010)

    Google Scholar 

  54. Kori, A., Krishnamurthi, G.: Zero shot learning for multi-modal real time image registration (2019). arXiv:1908.06213

  55. Krebs, J., Mansi, T., Delingette, H., Zhang, L., Ghesu, F.C., Miao, S., Maier, A.K., Ayache, N., Liao, R., Kamen, A.: Robust non-rigid registration through agent-based action learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 344–352. Springer, Berlin (2017)

  56. Krebs, J., Mansi, T., Mailhé, B., Ayache, N., Delingette, H.: Learning structured deformations using diffeomorphic registration. Preprint (2018) arXiv:1804.07172

  57. Krebs, J., Mansi, T., Mailhé, B., Ayache, N., Delingette, H.: Unsupervised probabilistic deformation modeling for robust diffeomorphic registration. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 101–109. Springer, Berlin (2018)

  58. Kuang, D., Schmah, T.: Faim—a convnet method for unsupervised 3D medical image registration. Preprint (2018). arXiv:1811.09243

  59. Lee, J.-G., Jun, S., Cho, Y.-W., Lee, H., Kim, G.B., Seo, J.B., Kim, N.: Deep learning in medical imaging: general overview. Korean J. Radiol. 18(4), 570–584 (2017)

    Google Scholar 

  60. Li, H., Fan, Y.: Non-rigid image registration using fully convolutional networks with deep self-supervision. Preprint (2017). arXiv:1709.00799

  61. Li, H., Fan, Y.: Non-rigid image registration using self-supervised fully convolutional networks without training data. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 1075–1078. IEEE, New York (2018)

  62. Liao, R., Miao, S., de Tournemire, P., Grbic, S., Kamen, A., Mansi, T., Comaniciu, D.: An artificial agent for robust image registration. In: AAAI, pp. 4168–4175 (2017)

  63. Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A., Van Ginneken, B., Sánchez, C.I.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)

    Google Scholar 

  64. Liu, C., Yuen, J., Torralba, A.: Sift flow: dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)

    Google Scholar 

  65. Liu, J., Pan, Y., Li, M., Chen, Z., Tang, L., Lu, C., Wang, J.: Applications of deep learning to mri images: a survey. Big Data Min. Anal. 1(1), 1–18 (2018)

    Google Scholar 

  66. Liu, M.-Y., Breuel, T., Kautz, J.: Unsupervised image-to-image translation networks. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, pp. 700–708. Curran Associates, Inc. (2017) http://papers.nips.cc/paper/6672-unsupervised-image-to-image-translation-networks.pdf

  67. Liu, Q., Leung, H.: Tensor-based descriptor for image registration via unsupervised network. In: 2017 20th International Conference on Information Fusion (Fusion), pp. 1–7. IEEE, New York (2017)

  68. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

  69. Lorenzi, M., Ayache, N., Frisoni, G.B., Pennec, X., (ADNI, A. D. N. I., et al.): LCC-demons: a robust and accurate symmetric diffeomorphic registration algorithm. NeuroImage 81, 470–483 (2013)

  70. Lv, J., Yang, M., Zhang, J., Wang, X.: Respiratory motion correction for free-breathing 3D abdominal MRI using CNN-based image registration: a feasibility study. Br. J. Radiol. 91, 20170788 (2018)

    Google Scholar 

  71. Ma, K., Wang, J., Singh, V., Tamersoy, B., Chang, Y.-J., Wimmer, A., Chen, T.: Multimodal image registration with deep context reinforcement learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 240–248. Springer, Berlin (2017)

  72. Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16(2), 187–198 (1997)

    Google Scholar 

  73. Mahapatra, D.: Elastic registration of medical images with gans. Preprint (2018). arXiv:1805.02369

  74. Mahapatra, D., Ge, Z., Sedai, S., Chakravorty, R.: Joint registration and segmentation of X-ray images using generative adversarial networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 73–80. Springer, Berlin (2018)

  75. Matthew, J., Hajnal, J.V., Rueckert, D., Schnabel, J.A.: LSTM spatial co-transformer networks for registration of 3D fetal US and MR brain images. In: Data Driven Treatment Response Assessment and Preterm, Perinatal, and Paediatric Image Analysis, pp. 149–159. Springer, Berlin (2018)

  76. Miao, S., Piat, S., Fischer, P., Tuysuzoglu, A., Mewes, P., Mansi, T., Liao, R.: Dilated FCN for multi-agent 2D/3D medical image registration. Preprint (2017). arXiv:1712.01651

  77. Miao, S., Wang, Z.J., Liao, R.: A cnn regression approach for real-time 2D/3D registration. IEEE Trans. Med. Imaging 35(5), 1352–1363 (2016a)

    Google Scholar 

  78. Miao, S., Wang, Z.J., Zheng, Y., Liao, R.: Real-time 2D/3D registration via CNN regression. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 1430–1434. IEEE, New York (2016)

  79. Myronenko, A., Song, X.: Intensity-based image registration by minimizing residual complexity. IEEE Trans. Med. Imaging 29(11), 1882–1891 (2010)

    Google Scholar 

  80. Nazib, A., Fookes, C., Perrin, D.: A comparative analysis of registration tools: traditional vs. deep learning approach on high resolution tissue cleared data. Preprint (2018). arXiv:1810.08315

  81. Neylon, J., Min, Y., Low, D.A., Santhanam, A.: A neural network approach for fast, automated quantification of dir performance. Med. Phys. 44(8), 4126–4138 (2017)

    Google Scholar 

  82. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch. In: NIPS-W (2017)

  83. Punithakumar, K., Boulanger, P., Noga, M.: A gpu-accelerated deformable image registration algorithm with applications to right ventricular segmentation. IEEE Access 5, 20374–20382 (2017)

    Google Scholar 

  84. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, pp. 91–99. Curran Associates, Inc. (2015) http://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-object-detection-with-region-proposal-networks.pdf

  85. Rivenson, Y., Zhang, Y., Günaydın, H., Teng, D., Ozcan, A.: Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl. 7(2), 17141 (2018)

    Google Scholar 

  86. Rohé, M.-M., Datar, M., Heimann, T., Sermesant, M., Pennec, X.: SVF-Net: learning deformable image registration using shape matching. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 266–274. Springer, Berlin (2017)

  87. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer, Berlin (2015)

  88. Rühaak, J., Heldmann, S., Kipshagen, T., Fischer, B.: Highly accurate fast lung CT registration. In: Medical Imaging 2013: Image Processing, Volume 8669, pp. 86690Y. International Society for Optics and Photonics (2013)

  89. Saalfeld, S., Fetter, R., Cardona, A., Tomancak, P.: Elastic volume reconstruction from series of ultra-thin microscopy sections. Nat. Methods 9(7), 717 (2012)

    Google Scholar 

  90. Salehi, S.S.M., Khan, S., Erdogmus, D., Gholipour, A.: Real-time deep registration with geodesic loss. Preprint (2018). arXiv:1803.05982

  91. Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Google Scholar 

  92. Sedghi, A., Luo, J., Mehrtash, A., Pieper, S., Tempany, C.M., Kapur, T., Mousavi, P., Wells III, W.M.: Semi-supervised deep metrics for image registration. Preprint (2018). arXiv:1804.01565

  93. Sheikhjafari, A., Noga, M., Punithakumar, K., Ray, N.: Unsupervised deformable image registration with fully connected generative neural network. In: International Conference on Medical Imaging with Deep Learning (2018)

  94. Shen, D.: Image registration by local histogram matching. Pattern Recogn. 40(4), 1161–1172 (2007)

    MathSciNet  MATH  Google Scholar 

  95. Shu, C., Chen, X., Xie, Q., Han, H.: An unsupervised network for fast microscopic image registration. In: Medical Imaging 2018: Digital Pathology, vol. 10581, p. 105811D. International Society for Optics and Photonics (2018)

  96. Simonovsky, M., Gutiérrez-Becker, B., Mateus, D., Navab, N., Komodakis, N.: A deep metric for multimodal registration. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 10–18. Springer, Berlin (2016)

  97. Sloan, J.M., Goatman, K.A., Siebert, J.P.: Learning rigid image registration-utilizing convolutional neural networks for medical image registration. In: 11th International Joint Conference on Biomedical Engineering Systems and Technologies, pp. 89–99. SCITEPRESS-Science and Technology Publications (2018)

  98. Smith, J.T., Yao, R., Sinsuebphon, N., Rudkouskaya, A., Un, N., Mazurkiewicz, J., Barroso, M., Yan, P., Intes, X.: Fast fit-free analysis of fluorescence lifetime imaging via deep learning. Proc. Natl. Acad. Sci. 116(48), 24019–24030 (2019)

    Google Scholar 

  99. Sokooti, H., de Vos, B., Berendsen, F., Lelieveldt, B.P., Išgum, I., Staring, M.: Nonrigid image registration using multi-scale 3D convolutional neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 232–239. Springer, Berlin (2017)

  100. Stergios, C., Mihir, S., Maria, V., Guillaume, C., Marie-Pierre, R., Stavroula, M., Nikos, P.: Linear and deformable image registration with 3D convolutional neural networks. In: Image Analysis for Moving Organ, Breast, and Thoracic Images, pp. 13–22. Springer, Berlin (2018)

  101. Sun, L., Zhang, S.: Deformable MRI-ultrasound registration using 3D convolutional neural network. In: Simulation, Image Processing, and Ultrasound Systems for Assisted Diagnosis and Navigation, pp. 152–158. Springer, Berlin (2018)

  102. Sun, Y., Moelker, A., Niessen, W.J., van Walsum, T.: Towards robust CT-ultrasound registration using deep learning methods. In: Understanding and Interpreting Machine Learning in Medical Image Computing Applications, pp. 43–51. Springer, Berlin (2018)

  103. Uzunova, H., Wilms, M., Handels, H., Ehrhardt, J.: Training CNNS for image registration from few samples with model-based data augmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 223–231. Springer, Berlin (2017)

  104. Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Diffeomorphic demons: efficient non-parametric image registration. NeuroImage 45(1), S61–S72 (2009)

    Google Scholar 

  105. Vialard, F.-X., Risser, L., Rueckert, D., Cotter, C.J.: Diffeomorphic 3D image registration via geodesic shooting using an efficient adjoint calculation. Int. J. Comput. Vis. 97(2), 229–241 (2012)

    MathSciNet  MATH  Google Scholar 

  106. Viola, P., Wells III, W.M.: Alignment by maximization of mutual information. Int. J. Comput. Vis. 24(2), 137–154 (1997)

    Google Scholar 

  107. Wang, G.: A perspective on deep imaging. Preprint (2016). arXiv:1609.04375

  108. Wang, Z., Schaul, T., Hessel, M., Van Hasselt, H., Lanctot, M., De Freitas, N.: Dueling network architectures for deep reinforcement learning. Preprint (2015). arXiv:1511.06581

  109. Wu, G., Kim, M., Wang, Q., Gao, Y., Liao, S., Shen, D.: Unsupervised deep feature learning for deformable registration of MR brain images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 649–656. Springer, Berlin (2013)

  110. Wu, G., Kim, M., Wang, Q., Munsell, B.C., Shen, D.: Scalable high-performance image registration framework by unsupervised deep feature representations learning. IEEE Trans. Biomed. Eng. 63(7), 1505–1516 (2016)

    Google Scholar 

  111. Yan, P., Xu, S., Rastinehad, A.R., Wood, B.J.: Adversarial image registration with application for MR and TRUS image fusion. Preprint (2018). arXiv:1804.11024

  112. Yang, Q., Yan, P., Zhang, Y., Yu, H., Shi, Y., Mou, X., Kalra, M.K., Zhang, Y., Sun, L., Wang, G.: Low dose CT image denoising using a generative adversarial network with wasserstein distance and perceptual loss. IEEE Trans. Med. Imaging 37, 1348–1357 (2018)

    Google Scholar 

  113. Yang, X.: Uncertainty quantification, image synthesis and deformation prediction for image registration. Ph.D. Thesis, The University of North Carolina at Chapel Hill (2017)

  114. Yang, X., Kwitt, R., Niethammer, M.: Fast predictive image registration. In: Carneiro, G., Mateus, D., Peter, L., Bradley, A., Tavares, J.M.R.S., Belagiannis, V., Papa, J.P., Nascimento, J.C., Loog, M., Lu, Z., Cardoso, J.S., Cornebise, J. (eds.) Deep Learning and Data Labeling for Medical Applications, pp. 48–57. Springer International Publishing, Cham (2016)

    Google Scholar 

  115. Yao, R., Ochoa, M., Intes, X., Yan, P.: Deep compressive macroscopic fluorescence lifetime imaging. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 908–911. IEEE, New York (2018)

  116. Yi, Z., Zhang, H., Tan, P., Gong, M.: Dualgan: unsupervised dual learning for image-to-image translation. arXiv Preprint (2017)

  117. Yoo, I., Hildebrand, D.G., Tobin, W.F., Lee, W.-C.A., Jeong, W.-K.: ssEMnet: serial-section electron microscopy image registration using a spatial transformer network with learned features. In: Cardoso, M.J., Arbel, T., Carneiro, G., Syeda-Mahmood, T., Tavares, J.M.R.S., Moradi, M., Bradley, A., Greenspan, H., Papa, J.P., Madabhushi, A., Nascimento, J.C., Cardoso, J.S., Belagiannis, V., Lu, Z. (eds.) Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 249–257. Springer International Publishing, Cham (2017)

    Google Scholar 

  118. Zhang, J.: Inverse-consistent deep networks for unsupervised deformable image registration. arXiv Preprint (2018). arXiv:1809.03443

  119. Zheng, J., Miao, S., Wang, Z.J., Liao, R.: Pairwise domain adaptation module for CNN-based 2-D/3-D registration. J. Med. Imaging 5(2), 021204 (2018)

    Google Scholar 

  120. Zhu, B., Liu, J.Z., Cauley, S.F., Rosen, B.R., Rosen, M.S.: Image reconstruction by domain-transform manifold learning. Nature 555(7697), 487 (2018)

    Google Scholar 

  121. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv Preprint (2017)

  122. Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, 12180, USA

    Grant Haskins, Uwe Kruger & Pingkun Yan

Authors
  1. Grant Haskins
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Uwe Kruger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pingkun Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pingkun Yan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was partially supported by NIH/NIBIB under awards R21EB028001 and R01EB027898, and NIH/NCI under a Bench-to-Bedside award.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Haskins, G., Kruger, U. & Yan, P. Deep learning in medical image registration: a survey. Machine Vision and Applications 31, 8 (2020). https://doi.org/10.1007/s00138-020-01060-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01060-x

Keywords

Search

Navigation