Elsevier

Medical Image Analysis

Volume 51, January 2019, Pages 61-88
Medical Image Analysis

Segmentation and classification in MRI and US fetal imaging: Recent trends and future prospects

https://doi.org/10.1016/j.media.2018.10.003Get rights and content

Highlights

  • State-of-the-art segmentation and classification methods for fetal brain, lungs, liver, heart, placenta and the whole fetus in magnetic resonance imaging and (2D / 3D) ultrasound are reviewed for the first time.

  • A total of 123 relevant works have been covered and discussed.

  • Potential applications of the surveyed methods into clinical settings are inspected.

  • Feasible and non-previously tackled computer-assisted fetal surgical planning areas of research are outlined.

Abstract

Fetal imaging is a burgeoning topic. New advancements in both magnetic resonance imaging and (3D) ultrasound currently allow doctors to diagnose fetal structural abnormalities such as those involved in twin-to-twin transfusion syndrome, gestational diabetes mellitus, pulmonary sequestration and hypoplasia, congenital heart disease, diaphragmatic hernia, ventriculomegaly, etc. Considering the continued breakthroughs in utero image analysis and (3D) reconstruction models, it is now possible to gain more insight into the ongoing development of the fetus. Best prenatal diagnosis performances rely on the conscious preparation of the clinicians in terms of fetal anatomy knowledge. Therefore, fetal imaging will likely span and increase its prevalence in the forthcoming years. This review covers state-of-the-art segmentation and classification methodologies for the whole fetus and, more specifically, the fetal brain, lungs, liver, heart and placenta in magnetic resonance imaging and (3D) ultrasound for the first time. Potential applications of the aforementioned methods into clinical settings are also inspected. Finally, improvements in existing approaches as well as most promising avenues to new areas of research are briefly outlined.

Introduction

Prenatal imaging technology for fetal diagnosis has rapidly evolved (Yan (2017)). Two-dimensional ultrasound (2D US) is the primary screening modality for pregnancy evaluation because of its relative low cost, real-time imaging, lack of harmful effects to both fetus and mother, and high resolution. Three- and four -dimensional ultrasound (3D / 4D US) with additional sonography modalities such as color / power Doppler are increasingly available and have been successfully employed to detect fetal structural abnormalities (Roy-Lacroix et al. (2017a)).

Although US is still the standard tool for fetal imaging, it provides a small field-of-view, limited soft-tissue acoustic contrast, beam attenuation by adipose tissue, and poor image quality. Furthermore, important limitations arise in the context of an intrauterine environment, such as operator variability, fetal position, effects due to gestational age (poor visualization, skull ossification), and fetal tissue definition (Roy-Lacroix et al. (2017a); Reddy et al. (2008)). In cases where abnormalities cannot be adequately assessed with US, fetal magnetic resonance imaging (MRI) can be a good alternative. Early studies using MRI in the evaluation of fetal morphology were hindered due to image degradation by fetal motion on MRI standard sequences and the relatively long acquisition time. Nevertheless, with ultrafast MRI techniques such as single-shot fast-spin echo and half-Fourier acquisition turbo spin echo (HASTE), fetal motion artifacts are minimized. Dramatic reductions in image acquisition time and improved acquisition protocols for multi-planar imaging have enabled MRI to be used in fetal evaluation (Story and Rutherford (2015)). High-quality fetal images are now routinely achieved without maternal or fetal sedation. However, limitations of fetal MRI include the need of equipment availability and radiology expertise, higher cost, and longer time to perform a complete examination.

The increasing advancement and use of fetal imaging for computer-assisted prenatal organ assessment provides essential clinical feedback to design improved ad hoc surgical frameworks. A better understanding of fetal pathophysiology and early diagnosis has paved the way for the development of in utero therapies rather than postnatally (Maselli and Badillo (2016)). In this way, (3D) US and MRI provide crucial anatomic information that can be helpful in planning antenatal care and surgical procedures, as well as predicting fetal development and outcome. Fetal surgery has emerged as a multidisciplinary field capable of improving fetus outcomes for a wide range of interventions, such as laser treatment for twin-to-twin transfusion syndrome (TTTS) and open fetal surgery for spina bifida (Pratt et al. (2015); Slaghekke et al. (2014); Adzick et al. (2011)). Fetal interventions are expected to increase in breadth and prevalence as novel surgical or computational procedures demonstrate to be clinically valid (Pratt et al. (2015); Mathis et al. (2015)). As in any surgery, best results are achieved when doctors are prepared pre-operatively with a detailed understanding of the baby’s anatomy thanks to fetus-specific 3D visualizations obtained from medical images.

The pre-operative phase in fetal surgery involves the following steps (see Fig. 1) (Pratt et al. (2015)). First, images of the target structures are taken. Segmentation and classification methods are then applied to distinguish between the regions of interest and the background organs and tissues that can be discarded. The fusion of several image modalities allows completing the target structures from the different partial image views. Next, volume rendering algorithms are applied in the reconstruction stage to yield a 3D model in which the structural relationships are disclosed. In the simulation phase, surgeons benefit from this model to visualize the anatomical structures from different perspectives, and to understand their intricate relationships.

Moreover, segmentation and classification techniques can also provide imaging biomarkers for the prediction of fetal development and outcome. The shape, volume, morphometry and texture of fetal organs can be characterized during pregnancy to assess fetal health and predict complicated pregnancies (Dahdouh et al. (2018)). Early biomarkers of fetal organs disease that may impair fetal growth and well-being open up brand-new opportunities to intervene and protect vulnerable fetuses.

This review attempts to fill existing gaps in the literature. We sought to uncover both the literature in fetal imaging and to illustrate the usefulness of novel approaches to provide additional and improved diagnostic information. To the best of our knowledge, the state-of-the-art of segmentation and classification methods for fetal organs such as placenta, lungs, liver and heart, or even the whole fetus, in (3D) US and MRI have not been reviewed so far. Considering the relevance and the steady progress made in the field, we deemed it necessary to survey them in detail. For completeness, fetal brain is also covered. While several reviews of fetal brain MRI exist (see Section 3.2.1), considerably less literature can be found on fetal brain US. To our knowledge, this is the first review that addresses it.

The contents are structured as follows. Section 2 describes the search criteria for this work. Section 3 reviews the main works on fetal MRI and 2D/3D US imaging, categorized by fetal organs: the whole fetus, brain, placenta, lungs, liver, thorax, and heart. Section 4 provides examples of fetal surgery applications for which the methods reviewed in Section 3 can be employed at the pre-operative phase. Section 5 briefly overviews the validation performed on the state-of-the-art reviewed. Lastly, Section 6 discusses the most useful contributions to the field by highlighting some cutting-edge ideas.

Section snippets

Method

A systematic review on the use of segmentation and classification methods in fetal imaging has been performed. To scrutinize the literature, medical subject headings (MeSH)1 and keywords related to the scope of this review (segmentation, classification, computer vision, machine learning, fetal MRI and (3D) US imaging) were combined with medical terms associated to prenatal diagnosis. PUBMED2, EMBASE3

Categorization by fetal organ

The main challenge to automatically localize and classify the fetal organs such as brain, placenta, heart, lungs and liver, or even the whole fetus is the unpredictable fetus position and arbitrary orientation. This section gathers 123 articles that show quantitative / qualitative and robust validation of their methods, these have been assessed clinically by doctors and have undergone a peer-review process.

Examples of fetal surgery applications

There have been attempts to implement segmentation and classification algorithms for the aforementioned organs in real surgeries at pre-operative stage. In the following, examples of fetal diseases for which these scientific approaches have been applied (or could be employed in the near future) at the pre-operative level are highlighted.

General performance assessment

Table 2, Table 3, Table 4, Table 5 summarize the validation done on the 123 articles discussed in Section 3. They involve all fetal image segmentation and classification methods in terms of the fetal organs here presented, namely brain, placenta, lungs and liver (thorax), heart, and the whole fetus.

From Table 2, Table 3, Table 4, Table 5, some general comments on the state-of-art reviewed can be made.

  • In the field of medical imaging, some simulation, phantom, and clinical studies are

Discussion and future prospects

Fetal MRI has gained wide acceptance in clinical practice as a valuable tool to complement prior prenatal (3D) US findings. Automatic segmentation and classification algorithms can nowadays facilitate the diagnosis of in utero abnormalities of fetal organs. In addition, thanks to the major breakthroughs in medical equipments and the recent advancements in the image acquisition protocols, more scalable and high-quality population reference studies can be performed. Crucial information is

Conclusion

This work provides the most recent landmarks on segmentation and classification methodologies in the context of fetal imaging. A total of 123 relevant studies in which state-of-the-art methods are used to analyze fetal structural anatomies to identify possible abnormalities or complications have been surveyed. Although fetal MRI and (3D) US are still improving, with higher resolution and enhanced signal-to-noise ratio, new processing approaches are emerging. Given the constraints that

Acknowledgements

This work was supported by CELLEX foundation and the Google Women Techmakers scholarship awarded to Jordina Torrents-Barrena. Also this work was funded by the Spanish Ministry of Economy and Competitiveness under the María de Maeztu Units of Excellence Programme [MDM-2015-0502].

References (164)

  • A. Makropoulos et al.

    A review on automatic fetal and neonatal brain MRI segmentation

    Neuroimage

    (2018)
  • M.A. Maraci et al.

    A framework for analysis of linear ultrasound videos to detect fetal presentation and heartbeat

    Med. Image Anal.

    (2017)
  • R.J. Moore et al.

    In utero perfusing fraction maps in normal and growth restricted pregnancy measured using IVIM echo-planar MRI

    Placenta

    (2000)
  • D. Ni et al.

    Standard plane localization in ultrasound by radial component model and selective search

    Ultrasound Med. Biol.

    (2014)
  • S. Nirmala et al.

    Markov random field segmentation based sonographic identification of prenatal ventricular septal defect

    Procedia Computer Science - 7th International Conference on Communication, Computing and Virtualization

    (2016)
  • N.S. Adzick et al.

    A randomized trial of prenatal versus postnatal repair of myelomeningocele

    Eng. J. Med.

    (2011)
  • A. Alansary et al.

    Fast fully automatic segmentation of the human placenta from motion corrupted MRI

    Medical Image Computing and Computer-Assisted Intervention 2016 (MICCAI 2016)

    (2016)
  • J. Anquez et al.

    Automatic segmentation of head structures on fetal MRI

    IEEE International Symposium on Biomedical Imaging: From Nano to Macro

    (2009)
  • J. Anquez et al.

    Automatic segmentation of antenatal 3d ultrasound images

    IEEE Trans. Biomed. Eng.

    (2013)
  • J. Anquez et al.

    Segmentation of the fetal envelope on ante-natal MRI

    IEEE International Symposium on Biomedical Imaging: From Nano to Macro

    (2010)
  • A.A. Baschat

    Fetal responses to placental insufficiency: an update

    BJOG:

    (2004)
  • C.F. Baumgartner et al.

    Sononet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound

    IEEE Trans. Med. Imaging

    (2017)
  • C.F. Baumgartner et al.

    Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks

    International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)

    (2016)
  • B.G. Becker et al.

    Automatic segmentation of the cerebellum of fetuses on 3D ultrasound images, using a 3D point distribution model

    International Conference of the IEEE Engineering in Medicine and Biology Society

    (2010)
  • A. Benavides-Serralde et al.

    Three-dimensional sonographic calculation of the volume of intracranial structures in growth-restricted and appropriate-for-gestational age fetuses

    Ultrasound Obstetrics Gynecol.

    (2009)
  • O.M. Benkarim et al.

    Toward the automatic quantification of in utero brain development in 3d structural MRI: a review

    Hum. Brain Mapp.

    (2017)
  • K.N. Bhanu-Prakash et al.

    Fetal lung maturity analysis using ultrasound image features

    IEEE Trans. Inf. Technol. Biomed.

    (2002)
  • S. Bhattacharya et al.

    Systems biology approaches to identify developmental bases for lung diseases

    Pediatr. Res.

    (2013)
  • L. Bibin et al.

    Hybrid 3d pregnant woman and fetus modeling from medical imaging for dosimetry studies

    Int. J. Comput. Assist. Radiol. Surg.

    (2009)
  • E. Bonet-Carne et al.

    Quantitative ultrasound texture analysis of fetal lungs to predict neonatal respiratory morbidity

    Ultrasound Obstetrics Gynecol.

    (2015)
  • G. Carneiro et al.

    Semantic-based indexing of fetal anatomies from 3-D ultrasound data using global / semi-local context and sequential sampling

    IEEE Conference on Computer Vision and Pattern Recognition

    (2008)
  • G. Carneiro et al.

    Detection and measurement of fetal anatomies from ultrasound images using a constrained probabilistic boosting tree

    IEEE Trans. Med. Imaging

    (2008)
  • M. Ceresa et al.

    Surgical planning system for twin-to-twin transfusion syndrome fetal surgery

    International Conference of Computed Assisted Radiology and Surgery (CARS)

    (2017)
  • M. Ceresa et al.

    A novel surgical planning system for TTTS

    Congreso Anual de la Sociedad Espaola de Bioingeniería

    (2017)
  • J.J. Cerrolaza et al.

    Fetal skull segmentation in 3D ultrasound via structured geodesic random forest

    International Workshop on Fetal, Infant and Ophthalmic Medical Image Analysis (MICCAI)

    (2017)
  • C. Chen et al.

    Evaluation of placental maturity by the sonographic textures

    Arch. Gynecol. Obstet.

    (2011)
  • H. Chen et al.

    Automatic fetal ultrasound standard plane detection using knowledge transferred recurrent neural networks

    International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)

    (2015)
  • H. Chen et al.

    Standard plane localization in fetal ultrasound via domain transferred deep neural networks

    IEEE J. Biomed. Health Inform.

    (2015)
  • H. Chen et al.

    Fetal abdominal standard plane localization through representation learning with knowledge transfer

    International Workshop on Machine Learning in Medical Imaging

    (2014)
  • S. Chikop et al.

    Segmentation and visualization of brain and lung volumes in fetal MRI using active contours and morphological operators

    Int. Society for Magnetic Resonance in Medicine (ISMRM)

    (2014)
  • M.Y. Choong et al.

    Variational level set algorithm in image segmentation for foetus ultrasound imaging system

    Int. J. Simulation

    (2012)
  • T. Cobo et al.

    Feasibility and reproducibility of fetal lung texture analysis by automatic quantitative ultrasound analysis and correlation with gestational age

    Fetal. Diagn. Ther.

    (2012)
  • R. Cuingnet et al.

    Where is my baby? a fast fetal head auto-alignment in 3D-ultrasound

    IEEE International Symposium on Biomedical Imaging: From Nano to Macro

    (2013)
  • S. Dahdouh et al.

    In vivo placental MRI shape and textural features predict fetal growth restriction and postnatal outcome

    J. Magn. Reson. Imaging

    (2018)
  • S. Dahdouh et al.

    Segmentation of fetal envelope from 3D ultrasound images based on pixel intensity statistical distribution and shape priors

    IEEE 10th International Symposium on Biomedical Imaging

    (2013)
  • Y. Deng et al.

    Automated detection of fetal cardiac structure from first-trimester ultrasound sequences

    3rd International Conference on Biomedical Engineering and Informatics

    (2010)
  • D.E.O. Dewi et al.

    Review on advanced techniques in 2-d fetal echocardiography: an image processing perspective

    Adv. Med. Diagnostic Technol.

    (2014)
  • I. Dindoyal et al.

    Automatic segmentation of low resolution fetal cardiac data using snake with shape priors

    5th International Symposium on image and Signal Processing and Analysis

    (2007)
  • I. Dindoyal et al.

    Level set snake algorithms on the fetal heart

    4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro

    (2007)
  • I. Dindoyal et al.

    2D / 3d fetal cardiac dataset segmentation using a deformable model

    Med. Phys.

    (2011)
  • Cited by (81)

    • Advances in the Application of Artificial Intelligence in Fetal Echocardiography

      2024, Journal of the American Society of Echocardiography
    • A review on deep-learning algorithms for fetal ultrasound-image analysis

      2023, Medical Image Analysis
      Citation Excerpt :

      Survey papers in the field have been published in the last years, even if a number of them focuses on the broader area of medical-image analysis (Liu et al., 2019) or surveys also DL algorithms for US image analysis outside the fetal field (Van Sloun et al., 2019; Akkus et al., 2019; Ouahabi and Taleb-Ahmed, 2021; Shen et al., 2021; Zaffino et al., 2020). Survey papers specifically dealing with US fetal images include: Torrents-Barrena et al. (2019), Song et al. (2021) and Sree and Vasanthanayaki (2019), where segmentation and classification algorithms are covered; Garcia-Canadilla et al. (2020) and Morris and Lopez (2021), that survey methods for fetal cardiology images; Rawat et al. (2018) and Bushra and Shobana (2021), that briefly summarize DL methods for fetal abnormality detection; and Chen et al. (2021c) and Diniz et al. (2020), that analyze research papers from a clinical perspective. An updated review that surveys the most recent work in the field of fetal US image analysis with DL could be a valuable and compact source of information for young researchers, and a reference overview document for those already working in the field.

    View all citing articles on Scopus

    Conflict of interest: There are no conflicts of interest.

    View full text