Elsevier

PET Clinics

Volume 17, Issue 1, January 2022, Pages 183-212
PET Clinics

AI-Based Detection, Classification and Prediction/Prognosis in Medical Imaging: Towards Radiophenomics

https://doi.org/10.1016/j.cpet.2021.09.010Get rights and content

Section snippets

Key points

  • Artificial intelligence (AI) techniques are being increasingly explored in medical imaging. Innovations in machine learning (ML) and deep learning (DL) have helped unlock the potentials of AI for successful applications.

  • Patient health information, including demographic information, electronic medical record notes, diagnostic imaging at different time-points, and radiologist reports, along with radiomic features can be used as input to AI techniques for detection, classification, and outcome

Advanced image quantification; the field of radiomics

Medical images contain a significant minable and potentially valuable quantitative information beyond what is nowadays captured in routine clinical evaluations, motivating the field of radiomics.28,29 An array of AI techniques in the field of medical imaging have emerged in the past decade to derive imaging biomarkers based on this information, using explicit (ie, handcrafted/engineered) radiomics features or deep radiomics features (ie, derived via deep neural networks (NNs)). These techniques

Radiomics signatures

Radiomics analyses can augment visual assessments made by radiologists.44 AI techniques can perform quantitative high-throughput image phenotyping (extracting numerous image-based features) and identifying important discriminative features that individually or in combination form an effective radiomics signature for a given task. The field of radiomics has been introduced and elaborated in an accompanying article by Orlhac and colleagues.29 The challenging task of detecting suspicious regions

Clinical utility of artificial intelligence-based detection in positron emission tomography/computed tomography: current status

Detection of organs or lesions as regions of interests (ROIs) or volumes of interests (VOIs) is an important step toward the classification of the regions.69 Besides, most existing segmentation approaches have an embedded detection step; we discuss segmentation methods in another work [see Yousefirizi and colleagues’ article, “Towards High-throughput AI-based Segmentation in Oncological PET Imaging,” in this issue]. End-to-end detection systems developed by AI techniques can effectively remove

Clinical utility of artificial intelligence-based classification in positron emission tomography/computed tomography: current status

Classification is considered the most popular area in which CNNs have been used; for example, AlexNet, ResNet, DenseNet, VGG network, and others.79 Similarly, in medical imaging, AI techniques have been widely used for the extraction of feature toward: (i) classification of suspicious lesions and tumor subtypes, as well as (ii) prediction/prognostication tasks, stratifying/classifying patients into risk groups.111,112 We review both frontiers next.

Temporal changes in radiomics features: dynamic-radiomics and delta-radiomics

Radiomics signatures of medical images are usually derived from static images. It has been shown that temporal changes for example in tracer uptake of PET scans can also reveal new aspects of tumor biology.142, 143, 144, 145 Analysis of features extracted from the temporal analysis of dynamic PET scans can be referred to as “dynamic-radiomics” (micro-scale temporal changes). By contrast, “delta-radiomics” refer to the analysis of features derived from the comparison of a study with prior images

Summary

Considering the strength of AI techniques in performing effective image phenotyping including robust identification of patterns beyond visual assessments, there is significant potential for use in accelerating and automating detection and classification tasks in medical imaging and as an example oncologic PET imaging, as reviewed in the present work. AI techniques can also link radiomics signatures and biological properties extracted by radiologists and nuclear medicine physicians in the

Clinics care points

  • Stratification of cancer into reliably distinct risk subgroups enables personalization of treatment.

  • Detection of organs or lesions as regions of interests (ROIs) or volumes of interests (VOIs) is an important step towards classification.

  • Classification tasks for identification and characterization of lesions have significant potential in clinical workflows. Classification in the context of prediction/prognosis also enables risk stratification.

  • Radiomics mines for potentially significant

Acknowledgements

This project was in part supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant RGPIN-2019-06467, and the Canadian Institutes of Health Research (CIHR) Project Grant PJT-173231.

Disclosure

Authors do not have anything to disclose regarding conflict of interest with respect to this article.

First page preview

First page preview
Click to open first page preview

References (194)

  • M. Vaidya et al.

    Combined PET/CT image characteristics for radiotherapy tumor response in lung cancer

    Radiother Oncol

    (2012)
  • A. Kotrotsou et al.

    Radiomics in brain tumors: an emerging technique for characterization of tumor environment

    Magn Reson Imaging Clin

    (2016)
  • Y. Zhang et al.

    Radiomics-based prognosis analysis for non-small cell lung cancer

    Scientific Rep

    (2017)
  • J. Kim et al.

    Training of deep convolutional neural nets to extract radiomic signatures of tumors

    J Nucl Med

    (2019)
  • B. Gaonkar et al.

    Eigenrank by committee: Von-Neumann entropy based data subset selection and failure prediction for deep learning based medical image segmentation

    Med Image Anal

    (2021)
  • D. Karimi et al.

    Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis

    Med Image Anal

    (2020)
  • L. Bi et al.

    Automatic detection and classification of regions of FDG uptake in whole-body PET-CT lymphoma studies

    Comput Med Imaging Graphics

    (2017)
  • S. Afshari et al.

    Automatic localization of normal active organs in 3D PET scans

    Comput Med Imaging Graphics

    (2018)
  • A. Ben-Cohen et al.

    Cross-modality synthesis from CT to PET using FCN and GAN networks for improved automated lesion detection

    Eng Appl Artif Intelligence

    (2019)
  • H. Xie et al.

    Automated pulmonary nodule detection in CT images using deep convolutional neural networks

    Pattern Recognition

    (2019)
  • Y. Shen et al.

    An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization

    Med image Anal

    (2021)
  • D. Gu et al.

    On the performance of lung nodule detection, segmentation and classification

    Comput Med Imaging Graphics

    (2021)
  • C.P. Langlotz

    RadLex: a new method for indexing online educational materials

    Radiographics

    (2006)
  • V. Chernyak et al.

    LI-RADS: future directions

    Clin Liver Dis

    (2021)
  • M.A. Bruno et al.

    Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction

    Radiographics

    (2015)
  • K. Daniel

    Thinking, fast and slow

    (2017)
  • Y.W. Kim et al.

    Fool me twice: delayed diagnoses in radiology with emphasis on perpetuated errors

    Am J roentgenology

    (2014)
  • L. Zhaoping et al.

    Understanding vision: theory, models, and data

    (2014)
  • M. Riesenhuber et al.

    Hierarchical models of object recognition in cortex

    Nat Neurosci

    (1999)
  • M. Bar

    A cortical mechanism for triggering top-down facilitation in visual object recognition

    J Cogn Neurosci

    (2003)
  • Kim JU, Kim ST, Kim ES, et al. Towards high-performance object detection: Task-specific design considering...
  • N.M. Murray et al.

    Artificial intelligence to diagnose ischemic stroke and identify large vessel occlusions: a systematic review

    J neurointerventional Surg

    (2020)
  • J. Chamberlin et al.

    Automated detection of lung nodules and coronary artery calcium using artificial intelligence on low-dose CT scans for lung cancer screening: accuracy and prognostic value

    BMC Med

    (2021)
  • D.J. Winkel et al.

    Evaluation of an AI-based detection software for acute findings in abdominal computed tomography scans: toward an automated work list prioritization of routine CT examinations

    Invest Radiol

    (2019)
  • U. Food et al.

    Computer-assisted detection devices applied to radiology images and radiology device data—Premarket notification [510 (k)] submissions

    (2012)
  • Center for Devices, & Radiological Health. (n.d.). Clinical Performance Assessment: Considerations for CAD Devices....
  • S.K. Zhou et al.

    Handbook of medical image computing and computer assisted intervention

    (2019)
  • D. Houle et al.

    Phenomics: the next challenge

    Nat Rev Genet

    (2010)
  • L. Hoyles et al.

    Molecular phenomics and metagenomics of hepatic steatosis in non-diabetic obese women

    Nat Med

    (2018)
  • K.M. Zbuk et al.

    Cancer phenomics: RET and PTEN as illustrative models

    Nat Rev Cancer

    (2007)
  • G. Bourdais et al.

    Large-scale phenomics identifies primary and fine-tuning roles for CRKs in responses related to oxidative stress

    PLoS Genet

    (2015)
  • Ş. Kafkas et al.

    Linking common human diseases to their phenotypes; development of a resource for human phenomics

    J Biomed semantics

    (2021)
  • M. Hatt et al.

    Radiomics in PET/CT: more than meets the eye?

    J Nucl Med

    (2017)
  • N. Horvat et al.

    Novel imaging techniques of rectal cancer: what do radiomics and radiogenomics have to offer? A literature review

    Abdom Radiol

    (2019)
  • C.P. Langlotz et al.

    A roadmap for foundational research on artificial intelligence in medical imaging: from the 2018 NIH/RSNA/ACR/The Academy Workshop

    Radiology

    (2019)
  • A.R.M. Al-shamasneh et al.

    Artificial intelligence techniques for cancer detection and classification: review study

    Eur Scientific J

    (2017)
  • B. D’Amore et al.

    Role of machine learning and artificial intelligence in interventional oncology

    Curr Oncol Rep

    (2021)
  • D. Visvikis et al.

    Artificial intelligence, machine (deep) learning and radio (geno) mics: definitions and nuclear medicine imaging applications

    Eur J Nucl Med Mol Imaging

    (2019)
  • Oquab M, Bottou L, Laptev I, et al. Is object localization for free?-weakly-supervised learning with convolutional...
  • Zhou B, Khosla A, Lapedriza A, et al. Learning deep features for discriminative localization. in Proceedings of the...
  • Cited by (0)

    View full text