Skip to main content

Advertisement

SpringerLink
Log in

Deep learning-based breast cancer classification through medical imaging modalities: state of the art and research challenges

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Breast cancer is a common and fatal disease among women worldwide. Therefore, the early and precise diagnosis of breast cancer plays a pivotal role to improve the prognosis of patients with this disease. Several studies have developed automated techniques using different medical imaging modalities to predict breast cancer development. However, few review studies are available to recapitulate the existing literature on breast cancer classification. These studies provide an overview of the classification, segmentation, or grading of many cancer types, including breast cancer, by using traditional machine learning approaches through hand-engineered features. This review focuses on breast cancer classification by using medical imaging multimodalities through state-of-the-art artificial deep neural network approaches. It is anticipated to maximize the procedural decision analysis in five aspects, such as types of imaging modalities, datasets and their categories, pre-processing techniques, types of deep neural network, and performance metrics used for breast cancer classification. Forty-nine journal and conference publications from eight academic repositories were methodically selected and carefully reviewed from the perspective of the five aforementioned aspects. In addition, this study provided quantitative, qualitative, and critical analyses of the five aspects. This review showed that mammograms and histopathologic images were mostly used to classify breast cancer. Moreover, about 55% of the selected studies used public datasets, and the remaining used exclusive datasets. Several studies employed augmentation, scaling, and image normalization pre-processing techniques to minimize inconsistencies in breast cancer images. Several types of shallow and deep neural network architecture were employed to classify breast cancer using images. The convolutional neural network was utilized frequently to construct an effective breast cancer classification model. Some of the selected studies employed a pre-trained network or developed new deep neural networks to classify breast cancer. Most of the selected studies used accuracy and area-under-the-curve metrics followed by sensitivity, precision, and F-measure metrics to evaluate the performance of the developed breast cancer classification models. Finally, this review presented 10 open research challenges for future scholars who are interested to develop breast cancer classification models through various imaging modalities. This review could serve as a valuable resource for beginners on medical image classification and for advanced scientists focusing on deep learning-based breast cancer classification through different medical imaging modalities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  • Abraham A (2005) Artificial neural networks. In: Peter H, Sydenham RT (eds) Handbook of measuring system design. Wiley, London, pp 901–908

    Google Scholar 

  • Adoui ME, Drisis S, Benjelloun M (2017) Analyzing breast tumor heterogeneity to predict the response to chemotherapy using 3D MR images registration. Paper presented at the Proceedings of the 2017 international conference on smart digital environment, Rabat, Morocco. http://delivery.acm.org/10.1145/3130000/3128137/p56-el_adoui.pdf?ip=103.18.2.251&id=3128137&acc=ACTIVE%20SERVICE&key=69AF3716A20387ED%2EE7759EC8BE158239%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&__acm__=1527212708_ca160ac108047af6b06969e33a701e4f. Accessed 15 Aug 2018

  • Ahn SJ, Kim YS, Kim EY, Park HK, Cho EK, Kim YK et al (2013) The value of chest CT for prediction of breast tumor size: comparison with pathology measurement. World J Surg Oncol 11:130. https://doi.org/10.1186/1477-7819-11-130

    Article  Google Scholar 

  • Aksebzeci BH, Kayaalti Ö (2017) Computer-aided classification of breast cancer histopathological images. Paper presented at the 2017 Medical Technologies National Congress (TIPTEKNO)

  • Amit G, Ben-Ari R, Hadad O, Monovich E, Granot N, Hashoul S (2017) Classification of breast MRI lesions using small-size training sets: comparison of deep learning approaches. Paper presented at the progress in biomedical optics and imaging—proceedings of SPIE

  • Amodei D, Ananthanarayanan S, Anubhai R, Bai J, Battenberg E, Case C et al (2016) Deep speech 2: end-to-end speech recognition in English and mandarin. Paper presented at the International conference on machine learning

  • Antropova N, Abe H, Giger ML (2018a) Use of clinical MRI maximum intensity projections for improved breast lesion classification with deep convolutional neural networks. J Med Imaging 5(1):6. https://doi.org/10.1117/1.jmi.5.1.014503

    Article  Google Scholar 

  • Antropova N, Huynh B, Giger M (2018) Recurrent neural networks for breast lesion classification based on DCE-MRIs. Paper presented at the progress in biomedical optics and imaging—proceedings of SPIE

  • Araujo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C et al (2017) Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12(6):14. https://doi.org/10.1371/journal.pone.0177544

    Article  Google Scholar 

  • Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C et al (2017) Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12(6):e0177544

    Article  Google Scholar 

  • Arefan D, Talebpour A, Ahmadinejhad N, Asl AK (2015) Automatic breast density classification using neural network. J Instrum. https://doi.org/10.1088/1748-0221/10/12/t12002

    Article  Google Scholar 

  • Arevalo J, González FA, Ramos-Pollán R, Oliveira JL, Lopez MAG (2015) Convolutional neural networks for mammography mass lesion classification. Paper presented at the 2015 37th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

  • Bakkouri I, Afdel K (2017) Breast tumor classification based on deep convolutional neural networks. Paper presented at the Proceedings—3rd international conference on advanced technologies for signal and image processing, ATSIP 2017

  • Bardou D, Zhang K, Ahmad SM (2018) Classification of breast cancer based on histology images using convolutional neural networks. IEEE Access. https://doi.org/10.1109/access.2018.2831280

    Article  Google Scholar 

  • Barr RG (2012) Sonographic breast elastography: a primer. J Ultrasound Med 31(5):773–783

    Article  Google Scholar 

  • Bayramoglu N, Kannala J, Heikkila J (2017) Deep learning for magnification independent breast cancer histopathology image classification. Paper presented at the Proceedings—international conference on pattern recognition

  • Bebis G, Georgiopoulos M (1994) Feed-forward neural networks. IEEE Potentials 13(4):27–31

    Article  Google Scholar 

  • Bejnordi BE, Lin J, Glass B, Mullooly M, Gierach GL, Sherman ME et al (2017a) Deep learning-based assessment of tumor-associated stroma for diagnosing breast cancer in histopathology images. Paper presented at the 2017 IEEE 14th international symposium on biomedical imaging (ISBI 2017)

  • Bejnordi BE, Zuidhof G, Balkenhol M, Hermsen M, Bult P, van Ginneken B et al (2017b) Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images. J Med Imaging 4(4):8. https://doi.org/10.1117/1.jmi.4.4.044504

    Article  Google Scholar 

  • Bekker AJ, Greenspan H, Goldberger J (2016) A multi-view deep learning architecture for classification of breast microcalcifications. Paper presented at the Proceedings—international symposium on biomedical imaging

  • Bengio Y (2009) Learning deep architectures for AI. Found Trends® Mach Learn 2(1):1–127. https://doi.org/10.1561/2200000006

    Article  MathSciNet  MATH  Google Scholar 

  • Beutel J, Kundel HL, Van Metter RL (2000) Handbook of medical imaging, vol 1. SPIE Press, Bellingham

    Google Scholar 

  • Bevilacqua V, Brunetti A, Triggiani M, Magaletti D, Telegrafo M, Moschetta M (2016) An optimized feed-forward artificial neural network topology to support radiologists in breast lesions classification. Paper presented at the GECCO 2016 companion—proceedings of the 2016 genetic and evolutionary computation conference

  • Breast Cancer Imaging (2018) Breast Cancer Imaging. Retrieved from http://www.aboutcancer.com/breast_cancer_imaging.htm. Accessed 20 Aug 2018

  • Byra M, Piotrzkowska-Wroblewska H, Dobruch-Sobczak K, Nowicki A (2017) Combining Nakagami imaging and convolutional neural network for breast lesion classification. Paper presented at the IEEE international ultrasonics symposium, IUS

  • Cao J, Qin Z, Jing J, Chen J, Wan T (2016) An automatic breast cancer grading method in histopathological images based on pixel-, object-, and semantic-level features. Paper presented at the 2016 IEEE 13th international symposium on biomedical imaging (ISBI)

  • Carneiro G, Nascimento J, Bradley AP (2015) Unregistered multiview mammogram analysis with pre-trained deep learning models. In: International conference on medical image computing and computer-assisted intervention. Springer, Cham, pp. 652–660

  • Carneiro G, Nascimento J, Bradley AP (2017) Automated analysis of unregistered multi-view mammograms with deep learning. IEEE Trans Med Imaging 36(11):2355–2365. https://doi.org/10.1109/TMI.2017.2751523

    Article  Google Scholar 

  • Chan T, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2015) PCANet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24(12):5017–5032. https://doi.org/10.1109/TIP.2015.2475625

    Article  MathSciNet  MATH  Google Scholar 

  • Chang J, Yu J, Han T, Chang H, Park E (2017) A method for classifying medical images using transfer learning: a pilot study on histopathology of breast cancer. Paper presented at the 2017 IEEE 19th international conference on e-health networking, applications and services (Healthcom)

  • Chen H, Qi X, Yu L, Dou Q, Qin J, Heng P-A (2017a) DCAN: deep contour-aware networks for object instance segmentation from histology images. Med Image Anal 36:135–146. https://doi.org/10.1016/j.media.2016.11.004

    Article  Google Scholar 

  • Chen JM, Li Y, Xu J, Gong L, Wang LW, Liu WL, Liu J (2017b) Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: a review. Tumor Biol 39(3):12. https://doi.org/10.1177/1010428317694550

    Article  Google Scholar 

  • Cheng J-Z, Ni D, Chou Y-H, Qin J, Tiu C-M, Chang Y-C et al (2016) Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 6:24454. https://doi.org/10.1038/srep24454

    Article  Google Scholar 

  • Chris Rose DT, Williams A, Wolstencroft K, Taylor C (2006) DDSM: digital database for screening mammography. Retrieved from http://marathon.csee.usf.edu/Mammography/Database.html. Accessed 26 Aug 2018

  • Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P et al (2013) The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26(6):1045–1057. https://doi.org/10.1007/s10278-013-9622-7

    Article  Google Scholar 

  • Conceição RC, Medeiros H, Halloran MO, Rodriguez-Herrera D, Flores-Tapia D, Pistorius S (2014) SVM-based classification of breast tumour phantoms using a UWB radar prototype system. Paper presented at the 2014 XXXIth URSI general assembly and scientific symposium (URSI GASS)

  • Cruz-Roa A, Gilmore H, Basavanhally A, Feldman M, Ganesan S, Shih NNC et al (2017) Accurate and reproducible invasive breast cancer detection in whole-slide images: a deep learning approach for quantifying tumor extent. Sci Rep. https://doi.org/10.1038/srep46450

    Article  Google Scholar 

  • Dalkey N, Helmer O (1963) An experimental application of the Delphi method to the use of experts. Manag Sci 9(3):458–467

    Article  Google Scholar 

  • De S Silva SD, Costa MGF, De A Pereira WC, Filho CFFC (2015) Breast tumor classification in ultrasound images using neural networks with improved generalization methods. Paper presented at the Proceedings of the annual international conference of the IEEE Engineering in Medicine and Biology Society, EMBS

  • Deng L, Yu D (2014) Deep learning: methods and applications. Found Trends® Signal Process 7(3–4):197–387. https://doi.org/10.1561/2000000039

    Article  MathSciNet  MATH  Google Scholar 

  • Dhungel N, Carneiro G, Bradley AP (2017) A deep learning approach for the analysis of masses in mammograms with minimal user intervention. Med Image Anal 37:114–128. https://doi.org/10.1016/j.media.2017.01.009

    Article  Google Scholar 

  • Doi K (2007) Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput Med Imaging Graph 31(4–5):198–211

    Article  Google Scholar 

  • Dua D (2017) KT UCI machine learning repository. University of California, School of Information and Computer Science, Irvine. Retrieved from http://archive.ics.uci.edu/ml

  • Duch W, Jankowski N (1999) Survey of neural transfer functions. Neural Comput Surv 2(1):163–212

    Google Scholar 

  • Duraisamy S, Emperumal S (2017) Computer-aided mammogram diagnosis system using deep learning convolutional fully complex-valued relaxation neural network classifier. IET Comput Vision 11(8):656–662. https://doi.org/10.1049/iet-cvi.2016.0425

    Article  Google Scholar 

  • Elmore JG, Jackson SL, Abraham L, Miglioretti DL, Carney PA, Geller BM et al (2009) Variability in interpretive performance at screening mammography and radiologists’ characteristics associated with accuracy. Radiology 253(3):641–651

    Article  Google Scholar 

  • Ertosun MG, Rubin DL (2015) Probabilistic visual search for masses within mammography images using deep learning. Paper presented at the 2015 IEEE international conference on bioinformatics and biomedicine (BIBM)

  • Farahani N, Parwani AV, Pantanowitz L (2015) Whole slide imaging in pathology: advantages, limitations, and emerging perspectives. Pathol Lab Med Int 7:23–33

    Google Scholar 

  • Feng Y, Zhang L, Yi Z (2018) Breast cancer cell nuclei classification in histopathology images using deep neural networks. Int J Comput Assist Radiol Surg 13(2):179–191. https://doi.org/10.1007/s11548-017-1663-9

    Article  Google Scholar 

  • Ferri C, Hernández-Orallo J, Salido MA (2003) Volume under the ROC surface for multi-class problems. Paper presented at the European conference on machine learning

  • Fischer A, Igel C (2012) An introduction to restricted Boltzmann machines. Paper presented at the Iberoamerican Congress on pattern recognition

  • Fonseca P, Mendoza J, Wainer J, Ferrer J, Pinto J, Guerrero J, Castaneda B (2015) Automatic breast density classification using a convolutional neural network architecture search procedure. Paper presented at the Progress in biomedical optics and imaging—proceedings of SPIE

  • Fukushima K, Miyake S (1982) Neocognitron: a self-organizing neural network model for a mechanism of visual pattern recognition. Competition and cooperation in neural nets. Springer, Berlin, pp 267–285

    Google Scholar 

  • Gandomkar Z, Brennan PC, Mello-Thoms C (2018) MuDeRN: multi-category classification of breast histopathological image using deep residual networks. Artif Intell Med. https://doi.org/10.1016/j.artmed.2018.04.005

    Article  Google Scholar 

  • Gardezi SJS, Awais M, Faye I, Meriaudeau F (2017) Mammogram classification using deep learning features. Paper presented at the 2017 IEEE international conference on signal and image processing applications (ICSIPA)

  • Goceri E (2017) Advances in digital pathology. Paper presented at the international conference on applied analysis and mathematical modeling. Istanbul, Turkey

  • Goceri E (2018) Formulas behind deep learning success. Paper presented at the international conference on applied analysis and mathematical modeling. Istanbul, Turkey

  • Goceri E, Goceri N (2017) Deep learning in medical image analysis: recent advances and future trends. Paper presented at the international conferences computer graphics, visualization, computer vision and image processing. Istanbul, Turkey

  • Goceri E, Gooya A (2018) On the importance of batch size for deep learning. Paper presented at the international conference on mathematics. Istanbul, Turkey

  • Goceri E, Songul C (2018) Biomedical information technology: image based computer aided diagnosis systems. Paper presented at the international conference on advanced technologies. Antalya, Turkey

  • Gurcan MN, Boucheron L, Can A, Madabhushi A, Rajpoot N, Yener B (2009) Histopathological image analysis: a review. IEEE Rev Biomed Eng 2:147

    Article  Google Scholar 

  • Haarburger C, Langenberg P, Truhn D, Schneider H, Thüring J, Schrading S et al (2018) Transfer learning for breast cancer malignancy classification based on dynamic contrast-enhanced MR images. Paper presented at the Informatik aktuell

  • Hadad O, Bakalo R, Ben-Ari R, Hashoul S, Amit G (2017) Classification of breast lesions using cross-modal deep learning. Paper presented at the proceedings—international symposium on biomedical imaging

  • Han S, Kang HK, Jeong JY, Park MH, Kim W, Bang WC, Seong YK (2017a) A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys Med Biol 62(19):7714–7728. https://doi.org/10.1088/1361-6560/aa82ec

    Article  Google Scholar 

  • Han Z, Wei B, Zheng Y, Yin Y, Li K, Li S (2017b) Breast cancer multi-classification from histopathological images with structured deep learning model. Sci Rep. https://doi.org/10.1038/s41598-017-04075-z

    Article  Google Scholar 

  • Hannun A, Case C, Casper J, Catanzaro B, Diamos G, Elsen E et al (2014) Deep speech: scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567

  • Hassan NA, Yassin AH, Tayel MB, Mohamed MM (2016) Ultra-wideband scattered microwave signals for detection of breast tumors using artifical neural networks. Paper presented at the 2016 3rd International conference on artificial intelligence and pattern recognition, AIPR 2016

  • He X, Frey EC (2008) The meaning and use of the volume under a three-class ROC surface (VUS). IEEE Trans Med Imaging 27(5):577–588

    Article  Google Scholar 

  • Heath M, Bowyer K, Kopans D, Moore R, Kegelmeyer WP (2000) The digital database for screening mammography. Paper presented at the Proceedings of the 5th international workshop on digital mammography

  • Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  • Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  • Hofvind S, Hovda T, Holen ÅS, Lee CI, Albertsen J, Bjørndal H et al (2018) Digital breast tomosynthesis and synthetic 2D mammography versus digital mammography: evaluation in a population-based screening program. Radiology 287(3):787–794

    Article  Google Scholar 

  • Hussain A, Farooq K, Luo B, Slack W (2015). A novel ontology and machine learning inspired hybrid cardiovascular decision support framework. Paper presented at the 2015 IEEE symposium series on computational intelligence

  • Islam KT, Raj RG, Mujtaba G (2017) Recognition of traffic sign based on bag-of-words and artificial neural network. Symmetry 9(8):138

    Article  Google Scholar 

  • Jaffar MA (2017) Deep learning based computer aided diagnosis system for breast mammograms. Int J Adv Comput Sci Appl 8(7):286–290

    Google Scholar 

  • Jalalian A, Mashohor S, Mahmud R, Karasfi B, Saripan MIB, Ramli ARB (2017) Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection. Exclus J 16:113–137. https://doi.org/10.17179/excli201-701

    Article  Google Scholar 

  • James JJ, Wilson ARM, Evans AJ (2016) The breast. Retrieved from https://radiologykey.com/the-breast-2/. Accessed 28 Aug 2018

  • Jarrett K, Kavukcuoglu K, LeCun Y (2009) What is the best multi-stage architecture for object recognition? Paper presented at the 2009 IEEE 12th international conference on computer vision

  • Jiang F, Liu H, Yu S, Xie Y (2017) Breast mass lesion classification in mammograms by transfer learning. Paper presented at the ACM international conference proceeding series

  • Jing H, Yang Y, Nishikawa RM (2012) Regularization in retrieval-driven classification of clustered microcalcifications for breast cancer. J Biomed Imaging 2012:3

    Google Scholar 

  • Jirayucharoensak S, Pan-Ngum S, Israsena P (2014) EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. Sci World J 2014:1–10

    Article  Google Scholar 

  • Jyh-Horng C, Jinn Tsong T, Tung-Kuan L, Kao-Shing H, Hon-Yi S (2014) Predictive models for 5-year mortality after breast cancer surgery. Paper presented at the 2014 International conference on machine learning and cybernetics

  • Kahou SE, Bouthillier X, Lamblin P, Gulcehre C, Michalski V, Konda K et al (2016) Emonets: multimodal deep learning approaches for emotion recognition in video. J Multimodal User Interfaces 10(2):99–111

    Article  Google Scholar 

  • Kasban H, El-Bendary M, Salama D (2015) A comparative study of medical imaging techniques. Int J Inf Sci Intell Syst 4:37–58

    Google Scholar 

  • Keele S (2007) Guidelines for performing systematic literature reviews in software engineering. In: Technical report, Ver. 2.3 EBSE Technical Report. EBSE

  • Khan MHM (2017) Automated breast cancer diagnosis using artificial neural network (ANN). Paper presented at the 2017 3rd Iranian conference on signal processing and intelligent systems, New York

  • Khan AM, Rajpoot N, Treanor D, Magee D (2014) A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE Trans Biomed Eng 61(6):1729–1738

    Article  Google Scholar 

  • Kim DH, Kim ST, Ro YM (2016) Latent feature representation with 3-D multi-view deep convolutional neural network for bilateral analysis in digital breast tomosynthesis. Paper presented at the 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP)

  • Abdullah-Al N, Bin Ali F, Kong YN, IEEE (2017) Histopathological breast-image classification with image enhancement by convolutional neural network. Paper presented at the 2017 20th International conference of computer and information technology, New York

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Paper presented at the Advances in neural information processing systems

  • Kumar D, Kumar C, Shao M (2017a) Cross-database mammographic image analysis through unsupervised domain adaptation. Paper presented at the 2017 IEEE international conference on big data (big data)

  • Kumar I, Bhadauria HS, Virmani J, Thakur S (2017b) A classification framework for prediction of breast density using an ensemble of neural network classifiers. Biocybern Biomed Eng 37(1):217–228. https://doi.org/10.1016/j.bbe.2017.01.001

    Article  Google Scholar 

  • Lakhani P, Sundaram B (2017) Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 284(2):574–582

    Article  Google Scholar 

  • Landgrebe TC, Duin RP (2008) Efficient multiclass ROC approximation by decomposition via confusion matrix perturbation analysis. IEEE Trans Pattern Anal Mach Intell 30(5):810–822

    Article  Google Scholar 

  • Lebron L, Greenspan D, Pandit-Taskar N (2015) PET imaging of breast cancer: role in patient management. PET Clinics 10(2):159–195. https://doi.org/10.1016/j.cpet.2014.12.004

    Article  Google Scholar 

  • Lee H, Chen Y-PP (2015) Image based computer aided diagnosis system for cancer detection. Expert Syst Appl 42(12):5356–5365. https://doi.org/10.1016/j.eswa.2015.02.005

    Article  Google Scholar 

  • Leod PM, Verma B (2016) Polynomial prediction of neurons in neural network classifier for breast cancer diagnosis. Paper presented at the Proceedings—international conference on natural computation

  • Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88. https://doi.org/10.1016/j.media.2017.07.005

    Article  Google Scholar 

  • Liu F, Hernandez-Cabronero M, Sanchez V, Marcellin MW, Bilgin A (2017) The current role of image compression standards in medical imaging. Information 8(4):131

    Article  Google Scholar 

  • Lo C, Shen Y-W, Huang C-S, Chang R-F (2014) Computer-aided multiview tumor detection for automated whole breast ultrasound. Ultrason Imaging 36(1):3–17. https://doi.org/10.1177/0161734613507240

    Article  Google Scholar 

  • Macenko M, Niethammer M, Marron JS, Borland D, Woosley JT, Guan X et al (2009) A method for normalizing histology slides for quantitative analysis. Paper presented at the IEEE international symposium on biomedical imaging: from nano to macro, 2009. ISBI’09

  • McCann MT, Ozolek JA, Castro CA, Parvin B, Kovacevic J (2015) Automated histology analysis: opportunities for signal processing. IEEE Signal Process Mag 32(1):78–87

    Article  Google Scholar 

  • McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133. https://doi.org/10.1007/BF02478259

    Article  MathSciNet  MATH  Google Scholar 

  • Mehdy MM, Ng PY, Shair EF, Saleh NIM, Gomes C (2017) Artificial neural networks in image processing for early detection of breast cancer. Comput Math Methods Med. https://doi.org/10.1155/2017/2610628

    Article  Google Scholar 

  • Mendel KR, Li H, Sheth D, Giger ML (2018) Transfer learning with convolutional neural networks for lesion classification on clinical breast tomosynthesis. Paper presented at the Progress in biomedical optics and imaging—proceedings of SPIE

  • MFMER (2018) Breast MRI. Retrieved from https://www.mayoclinic.org/tests-procedures/breast-mri/about/pac-20384809. Accessed 30 Aug 2018

  • Mina LM, Mat Isa NA (2015) Breast abnormality detection in mammograms using artificial neural network. Paper presented at the I4CT 2015—2015 2nd international conference on computer, communications, and control technology, art proceeding

  • Moon M, Cornfeld D, Weinreb J (2009) Dynamic contrast-enhanced breast MR imaging. Magn Reson Imaging Clin N Am 17(2):351–362

    Article  Google Scholar 

  • Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, Cardoso JS (2012) Inbreast: toward a full-field digital mammographic database. Acad Radiol 19(2):236–248

    Article  Google Scholar 

  • Moura DC, López MAG (2013) An evaluation of image descriptors combined with clinical data for breast cancer diagnosis. Int J Comput Assist Radiol Surg 8(4):561–574

    Article  Google Scholar 

  • Murtaza G, Shuib L, Mujtaba G, Raza G (2019) Breast cancer multi-classification through deep neural network and hierarchical classification approach. Multimed Tools Appl. https://doi.org/10.1007/s11042-019-7525-4

    Article  Google Scholar 

  • Nahid AA, Kong Y (2017a) Involvement of machine learning for breast cancer image classification: a survey. Comput Math Methods Med. https://doi.org/10.1155/2017/3781951

    Article  MathSciNet  Google Scholar 

  • Nahid AA, Kong YA (2017b) Local and global feature utilization for breast image classification by convolutional neural network. Paper presented at the 2017 International conference on digital image computing—techniques and applications, New York

  • Nahid AA, Kong Y (2018) Histopathological breast-image classification using local and frequency domains by convolutional neural network. Information (Switzerland). https://doi.org/10.3390/info9010019

    Article  Google Scholar 

  • Nahid AA, Mehrabi MA, Kong Y (2018) Histopathological breast cancer image classification by deep neural network techniques guided by local clustering. Biomed Res Int. https://doi.org/10.1155/2018/2362108

    Article  Google Scholar 

  • Nascimento CDL, Silva SDS, da Silva TA, Pereira WCA, Costa MGF, Costa Filho CFF (2016) Breast tumor classification in ultrasound images using support vector machines and neural networks. Revista Brasileira de Engenharia Biomedica 32(3):283–292. https://doi.org/10.1590/2446-4740.04915

    Article  Google Scholar 

  • Nejad EM, Affendey LS, Latip RB, Ishak IB (2017) Classification of histopathology images of breast into benign and malignant using a single-layer convolutional neural network. Paper presented at the ACM international conference proceeding series

  • Nweke HF, Teh YW, Alo UR, Mujtaba G (2018) Analysis of multi-sensor fusion for mobile and wearable sensor based human activity recognition. Paper presented at the Proceedings of the international conference on data processing and applications

  • Nweke HF, Teh YW, Mujtaba G, Al-garadi MA (2019) Data fusion and multiple classifier systems for human activity detection and health monitoring: review and open research directions. Inf Fusion 46:147–170

    Article  Google Scholar 

  • Pack C, Shin S, Choi HD, Jeon SI, Kim J (2016) Optimized multilayer perceptron using dynamic learning rate based microwave tomography breast cancer screening. Paper presented at the proceedings of the ACM symposium on applied computing

  • Pan X, Li L, Yang H, Liu Z, Yang J, Zhao L, Fan Y (2017) Accurate segmentation of nuclei in pathological images via sparse reconstruction and deep convolutional networks. Neurocomputing 229:88–99. https://doi.org/10.1016/j.neucom.2016.08.103

    Article  Google Scholar 

  • Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. Paper presented at the BMVC

  • Paula EL, Ladeira M, Carvalho RN, Marzagão T (2016) Deep learning anomaly detection as support fraud investigation in brazilian exports and anti-money laundering. Paper presented at the 2016 15th IEEE international conference on machine learning and applications (ICMLA)

  • Qiu Y, Yan S, Gundreddy RR, Wang Y, Cheng S, Liu H, Zheng B (2017) A new approach to develop computer-aided diagnosis scheme of breast mass classification using deep learning technology. J X-Ray Sci Technol 25(5):751–763. https://doi.org/10.3233/XST-16226

    Article  Google Scholar 

  • Radiological Society of North America, I. R. (2018) RadiologyInfo for patients. Retrieved from https://www.radiologyinfo.org/en/info.cfm?pg=genus. Accessed 2 Sep 2018

  • Rasti R, Teshnehlab M, Phung SL (2017) Breast cancer diagnosis in DCE-MRI using mixture ensemble of convolutional neural networks. Pattern Recogn 72:381–390. https://doi.org/10.1016/j.patcog.2017.08.004

    Article  Google Scholar 

  • Rebecca Sawyer Lee FG, Hoogi A, Rubin D (2016) Curated breast imaging subset of DDSM dataset. The Breast Cancer Imaging Archieve. Retrieved from https://wiki.cancerimagingarchive.net/display/Public/CBIS-DDSM#4413fe70f2bb4159b326a3f07fa6e6a9. Accessed 10 Sep 2018

  • Reinhard E, Adhikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Comput Graphics Appl 21(5):34–41

    Article  Google Scholar 

  • Rouhi R, Jafari M, Kasaei S, Keshavarzian P (2015) Benign and malignant breast tumors classification based on region growing and CNN segmentation. Expert Syst Appl 42(3):990–1002. https://doi.org/10.1016/j.eswa.2014.09.020

    Article  Google Scholar 

  • Rubin R, Strayer DS, Rubin E (2008) Rubin’s pathology: clinicopathologic foundations of medicine. Lippincott Williams & Wilkins, Philadelphia

    Google Scholar 

  • Ruifrok AC, Johnston DA (2001) Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol 23(4):291–299

    Google Scholar 

  • Sadaf A, Crystal P, Scaranelo A, Helbich T (2011) Performance of computer-aided detection applied to full-field digital mammography in detection of breast cancers. Eur J Radiol 77(3):457–461

    Article  Google Scholar 

  • Saidin N, Sakim HAM, Ngah UK, Shuaib IL (2012) Segmentation of breast regions in mammogram based on density: a review. arXiv preprint arXiv:1209.5494

  • Samala RK, Chan HP, Hadjiiski LM, Helvie MA, Cha KH, Richter CD (2017) Multi-task transfer learning deep convolutional neural network: application to computer-aided diagnosis of breast cancer on mammograms. Phys Med Biol 62(23):8894–8908. https://doi.org/10.1088/1361-6560/aa93d4

    Article  Google Scholar 

  • Samala RK, Chan HP, Hadjiiski LM, Helvie MA, Richter C, Cha K (2018) Evolutionary pruning of transfer learned deep convolutional neural network for breast cancer diagnosis in digital breast tomosynthesis. Phys Med Biol 63(9):8. https://doi.org/10.1088/1361-6560/aabb5b

    Article  Google Scholar 

  • Sathish D, Kamath S, Rajagopal KV, Prasad K (2016) Medical imaging techniques and computer aided diagnostic approaches for the detection of breast cancer with an emphasis on thermography—a review. Int J Med Eng Inf 8(3):275–299. https://doi.org/10.1504/IJMEI.2016.077446

    Article  Google Scholar 

  • Schneider M, Yaffe M (2000) Better detection: improving our chances. Paper presented at the Digital mammography: 5th international workshop on digital mammography IWDM

  • Selvathi D, Aarthy Poornila A (2018) Deep learning techniques for breast cancer detection using medical image analysis. In: Hemanth J, Balas VE (eds) Biologically rationalized computing techniques for image processing applications. Springer, Cham, pp 159–186

    Chapter  Google Scholar 

  • Sert E, Ertekin S, Halici U (2017) Ensemble of convolutional neural networks for classification of breast microcalcification from mammograms. Paper presented at the Proceedings of the annual international conference of the IEEE engineering in medicine and biology society, EMBS

  • Shan J, Alam SK, Garra B, Zhang YT, Ahmed T (2016) Computer-aided diagnosis for breast ultrasound using computerized Bi-rads features and machine learning methods. Ultrasound Med Biol 42(4):980–988. https://doi.org/10.1016/j.ultrasmedbio.2015.11.016

    Article  Google Scholar 

  • Shen D, Wu G, Suk H-I (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248. https://doi.org/10.1146/annurev-bioeng-071516-044442

    Article  Google Scholar 

  • Siddiqui MF, Mujtaba G, Reza AW, Shuib L (2017) Multi-class disease classification in brain MRIs using a computer-aided diagnostic system. Symmetry 9(3):37

    Article  MathSciNet  Google Scholar 

  • Sivachitra M, Vijayachitra S (2015) Classification of post operative breast cancer patient information using complex valued neural classifiers. Paper presented at the 2015 International conference on cognitive computing and information processing (CCIP)

  • Sohn K, Zhou G, Lee C, Lee H (2013) Learning and selecting features jointly with point-wise gated Boltzmann machines. Paper presented at the Proceedings of the 30th international conference on international conference on machine learning—volume 28, Atlanta, GA, USA

  • Sophie Softley Pierce, P. M., Breast Cancer Care (2017) Three quarters of NHS Trusts and Health Boards say ‘not enough’ care for incurable breast cancer patients

  • Spanhol FA, Oliveira LS, Petitjean C, Heutte L (2016a) Breast cancer histopathological image classification using convolutional neural networks. Paper presented at the Proceedings of the international joint conference on neural networks

  • Spanhol FA, Oliveira LS, Petitjean C, Heutte L (2016b) A dataset for breast cancer histopathological image classification. IEEE Trans Biomed Eng 63(7):1455–1462

    Article  Google Scholar 

  • Spanhol FA, Oliveira LS, Cavalin PR, Petitjean C, Heutte L (2017) Deep features for breast cancer histopathological image classification. Paper presented at the 2017 IEEE international conference on systems, man, and cybernetics (SMC)

  • Suckling J, Parker J, Dance D, Astley S, Hutt I, Boggis C et al (1994) The mammographic image analysis society digital mammogram database. Paper presented at the Exerpta Medica. International Congress series

  • Suckling J, Parker J, Dance D, Astley S, Hutt I, Boggis C et al (2015) Mammographic Image Analysis Society (MIAS) database v1, p 21

  • Sun J, Binder A (2017) Comparison of deep learning architectures for H&E histopathology images. Paper presented at the 2017 IEEE conference on big data and analytics (ICBDA)

  • Sun Y, Chen Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. Paper presented at the Advances in neural information processing systems

  • Sun W, Tseng TB, Zhang J, Qian W (2017) Enhancing deep convolutional neural network scheme for breast cancer diagnosis with unlabeled data. Comput Med Imaging Graph 57:4–9. https://doi.org/10.1016/j.compmedimag.2016.07.004

    Article  Google Scholar 

  • Sutton RS, Barto AG (1998) Introduction to reinforcement learning, vol 135. MIT Press, Cambridge

    MATH  Google Scholar 

  • Svozil D, Kvasnicka V, Pospichal J (1997) Introduction to multi-layer feed-forward neural networks. Chemometr Intell Lab Syst 39(1):43–62

    Article  Google Scholar 

  • Tan T, Platel B, Twellmann T, van Schie G, Mus R, Grivegnee A et al (2013) Evaluation of the effect of computer-aided classification of benign and malignant lesions on reader performance in automated three-dimensional breast ultrasound. Acad Radiol 20(11):1381–1388. https://doi.org/10.1016/j.acra.2013.07.013

    Article  Google Scholar 

  • Tataroğlu GA, Genç A, Kabakçı KA, Çapar A, Töreyin BU, Ekenel HK et al (2017) A deep learning based approach for classification of CerbB2 tumor cells in breast cancer. Paper presented at the 2017 25th Signal processing and communications applications conference (SIU)

  • Tessa S, Keith JFM (2018) The difference between an MRI and CT scan. Retrieved from https://www.healthline.com/health/ct-scan-vs-mri. Accessed 13 Sep 2018

  • Ting FF, Sim KS, IEEE (2017) Self-regulated multilayer perceptron neural network for breast cancer classification. Paper presented at the 2017 International conference on robotics, automation and sciences, New York

  • Tsui P-H, Yeh C-K, Chang C-C, Liao Y-Y (2008) Classification of breast masses by ultrasonic Nakagami imaging: a feasibility study. Phys Med Biol 53(21):6027

    Article  Google Scholar 

  • Tsui P-H, Ho M-C, Tai D-I, Lin Y-H, Wang C-Y, Ma H-Y (2016) Acoustic structure quantification by using ultrasound Nakagami imaging for assessing liver fibrosis. Sci Rep 6:33075

    Article  Google Scholar 

  • Ultrasound (2018) General ultrasound. Retrieved from https://www.radiologyinfo.org/en/info.cfm?pg=genus. Accessed 17 Sep 2018

  • van Zelst JCM, Tan T, Clauser P, Domingo A, Dorrius MD, Drieling D et al (2018) Dedicated computer-aided detection software for automated 3D breast ultrasound; an efficient tool for the radiologist in supplemental screening of women with dense breasts. Eur Radiol. https://doi.org/10.1007/s00330-017-5280-3

    Article  Google Scholar 

  • Vestjens JHMJ, Pepels MJ, de Boer M, Borm GF, van Deurzen CHM, van Diest PJ, Tjan-Heijnen VCG (2012) Relevant impact of central pathology review on nodal classification in individual breast cancer patients. Ann Oncol 23(10):2561–2566. https://doi.org/10.1093/annonc/mds072

    Article  Google Scholar 

  • Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P-A (2010a) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408

    MathSciNet  MATH  Google Scholar 

  • Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P-A (2010b) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408

    MathSciNet  MATH  Google Scholar 

  • Wan T, Cao J, Chen J, Qin Z (2017) Automated grading of breast cancer histopathology using cascaded ensemble with combination of multi-level image features. Neurocomputing 229:34–44. https://doi.org/10.1016/j.neucom.2016.05.084

    Article  Google Scholar 

  • Wang Y, Xu W (2018) Leveraging deep learning with LDA-based text analytics to detect automobile insurance fraud. Decis Support Syst 105:87–95

    Article  Google Scholar 

  • Wang J, Yang Y (2018) A context-sensitive deep learning approach for microcalcification detection in mammograms. Pattern Recogn 78:12–22. https://doi.org/10.1016/j.patcog.2018.01.009

    Article  Google Scholar 

  • Wang H, Cruz-Roa A, Basavanhally A, Gilmore H, Shih N, Feldman M et al (2014) Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. J Med Imaging. https://doi.org/10.1117/1.jmi.1.3.034003

    Article  Google Scholar 

  • Wang D, Wu K, Gu C, Guan X (2017) Time efficient cell detection in histopathology images using convolutional regression networks. Paper presented at the 2017 36th Chinese control conference (CCC)

  • Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82

    Article  Google Scholar 

  • World Health Organization (2018) Cancer. Retrieved from http://www.who.int/en/news-room/fact-sheets/detail/cancer. Accessed 20 Sep 2018

  • Wu K, Chen X, Ding M (2014) Deep learning based classification of focal liver lesions with contrast-enhanced ultrasound. Optik Int J Light Electron Opt 125(15):4057–4063

    Article  Google Scholar 

  • Wu J, Shi J, Li Y, Suo J, Zhang Q (2016) Histopathological image classification using random binary hashing based PCANet and bilinear classifier. Paper presented at the 2016 24th European signal processing conference (EUSIPCO)

  • Xu J, Luo X, Wang G, Gilmore H, Madabhushi A (2016) A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images. Neurocomputing 191:214–223. https://doi.org/10.1016/j.neucom.2016.01.034

    Article  Google Scholar 

  • Xu J, Zhou C, Lang B, Liu Q (2017) Deep learning for histopathological image analysis: towards computerized diagnosis on cancers. In: Lu L, Zheng Y, Carneiro G, Yang L (eds) Deep learning and convolutional neural networks for medical image computing: precision medicine, high performance and large-scale datasets. Springer, Cham, pp 73–95

    Chapter  Google Scholar 

  • Yassin NIR, Omran S, El Houby EMF, Allam H (2018) Machine learning techniques for breast cancer computer aided diagnosis using different image modalities: a systematic review. Comput Methods Programs Biomed 156:25–45. https://doi.org/10.1016/j.cmpb.2017.12.012

    Article  Google Scholar 

  • Youk JH, Gweon HM, Son EJ (2017) Shear-wave elastography in breast ultrasonography: the state of the art. Ultrasonography 36(4):300–309. https://doi.org/10.14366/usg.17024

    Article  Google Scholar 

  • Yousefi M, Krzyżak A, Suen CY (2018) Mass detection in digital breast tomosynthesis data using convolutional neural networks and multiple instance learning. Comput Biol Med 96:283–293. https://doi.org/10.1016/j.compbiomed.2018.04.004

    Article  Google Scholar 

  • Zhang Q, Xiao Y, Dai W, Suo JF, Wang CZ, Shi J, Zheng HR (2016) Deep learning based classification of breast tumors with shear-wave elastography. Ultrasonics 72:150–157. https://doi.org/10.1016/j.ultras.2016.08.004

    Article  Google Scholar 

  • Zhang X, Zhang Y, Han EY, Jacobs N, Han Q, Wang X, Liu J (2017) Whole mammogram image classification with convolutional neural networks. Paper presented at the 2017 IEEE international conference on bioinformatics and biomedicine (BIBM)

  • Zheng Y, Jiang Z, Xie F, Zhang H, Ma Y, Shi H, Zhao Y (2017) Feature extraction from histopathological images based on nucleus-guided convolutional neural network for breast lesion classification. Pattern Recogn 71:14–25. https://doi.org/10.1016/j.patcog.2017.05.010

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by University Malaya Research Grant Program—AFR (Frontier Science) (RG380-17AFR).

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Information Technology, University of Malaya, 50603, Kuala Lumpur, Malaysia

    Ghulam Murtaza, Liyana Shuib, Ainuddin Wahid Abdul Wahab, Ghulam Mujtaba, Henry Friday Nweke & Fariha Zulfiqar

  2. Department of Computer Science, Sukkur IBA University, Sukkur, Pakistan

    Ghulam Murtaza

  3. Department of Computer Science, PAF-KIET, Karachi, Pakistan

    Ghulam Mujtaba

  4. Computer Science Department, Ebonyi State University, P.M.B 053, Abakaliki, Ebonyi State, Nigeria

    Henry Friday Nweke

  5. Qatar University, Ibn Khaldoon Hall, Doha, Qatar

    Mohammed Ali Al-garadi

  6. St. James’s Hospital, James’s Street, Ushers, Dublin 8, Ireland

    Ghulam Raza

  7. Diagnostic Imaging and Radiotherapy Programme, Faculty of Health Sciences Universiti Kebangsaan Malaysia, Jalan Raja Muda Abdul Aziz, 50300, Kuala Lumpur, Malaysia

    Nor Aniza Azmi

Authors
  1. Ghulam Murtaza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liyana Shuib
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ainuddin Wahid Abdul Wahab
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ghulam Mujtaba
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ghulam Mujtaba
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Henry Friday Nweke
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mohammed Ali Al-garadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Fariha Zulfiqar
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ghulam Raza
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Nor Aniza Azmi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ghulam Murtaza or Liyana Shuib.

Ethics declarations

Conflict of interest

The authors have no conflict of interests to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Appendix

Appendix

See Tables 11 and 12.

Table 11 List of questions used as quality evaluation criteria (QEC)
Full size table
Table 12 Quality evaluation criteria applied on 56 studies
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Murtaza, G., Shuib, L., Abdul Wahab, A.W. et al. Deep learning-based breast cancer classification through medical imaging modalities: state of the art and research challenges. Artif Intell Rev 53, 1655–1720 (2020). https://doi.org/10.1007/s10462-019-09716-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-019-09716-5

Keywords

Search

Navigation