Skip to main content
SpringerLink
Log in

Detection of mitotic HEp-2 cell images: role of feature representation and classification framework under class skew

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

We propose and analyze a framework to detect and identify the mitotic type staining patterns among different non-mitotic (interphase) patterns on HEp-2 cell substrate specimen images. This is considered as a principal task in computer-aided diagnosis (CAD) of the autoimmune disorders. Due to the rare appearance of mitotic patterns in whole slide/specimen images, the sample skew between mitotic and non-mitotic patterns is an important consideration.

We suggest to apply some effective samples skew balancing strategies for the task of classification between mitotic v/s interphase patterns. Another aspect of this study is to consider the morphology and texture-based differences between both the classes that can be incorporated through effective morphology and texture-based descriptors, including the Gabor and LM (Leung-Malik) filter banks and also through some contemporary filter banks derived from convolutional neural networks (CNN).

The proposed framework is evaluated on a public dataset and we demonstrate good performance (0.99 or 1 Matthews correlation coefficient (MCC) in many cases), across various experiments. The study also presents a comparison between hand-engineered and CNN-based feature representation, along with the comparisons with state-of-the-art approaches. Hence, the framework proves to be a good solution for the mentioned skewed classification problem.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Kumar Y, Bhatia A, Minz R (2009) Antinuclear antibodies and their detection methods in diagnosis of connective tissue diseases: A journey revisited. Diagn Pathol 4:1. https://doi.org/10.1186/1746-1596-4-1

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. NI of Health (U.S. Department of Health, H. Services) (2005) Progress in autoimmune diseases research, Tech Rep

  3. Foggia P, Percannella G, Soda P, Vento M (2013) Benchmarking HEp-2 cells classification methods. IEEE Trans Med Imaging 32(10):1878–1889. https://doi.org/10.1109/TMI.2013.2268163

    Article  PubMed  Google Scholar 

  4. Hobson P, Lovell BC, Percannella G, Saggese A, Vento M, Wiliem A (2016) Computer aided diagnosis for anti-nuclear antibodies HEp-2 images: Progress and challenges, Pattern Recognition Letters 82. Part 1:3–11. https://doi.org/10.1016/j.patrec.2016.06.013

    Article  Google Scholar 

  5. Tonti S, Di Cataldo S, Macii E, Ficarra E (2015) Unsupervised HEp-2 mitosis recognition in indirect immunofluorescence imaging. In: Proc IEEE Eng Med Biol Soc 8135–8138. https://doi.org/10.1109/EMBC.2015.7320282

  6. Paul A, Mukherjee DP (2015) Mitosis detection for invasive breast cancer grading in histopathological images. IEEE Trans Image Process 24:4041–4054. https://doi.org/10.1109/tip.2015.2460455

    Article  PubMed  Google Scholar 

  7. Miros A, Wiliem A, Holohan K, Ball L, Hobson P, Lovell BC (2015) A benchmarking platform for mitotic cell classification of ANA IIF HEp-2 images. In: Proc International Conference on Digital Image Computing: Techniques and Applications 1– 6. https://doi.org/10.1109/DICTA.2015.7371213

  8. Foggia P, Percannella G, Soda P, Vento M (2010) Early experiences in mitotic cells recognition on HEp-2 slides. In: Proc International Symposium on Computer Based Medical Systems 38–43. https://doi.org/10.1109/CBMS.2010.6042611

  9. Iannello G, Percannella G, Soda P, Vento M (2014) Mitotic cells recognition in HEp-2 images. Pattern Recogn Lett 45:136–144. https://doi.org/10.1016/j.patrec.2014.03.011

    Article  Google Scholar 

  10. Nguyen GH, Bouzerdoum A, Phung SL, (2009) Learning pattern classification tasks with imbalanced data sets. In: Pattern recognition, IntechOpen. https://doi.org/10.5772/7544

  11. Manivannan S, Li W, Akbar S, Wang R, Zhang J, McKenna SJ (2016) An automated pattern recognition system for classifying indirect immunofluorescence images of HEp-2 cells and specimens. Pattern Recognit 51:12–26. https://doi.org/10.1016/j.patcog.2015.09.015

    Article  Google Scholar 

  12. Fogel I, Sagi D (1989) Gabor filters as texture discriminator. Biol Cybern 61(2):103–113. https://doi.org/10.1007/BF00204594

    Article  Google Scholar 

  13. Leung T, Malik J (2001) Representing and recognizing the visual appearance of materials using three-dimensional textons. Int J Comput Vision 43(1):29–44. https://doi.org/10.1023/A:1011126920638

    Article  Google Scholar 

  14. Gupta K, Bhavsar A, Sao AK (2019) Detecting mitotic cells in HEp-2 images as anomalies via one class classifier. Comput Biol Med 111:103328. https://doi.org/10.1016/j.compbiomed.2019.103328

    Article  PubMed  Google Scholar 

  15. Gupta K, Bhavsar A, Sao AK (2018) CNN based mitotic HEp-2 cell image detection In: Proc International Joint Conference on Biomedical Engineering Systems and Technologies-Bioimaging 167–174. https://doi.org/10.5220/0006721501670174

  16. Ponomarev GV, Kazanov MD (2016) Classification of ANA HEp-2 slide images using morphological features of stained patterns. Pattern Recognit Lett 8(Part 1):79–84. https://doi.org/10.1016/j.patrec.2016.03.010

    Article  Google Scholar 

  17. Sarrafzadeh O, Rabbani H, Dehnavi AM, Talebi A (2016) Analyzing features by SWLDA for the classification of HEp-2 cell images using GMM. Pattern Recognit Lett 82(Part 1):44–55. https://doi.org/10.1016/j.patrec.2016.03.023

    Article  Google Scholar 

  18. Ensafi S, Lu S, Kassim AA, Tan CL (2016) Accurate HEp-2 cell classification based on sparse coding of superpixels, Pattern Recognition Letters 82. Part 1:64–71. https://doi.org/10.1016/j.patrec.2016.02.007

    Article  Google Scholar 

  19. Hobson P, Lovell BC, Percannella G, Vento M, Wiliem A (2015) Bench- marking human epithelial type 2 interphase cells classification methods on a very large dataset. Artif Intell Med 65(3):239–250. https://doi.org/10.1016/j.artmed.2015.08.001

    Article  PubMed  Google Scholar 

  20. Wiliem A, Hobson P, Lovell BC (2014) Discovering discriminative cell attributes for HEp-2 specimen image classification. In: Proc IEEE Winter Conference on Applications of Computer Vision 423–430. https://doi.org/10.1109/WACV.2014.6836071

  21. Gragnaniello D, Sansone C, Verdoliva L (2016) Cell image classification by a scale and rotation invariant dense local descrip- tor, Pattern Recognition Letters 82. Part 1:72–78. https://doi.org/10.1016/j.patrec.2016.01.007

    Article  Google Scholar 

  22. Perner P, Perner H, Miller B (2002) Mining knowledge for HEp-2 cell image classification. Artif Intell Med 26(1):161–173. https://doi.org/10.1016/S0933-3657(02)00057-X

    Article  PubMed  Google Scholar 

  23. Soda P, Iannello G (2009) Aggregation of classifiers for staining pattern recognition in antinuclear autoantibodies analysis. IEEE Trans Inf Technol Biomed 13(3):322–329. https://doi.org/10.1109/TITB.2008.2010855

    Article  PubMed  Google Scholar 

  24. Jia X, Shen L, Zhou X, Yu S (2016) Deep convolutional neural network based HEp-2 cell classification. In: Proc Int Conf Pattern Recognit 77–80. https://doi.org/10.1109/ICPR.2016.7899611

  25. BS D, Subramaniam K, HR N (2016) HEp-2 cell classification using artificial neural network approach. In: Proc Int Conf Pattern Recognit 84–89. https://doi.org/10.1109/ICPR.2016.7899613

  26. Foggia P, Percannella G, Saggese A, Vento M (2014) Pattern recognition in stained HEp-2 cells: Where are we now? Pattern Recogn 47(7):2305–2314. https://doi.org/10.1016/j.patcog.2014.01.010

    Article  Google Scholar 

  27. Wiliem A, Wong Y, Sanderson C, Hobson P, Chen S, Lovell BC (2013) Classification of human epithelial type 2 cell indirect immunofluoresence images via codebook based descriptors, arXiv preprint

  28. Hobson P, Lovell BC, Percannella G, Vento M, Wiliem A (2014) Classifying anti-nuclear antibodies HEp-2 images: A benchmarking platform. In: Proc International Conference on Pattern Recognition 3233–3238. https://doi.org/10.1109/ICPR.2014.557

  29. Gupta K, Gupta V, Bhavsar A, Sao AK (2015) Class-specific hierarchical classification for HEp-2 specimen images. In: Proc International Conference on Image Processing ICIP 641–645. https://doi.org/10.1109/ICIP.2015.7350877

  30. Li Y, Shen L, Yu S (2017) HEp-2 specimen image segmentation and classification using very deep fully convolutional network. IEEE Transactions on Medical Imaging (99):1561–1572. https://doi.org/10.1109/TMI.2017.2672702

  31. Phan HTH, Kumar A, Kim J, Feng D (2016) Transfer learning of a convolutional neural network for HEp-2 cell image classification. In: Proc IEEE International Symposium on Biomedical Imaging 1208–1211. https://doi.org/10.1109/ISBI.2016.7493483

  32. Roullier V, L´ezoray O, Thong TV, Elmoataz A (2011) Multi- resolution graph-based analysis of histopathological whole slide images: Application to mitotic cell extraction and visualization. Comput Med Imaging Graphics 35(7–8):603–615. https://doi.org/10.1016/j.compmedimag.2011.02.005

    Article  Google Scholar 

  33. Hobson P, Lovell BC, Percannella G, Saggese A, Vento M, Wiliem (2016) HEp-2 staining pattern recognition at cell and specimen levels: Datasets, algorithms and results, Pattern Recognition Letters 82, Part 1 12–22 Pattern Recognition Techniques for Indirect Immunofluorescence Images Analysis. https://doi.org/10.1016/j.patrec.2016.07.013

  34. Percannella G, Soda P, VentoM (2011) Mitotic HEp-2 cells recognition under class skew. In: Proc International Conference on Image Analysis and Processing 353–362. https://doi.org/10.1007/978-3-642-24088-137

  35. Gupta K, Thapar D, Bhavsar A, Sao AK (2019) Deep metric learning for identification of mitotic patterns of HEp-2 cell images, in: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. https://doi.org/10.1109/CVPRW.2019.00141

  36. Gupta K, Bhavsar A, Sao AK (2020) Identification of HEp-2 specimen images with mitotic cell patterns. Biocybern Biomed Eng 40(3):1233–1249. https://doi.org/10.1016/j.bbe.2020.07.003

    Article  Google Scholar 

  37. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: Synthetic minority over-sampling technique. J Artif Int Res 16(1):321–357

    Google Scholar 

  38. Zhu X, Yang Y (2008) A lazy bagging approach to classification. Pattern Recogn 41(10):2980–2992. https://doi.org/10.1016/j.patcog.2008.03.008

    Article  Google Scholar 

  39. Cao P, Zhao D, Zaiane O (2013) An optimized cost-sensitive SVM for imbalanced data learning. In: Advances in Knowledge Discovery and Data Mining 280–292. https://doi.org/10.1007/978-3-642-37456-2_24

  40. Chazotte B (2011) Labeling nuclear DNA using DAPI, Cold Spring Harbor Protocols (1) pdb–prot5556. https://doi.org/10.1101/pdb.prot5556

  41. Gupta K, Bhavsar A, Sao AK (2018) Mitotic cells detection for HEp-2 specimen images using threshold-based evaluation scheme. In: Proc SPIE Med Imaging 10581–10589. https://doi.org/10.1117/12.2293524

  42. Feng Y, Yang F, Zhou X, Guo Y, Tang F, Ren F, Guo J, Ji S (2019) A deep learning approach for targeted contrast-enhanced ultrasound based prostate cancer detection. IEEE/ACM Trans Comput Biol Bioinf 16(6):1794–1801

    Article  Google Scholar 

  43. Hussain E, Mahanta LB, Das CR, Choudhury M, Chowdhury M (2020) A shape context fully convolutional neural network for segmentation and classification of cervical nuclei in pap smear images. Artif Intell Med 107:101897. https://doi.org/10.1016/j.artmed.2020.101897

    Article  PubMed  Google Scholar 

  44. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Proc Advances in Neural Information Processing Systems 1097–1105. https://doi.org/10.1145/3065386

  45. Yang J, Jiang Y-G, Hauptmann AG, Ngo C-W (2007) Evaluating bag-of- visual-words representations in scene classification. In: Proc International Workshop on Multimedia Information Retrieval 197–206. https://doi.org/10.1145/1290082.1290111

  46. Chang C-C, Lin C-J (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27. https://doi.org/10.1145/1961189.1961199

  47. Garc´ıa V, Mollineda RA, S´anchez JS (2009) Index of balanced accuracy: A performance measure for skewed class distributions. In: Proc Pattern Recognition and Image Analysis 441–448. https://doi.org/10.1007/978-3-642-02172-557

  48. Boughorbel S, Jarray F, El-Anbari M (2017) Optimal classifier for imbalanced data using Matthews correlation coefficient metric. PLoS ONE 12(6):1–17. https://doi.org/10.1371/journal.pone.0177678

    Article  CAS  Google Scholar 

  49. Van der Maaten L, Hinton G (2008) Visualizing high-dimensional data using t-SNE. J Mach Learn Res 9:2579–2605

    Google Scholar 

  50. Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In: Proc IEEE Conference on Computer Vision and Pattern Recognition 1717–1724. https://doi.org/10.1109/CVPR.2014.222

  51. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large scale image recognition. In: Proc International Conference on Learning Representations

  52. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proc IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778. https://doi.org/10.1109/CVPR.2016.90

Download references

Author information

Authors and Affiliations

  1. School of Computing & Electrical Engineering, Indian Institute of Technology Mandi, Mandi, India

    Krati Gupta, Arnav Bhavsar & Anil K. Sao

Authors
  1. Krati Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arnav Bhavsar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anil K. Sao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Krati Gupta.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gupta, K., Bhavsar, A. & Sao, A.K. Detection of mitotic HEp-2 cell images: role of feature representation and classification framework under class skew. Med Biol Eng Comput 60, 2405–2421 (2022). https://doi.org/10.1007/s11517-022-02613-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02613-0

Keywords

Search

Navigation