Abstract
Histopathology image analysis can be considered as a Multiple instance learning (MIL) problem, where the whole slide histopathology image (WSI) is regarded as a bag of instances (i.e., patches) and the task is to predict a single class label to the WSI. However, in many real-life applications such as computational pathology, discovering the key instances that trigger the bag label is of great interest because it provides reasons for the decision made by the system. In this paper, we propose a deep convolutional neural network (CNN) model that addresses the primary task of a bag classification on a histopathology image and also learns to identify the response of each instance to provide interpretable results to the final prediction. We incorporate the attention mechanism into the proposed model to operate the transformation of instances and learn attention weights to allow us to find key patches. To perform a balanced training, we introduce adaptive weighing in each training bag to explicitly adjust the weight distribution in order to concentrate more on the contribution of hard samples. Based on the learned attention weights, we further develop a solution to boost the classification performance by generating the bags with hard negative instances. We conduct extensive experiments on colon and breast cancer histopathology data and show that our framework achieves state-of-the-art performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bejnordi, B.E., et al.: Deep learning-based assessment of tumor-associated stroma for diagnosing breast cancer in histopathology images. In: ISBI (2017)
Couture, H.D., Marron, J.S., Perou, C.M., Troester, M.A., Niethammer, M.: Multiple instance learning for heterogeneous images: training a CNN for histopathology. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 254–262. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_29
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR (2005)
Han, Y., et al.: Avoiding false positive in multi-instance learning. In: NIPS (2010)
Hou, L., et al.: Patch-based convolutional neural network for whole slide tissue image classification. In: CVPR (2016)
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: ICML (2018)
Kandemir, M., Zhang, C., Hamprecht, F.A.: Empowering multiple instance histopathology cancer diagnosis by cell graphs. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8674, pp. 228–235. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10470-6_29
Wu, L., Wang, Y., Gao, J., Li, X.: Where-and-when to look: deep Siamese attention networks for video-based person re-identification. IEEE Trans. Multimed. (2019)
Lin, W., Wang, Y., Li, X., Gao, J.: Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Trans. Cybern. 49(5), 1791–1802 (2019)
Kraus, O.Z., Ba, J.L., Frey, B.J.: Classifying and segmenting microscopy images with deep multiple instance learning. In: Bioinformatics (2016)
Liu, G., Wu, J., Zhou, Z.-H.: Key instance detection in multi-instance learning. In: ACML (2012)
Otsu, N.: A threshold selection method from gray-level histograms. In: SMCS (1979)
Pappas, N., Popescu-Belis, A.: Explaining the stars: weighted multiple-instance learning for aspect-based sentiment analysis. In: EMNLP (2014)
Pappas, N., Popescu-Belis, A.: Explicit document modeling through weighted multiple-instance learning. In: JAIR (2017)
Shrivastava, A., Gupta, A., Girshick, R.: Training region-based object detectors with online hard example mining. In: CVPR (2016)
Sirinukunwattana, K., et al.: Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. In: T-MI (2016)
Song, Y., Li, Q., Huang, H., Feng, D., Chen, M., Cai, W.: Low dimensional representation of fisher vectors for microscopy image classification. In: T-MI (2017)
Sun, M., Han, T.X., Liu, M.C., Khodayari-Rostamabad, A.: Multiple instance learning convolutional neural networks for object recognition. In: ICPR (2016)
Wang, D., et al.: Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016)
Xu, Y., et al.: Multiple clustered instance learning for histopathology cancer image classification, segmentation and clustering. In: CVPR, June 2012
Xu, Y., et al.: Deep learning of feature representation with multiple instance learning for medical image analysis. In: ICASSP (2014)
Wu, L., Wang, Y., Gao, J., Li, X.: Deep adaptive feature embedding with local sample distributions for person re-identification. Pattern Recogn. 73, 275–288 (2018)
Lin, W., Wang, Y., Li, X., Gao, J.: Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Trans. Cybern. 49(5), 1791–1802 (2019)
Wu, L., Wang, Y., Shao, L.: Cycle-consistent deep generative hashing for cross-modal retrieval. IEEE Trans. Image Process. (2019)
Acknowledgement
This research was funded by the Australian Government through the Australian Research Council and Sullivan Nicolaides Pathology under Linkage Project LP160101797.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, M., Wu, L., Wiliem, A., Zhao, K., Zhang, T., Lovell, B. (2019). Deep Instance-Level Hard Negative Mining Model for Histopathology Images. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11764. Springer, Cham. https://doi.org/10.1007/978-3-030-32239-7_57
Download citation
DOI: https://doi.org/10.1007/978-3-030-32239-7_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32238-0
Online ISBN: 978-3-030-32239-7
eBook Packages: Computer ScienceComputer Science (R0)
