Abstract
It is generally believed that the human visual system is biased towards the recognition of shapes rather than textures. This assumption has led to a growing body of work aiming to align deep models’ decision-making processes with the fundamental properties of human vision. The reliance on shape features is primarily expected to improve the robustness of these models under covariate shift. In this paper, we revisit the significance of shape-biases for the classification of skin lesion images. Our analysis shows that different skin lesion datasets exhibit varying biases towards individual image features. Interestingly, despite deep feature extractors being inclined towards learning entangled features for skin lesion classification, individual features can still be decoded from this entangled representation. This indicates that these features are still represented in the learnt embedding spaces of the models, but not used for classification. In addition, the spectral analysis of different datasets shows that in contrast to common visual recognition, dermoscopic skin lesion classification, by nature, is reliant on complex feature combinations beyond shape-bias. As a natural consequence, shifting away from the prevalent desire of shape-biasing models can even improve skin lesion classifiers in some cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
BA-Transfomer architecture proposed by Wang et al. [30], trained on ISIC2016-2018 challenge datasets.
- 2.
- 3.
Reproducible code available on GitHub https://github.com/adriano-lucieri/shape-bias-in-dermoscopy.
- 4.
References
Argenziano, G., Fabbrocini, G., Carli, P., De Giorgi, V., Sammarco, E., Delfino, M.: Epiluminescence microscopy for the diagnosis of doubtful melanocytic skin lesions: comparison of the ABCD rule of dermatoscopy and a new 7-point checklist based on pattern analysis. Arch. Dermatol. 134(12), 1563–1570 (1998)
Barata, C., Ruela, M., Mendonça, T., Marques, J.S.: A bag-of-features approach for the classification of melanomas in dermoscopy images: The role of color and texture descriptors. In: Scharcanski, J., Celebi, M. (eds.) Computer Vision Techniques for the Diagnosis of Skin Cancer, pp. 49–69. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-39608-3_3
Betta, G., Di Leo, G., Fabbrocini, G., Paolillo, A., Sommella, P.: Dermoscopic image-analysis system: estimation of atypical pigment network and atypical vascular pattern. In: IEEE International Workshop on Medical Measurement and Applications, 2006. MeMea 2006. pp. 63–67. IEEE (2006)
Cassidy, B., Kendrick, C., Brodzicki, A., Jaworek-Korjakowska, J., Yap, M.H.: Analysis of the ISIC image datasets: usage, benchmarks and recommendations. Med. Image Anal. 75, 102305 (2022)
Chen, G., Peng, P., Ma, L., Li, J., Du, L., Tian, Y.: Amplitude-phase recombination: Rethinking robustness of convolutional neural networks in frequency domain. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 458–467 (2021)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Dinnes, J., et al.: Visual inspection and dermoscopy, alone or in combination, for diagnosing keratinocyte skin cancers in adults. Cochrane Database Syst. Rev. 12(12), CD011901 (2018)
Gachon, J., Beaulieu, P., Sei, J.F., Gouvernet, J., Claudel, J.P., Lemaitre, M., Richard, M.A., Grob, J.J.: First prospective study of the recognition process of melanoma in dermatological practice. Arch. Dermatol. 141(4), 434–438 (2005)
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F.A., Brendel, W.: Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231 (2018)
Geman, S., Bienenstock, E., Doursat, R.: Neural networks and the bias/variance dilemma. Neural Comput. 4(1), 1–58 (1992)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems 27 (2014)
Hasan, M.K., Elahi, M.T.E., Alam, M.A., Jawad, M.T., Martí, R.: Dermoexpert: skin lesion classification using a hybrid convolutional neural network through segmentation, transfer learning, and augmentation. Informatics in Medicine Unlocked, p. 100819 (2022)
Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., Madry, A.: Adversarial examples are not bugs, they are features. In: Advances in Neural Information Processing Systems 32 (2019)
Kawahara, J., Daneshvar, S., Argenziano, G., Hamarneh, G.: Seven-point checklist and skin lesion classification using multitask multimodal neural nets. IEEE J. Biomed. Health Inf. 23(2), 538–546 (2019). https://doi.org/10.1109/JBHI.2018.2824327
Kirichenko, P., Izmailov, P., Wilson, A.G.: Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint arXiv:2204.02937 (2022)
Kittler, H., Rosendahl, C., Cameron, A., Tschandl, P.: Dermatoscopy: an algorithmic method based on pattern analysis. facultas (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25 (2012)
Kundu, R.V., Patterson, S.: Dermatologic conditions in skin of color: part i. special considerations for common skin disorders. Am. Family Phys. 87(12), 850–856 (2013)
López-Leyva, J.A., Guerra-Rosas, E., Álvarez-Borrego, J.: Multi-class diagnosis of skin lesions using the fourier spectral information of images on additive color model by artificial neural network. IEEE Access 9, 35207–35216 (2021)
Marques, J.S., Barata, C., Mendonça, T.: On the role of texture and color in the classification of dermoscopy images. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4402–4405. IEEE (2012)
Menzies, S.W., Ingvar, C., Crotty, K.A., McCarthy, W.H.: Frequency and morphologic characteristics of invasive melanomas lacking specific surface microscopic features. Arch. Dermatol. 132(10), 1178–1182 (1996)
Norman, G., Barraclough, K., Dolovich, L., Price, D.: Iterative diagnosis. Bmj 339 (2009)
Quiñonero-Candela, J., Sugiyama, M., Schwaighofer, A., Lawrence, N.D.: Dataset Shift in Machine Learning. Mit Press (2008)
Ruela, M., Barata, C., Mendonca, T., Marques, J.S.: On the role of shape in the detection of melanomas. In: 2013 8th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 268–273. IEEE (2013)
Ruela, M., Barata, C., Mendonça, T., Marques, J.S.: What is the role of color in dermoscopy analysis? In: Sanches, J.M., Micó, L., Cardoso, J.S. (eds.) IbPRIA 2013. LNCS, vol. 7887, pp. 819–826. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38628-2_97
Stolz, W.: Abcd rule of dermatoscopy: a new practical method for early recognition of malignant melanoma. Eur. J. Dermatol. 4, 521–527 (1994)
Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27 (2014)
Szegedy, C., et al.: Intriguing properties of neural networks. In: International Conference on Learning Representations (2014). http://arxiv.org/abs/1312.6199
Tang, P., Yan, X., Nan, Y., Xiang, S., Krammer, S., Lasser, T.: Fusionm4net: a multi-stage multi-modal learning algorithm for multi-label skin lesion classification. Med. Image Anal. 76, 102307 (2022)
Wang, J., Wei, L., Wang, L., Zhou, Q., Zhu, L., Qin, J.: Boundary-aware transformers for skin lesion segmentation. In: de Bruijne, M., Cattin, P.C., Cotin, S., Padoy, N., Speidel, S., Zheng, Y., Essert, C. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 206–216. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_20
Wang, Z., Yang, Y., Shrivastava, A., Rawal, V., Ding, Z.: Towards frequency-based explanation for robust CNN. arXiv preprint arXiv:2005.03141 (2020)
Wen, D., et al.: Characteristics of publicly available skin cancer image datasets: a systematic review. The Lancet Digital Health (2021)
Xu, Q., Zhang, R., Zhang, Y., Wang, Y., Tian, Q.: A fourier-based framework for domain generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14383–14392 (2021)
Yin, D., Gontijo Lopes, R., Shlens, J., Cubuk, E.D., Gilmer, J.: A fourier perspective on model robustness in computer vision. In: Advances in Neural Information Processing Systems 32 (2019)
Zalaudek, I., et al.: Time required for a complete skin examination with and without dermoscopy: a prospective, randomized multicenter study. Arch. Dermatol. 144(4), 509–513 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lucieri, A., Schmeisser, F., Balada, C.P., Siddiqui, S.A., Dengel, A., Ahmed, S. (2022). Revisiting the Shape-Bias of Deep Learning for Dermoscopic Skin Lesion Classification. In: Yang, G., Aviles-Rivero, A., Roberts, M., Schönlieb, CB. (eds) Medical Image Understanding and Analysis. MIUA 2022. Lecture Notes in Computer Science, vol 13413. Springer, Cham. https://doi.org/10.1007/978-3-031-12053-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-12053-4_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-12052-7
Online ISBN: 978-3-031-12053-4
eBook Packages: Computer ScienceComputer Science (R0)