Abstract
Self-organizing maps (SOMs) are well appropriate for visualizing high-dimensional data sets. Training SOMs on raw high-dimensional data with classic metrics often leads to problems arising from the curse-of-dimensionality effect. To achieve more valuable semantic maps of high-dimensional data sets, we assume that higher-level features are necessary. We propose to gather such higher-level features from pre-trained convolutional layers, i.e., filter banks of convolutional neural networks (CNNs). Appropriately pre-trained CNNs are required, e.g., from the same or related domains, or in semi-supervised scenarios. We introduce SOM quality measures and analyze the new approach on two benchmark image data sets considering different convolutional network levels.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Also rectangle shapes are possible.
- 2.
MUX and quality measures are currently not used for the training process itself.
- 3.
The distance matrix contains the pairwise distances between patterns respectively positions on the map.
- 4.
The weights of the neurons cannot be used directly for visualization in the ConvSOM as they have a different format.
References
András P (2002) Kernel-kohonen networks. Int J Neural Syst 12(02):117–135. https://doi.org/10.1142/S0129065702001084
Coelho, D.N., Barreto, G.A., Medeiros, C.M.S.: Detection of short circuit faults in 3-phase converter-fed induction motors using kernel SOMs. In: WSOM, pp 1–7 (2017). https://doi.org/10.1109/WSOM.2017.8020016
Dozono H, Niina G, Araki S (2016) Convolutional self organizing map. In: 2016 international conference on computational science and computational intelligence (CSCI), pp 767–771. https://doi.org/10.1109/CSCI.2016.0149
Ferles C, Papanikolaou Y, Naidoo KJ (2018) Denoising autoencoder self-organizing map (DASOM). Neural Netw 105:112–131. https://doi.org/10.1016/j.neunet.2018.04.016
Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43(1):59–69. https://doi.org/10.1007/BF00337288
Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84–90. https://doi.org/10.1145/3065386
Kruskal JB (1964) Nonmetric multidimensional scaling: a numerical method. Psychometrika 29(2):115–129. https://doi.org/10.1007/BF02289694
Kutics A, O’Connell C, Nakagawa A (2013) Segment-based image classification using layered-SOM. In: 2013 IEEE international conference on image processing, pp 2430–2434. https://doi.org/10.1109/ICIP.2013.6738501
LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551. https://doi.org/10.1162/neco.1989.1.4.541
MacDonald D, Fyfe C (2000) The kernel self-organising map. In: KES 2000, fourth international conference on knowledge-based intelligent engineering systems and allied technologies, proceedings (Cat. No. 00TH8516), vol 1, pp 317–320. https://doi.org/10.1109/KES.2000.885820
Meier A, Kramer O (2017) An experimental study of dimensionality reduction methods. In: Kern-Isberner G, Fürnkranz J, Thimm M (eds) KI 2017, advances in artificial intelligence. Springer International Publishing, pp 178–192. https://doi.org/10.1007/978-3-319-67190-1_14
Miikkulainen R (1990) Script recognition with hierarchical feature maps. Connection Sci 2(1–2):83–101. https://doi.org/10.1080/09540099008915664
Platon L, Zehraoui F, Tahi F (2017) Self-organizing maps with supervised layer. In: WSOM, pp 1–8. https://doi.org/10.1109/WSOM.2017.8020022
Schleif F (2017) Small sets of random Fourier features by kernelized matrix LVQ. In: WSOM, pp 1–5. https://doi.org/10.1109/WSOM.2017.8020026
Villmann T, Biehl M, Villmann A, Saralajew S (2017) Fusion of deep learning architectures, multilayer feedforward networks and learning vector quantizers for deep classification learning. In: 2017 12th international workshop on self-organizing maps and learning vector quantization, clustering and data visualization (WSOM), pp 1–8. https://doi.org/10.1109/WSOM.2017.8020009
Wang M, Zhou W, Tian Q, Pu J, Li H (2017) Deep supervised quantization by self-organizing map. In: Proceedings of the 25th ACM international conference on multimedia, MM 2017. ACM, New York, pp 1707–1715. https://doi.org/10.1145/3123266.3123415
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Elend, L., Kramer, O. (2020). Self-Organizing Maps with Convolutional Layers. In: Vellido, A., Gibert, K., Angulo, C., MartÃn Guerrero, J. (eds) Advances in Self-Organizing Maps, Learning Vector Quantization, Clustering and Data Visualization. WSOM 2019. Advances in Intelligent Systems and Computing, vol 976. Springer, Cham. https://doi.org/10.1007/978-3-030-19642-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-19642-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-19641-7
Online ISBN: 978-3-030-19642-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)