Morpho-Mnist: Quantitative assessment and diagnostics for representation learning
Open access
Date
2019Type
- Journal Article
Abstract
Revealing latent structure in data is an active field of research, having introduced excitingtechnologies such as variational autoencoders and adversarial networks, and is essentialto push machine learning towards unsupervised knowledge discovery. However, a majorchallenge is the lack of suitable benchmarks for an objective and quantitative evaluation oflearned representations. To address this issue we introduce Morpho-MNIST, a frameworkthat aims to answer: “to what extent has my model learned to represent specific factors ofvariation in the data?” We extend the popular MNIST dataset by adding a morphometricanalysis enabling quantitative comparison of trained models, identification of the rolesof latent variables, and characterisation of sample diversity. We further propose a setof quantifiable perturbations to assess the performance of unsupervised and supervisedmethods on challenging tasks such as outlier detection and domain adaptation. Data andcode are available athttps://github.com/dccastro/Morpho-MNIST. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000391770Publication status
publishedExternal links
Journal / series
Journal of Machine Learning ResearchVolume
Pages / Article No.
Publisher
MIT PressSubject
representation learning; generative models; empirical evaluation; disentanglement; morphometricsOrganisational unit
09579 - Konukoglu, Ender / Konukoglu, Ender
More
Show all metadata