Inferring semantic concepts from community-contributed images and noisy tags

In this paper, we exploit the problem of inferring images' semantic concepts from community-contributed images and their associated noisy tags. To infer the concepts more accurately, we propose a novel sparse graph-based semi-supervised learning approach for harnessing the labeled and unlabeled data simultaneously. The sparse graph constructed by datum-wise one-vs-all sparse reconstructions of all samples can remove most of the concept-unrelated links among the data, thus is more robust and discriminative than conventional graphs. More importantly, we propose an effective training label refinement strategy within this graph-based learning framework to handle the noise in the tags, by bringing in a dual regularization for both the quantity and sparsity of the noise. In addition, we construct an informative compact concept space with small semantic gap to infer the semantic concepts in this space to bridge the semantic gap. The relations among different concepts are inherently embedded in this space to help the concept inference. We conduct extensive experiments on a real-world community-contributed image database consisting of 55,615 Flickr images and associated tags. The results demonstrate the effectiveness of the proposed approaches and the capability of our method to deal with the noise in the tags. We further show that we could achieve comparable performance by inferring semantic concepts from training data with noisy tags versus training data with clean ground-truth labels.