Heterogeneous Data Integration using Confidence Estimation of Unseen Visual Data for Zero-shot Learning

Abstract

Zero-shot learning is a learning methodology that can be used to recognize concepts that have never been seen during the training phase. Recently, interest in zero-shot learning has been increased by embedding multi-modal data into common vector space through heterogeneous data integration methodology. However, since the existing methodologies compare heterogeneous data focusing on the similarity between each vector, the performance of zero-shot learning decreases when the number of semantic candidates increases. We propose a heterogeneous data integration methodology using a confidence estimator for unseen visual data which estimates that whether input data is unseen data or not and output confidence measure. The proposed methodology constructs a more efficient zero-shot learning model by applying estimated confidence of input unseen visual data to the visual-semantic distance obtained from heterogeneous data integration model. Experiments have shown that the proposed methodology can improve zero-shot learning performance for unseen data despite a small performance decrease in the seen data.