Distilling and Refining Domain-Specific Knowledge for Semi-Supervised Domain Adaptation

Abstract

We propose a novel framework, Distilling And Refining domain-specific Knowledge (DARK), for Semi-supervised Domain Adaptation (SSDA) tasks. The proposed method consists of three strategies: Multi-view Learning, Distilling, and Refining. In Multi-view Learning, to acquire domain-specific knowledge, DARK trains a shared generator and two domain-specific classifiers using the labeled source and target data. Then, in Distilling, two classifiers exchange the domain-specific knowledge with each other to exploit a cross-view consistency regularization using soft labels between differently augmented unlabeled target samples. During this, DARK leverages information from low-confidence unlabeled target samples in addition to the high-confidence unlabeled target samples. To prevent a trivial collapse problem caused by the low-confidence samples, we propose the utilization of a sample-wise dynamic weight based on prediction reliability (SDWR). Finally, in Refining, for class alignment, class confusion of the unlabeled target data is minimized considering the model maturity. Simultaneously, to maintain model consistency between the predictions of differently augmented unlabeled target samples, a bridging loss with SDWR is used. Consequently, the experimental results on the SSDA datasets demonstrate that DARK outperforms the state-of-the-art benchmark methods for SSDA tasks. The code can be found at https://github.com/Juh-yun/DARK. © 2022. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.