멀티모달 특징 분해 트랜스포머 기반 적외선 및 가시광선 영상 합성 기법

초록

We propose an infrared and visible image fusion algorithm using common and complementary multi-modal featuredecomposition transformers. First, the proposed algorithm extracts multiscale shallow features from infrared and visible images. Then, we develop common and complementary feature decomposition transformers, which decompose the features into common andcomplementary components for each modality. For better decomposition, we develop a decomposition loss by constraining thecommon features to be correlated, while the complementary features are uncorrelated. Finally, the fusion block generates the fusedimage by combining the common and complementary features. Experimental results on a public dataset demonstrate that theproposed algorithm outperforms conventional algorithms in quantitative and qualitative comparison.

키워드