Text-guided diffusion-based restoration of extremely compressed backgrounds for VCM

초록

Restoring high-quality images from severely degraded inputs is essential for video coding for machines (VCM), where background regions are compressed at extremely low bitrates. In this letter, we propose a novel text-guided diffusion-based restoration (TGDR) algorithm, which integrates semantic information from text captions to guide the restoration process. Specifically, we develop a refinement block that incorporates a transformer-based time-aware feature extractor to fuse visual features, time-step embeddings, and textual semantics adaptively to guide a pretrained diffusion model during the reverse denoising process. By incorporating both visual and textual information, TGDR effectively reconstructs complex structures and improves semantic consistency in highly compressed regions. Experimental results show that TGDR achieves superior performance compared to state-of-the-art algorithms.

키워드