Deep Learning Approach to Video Frame Rate Up-Conversion Using Bilateral Motion Estimation

Abstract

We propose a deep learning-based frame rate up-conversion algorithm using bilateral motion estimation. We first estimate bilateral motion fields by employing a convolutional neural network. Also, we approximate intermediate bi-directional motion fields, assuming linear motions between successive frames. Finally, we develop the synthesis network to produce an intermediate frame by merging the warped frames, which are obtained using the two kinds of motion fields. Experimental results demonstrate that the proposed algorithm generates high-quality intermediate frames on challenging sequences with large motions and occlusion, and outperforms state-of-the-art conventional algorithms.