MVS-GS: High-Quality 3D Gaussian Splatting Mapping via Online Multi-View Stereo

Abstract

This study addresses the challenge of online 3D model generation for neural rendering using an RGB image stream. Previous research has tackled this issue by incorporating Neural Radiance Fields (NeRF) or 3D Gaussian Splatting (3DGS) as scene representations within dense SLAM methods. However, most studies focus primarily on estimating coarse 3D scenes rather than achieving detailed reconstructions. Moreover, depth estimation based solely on images is often ambiguous, resulting in low-quality 3D models that lead to inaccurate renderings. To overcome these limitations, we propose a novel framework for high-quality 3DGS modeling that leverages an online multi-view stereo (MVS) approach. Our method estimates MVS depth using sequential frames from a local time window and applies comprehensive depth refinement techniques to filter out outliers. The refinement method produces temporally consistent depths by checking sequential geometric consistency, enabling accurate initialization of Gaussians in 3DGS. Furthermore, we introduce a parallelized backend module that optimizes the 3DGS model efficiently, ensuring timely updates with each new keyframe. Experimental results demonstrate that our method outperforms state-of-the-art dense SLAM methods, achieving an average PSNR improvement of approximately 2 dB on indoor scenes. Moreover, our method reliably produces consistent 3D models in complex outdoor scenes, where existing methods often fail due to tracking errors and depth noise. It also reconstructs large-scale aerial scenes effectively, achieving an average PSNR gain of about 10.28 dB over existing methods.