Supervising the Unseen: Pseudo-View Depth and Normal Priors for Sparse-View Gaussian Splatting

초록

Sparse-view novel view synthesis with 3D Gaussian Splatting (3DGS) is severely under-constrained: with only a handful of input images, the optimized Gaussians overfit the training views and produce blurry, geometrically inconsistent renderings at unseen viewpoints. Our method builds on an empirical observation: pretrained monocular geometric foundation models, which have learned strong priors from large-scale real-world images, can predict plausible depth and surface-normal maps even for the imperfect images rendered at novel views by a sparse-view 3DGS model. Based on this observation, we design a model-agnostic refinement stage that can be applied to any pretrained sparse-view 3DGS reconstruction. Starting from a converged sparse-view model, we synthesize a pool of pseudo views around the input cameras. A monocular depth-and-normal foundation model is then queried on both the ground-truth training images and the images rendered at the pseudo views, producing pseudo-geometric targets. The Gaussians are then refined with a scale-invariant ordinal-depth loss and a confidence-weighted normal-consistency loss that aligns normals derived analytically from the differentiably rendered depth with the surface normals predicted by the frozen foundation model. When applied to a representative sparse-view backbone, our method achieves consistent gains of roughly +0.3–0.5 dB PSNR and 4–12% relative LPIPS over the backbone across all 3/6/12-view settings on the Tanks and Temples and MVImgNet benchmarks. Among the compared specialized sparse-view baselines it ranks first in the hardest 3- and 6-view settings and remains competitive at 12 views. Ablation studies confirm that both the depth and the normal priors, and their application to rendered novel views, contribute to the final quality. © 2013 IEEE.

키워드