跨视角泼溅:基于地理参考图像的前馈视角合成
Cross-View Splatter: Feed-Forward View Synthesis with Georeferenced Images
把正射卫星影像当作全局几何先验,与 GPS 标注的地面照片在统一三维坐标系下对齐特征并前馈预测像素对齐高斯泼溅,为稀疏地面覆盖的户外大场景新视角合成提供了可复用的跨视角融合范式。
原文依据 fuses orthorectified satellite views with GPS-tagged ground photos to predict Gaussian splats in a unified 3D coordinate frame
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出跨视角泼溅(Cross-View Splatter),一种前馈方法,为地面与卫星拍摄的户外场景预测像素对齐的高斯泼溅。忠实的重建需要良好的相机覆盖,但对于大型户外场景,地面影像采集耗时且难以规模化获取。幸运的是,卫星影像可以提供易于通过公共 API 获取的全局几何先验。跨视角泼溅将正射校正后的卫星视图与带 GPS 标签的地面照片融合,在统一的三维坐标系中预测高斯泼溅。通过对齐地面与鸟瞰特征表示,与仅使用地面影像相比,我们的模型提升了场景覆盖与新视角合成效果。我们在精心整理的地理参考数据集以及从开放地图服务中挖掘的卫星—地形配对数据上进行训练。我们在一个新的地理参考影像新视角合成基准上评估本方法,从而可以与先前的最优方法进行比较。我们的代码与数据准备流程将发布于 https://nianticspatial.github.io/cross-view-splatter/。
英文原文摘要 1083 字符
We present Cross-View Splatter, a feed-forward method that predicts pixel-aligned Gaussian splats for outdoor scenes captured at ground level and by satellite. Faithful reconstructions require good camera coverage, but ground imagery is time-consuming and hard to capture at scale for large outdoor scenes. Fortunately, satellite imagery can provide a global geometric prior that is easy to access via public APIs. Cross-View Splatter fuses orthorectified satellite views with GPS-tagged ground photos to predict Gaussian splats in a unified 3D coordinate frame. By aligning ground and bird's-eye feature representations, our model improves scene coverage and novel-view synthesis, compared to ground imagery alone. We train on curated georeferenced datasets and paired satellite-terrain data, mined from open mapping services. We evaluate our method on a new benchmark for novel-view synthesis with georeferenced imagery, allowing comparison to prior state-of-the-art methods. Our code and data preparation will be available at https://nianticspatial.github.io/cross-view-splatter/.