DirectFisheye-GS：通过跨视角联合优化在高斯泼溅中实现原生鱼眼输入
DirectFisheye-GS: Enabling Native Fisheye Input in Gaussian Splatting with Cross-View Joint Optimization
将鱼眼相机模型直接集成进 3DGS 光栅化流程，避免去畸变预处理带来的信息损失与过拟合；其基于特征重叠的跨视角联合优化策略可迁移到现有针孔相机 3DGS 管线，用于抑制边缘浮空伪影。
原文依据 we integrate fisheye camera model into the original 3DGS framework, enabling native fisheye image input for training without preprocessing
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论3D 高斯泼溅（3DGS）使得从日常图像进行高效三维场景重建成为可能，并具备实时、高保真的渲染能力，极大推动了 VR/AR 应用的发展。鱼眼相机凭借更宽的视场角（FOV），有望以更少的输入实现高质量重建，近期备受关注。然而，由于 3DGS 依赖光栅化，多数涉及鱼眼相机输入的后续工作会先对图像去畸变再训练，这带来两个问题：1）图像边缘的黑边导致信息损失，抵消了鱼眼大视场的优势；2）去畸变的拉伸-插值重采样将每个像素值扩散到更大区域，稀释了细节密度，导致 3DGS 过拟合这些低频区域，产生模糊和浮空伪影。在本工作中，我们将鱼眼相机模型集成到原始 3DGS 框架中，无需预处理即可实现原生鱼眼图像输入训练。尽管建模正确，我们观察到重建场景在图像边缘仍会出现浮空物：畸变向周边递增，而 3DGS 原有的每轮随机选视角优化忽略了高斯跨视角的相关性，导致极端形状（如过大或拉长），从而降低重建质量。为此，我们引入一种特征重叠驱动的跨视角联合优化策略，在多个视角间建立一致的几何与光度约束——该技术同样适用于现有的基于针孔相机的管线。我们的 DirectFisheye-GS 在公开数据集上达到或超越最优性能。
英文原文摘要 1673 字符
3D Gaussian Splatting (3DGS) has enabled efficient 3D scene reconstruction from everyday images with real-time, high-fidelity rendering, greatly advancing VR/AR applications. Fisheye cameras, with their wider field of view (FOV), promise high-quality reconstructions from fewer inputs and have recently attracted much attention. However, since 3DGS relies on rasterization, most subsequent works involving fisheye camera inputs first undistort images before training, which introduces two problems: 1) Black borders at image edges cause information loss and negate the fisheye's large FOV advantage; 2) Undistortion's stretch-and-interpolate resampling spreads each pixel's value over a larger area, diluting detail density-- causes 3DGS overfitting these low-frequency zones, producing blur and floating artifacts. In this work, we integrate fisheye camera model into the original 3DGS framework, enabling native fisheye image input for training without preprocessing. Despite correct modeling, we observed that the reconstructed scenes still exhibit floaters at image edges: Distortion increases toward the periphery, and 3DGS's original per-iteration random-selecting-view optimization ignores the cross-view correlations of a Gaussian, leading to extreme shapes (e.g., oversized or elongated) that degrade reconstruction quality. To address this, we introduce a feature-overlap-driven cross-view joint optimization strategy that establishes consistent geometric and photometric constraints across views--a technique equally applicable to existing pinhole-camera-based pipelines. Our DirectFisheye-GS matches or surpasses state-of-the-art performance on public datasets.