TokenGS:用可学习词元将 3D 高斯预测与像素解耦
TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokens
将 3D 高斯预测从像素网格解耦为可学习词元,使前馈 3DGS 重建不再受输入分辨率和视图数限制,并支持在词元空间进行测试时优化,为动态场景重建和场景流估计提供了新思路。
原文依据 move from the standard encoder-only design to an encoder-decoder architecture with learnable Gaussian tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文重新审视了现代基于 Transformer 的前馈 3D 高斯泼溅(3DGS)预测中的若干关键设计选择。我们认为,沿相机光线将高斯均值回归为深度的常见做法并非最优,转而提出仅使用自监督渲染损失直接回归 3D 均值坐标。这一形式使我们能够从标准的仅编码器设计转向带有可学习高斯词元的编码器-解码器架构,从而将预测基元的数量与输入图像分辨率和视图数量解绑。我们由此得到的方法 TokenGS 对姿态噪声和多视图不一致性表现出更强的鲁棒性,同时自然支持在词元空间中进行高效的测试时优化,且不损害已学习到的先验。TokenGS 在静态和动态场景上均实现了最优的前馈重建性能,产生更正则化的几何和更均衡的 3DGS 分布,同时无缝恢复诸如静态-动态分解和场景流等涌现场景属性。
英文原文摘要 1113 字符
In this work, we revisit several key design choices of modern Transformer-based approaches for feed-forward 3D Gaussian Splatting (3DGS) prediction. We argue that the common practice of regressing Gaussian means as depths along camera rays is suboptimal, and instead propose to directly regress 3D mean coordinates using only a self-supervised rendering loss.This formulation allows us to move from the standard encoder-only design to an encoder-decoder architecture with learnable Gaussian tokens, thereby _unbinding_ the number of predicted primitives from input image resolution and number of views. Our resulting method, __TokenGS__, demonstrates improved robustness to pose noise and multiview inconsistencies, while naturally supporting efficient test-time optimization in token space without degrading learned priors. TokenGS achieves state-of-the-art feed-forward reconstruction performance on both static and dynamic scenes, producing more regularized geometry and more balanced 3DGS distribution, while seamlessly recovering emergent scene attributes such as static-dynamic decomposition and scene flow.