RnG：面向部分观测下完整三维建模的统一 Transformer
RnG: A Unified Transformer for Complete 3D Modeling from Partial Observations
把 KV-cache 当作隐式三维表示、用重建引导的因果注意力在注意力层面分离已观测与未观测区域，这一思路可直接迁移到稀疏视角重建、新视角合成以及任何需要「补全未见几何」的前馈三维模型。
原文依据 treating the KV-cache as an implicit 3D representation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类能够从有限的二维观测中感知三维世界。尽管近期的前馈可泛化三维重建模型能够从稀疏图像中恢复结构，但它们通常只表示已观测区域，未观测的几何则未被建模。这引出了一个根本性问题：我们能否从部分观测中推断出完整的三维结构？我们提出 RnG（Reconstruction and Generation，重建与生成），这是一个前馈 Transformer，通过预测隐式的完整三维表示来统一重建与生成。RnG 引入重建引导的因果注意力机制，在注意力层面将重建与生成分离，同时把 KV 缓存视为一种隐式三维表示。任意位姿都可以高效地查询该缓存，从而渲染出高保真的 RGBD 输出。RnG 在准确重建可见几何的同时，生成合理的未观测结构与外观。它在可泛化三维重建与新视角生成方面取得了最优性能，同时效率足以支持实时交互应用。
英文原文摘要 1081 字符
Humans perceive the 3D world from limited 2D observations. While recent feed-forward generalizable 3D reconstruction models can recover structures from sparse images, they typically represent only observed regions, leaving unseen geometry unmodeled. This raises a fundamental question: Can we infer complete 3D structure from partial observations? We present RnG (Reconstruction and Generation), a feed-forward Transformer that unifies reconstruction and generation by predicting an implicit, complete 3D representation. RnG introduces a reconstruction-guided causal attention mechanism that separates reconstruction and generation at the attention level while treating the KV-cache as an implicit 3D representation. Arbitrary poses can efficiently query this cache to render high-fidelity RGBD outputs. RnG accurately reconstructs visible geometry while generating plausible unseen structures and appearance. It achieves state-of-the-art performance in generalizable 3D reconstruction and novel view generation, while being efficient enough for real-time interactive applications.