JRM:无需对齐的多物体联合重建模型
JRM: Joint Reconstruction Model for Multiple Objects without Alignment
把多视角/多次扫描中同一物体的重复观测当作个性化生成问题,用 3D 流匹配在潜变量空间隐式聚合未对齐观测,可替代显式配准与刚性对齐流程,对处理关节运动等非刚性变化的多观测重建有直接参考价值。
原文依据 JRM is a 3D flow-matching generative model that implicitly aggregates unaligned observations in its latent space
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论以对象为中心的重建旨在通过独立物体的组合来恢复场景的三维结构。这种独立性虽然可以简化建模,却丢弃了本可提升重建质量的强信号,尤其是重复性——同一物体模型在单个场景内或跨扫描中被多次观测到。我们提出联合重建模型(JRM),将物体重建表述为一种个性化生成任务,从而利用这种重复性:多个观测共享一个应对所有观测保持一致的主体,同时仍遵循各自的特定姿态与状态。该方向的已有方法依赖跨观测的显式匹配与刚性对齐,因而对误差敏感,且难以扩展到非刚性变换。相比之下,JRM 是一个三维流匹配生成模型,它在潜变量空间中隐式聚合未对齐的观测,以数据驱动的方式学习产生一致且忠实的重建,而无需显式约束。在合成数据与真实数据上的评估表明,JRM 的隐式聚合消除了对显式对齐的需求,提升了对错误关联的鲁棒性,并能自然处理关节运动等非刚性变化。总体而言,JRM 在重建质量上优于独立重建与基于对齐的基线方法。
英文原文摘要 1378 字符
Object-centric reconstruction seeks to recover the 3D structure of a scene through composition of independent objects. While this independence can simplify modeling, it discards strong signals that could improve reconstruction, notably repetition where the same object model is seen multiple times in a scene, or across scans. We propose the Joint Reconstruction Model (JRM) to leverage repetition by framing object reconstruction as one of personalized generation: multiple observations share a common subject that should be consistent for all observations, while still adhering to the specific pose and state from each. Prior methods in this direction rely on explicit matching and rigid alignment across observations, making them sensitive to errors and difficult to extend to non-rigid transformations. In contrast, JRM is a 3D flow-matching generative model that implicitly aggregates unaligned observations in its latent space, learning to produce consistent and faithful reconstructions in a data-driven manner without explicit constraints. Evaluations on synthetic and real-world data show that JRM's implicit aggregation removes the need for explicit alignment, improves robustness to incorrect associations, and naturally handles non-rigid changes such as articulation. Overall, JRM outperforms both independent and alignment-based baselines in reconstruction quality.