编排动态物体世界
Choreographing a World of Dynamic Objects
把 2D 视频生成模型中的欧拉表示蒸馏成拉格朗日运动信息，从而无需类别专属数据即可合成多体 4D 动态；这一「从视频先验提取运动」的蒸馏思路可直接迁移到动态场景重建、运动迁移与机器人策略生成。
原文依据 a distillation-based pipeline to extract the rich Lagrangian motion information hidden in the Eulerian representations of 2D videos
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们所在的物理 4D（3D + 时间）世界中的动态物体不断演化、形变并与其他物体交互，从而产生多样的 4D 场景动态。本文提出一种通用生成流程 CHORD，用于编排动态物体与场景并合成此类现象。传统的基于规则的图形学流程依赖类别专属的启发式规则来创建这些动态，但费时费力且难以扩展。近期的学习方法通常需要大规模数据集，而这些数据集未必覆盖所有感兴趣的物体类别。与之不同，我们的方法通过提出一种基于蒸馏的流程，从 2D 视频的欧拉表示中提取其中隐含的丰富拉格朗日运动信息，从而继承视频生成模型的通用性。我们的方法通用、多能且与类别无关。我们通过实验生成多样化的多体 4D 动态来证明其有效性，展示其相较现有方法的优势，并说明其在生成机器人操作策略方面的适用性。
英文原文摘要 1119 字符
Dynamic objects in our physical 4D (3D + time) world are constantly evolving, deforming, and interacting with other objects, leading to diverse 4D scene dynamics. In this paper, we present a universal generative pipeline, CHORD, for CHOReographing Dynamic objects and scenes and synthesizing this type of phenomena. Traditional rule-based graphics pipelines to create these dynamics are based on category-specific heuristics, yet are labor-intensive and not scalable. Recent learning-based methods typically demand large-scale datasets, which may not cover all object categories in interest. Our approach instead inherits the universality from the video generative models by proposing a distillation-based pipeline to extract the rich Lagrangian motion information hidden in the Eulerian representations of 2D videos. Our method is universal, versatile, and category-agnostic. We demonstrate its effectiveness by conducting experiments to generate a diverse range of multi-body 4D dynamics, show its advantage compared to existing methods, and demonstrate its applicability in generating robotics manipulation policies.