基于3D点轨迹的生成式视频运动编辑
Generative Video Motion Editing with 3D Point Tracks
利用3D点轨迹作为稀疏对应关系来条件化视频扩散模型，实现相机与物体运动的联合编辑；3D轨迹提供的深度线索有助于解决遮挡和深度顺序问题，为视频运动编辑提供了可复用的控制信号。
原文依据 We present a track-conditioned V2V framework that enables joint editing of camera and object motion.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论相机和物体运动是视频叙事的关键。然而，精确编辑这些捕获的运动仍然是一项重大挑战，尤其是在复杂的物体运动下。当前的运动控制图像到视频（I2V）方法通常缺乏用于一致视频编辑的全场景上下文，而视频到视频（V2V）方法提供视角变化或基本的物体平移，但对细粒度物体运动的控制有限。我们提出了一种基于轨迹条件的V2V框架，能够联合编辑相机和物体运动。我们通过以源视频和表示源运动与目标运动的成对3D点轨迹为条件来调节视频生成模型，从而实现这一目标。这些3D轨迹建立了稀疏对应关系，将源视频中的丰富上下文迁移到新的运动，同时保持时空一致性。关键的是，与2D轨迹相比，3D轨迹提供了显式的深度线索，使模型能够解决深度顺序并处理遮挡，从而实现精确的运动编辑。我们的模型在合成数据和真实数据上分两阶段训练，支持多种运动编辑，包括联合相机/物体操控、运动迁移和非刚性变形，释放了视频编辑的新创意潜力。
英文原文摘要 1248 字符
Camera and object motions are central to a video's narrative. However, precisely editing these captured motions remains a significant challenge, especially under complex object movements. Current motion-controlled image-to-video (I2V) approaches often lack full-scene context for consistent video editing, while video-to-video (V2V) methods provide viewpoint changes or basic object translation, but offer limited control over fine-grained object motion. We present a track-conditioned V2V framework that enables joint editing of camera and object motion. We achieve this by conditioning a video generation model on a source video and paired 3D point tracks representing source and target motions. These 3D tracks establish sparse correspondences that transfer rich context from the source video to new motions while preserving spatiotemporal coherence. Crucially, compared to 2D tracks, 3D tracks provide explicit depth cues, allowing the model to resolve depth order and handle occlusions for precise motion editing. Trained in two stages on synthetic and real data, our model supports diverse motion edits, including joint camera/object manipulation, motion transfer, and non-rigid deformation, unlocking new creative potential in video editing.