通过合成视频学习可迁移的时序基元用于视频推理
Learning Transferable Temporal Primitives for Video Reasoning via Synthetic Videos
该工作表明，用程序化生成的合成视频（简单几何形状）训练 VLM 的时序基元（方向、速度、状态跟踪），能有效迁移到真实视频理解任务，为视频后训练提供了一条低成本、可扩展的数据构造路径。
原文依据 Despite training on simple geometric shapes, SynRL achieves substantial improvements across 15 benchmarks
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从图像理解到视频理解的转变，要求视觉语言模型（VLM）从识别静态模式转向对运动轨迹、速度变化和状态转换等时序动态进行推理。然而，当前的后训练方法因两个关键局限而效果不足：（1）现有数据集往往缺乏时序中心性，答案可以从孤立的关键帧推断出来，而不需要整体的时序整合；（2）由专有模型生成的训练数据在基本时序感知上存在系统性错误，例如混淆运动方向或误判速度。我们提出 SynRL，一个后训练框架，教授模型时序基元——时序理解的基本构建块，包括方向、速度和状态跟踪。我们的关键洞见是，这些从程序化生成的合成视频中学到的抽象基元能有效迁移到真实场景。我们将时序理解分解为短期感知基元（速度、方向）和长期认知基元（状态跟踪、回溯推理），通过基于代码的视频生成构建了 7.7K 思维链和 7K 强化学习样本，并带有真值帧级标注。尽管仅在简单几何形状上训练，SynRL 在涵盖时序定位、复杂推理和通用视频理解的 15 个基准上取得了显著提升。值得注意的是，我们的 7.7K 合成思维链样本优于 Video-R1 的 165K 真实世界样本。我们将此归因于基本的时序技能——例如逐帧跟踪变化和比较速度——从抽象合成模式有效迁移到复杂真实场景。这为视频后训练确立了一个新范式：通过精心设计的合成数据进行视频时序学习，提供了一条更具成本效益的扩展路径。代码已发布于 https://github.com/jiangsongtao/Synthetic-Video。
英文原文摘要 1999 字符
The transition from image to video understanding requires vision-language models (VLMs) to shift from recognizing static patterns to reasoning over temporal dynamics such as motion trajectories, speed changes, and state transitions. Yet current post-training methods fall short due to two critical limitations: (1) existing datasets often lack temporal-centricity, where answers can be inferred from isolated keyframes rather than requiring holistic temporal integration; (2) training data generated by proprietary models contains systematic errors in fundamental temporal perception, such as confusing motion directions or misjudging speeds. We introduce SynRL, a post-training framework that teaches models temporal primitives, the fundamental building blocks of temporal understanding including direction, speed, and state tracking. Our key insight is that these abstract primitives, learned from programmatically generated synthetic videos, transfer effectively to real-world scenarios. We decompose temporal understanding into short-term perceptual primitives (speed, direction) and long-term cognitive primitives (state tracking, retrodictive inference), constructing 7.7K CoT and 7K RL samples with ground-truth frame-level annotations through code-based video generation. Despite training on simple geometric shapes, SynRL achieves substantial improvements across 15 benchmarks spanning temporal grounding, complex reasoning, and general video understanding. Remarkably, our 7.7K synthetic CoT samples outperform Video-R1's 165K real-world samples. We attribute this to fundamental temporal skills--such as tracking frame-by-frame changes and comparing velocity--that transfer effectively from abstract synthetic patterns to complex real-world scenarios. This establishes a new paradigm for video post-training: video temporal learning through carefully designed synthetic data provides a more cost-efficient scaling path. Code is released on https://github.com/jiangsongtao/Synthetic-Video.