超越黄金数据：通过时间步选择性训练解决运动-视觉质量困境
Beyond the Golden Data: Resolving the Motion-Vision Quality Dilemma via Timestep Selective Training
对做视频扩散训练的人，这篇给出了一个可直接复用的数据使用策略：不必筛掉运动强但画质差（或反之）的样本，而是按时间步把不同质量倾向的数据分配到模型学习的不同阶段，从而在数据受限时仍能提升生成质量。
原文依据 quality-imbalanced data can produce gradients similar to golden data at appropriate timesteps
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期视频生成模型取得了令人瞩目的成果。然而，这些模型严重依赖于同时具备高视觉质量与高运动质量的高质量数据。本文识别出视频数据整理中的一个关键挑战：运动-视觉质量困境。我们发现，视觉质量与运动强度本质上呈现负相关，这使得同时在这两方面都出色的黄金数据难以获得。为应对这一挑战，我们首先考察了视频扩散模型的分层学习动态，并对质量退化样本进行了基于梯度的分析。我们发现，质量不均衡的数据在合适的时间步上可以产生与黄金数据相似的梯度。基于此，我们引入了训练过程中时间步选择这一新概念，并提出时间步感知的质量解耦（TQD），它修改数据采样分布以更好地匹配模型的学习过程。对于特定类型的数据，采样分布会向更高的时间步倾斜以偏向运动丰富的数据，而高视觉质量的数据更可能在较低的时间步被采样。通过大量实验，我们证明 TQD 能够仅使用分离的不均衡数据进行训练，并取得超越使用更好数据的常规训练的性能，从而挑战了视频生成中对完美数据必要性的认知。此外，我们的方法在使用高质量数据训练时也能提升模型性能，展示了其在不同数据场景下的有效性。
英文原文摘要 1579 字符
Recent advances in video generation models have achieved impressive results. However, these models heavily rely on the use of high-quality data that combines both high visual quality and high motion quality. In this paper, we identify a key challenge in video data curation: the Motion-Vision Quality Dilemma. We discovered that visual quality and motion intensity inherently exhibit a negative correlation, making it hard to obtain golden data that excels in both aspects. To address this challenge, we first examine the hierarchical learning dynamics of video diffusion models and conduct gradient-based analysis on quality-degraded samples. We discover that quality-imbalanced data can produce gradients similar to golden data at appropriate timesteps. Based on this, we introduce the novel concept of Timestep selection in Training Process. We propose Timestep-aware Quality Decoupling (TQD), which modifies the data sampling distribution to better match the model's learning process. For certain types of data, the sampling distribution is skewed toward higher timesteps for motion-rich data, while high visual quality data is more likely to be sampled during lower timesteps. Through extensive experiments, we demonstrate that TQD enables training exclusively on separated imbalanced data to achieve performance surpassing conventional training with better data, challenging the necessity of perfect data in video generation. Moreover, our method also boosts model performance when trained on high-quality data, showcasing its effectiveness across different data scenarios.