单因素物理视频到音频生成的基准评测
Benchmarking Single-Factor Physical Video-to-Audio Generation
为视频到音频生成提供了单因素反事实配对与单视频模式测试的评测协议，可直接用于诊断多模态生成模型对视觉流与文本条件的依赖程度，并揭示时序对齐与语义准确率之间的权衡。
原文依据 we introduce FlatSounds, a benchmark that audits the physical reasoning of V2A models
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论生成式视频到音频（V2A）模型能够生成高度逼真的音轨，但其是否捕捉到了背后的物理过程仍不清楚。现有评测强调感知真实感，却忽视了受控干预下的物理正确性。本文提出 FlatSounds，一个通过以下方式审计 V2A 模型物理推理能力的基准：1）仅改变单一物理因素的受控反事实配对；2）探测内部一致性与方向性趋势的单视频模式测试。这些设定检验生成的音频是否正确反映了特定物理属性与时间关系。对最优模型的评测揭示出一致的权衡：模型更多依赖文本描述而非视觉流来推断物理与语义。文本描述通常提升物理与语义准确率，却反常地损害时序对齐。结果凸显出有必要超越音频质量，转向直接从像素学习物理过程。最后，我们发现基于物理的指标在我们自己的数据上与人类偏好测试高度相关。
英文原文摘要 1157 字符
Generative video-to-audio (V2A) models produce highly plausible soundtracks, but it remains unclear whether they capture the underlying physical processes. Existing evaluations emphasize perceptual realism and overlook physical correctness under controlled interventions. In this paper, we introduce FlatSounds, a benchmark that audits the physical reasoning of V2A models through: 1) controlled counterfactual pairs in which a single physical factor is varied, and 2) single-video pattern tests that probe internal consistency and directional trends. These settings test whether the generated audio correctly reflects specific physical properties and timings. Our evaluation of state-of-the-art models reveals a consistent trade-off: models rely more on text captions than the visual stream to infer physics and semantics. Captions generally improve physical and semantic accuracy, but paradoxically degrade temporal alignment. Our results highlight the need to move beyond audio quality toward learning physical processes directly from pixels. Finally, we find that our physics-based metrics correlate strongly with human preference tests on our own data.