听见关键之声！文本条件的选择性视频到音频生成
Hear What Matters! Text-conditioned Selective Video-to-Audio Generation
把文本提示词当作显式选择器，通过补充 token 引导交叉注意力从视频编码器中分离出与提示词相关的声源视觉特征，这一「文本引导特征选择」思路可迁移到视觉中的条件生成与可控编辑任务。
原文依据 treats the text prompt as an explicit selector to distinctly extract prompt-relevant sound-source visual features from the video encoder
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出一项新任务：文本条件的选择性视频到音频（V2A）生成，即从包含多个对象的视频中仅生成用户意图指定的声音。这一能力在多媒体制作中尤为关键，因为音频轨道需要按每个声源单独处理，以实现精确的编辑、混音与创作控制。我们提出 SELVA，一种新颖的文本条件 V2A 模型，它将文本提示词作为显式选择器，从视频编码器中显著地提取与提示词相关的声源视觉特征。为了在高效微调视频编码器的同时抑制与文本无关的激活，所提出的补充 token 促进交叉注意力产生稳健的语义与时序定位。SELVA 进一步以自监督方式采用自主视频混合方案，以克服单声道音频轨道监督的缺失。我们在 VGG-MONOAUDIO 上评估 SELVA，这是为此类任务构建的干净单声源视频基准。大量实验与消融研究在音频质量、语义对齐和时序同步方面一致验证了其有效性。
英文原文摘要 1115 字符
This work introduces a new task, text-conditioned selective video-to-audio (V2A) generation, which produces only the user-intended sound from a multi-object video. This capability is especially crucial in multimedia production, where audio tracks are handled individually for each sound source for precise editing, mixing, and creative control. We propose SELVA, a novel text-conditioned V2A model that treats the text prompt as an explicit selector to distinctly extract prompt-relevant sound-source visual features from the video encoder. To suppress text-irrelevant activations with efficient video encoder finetuning, the proposed supplementary tokens promote cross-attention to yield robust semantic and temporal grounding. SELVA further employs an autonomous video-mixing scheme in a self-supervised manner to overcome the lack of mono audio track supervision. We evaluate SELVA on VGG-MONOAUDIO, a curated benchmark of clean single-source videos for such a task. Extensive experiments and ablations consistently verify its effectiveness across audio quality, semantic alignment, and temporal synchronization.