TRANSPORTER：从 VLM 流形迁移视觉语义
TRANSPORTER: Transferring Visual Semantics from VLM Manifolds
把 VLM 的 logit 分数通过最优传输耦合映射到 T2V 模型的语义嵌入空间，用生成视频作为解释载体——做 VLM 可解释性或探针研究时，可复用这套「logit→嵌入方向→条件生成」的流程来可视化模型内部语义。
原文依据 TRANSPORTER learns an optimal transport coupling to VLM's high-semantic embedding spaces
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频理解模型是如何得出答案的？尽管当前的视觉语言模型（VLM）能够对包含多样物体、动作表现与场景动态的复杂场景进行推理，但理解并控制其内部过程仍是一个未解难题。受文本到视频（T2V）生成模型近期进展的启发，本文提出 logits-to-video（L2V）任务，并给出一种与模型无关的方法 TRANSPORTER，用于生成能够刻画 VLM 预测背后潜在规则的视频。鉴于 T2V 模型具备高视觉保真度，TRANSPORTER 学习一个到 VLM 高语义嵌入空间的最优传输耦合。进而，logit 分数定义了用于条件视频生成的嵌入方向。TRANSPORTER 生成的视频能够反映在多样物体属性、动作副词与场景上下文上的描述变化。跨多个 VLM 的定量与定性评估表明，L2V 可为模型可解释性提供一个高保真、此前未被探索的新方向。
英文原文摘要 1072 字符
How do video understanding models acquire their answers? Although current Vision Language Models (VLMs) reason over complex scenes with diverse objects, action performances, and scene dynamics, understanding and controlling their internal processes remains an open challenge. Motivated by recent advancements in text-to-video (T2V) generative models, this paper introduces a logits-to-video (L2V) task alongside a model-independent approach, TRANSPORTER, to generate videos that capture the underlying rules behind VLMs' predictions. Given the high-visual-fidelity produced by T2V models, TRANSPORTER learns an optimal transport coupling to VLM's high-semantic embedding spaces. In turn, logit scores define embedding directions for conditional video generation. TRANSPORTER generates videos that reflect caption changes over diverse object attributes, action adverbs, and scene context. Quantitative and qualitative evaluations across VLMs demonstrate that L2V can provide a fidelity-rich, novel direction for model interpretability that has not been previously explored.