UniGame:将统一多模态模型变为自身的对手
UniGame: Turning a Unified Multimodal Model Into Its Own Adversary
提出在共享词元接口施加轻量扰动器的自对抗后训练框架,可复用于提升任意视觉-语言统一模型的理解-生成一致性与对抗鲁棒性。
原文依据 By applying a lightweight perturber at the shared token interface, UniGame enables the generation branch to actively seek and challenge fragile understanding
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论统一多模态模型(UMMs)在单一架构下于理解和生成两方面均展现出令人印象深刻的性能。然而,UMMs 仍存在一个根本性的不一致:理解倾向于紧凑的嵌入,而生成则倾向于富含重建信息的表示。这种结构性的权衡导致了决策边界错位、跨模态一致性下降,以及在分布偏移和对抗性偏移下脆弱性加剧。本文提出 UniGame,一个自对抗后训练框架,直接针对这些不一致性。通过在共享词元接口处施加轻量级扰动器,UniGame 使生成分支能够主动寻找并挑战脆弱理解,将模型本身变为自身的对手。实验表明,UniGame 显著提升了一致性(+4.6%)。此外,它在理解(+3.6%)、生成(+0.02)、分布外与对抗鲁棒性(在 NaturalBench 和 AdVQA 上分别提升 +4.8% 和 +6.2%)方面也取得了实质性改进。该框架与架构无关,仅引入不到 1% 的额外参数,并与现有后训练方法互补。这些结果将对抗性自博弈定位为一种通用且有效的原则,用以增强未来多模态基础模型的一致性、稳定性和统一能力。
英文原文摘要 1388 字符
Unified Multimodal Models (UMMs) have shown impressive performance in both understanding and generation with a single architecture. However, UMMs still exhibit a fundamental inconsistency: understanding favors compact embeddings, whereas generation favors reconstruction-rich representations. This structural trade-off produces misaligned decision boundaries, degraded cross-modal coherence, and heightened vulnerability under distributional and adversarial shifts. In this paper, we present UniGame, a self-adversarial post-training framework that directly targets the inconsistencies. By applying a lightweight perturber at the shared token interface, UniGame enables the generation branch to actively seek and challenge fragile understanding, turning the model itself into its own adversary. Experiments demonstrate that UniGame significantly improves the consistency (+4.6%). Moreover, it also achieves substantial improvements in understanding (+3.6%), generation (+0.02), out-of-distribution and adversarial robustness (+4.8% and +6.2% on NaturalBench and AdVQA). The framework is architecture-agnostic, introduces <1% additional parameters, and is complementary to existing post-training methods. These results position adversarial self-play as a general and effective principle for enhancing the coherence, stability, and unified competence of future multimodal foundation models.