通过空文本嵌入优化实现文本到图像扩散模型的测试时对齐
Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation
把测试时对齐的作用点从潜变量或噪声改到无分类器引导的无条件文本嵌入上，利用文本嵌入空间的语义结构约束优化轨迹，可直接复用到任何需要奖励微调或偏好对齐的扩散采样流程中，并缓解奖励攻击。
原文依据 aligns diffusion models by optimising the unconditional embedding in classifier-free guidance, rather than manipulating latent or noise variables
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论测试时对齐（TTA）旨在推理阶段使模型适配特定奖励。然而，现有方法往往对目标奖励函数优化不足或过度优化（奖励攻击）。我们提出空文本测试时对齐（Null-TTA），它通过优化无分类器引导中的无条件嵌入来对齐扩散模型，而非操纵潜变量或噪声变量。由于文本嵌入空间具有结构化的语义特性，这确保对齐发生在语义连贯的流形上，并防止奖励攻击（即利用非语义噪声模式提升奖励）。由于无分类器引导中的无条件嵌入充当模型生成分布的锚点，Null-TTA 即使不更新模型参数，也能直接将模型的生成分布导向目标奖励，而不仅仅是调整样本。得益于这些理想特性，我们证明 Null-TTA 在保持强跨奖励泛化能力的同时，实现了最优的目标测试时对齐。这确立了语义空间优化作为一种有效且有原则的 TTA 新范式。
英文原文摘要 1184 字符
Test-time alignment (TTA) aims to adapt models to specific rewards during inference. However, existing methods tend to either under-optimise or over-optimise (reward hack) the target reward function. We propose Null-Text Test-Time Alignment (Null-TTA), which aligns diffusion models by optimising the unconditional embedding in classifier-free guidance, rather than manipulating latent or noise variables. Due to the structured semantic nature of the text embedding space, this ensures alignment occurs on a semantically coherent manifold and prevents reward hacking (exploiting non-semantic noise patterns to improve the reward). Since the unconditional embedding in classifier-free guidance serves as the anchor for the model's generative distribution, Null-TTA directly steers model's generative distribution towards the target reward rather than just adjusting the samples, even without updating model parameters. Thanks to these desirable properties, we show that Null-TTA achieves state-of-the-art target test-time alignment while maintaining strong cross-reward generalisation. This establishes semantic-space optimisation as an effective and principled novel paradigm for TTA.