图像编辑与可控生成 · CVPR2026 ·2026.06.03

RAISE:面向免训练文本到图像对齐的需求自适应进化精修

RAISE: Requirement-Adaptive Evolutionary Refinement for Training-Free Text-to-Image Alignment

Liyao Jiang、Ruichen Chen、Chao Gao、Di Niu

与你有关的部分

把 T2I 生成当作需求驱动的测试时搜索:用结构化需求清单驱动 VLM 验证,只在未满足项上分配提示词重写、噪声重采样与指令编辑等精修动作,可作为即插即用的免训练推理框架套到任意扩散模型上,替代固定迭代预算的 scaling。

原文依据 evolving a population of candidates at inference time through a diverse set of refinement actions
可迁移到 文本到图像生成提示词对齐指令式图像编辑生成结果评测

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

近期文本到图像(T2I)扩散模型已实现显著的逼真度,但提示词与图像之间的忠实对齐仍具挑战,尤其是包含多个物体、关系和细粒度属性的复杂提示词。现有的免训练推理时缩放方法依赖固定的迭代预算,无法适应提示词难度;而经反思微调的模型需要精心构建的反思数据集,并对扩散模型与视觉语言模型进行大量联合微调,往往过拟合于反思路径数据,且缺乏跨模型的可迁移性。我们提出 RAISE(需求自适应自我改进进化),一个免训练、需求驱动的进化框架,用于自适应 T2I 生成。RAISE 将图像生成建模为需求驱动的自适应缩放过程,在推理时通过一组多样化的精修动作——包括提示词重写、噪声重采样和指令式编辑——进化候选样本种群。每一代候选都依据结构化的需求清单进行验证,使系统能够动态识别未满足项,并仅在需要处分配进一步计算。这实现了自适应测试时缩放,使计算投入与语义查询复杂度相匹配。在 GenEval 和 DrawBench 上,RAISE 取得最优的对齐效果(GenEval 总体 0.94),同时相比先前的缩放与反思微调基线产生更少的生成样本(减少 30-40%)和更少的 VLM 调用(减少 80%),展现出高效、可泛化且与模型无关的多轮自我改进能力。代码已发布于 https://github.com/LiyaoJiang1998/RAISE。

英文原文摘要 1676 字符

Recent text-to-image (T2I) diffusion models achieve remarkable realism, yet faithful prompt-image alignment remains challenging, particularly for complex prompts with multiple objects, relations, and fine-grained attributes. Existing training-free inference-time scaling methods rely on fixed iteration budgets that cannot adapt to prompt difficulty, while reflection-tuned models require carefully curated reflection datasets and extensive joint fine-tuning of diffusion and vision-language models, often overfitting to reflection paths data and lacking transferability across models. We introduce RAISE (Requirement-Adaptive Self-Improving Evolution), a training-free, requirement-driven evolutionary framework for adaptive T2I generation. RAISE formulates image generation as a requirement-driven adaptive scaling process, evolving a population of candidates at inference time through a diverse set of refinement actions--including prompt rewriting, noise resampling, and instructional editing. Each generation is verified against a structured checklist of requirements, enabling the system to dynamically identify unsatisfied items and allocate further computation only where needed. This achieves adaptive test-time scaling that aligns computational effort with semantic query complexity. On GenEval and DrawBench, RAISE attains state-of-the-art alignment (0.94 overall GenEval) while incurring fewer generated samples (reduced by 30-40%) and VLM calls (reduced by 80%) than prior scaling and reflection-tuned baselines, demonstrating efficient, generalizable, and model-agnostic multi-round self-improvement. Code is available at https://github.com/LiyaoJiang1998/RAISE.

标签

会议 CVPR2026
页码 22038-22048
Bibkey
来源 CVF Open Access