InverFill：面向高质量少步扩散图像修复的一步反演方法
InverFill: One-Step Inversion for Enhanced Few-Step Diffusion Inpainting
把输入掩码图像的语义通过一步反演注入初始噪声，替代随机高斯噪声，可直接套用到任何少步 T2I 模型的 blended sampling 流程中，在低 NFE 下改善修复区域与背景的协调性，且无需重训练。
原文依据 we propose InverFill, a one-step inversion method tailored for inpainting that injects semantic information from the input masked image into the initial noise
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期基于扩散的模型在图像修复中实现了照片级真实感，但需要大量采样步数，限制了实际应用。少步文本到图像模型提供了更快的生成速度，但将其朴素地应用于图像修复会在背景与修复区域之间产生较差的协调性和伪影。我们将这一原因追溯到随机高斯噪声初始化：在函数评估次数较少时，这种初始化会导致语义错位和保真度下降。为克服这一问题，我们提出 InverFill，一种专为图像修复设计的一步反演方法，它将来自输入掩码图像的语义信息注入初始噪声，从而实现高保真度的少步图像修复。InverFill 不训练修复模型，而是在混合采样流程中利用少步文本到图像模型，并以语义对齐的噪声作为输入，显著改进了朴素混合采样，甚至在低 NFE 下可与专用修复模型相媲美。此外，InverFill 不需要真实图像监督，仅增加极少的推理开销。大量实验表明，InverFill 能够稳定提升基线少步模型，在无需昂贵重训练或大量迭代优化的情况下改善图像质量和文本一致性。
英文原文摘要 1242 字符
Recent diffusion-based models achieve photorealism in image inpainting but require many sampling steps, limiting practical use. Few-step text-to-image models offer faster generation, but naively applying them to inpainting yields poor harmonization and artifacts between the background and inpainted region. We trace this cause to random Gaussian noise initialization, which under low function evaluations causes semantic misalignment and reduced fidelity. To overcome this, we propose InverFill, a one-step inversion method tailored for inpainting that injects semantic information from the input masked image into the initial noise, enabling high-fidelity few-step inpainting. Instead of training inpainting models, InverFill leverages few-step text-to-image models in a blended sampling pipeline with semantically aligned noise as input, significantly improving vanilla blended sampling and even matching specialized inpainting models at low NFEs. Moreover, InverFill does not require real-image supervision and only adds minimal inference overhead. Extensive experiments show that InverFill consistently boosts baseline few-step models, improving image quality and text coherence without costly retraining or heavy iterative optimization.