CTCal：通过跨时间步自校准重新审视文本到图像扩散模型
CTCal: Rethinking Text-to-Image Diffusion Models via Cross-Timestep Self-Calibration
把扩散模型内部小时间步的交叉注意力图当作显式监督信号，去校准大时间步的表征学习，是一种无需额外标注、模型无关的训练期对齐增强手段，可直接插入 SD 系列或 flow matching 模型以提升文本-图像细粒度对齐。
原文依据 CTCal leverages the reliable text-image alignment (i.e., cross-attention maps) formed at smaller timesteps with less noise to calibrate the representation learning at larger timesteps
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期文本到图像合成的进展在很大程度上由扩散模型推动，但实现文本提示与生成图像之间的精确对齐仍是一个持续存在的挑战。我们发现，这一困难主要源于传统扩散损失的局限性，它只能为建模细粒度文本-图像对应关系提供隐式监督。本文提出跨时间步自校准（CTCal），其依据的支撑性观察是：在扩散模型内部，建立准确的文本-图像对齐会随着时间步增大而变得越来越困难。CTCal 利用在噪声较少的小时间步上形成的可靠文本-图像对齐（即交叉注意力图），去校准噪声较多的大时间步上的表征学习，从而在训练过程中提供显式监督。我们进一步提出时间步感知的自适应加权，以实现 CTCal 与扩散损失的协调融合。CTCal 与模型无关，可无缝集成到现有的文本到图像扩散模型中，涵盖基于扩散的方法（如 SD 2.1）与基于流的方法（如 SD 3）。在 T2I-Compbench++ 与 GenEval 基准上的大量实验证明了所提 CTCal 的有效性与泛化性。我们的代码可在 https://github.com/xiefan-guo/ctcal 获取。
英文原文摘要 1412 字符
Recent advancements in text-to-image synthesis have been largely propelled by diffusion-based models, yet achieving precise alignment between text prompts and generated images remains a persistent challenge. We find that this difficulty arises primarily from the limitations of conventional diffusion loss, which provides only implicit supervision for modeling fine-grained text-image correspondence. In this paper, we introduce Cross-Timestep Self-Calibration (CTCal), founded on the supporting observation that establishing accurate text-image alignment within diffusion models becomes progressively more difficult as the timestep increases. CTCal leverages the reliable text-image alignment (i.e., cross-attention maps) formed at smaller timesteps with less noise to calibrate the representation learning at larger timesteps with more noise, thereby providing explicit supervision during training. We further propose a timestep-aware adaptive weighting to achieve a harmonious integration of CTCal and diffusion loss. CTCal is model-agnostic and can be seamlessly integrated into existing text-to-image diffusion models, encompassing both diffusion-based (e.g., SD 2.1) and flow-based approaches (e.g., SD 3). Extensive experiments on T2I-Compbench++ and GenEval benchmarks demonstrate the effectiveness and generalizability of the proposed CTCal. Our code is available at https://github.com/xiefan-guo/ctcal.