组扩散:通过解锁跨样本协作增强图像生成
Group Diffusion: Enhancing Image Generation by Unlocking Cross-Sample Collaboration
把注意力从单张图内的 patch 扩展到一组图像之间,在推理阶段联合去噪,是一种无需重训即可插到现有 DiT 上的推理策略;做图像生成的人可以直接试「组大小」这一新超参,并用其提出的定性指标监控跨样本注意力强度与 FID 的相关性。
原文依据 unlocking the attention mechanism to be shared across images, rather than limited to just the patches within an image
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文探索了扩散模型推理中一个尚未被利用的信号。以往所有方法在推理时都独立生成图像,而我们则追问:样本能否被协同生成。我们提出组扩散(Group Diffusion),让注意力机制在图像之间共享,而不再局限于单张图像内部的图块。这使得图像能够在推理阶段被联合去噪,同时学习图像内与图像间的对应关系。我们观察到明显的规模效应——更大的组规模带来更强的跨样本注意力与更好的生成质量。此外,我们引入一种定性度量来刻画这一行为,并表明其强度与弗雷歇初始距离(FID)密切相关。基于标准扩散 Transformer,我们的 GroupDiff 在 ImageNet-256x256 上取得最高 32.2% 的 FID 提升。我们的工作揭示了跨样本推理作为一种有效且此前未被探索的生成建模机制。
英文原文摘要 947 字符
In this work, we explore an untapped signal in diffusion model inference. While all previous methods generate images independently at inference, we instead ask if samples can be generated collaboratively. We propose Group Diffusion, unlocking the attention mechanism to be shared across images, rather than limited to just the patches within an image. This enables images to be jointly denoised at inference time, learning both intra and inter-image correspondence. We observe a clear scaling effect -- larger group sizes yield stronger cross-sample attention and better generation quality. Furthermore, we introduce a qualitative measure to capture this behavior and show that its strength closely correlates with FID. Built on standard diffusion transformers, our GroupDiff achieves up to 32.2% FID improvement on ImageNet-256x256. Our work reveals cross-sample inference as an effective, previously unexplored mechanism for generative modeling.