DiP：在像素空间中驯服扩散模型
DiP: Taming Diffusion Models in Pixel Space
把高分辨率像素空间扩散拆成「大图像块上的全局结构 + 轻量细节检测头恢复局部细节」两阶段，可在不依赖 VAE 的前提下把像素空间扩散的推理成本压到 LDM 水平，做高分辨率图像生成或想摆脱 VAE 信息损失时可直接复用这一解耦范式。
原文依据 a Diffusion Transformer (DiT) backbone operates on large patches for efficient global structure construction, while a co-trained lightweight Patch Detailer Head
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型面临生成质量与计算效率之间的根本性权衡。潜变量扩散模型（LDM）提供了一种高效的解决方案，但存在潜在的信息损失且无法端到端训练。相比之下，现有的像素空间模型绕过了 VAE，但在高分辨率合成上计算代价过高。为化解这一困境，我们提出 DiP，一种高效的像素空间扩散框架。DiP 将生成解耦为全局与局部两个阶段：Diffusion Transformer（DiT）骨干网络在大图像块上运行，以高效构建全局结构；而一个协同训练的轻量级图像块细节检测头则利用上下文特征恢复细粒度的局部细节。这一协同设计在不依赖 VAE 的情况下实现了与 LDM 相当的计算效率。DiP 相比先前方法推理速度最高提升 10 倍，而参数总量仅增加 0.3%，并在 ImageNet 256x256 上取得 1.79 的弗雷歇初始距离分数。
英文原文摘要 1024 字符
Diffusion models face a fundamental trade-off between generation quality and computational efficiency. Latent Diffusion Models (LDMs) offer an efficient solution but suffer from potential information loss and non-end-to-end training. In contrast, existing pixel space models bypass VAEs but are computationally prohibitive for high-resolution synthesis. To resolve this dilemma, we propose DiP, an efficient pixel space diffusion framework. DiP decouples generation into a global and a local stage: a Diffusion Transformer (DiT) backbone operates on large patches for efficient global structure construction, while a co-trained lightweight Patch Detailer Head leverages contextual features to restore fine-grained local details. This synergistic design achieves computational efficiency comparable to LDMs without relying on a VAE. DiP is accomplished with up to 10xfaster inference speeds than previous method while increasing the total number of parameters by only 0.3%, and achieves an 1.79 FID score on ImageNet 256x256.