基于条件引导调度的混合数据-流水线并行扩散加速
Accelerating Diffusion via Hybrid Data-Pipeline Parallelism Based on Conditional Guidance Scheduling
对做扩散模型部署的人，这篇给出了一个可直接复用的思路：把条件/无条件两条去噪路径当作数据划分维度，并按两条路径的去噪差异自适应切换流水线并行度，从而在多卡上把生成延迟压下来而不引入明显伪影。
原文依据 leverage the conditional and unconditional denoising paths as a new data-partitioning perspective
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型在高保真图像、视频与音频生成方面已取得显著进展，但其推理仍然计算开销高昂。然而，当前基于分布式并行的扩散加速方法存在明显的生成伪影，且无法实现与 GPU 数量成正比的实质性加速。为此，我们提出一种混合并行框架，将一种新的数据并行策略——基于条件的分区——与一种最优流水线调度方法——自适应并行切换——相结合，以在条件扩散模型中降低生成延迟并实现高质量生成。其核心思想是：(i) 将条件与无条件去噪路径作为一种新的数据划分视角；(ii) 根据这两条路径之间的去噪差异自适应地启用最优的流水线并行。我们的框架在两块 NVIDIA RTX 3090 GPU 上分别于 SDXL 和 SD3 上实现了 2.31 倍和 2.07 倍的延迟降低，同时保持了图像质量。该结果证实了我们的方法在基于 U-Net 的扩散模型与基于 DiT 的流匹配架构上的通用性。在高分辨率合成设置下，我们的方法在加速方面也优于现有方法。代码见 https://github.com/kaist-dmlab/Hybridiff。
英文原文摘要 1359 字符
Diffusion models have achieved remarkable progress in high-fidelity image, video, and audio generation, yet inference remains computationally expensive. Nevertheless, current diffusion acceleration methods based on distributed parallelism suffer from noticeable generation artifacts and fail to achieve substantial acceleration proportional to the number of GPUs. Therefore, we propose a hybrid parallelism framework that combines a novel data parallel strategy, condition-based partitioning, with an optimal pipeline scheduling method, adaptive parallelism switching, to reduce generation latency and achieve high generation quality in conditional diffusion models. The key ideas are to (i) leverage the conditional and unconditional denoising paths as a new data-partitioning perspective and (ii) adaptively enable optimal pipeline parallelism according to the denoising discrepancy between these two paths. Our framework achieves 2.31xand 2.07xlatency reductions on SDXL and SD3, respectively, using two NVIDIA RTX 3090 GPUs, while preserving image quality. This result confirms the generality of our approach across U-Net-based diffusion models and DiT-based flow-matching architectures. Our approach also outperforms existing methods in acceleration under high-resolution synthesis settings. Code is available at https://github.com/kaist-dmlab/Hybridiff.