Tea-Adapter：面向高效条件生成的教师适配器
Tea-Adapter: Teacher Adapter for Efficient Conditional Generation
用小型已调好的教师扩散模型通过逆蒸馏把精确控制能力迁移进十亿参数视频 DiT，配合 Mixture of Condition Experts 在统一结构里动态路由多条件输入，为可控视频生成提供了一条免全量微调、低显存可复用的适配器路线。
原文依据 a novel reverse distillation method that enables large video diffusion models to inherit precise control capabilities from smaller, efficiently tuned teacher diffusion models
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 Tea-Adapter，一种即插即用的适配器，旨在将较小教师模型中的条件知识高效地整合进较大的学生视频扩散模型。现有的可控视频 DiT 方法面临关键挑战：对十亿参数模型进行全量微调代价极高；同时 ControlNet 引入大量参数开销，且在新颖的多条件组合上灵活性有限。为克服这些问题，Tea-Adapter 提出一种新颖的逆蒸馏方法，使大型视频扩散模型能够从经过高效调优的较小教师扩散模型中继承精确的控制能力，从而无需全量微调。此外，我们注意到不同条件之间的内在关系，用混合条件专家（MCE）层取代级联式 ControlNet 设计。该结构在统一架构内动态路由多样的条件输入，同时支持单条件控制和多条件组合，且不产生额外训练开销。为实现跨尺度知识迁移，我们进一步设计了特征传播模块，以保证特征在视频帧之间高效且时序一致地传播。实验表明，Tea-Adapter 能够实现高保真、多条件的视频合成，使先进的可控视频生成在低资源硬件上成为可能。
英文原文摘要 1454 字符
We propose Tea-Adapter, a plug-and-play adapter designed to efficiently integrate conditional knowledge from a smaller teacher model into a larger student video diffusion model. Existing controllable video DiT methods face critical challenges: full fine-tuning of billion-parameter models is extremely expensive. At the same time, ControlNets introduce substantial parameter overhead and exhibit limited flexibility for novel multi-condition compositions. To overcome these issues, Tea-Adapter introduces a novel reverse distillation method that enables large video diffusion models to inherit precise control capabilities from smaller, efficiently tuned teacher diffusion models, eliminating the need for full fine-tuning. Moreover, recognizing the intrinsic relationships between different conditions, we replace the cascaded ControlNet design with a Mixture of Condition Experts (MCE) layer. This structure dynamically routes diverse conditional inputs within a unified architecture, supporting both single-condition control and multiple condition combinations without additional training cost. To achieve cross-scale knowledge transfer, we further develop a Feature Propagation Module to ensure efficient and temporally consistent feature propagation across video frames. Experiments demonstrate that Tea-Adapter enables high-fidelity, multi-condition video synthesis, making advanced, controllable video generation feasible on low-resource hardware.