MakeAnything：利用扩散 Transformer 进行多领域过程序列生成
MakeAnything: Harnessing Diffusion Transformers for Multi-Domain Procedural Sequence Generation
提出非对称 LoRA 微调策略（冻结编码器、自适应调整解码器）以平衡泛化与任务性能，可复用于其他基于 DiT 的生成任务；同时利用时空一致性约束实现从静态图像生成过程序列，为视觉生成中的时序建模提供思路。
原文依据 we introduce MakeAnything, a framework based on the diffusion transformer (DIT), which leverages fine-tuning to activate the in-context capabilities of DIT for generating consistent procedural sequences.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类智能的一个标志是能够通过结构化的多步骤过程创造复杂的人工制品。利用人工智能生成过程教程是一个长期存在但具有挑战性的目标，面临三个关键障碍：（1）多任务过程数据集的稀缺；（2）保持步骤之间的逻辑连续性和视觉一致性；（3）跨多个领域的泛化。为了应对这些挑战，我们提出了一个多领域数据集，涵盖 21 个任务和超过 24,000 个过程序列。在此基础上，我们引入了 MakeAnything，一个基于扩散 Transformer（DIT）的框架，它利用微调来激活 DIT 的上下文能力，以生成一致的过程序列。我们引入了用于图像生成的非对称低秩适配（LoRA），通过冻结编码器参数同时自适应调整解码器层，来平衡泛化能力和任务特定性能。此外，我们的 ReCraft 模型通过时空一致性约束实现图像到过程的生成，允许将静态图像分解为合理的创作序列。大量实验表明，MakeAnything 超越了现有方法，为过程生成任务设定了新的性能基准。
英文原文摘要 1302 字符
A hallmark of human intelligence is the ability to create complex artifacts through structured multi-step processes. Generating procedural tutorials with AI is a longstanding but challenging goal, facing three key obstacles: (1) scarcity of multi-task procedural datasets, (2) maintaining logical continuity and visual consistency between steps, and (3) generalizing across multiple domains. To address these challenges, we propose a multi-domain dataset covering 21 tasks with over 24,000 procedural sequences. Building upon this foundation, we introduce MakeAnything, a framework based on the diffusion transformer (DIT), which leverages fine-tuning to activate the in-context capabilities of DIT for generating consistent procedural sequences. We introduce asymmetric low-rank adaptation (LoRA) for image generation, which balances generalization capabilities and task-specific performance by freezing encoder parameters while adaptively tuning decoder layers. Additionally, our ReCraft model enables image-to-process generation through spatiotemporal consistency constraints, allowing static images to be decomposed into plausible creation sequences. Extensive experiments demonstrate that MakeAnything surpasses existing methods, setting new performance benchmarks for procedural generation tasks.