SketchAssist:面向语义编辑与精确局部重绘的实用助手
SketchAssist: A Practical Assistant for Semantic Edits and Precise Local Redrawing
将 DiT 与任务引导的混合专家(T-MoE)结合到低秩适配层中,为草图编辑提供统一框架,可复用于其他需要同时处理语义修改与局部重绘的可控生成任务。
原文依据 we integrate a Task-guided Mixture-of-Experts (T-MoE) into LoRA layers, enabling adaptive control over semantic and structural guidance.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论草图编辑需要同时处理高层语义变化和精确的局部重绘,这种组合对于稀疏且对风格敏感的线稿尤其具有挑战性。与自然图像不同,草图依赖极少的视觉线索,使得现有方法难以在保持整体连贯性的同时,协调全局语义修改与细粒度结构控制。我们提出 SketchAssist,一个交互式草图助手,它将指令引导的编辑与线稿引导的区域重绘统一起来,在保持整体构图的同时实现高效且可控的草图操作。为支持该任务,我们引入一个可控数据生成流程,该流程构建具有精确属性变化的结构化编辑序列,并在多步修改中保持结构对齐,同时通过风格保持变换扩展风格多样性。基于这些数据,SketchAssist 采用基于 DiT 的统一框架,使用多通道输入表示在单一接口中编码草图、掩码和引导信号。为进一步处理不同的编辑模式,我们将任务引导的混合专家(T-MoE)集成到低秩适配层中,实现对语义和结构引导的自适应控制。大量实验表明,该模型在两个任务上均达到最优性能,与近期方法相比,具有更强的指令遵循能力以及改进的结构和风格一致性。总之,我们的方法为草图编辑提供了一种实用且可控的解决方案。
英文原文摘要 1572 字符
Sketch editing requires jointly handling high-level semantic changes and precise local redrawing, a combination that is particularly challenging for sparse, style-sensitive line art. Unlike natural images, sketches rely on minimal visual cues, making it difficult for existing methods to reconcile global semantic modifications with fine-grained structural control while preserving overall coherence. We present SketchAssist, an interactive sketch assistant that unifies instruction-guided editing with line-guided region redrawing, enabling efficient and controllable sketch manipulation while preserving overall composition. To support this task, we introduce a controllable data generation pipeline that constructs structured edit sequences with precise attribute variations and maintains structural alignment across multi-step modifications, while expanding stylistic diversity via style-preserving transformations. Building on this data, SketchAssist adopts a unified framework based on DiT, using a multi-channel input representation to encode sketches, masks, and guidance signals within a single interface. To further handle different editing modes, we integrate a Task-guided Mixture-of-Experts (T-MoE) into LoRA layers, enabling adaptive control over semantic and structural guidance. Extensive experiments demonstrate state-of-the-art performance on both tasks, achieving strong instruction adherence and improved structural and style consistency compared to recent methods. Together, our method provide a practical and controllable solution for sketch editing.