Inter-Edit:首个面向交互式指令图像编辑的基准
Inter-Edit: First Benchmark for Interactive Instruction-Based Image Editing
提出以「简洁文本指令 + 不精确空间掩码」作为编辑条件的设定,并给出百万级模拟真实用户掩码的数据生成流水线,可直接用于训练和评测指令式编辑模型在弱定位输入下的鲁棒性。
原文依据 generates high-quality edits from a more intuitive combination: concise text instructions and imprecise spatial guidance
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论精确且可控的图像编辑仍是一项重大挑战。现有方法通常依赖文本提示,但仅凭描述实现准确的空间定位本身就很困难。基于掩码的方法虽然提供了更好的控制,但通常要求用户给出过于精确的标注,从而加重了用户负担并导致不自然的结果。为弥合这一差距,我们提出了交互式指令图像编辑(I^3E)任务,它从一种更直观的组合——简洁的文本指令与不精确的空间引导——生成高质量编辑。为应对合适数据严重匮乏的问题,我们提出了一条高效的数据生成流水线,用于构建 Inter-Edit,这是一个新的百万级训练数据集,可模拟真实用户掩码——并非严格与分割区域对齐。我们还提出了一个全面的基准,其中包含精心人工标注的测试集,覆盖多样化的、依赖定位的编辑场景以及真实的用户交互模式。为评测该任务,我们引入了一套新的位置感知指标,其与人类感知判断高度相关。最后,我们基于 Inter-Edit 训练了三个基线模型。大量实验表明,我们的方法显著提升了 I^3E 的性能,在定位和编辑质量上取得大幅改进,并优于现有最优模型。Inter-Edit 数据集及所有相关代码已在 https://github.com/Delong-liu-bupt/Inter-Edit 公开。
英文原文摘要 1550 字符
Precise and controllable image editing remains a significant challenge. Current methods often rely on text prompts, but achieving accurate spatial localization solely through descriptions is inherently difficult. Mask-based approaches, though offering better control, typically require overly precise user annotations, thus increasing user burden and leading to unnatural results. To bridge this gap, we introduce the **I**nteractive **I**nstruction-based **I**mage **E**diting (I^3E) task, which generates high-quality edits from a more intuitive combination: concise text instructions and imprecise spatial guidance. To address the critical lack of suitable data, we propose an efficient pipeline to generate Inter-Edit, a new million-scale training dataset that simulates realistic user masks---not strictly segment-aligned. We also present a comprehensive benchmark, featuring a meticulously human-annotated test set that captures diverse, localization-dependent editing scenarios and realistic user interaction patterns. To evaluate this task, we introduce a new suite of position-aware metrics that strongly correlate with human perceptual judgments. Finally, we develop three baseline models trained on Inter-Edit. Extensive experiments demonstrate that our methods significantly enhance I_3E performance, achieving substantial improvements in localization and edit quality, and outperforming existing state-of-the-art models. The Inter-Edit dataset and all related code are publicly available at https://github.com/Delong-liu-bupt/Inter-Edit.