Omni IIE Bench：面向图像编辑模型实用能力的基准
Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Models
为指令式图像编辑模型提供按语义尺度分层的诊断基准，可用于暴露模型在属性修改与实体替换之间的性能不一致，并指导多轮编辑一致性的评测协议设计。
原文依据 nearly all models exhibit a significant performance degradation when transitioning from low-semantic-scale to high-semantic-scale tasks
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管基于指令的图像编辑（IIE）已取得显著进展，现有基准通过混合评测来追求任务广度。这种范式掩盖了专业应用中一个关键的失效模式：模型在不同语义尺度任务上的性能不一致。为填补这一空白，我们提出 Omni IIE Bench，一个高质量、人工标注的基准，专门用于诊断 IIE 模型在实际应用场景中的编辑一致性。Omni IIE Bench 采用创新的双轨诊断设计：（1）单轮一致性，由属性修改与实体替换的共享上下文任务对组成；（2）多轮协调，涉及跨越语义尺度的连续对话任务。该基准通过极为严格的多阶段人工筛选流程构建，包括由计算机视觉研究生执行的质量标准以及由专业设计师开展的行业相关性审查。我们使用 Omni IIE Bench 对 8 个主流 IIE 模型进行了全面评测。我们的分析首次量化了一个普遍存在的性能差距：几乎所有模型在从低语义尺度任务过渡到高语义尺度任务时都表现出显著的性能下降。Omni IIE Bench 为开发下一代更可靠、更稳定的 IIE 模型提供了关键的诊断工具与洞见。
英文原文摘要 1474 字符
While Instruction-based Image Editing (IIE) has achieved significant progress, existing benchmarks pursue task breadth via mixed evaluations. This paradigm obscures a critical failure mode crucial in professional applications: the inconsistent performance of models across tasks of varying semantic scales. To address this gap, we introduce Omni IIE Bench, a high-quality, human-annotated benchmark specifically designed to diagnose the editing consistency of IIE models in practical application scenarios. Omni IIE Bench features an innovative dual-track diagnostic design: (1) Single-turn Consistency, comprising shared-context task pairs of attribute modification and entity replacement; and (2) Multi-turn Coordination, involving continuous dialogue tasks that traverse semantic scales. The benchmark is constructed via an exceptionally rigorous multi-stage human filtering process, incorporating a quality standard enforced by computer vision graduate students and an industry relevance review conducted by professional designers. We perform a comprehensive evaluation of 8 mainstream IIE models using Omni IIE Bench. Our analysis quantifies, for the first time, a prevalent performance gap: nearly all models exhibit a significant performance degradation when transitioning from low-semantic-scale to high-semantic-scale tasks. Omni IIE Bench provides critical diagnostic tools and insights for the development of next-generation, more reliable, and stable IIE models.