EduDiag：面向大型多模态模型错误追踪与纠正的教育诊断推理基准
EduDiag: A Benchmark for Educational Diagnostic Reasoning with Error Tracing and Correction on Large Multimodal Models
该论文提出的教育诊断推理任务要求LMM从错误答案逆向重建错误推理链并生成纠正反馈，为多模态推理评估提供了新方向；其构建的8K错误推理链数据集和GRPO优化策略可复用于提升视觉语言模型的推理与纠错能力。
原文依据 we introduce Edudiag benchmark, requiring LMMs to reconstruct erroneous reasoning chains from incorrect answers and generate corrective feedback.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型多模态模型（LMM）在多模态推理上取得了令人瞩目的性能，已成为智能问答系统进步的关键技术。在真实教育场景中，有效的教学远不止于提供答案。经验丰富的教师会分析学生的错误答案以追溯潜在错误并提供纠正反馈，这种能力被称为教育诊断推理，而现有LMM对此能力的探索仍不充分。为弥补这一研究空白，我们推出了EduDiag基准，要求LMM从错误答案中重建错误推理链并生成纠正反馈。通过AI辅助标注流程与严格的人工验证，我们构建了8K条错误推理链及相应反馈，涵盖三个代表性教育领域：常识、科学和数学。对28个领先LMM的广泛评估表明，EduDiag是一个具有挑战性的测试平台，即使领先的专有LMM也表现不佳，而对开源LMM进行监督微调（SFT）仅带来边际性能提升。此外，我们进行了分析实验，并识别出教育诊断推理的三个关键见解：（i）有效的错误追踪仍是主要瓶颈，而SFT模型仍无法反向识别常见错误。（ii）组相对策略优化（GRPO）缓解了这一瓶颈并提升了性能。（iii）经GRPO优化的LMM能够基于其自构建的错误推理链为多项选择题生成看似合理但具有挑战性的干扰项。我们相信EduDiag为评估先进LMM提供了新方向。
英文原文摘要 1736 字符
Large multimodal models (LMMs) have achieved impressive performance on multimodal reasoning, becoming crucial technology for the advancement of intelligent question-answering systems. In real-world educational scenarios, effective teaching extends far beyond providing answers. Experienced teachers analyze students' incorrect answers to trace underlying errors and provide corrective feedback, termed educational diagnostic reasoning, a capability that remains under-explored in existing LMMs. To bridge this research gap, we introduce Edudiag benchmark, requiring LMMs to reconstruct erroneous reasoning chains from incorrect answers and generate corrective feedback. Through an AI-assisted annotation pipeline with rigorous human verification, we create 8K erroneous reasoning chains and corresponding feedback, spanning three representative educational domains: commonsense, science, and mathematics. Extensive evaluation across 28 leading LMMs highlights Edudiag as a challenging testbed, where even leading proprietary LMMs struggle on it and supervised fine-tuning (SFT) on open-source LMMs achieves marginal performance gains. Moreover, we conduct analysis experiments and identify three critical insights for educational diagnostic reasoning: (i) Effective error tracing remains the primary bottleneck, while SFT models still fail to reversely identify errors that commonly occur. (ii) Group relative policy optimization (GRPO) mitigates this bottleneck and boosts performance. (iii) LMMs optimized with GRPO can generate plausible yet challenging distractors for multiple-choice questions based on their self-constructed erroneous reasoning chains. We believe Edudiag provides a new direction for evaluating the advanced LMMs.