超越多项选择：面向鲁棒视觉语言强化微调的可验证开放问答
Beyond Multiple Choice: Verifiable OpenQA for Robust Vision-Language RFT
为视觉语言模型提供可验证的开放问答数据构造与自动验证流程，可直接用于强化微调（RFT）和更可靠的基准评测，避免多项选择选项泄露带来的奖励信号偏差。
原文依据 We propose ReVeL (Rewrite and Verify by LLM), a framework that rewrites multiple-choice questions into open-form questions while keeping answers verifiable whenever possible.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多项选择问答（MCQA）一直是评估和强化微调（RFT）现代多模态语言模型的流行格式。其受限的输出格式允许简化、确定性的自动验证。然而，我们发现选项可能泄露可利用的信号，这使得准确率指标无法可靠地指示真实能力，并在RFT过程中鼓励显式或隐式的答案猜测行为。我们提出ReVeL（由LLM重写与验证），一个将多项选择题重写为开放形式问题、同时在可能时保持答案可验证的框架。该框架根据不同的答案类型对问题进行分类，并分别应用不同的重写和验证方案。当应用于RFT时，我们转换了2万个MCQA示例，并使用GRPO微调Qwen2.5-VL模型。在ReVeL-OpenQA上训练的模型在多项选择基准上达到与MCQA相当的准确率，并将开放问答准确率提升约六个百分点，表明相比基于MCQA的训练具有更好的数据效率和更鲁棒的奖励信号。当用于评估时，ReVeL还揭示了MCQA基准中高达20个百分点的分数膨胀（相对于开放问答），提高了评判准确率，并降低了成本和延迟。我们将公开代码和数据。
英文原文摘要 1338 字符
Multiple-choice question answering (MCQA) has been a popular format for evaluating and reinforcement fine-tuning (RFT) of modern multimodal language models. Its constrained output format allows for simplified, deterministic automatic verification.However, we find that the options may leak exploitable signals, which makes the accuracy metrics unreliable for indicating real capabilities and encourages explicit or implicit answer guessing behaviors during RFT. We propose ReVeL (Rewrite and Verify by LLM), a framework that rewrites multiple-choice questions into open-form questions while keeping answers verifiable whenever possible. The framework categorizes questions according to different answer types, apply different rewriting and verification schemes, respectively. When applied for RFT, we converted 20k MCQA examples and use GRPO to finetune Qwen2.5-VL models. Models trained on ReVeL-OpenQA match MCQA accuracy on multiple-choice benchmarks and improve OpenQA accuracy by about six percentage points, indicating better data efficiency and more robust reward signals than MCQA-based training. When used for evaluation, ReVeL also reveals up to 20 percentage points of score inflation in MCQA benchmarks (relative to OpenQA), improves judging accuracy, and reduces both cost and latency. We will release code and data publicly.