AV-Reasoner：面向多模态大模型的线索锚定音视频计数能力改进与基准构建
AV-Reasoner: Improving and Benchmarking Clue-Grounded Audio-Visual Counting for MLLMs
对做视频多模态推理的人，这篇提供了两点可复用：一是用 GRPO + 课程学习把相关任务（如时序定位、事件定位）的能力迁移到计数这类稀疏监督任务上；二是 CG-AV-Counting 的黑盒/白盒双协议评测，可作为检验模型推理链是否真正依赖视觉线索的测试床。
原文依据 we propose AV-Reasoner, a model trained with GRPO and curriculum learning to generalize counting ability from related tasks
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管视频理解已取得进展，当前多模态大模型在计数任务上仍表现不佳。现有基准受限于视频时长短、查询闭集、缺少线索标注以及多模态覆盖不足。本文提出 CG-AV-Counting，一个人工标注的线索锚定计数基准，包含 497 段长视频上的 1,027 个多模态问题与 5,845 条标注线索。它同时支持黑盒与白盒评测，可作为端到端计数与基于推理的计数的综合测试平台。为探索提升模型计数能力的途径，我们提出 AV-Reasoner，该模型采用 GRPO 与课程学习训练，从相关任务中泛化计数能力。AV-Reasoner 在多个基准上取得 SOTA 结果，验证了强化学习的有效性。然而实验表明，在域外基准上，语言空间中的推理带来的性能提升有限，说明仍需更鲁棒的跨域推理机制。
英文原文摘要 1025 字符
Despite progress in video understanding, current MLLMs struggle with counting tasks. Existing benchmarks are limited by short videos, close-set queries, lack of clue annotations, and weak multimodal coverage. In this paper, we introduce CG-AV-Counting, a manually-annotated clue-grounded counting benchmark with 1,027 multimodal questions and 5,845 annotated clues over 497 long videos. It supports both black-box and white-box evaluation, serving as a comprehensive testbed for both end-to-end and reasoning-based counting. To explore ways to improve model's counting capability, we propose AV-Reasoner, a model trained with GRPO and curriculum learning to generalize counting ability from related tasks. AV-Reasoner achieves SOTA results across multiple benchmarks, demonstrating the effectiveness of reinforcement learning. However, experiments reveal that on out-of-domain benchmarks, reasoning in the language space offers limited performance gains, suggesting the need for more robust cross-domain reasoning mechanisms.