构建具有人机协同监督的精确视频语言
Building a Precise Video Language with Human-AI Oversight
提出以「批判式人机协同」替代全人工写描述：模型先出预描述，专家只写纠错批判，再把批判与前后描述偏好用 SFT/DPO/推理时扩展训练开源 VLM，可直接迁移到任何需要高质量细粒度视频/图像描述数据的视觉任务。
原文依据 we introduce a critique-based human-AI (CHAI) oversight framework, where trained human experts provide correctional critiques
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频语言模型（VLM）通过自然语言学习对动态视觉世界进行推理。我们提出一套开放数据集、基准与可扩展监督方案，以实现精确的视频描述。首先，我们定义了一套结构化规范，用于描述主体、场景、运动、空间与相机动态，并以与专业视频创作者（如电影制作者）共同制定的数百个精心定义的视觉原语为基础。其次，为整理高质量描述，我们引入基于批判的人机协同（CHAI）监督框架：受训的人类专家对模型生成的预描述提供纠错性批判，将其修订为更优的后描述。这种分工将文本生成交给模型，使人类能更专注于核验，从而提升标注的准确性与效率。此外，这些批判以及预描述与后描述之间的偏好，为通过 SFT、DPO 与推理时扩展来改进开源模型（Qwen3-VL）在描述生成、奖励建模与批判生成方面的能力提供了丰富监督。我们的消融实验表明，由该监督框架保障的批判质量（精确率、召回率与建设性）直接决定下游性能。在适度的专家监督下，所得模型甚至优于 Gemini-3.1-Pro 等闭源模型。最后，我们将该方法应用于对大规模专业视频（如电影、广告、游戏）重新生成描述，并微调 Wan 等视频生成模型，使其更好地遵循长达 400 词的详细提示，从而在相机运动、角度、镜头、对焦、视角与取景等电影摄影方面实现更精细的控制。总体而言，我们的结果表明，精确的规范与人机协同监督是实现专业级视频理解与生成的关键。
英文原文摘要 1869 字符
Video-language models (VLMs) learn to reason about the dynamic visual world through natural language. We introduce a suite of open datasets, benchmarks, and recipes for scalable oversight that enable precise video captioning. First, we define a structured specification for describing subjects, scenes, motion, spatial, and camera dynamics, grounded by hundreds of carefully defined visual primitives developed with professional video creators such as filmmakers. Next, to curate high-quality captions, we introduce a critique-based human-AI (CHAI) oversight framework, where trained human experts provide correctional critiques to revise model-generated pre-captions into improved post-captions. This division of labor improves annotation accuracy and efficiency by offloading text generation to models, allowing humans to better focus on verification. Additionally, these critiques and preferences between pre- and post-captions provide rich supervision for improving open-source models (Qwen3-VL) on caption generation, reward modeling, and critique generation through SFT, DPO, and inference-time scaling. Our ablations show that critique quality in precision, recall, and constructiveness, ensured by our oversight framework, directly governs downstream performance. With modest expert supervision, the resulting model outperforms even closed-source models such as Gemini-3.1-Pro. Finally, we apply our approach to re-caption large-scale professional videos (e.g., films, commercials, games) and fine-tune video generation models such as Wan to better follow detailed prompts of up to 400 words, achieving finer control over cinematography including camera motion, angle, lens, focus, point of view, and framing. Overall, our results show that precise specification and human-AI oversight are key to achieving professional-level video understanding and generation.