SARL-STG：用于精炼多模态大语言模型时空视频定位的空间感知强化学习框架
SARL-STG: A Spatially Aware Reinforcement Learning Framework for Refining MLLMs in Spatio-Temporal Video Grounding
把开放词表检测器的空间定位质量作为强化学习奖励信号，反向驱动 MLLM 的时序推理，为视觉语言模型在视频定位任务中提供了一种可复用的「空间反馈时序」训练范式。
原文依据 a spatial knowledge-injected reward mechanism that uses spatial grounding quality as discriminative signals for temporal refinement
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论时空视频定位（STVG）要求模型在空间与时序两个维度上定位目标。尽管近期取得了进展，现有方法仍难以处理语言描述中复杂而细粒度的空间语义，导致误差从时序定位阶段向空间定位阶段传播。我们将这一局限归因于时序预测与空间预测之间缺乏迭代精炼。为应对这些挑战，我们提出 SARL-STG，这是首个面向 STVG 的基于强化学习的框架。它通过多阶段训练，在时序模块与空间模块之间进行动态交互，逐步精炼时空定位。具体而言，SARL-STG 包含：（1）一个统一架构，将用于时序推理的预训练多模态大语言模型与用于空间定位的开放词表检测器无缝集成；（2）一种分层强化学习训练策略，从粗粒度时序优化逐步过渡到细粒度时空优化；（3）一种注入空间知识的奖励机制，利用空间定位质量作为判别信号来精炼时序。为支持大规模训练，我们还构建了 STVG-Wild，一个带有多样化时空标注的大规模数据集。实验表明，我们的方法在多个基准（HCSTVG、VidSTG、Charades-STA 等）上取得了最优性能，显著减少了误差累积，并提升了时序与空间定位的准确率。
英文原文摘要 1459 字符
Spatio-Temporal Video Grounding (STVG) requires models to localize objects both spatially and temporally. Despite recent progress, existing methods struggle with complex and fine-grained spatial semantics in language descriptions, leading to error propagation from temporal to spatial grounding stages. We attribute this limitation to the lack of iterative refinement between temporal and spatial predictions. To address these challenges, we propose SARL-STG, the first RL-based framework for STVG. It progressively refines spatio-temporal grounding through multi-stage training with dynamic interaction between temporal and spatial modules. Specifically, SARL-STG contains: (1) a unified architecture that seamlessly integrates a pretrained MLLM for temporal reasoning with an open-vocabulary detector for spatial localization, (2) a hierarchical RL training strategy that progresses from coarse temporal to fine-grained spatio-temporal optimization, and (3) a spatial knowledge-injected reward mechanism that uses spatial grounding quality as discriminative signals for temporal refinement. To facilitate training at scale, we also construct STVG-Wild, a large-scale dataset with diverse spatio-temporal annotations. Experiments demonstrate that our method achieves state-of-the-art performance on multiple benchmarks (HCSTVG, VidSTG, Charades-STA, etc.), significantly reducing error accumulation and enhancing both temporal and spatial grounding accuracy.