SPOT：面向运动稳定 MLLM 引导视频分割的时空提示词优化
SPOT: Spatiotemporal Prompt Optimization for Motion-Stabilized MLLM-Guided Video Segmentation
提出一种仅通过约束 MLLM 输出行为（布朗桥损失和几何感知提示词质量损失）来提升视频分割时序一致性的方法，无需视频预训练或架构修改，为利用静态图像训练的视觉基础模型处理动态视频任务提供了可复用的正则化思路。
原文依据 a Brownian bridge loss that models object trajectories as endpoint-constrained Gaussian processes to ensure temporal smoothness
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLM）与视觉基础模型的协同框架在图像理解任务中展现出卓越性能，但在视频分割场景中面临严重的时序不一致挑战。现有方法主要依赖在静态图像上训练的 MLLM 生成逐帧分割提示词，忽略了视频运动的物理连续性。本文认为，视频理解任务的性能局限源于对模型输出行为的约束不足。因此，我们提出一种时空协同优化机制，仅通过约束 MLLM 输出行为即可实现时序一致的视频分割，无需大规模视频预训练或复杂的架构修改。我们的方法具有两个互补机制：布朗桥损失将物体轨迹建模为端点约束的高斯过程以确保时序平滑性，以及几何感知的提示词质量损失强制与目标结构的空间一致性。在指代表达视频分割和推理视频分割任务上的实验表明，我们的方法在 Ref-YouTube-VOS、Ref-DAVIS-2017、MeVIS、A2d-Sentences、JHMDB-Sentences 和 ReVOS 基准上显著超越最优技术。这项工作表明，对物理世界约束的显式建模可以释放静态训练的基础模型在动态视觉理解任务中的全部潜力。
英文原文摘要 1549 字符
The synergistic framework of multimodal large language models (MLLMs) and vision foundation models demonstrates exceptional performance in image understanding tasks, yet encounters severe temporal inconsistency challenges in video segmentation scenarios. Existing methods predominantly rely on MLLMs trained on static images to generate per-frame segmentation prompts, neglecting the physical continuity of video motion. This paper posits that performance limitations in video understanding tasks from inadequate constraints on model output behavior. Consequently, we propose a spatiotemporal co-optimization mechanism that achieves temporally consistent video segmentation solely by constraining MLLM output behavior, eliminating the need for large-scale video pretraining or complex architectural modifications. Our method features two complementary mechanisms: a Brownian bridge loss that models object trajectories as endpoint-constrained Gaussian processes to ensure temporal smoothness, and a geometry-aware prompt quality loss that enforces spatial consistency with target structures. Experiments on referring expression video segmentation and reasoning video segmentation tasks demonstrate that our method significantly surpasses state-of-the-art techniques on the Ref-YouTube-VOS, Ref-DAVIS-2017, MeVIS, A2d-Sentences, JHMDB-Sentences and ReVOS benchmarks. This work establishes that explicit modeling of physical world constraints can unlock the full potential of statically trained foundation models in dynamic visual understanding tasks.