图像分割 · CVPR2026 ·2026.06.03

VIRST:面向时空分割的视频指令推理助手

VIRST: Video-Instructed Reasoning Assistant for SpatioTemporal Segmentation

Jihwan Hong、Jaeyoung Do

与你有关的部分

将分割感知的视频特征通过时空融合注入视觉语言主干,并用时序动态锚框更新器在大运动、遮挡与重现下维持稳定时序线索,为指代视频分割与推理式分割提供可复用的端到端范式。

原文依据 employs the Temporal Dynamic Anchor Updater to maintain temporally adjacent anchor frames that provide stable temporal cues under large motion, occlusion, and reappearance
可迁移到 指代视频目标分割视频目标分割视觉语言推理

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

指代视频目标分割(RVOS)旨在根据自然语言描述在视频中分割目标对象。然而,将视觉语言模型与独立传播模块耦合的固定关键帧方法,往往无法捕捉快速变化的时空动态,也难以处理需要多步推理的查询,导致在运动密集型和推理导向的视频上性能急剧下降,超出静态 RVOS 基准的适用范围。为解决这些局限,我们提出 VIRST(面向时空分割的视频指令推理助手),这是一个在单一模型内统一全局视频推理与像素级掩码预测的端到端框架。VIRST 通过时空融合(STF)连接语义表示与分割表示,将分割感知的视频特征融合进视觉语言主干,并采用时序动态锚框更新器维护时序相邻的锚框帧,在大运动、遮挡与重现情况下提供稳定的时序线索。这一统一设计在真实且具有挑战性的条件下于多个 RVOS 基准上取得最优结果,展现出对指代导向与推理导向两类设定的强泛化能力。

英文原文摘要 1275 字符

Referring Video Object Segmentation (RVOS) aims to segment target objects in videos based on natural language descriptions. However, fixed keyframe-based approaches that couple a vision language model with a separate propagation module often fail to capture rapidly changing spatiotemporal dynamics and to handle queries requiring multi-step reasoning, leading to sharp performance drops on motion-intensive and reasoning-oriented videos beyond static RVOS benchmarks. To address these limitations, we propose VIRST (Video-Instructed Reasoning Assistant for Spatio-Temporal Segmentation), an end-to-end framework that unifies global video reasoning and pixel-level mask prediction within a single model. VIRST bridges semantic and segmentation representations through the Spatio-Temporal Fusion (STF), which fuses segmentation-aware video features into the vision-language backbone, and employs the Temporal Dynamic Anchor Updater to maintain temporally adjacent anchor frames that provide stable temporal cues under large motion, occlusion, and reappearance. This unified design achieves state-of-the-art results across diverse RVOS benchmarks under realistic and challenging conditions, demonstrating strong generalization to both referring and reasoning oriented settings.

标签

会议 CVPR2026
页码 3410-3420
Bibkey
来源 CVF Open Access