鲁棒性与安全 · CVPR2026 ·2026.06.03

鲁棒的提示式视频目标分割

Robust Promptable Video Object Segmentation

Sohyun Lee、Yeho Gwon、Lukas Hoyer、Konrad Schindler、Christos Sakaridis、Suha Kwak

与你有关的部分

把「记忆中的目标级表征」作为条件去门控低秩适配(LoRA 式)参数,使鲁棒化过程按被跟踪目标分别进行并保持时序一致;这套 memory-conditioned gated adaptation 可直接迁移到其他提示式视频模型(如 SAM2 类跟踪器)做测试时/轻量鲁棒适配。

原文依据 MoGA leverages object-specific representations maintained in memory across frames to condition the robustification process
可迁移到 promptable video object segmentationvideo object segmentationvideo object tracking

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

提示式视频目标分割(PVOS)模型在输入损坏下的性能会显著下降,这阻碍了 PVOS 在安全关键领域的部署。本文首次对鲁棒 PVOS(RobustPVOS)进行了系统性研究。我们首先构建了一个新的综合性基准,包含两个真实世界评测数据集,涵盖 351 段视频片段和 2,500 余个目标掩码,均处于真实世界的恶劣条件下。同时,我们对现有 VOS 数据集施加多样且随时间变化的损坏,从而生成合成训练数据。此外,我们提出了一种新的 RobustPVOS 方法,称为记忆-目标条件门控秩适配(Memory-object-conditioned Gated-rank Adaptation,MoGA)。成功实现 RobustPVOS 的关键有两点:有效处理目标特定的退化,以及确保预测的时序一致性。MoGA 利用跨帧保存在记忆中的目标特定表征来对鲁棒化过程进行条件化,使模型能够以时序一致的方式对每个被跟踪目标区别处理。在我们基准上的大量实验验证了 MoGA 的有效性,在合成与真实世界数据集上、在多种损坏类型下均取得一致且显著的提升,为未来 RobustPVOS 研究建立了强有力的基线。我们的基准已公开于 https://sohyun-l.github.io/RobustPVOS_project_page/。

英文原文摘要 1398 字符

The performance of promptable video object segmentation (PVOS) models substantially degrades under input corruptions, which prevents PVOS deployment in safety-critical domains. This paper offers the first comprehensive study on robust PVOS (RobustPVOS). We first construct a new, comprehensive benchmark with two real-world evaluation datasets of 351 video clips and more than 2,500 object masks under real-world adverse conditions. At the same time, we generate synthetic training data by applying diverse and temporally varying corruptions to existing VOS datasets. Moreover, we present a new RobustPVOS method, dubbed Memory-object-conditioned Gated-rank Adaptation (MoGA). The key to successfully performing RobustPVOS is two-fold: effectively handling object-specific degradation and ensuring temporal consistency in predictions. MoGA leverages object-specific representations maintained in memory across frames to condition the robustification process, which allows the model to handle each tracked object differently in a temporally consistent way. Extensive experiments on our benchmark validate MoGA's efficacy, showing consistent and significant improvements across diverse corruption types on both synthetic and real-world datasets, establishing a strong baseline for future RobustPVOS research. Our benchmark is publicly available at https://sohyun-l.github.io/RobustPVOS_project_page/.

标签

会议 CVPR2026
页码 39838-39847
Bibkey
来源 CVF Open Access