通过蒸馏辅助的测试时自适应引导视频语义分割模型
Bootstrapping Video Semantic Segmentation Model via Distillation-assisted Test-Time Adaptation
将图像语义分割模型通过蒸馏 SAM2 的时序知识并配合轻量时序融合模块，在测试时自适应为视频语义分割模型，为无标注视频分割提供可复用范式。
原文依据 DiTTA distills SAM2's temporal segmentation knowledge into the ISS model during a brief, single-pass initialization phase
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论全监督视频语义分割（VSS）严重依赖密集标注的视频数据，限制了实际应用。另一种方案是逐帧应用预训练的图像语义分割（ISS）模型，避免了标注成本，但忽略了关键的时序一致性。近期的基础模型如 SAM2 能够实现高质量的掩码传播，但由于语义理解有限和计算开销大，直接用于 VSS 仍不实用。本文提出 DiTTA（蒸馏辅助的测试时自适应），一种新颖的框架，通过高效的测试时自适应（TTA）将 ISS 模型转换为时序感知的 VSS 模型，无需标注视频。DiTTA 在短暂的、单次遍历的初始化阶段将 SAM2 的时序分割知识蒸馏到 ISS 模型中，并辅以一个轻量级的时序融合模块来聚合跨帧上下文。关键的是，即使使用非常有限的部分视频片段（例如初始 10%）进行自适应，DiTTA 也能实现稳健的泛化，显著优于在推理过程中反复调用 SAM2 的零样本细化方法。在 VSPW 和 Cityscapes 上的大量实验证明了 DiTTA 的有效性，相对于全监督 VSS 方法取得了有竞争力或更优的性能，从而为现实世界的 VSS 任务提供了一种实用且无需标注的解决方案。
英文原文摘要 1393 字符
Fully supervised Video Semantic Segmentation (VSS) relies heavily on densely annotated video data, limiting practical applicability. Alternatively, applying pre-trained Image Semantic Segmentation (ISS) models frame-by-frame avoids annotation costs but ignores crucial temporal coherence. Recent foundation models such as SAM2 enable high-quality mask propagation yet remain impractical for direct VSS due to limited semantic understanding and computational overhead. In this paper, we propose DiTTA (Distillation-assisted Test-Time Adaptation), a novel framework that converts an ISS model into a temporally-aware VSS model through efficient test-time adaptation (TTA), without annotated videos. DiTTA distills SAM2's temporal segmentation knowledge into the ISS model during a brief, single-pass initialization phase, complemented by a lightweight temporal fusion module to aggregate cross-frame context. Crucially, DiTTA achieves robust generalization even when adapting with highly limited partial video snippets (e.g., initial 10%), significantly outperforming zero-shot refinement approaches that repeatedly invoke SAM2 during inference. Extensive experiments on VSPW and Cityscapes demonstrate DiTTA's effectiveness, achieving competitive or superior performance relative to fully-supervised VSS methods, thus providing a practical and annotation-free solution for real-world VSS tasks.