MatAnyone 2:通过学习型质量评估器扩展视频抠图
MatAnyone 2: Scaling Video Matting via a Learned Quality Evaluator
提出无真值质量评估器(QE)生成像素级评估图,可用于训练时在线抑制错误区域或离线筛选数据构建大规模视频抠图数据集,为视频抠图提供可复用的质量反馈与数据扩展方案。
原文依据 a learned Quality Evaluator (QE) that assesses semantic and boundary quality of alpha mattes without ground truth
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频抠图仍受限于现有数据集的规模和真实感。虽然利用分割数据可以增强语义稳定性,但缺乏有效的边界监督往往导致抠图结果类似分割、缺少精细细节。为此,我们引入一个学习型质量评估器(QE),无需真值即可评估alpha抠图的语义和边界质量。它生成逐像素的评估图,识别可靠区域和错误区域,从而实现细粒度质量评估。QE通过两种方式扩展视频抠图:(1)在训练中作为在线抠图质量反馈以抑制错误区域,提供全面监督;(2)作为离线数据筛选模块,结合领先的视频和图像抠图模型的优势,提升标注质量。这一过程使我们构建了一个大规模真实世界视频抠图数据集VMReal,包含28K个片段和2.4M帧。为处理长视频中的大幅外观变化,我们引入参考帧训练策略,在局部窗口之外纳入长程帧以进行有效训练。我们的MatAnyone 2在合成和真实世界基准上均达到最优性能,在所有指标上超越先前方法。
英文原文摘要 1303 字符
Video matting remains limited by the scale and realism of existing datasets. While leveraging segmentation data can enhance semantic stability, the lack of effective boundary supervision often leads to segmentation-like mattes lacking fine details. To this end, we introduce a learned Quality Evaluator (QE) that assesses semantic and boundary quality of alpha mattes without ground truth. It produces a pixel-wise evaluation map that identifies reliable and erroneous regions, enabling fine-grained quality assessment. The QE scales up video matting in two ways: (1) as an online matting-quality feedback during training to suppress erroneous regions, providing comprehensive supervision, and (2) as an offline selection module for data curation, improving annotation quality by combining the strengths of leading video and image matting models. This process allows us to build a large-scale real-world video matting dataset, VMReal, containing 28K clips and 2.4M frames. To handle large appearance variations in long videos, we introduce a reference-frame training strategy that incorporates long-range frames beyond the local window for effective training. Our MatAnyone 2 achieves state-of-the-art performance on both synthetic and real-world benchmarks, surpassing prior methods across all metrics.