UniRefiner：通过对比寄存器教预训练ViT自行处置伪影
UniRefiner: Teaching Pre-trained ViTs to Self-Dispose Dross via Contrastive Register
提出用对比寄存器（contrastive registers）在少量微调下净化预训练ViT中的伪影词元，可直接用于提升任何ViT在分割、深度估计等密集预测任务上的空间表征质量。
原文依据 UniRefiner uses contrastive registers to explicitly isolate and redistribute spurious tokens via a dual objective
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于视觉Transformer（ViT）的表征学习发展迅速，然而大规模模型在空间敏感任务中的效用受到伪影词元的阻碍。此前缓解这一问题的努力有限，通常将这些伪影狭隘地定义为简单的高范数离群点。我们认为这一范围不够充分。对于密集预测任务，我们主张任何未能编码位置对齐语义的词元都应被视为伪影。这一定义拓宽后揭示了一个更复杂的问题，促使我们系统地对破坏空间表征的三种基本伪影词元类型进行分类和刻画。基于这一全面诊断，我们提出UniRefiner，一个通用的精炼框架，教预训练ViT自行处置这些伪影。UniRefiner使用对比寄存器，通过双重目标显式隔离并重新分配伪影词元：（i）将图像词元与过滤后的常规词元对齐以保留语义，（ii）将寄存器词元与检测到的伪影词元对齐以捕获伪影信号。我们的方法仅需在5k图像上微调几个epoch即可精炼多种ViT，包括EVA-CLIP-8B和InternViT-6B等巨型模型。实验证明了一致且显著的改进：值得注意的是，精炼后的EVA-CLIP-8B在ADE20K上达到51.9%的平均交并比（+9.4%），超过了DINOv2（49.1%）等专用视觉模型，同时零样本分割精度最高提升22%。UniRefiner释放了现有大规模基础模型的潜在空间能力，为其更广泛的应用铺平了道路。
英文原文摘要 1681 字符
Representation learning with Vision Transformers (ViTs) has advanced rapidly, yet the utility of large-scale models in spatially sensitive tasks is hindered by spurious tokens. Prior efforts to mitigate this have been limited, often defining these artifacts narrowly, for example, as simple high-norm outliers. We argue that this scope is insufficient. For dense prediction tasks, we posit that any token failing to encode location-aligned semantics should be treated as a spurious artifact. This broader definition reveals a more complex problem, leading us to systematically categorize and characterize three fundamental types of spurious tokens that corrupt spatial representations. Based on this comprehensive diagnosis, we propose UniRefiner, a universal refinement framework that teaches pre-trained ViTs to self-dispose of these artifacts. UniRefiner uses contrastive registers to explicitly isolate and redistribute spurious tokens via a dual objective: (i) it aligns image tokens with filtered regular tokens to preserve semantics, and (ii) it aligns register tokens with detected spurious tokens to capture the spurious signals. Our method requires only a few epochs of fine-tuning on 5k images to refine diverse ViTs, including massive models like EVA-CLIP-8B and InternViT-6B. Experiments demonstrate consistent and significant improvements: notably, the refined EVA-CLIP-8B achieves 51.9% mIoU on ADE20K (+9.4%), surpassing specialized vision models like DINOv2 (49.1%), while zero-shot segmentation accuracy improves by up to 22%. UniRefiner unlocks the latent spatial potential of existing large-scale foundation models, paving the way for their broader application.