NAF：基于邻域注意力滤波的零样本特征上采样
NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filtering
NAF 提供了一种与具体 VFM 无关的零样本特征上采样器，可直接插在任意视觉基础模型（如 DINOv2、SAM）之后，为分割、深度估计等像素级任务恢复高分辨率特征，无需针对每个 VFM 重新训练；其跨尺度邻域注意力 + RoPE 的权重预测方式也可迁移到其他需要内容自适应滤波的环节。
原文依据 NAF operates zero-shot: it upsamples features from any VFM without retraining
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉基础模型（VFM）提取的是空间下采样后的表征，这给像素级任务带来了挑战。现有的上采样方法面临一个根本性权衡：经典滤波器速度快、适用面广，但依赖固定的形式；现代上采样器通过可学习的、针对特定 VFM 的形式获得了更优的精度，代价是每换一个 VFM 就要重新训练。我们提出邻域注意力滤波（NAF），通过跨尺度邻域注意力和旋转位置编码（RoPE）学习自适应的空间与内容权重，且仅由高分辨率输入图像引导，从而弥合了这一差距。NAF 以零样本方式工作：无需重新训练即可对任意 VFM 的特征进行上采样，使其成为首个在多个下游任务上超越针对特定 VFM 的上采样器并达到最优性能的、与 VFM 无关的架构。它保持高效率，可扩展到 2K 特征图，并以 18 FPS 重建中间分辨率特征图。除特征上采样之外，NAF 在图像复原上也展现出强劲性能，凸显了其通用性。代码与检查点见 https://github.com/valeoai/NAF。
英文原文摘要 1179 字符
Vision Foundation Models (VFMs) extract spatially downsampled representations, posing challenges for pixel-level tasks. Existing upsampling approaches face a fundamental trade-off: classical filters are fast and broadly applicable but rely on fixed forms, while modern upsamplers achieve superior accuracy through learnable, VFM-specific forms at the cost of retraining for each VFM. We introduce Neighborhood Attention Filtering (NAF), which bridges this gap by learning adaptive spatial-and-content weights through Cross-Scale Neighborhood Attention and Rotary Position Embeddings (RoPE), guided solely by the high-resolution input image. NAF operates zero-shot: it upsamples features from any VFM without retraining, making it the first VFM-agnostic architecture to outperform VFM-specific upsamplers and achieve state-of-the-art performance across multiple downstream tasks. It maintains high efficiency, scaling to 2K feature maps and reconstructing intermediate-resolution maps at 18 FPS. Beyond feature upsampling, NAF demonstrates strong performance on image restoration, highlighting its versatility. Code and checkpoints are available at https://github.com/valeoai/NAF.