Upsample Anything:一个简单且难以超越的特征上采样基线
Upsample Anything: A Simple and Hard to Beat Baseline for Feature Upsampling
把高斯泼溅与联合双边上采样结合成逐图像测试时优化的各向异性高斯核,可在不重训练的情况下把任意 ViT 的 14x/16x 下采样特征、深度图或概率图恢复为像素级高分辨率输出,适合作为分割/深度等密集预测任务中冻结基础模型后的通用后处理插件。
原文依据 a lightweight test-time optimization (TTO) framework that restores low-resolution features to high-resolution, pixel-wise outputs without any training
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 Upsample Anything,一个轻量级的测试时优化(TTO)框架,无需任何训练即可将低分辨率特征恢复为高分辨率的逐像素输出。尽管视觉基础模型在多种下游任务上展现出强大的泛化能力,但其表征通常被下采样 14 倍/16 倍(如 ViT),这限制了它们在像素级应用中的直接使用。现有的特征上采样方法依赖针对特定数据集的重训练或繁重的隐式优化,限制了可扩展性与泛化能力。Upsample Anything 通过一种简单的逐图像优化解决了这些问题:该优化学习一个各向异性高斯核,融合空间线索与值域线索,从而有效地将高斯泼溅与联合双边上采样联系起来。所学到的核充当一个通用的、边缘感知的算子,可无缝迁移到不同架构与模态之间,实现对特征、深度或概率图的精确高分辨率重建。它在每张 224x224 图像上仅需约 0.419 秒,并在语义分割、深度估计以及深度图与概率图上采样上均取得了最优性能。项目主页:https://seominseok0429.github.io/Upsample-Anything/
英文原文摘要 1314 字符
We present Upsample Anything, a lightweight test-time optimization (TTO) framework that restores low-resolution features to high-resolution, pixel-wise outputs without any training. Although Vision Foundation Models demonstrate strong generalization across diverse downstream tasks, their representations are typically downsampled by 14x/16x (e.g., ViT), which limits their direct use in pixel-level applications. Existing feature upsampling approaches depend on dataset-specific retraining or heavy implicit optimization, restricting scalability and generalization. Upsample Anything addresses these issues through a simple per-image optimization that learns an anisotropic Gaussian kernel combining spatial and range cues, effectively bridging Gaussian Splatting and Joint Bilateral Upsampling. The learned kernel acts as a universal, edge-aware operator that transfers seamlessly across architectures and modalities, enabling precise high-resolution reconstruction of features, depth, or probability maps. It runs in only ~0.419 \text s per 224x224 image and achieves state-of-the-art performance on semantic segmentation, depth estimation, and both depth and probability map upsampling. Project page: \href https://seominseok0429.github.io/Upsample-Anything/ https://seominseok0429.github.io/Upsample-Anything/