教 DINOv3 理解部分 3D 几何:一种自监督的几何感知方法
Teaching DINOv3 About Partial 3D Geometry: A Self-Supervised Geometry-Aware Approach
用 LoRA 自监督微调 DINOv3 特征,使其对部分观测(partial shape)鲁棒,可直接替换部分形状匹配与左右对称预测流程中的特征描述子。
原文依据 we propose a method to refine DINO features using LoRA-based self-supervised learning, enabling the generation of feature descriptors that are robust to partiality
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论部分形状匹配是 3D 视觉中一个关键但尚未被充分探索的问题,与现实场景高度相关,因为现实中的形状往往只能被部分观测。现有特征描述子在这一设定下存在困难:传统表示要么难以处理部分形状的边界,要么严重依赖形状的空间位置。尽管已有方法采用 DINO 特征进行部分形状匹配,但这些特征本身并不适合处理部分观测。在本工作中,我们提出一种基于 LoRA 的自监督学习方法来精炼 DINO 特征,从而生成对部分性鲁棒的特征描述子。相比传统特征或视觉基础模型特征,我们的特征显著提升了部分形状匹配的性能。此外,将其集成到现有的部分形状匹配流程中,我们在部分形状匹配与左右预测基准上取得了最优结果。
英文原文摘要 1017 字符
Partial shape matching is a crucial yet underexplored problem in 3D vision, with significant relevance to real-world scenarios where shapes are often only partially observed. Existing feature descriptors face difficulties in this setting, as traditional representations either struggle with the boundaries of partial shapes or heavily depend on the shape's spatial position. While existing approaches have employed DINO features for partial shape matching, these features are not inherently suited for handling partial observations. In this work, we propose a method to refine DINO features using LoRA-based self-supervised learning, enabling the generation of feature descriptors that are robust to partiality. Our features substantially improve performance on partial shape matching compared to traditional or vision foundation features. Additionally, when integrated into existing partial shape matching pipelines, we achieve state-of-the-art results on partial shape matching and left-right prediction benchmarks.