CLIPoint3D:基于语言锚定的少样本无监督三维点云域自适应
CLIPoint3D: Language-Grounded Few-Shot Unsupervised 3D Point Cloud Domain Adaptation
把点云投影成多张深度图后直接复用冻结的 CLIP 主干,配合知识驱动的提示词微调和熵引导的视角采样,为 3D 域自适应提供了一条无需训练重型 3D 编码器的轻量路线;其中的最优传输对齐损失与不确定性感知原型对齐损失也可迁移到其他跨域特征对齐场景。
原文依据 projects 3D samples into multiple depth maps and exploits the frozen CLIP backbone
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期诸如 CLIP 的视觉语言模型展现出令人印象深刻的跨模态推理能力,并已从图像扩展到三维感知。然而,这些模型在域偏移下仍然脆弱,尤其是在从合成点云自适应到真实世界点云时。传统的三维域自适应方法依赖沉重的可训练编码器,虽能取得较高精度,却以效率为代价。我们提出 CLIPoint3D,这是首个基于 CLIP 构建的少样本无监督三维点云域自适应框架。该方法将三维样本投影为多张深度图,并利用冻结的 CLIP 主干,通过一种知识驱动的提示词微调方案对其进行精化,该方案将高层语言先验与来自轻量三维编码器的几何线索相结合。为有效适配任务特定特征,我们对 CLIP 的编码器施加参数高效微调,并设计了一种熵引导的视角采样策略以挑选置信度高的投影。此外,基于最优传输的对齐损失与不确定性感知的原型对齐损失协同弥合源域与目标域之间的分布差距,同时保持类别可分性。在 PointDA-10 与 GraspNetPC-10 基准上的大量实验表明,CLIPoint3D 相较基于 CLIP 的基线以及传统基于编码器的基线均取得 3-16% 的稳定精度提升。项目主页:https://sarthakm320.github.io/CLIPoint3D。
英文原文摘要 1400 字符
Recent vision-language models (VLMs) such as CLIP demonstrate impressive cross-modal reasoning, extending beyond images to 3D perception. Yet, these models remain fragile under domain shifts, especially when adapting from synthetic to real-world point clouds. Conventional 3D domain adaptation approaches rely on heavy trainable encoders, yielding strong accuracy but at the cost of efficiency. We introduce CLIPoint3D, the first framework for few-shot unsupervised 3D point cloud domain adaptation built upon CLIP. Our approach projects 3D samples into multiple depth maps and exploits the frozen CLIP backbone, refined through a knowledge-driven prompt tuning scheme that integrates high-level language priors with geometric cues from a lightweight 3D encoder. To adapt task-specific features effectively, we apply parameter-efficient fine-tuning to CLIP's encoders and design an entropy-guided view sampling strategy for selecting confident projections. Furthermore, an optimal transport-based alignment loss and an uncertainty-aware prototype alignment loss collaboratively bridge source-target distribution gaps while maintaining class separability. Extensive experiments on PointDA-10 and GraspNetPC-10 benchmarks show that CLIPoint3D achieves consistent 3-16% accuracy gains over both CLIP-based and conventional encoder-based baselines. Project page: https://sarthakm320.github.io/CLIPoint3D.