PatchAlign3D：面向密集三维形状理解的局部特征对齐
PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understanding
提出仅编码器的点云 Transformer，通过两阶段训练（DINOv2 密集特征蒸馏 + 多正对比对齐部件文本嵌入）实现零样本 3D 部件分割，单次前向推理无需多视角渲染，为 3D 局部理解提供高效可复用的表征学习方案。
原文依据 Our 3D encoder achieves zero-shot 3D part segmentation with fast single-pass inference without any test-time multi-view rendering
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前的三维形状基础模型在全局任务（检索、分类）上表现出色，但在局部部件级推理上迁移能力较差。近期方法利用视觉与语言基础模型，通过多视角渲染和文本查询直接求解密集任务。这些流程虽有前景，但需要对多个渲染视角进行昂贵的推理，严重依赖大语言模型（LLM）的提示词工程来生成描述，且未能利用形状固有的三维几何信息。为弥补这一差距，我们引入一种仅编码器的三维模型，可直接从点云生成语言对齐的图像块级特征。我们的预训练方法建立在现有数据引擎之上，该引擎通过将多视角 SAM 区域与 VLM 描述配对来生成部件标注的三维形状。利用这些数据，我们分两阶段训练点云 Transformer 编码器：（1）将视觉编码器（如 DINOv2）的密集二维特征蒸馏到三维图像块中；（2）通过多正对比目标将这些图像块嵌入与部件级文本嵌入对齐。我们的三维编码器实现了零样本三维部件分割，推理快速且为单次前向，无需任何测试时的多视角渲染，同时在多个三维部件分割基准上显著优于先前基于渲染和前馈的方法。
英文原文摘要 1352 字符
Current foundation models for 3D shapes excel at global tasks (retrieval, classification) but transfer poorly to local part-level reasoning. Recent approaches leverage vision and language foundation models to directly solve dense tasks through multi-view renderings and text queries. While promising, these pipelines require expensive inference over multiple renderings, depend heavily on large language-model (LLM) prompt engineering for captions, and fail to exploit the inherent 3D geometry of shapes. We address this gap by introducing an encoder-only 3D model that produces language-aligned patch-level features directly from point clouds. Our pre-training approach builds on existing data engines that generate part-annotated 3D shapes by pairing multi-view SAM regions with VLM captioning. Using this data, we train a point cloud transformer encoder in two stages: (1) distillation of dense 2D features from visual encoders such as DINOv2 into 3D patches, and (2) alignment of these patch embeddings with part-level text embeddings through a multi-positive contrastive objective. Our 3D encoder achieves zero-shot 3D part segmentation with fast single-pass inference without any test-time multi-view rendering, while significantly outperforming previous rendering-based and feed-forward approaches across several 3D part segmentation benchmarks.