SemLT3D:语义引导的专家蒸馏用于纯视觉长尾三维目标检测
SemLT3D: Semantic-Guided Expert Distillation for Camera-only Long-Tailed 3D Object Detection
针对纯视觉三维目标检测中的长尾分布问题,提出语言引导的混合专家路由与CLIP语义投影蒸馏,可复用于其他长尾三维感知任务以增强稀有类表征。
原文依据 a language-guided mixture-of-experts module that routes 3D queries to specialized experts according to their semantic affinity
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论纯视觉三维目标检测已成为自动驾驶中替代激光雷达的一种经济高效且可扩展的方案,但现有方法主要关注整体性能,而忽视了真实数据集中固有的严重长尾不平衡问题。实践中,许多稀有但安全关键的类别(如儿童、婴儿车或应急车辆)严重缺乏代表性,导致学习有偏且性能下降。显著的类间歧义(如视觉相似的子类)和较大的类内多样性(如外观、尺度、姿态或上下文差异很大的物体)进一步加剧了这一挑战,共同阻碍了可靠的长尾识别。本文提出SemLT3D,一个语义引导的专家蒸馏框架,旨在通过语义先验丰富稀有类别的表示空间。SemLT3D包含:(1) 语言引导的混合专家模块,根据三维查询的语义亲和度将其路由到专门的专家,使模型能更好地解耦易混淆类别并专注于尾部数据分布;(2) 语义投影蒸馏流程,将三维查询与CLIP引导的二维语义对齐,从而在不同视觉表现下生成更连贯且更具判别性的特征。尽管动机源于长尾不平衡,SemLT3D中的语义结构化学习也提升了在更广泛外观变化和挑战性角落情况下的鲁棒性,为更可靠的纯视觉三维感知迈出了原则性的一步。
英文原文摘要 1613 字符
Camera-only 3D object detection has emerged as a cost-effective and scalable alternative to LiDAR for autonomous driving, yet existing methods primarily prioritize overall performance while overlooking the severe long-tail imbalance inherent in real-world datasets. In practice, many rare but safety-critical categories such as children, strollers, or emergency vehicles are heavily underrepresented, leading to biased learning and degraded performance. This challenge is further exacerbated by pronounced inter-class ambiguity (e.g., visually similar subclasses) and substantial intra-class diversity (e.g., objects varying widely in appearance, scale, pose, or context), which together hinder reliable long-tail recognition. In this work, we introduce SemLT3D, a Semantic-Guided Expert Distillation framework designed to enrich the representation space for underrepresented classes through semantic priors. SemLT3D consists of: (1) a language-guided mixture-of-experts module that routes 3D queries to specialized experts according to their semantic affinity, enabling the model to better disentangle confusing classes and specialize on tail distributions; and (2) a semantic projection distillation pipeline that aligns 3D queries with CLIP-informed 2D semantics, producing more coherent and discriminative features across diverse visual manifestations. Although motivated by long-tail imbalance, the semantically structured learning in SemLT3D also improves robustness under broader appearance variations and challenging corner cases, offering a principled step toward more reliable camera-only 3D perception.