3D-VCD：通过视觉对比解码缓解3D-LLM具身智能体的幻觉
3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding
把2D视觉对比解码的思路迁移到3D具身场景：在推理时对3D场景图施加类别标签交换、坐标加噪等扰动来构造"负"上下文，从而无需训练即可抑制3D-LLM的幻觉，可直接用于3D视觉语言模型的可靠性提升。
原文依据 applying novel distortions directly to a 3D scene graph, such as swapping object category labels or noising positional coordinates
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型语言模型正越来越多地被用作3D具身智能体的认知核心，以实现复杂的环境推理。然而，这些智能体往往会继承幻觉这一关键缺陷，常常无法将其回答建立在自身的3D视图之上。视觉对比解码（VCD）是一种强大的免训练方法，可用于缓解基于2D图像的模型中的幻觉，但尚未被适配到复杂的3D具身环境中。本文以成为首个弥合这一空白的工作为雄心目标，提出了面向3D具身智能体的VCD框架。我们的方法在推理阶段运行，通过生成一个"负"3D上下文来工作——它不是对图像进行模糊处理，而是直接对3D场景图施加新颖的扰动，例如交换物体类别标签或对位置坐标加噪。我们在标准评测基准上评估了该方法，发现它持续优于现有模型。例如，在3D-POPE的随机类别上，我们的3D-VCD方法将Yes率从99.9%降至75.1%，同时将精确率从50.0%提升至62.2%。这些结果表明，我们的免训练方法有效抑制了幻觉，使3D智能体显著更加可靠且更具依据性。
英文原文摘要 1297 字符
Large Language Models are increasingly integrated as the cognitive core of 3D embodied agents to enable complex environmental reasoning. However, these agents tend to inherit the critical flaw of hallucination, often failing to ground their responses to their 3D view. While Visual Contrastive Decoding (VCD) is a powerful training-free method for mitigating hallucinations in 2D image-based models, it has not been adapted to the complex 3D embodied environment. In this paper, we embarked on the ambitious goal of being the very first to bridge this gap by introducing a VCD framework for 3D embodied agents. Our method operates at inference time by generating a "negative" 3D context, not by blurring an image, but by applying novel distortions directly to a 3D scene graph, such as swapping object category labels or noising positional coordinates. We evaluate our approach on standard evaluation benchmarks and find that it consistently outperforms existing models. For example, in the random category of 3D-POPE, our 3D-VCD method reduced Yes-rate from 99.9% to 75.1% while simultaneously increasing precision from 50.0% to 62.2%. These results demonstrate that our training-free approach effectively curbs hallucination, yielding 3D agents that are significantly more reliable and grounded.