通过锚定描述实现判别性感知以用于推理分割
Discriminative Perception via Anchored Description for Reasoning Segmentation
用描述性 caption 与上下文做语义对比来构造判别性奖励,可作为一种可复用的 RL 奖励设计思路,用于任何需要模型聚焦目标区域的 grounding/分割任务,同时产出与分割对齐的可解释理由。
原文依据 we propose DPAD to compel the model to generate a descriptive caption of the referred object
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论推理分割日益采用强化学习来生成解释性推理链,以引导多模态大语言模型。然而这些几何奖励主要局限于引导最终定位,无法判别推理过程是否仍锚定在被指称区域,还是偏离到无关上下文中。缺乏这种判别性引导,模型的推理往往退化为发散而冗长的推理链,最终无法在复杂场景中消歧并感知目标。这表明需要以判别性感知来补充强化学习目标,即一种主动将目标与其上下文区分开的能力。为实现这一点,我们提出 DPAD,迫使模型生成被指称对象的描述性 caption,随后通过对比该 caption 与被指称对象的语义相关性同更广泛上下文之间的差异来进行显式判别。通过优化这一判别能力,模型被迫关注目标的独特属性,从而得到更收敛、更高效的推理链。该描述性 caption 还充当与分割对齐的可解释性依据。基准实验证实了本方法的有效性,取得了显著性能提升,ReasonSeg 上的 cIoU 提升 3.09%,推理链长度减少约 42%。代码见 https://github.com/mrazhou/DPAD。
英文原文摘要 1529 字符
Reasoning segmentation increasingly employs reinforcement learning to generate explanatory reasoning chains that guide Multimodal Large Language Models. While these geometric rewards are primarily confined to guiding the final localization, they are incapable of discriminating whether the reasoning process remains anchored on the referred region or strays into irrelevant context. Lacking this discriminative guidance, the model's reasoning often devolves into unfocused and verbose chains that ultimately fail to disambiguate and perceive the target in complex scenes. This suggests a need to complement the RL objective with Discriminative Perception, an ability to actively distinguish a target from its context. To realize this, we propose DPAD to compel the model to generate a descriptive caption of the referred object, which is then used to explicitly discriminate by contrasting the caption's semantic relevance to the referred object against the wider context. By optimizing for this discriminative capability, the model is forced to focus on the unique attributes of the target, leading to a more converged and efficient reasoning chain. The descriptive caption also serves as an interpretability rationale that aligns with the segmentation. Experiments on the benchmarks confirm the validity of our approach, delivering substantial performance gains, with the cIoU on ReasonSeg increasing by 3.09% and the reasoning chain length decreasing by approximately 42%. Code is available at https://github.com/mrazhou/DPAD.