Granulon：以自适应多粒度语义唤醒像素级视觉编码器，服务多模态大语言模型
Granulon: Awakening Pixel-Level Visual Encoders with Adaptive Multi-Granularity Semantics for MLLM
用文本条件动态控制视觉编码器的抽象粒度，并通过粒度引导的池化与关系聚类生成紧凑视觉词元，为多模态大模型中的细粒度视觉理解提供可复用方案。
原文依据 Granulon introduces a text-conditioned granularity Controller that dynamically adjusts the visual abstraction level according to the semantic scope of the textual input
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期多模态大语言模型的进展主要依赖基于 CLIP 的视觉编码器，这类编码器强调全局语义对齐，但在细粒度视觉理解上表现不佳。相比之下，DINOv3 提供了强大的像素级感知能力，却缺乏粗粒度语义抽象，导致多粒度推理能力有限。为弥补这一差距，我们提出 Granulon，一种基于 DINOv3 的新型多模态大语言模型，具备自适应粒度增强能力。Granulon 引入了一个文本条件粒度控制器，可根据文本输入的语义范围动态调整视觉抽象层级；同时设计了一个自适应词元聚合模块，执行粒度引导的池化与关系感知聚类，以生成紧凑且语义丰富的视觉词元。该设计使得单次前向传播内即可实现统一的“像素到细粒度到粗粒度”推理。大量且可解释的实验表明，Granulon 在相同设置下准确率提升 30%、幻觉减少 20%，优于所有视觉编码器。代码见补充材料。
英文原文摘要 1106 字符
Recent advances in multimodal large language models largely rely on CLIP-based visual encoders, which emphasize global semantic alignment but struggle with fine-grained visual understanding. In contrast, DINOv3 provides strong pixel-level perception yet lacks coarse-grained semantic abstraction, leading to limited multi-granularity reasoning. To address this gap, we propose Granulon, a novel DINOv3-based MLLM with adaptive granularity augmentation. Granulon introduces a text-conditioned granularity Controller that dynamically adjusts the visual abstraction level according to the semantic scope of the textual input, and an Adaptive Token Aggregation module that performs granularity-guided pooling and relation-aware clustering to produce compact, semantically rich visual tokens. This design enables unified "pixel-to-fine-to-coarse" reasoning within a single forward pass. Extensive and interpretable experiments demonstrate that Granulon improves accuracy by 30% and reduces hallucination by 20%, outperforming all visual encoders under identical settings. Code is available at the Supplementary.