MODIX：面向视觉语言模型的免训练多模态信息驱动位置索引缩放
MODIX: A Training-Free Multimodal Information-Driven Positional Index Scaling for Vision-Language Models
对做 VLM 推理与注意力分析的人，这篇给出了一个免训练、不改参数的位置索引重标定手段：用协方差熵衡量模态内信息密度、用跨模态对齐衡量模态间交互，据此缩放位置步长，可直接插到现有 VLM 的注意力计算前，用来缓解冗余视觉 token 抢占注意力的问题。
原文依据 we propose MODIX (Multimodal Information-Driven Positional IndeX Scaling), a training-free framework that dynamically adapts positional strides based on modality-specific contributions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）在多模态理解方面已取得显著进展，但其位置编码机制仍不够理想。现有方法对所有 token 统一分配位置索引，忽略了模态内部与模态之间信息密度的差异，导致注意力分配低效：冗余的视觉区域占据主导，而信息丰富的内容反而被低估。我们将位置粒度视为一种隐式资源，提出 MODIX（多模态信息驱动的位置索引缩放），这是一个免训练框架，依据各模态的贡献动态调整位置步长。MODIX 通过基于协方差的熵对模态内密度建模，并通过跨模态对齐对模态间交互建模，从而得到统一分数，再用这些分数重新缩放位置索引，为信息丰富的模态分配更细的粒度、压缩冗余模态，且无需对模型参数或架构做任何修改。在多种架构与基准上的实验表明，MODIX 能持续提升多模态推理能力，并根据任务相关的信息分布自适应地重新分配注意力，这说明在多模态序列建模中，位置编码应被视为 Transformer 的一种自适应资源。
英文原文摘要 1367 字符
Vision-Language Models (VLMs) have achieved remarkable progress in multimodal understanding, yet their positional encoding mechanisms remain suboptimal. Existing approaches uniformly assign positional indices to all tokens, overlooking variations in information density within and across modalities, which leads to inefficient attention allocation where redundant visual regions dominate while informative content is underrepresented. We identify positional granularity as an implicit resource and propose MODIX (Multimodal Information-Driven Positional IndeX Scaling), a training-free framework that dynamically adapts positional strides based on modality-specific contributions. MODIX jointly models intra-modal density via covariance-based entropy and inter-modal interaction via cross-modal alignment to derive unified scores, which rescale positional indices to allocate finer granularity to informative modalities while compressing redundant ones, without requiring any modification to model parameters or architecture. Experiments across diverse architectures and benchmarks demonstrate that MODIX consistently improves multimodal reasoning and adaptively reallocates attention according to task-dependent information distributions, suggesting that positional encoding should be treated as an adaptive resource in Transformers for multimodal sequence modeling.