PAS：视频大语言模型时序编码的免训练稳定器
PAS: A Training-Free Stabilizer for Temporal Encoding in Video LLMs
对做视频 LLM 的人，这是一个即插即用的推理期补丁：不改结构、不重训，只在各注意力头上加相反相位偏移再聚合，就能让多模态 RoPE 的时序核变平滑、注意力对帧时序抖动更稳；可直接套到任意用 3D RoPE 的视频 LLM 上提升时序鲁棒性。
原文依据 PAS preserves the per-head spectrum magnitude, while the aggregation effectively smooths the temporal kernel and reduces phase sensitivity
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频大语言模型存在时序不一致性：帧时序的微小偏移就可能翻转注意力并压制相关帧。我们将这一不稳定性追溯到将旋转位置编码通过多模态 RoPE 扩展到视频的常见做法。由此诱导出的逆傅里叶时间核会呈现帧尺度的波纹，这些波纹以不同因子乘到相邻帧上，从而扰动了本应由原始查询—键内积主导的注意力。我们提出相位聚合平滑（PAS），一种简单、免训练的机制：在各注意力头上施加小的相反相位偏移，再聚合其输出。PAS 保持每个头的频谱幅度，而聚合则有效平滑了时间核并降低了相位敏感性，同时不改变位置编码结构。我们的分析表明，经 RoPE 旋转后的 logit 可近似为被时间核缩放的内容点积；平滑该核可带来注意力对小时序偏移的 Lipschitz 稳定性；在满足奈奎斯特采样条件的采样下，多相位平均可衰减高频波纹，同时保持各头频谱。在匹配词元预算下于多个视频理解基准上的实验显示出一致提升，且计算开销可忽略。PAS 为视频大语言模型中鲁棒的时序编码提供了一种即插即用的升级方案。
英文原文摘要 1371 字符
Video LLMs suffer from temporal inconsistency: small shifts in frame timing can flip attention and suppress relevant frames. We trace this instability to the common extension of Rotary Position Embeddings to video through multimodal RoPE. The induced inverse Fourier time kernel exhibits frame-scale ripples that multiply adjacent frames by different factors, which perturbs attention that should otherwise be governed by the raw query key inner product. We present Phase Aggregated Smoothing (PAS), a simple, training-free mechanism that applies small opposed phase offsets across heads and then aggregates their outputs. PAS preserves the per-head spectrum magnitude, while the aggregation effectively smooths the temporal kernel and reduces phase sensitivity without changing the positional encoding structure. Our analysis shows that the RoPE rotated logit can be approximated as a content dot product scaled by a time kernel; smoothing this kernel yields Lipschitz stability of attention to small temporal shifts; multi phase averaging attenuates high frequency ripples while preserving per-head spectra under Nyquist-valid sampling. Experiments on multiple video understanding benchmarks under matched token budgets show consistent improvements with negligible computational overhead. PAS provides a plug and play upgrade for robust temporal encoding in Video LLMs.