VLA 模型比你想象的更具泛化性:重新审视物理建模与空间建模
VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modeling
对做 VLA/机器人策略的视觉研究者:只需在 ViT 视觉词元上做全局仿射调制(4K 参数)或低秩更新,就能把新视角下的成功率从 48.5% 拉到 87–91%,说明预训练 VLA 的视觉表征本身已具备未被利用的鲁棒性,瓶颈在空间建模对齐而非物理建模;这套轻量视觉重校准可直接复用到自己的 VLA 微调流程中。
原文依据 improves Libero viewpoint accuracy from 48.5% to 87.1% with only 4K parameters
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉-语言-动作(VLA)模型在分布内表现优异,但在新的相机视角和视觉扰动下会显著退化。我们表明,这种脆弱性主要源于空间建模中的错位,而非物理建模。为此,我们提出一种单样本自适应框架,通过轻量、可学习的更新来重新校准视觉表征。我们的第一种方法——特征词元调制(FTM)——对视觉词元施加全局仿射变换,仅用 4K 参数便将 Libero 视角准确率从 48.5% 提升至 87.1%。在此基础上,特征线性适配(FLA)向 ViT 编码器引入低秩更新,以 4.7M 参数达到 90.8% 的成功率——以远低的成本匹配 LoRA 规模的微调。这些结果共同揭示了预训练 VLA 模型中大量尚未被利用的鲁棒性,并表明有针对性的、极小的视觉适配就足以恢复视角泛化能力。
英文原文摘要 992 字符
Vision-language-action (VLA) models achieve strong in-distribution performance but degrade sharply under novel camera viewpoints and visual perturbations. We show that this brittleness primarily arises from misalignment in Spatial Modeling, rather than Physical Modeling. To address this, we propose a one-shot adaptation framework that recalibrates visual representations through lightweight, learnable updates. Our first method, Feature Token Modulation (FTM), applies a global affine transformation to visual tokens and improves Libero viewpoint accuracy from 48.5% to 87.1% with only 4K parameters. Building on this, Feature Linear Adaptation (FLA) introduces low-rank updates to the ViT encoder, achieving 90.8% success with 4.7M parameters--matching LoRA-scale finetuning at far lower cost. Together, these results reveal substantial untapped robustness in pretrained VLA models and demonstrate that targeted, minimal visual adaptation is sufficient to restore viewpoint generalization.