ViT$^3$：释放视觉中的测试时训练
ViT$^3$: Unlocking Test-Time Training in Vision
把注意力重写为测试时的在线学习问题，用键值对在线构建紧凑内模型，为视觉主干提供线性复杂度且可并行化的替代方案；做视觉骨干替换或长序列/高分辨率视觉建模时，可直接把 ViT^3 当作 Mamba、线性注意力的对照基线。
原文依据 TTT reformulates attention operation as an online learning problem, constructing a compact inner model from key-value pairs at test time.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论测试时训练（TTT）近期成为高效序列建模的一个有前景的方向。TTT 将注意力操作重新表述为一个在线学习问题，在测试时由键值对构建紧凑的内模型。这一重新表述开辟了丰富而灵活的设计空间，同时实现了线性计算复杂度。然而，设计出强大的视觉 TTT 仍具挑战性：内模块与内训练的基本选择缺乏全面的理解与实用指南。为填补这一关键空白，本文对面向视觉序列建模的 TTT 设计开展了系统的实证研究。通过一系列实验与分析，我们提炼出六条实用洞见，为有效的视觉 TTT 确立设计原则，并为未来改进指明路径。这些发现最终凝结为视觉测试时训练（ViT^3）模型，一种纯 TTT 架构，具备线性复杂度与可并行计算。我们在多种视觉任务上评估 ViT^3，包括图像分类、图像生成、目标检测与语义分割。结果表明，ViT^3 稳定地匹配或超越先进的线性复杂度模型（如 Mamba 与线性注意力变体），并有效缩小了与高度优化的视觉 Transformer 之间的差距。我们希望本项研究与 ViT^3 基线能够促进视觉 TTT 模型的未来工作。代码：github.com/LeapLabTHU/ViTTT。
英文原文摘要 1484 字符
Test-Time Training (TTT) has recently emerged as a promising direction for efficient sequence modeling. TTT reformulates attention operation as an online learning problem, constructing a compact inner model from key-value pairs at test time. This reformulation opens a rich and flexible design space while achieving linear computational complexity. However, crafting a powerful visual TTT design remains challenging: fundamental choices for the inner module and inner training lack comprehensive understanding and practical guidelines. To bridge this critical gap, in this paper, we present a systematic empirical study of TTT designs for visual sequence modeling. From a series of experiments and analyses, we distill six practical insights that establish design principles for effective visual TTT and illuminate paths for future improvement. These findings culminate in the Vision Test-Time Training (ViT^3) model, a pure TTT architecture that achieves linear complexity and parallelizable computation. We evaluate ViT^3 across diverse visual tasks, including image classification, image generation, object detection, and semantic segmentation. Results show that ViT^3 consistently matches or outperforms advanced linear-complexity models (e.g., Mamba and linear attention variants) and effectively narrows the gap to highly optimized vision Transformers. We hope this study and the ViT^3 baseline can facilitate future work on visual TTT models. Code: github.com/LeapLabTHU/ViTTT.