FOZO：面向测试时自适应的纯前向零阶提示词优化
FOZO: Forward-Only Zeroth-Order Prompt Optimization for Test-Time Adaptation
提供了一种无需反向传播、仅靠前向计算与零阶梯度估计即可在测试时优化提示词的方案，可直接用于低端设备或 INT8 量化模型上的在线域适应，避免修改模型权重。
原文依据 FOZO leverages a memory-efficient zeroth-order prompt optimization
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论测试时自适应（TTA）对于使深度学习模型能够应对真实世界的数据分布偏移至关重要。然而，现有方法存在显著局限：基于反向传播的方法由于计算与内存需求高，且在自适应过程中倾向于修改模型权重，不适合低端部署设备；而传统的无反向传播技术则表现出受限的自适应能力。在本工作中，我们提出纯前向零阶优化（FOZO），一种新颖且实用的无反向传播 TTA 范式。FOZO 利用内存高效的零阶提示词优化，其目标函数同时优化中间特征统计量与预测熵。为确保在分布外数据流上高效且稳定的自适应，我们在零阶梯度估计中引入动态衰减的扰动尺度，并在 TTA 数据流假设下从理论上证明了其收敛性。在 ImageNet-C、ImageNet-R 和 ImageNet-Sketch 上的大量持续自适应实验表明 FOZO 具有优越性能，在 ImageNet-C（5K，level 5）上达到 59.52% 的 Top-1 准确率，优于主要的基于梯度的方法以及当前最优的纯前向方法 FOA（58.13%）。此外，FOZO 在量化（INT8）模型上展现出强泛化能力。这些发现表明，FOZO 是资源受限场景下 TTA 部署极具竞争力的解决方案。代码：https://github.com/eVI-group-SCU/FOZO
英文原文摘要 1547 字符
Test-Time Adaptation (TTA) is essential for enabling deep learning models to handle real-world data distribution shifts. However, current approaches face significant limitations: backpropagation-based methods are not suitable for low-end deployment devices, due to their high computation and memory requirements, as well as their tendency to modify model weights during adaptation; while traditional backpropagation-free techniques exhibit constrained adaptation capabilities. In this work, we propose Forward-Only Zeroth-Order Optimization (FOZO), a novel and practical backpropagation-free paradigm for TTA. FOZO leverages a memory-efficient zeroth-order prompt optimization, which is led by objectives optimizing both intermediate feature statistics and prediction entropy. To ensure efficient and stable adaptation over the out-of-distribution data stream, we introduce a dynamically decaying perturbation scale during zeroth-order gradient estimation and theoretically prove its convergence under the TTA data stream assumption. Extensive continual adaptation experiments on ImageNet-C, ImageNet-R, and ImageNet-Sketch demonstrate FOZO's superior performance, achieving 59.52% Top-1 accuracy on ImageNet-C (5K, level 5) and outperforming main gradient-based methods and SOTA forward-only FOA (58.13%). Furthermore, FOZO exhibits strong generalization on quantized (INT8) models. These findings demonstrate that FOZO is a highly competitive solution for TTA deployment in resource-limited scenarios. Code: https://github.com/eVI-group-SCU/FOZO