油量表：在大规模多模态模型中提前估计思维链长度
Fuel Gauge: Estimating Chain-of-Thought Length Ahead of Time in Large Multimodal Models
对做多模态大模型推理部署的人，这篇给出了一个在生成前预测思维链长度的隐变量信号，可直接用于预分配 KV cache 以缓解显存碎片，也可作为自适应推理长度控制（避免过思考/欠思考）的调度依据。
原文依据 we propose Fuel Gauge, the first method which extracts this hidden signal and predicts CoT length ahead of time
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论推理型大规模多模态模型（LMM）已成为许多应用的事实选择。然而，这些模型依赖冗长且在运行时难以预测的思维链（CoT）过程，常常导致计算资源利用效率低下（源于内存碎片）以及精度次优（源于欠思考与过思考）。经验证据表明，思维链长度与任务难度之间存在强相关性。这表明，可以基于一个表示可用于支撑推理过程的“燃料”量的隐参数，提前估计思维链长度。基于这一洞见，我们提出 Fuel Gauge，这是首个提取该隐信号并提前预测思维链长度的方法。我们在两个下游任务上展示了 Fuel Gauge 的实用性：预测式 KV cache 分配，用于解决 LMM 服务系统中的内存碎片问题；以及思维链长度调制，用于缓解欠思考与过思考。在纯文本、图文与视频文本问答基准上针对 LMM 的大量实验，证明了 Fuel Gauge 的有效性、泛化性与实用价值。例如，在 GPQA-Diamond 基准上，我们的 Fuel Gauge 相比基线将思维链长度预测误差降低到不足一半；这转化为内存分配频率降低 13.37 倍。
英文原文摘要 1399 字符
Reasoning Large Multi-modality Models (LMMs) have become the de facto choice for many applications. However, these models rely on a Chain-of-Thought (CoT) process that is lengthy and unpredictable at runtime, often resulting in inefficient use of computational resources (due to memory fragmentation) and sub-optimal accuracy (due to under- and over-thinking). Empirical evidence shows a strong correlation between CoT length and task difficulty. This suggests that the CoT length can be estimated ahead of time based on a hidden parameter representing the amount of "fuel" available to support the reasoning process. Based on this insight, we propose Fuel Gauge, the first method which extracts this hidden signal and predicts CoT length ahead of time. We demonstrate the utility on the Fuel Gauge on two downstream tasks: predictive KV cache allocation, which addresses memory fragmentation in LMM serving systems, and CoT length modulation, which mitigates under-thinking and over-thinking. Extensive experiments on LMMs across text-only, image-text, and video-text question answering benchmarks demonstrate the effectiveness, generalizability, and practical value of Fuel Gauge. For example, on the GPQA-Diamond benchmark, our Fuel Gauge achieves less than half the CoT length prediction error compared to the baseline; this translates into a 13.37x reduction in the memory allocation frequency.