E$^2$-SCI：基于信用惯性的弹性边云推测解码
E$^2$-SCI: Elastic Edge-Cloud Speculative Decoding via Credit Inertia
推测解码的接受率在相邻词元窗口间存在时序一致性（credit inertia），可据此动态调整草稿词元的验证阈值，并以异步流水线隐藏延迟；做视觉语言模型或扩散模型少步采样加速时，可把这一「按历史表现弹性放宽/收紧验证」的思路迁移到多步生成或自回归解码的验证环节。
原文依据 we identify a phenomenon we term credit inertia, where acceptance rates of adjacent token windows exhibit strong temporal consistency
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在边云环境中，推测解码的效率受到上行传输与云端验证的严重制约。本文识别出一种我们称为信用惯性的现象，即相邻词元窗口的接受率表现出很强的时序一致性：紧随近期表现良好窗口之后的词元更可能通过验证，而紧随表现不佳窗口之后的词元则更可能验证失败。受此观察启发，我们提出 E^2-SCI，一种弹性边云推测解码框架，它依据近期历史表现动态调整草稿词元的验证阈值。该自适应机制使系统对历史表现强的窗口更为宽松、对表现弱的窗口更为严格，从而有效利用时序一致性降低整体延迟。我们进一步引入渐进前瞻并发（PLC），以异步方式将草稿生成与验证流水化，从而隐藏延迟。在多个基准上的实验表明，E^2-SCI 在 DeepSeek-R1-Distill-Qwen（1.5B/32B）上达到超过 9.4 词元/秒，相比 FSD 基线取得 88.5% 的加速，同时保持精度。值得注意的是，E^2-SCI 可与现有框架无缝集成，展现出广泛的适用性与更优的效率—质量权衡。
英文原文摘要 1366 字符
In edge-cloud environments, efficiency of speculative decoding is heavily constrained by uplink transmission and cloud-side verification. In this work, we identify a phenomenon we term credit inertia, where acceptance rates of adjacent token windows exhibit strong temporal consistency. Tokens following recently well-performing windows are likely to pass verification, whereas tokens following poorly performing windows are likely to fail. Motivated by this observation, we propose E^2-SCI, an elastic edge-cloud speculative decoding framework that dynamically adjusts draft token verification thresholds based on recent historical performance. This adaptive mechanism allows system to be more permissive for windows with strong historical performance and stricter for windows with weak performance, effectively leveraging temporal consistency to reduce overall latency. We further introduce Progressive Lookahead Concurrency (PLC), which pipelines draft generation and verification asynchronously to hide latency. Experiments across multiple benchmarks show that E^2-SCI achieves over 9.4 tokens/s on DeepSeek-R1-Distill-Qwen (1.5B/32B), delivering an 88.5% speed improvement over FSD baseline while maintaining accuracy. Notably, E^2-SCI integrates seamlessly with existing frameworks, demonstrating broad applicability and superior efficiency-quality trade-offs.