VL-RouterBench：视觉语言模型路由基准
VL-RouterBench: A Benchmark for Vision-Language Model Routing
为视觉语言模型的路由/级联部署提供了可复现的样本-模型质量与成本矩阵及评测协议，做多模型推理系统或按预算选模型时可直接复用其数据构造与排序打分方式。
原文依据 We present VL-RouterBench to assess the overall capability of VLM routing systems systematically.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模型路由已从一项工程技术演变为关键基础设施，然而现有工作缺乏用于评估视觉语言模型（VLM）的系统化、可复现的基准。我们提出 VL-RouterBench，以系统化地评估 VLM 路由系统的整体能力。该基准以 VLM 的原始推理与打分日志为基础，在样本-模型对上构建质量矩阵与成本矩阵。在规模上，VL-RouterBench 覆盖 3 个任务组共 14 个数据集、合计 30,540 个样本，包含 15 个开源模型与 2 个 API 模型，产生 519,180 个样本-模型对，输入输出词元总量达 34,494,977。评测协议联合度量平均准确率、平均成本与吞吐量，并以归一化成本与准确率的调和平均数构建排序分数，从而支持在不同路由配置与成本预算之间进行比较。在该基准上，我们评估了 10 种路由方法与基线，观察到显著的可路由增益，而当前最优路由器与理想 Oracle 之间仍存在明显差距，表明通过更细粒度的视觉线索与文本结构建模来改进路由器架构仍有相当大的空间。我们开源了完整的数据构造与评测工具链，以促进多模态路由研究中的可比性、可复现性与实际部署。
英文原文摘要 1388 字符
Multi-model routing has evolved from an engineering technique into essential infrastructure, yet existing work lacks a systematic, reproducible benchmark for evaluating vision-language models (VLMs). We present VL-RouterBench to assess the overall capability of VLM routing systems systematically. The benchmark is grounded in raw inference and scoring logs from VLMs and constructs quality and cost matrices over sample-model pairs. In scale, VL-RouterBench covers 14 datasets across 3 task groups, totaling 30,540 samples, and includes 15 open-source models and 2 API models, yielding 519,180 sample-model pairs and a total input-output token volume of 34,494,977. The evaluation protocol jointly measures average accuracy, average cost, and throughput, and builds a ranking score from the harmonic mean of normalized cost and accuracy to enable comparison across router configurations and cost budgets. On this benchmark, we evaluate 10 routing methods and baselines and observe a significant routability gain, while the best current routers still show a clear gap to the ideal Oracle, indicating considerable room for improvement in router architecture through finer visual cues and modeling of textual structure. We open-source the complete data construction and evaluation toolchain to promote comparability, reproducibility, and practical deployment in multimodal routing research.