共识熵:利用多 VLM 一致性实现自验证与自改进的 OCR
Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCR
提出一种无需训练、模型无关的「共识熵」指标,用多个 VLM 在输出空间的一致性来估计单样本可靠性;做 VLM 输出筛选、伪标签清洗或集成解码时可直接复用该打分与自适应路由策略。
原文依据 a training-free, model-agnostic metric that estimates output reliability by measuring inter-model agreement entropy
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论光学字符识别(OCR)是视觉语言模型(VLM)的基础能力,也是为 LLM 训练生成高质量数据的关键环节。然而,尽管平均 OCR 准确率有所提升,最优的 VLM 在检测样本级错误方面仍存在困难,且缺乏有效的无监督质量控制手段。我们提出共识熵(Consensus Entropy, CE),一种无需训练、与模型无关的指标,通过度量模型间一致性熵来估计输出的可靠性。其核心洞见在于:正确预测在输出空间中趋于收敛,而错误预测则趋于发散。基于 CE,我们构建了 CE-OCR,一个轻量的多模型框架,通过集成一致性来验证输出、挑选最佳输出,并借助自适应路由进一步提升效率。实验表明,CE 在质量验证上表现稳健,相较 VLM-as-Judge 将 F1 分数提升 42.1%。CE-OCR 在同等成本下取得一致的 OCR 增益,优于自一致性与单模型基线。值得注意的是,CE 无需训练或监督,可实现即插即用集成。
英文原文摘要 1060 字符
Optical Character Recognition (OCR) is fundamental to Vision-Language Models (VLMs) and high-quality data generation for LLM training. Yet, despite progress in average OCR accuracy, state-of-the-art VLMs still struggle with detecting sample-level errors and lack effective unsupervised quality control.We introduce Consensus Entropy (CE), a training-free, model-agnostic metric that estimates output reliability by measuring inter-model agreement entropy. The core insight is that correct predictions converge in output space, while errors diverge. Based on CE, we develop CE-OCR, a lightweight multi-model framework that verifies outputs by ensemble agreement, selects the best outputs, and further improves efficiency through adaptive routing. Experiments demonstrate that CE is robust for quality verification, improving F1 scores by 42.1% over VLM-as-Judge. CE-OCR achieves consistent OCR gains, outperforming self-consistency and single-model baselines at the same cost. Notably, CE requires no training or supervision, enabling plug-and-play integration.