LLM 引导的概率融合用于标签高效的文档版面分析
LLM-Guided Probabilistic Fusion for Label-Efficient Document Layout Analysis
把 OCR-LLM 推断出的层级版面结构当作结构化先验，与教师检测器的视觉预测做逆方差概率融合来生成伪标签，并用实例自适应门控替代固定权重；做半监督检测或需要引入外部语言模型先验来提升伪标签质量时可直接复用这套融合与门控机制。
原文依据 an OCR-LLM pipeline infers hierarchical regions which are combined with teacher detector outputs through inverse-variance fusion to generate refined pseudo-labels
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管半监督学习已取得进展，文档版面理解仍然是数据密集型的任务。本文提出一个框架，通过基于原理的概率加权，将视觉预测与来自文本预训练 LLM 的结构先验相融合，从而增强半监督检测。给定无标注文档，OCR-LLM 流水线推断出层级区域，这些区域通过逆方差融合与教师检测器的输出相结合，以生成精炼后的伪标签。本方法在不同模型规模上均表现出一致的增益。在使用轻量级 SwiftFormer 骨干网络（26M 参数）时，仅使用 PubLayNet 上 5% 的标签即达到 88.2±0.3 AP。当应用于文档预训练的 LayoutLMv3（133M 参数）时，我们的融合框架达到 89.7±0.4 AP，超过采用标准半监督学习的 LayoutLMv3（89.1±0.4 AP，p=0.02），并与需要 1 亿页以上多模态预训练的 UDOP [??]（89.8 AP）持平。这表明 LLM 结构先验与轻量级架构和预训练架构均具有互补性。主要发现包括：(1) 学习到的实例自适应门控相比固定权重提升 +0.9 AP，且数据依赖的 PAC 界能够正确预测收敛；(2) 开源 LLM 可实现隐私保护部署且损失极小（Llama-3-70B：轻量级下 87.1 AP，配合 LayoutLMv3 时 89.4 AP）；(3) LLM 提供了超出简单文本启发式的定向语义消歧（占 18.7% 的情况，增益 +3.8 AP）。系统总成本包括每 5 万页 12 美元的 GPT-4o-mini API 费用，或本地 Llama-3-70B 的 17 GPU 小时，并在多次训练运行中摊销。
英文原文摘要 1588 字符
Document layout understanding remains data-intensive despite advances in semi-supervised learning. We present a framework that enhances semi-supervised detection by fusing visual predictions with structural priors from text-pretrained LLMs via principled probabilistic weighting. Given unlabeled documents, an OCR-LLM pipeline infers hierarchical regions which are combined with teacher detector outputs through inverse-variance fusion to generate refined pseudo-labels.Our method demonstrates consistent gains across model scales. With a lightweight SwiftFormer backbone (26M params), we achieve 88.2\pm0.3 AP using only 5% labels on PubLayNet. When applied to document-pretrained LayoutLMv3 (133M params), our fusion framework reaches 89.7\pm0.4 AP, surpassing both LayoutLMv3 with standard semi-supervised learning (89.1\pm0.4 AP, p=0.02) and matching UDOP [??] (89.8 AP) which requires 100M+ pages of multimodal pretraining. This demonstrates that LLM structural priors are complementary to both lightweight and pretrained architectures. Key findings include: (1) learned instance-adaptive gating improves over fixed weights by +0.9 AP with data-dependent PAC bounds correctly predicting convergence; (2) open-source LLMs enable privacy-preserving deployment with minimal loss (Llama-3-70B: 87.1 AP lightweight, 89.4 AP with LayoutLMv3); (3) LLMs provide targeted semantic disambiguation (18.7% of cases, +3.8 AP gain) beyond simple text heuristics.Total system cost includes \12 for GPT-4o-mini API or 17 GPU-hours for local Llama-3-70B per 50K pages, amortized across training runs.