破解证据稀疏:面向长文档理解的智能体上下文工程
Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding
对做多模态长文档问答的研究者,这篇提供了一个可复用的「检索后二次精炼」范式:用多个智能体在检索到的页面里筛出表格、图表等显著视觉证据,再蒸馏成证据密集的多模态上下文喂给 VLM,可直接迁移到多页文档 VQA 与图表理解流程中。
原文依据 filters for salient visual evidence such as tables and charts, and analyzes the query to devise a reasoning strategy
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文档理解是一项长期存在的实际任务。视觉语言模型(VLM)已逐渐成为该领域的主要方法,在单页任务上表现出有效性能。然而,在处理长文档时其效果会下降。在此类场景中,线索往往分散在多个页面和多种模态之间,冗长输入带来的冗余也会损害模型的判断。尽管检索增强生成通过筛选与问题相关的内容缓解了这一问题,但检索结果中仍包含大量冗余。为解决上述局限,我们提出 SLEUTH,一个多智能体框架。具体而言,SLEUTH 以由粗到细的流程编排一个检索器和四个协作智能体。该框架在检索到的页面中识别关键的文本与视觉线索,筛选表格、图表等显著视觉证据,并分析查询以制定推理策略。它最终合成一个经过蒸馏、证据密集的多模态上下文以生成最终预测。SLEUTH 与模型无关且可扩展。当与先进的 VLM 主干模型配合使用时,它在多个长文档基准上持续提升性能,取得 SOTA 结果。消融实验验证了各模块的有效性,并证实了我们分层精炼范式的优势。
英文原文摘要 1366 字符
Document understanding is a long-standing practical task. Vision-Language Models (VLMs) have gradually become a primary approach in this domain, demonstrating effective performance on single-page tasks. However, their effectiveness diminishes when handling long documents. In such scenarios, clues are often scattered across multiple pages and modalities, and redundancy from lengthy inputs can impair the model's judgment. While retrieval-augmented generation mitigates this issue by filtering for question-relevant content, the retrieved results still contain substantial redundancy. To address these limitations, we propose SLEUTH, a multi-agent framework. Concretely, SLEUTH orchestrates a retriever and four collaborative agents in a coarse-to-fine process. The framework identifies key textual and visual clues within the retrieved pages, filters for salient visual evidence such as tables and charts, and analyzes the query to devise a reasoning strategy. It ultimately synthesizes a distilled, evidence-dense multimodal context to generate the final prediction. SLEUTH is model-agnostic and scalable. When paired with advanced VLM backbones, it consistently improves performance on multiple long-document benchmarks, achieving SOTA results. Ablation studies verify each module's effectiveness and confirm the benefits of our hierarchical refinement paradigm.