RxnCaption：将反应图解析重构为视觉提示引导的图像描述
RxnCaption: Reformulating Reaction Diagram Parsing as Visual Prompt Guided Captioning
将坐标预测式解析重构为图像描述任务，并利用视觉提示（边界框与索引）引导 LVLM，为文档图像的结构化信息提取提供了可复用的范式。
原文依据 reformulates the traditional coordinate prediction driven parsing process into an image captioning problem
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大规模化学反应数据集对于化学领域的人工智能研究至关重要。然而，现有的化学反应数据通常以论文中的图像形式存在，导致其无法被机器读取，也无法用于训练机器学习模型。为应对这一挑战，我们提出了 RxnCaption 框架，用于化学反应图解析（RxnDP）任务。该框架将传统的坐标预测驱动的解析过程重构为图像描述问题，而大型视觉语言模型（LVLM）能够自然地处理此类问题。我们引入了一种称为“边界框与索引作为视觉提示”（BIVP）的策略，该策略利用我们最优的分子检测器 MolYOLO，直接在输入图像上预先绘制分子边界框和索引。这将下游解析转变为自然语言描述问题。大量实验表明，BIVP 策略在简化模型设计的同时，显著提升了结构提取质量。我们进一步构建了 U-RxnDiagram-15k 数据集，其规模比先前真实世界文献基准大一个数量级，并包含一个在四种布局原型上均衡的测试子集。实验表明，RxnCaption-VL 在多个指标上达到了最优性能。我们相信，我们的方法、数据集和模型将推动化学文献中的结构化信息提取，并促进人工智能在化学领域更广泛的应用。
英文原文摘要 1431 字符
Large-scale chemical reaction datasets are crucial for AI research in chemistry. However, existing chemical reaction data often exist as images within papers, making them not machine-readable and unusable for training machine learning models. In response to this challenge, we propose the RxnCaption framework for the task of chemical Reaction Diagram Parsing (RxnDP). Our framework reformulates the traditional coordinate prediction driven parsing process into an image captioning problem, which Large Vision Language Models (LVLMs) handle naturally. We introduce a strategy termed "BBox and Index as Visual Prompt" (BIVP), which uses our state-of-the-art molecular detector, MolYOLO, to pre-draw molecular bounding boxes and indices directly onto the input image. This turns the downstream parsing into a natural-language description problem. Extensive experiments show that the BIVP strategy significantly improves structural extraction quality while simplifying model design. We further construct the U-RxnDiagram-15k dataset, an order of magnitude larger than prior real-world literature benchmarks, with a balanced test subset across four layout archetypes. Experiments demonstrate that RxnCaption-VL achieves state-of-the-art performance on multiple metrics. We believe our method, dataset, and models will advance structured information extraction from chemical literature and catalyze broader AI applications in chemistry.