数据集与评测 · CVPR2026 ·2026.06.03

ChartNet:面向鲁棒图表理解的百万级高质量多模态数据集

ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding

Jovana Kondic、Pengyuan Li、Dhiraj Joshi、Isaac Sanchez、Ben Wiesel、Shafiq Abedin 等 24 人

与你有关的部分

对做图表理解或多模态文档理解的视觉研究者,ChartNet 提供了百万级、五元对齐(绘图代码/图像/数据表/摘要/带推理问答)的监督信号,可直接用于微调 VLM 并检验其在几何图形与结构化数值联合推理上的泛化能力。

原文依据 ChartNet leverages a novel code-guided synthesis pipeline to generate 1.5 million diverse chart samples spanning 24 chart types and 6 plotting libraries.
可迁移到 图表理解视觉问答多模态预训练文档图像理解

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

理解图表要求模型对几何视觉模式、结构化数值数据和自然语言进行联合推理——当前视觉语言模型(VLM)在这一能力上仍然有限。我们提出 ChartNet,一个高质量、百万级的多模态数据集,旨在推进图表的解读与推理。ChartNet 采用一种新颖的代码引导合成流程,生成 150 万个多样化的图表样本,覆盖 24 种图表类型和 6 个绘图库。每个样本由五个对齐的组成部分构成:绘图代码、渲染后的图表图像、数据表、自然语言摘要,以及带推理的问答,从而提供细粒度的跨模态对齐。为覆盖图表理解的全部范围,ChartNet 还包含专门子集,涵盖人工标注数据、真实世界数据、安全性和定位。此外,一套严格的质量过滤流程确保了视觉保真度、语义准确性以及图表表示之间的多样性。在 ChartNet 上微调可在多个基准上稳定提升结果,表明其作为多模态模型大规模监督信号的有效性。作为同类中最大的开源数据集,ChartNet 旨在支持具备鲁棒且可泛化的数据可视化理解能力的基础模型的研发。该数据集已公开于 https://huggingface.co/datasets/ibm-granite/ChartNet

英文原文摘要 1446 字符

Understanding charts requires models to jointly reason over geometric visual patterns, structured numerical data, and natural language -- a capability where current vision-language models (VLMs) remain limited. We introduce ChartNet, a high-quality, million-scale multimodal dataset designed to advance chart interpretation and reasoning. ChartNet leverages a novel code-guided synthesis pipeline to generate 1.5 million diverse chart samples spanning 24 chart types and 6 plotting libraries. Each sample consists of five aligned components: plotting code, rendered chart image, data table, natural language summary, and question-answering with reasoning, providing fine-grained cross-modal alignment. To capture the full spectrum of chart comprehension, ChartNet additionally includes specialized subsets encompassing human annotated data, real-world data, safety, and grounding. Moreover, a rigorous quality-filtering pipeline ensures visual fidelity, semantic accuracy, and diversity across chart representations. Fine-tuning on ChartNet consistently improves results across benchmarks, demonstrating its utility as large-scale supervision for multimodal models. As the largest open-source dataset of its kind, ChartNet aims to support the development of foundation models with robust and generalizable capabilities for data visualization understanding. The dataset is publicly available at https://huggingface.co/datasets/ibm-granite/ChartNet

标签

会议 CVPR2026
页码 15922-15932
Bibkey
来源 CVF Open Access