GeoTikzBridge：推进面向几何感知与推理的多模态代码生成
GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoning
通过生成 TikZ 代码作为中间表示来增强 MLLM 的细粒度几何感知，这种「代码即表示」的思路可迁移到需要精确几何输出的视觉任务（如场景图生成、CAD 重建），也可作为即插即用的推理模块提升几何问题求解。
原文依据 enhances local geometric perception and visual reasoning through tikz-based code generation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLMs）近期展现出卓越的感知与推理能力。然而，它们难以感知细粒度的几何结构，这限制了其几何理解与视觉推理能力。为此，我们提出 GeoTikzBridge，一个通过基于 tikz 的代码生成来增强局部几何感知与视觉推理的框架。在该框架内，我们构建了两个模型，并由两个互补的数据集提供支持。GeoTikzBridge-Base 模型在 GeoTikz-Base 数据集上训练，该数据集是迄今为止最大的图像到 tikz 数据集，包含 250 万对数据（比现有开源数据集大 16 倍）。这一过程通过迭代数据扩展和局部几何变换策略实现。随后，GeoTikzBridge-Instruct 在 GeoTikz-Instruct 数据集上微调，这是首个支持视觉推理的指令增强 tikz 数据集。大量实验结果表明，我们的模型在开源 MLLMs 中达到了最优性能。此外，GeoTikzBridge 模型可作为任何 MLLM（LLM）的即插即用推理模块，提升几何问题求解中的推理性能。
英文原文摘要 1219 字符
Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable perceptual and reasoning abilities. However, they struggle to perceive fine-grained geometric structures, constraining their ability of geometric understanding and visual reasoning. To address this, we propose GeoTikzBridge, a framework that enhances local geometric perception and visual reasoning through tikz-based code generation. Within this framework, we build two models supported by two complementary datasets. The GeoTikzBridge-Base model is trained on GeoTikz-Base dataset, the largest image-to-tikz dataset to date with 2.5M pairs (16 xlarger than existing open-sourced datasets). This process is achieved via iterative data expansion and a localized geometric transformation strategy. Subsequently, GeoTikzBridge-Instruct is fine-tuned on GeoTikz-Instruct dataset which is the first instruction-augmented tikz dataset supporting visual reasoning. Extensive experimental results demonstrate that our models achieve state-of-the-art performance among open-sourced MLLMs. Furthermore, GeoTikzBridge models can serve as plug-and-play reasoning modules for any MLLM(LLM), enhancing reasoning performance in geometric problem-solving.