生成模型与扩散 · CVPR2026 ·2026.06.03

IntroSVG:通过内省式生成器-评论器框架从渲染反馈中学习文本到SVG生成

IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework

Feiyu Wang、Jiayuan Yang、Zhiyuan Zhao、Da Zhang、Bingyu Li、Peng Liu 等 7 人

与你有关的部分

将渲染后的视觉反馈引入自回归生成循环,通过生成器-评论器双角色VLM和DPO对齐,为文本到矢量图形生成提供了可复用的闭环优化范式。

原文依据 the autoregressive training process does not incorporate visual perception of the final rendered image
可迁移到 文本到SVG生成矢量图形生成视觉反馈优化

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

可缩放矢量图形(SVG)因其固有的可缩放性和可编辑性,在数字设计中处于核心地位。尽管视觉语言模型(VLM)在内容生成方面取得了显著进展,但现有的文本到SVG生成方法受限于一个核心挑战:自回归训练过程未纳入对最终渲染图像的视觉感知,这从根本上制约了生成质量。为解决这一局限,我们提出了内省式SVG生成框架(IntroSVG)。该框架的核心是实例化一个在闭环中运行、同时承担生成器和评论器双重角色的统一VLM。具体而言,通过监督微调(SFT),模型学习起草SVG并对其渲染输出提供反馈;此外,我们系统地将早期失败转化为高质量的纠错训练数据,从而增强模型鲁棒性。随后,我们利用高容量教师VLM构建偏好数据集,并通过直接偏好优化(DPO)进一步对齐生成器的策略。在推理阶段,优化后的生成器和评论器以迭代的“生成-审查-精炼”循环协同工作,从不完美的中间草稿出发,自主提升输出质量。实验结果表明,我们的方法在多个关键评估指标上达到了最优性能,生成的SVG具有更复杂的结构、更强的语义对齐和更高的可编辑性。这些结果证实了将显式视觉反馈纳入生成循环的有效性。

英文原文摘要 1664 字符

Scalable Vector Graphics (SVG) are central to digital design due to their inherent scalability and editability. Despite significant advancements in content generation enabled by Visual Language Models (VLMs), existing text-to-SVG generation methods are limited by a core challenge: the autoregressive training process does not incorporate visual perception of the final rendered image, which fundamentally constrains generation quality. To address this limitation, we propose an Introspective SVG Generation Framework (IntroSVG). At its core, the framework instantiates a unified VLM that operates in a closed loop, assuming dual roles of both generator and critic. Specifically, through Supervised Fine-Tuning (SFT), the model learns to draft SVGs and to provide feedback on their rendered outputs; moreover, we systematically convert early-stage failures into high-quality error-correction training data, thereby enhancing model robustness. Subsequently, we leverage a high-capacity teacher VLM to construct a preference dataset and further align the generator's policy through Direct Preference Optimization (DPO). During inference, the optimized generator and critic operate collaboratively in an iterative "generate-review-refine" cycle, starting from imperfect intermediate drafts to autonomously improve output quality. Experimental results demonstrate that our method achieves state-of-the-art performance across several key evaluation metrics, generating SVGs with more complex structures, stronger semantic alignment, and greater editability. These results corroborate the effectiveness of incorporating explicit visual feedback into the generation loop.

标签

会议 CVPR2026
页码 615-625
Bibkey
来源 CVF Open Access