dMLLM-TTS:面向扩散多模态大语言模型的自验证高效测试时扩展
dMLLM-TTS: Self-Verified and Efficient Test-Time Scaling for Diffusion Multi-Modal Large Language Models
把 dMLLM 自身的图像理解能力当作验证器,配合 O(N+T) 的分层搜索在采样轨迹与迭代精修两个维度上做测试时扩展,可直接迁移到任意扩散式多模态大模型的推理阶段以提升文生图质量并降低搜索开销。
原文依据 a self-verified feedback mechanism that leverages the dMLLMs' intrinsic image understanding capabilities to assess text-image alignment
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散多模态大语言模型(dMLLM)近期作为一种统一图像生成与理解的新型架构出现。然而,如何为其设计有效且高效的测试时扩展(TTS)方法以充分释放其生成潜力,仍是一个尚未充分探索的挑战。为此,我们提出 dMLLM-TTS,一个在两个互补扩展轴上运行的新框架:(1)轨迹探索扩展,用于增强所生成假设的多样性;(2)迭代精修扩展,用于稳定生成。常规 TTS 方法通常在这两个维度上执行线性搜索,带来 O(NT) 的显著计算开销,并且需要外部验证器来完成 best-of-N 选择。为克服这些局限,我们提出两项创新。第一,我们设计了一种复杂度为 O(N+T) 的高效分层搜索算法,能够自适应地扩展与剪枝采样轨迹。第二,我们引入自验证反馈机制,利用 dMLLM 内在的图像理解能力评估文本-图像对齐,从而无需外部验证器。在 GenEval 基准上针对三个代表性 dMLLM(如 Lumina-DiMOO、MMaDA、Muddit)的大量实验表明,我们的框架在显著提升生成质量的同时,相比线性搜索实现了最高 6 倍的效率提升。
英文原文摘要 1394 字符
Diffusion Multi-modal Large Language Models (dMLLMs) have recently emerged as a novel architecture unifying image generation and understanding. However, developing effective and efficient Test-Time Scaling (TTS) methods to unlock their full generative potential remains an underexplored challenge. To address this, we propose dMLLM-TTS, a novel framework operating on two complementary scaling axes: (1) trajectory exploration scaling to enhance the diversity of generated hypotheses, and (2) iterative refinement scaling for stable generation. Conventional TTS approaches typically perform linear search across these two dimensions, incurring substantial computational costs of O(NT) and requiring an external verifier for best-of-N selection. To overcome these limitations, we propose two innovations. First, we design an efficient hierarchical search algorithm with O(N+T) complexity that adaptively expands and prunes sampling trajectories. Second, we introduce a self-verified feedback mechanism that leverages the dMLLMs' intrinsic image understanding capabilities to assess text-image alignment, eliminating the need for external verifier. Extensive experiments on the GenEval benchmark across three representative dMLLMs (e.g., Lumina-DiMOO, MMaDA, Muddit) show that our framework substantially improves generation quality while achieving up to 6x greater efficiency than linear search.