超越像素模拟:基于诊断语义令牌和原型控制的病理图像生成
Beyond Pixel Simulation: Pathology Image Generation via Diagnostic Semantic Tokens and Prototype Control
利用冻结的病理多模态大模型提取诊断语义令牌,并结合原型库实现细粒度形态控制,为医学图像生成提供了可复用的语义控制范式。
原文依据 UniPath implements Multi-Stream Control: a Raw-Text stream; a High-Level Semantics stream that uses learnable queries to a frozen pathology MLLM to distill paraphrase-robust Diagnostic Semantic Tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在计算病理学中,理解与生成沿着不同的路径发展:先进的理解模型已经展现出诊断级别的能力,而生成模型主要是在模拟像素。进展仍受到三个相互关联因素的阻碍:大规模高质量图文语料的稀缺;缺乏精确、细粒度的语义控制,导致依赖非语义线索;以及术语异质性,即同一诊断概念的不同表述阻碍了可靠的文本条件控制。我们提出了 UniPath,一个语义驱动的病理图像生成框架,它利用成熟的诊断理解能力来实现可控生成。UniPath 实现了多流控制:原始文本流;高层语义流,该流使用可学习查询访问冻结的病理多模态大模型,以提炼出对复述鲁棒的诊断语义令牌,并将提示扩展为诊断感知的属性包;以及原型流,通过原型库提供组件级的形态控制。在数据方面,我们整理了一个包含 265 万图文对的数据集和一个精细标注的高质量 6.8 万子集,以缓解数据稀缺问题。为了进行全面评估,我们建立了一个针对病理学量身定制的四级评估体系。大量实验表明 UniPath 达到了最先进的性能,包括 Patho-FID 为 80.9(比第二名好 51%)以及细粒度语义控制达到真实图像的 98.7%。数据集和代码可从 https://github.com/Hanminghao/UniPath 获取。
英文原文摘要 1574 字符
In computational pathology, understanding and generation have evolved along disparate paths: advanced understanding models already exhibit diagnostic-level competence, whereas generative models largely simulate pixels. Progress remains hindered by three coupled factors: the scarcity of large, high-quality image-text corpora; the lack of precise, fine-grained semantic control, which forces reliance on non-semantic cues; and terminological heterogeneity, where diverse phrasings for the same diagnostic concept impede reliable text conditioning. We introduce UniPath, a semantics-driven pathology image generation framework that leverages mature diagnostic understanding to enable controllable generation. UniPath implements Multi-Stream Control: a Raw-Text stream; a High-Level Semantics stream that uses learnable queries to a frozen pathology MLLM to distill paraphrase-robust Diagnostic Semantic Tokens and to expand prompts into diagnosis-aware attribute bundles; and a Prototype stream that affords component-level morphological control via a prototype bank. On the data front, we curate a 2.65M image-text corpus and a finely annotated, high-quality 68K subset to alleviate data scarcity. For a comprehensive assessment, we establish a four-tier evaluation hierarchy tailored to pathology. Extensive experiments demonstrate UniPath's SOTA performance, including a Patho-FID of 80.9 (51% better than the second-best) and fine-grained semantic control achieving 98.7% of the real-image. The dataset and code can be obtained from https://github.com/Hanminghao/UniPath.