IVAAN:面向细胞核分析的属性引导文本提示生成实例级视觉语言对齐
IVAAN: Instance-level Vision-Language Alignment via Attribute-Guided Text Prompts Generation for Nuclei Analysis
用真值掩码自动生成属性引导的文本描述,再以对比学习把实例级视觉表征与语义锚点对齐,并学习类内多个原型 token 建模子模态——这套「无人工文本标注的实例级视觉语言对齐 + 原型语义反馈」可直接迁移到其他缺乏实例级文本标注的密集预测任务(如细胞/病灶/遥感目标分割)。
原文依据 We introduce an instance-level vision-language framework that derives attribute-guided textual descriptions from ground-truth masks.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论细胞核实例分割与分类是病理学中的基础任务,但由于严重的类别不平衡以及器官和染色带来的变异性,仍然具有挑战性。尽管视觉语言方法能够注入显式语义线索,从而在不平衡条件下减少虚假的上下文偏置,但实例级文本标注的缺失限制了它们在细胞核级分析中的效用。我们提出一个实例级视觉语言框架,从真值掩码中导出属性引导的文本描述。随后通过对比学习将视觉表征与这些语义文本锚点对齐,在实例级别将形态与语义耦合起来。为了在保持器官一致的类别语义的同时捕捉类内变异,我们学习多个类别特定的 token,它们作为原型表示一个类别内多样的子模态,对形态相似的细胞核进行汇总。我们的方法无需人工文本标签即可同时提升分割与分类性能,表明语言引导的实例对齐结合基于原型的语义反馈能够产生更具判别性和泛化性的细胞核表征。
英文原文摘要 1177 字符
Nuclei instance segmentation and classification are fundamental but remain challenging in pathology due to severe class imbalance and organ- and stain-induced variability. While vision-language approaches can inject explicit semantic cues that reduce spurious contextual bias under imbalance, the absence of instance level textual annotations has limited their utility for nucleus-level analysis. We introduce an instance-level vision-language framework that derives attribute-guided textual descriptions from ground-truth masks. We then align visual representations with these semantic text anchors via contrastive learning, coupling morphology with semantics at the instance level. To capture intra-class variations while maintaining organ-consistent class semantics, we learn multiple class-specific tokens that act as prototypes representing diverse submodes within a class, summarizing morphologically similar nuclei. Our approach improves both segmentation and classification without manual text labels, indicating that language-guided instance alignment combined with prototype-based semantic feedback yields more discriminative and generalizable nuclei representations.