任意到任意的 3D 扩散模型与知识迁移:一项放疗计划研究
Any2Any 3D Diffusion Models with Knowledge Transfer: A Radiotherapy Planning Study
把预训练视频扩散骨干的时空先验迁移到体素级 3D 预测任务上,并用「模态+角色」感知的 Any2Any 条件接口替代交叉注意力,为视觉研究者提供了一个可复用的思路:如何在不改动骨干结构的前提下,把异构条件(图像、掩码、束流描述符)统一注入扩散模型,并用奖励后训练对齐领域偏好。
原文依据 transfers prior knowledge from pretrained video diffusion models to clinically meaningful dose prediction
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论体素级剂量预测是放疗计划中关键但具有挑战性的任务,因为从零开始训练的定制模型往往难以在不同机构、扫描仪和计划协议之间泛化。与此同时,在十亿级视觉数据上预训练的大型生成骨干已学到较强的几何与结构先验,这些先验可能迁移到其源域之外。我们提出 DiffKT3D,一个任意到任意(Any2Any)的 3D 扩散框架,将预训练视频扩散模型中的先验知识迁移到具有临床意义的剂量预测任务上。为支持对异构临床输入(包括 CT、靶区与器官掩码、体表掩码以及束流描述符)的灵活条件化,我们引入一种模态与角色感知的任意到任意建模方式,在避免使用交叉注意力的同时保留统一的条件接口。我们进一步采用强化学习(RL)后训练,并配合具有临床信息的 Scorecard 奖励,使生成结果与机构的治疗偏好对齐。在 GDP--HMM 挑战赛上,DiffKT3D 相对最佳挑战赛方案将体素级 MAE 从 2.07 降至 1.93,同时改善了图像质量与临床偏好对齐程度。这些结果表明,预训练扩散先验结合灵活的多模态条件化与 Scorecard 对齐的后训练,为放疗剂量预测提供了一条有效途径。
英文原文摘要 1379 字符
Voxel-wise dose prediction is a critical yet challenging task in radiotherapy (RT) planning, as bespoke models trained from scratch often struggle to generalize across institutions, scanners, and planning protocols. Meanwhile, large generative backbones pretrained on billion-scale visual data have learned strong geometric and structural priors that may transfer beyond their source domain. We propose DiffKT3D, an Any2Any 3D diffusion framework that transfers prior knowledge from pretrained video diffusion models to clinically meaningful dose prediction. To support flexible conditioning over heterogeneous clinical inputs, including CT, target and organ masks, body mask, and beam descriptors, we introduce a modality- and role-aware Any2Any formulation that avoids cross-attention while retaining a unified conditioning interface. We further apply reinforcement-learning (RL) post-training with a clinically informed Scorecard reward to align generations with institutional treatment preferences. On the GDP--HMM challenge, DiffKT3D reduces voxel-wise MAE from 2.07 to 1.93 relative to the top challenge solution, while also improving image quality and clinical preference alignment. These results show that pretrained diffusion priors, combined with flexible multimodal conditioning and scorecard-aligned post-training, provide an effective approach to RT dose prediction.