NeuroSeg 遇上 DINOv3:通过 DINOv3 初始化将 2D 自监督视觉先验迁移至 3D 神经元分割
NeuroSeg Meets DINOv3: Transferring 2D Self-Supervised Visual Priors to 3D Neuron Segmentation via DINOv3 Initialization
把 DINOv3 的 2D 自监督表征通过 filter inflation 迁移到 3D 神经元体数据分割,为标注稀缺的体成像任务提供了一条可复用的数据高效初始化路径;拓扑感知骨架损失也可直接用于其他管状/树状结构的重建。
原文依据 we design an inflation-based adaptation strategy that inflates 2D filters into 3D operators, preserving semantic priors from DINOv3
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论以 DINOv3 为代表的 2D 视觉基础模型在大规模自然图像上以自监督方式训练,已展现出强大的零样本泛化能力,能够捕获丰富的全局上下文与细粒度结构线索。然而,面向下游体成像神经影像的同类 3D 基础模型仍然缺失,这主要归因于 3D 图像采集的困难以及高质量标注的稀缺。为填补这一空白,我们提出将 DINOv3 学到的 2D 视觉表征适配到 3D 生物医学分割模型,从而实现更数据高效且形态更忠实的神经元重建。具体而言,我们设计了一种基于膨胀的适配策略,将 2D 滤波器膨胀为 3D 算子,在适配 3D 神经元体数据块的同时保留来自 DINOv3 的语义先验。此外,我们引入拓扑感知的骨架损失,以显式约束基于图的神经元树突重建的结构保真度。在四个神经元成像数据集(包括来自 BigNeuron 的两个数据集以及 NeuroFly 和 CWMBS 两个公开数据集)上的大量实验表明,该方法在重建精度上相较当前最优方法取得一致提升,在整体结构平均值上平均提升 2.9%,在不同结构平均值上平均提升 2.8%,在不同结构百分比上平均提升 3.8%。
英文原文摘要 1321 字符
2D visual foundation models, such as DINOv3, a self-supervised model trained on large-scale natural images, have demonstrated strong zero-shot generalization, capturing rich global context and fine-grained structural cues. However, an analogous 3D foundation model for downstream volumetric neuroimaging remains lacking, largely due to the challenges of 3D image acquisition and the scarcity of high-quality annotations. To address this gap, we propose to adapt the 2D visual representations learned by DINOv3 to a 3D biomedical segmentation model, enabling more data-efficient and morphologically faithful neuronal reconstruction. Specifically, we design an inflation-based adaptation strategy that inflates 2D filters into 3D operators, preserving semantic priors from DINOv3 while adapting to 3D neuronal volume patches. In addition, we introduce a topology-aware skeleton loss to explicitly enforce structural fidelity of graph-based neuronal arbor reconstruction. Extensive experiments on four neuronal imaging datasets, including two from BigNeuron and two public datasets, NeuroFly and CWMBS, demonstrate consistent improvements in reconstruction accuracy over SoTA methods, with average gains of 2.9% in Entire Structure Average, 2.8% in Different Structure Average, and 3.8% in Percentage of Different Structure.