3D 视觉与重建 · CVPR2026 ·2026.06.03

更自然、更真实:面向人脑 3D 视觉解码的对象感知高斯泼溅

More Natural, More Real: Object-aware Gaussian Splatting for 3D Visual Decoding from Human Brain

Haodong Jing、Dongyao Jiang、Jixin Wang、Junhao Jia、Yanshu Li、Yongqiang Ma 等 7 人

与你有关的部分

把 3DGS 的初始控制参数(位置/颜色/语义锚点)改由 fMRI/EEG 特征回归得到,并用多视图稳定器约束跨视角一致性——这套「神经信号→3DGS 属性」的解码管线可迁移到任何需要从低维/噪声条件生成可控 3D 表示的场景(如条件式 3D 生成、脑机接口驱动的资产创建)。

原文依据 BrainGS incorporates a Fusion Time-Spatial Network to achieve comprehensive encoding of the brain, combined with the Multi-Attribute Controller (MAC)
可迁移到 novel view synthesis3D reconstructionbrain decodingconditional 3D generation

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

探索人类对立体世界的视觉感知与理解,是计算神经科学中的一个重要课题。近期研究已提供了丰富的 Brain-3D 数据集,并对 3D 视觉重建开展了初步探索。然而,现有研究难以捕捉 3D 刺激视图动态变化的差异,在高保真重建与渲染方面仍有改进空间。3D 高斯泼溅(3DGS)近来在立体视图合成上取得了显著进展。受其启发,我们提出 BrainGS——一个从大脑中解码出更真实 3D 物体的创新框架。BrainGS 引入融合时空网络以实现对大脑信号的全面编码,并结合多属性控制器(MAC),以视觉、语义和颜色为锚点对特征进行解耦,有效学习 Brain-3D 的特征分布,为 3DGS 提供初始控制。多视图稳定器(MVS)克服了捕捉 3D 物体多视图变化的难题,构建出更鲁棒的视点表征。在 fMRI/EEG 上的全面实验与讨论表明 BrainGS 取得了 SOTA 性能(2.936 FPD,0.202 LPIPS),提供了可靠的神经层面解释,为理解大脑立体视觉提供了新见解。

英文原文摘要 1359 字符

Exploring human visual perception and understanding of the stereoscopic world represents a significant topic in computational neuroscience. Recent studies have provided rich Brain-3D datasets, conducted preliminary explorations into 3D visual reconstruction. However, existing research struggles to capture the differences in dynamic changes of 3D stimulus views, and there remains room for improvement in high-fidelity reconstruction and rendering. 3D Gaussian Splatting (3DGS) has recently achieved significant progress in stereoscopic view synthesis. Inspired by it, we propose BrainGS -- an innovative framework for decoding more realistic 3D objects from the brain. BrainGS incorporates a Fusion Time-Spatial Network to achieve comprehensive encoding of the brain, combined with the Multi-Attribute Controller (MAC), it decouples features using visual, semantic, and color as anchors, effectively learning the feature distribution of Brain-3D and providing initial control for 3DGS. The Multi-View Stabilizer (MVS) overcomes the challenge of capturing multi-view changes of 3D objects, creating more robust viewpoint representations. Comprehensive experiments and discussions on fMRI/EEG show the SOTA performance (2.936 FPD, 0.202 LPIPS) of BrainGS, providing reliable neural interpretations, offering new insights into brain stereovision understanding.

标签

会议 CVPR2026
页码 19033-19044
Bibkey
来源 CVF Open Access