PoInit-of-View:视图初始化投毒可跨多种三维重建系统迁移
PoInit-of-View: Poisoning Initialization of Views Transfers Across Multiple 3D Reconstruction Systems
针对 SfM 前端的关键点检测与特征匹配注入跨视角梯度不一致扰动,可作为研究三维重建流程前端脆弱性、以及设计对应防御或鲁棒特征匹配的切入点;其黑盒迁移设定(3DGS→NeRF)也提示可用 SfM 作为代理模型生成可迁移攻击。
原文依据 optimizes adversarial perturbations to intentionally introduce cross-view gradient inconsistencies at projections of corresponding 3D points
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论针对三维重建系统输入视图的投毒问题近来已有研究。然而,现有研究只是将对抗梯度整体地通过三维重建流程反向传播,并未揭示植根于流程特定模块中的新脆弱性。本文认为,作为众多广泛使用的重建系统的几何核心,运动恢复结构(SfM)可以被针对性地攻击以取得强投毒效果。为此,我们提出 PoInit-of-View,它优化对抗扰动,在对应三维点的投影处刻意引入跨视角梯度不一致。这些不一致会破坏关键点检测与特征匹配,从而污染 SfM 中的姿态估计与三角化,最终导致渲染视图质量低下。我们还给出了将跨视角不一致性与对应关系崩塌相联系的理论分析。实验结果表明,PoInit-of-View 在多种三维重建系统与数据集上均有效,在黑盒迁移设定(如 3DGS 到 NeRF)下,峰值信噪比超过基于单视图的方法 25.1%,结构相似度超过 16.5%。
英文原文摘要 1206 字符
Poisoning input views of 3D reconstruction systems has been recently studied. However, existing studies simply backpropagate adversarial gradients through the 3D reconstruction pipeline as a whole, without uncovering the new vulnerability rooted in specific modules of the pipeline. In this paper, we argue that structure-from-motion (SfM), as the geometric core of many widely used reconstruction systems, can be targeted to achieve strong poisoning effects. To this end, we propose PoInit-of-View, which optimizes adversarial perturbations to intentionally introduce cross-view gradient inconsistencies at projections of corresponding 3D points. These inconsistencies disrupt keypoint detection and feature matching, thereby corrupting pose estimation and triangulation within SfM and eventually resulting in low-quality rendered views. We also provide a theoretical analysis connecting cross-view inconsistency to correspondence collapse. Experimental results demonstrate the effectiveness of PoInit-of-View on diverse 3D reconstruction systems and datasets, surpassing the single-view-based method by 25.1% percent in PSNR and 16.5% percent in SSIM in black-box transfer settings, such as 3DGS to NeRF.