FISHuman：基于多视图 4D 重拓扑的细粒度单图三维人体重建
FISHuman: Fine-grained Single-image 3D Human Reconstruction via Multi-view 4D Remeshing
把多视图视频扩散模型生成的 RGB/法线序列当作重建先验，再用 4D remeshing 逐顶点跟踪形变、解耦共享规范网格与瞬时变化，从而在多视图不一致的引导下仍能优化出统一 UV 纹理——这套「扩散先验 + 形变跟踪 + UV 融合」的流程可直接迁移到单图物体/人物资产重建与动画管线。
原文依据 we introduce a novel 4D remeshing module that explicitly disentangles the learning of the globally shared canonical mesh and transient variations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论单图三维人体重建因其便捷性和在各应用中的高需求而具有重要前景。已有方法通过采用二维多视图扩散模型生成辅助视图作为重建先验取得了巨大进展，但这些方法受困于三维不一致性和有限的泛化能力。本文提出 FISHuman，旨在从单视图输入生成细粒度、高保真且内容多样的三维人体，提供可直接用于生产的三维资产。我们设计了一套精细的工作流，从多视图不一致的引导中重建动态三维网格。具体而言，我们改造了一个基于双流 Transformer 的视频扩散模型，用以生成跨模态对齐的多视图 RGB 与法线序列。我们发现，直接采用静态三维重建会因生成帧缺乏三维感知而导致几何畸变和纹理模糊。为解决该问题，我们引入一种新颖的 4D 重拓扑模块，通过在不同视角下跟踪逐顶点形变，显式地解耦全局共享规范网格的学习与瞬时变化的学习。形变网格的拓扑一致性天然地使得能够优化统一的 UV 表示，从而有效整合各帧的外观属性。定性与定量实验结果均表明，本方法在外观真实感、几何精细度和泛化多样性方面优于先前工作。我们还展示了重建出的虚拟形象在动画与三维编辑等下游应用中的适用性。
英文原文摘要 1721 字符
Single-image 3D human reconstruction holds significant promise due to its convenience and high demand in various applications. Previous methods have garnered tremendous progress by employing 2D multi-view diffusion models to generate auxiliary views as reconstruction priors, but they struggle with 3D inconsistencies and limited generalization capabilities. In this paper, we present FISHuman, which aims to generate fine-grained, high-fidelity, and content-wise diverse 3D humans from a single-view input, providing production-ready 3D assets. We propose an elaborately designed workflow that reconstructs dynamic 3D meshes from multi-view inconsistent guidance. Specifically, we adapt a dual-stream transformer-based video diffusion model to generate cross-modally aligned multi-view RGB and normal sequences. We find that naively employing static 3D reconstruction can lead to geometric distortions and texture blurriness, due to the lack of 3D awareness within the generated frames. To address this, we introduce a novel 4D remeshing module that explicitly disentangles the learning of the globally shared canonical mesh and transient variations by tracking per-vertex deformations under different viewpoints. The topological consistency of the deformed meshes inherently enables the optimization of a unified UV representation that effectively integrates appearance attributes across frames. Both qualitative and quantitative experimental results demonstrate the superiority of our method over prior works in terms of appearance realism, geometric fineness, and generalization diversity. We also showcase the applicability of our reconstructed avatars for downstream applications including animation and 3D editing.