深度特征变形权重
Deep Feature Deformation Weights
把深度视觉特征的邻近性直接当作网格变形权重,并用重心特征蒸馏让蒸馏复杂度与网格分辨率解耦——这套思路可迁移到任何需要从2D渲染信号反推3D表面稠密属性的任务(如表面属性场、可变形重建、实时蒙皮权重预测)。
原文依据 We introduce barycentric feature distillation, an improved feature distillation pipeline which leverages the full visual signal from shape renders
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于控制柄的网格变形是计算机图形学中的经典范式,能够通过稀疏控制实现直观编辑。经典技术快速且精确,但要求用户事先知道理想的控制柄位置,这可能不直观且不一致。控制柄集合难以调整,因为权重通常通过由控制柄定义的能量函数进行优化。另一方面,现代数据驱动方法提供语义编辑,但牺牲了细粒度控制与速度。我们提出一种兼得两者优势的技术:深度特征邻近性产生平滑、视觉感知的变形权重,且无需额外正则化。重要的是,这些权重可对任意表面点实时计算,而先前方法需要昂贵的优化。我们引入重心特征蒸馏,一种改进的特征蒸馏流程,利用形状渲染的完整视觉信号,使蒸馏复杂度对网格分辨率保持鲁棒。这使得高分辨率网格可在数分钟内处理,而先前方法可能需要数小时。我们通过特征空间约束和局部性加权保留并扩展了经典性质。我们的场表示支持自动视觉对称性检测,并据此生成保持对称性的变形。我们展示了一个概念验证应用,可在消费级机器上实时为多达100万面的网格生成变形。项目页面:https://threedle.github.io/dfd。
英文原文摘要 1538 字符
Handle-based mesh deformation is a classic paradigm in computer graphics which enables intuitive edits from sparse controls. Classical techniques are fast and precise, but require users to know ideal handle placement apriori, which can be unintuitive and inconsistent. Handle sets cannot be adjusted easily, as weights are typically optimized through energies defined by the handles. Modern data-driven methods, on the other hand, provide semantic edits but sacrifice fine-grained control and speed. We propose a technique that achieves the best of both worlds: deep feature proximity yields smooth, visual-aware deformation weights with no additional regularization. Importantly, these weights are computed in real-time for any surface point, unlike prior methods which require expensive optimization. We introduce barycentric feature distillation, an improved feature distillation pipeline which leverages the full visual signal from shape renders to make distillation complexity robust to mesh resolution. This enables high resolution meshes to be processed in minutes versus potentially hours for prior methods. We preserve and extend classical properties through feature space constraints and locality weighting. Our field representation enables automatic visual symmetry detection, which we use to produce symmetry-preserving deformations. We show a proof-of-concept application which can produce deformations for meshes up to 1 million faces in real-time on a consumer-grade machine. Project page at https://threedle.github.io/dfd.