Refacade：使用给定参考纹理编辑物体
Refacade: Editing Object with Given Reference Texture
提出基于配对3D网格渲染训练纹理移除器，并结合拼图置换破坏参考图全局布局，实现从参考物体到目标物体的局部纹理迁移；该解耦思路可复用于视频编辑中的外观控制与结构保持。
原文依据 we employ a texture remover trained on paired textured/untextured 3D mesh renderings to remove appearance information while preserving geometry and motion of source videos
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型的最新进展在图像和视频编辑方面取得了显著进步，但一些任务仍未得到充分探索。本文我们将物体重纹理扩展至视频领域，该任务将参考物体中的局部纹理迁移到图像或视频中的目标物体上。为完成此任务，一种直接方案是使用以源结构和参考纹理为条件的ControlNet。然而，该方法因两个原因导致可控性有限：以原始参考图像为条件引入了不必要的结构信息，且该方法未能解耦源物体的视觉纹理与结构信息。为解决此问题，我们提出名为Refacade的方法，其包含两个关键设计，以在图像和视频中实现精确且可控的纹理迁移。首先，我们采用在配对的有纹理/无纹理3D网格渲染上训练的纹理移除器，以移除外观信息同时保留源视频的几何与运动。其次，我们使用拼图置换破坏参考图的全局布局，促使模型关注局部纹理统计而非物体的全局布局。大量实验表明，该方法在视觉质量、精确编辑和可控性方面表现优异，在定量和人工评估中均优于强基线。
英文原文摘要 1381 字符
Recent advances in diffusion models have brought remarkable progress in image and video editing, yet some tasks remain underexplored. In this paper, we extend Object Retexture into video domain, which transfers local textures from a reference object to a target object in images or videos. To perform this task, a straightforward solution is to use ControlNet conditioned on the source structure and the reference texture. However, this approach suffers from limited controllability due to two reasons: conditioning on the raw reference image introduces unwanted structural information, and this method fails to disentangle visual texture and structure information of the source. To address this problem, we proposed a method, namely Refacade, that consists of two key designs to achieve precise and controllable texture transfer in both images and videos. First, we employ a texture remover trained on paired textured/untextured 3D mesh renderings to remove appearance information while preserving geometry and motion of source videos. Second, we disrupt the reference's global layout using a jigsaw permutation, encouraging the model to focus on local texture statistics rather than global layout of object. Extensive experiments demonstrate superior visual quality, precise editing, and controllability, outperforming strong baselines in both quantitative and human evaluations.