生成模型与扩散 · CVPR2026 ·2026.06.03

LumiX:结构化且连贯的文本到本征图生成

LumiX: Structured and Coherent Text-to-Intrinsic Generation

Xu Han、Biao Zhang、Xiangjun Tang、Xianzhi Li、Peter Wonka

与你有关的部分

Query-Broadcast Attention 让所有本征图共享同一组 query,从而在联合扩散采样中保持跨图结构一致;Tensor LoRA 用张量化低秩适配建模跨图关系,可作为多输出联合生成任务中参数高效训练与一致性的通用手段。

原文依据 Query-Broadcast Attention, a mechanism that ensures structural consistency by sharing queries across all maps in each self-attention block
可迁移到 text-to-image generationintrinsic image decompositionmulti-task dense prediction

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

本文提出 LumiX,一个用于连贯文本到本征图生成的结构化扩散框架。以文本提示为条件,LumiX 联合生成一整套本征图(如反照率、辐照度、法线、深度和最终颜色),从而对底层场景提供结构化且物理一致的描述。这一能力由两项关键贡献实现:1)Query-Broadcast Attention,一种在每个自注意力块中让所有图共享 query 的机制,用以保证结构一致性;2)Tensor LoRA,一种基于张量的适配方法,以参数高效的方式建模跨图关系,从而实现高效的联合训练。这些设计共同使稳定的联合扩散训练与多种本征属性的统一生成成为可能。实验表明,LumiX 能产生连贯且具有物理意义的结果,与当前最优方法相比,对齐度提升 23%,偏好得分更优(0.19 对 -0.41),并且能在同一框架内执行以图像为条件的本征分解。

英文原文摘要 1046 字符

We present LumiX, a structured diffusion framework for coherent text-to-intrinsic generation. Conditioned on text prompts, LumiX jointly generates a comprehensive set of intrinsic maps (e.g., albedo, irradiance, normal, depth, and final color), providing a structured and physically consistent description of an underlying scene. This is enabled by two key contributions: 1) Query-Broadcast Attention, a mechanism that ensures structural consistency by sharing queries across all maps in each self-attention block. 2) Tensor LoRA, a tensor-based adaptation that parameter-efficiently models cross-map relations for efficient joint training. Together, these designs enable stable joint diffusion training and unified generation of multiple intrinsic properties. Experiments show that LumiX produces coherent and physically meaningful results, achieving 23% higher alignment and a better preference score (0.19 vs. -0.41) compared to the state of the art, and it can also perform image-conditioned intrinsic decomposition within the same framework.

标签

会议 CVPR2026
页码 21942-21952
Bibkey
来源 CVF Open Access