扩散 Transformer 中空间关系生成的电路机制
Circuit Mechanisms for Spatial Relation Generation in Diffusion Transformers
用电路级（cross-attention head 级）机制分析拆解 DiT 如何从文本 token 读取空间关系，为诊断和修复文生图模型的空间关系失败提供了可复用的分析范式：从零训练受控小模型、按文本编码器类型对比内部电路，并据此解释 OOD 鲁棒性差异。
原文依据 we adopt a mechanistic interpretability approach to investigate how a DiT can generate correct spatial relations between objects
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散 Transformer（DiT）极大推动了文本到图像生成的发展，但模型仍难以按文本提示中指定的方式生成物体之间正确的空间关系。本研究采用机制可解释性方法，考察 DiT 如何生成物体之间正确的空间关系。我们从零开始训练了不同规模、搭配不同文本编码器的 DiT，使其学习生成包含两个物体的图像，且这两个物体的属性与空间关系均由文本提示指定。我们发现，尽管所有模型都能以接近完美的准确率学会该任务，但其底层机制因文本编码器的选择而差异巨大。当使用随机文本嵌入时，空间关系信息通过一个两阶段电路传递给图像 token，该电路涉及两个交叉注意力头，分别读取文本提示中的空间关系与单个物体的属性。当使用预训练文本编码器（T5）时，DiT 采用另一种电路，利用文本 token 内部的信息融合，从单个文本 token 中同时读取空间关系与单物体信息。我们进一步表明，尽管两种设定在域内性能上相近，但它们对域外扰动的鲁棒性存在差异，这或许暗示了在真实场景中生成正确关系的困难。
英文原文摘要 1448 字符
Diffusion Transformers (DiTs) have greatly advanced text-to-image generation, but models still struggle to generate the correct spatial relations between objects as specified in the text prompt. In this study, we adopt a mechanistic interpretability approach to investigate how a DiT can generate correct spatial relations between objects. We train, from scratch, DiTs of different sizes with different text encoders to learn to generate images containing two objects whose attributes and spatial relations are specified in the text prompt. We find that, although all the models can learn this task to near-perfect accuracy, the underlying mechanisms differ drastically depending on the choice of text encoder. When using random text embeddings, we find that the spatial-relation information is passed to image tokens through a two-stage circuit, involving two cross-attention heads that separately read the spatial relation and single-object attributes in the text prompt. When using a pretrained text encoder (T5), we find that the DiT uses a different circuit that leverages information fusion in the text tokens, reading spatial-relation and single-object information together from a single text token. We further show that, although the in-domain performance is similar for the two settings, their robustness to out-of-domain perturbations differs, potentially suggesting the difficulty of generating correct relations in real-world scenarios.