Talk2Move：基于强化学习的场景中文本指令物体级几何变换
Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenes
将强化学习（GRPO）与扩散模型结合，通过设计物体级空间奖励（位移、旋转、缩放）实现文本指令驱动的物体几何变换，为可控图像编辑提供了一种无需配对监督的优化范式。
原文依据 Talk2Move employs Group Relative Policy Optimization (GRPO) to explore geometric actions through diverse rollouts generated from input images and lightweight textual variations, removing the need for costly paired data.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 Talk2Move，一个基于强化学习（RL）的扩散框架，用于场景中物体的文本指令空间变换。通过自然语言在场景中空间地操控物体对多模态生成系统而言是一项挑战。现有的基于文本的操控方法可以调整外观或风格，但由于缺乏配对监督和像素级优化的限制，它们难以执行物体级几何变换——例如平移、旋转或调整物体大小。Talk2Move 采用组相对策略优化（GRPO），通过从输入图像和轻量文本变化生成多样化 rollout 来探索几何动作，从而无需昂贵的配对数据。一个空间奖励引导模型将几何变换与语言描述对齐，而离策略步骤评估和主动步骤采样通过聚焦信息丰富的变换阶段来提高学习效率。此外，我们设计了以物体为中心的空间奖励，直接评估位移、旋转和缩放行为，从而实现可解释且连贯的变换。在精心整理的基准上的实验表明，Talk2Move 实现了精确、一致且语义忠实的物体变换，在空间准确性和场景连贯性上均优于现有的文本引导编辑方法。
英文原文摘要 1399 字符
We introduce Talk2Move, a reinforcement learning (RL) based diffusion framework for text-instructed spatial transformation of objects within scenes. Spatially manipulating objects in a scene through natural language poses a challenge for multimodal generation systems. While existing text-based manipulation methods can adjust appearance or style, they struggle to perform object-level geometric transformations--such as translating, rotating, or resizing objects--due to scarce paired supervision and pixel-level optimization limits. Talk2Move employs Group Relative Policy Optimization (GRPO) to explore geometric actions through diverse rollouts generated from input images and lightweight textual variations, removing the need for costly paired data. A spatial reward guided model aligns geometric transformations with linguistic description, while off-policy step evaluation and active step sampling improve learning efficiency by focusing on informative transformation stages. Furthermore, we design object-centric spatial rewards that evaluate displacement, rotation, and scaling behaviors directly, enabling interpretable and coherent transformations.Experiments on curated benchmarks demonstrate that Talk2Move achieves precise, consistent, and semantically faithful object transformations, outperforming existing text-guided editing approaches in both spatial accuracy and scene coherence.