Ego-InBetween：在第一人称视频中生成物体状态转移
Ego-InBetween: Generating Object State Transitions in Ego-Centric Videos
把「初始态→目标态」的中间帧生成显式建模为多步状态转移，并用 VLM 推断转移步骤作为生成条件，为视频生成中的长程物理一致性提供了一条可复用的「先推理后生成」范式。
原文依据 generating intermediate frames that depict object transformations between initial and target states under a brief action instruction
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论理解物理变化过程对人类认知与人工智能系统都至关重要，尤其是从第一人称视角出发，因为该视角是人与机器在动作建模之间沟通的关键桥梁。我们将这一建模过程定义为第一人称指令视觉状态转移（Egocentric Instructed Visual State Transition, EIVST），其任务是在一段简短动作指令下，生成描绘物体从初始状态到目标状态之间变化的中间帧。EIVST 对当前生成模型提出了两点挑战：（1）理解初始状态与目标状态的视觉场景，并从第一人称视角推理转移步骤；（2）生成与给定指令一致、同时在两个视觉状态之间保持物体外观一致的中间转移过程。为应对这些挑战，我们提出 EgoIn 框架。该框架首先使用 TransitionVLM 推断两个给定状态之间的多步转移过程，TransitionVLM 在我们构建的数据集上经过微调，以更好地适配该任务并减少幻觉信息。随后，它基于所提出的转移条件模块产生的转移条件生成一段帧序列。此外，我们引入物体感知辅助监督，以在整个转移过程中保持一致的物体外观。在人物-物体与机器人-物体交互数据集上的大量实验表明，EgoIn 在生成语义有意义且视觉连贯的变化序列方面具有更优性能。
英文原文摘要 1547 字符
Understanding physical transformation processes is crucial for both human cognition and artificial intelligence systems, particularly from an egocentric perspective, which serves as a key bridge between humans and machines in action modeling. We define this modeling process as Egocentric Instructed Visual State Transition (EIVST), which involves generating intermediate frames that depict object transformations between initial and target states under a brief action instruction. EIVST poses two challenges for current generative models: (1) understanding the visual scenes of the initial and target states and reasoning about transformation steps from an egocentric view, and (2) generating a consistent intermediate transition that follows the given instruction while preserving object appearance across the two visual states. To address these challenges, we propose the EgoIn framework. It first infers the multi-step transition process between two given states using TransitionVLM, fine-tuned on our curated dataset to better adapt to this task and reduce hallucinated information. It then generates a sequence of frames based on transition conditions produced by the proposed Transition Conditioning module. Additionally, we introduce Object-aware Auxiliary Supervision to preserve consistent object appearance throughout the transition. Extensive experiments on human-object and robot-object interaction datasets demonstrate EgoIn's superior performance in generating semantically meaningful and visually coherent transformation sequences.