ST4R-Splat：4D 高斯泼溅中的时空指代分割
ST4R-Splat: Spatio-Temporal Referring Segmentation in 4D Gaussian Splatting
将指代分割从静态 3D 扩展到动态 4D 高斯泼溅，提出时间不变实例嵌入与实例级时序状态映射，可直接用于语言驱动的 4D 场景理解与编辑。
原文依据 We introduce Spatio-Temporal Referring Segmentation in 4D Gaussian Splatting (STRS-4DGS), a novel task aiming to jointly identify and segment a target instance across space and time given a referring expression.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论通过自然语言理解动态 4D 环境中的物体至关重要，但仍未被充分探索。现有方法主要关注静态 3D 指代分割或开放词表 4D 查询，难以在显式 4D 重建中定位复杂的时空指代表达。我们引入 4D 高斯泼溅中的时空指代分割（STRS-4DGS），这是一个新任务，旨在给定指代表达的情况下，跨空间和时间联合识别并分割目标实例。为此，我们提出 ST4R-Splat，这是首个用于 STRS-4DGS 的框架。具体而言，我们的框架包含一个实例感知 4D 高斯指代场，为稳健的空间定位分配时间不变的嵌入，以及一个实例级时序状态映射模块，直接在特征空间中实现视角无关的时序定位。为提供丰富的时空语义监督，我们开发了一个基于多模态大语言模型的自动描述生成流程，生成解耦的空间和时间文本描述。在我们新构建的 STRS-4DGS 基准上评估，我们的方法在时间无关和时间敏感指标上均显著优于适配的最优基线，为语言驱动的 4D 场景理解奠定了坚实基础。
英文原文摘要 1308 字符
Understanding objects in dynamic 4D environments via natural language is crucial yet underexplored. While existing methods focus on static 3D referring segmentation or open-vocabulary 4D querying, they struggle to ground complex spatio-temporal referring expressions in explicit 4D reconstructions. We introduce Spatio-Temporal Referring Segmentation in 4D Gaussian Splatting (STRS-4DGS), a novel task aiming to jointly identify and segment a target instance across space and time given a referring expression. To tackle this, we propose ST4R-Splat, the first framework for STRS-4DGS. Specifically, our framework incorporates an Instance-Aware 4D Gaussian Referring Field that assigns time-invariant embeddings for robust spatial grounding, and an Instance-Level Temporal State Mapping module that enables view-independent temporal localization directly in feature space. To provide rich spatio-temporal semantic supervision, we develop an automatic, MLLM-based captioning pipeline that generates decoupled spatial and temporal textual descriptions. Evaluated on our newly constructed STRS-4DGS benchmark, our method significantly outperforms adapted state-of-the-art baselines across both time-agnostic and time-sensitive metrics, establishing a strong foundation for language-driven 4D scene understanding.