4DP-QA：面向视觉语言模型 4D 感知的可扩展问答
4DP-QA: Scalable QA for 4D Perception in Vision Language Models
提出 True-Motion Tracking 这一固定参考系下的跟踪表述，把相机运动与物体运动解耦，可直接用于构建视频/4D 场景的 VLM 训练数据与评测基准，缓解 VLM 只能通过 2D 投影间接观察运动的问题。
原文依据 casting tracking in both the traditional way and in a novel, fixed reference system, dubbed True-Motion Tracking
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管近期取得了进展，视觉语言模型（VLM）仍难以把握世界的动态。我们注意到，对 4D 场景进行推理本身已颇具挑战，而两个因素使其更加复杂。首先，VLM 只能通过运动在 2D 图像上的投影来间接观察运动。其次，现有数据集未能将物体运动与相机运动解耦。为应对这些挑战，我们提出了一条聚焦于运动相关场景理解的问答生成流水线。我们特别处理了相机运动与物体运动的纠缠问题：既采用传统方式进行跟踪，也采用一种新颖的固定参考系进行跟踪，后者被称为 True-Motion Tracking，能够提供对运动的直观描述。基于该流水线，我们生成了一个包含 40 万样本的大规模训练数据集 4DP-QA（4D Perception QA），以及一个包含 2200 个样本的基准 4DP-QA-Bench。在该数据集上训练现有模型可在外部基准上取得性能提升，验证了方法的有效性。
英文原文摘要 1014 字符
Despite recent advances, Vision Language Models (VLMs) still struggle to grasp the dynamics of the world. We note that the ability to reason about a 4D scene, challenging in itself, is further complicated by two factors. First, VLMs observe motion indirectly via its projection onto 2D images. Second, existing datasets fail to disentangle object and camera motion. To address these challenges, we present a QA generation pipeline that focuses on motion-related scene understanding. We take particular care of the entanglement of camera and object motion by casting tracking in both the traditional way and in a novel, fixed reference system, dubbed True-Motion Tracking, which provides an intuitive description of motion. From this pipeline, we generate a large-scale training dataset of 400K samples, 4DP-QA (4D Perception QA), and a 2.2K-sample benchmark, 4DP-QA-Bench. Training existing models on our dataset yields performance improvements on an external benchmark, validating the effectiveness of our method.