无像素感知：从相机轨迹中获取感知
Seeing without Pixels: Perception from Camera Trajectories
把相机位姿轨迹当作一种轻量模态，用对比学习对齐到语言嵌入空间，为视频理解提供了一条不依赖像素的表征路径；做视频-文本检索或第一人称动作理解时，可考虑把轨迹编码器作为额外或替代的视觉分支。
原文依据 we propose a contrastive learning framework to train CamFormer, a dedicated encoder that projects camera pose trajectories into a joint embedding space, aligning them with natural language
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论能否不看视频像素、仅凭相机轨迹——即相机在空间中划出的路径——来感知视频内容？本文首次系统性地研究了这一看似不可思议的问题。为此，我们提出一个对比学习框架来训练 CamFormer，这是一个专用编码器，将相机位姿轨迹投影到联合嵌入空间，并使其与自然语言对齐。我们发现，与表面上的简单性相反，相机轨迹是一种信息量极高的信号，可用于揭示视频内容。换言之，「你如何移动」确实能为「你在做什么」（第一人称）或「你在观察什么」（第三人称）提供有价值的线索。我们在从跨模态对齐到分类与时序分析的一系列多样化下游任务上，展示了所学 CamFormer 嵌入的通用性。重要的是，我们的表征在多种相机位姿估计方法下都具有鲁棒性，包括高保真多传感器估计器和标准的仅 RGB 估计器。我们的发现确立了相机轨迹作为一种轻量、鲁棒且通用的模态，用于感知视频内容。
英文原文摘要 1166 字符
Can one perceive a video's content without seeing its pixels, just from the camera trajectory--the path it carves through space? This paper is the first to systematically investigate this seemingly implausible question. Towards this end, we propose a contrastive learning framework to train CamFormer, a dedicated encoder that projects camera pose trajectories into a joint embedding space, aligning them with natural language. We find that, contrary to its apparent simplicity, the camera trajectory is a remarkably informative signal to uncover video content. In other words, "how you move" can indeed provide valuable cues about "what you are doing" (egocentric) or "observing" (exocentric). We demonstrate the versatility of our learned CamFormer embeddings on a diverse suite of downstream tasks, ranging from cross-modal alignment to classification and temporal analysis. Importantly, our representations are robust across diverse camera pose estimation methods, including both high-fidelity multi-sensored and standard RGB-only estimators. Our findings establish camera trajectory as a lightweight, robust, and versatile modality for perceiving video content.