VidTAG：面向全球尺度的时序对齐视频到 GPS 地理定位与去噪序列预测
VidTAG: Temporally Aligned Video to GPS Geolocalization with Denoising Sequence Prediction at a Global Scale
把视频地理定位从图像检索式画廊匹配转为帧到 GPS 坐标的检索，并用 TempGeo 对齐帧嵌入、GeoRefiner 编码器-解码器做时序一致的 GPS 序列精修；对做视觉定位、跨模态检索与视频时序建模的人，这套「坐标库替代图像库 + 时序对齐精修」的范式可直接迁移到轨迹估计类任务。
原文依据 We propose VidTAG, a dual-encoder framework that performs frame-to-GPS retrieval using both self-supervised and language-aligned features.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频地理定位任务旨在确定视频来源的精确 GPS 坐标并绘制其轨迹，在取证、社交媒体与探索等场景中具有应用价值。现有的基于分类的方法仅在粗粒度的城市级别上运行，无法捕捉细粒度细节；而图像检索方法在全球尺度上不切实际，因为需要构建大规模图像库，而这难以实现。相比之下，构建 GPS 坐标库则简单且成本低廉。我们提出 VidTAG，一个双编码器框架，利用自监督特征与语言对齐特征执行帧到 GPS 的检索。为解决视频预测中的时序不一致问题，我们引入 TempGeo 模块，用于对齐帧嵌入；以及 GeoRefiner 模块，一种编码器-解码器架构，利用上述对齐后的帧嵌入对 GPS 特征进行精修。在 Mapillary（MSLS）与 GAMa 数据集上的评估表明，本模型能够生成时序一致的轨迹并优于基线，在 1 公里阈值上相较 GeoCLIP 提升 20%。在全球粗粒度视频地理定位（CityGuessr68k）上，我们还以 25% 的优势超越了当前最优。我们的方法实现了细粒度视频地理定位，并为未来研究奠定了坚实基础。更多细节见项目网页：parthpk.github.io/vidtag webpage。
英文原文摘要 1436 字符
The task of video geolocalization aims to determine the precise GPS coordinates of a video's origin and map its trajectory; with applications in forensics, social media, and exploration. Existing classification-based approaches operate at a coarse city-level granularity and fail to capture fine-grained details, while image retrieval methods are impractical on a global scale due to the need for extensive image galleries which are infeasible to compile. Comparatively, constructing a gallery of GPS coordinates is straightforward and inexpensive. We propose VidTAG, a dual-encoder framework that performs frame-to-GPS retrieval using both self-supervised and language-aligned features. To address temporal inconsistencies in video predictions, we introduce the TempGeo module, which aligns frame embeddings, and the GeoRefiner module, an encoder-decoder architecture that refines GPS features using said aligned frame embeddings. Evaluations on Mapillary (MSLS) and GAMa datasets demonstrate our model's ability to generate temporally consistent trajectories and outperform baselines, achieving a 20% improvement at the 1 km threshold over GeoCLIP. We also beat current State-of-the-Art by 25% on global coarse grained video geolocalization (CityGuessr68k). Our approach enables fine-grained video geolocalization and lays a strong foundation for future research. More details on the project webpage: parthpk.github.io/vidtag webpage.