图到帧 RAG：面向免训练与可审计视频推理的视觉空间知识融合
Graph-to-Frame RAG: Visual-Space Knowledge Fusion for Training-Free and Auditable Video Reasoning
把外部知识渲染成单张「推理帧」拼进视频，让 LMM 在统一视觉域内做联合推理，为视觉问答/视频推理提供了一种免训练、可审计的知识注入接口，可直接替换文本或多样本检索增强的拼接方式。
原文依据 renders it as a single reasoning frame appended to the video. LMMs then perform joint reasoning in a unified visual domain
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当视频推理需要外部知识时，许多基于大型多模态模型（LMM）的系统采用检索增强来补充缺失的上下文。然而，追加文本或多片段证据会把异构信号强行压入同一个注意力空间。我们观察到，即便在非长视频上也会出现注意力被稀释和认知负荷升高的问题。瓶颈不仅在于检索什么，还在于如何表示外部知识并将其与视频主干融合。我们提出图到帧 RAG（G2F-RAG），一种免训练且可审计的范式，在视觉空间中传递知识。在离线阶段，一个智能体构建与具体问题无关的视频知识图谱，整合实体、事件、空间关系以及关联的世界知识。在在线阶段，一个分层多智能体控制器判断是否需要外部知识，检索出最小充分子图，并将其渲染为单张推理帧追加到视频之后。随后 LMM 在统一的视觉域内进行联合推理。该设计降低了认知负荷，并留下显式、可检查的证据链。G2F-RAG 可即插即用地跨主干与规模使用。它在多个公开基准上取得一致增益，在知识密集型设定下提升更大。消融实验进一步证实知识表示与传递方式的重要性。G2F-RAG 将检索重新表述为视觉空间知识融合，以实现稳健且可解释的视频推理。
英文原文摘要 1467 字符
When video reasoning requires external knowledge, many systems with large multimodal models (LMMs) adopt retrieval augmentation to supply the missing context. Appending textual or multi-clip evidence, however, forces heterogeneous signals into a single attention space. We observe diluted attention and higher cognitive load even on non-long videos. The bottleneck is not only what to retrieve but how to represent and fuse external knowledge with the video backbone.We present **Graph-to-Frame RAG (G2F-RAG)**, a training free and auditable paradigm that delivers knowledge in the visual space. On the offline stage, an agent builds a problem-agnostic video knowledge graph that integrates entities, events, spatial relations, and linked world knowledge. On the online stage, a hierarchical multi-agent controller decides whether external knowledge is needed, retrieves a minimal sufficient subgraph, and renders it as a single reasoning frame appended to the video. LMMs then perform joint reasoning in a unified visual domain. This design reduces cognitive load and leaves an explicit, inspectable evidence trail.G2F-RAG is plug-and-play across backbones and scales. It yields consistent gains on diverse public benchmarks, with larger improvements in knowledge-intensive settings. Ablations further confirm that knowledge representation and delivery matter. G2F-RAG reframes retrieval as visual space knowledge fusion for robust and interpretable video reasoning.