META:面向长视频理解的工具轨迹自适应元进化
META: Meta Evolution of Tool Trajectory Adaptation for Long-Video Understanding
对做长视频理解的视觉研究者,这篇提供了免训练、模型无关的智能体框架:把重复出现的多步工具调用轨迹抽象为可复用的宏工具,并把失败轨迹蒸馏成结构化失败先验来约束后续工具选择,从而在不更新参数的情况下缩短推理链、提升时序感知能力,可直接套用到现有 VLM 的长视频问答流程上。
原文依据 abstracts recurring multi-step patterns into reusable macro-tools, and distills failed executions into structured failure priors
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论长视频理解仍然面临挑战,原因在于极端的时序冗余、稀疏却具有决定性的事件,以及视觉语言模型(VLM)在长程推理上的不稳定性。现有的基于智能体的方法会调用外部微工具,但这些方法保持静态,对每个任务都要反复重建细粒度操作的长链,无法获得可复用的多步感知技能。我们提出 META,这是首个能够自我进化其工具增强推理的免训练智能体。META 通过求解与进化两个循环运作:它分析自身的工具轨迹,将反复出现的多步模式抽象为可复用的宏工具,并把失败的执行蒸馏为结构化的失败先验,用以改进工具的使用方式。通过符号化的整合与剪枝,META 逐步缩短推理路径,并获得更通用的感知与时序能力——且无需任何参数更新。META 在长视频基准上取得了最优性能,展示了一种可扩展、模型无关的长视频理解范式,无需额外训练即可持续进化。
英文原文摘要 1129 字符
Long-video understanding remains challenging due to extreme temporal redundancy, sparse yet decisive events, and the instability of long-horizon reasoning in visual-language models (VLMs). Existing agent-based methods invoke external micro-tools but remain static, repeatedly rebuilding long chains of fine-grained operations for each task without acquiring reusable multi-step perceptual skills.We propose META, the first training-free agent capable of self-evolving its tool-augmented reasoning. META operates through dual Solving and Evolving loops: it analyzes its own tool trajectories, abstracts recurring multi-step patterns into reusable macro-tools, and distills failed executions into structured failure priors that refine tool usage. Through symbolic consolidation and pruning, META progressively shortens reasoning paths and acquires more general perceptual and temporal abilities--without any parameter updates. META achieves state-of-the-art performance on long-video benchmarks, demonstrating a scalable, model-agnostic paradigm for long-video understanding that can continually evolve without additional training.