AutoCut：基于多模态离散化与可控生成的端到端广告视频编辑
AutoCut: End-to-end advertisement video editing based on multimodal discretization and controllable generation
把视频、音频用残差向量量化离散成与文本对齐的统一词元，再用多模态大语言模型自回归预测词元序列来驱动视频剪辑与生成，为视觉研究者提供了一条「离散词元 + 语言模型」做长视频可控编辑的可复用路线。
原文依据 applies residual vector quantization to discretize them into unified tokens aligned with textual representations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论短视频已成为数字广告的主要媒介，需要可扩展且高效的内容创作方式。然而，当前的工作流与 AI 工具彼此割裂且局限于单一模态，导致制作成本高、整体效率低。为解决这一问题，我们提出 AutoCut，一个基于多模态离散化与可控编辑的端到端广告视频编辑框架。AutoCut 采用专用编码器提取视频与音频特征，随后通过残差向量量化将其离散为与文本表示对齐的统一词元，从而构建共享的视频-音频-文本词元空间。在基础模型之上，我们通过多模态对齐与监督微调相结合，进一步开发了用于视频编辑的多模态大语言模型，在统一的编辑框架内支持涵盖视频选择与排序、脚本生成以及背景音乐选择的任务。最后，一套完整的制作流水线将预测的词元序列转换为可部署的长视频输出。在真实广告数据集上的实验表明，AutoCut 在降低制作成本与迭代时间的同时，显著提升了一致性与可控性，为可扩展的视频创作开辟了道路。代码与数据见 https://github.com/AdAutoCut/Autocut
英文原文摘要 1371 字符
Short-form videos have become a primary medium for digital advertising, requiring scalable and efficient content creation. However, current workflows and AI tools remain disjoint and modality-specific, leading to high production costs and low overall efficiency. To address this issue, we propose AutoCut, an end-to-end advertisement video editing framework based on multimodal discretization and controllable editing. AutoCut employs dedicated encoders to extract video and audio features, then applies residual vector quantization to discretize them into unified tokens aligned with textual representations, constructing a shared video-audio-text token space. Built upon a foundation model, we further develop a multimodal large language model for video editing through combined multimodal alignment and supervised fine-tuning, supporting tasks covering video selection and ordering, script generation, and background music selection within a unified editing framework. Finally, a complete production pipeline converts the predicted token sequences into deployable long video outputs. Experiments on real-world advertisement datasets show that AutoCut reduces production cost and iteration time while substantially improving consistency and controllability, paving the way for scalable video creation. Code and data are available at https://github.com/AdAutoCut/Autocut