Stitch-a-Demo：从多步描述构建视频演示
Stitch-a-Demo: Creating Video Demonstrations from Multistep Descriptions
把「多步文本描述→连贯视频演示」建模为跨片段检索与拼接问题，其弱监督数据构造与难负样本设计可直接迁移到视频检索、程序性视频理解与图文对齐任务中。
原文依据 a novel retrieval-based method to assemble a video demonstration from a multistep description
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在从文本描述获取视觉图示时，现有方法通常取一段单一文本上下文——一句说明文字或一段动作描述——并检索或生成与之匹配的视觉上下文。然而，先前工作无法对多步描述（例如一份烹饪食谱或一份园艺操作手册）进行视觉图示，而孤立地处理每一步描述又会导致演示不连贯。我们提出 Stitch-a-Demo，一种新颖的基于检索的方法，用于从多步描述组装出一段视频演示。所得视频包含的片段（可能来自不同来源）既准确反映所有步骤描述，又在视觉上保持连贯。我们设计了一套训练流程，构造包含多样流程的大规模弱监督数据，并注入难负样本，以同时提升正确性与连贯性。在真实场景的教学视频上验证，Stitch-a-Demo 取得了最优性能，提升幅度最高达 29%，并在人类偏好研究中取得显著优势。
英文原文摘要 1083 字符
When obtaining visual illustrations from text descriptions, today's methods take a description with a single text context--a caption, or an action description--and retrieve or generate the matching visual context. However, prior work does not permit visual illustration of multistep descriptions, e.g. a cooking recipe or a gardening instruction manual, and simply handling each step description in isolation would result in an incoherent demonstration. We propose Stitch-a-Demo, a novel retrieval-based method to assemble a video demonstration from a multistep description. The resulting video contains clips, possibly from different sources, that accurately reflect all the step descriptions, while being visually coherent. We formulate a training pipeline that creates large-scale weakly supervised data containing diverse procedures and injects hard negatives that promote both correctness and coherence. Validated on in-the-wild instructional videos, Stitch-a-Demo achieves state-of-the-art performance, with gains up to 29% as well as dramatic wins in a human preference study.