和谐画布：面向视觉引导音乐风格迁移的无反转编辑
Harmonic Canvas: Inversion-Free Editing for Visually-Guided Music Style Transfer
把基于 flow 的无反转编辑框架从图像迁移到音频，用 CLIP+ViT 双编码器融合视觉嵌入作为生成条件，并用可微归一化色度约束在生成流上保持内容一致性——这套「视觉条件 + 无反转流编辑 + 内容保持约束」的组合可直接复用到图像/视频的风格化编辑与视觉引导生成。
原文依据 We introduce a flow-based, inversion-free framework for multimodal music style transfer that unifies textual and visual guidance.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论音乐风格迁移（MST）旨在以新的风格形式重新诠释已有音乐作品，同时保持其旋律的连贯性。以文本或音频为条件的传统方法忽视了音乐风格深刻的多模态特性。视觉氛围——体现在色彩、光照与构图上——所编码的情感属性与音色、节奏和和声相呼应，然而在音乐风格迁移的语境中尚未得到充分探索。我们提出一个基于流（flow）的无反转框架，用于统一文本与视觉引导的多模态音乐风格迁移。我们的方法应对两个挑战：（1）通过一个融合 CLIP 与 ViT 嵌入的双编码器融合模块，捕捉超越语言的跨模态语义；（2）使用一个可微的归一化色度约束，沿生成流调控音级一致性，从而保持旋律身份。我们将 MeLBench 与 MusicCaps 集合重新组织并扩展为一个按流派结构化的多模态数据集，以支持风格感知的分析。定量与感知评估表明，我们的方法在控制力、结构保真度和跨模态表现力方面均取得更优表现，凸显了视觉感知在音乐生成中的作用。我们的演示已在线提供。
英文原文摘要 1285 字符
Music style transfer (MST) aims to reinterpret existing musical pieces in new stylistic forms while maintaining their melodic coherence. Conventional approaches conditioned on text or audio overlook the profoundly multimodal character of musical style. Visual ambience -- reflected in color, lighting, and composition -- encodes affective attributes that parallel timbre, rhythm, and harmony, which, however, remain underexplored in MST context. We introduce a flow-based, inversion-free framework for multimodal music style transfer that unifies textual and visual guidance. Our approach tackles two challenges: (1) capturing cross-modal semantics beyond language through a dual-encoder fusion module that merges CLIP- and ViT-derived embeddings, and (2) preserving melodic identity using a differentiable normalized chroma constraint that regulates pitch-class consistency along the generative flow. We reorganize and extend the MeLBench and MusicCaps collections into a genre-structured multimodal dataset to support style-aware analysis. Quantitative and perceptual evaluations demonstrate that our approach achieves superior control, structural fidelity, and cross-modal expressiveness, underscoring the role of visual perception in music generation. Our demo is available online.