图像编辑与可控生成 · CVPR2026 ·2026.06.03

PhotoFramer:多模态图像构图指令

PhotoFramer: Multi-modal Image Composition Instruction

Zhiyuan You、Ke Wang、He Zhang、Xin Cai、Jinjin Gu、Tianfan Xue 等 8 人

与你有关的部分

把「构图指导」建模为文本指令加示例图的双输出任务,并给出用多视角数据加退化模型合成「构图差—构图好」配对的可复用数据构造流程,可直接迁移到其他需要成对编辑监督的图像生成任务。

原文依据 we finetune a model that jointly processes and generates both text and images
可迁移到 instruction-based image editingimage compositiontext-to-image generationimage cropping

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

构图在拍照过程中至关重要,然而许多普通用户难以拍出构图良好的照片。为提供构图指导,我们提出 PhotoFramer,一个多模态构图指令框架。给定一张构图不佳的图像,PhotoFramer 首先用自然语言描述如何改进构图,然后生成一张构图良好的示例图像。为训练这样一个模型,我们构建了一个大规模数据集。受人类拍照方式的启发,我们将构图指导组织为层次化的子任务:平移、放大和视角变换任务。平移与放大数据从现有裁剪数据集中采样,而视角变换数据则通过两阶段流程获得。首先,我们从多视角数据集中采样具有不同视角的图像对,并训练一个退化模型,将构图良好的照片转换为构图不佳的照片。其次,我们将该退化模型应用于专家拍摄的照片,合成构图不佳的图像以构成训练对。利用该数据集,我们微调了一个能够联合处理并生成文本与图像的模型,从而实现带有示例说明的可操作文本指导。大量实验表明,文本指令能够有效引导图像构图,且将其与示例结合相比仅使用示例的基线可带来一致的提升。PhotoFramer 为构图助手迈出了实用的一步,使专家摄影先验能够为普通用户所用。

英文原文摘要 1472 字符

Composition matters during the photo-taking process, yet many casual users struggle to frame well-composed images. To provide composition guidance, we introduce PhotoFramer, a multi-modal composition instruction framework. Given a poorly composed image, PhotoFramer first describes how to improve the composition in natural language and then generates a well-composed example image. To train such a model, we curate a large-scale dataset. Inspired by how humans take photos, we organize composition guidance into a hierarchy of sub-tasks: shift, zoom-in, and view-change tasks. Shift and zoom-in data are sampled from existing cropping datasets, while view-change data are obtained via a two-stage pipeline. First, we sample pairs with varying viewpoints from multi-view datasets, and train a degradation model to transform well-composed photos into poorly composed ones. Second, we apply this degradation model to expert-taken photos to synthesize poor images to form training pairs. Using this dataset, we finetune a model that jointly processes and generates both text and images, enabling actionable textual guidance with illustrative examples. Extensive experiments demonstrate that textual instructions effectively steer image composition, and coupling them with exemplars yields consistent improvements over exemplar-only baselines. PhotoFramer offers a practical step toward composition assistants that make expert photographic priors accessible to everyday users.

标签

会议 CVPR2026
页码 10197-10207
Bibkey
来源 CVF Open Access