ViHOI：借助视觉先验的人体-物体交互合成
ViHOI: Human-Object Interaction Synthesis with Visual Priors
把大视觉语言模型的多层特征经 Q-Former 适配器压缩成先验 token，作为扩散生成 3D 人体-物体交互运动的条件；视觉研究者可复用这套「VLM 特征→紧凑先验→条件生成」的跨模态条件注入范式，并用文生图合成参考图提升对未见物体的泛化。
原文依据 we design a Q-Former-based adapter that compresses the VLM's high-dimensional features into compact prior tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论生成真实且物理合理的 3D 人体-物体交互（HOI）仍是运动生成中的关键挑战。一个主要原因是仅用文字描述这些物理约束十分困难。为克服这一局限，我们提出一种新范式：从易于获取的 2D 图像中提取丰富的交互先验。具体而言，我们提出 ViHOI，一个新框架，使基于扩散的生成模型能够利用来自 2D 图像的丰富且任务特定的先验来提升生成质量。我们采用大型视觉语言模型（VLM）作为强大的先验提取引擎，并采用层解耦策略获取视觉先验与文本先验。同时，我们设计了一个基于 Q-Former 的适配器，将 VLM 的高维特征压缩为紧凑的先验 token，从而显著促进扩散模型的条件训练。我们的框架在由数据集渲染得到的运动图像上训练，以确保视觉输入与运动序列之间严格的语义对齐。在推理阶段，它利用由文本到图像生成模型合成的参考图像，以提升对未见物体和交互类别的泛化能力。实验结果表明，ViHOI 取得了最优性能，在多个基准上优于现有方法，并展现出更优的泛化能力。本工作的代码将发布于 https//github.com/MPI-Lab/ViHOI。
英文原文摘要 1474 字符
Generating realistic and physically plausible 3D Human-Object Interactions (HOI) remains a key challenge in motion generation. One primary reason is that describing these physical constraints with words alone is difficult. To address this limitation, we propose a new paradigm: extracting rich interaction priors from easily accessible 2D images. Specifically, we introduce ViHOI, a novel framework that enables diffusion-based generative models to leverage rich, task-specific priors from 2D images to enhance generation quality. We utilize a large Vision-Language Model (VLM) as a powerful prior-extraction engine and adopt a layer-decoupled strategy to obtain visual and textual priors. Concurrently, we design a Q-Former-based adapter that compresses the VLM's high-dimensional features into compact prior tokens, which significantly facilitates the conditional training of our diffusion model. Our framework is trained on motion-rendered images from the dataset to ensure strict semantic alignment between visual inputs and motion sequences. During inference, it leverages reference images synthesized by a text-to-image generation model to improve generalization to unseen objects and interaction categories. Experimental results demonstrate that ViHOI achieves state-of-the-art performance, outperforming existing methods across multiple benchmarks and demonstrating superior generalization. The code for this work will be released at https//github.com/MPI-Lab/ViHOI.