渲染即适配:面向视线估计的无监督个性化适配
Render-to-Adapt: Unsupervised Personal Adaptation for Gaze Estimation
把「用渲染器合成图像、再以预测结果作为伪标签做自监督」这套闭环,用于单用户少样本无标签的模型校准;做视线/头部姿态等细粒度人脸任务的人,可以直接把 R2A 的渲染-一致性信号搬到自己的个性化适配上,替代容易在单用户上失效的 UDA。
原文依据 R2A utilizes a gaze-conditioned renderer to synthesize new images based on the model's gaze predictions, and enforces eye-region consistency as a label-free signal
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于深度学习的视线估计方法在用户与环境不断变化的真实场景中往往出现显著性能下降。为解决该问题,近期多数方法采用无监督域自适应(UDA)来弥合源域与目标域之间的差距。然而这一范式与真实场景并不一致:系统通常只需适配单个新用户。因此,本文提出一种更实用的范式——无监督个性化适配(UPA),即利用来自单个新用户的少量无标签图像对预训练模型进行校准。传统的 UDA 方法无法保证对每一位用户都带来提升,在此设定下其平均性能往往更低。针对该问题,本文提出 Render-to-Adapt(R2A),一个专为 UPA 任务设计的自监督框架。给定预训练的视线模型,R2A 利用以视线为条件的渲染器,根据模型的视线预测合成新图像,并以眼区一致性作为无标签信号加以约束,从而增强个性化视线估计。我们在重新设计的跨数据集个性化适配基准上评估 R2A。实验结果表明,R2A 在全部个体上均能稳定提升性能,并显著优于现有最先进方法。
英文原文摘要 1332 字符
Deep learning-based gaze estimation methods tend to suffer from substantial performance drop in real-world scenarios with varying users and environments. To tackle this issue, most recent approaches employ Unsupervised Domain Adaptation (UDA) to bridge the gap between source and target domains. However, this paradigm is misaligned with real-world scenarios, where the system typically needs to adapt to only a single new user. Therefore, this paper advocates a more practical paradigm: Unsupervised Personal Adaptation (UPA), which calibrates a pre-trained model using a few unlabeled images from a single new user. Conventional UDA methods do not guarantee improvements for every user and often yield lower average performance in this setting. To address this problem, we propose Render-to-Adapt (R2A), a self-supervised framework specifically designed for the UPA task. Given a pretrained gaze model, R2A utilizes a gaze-conditioned renderer to synthesize new images based on the model's gaze predictions, and enforces eye-region consistency as a label-free signal to enhance personalized gaze estimation. We evaluate R2A on a re-designed cross-dataset personal adaptation benchmark. Experimental results show that R2A consistently improves performance across all individuals and significantly outperforms existing SOTA methods.