人脸验证的“金发姑娘”测试集
Goldilocks Test Sets for Face Verification
该论文提出的三个测试集（Hadrian、Eclipse、ND-Twins）和“Goldilocks”测试集构建规则，为评估人脸识别模型在面部属性变化和相似身份上的鲁棒性提供了新基准，可直接用于诊断现有模型在真实场景下的弱点。
原文依据 We propose three challenging test sets to reveal important but ignored weaknesses of the existing FR algorithms.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前知名测试集上报告的人脸验证准确率已趋于饱和。因此，一些困难的测试集通过降低图像质量或向图像中添加伪影来构建。然而，我们认为测试集可以在不人为降低图像质量的情况下具有挑战性，因为人脸识别（FR）模型在正确识别以下情况时存在困难：1）具有较大人脸属性差异的同一身份对（即真实对），2）具有较小人脸属性差异的不同身份对（即冒名对），以及3）相似外观身份对（例如双胞胎和亲属）。我们提出了三个具有挑战性的测试集，以揭示现有FR算法重要但被忽视的弱点。为了在面部属性变化上挑战模型，我们提出了Hadrian和Eclipse，分别针对面部毛发差异和面部曝光差异。两个测试集中的图像均为高质量且在受控环境中采集。为了在相似外观人物上挑战FR模型，我们提出了ND-Twins，其包含来自专用双胞胎数据集的图像。LFW测试协议被用于构建所提出的测试集。此外，我们引入了额外规则来组装“Goldilocks1”级别的测试集，包括：1）限制困难样本的出现次数，2）在人口统计群体间进行等概率评估，3）约束验证折之间的身份重叠。定量上，在不进一步处理图像的情况下，所提出的测试集具有与现有添加伪影的测试集相当或更高的难度。数据集可在以下网址获取：https://github.com/HaiyuWu/SOTA-Face-Recognition-Train-and-Test。
英文原文摘要 1741 字符
Reported face verification accuracy has reached a plateau on current well-known test sets. As a result, some difficult test sets have been assembled by reducing the image quality or adding artifacts to the image. However, we argue that test sets can be challenging without artificially reducing the image quality because the face recognition (FR) models suffer from correctly recognizing 1) the pairs from the same identity (i.e., genuine pairs) with a large face attribute difference, 2) the pairs from different identities (i.e., impostor pairs) with a small face attribute difference, and 3) the pairs of similar-looking identities (e.g., twins and relatives). We propose three challenging test sets to reveal important but ignored weaknesses of the existing FR algorithms. To challenge models on variation of facial attributes, we propose Hadrian and Eclipse to address facial hair differences and face exposure differences. The images in both test sets are high-quality and collected in a controlled environment. To challenge FR models on similar-looking persons, we propose ND-Twins, which contains images from a dedicated twins dataset. The LFW test protocol is used to structure the proposed test sets. Moreover, we introduce additional rules to assemble "Goldilocks1" level test sets, including 1) restricted number of occurrence of hard samples, 2) equal chance evaluation across demographic groups, and 3) constrained identity overlap across validation folds. Quantitatively, without further processing the images, the proposed test sets have on-par or higher difficulties than the existing test sets that add artifacts to the images. The datasets are available at: https://github.com/HaiyuWu/SOTA-Face-Recognition-Train-and-Test.