TY  - CONF
AU  - Tomas Sixta
AU  - Julio C. S. Jacques Junior
AU  - Pau Buch Cardona
AU  - Eduard Vazquez
AU  - Sergio Escalera
A2  - ECCVW
PY  - 2020//
TI  - FairFace Challenge at ECCV 2020: Analyzing Bias in Face Recognition
T2  - LNCS
BT  - ECCV Workshops
SP  - 463
EP  - 481
VL  - 12540
N2  - This work summarizes the 2020 ChaLearn Looking at People Fair Face Recognition and Analysis Challenge and provides a description of the top-winning solutions and analysis of the results. The aim of the challenge was to evaluate accuracy and bias in gender and skin colour of submitted algorithms on the task of 1:1 face verification in the presence of other confounding attributes. Participants were evaluated using an in-the-wild dataset based on reannotated IJB-C, further enriched 12.5K new images and additional labels. The dataset is not balanced, which simulates a real world scenario where AI-based models supposed to present fair outcomes are trained and evaluated on imbalanced data. The challenge attracted 151 participants, who made more 1.8K submissions in total. The final phase of the challenge attracted 36 active teams out of which 10 exceeded 0.999 AUC-ROC while achieving very low scores in the proposed bias metrics. Common strategies by the participants were face pre-processing, homogenization of data distributions, the use of bias aware loss functions and ensemble models. The analysis of top-10 teams shows higher false positive rates (and lower false negative rates) for females with dark skin tone as well as the potential of eyeglasses and young age to increase the false positive rates too.
UR  - https://link.springer.com/chapter/10.1007/978-3-030-65414-6_32
L1  - http://refbase.cvc.uab.es/files/SJB2020.pdf
UR  - http://dx.doi.org/10.1007/978-3-030-65414-6_32
N1  - HUPBA
ID  - Tomas Sixta2020
ER  -