Large-Scale Face Image Retrieval System at Attribute Level Based on Facial Attribute Ontology and Deep Neuron Network

Abstract

From the emerging of Deep Convolution Neural Network (DCNN), the Visual Information Retrieval would have good prospects for visual features automatically extracted at high semantic levels. However, the deep features could not be robust to some challenges as one-to-many and many-to-one relationships between face identifiers and facial attributes in querying on the face identifier level. To solve these issues at the large-scale level, we proposed a face retrieval system by using the “divide and conquer” method: query by attributes instead of querying by the identifier. We used Facial Attributes in the Fast-Filter stage, after that, our proposed system would retrieve the Face Identifier from the retrieved candidates. DCNN is very useful in the facial attribute learning because of the same network architecture for multiple-attribute groups. We built the attribute learning model following the bottom-up and top-down process. The bottom-up process uses DCNNs with the corresponding face parts and the top-down process is based on our proposed Facial Attribute Ontology (FAO). FAO supports multi-task learning in DCNN, re-usability for other retrieval tasks, flexibility in intelligent queries. We experimented our proposed method on the LFWA and CelebA dataset; our system achieved the average precision at 85.68%, this result is higher than some state-of-the-art methods. In more details, we also outperformed at 25 on 40 attribute detectors. Moreover, we speeded up the retrieval process based on the multi-attribute space and the indexing method named Hierarchical K-means++. At last, on retrieval experiments, we gathered 0.79 and 0.82 MAP-score average for one attribute query in LFWA and CelebA respectively.