doiSerbia

Comparative evaluation of keypoint detectors for 3d digital avatar reconstruction

Gajić Dušan (University of Novi Sad, Faculty of Technical Sciences, Novi Sad, Serbia)
Gojić Gorana (University of Novi Sad, Faculty of Technical Sciences, Novi Sad, Serbia)
Dragan Dinu (University of Novi Sad, Faculty of Technical Sciences, Novi Sad, Serbia)
Petrović Veljko (University of Novi Sad, Faculty of Technical Sciences, Novi Sad, Serbia)

Three-dimensional personalized human avatars have been successfully utilized in shopping, entertainment, education, and health applications. However, it is still a challenging task to obtain both a complete and highly detailed avatar automatically. One approach is to use general-purpose, photogrammetry-based algorithms on a series of overlapping images of the person. We argue that the quality of avatar reconstruction can be increased by modifying parts of the photogrammetry-based algorithm pipeline to be more specifically tailored to the human body shape. In this context, we perform an extensive, standalone evaluation of eleven algorithms for keypoint detection, which is the first phase of the photogrammetry-based reconstruction pipeline. We include well established, patented Distinctive image features from scale-invariant keypoints (SIFT) and Speeded up robust features (SURF) detection algorithms as a baseline since they are widely incorporated into photogrammetry-based software. All experiments are conducted on a dataset of 378 images of human body captured in a controlled, multi-view stereo setup. Our findings are that binary detectors highly outperform commonly used SIFT-like detectors in the avatar reconstruction task, both in terms of detection speed and in number of detected keypoints.

Keywords: Detector, Photogrammetry-based reconstruction, 3D human avatar, Structure from Motion, Multi-view Stereo

Projects of the Serbian Ministry of Education, Science and Technological Development, Grant no. TR32044 (2011-2020), Grant no. ON174026 (2011-2020), and Grant no. III44006 (2011-2020