Comparative study of localization metrics for the evaluation of image interpretation systems

Baptiste Hemery; Helene Laurent; Bruno Emile; Christophe Rosenberger

doi:10.1117/1.3446803

1 April 2010 Comparative study of localization metrics for the evaluation of image interpretation systems

Baptiste Hemery, Helene Laurent, Bruno Emile, Christophe Rosenberger

Author Affiliations +

Journal of Electronic Imaging, Vol. 19, Issue 2, 023017 (April 2010). https://doi.org/10.1117/1.3446803

Abstract

Image interpretation is particularly important in many real applications (video monitoring, biometrics, etc.). Due to the proliferation of image interpretation systems in the literature, their evaluation still remains a crucial stake. Among all the tasks in this field, the quality of object localization is often evaluated through an evaluation metric. We propose to review these techniques and study their reliability. We first propose a generic definition of a localization algorithm. Then, different state of the art techniques to evaluate image interpretation results are detailed. Secondly, we focus on metrics that enable us to evaluate localization results. We propose a general methodology to analyze the behavior of an evaluation metric, considered here as a black box (its definition is not even supposed to be known). We define the properties that these metrics should fulfill. We then perform a comparative study of 33 localization metrics from the state of the art. Experimental results conducted on a large and significant image database permit us to determine metrics that should be used in the future for the evaluation of object localization results.

©(2010) Society of Photo-Optical Instrumentation Engineers (SPIE)

Citation Download Citation

Baptiste Hemery, Helene Laurent, Bruno Emile, and Christophe Rosenberger "Comparative study of localization metrics for the evaluation of image interpretation systems," Journal of Electronic Imaging 19(2), 023017 (1 April 2010). https://doi.org/10.1117/1.3446803

Published: 1 April 2010

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available