Elsevier

Information Fusion

Volume 33, January 2017, Pages 71-85
Information Fusion

Overview of the combination of biometric matchers

https://doi.org/10.1016/j.inffus.2016.05.003Get rights and content

Highlights

  • The paper presents the methodologies on information fusion in the biometric field.

  • The methodologies, architectures, and benchmarks related to unimodal and multimodal fusion of biometric systems are discussed.

  • The state of the art in the combination of biometric matchers is provided.

  • A case study for the experimental evaluation of methods for biometric fusion at score level is presented.

  • Some possible directions for future research are suggested.

Abstract

Biometric identity verification refers to technologies used to measure human physical or behavioral characteristics, which offer a radical alternative to passports, ID cards, driving licenses or PIN numbers in authentication. Since biometric systems present several limitations in terms of accuracy, universality, distinctiveness, acceptability, methods for combining biometric matchers have attracted increasing attention of researchers with the aim of improving the ability of systems to handle poor quality and incomplete data, achieving scalability to manage huge databases of users, ensuring interoperability, and protecting user privacy against attacks. The combination of biometric systems, also known as “biometric fusion”, can be classified into unimodal biometric if it is based on a single biometric trait and multimodal biometric if it uses several biometric traits for person authentication.

The main goal of this study is to analyze different techniques of information fusion applied in the biometric field. This paper overviews several systems and architectures related to the combination of biometric systems, both unimodal and multimodal, classifying them according to a given taxonomy. Moreover, we deal with the problem of biometric system evaluation, discussing both performance indicators and existing benchmarks.

As a case study about the combination of biometric matchers, we present an experimental comparison of many different approaches of fusion of matchers at score level, carried out on three very different benchmark databases of scores. Our experiments show that the most valuable performance is obtained by mixed approaches, based on the fusion of scores. The source code of all the method implemented for this research is freely available for future comparisons1.

After a detailed analysis of pros and cons of several existing approaches for the combination of biometric matchers and after an experimental evaluation of some of them, we draw our conclusion and suggest some future directions of research, hoping that this work could be a useful start point for newer research.

Introduction

Biometrics refers to technologies used to measure human physical or behavioral characteristics such as iris, face, fingerprints, retina, hand geometry, voice or signatures and using such measures to detect and recognize individuals. Biometric identity verification offers a radical alternative to passports, ID cards, driving licenses or PIN numbers in authentication. Since biometrics authentication uses unique physical traits, the user is not required to carry any additional ID document. Moreover, unlike most traditional authorization systems such as personal identification numbers (PINs), passwords, or ID card, biometric credentials cannot be lost, forgotten, guessed, or easily cloned. Most common biometric systems include an enrollment and an identification/verification phase. Enrollment consists in the acquisition by a scanner of a "live sample" of the biometric of the person to be identified, followed by processing and storing as a template. Verification involves matching a captured biometric sample against the enrolled template that is stored in order to identify/verify user identity [1].

Since the first elementary fingerprint recognition system was proposed in early 20 century, the research community has spent energy to find out new biometric modalities, that is any physical or behavioral characteristic which satisfies the conditions of universality, discriminative amongst the population, invariance against time, easily collectible and difficult to reproduce/cheat. Based on the above criteria, several distinctive traits have been identified [2]: physiological (e.g. fingerprint, face, iris), behavioral (e.g. signature, gait, voice), medico-chemical (e.g. DNA, ECG) and soft (e.g. height, gender, ethnicity).

Biometric Identification is a One-to-Many matching of the captured biometric sample against all stored templates in order to determine a person's identity even without his/her knowledge or consent. For example, using a latent fingerprint to identify a criminal or scanning a crowd with a camera and using face recognition technology to find someone. In identification the user's biometric input is compared with the templates of all the persons enrolled in the database and the system outputs either the identity of the person whose template has the highest degree of similarity with the user's input or a rejection decision indicating that the user is not present in the DB. An extension to identification is screening, where the biometric system is called to guarantee that a particular individual does not belong to a watch list of identities.

Biometric Verification is a One-to-One matching of the captured biometric sample against the template of the person he/she claims to be, the identity claim is accepted as “genuine” if the degree of similarity is sufficiently high, as “impostor” otherwise. For example, fingerprint or retinal scans can be used to grant access to restricted areas or a bank account [1]. Many biometric applications (i.e. the FBI-IAFIS and US-VISIT IDENT program) work in the identification mode, and since the number of enrolled users can be very huge identification is significantly more challenging than verification.

Different biometric systems share a common general flow (Fig. 1), which is composed by four main components:

  • Acquisition module: The first component of a biometric system is acquisition of the biometric data of an individual from a biometric sensor hardware. For face and iris images, the sensor is typically a camera, for fingerprints, the sensor is typically a scanner, for voice data, the sensor is a microphone. The quality of the acquisition module has a significant impact on the performance of the system which is sensitive to the environmental conditions (i.e. changes in brightness of an image), quality of sensor (i.e. dpi of the image), human factor (i.e. pose variations).

  • Feature extraction module: The acquired data is pre-processed to remove noise or other abnormalities present and then subjected to the feature extraction process in order to extract biometrical values that ideally must describe uniquely an individual, so that biometric data collected from one individual, at different times, are “similar”, while those collected from different individuals are “dissimilar”. For example, the position and orientation of minutiae points in a fingerprint image are used in a fingerprint system. The features extracted during enrollment are stored in a template, which is a possibly small and easy to process. In order to improve interoperability among different biometric systems there exist proposals of standard format of templates, i.e. for fingerprint they are based only on minutiae points.

  • Matching module: In this module, which is not used during enrollment, the feature values from an unknown individual are compared against those in the stored template by generating a matching score indicating the degree of similarity between a pair of biometrics data. The score should be high for features from the same individuals and low for those from different ones. For example in a fingerprint system, the number of matching minutiae points between the query and the template can be returned as a matching score. Usually matching is a difficult pattern-recognition problem due to large intra-class variations (caused by bad acquisition, noise, different environmental condition, distortions, etc.) and large inter-class similarity (i.e. differencing identical twins is very difficult in face recognition).

  • Decision component: In this module the user's identity is established (identification) or a claimed identity is accepted/rejected (verification) based on the matching score. Usually the final decision is taken by comparing the matching score to a fixed threshold, which is selected according to consideration about the degree on security required by the application.

Unfortunately biometric systems also presents several limitations which in some cases make the performance of one single biometric modality insufficient for the related application in terms of accuracy, universality, distinctiveness, acceptability. Main limitations of biometric systems [3] are related to (i) variable environmental conditions (i.e. noise, changes in illumination, pose) which may heavily affect the accuracy of the system, in particular when acquisition is not performed in constrained conditions, (ii) large intra-class variations caused by acquisition in different conditions or aging effects, (iii) non-universality of some biometric credential due to illness or disabilities, (iv) spoof attacks that are performed by falsifying a biometric trait and then presenting this falsified information to the biometric system.

In order to overcome such limitations, methods for combining biometric matchers have attracted increasing attention of researchers [4] with the aim of improving the ability of systems to handle poor quality and incomplete data, achieve scalability to manage huge databases of users, ensure interoperability and protect user privacy against attacks. The combination of biometric systems, also known as “biometric fusion”, can be classified into two groups [5]: unimodal biometric systems perform person recognition based on a single source of biometric information which is processed using different approaches, multimodal biometric systems acquire and use several biometric traits for person authentication. Some samples of possible sources of information, in a unimodal and a multimodal system, are depicted in Fig. 2.

Unimodal systems include [6] multi-sample techniques which fuse information from the same input data, multi-instance and multi-sensor which perform multiple acquisition based on the same or different sensor and multi-algorithm which uses different approaches (e.g. a minutiae based and an image based approach for fingerprint recognition). For example, unimodal fusion can be effective to match face images obtained before and after plastic surgery: in [7] a fusion approach is proposed that combines information from the face and ocular regions to enhance recognition performance in presence of plastic surgery.

Multimodality has several advantages over unimodality since it is able to overcome the limitations of unimodal biometric systems due to non-universality of some traits, it can achieve higher recognition accuracy, it is less sensitive to variable environmental conditions, and it is harder to forge. Multimodal systems can reduce the Failure-to-Enroll Rate (FTER) and Failure-to-Capture Rate (FTCR) and ensure a larger sufficient population coverage: for instance, it is estimated that 2% of the population may not be able to provide a fingerprint due to medical/genetic/accidental/temporary conditions [8]. Moreover, a multimodal system may have a higher fault tolerance than a unimodal one, since it can be able to operate even in absence some biometric sources become unreliable due to sensor or software malfunction, or to a deliberate user manipulation. Multimodal systems are also more resistant to spoof attacks, because it is more difficult for the attacker to simultaneously spoof multiple biometric sources. However, multimodality has also some drawbacks related to additional operational costs, enrollment times and security risks [9]. The protection of multi-biometric templates is particularly critical, since they contain information regarding multiple traits of the same subject which can be used for a reconstruction of original biometric traits (e.g. fingerprints [10]). Another drawback of multimodal systems is that often users feel inconvenience from the need of several acquisition steps, therefore particularly interesting are those methods which consider traits close together, which can be acquired using a single input device: i.e. ocular biometric traits such as iris, periocular, retina, and eye movement [11], finger traits such as finger veins and fingerprints [12], [13].

The main goal of this study is to analyze different techniques of information fusion applied in the biometric field. The remainder of the paper is organized as follows: in Section 2 the advantages of combining classifiers is discussed and a taxonomy for the biometric system fusion is presented, Section 2 also includes an overview of existing studies of biometric fusion at different levels, including both unimodal and multimodal biometric systems; Section 3 presents existing performance indicators used to compare biometric systems and Section 4 introduces the problem of system evaluation by discussing the need of real multimodal datasets and suggesting a list of existing benchmarks; Section 5 presents a case study for the evaluation of score based biometric fusion techniques, detailing some approaches experimentally evaluated in Section 6; finally Section 7 concludes the paper with some suggestions for further investigations.

Section snippets

Biometric system fusion

In the last years, there have been a number of proposals on how to combine different classifiers for improving the performance of a stand-alone method. Combined approaches, also called Multiple Classifier Systems (MCS) or classifier ensembles [14] are usually based on combining the outputs of elementary classifiers for a given classification problem. In several pattern recognition fields, including biometric systems, many advantages of MCS vs. stand-alone methods have been pointed out [15] : (i)

Performance evaluation

The performance of a biometric system is influenced by environmental factors at the acquisition (i.e. temperature, humidity, illumination conditions) and performance factors (i.e. quality of the sensor, composition of target user population, and robustness of recognition). In order to quantify the performance of a biometric system, its accuracy is measured in terms of (i) sample acquisition error and (ii) recognition performance [2].

Acquisition errors measure environmental conditions

Benchmark databases

The availability of benchmark databases is crucial to evaluate new biometric systems. Collect biometric data is an onerous task, mainly when multimodal databases are required, due to the extra effort needed for the acquisition of multiple biometric traits. For this reason, in the literature, most of first multimodal databases were synthetically generated by defining a “chimeric user” obtained coupling different biometric traits not acquired from the same person. Some researchers [57] observed a

Score level fusion: a case study

In order to discuss and compare some of the several approaches proposed in the literature for the fusion of biometric systems at the score level, we propose a case study based on three biometric score benchmarks available in the literature.

The aim of this work is to find out the best fusion strategy in terms of recognition performance and to evaluate the improvement obtained with respect to a non-combined system. Although feature level fusion can exploit a richer source of information than

Experiments

Since the case study here presented is related to score level fusion the experiments have been carried out on three databases of scores, two multimodal datasets and a unimodal fingerprint database:

  • The NIST BSSR1 (BSSR) is a multimodal biometric database of scores freely available from the NIST20. It includes a set of raw output similarity scores from two face recognition systems and one fingerprint system. The release includes true multimodal

Conclusions

Nowadays there are a growing number of applications that need authentication and identification processes: the defense requires methods to determine with certainty an individual's true identity, homeland security requires technologies to secure the borders, airports and ferries, financial transactions need strong authentication processes and fraud prevent solutions, enterprise solutions require the oversight of people, technologies and processes.

Unfortunately, there is no security framework

References (90)

  • G.L. Marcialis et al.

    Personal identity verification by serial fusion of fingerprint and face matchers

    Pattern Recognit.

    (2009)
  • L. Nanni et al.

    Descriptors for image-based fingerprint matchers

    Expert Syst. Appl.

    (2009)
  • N. Poh et al.

    A user-specific and selective multimodal biometric fusion strategy by ranking subjects

    Pattern Recognit.

    (2013)
  • L. Nanni et al.

    Combining biometric matchers by means of machine learning and statistical approaches

    Neurocomputing

    (2015)
  • S. Bharadwaj et al.

    QFuse: online learning framework for adaptive biometric system

    Pattern Recognit.

    (2015)
  • J. Fierrez-Aguilar et al.

    Discriminative multimodal biometric authentication based on quality measures

    Pattern Recognit.

    (2005)
  • R. Cappelli et al.

    Fingerprint verification competition 2006

    Biometric Technol. Today.

    (2007)
  • HoC.C. et al.

    MMU GASPFA: a COTS multimodal biometric database

    Pattern Recognit. Lett.

    (2013)
  • M. Hanmandlu et al.

    Score level fusion of multimodal biometrics using triangular norms

    Pattern Recognit. Lett.

    (2011)
  • J. Peng et al.

    Multimodal biometric authentication based on score level fusion of finger biometrics

    Opt. - Int. J. Light Electron Opt.

    (2014)
  • L.I. Kuncheva et al.

    Decision templates for multiple classifier fusion: an experimental comparison

    Pattern Recognit.

    (2001)
  • L. Nanni et al.

    Likelihood ratio based features for a trained biometric score fusion

    Expert Syst. Appl.

    (2011)
  • N. Poh et al.

    A multimodal biometric test bed for quality-dependent, cost-sensitive and client-specific score-level fusion algorithms

    Pattern Recognit.

    (2010)
  • P. Wild et al.

    Robust multimodal face and fingerprint fusion in the presence of spoofing attacks

    Pattern Recognit

    (2016)
  • QiX. et al.

    LOAD: local orientation adaptive descriptor for texture and material classification

    Neurocomputing

    (2016)
  • ZhangD.D.

    Automated Biometrics: Technologies and Systems

    (2013)
  • A. Ross et al.

    Multimodal biometrics : an overview

    (2004)
  • A.A. Ross et al.

    Handbook of Multibiometrics

    (2006)
  • K. Nandakumar et al.

    Likelihood ratio-based biometric score fusion

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2008)
  • R. Jillela et al.

    Mitigating effects of plastic surgery: fusing face and ocular biometrics

  • D. Maio et al.

    Fvc2004: third fingerprint verification competition

    Biometric Authentication

    (2004)
  • C. Rathgeb et al.

    Multi-Biometric Template Protection: Issues and Challenges

  • R. Cappelli et al.

    Fingerprint image reconstruction from standard templates

    Analysis

    (2007)
  • Y.H. Park et al.

    A multimodal biometric recognition of touched fingerprint and finger-vein

  • L.I. Kuncheva

    Combining pattern classifiers: methods and algorithms (Kuncheva, L.I.; 2004) [book review]

    IEEE Trans. Neural Netw.

    (2007)
  • N. Mohamad et al.

    Investigation of information fusion in face and palmprint multimodal biometrics

  • L. Nanni et al.

    Classifier Ensemble Methods

    (2015)
  • L. Nanni et al.

    Using ensemble of classifiers in Bioinformatics

  • A. Ross et al.

    Feature level fusion using hand and face biometrics

  • YangF.

    Development of a fast panoramic face mosaicking and recognition system

    Opt. Eng.

    (2005)
  • ZhangY. et al.

    A Hybrid Swipe Fingerprint Mosaicing Scheme

  • F. Omri et al.

    Pixel level fusion of multispectral face images: short review

  • A. Nagar et al.

    Multibiometric cryptosystems based on feature-level fusion

    IEEE Trans. Inf. Forensics Secur.

    (2012)
  • C.H. Chan et al.

    Multiscale local phase quantization for robust component-based face recognition using kernel fusion of multiple descriptors

    IEEE Trans. Pattern Anal. Mach. Intell

    (2013)
  • S. Shekhar et al.

    Joint sparse representation for robust multimodal biometrics recognition

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2014)
  • Cited by (124)

    • Type-2 fuzzy blended improved D-S evidence theory based decision fusion for face recognition

      2022, Applied Soft Computing
      Citation Excerpt :

      Feature and sensor-level fusion is more discriminative than post-classification levels due to their ability to preserve low-level information. In this regard, it is worth noting that the researchers commonly used post-classification fusion due to its ease of processing whereas, fusion of the data coming from multiple sources becomes difficult due to outliers, incompatibility and dimension [4,6,13,14]. Multiple outcomes obtained from different classifiers are ensemble by means of decision fusion method to elevate the performance of a decision system [16].

    View all citing articles on Scopus
    View full text