Fingerprint recognition under the influence of image sensor ageing

Fingerprint recognition performance is affected by many factors. One of these are defective pixels caused by ageing effects of the image sensor. We investigate the impact of these image sensor ageing related pixel defects on the performance of different fingerprint matchers (NBIS, VeriFinger, FingerCode and Phase Only Correlation). The matchers are compared against each other to quantify the differences in the impact. In practice also other influences besides image sensor ageing related effects are present. As we aim to evaluate the impact of the defective pixels only, disregarding template ageing and other external influences, it is not possible to use real image data. Instead an experimental study using an ageing simulation algorithm including hot and stuck pixels is conducted on the FVC2002 and FVC2004 data sets.


Introduction
Fingerprint recognition systems are well established nowadays because of their advantages over password or token based authentication.Most fingerprint scanners are based on an optical image sensor.The quality of the acquired fingerprint image can be degraded by many factors, e.g.bad finger surface conditions, dirt on the sensor's surface, external noise and misplacement.Another type of distortion impacting the image quality, originating from the fingerprint scanner itself, are image sensor defects caused by sensor ageing effects [1].These defects become noticeable in the form of isolated defective pixels, appearing as point like, spiky shot noise in the output images.Some example fingerprint images containing defective pixels can be seen in Fig. 1.
The statistical nature of these pixel defects as well as the source causing the defects are well studied in the literature for typical image sensors equipped in consumer cameras [2,3].In the domain of biometric recognition systems, the impact of image sensor ageing on the recognition performance has not received much attention.There is the work of Bergmüller et al. [4] regarding image sensor ageing in iris recognition, our previous work on finger vein [5] and hand vein recognition systems [6] as well as our work on fingerprint recognition [7], to which this current work is an extension to.The main extension compared with our previous works is a more systematic evaluation methodology featuring a more comprehensive test setup which is split into three different 'scenarios' in order to quantify the sensor ageing's impact on the recognition performance in situations corresponding to different real-world settings, including additional tests with aged templates for all the tested recognition schemes and on all four data sets.Moreover, the third scenario is completely new if it comes to fingerprints: different state-of-the-art denoising methods are thoroughly tested and evaluated in order to find the best performing method which is capable of mitigating the influence of image sensor ageing.Of course, the results of our previous investigations regarding fingerprint recognition [7] are also included.
Image sensor ageing belongs to the template ageing effect in biometrics.Other reasons for template ageing are biological ageing of the subject, changes in subject behaviour and changes in the acquisition conditions.All of these issues lead to a lower recognition performance.The aim of our study is to isolate the impact of image sensor ageing and to find out which role it plays in the scope of biometric template ageing.Using real-world fingerprint images is therefore not possible because there are always other effects in addition to image sensor ageing present like subject ageing and changes in environmental conditions, which further degrade the recognition performance.To be able to quantify the impact of image sensor ageing related pixel defects exclusively, we utilise an ageing simulation algorithm.This algorithm simulates the defective pixels caused by image sensor ageing according to our simplified pixel model and generates several 'aged' data sets out of an input fingerprint data set.Our empirical evaluations include different types of feature extraction and matching schemes as they may react differently to the image sensor ageing related effects.We evaluate two minutiae-based matchers, one ridge feature-based matcher and one correlation-based matcher.The simulations and the subsequent evaluations are performed on the FVC2002 [8] and FVC2004 [9] fingerprint data sets.The matchers' recognition performance is quantified in terms of the EER [equal error rate -point where false match rate (FMR) = false non-match rate (FNMR)], the FMR1000 (FNMR at a FMR of 0.1%) and ZeroFMR (FNMR at a FMR of 0%).Due to the nature of the defective pixels, our evaluations correspond to investigating the robustness of the fingerprint matchers against spiky shot noise.We systematically evaluate three different scenarios: i.The first scenario is the 'basic' one, corresponding to the most common case: only the probe images are aged using our simulation algorithm while the stored template images are unaltered.In a practical deployment of a fingerprint recognition system, a new fingerprint sensor is equipped and the database is established.The sensor stays in use several years and suffers from increasing ageing effects during that time, causing the image sensor to develop more and more defective pixels which of course only affect the newly acquired probe fingerprints.ii.In the second scenario not only the probe images but also the stored template images are aged using the same pixel defects as the probe ones.This corresponds to the application of a biometric sensor which is already in use for several years and thus shows some ageing related effects while establishing the biometric database.Of course the ageing related effects increase over time like in the first scenario, but in contrast to the first one, in the current scenario the focus is not on this increase but on the recognition performance right at the time of acquisition.In other words, we want to find out if an 'old' biometric sensor's baseline performance is worse compared with a new one, not accounting for additional ageing influences resulting from long timespans between the capturing sessions.Thus, the same pixel defects (same locations and parameters) are used for both, template and probe images, as this investigation focuses on images taken with the same, old biometric sensor and the pixel defects caused by image sensor ageing are permanent.iii.The third scenario is basically equivalent to the first one except that the aged probe images are denoised using one of several different denoising filters before the preprocessing and feature extraction in order to lower the influence of image sensor ageing.Again the template images are unaged (and not denoised).Most commercial biometric sensors use image enhancement techniques including denoising to reduce external influences, thus this scenario is directly related to practical deployments of a typical fingerprint recognition system.
The rest of this paper is organised as follows.Section 2 gives a brief overview on image sensor ageing, presents the pixel defect model and explains the ageing simulation algorithm.Section 3 outlines the four fingerprint matchers.The different image denoising methods are explained in Section 4. Section 5 describes the experimental setup and provides the evaluation results.Section 6 concludes this work and gives an outlook on future work.

Image sensor ageing
Most biometric sensors contain some kind of image sensor, especially optical fingerprint scanners.An image sensor is an analogue device, which basically consists of an array of photosensitive cells, called pixels.Like every other electronic device, an image sensor ages.Ageing becomes noticeable in form of defective pixels, showing different characteristics than at manufacturing time, even if the sensor is not in use.They appear as spiky shot noise in the output image and hence degrade the image quality.

Mechanism causing the defects
Pixel defects are permanent, i.e. once they appeared they do not heal over time, inter-defect times follow an exponential distribution [1] and the defects are randomly distributed over the sensor area.
According to the literature [2], the main defect causing source is cosmic ray radiation, actually the neutrons of the cosmic rays.
Leung et al. [10] applied statistical analysis to the spatial and temporal distribution of defects.They showed that the spatial distribution of defects across the sensor area follows a normal random distribution with no significant bias towards short or long distances, i.e. no defect clustering.They also pointed out that the number of defects increases linearly with time, indicating a constant defect rate.Both is in contradiction to material degradation as defect source.For more details, the interested reader is referred to their original papers [2,3,10].We focus on in-field defects only.On the contrary, manufacture time defects occur during the fabrication process and may also include stuck and abnormal sensitivity pixels but are usually corrected by factory calibration, i.e. simply masked out.

Defect types
The most prominent defect types that a sensor develops over its lifetime are hot and stuck pixels.
A hot pixel has an illumination independent component, which increases linearly with exposure time.It appears as a bright spot with a fixed location in the output image.A partially stuck hot pixel has an additional component (offset) that is independent from the illumination and exposure time.
The output of a stuck pixel is always the same arbitrary but fixed value c in the range 0 ≤ c ≤ 255 (assuming an 8-bit image).Thus, its output is constant under all illuminations and exposure time independent.Stuck pixels mainly appear as factory time defects and Leung et al. [3] claim that they never found a true stuck pixel in field.

Pixel defect model
A pixel model describes either the raw output of a single pixel or the whole sensor considering the incoming illumination and the impact of pixel defects.We adopted the pixel model of Bergmüller et al. [4], which is a simplified version of J. Fridrich's pixel model [1].As we aim for reproducible tests, modelling noise and environmental influences should be minimised.In fact, they can be eliminated completely in a simulation.All images are taken with the same sensor using the same exposure settings, thus the exposure time is constant and the photo-response non-uniformity can be eliminated.The dark current level is usually low for short exposure times used for capturing fingerprint images.This yields the simplified pixel model (y is the output of a single pixel, i is the incident illumination) If the dark current d of a pixel is extremely high, it is often denoted as hot pixel.Whereas if the offset c is high, this results in a saturated pixel and is sometimes denoted as stuck pixel.The definitions in the literature are not consistent, so we define the following pixel model for our experiments: where the first one is light independent and has a constant value c, denoted as stuck pixel.The second one adds an offset to the incident illumination and is referred to as hot pixel.The pixel model for 8-bit grey-scale images is where C and D are the defect matrices (same size w × h as the image, storing the stuck pixel value or hot pixel offset value, respectively), I is the incident illumination and x and y are the pixel's coordinates.A pixel's output Y(x,y) saturates at 0 and 255 if interval borders are exceeded.This pixel model is the basis for the ageing simulation algorithm.

Image sensor ageing simulation algorithm
We use the same simulation algorithm as in our previous work [5], which is an extended version of the original algorithm proposed by Bergmüller et al. [4]  offset d is added to a pixel's value for hot pixels and its value is replaced by c for stuck pixels.The defect matrices C and D are calculated recursively to comply with the once defective -always defective principle.

Empiric formula for estimating defect growth rate
Chapman et al. [11] derived an empirical formula for estimating the defect growth rate based on the sensor technology (Charge-Coupled Device (CCD) or Active Pixel Sensor (APS)) and on sensor design parameters like sensor area, pixel size and gain (adjusted by the ISO setting) where ρ is the defect density (defects/year/mm), A is the number of defects/year/mm if the pixel size is 1 m, S is the pixel size, ISO is the ISO value according to the ISO setting of the image sensor and B and C are constants depending on the sensor type (CCD: A = 0.0141, B = −2.25 and C = 0.69; APS: A = 0.0742, B = −3.07 and C = 0.5).

Fingerprint recognition
Today minutiae-based matchers are the most widely used ones.Our aim is to test different classes of fingerprint recognition algorithms as different types of fingerprint recognition schemes react differently to image degradations.Therefore, we do not only consider minutiae-based matchers but three fundamentally different types of fingerprint feature extraction and matching schemes: Correlation-based matchers: The fingerprint images are used in their entirety, the global ridge and furrow (i.e.valley) structure of a fingerprint is decisive.Images are correlated at different rotational and translational alignments, image transform techniques may be utilised for that purpose.As a representative of this class, we use a custom implementation of the Phase Only Correlation (POC) matcher [12].
Ridge feature-based matchers: They deal with the overall ridge and furrow structure in the fingerprint, yet in a localised manner.Characteristics like local ridge orientation or local ridge frequency are extracted to generate a set of appropriate features representing the individual fingerprint.As a representative of this class, we use a custom implementation of the FingerCode (FC) approach [13].
Minutiae-based matchers: The set of minutiae within each fingerprint is determined and stored as list, each minutia being represented (at least) by its location and direction.The matching process then basically tries to establish an optimal alignment between the minutiae sets of two fingerprints to be matched, resulting in a maximum number of pairings between minutiae from one set with compatible ones from the other set.As the first minutiae-based matcher, we apply mindtct and bozorth3 from the 'NIST Biometric Image Software' (NBIS) package (available at http://fingerprint.nist.gov/NBIS/)for minutiae detection and matching, respectively.The second minutiae-based matcher we use is VeriFinger, developed by Neurotechnology (VeriFinger SDK 9.0 available at http://www.neurotechnology.com/verifinger.html),denoted as VF.For more details on the SDK and the FC and POC approach, the interested reader is referred to recent work [14].

Image denoising
Different denoising methods were evaluated in the context of scenario 3 in order to mitigate the impact of the image sensor ageing related pixel defects.The main goal is to preserve the baseline recognition accuracy (for the unaged images) while improving the recognition accuracy of the aged images.The denoising methods are described in the following.A fingerprint image suffering from image sensor ageing effects and its denoised versions are shown in Fig. 2.

Median filtering + adaptive wiener filtering (MWF):
The most basic denoising method is median filtering of the image followed by an adaptive Wiener filtering.The pixel defects caused by image sensor ageing are similar to spiky shot noise and this kind of filtering is well suited to remove salt and pepper noise from images.Our own custom implementation is applied during the evaluations.
Bayesian estimate (Bayes): This denoising approach by Wong et al. [15] was originally intended to remove speckle noise from optical tomography retinal images.At first the image data is projected into the logarithmic space to decouple the noise-free data and the speckle noise.There a general Bayesian least squares estimate of the noise-free data is found.The required estimate of the posterior distribution of the noise-free data is calculated in a non-parametric fashion by a conditional posterior sampling approach.As the speckle noise is data dependent, this approach should enable significant noise suppression while preserving image details.The implementation of Markus Mayer and Martin Kraus has been utilised (publicly available at: https://www5.cs.fau.de/research/software/idaa/).
BM3DSharp (BM3DS): Dabov et al. [16] proposed an extension of their original BM3D denoising approach [17].It combines the block-matching and three-dimensional (3D) filtering approach of BM3D with alpha-rooting, which is a transform-domain sharpening technique.The original BM3D image denoising strategy works by exploiting groups of similar 2D image fragments, combining them into 3D arrays.Collaborative filtering is then applied to these 3D groups.This is done in three successive steps: 3D transformation of a group, shrinkage of the transform spectrum (by hard thresholding) and inverse 3D transformation.Alpha-rooting is applied to the hard-thresholded 3D transform spectrum before the inverse 3D transform.The implementation of Alessandro Foi (publicly available at: http://www.cs.tut.fi/~foi/GCF-BM3D/ and http://www.cs.tut.fi/~lasip/) has been utilised.

Local polynomial approximation intersection of confidence interval (LPA-ICI):
This approach by Katkovnik et al. [18] is based on the spatially adaptive restoration of a multi-variable anisotropic function, represented by uniformly sampled noisy data.Anisotropic discrete directional kernel estimates are found using the local polynomial approximation (LPA).The non-linearity is incorporated by an intersection of confidence intervals (ICI).A set of directional-LPA convolution kernels for several specified directions is used to obtain a set of directional varying-scale estimates.These are compared using the ICI rule, resulting in an adaptive scale estimate for every pixel in the image and each direction.The estimates are then fused to obtain the final anisotropic LPA-ICI estimate.The LPA-ICI implementation of Alessandro Foi has been utilised (http://www.cs.tut.fi/~foi/GCF-BM3D/ and http://www.cs.tut.fi/~lasip/).

Shape-adaptive Discrete Cosine Transform (DCT) (SADCT):
The SADCT was originally proposed by Foi et al. [19] for video compression but can be applied for still image denoising too.Therefore, SADCT is used in combination with the LPA-ICI technique.LPA-ICI defines the shape of the transform's support for SADCT.At first the mean is subtracted from the input image.Then the SADCT is computed by cascaded application of 1D varyinglength DCT transforms, which are normalised in order to get a normalised SADCT transform.Denoising is then performed either by hard thresholding or Wiener filtering in the SADCT domain for every point in the original input image.The implementation of Alessandro Foi has been utilised (http://www.cs.tut.fi/~foi/GCF-BM3D/ and http://www.cs.tut.fi/~lasip/).
Adaptive pixel defect filtering (APDF): This is our own custom extension of the basic median and Wiener filter based denoising approach.At first the image is filtered using a median filter followed by an adaptive Wiener filter to create an estimation image.Then each pixel in the original image is compared with the corresponding pixel in the filtered image.If the difference is above a predefined threshold, the output pixel value is set to the value of the filtered estimate, else it is set to the original pixel value.

Experiments
We conduct our empirical study based on the FVC2002/FVC2004 DB1 and DB2.DB3 was acquired using a thermal sensor, whereas DB4 contains synthetically generated images, but both are not used because we focus on optical image sensors only.FVC2004's test procedure to determine the FMR, FNMR in order to calculate the EER, FMR1000 and ZeroFMR, is adopted.For details on the databases itself and the test protocol please refer to [8,9].

Experimental settings
To apply our image sensor ageing simulation algorithm, the first step is to set the simulation parameters, i.e. we have to determine the defect growth rate.Bergmüller et al. [4] had two iris data sets available, captured with the same sensor and a time span of 4 years in between.They estimated the relative growth in the number of defective pixels using a statistical approach based on the captured images, resulting in a defect rate of 0.6659 defects/(MP*year).We have no such data sets for fingerprints, hence we are not able to determine the defect growth rate based on real sensor data.Instead we use the formula of Chapman et al. [11] to derive the approximate defect growth rate from the technical data of the DigitalPersona U.are.U 4000B (DigitalPersona now belongs to CrossMatch and the U.are.U 4000B has been discontinued.No information about the U.are.U 4000B can be found on the CrossMatch website but there is a reference to this fingerprint scanner on the Neurotechnology website: http:// www.neurotechnology.com/fingerprint-scanner-digitalpersona-uare-u-4000.html)fingerprint scanner, as a representative of commonly equipped fingerprint scanners.The defect growth rates of similar sensors might differ, but they should be in the same range.There is no data concerning the image sensor used in the U.are.U 4000B fingerprint scanner available, thus we disassembled the device and calculated the sensor area based on an image of the sensor chip (see Fig. 3) and its outside dimensions.The outside dimensions (dark yellow square) of the image sensor chip are 10.6 × 10.6 mm, measured using a digital calliper rule or 410 × 410 pixels in the image, respectively.The actual image sensor area (depicted as red to yellow rectangle in the left and blue rectangle in the right picture) is about 77 × 66 pixels.The relation of (77/410) × 10.6 and (66/410) × 10.6 mm yields the dimensions of the actual sensor area of 1.99 × 1.71 mm.
The U.are.U 4000B has a resolution of 356 × 328 pixels.Thus the pixel size is: (1.99/356) × (1.71/328) = 5.59 × 5.21 μm.We assume a quadratic pixel size of 5.4 μm and ISO level 400 as there is no information available at which ISO level the FVC2002/ FVC2004 images were captured.Consequently, the resulting defect rate is ρ = 3.404 × 0.0742 × 5.4 −3.07 × 400 0.5 = 0.0285 defects/year = 0.244 defects/(MP * year) According to Chapman et al. [11] and Theuwissen [2], the additional offset I Offset of hot pixels or the dark current value, respectively, follows an exponential distribution, i.e. hot pixels with a lower amplitude are more likely to occur.The exponential distribution's parameter μ = 0.15 was estimated based on their data.
In practice, only very few defective pixels occur under normal conditions as indicated by the theoretical defect rate of 0.244 defects/(MP * year).
At first we started with 1 defect/(MP * year) (less than one defect cannot be simulated), resulting in 10 defects/MP after 10 years which had no effect at all on the recognition accuracy.Then we successively increased the defect rate (2, 5, 10, 20, 50, 70, 100, 200 and 400) but again there was no significant influence on the recognition accuracy in most cases.A timespan of 10 years with a defect rate of 400 defects/ (MP*year) results in 4000 defects/MP, which is a good starting point to notice at least some influence.Thus, we run our simulations with a defect rate of 4000 defects/(MP*year), a time span of 10 years and time steps of 1 year.
Such a high defect rate might not occur in practice.Nevertheless, higher defect rates than the estimated theoretical one might occur for instance in the following scenarios: • The intensity of the cosmic ray radiation is proportional to the altitude, i.e. it increases with increasing altitude.At 9144 m (30,000 ft), which is the usual flight level of long distance airplanes, it is about 300-400 times higher than at ground level.If a fingerprint or other biometric scanner is equipped on an airplane for authentication purposes, this scanner will be subjected to this higher level of radiation and thus to increased pixel defect rates.• Electrical stress imposed to fingerprint scanners might lead to additional defects on the image sensor which further increases the pixel defect rate by several orders of magnitude.
We run the simulations for hot pixels only, stuck pixels only and combined stuck and hot pixels.To mitigate statistical fluctuations due to the random locations and amplitudes of the defects, all tests are run five times (except for FC and POC, due to their long runtime only one run is performed) and the mean value of all runs is the final result.The experiments are run in the three different scenarios motivated in the introduction: in scenario 1 (S1) only the probe images are aged.In scenario 2 (S2) the template images are aged too, whereas in scenario 3 (S3) the probe images are aged and then denoised while the templates are again unaged.One could also think of a fourth scenario (S4), a combination of S2 and S3, in which the templates and probe images are both aged and then subsequently denoised.We evaluated this scenario for NBIS and VF and found out that the general trend is the same as in scenario S3 with some minor fluctuations (no general improvement nor worsening).Thus, this is not included as a separate scenario in the paper.
All results are evaluated in terms of the relative difference between the aged and the baseline EER/ZeroFMR/FMR1000 value.These relative differences are calculated as follows: where r a denotes the relative increase/decrease (in percentage terms) for a given year a and evaluated on the database DB, V i is the performance indicator for the ith recognition algorithm, DB 0 is the original (unaged) database and DB a is the aged database at age a.

Experimental results
Scenario S1: Table 1 (FVC2004 DB1) shows the S1 results for FVC2004 DB1.First of all, the baseline values are given as absolute values, followed by the relative increases (in %) at a defect density of 40,000 hot pixels per MP, 40,000 stuck pixels per MP and 40,000 combined hot and stuck pixels per MP.According to the baseline values, VF is by far the best performing matcher, followed by FC, closely followed by NBIS.The worst performing matcher is POC.This ranking is not changed across the whole tested range.Looking at the solid lines in Fig. 4a, showing the EER increases for combined hot and stuck pixels, NBIS is influenced most, followed by FC and VF.POC is influenced least.This is in agreement with the FMR1000 and ZeroFMR values.POC shows a negative EER trend indicating that its performance increases with an increasing number of defective pixels which might be caused by an amplification of the frequency ranges in which the fingerprint information is contained relative to the noise introduced by the defective pixels.In general, the matching performance drops with an increasing defect density although the highest increase in EER is only about 16.5%.The results for FVC2004 DB2 are shown in Table 1 (FVC2004 DB2) and Fig. 4b (solid lines).Given the unaged images VF performs best, followed by NBIS, then FC and POC performing worst.On DB2 both minutiae-based matchers perform better than the correlation-and the ridge feature-based one.POC is again most stable, followed by VF and NBIS.All three are nearly influenced to the same extent, while FC is heavily influenced on DB2.In general, on DB2 hot pixels cause the least decrease in recognition performance, followed by hot and stuck pixels combined.Stuck pixels lead to the highest decrease.On DB1, hot and stuck pixels combined lead to the most severe effects.
Comparing the results in Table 1 (FVC2004 DB1) and Table 1 (FVC2004 DB2) unveils that the influence of pixel defects on DB2 is higher than on DB1.The absolute area (in pixels) covered by the fingerprint in DB1 images is larger than in DB2 images.Consequently, the ridges and valleys are wider and thus less affected by single pixel defects.Hot pixels have less influence on DB1 than on DB2.This is quite obvious as the background in the DB1 images is all white.A hot pixel simply adds an offset to the pixel value and if the pixel is already white, it stays white, i.e. no change.Moreover, stuck pixels have less influence on DB1.This is due to the higher contrast compared with DB2 images, i.e. the ridge lines appear darker and together with the bright background this reduces the influence of single pixel defects.
The results for FVC2002 DB1 are shown in Table 2 (FVC2002 DB1).Looking at the baseline values, it can be seen that VF again clearly performs best, followed by NBIS and then by FC.POC performs worst.According to the values in the table and the solid lines in Fig. 4c, FC and POC are influenced less by defective pixels compared with NBIS and VF.VF consistently shows the highest relative EER increases, however its absolute performance remains the best one compared with the other matchers across the whole tested range.The two minutiae-based matchers achieve a higher performance than FC and POC, but are more sensitive to defective pixels.Similar to FVC2004 DB2, hot pixels cause the least influence, followed by hot and stuck pixels combined and stuck pixels lead to the highest influence.
Table 2 (FVC2002 DB2) and the solid lines in Fig. 4d depict the results on FVC2002 DB2.Again VF clearly outperforms all other matchers.NBIS is ranked second, followed by FC and POC performs worst.The performance of VF even increases with an increasing number of defects.In contrast to DB1, the two minutiaebased matchers are influenced less than FC and POC on DB2.Again hot pixels lead to the least influence while stuck pixels lead to the most severe one.Hot and stuck pixels combined are in between.Comparing the FVC2002 and FVC2004 results indicates that the minutiae-based matchers perform better and are less IET Biom.This is an open access article published by the IET under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/)influenced by defective pixels than the non-minutiae-based ones for higher quality fingerprints (FVC2002).
Scenario S2: Table 3 lists the results on all tested data sets for scenario S2.The dashed lines in Figs.4a and b show the hot+stuck pixels results for scenario S2 on the FVC2004 data sets DB1 and DB2, respectively.Looking at VF there is no clear trend if VF with aged templates (S2) or VF with only probes aged (S1) performs better.On DB1 NBIS S2 and NBIS S1 perform nearly equally for hot pixels.However, for stuck and hot+stuck pixels, NBIS S2 performs better than NBIS S1 which is in a way the expected behaviour.If the templates are aged too, the images become more similar and this should lead to improved matching results.FC and POC S2 perform much worse than in the S1 case starting at about 25,000 defects/MP for FC and 10,000 defects/MP for POC on DB1.This is due to an increased FMR.The impostor scores become higher and are equal or sometimes even higher than the genuine ones.We assume that FC and POC are not able to cope with the kind of noise introduced by the high number of stuck pixels.The fingerprint information is suppressed by the noise which is the same in every image and makes all the images looking more similar for FC and POC.Note that the FC and POC results for more than 24,000 defects/MP are not depicted in the figure (only the beginning of the steep slopes is visible) to be able to distinguish the curves of NBIS and VF better.
Analysing the results of scenario S2 on FVC2004 DB2 unveils that VF S2 behaves similar to S1.For NBIS the situation is completely different.NBIS S1 clearly outperforms NBIS S2 with increasing defect density.This time the images looking more similar leads to an increase in the FMR and results in an increased EER.The performance of FC and POC S2 is inferior compared with the S1 case on DB2.Again for stuck pixels FC and POC are heavily influenced.The reason is the same as for DB1, the increased FMR.
The dashed lines in Figs.4c and d show the S2 results on FVC2002 DB1 and DB2, respectively.The situation is similar to FVC2004.The results of VF and NBIS do not significantly differ for S1 and S2 on DB2.NBIS S2 performs worse than NBIS S1 on DB1.FC and POC are another time influenced more in the S2 case, especially if stuck pixels are present.Again starting from a certain number of stuck pixel defects, POC is highly influenced and its EER rises sharply on DB1.It is less influenced on DB2 but still remains the most affected matcher.
Scenario S3: Scenario S3 evaluates different denoising methods.The results on FVC2004 DB1 and DB2 can be seen in Table 4.In Figs.5a and b, the dashed lines represent the S3 results  for NBIS and the dashed-solid lines for VF, respectively.The baseline results (at 0 defects/MP) are relative relating to the S1 baseline values.The figures (5a to 5d as well as 6a to 6d) include the S1 results (solid lines) to serve as a basis for the comparison and the results for the three best performing denoising methods (i.e.achieving the lowest EERs or the lowest relative increase in the EERs).
Focusing on DB1 and NBIS, the APDF denoising achieves the best performance and is even able to improve NBIS' EER compared with its S1 baseline one starting at 48,000 defects/MP (negative values in the figure).BM3DS has nearly the same performance up to 48,000 defects/MP.Starting from that point the results get worse but still remain better than without denoising.MWF is worse than APDF and BM3DS but starting from about 12,000 defects/MP it is able to improve the results compared with the S1 case.For VF on DB1 APDF is the best denoising method, followed by SADCT.SADCT's performance drops at 48,000 defects/MP.BM3DS as third best denoising approach is also able to improve the results compared with the S1 case.
Moving on to FVC2004 DB2, the three best performing denoising methods (BM3DS, LPA-ICI and APDF) improve NBIS' performance across the whole range.BM3DS performed best, followed by APDF and MWF.In contrast to DB1, APDF improved NBIS' results over BM3DS starting from 48,000 defects/MP (it is worse than BM3DS for such a high number of defects on DB1).LPA-ICI and BM3DS are able to improve VF's performance on DB2.APDF is only able to improve VF's performance starting from 56,000 defects/MP, between 10,000 and 56,000 defects/MP the performance decreases.
POC's results on DB1 and DB2 can be seen in Table 6 as well as in Figs.6a and b, respectively.On DB1, LPA-ICI is able to improve POC's performance up to 32,000 defects/MP, whereas Bayes and SADCT are able to improve POC's performance starting from 40,000 defects/MP.The other denoising methods do not further improve POC's performance (note that most points are below the 0% line meaning POC's performance is already better than its S1 baseline one).On DB2, all denoising methods except MWF are able to reduce the influence of the defective pixels for POC.APDF is able to improve POC's performance compared with its S1 baseline one across the whole range.BM3DS and Bayes achieve an improvement up to about 50,000 defects/MP.
MWF and APDF are able to improve FC's performance on DB1 (Fig. 6a) starting from 8000 defects/MP but not in the baseline case.BM3DS is able to regain some performance for FC between 26,000 and 56,000 defects/MP.BM3DS and LPA-ICI are able to improve FC's performance over the S1 baseline one if only hot pixels are present on DB1.Fig. 6b (note that the FC S1 curve is way beyond the 25% line starting from 8000 defects/MP and thus not shown in the figure) reveals that BM3DS is not only able to completely mitigate the impact of the defective pixels but also improves FC's performance compared with its S1 baseline one across the whole tested range on DB2.Bayes and APDF are able to improve FC's EER compared with the S1 baseline one up to 32,000 defects/MP.All denoising methods except MWF generally increase FC's performance in the hot pixel only case.
Table 5 as well as Figs.5c and d show the results for NBIS and VF on FVC2002 DB1 and DB2, respectively.On FVC2002 DB1, the best performing denoising method for NBIS is Bayes, followed by APDF and BM3DS.BM3DS is only able to improve NBIS' results compared with the S1 case up to 24,000 defects/MP.Improving the results of VF seems to be more difficult as only BM3DS is able to achieve that.LPA-ICI, as the second best performing denoising method, is not really able to improve the results and also SADCT makes the results worse.On DB2 (Table 6), the situation is mirrored compared with DB1.All three best performing denoising methods, Bayes, LPA-ICI and MWF are able to improve VF's S3 results over the S1 ones (except MWF for more than 28,000 defects/MP).None of the three best performing denoising methods (BM3DS, SADCT and APDF) is able to improve the results for NBIS on DB2.POC's and FC's results on FVC2002 DB1 and DB2 are shown in Table 7 and Figs.6c and d, respectively.On DB1 only APDF is able to improve POC's performance in general.On DB2, Bayes boosts up POC's performance starting from 4000 defects/MP up to 30,000 defects/MP and BM3DS as well as SADCT from 8000 defects/MP, respectively.MWF raises FC's performance over the  S1 baseline one across the whole range on DB1, whereas Bayes and BM3DS are able to mitigate all effects of the defective pixels up to about 30,000 defects/MP.BM3DS, SADCT and APDF boost up FC's performance over the S1 baseline one on DB2 but do not improve the overall performance of FC up to 16,000 defects/MP because its S3 performance is already better than its S1 baseline  one.On DB1 as well as on DB2, denoising is less effective for stuck pixels than for hot pixels only if applied to FC or POC in most cases.

Results summary:
The scenario S1 results revealed that the minutiae-based matchers are less influenced by defective pixels for higher quality fingerprints.In general, hot pixels have the least influence, followed by hot and stuck pixels combined and if the number of defects remains the same while only stuck pixels are present these have the highest influence, i.e. these lead to the highest decrease in the recognition performance.In addition, the relative influence of the defective pixels is depending on the database.In any case, for a realistic number of defective pixels in conventional environments and operating conditions, there is no noticeable influence on the recognition performance.
Our experiments in the context of scenario S2 (in which the templates are also aged) showed that the results get worse for most matchers (especially FC and POC) on all tested data sets.Thus, it constitutes a clear disadvantage to use an already 'old' fingerprint scanner for establishing a new biometric database.
In the context of scenario S3, it became apparent that denoising is able to reduce the effects of the sensor ageing related pixel defects, in many cases it completely mitigates them, sometimes even improving the performance compared with the baseline one performance.However, in a few cases (for VF on two of the data sets) no general improvement over the whole range was possible.Most denoising methods work better if only hot pixels are present.The best suitable denoising method is highly depending on the particular data set and recognition scheme.Thus, no overall best denoising method can be determined.Nevertheless, in almost all cases it is possible to choose a suitable denoising method for the specific combination of matcher, database, defect type and defect density in order to retain the baseline performance of the recognition system regardless of the presence of image sensor ageing related pixel defects.

Conclusion
We investigated the influence of image sensor ageing related pixel defects within the biometric template ageing phenomenon on fingerprint recognition systems using three different scenarios, including only probe images aged, templates aged and probes aged and denoised while templates are unaged.As it is not possible to investigate solely the impact of image sensor ageing using actual fingerprint images acquired by a real fingerprint sensor, disregarding all other template ageing effects and external influences we used an image sensor ageing simulation algorithm to generate the aged fingerprint images.At first we exemplarily estimated the defect growth rate based on a real sensor's characteristics.On the one hand, this theoretical rate is quite low and does not lead to any influence on the recognition accuracy.On the other hand, the models of Chapman et al. and J. Fridrich are only theoretical and not able to cover all the effects occurring inside a sensor in real-world applications leading not only to higher defect growth rates but also other effects having a negative impact on the image quality.To take this into account, we used a much higher defect rate during our experiments.
Although there is a noticeable drop in the recognition performance for unrealistic high numbers of pixel defects, most matchers are quite stable in general.Higher defect rates than the theoretical one of 0.244 defects/(MP*year) are likely to occur in practice because other external influences are present.This would still only lead to a few defective pixels over a reasonable sensor lifetime of 10 years.We showed that <1000 defects/MP have no impact on the recognition accuracy.For a higher number of defects the influence is highly depending on the data set and the recognition scheme/matcher.Moreover, the performance of some recognition schemes even increased if defective pixels were present.In addition, several of the tested image denoising techniques were able to reduce or even cancel out the effect of the defective pixels across the whole tested range depending on the particular data set and recognition scheme.By choosing an adaptive denoising method depending on the combination of defect density, data set and recognition scheme it becomes possible to completely mitigate the effects of image sensor ageing induced defective pixels even for extremely high defect rates.Thus, the The upshot of all this is that defective pixels caused by image sensor ageing do not seem to be a problem in usual practical deployments of fingerprint recognition.
Our future work will include tests with real fingerprint scanners.We acquired several off-the-shelf commercial fingerprint scanners which we plan to store inside an airplane to induce higher defect rates and evaluate the sensors at regular time intervals to be able to quantify the image sensor ageing impact (in terms of visible defects).This will help us to refine the pixel defect model and taking all the ageing effects occurring inside a real-world fingerprint sensor into consideration.

Fig. 1
Fig. 1 Example fingerprint images from artificially aged sensors (by simulating image sensor ageing)

Fig. 3
Fig. 3 Image sensor inside the U.are.U 4000B fingerprint scanner

Table 1
This is an open access article published by the IET under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/)

Table 3
This is an open access article published by the IET under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/)

Table 6
This is an open access article published by the IET under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/)contribution of defective pixels caused by image sensor ageing to the template ageing effect in fingerprint recognition is negligible.