A complete fuzzy discriminant analysis approach for face recognition
Introduction
Face recognition is a natural and direct biometric method, which has been researched in various areas such as computer vision, image processing, and pattern recognition. In practice, as one of the most frequently used kernel-based learning machine, support vector machine (SVM) [3], [17] is a newly emerging technique for learning relationships in data within the framework of statistical theory. It outperforms conventional classifiers especially when the number of training data is small and there is no overlap between classes. For the conventional methods, an n-class problem is converted into n two-class problems or n(n − 1)/2 two-class problems. But unclassifiable regions still remain by those two methods. To resolve unclassified regions for the classification, Shigeo [1], [15] proposed fuzzy support vector machine (FSVM) method represented with the fuzzy membership function while realizing the same classification results with that of the conventional pair-wise classification. However, a serious drawback of FSVM is that the computation requirement increases rapidly with the increase of the number of classes and training sample size. To address this problem, an improved FSVM method that combines the FSVM and decision tree, called DT-FSVM, is proposed firstly.
On the other hand, we know that the most well-known feature extraction techniques used for face recognition are those of Eigenface [16] and Fisherface [2]. The Eigenface method relies on a transformation of feature vectors by utilizing principal components (PCA). In essence, the PCA dwells on a linear projection of a high-dimensional face image space [4], [7] into a new low-dimensional feature space [20], [21]. The major problem of the Eigenface technique is that it can be affected by variations in light direction, different face poses and diversified face expressions. The second well-known approach of Fisherface is insensitive to large variation in the conditions we have enumerated above, which performs PCA on the training data and followed by Fisher's linear discriminant analysis (LDA) [10], [12], [14], [19], [23], [25], [28]. It has been one of the effective algorithms due to its power of extracting the most discriminatory features. However, the most existing LDA based algorithms are employed to dwell on the concept of a binary (yes/no) class assignment meaning that the samples are assigned to the given classes (categories) definitely. Evidently, as the samples are significantly affected by numerous environmental conditions (such as face images are affected by illumination, expression, etc.), it is advantageous to investigate these factors and quantify their impact on their “internal” class assignment [11]. Interestingly, the idea of such class assignment has been around for a long time and can be dated back to the results published by Keller et al. [9] coming under the notion of a fuzzy k-nearest neighbors classifier.
Therefore, the remaining problem of our framework is to revisit the F-LDA feature extraction technique [11] and augment it by some improved mechanisms of fuzzy set [26]. Considering the fact that the outlier samples in the patterns may have some adverse influence on the classification result, we developed a novel F-LDA algorithm using a relaxed normalized condition in the definition of fuzzy membership function. Compared with the F-LDA algorithm, the presented method computes the discriminant vectors associated with the membership grade from each training sample, which is theoretically effective to overcome the classification limitation originated from the outlier samples. Finally, by making full use of fuzzy set theory, a complete F-LDA (CF-LDA) framework is developed by combining the reformative F-LDA (RF-LDA) feature extraction method and DT-FSVM classifier.
This paper is organized as follows. Section 2 gives a brief introduction of FSVM classifier and its improved algorithm for the multi-classes problems. Section 3 provides a concise summary of the technique of F-LDA method and introduces all required notation. The two sections listed above can serve as a prerequisite to the complete fuzzy discriminant analysis approach outlined in Section 4. Section 5 reports on comprehensive simulation results completed for several commonly used face image databases such as ORL and NUST603 (603 Laboratory, School of Computer Science and Technology, Nanjing University of Science and Technology). Finally, concluding comments are presented in Section 6.
Section snippets
Conventional pair-wise classification
The SVM algorithm computes for the separating hyperplane whose margin of separation between positive and negative samples is maximized. Since the extension to nonlinear decision function is straightforward, to simplify discussions, we consider linear decision functions. Let the decision function for class i against class j, with the maximum margin, bewhere is the m-dimensional vector, bij is a scalar, and Dij(x) = −Dji(x).
For the input vector x we calculate
Conventional F-LDA approach
In this section, we are concerned with face recognition using F-LDA algorithm [11] and its fuzzy set based augmentation. The well-known Fisherface method is relatively insensitive to substantial variations in light direction, face pose, and facial expression. This is accomplished by using both PCA and LDA analysis. What makes most of methods in face recognition (including the Fisherface approach) similar is an assumption about the same level of typicality (relevance) of each face to the
How to develop a complete F-LDA approach
Up to now, the F-LDA method combined with FKNN algorithm is considered to regain the statistical properties of the patterns such as the mean value and scatter covariance matrices. However, after investigating the membership allocation formula, we find that the method attempts to “fuzzify” or refine the membership grades of the labeled patterns only by fuzzifying the each class center. How can we make full use of the distribution information of each sample to the redefinition of scatter
Experimental results
The proposed CF-LDA approach is used for face recognition and tested on two available public face databases, i.e., the NUST603 [27] and ORL [13] face image databases. To evaluate the CF-LDA method properly, we also include experimental results for Fisherface [2], D-LDA [24], F-LDA and RF-LDA methods, respectively.
Conclusions
In this paper, we have proposed a novel complete fuzzy discriminant analysis approach to accomplish the mission of face recognition. This algorithm is based on the RF-LDA feature extraction method and DT-FSVM classifier, which has increased discriminatory capability and is more adaptive compared with the traditional F-LDA method. Meanwhile, using reformative fuzzy membership function will ensure that the different sample in the training set can make different contribution to the redefinition of
Acknowledgements
The authors would like to thank the anonymous reviewers for their constructive advice. This work is supported by the National Science Foundation of China (nos. 60632050, 60572034, 90820002), Program for New Century Excellent Talents in University of China (Grant no. NCET-06-0487), Natural Science Foundation of Jiangsu Province (Grant no. BK2006081) and Program for Innovative Research Team of Jiangnan University (Grant no. JNIRT0702).
References (28)
- et al.
A new LDA-based face recognition system which can solve the small sample size problem
Pattern Recognition
(2000) - et al.
Boosting k-nearest neighbor classifier by means of input space projection
Expert Systems with Applications
(2009) - et al.
Solving the small sample size problem in face recognition using generalized discriminant analysis
Pattern Recognition
(2006) - et al.
Discrimination of disease-related non-synonymous single ucleotide polymorphisms using multi-scale RBF kernel fuzzy support vector machine
Pattern Recognition Letters
(2009) - et al.
Feature extraction for one-class classification problems: enhancements to biased discriminant analysis
Pattern Recognition
(2009) - et al.
Face recognition using a fuzzy Fisherface classifier
Pattern Recognition
(2005) - et al.
A note on two-dimensional linear discriminant analysis
Pattern Recognition Letters
(2008) - et al.
Linear dimensionality reduction by maximizing the Chernoff distance in the transformed space
Pattern Recognition
(2008) - et al.
Fuzzy least squares support vector machines for multiclass problems
Neural Networks
(2003) - et al.
A novel class-dependence feature analysis method for face recognition
Pattern Recognition Letters
(2008)
Why can LDA be performed in PCA transformed space
Pattern Recognition
Neighborhood discriminant projection for face recognition
Pattern Recognition Letters
A direct LDA algorithm for high-dimensional data—with application to face recognition
Pattern Recognition
Fuzzy set
Information and Control
Cited by (50)
Swarm intelligence inspired classifiers for facial recognition
2017, Swarm and Evolutionary ComputationCitation Excerpt :The best recognition rates are obtained when using the Gabor magnitude as characteristic features in combination with LDA as a reduction space method. Furthermore, this study shows that the recognition accuracy rates of the proposed algorithms (DT-BPSO, BA) are higher than those based DT-FSVM [39] and on bacteria foraging optimization (BFO) [53,54]. Face recognition accuracy is dependent on three related steps namely, features extraction, dimensionality reduction and classification.
Heterogeneous face matching using geometric edge-texture feature (GETF) and multiple fuzzy-classifier system
2016, Applied Soft Computing JournalExtended semi-supervised fuzzy learning method for nonlinear outliers via pattern discovery
2015, Applied Soft Computing JournalCitation Excerpt :The F-MMC [28] utilizes generalized singular value decomposition to the maximum margin criterion (MMC) criterion, and the fuzzy scatter matrices are simultaneously redefined by means of a membership. RF-LDA is our previous work [29] that extends F-LDA and includes complete fuzziness in the calculation of between-class and within-class scatter matrices. Meanwhile, RF-LDA is a part of the foundation of this article.
Face recognition based on curvelets and local binary pattern features via using local property preservation
2014, Journal of Systems and SoftwareA fuzzy supervised learning method with dynamical parameter estimation for nonlinear discriminant analysis
2013, Computers and Mathematics with ApplicationsCitation Excerpt :How can we take full advantage of the distribution information of each sample to the redefinition of scatter matrices? In our previous research in Ref. [13], we extended the F-LDA and included complete fuzziness in the calculation of between-class scatter matrix and within-class scatter matrix. The proposed method HRF-LDA is inspired by an improved fuzzy LDA, which ranked each feature according to the relevancy between the outlier patterns and its counterparts by calculating the membership grades.