A framework for selection and fusion of pattern classifiers in multimedia recognition
Introduction
The ever growing presence of sensors in our daily lives led us to the so called big-data revolution and within this shear amount of data, visual data is of particular interest. Citing a recent The New York Times article (Brooks, 2013) “The philosophy of data”, surely the rising philosophy of the day, is data-ism in which everyone wants to take advantage of data as it is the holy grail of contemporaneity. But, amidst such a massive amount of data, the question is how to process such data to actually come out with useful conclusions?
Visual data is of particular interest in this revolution. The explosion of visual data makes us face many new challenges unthinkable two decades ago. Image and video classification tasks have been inserted in different and complex applications (e.g., data categorization in search, biometric recognition, and document indexing through visual content, object recognition, etc.) and the use of machine learning-based solutions has become the most popular approach for several applications. However, there is no silver bullet that solves all the problems which means that it is not possible to characterize all images of different domains with the same description method nor is it possible to use the same learning method to achieve good results in any kind of application.
In this work, we investigate the combination of several learning methods and image descriptors aiming at creating more effective classifiers. We propose a framework for automatically combining the most discriminative classifiers using the support vector machine technique (SVM), as well as exploring the use of diversity measures to select the less-correlated, yet effective, classifiers. We have performed experiments that demonstrate that the proposed framework for classifier fusion yields comparable results to the traditional fusion approaches but using less learning and description methods as well as not incurring in the curse of dimensionality problems, which are common to some fusion techniques. Another major advantage of the proposed method is that it yields good classification results using small training examples being more robust to the small sample size problem common in many classification techniques (Bishop, 2006).
This paper extends upon two previous conference papers of ours (Faria et al., 2012a, Faria et al., 2012b) and differs regarding the following aspects: (i) the related work section was thoroughly extended; (ii) a new approach for classifier selection is introduced; (iii) other diversity measures have been included as additional criteria for selecting the most appropriate classifiers to be combined; (iv) additional classification problems have been considered; (v) more experiments for comparing the proposed framework with other state-of-the-art ensemble approaches were performed. It is also worth mentioning that the framework proposed in Faria et al., 2012a, Faria et al., 2012b were used as baselines in our experiments.
We organized the remainder of this paper in five sections. Section 2 presents related work and background concepts necessary for the understanding of this paper. Section 3 describes the steps of the proposed framework for classifier fusion and for selecting the most appropriate classifiers based on diversity measures. Section 4 shows the experimental protocol we devised to validate our work, while Section 5 discusses the results. Finally, Section 6 concludes the paper and points out future research directions.
Section snippets
Related work
This section presents related work on image classification and information fusion (Section 2.1), as well as introduces background concepts necessary for the full understanding of this work (Section 2.2).
The classifier fusion framework
This section presents our framework for classifier fusion and selection.
Experimental protocol
In this section, we present the experimental setup we used in this work.
Experiments and discussion
This section discusses the results regarding the effectiveness and efficiency of the proposed framework using four different datasets (Caltech, Coffee, Freefoto, and Fruits). Before going to the results, we present an experiment showing that diversity measures can provide different opinions and the research possibilities this might open.
Final remarks and future work
This paper presented a framework for fusion and selection of classifiers using a diversity measure analysis and meta-learning on top of classifiers outcomes. Particularly in this paper, we have used the support vector machine technique (SVM) but other learning methods could be used as well. We also compared several different learning methods and image descriptors in four different classification problems (scene/object classification and crop/produce recognition) showing the proposed method is
Acknowledgments
The authors are grateful to CAPES (Grants 1260-12-0), CNPq (Grants 304352/2012-8, 484254/2012-0 and 306580/2012-8), FAPESP (Grants 2010/14910-0, 2012/18768-0 and 2010/05647-4), and Microsoft Research for the financial support.
References (71)
- et al.
Fingerprint classification using one-vs-all support vector machines dynamically ordered with naive Bayes classifiers
Pattern Recogn.
(2008) - et al.
Image retrieval based on shape similarity by edge orientation autocorrelogram
Pattern Recogn.
(2003) - et al.
Classifier ensembles: select real-world applications
Inf. Fusion
(2008) - et al.
Comparative study of global color and texture descriptors for web image retrieval
J. Visual Commun. Image Represent.
(2012) - et al.
A novel algorithm for feature selection using Harmony search and its application for non-technical losses detection
Comput. Electric. Eng.
(2011) - et al.
Automatic fruit and vegetable classification from images
Elsevier Comput. Electron. Agric.
(2010) - et al.
Texture recognition and image retrieval using gradient indexing
J. Visual Commun. Image Represent.
(2000) - et al.
Innovations in fingerprint capture devices
Pattern Recogn.
(2003) - et al.
Object-based land cover classification of shaded areas in high spatial resolution imagery of urban areas: a comparison study
Remote Sens. Environ.
(2009) - Antonie, M.-L., Zaïane, O.R., Coman, A. 2001. Application of data mining techniques for medical image classification....
Parallel consensual neural networks
IEEE Trans. Neural Networks
Pattern Recognition and Machine Learning (Information Science and Statistics)
Bagging predictors
Mach. Learn.
Random forests
Mach. Learn.
LIBSVM: a library for support vector machines
ACM Trans. Intell. Syst. Technol.
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
What does classifying more than 10,000 image categories tell us?
Interactive multiscale classification of high-resolution remote sensing images, selected topics in applied earth observations and remote sensing
IEEE J. PP
Is combining classifiers with stacking better than selecting the best one?
Mach. Learn.
The Elements of Statistical Learning
Scale-sets image analysis
Int. J. Comput. Vision
The Meta-Pi network: building distributed knowledge representations for robust multisource pattern recognition
IEEE Trans. Pattern Anal. Mach. Intell.
Cited by (42)
A method for automatic identification and separation of wildlife images using ensemble learning
2023, Ecological InformaticsAn ensemble classifier for vibration-based quality monitoring
2022, Mechanical Systems and Signal ProcessingCitation Excerpt :Since these two approaches are complementary to each other, the best solution would be their combination. However, there is limited literature that addresses both approaches [51]. In this paper, an information-based framework has been proposed to select proper classifiers and then combine them to achieve the highest possible accuracy.
A role of computer vision in fruits and vegetables among various horticulture products of agriculture fields: A survey
2020, Information Processing in AgricultureFeature selection in multimedia: The state-of-the-art review
2017, Image and Vision ComputingA Study of the Neuro Learning Model to Diagnosis of the (COVID-19)
2023, Lecture Notes in Networks and SystemsPusion - A Generic and Automated Framework for Decision Fusion
2023, Proceedings - International Conference on Data Engineering