Abstract
In the past few years, a lot of attention has been devoted to multimedia indexing by fusing multimodal informations. Two kinds of fusion schemes are generally considered: The early fusion and the late fusion. We focus on late classifier fusion, where one combines the scores of each modality at the decision level. To tackle this problem, we investigate a recent and elegant well-founded quadratic program named MinCq coming from the machine learning PAC-Bayesian theory. MinCq looks for the weighted combination, over a set of real-valued functions seen as voters, leading to the lowest misclassification rate, while maximizing the voters’ diversity. We propose an extension of MinCq tailored to multimedia indexing. Our method is based on an order-preserving pairwise loss adapted to ranking that allows us to improve Mean Averaged Precision measure while taking into account the diversity of the voters that we want to fuse. We provide evidence that this method is naturally adapted to late fusion procedures and confirm the good behavior of our approach on the challenging PASCAL VOC’07 benchmark.
Chapter PDF
References
Atrey, P.K., Hossain, M.A., El-Saddik, A., Kankanhalli, M.S.: Multimodal fusion for multimedia analysis: a survey. Multimedia Syst. 16(6), 345–379 (2010)
Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines (2001)
Devroye, L., Gyorfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition. Springer (1996)
Dietterich, T.G.: Ensemble methods in machine learning. In: Multiple Classifier Systems, pp. 1–15 (2000)
Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL visual object classes challenge 2007 (VOC 2007) results (2007)
Fakeri-Tabrizi, A., Amini, M.-R., Gallinari, P.: Multiview semi-supervised ranking for automatic image annotation. In: ACM Multimedia, pp. 513–516 (2013)
Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proc. of ICML, pp. 148–156 (1996)
Fürnkranz, J., Hüllermeier, E.: Preference Learning. Springer (2010)
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. TPAMI 20, 226–239 (1998)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms (2004)
Lacasse, A., Laviolette, F., Marchand, M., Germain, P., Usunier, N.: PAC-Bayes bounds for the risk of the majority vote and the variance of the gibbs classifier. In: NIPS (2006)
Laviolette, F., Marchand, M., Roy, J.-F.: From PAC-Bayes bounds to quadratic programs for majority votes. In: ICML (2011)
Leonard, D., Lillis, D., Zhang, L., Toolan, F., Collier, R.W., Dunnion, J.: Applying machine learning diversity metrics to data fusion in information retrieval. In: Clough, P., Foley, C., Gurrin, C., Jones, G.J.F., Kraaij, W., Lee, H., Mudoch, V. (eds.) ECIR 2011. LNCS, vol. 6611, pp. 695–698. Springer, Heidelberg (2011)
Ma, A.J., Yuen, P.C., Lai, J.-H.: Linear dependency modeling for classifier fusion and feature combination. TPAMI 35(5), 1135–1148 (2013)
McAllester, D.A.: PAC-bayesian model averaging. In: COLT, pp. 164–170 (1999)
Re, M., Valentini, G.: Ensemble methods: a review. In: Advances in machine learning and data mining for astronomy, pp. 563–582 (2012)
Snoek, C., Worring, M., Smeulders, A.W.M.: Early versus late fusion in semantic video analysis. In: ACM Multimedia, pp. 399–402 (2005)
Sun, S.: A survey of multi-view machine learning. Neural Computing and Applications 23(7-8), 2031–2038 (2013)
Wickramaratna, J., Holden, S., Buxton, B.F.: Performance degradation in boosting. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 11–21. Springer, Heidelberg (2001)
Wolpert, D.H.: Stacked generalization. Neural Networks 5(2), 241–259 (1992)
Wu, Y., Chang, E.Y., Chang, K.C.-C., Smith, J.R.: Optimal multimodal fusion for multimedia data analysis. In: ACM Multimedia, pp. 572–579 (2004)
Yue, Y., Finley, T., Radlinski, F., Joachims, T.: A support vector method for optimizing average precision. In: SIGIR, pp. 271–278 (2007)
Zhang, T.: Statistical analysis of some multi-category large margin classification methods. JMLR 5, 1225–1251 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Morvant, E., Habrard, A., Ayache, S. (2014). Majority Vote of Diverse Classifiers for Late Fusion. In: Fränti, P., Brown, G., Loog, M., Escolano, F., Pelillo, M. (eds) Structural, Syntactic, and Statistical Pattern Recognition. S+SSPR 2014. Lecture Notes in Computer Science, vol 8621. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44415-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-662-44415-3_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-44414-6
Online ISBN: 978-3-662-44415-3
eBook Packages: Computer ScienceComputer Science (R0)