Abstract
Random Forests (RF) are a successful ensemble prediction technique that uses majority voting or averaging as a combination function. However, it is clear that each tree in a random forest may have a different contribution in processing a certain instance. In this paper, we demonstrate that the prediction performance of RF may still be improved in some domains by replacing the combination function with dynamic integration, which is based on local performance estimates. Our experiments also demonstrate that the RF Intrinsic Similarity is better than the commonly used Heterogeneous Euclidean/Overlap Metric in finding a neighbourhood for local estimates in the context of dynamic integration of classification random forests.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36(1,2), 105–139 (1999)
Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: Proc. 7th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining KDD 2001, pp. 245–250. ACM Press, New York (2001)
Blake C.L., Keogh, E., Merz, C.J.: UCI repository of machine learning databases. Dept. of Information and Computer Science, University of California, Irvine, CA (1999), http://www.ics.uci.edu/~mlearn/MLRepository.html
Breiman, L.: Bias, Variance, and Arcing Classifiers, Tech. Report 486, Statistics Dept., University of California, Berkeley, USA (1996)
Breiman, L.: Random Forests. Machine Learning 45(1), 5–32 (2001)
Kohavi, R., Wolpert, D.: Bias plus variance decomposition for zero-one loss functions. In: Proc. 13th Int. Conf. on Machine Learning, pp. 275–283. Morgan Kaufmann, San Francisco (1996)
Robnik-Šikonja, M.: Improving random forests. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS, vol. 3201, pp. 359–370. Springer, Heidelberg (2004)
Rooney, N., Patterson, D., Anand, S., Tsymbal, A.: Dynamic integration of regression models. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 164–173. Springer, Heidelberg (2004)
Schaffer, C.: Selecting a classification method by cross-validation. Machine Learning 13, 135–143 (1993)
Tsymbal, A., Pechenizkiy, M., Cunningham, P.: Sequential genetic search for ensemble feature selection. In: Proc. 19th Int. Joint Conf. on Artificial Intelligence IJCAI 2005, pp. 877–882. Morgan Kaufmann, San Francisco (2005)
Tsymbal A., Pechenizkiy M., Cunningham P.: Dynamic integration with random forests. Tech. Report TCD-CS-2006-23, Dept. of Computer Science, Trinity College Dublin, Ireland (2006), available online at: http://www.cs.tcd.ie/publications/tech-reports/reports.06/
Tsymbal, A., Puuronen, S.: Bagging and boosting with dynamic integration of classifiers. In: Zighed, D.A., Komorowski, J., Żytkow, J.M. (eds.) PKDD 2000. LNCS, vol. 1910, pp. 116–125. Springer, Heidelberg (2000)
Wilson, D.R., Martinez, T.R.: Improved heterogeneous distance functions. Journal of Artificial Intelligence Research 6(1), 1–34 (1997)
Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools With Java Implementations. Morgan Kaufmann, San Francisco (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tsymbal, A., Pechenizkiy, M., Cunningham, P. (2006). Dynamic Integration with Random Forests. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds) Machine Learning: ECML 2006. ECML 2006. Lecture Notes in Computer Science(), vol 4212. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11871842_82
Download citation
DOI: https://doi.org/10.1007/11871842_82
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45375-8
Online ISBN: 978-3-540-46056-5
eBook Packages: Computer ScienceComputer Science (R0)