Abstract
Computer-simulated experiments have become a cost effective way for engineers to replace real experiments in the area of product development. However, one single computer-simulated experiment can still take a significant amount of time. Hence, in order to minimize the amount of simulations needed to investigate a certain design space, different approaches within the design of experiments area are used. One of the used approaches is to minimize the time consumption and simulations for design space exploration through response surface modeling. The traditional methods used for this purpose are linear regression, quadratic curve fitting and support vector machines. This paper analyses and compares the performance of four machine learning methods for the regression problem of response surface modeling. The four methods are linear regression, support vector machines, M5P and random forests. Experiments are conducted to compare the performance of tree models (M5P and random forests) with the performance of non-tree models (support vector machines and linear regression) on data that is typical for concept evaluation within the aerospace industry. The main finding is that comprehensible models (the tree models) perform at least as well as or better than traditional black-box models (the non-tree models). The first observation of this study is that engineers understand the functional behavior, and the relationship between inputs and outputs, for the concept selection tasks by using comprehensible models. The second observation is that engineers can also increase their knowledge about design concepts, and they can reduce the time for planning and conducting future experiments.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmed, M., Qin, N.: Comparison of response surface and kriging surrogates in aerodynamic design optimization of hypersonic spiked blunt bodies. In: 13th International Conference on Aerospace Sciences and Aviation Technology, 26–28th May, Military Technical College, Kobry Elkobbah, Cairo, Egypt (2009)
Bell, T.E., Bixler, D.C., Dyer, M.E.: An extendable approach to computer-aided software requirements engineering. IEEE Trans. Softw. Eng. 1, 49–60 (1977)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Carley, K.M., Kamneva, N.Y., Reminga, J.: Response surface methodology. Technical report, DTIC Document (2004)
Chen, K.Y., Wang, C.H.: Support vector regression with genetic algorithms in forecasting tourism demand. Tour. Manag. 28(1), 215–226 (2007)
Couckuyt, I., Gorissen, D., Rouhani, H., Laermans, E., Dhaene, T.: Evolutionary regression modeling with active learning: an application to rainfall runoff modeling. In: Kolehmainen, M., Toivanen, P., Beliczynski, B. (eds.) ICANNGA 2009. LNCS, vol. 5495, pp. 548–558. Springer, Heidelberg (2009)
Crombecq, K., Couckuyt, I., Gorissen, D., Dhaene, T.: Space-filling sequential design strategies for adaptive surrogate modelling. In: The First International Conference on Soft Computing Technology in Civil, Structural and Environmental Engineering (2009)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Freitas, A.A.: Comprehensible classification models: a position paper. SIGKDD Explor. Newsl. 15(1), 1–10 (2013). http://doi.acm.org/10.1145/2594473.2594475
Gorissen, D., Couckuyt, I., Demeester, P., Dhaene, T., Crombecq, K.: A surrogate modeling and adaptive sampling toolbox for computer based design. J. Mach. Learn. Res. 11, 2051–2055 (2010)
Gramacy, R.B., Le Digabel, S.: The mesh adaptive direct search algorithm with treed Gaussian process surrogates. Groupe d’études et de recherche en analyse des décisions (2011)
Gu, H., Yang, L., Hu, Z., Yu, J.: Surrogate models for shape optimization of underwater glider, pp. 3–6, February 2009
Kohavi, R., et al.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI, vol. 14, pp. 1137–1145 (1995)
Nikolos, I.K.: On the use of multiple surrogates within a differential evolution procedure for high-lift airfoil design. Int. J. Adv. Intell. Paradigms 5, 319–341 (2013)
Pos, A., Borst, P., Top, J., Akkermans, H.: Reusability of simulation models. Knowl.-Based Syst. 9(2), 119–125 (1996)
Quinlan, J.R., et al.: Learning with continuous classes. In: Proceedings of the 5th Australian Joint Conference on Artificial Intelligence, Singapore, vol. 92, pp. 343–348 (1992)
Quinn, G.P., Keough, M.J.: Experimental Design and Data Analysis for Biologists. Cambridge University Press, Cambridge (2002)
Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)
Sheskin, D.J.: Handbook of Parametric and Nonparametric Statistical Procedures. CRC Press, Boca Raton (2003)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (2000)
Wang, Y., Witten, I.H.: Inducing model trees for continuous classes. In: Proceedings of the Ninth European Conference on Machine Learning, pp. 128–137 (1997)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2011)
Acknowledgments
This work was supported by the Knowledge Foundation through the research profile grants Model Driven Development and Decision Support and Scalable Resource-efficient Systems for Big Data Analytics.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Dasari, S.K., Lavesson, N., Andersson, P., Persson, M. (2015). Tree-Based Response Surface Analysis. In: Pardalos, P., Pavone, M., Farinella, G., Cutello, V. (eds) Machine Learning, Optimization, and Big Data. MOD 2015. Lecture Notes in Computer Science(), vol 9432. Springer, Cham. https://doi.org/10.1007/978-3-319-27926-8_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-27926-8_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-27925-1
Online ISBN: 978-3-319-27926-8
eBook Packages: Computer ScienceComputer Science (R0)