Abstract
Multi-target regression is a predictive task involving multiple numerical outputs per instance. In the domain of multi-label classification there exist a large number of techniques that successfully model outputs together. Classifier Chains is one such example that is naturally extendable to the multi-target regression task (as Regressor Chains). However, although this method is straightforward to adapt to the regression setting, large improvements over independent models (as seen already in the multi-label classification context over the recent decade) have not as of yet been obtained from Regressor Chains. One of the reasons for this is the adoption of squared-error-based loss metrics which do not require consideration of joint-target modeling. In this paper, we consider cases where the predictive distribution can be multi-modal. Such a scenario, which easily manifests in real-world tasks involving uncertainty, motivates a different loss metric and, thereby, a different approach. We thus present a new method for multi-target regression: Multi-Modal Ensemble of Regressor Chains (mmERC), which performs competitively on datasets exhibiting a multi-modal distribution, both against independent regressors and state-of-the-art ensembles of regressor chains. We argue that such distributions are not sufficiently considered in the regression and particularly multi-target regression literature.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bassett, R., Deride, J.: Maximum a posteriori estimators as a limit of Bayes estimators. Math. Program. 174, 129–144 (2018). https://doi.org/10.1007/s10107-018-1241-0
Burger, M., Lucka, F.: Maximum a posteriori estimates in linear inverse problems with log-concave priors are proper Bayes estimators. Inverse Probl. 30(11), 114004 (2014). https://doi.org/10.1088/0266-5611/30/11/114004
Dembczyński, K., Waegeman, W., Hüllermeier, E.: An analysis of chaining in multi-label classification. In: ECAI: European Conference of Artificial Intelligence, vol. 242, pp. 294–299. IOS Press (2012)
Feng, Y., Fan, J., Suykens, J.A.: A statistical learning approach to modal regression. J. Mach. Learn. Res. 21(2), 1–35 (2020)
Fenga, Y., Huang, X., Shi, L., Yang, Y., Suykens, J.: Learning with the maximum correntropy criterion induced losses for regression. J. Mach. Learn. Res. 16, 993–1034 (2015)
Hendry, A.P., Huber, S.K., León, L.F.D., Herrel, A., Podos, J.: Disruptive selection in a bimodal population of Darwin’s finches. Proc. Roy. Soc. B: Biol. Sci. 276(1657), 753–759 (2008). https://doi.org/10.1098/rspb.2008.1321
Ho, T.K.: Random decision forests. In: Proceedings of 3rd International Conference on Document Analysis and Recognition, vol. 1, pp. 278–282 (1995). https://doi.org/10.1109/ICDAR.1995.598994
Lloyd, S.: Least squares quantization in PCM. IEEE Trans. Inf. Theory 28(2), 129–137 (1982). https://doi.org/10.1109/TIT.1982.1056489
Melki, G., Cano, A., Kecman, V., Ventura, S.: Multi-target support vector regression via correlation regressor chains. Inf. Sci. 415–416, 53–69 (2017). https://doi.org/10.1016/j.ins.2017.06.017
de Mendiburu, F., de Mendiburu, M.: Package ‘agricolae’ (2019). https://cran.r-project.org/package=agricolae
Moyano, J.M., Gibaja, E.L., Ventura, S.: An evolutionary algorithm for optimizing the target ordering in ensemble of regressor chains. In: 2017 IEEE Congress on Evolutionary Computation (CEC), pp. 2015–2021. IEEE (2017)
Paliwal, S., Iglesias, P.A., Campbell, K., Hilioti, Z., Groisman, A., Levchenko, A.: MAPK-mediated bimodal gene expression and adaptive gradient sensing in yeast. Nature 446(7131), 46–51 (2007). https://doi.org/10.1038/nature05561
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Read, J., Martino, L.: Probabilistic regressor chains with Monte Carlo methods. Neurocomputing 413, 471–486 (2020). https://doi.org/10.1016/j.neucom.2020.05.024
Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains: a review and perspectives. J. Artif. Intell. Res. (JAIR) 70, 683–718 (2021)
Spyromitros-Xioufis, E., Tsoumakas, G., Groves, W., Vlahavas, I.: Multi-target regression via input space expansion: treating targets as inputs. Mach. Learn. 104(1), 55–98 (2016). https://doi.org/10.1007/s10994-016-5546-z
Vasconcelos, J.C.S., Cordeiro, G.M., Ortega, E.M.M.: Édila Maria de Rezende: a new regression model for bimodal data and applications in agriculture. J. Appl. Stat. 48(2), 349–372 (2021). https://doi.org/10.1080/02664763.2020.1723503
Waegeman, W., Dembczyński, K., Hüllermeier, E.: Multi-target prediction: a unifying view on problems and methods. Data Min. Knowl. Disc. 33(2), 293–324 (2018). https://doi.org/10.1007/s10618-018-0595-5
Xu, D., Shi, Y., Tsang, I.W., Ong, Y.S., Gong, C., Shen, X.: Survey on multi-output learning. EEE Trans. Neural Netw. Learn. Syst. 31(7), 2409–2429 (2019)
Yao, W., Li, L.: A new regression model: modal linear regression. Scand. J. Stat. 41(3), 656–671 (2014). https://doi.org/10.1111/sjos.12054
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Antonenko, E., Read, J. (2022). Multi-modal Ensembles of Regressor Chains for Multi-output Prediction. In: Bouadi, T., Fromont, E., Hüllermeier, E. (eds) Advances in Intelligent Data Analysis XX. IDA 2022. Lecture Notes in Computer Science, vol 13205. Springer, Cham. https://doi.org/10.1007/978-3-031-01333-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-01333-1_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-01332-4
Online ISBN: 978-3-031-01333-1
eBook Packages: Computer ScienceComputer Science (R0)