Abstract
Not all real-world data are labeled, and when labels are not available, it is often costly to obtain them. Moreover, as many algorithms suffer from the curse of dimensionality, reducing the features in the data to a smaller set is often of great utility. Unsupervised feature selection aims to reduce the number of features, often using feature importance scores to quantify the relevancy of single features to the task at hand. These scores can be based only on the distribution of variables and the quantification of their interactions. The previous literature, mainly investigating anomaly detection and clusters, fails to address the redundancy-elimination issue. We propose an evaluation of correlations among features to compute feature importance scores representing the contribution of single features in explaining the dataset’s structure.
Based on Coalitional Game Theory, our feature importance scores include a notion of redundancy awareness making them a tool to achieve redundancy-free feature selection. We show that the deriving features’ selection outperforms competing methods in lowering the redundancy rate while maximizing the information contained in the data. We also introduce an approximated version of the algorithm to reduce the complexity of Shapley values’ computations.
C. Balestra—This research was supported by the research training group Dataninja (Trustworthy AI for Seamless Problem Solving: Next Generation Intelligence Joins Robust Data Analysis) funded by the German federal state of North Rhine-Westphalia.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The first 50 features in the Big Five dataset are the categorical answers to the personality test’s questions and are divided into 5 personalities’ traits (10 questions for each personality trait). To apply the full algorithm, we select questions from different personalities and restrict to 10000 instances.
- 2.
We restrict to the 5000 highest-rated players by the overall attribute.
- 3.
References
Burgess, M.A., Chapman, A.C.: Approximating the Shapley value using stratified empirical Bernstein sampling. In: IJCAI (2021)
Cai, D., Zhang, C., He, X.: Unsupervised feature selection for multi-cluster data. In: KDD (2010)
Castro, J., Gómez, D., Tejada, J.: Polynomial calculation of the Shapley value based on sampling. Comput. Oper. Res. 36, 1726–1730 (2009)
Catav, A., et al.: Marginal contribution feature importance - an axiomatic approach for explaining data. In: ICML (2021)
Cheng, C.-H., Fu, A., Zhang, F.: Entropy-based subspace clustering for mining numerical data. In: KDD (1999)
Cohen, S., Dror, G., Ruppin, E.: Feature selection via coalitional game theory. Neural Comput. 19, 1939–1961 (2007)
Cover, T.M., Thomas, J.A.: Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, Hoboken (2006)
He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: NIPS (2006)
Lazo, A.V., Rathie, P.: On the entropy of continuous probability distributions. Trans. Inf. Theory 24(1), 120–122 (1978)
Li, J., et al.: Feature selection: a data perspective. ACM Comput. Surv. 50, 1–45 (2017)
Li, Z., Yang, Y., Liu, J., Zhou, X., Lu, H.: Unsupervised feature selection using nonnegative spectral analysis. In: AAAI (2012)
Lu, Y., Cohen, I., Zhou, X.S., Tian, Q.: Feature selection using principal feature analysis. In: MM (2007)
Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: NIPS (2017)
Nguyen, H., Müller, E., Vreeken, J., Keller, F., Böhm, K.: CMI: an information-theoretic contrast measure for enhancing subspace cluster and outlier detection. In: SDM (2013)
Pfannschmidt, K., Hüllermeier, E., Held, S., Neiger, R.: Evaluating tests in medical diagnosis: combining machine learning with game-theoretical concepts. In: Carvalho, J.P., Lesot, M.-J., Kaymak, U., Vieira, S., Bouchon-Meunier, B., Yager, R.R. (eds.) IPMU 2016. CCIS, vol. 610, pp. 450–461. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40596-4_38
Reshef, D., et al.: Detecting novel associations in large data sets. Science 334, 1518–1524 (2011)
Rozemberczki, B., et al.: The Shapley value in machine learning (2022)
Shapley, L.S.: A value for N-person games. Contributions to the Theory of Games (1953)
Shekar, A.K., Bocklisch, T., Sánchez, P.I., Straehle, C.N., Müller, E.: Including multi-feature interactions and redundancy for feature ranking in mixed datasets. In: Ceci, M., Hollmén, J., Todorovski, L., Vens, C., Džeroski, S. (eds.) ECML PKDD 2017. LNCS (LNAI), vol. 10534, pp. 239–255. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71249-9_15
Solorio-Fernández, S., Carrasco-Ochoa, J.A., Martínez-Trinidad, J.F.: A review of unsupervised feature selection methods. Artif. Intell. Rev. 53(2), 907–948 (2019). https://doi.org/10.1007/s10462-019-09682-y
Strumbelj, E., Kononenko, I.: An efficient explanation of individual classifications using game theory. J. Mach. Learn. Res. 11, 1–18 (2010)
van Campen, T., Hamers, H., Husslage, B., Lindelauf, R.: A new approximation method for the Shapley value applied to the WTC 9/11 terrorist attack. Soc. Netw. Anal. Min. 8, 1–12 (2018)
Vergara, J.R., Estévez, P.A.: A review of feature selection methods based on mutual information. Neural Comput. Appl. 24(1), 175–186 (2013). https://doi.org/10.1007/s00521-013-1368-0
Wang, S., Tang, J., Liu, H.: Embedded unsupervised feature selection. In: AAAI (2015)
Yang, Y., Shen, H., Ma, Z., Huang, Z., Zhou, X.: \(l_{2,1}\)-norm regularized discriminative feature selection for unsupervised learning. In: IJCAI (2011)
Zhao, Z., Liu, H.: Spectral feature selection for supervised and unsupervised learning. In: ICML (2007)
Zhu, X., Wang, Y., Li, Y., Tan, Y., Wang, G., Song, Q.: A new unsupervised feature selection algorithm using similarity-based feature clustering. Comput. Intell. 35, 2–22 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Balestra, C., Huber, F., Mayr, A., Müller, E. (2022). Unsupervised Features Ranking via Coalitional Game Theory for Categorical Data. In: Wrembel, R., Gamper, J., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Big Data Analytics and Knowledge Discovery. DaWaK 2022. Lecture Notes in Computer Science, vol 13428. Springer, Cham. https://doi.org/10.1007/978-3-031-12670-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-12670-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-12669-7
Online ISBN: 978-3-031-12670-3
eBook Packages: Computer ScienceComputer Science (R0)