Enhancing functional efficiency in information-extreme machine learning with logistic regression ensembles

Oleksandr Papchenko, Borys Kuzikov, Oksana Shovkoplias

Abstract


The subject matter of the article is the application of supervised machine learning for the task of object class recognition. The goal is enhancing functional efficiency in information-extreme technology (IET) for object class recognition. The tasks to be solved are: to analyse possible ways of increasing the functional efficiency IET approach; implement an ensemble of models that include logistic regression for prioritizing recognition features and an IEI learning algorithm; compare the functional efficiency of the resulting ensemble of models on well-known dataset with classic approach and results of other researchers. The methods: The method is developed within the framework of the functional approach to modelling natural intelligence applied to the problem of object classification. The following results were obtained: The study tries to augment existing IET to support feature prioritization as part of the object class recognition algorithm. The classical information-extreme algorithm treats all input features equivalently important in forming the decisive rule. As a result, the object features with strong correlation are not prioritized by the algorithm's decisive mechanism – resulting in decreasing functional efficiency in exam mode. The proposed approach is solving this problem by applying a two-stage approach. In the first stage the multiclass logistic regression applied to the input training features vectors of objects to be classified – formed the normalized training matrix. To prevent overfitting of the logistic regression, a model the L2(ridge) regularization method was used. On the second stage, the information-extreme method as input takes the result of the first stage. The geometrical parameters of class containers and the control tolerances on the recognition features were considered as the optimization parameters. Conclusions. The proposed approach increases MNIST (Modified National Institute of Standards and Technology) dataset classification accuracy compared with the classic information-extreme method by 26,44%. The proposed approach has a 3.77% lower accuracy compared to neural-like approaches but uses fewer resources in the training phase and allows retraining the model, as well as expanding the dictionary of recognition classes without model retraining.


Keywords


supervised machine learning; information-extreme machine learning; machine learning parameter regularization; algorithms ensemble; information criterion; optimization

Full Text:

PDF

References


Jiang, T., Gradus, J. L., & Rosellini, A. J. Supervised machine learning: a brief primer. Behavior therapy, 2020, vol. 51, iss. 5, pp. 675-687. DOI: 10.1016/j.beth.2020.05.002.

Pugliese, R., Regondi, S., & Marini, R. Machine learning-based approach: global trends, research directions, and regulatory standpoints. Data science and management, 2021, vol. 4, pp. 19-29. DOI: 10.1016/j.dsm.2021.12.002.

Sarker, I. H. Machine learning: algorithms, real-world applications and research directions. SN computer science, 2021, vol. 2, no. 3, article no 160, 21 p. DOI: 10.1007/s42979-021-00592-x.

Ciresan D., Meier, U., & Schmidhuber J. Multi-column deep neural networks for image classification. 2012 IEEE conference on computer vision and pattern recognition (CVPR), Providence, RI, 2012, pp. 3642-3649. DOI: 10.1109/cvpr.2012.6248110.

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research, 2014, vol. 15, no. 56, pp. 1929-1958. Available at: http://jmlr.org/papers/v15/srivastava14a.html (accessed: 28.08.2023).

Albawi, S., Mohammed, T. A., & Al-Zawi, S. Understanding of a convolutional neural network. 2017 international conference on engineering and technology (ICET), Antalya, Turkey, 2017, pp. 1-6. DOI: 10.1109/ICEngTechnol.2017.8308186.

Han, D., Liu, Q., & Fan, W. A new image classification method using CNN transfer learning and web data augmentation. Expert systems with applications, 2018, vol. 95, pp. 43-56. DOI: 10.1016/j.eswa.2017.11.028.

Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. Edge computing: vision and challenges. IEEE internet of things, 2016, vol. 3, iss. 5, pp. 637-646. DOI: 10.1109/jiot.2016.2579198.

Yang, Q., Liu, Y., Chen, T., & Tong, Y. Federated machine learning. ACM transactions on intelligent systems and technology, 2019, vol. 10, iss. 2, article no. 12. 19 p. DOI: 10.1145/3298981.

Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., & Ramage, D. Federated learning for mobile keyboard prediction. Available at: https://arxiv.org/abs/1811.03604 (accessed: 28.08.2023). DOI: 10.48550/arXiv.1811.03604

Preuveneers, D., Tsingenopoulos, I., & Joosen, W. Resource usage and performance trade-offs for machine learning models in smart environments. Sensors, 2020, vol. 20, iss. 4, article no. 1176. 27 p. DOI: 10.3390/s20041176.

Bouguettaya, A., Kechıda, A., & Taberkıt, A. M. A Survey on Lightweight CNN-Based Object Detection Algorithms for Platforms with Limited Computational Resources. International Journal of Informatics and Applied Mathematics, 2019, vol. 2, iss. 2, pp 28-44. Available at: https://dergipark.org.tr/en/pub/ijiam/issue/52418/654318. (accessed: 28.08.2023).

Tamburrini, G. The AI carbon footprint and responsibilities of AI scientists. Philosophies, 2022, vol. 7, iss. 1, article no. 4. DOI: 10.3390/philosophies7010004.

Dovbysh, A. S., Shelehov, I. V., Romaniuk, A. M., Moskalenko, R. A., & Savchenko, T. R. Decision-Making support system for diagnosis of breast oncopathologies by histological images. Cybernetics and systems analysis, 2023, vol. 59, iss. 3, pp. 493-502. DOI: 10.1007/s10559-023-00584-0.

Dovbysh, A. S., & Piatachenko, V. Y. Hierarchical clustering approach for information-extreme machine learning of hand brush prosthesis. Proceedings of the 5th international conference on computational linguistics and intelligent systems, Kharkiv, 2021, pp. 1706-1715.

Naumenko, I., Myronenko, M., & Savchenko, T. Information-extreme machine training of on-board recognition system with optimization of RGB-component digital images. Radioelectronic and computer systems, 2021, no. 4, pp. 59-70. DOI: 10.32620/reks.2021.4.05.

Dovbysh, A., Liubchak, V., Shelehov, I., Simonovskiy, J., & Tenytska, A. Information-extreme machine learning of a cyber attack detection system. Radioelectronic and computer systems, 2022, no. 3, pp. 121-131. DOI: 10.32620/reks.2022.3.09.

Dovbysh, A., Shelehov, I., Prylepa, D., & Golub, I. Information synthesis of adaptive system for visual diagnostics of emotional and mental state of a person. Eastern-European Journal of Enterprise Technologies, 2016, vol. 4, iss. 9(82), pp. 11-17. DOI: 10.15587/1729-4061.2016.75683.

Dovbysh, A., Shelehov, І., Pylypenko. S., & Berest, O. Estimation of Informativeness of Recognition Signs at Extreme Information Machine Learning of Knowledge Control System. CEUR Workshop Proceedings, 2019, article no. 2362. 10 p.

Deng, L. The MNIST database of handwritten digit images for machine learning research. IEEE signal processing magazine, 2012, vol. 29, iss. 6, pp. 141–142. DOI: 10.1109/msp.2012.2211477.

Mehta, P., Bukov, M., Wang, C.-H., Day, A. G. R., Richardson, C., Fisher, C. K., & Schwab, D. J. A high-bias, low-variance introduction to Machine Learning for physicists. Physics reports, 2019, vol. 810, pp. 1-124. DOI: 10.1016/j.physrep.2019.03.001.

Ng, A. Y. Feature selection, L1 vs. L2 regularization, and rotational invariance. Proceedings of the twenty-first international conference on Machine learning (ICML '04), New York, NY, USA. 78 p. DOI: 10.1145/1015330.1015435.

Baldominos, A., Saez, Y., & Isasi, P. A Survey of Handwritten Character Recognition with MNIST and EMNIST. Applied Sciences, 2019, vol. 9, iss. 15, article no. 3169. DOI: 10.3390/app9153169.

Islam, M. A., & Iacob, I. E. Manuscripts Character Recognition Using Machine Learning and Deep Learning. Modelling, 2023, vol. 4, iss. 2, pp. 168-188. DOI: 10.3390/modelling4020010.




DOI: https://doi.org/10.32620/reks.2023.4.06

Refbacks

  • There are currently no refbacks.