Abstract
This research introduces an innovative hybrid intelligence framework leveraging multimodal data to automate the evaluation of core soft skills, including decision-making, conflict resolution, and creativity. The model applies the principles of the Granular Linguistic Model of Phenomena (GLMP), a sophisticated method that delineates phenomena at varying granularity levels, ensuring a detailed analysis of the exhibited skills. The process involves mining significant behavioural features from diverse data sources, specifically video, audio, and text, employing deep learning algorithms. The extracted features are then subjected to the GLMP, representing the students’ behaviour in a structured, interpretable format across multiple granularities. The GLMP application yields an exhaustive set of granular linguistic prompts that encapsulate the complexity of the identified soft skills. This multimodal information feeds into a fuzzy logic-based detector that evaluates the defined soft skills. This integrative approach merges granular linguistic modelling with multimodal data, enabling a comprehensive and accessible understanding of the students’ soft skills. The implications of this approach extend beyond the academic sphere, finding utility in broader contexts such as college admissions and job recruitment, where objective skill evaluation is crucial. This research underscores the value of multimodal integration within the GLMP framework, highlighting its critical role in translating raw data into actionable insights. It further illuminates the potential of such methods in enhancing real-world decision-making processes and outcomes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akoa, B.E., Simeu, E., Lebowsky, F.: Using statistical analysis and artificial intelligence tools for automatic assessment of video sequences. In: Eschbach, R., Marcu, G.G., Rizzi, A. (eds.) Color Imaging XIX: Displaying, Processing, Hardcopy, and Applications, vol. 9015, p. 90150O. International Society for Optics and Photonics, SPIE (2014). https://doi.org/10.1117/12.2044797
de Anda-Trasviña, A., Nieto-Garibay, A., Gutiérrez, J.: Natural language report of the composting process status using linguistic perception. Appl. Soft Comput. 127, 109357 (2022)
Conde-Clemente, P., Alonso, J.M., Trivino, G.: Toward automatic generation of linguistic advice for saving energy at home. Soft. Comput. 22(2), 345–359 (2018)
Giannakakis, G., et al.: Stress and anxiety detection using facial cues from videos. Biomed. Signal Process. Control 31, 89–101 (2017). https://doi.org/10.1016/j.bspc.2016.06.020
Kaehler, A., Bradski, G.: Learning OpenCV 3 - Computer Vision in C++ with the OpenCV Library. O’Reilly Media, Inc. (2016)
Losada, D.E., Gamallo, P.: Evaluating and improving lexical resources for detecting signs of depression in text. Lang. Resour. Eval. 54(1), 1–24 (2020). https://doi.org/10.1007/s10579-018-9423-1
Min, Q., Zhou, Z., Li, Z.: An approach to automatic evaluation of instructional videos. In: 2021 the 5th International Conference on Management Engineering, Software Engineering and Service Sciences, pp. 63–68. ICMSS 2021, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3459012.3459022
Nayak, S., Kumar, S., Agarwal, D., Parikh, P.: AI-enabled personalized interview coach in Rural India. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds.) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium, pp. 89–93. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-11647-6_15
Radford, A., et al: Introducing Whisper (2022). https://openai.com/blog/whisper/
Solé-Beteta, X., Navarro, J., Gajšek, B., Guadagni, A., Zaballos, A.: A data-driven approach to quantify and measure students’ engagement in synchronous virtual learning environments. Sensors 22(9) (2022). https://doi.org/10.3390/s22093294
Sun, Y., Nomiya, H., Hochin, T.: Automatic evaluation of motion picture contents by estimation of fgacial expression intensity. In: 2019 20th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), pp. 227–232 (2019). https://doi.org/10.1109/SNPD.2019.8935660
Trivino, G., Sanchez, A., Montemayor, A.S., Pantrigo, J.J., Cabido, R., Pardo, E.G.: Linguistic description of traffic in a roundabout. In: International Conference on Fuzzy Systems, pp. 1–8. IEEE (2010)
Acknowledgements
The Spanish Government has partially supported this work under the grant SAFER: PID2019-104735RB-C42 (ERA/ERDF, EU), and project PLEC2021-007681 funded by MCIN/AEI /10.13039/501100011033 and by the European Union NextGenerationEU/ PRTR.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Guerrero-Sosa, J.D.T., Romero, F.P., Menendez, V.H., Serrano-Guerrero, J., Olivas, J.A., Montoro-Montarroso, A. (2023). Granular Linguistic Model Based Multimodal Data Integration for Automated Evaluation of Core Soft Skills. In: Bravo, J., Urzáiz, G. (eds) Proceedings of the 15th International Conference on Ubiquitous Computing & Ambient Intelligence (UCAmI 2023). UCAmI 2023. Lecture Notes in Networks and Systems, vol 842. Springer, Cham. https://doi.org/10.1007/978-3-031-48642-5_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-48642-5_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48641-8
Online ISBN: 978-3-031-48642-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)