Analysis of Competency Assessment of Educational Innovation in Upper Secondary School and Higher Education: A Mapping Review

Abstract

Despite the plethora of studies reported during the last decade in relation to educational innovation in teaching and assessment of competencies, a consensus is seemingly lacking on a definition that establishes the scope and boundaries competency assessment. This research gap motivated a systematic review of the literature published on the topics of “educational innovation in teaching” and “assessment of competencies” in upper secondary and higher education during the period from January 2016 to March 2021. The main objective of the study was to define and evaluate educational innovation in teaching and assessment of competencies in upper secondary and higher education following PRISMA guidelines for a systematic literature review (SLR) on a curated corpus of 320 articles. We intended to answer the following questions: (1) What do “educational innovation in teaching” and “assessment of competencies” represent for upper secondary and higher education? (2) How are they evaluated? Lastly, (3) are efforts exerted toward the standardization of transversal competencies? The SLR seeks answers to these questions by examining nine research sub-queries. The result indicated that the greatest effort toward educational innovation in competencies was made at the higher education level and targeted students. Competencies were revised through associations with the Sustainable Development Goals of the 2030 Agenda. In addition, the methodologies used for teaching and evaluation of competencies were reviewed. Finally, the study discussed which technologies were used to develop the proficiencies of students.

1. Introduction

The 17 Sustainable Development Goals (SDGs) of the United Nations 2030 Agenda are an integrated framework of human, social, and environmental development objectives developed by member states of the United Nations [1]. One of these objectives is access to and quality of education (Objective 4). Through their role in human formation, knowledge production, and innovation, universities play an essential role in achieving SDGs. Universities have endeavored to include interdisciplinary and sustainability principles in their curricula [2] and have created tools for measuring key sustainability competencies [3]. At its 40th Session in Paris 2019, the Executive Board of the United Nations Educational, Scientific and Cultural Organization (UNESCO) proposed a new framework of Education for Sustainable Development for 2030, which aims to build a just and sustainable world using a threefold approach: (a) individual transformation, (b) societal transformation, and (c) technological advancement [1]. Universities have also attempted to measure success in delivering SDGs [4]. The Times Higher Education Impact Rankings, which was introduced in 2019 to measure the social and economic impacts of institutions, has recognized the contributions of more than 700 universities worldwide in tackling the most significant global challenges [5]. Although an increasing number of universities are aligning their activities with SDGs, a need has emerged to document the wide variety of educational innovation activities relevant to sustainable development being undertaken by universities.

Reforming higher education without reforming secondary education is futile; higher education has a long history of being sluggish in absorbing advances for various reasons [6]. Moreover, enhancing the productivity of higher education is particularly challenging because it is complex (due to the cohesiveness and continuity of science) and labor-intensive [7]. Conversely, upper secondary schools are much more restrictive than colleges because they prioritize the health and safety of students over their training for real life and employment [8]. In a similar manner, a key issue emerged with the observation that upper secondary schools and higher education systems are relatively dissociated from one another from the point of view of student evaluation. For instance, in secondary education, students are disconnected from the rest of the world, and a significant gap exists between performance in upper secondary school as assessed by former upper secondary students and outcomes of college admission exams [9]. In other words, obtaining impeccable scores in admission exams is not guaranteed, in principle, for a student who is perceived to obtain good grades in the majority of subjects. Another example is that attendance is mandatory in upper secondary schools but rarely taken in lectures at universities; nonetheless, attendance is a major predictor of student success at both educational levels. Similarly, assignments in upper secondary schools are designed to be limited due to their objective of conducting a straightforward assessment, such that students can perform an activity or a subject. Consequently, assignments help students make decisions about which areas they will like to further explore when they eventually enroll in a college major. In contrast, once a student is pursuing a college major or starting a professional career, deciding on a subject from a curriculum or applying to an industry sector leads to greater consequences [10,11]. In addition, assessments in higher education vary and are designed to stimulate critical thinking, creative thinking, and autonomous learning. These assessments also imitate scenarios similar to professional conditions, and the majority of studies in this field are conducted outside classrooms; therefore, assignments in higher education result in higher-order thinking. In the same manner, objectives differ between higher education and upper secondary school because the objective of the first is to offer education that enables graduates to join the economic sector or labor activities, whereas the second prioritizes giving students the necessary foundations to be eventually incorporated into universities as an intermediate step prior to entering the professional world. For these reasons, analyzing studies on educational innovation and evaluation of these innovations at both educational levels is important. Institutions in various countries offer education to students at both educational levels. These institutions design educational programs where students are given continuity in their planning at these two levels. Similarly, although the two educational levels pursue different objectives for students, they are linked through students, who, after finishing one educational level, go on to the other. Consequently, strengths or deficiencies presented by students may influence their performance at a later educational level.

Technological changes in artificial intelligence, machine learning, big data, data mining, and augmented reality, among others, are exerting significant impacts on various fields, including education [12]. In the digital era, humans live in an environment filled with technology and easy access to extensive information. If humans continue to employ the present paradigm at work, then they will undoubtedly lose to artificial intelligence [13]. Therefore, the paradigm must shift; that is, schools must be able to provide students with intellectually stimulating experiences and opportunities for them to develop 21st-century competencies [14]. The reason for this notion is that robots (artificial intelligence) will lack certain abilities such as critical thinking, problem-solving, communication, and collaboration. Universally applicable practices are being implemented and evaluated in various educational fields [15], and specialized educational programs are being designed using these technologies [16]. Furthermore, recent technological advances have modified learning dynamics and teaching methods, which generated new challenges in educational systems such as an increased focus on informal education, a widening gap in prior knowledge, and disparity among education and training, available career options, and workforce development [17]. Due to the current methodologies for education delivery, universities have endeavored to make changes in their teaching processes. Although education plays a vital role in developing human innovation skills, several studies have suggested that institutions of higher education cannot fulfill this demand [9]. As such, a need has emerged to update pedagogical practices and to develop assessment tools for measuring and develop the innovation capacity of individuals [18]. In this vein, an evident notion is that creating a school culture of innovation maximizes instruction, enhances learning, and increases student achievement [19].

Curricular efforts were exerted to develop transversal competencies. For example, the Tecnologico de Monterrey in Mexico recently implemented a new model named TEC21. This educational model incorporates seven transversal competencies into all undergraduate curriculum programs [20]. Over the last years, a rapid increase in the number of publications regarding the evaluation of sustainability competencies were observed [21]. UNESCO identified eight core sustainability competencies, namely, self-awareness, strategic, critical thinking, integrated problem-solving, collaboration, normative, system thinking, and anticipatory [22]. In addition, the OECD listed skills that have also been outlined in the labor sector and prioritized as the most relevant ones that people should possess, which are, therefore, considered future work skills [23,24]. These are skills, attitudes, values (communication/multi-literacies, cooperation/collaboration, critical thinking, problem solving, empathy, respect, and persistence/resilience), frameworks of learning concepts (student agency, creative thinking, responsibility, and conflict resolution), cycles of competency development (i.e., anticipation, action, and reflection), and complex competencies (global competency and literacy for sustainable development, entrepreneurship/enterprising, digital literacy, and computational thinking/coding). Although the abovementioned efforts elucidate or define core competencies relevant to educational innovation, literature review studies that demonstrate if such competencies are the focus of educational practices are lacking.

Moreover, a lack of consensus in the literature exists in relation to the definition of “educational innovation.” For instance, the definition of the term educational innovation as the implementation of information technologies in education is common. However, a search in the Education Resources Information Center (ERIC) Thesaurus [25] for educational innovation produces various keywords associated with the term such as educational change, educational development, and educational improvement. In essence, five years have passed since López and Heredia defined educational innovation in 2017 [26] as the implementation of a significant change in the teaching–learning process in terms of materials used, methods of delivery of sessions, and contents or contexts that imply teaching. As a result, the perceived difference may be related to the quality or novelty of the improved element, as well as the contribution of value to the teaching–learning process [26]. Nevertheless, verifying and modernizing this concept are necessary steps, such that it can be adjusted and aligned with the 2030 mission of the Tecnologico de Monterrey and the SDGs of the United Nations. From the perspective of relating the SDGs to educational programs, these SDGs are beginning to be incorporated an in intensive manner (see [27,28,29,30,31,32,33] for examples). Moreover, this systematic review study intends to contribute to the elucidation of the process and method of implementing educational innovation in teaching and evaluation of competencies at the upper secondary and higher education levels. Therefore, based on literature published from 2016 to March 2021, the study focused on reviewing the current definition of educational innovation in upper secondary and higher education in terms of competency development and assessment. Moreover, we searched for a framework for unifying the definition and understanding of competencies.

The objectives of this systematic literature review (SLR) through mapping were to understand the characteristics of related studies, frequently examined research topics, types of variables and factors, and educational innovations conducted in the field. To answer these questions, we dissected the original knowledge that, in turn, can enable the development of a theoretical framework to provide better guidance for trainers, researchers, and decision-makers. The objective of this work was to support structural changes in education required by the 2030 Agenda.

The remainder of the paper is structured as follows. Section 2 outlines the methodology and guidelines that underpin this systematic review of the literature. Section 3 provides an analysis and interpretation of metadata regarding the 320 documents analyzed through graphs and tables. Section 4 presents a comparison of studies on transversal competencies to establish the perspective of the study information and describes the limitations of this work. Finally, Section 5 concludes and describes the major observations regarding the three research questions and areas of opportunity for future work.

2. Materials and Methods

An SLR is a review of published articles on a certain topic where a predefined methodology is followed to synthesize available evidence. Webster and Watson dictionary (2002) mention that an SLR must consist of defined and planned steps with clear objectives. In this review, a realization of research questions is conducted to obtain a structured summary and to identify patterns or trends. In the field of higher education, many SLRs that were recently published follow a similar process [34,35]. Nonetheless, the authors mention that a described methodology can also be conducted with synthesized steps that cover the same activities [34,35].

In the current study, the process began with the definition of a review protocol that specifies the research questions to be addressed and the methods used to conduct the review. After collecting and filtering articles of interest, the next step was to analyze and compare the evidence to identify patterns or trends in the presented questions or the lack of consensus in the literature. Given previous methods used for SLR [36], the current study designed a six-step workflow as follows:

• Verification of the existence of available literature in the field of interest and review protocol;
• Definition of search strategies;
• Documentation of the search strategy;
• Definition of inclusion and exclusion criteria;
• Identification of aspects to examine in each selected article;
• Synthesis of the findings (study’s discussion).

2.1. Verification of Available Literature and Review Protocol

The first step was to verify the available literature and define the research protocol. The research criteria were defined in papers published during the last five years (January 2016 to March 2021) in the English language. Furthermore, the main keywords and types of document (scientific articles) were defined to limit the database search.

The research objectives of the review study were defined as follows:

Objective I: How is educational innovation being defined in upper secondary and higher education in terms of competencies?

Objective II: How are educational innovation projects evaluated to develop competencies?

Objective III: Is there a reference framework to unify the understanding of competency development?

Objective IV: In studies on educational innovation, has the link between education and SDGs been established?

2.2. Documentation of Search Strategies and the Inclusion/Exclusion Criteria

As a technical language, educational innovation according to the ERIC Thesaurus [25] refers to an association or relationship among various terms as follows:

• Educational Change;
• Educational Development;
• Educational Environment;
• Educational Improvement;
• Educational Research;
• Educational Technology;
• Educational Assessment;
• Experimental Colleges;
• Experimental Curriculum;
• Experimental Schools;
• Experimental Teaching;
• Nontraditional Education;
• Research and Development.

The above list of numerous words also provides additional evidence and criteria on the subject that no consensus exists on the essence of “educational innovation.” Therefore, this study intended to use a set of criteria for the search keywords to find as many primary studies as possible related to the research questions using an unbiased strategy. Toward this end, we opted to use only the keywords “educational innovation” and “assessment” to simplify the search. Each term was reviewed in the Scopus and Web of Science (WoS) databases in scientific articles published in the last five years (

Table 1. Inclusion, exclusion, and quality criteria for the mapping systematic review study.

Inclusion Criteria	Exclusion Criteria	Quality Criteria
Studies on educational innovation for competency assessment in higher education and upper secondary education in WoS and Scopus databases	Studies that do not address educational innovation for competency assessment in higher education and upper secondary education	Contribution of valuable information on educational innovation for competency assessment in higher education and upper secondary education
Scientific articles	Proceedings, review articles, book chapters, and books and ESCI	Consistency between objectives, applied methods, and results
Scientific articles with a DOI registry	Scientific articles with a DOI registry
Scientific articles in written in the English language	Scientific articles are not written in the English language
Articles published in the years 2016 to March 2021	Articles not published in the years 2016 to March 2021

). The search string only returned full articles and excluded review articles, books, book chapters, and proceedings. The selected scientific articles were in written in the English language.

2.3. Identification of Aspects to Examine in Each Selected Article and Data Synthesis

The documents of the obtained metadata featured the following characteristics: scientific articles that included the topics “educational innovation” and “assessment” in the title, abstract, or keywords; published between January 2016 and March 2021; and written in the English language. Furthermore, documents that were book chapters, review articles or articles published in conference proceedings, works published in languages apart from English, and works published before January 2016 were excluded (

Table 2. Defined search string.

Keywords Used	Search String	Language	Time Frame	Types of Documents	Databases
educational innovation + assessment	TITLE-ABS-KEY (educational) AND TITLE- ABS-KEY (innovation) AND TITLE- ABS-KEY (assessment)	English	2016–2021	Articles, review articles, proceedings, book chapters, and books	Scopus and WoS

). Once filters were applied and the search chain was defined, we strategically analyzed the summaries, results, discussion, and conclusions of the selected articles to facilitate and expedite data collection using an information organizer (e.g., databases, tables, and figures).

In total, 829 documents were obtained from the Scopus database and 626 from WoS. Data extracted from the search engines were name(s) of author(s), titles, DOI, abstracts, and keywords. The two lists were then combined in Excel, and repetitive articles were omitted, which led to a total of 765 articles (Scopus: 486; WoS: 343). Finally, a review of all abstracts was performed to eliminate articles that were irrelevant to the objectives of this study, such as articles that did not refer to upper secondary or higher education or were not primarily focused on the theme of educational innovation.

The PRISMA methodology [37,38,39] includes a diagram (Figure 1) that presents the inclusion and exclusion criteria designed for this study. The final metadata sample consisted of 320 studies. A list of the 320 studies analyzed (metadata) is available in the Supplementary Materials.

Table 3. Research questions.

Research Questions	Answers
RQ1: Is the research study focused on learning or evaluating a competency?	A. Learning a competence. B. Measuring a competence.
RQ2: Which competence was being evaluated or taught? RQ2B: How is the competence standardized according to UNESCO?	(for RQ2B) A. Critical and innovative thinking. B. Interpersonal skills. C. Intrapersonal skills. D. Global citizenship E. Media and information literacy.
RQ3: Who benefits from the study (who is the study population)?	A. Students. B. Professors. C. Both. D. Alumni.
RQ4: If the participants in RQ3 are students, then at which educational level are they enrolled?	A. Highschool. B. Undergraduate and graduate. C. Both.
RQ5: What type of educational methodology is applied in the teaching study?	A. PBL (Project-Based Learning). B. CBL (Challenge-Based Learning). C. LOtC (Learning Outside the Classroom). D. SBL (Society-Based Learning). E. Learning by doing F. etc.
RQ6: Which SDG is related to the educational innovation study?	1. End poverty. 2. Zero hunger. 3. Health and wealth. 4. Quality education. 5. Gender equality. 6. Clean water and sanitation. 7. Affordable energy and non-contaminants. 8. Decent work and economic growth. 9. Industry, innovation, and infrastructure. 10. Reduction of inequalities. 11. Sustainable cities and communities 12. Production and responsible consumption. 13. Climate action. 14. Submarine life. 15. Terrestrial ecosystem life. 16. Peace, justice, and solid institutions. 17. Partnerships to achieve objectives.
RQ7: How is the evaluation of educational innovation being carried out?	A. Self-evaluation. B. Peer evaluation of the participants. C. Through an examination (external/institutional accreditation).
RQ8: How many times was the evaluation carried out? (How many interventions does the measuring instrument have)?	A. One time only. B. Periodic/partial performance evaluations. C. Evaluation by academic period. D. Annual evaluation.
RQ9: Was the technology used in the study imported from another field?	A. No technology was used. B. It was created for education but can be exported to other fields. C. It was created/used in another field but was imported for education. D. It was created for education, and it can only be used for education.

presents the nine sub-queries used to elucidate the three main research objectives discussed in step one of this method. The goals were to determine patterns or trends in educational innovation in upper secondary and higher education and to identify the characteristics of articles, findings, digital technologies, types of designs, and lines of educational innovation that contributed to such trends. Therefore, each article was reviewed in the context of these questions, which was part of the meta-analysis (data available in the Supplementary Materials, Table S1).

The nine research questions emerged from preliminary bibliographic searches conducted by the authors and the perceived challenges (and benefits) that they present in the application of the methodology in terms of conducting the studies and the evaluation of activities by students and research groups. Alternatively, the answers were derived from the theoretical support on which the study was based. The research motivation was based on the opportunity to contribute to new methods for examining educational innovation.

3. Results

The questions shown in Table 3 were answered for each scientific article in the metadata obtained. Next, we searched for patterns or trends in educational innovation by correlating the results obtained for two or more sub-queries. For example, when we analyzed the results for RQ1 (Is the research study focused on learning or evaluating a competency?) and RQ3 (Who benefits from the study or who is the study population?), we concluded that the majority of studies in the literature focus on developing or learning competency among students. Similarly, when the results of RQ1 and RQ4 (If the participants in RQ3 are students, then at which educational level are they enrolled?) are compared, we found that the majority of the studies that focus on developing or learning a competency were performed at the higher educational level. Figure 2A,B plot these results.

Figure 2A indicates that articles that focus on students developing or learning a competence represented nearly 41% of all studies. In contrast, approximately 22% of the studies focused on measuring or evaluating a learned competence. The second group by frequency were studies where the populations are professors and students: 9% for developing/learning a competence and slightly higher for measuring competence (approximately 11%). A few studies tackled the problem of developing/learning competence or measuring a competence among professors only (6% and 7%, respectively).

Furthermore, Figure 2B depicts the distribution of studies that focus on developing/learning a competence or measuring a competence according to levels of education: Higher Education, Upper Secondary Education, other (administrative areas), or both. A predominant part of the studies focused on developing/learning a competence and measuring a competence in higher education (52% and 34%, respectively). The least number of studies were conducted in populations that include managers, administrators, and staff, among others, in higher education (other). Moreover, Figure 2B presents that studies conducted in upper secondary education, at both educational levels and in administrative areas of educational institutions, represent approximately 8% as a whole. Thus, the majority of studies conducted in academia have been developed in higher education.

Figure 3 illustrates the results obtained for RQ2 (Which competence was being evaluated or taught?) and RQ2B (How is competence standardized according to UNESCO?). The figure is a word cloud chart that displays the most frequently presented competencies in the literature. The study observed that a significant variety of competencies have been examined. However, such competencies were frequently present in studies related to medicine/health, digital competence, assessment skills, applications, and learning, among others.

In addition, the current study demonstrated that no consensus exists in the literature regarding the classification of competencies. In other words, no common taxonomy is used to organize various competencies in a standardized manner. Nevertheless, it was observed that academia, international organizations, and public policy managers in government are beginning to focus on key basic competencies that are global and, therefore, transferable [24,40]. Studies that analyzed competencies required in the labor field projected cognitive and learning abilities, yet programs that involve mobility become attractive when considering employability, management, and career development such as life skills [24].

UNESCO catalogs a set of competencies in education [41,42]; consequently, we homologated the competencies obtained in our metadata with those of UNESCO (Figure 3B) [41,42]. Figure 3B highlights the obtained results for RQ2B. The set of competencies associated with critical and innovative thinking represents 31% of our metadata. In total, 21% of the studies classify competencies into media and information, whereas those grouped under interpersonal and intrapersonal skills are represented in 18% and 17% of the studies, respectively. The least represented group of competencies was global citizenship. Notably, the sum of all studies represented in the graph does not add up to 100% because several studies incorporated more than one competence. As a result, these studies were quantified as the percentage for both competencies.

In addition, we analyzed the results obtained for RQ2B according to the academic level described in RQ3 (Figure 4). The study found that the frequency of competencies previously analyzed in Figure 3B was maintained when accounting for academic level. Again, the studies were conducted in higher education (blue bars). At a much-reduced level, studies on competencies were conducted at the upper-middle level (orange bars), whereas studies at both levels simultaneously (gray bars) were nearly non-existent. The last result is expected because implementing an educational innovation study to teach skills and assess them at both educational levels entails high levels of difficulty due to the disparity of curricular plans and the degree of maturity of students. Furthermore, the distribution of the number of studies that the various groupings of competencies consider is similar to those developed at upper secondary education. In the same manner, pointing out that the sum of the percentages presented in Figure 4 is 97.87% is important because the remaining 2.13% is associated with studies that were conducted on students (the sample of studies corresponds to managers and administrative personnel, among others).

Figure 5 plots the results for RQ2B and RQ3; thus, it correlates the standardized group of competencies according to UNESCO [41,42] by participants (students, professors, or both). Again, the most represented participant group in all studies is that of students.

The sum of all gray (students), orange (teachers), and blue (simultaneous application of students and teachers) bars does not add up to 100% because the remaining percentage is associated with studies where the sample of the subject was directors and administrators, among others. The study observed in detail that when accounting for participants in the study, the result of frequencies of standardized competencies in the metadata is similar to those in Figure 3B. The competence that binds critical and innovative thinking skills is the most studied, followed by media and information. Moreover, competencies that fall under global citizenship are the least examined.

The results for RQ5 (What type of educational methodology is applied in the teaching study?) and RQ7 (How is the evaluation of educational innovation being conducted?) were plotted in the three-dimensional graph in Figure 6. Thus, analysis considers the correlation between educational methods and evolution approaches in the metadata.

Evidence exists of a significant effort to develop innovative educational methodologies, such as learning by doing (15.8%), e-learning (10.8%), project-based learning (PBL; 8.6%), society-based learning (7.2%), learning outside the classroom (LOtC; 6.5%), challenge based learning (6.3%), assessment (5.4%), and active learning (4.7%). However, 34.7% of the studies did not report using an innovative educational method or focused only on assessment. Under assessment, we found several reported approaches: self-evaluation (students assesses their progress or achievements in the class), peer evaluation (students assess the progress of peers classmates or group members), roundtable discussions (group or individual public presentations, as well as seminars), interviews (assessment is conducted in the form of a formal conversation between a teacher and a student), and institutional exams (traditional form of a written exam). The study did not find the same level of innovation for the assessment of competencies. As Figure 6 shows, the most used method for assessment continues to the traditional one, which is based on tests or exams (external or institutional). Although new assessment methods, such as self- or peer evaluation, roundtable discussions, or interviews can be found, many studies use examination for assessment. Studies that did not report any innovation in educational methods (NA in Figure 6) lacked any innovation in assessment methods. Nevertheless, the examination is also a dominant approach in studies that report learning by doing (50%), project-based learning (53%), and learning outside the classroom (58%) as educational methods. The innovation or diversification of the assessment of competencies was most visible in e-learning methods, where only 35% of the studies used examination, and the rest used self-evaluation (18.7%), external evaluation (18.7% studies), peer evaluation (6%), and research-based evaluation (2%). Moreover, 19.6% of the studies related to e-learning did not report the assessment approach.

Subsequently, to address RQ6 (Which SDG is related to the study on educational innovation?), we designed a Sankey diagram with the correlation between study subjects (samples of the research), academic level where the studies were implemented, and the relationship of competencies examined in relation to SDGs (Figure 7).

We found that many of the studies were conducted on students (72%) followed by a non-negligible number of studies directed at teachers (25%). Notably, a few of the articles targeted other segments of the population, as indicated by the greenish rectangle in the lower-left corner of the Sankey graph (approximately 3%). Unsurprisingly, the majority of studies were conducted in higher education (88%). Alternatively, the proportion of studies related to upper secondary education is lower (4%) than that associated with other contexts (5.5%).

Another notable aspect is that 30% of the studies addressed SDG3 (good health and well-being). This finding can presumably be associated with the fact that medicine has been the discipline that pioneered competency-based learning. However, interest in addressing other SDGs seems to be emerging. For clarity, we divided the SDGs into two columns in the Sankey chart. The right panel presents a set of SDGs with only one article that used theme as reference. The other SDGs (left column) were referred to in two or more articles. Thus, future studies should focus on developing awareness about SDGs, which are currently under-represented in education, and should produce instruments for including SDGs in the body of knowledge taught at all levels of education. Moreover, another important point is that out of the studies derived from our metadata, Figure 7 points to the distribution of such studies in relation to SDGs, where Objective 4 (quality education) was excluded because all studies were related to it.

Finally, we related the information obtained for RQ7 (How is the evaluation of educational innovation being conducted?) with RQ8 (How many times was evaluation carried out? or How many interventions does the measuring instrument have?), and RQ9 (Was the technology used in the study imported from another field?).

Table 4. Percentage of distribution of answers from scientific articles obtained for RQ7, RQ8, and RQ9.

RQ7. How Is the Evaluation of Educational Innovation Being Conducted?	Percentage (%)
Through an examination (external/or institutional accreditation)	45.24
Student evaluates himself	13.74
Through an external examination	8.24
Peer evaluation of the participants	6.76
Not defined	6.76
Other	19.26
RQ8. How Many Times Was Evaluation Conducted? (How Many Interventions Does the Measuring Instrument Have?)	Percentage (%)
One time only	48.63
Evaluation by academic period	22.04
Periodic/partial performance evaluations	14.77
Not defined	9.31
Annual evaluation	3.40
Other	1.85
RQ9. Was the Technology Used in the Study Imported from Another Field?	Percentage (%)
No technology was used	33.54
It was created/used in another field, but was imported for education	32.05
It was created for education, and it can only be used for education	20.72
Other	13.69

summarizes the distribution of answers obtained for these questions.

Thus far, the preferred method for evaluating educational innovation is through examinations (internal or external), which represents nearly half of the selected articles (45.24%). The second most frequent method was self-evaluation (13.74%) followed by external evaluation (8.24%). The category “Others” (Table 4) represents options such as roundtable discussions, interviews, use of performance indicators, public demonstration, taxonomy of educational processes, and surveys based on previous research, among others. Additionally, the results for RQ8 suggest that evaluation was performed only once in the majority of articles (48.63%). This result is relatively important because the conclusions and results of educational innovation were reported after only one intervention using a measuring instrument in nearly half of the articles. The second and third options correspond to evaluations by academic period (22.02%) and periodic/partial evaluations (14.77%). Moreover, the results for RQ9 reveal that the context of innovation reported does not rely on technology (35.68%) or that technology was imported from other fields (34%). Only 22% of the cases was the technology created exclusively for education. The category “Others” represents options with less than 4% such as cellular, virtual reality software, videos, repositories, information and communication technologies, and board games, among others.

4. Discussion

4.1. Discussion of the Study

The analysis shed new light on the perception of research studies about future skills and competencies that different institutions identified as important. In other words, although no defined global consensus currently exists on which soft or transferable skills must be taught at upper secondary school and higher education, several reference studies previously outlined certain relevant competencies that can be associated with and prioritized in teaching. For example,

Table 5. Relation of the results of the present review with certain competencies and skills addressed in other organizations or studies.

TEC21 Competencies [20]	Future Work Skill [40]	Definition and Selection of Key Competencies [24]	Classification Competences (Research Question Q2B) [41,43]	Sustainable Competencies [22]	Percentages of Analyzed Studies in This Review
Self-knowledge and management.	Intellect management.	The ability to form and conduct life plans and personal projects.	Intrapersonal Skills.	Self-awareness.	20.31%
Innovative entrepreneurship.	Not considered.	Not considered.	Critical and Innovative Thinking.	Strategic.	33.12%
Reasoning to address complexity.	Novel and adaptive thinking.	The ability to manage and resolve conflicts.		Critical thinking/Integrated problem-solving.
Social intelligence.	Social intelligence.	The ability to relate well to others.	Interpersonal Skills.	Collaboration.	20%
Commitment to ethics and citizenship.	Not considered.	The ability to cooperate.	Global Citizenship.	Normative.	4.06%
Communication.	Not considered.	The ability to use language, symbols, and text interactively.	Media and Information Literacy.	Not considered.	25.31%
Digital transformation.	Computational thinking, Command of new media, and Virtual collaboration.	The ability to use technology interactively.		Not considered.
NOT CONSIDERED.	Not considered.	Not considered.	Not considered.	Systems thinking.	0%
NOT CONSIDERED.	Not considered.	Not considered.	Not considered.	Anticipatory.	0%

plots the relationship between competencies that were examined by or implemented in different entities and illustrates which ones were addressed in the literature.

The table indicates that Future work skills and the OECD do not consider entrepreneurship in their list. Future work skills also does not consider Ethics and Communication. However, all skills and competencies of the organizations and studies presented in the table exhibit certain degrees of relation to the competencies identified by UNESCO.

The main contrast that can be observed between institutions of higher education and upper secondary education is that higher education places a greater emphasis on learning outcomes and performance outcomes, which can lead to changes in goals. Alternatively, upper secondary education focuses on preparing students for university and refining basic knowledge; thus, it is less likely to participate in educational innovation with respect to higher education. Some in the higher education sector are more concerned about philosophy because universities are being increasingly perceived as manufacturers of job skills, which fuels the pressure for uniformity at the expense of the visions of other institutions of higher education [10,11]. High school and higher education share certain parallels in that they can both educate students for careers outside the classroom. Simply put, they teach students to become self-taught, listen, collaborate, lead, be creative and innovative, manage their time, and dedicate efforts toward continuous and sustained work in activities and in classes of subjects that are important for both levels of education.

Furthermore, another important point is that a new driver has emerged in the current role of managing education such as the COVID-19 pandemic. The crisis has exerted a considerable influence on medical education, particularly in terms of teaching and assessment [44], or evaluating complementary personality traits in students such as it is sensation seeking associated with risky behaviors [45]. Similarly, the pandemic affected other sectors such as online commerce [46]. However, as the world began to resurface from this challenging period, a possibility exists that the pandemic will leave long-term changes in the foundational elements of educational delivery. Such changes may include increased integration of “flipped learning” and “telepresence” as methods of content delivery, as well as a re-imagination of current school examination systems. The common theme is that the COVID-19 pandemic has served as a catalyst required to diversify the format and delivery of teaching and assessment. In doing so, it has depicted the intrinsic advantages of many of these contemporary approaches relative to their conventional archetypes. The majority of these educational changes were initially due to urgency; however, many of these changes will likely remain in other refined forms as preferred methods of teaching and assessment in the future.

4.2. Limitations of the Study

WoS and Scopus were the only databases used in this study. In the methodology of the SLR, analysis of scientific studies was conducted by including only scientific articles from January 2016 to March 2021 (n = 320), with a DOI registry, and written in the English (Table 1). Moreover, the systematic review was conducted using a predefined search string; thus, it is limited by the scheme described in Table 2.

5. Conclusions

5.1. Conclusions of the Study

The objectives of the study were to answer the three main research questions by collecting data according to nine sub-queries. Consequently, we highlight the conclusions regarding the data obtained and the three main questions in the following paragraphs.

Objective I: How is educational innovation being defined in upper secondary and higher education for competencies?

Numerous studies have widely used the concept of educational innovation. However, a clear definition that stratifies the constructs is lacking. In addition, a lack of standardized parameters is noted for teaching and evaluating technical or transversal skills. RQ1–RQ6 and RQ9 provide information that can elucidate certain aspects of Objective I.

We observed that the main trend is to teach and evaluate competencies among students in higher education. The most studied competencies are digital competence in the field of medicine/health. According to the classification proposed for the standardization of competencies presented by UNESCO in 2019 [1], the predominant competency analyzed in the studies is critical and innovative thinking (33.12%; Figure 3B). This competency is also the one most frequently analyzed in studies that focus on higher education (29.09%; Figure 4) In 20.76% of the studies that focus on this competence in higher education, the participants were students (Figure 5). The review also revealed that the most frequently used methodology for educational teaching was learning by doing (15.8%; Figure 6).

Furthermore, we analyzed the relationship between educational innovation in competencies and SDGs of the 2030 Agenda and found that the most frequent participants in studies that address a few of the SDGs are students in higher education (Figure 7). The SDG that was most frequently discussed is SDG3 (good health and well-being); however, all studies are considered directly aligned with Objective 4 (quality education).

Finally, we observed that technology was imported from other areas outside of education for studies that used certain forms of innovative technology. Nonetheless, 33.54% of the studies did not use any technology (Table 4); therefore, a niche of opportunity is presented for exploring other methods of integrating technology in educational practices and developing modern technology specifically for educational purposes.

Objective II: How are educational innovation projects being evaluated to develop competencies?

Analysis related to RQ7 and RQ8 provides hints for answering Objective II (Figure 6). The most frequently used methods for competency assessment continue to be those based on tests or examinations (external or institutional), with the majority of studies reporting only a single assessment period. Although educational methodologies are becoming increasingly diversified and creative in terms of applied skills (e.g., learning by doing and project-based learning), assessment in the majority of cases is performed using a standard examination based on tests or exams. Students have criticized standard examinations and tests as being less fair than assessments that require the active involvement of students [47]. Conversely, e-learning studies report the most diversified approaches for assessment. The fact that e-learning ranks first in terms of diversification and innovation in assessment attests to the fact that the change in communication tools or technological context has spurred innovation in evaluation. Nevertheless, the integration of technological tools for innovation in the assessment of competencies in education lacks scholarly attention. Thus, future research should focus on the potential of technological tools in advancing innovation in evaluation to promote new educational methods.

Objective III: Is there a reference framework for unifying the understanding of competence development?

Objective III was addressed using information obtained from RQ2B and RQ6. We concluded that no consensus exists on the standardization of competence, although we found several examples in the field of Future Work Skills [40] in a sectoral manner and the definition and selection of key competencies according to OECD [23], Classification Competences by UNESCO in 2019 [1], those presented by Tecnologico de Monterrey in its new educational model TEC21 for all graduate programs, and undergraduate courses offered by various schools [20,43]. The most frequently examined competence is critical and innovative thinking (33.12%; Figure 3B) followed by other important competencies, such as the ability to manage and resolve conflicts (OECD) [23], novel and adaptive thinking [40], innovative entrepreneurship, and reasoning to address complexity [20,43].

5.2. Future Work

Recent research into the culture of the higher education workforce indicates a deteriorating level of trust in the sector, frustration about the direction of education management, and increased levels of poor mental health. Every day, students and academics continue to face important mental health challenges that vary in severity and have become more pronounced due to the COVID-19 pandemic. Therefore, focusing on studies that reinforce the teaching of transferable skills that are aimed toward, for example, the resilience, stress management and mitigation, work exhaustion, and health of all stakeholders in the sector (e.g., teachers, administrators, staff, and students) is imperative for future studies. Moreover, generating studies on educational innovation in terms of competence assessment is crucial. Finally, modern technologies that will focus on improving competence learning for professors and students need to be developed.