A Meta-Analysis of Use of Serious Games in Education over a Decade

It seems necessary to review the literature to explore the effectiveness of serious games in education, since the number of studies on serious games is surging up. This study systematically reviewed the literature within around a recent decade. The trend of the number of publications related to use of serious games in education was firstly clarified based on the data retrieved from major databases. Secondly, various factors were determined that influenced the effect of serious game assisted learning. Themajor section identified both advantages and disadvantages of use of serious games in education. Use of serious games in medical science has been rising in a recent decade, which is thus highlighted in this study. Attitudes toward use of serious games in education were explored, as well as the new development of use of serious games in education. Future theoretical and practical exploration might need interdisciplinary cooperation.


Introduction
The past decade has been witnessing fast development of information technologies, together with rapid development of serious games.With the increasing application of technology assisted education, there is an urgent need to investigate the effect of serious game assisted learning.An increasing number of studies on serious games are also emerging, which needs to be reviewed to show future research directions and reveal the features of successful serious games.Designs of serious games and their application to education are also in need of the review of literature, which helps designers and teachers to better their professional work in the future.
Different from entertaining games, serious games are designed for an educational rather than an entertaining purpose [1].Serious games are referred to as entertaining tools with a purpose of education, where players cultivate their knowledge and practice their skills through overcoming numerous hindrances during gaming.Players' performances are scored during the gaming process [2].In case players overcome a hindrance, they will obtain some awards such as scores, advancement, and power.Educational elements can be integrated into the gameplay, which will be subconsciously acquired by the players during the gaming process.
One of the reasons for the effectiveness of serious games in education may be their influence on learners' mood.Gaming, as an entertainment outlet, plays an important role in mood formation such as sadness, happiness, and anger [1].Effective serious games attempt to form positive mood in order to encourage players to continue the play, leading to increased interest in gameplay, as well as better academic performances.Happy and active players can certainly be absorbed in serious game assisted learning.
This review study, therefore, aims to find out the main features of successful serious games and put forward constructive suggestions for designers via reviewing significant works in the recent decade.We attempt to answer the following research questions: (1) What is the trend of the number of publications in a recent decade?(2) What are the factors influencing the effect of serious game assisted learning?(3) What are the positive findings in use of serious games in education?(4) What are the negative findings in use of serious games in education?(5) Is use of serious gaming effective in medical education?(6) What attitudes do learners/players hold toward use of serious games in education?(7) What is the development of use of serious games in education?

Research Methods
In order to answer the research questions, we obtained 792 results including 729 articles, 49 reviews, 15 proceeding papers, 7 editorial comments, and 5 meeting abstracts by searching "Web of Science" (involving six databases, i.e., SCI-EXPANDED, SSCI, A&HCI, ESCI, CCR-EXPANDED, and IC) with the subjects "serious game" AND/OR "serious gaming" AND/OR "education" AND/OR "learning".The quality of publications was assessed by a 5-point criterion [3]: (1) quality of research design for answering research questions.The quality is classified into "high" if the study is conducted through a rigid design, e.g., combining quantitative with qualitative research; The quality is considered "medium" if the study is carried out through a less rigid design, e.g., a quasi-experimental design; the quality is reduced to "low" if the study is loosely designed, e.g., self-report description; (2) appropriate methodology or analysis; (3) generalizability of the results; (4) relatedness of conceptual framework, context, sample, and measures to research questions; and (5) reliability of findings.Each of the selected publications was scored based on the above criteria and reviewed by two researchers.The interrater reliability for the scoring was .83,reaching high consistency between two raters in terms of quality of papers.The papers with high quality were selected in this study.
By perusing titles and abstracts, we initially screened those unrelated to serious game assisted learning.Then 351 publications remained.The second screening excluded 49 reviews, 15 proceeding papers, 7 editorial comments, and 5 meeting abstracts, after which 275 articles survived.The third screening excluded those indexed in ESCI, CCR-EXPANDED, and IC because these databases were not considered main sources to retrieve highly quality serious game related papers.We also excluded those in the fields of computer science software engineering, computer science artificial intelligence, computer science information systems, computer science theory methods, engineering environments, environmental sciences, communication, critical care medicine, emergency medicine, ergonomics, telecommunications, water resources, and other unrelated disciplines.After this, the publications were reduced to 76 peer reviewed articles indexed in SCI-EXPANDED, SSCI, and A&HCI.Then two researchers independently read the remaining articles to exclude those unrelated to serious game assisted learning.Both of them reached an agreement on 39 articles to be included in the review analysis.

Results
All the selected papers were systematically reviewed and rigidly analyzed, whose results were shown below.

The Trend of the Number of Publications.
The number of publications was then analyzed using the tool in "Web of Science", whose result is shown in Figure 1.
As shown in Figure 1, the year 2009 began to witness publications about serious game assisted education.With time elapsing, the number of publications steadily rises until the peak in 2017.The year 2018 also predicts a large number of publications since the data were obtained in September 2018.Far more publications may be waiting to be published and indexed in "Web of Science".The reasons for the increasing number of publications should be explored, among which influencing factors in serious games assisted education should be firstly discussed.

Factors Influencing the Effect of Serious Game Assisted
Learning.Numerous factors may exert more or less influences on the effect of serious game assisted learning.They might include the perceived usefulness of the game and the interaction of students with colleagues, instead of effort expectancy [4].It has been widely accepted that serious games, as a tool integrated into many courses, are playing an important role in learning and helping learners focus on the target subject.Perceived usefulness, ease of use, and goal clarity were indicators of satisfaction and effectiveness in use of serious games, which was confirmed by a survey and a group of participants.When learners clearly predict the goals and ease of use, they tend to focus on the contents and enjoy themselves [5].
In addition, serious games should be appropriate for various learners and target contents.Those catering for different learner needs and applicable in different academic contexts could more likely achieve success than those failing to do so.Relationships between learning attributes and gaming mechanics, an important influencing factor, should therefore be considered and implemented by teachers, which could be integrated into teaching plans and learning process.This relationship played an important role in improving learning effectiveness and in enhancing learning experience [6].
Specific influencing factors have recently been explored.Serious game assisted learning was greatly influenced by five factors, i.e., backstory and production, realism, artificial intelligence and adaptivity, interaction, and feedback and debriefing.Backstory and production refer to the information about the effect of serious games.Realism of serious games means the degree to which the game could meet users' expectation.Artificial intelligence and adaptivity refer to an approach using computing algorithms to facilitate the usergame interaction in order to provide service to cater for users' individual needs.The interaction in a serious game includes user-user, user-game, and user-instructor mutual communication.Feedback refers to the evaluation of the serious game, as well as the response of the game to users.Debriefing means communication sessions when information is shared and examined after the gameplay.Learners should seriously consider these factors to achieve success in learning, and designers of serious games should also integrate these factors into the game production [7].
Gaming easiness and instruction were also demonstrated influencing factors.Easiness in use and gaming instructiveness were more important factors than enjoyment to encourage gameplay participation.Enjoyment and motivation were thus not reported as influencing factors.Although enjoyment and motivation were positively correlated, no significant gain in learning outcomes was revealed due to both factors [8].
Surprises could also be considered an important factor that influences the effectiveness of use of serious games in education.Surprises in serious games improved learning effect through stimulating cognitive structures.Surprise might bring benefits to learners at an advanced level in terms of proportional reasoning skills.Thus narrativeness that produced surprises could be used to facilitate reasoning skills and improve learning effect [9].Surprising events could reveal the potential of serious games, where learners could obtain significantly more knowledge structures and foster more in-depth learning than those without surprising events [10].
Except for the learning contexts on campus, workplace out of campus was also a concern of scholars.In the business settings, the influencing factors in a serious game included instructional content, game dimensions, game cycle, debriefing, perceived educational value, transfer of learnt skills, and intrinsic motivation [11].
Types of serious games and age of learners are also influencing factors that have been academically explored.Learners at a younger age performed significantly better in serious game assisted learning than those at an older age [1].Games with different features also exert different influences on learning effects.Open ended serious games improved students' specific skills and better their learning performance via problem solving than closed games [12].

Positive Findings in Use of Serious Games in Education.
There are a huge number of findings in serious game assisted learning, most of which are supportive, coupled with a few negative results.Serious gaming immersed learning could facilitate learners' holistic understanding of scientific conceptions due to the improved performances on science and the prolonged retention of science knowledge.Gaming immersion was also positively correlated with performance of science learning [7].However, in both serious game and nongame assisted learning approaches, no significant differences in undesirable aspects (e.g., complaint or disturbance that influences other players' gameplay experience) were revealed among teachers and learners [13].
Serious games proved effective in enhancing cognitive abilities and affect, as well as pleasant mood, in general learning.Through analyzing 46 empirical studies, serious games were reported helpful for learners to obtain cognitive abilities, and increase positive affect of learning [14].Serious games could increase players' overall pleasant mood and their happiness level based on their self-reports.Players felt happier with serious game assisted learning and female players achieved more success in this learning approach.
Educational technologies, such as serious games and mobile applications, improved learners' academic achievements and encourage their participation in learning activities.Educational or serious games can act as effective tools to improve teaching in the sciences [14].Serious games were deemed as an effective pedagogical medium to cater for learners' various requirements and expectations.Despite the fact that no significant differences in academic achievements were found; significantly more positive attitudes toward serious game assisted learning were revealed compared with traditional paper-based learning.Serious game assisted pedagogy could provide flexible learning for different learners who could move beyond the limitations of traditional learning [15].Aided with serious games and other educational technologies, learners could choose time and venue they feel convenient to learn, without being limited to ossified schedules and places as in traditional learning.
Serious game-based learning proved significantly more effective than the nongame-based learning.Learners were engaged in serious game-based learning significantly longer than the nongame-based learning.In the former learning approach, learners and teachers were significantly more motivated, desirable, helpful, and less hindered compared with the latter.It was empirically evidenced that learners who learnt through playing serious games were scored significantly higher than those who did not learn through gameplay although significant differences in knowledge tests were not
Negative findings (1) The nature of serious games negatively influenced the relationship between mental workload and learning effect [22].
(2) No significant differences in in-depth learning were found among learners [23].
(3) Some serious games aggravated the mental workload and decreased the learning effectiveness [24].
revealed [16].Serious games were also reported effective in the field of architecture education.Previous use of serious games obtained positive results in architecture education.Architecture involves architectural art and technology, as well as the aesthetic and practical aspects of architectural art.Although they are clearly different, they are closely related, and their components vary greatly with the specific circumstances and buildings.Undergraduate architecture majors include architecture, urban and rural planning, landscape architecture, etc. Use of serious games in architecture education could enhance students' practical and theoretical knowledge, which was reported by many students majoring in architecture.
In the context of business, use of serious games was also effective in improving cross cultural communication competence between people from different countries [11].Game-based learning was also effective in social learning in favor of team opinions [17], as well as in script collaboration based professional learning and improvements on learner satisfaction [18].This was in line with the findings of another meta-analytical review that computer games create a complex learning situation where instructional support (e.g., scripted collaboration) is needed to facilitate cognitive learning [19].Serious game use could also improve social interaction abilities by integrating metacognitive strategies into gaming, leading to better academic achievements and learning activity engagement.It could also improve writing skills via thinking aloud and modeling in the gaming process [20].
Game-based learning was also evidenced effective in sociocultural learning in terms of cognitive and motivational effects [21].Despite the fact that the majority of studies positively evaluated the effect of serious games on learning, social interactions, and cultural communication, negative results could not be ignored.

Negative Findings in Use of Serious Games in Education.
Some negative results were, however, also found especially in terms of the correlations between mental workload and learning effect.The nature of serious games negatively influenced the relationship between mental workload and learning effect [22].In case the serious game makes the mental workload heavier, the learning effect tends to be negatively influenced, and vice versa.It is therefore no wonder to find that, in the serious game "Peacemaker" (Impact Games 2007) assisted learning, no significant differences in in-depth learning were found among learners [23].This type of serious game might have aggravated the mental workload and thus decreased the learning effectiveness since the mental workload such as the change of Heart Rate significantly indicated the learning outcomes at a certain difficulty level [24].
In order to clearly present a scenario and enhance the readability of the paper, influencing factors and positive and negative findings are summarized in Table 1.

Use of Serious Gaming in Medical Education.
A huge number of studies have been devoted to professional fields, e.g., medical education.Serious games in the field of medical education have been widely used due to their positive outcomes in learning and learner participation.Usability, motivation, flow state, affective engagement, and learning were determined, which revealed that serious games were frequently used with a high level of engagement.However, the negative affect in the use of serious games increased, followed by a decreased negative affect in physiotherapists and a rise in learner positive affect after the prototype [25].The disabled could also benefit from use of serious games in learning.
A framework involving three major components, i.e., learner profiling, learning adaptation, and learning evaluation, was designed for disabled people to engage in learning process by integrating learning analytics into serious games, which could be meaningful for the design, implementation, evaluation, and adaptation of serious games for education [26].Serious game assisted learning also proved effective in learning for those caught with autism spectrum disorders.
A digital serious game was designed for children and young people with autism spectrum disorders to learn geographical knowledge.Several learning strategies were embedded for them to learn collaboratively assisted with computers.The serious game improved their geographical knowledge learning and brought considerable benefits to learners.The serious game design process and future research should shed light on the facilitation of learner involvement in academic activities, together with social engagement within the classroom [27].
Use of serious games in neuroscience learning was also explored, which reported positive outcomes.Learners assisted with serious games acquired significantly more neuroscience knowledge than those without the aid of serious games.When interacting with serious games, they tended to perform more cognitive and metacognitive strategies to better integrate academic activities into individual learning styles and the cognitive operation also became significantly more efficient than the nonserious game assisted learning [28].
Generally, the medical science has recently witnessed clearly more studies on serious game assisted learning compared with other fields and most of studies in medical science supported use of serious games.Merely positive findings in use of serious game assisted learning may be not convincing to comprehensively evaluate the effect of serious games.Attitudes of learners, teachers, and practitioners should also be reviewed as an important element to assess serious game assisted learning.

Attitudes toward Use of Serious Games in Learning.
The conception of attitudes toward serious games design using advanced information technology has been acknowledged as a critical element in the use and acceptance of information technology [29].Understanding learners' attitude toward serious game assisted learning is crucial for scholars to design appropriate teaching outlines to meet learners' different needs and for practitioners to design proper serious games to facilitate learning effectiveness.Different attitudes mirror different needs of learners and practitioners, on which developers and instructors of serious games should base theoretical and practical framework.
Learners held positive attitudes, positive cognitive perceptions, and high positive and low negative affective perceptions regarding various serious games assisted learning.Simulations were demonstrated more supportive for the comprehension and application of knowledge while producing a less positive affect than quizzes and adventures.Significant gender differences were found, where females had significantly higher perceptions of negative affective quality than males.When serious games were generally addressed, learners could answer questions about specific games in more detail.Female held a significantly more positive attitude and higher perceptions of positive affective quality than males in terms of all the three tested games.It is important to separately study different types of serious games and seriously take the gender variable into account when exploring learner attitudes and perceptions in serious games assisted learning [30].
Serious games might encourage learners to hold positive attitudes toward academic tasks with strong self-regulation if they were immersed in the gaming situation.Positive attitudes help learners to produce better academic achievements than negative attitudes.Thus it is reasonable to conclude that serious gaming leads to significantly more positive attitudes than traditional learning.Serious games installed on smart phones, iPods or other portable devices, could realize mobile learning.Different from traditional classroom-based learning, mobile learning is not limited to fixed classrooms.Learners tend to be attracted by its mobility and convenience.They usually hold positive attitudes especially toward this attractive learning approach that easily triggers their interest and motivation [31].In order to cultivate positive attitudes toward serious game assisted learning, the way to develop serious games should be carefully taken into account.

Development of Use of Serious Games in Education.
When developing serious games, it is important to develop a tool to track gameplay or learning progress.Although use of serious games in learning could raise educational quality and learner satisfaction, learners tended to concentrate on gameplay rather than learning in the serious game assisted learning.This in fact not only created both fun and learning atmosphere simultaneously but also made it difficult for researchers and teachers to track learning progress during the gameplay process.Success in tracking learning progress could obtain immediate feedback for teachers to moderate teaching objectives and contents.A real-time analytical tool was designed to track the dynamic and transient learning and gameplay progress, by which teachers could keep up with learning progress, provide learning suggestions, and revise teaching goals and progress in time.This tool was positively evaluated and future promulgation was necessary [32].Besides, effective pedagogical approaches should be carefully integrated into gameplay designs and academic objectives.
Despite the fact that serious game assisted learning has received great consensus on the effectiveness, methodologies and tools are still deficient to design, analyze and support this learning approach.The major objective of the Games and Learning Alliance European Network of Excellence on Serious Games is to address this issue, with pedagogies as its main focus (https://seriousgamessociety.org/).The fundamental design of serious game assisted learning is to integrate learning objectives into gameplay, through which educational aim is realized.A Learning Mechanics-Game Mechanics (LM-GM) model was designed to support serious game assisted learning.Pedagogical and gaming elements were both taken into account in the learning process.Gameplay mechanics and pedagogical elements were included in the LM-GM model.This model could help teacher evaluate the effectiveness of a serious game and perceive how to carry out the gameplay assisted learning and teaching.The effectiveness of the model was demonstrated and it is beneficial to future research into serious game assisted learning [33].An interesting serious game assisted learning approach has been recently raised, which is beneficial to the development of serious game assisted learning.
A new serious game assisted learning, referred to as transmedia learning, was designed to deliver learning or training contents through various media to rapidly implement immersive learning or training.The transmedia storytelling supported by entertainment, advertisement, and game industries could involve audiences in the story [34].New learning strategies were provided for learners to receive training or learning, where transmedia storytelling components, i.e., wireless connections, serious games, videos, Wechat, Messenger, graphic novels, machinima, blogs, and other educational games, were integrated into learning or training process.
The relationship between learning and game play is meaningful to explore in order to make preparations for game designers.A computational model was designed to study the relationship between learning and gameplay.It could determine the details of gameplay under different conditions.Stable and effective, the model could help researchers to explore quantitative factors between gameplay variables, perception of gameplay, and game designs [35].With these mentioned issues addressed, the possibilities of birth of feasible serious game assisted learning approaches will be enlarged.

Discussion
The serious games are mostly reported effective in education although some studies arrive at negative conclusions.It cannot be neglected that serious games should be assessed before they are applied to education.The proposed assessment procedure in a serious game was made of three steps, i.e., knowledge elicitation, representation, and evaluation.Compared with verbal assessment, structural assessment in a serious game might provide a significantly more in-depth perception of important concepts in a field.Four guidelines proposed to make use of structural assessment were provided in a serious game, i.e., (1) securing the proper domain for structural assessment, (2) choosing a proper referent for the target participants, (3) determining the concepts for structural assessment, and (4) analyzing the graphical knowledge representations in order to gather enough data of the structure quality [36].The assessment of serious games improved the quality of game products, which was beneficial to game design and teaching.
The multidimensional examinations on serious games and learning outcomes paved a solid foundation for future research into diverse serious game assisted learning and teaching.Various factors identified in this study help serious game designers and scholars to step further into this field by teaching and designing educational games based on the determined influencing elements.Links established in the study between positive findings, negative results, attitudes, and rising developments in serious games used in education also provide a comprehensive reference for professionals and practitioners.The highlight on the medical field shows a pioneering research direction, leading to studies on other areas such as language acquisition, health education, and environmental education.
The research methods used in this study ensured the reliability of the findings.The papers were all selected based on the rigid criteria, which enhanced the quality of works included in this study.Reviewed contents involve arts and humanities, science, engineering, and medicine, providing a full-scale reference for future research and serious game designs.Although an empirical study might produce more convincing findings, this study, with rigid qualitative method, might be able to produce reliable results due to its reasonable publication sampling, database selection, and rigid designing.
The factors that influence serious game assisted learning could act as a reference for future gaming and teaching designs.Since perceived usefulness, ease of use and goal clarity are important factors attracting learners to use serious games, designers should take them into serious consideration when designing serious games and teachers should also try to integrate these factors into teaching so as to improve the learning effectiveness and encourage learners to participate into learning activities [5].
Assessment of players' performances is a difficult issue to address, which is drawing much attention.Scoring players' performances should include various factors such as type of games, teaching objective, and gaming context.A scoring mechanism containing four categories, i.e., assessment aim, implementation, integration, and primary assessment type, was used to assess learners' skills during the serious gaming process.Players' assessment aimed at a formative rather than a confirmative conclusion, which was often carried out during the playing process.However, there are still some limitations regarding gaming scoring.Many elements in playing such as scoring and state monitoring were ignored.
The assessment purpose and certificates of attained skills were usually provided outside the context of gaming, which might not be able to encourage players to engage in playing.The prescribed implementations might also confine gaming achievements.Future scoring of serious gaming might take these limitations into account in order to improve the effectiveness of learning through serous gaming (Juan et al., 2017).For example, assessment could be carried out during the gameplay process coupled with timely feedback to the performance.As for language learners, the feedback could be written or recorded in the target language so that players' language skills could be improved when reading or listening to the information.

Major Findings.
This study systematically reviewed the works in a recent decade and provided worthwhile information for researchers and practitioners.Findings generally align with conclusions in other meta-analytical reviews (e.g., [14,19]).Specifically, the trend of the number of publications related to use of serious games in education was firstly clarified based on the data retrieved from major databases.This pictured the research and showed a clear direction for future research.This study, secondly, determined various factors that influenced the effect of serious game assisted learning, reminding learners, teachers, and designers of the important elements in use of serious games in education.
The major section of this study aims to identify both advantages and disadvantages of use of serious games in education.By exploring both positive and negative findings, the study sheds light on both sides of a coin.Use of serious games in medical science has been rising in a recent decade, which is thus highlighted in this study.Attitudes toward use of serious games in education were explored in order to help professionals and practitioners to perceive different viewpoints.The new development of use of serious games in education was also reviewed to pave a solid foundation for improvements on serious game assisted learning and teaching via updated technologies and tools.

5.2.
Limitations.This study, however, cannot cover all of the related publications throughout the world.Rather, it is limited to a few major databases.Other databases such as EI Compendex, and Elsevier were not included.This might lead to the deficiency of selected papers and under-representation of the literature.

Future Research Directions.
Use of serious games in education is a complicated issue to address, which needs multidisciplinary efforts.Future research directions could focus on interdisciplinary cooperation between related subjects, such as education, computer science, statistics, and psychology.

2 InternationalFigure 1 :
Figure 1: A histogram of related publications ranging from 2009 to 2018

Table 1 :
A summary of influencing factors, positive and negative findings.