2.1. Information sources

A comprehensive computerized search of the following electronic databases was performed: (i) Web of Science (all databases); (ii) Scopus; (iii) SPORTDiscus; (iv) PubMed; and (v) Cochrane library (Cochrane Database of Systematic Reviews). The search process for relevant publications had no restriction regarding year of publication and included SR and SRMA articles retrieved until 15^th of August of 2020. The following search strings were employed: (“small-sided games” OR “sided-games” OR “drill-based games” OR “SSG” OR “conditioned games” OR “small-sided and conditioned games” OR “reduced games” OR “play formats”) AND (“team sport” OR football OR soccer OR futsal OR handball OR volleyball OR basketball OR hockey OR rugby OR cricket OR “water polo” OR lacrosse OR softball OR korfball) AND (“systematic review” OR “meta-analysis”). Additionally, the reference lists of the studies retrieved were manually searched to identify potentially eligible studies not captured by the electronic searches. Finally, an external expert (Ph.D., assistant professor and ten years of publications in SSGs, with international publications in the topic) has been contacted in order to verify the final list of references included in this umbrella review in order to understand if there was any study that was not detected through our research. The inclusion and exclusion criteria were also provided to the external expert (as suggested by Cochrane’s guidelines) [16], however, no information about which databases to consult or search strategies to use were provided aiming tot create a bias in the expert. This process (double-check of an external expert) is recommended by PRISMA guidelines and also is part of the assessment items in AMSTAR-2 [17]. The external expert did not add new articles for inclusion, thus confirming the accuracy of the initial search.

2.2. Eligibility criteria

The inclusion criteria for this umbrella review were as follows: (i) only SR or SRMA in SSGs (not limited to the type of study designs included in the SR or SRMA) in team ball sports; (ii) any SR or SRMA in SSGs that included outcomes related to an internal and external load (acute responses to SSGs), fitness variables (adaptations after a certain period of SSGs-based intervention) technical/tactical measures (acute responses or adaptations), psychological or pedagogical dimensions (acute responses or adaptations); and (iii) peer-reviewed SR and SRMA written in English that provided full-text. Studies were excluded on the basis that they: (i) were not SR or SRMA (e.g., narrative reviews, brief reviews, scoping reviews, empirical articles, methodological proposals); (ii) did not include relevant data in SSGs and/or team ball sports; (iii) were not fully written in English; and (iv) consisted of abstracts only, without accompanying full-texts.

The screening of the title, abstract and reference list of each study to locate potentially relevant studies was independently performed by two of the authors (FMC and HS). Additionally, they reviewed the full version of the included papers in detail to identify articles that met the selection criteria. A third author (JA) participated to resolve discrepancies regarding the selection process.

2.3. Data extraction

A data extraction was prepared in Microsoft Excel sheet (Microsoft Corporation, Readmon, WA, USA) in accordance with the Cochrane Consumers and Communication Review Group’s data extraction template [18]. The Excel sheet was used to assess inclusion requirements and subsequently tested for all selected studies. The process was independently conducted by two of the authors (FMC and HS). Any disagreement regarding study eligibility was resolved by a third author (JA). Full text articles excluded, with reasons, were recorded. All the records were stored in the sheet.

2.4. Data items

The following information was extracted from the included SR and SRMA: (i) number of original articles included (n), age-group (youth, adults or both), sex (men, women or both), competitive level (if available), design (SR or SRMA) and type of original articles included (experimental, observational analytic or both); (ii) identification of the effects (acute or adaptations), dimension of analysis (internal load, external load, technical, tactical, recovery/fatigue/readiness, psychological), outcomes explored, and main findings.

2.5. Assessment of methodological quality

The Assessing the Methodological Quality of Systematic Reviews (AMSTAR-2) tool [17] was used to assess the methodological quality of the SR and SRMA included in this umbrella-review. The AMSTAR-2 it is a rating system that classify all reviews’ quality level into critically low, low, moderate and high. The system classifies 16 items, namely [17]: (i) information about the use of PICO; (ii) statement about the methods made before conducting research; (iii) explanation for inclusion of study designs; (iv) use of comprehensive literature search strategy; (v) study selection made in duplicate; (vi) data extraction in duplicate; (vii) list of excluded studies and reasons; (viii) describe included studies in detail; (ix) assessing the risk of bias; (x) report sources of funding for the included studies; (xi) appropriate statistical methods used in the meta-analysis; (xii) assess the potential impact of risk of bias on the results; (xiii) consider the risk of bias in primary outcomes when interpreting/discussing the results; (xiv) appropriate explanation about heterogeneity observed in the results; (xv) conduct an adequate investigation of publication bias and discuss its likely impact on the results, and (xvi) report potential sources of conflict of interest. The quality of each eligible SR and SRMA was analysed by two researchers (FMC and HS) independently. If the assessment was unclear, consensus was achieved with the help of a third author (JA).

The overall confidence in the results of the systematic reviews proposed by the AMSTAR-2 tool was defined as: (1) high—No, or one non-critical weakness: the systematic review provides an accurate and comprehensive summary of the results; (2) moderate—more than one non-critical weakness but no critical flaws: the systematic review provides an accurate summary of the results; (3) low—one critical flaw, with or without non-critical weaknesses: the systematic review may not provide an accurate and comprehensive summary of the results; (4) critically low—more than one critical flaw, with or with-out non-critical weaknesses: the review should not be relied on to provide an accurate and comprehensive summary of the results [17].

Data were retrieved by two authors (FMC and HS) and checked by a third author (JA). A specifically designed template for data extraction was developed. For each included SR or SRMA, the following items were extracted: study citation details, purpose of the study and context of analysis, type of analysis, match context, individual and environmental constraints and, main outcomes. Since the data were descriptively reported in this umbrella review, there was no statistical analysis.

3.1. Study identification and selection

The searching of databases identified a total of 176 titles. These studies were then exported to reference manager software (EndNote^TM X9, Clarivate Analytics, Philadelphia, PA, USA). Duplicates (88 references) were subsequently removed either automatically or manually. The remaining 88 articles were screened for their relevance based on titles and abstracts, resulting in the removal of a further 65 studies. Following the screening procedure, 23 articles were selected for in depth reading and analysis. After reading full texts, a further 11 studies were excluded due to not being SR or SRMA (n = 8) and not being written in English (n = 3) (Fig 1). From the 12 papers included in this umbrella review, 8 were SR and 4 were SRMA. The chronological analysis of the articles considered in this umbrella review revealed the recent developments in this area of research, highlighting that 8 of 12 articles (67%) were published between 2019 and 2020 (last two years) and the oldest SR were published in 2011 (9 years ago).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA flow diagram highlighting the selection process for the studies included in the umbrella-review.

https://doi.org/10.1371/journal.pone.0247067.g001

3.2. Study characteristics and qualitative synthesis

The characteristics of the 12 SR and SRMA included in the umbrella review can be found in Table 1, while the summary of the main evidence found in each review can be found in Table 2. Notably, most studies included in the reviews lasted between six and eight weeks of training, sometimes up to 10 or 12 weeks, occasionally under six weeks, and one odd study analyzed by [19] lasted 18 weeks. Most studies used single groups or parallel, non-randomized groups. In the reviews conducted on soccer [9, 19], only two research articles in each included randomization. In the review by [20], only one article provided randomization. Seven reviews did not report on randomization of the included studies or explicitly used observational studies (i.e., single-group) only [21]. One review even explicitly acknowledged the lack of randomization of original research papers as a major shortcoming [22]. The exception was the review on team sports by [23], in which 14 of the 16 studies had followed randomized interventions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of SR and SRMA characteristics.

https://doi.org/10.1371/journal.pone.0247067.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary of main findings presented in the SR and SRMA.

https://doi.org/10.1371/journal.pone.0247067.t002

Of the included articles, eight focused on soccer [6, 21, 22, 24–28], one focused on basketball [29], and three examined team ball sports [20, 23, 30]. This reveals that most of the existing systematized knowledge is derived from applications in soccer. Additionally, three of the reviews analyzed only men [6, 19, 27], six analyzed both sexes [9, 20, 23, 28–30], and the remaining three did not report participants’ sex [21, 22, 25]. Regarding age groups, three reviews included only youth players [6, 27, 30], and the remaining nine included both youth and adults. Internal load monitoring was reported in only four reviews [9, 19, 28, 29]; external load monitoring was also reported in three of those four reviews [19, 28, 29] and none of the others, meaning that the load produced by SSGs is not well-known or, at least, not well-systematized in reviews.

In terms of dimensions of analysis, only five of the 12 reviews analyzed the tactical dimension [19, 21, 22, 28, 30], which was unexpected given that one of the main goals of SSGs is to promote the learning of game principles. Also, only five reviews analyzed the technical dimension [19, 21, 25, 28, 30], meaning that fewer than 50% of the studies have verified the impact of SSGs on skills. Among the SR and SRMA analyzed, six exclusively focused on reporting acute effects [21, 22, 25, 28–30], four of them only on longer-term adaptations [6, 20, 23, 27], while the remaining summarized both acute effects and adaptations [9, 19]. From those exclusively focused on acute effects, one exclusively focused on the tactical dimension [22], one focused only on the technical dimension [25], and the remaining works combined more than one of these dimensions. The psychological adaptations were limited to acute enjoyment of engaging with SSGs.

All of the reviews exclusively focusing on longer-term adaptations have only reported variations in fitness status [6, 7, 20, 27], and therefore, the long-term tactical, technical, and psychological adaptations are not known. Combined with a lack of longitudinal randomized trials, the actual long-term adaptations and learning promoted by SSGs are currently unknown. Table 3 presents a synthesis of the short-term, acute effects and adaptations of SSGs, while Table 4 synthesizes the long-term adaptations.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Short-term, acute effects and adaptations of SSGs: Summary table.

https://doi.org/10.1371/journal.pone.0247067.t003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Longer-term adaptations to SSG-based programs: Summary table.

https://doi.org/10.1371/journal.pone.0247067.t004

A conceptual overview elaborated by the authors of this umbrella review can be seen in Fig 2. Hopefully, this overview helps in systematizing the complexity of the field and presenting it in an intelligible manner.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Main small-sided games topics.

https://doi.org/10.1371/journal.pone.0247067.g002

3.3. Methodological quality

The overall methodological quality of the 24 included papers is summarized in Table 5. Based on the overall confidence rating obtained through AMSTAR-2 [17], we concluded that the overall confidence in the results of five reviews was rated as “critically low” and seven reviews were rated as “low.” Considering the recent development of research on small-sided games in team sports, we decided not to exclude any of the studies based on their quality assessment because we believe that the proposal for methodological improvements in conducting systematic reviews in this area of research is of relevant importance at this stage. In addition, we consider that the aspects underlying obtaining the scores mentioned above do not significantly question the results presented in the studies included in this review. Lastly, it has been found that similar results were obtained for other areas of research when applying the AMSTAR-2 tool [31].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. AMSTAR 2 assessment of each included SR or SRMA.

https://doi.org/10.1371/journal.pone.0247067.t005

One factor that contributed to this low level of confidence was related to the lack of protocol registration in all of the analyzed systematic reviews and meta-analyses. In this sense, we recommend the registration of protocols before starting the reviews in one of the available platforms (e.g., Campbell, Cochrane, Inplasy, Open Science Framework, PROSPERO, among others). Importantly, all of the included studies were performed after the publication of the PRISMA statement in 2009 [32]. Additionally, in 50% of the studies, the authors did not explain their criteria for selection of the study designs and did not perform the study selection and extraction data in duplicates, which warrants concerns related to the rigor of the reviews due to a lack of error-correction methods.

Moreover, it should be noted that around 25% of the included papers did not apply a satisfactory technique for assessing the risk of bias (RoB) of the individual studies that were included in the review. However, the RoB of individual studies plays a major role when interpreting and discussing their results and may, therefore, profoundly impact the conclusions derived from the reviews, both qualitatively and quantitatively. Additionally, when heterogeneity is observed in the results, authors should provide a satisfactory explanation and critical discussion for it. Our umbrella review highlights the need to improve the quality of systematic reviews in this field of research. Otherwise, consensus and guidelines may be established based on unsubstantiated or fragile conclusions.

4.1. Acute effects of SSGs: Internal and external loads

This current umbrella review showed that among the 12 articles summarizing acute effects, one exclusively centered on internal load (i.e., biological responses to the demands imposed by the SSG) [26], and three included internal and external loads [10, 19, 29]. Within internal load, the most common measures analyzed were the heart rate (HR) during exercise, rate of perceived exertion (RPE) after exercise, and blood lactate concentration (BLa). For external load measures, the most recurrent measures were the distances covered at different speed thresholds.

Considering the acute effects of SSGs on internal load, it was found that different task constraints seem to work independently and/or concurrently to promote significant variations in the biological response [10, 19, 26, 29]. Interestingly, across the reviews dedicated to acute effects on internal and external loads, the task constraints were always presented based on the format of play (smaller vs. larger formats and, sometimes, the influence of balanced vs. unbalanced formats) and pitch configuration (mainly the pitch size and the influence of different relative areas of play). Remaining constraints, such as the scoring method, action restrictions, or tactical/strategic mission, were also often reported, though with less emphasis. The training regimen was predominantly reported in the effects on the internal and external loads as well. A summary of evidence about the effects of these constraints on internal and external loads is provided in Table 3. Despite the relevance of task constraints in directly influencing internal and external load demands of SSGs, contextual factors, such as age group, expertise, competitive level, or maturation, seemed to also act as moderators [19, 28]. Surprisingly, fitness level was not tested as a possible covariable for physiological and physical responses in SSGs.

In the reviewed articles, it was observed that the main interactions tested between task constraints were between formats and relative area per player [26]. However, more studies should be conducted on the concurrent effects of different task constraints. Additionally, future systematic reviews should add sections discussing intra- and inter-individual variability during SSGs [33–35]. For that purpose, covariables must be added, and longer randomized studies must be promoted to describe how variable workload outcomes can be obtained during the same SSG. The influence of day of the week (and recovery status), readiness, and other important contextual factors should also be added for a better understanding of how and when to use specific SSGs in the weekly training schedule.

In summary (see Table 3), internal load tends to be greater when using smaller game formats (i.e., fewer players), larger pitches, limitations imposed on ball touches, special defensive demands or rules (e.g., man-to-man marking), and regular encouragement, while the inclusion of goalkeepers (soccer) usually reduces internal load. The effects of training regimen and contextual factors on the internal load produced by SSGs are either unknown or inconsistent. The results are similar for external load, with two relevant differences: (i) findings consistently suggest that intermittent training regimens produce greater loads than continuous regimens and (ii) greater competitive levels tend to increase external load during games.

4.2. Acute effects of SSGs: Technical/tactical dimensions

Among the included SR, four of them reported acute effects on both tactical and technical outcomes [10, 19, 21, 30], one exclusively focused on tactical behavior [22], and one examined only technical actions [36]. Regarding the technical actions, the great majority of the studies were conducted using an observational methodology (coding the actions). Meanwhile, for tactical behavior, two main approaches were performed (observational analysis–coding behaviors; position-based data analysis–using metrics).

Task constraints predominantly focused on format and pitch configuration. Other constraints, such as scoring method, action restrictions, or tactical instructions, were also considered, but to a lesser extent. Interestingly, tactical instructions were more widely reported in this section, possibly due to the more pronounced effects on behaviors and the collective organization of players. A summary of the main findings and their consistency is reported in Table 3. In the particular case of technical and tactical actions, contextual factors, such as mental fatigue, expertise, or skill level, and age group, were explored and were consistently reported as determinants to justify variations in tactical behaviors and technical actions. Curiously, fitness status had no significant impact on technical actions and tactical behaviors [22, 36].

Due to the intimate relationship between behavioral and physical demands in the game, a greater exploration of the topic in the SR would be expected. However, it seems that such an approach has not been regularly conducted. More original works, such as ones combining analyses of tactical behavior, collective organization, and physical demands [11, 37], should be carried out. This would be helpful because a more robust overview of the effects of task constraints on all performance dimensions (e.g., tactical/technical, physical/physiological, psychological) would help to better identify the most adequate for specific situations.

In summary (see Table 3), it has been found that game format, pitch configuration, assigned tactical/strategic missions, and contextual factors (e.g., match status, expertise level) consistently affect acute assessments. Meanwhile, scoring method provides inconsistent results, and there is no evidence found regarding action restriction. As for the assessment of technical actions or skill, game format, pitch configuration, scoring method, action restriction, tactical/strategic mission, and contextual factors all affect acute assessments, especially concerning the frequency and success of the actions. Indeed, only one of the constraint categories (training regimen) failed to provide consistent effects.

4.3. Adaptations to SSG-based programs

The use of SSG-based programs to promote changes in physical performance, technical/tactical skills, and the psychological dimension was summarized in six articles. Two SRMA [23, 27] were specifically dedicated to identifying the influence in different performance measures and technical skills, while one SRMA [6] exclusively focused on the effects on aerobic performance compared to other training methods (conventional endurance), and another compared the effects of SSGs with other training methods on the change-of-direction ability [20].

Across the SRMA, the positive effects of SSG-based programs on aerobic performance were consistent, mainly emphasizing that there were no significant differences from other running-based methods [6, 23, 27]. Going into more detail, the majority of intervention studies using SSGs lasted six to eight weeks, with two to three sessions/week, using 2 vs. 2 to 4 vs. 4 formats performed four to seven times per session (repetitions), 2–4 minutes each repetition and with a work-to-rest ratio between 1:0.5 and 1:1 [6, 23, 27]. However, the evidence for positive effects on repeated sprint ability, jumping performance, linear sprinting, or change-of-direction ability was not so consistent and unequivocal as in the case of aerobic performance [23, 27]. This might create an opportunity for more research, mainly that combining SSG-based programs with other training methods. Some original research has tested the combination of SSGs and running-based, high-intensity interval training [38, 39] or even SSGs combined with strength training in the weight room [40]. However, studies combining SSGs and other training methods are recent and few, and so more consistent research should be performed.

Additionally, the use of an SSG-based training approach across the week needs to also be researched. In the reviews analyzed, this topic was not addressed. However, it seems important to analyze how the distribution of different SSGs across the week (not merely as training intervention) can be organized and implemented. The original cohort study conducted recently tried to demonstrate the effects of a weekly schedule based on SSGs [40]. However, more robust study designs should be developed to make comparisons with other interventions. Additionally, there is a lack of discussion and research in original works about the individual response profile to SSG-based programs [41]. The interaction of response to fitness and tactical/technical baseline levels, competitive level, and remaining workload made across the week should be analyzed to gain a more specific idea about the impact of SSGs in players.

In summary (see Table 4), SSGs consistently and homogeneously produce improvements in aerobic capacity that are similar to those provided by more traditional aerobic training methods. However, neuromuscular adaptations, as measured by strength, speed, and agility tests, among others, are very heterogeneous, and the evidence currently suggests that a combination of SSGs with other, complementary training methods is warranted. Tactical and technical improvements seem to occur, but research on the topic is still scarce, especially given that two traditional arguments for the implementation of SSGs are a better interpretation of the game (with consequently better tactical performance) and promote execution opportunities that allow players to contextually improve their skills [42]. Finally, analyses of psychological constructs in the context of SSGs have been limited to enjoyment, which is highly restrictive and provides a unidimensional look into a multifactorial reality. Furthermore, associations or correlations between enjoyment and other psychological constructs are not always linear [43, 44], and complex relationships may have to be addressed, as they can mediate the experience.

4.4. Limitations of the current research on SSGs

The existing systematic reviews of research on SSGs have revealed some shortcomings (i.e., limitations that reduce the scope and trustworthiness of the existing body of knowledge). Noteworthy limitations include (i) the limited application of randomized experimental designs and over-reliance on observational studies; (ii) within experimental studies, an over-reliance on single group interventions and with few sessions, lacking comparative power; (iii) a lack of pre-registration of the review protocols, meaning that initial protocols may easily be changed a posteriori to better fit the emerging data; (iv) the overall poor methodological quality of reviews and meta-analyses as assessed through AMSTAR-2; (v) a focus on soccer by most of the research, with very little systematized knowledge being derived about the impact of SSGs in other sports; (vi) the fact that the known effects of manipulating task constraints are largely related to soccer and perhaps not generalizable to other sports; (vii) the almost complete neglect of psychological variables and their interaction with other variables; and (viii) the inter- and intra-individual variation in response to SSGs remaining unexplored.

4.5. Directions for future SSG research

In light of the limitations that were identified, research on SSGs could benefit from the following: (i) design experimental studies lasting 12 or more weeks with parallel non-randomized (e.g., teams from the same age group but from different clubs) groups (or, preferably, randomized groups); (ii) pre-register the full protocol, either for a trial or for a systematic review; (iii) expand research to different team sports and explore to what extent the same types of SSGs and manipulation of task constraints induce similar or distinct effects; (iv) focus more intensely on long-term effects, thereby providing more reliable indicators of learning; (v) explore how variables of different natures (e.g., physical vs. technical) interact in such experiments; (vi) incorporate several complementary psychological variables, both as primary outcomes (to assess the impact of SSGs on these variables) and mediators of other variables (e.g., physiological measures, changes in tactical behavior); and (vii) explore the role of inter- and intra-individual variability in response to similar SSGs.

Admittedly, a single study will not be able to address all of these aspects, and so compromises will have to be made. Furthermore, since age, expertise level, and other covariables will likely have a profound impact on the results, any generalizations of data from very specific samples should be made with great caution.