What are the effects of different elements of media on radicalization outcomes? A systematic review

Abstract Background Most national counter‐radicalization strategies identify the media, and particularly the Internet as key sources of risk for radicalization. However, the magnitude of the relationships between different types of media usage and radicalization remains unknown. Additionally, whether Internet‐related risk factors do indeed have greater impacts than other forms of media remain another unknown. Overall, despite extensive research of media effects in criminology, the relationship between media and radicalization has not been systematically investigated. Objectives This systematic review and meta‐analysis sought to (1) identify and synthesize the effects of different media‐related risk factors at the individual level, (2) identify the relative magnitudes of the effect sizes for the different risk factors, and (3) compare the effects between outcomes of cognitive and behavioral radicalization. The review also sought to examine sources of heterogeneity between different radicalizing ideologies. Search Methods Electronic searches were carried out in several relevant databases and inclusion decisions were guided by a published review protocol. In addition to these searches, leading researchers were contacted to try and identify unpublished or unidentified research. Hand searches of previously published reviews and research were also used to supplement the database searches. Searches were carried out until August 2020. Selection Criteria The review included quantitative studies that examined at least one media‐related risk factor (such as exposure to, or usage of a particular medium or mediated content) and its relationship to either cognitive or behavioral radicalization at the individual level. Data Collection and Analysis Random‐effects meta‐analysis was used for each risk factor individually and risk factors were arranged in rank‐order. Heterogeneity was explored using a combination of moderator analysis, meta‐regression, and sub‐group analysis. Results The review included 4 experimental and 49 observational studies. Most of the studies were judged to be of low quality and suffer from multiple, potential sources of bias. From the included studies, effect sizes pertaining to 23 media‐related risk factors were identified and analyzed for the outcome of cognitive radicalization, and two risk factors for the outcome of behavioral radicalization. Experimental evidence demonstrated that mere exposure to media theorized to increase cognitive radicalization was associated with a small increase in risk (g = 0.08, 95% confidence interval [CI] [−0.03, 19]). A slightly larger estimate was observed for those high in trait aggression (g = 0.13, 95% CI [0.01, 0.25]). Evidence from observational studies shows that for cognitive radicalization, risk factors such as television usage have no effect (r = 0.01, 95% CI [−0.06, 0.09]). However, passive (r = 0.24, 95% CI [0.18, 0.31]) and active (r = 0.22, 95% CI [0.15, 0.29]) forms of exposure to radical content online demonstrate small but potentially meaningful relationships. Similar sized estimates for passive (r = 0.23, 95% CI [0.12, 0.33]) and active (r = 0.28, 95% CI [0.21, 0.36]) forms of exposure to radical content online were found for the outcome of behavioral radicalization. Authors' Conclusions Relative to other known risk factors for cognitive radicalization, even the most salient of the media‐related risk factors have comparatively small estimates. However, compared to other known risk factors for behavioral radicalization, passive and active forms of exposure to radical content online have relatively large and robust estimates. Overall, exposure to radical content online appears to have a larger relationship with radicalization than other media‐related risk factors, and the impact of this relationship is most pronounced for behavioral outcomes of radicalization. While these results may support policy‐makers' focus on the Internet in the context of combatting radicalization, the quality of the evidence is low and more robust study designs are needed to enable the drawing of firmer conclusions.


| What are the main findings of this review?
Results from experimental studies indicate that simple, one-time exposure to mediated content is associated with a small increase in the risk of radicalization. For individuals high in trait aggression, there is an equally small increase in risk.
The results from observational studies show that the magnitude of the relationship between simple usage of different media and radicalization is essentially inconsequential, with estimates close to zero. However, when usage of media was measured with reference to specific types of content or activities, such as the consumption of or posting of political content, small but potentially meaningful relationships are found.
For some attitudinal risk factors, such as perceptions of media bias, estimates indicate only small relationships with radicalization, whereas the estimates for others, such as attachment to online networks, point to more salient relationships. The largest estimates pertain to both passive and active forms of Internet-based exposure to content defined as specifically radical.
The reliability of these results is tempered by multiple sources of bias inherent in the cross-sectional studies, as well as the experimental studies that give rise to these results, including the inability to establish temporal ordering. Additionally, alternative theoretical explanations suggest that the results may suffer from confounding with factors such as age, gender, and self-control.

| What do the findings of this review mean?
Simple media consumption is unlikely to be associated with any significant risk of radicalization, including one-time exposures to radical content. Despite alternative hypotheses that increases in risk are a function of certain psychological traits, such as aggression, differences in the magnitude of estimates are negligible.
On the other hand, Internet-mediated exposure to radical content, whether passive or active, is associated with a significantly stronger relationship with radicalization than other types of mediarelated risk factors. When compared to other known risk factors for cognitive radicalization (non-media), the relative magnitudes of the estimates are moderate. However, when compared to other known risk factors for behavioral radicalization, the relative sizes of the estimates are considerable.
The ability to draw conclusions from the results is limited by the fact that studies suffer from multiple sources of bias.
1.6 | How up-to-date is this review?
The review authors searched for studies up to mid-2020.

| The media-radicalization relationship
Since the early days of terrorism research, scholars have consistently emphasized the integral role of the media as a key factor in promoting terrorism. The media's reporting on and portrayal of terrorist attacks and groups provides them with "free advertising." This exposure provides terrorists, groups, and their proponents with a vehicle for highlighting their grievances and spreading their message. In doing so, terrorists aim to attract new supporters and recruits, and strengthen their image among their current supporters (Weimann, 2005). The media also reports on and portrays a range of social issues that may highlight forms of discrimination or violence against different groups-both locally and abroad-which can increase feelings of anger, deprivation, and identification with the oppressed group. All these factors can increase the likelihood that an individual will support what they perceive to be the use of defensive violence, or even engage in it themselves (Wolfowicz et al., 2020).
These types of effects have received extensive attention in the ever-growing literature on individual radicalization, which focuses on the factors that increase (or decrease) the likelihood of the development of attitudes that justify terrorism, and which could therefore underpin a turn to radical behaviors such as terrorism. In this literature, which has primarily developed only in the last 15 years, research on the role of traditional media has shifted to focusing on the role of new media, namely the Internet, and social media in particular. These medias have been successfully leveraged by radical groups and their supporters to spread their message, greatly increase their support base, and from this base generate new recruits (Scrivens & Conway, 2019).
There is little doubt that media exposure in general can and does impact a range of cognitive outcomes, from emotions to attitudes and | 3 of 50 perceptions, as well as behavioral outcomes, whether on account of cognitive changes, imitation, or other mechanisms. An extensive literature exists that demonstrates these effects with respect to a range of deviant, criminal, and violent cognitive and behavioral outcomes. However, despite what seems to be a significant amount of attention given to the topic of media effects on radicalization, which encompasses a form of violent cognition (a positive evaluation of violence) and behavioral outcomes (Wolfowicz et al., 2020), there is a worrying lack of quantitative research LaFree, 2017;Whittaker, 2018).

| Media-related risk factors
Like the broader literature of media effects on violent cognitions and behaviors, there is a debate about the magnitude, and even the direction of the effects of media on radical cognitions and behaviors (Ferguson, 2016). Previous research has identified significant overlaps between radical cognitions and behaviors with violent cognitions and behaviors, as well as the risk and protective factors for these outcomes (Wolfowicz et al., 2020). Given these overlaps, it would be reasonable to expect that the media effects for general criminal and criminal-analogous outcomes should be similar for radical outcomes.
In this respect, it can be expected that there are differential effects for different mediums and content (Awan, 2007a(Awan, , 2007bAwan et al., 2011).
Some of the earliest literature on modern terrorism emphasized the potential role of television news-broadcast media in fostering the phenomenon. As Social cognitive theorist Albert Bandura (1990) wrote, "They [terrorists] use television as the main instrument for gaining sympathy and supportive action" (p. 8). As a multidimensional media, providing news, entertainment, and broadcast, television can potentially affect sympathy and support (cognitive radicalization), or supportive action (behavioral radicalization) in multiple ways (Matusitz, 2013). First, as a source of exposure to violent imagery, either real or fictitious, television can activate aggression and increase desensitization. These images, as well as general news and political news about social issues and conflicts, can also serve as source of imitation, and grievance.
Television can also serve as a medium for the broadcasting of both explicit and more subtly ideological messaging, from politicians and commentators to religious sermons and other programming (Matusitz, 2013).
Some studies have found that when terror attacks receive more television coverage, interest in associated groups on the Internet increases (Jetter, 2019). The interplay between news and broadcasting can also potentially contribute to radicalization. For example, certain populations may be exposed to elements of a materialistic western culture through television. These depictions may subsequently be referred to in a disparaging way by preachers (Mousseau, 2011). Television, news media, and selected propaganda items may expose a receiver to images, scenes, and messages that highlight attacks on a group with whom they identify. For example, items that highlight the acts of extremists representing an opposing group (e.g., right-wing vs. left wing), or military or state attacks on in-group members. Interviews with former terrorists and non-violent radicals reveal that these forms of exposure can play a central role at different stages of the radicalization process (Baugut & Neumann, 2020a). Additionally, mainstream media reports of attacks against certain groups, both locally and abroad, may also be subjectively selected by propagandists and then re-distributed through other mediums. This form of "hybrid propaganda" has been found to have played a key role in the radicalization of radical offenders in Europe (Baugut & Neumann, 2019a).
Another key factor is the potential for news-media bias to increase feelings of hostility. In a laboratory experiment Neumann et al. (2018) exposed Muslim participants, who differed in their religious-fundamentalist beliefs, to either positive or negative news stories about Muslims. The study found that negative presentations were associated with perceived media bias toward Muslims and increased participants' feelings of general anger. The effect was greater for participants who had higher scores on the religiousfundamentalist beliefs scale. These results are in line with evidence concerning the "hostile media effect." A meta-analysis found that the relationship between increased media consumption and perceptions of a hostile media were quite modest, with an average effect of r = 0.296. The study found that the medium (e.g., newspapers, television etc.) did not have a statistically significant impact on this result. However, greater involvement with the issue depicted in the media had a statistically significant, positive impact (Hansen & Kim, 2011).
A recent study based on interviews with dozens of incarcerated terrorism offenders found that they believed that exposure to media did not have a significant impact on their beliefs or attitudes.
However, they also believed that the media did have a significant impact on others and played a role in formulating negative attitudes toward their in-group within the general population (Baugut & Neumann, 2020b. This type of media perception is known as the "third person effect," which posits that individuals perceive the media to have greater sway and influence over others than over themselves (Paul et al., 2000). That the terrorists from the above noted study all displayed evidence of the third-person effect provides an early indication that a potentially wide variety of media effects may be applicable to radicalization.
In recent years there has been a growing trend in which radical groups have used the medium of videogames to indoctrinate and to recruit. Different groups, such as ISIS, have created high-quality games in which players operate as terrorists, killing opponents who represent the groups' real-life adversaries. Beyond these tailor-made games, the interactive features of mainstream games are rife with hate speech, racism, and threats of violence. It could be theorized that even these "ordinary" video games can increase the risk of radicalization, as they do for generalized violent cognitions and behaviors. However, as with respect to the research on video game effects more generally, there have been suggestions that video games could provide prosocial effects and thereby reduce the risk of radicalization (Menendez-Ferreira et al., 2020). 1 Just as in the broader field of media-effects research, while television and the Internet have commanded the bulk of researchers' attention, the literature has also discussed the role of other forms of traditional media in radicalization. For example, the role of radio has been discussed in the context of right-wing radicalization (Hackett & Soares, 2015;Hamm & Spaaij, 2017) and radicalization in certain parts of Africa (Yanagizawa-Drott, 2014). Similarly, the role of music has figured prominently in research on right-wing radicalization (Gaudette et al., 2020;Pieslak, 2015); and more recently regarding Islamist radicalization (Pieslak, 2015). Research has also focused on multimodal mediums, such as music videos, videogames, and shortclip productions, with such content believed to have stronger effects on emotional and cognitive outcomes, in line with the objectives of radical groups (Winkler & Pieslak, 2019).
Despite the move to digital sources, print-media remains an important source for news and for conveying different ideological positions in many places around the world. In Pakistan, the ideological slant local newspapers have given to the portrayal of drone strikes on terrorist targets has been referred to as a "major source" of radicalization (Raza & Awan, 2013). Additionally, across the Middle East, and even in places such as downtown London, books such as Adolf Hitler's Mein Kampf and the fictional Protocols of the Elders of Zion are sold alongside more recent radical publications (Patterson, 2016). These books and others, such as The Turner Diaries (1978) have been implicated in the radicalization of several terrorists involved in attacks, including the Oklahoma City Bombings Mills et al., 2019).
Despite the importance of these "old media," it cannot be denied that most of the attention in recent years has shifted to examining and understanding the role of the Internet as a media source for radicalization. However, just as Internet technology has undergone significant change over the last two decades, so too has the focus of research. Extremist groups were early adopters of the world wide web, with web pages created as early as 1996 which were used to promote their causes, provide access to content, and enable discussion between supporters (Erez et al., 2011). The move to discussion forums was a key innovation for radical groups as it not only provided a place for like-minded individuals to communicate but for the curious to hear from the more well versed and experienced.
Already in 1995, members of the neo-Nazi movement had created StormFront, a platform which still operates today with more than 150,0000 active members (Bowman-Grieve, 2009). Discussion forums continue to be viewed as a media-related risk factor for radicalization (Moskalenko et al., 2022). They are even central to some ideologies and movements, such as INCELS (involuntary celibates), which are guided by a deeply misogynistic ideology. While INCEL discussions groups have many thousands of members, high-profile attackers have often been found to have been highly active in their postings (Moskalenko et al., 2022).
Today, social media platforms, such as Facebook, Twitter, and YouTube, have garnered much of the attention in discussions about the potential for media to serve as a risk factor for radicalization.
Together with discussion forums and chat rooms, social media provides opportunities for likeminded individuals to connect, to seek reinforcement for previously held beliefs, and share content. In this regard, these platforms provide for the sharing of multimodal content, such as digital images and videos. Multimodal content is likely to have stronger psychological effects than other unimodal or bimodal content (Conway & MacDonald, 2020). As with the study of media-effects more generally, different platforms offer opportunities for both passive and active forms of consumptions. Traditional media platforms, such as print media and television, provide opportunities for passive exposure, in which a consumer is not required to participate or engage in any specific way. Internet-based platforms however, whether discussion forums, social media, or mobile communication applications, in addition to opportunities for passive exposure, also provide opportunities for active exposure, in which the consumer can actively interact with the media, or with others who are also interacting with or through the media (Coyne et al., 2018). Radical groups and ideologies of all stripes have some sort of presence on these platforms (Holbrook, 2021;Weimann, 2016). These are also the type of platforms where users are most likely to seek out radical content produced by groups such as Al-Qaeda and ISIS (Frissen, 2021).

| The nature of the relationship between media-related factors and radicalization
The potentially negative cognitive and behavioral effects of media have been analyzed for close to a century, with the first studies examining the effects of movies. With the proliferation of home TV sets in the 1950s, and several sensationalized cases of violence described in the media as having been the result of imitation of TV violence, this field of research expanded greatly (Sparks et al., 2009). In seeking to explain and test the ways in which media may potentially increase the risk of deviant or criminal cognitions and behaviors, researchers have drawn extensively on social learning and social cognitive perspectives. Social learning has also provided the basis for several other prominent, media-specific theories (Allen et al., 2018;Phillips, 2017).
According to criminology's social learning theory, deviant attitudes and behaviors are learned through the same mechanisms as normative attitudes and behaviors, namely through differential associations, the definition that they provide, imitation, and differential reinforcement. Differential associations can be peers, family, role models or other individuals. Their opinions and positions on a given issue or behavior, as well as the behaviors they themselves engage in, provide an individual with a balance of definitions in favor or against the given behavior. The experiencing and observing of the behaviors of differential associations also provides a source of imitation of what those behaviors should look like. Additionally, through experiences 1 A video-game tournament using the FIFA soccer game was conducted in Malmo, Sweden by the NGO Kommon-Ground as a pilot CVE intervention. The goal of the intervention was to promote intergroup cooperation and other prosocial factors. | 5 of 50 and observations, an individual will gauge what sort of reinforcement -positive or negative-engagement in the behavior tends to garner.
When an individual has a greater proportion of differential associations holding a certain attitude, or engaging in a behavior, when they are exposure to it more often, and when they see that it generates positive reinforcement, they are more likely to adopt the attitude or behavior. In this regard, learning can also be mediated, and pieces of media-content, from books to television programs can also serve as differential associations. Even more so on the Internet, individuals can establish or maintain associations with, or follow the behaviors of actual individuals (Akers, 1998). Differential associations are conditioned by frequency of interaction, duration of association, intensity of association, and the subjective priority (importance) ascribed to them. Sometimes the intensity of mediated differential associations can be even stronger than for non-mediated ones. Additionally, mediated learning may provide for a greater source of imitation due to its visual nature-as in the case of television, movies, videogames and the Internet-and the frequency and repetitiveness of exposures to content and the definitions, or messages that they provide with respect to a given attitude or behavior (Akers, 1998).
Another variant of SLT that was put forth by Bandura (1978) provided the basis for his development of a more cognitively oriented theory, Social Cognitive Theory (SCT). While the theory operates according to the same mechanisms of observation, experience and learning, the way in which it can lead to deviant outcomes is through desensitization and moral disengagement. For example, repeated exposure to messages that demonize or dehumanize a particular individual, type of individual, group or thing, can lead to a moral disengagement in which violence becomes legitimate (Bandura, 1977(Bandura, , 1990(Bandura, , 2001(Bandura, , 2009).
Other variants of both SLT and SCT have been developed to focus on media-effects specifically. For example, the General Aggression Model (GAM) holds that exposure to media depicting violence, or which includes pro-violent messaging, primes aggressive thoughts and emotions, arousing and activating aggressive cognitions at the neural level. While arousal and activations may initially be measured in attitudinal expressions, such as in the case of learning theory, it is these cognitive effects operating at the level of neural pathways that may ultimately manifest in outward behaviors (Allen et al., 2018). Both social learning and cognitive perspectives also give weight to the role of reciprocal determinism. That is, just as "birds of a feather flock together," and individuals tend to gravitate toward those with beliefs similar to their own, they also have a tendency to select bias-congruent media. As such, pre-existing biases shape media consumption, and consumption reinforces and confirms prior beliefs (Huesmann & Taylor, 2006). This process has been described as a reinforcing spiral (Slater, 2007), and can lead to the formation of ideologically homogenous "echo chambers," insular networks dominated by one-sided presentations of ideas, opinions, and attitudes (Stevens & Neumann, 2009;Sunstein, 2009).
The meta-analytic contributions to the field of media-effects provide strong evidence for the social learning and cognitive perspectives. On the one hand, it has been found that pro-social media exposure improves pro-social cognitions and behaviors, whilst anti-social or deviant media exposure promoted deviant cognitions and behaviors. Similarly, pro-social media has been found to reduce the likelihood of aggression, whereas anti-social, deviant, and violent media has been found to increase it. Moreover, great consistency has been found between these positive and negative effects, which have been found to have nearly identical magnitudes (Coyne et al., 2018;Greitemeyer & Mügge, 2014;Mares & Woodard, 2005).
In opposition to the media socialization paradigm, in which media usage is seen as a predictor of the outcome, there is the media selectivity paradigm, in which media usage is an outcome of the theorized predictors (Slater, 2008). Due in large part to the designs of studies in the field of media-effects, it is often difficult to disentangle whether certain types of media usage increase outcomes such as aggression, or whether those high in aggression are simply more prone to engage with problematic media. Similarly, failure to control for potential confounder, especially those known to be associated with the outcome, such as gender, mean that effects may simply represent the relationship between the third variable and the outcome. Some studies have found that when controlling for such factors, effect size estimates are as much as half of bivariate correlations. These findings provide support for the media-selectivity perspective as they demonstrate that a large proportion of the variance is explained by individual characteristics (Anderson et al., 2010).
A related issue is likely to exist with respect to radicalization as well. One of the most important risk factors for radicalization is low self-control, as well as related risk factors such as thrill/sensation seeking (Wolfowicz, Litmanovitz, et al., 2021). Some research indicates that individuals with low self-control may be more likely to engage with problematic media in several ways. First, they may be more prone to spending more time engaging with media in general, as well as developing Internet addiction (Li et al., 2021). By spending more time using media, they may therefore be at a heightened risk of encountering content that could increase risk for radicalization.
Indeed, there is already some evidence that low self-control is a predictor of exposure to radical content (Pauwels et al., 2020;Hawdon et al., 2019). Additionally, an individual's self-control mediates the influence that exposure to media has on them. Based on these issues, it has been suggested that self-control perspectives inherently suggest that the relationship between media and deviance is a spurious one (Hermann, 2011). 2.3.1 | Meta-analytic contributions to the study of media effects While there are some exceptions (e.g., Ferguson et al., 2020), over the years there have been a slew of meta-analyses that have produced fairly consistent results showing that exposure to violent television (and movies) has a modest effect on violent cognitions and behaviors (see Comstock et al., 2014 for a review of previous systematic reviews). When it comes to the Internet and social media as mediums, most meta-analytic research point to small relationships (of about r = 0.10) with psychological outcomes-such as self-esteem and depression (Appel et al., 2020). However, when it comes to delinquent behaviors, average effect sizes of r = 0.21 have been found (Vannucci et al., 2020). This effect size is almost identical in magnitude to the effect sizes that have been found for exposure to television violence (Comstock et al., 2014). Other forms of media exposure have also been found to have pooled effect sizes of a similar magnitude on analogous outcomes, such as the effects of music on anti-social behaviors (Timmerman et al., 2008) and violent video games (e.g., Comstock et al., 2014;Ferguson et al., 2020).
The findings mirror those of other areas of investigation into risk factors for related cognitive and behavioral outcomes, (such as violent cognitions and behaviors, or suicidal thoughts and behaviors), in which effect sizes are almost always larger for the cognitive outcome-which is generally more prevalent. For factors where effect sizes are found to be larger for the behavioral outcomes, it has been suggested that this could be an indication of them serving as mediators of the continuity between the given cognitive and related behavior (Wolfowicz, Weisburd, et al., 2021a). Of all media-related factors, such a finding has been made only with respect to violent video-games (e.g., Comstock et al., 2014;Ferguson et al., 2020). This evidence points to a small but potentially important role for at least some mediums to play a role not only in the risk of development of deviant cognitions and behaviors but as a mediator, in which in increases the risk of deviant behavior for those who already possess the related deviant cognition (Ajzen & Fishbein, 1980;Bosco et al., 2015;Wolfowicz, Weisburd, et al., 2021a).
Several lessons can be learned from the collective results from the literature. First, the magnitudes of the effects between television and video games are highly similar, even though the mediums are fundamentally and qualitatively different. While television offers a one-way, passive form of exposure, videogames offer a two-way passive-active form of exposure. Secondly, the magnitudes of the effects for most media-related risk factors are considerably smaller than analogous risk factors from the offline domain, such as social learning factors relating to exposure to violence, and having deviant associations. Third, the magnitudes of the effects for media-related risk factors do differ based on the nature of content. This indicates that mere exposure to or usage of a medium is not sufficient. Rather it is the mediated exposure to specific types of content across different mediums that increases risk of deviant outcomes to varying degrees (Anderson & Dill, 2000;Anderson et al., 2007;Coyne et al., 2018;Gentile et al., 2004;Martins & Weaver, 2019).

| Why it is important to do the review
As per the above, despite an extensive literature on media effects on aggression and other deviant outcomes, and in particular the extensive meta-analytic contribution to this literature, to date there has been little movement to integrate the study of media-effects on radicalization. Specifically, there have been few attempts to apply the theoretical and methodological frameworks used in the literature on media effects to this relatively new area of study . In response to this issue, the current study had the objective of integrating media effects on radicalization into the broader media effects literature as it pertains to criminal and criminal analogous outcomes more generally.

| OBJECTIVES
This systematic review sought to first identify what the different media-related risk factors for radicalization are, for which quantitative estimates exist. By organizing the identified factors in a rankorder according to their pooled effect sizes, we sought to identify the relative magnitude of the estimates and contrast them between the different outcomes of radicalization. We also sought to explore whether heterogeneity can be explained by the different context and settings in which studies are carried out. The review addressed these objectives by seeking to answer four central research questions: 1) What are the different media-related risk factors for cognitive radicalization?
2) What are the different media-related risk factors for behavioral radicalization?
3) What are the relative magnitudes of the effect sizes for the different risk factors across the different outcomes? 4) Can heterogeneity within the strength of relationships be explained by differences between ideologies (e.g., right-wing and Islamist inspired), and regions (e.g., EU and the US)?

| METHODS
The methods for this review were pre-determined in a systematic review protocol published in the Campbell Collaboration journal (Wolfowicz et al., 2021a). Below we re-iterate the inclusion and exclusion criteria and the methods used in this review.

| Types of studies
The review sought to identify, collate, and synthesize observational studies, namely cross-sectional, longitudinal, and case-control designs. The outcomes from these studies were expected to be derived primarily from self-reports. However, the review also sought to include studies based on clinical reports (e.g., practitioner coded data), administrative data (e.g., from security services), and secondary databases constructed from multiple open sources (e.g., Profiles of Individual Radicalization in the United States [PIRUS]).
Experimental studies were included when the experimental manipulation involved some variation (e.g., at least two conditions) | 7 of 50 of some form of human-media interaction, such as exposure to news-media, videos. That is, the experimental condition differed in terms of the type of media that participants were exposure to, or the contents of the media that participants were exposed to assess differences in one of the eligible outcomes. This included labbased experiments, Internet-based experiments, and experiments using vignette designs. For these types of studies to be included, the treatment or control must have included at least one condition in which the exposure was theorized to increase the likelihood of radicalization. These criteria also served to distinguish the studies and this review from research (including systematic review) on counter-narratives (e.g., Carthy et al., 2020). While these studies' outcomes were also expected to be derived primarily from selfreports, they were also open to inclusion if based on clinical reports or observations (e.g., practitioner coded data). These studies were eligible for inclusion irrespective of the method of assignment (e.g., random, random blocked).
For all study types, it was necessary for them to display variation on the dependent variable, (e.g., a proportion of the sample displayed, or didn't display the outcome of interest in the case of dichotomous measures or displayed it to varying degrees in the case of ordinal, discrete or continuous measures).
Cross-sectional and longitudinal studies were included without any limitations on the nature of the sample (e.g., gender or age composition) or sampling method (e.g., Random, Representative, Quota, Convenience, Purposive, Snowball etc.).
For case-control designs, studies were included when the "case" sample was made up of a group of individuals who displayed one or more of the outcomes of interest (see below "Types of outcome measures") as assessed by self-reports, clinical reports, or administrative decisions (such as a clear violation of the law in line with the outcomes of interest described below). For these studies to have been included, it was necessary that the control sample would be made up of one of the following types of individuals: 1) Individuals assessed for or who were suspected of having displayed one of the outcomes of interest but who were found not to, or who were found to display it to a lesser degree than the "case" sample or some chosen threshold.
2) Individuals from the general population not displaying the outcome of interest or displaying the outcome of interest to varying degrees that is representative of the natural distribution in the general population.
3) Individuals not displaying the outcome of interest but who display a related, excluded outcome (see below "Types of outcome measures").
For these studies, controls could be chosen based on having been part of a single cohort (as in the case of prospective studies) or derived from a relevant data set (such as administrative records).
Studies were also eligible for inclusion if the matched controls were derived from a separate, related data set, as is often used in retrospective studies. Studies were also eligible for inclusion when the control sample was derived from the general population and included individuals considered to be at risk for displaying the outcome of interest specifically, or a distribution of individuals displaying or not displaying (or displaying to varying degrees) the outcome of interest.
For longitudinal studies, we included both prospective or retrospective studies, as well as whether they are cohort or panel studies, that measured the indicator and outcome of interest at least two different time points. We set no limits on the time between measurement but expected it to be a period of at least a few months for a study to be considered to be a true longitudinal design. For cohort studies, whether prospective or retrospective, we included studies in which the cohort had some shared characteristic, such as being from a specific age group, geographic setting, or social setting at the time of selection. For panel studies, whether prospective or retrospective, we included studies irrespective of their sampling (e.g., Random, Representative etc.).
Observational studies and experimental studies will be treated separately in the analysis.

| Types of participants
Previous reviews of the literature have highlighted the dearth of research concerning the media-radicalization relationship at the individual level. It has been found that most of the research focuses on the Internet, and specifically on platforms, content, groups, and networks. Whilst important, this type of research represents a distinct line of inquiry and is better described as the analysis of "online extremism" rather than "radicalization" (Odag et al., 2019).
This type of research does not provide the type of information-due to the units of analysis-that can inform us about the media effects on radicalization (Winter et al., 2020).
As such, given that previous reviews have already dealt with this distinct line of inquiry, the current review sought to address the gap in the literature concerning the individual level. The review therefore included studies in which the participants, or units of analysis were individuals. The review therefore excluded studies where the unit of analysis was some form of media content, network(s), group(s), or other units of analysis that were unable to provide information pertaining to this review's research questions and objectives.
The review placed no restrictions on inclusion based on the individual characteristics of the participants. The review included all samples regardless of the participants' age, gender, ethnicities, religion, or the type of radicalizing doctrine being investigated (e.g., right-wing, left-wing, religious etc.). Previous systematic reviews have found that such factors rarely impact the pooled estimates in meta-analyses of indicators pertaining to the same radicalization outcomes taken up by the current review (Wolfowicz et al., 2020). As such, while we combined studies examining the same indicators into a single analysis regardless of conceptual differences between the participants, where possible we explored participant population factors as possible sources of heterogeneity using meta-regression, and moderator analysis (see section on "Assessment of Heterogeneity").

| Types of factors
The review sought to include studies that examine at least one media-related factor as an independent variable that provides an indication of correlation with one of the outcomes of interest. A media-related factor was considered to be an exposure or experiential based indicator whose source is in some form of media.
For example, a known risk factor for radicalization is exposure to violence (Wolfowicz et al., 2020). This review considered in-person exposure to violence to be excluded but mediated exposure to violence (through television, social media, or other forms of media) would be an eligible indicator. In line with the review's objectives and research questions, the review made no predeterminations as to what media-related factors may exist. Nevertheless, in Table 1 we provide some examples of the types of factors that were expected to be identified.

| Types of outcome measures
In line with the literature and previous systematic reviews (e.g., Wolfowicz, Weisburd, et al., 2021b), this review examined two distinct, albeit interrelated outcomes, namely the cognitive and behavioral outcomes of radicalization. To ensure that the review included studies with comparable outcomes, the review limited its inclusion to studies whose dependent variables were assessed to be in line with the outcomes of McCauley and Moskalenko's (2017) Two-Pyramid Model of radicalization (TPM). Broadly speaking, the TPM was selected because rather than being a traditional process models of radicalization, it is an outcome-based typological model.
The TPM differentiates between the cognitive (attitudinal) and behavioral outcomes of radicalization, and also distinguishes between legitimate, legal forms of activism that are often incorrectly conflated with radicalization, and the types of illegal, radical behaviors that are of concern to society and authorities (Moskalenko & McCauley, 2009). The TPM's typologies provide a clear set of criteria for determining if an outcome falls in to one of its categories, as well as for the categorization of the outcome (Wolfowicz et al., 2020). Moreover, when studying related cognitive and behavioral outcomes in parallel, the cognitive outcome should have a high level of specificity with reference to the behavioral outcome (Fishbein & Ajzen, 1975). This criterion is met by the TPM's typologies as described below (Figure 1).
In line with the TPM, studies were classified as examining either cognitive or behavioral radicalization outcomes, with each of these outcomes including two sub-categories. With respect to cognitive radicalization the review included studies that examined radical attitudes or intentions. Studies examining radical attitudes were included when the dependent variable assessed support for, or justification of the use of radical violence, which is the use of violence toward persons in the name of a cause. This definition strikes a good balance between specificity and sensitivity, leaving room for the inclusion of violence motivated by a range of ideologies, and excluding non-ideological forms of violence.
Studies examining radical intentions were included when the dependent variable assessed intentions toward engaging in, or an expression of a willingness or desire toward the commission of radical violence. In both cases, studies were included irrespective of their use of single or multiple item outcome measures, if they included specific references to radical violence generally, or specific instances of radical violence. That is, while the review placed no specific restrictions on which measures were used, inclusion was limited to studies whose measures included such explicit reference to the use of some form of radical violence in the name of a cause.
While some reviews examining a broad, and large number of studies have previously assessed the relationship between individual factors radical attitudes and intentions separately (Wolfowicz, Weisburd, et al., 2021a), others have found it useful to combine them as a single measure of cognitive radicalization (Emmelkamp et al., 2020). Given the high degree of similarity in measurements used, and the fact that we did not expect as large a number of studies as has been found in previous reviews (e.g., Wolfowicz, Weisburd, et al., 2021a), we grouped studies examining radical attitudes and intentions together as a single outcome for cognitive radicalization.  When a study reported on two separate outcomes for cognitive radicalization (e.g., radical attitudes and radical intentions), two separate analyses were conducted in which alternative effect sizes were used. Alternatively, when two or more studies contributing to a particular analysis reported on two or more outcomes, sub-group analysis was conducted. Where the analysis for any particular factor included effect sizes derived from at least two studies for each of the outcome categories (attitudes and intentions), moderator analysis was performed to assess the degree of between outcome measurement heterogeneity (see "Sub-group analyses and investigation of heterogeneity").
With respect to behavioral radicalization, this review included studies that examined engagement in radical behaviors, which are sub-terroristic forms of illegal, radical violence (to be distinguished from legal, non-violent forms of activism), as well as studies that examined involvement in terrorism. Regarding radical violence, the review included studies that assessed self-reports of prior engagements in radical violence (e.g., Pauwels & Schils, 2016), as well as studies based on administrative data, or databases derived from both official and open sources. Regarding terrorism, the review considered terrorism to be any form of terrorism offending (planned or successful terrorism offending) as defined by the law of the country in which the offence occurred. The review also included studies whose samples were drawn from data from official and open sources (e.g., PIRUS). Studies examining both self-reported forms of radical violence and terrorism involvement were combined as a single outcome of behavioral radicalization. It was determined that in a case in which there were more than two studies from each of these categories in a single analysis, moderator analysis would be used to assess whether any between-outcome measurement heterogeneity exists (see "Sub-group analyses and investigation of heterogeneity").

| Duration of follow-up
While no limitations were placed on the duration of follow-up, it was determined that if a sufficient number of longitudinal studies would be identified, they would be grouped together by the time interval to follow up. We considered the length of time between two time points from 0 to 12 months as a short-term follow up period, 12-24 months as a midterm follow-up period, and 24> months as a long-term follow up period. It was determined that if a sufficient number of such studies were identified, a separate analysis would be conducted for each of the time point categories.

| Types of settings
No limitations were placed on the settings from which the samples of studies originated. Where possible, moderator analysis was used to identify differences in effects between different countries or regional settings.

| Electronic searches
Pilot searches were conducted to identify the most appropriate combination of search terms and strings that would achieve the optimal balance between sensitivity and specificity. To be able to properly address our first research question, we excluded from the search terms any reference to specific indicators/independent variables (e.g., television), so as to not to pre-determine or bias what factors would figure in the review (Shenderovich et al., 2016;Wolfowicz et al., 2020). Rather, in their place, we utilized general terms to capture all forms of media. At the recommendation of the editors from the Campbell Collaboration, who provided their input on the search strategy, we did include direct references to the Internet, as researchers are less likely to use terms like "media" to refer to the Internet. The search string also sought to ensure that the search results would be limited, as much as possible, to quantitative studies.
During the revision process, searches were re-conducted using the revised search string: Center database as well as International Political Science Abstracts (IPSA), we were unable to access these databases. We opted to include the Worldwide Political Science Abstracts database, which was not included in the original protocol, given that it covers the IPSA.
Searches were carried out in August 2020 and were run again in March 2022 before publication of the review as part of the review process.

| Searching other resources
Given that the topic of radicalization is known to be addressed by a range of disciplines (Wolfowicz et al., 2020), searches were carried out in several different sources. An initial search was carried out on the Campbell Collaboration and Cochrane libraries. We also carried out searches to identify existing reviews and screen their reference lists to identify possible studies that met the inclusion criteria. While the searches were being carried out, we also contacted several recognized experts (researchers and organizations) and presented them with our review topic, criteria, and initial results, with a request that they direct us to any missing studies that they may be familiar with that could meet the inclusion criteria. For included studies, which were subjected to full-text screening, we also reviewed their literature reviews and reference lists to identify whether they include any additional studies that may meet the inclusion criteria.

| Selection of studies
After searches were performed, all search results were downloaded into Endnote X9 and stored in a shared library that all reviewers and research assistants had access to. A double screening process was implemented in which two reviewers screened the titles and abstracts of retrieved items to assess whether they related in any way to the topics of interest. The screening was performed directly in Endnote, where the reference window displayed the titles and abstracts. All studies considered to potentially meet the inclusion criteria were copied to a separate Endnote folder entitled "First

| Data extraction and management
In addition to effect sizes, we also coded several study-level characteristics that are known to potentially impact heterogeneity and thereby affect the results or the way in which they should be interpreted. Where possible, we examined the effects of study level characteristics using meta-regression and moderator analysis. Since there is no consensus as to the minimum number of studies needed for conducting such analysis, we adopted a minimalist's approach in which the minimum number of studies needed for examining a continuous variable is 6 (Fu et al., 2011) and 2 studies per category for categorical variables (Marino et al., 2018). Rather than using meta-regression to justify the use of moderator analysis as originally determined in the protocol, we carried out moderator analysis for all factors for which a sufficient number of effect sizes existed. Table 2 includes the different characteristics that were coded.
All coding was performed by two coders, one of whom is the first author. Coding was carried out using Excel. At the end of the coding stage, coder inter-reliability was assessed. In all instances in which the coding of effect sizes differed between the coders, the coders recoded the data and conducted the analysis jointly. Similarly, the coding of study-level characteristics was coded by the same two coders.
Identified differences were rectified through a joint re-coding.

| Assessment of risk of bias in included studies
While the review sets few limitations on inclusion based on study quality, several measures of study quality will be coded as study-level characteristics to assess risk of bias. Where possible, we will attempt to analyze how these factors may impact the results using metaregression. Where this is not possible, the risk of bias elements of studies will be raised in the discussion of the results. For observational studies, we will carry out the assessment of risk based on the most relevant items derived from the ROBINS-E (Risk-of-bias in non-randomized studies of exposure) assessment tool. The selected risk of bias items that will be assessed are listed below in Table 3.
In the ROBINS-E guidance, overall ratings for risk of bias are determined by the highest risk of bias rating in each individual domain. This means that all domains essentially contribute equally to T A B L E 2 Study-level characteristics.

Variable Description
Effect size derivation Categorical: Effect size from bivariate or partial effect size. the overall judgment of a study. It has been noted that one of the limitations of this approach is that studies with high risk of bias in only one domain may be assessed to be of equally low quality as studies with high risk of bias in multiple domains. As such, we report the full risk of bias assessment so that users can easily see the different sources of bias that were identified in the included studies (Bero et al., 2018).
For randomized experimental studies, we used an adapted version of Cochrane's Risk of Bias tool for randomized trials (RoB2).
As with the observational studies, risk of bias relates to both the study and outcome levels, but there are two additional risk domains that differentiate experimental from non-experimental studies, The following items from RoB2 were included (Table 4).
Similar to the observational studies, experimental studies were assessed as being of low, high, or unclear risk of bias overall according to the highest assessment from any single domain. That is, if a study was assessed to have high risk of bias in one domain, we classify the entire study as being at high risk of bias.

| Measures of effects
There is a lack of consensus as to whether meta-analysis based on observational studies should give preference to effect sizes derived from bivariate correlations, or correlations derived from the results of T A B L E 3 Risk of bias measures for observational studies.

Variable Description
Dependent variable(s) Was the dependent variable measured using a validated instrument?
Independent variable(s) Was exposure to the independent variable measured using a validated instrument?

Measurement source
Was the source of the measurement for the independent/dependent variable based on self-reported, clinician reported, administrative reported, or other type of data?
Sampling methodology • Did the study use Systematic, stratified, convenience, quota, purposive, snowball, or another type of sampling?
Temporal ordering Can temporal ordering be established between the independent and dependent variable(s)?
Analytical rigor Did the statistical models used adequately control for key confounding variables?
Missing data Was missing data an issue in the study, and if so, how was it dealt with?

Results
Were results reported in a non-biased way (e.g., were non-significant results also reported)?
Reporting bias Does the study indicate that it has the data to report on a key relationship but fails to do so?
Outcome bias Does the study indicate that it has the data to report on a relationship with an alternative outcome but fails to do so?
T A B L E 4 Risk of Bias measures for experimental studies.

1.1
Was the allocation sequence random?
1.3 Did baseline differences between intervention groups suggest a problem with the randomization process?

2.6
Was an appropriate analysis used to estimate the effect of assignment to intervention?

2.7
Was there potential for a substantial impact (on the result) of the failure to analyze participants in the group to which they were randomized?

5.1
Were the data that produced this result [the results for this study] analyzed in accordance with a pre-specified analysis plan that was finalized before unblinded outcome data were available for analysis? multivariate regression models that have controlled for confounders.
While standardized coefficients derived from multivariate models may produce estimates closer to the "true" value, there is significant between-study variations in model specifications, and studies are often not comparable (Hanushek & Jackson, 1977;Hunter & Schmidt, 2004).
For this reason, many researchers give preference to bivariate correlations, which offer a consistent and uncontaminated measure across studies (Pratt et al., 2014). The decision whether to use bivariate, multivariate, or a combination of these effect sizes also depends on a study's objectives (Aloe et al., 2016). One of our objectives was to identify the relative magnitude of the effects for different media-related factors, and bivariate correlations provide more stable estimates for developing a rank-order of estimates among multiple factors (Hedges & Olkin, 1985). As such, we follow the approach that gives preference to bivariate correlations but allows for the inclusion of effect sizes standardized from regression models as supplementary effect sizes when that is all that is available. We will use moderator analyses to assess and report the effects of this approach (Aloe et al., 2016). We believe that this approach is the one most suited to the current review. It is also in line with related works that have been conducted (e.g., Najaka et al., 2001;Wong et al., 2010), and serves to enable the inclusion of the greatest possible number of effect sizes (Borenstein et al., 2009).
All effect sizes were standardized as Fisher's Z transformed variables (Borenstein et al., 2009;Rosenthal, 1984), which was also used for reporting of results. Effect sizes were derived from bivariate correlations, primarily from correlation matrices, or calculated from descriptive data such as means and standard deviations, t tests, χ 2 , analysis of variances (ANOVAs), and other classical hypothesis tests.
The calculation of bivariate effect sizes from descriptive data was carried out using the formulas and conventions of Lipsey and Wilson (2001), and the "Practical Meta-Analysis Effect Size Calculator" available through the Campbell Collaboration website.
For effect sizes derived from bivariate sources, standard errors were calculated based on the variance of the z transformed variable (1/n − 3), in which the standard error is calculated as being the square root of the variance.
Where a study reported only the results from linear regression models where the independent variable (IV) and dependent variable (DV) are both continuous we calculated r as x y In situations in which standard deviations (SD) were not reported, or when the IV was dichotomous and the DV continuous, or where the IV was ordinal or continuous and the DV was dichotomous, r was calculated based on the t ratio (B/SE): In instances in which both IV and DV were dichotomous, and only B was reported we first calculated Cohen's d and then converted this to r as follows: For these conversions, standard errors were calculated, whenever possible, based on a rescaling of the model-based standard error, which is calculated as When the SE was not reported, it was calculated from reported confidence intervals (CIs). In the case of 95% CIs, the SE was calculated as For experimental studies, while it is common to calculate the standard mean difference, Cohen's d, there is a tendency for overestimation of effects in small samples (Hedges, 1981). As such, we determined to use Hedge's g, which corrects for this bias. To calculate Hedge's g, we would first calculate the correction factor, j as With Cohen's d already having been calculated based on the means and standard deviations of the groups or other summary statistics, we calculate g = J × d. Subsequently we calculate the standard error as the square root of V g , with V g calculated as

| Dealing with missing data
When a study was missing information pertaining to study level characteristics or effect sizes, the following actions were taken in the order in which they appear: 5. Contact the authors with a request to provide the missing data.
It was originally determined that in a case in which a study reported on the effect of a particular factor as not having been statistically significant and no additional information was given or had been acquired through the above steps, an effect size of zero would be entered into the meta-analysis (Durlak & Lipsey, 1991).
Additionally, it was determined that if a study presented quantitative | 13 of 50 findings that were unable to be synthesized in the meta-analysis, because the statistic presented does not enable conversion or standardization, or there is not enough information available, the above steps would be followed.

| Assessment of heterogeneity
Heterogeneity was assessed using Cochran's Q and its associated p value, as well as the I 2 statistics. I 2 scores of >75 indicate high, >50 moderate, >25 medium heterogeneity, and >0 low heterogeneity.
When I 2 = 0 it indicates an absence of heterogeneity.

| Assessment of reporting biases 4.3.7.1 | Between studies
Reporting bias is a known issue with meta-analytic studies.
Commonly referred to as the 'file-drawer problem', it is widely understood that researchers tend to avoid publishing non-significant results, or occasionally results with depicting exceptionally small effects. As a result, the identifiable literature may not be representative of all the studies that have been conducted on a given issue, and the results from pooling these studies may overestimate the true effect size (Rosenthal, 1979). In this review two methods were used to assess reporting bias.
While the published protocol stated that we would use Rosenthal's (1979) Fail-Safe N test, based on comments received, as well as the approach taken in previous reviews (Wolfowicz, Weisburd, et al., 2021a), we replaced this approach with Egger's regression test (Egger et al., 1997). In Egger's regression test, the standardized effect sizes are regressed on their precisions. This is the equivalent of a weighted regression of the effect sizes on their standard errors, in which weighting is on the inverse variance. In the absence of publication bias, the intercept is expected to be zero (Rothstein et al., 2006). A statistically significant intercept above zero indicates the presence of publication bias.
The second approach noted in the protocol was maintained, namely the Trim-and-fill method (Duval, 2005;Duval & Tweedie, 2000a, 2000b. This method estimates the number of studies missing on the extremities of a funnel plot and augments the observed data to create a more symmetric distribution. Based on this, adjusted estimates and heterogeneity statistics are generated and enable the assessment of the degree to which the results are sensitive to reporting bias. As both methods suffer from their own limitations (Egger et al., 2001;Sterne et al., 2000), they have often been used complimentarily, including both in risk factor related research (e.g., Assink et al., 2015;Vazsonyi et al., 2017;Wolfowicz, Weisburd, et al., 2021a).

| Within studies
A common issue in different literatures concerns outcome reporting bias, in which studies may not report on the relationship between variables and certain outcomes even when it is clear that they have enough information to report on such relationships. In some cases, a study may make no mention of possible alternative outcomes, even though the researchers have collected data pertaining to it. To assess and account for outcome reporting bias, we will search for online supplementary materials and open access data for all included studies to identify any additional sources of reporting bias as they pertain to unreported outcomes. For studies for which supplementary materials or original data is found, we will code and report as to whether such materials provide evidence to support the identification of either of the two types of reporting bias described above.

| Data synthesis
Meta-analysis was conducted for each factor for which at least two unique effect sizes were found. All analyses were carried out using Biostat's Comprehensive Meta-Analysis (CMA) software (Borenstein et al., 2009). Random effects models were used for all analyses to account for heterogeneity. CMA V3 uses the Method of Moments approach of DerSimonian and Laird (1986) as the random effects estimator for τ 2 . Results were arranged in rank order according to the size of the pooled estimates for the different factors analyzed.

| Sub-group analysis and investigation of heterogeneity
To further investigate and understand sources of heterogeneity, and to identify ways in which estimates for different factors may fluctuate between different contexts and conditions, a number of additional analyses were planned.
As described above, meta-regression was used to assess the effects of continuous variables, and moderator analysis was used to assess the effects of categorical variables. The following variables were examined: • Region from which the sample was derived (EU, US, and Other).
• Ideological strain examined by the study (non-specific/mixed ideologies, Right-wing, Left-wing, Islamist, and Other (which included separatist and ethno-nationalist).
• Year of data collection.
• Mean age of study sample.
• Proportion of males in study sample. We adopted a minimalist approach in which a minimum of two studies from each category is needed to perform meta-regressions for categorical variables, and a minimum of six studies is needed for continuous variables. For categorical variables, wherever the meta-regression provides a statistically significant effect (p < 0.10), moderator analyses will be used to assess the degree of between category heterogeneity and differences in the estimates.
As our approach was to identify and utilize bivariate correlations as our preferred effect size, and only utilize effect sizes derived from multivariate models when bivariate calculations were not possible, we did not assess differences in the effect sizes from different studies based on the other (both included and excluded) covariates in a study. However, for any analysis for any factor that included at least two effect sizes derived from different sources (Bivariate or standardized partial effect sizes), we assessed the impact of combining the effect sizes through moderator analysis. The moderator analysis was used to identify the degree of between category heterogeneity and differences in the estimates across the categories.
As per the above, we did not carry out meta-regressions as a justification for this analysis as was originally considered in the protocol, as it was determined that this was an unnecessary step.

| Sensitivity analysis
Sensitivity analysis was carried out for each analysis that included 3 or more studies by using the "leave-one-out" method. In this method the meta-analysis is iteratively repeated k times (k = the number of studies), with a different study excluded at each iteration and the estimates and heterogeneity statistics re-calculated. The results provide for the ability to examine whether a specific study is responsible for greatly influencing the results (Viechtbauer & Cheung, 2010). We report on those analyses in which the removal of a single study was found to reduce heterogeneity by at least one level, with the levels being low (I 2 < 25), moderate (I 2 < 50), high (I 2 < 75), and very high (I 2 > 75).

| Results of the search
Our initial systematic searches were carried out in August 2020 and resulted in the retrieval of 36,732 unique items which were subsequently transferred to an EndNote library. Once imported to the library and fully updated, an initial screen was carried out based on their titles and abstracts. The initial screening sought to remove all studies that were clearly not related to the general topic of radicalization (or associated terms) in any way (such as studies from medicine examining "free radicals," environmental studies examining "extreme" weather, and studies examining more closely related topics such as fear of terrorism, or the impact of terrorism on different markets). In addition, all duplicates were removed at this stage.
Following this process, we were left with 569 items, which together with 4 items we received from experts, left 573 items to be reviewed.
The second screening stage included a more careful reading of the abstracts to identify whether studies dealt with a topic related to the current review. In some cases, where the abstracts were not clear as to what was being examined, a scanning of the methods sections of the papers was carried out. This process led to the removal of a further 469 items, as most of these studies were found to be examining group, platform, or content level dynamics, or using proxies for radicalization that fell outside of the inclusion criteria, such as voting for populist right-wing parties in democratic elections.
For the remaining set of 104 studies, we downloaded the full-text PDF for each item and attached them to their respective items in the EndNote library. From here, we reviewed the methodology sections of each item and identified whether the dependent variable(s) was in line with the inclusion criteria, and whether any of the independent variables measured a media-related factor. The final inclusion was 41 studies. In addition to these studies, we included 1 study that was sent to us by an expert (others that were received did not meet the inclusion criteria), 4 studies that we were familiar with, and which did not appear in our searches, and 1 unpublished study by the review's authors, for a total of 47 included studies. 2 In March 2022, following consultations with the editorial team of Campbell Collaboration's Crime and Justice Group, we re-ran the searches using a slightly modified search syntax, limiting search results to those published before August 2020 when the initial searches were carried out. These updated searches resulted in the retrieval of an additional 11,191 unique items. Following the same process as the initial screening, 87 potentially relevant items were identified in the first screening. However, upon a more careful inspection of the titles and abstracts, 20 of these items were removed. Among the remaining 67 studies, we excluded N = 43 based on their outcome not meeting the inclusion criteria, N = 4 for not having a relevant indicator variable, N = 6 for their unit of analysis (not individual level), and N = 5 for being qualitative, and N = 1 for having an invalid control group. 3 There were a total of four studies for which we were unable to locate a copy of the study online. After contacting the authors, two of these studies were provided to us, one of the authors failed to respond to our request, and another author (who was the sole author of the study) passed away some year ago.
While N = 6 studies met the inclusion criteria, N = 1 of them was the dissertation that preceded an already included study and did not include any new information (Luchsinger, 2018), and N = 2 others were published by the same author of an already included study (Pauwels & Schils, 2016) and did not provide any new information to be synthesized. and East-Asian countries (7.9%). There was an additional study which had a mixed sample of Europe and North America, as well as for North America and Asia. One study focused on a South American sample, and another on Australia. Table 5 provides a description of the included studies (more details are available in the Supporting Information Appendix).
From experimental studies, we extracted 12 effect sizes that pertained to manipulated exposure to mediated content theorized to increase radicalization. We also extracted an additional 4 effect sizes pertaining to relevant interactions between treatment exposure and trait aggression. From observational studies, we identified 23 mediarelated factors for which at least two effect sizes were identified, with a total of 110 effect sizes extracted for the primary analyses, and an additional 10 effect sizes extracted pertaining to sub-groups. Table 6 provides a description of these factors, which are listed in alphabetical order.
While the descriptions for many of the factors are quite straightforward, some are deserving of further elaboration, particularly what we refer to as "radical content." We consider "radical content" to be mediated content that is produced by individuals or groups with the goal of, or potential of increasing support or justification of terrorism (the use of violence in the name of a cause or ideology), or terrorist groups. This content usually makes use of violent images or messages which seek to generate sympathy or  study took a slightly different approach and examined whether participants had been exposed to violent content that they felt promoted violence against civilians and security forces. In the review, we distinguish between passive exposure to and the active seeking out of this type of radical content.

| Observational studies
Most of the observational studies that were excluded during the screening process failed to report on the relationship between a media-related factor and an included outcome of radicalization. In most instances, studies were excluded for failing to meet both criteria, and not just one.

| Experimental studies
The search results identified several studies that referred to themselves explicitly as "experimental" or otherwise described themselves in a way that indicated that an experiment was carried out. However, following full-text screenings it was identified that these studies could not be properly considered to be experimental due to the lack of a control group. with the different conditions. As per the study, the main independent variable was "grievances," and no group was exposed to a neutral condition. As such, the study's treatment does not represent a form of included media-exposure since it did not include a neutral condition. A similar approach was taken by Lemieux et al. (2017), and as such, this study was excluded on the same basis.
Additionally, some studies that otherwise would have met the inclusion criteria for being experimental studies examining an appropriate media-related factor were excluded based on examining an outcome that failed to meet the inclusion criteria. For example, Shortland et al. (2021) examined a theoretical aggressive response to a car accident.

| Risk of bias in included studies
The risk of bias assessment is contained in  outcomes, however these adapted instruments have either not been validated or otherwise lack evidence that they are valid measures for the outcomes of interest (e.g., Eyal et al., 2006;Gvirsman et al., 2016;Kremerman et al., 2012).

| Independent variables
Among the experimental studies, treatment included manipulated exposure to a piece of mediated content that was theorized to increase the risk of radicalization compared to a neutral piece of content, in line with standard practices in media-effects research (Ferguson & Savage, 2012). For two studies ( delinquency (Ferguson & Savage, 2012). When self-reports are used to measure exposure to specific types of content, there is an added source of potential bias, as specific interest in the topic of the content (e.g., politics) may influence memory recall of exposure to it (Donohew et al., 1998;Slater, 2004). This added source of potential bias is relevant to those risk factors included in the review which measure exposure to specific types of content (e.g., political, radical, or violent).

| Report type
The studies were primarily based on self-reported measures of both the dependent and independent variables. One of the studies measuring behavioral radicalization was based on a database derived from open sources that included a group of violent radicals and a control group of non-violent radicals (Sahani, 2018). Another study measuring behavioral radicalization was also based on an opensource database of convicted terrorists and compared it with a general population survey relying on self-reported measures (Clemmow et al., 2020).
T A B L E 6 Description of media-related factors included in meta-analysis.

Factor Description
Active exposure Active engagement with/seeking out radical content      While the broader radicalization literature includes some examples of clinical assessments of radicalization, these are usually made as part of risk assessments for individuals that have been referred to various authorities (e.g., Pfundmair et al., 2022). Selfreports of cognitive radicalization remain the dominant report type in the literature and are even common for measuring sub-terroristic forms of radical behaviors (see Wolfowicz, Weisburd, et al., 2021a).
The studies included in this review are therefore representative of the types of studies found in the broader literature on radicalization.

| Sampling strategy
Of the included studies, the most common strategy was to employ random sampling (N = 17), followed by representative sampling (N = 9) and stratified sampling (N = 7). Purposive sampling was used in N = 4 studies, Convenience sampling in N = 5 studies, snowball sampling in N = 2 studies, probability sampling in N = 2 studies, non-probability in N = 1 study, and mixed sampling strategy in another N = 1 study.

| Temporal ordering
In experimental studies, it was not evident that Time 1 measures of the outcome were controlled for, nor did any of the studies report on the collection of such measures.
Regarding observational studies, the review included only two longitudinal studies. These studies did not report on the same set of factors and as such it was not possible to conduct a separate metaanalysis for longitudinal studies (Gvirsman et al., 2016;Zhu et al., 2020).
For two studies that examined radical behaviors (Clemmow et al., 2020;Sahani, 2018), exposure to the risk factors inherently preceded the outcome of radical offending behaviors.
It is not possible to determine that exposure to the independent variable preceded the onset of the dependent variable for the remainder of the studies (N = 45) as they are cross-sectional.
Additionally, in the two experimental studies, pre-test measures of the outcome were not taken.  conduct a separate analysis on partial effect sizes controlling for similar sets of confounding variables.

| Missing data
Few of the studies noted any issues with missing data. However, this is not necessarily an indication that missing data did not exist as it is very common for publications to fail to note this issue. A few studies that noted the presence of missing data noted that it was handled either through exclusion from the analysis or using estimation techniques.

| Non-significant results
There was no clear indication of non-significant result reporting bias in all but one of the studies (Calenda & Mosca, 2007), which stated that certain relationships existed but failed to report on them.

| Excluded variables
There were N = 3 studies that explicitly mentioned variables that were apparently available in the data but then absent in reporting of the results. There were also several studies that relied on secondary data which are known to contain other potentially relevant variables that were not included (N = 6). There were N = 5 studies that had the ability to report on different types of media-related factors separately but only reported on them as an aggregate measure. For those studies for which such bias was not evident, we are still unable to fully discount the possibility that variables were omitted as studies were not pre-registered. 4

| Excluded outcomes
Two studies (Frissen, 2019(Frissen, , 2020) relied on a combined measure of cognitive radicalization, made up of both the SyFor and Activism-

Radicalism-Intentions-Scale (ARIS). While other radicalization studies
not included in this review have taken a similar approach (e.g., Rousseau et al., 2019), these scales may represent distinct constructs (Wolfowicz et al., 2020) and it thus would have been informative to have had the ability to assess effect sizes as they pertain to each of the scales separately.
In one study (Calenda & Mosca, 2007) correlations between different versions of the dependent variable and independent variables are reported on. In some cases, this study reported on individual items that made up the dependent variable, whereas in other cases they report on correlations with the index dependent variable.
In another study (Mulyana, 2020), insufficient information is given about the dependent variable. However, as it is described, it appears that an additional measure of radical intentions is available but not reported on.
In Pauwels and Schils (2016) study (see also Pauwels et al., 2020), only one version of the dependent variable is reported. However, in data supplied by the authors multiple versions of the dependent variable were available.
There was no evidence of pre-published or registered protocols for any of the studies. As such, even where there was no evidence of excluded outcome bias, we are still unable to discount the possibility that they employed more than one outcome measure but only reported on one. We are also unable to discount the possibility that additional models were used in which the dependent variable was measured differently (e.g., dichotomized). 5

| Allocation sequence
For three of the experimental studies, allocation sequence was reported as having been made at random. Allocation sequence was not explicitly mentioned in n = 1 study (Azeem et al., 2019).

| Baseline differences
Experimental studies differed significantly with respect to the reporting of baseline differences between groups. Azeem et al. (2019) reported some basic characteristics and checked for differences using ANOVA. Another study included extensive baseline characteristics in the supplemental materials (Jones & Paris, 2018). Two other studies make no mention of assessing baseline characteristics, nor were such characteristics detailed.
However, these studies did include age and gender as controls in their statistical models (Kalmoe, 2014;Shortland et al., 2020).

| Appropriate analysis
Two of the experimental studies used OLS regression to analyze the data and report results (Kalmoe, 2014;Shortland et al., 2020), and another used a one-way ANOVA (Azeem et al., 2019). Another study compared groups using t tests (Jones & Paris, 2018 were also available that provided additional information.

| Failure to analyze participants
For experimental studies there was no indication of a potential for a substantial impact (on the result) due to a failure to analyze participants in the group to which they were randomized. 4 Pre-registration of studies, such as in the form of a research protocol, are rare in these fields. However, pre-registration helps to reduce this form of bias as researchers would For experimental studies, there is no evidence of a pre-specified analysis plan having been in place before data were available for analysis.

| Multiple outcomes
For experimental studies there was no evidence that results were selected on the basis of the results from multiple outcome measurements.

| Multiple analyses
For experimental studies, given the choice of analysis (OLS) over a simpler comparison of means (such as t tests), we cannot discount the possibility that results were selected based on results stemming from multiple analyses. However, it is of note that the studies also reported on results that were not statistically significant.

| Observational studies
A total of 23 factors were identified for the outcome of cognitive radicalization, consisting of 3 factors with negative associations and 20 factors with positive associations with the outcome. Only two factors were identified for the outcome of behavioral radicalization (see Table 6 for a description of these factors). Following the classification of very small, small, moderate, and large estimates, most of the factors had very small-small estimates (Hopkins, 2002).
Among the factors with negative associations were Internet Access, Newspaper reading, and Radio listening. In addition to the effect sizes being exceptionally small, they were also not statistically significant. Among factors with positive associations, 12 had very small estimates ranging from r = 0.01 to 0.08. These included, in rank order: Television consumption, Facebook usage, using the Internet to consume political news, engaging in content piracy, time spent online, trust in media, Twitter usage, posting of political content online, general media consumption, online network participation, technical skills, and being a victim of cyberbullying. The next tier included factors with small to moderate estimates (r = 0.10 to 0.26). Following the rank order, similar estimates were found for factors pertaining to general components of Internet and media usage. This included: Selfcensorship online, hacking, usage of online discussion forums, attachment to online networks, active seeking of radical content online, passive consumption of radical content online, and consumption of ISIS news. An estimate that can be categorized as large was found for a single factor, namely willingness to engage in ideologically motivated cyber-attacks (r = 0.57).
The results show that certain attitudinal and experiential factors, as they pertain to the Internet, have effect sizes that are not substantively different from general Internet usage. They also show that more specific forms of online activity, as well as some attitudinal factors, in particular a willingness to engage in ideologically motivated cyber-attacks, demonstrate more robust relationships with radicalization than general forms of consumption.
As opposed to cognitive radicalization, for behavioral radicalization the included studies only provided effect sizes pertaining to two factors for which meta-analysis was possible, namely passive and active forms of consumption and engagement with radical content.

| Experimental studies
The experimental studies included in this review were combined in a single meta-analysis to examine the effects of 'one-off exposure to radicalizing content'. However, the nature of the exposure differed between the studies. Kalmoe (2014) Shortland et al. (2020) was the only study to expose the treatment group to an explicitly radical video, being an approximately 6-min-long propaganda video produced by the ISIS terrorist group. Two of the studies, which included a total of four samples (Kalmoe, 2014;Shortland et al., 2020), also examined interactions between exposure to the respective experimental conditions and trait aggression ( Table 9).
The results of the random effects meta-analysis (k = 12, n = 3196) found a pooled estimate of g = 0.08 [95% CI = −0.03, 0.19], which presented with more than a moderate degree of heterogeneity to radicalize was associated with a small increase in risk, larger than exposure on its own (Figure 4). 6 As per the protocol, results from experimental studies were to be calculated and reported as Hedge's g. However, four of the effect sizes were calculated from unstandardized regression coefficients. On account of this, we also report here that the reported estimate is equivalent to r = 0.04 [95% CI = −0.02, 0.09]. Similarly, the estimate for the interaction between exposure and trait aggression was r = 0.06.

| Heterogeneity
As per Table 3, heterogeneity was found to be very high (I 2 > 75) for 13/23 factors derived from observational studies for cognitive radicalization, and both the factors pertaining to behavioral radicalization.
Heterogeneity was high for one additional factor, moderate for two factors, and absent for seven factors, of which four factors were made up of only two effect sizes. Heterogeneity was moderate for both analyses of experimental studies. As such, and following the protocol, we sought to investigate a range of potential sources of heterogeneity.

| Region
There were an insufficient number of studies to enable examining heterogeneity between countries. As such, studies were pooled together by region or country type to facilitate a meaningful analysis.
The analysis was carried out for ten of the factors pertaining to cognitive radicalization, and one factor pertaining to behavioral radicalization (see Table 11).
For cognitive radicalization, statistically significant differences were found for only one factor, and marginally significant differences were found for two additional factors. For General media use, the estimate for studies from EU countries (k = 6) of r = 0.12 was significantly larger (Q between = 6.788, p = 0.009) than the estimate for studies from other countries (k = 5) of .03. For Posting of political opinions online, there was marginally significant between-group heterogeneity (Q between = 2.743, p = 0.098) for EU studies (k = 3), which had a pooled estimate of r = 0.06 and other countries (k = 2), which had a pooled estimate of r = 0.09. For Twitter usage, the estimate for EU-based studies (k = 2) of r = 0.13, was larger (Q between = 5.068, p = 0.079) than the estimate for studies from the US (k = 2) of r = 0.04 and from other countries (k = 2) of r = 0.02.

| Ideological strain
Due to the small number of studies identified in the review, it was only possible to analyze heterogeneity based on the ideological strain examined for N = 7 risk factors, among which there were no cases of statistically significant differences between-ideology heterogeneity. For experimental exposure, while studies examining Islamist radicalization had a noticeably larger estimate, the between-group heterogeneity was not statistically significant ( T A B L E 10 Meta-regressions for study-level characteristics.  While the differences in the size of the estimates is apparent, the difference was only marginally significant as indicated by the extent of between-group heterogeneity (Q between = 2.755, p = 0.097).

| Leave-one-out analysis
Leave-one-out analyses were carried out for all analyses that included >3 effect sizes and for which the I 2 statistic indicated any heterogeneity (>0). The analyses are presented for all factors for which the removal of a single study led to a significant reduction of heterogeneity. Significant reductions in heterogeneity were found for 9/14 factors analyzed. Heterogeneity was reduced from very high to high for two factors. Heterogeneity was reduced to 0 for four of the factors, although for one of these factors the removal of the one study left only two effect sizes in the analysis. For one factor, ISIS T A B L E 12 Moderator analysis for ideological strain.  (2) Beheading vid 0.16*** 0.11, 0.21 1.101 9.186 2189 (2) Abbreviations: 95% CI, 95% lower and upper confidence intervals; I 2 , heterogeneity statistic; k, number of studies; N, total sample size; Q, Cochran's Q heterogeneity statistic and χ 2 test p value; r, correlation coefficient.  Table 3 as representing the true rank order of the relative magnitude of effects.
At the same time, most of the factors with small, and essentially inconsequential effects, display little or no difference in this analysis, indicating that they are more robust to the impacts of outlier bias (Table 14).

| Single effect sizes
The systematic review identified several other media-related factors for which only a single effect size was found, and thus meta-analysis was not possible. However, it is important to consider the nature of these factors as part of the broader scope of the review. . Finally, the study also examined the extent to which respondents felt that illegally accessing computer systems was beneficial for society, which was also associated T A B L E 14 Leave-one-out analysis. Note: The estimates for factors pertaining to factors under the categories of "cognitive" and "behaviors" are reported as r correlations and for "experimental" as Hedge's g.

| Efficacy of online activities
Brunsting and Postmes (2002)  . It is of note that increased parental involvement/control has been found to be a protective factor against radicalization, albeit with somewhat larger effect sizes (Wolfowicz, Weisburd, et al., 2021a).

| Other online platforms
While the review only found enough studies to meta-analyze

| Network characteristics
In a longitudinal study, Zhu et al. (2020) found highly significant effects for the role of network heterogeneity as a mediator of social learning variables and a predictor of self-reported radical behaviors (r = 0.30, 95% CI [0.17, 0.42]).

| Publication bias
Publication bias analysis was carried out for all factors that included a minimum of three effect sizes. Egger's regression was found to be marginally significant for only one factor, Network attachment, and was not statistically significant for all other factors. The Trim-and-Fill method identified missing effect sizes for nine factors for cognitive radicalization, one of which was Network attachment, and both factors pertaining to behavioral radicalization. In all such cases, between 1 and 2 missing studies were identified, and the maximum difference between the original estimate and the adjusted estimate did not exceed r = 0.09 (network attachment). Thus, most of the factors analyzed were robust against publication bias and the differences emerging from the trim-and-fill analysis do not change the interpretation of the results (see Table 15).

| Summary of main results
Despite a robust literature on media effects and criminological outcomes (e.g., violent cognitions and behaviors), the literature on media effects and radicalization is quite underdeveloped. The bulk of the research has been devoted to examining radical content itself, or the way in which different platforms are used as part of the strategies of radical groups to spread their messages and ideologies. Relatively little research has been carried out to examine the effects of exposure to these platforms and content on individuals' radicalization . What individual level research has been conducted, has mostly been limited to the analysis of mere passive exposure to radical content (Richards & Wood, 2020;Wolfowicz, Weisburd, et al., 2021a). It remains that little is known about differences in the effects of media-related factors on radicalization, including with regard to mediums (e.g., television and Internet) and types of content (e.g., political, violent, and radical; Scrivens & Conway, 2019).
In this systematic review we sought to identify the different media-related factors for which there exists quantitative estimates concerning their relationship with the two primary outcomes of radicalization: cognitive and behavioral. The primary objective of the review was to identify which factors had the strongest relative relationships with these outcomes, and to integrate the findings A sub-set of experimental studies examined the interaction between treatment and trait-aggression, which is theorized to (a) Abbreviations: 95% CI, 95% lower and upper confidence intervals; Egger's test, test statistics and associated p value; g, Hedge's g; Q, Cochran's Q heterogeneity statistic; r, correlation coefficient; r adjusted /g adjusted , correlation/Hedge's g adjusted after trim-and-fill; T&F, Trim-and-Fill.
increase the effects of exposure or (b) confound the effects of exposure. Indeed, the pooled estimate derived from four samples was somewhat larger than the pooled estimate for simple exposure described above. Due to the small number of effect sizes, it was not possible to investigate possible sources of heterogeneity. However, a leave-one-out analysis found that the removal of one study reduced heterogeneity to zero. In doing so, the size of the estimate increased, indicating a possible downward bias of the pooled results.
The results from the analysis of observational studies provide a degree of corroboration to the findings from the experimental studies. The results showed that for factors measuring exposure to mediums, irrespective of the medium, the magnitude of the effects is exceptionally small. Even when factors measure exposure to specific types of content (e.g., political or violent), irrespective of the medium, estimates remained exceptionally small. The relative magnitude of the effects for these factors were virtually indistinguishable from those that were not content-specific. In moving down the rank-order, only the effect sizes for seven factors were larger than r = 0.10, and only five of these were larger than r = 0.20. What distinguishes these factors from those noted above is that they were all Internet-related factors. Among these factors, the smallest effect sizes were for online self-censorship and posting on forums. These were followed by larger estimates for online network attachment, active seeking of radical content online, passive exposure to radical content online, consumption of ISIS news, and willingness to engage in ideologically motivated cyber-attacks.
While caution is certainly warranted in interpreting the results, they do point to similarities with evidence from the broader field of media-effects research on deviant outcomes which has also found that the more specific content type is to the cognition being measured, the larger the statistical relationship (Anderson & Dill, 2000;Anderson et al., 2007;Coyne et al., 2018;Gentile et al., 2004;Martins & Weaver, 2019). They also support the idea that as a medium, the Internet can have greater impact on cognitions and attitudes than television for example (Shah et al., 2001). In this regard, it has previously been found that the active posting of political content online has a greater impact on outcomes such as civic engagement than exposure to political content through other mediums (e.g., Shah et al., 2005). The potentially greater impact of the Internet over other mediums may also be explained by the various types of attitudes, emotions, and behaviors that individuals can develop toward it. For some, attachment to their online networks can be even stronger than their attachment to offline networks (Neumann, 2013). In this regard we found that a tendency toward self-censorship, and feelings of attachment to online networks, have-relatively speaking-robust relationships with cognitive radicalization. These results demonstrate that like with offline associations and networks, individuals are susceptible to social control online as well (Wolfowicz, Weisburd, et al., 2021a). However, the analysis demonstrated that attachment to online networks is sensitive to outlier bias. As such, the true magnitude of the effect for this factor may be smaller than what was found in the primary analysis.
In this regard, in a larger systematic review and meta-analysis of general risk and protective factors for radicalization, it was found that different forms of offline social bonds have relatively robust relationships with radicalization (Wolfowicz et al., 2020). The review found that while certain social bonds can have risk effects, others have protective effects. It has long been found that increased Internet usage can correspond with a decreased engagement with offline social bonds, especially those that can have a protective effect such as parents (Kraut et al., 1998). On the other hand, increased involvement with online networks may serve as a replacement for weak offline bonds. Weak offline social bonds, as well as stronger online network attachment have been found to increase the likelihood of exposure to online radical content (Oksanen et al., 2014).
Additionally, greater online network attachment increases the likelihood of agreement with the messages of such content (Bernatzky et al., 2021).
Similar to what has been found in the broader literature on media effects and deviant outcomes (both cognitive and behavioral), even for the most important factors, effect sizes are objectively small (Valkenburg et al., 2016). However, with reference to the metaanalysis of Wolfowicz, Litmanovitz, et al. (2021), the estimates for passive and active exposure as they pertain to cognitive outcomes would be situated at approximately the halfway point of the rankorder of known risk factors. More striking is that for behavioral radicalization, the effect sizes would rank much higher, with estimates being quite close to those of some of the most important criminogenic factors, including thrill seeking (r = 0.19), low selfcontrol (r = 0.28), deviant peers (r = 0.30), and radical cognitions themselves (r = 0.30).
While the results indicate that traditional forms of media (even those including general political or violent content) may have essentially insignificant relationships with radicalization, they also serve to allay the minimalist perspective that Internet usage has little relationship with radicalization (Awan, 2007a). However, given the nature of the data we are unable to say what the nature of this relationship is. It is certainly possible that through social learning processes, exposure to certain types of content and associations increases radicalization. This process can be accelerated through the formation of stronger social bonds, and the adoption of group-based norms and values. On the other hand, it is perhaps equally possible that nature of the relationship between Internet usage and radicalization is more one of amplification. That is, in the context of radicalization, Internet usage's most significant effect is in providing already cognitive radicalized or radicalizing individuals with the opportunity to engage with content and associations who can provide confirmation for their beliefs, and thereby increase the potential for engaging in radical behaviors. This assumption is analogous to evidence on media effects in which it has been found that individuals with aggressive tendencies preferentially search out violent media content, which in turn reinforces, or increases their aggression, and thereby increase the potential for engaging in aggressive behaviors (Slater et al., 2003). Similar to evidence that engagement in radical behaviors may be the biggest risk factor for | 35 of 50 radical cognitions (Moskalenko & McCauley, 2020), it is necessary to consider the possibility that radicalization is as likely to be a risk factor for exposure to radical content as exposure to radical content is for radicalization. Previously held radical and violent beliefs have been found to be predictive of engagement with radical content (Hawdon & Costello, 2020;Pauwels et al., 2020;Hawdon et al., 2019).
These two possibilities are analogous to the competing paradigms of media socialization (where problematic media usage is a predictor of deviant outcomes) and media selectivity paradigm (where problematic media usage is an outcome of deviance (Slater, 2007). A third possibility that must be considered it that other risk factors for radicalization may draw individuals to engagement with radical content. These factors may be socio-demographic, in particular age and gender, or may related to certain propensities, such as low selfcontrol and sensation seeking (Pauwels et al., 2020;Hawdon et al., 2019). As Felson (1996) argues, it is possible that mediaeffects on violent outcomes (both cognitions and behaviors) represent imitations of the self-control displayed by the characters that feature in the content, rather than violence in and of itself. More research is needed in order for the nature of the media-radicalization relationship to be better understood.
The role of media effects in criminality (and radicalization) has traditionally been conceptualized, like other risk factors, as being a moderator between attitudes and behavior (Kanz, 2016). While metaanalysis is not suitable for examining this type of theorized path model, meta-analytic results can sometimes point to the existence of such a relationship. Generally, when analyzing related cognitive and behavioral outcomes, effect sizes for risk factors are almost always larger for the cognitive outcomes. In those instances where effects are found to be larger for the behavioral outcome, it is suggestive of the variable being a moderator of the continuity between the cognition and behavior (Ajzen & Fishbein, 1980;Bosco et al., 2015;Wolfowicz, Weisburd, et al., 2021a). In this regard, our review found that the effect sizes for passive and active exposure to online radical content on behavioral radicalization were effectively identical to those for cognitive radicalization. While in and of itself this does not tell us much about the nature of the relationship, it does provide indication that it is worthwhile to further explore the potential moderating effects of these factors.
Overall, a degree of consistency can be seen between our findings and those concerning media-effects for violent cognitions and behaviors more generally. However, the current state of the body of evidence still has some way to go before we can make any claims of causality, which can currently not be made. Taken together, the results of this review should serve as a source of direction for research. In recent years there has been significant growth in criminological applications to the study of radicalization and terrorism with respect to a wide range of issues. To a large degree it can be said that the degree of overlap between radicalization/terrorism and other forms of deviance is greater than the degree of divergence (LaFree et al., 2020). With this being the case, we hope that this review will serve as a source of inspiration for better integrating the rich study of media effects into the radicalization research agenda.
6.2 | Overall completeness and applicability of evidence  (Hopf et al., 2008). Moreover, the results of the review highlight the lack of case-control studies comparing violent and non-violent extremists (cognitive and behavioral radicalization).
These types of studies are needed to identify what types of media-related factors, activities, and behaviors may differentiate between these two groups. The absence of evidence derived from this type of inquiry represents a significant hindrance to the development of more evidence-based policies and practices (Freilich & LaFree, 2016;Scrivens et al., 2020).
In addition, our review identified several factors for which only a single effect size was found, and as such no meta-analysis was conducted. With regard to mediums, single effect sizes were found for Video-game play (Eyal et al., 2006), Music (Manzoni et al., 2019), and YouTube usage . We also found single effect sizes for specific aspects of Internet usage, namely network structure characteristics such as network heterogeneity (Zhu et al., 2020), and user-profile level activity such as receiving or giving of "likes" (Goede et al., 2019). Regarding network structure characteristics, one of the most frequently referred to mechanisms of online radicalization is that of the echo chamber, which describes an insular, isolated, and homogenous network. There is now an extensive literature that has found statistically significant relationships between several different network-level characteristics (such as density, homogeneity, tie strength etc.) and a range of cognitions and behaviors (Chancellor & De Choudhury, 2020). However, as demonstrated by our results, there is little evidence concerning echo-chamber related factors and individual radicalization outcomes (Macdonald & Whittaker, 2019). Regarding specific aspects of online behaviors, there is also a growing body of research that demonstrates that factors such as posting frequency, sharing, and the receiving of likes and comments, may be highly predictive of a range of cognitive and behavioral outcomes (Chancellor & De Choudhury, 2020). While some studies have demonstrated the usefulness of such factors in improving automated detection of radicalization online (e.g., Nouh et al., 2019;Wolfowicz, Perry, et al., 2021), our review highlights a lack of evidence concerning their relationship with radicalization itself.
Overall, it is evident that there are additional media-related risk factors for radicalization for which there is currently insufficient evidence to enable a meaningful synthesis. Given the potential importance of these factors, as well as those that were analyzed These factors represent areas that demand further research.

| Quality of the evidence
Like other research on risk factors that relies primarily on crosssectional studies, the nature of the relationships examined in the study can best be classified as putative risk factors. That is, whilst the effects demonstrate correlations in the theorized direction, they lack confirmation of the temporal ordering needed for classification as risk or protective factors (Wolfowicz, Weisburd, et al., 2021a). As a reflection of the quality of the studies in the literature, this represents a limitation on the types of inferences that can be made considering the current state of the evidence. Indeed, the review included one longitudinal study that did control for previously held beliefs and mediated exposure to violence and found that in doing so, there was no relationship between exposure and later radical attitudes (Gvirsman et al., 2016). Whilst conclusions cannot be drawn from a single longitudinal study, previous research on media effects in criminology has found that when controlling for such factors, media effects are substantially reduced when compared to cross-sectional studies (Anderson et al., 2010).
Relatedly, the included studies rely on self-reports of media exposure and usage. It has often been noted that individuals have difficulties in accurately recalling their actual media use and consumption, which can often mean significant over and under reporting (Prior, 2009a(Prior, , 2009b(Prior, , 2012. While these are sometimes the best measures of individual exposure that are available (Slater, 2004), they do introduce a source of potential bias.
As with all studies dealing with observational data, the most important quality criteria pertain to measures of the dependent and independent variables, appropriate sampling, and appropriate comparisons made statistically (Murray et al., 2009

| Independent variables
The nature of the independent variables examined in this review are like those that are examined in the broader study of media-effects.
That is, the independent variables capture self-reports pertaining to media usage and consumption. While these types of measures are quite standard in media-effects research, they are sensitive to under and over estimations of exposure time (Prior, 2009a(Prior, , 2009b(Prior, , 2012. Additionally, self-reports are sensitive to memory recall bias, especially when the reports pertain to topical content toward which respondents possess differential levels of interest (Donohew et al., 1998;Slater, 2004 aggression, and gender. It has previously been suggested that bivariate correlations between violent media consumption and violent outcomes may simply represent gender effects, or the effects of trait aggression, which is also highly correlated to gender (Ferguson, 2015). Additionally, these and other factors may impact differential exposure to media (Savage & Yancey, 2008). Given the nature of the data, we were unable to analyze effects adjusted for such factors and we were unable to account for this potential source of bias. As such, whilst the effects identified in this review certainly follow the theorized direction, it is also entirely possible that more radical individuals are simply more likely to engage in the types of media behaviors analyzed.

| Publication bias and sensitivity of the results
Publication bias and sensitivity analysis were limited to factors for which there were more than three effect sizes. For most factors, there was no evidence of significant publication bias or sensitivity to | 37 of 50 outliers. For those factors for which these analyses indicated changes to the pooled estimates, the differences were exceptionally small and do not change the substantive findings. However, for one factor, network attachment, the results are inconclusive. The leave-one-out analysis indicated that one study significantly contributed to an inflated estimate. In removing this study, the pooled estimate was reduced from r = 0.20 to r = 0.07. On the other hand, the Trim-and-Fill analysis imputed two missing effect sizes to the right of the pooled estimate, increasing the estimate to r = 0.29. These results indicate that until additional effect sizes can be synthesized, the results for this factor should be interpreted cautiously. The only other factor for which there was significant evidence of publication bias was for exposure to radicalizing exposure in experimental studies.
Here, the Trim-and-Fill analysis indicated a small reduction in the effect size.
6.4 | Limitations and potential biases in the review process

| Review process
We acknowledge that there are studies that may provide important evidence concerning the risk and protective factors for radicalization that were not included in this review on account of their outcome measures failing to meet the review's inclusion criteria. For example, some studies are known to assess "willingness to die for a cause/group." However, as described above, such studies were excluded since a willingness to die does not necessarily indicate a willingness to use violence against others (e.g., Bélanger et al., 2014).
Other studies have examined the effects of different forms of media consumption on outcomes such as racism and hate-speech (Soral et al., 2020). Yet others, such as Machackova and Šerek (2017), analyzed how online political activity predicted the acceptance of "non-conventional" activism, which included a vague reference to "run-ins with authorities." We do not believe that the review's results are biased because of having excluded such studies. Rather it means that our results are based on a more homogenous set of outcomes and should therefore be more robust from a metaanalysis perspective. However, we can note here that the findings from these studies demonstrate a significant degree of overlap with our results.
It is also important to highlight the fact that several of the metaanalyses performed in this review are based on a very small number of effect sizes (as few as two) and some of those with a greater number of effect sizes demonstrate significant heterogeneity. As such, caution must be used in drawing inferences from our analyses and interpreting our results.
Another potential limitation of this review is language. While it is common to assume that the most important studies are at least indexed in databases in English, we cannot discount the possibility that studies in other languages exist that we were unable to identify.
Indeed, as our results found a handful of studies examining samples in Asia, we suspect that additional studies from these countries may exist in local languages. Future attempts to synthesize the body of evidence may consider dedicating more significant resources to attempting to identify studies published in other languages.
Whilst we encourage replication, we acknowledge that the authors' familiarity with the literature may have impacted the number of studies that passed through the different screening stages. As such, while a replication will likely reach similar substantive findings, differences in inclusions and exclusions at different stages would be expected.

| Agreements and disagreements with other studies or reviews
To date, only one other review has specifically focused on the mediaradicalization relationship, placing a specific focus on the Internet.
That review was unable to provide any sort of quantitative synthesis however, and it also included studies that were not individual-centric.
Nevertheless, the conclusions of that review were that Internet usage can be a source of risk for radicalization (Hassan et al., 2018), which receives support by the results of the current review. In a larger review of risk and protective factors for radicalization in OECD countries, Wolfowicz, Litmanovitz, et al. (2021)  Evidence from the broader field of research on media effects for outcomes related to violent cognitions and behaviors demonstrates some degrees of agreement with the results of the current review.
First, Martins and Weaver (2019) find that the degree of specificity between the type of content and aggression has a significant impact on effect sizes, similar to the results of our review. Here, they found that while exposure to general media had an effect of r = 0.08, exposure to general violent media had an effect of r = 0.15, and relational violent media (specific to aggression outcomes measured) had an effect of r = 0.21. Figure 5 provides a juxtaposition of the results from Wolfowicz, Litmanovitz, et al. (2021) and Martins and Weaver (2019) with the current review.
Moreover, in recent years, been primarily focused on the effects of video games. While the current review was unable to examine the effects of video games, one similar finding is that the effects of violent video games are stronger for behavioral than cognitive outcomes of aggression (Ferguson et al., 2020). Another area of overlap relates to the relative magnitude of media effects compared to other known risk factors. In this regard, whether relating to aggressive cognitions or behaviors, media effects generally rank at about the halfway mark in rank orders of other commonly examined risk factors (Comstock et al., 2014).

| Implications for practice and policy
The review only identified two factors for behavioral radicalization, passive and active exposure to radical content, and the quality of the evidence is poor. Nevertheless, the relative magnitude of the effects suggests that relationships for these factors are relatively robust. The findings therefore support a policy focus on the Internet that views it as a potential source of risk. In practice, problematic Internet usage, and engagement with radical content online in particular, should be investigated as being added as a risk factor item in risk assessment. It would seem intuitive for interventions to aim at reducing the availability of radical content online. This is especially important given that a sizable proportion of individuals, especially youth and young adults, may actively seek out, engage with, or accidentally come to be exposed to radicalizing content (e.g., Frissen, 2021;Grizzle & Tornero, 2016). Simultaneously, whether through counter-narratives or other means, it is important to seek ways to reduce the influence of radical content. However, in developing strategies it is important to weigh other considerations.
In this regard, it is possible that by permitting individuals engagement with radical content online, it may provide them with a non-violent outlet to voice their grievances. Additionally, the monitoring of online activities is an important aspect of counterterrorism efforts and may provide an opportunity for early prevention (Wolfowicz, Weisburd, et al., 2021a). Whether it pertains to the Internet or other forms of media, policy efforts to control access and availability to content, as well as the leveraging of the Internet for surveillance and prevention efforts, should be carefully considered against possible impingements on free speech and civil liberties (Hasisi et al., 2019). It is therefore important for policy makers to support research than can underpin the development of more evidence-based policies and practices, especially case control research that can serve to identify factors that differentiate between violent and non-violent radicals (e.g., Scrivens et al., 2021;Wolfowicz, Weisburd, et al., 2021a).
Currently, the most common strategies to combat the risks of Internet-based radicalization are (1) content removal, (2) counternarratives, and (3) digital literacy. However, the evidence based for each of these approaches is lacking (Zeiger & Gyte, 2020).
Additionally, in some cases there is evidence of the potential for iatrogenic effects. Below we discuss these issues in brief and highlight how the results of our analysis relate to them, as well as potential new areas that should potentially be considered.
With respect to the "takedown" of content, profiles, and pages, government agencies actively remove radicalizing content or compel IT companies to carry out such removal. In recent years, companies such as Twitter have taken an active approach to the removal of radical content and profiles. However, this approach has been criticized for failing to reduce access, as new profiles quickly replace those removed . Additionally, radical groups may use content removal as being demonstrative of their claims against governments and societies as being discriminatory toward their group (Neumann, 2013). Moreover, by removing radicals from more open platforms (such as Twitter), they may move to platforms that are more difficult to monitor (such as Telegram). Furthermore, content filtering and removal is usually driven by algorithms, and inevitably leads to the removal of legitimate content and may impinge on civil liberties (Hasisi et al., 2019). There is indeed evidence that following a crackdown on ISIS-related Twitter accounts, the group has migrated to Telegram. And while their reach through Telegram may be significantly smaller than it was on Twitter, it is believed that the move to the platform has led to a more radicalized and committed following . Given that the magnitude of the estimates for radical content exposure are objectively only small to moderate, and the potential for confounding is more likely to be associated with overestimation rather than underestimation, policy makers should carefully consider the costs and benefits of takedown policies.
With respect to counter-narratives, these are meant to balance out. According to social learning theory, it is the balance of definitions favoring or dissenting from a particular belief or behavior that determine whether an individual will adopt the given belief or F I G U R E 5 A comparison of the current review with Wolfowicz, Litmanovitz, et al. (2021) and Martins and Weaver (2019) | 39 of 50 behavior (Akers, 1998). However, as identified by a recent Campbell Collaboration systematic review, the evidence on the effects of counter-messaging and counter-narratives is still underdeveloped (Carthy et al., 2020). There is currently mixed evidence concerning potential iatrogenic effects as well. While the results of a recent experimental study found that counter-messaging can increase radicalization for those who already hold radical attitudes (Bélanger et al., 2020), a much larger study carried out in cooperation with Facebook found no evidence of counter-messaging increasing radicalization (e.g., Saltman et al., 2021). As such, further research on developing counter-narratives is needed. One possible direction that the results of this review would support, is that like counterradicalization strategies that target underlying risk factors as opposed to radical beliefs themselves (Wolfowicz et al., 2020), counternarratives should target alternative, risk-related topics. In addition, counter-narrative strategies need to take into consideration how elements of the Internet itself, such as personalization algorithms, may be leveraged to reduce access to radicalizing content, and facilitate the distribution of counter-messaging (Rieger et al., 2019).
Another potential avenue is to take a more positive-oriented approach. Recent meta-analytic evidence shows that exposure to pro-social media can serve as a protective factor against a range of negative outcomes, including aggressive cognitions and behaviors (Coyne et al., 2018).
With respect to digital literacy, which figures prominently in the literature, there is less evidence. Despite reason to believe that digital literacy can reduce susceptibility to radical content online (Grizzle & Tornero, 2016), the little quantitative evidence that exists is not promising. For example,  study of Indonesia students found no correlation between digital literacy, understanding about the nature and dangers of Internet-facilitated radicalization, and radicalization itself. We note that our results find a small risk effect for technological skills and radicalization. Of course, this result must be approached with caution given the quality of the evidence from which it is derived. Like other risk factors that pertain to the development of individual skills, further investigation into the effects of digital literacy is needed, especially given its focus in practice and policy (Wolfowicz et al., 2022).
The results of the review point to additional areas that may be considered relevant for practice and policy, in particular combatting Internet addiction. In this regard, the review found that attachment to online networks, which is related to Internet addiction, is associated with increased risk for cognitive radicalization. While it is important to interpret these results with caution in light of the low quality of the studies, Internet addiction has been found to be associated with a range of negative psychological consequences (Lopez-Fernandez & Kuss, 2020). As policy makers are already seeking to combat Internet addiction as a general societal problem (Lopez-Fernandez & Kuss, 2020), there may be cross-cutting benefits in countering radicalization as well. So too, the review found that cyberbullying victimization, and self-censorship online are associated with small risk estimates. While again it is important to temper interpretations of the results against the low quality of the evidence, issues of cyberbullying are already part of the broader agenda of social issues for many countries and may represent an area with cross-cutting implications.

| Implications for research
Overall, the results of this review serve to integrate the study of media effects and radicalization into the broader meta-analytic contributions to the study of media effects on other criminal and criminal-analogous outcomes. The findings demonstrate a significant degree of overlap with research on outcomes such as violent and aggressive cognitions and behaviors, as well as cognitions and behaviors pertaining to specific types of crime. However, unlike other comparisons between related cognitive and behavioral outcomes, the magnitude of the effects between cognitive and behavioral radicalization for the most relevant factors are almost identical. As such, further research into the role of the Internet in radicalization is certainly justified and needed.
At the same time, the review highlights a relative lack of quantitative research in general and identifies a relatively limited scope of factors examined to date. There is a need to broaden the scope of the types of media-related factors that are analyzed, and the methods used in their research. This is demonstrated by several single effect sizes identified in the review. Research also needs to consider other research designs which can perhaps be more informative. In particular, case-control studies comparing radicalized and non-radicalized individuals, or behaviorally radicalized with cognitively radicalized individuals may serve to identify what types of media effects may differentiate between these groups. This is the type of research that may serve to improve the effectiveness of online surveillance technologies and prevention efforts more generally. Unfortunately, to date, there are a very limited number of studies that have taken this approach (e.g., Scrivens et al., 2021;. Additionally, social network analysis is a promising methodology for examining a range of factors, including those pertaining to the theorized effects of echo chambers. In this regard, a recent study found that certain types of network structure characteristics, such as network insularity, may be predictors of radicalization. While there are a number of studies that employ network analysis at the group level, more research is needed at the individual level (Wolfowicz, Weisburd, et al., 2021a).
There are of course a number of systematic issues that should lead us to question the strength and direction of the factors examined in this review. As stated above, differential engagement with different types of media usage, and its influence, may reflect confounding factors such as age, gender, and self-control. As has been demonstrated by inquiries into media-effects for outcomes some as aggression, accounting for these confounders can significantly reduce the magnitude of observed bivariate relationships.
Indeed, the likelihood of exposure to radical content online has been found to be predicted by such factors (Hawdon et al., 2019). That the studies included in this review generally did not do so, should serve as a call to researchers to ensure that future work accounts for such issues.
Another issue that researchers should consider is how radicalization is measured. While it is true that more validated instruments have been developed only recently, using these instruments will help to ensure a greater degree of comparability. It is likely that the different types of measures used by the included studies serve to account for at least some of the observed heterogeneity, which could stem both from differences in the constructs being captured and in measurement scaling. Whilst we do not recommend any particular instrument over another, examples from studies included in this review are SyFor (Bhui et al., 2014) and ARIS (Moskalenko & McCauley, 2009).
It is also of note that whilst only small, there are statistically significant differences in the effects for both general media usage (for cognitive radicalization) and radical content consumption (for behavioral radicalization) between regions, with the effects for both factors being larger for European studies. Previous research has found that only a small number of risk factors for radicalization display regional heterogeneity . Future research may find benefit in examining the issue from a cultural perspective, which may consider how different cultures relate to media usage more generally or use media differently. It may also be the case that the differences simply reflect different frequencies of media usage, which can also be attached to culture. These are issues that future research should take into consideration or otherwise could serve as important lines of inquiry in and of themselves.
Relatedly, Internet usage continues to increase, in part due to the accessibility offered by mobile devices. Increased usage may increase the likelihood for exposure to radical content, a function of routine activities (Hawdon et al., 2019), whereas more frequent exposure may increase the potential influence that such content has on its consumers, a function of social learning. Our analysis found a marginally significant effect for year of data collection on passive exposure to radical content, in which more recent studies are associated with larger effects. Future research should consider how the year of data collection may influence results, and how results may reflect trends in media usage more generally.
Lastly, it is important to consider how the development of longitudinal research may impact our understanding of media effects in radicalization. In one longitudinal study, by Gvirsman et al. (2016), mediated exposure to violence did not influence radicalization when controlling for prior exposure and previously held radical attitudes.
However, in another study, by Zhu et al. (2020), online political/ radical activity as well as online network heterogeneity did predict radical behaviors, even when controlling for previous online activity and radical behaviors. More longitudinal studies are needed to clarify the nature and magnitude of the effects of these factors. In this respect, the current state of the literature is like the body of evidence concerning risk factors for radicalization more generally, which is still reliant primarily on cross-sectional designs (Wolfowicz, Weisburd, et al., 2021a).
We believe that the current evidence concerning media effects in radicalization is quite clear in at least one respect: That media exposure alone is unlikely to lead to radicalization. Indeed, there are very few cases of terrorists who were wholly radicalized by the media (Meleagrou-Hitchens et al., 2017). Even in cases where the Internet has apparently played an important role in radicalization processes, offline processes have been found to be more important (von Behr et al., 2013). Indeed, there are over a hundred different risk factors for radicalization that are not related to media (Wolfowicz, Weisburd, et al., 2021a). It is therefore important for research to consider the interplay between online and offline exposures to radicalizing messages. That is, exposure to radicalizing content, especially in instances where it contributes to risk, may not occur in a vacuum. It may be related to an individual's predispositions, or other aspects of their lives, including meeting new peers who themselves may hold radical attitudes or share such content with them.
The current evidence suggests that the field could benefit greatly from recent contributions from criminology to the field of radicaliza-  (Wolfowicz, Weisburd, et al., 2021a), they may operate as key moderators of the media-radicalization relationship. Similarly, researchers should also consider what the key outcomes are of media's effects on radicalization. As stated above, cognitive radicalization rarely leads to behavioral radicalization (McCauley & Moskalenko, 2017). However, it could be associated with other negative outcomes. In this regard, exposure to violent media has been seen to increase the risk of violent cognitions, which can negatively affect psychological well-being, including factors such as life-satisfaction and attachment (Huesmann, 2007;Jahangir et al., 2014;Reinecke & Oliver, 2016). It is of note that such factors have robust relationships with radicalization as well (Wolfowicz, Weisburd, et al., 2021a).
We believe that the results of this review, which provide modest estimates for media effects on radicalization, but which suffer from numerous potential sources of bias, supports the assertion that "The power of radical messages online and offline and the powerful discourses brought about by recruiters should neither be overestimated or underestimated" (Pauwels & Hardyns, 2018, p. 36).

DECLARATIONS OF INTEREST
The review's authors authored one presently unpublished study that was included in the current review.

PLANS FOR UPDATING THE REVIEW
The lead author will be responsible for ensuring that the review is updated in 3-5 years from the date of its first publication if no comparable review is published by that time.

DIFFERENCES BETWEEN PROTOCOL AND REVIEW
The review deviated from the protocol with respect to the databases that were included in the final searches. In particular, instead of search in International Political Science Abstracts (ISPA), searches were performed in Worldwide Political Science Abstracts, which includes ISPA.
The review also deviated from the protocol with respect to the publication bias analysis. While the protocol stated the Fail-Safe N test would be conducted, this approach was changed to Egger's regression following comments from the reviewers.
Another deviation from the protocol pertained to the way in which meta-regression and moderator analyses were applied. In the protocol it was stated that for categorical variables, metaregression would be used, and where found to be statistically significant, this would be followed up with moderator analysis.
Following comments from a Campbell Collaboration editor, this approach was abandoned. Instead, wherever there was sufficient data, all categorical variables were investigated using moderator analysis.
In addition, the review included some ad-hoc analyses that were not noted in the protocol. Relatedly, it was not possible to assess the impact of some of the study level characteristics listed in the protocol due to an insufficient number of studies/effect sizes for each category. This was the case for "Different measures of outcomes" and "Different types of studies."