The name of the game: Promoting resilience against extremism through an online gaming campaign

Extremist and terrorist groups are known to have used games and gaming successfully in their online recruitment and indoctrination campaigns. What are the possibilities for using online games to generate resilience against extremism? Current research on online counter and alternative narratives generally acknowledges effects in terms of awareness raising, yet is skeptical when it comes to impact, mostly owing to the limited evidence and empirical research in this area. A related problem with the early counter and alternative narrative campaigns, however, has been not only a lack of initial preparatory work on understanding the radicalization process itself — specifically, on how narratives are produced and disseminated — but also on aspects relating to the audience, the messenger, and the communications strategy more broadly. In designing DECOUNT [1], an online game that incorporates both counter and alternative narratives, we have taken these aspects into consideration and created an accessible, easy-to-use product by first researching the following: individual radicalization processes; extremist online propaganda narratives and imagery; and the preferences and issues of the target population. We then created and tested the game and advertised it on particular social media platforms (Instagram, YouTube and Twitch) with appropriate targeting. Finally, we carried out a number of evaluation procedures, including a qualitative assessment and two quasi-experiments. This paper outlines this preparatory, creative, and evaluative work, and contextualizes it within the literature on the role played by online platforms and content both in individual radicalization processes and in creating resilience.

The potential influence of violent games on individual violent behavior has received a good deal of scholarly attention in recent years. In 2020 the American Psychological Association (APA) arrived at the conclusion that there is no causal relationship between violent games and violent behavior (American Psychological Association, 2020). The relationship between violent games and individual radicalization leading to violent extremism, however, has only recently captured the attention of scholars in the field. How does the gamification of extremist online propaganda influence the individual viewer? What is the effect on individual radicalization processes?
In her article on the influence of gamification on individual radicalization, Linda Schlegel (2020b) identifies three mechanisms through which games could potentially facilitate radicalization. The first acts by gratifying psychological needs. Drawing on the psychological theory of self-determination, she argues that "gamification can help to meet three basic psychological needs: feeling competent, feeling autonomous, and experiencing social relatedness", and as such it "increases the likelihood that users will continue to participate, thereby normalizing the engagement with extremist content and potentially facilitating radicalization processes" (Schlegel, 2020b). The second mechanism concerns the teaching of ideology. According to Schlegel, "[i]t is not unreasonable to assume that further developments [concerning the teaching of ideology] will incorporate more gamification elements to increase user engagement with ideological contents and facilitate radicalization by making the learning of ideological concepts engaging and 'fun'" (Schlegel, 2020b). Finally, gaming might facilitate individual radicalization by increasing the social appeal of extremist tools: "Competition and a sense of achievement may increase the willingness to engage with extremist content in order to collect even more points" (Schlegel, 2020b).
Although the gamification of extremist online propaganda is not a new phenomenon, recent developments suggest that it is beginning to play an even more central role in the online recruitment efforts of extremist organizations. While empirical research on the topic is still in its infancy, existing literature indicates that the psychological effects of using games in online propaganda should not be overlooked. Given the abovementioned implications concerning the effectiveness of narratives on behavior and attitudes, this paper is concerned with the following question: is it possible to tap into modern gaming culture and create games that have the potential to increase resilience against extremism?

The effectiveness of counter and alternative narratives in preventing violent extremism/countering violent extremism (PVE/CVE)
In response to IS's recruitment of thousands of foreign fighters -a success due in part to their sophisticated social media strategy -governments and civil society organizations have looked towards counter and alternative narratives as part of their broader CVE efforts. According to the European Commission's Radicalization Awareness Network (RAN), alternative narratives " [u]ndercut violent extremist narratives by focusing on what we are 'for' rather than 'against'", and by delivering a " [p]ositive story about social values, tolerance, openness, freedom and democracy", while counter-narratives " [d]irectly deconstruct, discredit and demystify violent extremist messaging" by challenging "ideologies through emotion, theology, humour, exposure of hypocrisy, lies and untruths" (Radicalisation Awareness Network, 2015).
A number of projects involving the production and dissemination of counter and alternative narratives have since been established -some by government agencies, others by civil society. One of the recurring problems has been the lack of preparatory work on a number of aspects: in particular, on understanding how narratives work and how to obtain knowledge about, and gain the trust of, the target audience. Additionally, researchers in the field have criticized the lack of empirical data on the impact these social media campaigns have on the target group, especially in the case of counter-narratives (e.g., Ferguson, 2016;Rosand and Winterbotham, 2019). In a recent analytical brief, the Counter-Terrorism Committee's Executive Directorate (CTED) of the United Nations Security Council identifies two challenges arising in counter and alternative narrative campaigns: the fact that they are often based on untested assumptions, and disregard for the gender perspective (CTED, 2020). The CTED also points out a lack of monitoring and evaluation, which casts doubt on the effectiveness of counter-narrative campaigns (CTED, 2020). Importantly, it should be ensured that counter-narratives do not risk exacerbating the problem: a recent experimental study suggests that presenting counter-narratives alone can have an adverse effect on individuals who are vulnerable to radicalization. For example, participants with a deep need of cognitive closure expressed greater support for IS after being presented with counter-narratives (Bélanger, et al., 2020). Findings such as these reinforce the notion that, to prevent online recruitment by extremist groups, it is not enough just to counter extremist narratives: it is also necessary to offer alternatives.
While evaluations of counter and alternative narrative campaigns in the field of PVE/CVE still remain scarce, insights from the literature on the effectiveness of narratives on attitudes and behavior indicate that such effects are indeed possible. Studies show that exposure to information can successfully influence beliefs, attitudes, intentions, and behavior when that information is delivered in the form of a narrative (Braddock and Dillard, 2016). In that context, scholars have suggested that it is important not only to identify target themes, but also to take into account the style in which these themes are presented (Braddock and Horgan, 2016). Furthermore, an experimental study revealed that identification with a main character is an important factor in narrative persuasion, i.e., in the effect a narrative has on the attitudes of the receiver (de Graaf, et al., 2012). In the experimental study, participants read a story delivered by one of two characters -who had conflicting goals. The results revealed that both the player's identification with a character and the extent to which player attitudes remained constant were influenced by perspective (de Graaf, et al., 2012). Valuable inferences can thus be drawn for the development of counter and alternative narrative campaigns, if literature on the psychological effects of narratives is taken into account.

DECOUNT -The game
The game's design addresses and illustrates these issues. DECOUNT is a role-play game: in each of three stories, it focuses on a particular character whose evolution in the game depends on the decisions the player takes. The lifeworlds of the two male and two female characters are situated in the jihadi and right-wing extremist scenes. The game is structured following a theoretical model developed earlier (Pisoiu, 2011), which conceptualized radicalization as a sequence of choices motivated by gain but shaped by an alternative system of interpretative frameworks.
A problem besetting the early counter and alternative narrative campaigns was that insufficient preparatory work had been done to understand the radicalization process itself. A large part of this project, therefore, was devoted to examining individual radicalization processes. As a first step in mapping them for the purposes of the game, the team interviewed 31 current and former radicalized right-wing and Islamist individuals in Austria, to learn about their perspectives and reflections on their own radicalization processes and to see how they related to various types of media, including video games. Radicalization takes place gradually; its beginnings are unremarkable and are not registered for what they are by the individuals in question. Self-observation usually comes later. However, certain situations are conducive to first contacts, which often take place at school, within a group of friends, or in the family. The progression towards radicalization was usually described by interviewees as seamless, as 'drifting': in retrospect, they found it hard to explain, or even to understand, why they took certain steps. They could, however, relate occurrences, which is why the narrative structure of the interviews proved particularly useful when it came to writing authentic scripts for the browser game. At the macro level, particular events and trends in world and national politics, such as the war in Syria or polarizing discourses at home, acted as facilitators, as they raised questions or created perceptions of discrimination. In some cases, criminality and gangs preceded or were even interwoven into the radicalization process.
While they were in radical scenes, individuals did not perceive themselves or their actions as being anything out of the ordinary; this can often be because the group and/or the wider social environment subscribe to at least some basic ideas about what is wrong in society. It does, however, confirm the relevance of frames and narratives as filters through which reality is perceived, thereby highlighting the importance of understanding and deconstructing them. In the right-wing spectrum we observed broader narratives of 'good' and 'bad' foreigners, of various types of disadvantage afflicting the natives as opposed to refugees, or of there being simply 'too many' foreigners. Within the jihadi spectrum there was not so much a narrative of anti-Muslim discrimination as a perception of being at a disadvantage as foreigners, while the Syrian conflict came across as a strong presence in the collective identity. That said, once radicalization led to visible changes in appearance, references to these changes or increased attention from authorities were interpreted as discrimination against Muslims. In several cases of Islamist radicalization, police brutality was mentioned as an illustration of discriminatory treatment.
Motivation to join sometimes included a desire to make up for past mistakes. Particular individuals also played an important role, whether as mentors, sounding boards, or helpers who intervened in crisis situations such as bullying. It was also possible to identify spatial indicators for radicalization processes: particular parts of a city, or hotspots such as certain shopping malls, parks, or gyms. Individual roles appeared broadly in the form of helper -hero or resurrecting hero -a reassertion of masculinity after a loss of self-esteem. The reasons for such a loss were manifold, ranging from debt to addiction and unemployment. In some cases, the group and the ideology conveyed a sense of psychological stability. For these individuals, becoming part of an extremist group appeared to be the first good choice after a series of bad choices they had made in the past. That said, a loss of self-esteem or other form of crisis did not appear to be necessary to motivate all individuals to join: in many cases, individuals literally 'tagged along', and their main motivation was to be accepted and recognized by the group. For others, it was simply a way to raise their status in their wider social environment. For these individuals especially, it was important to be recognizable as a member of the relevant subculture, e.g., by wearing certain clothing. In some cases, manliness -often associated with a military appearance -played an important role. For many of the interviewees, interest in (international) politics was limited to events featured in the narratives of the relevant extremist group. At the same time, conspiracy theories were to some degree present in all cases; in particular, the conspiracy around 9/11 was named across the ideological spectrum. Conspiracy narratives reinforced the 'us versus them' attitude and the willingness to fight for higher goals, like justice. Historical grievances and their originators were identified as problems, and integrated into the overall narrative. In many cases, pursuing solutions was portrayed as a duty for followers of the movement, leading to a form of guilty conscience -or at least, cognitive dissonance -in some of those who did not immediately comply. Depending on the radical group, the call for collective action was to fight either against foreigners or against people of a different faith. The enemies identified were mostly the same: Americans, Jews, feminists, and homosexuals, plus Muslims from certain other branches of Islam and 'unbelievers' in general, or -for the 'hardcore' types -non-Austrians/non-Germans in general. Contradictions within an ideology, or between statements and actions, acted in some cases as triggers for the disengagement process.
Another finding in this research was the role that social relationships, for example with family, friends, and the community, can play in the radicalization process. Some of the interviewees came from unstable families or had been neglected by their parents. In all cases, however, the groups offered protection, community, being there for each other, and a sense of togetherness, of comradeship (or sisterhood). The radical groups also provided a cause to fight for. Some interviewees got into trouble with their families or certain family members when they joined radical groups; in some cases this led to their being repudiated, which in turn reinforced their radicalization. Repudiators here included their children and, in one instance, a girlfriend. In other cases, it appeared that this reaction from family members acted as a pull factor, drawing them away from the radical group.
The interviews also involved naming particular music, symbols, videos, and text that had been especially influential in their radicalization. In addition to being sources of information, these had psychological effects that ranged from conveying a sense of calm, belonging, or status, to motivating them to fight. The interviewees also gave information on their use of the Internet and social media. The 'echo chamber' effect was confirmed by the exclusive reference to individuals and sources of information from within the scene. YouTube, Facebook, Telegram, and WhatsApp were mentioned, as well as particular websites and forums. Almost all of the interviewees indicated that they played video games -some of them for up to four hours a day.
The insights we gained from the interviews were catalogued in a spreadsheet, with situations and consequences; all of these were categorized by degrees of radicalization. This provided raw material for conceptualizing the stories of the four main characters in the DECOUNT browser game. As well as interviewing people, we analyzed propaganda on relevant social media and other Internet resources, depending on where the relevant groups were present. This source gave us additional material for the story timelines and the propaganda messages included in the dialogues. The purpose of the analysis of this online material was to identify propaganda frames, persuasion mechanisms, and individual roles in the right-wing extremist and jihadi scenes in Austria. We selected and analyzed material on social media (Telegram, Twitter, Facebook, YouTube) and websites that were found to be relevant and available, primarily in Austria, but we also selected sources from elsewhere in the German-speaking world, insofar as links were found to Austria. In the right-wing extremist spectrum we focused on the 'New Right' and the neo-Nazi spectrum, with the Identitarian Movement Austria (Identitäre Bewegung Österreich -IBÖ), Unwiderstehlich Österreich, and unzensuriert.at; in the jihadist spectrum we focused on the Salafi and jihadi Telegram channels, and on a select sample of online videos which were found to be influential in the radicalization of Austrian jihadis. The content was examined by applying frame analysis (Benford and Snow, 2000). "Framing processes have come to be regarded, alongside resource mobilization and political opportunity processes, as a central dynamic in understanding the character and course of social movements" (Benford and Snow, 2000). In the course of a frame analysis, diagnostic frames, prognostic frames, motivational frames, and counterframes are identified within the data. Passages in which a problem is identified (in our case, by an extremist group online) are coded as diagnostic frames. Articulations of proposed solutions are subsumed under prognostic frames, and calls for collective action are coded as motivational frames. A number of rhetorical mechanisms were also considered, such as the use of empirical evidence or the authority of the frame articulators. Apart from text, we also collected video and audio material, which we used in the game in a slightly modified form.
The results of the frame analysis were taken into account when producing the DECOUNT browser game. During the course of the game the player meets different characters, some of whom belong to an extremist group. In trying to persuade the protagonist to join their ranks, these characters resort to the framings identified in the frame analysis. In the female Identitarian story, for example, the protagonist is confronted with the motivational frame for engaging in patriotic activism from a particular perspective, namely as a victim of sexual violence. She is then confronted with xenophobic and conspiratorial narratives that are ideologically inconsistent, and so has to choose whether or not to abandon her initial political agendas. The browser game is set up as a simulation of a social media application in which the news feed consists of postings by friends. Over time, this news feed gradually comes to contain more and more extremist material, as the main character befriends more individuals from the extremist milieu.

Creating and testing the game
Based on the interviews and the scenes identified therein, as well as the framings identified in the abovementioned analysis, four stories were conceptualized. Each story features a main character -one of two female and two male protagonists -in the context of either jihadi or right-wing extremism. To summarize the four stories: Marco spends most of his time with his clique, who seldom have good plans and repeatedly have problems with the police. They are interested in soccer, MMA, and rap, and regularly hang out in shisha bars. Marco also has problems at home, and at one point he falls out with his clique. Suddenly, an old friend appears and offers to help him change his lifestyle. Gradually, politics and religion become increasingly important for Marco, as can be seen on his timeline. This story contains three main narratives which are deconstructed in discussions with the protagonist's uncle. At the beginning of the radicalization process, a political issue that typically features in jihadist propaganda is addressed: namely, the oppression and murder of Muslims. The first discussion suggests that one can also engage in legal political activity and fight in this way for political or humanitarian goals. The second narrative concerns the clothing rules typically applied to women in jihadist circles. The protagonist reprimands a woman whose style of dress is deemed inappropriate under these rules, leading to a discussion on women's rights and the right to freedom of choice. The third incident relates to a decision to use violence against a drug dealer. Here the dangers of peer pressure versus the advantages of critical thinking are discussed.
Because they go to different schools after the summer vacation, Jasmin and her friends part ways. Jasmin finds it difficult to make friends in her new class, looks for advice online and, while surfing the Internet, comes across the topic of beauty in Islam. The content makes her curious, and much of it is presented in a scientifically convincing manner. Meanwhile, two old friends appear, veiled, on social media, and invite her to join a secret group. At the same time, she falls down a rabbit hole while looking for friends and beauty tips online. Jasmin's radicalization takes place in the context of the typical teenage problems of changing school, discovering and questioning one's own appearance, and conflicts with siblings and parents. The protagonist is an intelligent girl whose search for friendship and knowledge leads her into radical circles. Jasmin is confronted with major issues as the dichotomous worldviews of jihadist propaganda and dogmas, patriarchal structures, and their role models are taken up and problematized during the story, and are included in both the narrative and the counternarratives. Like in Marco's story, the oppression of Muslims features in Jasmin's radicalization, as a political issue. Jasmin is 'searching', and is therefore receptive to the recruiting strategies of extremist groups. Typical radicalization mechanisms such as supposedly empirical 'evidence' for ideological propositions, as well as persecution mania, are also discussed.
Franziska is interested in environmental issues and wants to become an influencer. When she moves to the big city to study journalism, she leaves a whole world behind -her home in the countryside, her father, who is a passionate beekeeper, and her friends. The environment seems to be an important issue also in the city: two campaigning groups would like to enlist Franziska's commitment and skills. Although both plan similar activities, they define environmental protection very differently, so Franziska has to decide where her priorities lie and how far she will go for her career. Franziska's radicalization is initiated by her relocation and her environmental activism. Her radicalization takes place predominantly at an intellectual level. The right-wing extremist content in this story is conveyed in a more subtle way, typical of the New Right. The themes are: 'mass migration' and the resulting change in society, and the state's alleged support for mass immigration and (the allegedly ensuing) crime. Other themes are the 'lying press' and the presumed democratic suitability of right-wing ideology, as well as the supposedly strong role of women (in what is in fact a deeply patriarchal extremist scene). Franziska becomes immersed in the propaganda created by the authoritarian group and must make some important decisions.
Jens changes school and starts in a new class. The subject of National Socialism is currently on the history teacher's agenda, and the class is preparing for a trip to the former Mauthausen concentration camp. Not all students are enthusiastic about the idea. One in particular is ambivalent, which is why the others cut him off. Jens has to take a position and is lured into a new world. Since the arguments of contemporary extremist movements on the political right sometimes resemble those heard in the era of National Socialism, it made sense for the game to start by discussing the importance of historical knowledge for recognizing right-wing movements' argumentation patterns, and their consequences. Various radicalization mechanisms typical of the right-wing extremist scene are presented in the course of the story: the influence of right-wing music, distancing from friends and a subsequent immersion in the extremist scene; the desire for friendship, recognition, and status; and the professionalization of street and underground combat. Typical right-wing extremist topics such as xenophobia or the crimes allegedly committed by foreigners are criticized less by means of dialogue than by grotesque representations on the timeline.
To produce effective, impactful alternative narratives it is necessary to understand the preferences and concerns of the target audience -in our case, individuals vulnerable to radicalization and young people in general. The project therefore involved understanding the preferences and interests of the target audience, in particular those who were vulnerable. The goal was on the one hand to learn about how the target group used social media, and on the other, to gather information on the aesthetics and themes they find interesting, socio-political issues they have encountered and observed, and their ideas about current and future sociopolitical roles. The project involved in-depth interviews with 10 vulnerable young people and two focus groups, and an analysis of existing studies on vulnerable youth in Vienna and Austria. Additionally, an expert workshop and three feedback sessions with practitioners from related fields such as social work, youth work, deradicalization etc. were organized. In analysing the target audience for counter and alternative narratives, the main focus was on: Social media use: which platforms are interesting, what time of day they are used at, etc. The rationale here was that knowledge of the everyday use of social media within the target group is important for setting up a successful online campaign. The information gathered was used in the design and dissemination of the online components. Results indicated for example that Instagram, rather than Facebook or Twitter, was popular among the target group, and that video games were popular with the majority of the respondents.
Content/aesthetics/game concept: gather information on the aesthetics and themes the target group finds appealing, socio-political issues they have encountered and observed, and their ideas about their current and future socio-political roles. Relevant political topics at the time included, for example, deportations and the xenophobic discourse of far-right parties and their leaders, as well as conflict and war abroad. This information was used to write the scripts for the online game and to provide ideas on content for the videos.
For the artistic design we tested the proposals of four artists, finally selecting two on the basis of the target group's preferences.
While producing the game we regularly tested the prototype on the target audience, to assess the relevance and appropriateness of the design and the dialogues.
As well as gathering direct input from representatives of the target audience, from practitioners who are involved with the target audience in their daily work we obtained information on the following two aspects: Target audience: The scope and characteristics of the target group had to be defined as specifically as possible. A number of age-specific topics were also suggested, such as identity and sexuality issues.
Online platform: What should the online platform look like? What do practitioners need, in their work with vulnerable youth?
Additionally, designs produced in the course of the DECOUNT project were tested on the focus groups. Six drafts of possible designs for the online game, and a prototype of it, were presented to the group's participants. To find artists who would illustrate the characters, their assets and other components of the online game, six artists from Germany and Austria were initially asked for sample drawings which could then be tested on the target group. More specifically, they were asked to submit a sample drawing of two main characters, one male and one female, and three cartoon-style panels depicting a fistfight between youngsters. The participants were then presented with printed versions of these sample drawings and asked to indicate their favorite by placing a sticker on the one they liked best. Next, they were asked to explain their choice. The focus groups were also used to test the prototype of the online game.
To gain deeper insights into the issues discussed by the focus groups, 10 interviews were held with adolescents from the target group. As with the focus groups, the interview questions concentrated on sociopolitical issues and on social media use. Interview guidelines, including more detailed questions than for the focus groups, had been drafted beforehand. They included questions on the interviewees' use of social media ("Which social media platforms do you use in your daily life, and for what purposes?"), online games ("What is your preferred online game, and why?"), online videos ("What kind of videos are shared among your peers?"), socio-political topics ("What political issue is discussed among you and your peers at the moment?"), aesthetics ("Can you show me an example of a meme that is popular with you and your peers at the moment? What do you like about it?") and roles ("Did one of you ever start an online campaign?"). Once the focus groups had given their consent, they were recorded, and the content was later transcribed and analyzed.

Evaluation and discussion of the results
To assess the impact of the browser game intervention, a mixed-method research design was used. A quasiexperiment was conducted to find out whether the target group could be dissuaded from promoting terrorism and violent extremism and/or political violence; this approach entailed measuring participants' attitudes before and after the video game intervention. To gain deeper insights into the intervention's resonance among the target group, a focus group was set up.
The main objective of the quasi-experiment was to examine whether or not the video game intervention led to a change in (political) attitudes among the target group. In the case of a positive change, a secondary aim was to investigate whether or not this was more likely when the video game intervention was embedded in a PVE/CVE workshop. With this in mind, two different experimental conditions were developed and evaluated: In Experimental Condition 1, participants played the video game and then took part in a workshop conducted by experts in the field of PVE/CVE; in Experimental Condition 2, the participants merely played the game. Any change in attitudes was measured using pre-and post-test questionnaires.
In accordance with the definition of the video game's target group as being juveniles (male and female) between 15 and max. 25 years old, in schools and youth centres, 74 adolescents (mean age = 18.7, standard deviation = 2.85, 33 female, 40 male) from these institutions took part in the quasi-experiment.
To make it possible to measure the effect of the video game in quantitative terms, its previously defined goals relating to extremist attitudes were formulated as variables so that their degree of attainment could be captured by a questionnaire. Extra dimensions were added, giving a total of five dimensions and ten subdimensions, to reflect the degree to which different attitudes had changed among participants. The items were selected from well-established self-report questionnaires. Participants could rate how much they agreed with the statements in the questionnaire on a five-point Likert scale (1 = I strongly disagree; 2 = I disagree; 3 = I neither agree nor disagree; 4 = I agree; 5 = I strongly agree).
The goal of the video game was primarily to affect attitudes towards extremism and to illustrate the radicalization process. Items on 'extremist narratives' were therefore included, addressing several aspects of the different extremist narratives of right-wing and Islamist-extremist/jihadist groups that feature in the video game: among others, items addressing the 'conspiracy theories' dimension, which can be found in the narratives of both kinds of groups. To measure the participants' agreement with conspiracy-theory elements, items on the 'Verschwörungsmentalität' (conspiracy mentality) dimension were adopted (Imhof and Decker, 2013). Further items on 'conspiracy theories' were adopted from the 'Neurechte Einstellungen' (new-right attitudes) scale by . As well as conspiracy theories, particular gender roles are an aspect of extremist narratives and were therefore included in the item series for the quasi-experiment. Items on gender roles were adopted from the 'Sexismus traditionell' (traditional sexism) dimension as defined in the Gespaltene Mitte -Feindselige Zustände study . A further aspect of narratives on both the right-wing extremist and the Islamist-extremist/jihadist spectrum is the legitimization of violence. In order to measure the effect that the video game intervention had on the participants' 'attitude towards violence', items by Ulbrich-Herrmann (1995) were included.
A number of other dimensions were added to help us explore the broader applicability of the game: democratic values, critical thinking and media literacy, and radicalization as a process. For democratic values, well-established items from different series were compiled to form three dimensions in the evaluation questionnaire. For the 'human rights' dimension, items on 'Gleichheits-und Pluralitätsnormen' (equality and pluralism norms) from the Leipziger Autoritarismus Studie 2018 (Decker and Brähler, 2018) were included. For the 'democratic engagement' dimension, items from the 'politische Deprivation' (political deprivation) dimension of the same study were selected (Decker and Brähler, 2018). For the 'diversity' dimension, items from the 'Reflexion Sozialer Stereotype' (reflection of social stereotypes) scale were included (Christoph and Reinders, 2012).
Another goal of the video game is to support the development of critical media literacy. No standardized items on this relatively new theoretical concept are yet available, so for this quasi-experiment new items were formulated on the basis of existing literature (Frischlich, 2019) [3]. In order to measure whether the video game intervention had an effect on the perceived 'autonomy' of the participants and their 'critical thinking', three items of the Kurzskala Autoritarismus (KSA-3; short authoritarianism scale) were included. In particular, the 'autoritäre Unterwürfigkeit' (authoritarian submissiveness) dimension was adopted -a theoretical concept which can be described as submissiveness to the established authorities and a general acceptance of what they say and do (Beierlein, et al., 2014).
An important goal of the video-game intervention was to develop an understanding of individual 'radicalization as a process'. Research has not yet come up with standardized items on this very specific issue; accordingly, new items were formulated on the basis of existing literature (Pisoiu and Hain, 2017) [4].
In addition to measuring the effect that playing the video game had on some of the target group's political attitudes, the evaluation was designed to examine attitudes directly connected to topics addressed in the scripts of the game, so the extent to which the participants agreed with statements about the content of the storylines was examined. The statements were formulated along the lines of the dimensions included in the pre-/post-test questionnaires and the goals of the video game, respectively, and were included in the followup questionnaire only. Participants could rate how much they agreed with the statements on a five-point Likert scale (1 = I strongly disagree; 2 = I disagree; 3 = I neither agree nor disagree; 4 = I agree; 5 = I strongly agree).
The design of the quasi-experiment was a 2 (intervention type) x 2 (pre-/post-questionnaire) design. The 'intervention type' independent variable was manipulated between subjects (differences in the delivery of the intervention), while the other independent variable ,'questionnaire', was manipulated within subjects (before/after). The dependent variables were the mean score of the pre-/post-test questionnaires both as a whole and for the different sub-dimensions of the questionnaire.
After the quasi-experiment, eight individuals were recruited for the focus group. Using the interview guide, the moderator (a member of the research team) initiated a discussion among the participants. This took around 30 minutes and was recorded for the purpose of analysis. The recording was transcribed and analyzed by means of categories based on the transcript.
Overall, the analysis revealed that the video game intervention had a significant effect on participants' attitudes towards 'extremist narratives', whether or not it was embedded in a workshop: after playing the video game, they agreed less with extremist statements.
In participants who only played the video game, no further significant differences in attitude before or after the intervention could be determined. In other words, no attitude changes appeared among the participants in Experimental Condition 2 with regard to 'democratic values', 'critical media literacy', 'autonomy and self-reflection', or the 'radicalization process' itself. At this point it should be noted, however, that when only participants from a certain recruitment pool -in this case, an institute of applied sciences -were included in the analysis (64 percent of participants in Condition 2), the mean scores of the post-test questionnaire were significantly lower than on the pre-test questionnaire (also in Condition 2).
Among the workshop participants, a significant difference was detected between the mean scores on the whole questionnaire before and after the intervention. The mean score was significantly lower after they had taken part in the workshop, which shows that they then agreed significantly less with the statements in the questionnaire. This finding supports the assumptions formulated for the video game intervention. An analysis of the sub-dimensions revealed exactly which assumptions: specifically, it became clear that, besides the effect on the 'extremist narratives' scale, mentioned above, this significant difference could be attributed to the decrease in authoritarian attitudes (KSA-3 scale) after the intervention. That is, after playing the video game in the course of a workshop, participants in this condition scored significantly lower on the 'Short authoritarianism scale', a sub-dimension of the 'autonomy and self-refection' scale of the current evaluation. In other words, after the video-game intervention in combination with a workshop, participants' authoritarian attitudes diminished. This finding is especially interesting against the background of observations made in the focus group, revealing that the video game stimulates reflection on the appeal that social groups have for an individual, and on dynamics within a group more generally. Together, these findings represent tentative evidence that the video game intervention performs well in fostering autonomy and critical thinking. Also in Experimental Condition 1, however, there was no significant effect on the mean scores for the 'democratic values', 'critical media literacy', or 'radicalization as process' dimensions.
The participants' self-assessment in the following evaluation method showed effects along all dimensions. On the follow-up questionnaire, three statements in their self-assessment stood out: "Through the video game, I learned what narratives extremist groups use to lure people", "The video reminds you to question your friends' views critically too" and "The video game demonstrates that it is not always easy, but it is important, to make sure that different people can live together peacefully in a country." This finding is an indicator that, at least according to the participants' own self-assessment, the video game was exceptionally successful in: fostering critical thinking and autonomy; raising awareness of extremist narratives; and promoting the democratic value of diversity.
The focus group showed, in the discussion on the video game intervention, that participants agreed they had learned quite a lot about the radicalization process as such and about how extremist organizations recruit individuals both online and off-line. They also mentioned, however, that the game could have worked better by offering players more alternative opportunities to fight injustice in a democratic way. Interestingly, a new target group was identified in the course of the guided discussion: individuals who have cases of individual radicalization in their own environment and are seeking support.
What are the limitations of the current evaluation? The mixed results concerning the effect on participants' attitudes in the quasi-experiment might be due to methodological constraints. Data collection was severely impeded by the COVID-19 restrictions introduced by the Austrian government, which meant that the quasiexperiment had to be conducted in four different institutions. This undoubtedly affected the validity of the findings, as conducting it with participants who all came from the same institution would clearly have limited the external factors. The difference in results in Condition 2, in particular (when participants from a youth center were tested by youth workers, unsupervised by members of the research team), demonstrates that repeating the quasi-experiment without the constraints of the COVID-19 pandemic would most probably yield more insightful results, especially regarding the video game-only condition.
COVID-19 regulations also had a negative effect on the quality of the workshop held at the job training program. As the PVE/CVE trainers were not allowed to enter this institution in person, the workshop had to be conducted digitally via a video call application. Unfortunately, severe technical problems during this online workshop further complicated the collection of data for the quasi-experiment.
Another organizational issue relating to the COVID-19 regulations impeded data collection. Spill-over effects might have been prevented if participants could have filled in the pre-test questionnaire the day before the intervention -owing to organizational difficulties, however, they filled in both the pre-and post-test questionnaires on the same day. This makes it quite likely that they more or less remembered the questions, and tried conscientiously to answer as they had before.
Another limitation arising from the Austrian government's COVID-19 regulations concerns the sample size. Restrictions on visits to educational institutions by outsiders made it impossible, for this evaluation, to recruit enough participants to include a control group in the design of the study. Yet proper control groups maintain internal validity, because they reduce the probability that changes in the dependent variable can be explained by anything other than the independent variable. Given the methodological limitations and the promising results concerning the change in certain attitudes as a result of the video game intervention, a second investigation into its impact, using a larger sample size, would be of great interest.
In this article we have outlined our preparatory work and our implementation and evaluation of a game in the context of preventing extremism. Results deriving from first-hand research with radicalized and vulnerable youth informed both the creation of the game and decisions on design and marketing. An initial evaluation of results indicated changes in the participants' attitude to extremism and a better overall understanding of radicalization processes. At the same time, it became apparent that more intensive work is required on topics relating to democracy and critical thinking -and that young people today are in urgent need of civic education.

About the authors
Daniela Pisoiu is a senior researcher at the Austrian Institute for International Affairs (Österreichische Institut für Internationale Politik, OIIP) and fellow at SCENOR. Her fields of research are terrorism, radicalization, extremism, comparative regional security, and American and European foreign and security policy. She completed her Ph.D. at the Centre for the Study of Terrorism and Political Violence at the University of St Andrews, and has conducted fieldwork on radicalization in Austria, Germany, and France, as well as other European countries. 1. The video game, together with the Web site (extremismus.info) and a short film, were created as part of a project titled DECOUNT, co-funded by the EU's Internal Security Fund. The aim of this interdisciplinary project was to provide evidence-based materials on extremism prevention in the context of a counter and alternative narratives campaign.