Research quality and psychological theory in publications on school shooters with multiple victims - A systematic review of the literature

Abstract School shooting homicide events generate considerable attention. A substantial number of research reports have tried to explain the phenomenon. However, the outcome of these studies has produced a conflicting picture of the issue. Our systematic review explored the quality of research in publications on school shooters. Research quality was assessed concerning description of design, method and interpretation of results according to PRISMA and CRD criteria. We investigated evidence of the impact of psychological theories on how research was designed and interpreted. A total of 10 papers met the criteria for inclusion in the review. With a few exceptions, the research quality was low. Only three studies contained a separate methods section. Two out of ten studies reported from an interview with a school shooter. Secondary sources such as school, hospital and/or psychological evaluations were used in four studies, while the rest had only applied tertiary data sources. There was a void of psychological theoretical analysis to inform the creation of relevant research designs. No study discussed psychological theories to inform inference from empirical data to conclusion. Higher quality of research and enhanced focus on theoretical understanding of psychological factors in school shooting are called upon.


ABOUT THE AUTHORS
Pål Grøndahl is a specialist in clinical psychology. Pål Grøndahl obtained his PhD in forensic psychiatric quality and use of methods. He is a senior researcher at the Centre for Research and Education in Forensic Psychiatry at Oslo University Hospital in Norway. He has worked as a forensic psychiatric expert witness for the past 15 years.
Stål Bjørkly is a specialist in clinical psychology. Stål Bjørkly obtained PsD in assessment and treatment of violence in psychotic patients. Stål Bjørkly is a professor in clinical psychology at Molde University College, Norway since 1998. Stål Bjørkly is a research consultant at the Centre for Research and Education in Forensic Psychiatry, Oslo Norway since 2002. Stål Bjørkly published two books and about 80 international articles and book chapters on violence in persons with mental disorders.

PUBLIC INTEREST STATEMENT
A school shooting homicide episode is a tragic and highly dramatic event where a student kills his classmates and teachers. Such events generate considerable media attention. In the wake of such killings, a lot of studies have tried to explain the phenomenon and several of these studies have developed certain typologies of the school shooters. We wanted to explore the quality of research in publications on school shooters. After a literature search, we found that 10 studies met our specified inclusion criteria. We found that the quality of the research concerning school shooters was not good enough to contribute to valid explanations to why such tragedies occur. Several of the studies used only data from the media and Internet and only two studies had conducted an interview with a school shooter. The studies rarely used psychological theories to explain these adverse events.

Introduction
Intermittently tragic school massacres occur where a pupil or student has shot dead fellow students and teachers at a school, college or university. One recent example of an incident took place at Umpqua Community College, in the United States on 1 October 2015. A 26-year-old student shot and killed one teacher and eight students inside a classroom where he took classes. Nine other students were injured. After the fatal act, the perpetrator killed himself. Like in any other school shooting, the motive and possible causes for the killing appeared not to be easily accessible. This state of obscurity brings about speculative explanations from people in general and in the media. It is of paramount importance that mental health professionals and other relevant professions counteract these speculations by communicating reliable and valid knowledge. However, in our view, attempts made to find simple explanations based on personality traits, psychopathology or social context factors among school massacre, perpetrators seem to dominate several of the professional contributions in the field. The premise of these assumptions appears to be that school shootings have common causal dynamics because they share common behavioural and contextual topography.
Attempts to explain school shootings have been done in public investigation reports, case studies and reviews. The outcome of these studies has produced a mixed and even conflicting picture of the phenomenon. Several authors have noted that there is little evidence of common denominators for school shootings and the perpetrators of such acts (Leuschner et al., 2011;Levin & Madfis, 2009;O'Toole, 2000;Weisbrot, 2008). Despite such warnings, authors are still concerned with the possibility of assessing school shooters and develop risk profiles (O'Toole, 2000;Twemlow et al., 2002). However, Thompson and Kyle (2005) warn against too much focus on school violence and states that: "some may be tempted to overemphasise the nature of school violence as they develop interventions" (Thompson & Kyle, 2005 p. 420). Despite this, others have developed hypotheses and models to explain school shootings. Several studies of the characteristics of school shooters have emphasized that the perpetrators are lonely, alienated by their peers and are victims of bullying (Dill, Redding, Smith, Surette, & Cornell, 2011;Kimmel & Mahler, 2003;Twemlow et al., 2002;Weisbrot, 2008). Yet, it is noted that such characteristics can be found in many students who never do show any signs of violence (Cornell, 2011;Leuschner et al., 2011). Other researchers claim that school shooters leak their plans in advance of an attack and suggest that this is an important warning signal (Leuschner et al., 2011;Preti, 2008;Twemlow et al., 2002;Weisbrot, 2008). Levin and Madfis (2009) have proposed a sequential and additive/cumulative strain model. The purpose of their model was to take into account the accumulation of several factors ultimately leading up to a school shooting incidence. Such a model intended to avoid a monolithic or unidirectional explanation of a complex phenomenon.
Their model contains five sequences: prolonged stress, strain without social control, acute stress, planning, and finally the atrocity. The transition to a new stage is dependent on the presence of the preceding stage, which largely also explains that school shootings are a rare but fatal phenomenon (Levin & Madfis, 2009). Some scholars have emphasized that most school massacres have occurred in suburbs rather than in big cities. They suggest that one reason why this is happening in the suburbs may be that smaller communities have a more normative approach to how one "should" be than in larger cities (Kiilakoski & Oksanen, 2011).
The probability of being killed in a school massacre is extremely low, even in the US where most of the school shooting incidents have taken place. This type of violence represents approximately 1% of homicides that occur among school-age youths (CDC, C. f. D. C, 2008). Though the probability of being killed in such incidents is very low, school shootings naturally creates legitimate fear, resentment and demands for preventive measures. However, the low base rate minimizes the likelihood of finding specific risk factors that can identify school shooters. Consequently, it may be a better basis for prevention if one could identify risk factors in these persons, not in terms of committing homicide but rather concerning violence in general. This invites for exploring two research issues associated with school shootings: the extent and results of research on the use of structured risk assessment tools for violence, and the application of psychological theory to explain the phenomenon. The conglomerate of explanations and research approaches to school shooting incidents may just reflect that different fields of research and different research traditions are involved. Yet, it appears that a great majority of research has had a descriptive and pragmatic approach to explaining and understanding school shootings, and consequently that theory-based understanding and research is scarce within this field.
Theories have several important features or characteristics. First, a theory is meant to synthesize an understanding of a phenomenon in order to obtain accuracy and simplicity. Second, it has to guide our comprehension by being focused, discriminative and selective to categorize our way of thinking. Third, it has a generative function that may vary from heuristic and loose assumptions to highly structured and formalized algorithms. Finally, a theory has a predictive function by contributing to comprehend presence and future. The diversity of research approaches involved in analysing school shooters has the potential of enriching our understanding. On the other hand, a dominance of "a-theoretical" contributions may create confusion and over-simplicity because they ignore the synthesizing, generative and predictive assets of theories. Psychological theories form a significant platform to understand how cognitive and emotional factors are involved in motivating behaviour. Wide spectrums of theories from developmental and clinical psychology are of interest for analysis of school shooters. Naturally, theories of aggression are of special relevance. Social learning theory (Bandura, 1973) and the frustration-aggression hypothesis (Dollard, Miller, Doob, Mowrer, & Sears, 1939) may provide relevant theoretical approaches to improve the understanding of school shootings. The same goes for more recent developments within theories of aggression, such as the General Aggression Model (GAM); e.g. (Bushman & Anderson, 2002).
In a recent systematic review, Sommer and co-workers (2014) concluded that social dynamics can play an important role in becoming a school shooter (Sommer, Leuschner, & Scheithauer, 2014). However, they also found a diversity of social dynamic interactions that were associated with the atrocity, and they failed to identify a common social dynamic interaction present in all cases of school shootings. They also emphasized the significant contribution of the perpetrator's perceptions of social dynamic interaction, and failure in emotion regulation and coping strategies. Still, they did not analyse social dynamics within a frame of psychological theory, leaving scrutiny of possible psychological explanations of school shooting undone. A preliminary search of the literature on school shooters yielded no publications that applied psychological theories to understand the phenomenon. Taken together, these findings lead us to do a systematic review of scientific research publications on school shootings with a main emphasis on research quality and the role of psychological theories and models.
We addressed this by exploring the following research questions concerning school shooters: (1) What is the evidence-based knowledge of risk factors? (2) What quality constitutes the research that has generated the current understanding? (3) What impact has psychological theories had on informing the understanding and design of this research?

Data collection and inclusion criteria for study type
Initially, we did a systematic search for English-language articles, chapters and books in the following databases: Medline (1967to April 2013), PsycInfo, (1967to April 2013 and Science Direct (1995to April 2013). An update from April 2013 to January 2015 was conducted. The search terms were School or schools and mass or multiple and shooting or murder or homicide or killing. A mass killing has been defined as an antisocial act of killing several people in a single episode or event (Levin & Madfis, 2009). Killings related to military or police service operations fall outside this definition. School shootings are often characterized by a single episode in which a student more or less randomly shoots classmates/teachers and where the perpetrator often die by his own hand or is killed in a shootout with police (Preti, 2008). Inclusion criteria for school shooting were (1) perpetrators who went to school where the attack occurred, or were expelled from this school, and (2) with multiple (three or more victims who were killed or injured at the actual teaching place). This limitation was done because we wanted to analyse school massacres and not cases where single or double homicide occurred due to more "ordinary" interpersonal conflicts. The definition is taken from the Federal Bureau of Investigation's (FBI) Crime Classification Manual (Douglas, Burgess, Burgess, & Ressler, 1992). Publications that only covered aftermath of school shootings for fellow students, gun control issues, healthcare systems or communities were not included in this review. The same applies to single murder or violence and multiple murders of young people outside the school context. We also excluded works that only addressed school shooting as a theoretical issue without presenting any empirical data e.g. (Kimmel & Mahler, 2003;Levin & Madfis, 2009;Preti, 2008). In accordance with previous method, recommendations for systematic literature reviews publications were not included if they had just been published as abstracts in conference proceedings (Jones, 2004;Knipschild, 1994). The authors also conducted hand searches in the reference lists of the retrieved papers. The data extraction was done according to the PRISMA checklist (Moher, Liberati, Tetzlaff, & Altman, 2009).

Sifting retrieved citations
We followed a traditional three-step procedure for literature review where we began sorting out from the title, abstract and a final full-text review (Jones, 2004). The authors who are experienced researchers and clinicians independently extracted studies according to the predesigned inclusion and assessment criteria. The final decision on inclusion of full-text papers was taken after consensus agreement between the authors (Figure 1). Papers that at any step failed to meet the inclusion criteria were excluded. Both literature reviews with case illustrations and case reviews were accepted for inclusion. The main difference between these publication types was that the scope of literature reviews was to provide a comprehensive presentation of variables such as prevalence and characteristics found in the literature, often for a limited time period. The case reviews aimed to use multiple cases to illustrate diversity, a new typology or warning signs. However, for inclusion they had to present empirical data and not only theoretical analysis of the phenomenon.

Assessment criteria for the empirical content of publications
Each included publication was analysed on the basis of the following information: number of cases, the number of killed and wounded, inclusion criteria for selection of case(s) and the main findings or conclusions. The main focus on empirical findings in the included literature was the following: the perpetrators' mental condition, clinical characteristics, typology, risk assessment and other psychosocial factors, with their possible association to the shooting incident. Methodological quality was assessed across 19 criteria from the Centre for Reviews and Dissemination's guidance for undertaking reviews in health care (CDC, C. f. D. C, 2008). However, the retrieved studies failed to meet but a few of these criteria. Therefore, the methodological quality was analysed with a shortened version of the CRD guidance without the following items: 2.4 what was the response rate? 3.2 is the measure of recidivism robust? 3.3 was the follow-up period adequate? And 3.4 what was the drop-out rate? However, it included to evaluate the aim of each article and to assess the research methods used in each article concerning the following variables: (1) Presence of a methods section, (2) Type of data sources, (3) Information about the perpetrator and (4) Research design and inferences made from the data (see Table 1).

Data sources
Sources of information were rated according to three quality levels: Primary: clinical interview with perpetrator, survivors or witnesses. Secondary: school, psychiatric, psychological, hospital, social and criminal databases, or other scholarly literature. The tertiary level covered information obtained from the media, and/or unauthorized Internet sites.

Information about the perpetrator
Information concerning psychosocial factors and clinical findings, and interpretations was split to determine whether the publication referred to information obtained before and/or after the shooting.

Design
Ratings of the design of each publication were categorized into weak, fair, good or very good. The minimum requirement for a quality rating of "Fair" was that the publication (1) had a method section or any other clear method description with inclusion and exclusion criteria for being a school shooter, and (2) used secondary data sources. A good rating would require the use of primary data sources. To obtain a very good rating the design would include a comparative study of a matched sample of school shooters and students with other types of school violence.

Interpretation
This part deals with our evaluation on a four-point scale of the quality of the inference from empirical data to conclusions in the publications. "Weak" means that there was a significant discrepancy between empirical data and conclusions, limitations to the design were not addressed, and no theoretical considerations were presented. "Fair" signifies that there was only minor imbalance between data and conclusions, that theoretical considerations were only partly integrated, but limitations of the design were not adequately discussed. "Good" reflects a good balance between empirical data and conclusions, integrated theoretical considerations and limitations of the design were sufficiently discussed. Finally, "Very good" indicates that the balance between data and conclusions was very good, theoretical considerations were presented in detail and design limitations were addressed to the point. Information concerning the perpetrator before the shooting was gathered retrospectively. 4 Information concerning the perpetrator after the shooting was gathered retrospectively. 5 Overall evaluation of the method used to obtain empirical data in the study: weak, fair, good and very good. 6 Inference from empirical data to conclusion concerning finding: weak, fair, good, very good. 7 Ten cases. 8 One case.

Information about perpetrator
Research quality

Assessment criteria for the theoretical content in the publications
Each article was scrutinized for the use of theories and models of psychology in the design of the study, and in the process of inferring from data to conclusions concerning school shooters. A positive identification of the use of theories from developmental psychology, clinical psychology or theories of aggression was obtained by separate coding of each article followed by consensus agreement between the authors. Rating options were present, partially present and not present. The main criteria were that psychological models or theories were used in the: (1) design of the study, (2) interpretation and synthesis of the empirical data and (3) whether the validity of a psychological theory was tested in the study. A psychological typology based on personality traits is an example of a psychological model. Diagnostic features and symptoms such as social withdrawal or a delusion are not. Notwithstanding, just referring to the name of a model or a theory without further elaboration and integration of its relevance in the analysis of data did not meet the criteria.

Use of risk assessment tools
We scrutinized each publication for use of actuarial (e.g. the Violence Risk Appraisal Guide (VRAG), (Harris, Rice, & Quinsey, 1993) or structured professional judgement tools (e.g. Assessing Risk for Violence (HCR-20 V3 ), (Douglas, Hart, Webster, & Belfrage, 2013) for risk assessment of violence. The minimum requirement for a positive quality score of 1 was referring to such tools. A positive quality score of 2 was obtained if any of these instruments had been used to assess the school shooters.

Results
A total of 422 hits were reduced to 10 using the three-step procedure for inclusion in our review.
Hand searches resulted in one additional article and a final result of eight articles, one report and one book chapter for full text review (Figure 1).
We distinguished between case-based publications and theoretical reviews with case descriptions. We found five selective case reviews and five literature reviews (Table 1).
Apart from two articles and one report from 1999 to 2003, the included papers were published between 2008 and 2013. Nine of the papers had authors from the US and one originated from

Study Basis and aim of article Main findings Use of theories of psychology
McGee and DeBernardo (1999) Selective case review 1 Cases selected to present and analyse school shooters (ss) characterized as "Classroom avengers".
A compilation of 44 inclusion and 45 exclusion criteria pertaining to socio-demographical and clinical character-ristics of being a "Classroom Avenger."

Theory-based interpretation No
Empirical testing of theory No Vossekuil et al. (2002) Identify information that could be obtainable, or "knowable," prior to an attack. Information analysed and evaluated to produce a factual, accurate knowledge base on targeted school attacks.
Identified 37 incidents of targeted schoolbased attacks, committed by 41 individuals over a 25-year period.

Theory-based interpretation No
Empirical testing of theory No Wike and Fraser (2009) Selective review of the literature. No clear profile found though case comparisons and anecdotal data show common features: (1) fascination for and access to guns; (2) Leakage of plan of the act in advance. Special characteristics of perpetrators: (1) Depression, anger and suicidal tendencies (2) Rejection and victimization by peers.

Theory-based design No
To examine the nature of school shootings. Discuss individual characteristics of perpetrators and the vulnerabilities of schools were shootings have occurred. Review plausible prevention strategies.

Theory-based interpretation No
Empirical testing of theory No Rocque (2012) A selective review of the literature. The media has tended to overreact to school shootings. Empirical data are sparse, anecdotal and mainly based on case studies. There is a shortage of sophisticated theories on school shootings. Policies have mostly involved "target hardening" measures.

Theory-based interpretation Partly
Empirical testing of theory No Dutton et al. (2013) Selective case review. Casuistic study of four persons, three were ss.
Perpetrators suffer from severe pathology. They are fixated and obsessed with rejection by others. They formulate plans to annihilate the transgressors and justify this as vengeance for the transgressions made against them. These perceptions are more consistent with paranoid thinking than with psychopathy.

Theory-based design No
Theory-based interpretation Partly Proposes that "psychological diagnosis" of mass murderers is often based on symptoms that are shared with other diagnoses. They argue that some mass shooters reveal profound disturbances in ego-identity and paranoia.
Empirical testing of theory No Langman (2013) Selective case review of the literature to classify 35 shooters into a typology of perpetrators.
The ss were classified into a typology comprising 7 psychopathic, 11 (14) psychotic and 8 traumatized perpetrators. The rest of the shooters were classified as either uncategorized or tentative.

Theory-based interpretation No
To provide a data-based research foundation by analysing a large enough sample of rampage ss.

Empirical testing of theory No
Canada. The school shooter cases described and analysed in the publications were all from the US, and there was a substantial overlap of some of the most notorious school shooters in the papers. The number of trait, personality and behaviour characteristics of school shooters ranged from 3 to 44 in the publications, and hence there was some overlap of these characteristics among the publications.

Findings concerning method, data sources and research quality
None of the studies were systematic reviews. All of the included papers analysed selective samples of school shooters adapted to the purpose of the individual research. Four studies contained a separate methods section stating how they collected data and what sources they used. In the remaining publications, this was partly accounted for in bits and pieces, yet not explicitly described and definitely not easy to find ( Table 2).
All publications had information about the perpetrator from the time before the shooting, but only half of them presented information from the time after the atrocity. Regarding research quality, no study did qualify for having a good or a very good design. One study was considered to have a good interpretation of their data.

Findings concerning the use of psychological theory in design, interpretation and empirical testing
Both authors scrutinized the articles separately and rated the articles' use of theoretical considerations as not present, partly present or present concerning (1) theory-based design, (2) theory-based interpretation and (3) empirical testing of theory. We achieved 74% absolute agreement for our ratings and this was interpreted as adequate, particularly because every disagreement was followed by a consensus decision by the raters. According to Neuendorf (2002), 80-90% agreement is nearly always acceptable and even 70% agreements may be appropriate in some studies. All disagreements were of a minor character, usually in the shape of "Not present" vs. "Partially present". However, this divergence was limited to whether the publication had a theory-based interpretation or not. When we reanalysed the articles to obtain a consensus decision, we found that some articles only referred to relevant concepts, such as rejection and attachment problems, in a descriptive way. Three publications provided partly use of theory-based interpretation of findings. Still, we found no evidence of any actual use of relevant psychological theories to generate a more comprehensive and dynamic understanding of these interpersonal problems and why they should result in school shooting. Not a single publication was rated to have a theory-based design or to report findings from an empirical testing of a psychological model or theory. No use of or reference to structured judgement or actuarial risk assessment tools was found in the 10 publications.

Discussion
The main findings of the review were: (1) Empirical research is scarce, (2) The quality of the research is mostly inadequate, and (3) The role of theories of psychology is virtually absent in the research of school shooters. The search in the literature yielded 10 studies, five case reviews and five literature reviews. Only four studies contained a separate methods section stating their data collection method and what sources they had used. Two studies presented data from primary sources, in the shape of interviews with one or more school shooters. Secondary sources such as school, hospital and/or psychological and psychiatric evaluations were used in four studies, while the rest had only applied tertiary data sources. We found that all gave information about the perpetrator before the shooting, but only five had obtained any information from shooters that had survived.
We found that several authors tried to develop common denominators and profiles of the attackers. However, it appears to be very little evidence-based support for specific typologies or profiles of school shooters. The presented typologies expressed as risk profiles may appear to have good face value. However, close scrutiny disrobes flaws concerning the discriminative validity of these typologies. A given school shooter may have a certain background and exhibit certain characteristics (young, white, bullied, lonely, feminine, socially alienated, psychopathic or psychotic). Clinically, it does make sense that such a person may develop a rage and explode in a fatal violent act. Still, the vast majority of those who have been bullied never commit any (serious) act of violence. To put it in another way, for each school shooter that fits into a given profile there are thousands of other nonviolent young boys that also fit the same profile. In the most thorough report, we reviewed it is clearly stated that: "There is no accurate or useful 'profile' of students who engaged in targeted school violence." (Vossekuil, Fein, Bornum, & Modzeleski, 2002). It is therefore almost impossible to foresee or predict the "typical" school shooter on the basis of such typologies. A whole spectrum of different school shooter profiles, ranging from the classroom avenger to the psychotic or psychopathic perpetrator, has been suggested. For example, Leary, Kowalski, Smith, & Phillips (2003) argued that in 13 of 15 cases that they reviewed the perpetrator had experienced social rejection. In their discussion section, they claimed that "certain personality disorders are characterized by aggressiveness, paranoia, low impulse control, lack of empathy for other people, and even sadistic behaviors, all of which may lower one's threshold for violence" and then they referred to Millon (1981). However, we found no research evidence for their use of this theoretical approach to measure and document the association between these characteristics and school shootings. In fact, it looks as though the lack of empirical testing and support to school shooter typologies has contributed to construction of new profiles rather than to inspire empirical validation research.
The models presented about school shooters, that is, their motives, psychiatric typologies or risk profiles, are primarily based on case studies. Generally, these studies have comprised very few cases which weaken the validity of the proposed models. In addition, we found that the case studies were rarely based on primary sources. Only two of the studies included in the review contained information obtained from a primary source. Researchers conducted interviews with 10 perpetrators of school attacks (Vossekuil et al., 2002) and a single interview of a school shooter (Reuter-Rice, 2008).
The rest of the studies were based on secondary and tertiary sources procured retrospectively. Profiling based on such alleged perpetrator characteristics (e.g. fascination of weapons, bullied feminine boys, paranoid thinking or socially alienated boys) can lead to false positives and unfortunate stigma (Leuschner et al., 2011).
The explanatory models presented can at best provide hypothetical and tentative explanations of the dynamics of school shootings. No single study had a theory-based design. As a corollary, we found no investigation with empirical testing of psychological theories of aggression. However, three articles partially used theory-driven interpretations in their explanation of school shooters. Based on psychodynamic theory, Leary and co-workers suggested that rejection may precipitate painful feelings of shame and that this may provoke anger and aggression. Still, they emphasized retribution as the primary motive in most of the school shootings. Rocque (2012) referred to Fast who claimed that rampage shootings in schools were ceremonial violence caused by several factors, such as brain damage, mental illness and social isolation (Fast, 2008). The main motive was to gain status and prestige. Dutton and collaborators leaned upon Harry Stack Sullivan's psychodynamic interaction theory (Sullivan, 1956). A deep sense of inferiority generates anxiety and chronic feelings of shame and humiliation. To cope with this, transfer of blame away from the self and onto others nurtures the development of a paranoid personality. However, these somehow overlapping theoretical considerations fail to answer the basic question; do they actually explain why some of these persons become extremely violent and others not? Even more so, do they explain school shootings? In our opinion, they don't and the reason for this is the absence of any empirical evidence of the predictive validity of these theoretical models.
At present, no single explanatory model is developed to inform risk assessment of who will cross the line and actually start to kill their schoolmates and teachers. However, the more general a model is the more likely it will generate false positives and misidentification of key risk characteristics. Based on the current knowledge of school shooters, one claim appears to be as good as the other. Hence, the existing literature based on typologies, profiles and theories may at best offer some knowledge about the dynamics behind one or more of the episodes that have taken place. Still, they do not identify risk factors that can enhance prediction and prevention.
We also found that no single study had used or evaluated the potential of actuarial or structured professional risk assessment tools. One interpretation of this is that researchers in this field make a clear distinction between school shooters and other violence perpetrators. In other words, school shooters are seen as belonging to a unique or exclusive subgroup of violent perpetrators compared to other perpetrators of serious violence. However, if this reflects a deliberate choice one would expect an explanation and a rationale for this point of view. This may, however, turn out to be a valid choice, but without any research evidence for the distinction further studies are needed to clarify the issue.
A limitation to our review is that we may have overlooked some relevant studies. However, this possible flaw is to some extent modified by the absence of new relevant papers in the reference lists of the articles we reviewed. Still, we must keep the possibility open that there may be unpublished reports with both better designs and more valid results than those found in our review. Even if the absolute inter-rater agreement for this research was adequate, there is always the risk of confirmation bias in this type of investigation. This weak point must be taken into consideration when interpreting our findings.
Future research may want to abandon further attempts to make profiles and predictions based on very small numbers of persons and incidents. Although researchers must expect to be confronted with inherent ethical and design obstacles, we request a stronger effort to use primary sources to inform the development of efficient prevention measures. The rationale behind this change of main focus from risk assessment to prevention is to invest resources that can mitigate risk of violence in general in the school context. At present, this broad-spectrum approach appears to be the closest one gets to reduce the risk of school shootings. A series of strategies have been suggested to prevent future school shootings (e.g. Wike and Fraser, 2009). So far, the evidence for any efficient strategy tailor-made to reduce rates of school shooting does not exist. The main reason for this is the complexity of the phenomenon and failure to identify specific warning signs and risk factors for school shootings in current research. A recent meta-analysis of 36 studies (N = 113,778) showed a moderate negative association between students' perceptions of school climate and violence. However, attempts to identify explanatory factors such as students' and school' characteristics for this relationship failed (Steffgen, Recchia, & Viechtbauer, 2013). Even though the base rate of violence is dramatically higher than for school shootings, this finding illustrates the complexity of identifying risk factors for school violence in general.
To sum up, there exists no substantial scientific base for assuming that valid risk factors or profiles can be identified to predict or prevent any event with a base rate as extremely low as multiple school shootings actually have. This fact contradicts the view that it is feasible to develop reliable and valid school shooter profiles. Despite this, the belief in profiling appears to persist within this field of research, and even the lack of empirical evidence concerning psycho-social typologies in general appears to have no impact. Hence, we recommend a change from a limited focus on school shootings to research on warning signs and risk factors for school violence in general. There are good reasons to believe that this research will generate findings that are relevant to develop efficient violence prevention strategies to mitigate school violence. Still, whether this will have a positive effect to lower the rates of school shootings is an empirical question.