The Safe pilot study: A prospective naturalistic study with repeated measures design to test the psychosis - violence link in and after discharge from forensic facilities

The research evidence is very strong for high recidivism rates of violence after discharge from forensic facilities. Big data research has found that a substantial proportion of the forensic population with relapse into violence has a psychosis diagnosis and a criminal record. However, more research on the association between psychotic symptoms and violence may inform and enhance risk assessment, prevention, and treatment. We conducted a prospective naturalistic study with a repeated measures design in a sample of 22 psychotic patients during follow-up after discharge from forensic mental health facilities. We had three aims: to test the predictive validity of three psychotic symptom scales for violence, to analyze main and interaction effects between psychotic symptoms and previous criminal conviction, and to explore the feasibility and potential benefit of the repeated measures design for prospective follow-up research. Interpreted within the limitation of the small sample size, the results were promising for all scales, particularly for adjusted effects without interaction. Two scales remained significant when their interaction with criminal conviction was adjusted. This indicates that risk judgment of psychotic patients with criminal conviction can be improved by adding measurement of fluctuations in psychotic symptoms. The repeated measures design was instrumental in this research.


Introduction
Despite some exceptions (e.g., Hayes et al., 2014), research indicates high violent recidivism rates after discharge from forensic facilities (e.g., Fazel et al., 2016;Krona et al., 2016;Tabita, et al., 2012). In searching for clinical predictors of violent recidivism, diagnoses such as schizophrenia (e.g., Douglas et al., 2009), personality disorder (e.g., O'Connell and Marcus, 2019), childhood victimization (Krona et al., 2016) and drug abuse (e.g., Walter et al., 2019) have been identified. Although diagnoses may be significant risk factors of violence, their clinical relevance ostensibly is low, relative to a symptom-level approach. Certainly, in the development and implementation of violence risk management plans and therapy in clinical contexts, assessment of psychotic symptoms has greater potential for clinical relevance and risk prevention (e.g., Keers et al., 2014;van Dongen et al., 2014). Further, symptom-level measurement has been found meta-analytically to have stronger associations with violence than diagnostic-level measurement (Douglas et al., 2009). This paper reports findings from a prospective investigation of psychotic symptoms as dynamic violence risk factors. To be considered dynamic, a risk factor must be shown to change in nature, frequency, or severity, and these changes must be associated with subsequent changes in violence risk (e.g., de Vries Robbé et al., 2015).
Three systematic reviews concerning the psychosis-violence link were published in 2013 (Nederlof et al., 2013;Reagu et al., 2013;Witt et al., 2013). The review by Witt and co-workers tested a series of dynamic risk factors, but failed to focus on emotional distress and psychosis. The two others found that emotional distress (particularly anger and anxiety) increased the risk of violence. Still, both reviews investigated psychosis at a general diagnostic level. This categorization is likely too crude to inform risk assessment of violence on the individual level because it treats all persons in a uniform manner, despite great variability in symptoms and associated features across people. For instance, when psychotic symptoms are accompanied by emotional distressespecially angertheir connection to violence is strengthened (e.g., Peterson et al., 2014;Ulrich et al., 2018;Van Dongen et al., 2012). So far, research indicates that emotional distress, predominantly anger, serves as a catalyst between psychotic symptoms and violence, echoing earlier theorizing (Junginger, 1996;Taylor, 1998).
In the last decade, research has shown a strong association between psychosis and subsequent criminal convictions and violent acts (e.g., Krona et al., 2016;Purcell et al., 2015;Winsper et al.,2013;Witt et al., 2015). Purcell and collaborators (2015) conducted a cross-sectional investigation of 271 young help-seeking persons at risk for psychosis and a control group that was not at risk for psychosis (n = 440). The at-risk for psychosis persons had not been charged or convicted more than the controls for violent offenses, but they were twice as often charged and convicted for non-violent offenses. This was the first study to address the association between previous non-violent criminal offending and increased violence risk in an at-risk group consisting of people who had not yet developed psychosis and who had previously not been convicted of violent crimes. However, due to the use of a cross-sectional instead of a prospective design, this study could not provide evidence for the previous non-violent criminal offending-violence link.
In a prospective study, Winsper and colleagues (2013) followed 670 first-episode psychosis (FEP) cases at 6-and 12-months post-discharge. Latent class growth analyses identified four subgroups of premorbid delinquency: stable low; adolescent-onset high to moderate; stable moderate; and stable high. Results showed that increased risk of violence was partially motivated by psychotic symptoms in the moderate delinquency group, while being in the stable high premorbid delinquency group had a direct effect on risk of violence. They concluded that there seemed to be different trajectories to violence during FEP, but also emphasized previous delinquency as a strong risk factor of violence. Witt et al. (2015) conducted an epidemiology study with almost 14, 000 persons with schizophrenia. Violent and non-violent criminal history factors were most strongly associated with subsequent violence, even when controlling for young age and comorbid substance abuse disorder. However, Krona and colleagues (2016) conducted a follow-up register study in a cohort of 125 discharged forensic patients, of whom 91 had psychosis. Their main finding was that childhood adversities together with early debut in non-violent criminality was a risk factor for general criminal recidivism, but not for violent crime. In sum, there is research that indicates that a psychotic person with a violent or non-violent criminal record poses a higher risk of violence than one without criminal conviction. Of course, such is true of persons without mental disorder as well (Andrews et al., 2006). So far, the research designs used for tests of psychotic symptoms and criminality as risk factors of violence have been predominantly population-based epidemiological surveys or register studies, retrospective criminal file studies, and cross-sectional designs. In sum: (a) prospective investigations are scarce, (b) explanatory and outcome variables were methodologically limited, and (c) there is a paucity of studies combining criminal conviction and psychotic symptoms as predictors of violence. This paper reports from the Safe pilot project that was conducted in five Norwegian forensic mental health units. It had four major research topics: psychopathology and general level of functioning; risk assessment; risk management strategies; and recidivism (Bjørkly, 2004;Bjørkly et al., 2019). One of the main aims of the risk assessment part was to investigate dynamic risk factors for violence at the individual level. The possible dynamic risk factors for violence in the Safe pilot project included dissociation, hopelessness, insight (inversely), resilience (inversely), and psychotic symptoms. Our main focus in this paper is on the measurement of psychotic symptoms and their association with violence in patients before and after discharge from high and medium security psychiatric facilities. We used three validated instruments to assess hallucinations and delusions. One reason for choosing these scales was that they included measures of emotional distress, such as anger and anxiety, related to the actual psychotic symptom.
We investigated the association between having a criminal conviction and psychotic symptoms, on the one hand, and violence during follow-up, on the other. The reason for this was that within the context of violence risk assessment and prevention, criminal conviction is at best a historical risk factor and of limited use in terms of risk management. For patients with criminal convictions who pose some risk of violence, we need additional changeable risk factors to optimize assessment of future risk. Hence, we sought to determine whether dynamic measurement with psychotic symptom scales could improve upon the risk assessment of patients with previous criminal conviction. If so, one could combine a historical and clinical risk factor with potential to identify risk and also instigate preventive measures, tailored treatment, and further development for the patient.
When we analyzed dynamic risk factors in the Safe pilot study, we used a prospective repeated measurements design to estimate the association between risk assessment and violent recidivism inside or after discharge from medium and maximum secure units. To our knowledge, no research has tested the psychotic symptoms-emotional distress-violence link in a prospective repeated measures design. The primary strengths of the repeated measures design are that it makes experimental, observational, and intervention studies more efficient and helps to keep the error variance low. This aids in keeping the validity of the results higher, while still allowing for smaller than usual subject groups. We conducted a search of the literature in Ovid MEDLINE(R), Embase, and PsychINFO from 1975 to August 1, 2020, on prospective repeated measurements design in follow-up studies before and after discharge from forensic units (search terms on request). No relevant publications were found. This resulted in our third research aim (see aim 3 below).
One important factor in this study is that a battery of dynamic risk factors was used three times during the year before discharge and one time after discharge. Within the limitations set by the small sample size, the main aims, controlling for confounders such as drug abuse and a history of childhood victimization, of the present research were to: 1 Explore the association between three psychotic symptom scales and risk of violence during follow-up; 2 Analyse possible main and interaction effects between measures of psychotic symptoms and previous criminal conviction on violence during follow-up; and 3 Examine the feasibility and potential benefit of the repeated measurements design for prospective follow-up research on the psychosis-violence link.

Setting
One year before the Safe pilot study finished, there were 30 beds in high security and 135 beds in medium security in the two regional healthcare trusts we recruited from. They served a population of about 4 million inhabitants (Statistics Norway [SSB], 2015).
The study involved one maximum security forensic psychiatric ward (n = 10 beds) and four medium secure wards (n = 48 beds) in the southern and western parts of Norway. The research was conducted with a naturalistic, repeated measures design and lasted from 2006 to 2016.

Participants
The Safe pilot project had 27 patients recruited from the maximum and four medium secure wards. Only 22 of these had psychosis. The sample with psychosis consisted of 3 women and 19 men. There were two good reasons for focusing on history of violence and psychosis. First, About 90% of Norwegian forensic patients have violence as inclusion criterion for admission. Second, it is important to find splitting factors for violence recidivism within the violent group in a prospective study. This time we focused on psychotic symptoms, given their importance and wide-spread occurrence in this population. The great majority were of Norwegian ethnicity and citizenship.
Seventeen had never been in an intimate relationship and for the ones who had been (n = 5, 23%), it was prior to entering the Safe pilot project. At the start of the pilot project, 13 (59%) patients were without a permanent address, 4 (18%) rented housing, 3 (14%) lived with relatives, and 2 (9%) owned their housing. Their main income was disability pension or other social support. The highest education level was primary school for 7 patients (32%), 2 (18%) out of 11 who had started upper secondary school passed, and 1 out of 3 had managed to get a university degree.
Fourteen patients had long stays in general psychiatry before forensic mental care (M = 584.33 days, SD = 1021.74). Fourteen patients were admitted to forensic inpatient treatment before the current stay. Mean number of days in the previous stays was 875.86 (SD = 1326.50). Seven patients had been treated in substance abuse institutions (M = 139.14 days; SD = 205.86). According to the ICD-10 criteria (WHO, 1992), all patients in the sample had a main diagnosis within the schizophrenia spectrum disorders (17 (77%) patients with paranoid, 3 (14%) with hebephrenic schizophrenia, and 2 (9%) with schizoaffective disorder, unspecified). Four patients had a secondary diagnosis of substance-induced psychosis, and one patient had dissocial personality disorder.
The most serious violent acts before being admitted to the current stay included homicide (3 patients and 4 first degree murdersone patient had committed two grave sadistic homicides) and severe physical assault (n = 19). The mean age at discharge was 34.08 (SD = 6.19). Nine patients were discharged to community housing with partial staff follow-up from community health and social services. Two patients moved to community mental health centres. Six patients were discharged to locked wards in general psychiatry and 5 were not discharged from the forensic facilities during the project. Given that we were not allowed to record information on people who were not enrolled into the study, we have no information on the total number of patients in the wards during the study period. However, roughly 80% of forensic patients in Norway would meet our inclusion criteria of having a schizophrenia spectrum disorder and a history of violence.

Research design
In this naturalistic observational study, we used a prospective repeated measures design to compare violent and non-violent patients during follow-up from the first measurement to at least one year after discharge. In this paper, we present the first main aim of the investigation, which addressed the link between psychotic symptoms and violence measured with three scales for psychotic symptoms (see 2.4). The sample (n = 22) was measured over three times with the three scales (M = 4.2 times); there were over 90 complete measurements. Out of these, 2 (9%) had only three assessments, 15 (68%) had four measurements, 3 (14%) had five, and 2 (9%) had six measurements.

Measures
We used the following instruments to measure psychosis: The Psychotic Symptom Rating Scales (PSYRATS A (11 items) & B (6 items); Haddock et al., 1999) and the Psychiatric Epidemiological Interview (PERI, Dohrenwend et al., 1986). PSYRATS A measures auditory hallucinations, PSYRATS B measures delusions, and the PERI is used to detect Perceived Threat and Control Override (TCO) delusions (3 items for persecutory delusion with loss of control over own thoughts and emotions, Link et al., 1998). One important advantage of PSYRATS A and B is that they measure emotional distress associated with hallucinations and delusions. The scales have good to very good inter-rater reliability and good predictive validity. They were used at least three times after baseline with three months between each measurement.
Violence was defined as intentional attempts at, threats of, or actual and intended infliction of bodily injury or harm on another person. The main criteria in this definition are from the MacArthur Violence Risk Assessment Study (Monahan & Appelbaum, 2000): physical assaults leading to physical injury in another person; the use of a dangerous object/weapon against another person; threats of using a weapon against another person; and the use of physical force in connection with sexual offenses. Physical and verbal threats that clearly implied an imminent physical assault were included in our study. Damage to property and harm to self did not meet the criteria for violence, and fire setting was only included if it was done with the intent to cause bodily harm to other people.
Information about type and numbers of violence acts for each patient was recorded in the REFA form (Report Form for Aggressive Episodes; Bjørkly, 1996). In addition, the Safe research coordinators scrutinized ward reports and medical records in the electronic journal system. To obtain consensus agreement on aggressive acts, they interviewed and discussed with clinicians in charge of the patients during follow-up in hospital or community services. They also recorded type of treatment unit and follow-up time after discharge.

Procedure
Recruitment of Safe pilot patients was based on decisions from the clinicians who were responsible for their treatment in the secure wards to discharge the actual patient within the coming year. Patients who were evaluated to be ready for discharge within 9 to 12 months were included, and, since the study had a naturalistic design, the Safe research coordinators had no formal impact concerning discharge decisions. The clinician in charge of the patient contacted the Safe project coordinator for inclusion in the project. The final decision was taken in the Safe pilot project meetings. The main premise for inclusion in the research project was that the observer-rated repeated measurements were feasible for the actual patient. Only patients with a history of violence were included. There were no additional inclusion criteria. No application was declined by the research group during the project. The observer-rated assessments were coded by the research coordinator based on clinicians' and nurses' information from observations of the patients. The research coordinators' role was to collect and secure data.
The Safe pilot project was approved by the Regional Committee for Medical and Health Research Ethics South-East in Norway. The first part of the study was approved on the condition that informed consent was obtained from the patients (REK East-Norway, ref 27-6003 1.2005.2678). However, because the great majority of patients withdrew their consent when they relapsed into, for example, an acute psychotic phase, it was impossible to do this research. After three years and a new application to the Regional Ethics Committee, the project was granted approval without informed consent, provided mandatory communication of information about the project to the participants (REK South-East, ref 1.2005.2678, 2009/167). The main reason for the approval was that the project applied observer-rated assessment by clinical staff that the patient knew and that this type of procedure was identical to the routine clinical assessments in the wards.

Statistical analysis
The data were cleaned and quality controlled by two research assistants and the first author and coded into SPSS files (IBM SPSS Statistics 25). Univariate and bivariate analyses were conducted with IBM SPSS Statistics 26. Stata 16 was used for calculation of exact p-values and for analysis of repeated measurements. Exact p-values were used due to small sample sizes. However, no exact tests exist for analysis of repeated measurements. A usual significance level of 5% was used. When comparing independent data for the violent (n = 7 patients) and the nonviolent subgroups (n = 15 patients), we used the Mann-Whitney (MW) exact test for non-normal scores, Fisher-Freeman-Halton (FFH) exact test for associations presented in RxC contingency tables, and Fisher exact test for 2 × 2 contingency tables. A generalized linear mixed model (GLMM) with logit link was used for analysis of the occurrence of violent acts for repeated measures over the five time points. Linear mixed models were used to analyse the association between the total number of violent acts (N = 46) and psychotic symptoms. We used risk difference (RD) and the McNemar exact test for the analyses of changes in risk of violence over the five time points. Estimates of Variation Inflation Factor (VIF) was used to test for multicollinearity between independent variables and interpretation of normal P-P plots for normality of residuals in linear regression. We also compared models with only main effects and models with adjustments for interaction terms, and tested models with only main effects before interaction terms were entered in the model. Only significant interaction terms were included (p ≤.05). Because we used an exploratory research design, no sample size calculation was performed. We did not estimate observed power, i.e., post hoc statistical power, since it gives no additional information beyond reporting the pvalues and the confidence intervals (Hoenig & Heisey, 2001).

Results
Six men and one woman were violent during follow-up. The total was 46 violent acts dispersed across 25 verbal threats, 12 physical threats, and 9 physical assaults. Four violent patients were in community services where the totality of violence included two physical assaults and three verbal threats. Corresponding figures for the three violent persons who happened to be in locked wards was 22 verbal threats, 12 physical threats, and 7 physical assaults. One patient was in community services on the first and second measurement points (two physical assaults) before he was readmitted to medium security and stayed there during the last three measurement points (three verbal threats, three physical threats, and three physical assaults).

Univariate comparison of sociodemographic variables, previous treatment, and follow-up time between the violent and non-violent groups
We tested possible differences between the violent (n = 7) and the non-violent (n = 15) groups for age, ethnicity, previous treatment in general psychiatry before the current stay in the forensic unit, and follow-up time after discharge from the forensic psychiatry unit (see Table 1). We found no significant differences.

Early history of victimization and/or perpetration of violence
We compared the groups for historical factors of violence risk, but failed to find any significant difference between the groups: Age at first violent act (up to 18 years), violent (n = 6): M = 15.50 years, SD = 1.87 vs. non-violent (n = 12): M = 13.75 years, SD = 3.28, (MW exact test, p = 0.39). Numbers and types of violence as victim and/or perpetrator are presented in Table 2.

Substance abuse
We analysed two factors related to substance abuse and violence and found no significant differences between patients with and without violence during follow-up. Comorbid substance abuse diagnosis: violent (n = 1) vs non-violent (n = 4) (Fisher exact test, p = 1.00); a history of violence conviction under the influence of drugs, no drugs = 4 in violent vs 9 in non-violent group; narcotics = 0 vs 2; narcotics and alcohol = 3 vs 4 (FFH exact test, p = 0.67).

Previous criminal convictions
Ten patients in the non-violent group had at least one criminal conviction (drug dealing: 1 patient, grievous bodily harm: 6, and homicide: 3). Corresponding figures for the violent group were drug dealing: 1, grievous bodily harm: 4, and no homicide. No significant between-group difference was found (Fisher exact test, p = 0.55).

Analysis of repeated measures
We used matched pairs analyses to estimate the association of paired data over five time points. The association was significant for two periods between time point 1 to 4 and 1 to 5, with a reduced risk of violence of 27.3 percentage points in both cases (RD = 0.27, CI 95% = .041, .504; McNemar exact test p = .031). We analysed the association for PSYRATS A and B and violence with generalized linear mixed models with logit link. Both failed to produce significant results. Hence, we analysed the aggregated mean value of PSYRATS A and B scores with a significant result. A one-unit change in the mean value in PSYRATS A and B scores gives a 5 times increased odds for patients belonging to the violent group (OR = 5.18; p = .054; CI 95% = 1.02, 26.34). The TCO scale failed to produce significant results in the generalized linear mixed analysis.

Table 1
Comparison of age, ethnicity, previous treatment in general psychiatry, and follow-up time for patients in non-violent (n = 15) and violent group (n = 7).  First, we tested the unadjusted association between three measures of psychotic symptoms (PSYRATS A, PSYRATS B, and TCO) and criminal records with violent acts during follow-up (n = 7 patients with 46 acts of violence). Only scores on TCO had a significant association with violent acts (β = .30, p = .05, CI 95% = 0.000 -0.600) (Table 3), while PSYRATS A (p = .08) and the aggregated mean of PSYRATS A and B (p = 0.10) showed trends towards significance.
Next, we estimated the adjusted association between the scores of PSYRATS A, PSYRATS B, PSYRATS A and B (mean), TCO, and criminal record, and number of violent acts during follow-up.
In the analysis of adjusted effects without interaction term, all variables were significant (p range = .002 -.04 and β range = 09 -1.04). In the last step, the interaction term for each instrument and criminal record was entered together. Only PSYRATS A (β = 0.59, p = .019, CI 95% = .10 -.93) and TCO (β = 0.72, p = 0.008; CI 95% = 0.19 -1.25) had a significant positive interaction effect with criminal record. Yet, comparing the main effect of PSYRATS A without and with interaction term was halved, and the main effect of TCO was reduced by about 20%. The standardized residuals for selected and unselected cases had an approximately normal distribution. No multicollinearity was found (VIF range = 2.45-4.98).

Discussion
Our ambition with this small-scale, repeated measurements research was to generate hypotheses for further validation in larger-scale research. Despite its modest scope, we still completed nearly 100 evaluations on 22 patients across about a mean length of two and a half years follow-up, both within and outside of secure forensic facilities. We found a significant reduced risk of violence between the first and last two measurement points in the repeated measures design. This may reflect a change for the better for patients during follow-up after discharge. However, even if this study did not focus on effects of risk management or treatment, one interpretation may be that the shift of treatment context may have enhanced the patients' quality of life. A more speculative view is that the effect of repeated clinical assessments of psychotic symptoms may have informed the interaction and communication between clinical personnel and patients in a positive way. These are observations to be studied systematically in future research.
We analyzed the association between scores from three psychotic symptom scales -PSYRATS A (auditory hallucinations), PSYRATS B (delusions), and PERI (TCO delusions) -and violence during follow-up using logistic regression, without significant results. However, when we calculated and tested the aggregated mean value of PSYRATS A and B, we found a strong positive association with violence during follow-up. This may be so because we combined hallucinations and delusions into one predictor variable with more variability and information. Moreover, the low statistical power was reflected in a wide confidence interval and that must be taken into consideration when interpreting this finding.
The second scope was to combine the repeated measures results of psychotic symptoms with criminal conviction, a historical risk factor of violence. Naturally, there exists no specialized treatment for having a history of previous criminal convictions. Psychotic symptoms, on the other hand, are dynamic, fluctuating, and amenable to treatment. Therefore, we wanted to explore whether psychotic symptoms had an impact on risk of violence in patients with criminal convictions and to explore how the two risk factors interacted. We started with analyzing unadjusted effects (Table 3). TCO was the only significant explanatory variable for violence during follow-up. PSYRATS A and the aggregated mean of PSYRATS A+B trended toward significance. Criminal conviction was not significant.
In the second step (Table 4), we tested PSYRATS A adjusted for effects of criminal conviction without interaction term. Both variables were significant. When we explored the nature and effect of the interaction term, it turned out to be the only significant variable. Similar results were found for TCO for both adjusted effects without and with interaction term. This offers support to an interaction effect between a historical and a clinical risk factor. Yet, PSYRATS B and the aggregated mean of PSYRATS A+B, respectively, obtained only significant results in the adjusted effects analysis without interaction terms.
A majority of our participantsall of whom had mental illness and previous criminal convictionswere not violent in this sample. Based on our findings, we suggest that dynamic risk factors combined with criminal conviction may optimize assessment of risk of future violence in this group. For psychotic persons with a history of criminal conviction, our results indicate that the interaction between repeated dynamic measurements with scales for auditory hallucinations and persecutory delusions may be scrutinized further to explore if they enhance predictive validity in assessment of risk of violence. However, for delusions there was no interaction effect, and the main effect was low with a significant trend, while the main effect of criminal conviction was high with a marginal significance estimate. Good to very good results were found for the aggregated mean effects of hallucinations and delusions and criminal conviction. This variation of results may illustrate the statistical power issue for this research. Hence, further research with a larger sample size is called for.  It has been claimed that small-scale research has some advantages that big data designs often lack (e.g., Stewart & Davis, 2016). Big data in mental health research has so far been applied in epidemiological archival and genetic studies (Weissman, 2020;Favaretto et al., 2020). First, our research format brings research closer to the psychotic and violent person's inner life and to the context of professionals' clinical work. Obtaining precise information on whether symptoms of mental disorder were actually present at the time of a violent act will never be possible in big data designs. Having obtained this information it should be an integrated part in tailored treatment and risk management with emotion regulation as a catalyst. We also had more nuanced and better violence outcome measures, ranging from verbal threats to homicide. Second, the clinicians who were responsible for the treatment of the Safe pilot project patients managed the inclusion, the discharge decision, and choice of location. They participated in nine project meetings per year during the study period. This secured both the quality of design and measurement. There were no external researchers involved in the clinical sites, which strengthens the external validity of our findings. Third, repeated measurements of psychotic symptoms provided better internal validity concerning the association between symptoms and violence. Our design also allowed for the use of the nearest measurement point before the violent act occurred in our statistical analysis. Fourth, by using a repeated measures design, we avoided dealing with the effects of individual differences that can occur in case-control studies. Fifth, we have not found other research in the forensic mental health context that has reported results from measuring temporal proximity between psychotic symptoms and violence with a repeated measure design (e.g., Coid, 2020). Sixth, it is advantageous that we monitored the follow-up of discharged forensic patients in different sites and health care levels. Finally, our main findings were controlled for a series of highly relevant and accurately measured historical and present risk factors of violence. This made it easier to interpret the findings as true effects.

Limitations
First, repeated measures designs have some disadvantages compared to case-control designs. The most serious flaw is known as the order effect. In treatment research, it is related to the order that treatments are given but not due to the treatment itself. Since our study had an observational design, the order effect was not relevant. We also controlled confounding and bias by using a linear mixed model approach. Second, even if repeated measures require fewer participants than a case-control study, our sample had only 22 participants and, in particular, the subgroup analysis between violent and non-violent patients had very low statistical power. Yet, we found significant differences between the subgroups. Third, we wanted to use a naturalistic design especially concerning discharge decision. Hence, we did not randomize our sample. Even if it were possible, it would have taken too much time to reach a sufficient number of patients. Fourth, the use of criminal conviction as predictor of violence is a static risk factor with some degree of uncertainty. Due to the small sample we could not split it up into violent and non-violent criminal convictions. Fifth, the scope of our research precluded inclusion of other potentially important risk factors, and interactions between psychosis and other risk factors (such as between substance use and psychosis, Douglas et al., 2009;Monahan et al., 2001). Finally, the naturalistic design provided high external validity concerning discharge and follow-up from forensic facilities in Norway. Still, the generalizability to other countries with other health system may be limited.

Conclusion
This study had three research aims. First, we explored the association between three scales for psychotic symptoms and violence. The results were promising for all scales, particularly for adjusted effects without interaction. This offered support for use of these tools in risk assessment of persons with psychosis. Second, when the interaction term with criminal conviction combined with one scale at a time was entered, only the scale for hallucinations and TCO remained significant. Risk assessments of psychotic patients with criminal conviction may be informed of fluctuations in violence risk by observing and tracking psychotic symptoms on a repeated basis. Third, the repeated measurements design made possible the use of a small sample. Compared to big data research with focus on big cohort or population studies, our design allowed for a deeper investigation of the phenomenology of psychotic symptoms and their impact on concomitant fluctuations in violence risk.

Declaration of Competing Interest
This reseach and publication have no conflict of interest ever. The authors have no commercial or ethical conflict of interest that may have biased any results.

Role of the funding source
There has not been any funding for this project besides our wages as researchers