Risk of Violence by Psychiatric Patients: Beyond the "Actuarial Versus Clinical" Assessment Debate

Two statements about risk assessment that were usually regarded as true 30 years ago are not usually regarded as true today. The first is that psychiatric patients as a group are no more likely to engage in violent behavior than the general population ( 1 ). The second is that psychiatrists and psychologists are not able to predict which of their patients will act violently ( 2 ). The truth of the first statement is now seen as depending on, among other things, the group of patients being described ( 3 ). The second statement is now seen as untrue. A range of methods consistently predict violence at levels of accuracy better than chance ( 4 ).

These methods are usually characterized as actuarial, in which the prediction derives from a mathematical combining of known risk factors, or clinical ( 5 ). In violence risk research the adoption and adaptation of statistical approaches including the receiver operator characteristic (ROC) curve and the "number needed to treat" measure have improved researchers' ability to describe predictive accuracy. This article describes the levels of accuracy that can be achieved, examines the prospects for improvement, discusses the implications for clinical practice, and makes suggestions for future research.

The state of knowledge

Overall accuracy

The accuracy of validated actuarial and clinical approaches to the prediction of violence in populations with mental disorders improved between 1970 and 2000 ( Figure 1 ). The improvement was limited and characterized by a wide range in the published results. Actuarial methods outperformed clinical methods ( 4 ).

Before research can inform clinical practice, however, clinicians need to know not only that an improvement has occurred but also what level of accuracy that improvement now permits. Recent suggestions that actuarial approaches are inapplicable to the management of individual cases ( 6 ) have been challenged ( 7 , 8 ). Instead, actuarial methods and the instruments they employ are usually held to possess a predictive validity that can be measured using the same statistical techniques that are used elsewhere in medicine.

The area under the ROC curve (AUC) ranges from .5 (random) to 1.0 (perfect prediction) and is used in medicine and surgery to describe the accuracy of screening techniques ( 9 , 10 ). Its particular value as a measure of predictive accuracy derives from its being independent of the base rate and of the "cutting score," the point on a continuous scale at which a person is classified as dangerous in each validation study. Validation of the revised version of Hare's Psychopathy Checklist ( 11 ), the Violence Risk Assessment Guide (VRAG) ( 12 ), and the Historical, Clinical and Risk Management Scheme (HCR-20) ( 13 ) in the 1990s demonstrated AUCs of .72, .75, and .76 ( 14 , 15 , 16 ). This level of accuracy is similar to that achieved by criminal justice agencies in predicting sex-offender recidivism ( 17 ).

Since 2000 further validation of these instruments has generated AUCs ranging from .61 ( 18 ) to .82 ( 19 ). Their accuracy seems to generalize to nonforensic populations. The HCR-20 predicted violence in a general psychiatric sample with an AUC of .72 for men and .77 for women ( 20 ). It generalizes also to samples outside North America. A Danish validation of the VRAG reported an AUC of .73 ( 21 ). Newer instruments perform to a similar standard ( 22 ). A replication of the iterative classification tree, the algorithm on which Classification of Violence Risk software ( 23 ) operates, was published in 2005 and demonstrated AUCs of .63 and .70, depending on the outcome measure ( 24 ).

Calculations of the AUC most often assume an optimal cutoff score, something that can only be calculated after the outcome for each subject is known. Where the optimal score is not known, accuracy will be less ( 25 ). Translating figures for the AUC into numbers that are clinically meaningful is difficult for other reasons, too, not least because the clinical judgments that they might inform, such as whether to admit a patient to the hospital, derive from many considerations, not just one. One approach is to ask the following question: if a particular instrument was used as a screening test and those identified as likely to be violent were not discharged, over any given period how many patients would need to be detained to prevent one unwanted act?

This statistic, the number needed to detain, is the inverse of positive predictive value and analogous to the number needed to treat. It derives from sensitivity, specificity, and base rate ( 26 ). The AUC of .75, of the VRAG, was generated with a procedure that classified patients as violent or nonviolent with a sensitivity level of .73 and a specificity level of .63 ( 15 ). Used as a screening test where the base rate of violence is 10% and where, as a result, an unselective approach would lead to the detention of ten people in order to prevent one from acting violently, the VRAG would require the detention of five people to achieve the same end.

The number needed to detain rises as prevalence of violence falls ( Figure 2 ). One consequence is that the number of mistakes rises when unusual acts, such as acts of serious violence, are sought to be prevented. At the base rate recorded in the Epidemiologic Catchment Area study, where 17% of the sample self-reported violence in the previous 12 months, the number needed to detain to prevent an act of violence is 3.5. The outcome measure in the study was not conditional on violence that resulted in injury, however. When injury is required by the outcome measure, the base rate falls. In the CATIE (Clinical Antipsychotic Trials of Intervention Effectiveness) study, the six-month prevalence of assault with a weapon or causing serious injury was 3.6% ( 27 ). Here the number needed to detain at a sensitivity of .73 and a specificity of .63 is 15.

A second consequence of the relationship between number needed to detain and prevalence shown in Figure 2 is that at low base rates, prevalence becomes more important, and the psychometric qualities of a test less important, in determining the number of correct predictions. At the prevalence rates seen in most psychiatric outpatient settings, even a substantial improvement in an instrument's psychometric qualities may have limited effect. At the 3.6% base rate in the CATIE study, for instance, a 20% increase in sensitivity, all other things being equal, reduces the number needed to detain only to 13.

Relative accuracy in particular situations

Different risk factors apply and the same factors apply with different weights in different samples ( 28 ). Mental disorder seems to be more important, for instance, when base rates are low ( 29 ). Presumably for these reasons, the same instruments achieve different levels of accuracy in different populations ( 30 , 31 ). This has cost implications for actuarial approaches: the same techniques must be revalidated before their accuracies can be stated. By implication, although it is mentioned less often, equivalent concerns apply to clinical approaches. This heterogeneity also suggests, however, that in particular situations greater accuracy may be achievable.

For instance, are predictions more accurate when they cover only the near future? Reviews that control for base rates suggest that the accuracy of clinical and actuarial predictions is unaffected by the duration over which the prediction is made ( 32 ). Clinicians have expressed the opposite view, that predictions are more accurate when they are made over short periods ( 33 , 34 , 35 ). In theory, because clinicians talk to patients, they can have access to the reasons someone might act violently ( 5 ). Because much violence associated with a psychiatric disorder is not planned, however, it may be that any consequent improvement in the ability to predict violence is too short-lived to be demonstrated by the usual research methods ( 32 ).

Prediction may also be more feasible when patients have particular complaints. "Threat control override" symptoms, combining persecutory ideas and passivity experiences, have been linked to violent behavior in patient and community samples from the United States ( 36 , 37 ) and Israel ( 38 ). The association has not been demonstrated consistently, however, and if it does exist it seems to operate differently in different populations. At least for threat control override symptoms and violence occurring within the same 12-month period, the association seems to disappear when substance abuse is controlled for ( 37 ). It may disappear also when the client is in treatment ( 39 ). Studies of discharges from inpatient units have also shown no consistent association ( 40 , 41 ), although this may be due to a failure to allow for a differential effect among men and women ( 42 ).

A third possibility is that particular types of offending "breed true," offering the potential to reduce violence by identifying particular situations and potential victims. Criminological research demonstrates little of the "specialization" in offending that would allow this, however ( 43 , 44 , 45 ), and what there is may not help with prevention. Although sexual offenders are more likely than others to commit the same type of offense on successive occasions ( 46 ), a sexual offense against a child is still seven times more likely to be committed by a general offender than by someone with a history of sexual convictions ( 47 ). Fewer data are available for psychiatric samples. It may be that patients who have assaulted people before hospitalization are more likely to choose the same victims when they assault people after they are discharged ( 48 ). Those who offend after release from maximum-security hospitals, on the other hand, seem to display the same heterogeneity, between pre- and postdischarge offenses, evident in criminological research ( 49 ).

The potential for change in the current state of knowledge

Used as screening tests to detain the risky at the rates of violence usually encountered in psychiatric populations, therefore, current approaches can prevent the violent acts of a few only by detaining many. Combining the results of several instruments seems not to help ( 50 ). There are some reasons to expect, however, that accuracy will improve.

First, some of the validation studies that have been conducted, for instance on people who have already committed criminal acts, are victims of Gordon's partialing error ( 51 ). Gordon noted that the accuracy of any prediction is reduced when it is applied to a sample, such as a group of patients detained on the grounds of dangerousness, which is less heterogeneous, in terms of the independent variables linked to offending, than the general population. Where all of the patients have criminal records, for instance, criminal record is a poor predictor. Second, the distribution of predictive accuracy when plotted against variables that contribute to risk, such as gender, is not known. There are suggestions that clinicians are more accurate when predicting violence among men ( 52 ), and it may be that there are other patient groups for whom more accurate predictions are possible.

There are also, however, reasons to suspect that any future improvement in predictive accuracy will be limited. First, although any improvement shown in Figure 1 appears marginal, what seems unequivocally to have improved is methodology. Early reports used retrospective methods and, as independent variables, either release decisions ( 53 , 54 ) or variables obtained for other purposes ( 55 ). The outcome variable typically derived from a single source ( 56 ).

The MacArthur Violence Risk Assessment Study, by contrast, used a sample of 1,136 hospital discharges from multiple sites, 134 variables chosen for their relationship to violence, a prospective method, and an outcome measure derived from multiple interviews and collateral information ( 24 , 57 , 58 ). Most methodological flaws—the lack of a reliable measure of violent behavior, for instance—will reduce accuracy. The improvement in accuracy shown in Figure 1 may be a product of improved methodology. The important questions then become what further improvement is possible and whether the same results can be achieved in clinical settings.

Second, most models of prediction have as a premise that violence is stochastic, or random, only at the level of the population. That is to say, certain individuals within each discharge cohort will offend, and any limits to predictive accuracy derive from an inability to identify those individuals. It is possible, however, that violence is stochastic also at the level of the individual ( 51 ) and that if the tape could somehow be rerun, a different group of patients would offend. Even an intermediate position, where violence is stochastic at both levels, implies a ceiling on the accuracy that can be achieved by both clinical and actuarial approaches.

Potential ways forward

Psychiatric texts nevertheless continue to emphasize the contribution of violence risk assessment to clinical management ( 59 , 60 ). As a result, the question of how this should be done remains an important one for psychiatric research. One way forward would be to make a clearer distinction between two types of variables that correlate with patient violence.

Epidemiological and behavioral variables

Violent behavior is commoner, by a factor of 3, among psychiatric patients than among people who have never received psychiatric treatment ( 61 ). Men with schizophrenia are four times more likely than other men to be convicted of a crime of violence ( 62 ). Violence is commoner, by a factor of 6, in cross-sectional community surveys when the individual has a mental disorder ( 63 ). Using control groups from the same neighborhood reduces the effect of social drift and leads to lower odds ratios ( 64 ). Higher ratios are reported in countries where the crime rate is lower ( 65 , 66 , 67 ). These reports are forms of epidemiology. They demonstrate patterns of association between violence and mental disorder.

Odds ratios associated with past behavior are typically higher. Members of the 1945 Philadelphia Birth Cohort who were chronically delinquent as teenagers were 14 times more likely to be convicted four or more times as adults ( 68 ). English inner-city boys convicted as juveniles were nine times more likely than their unconvicted peers to be convicted again between the ages of 25 and 32 ( 69 , 70 ). An adult with a previous violent conviction is 14 times more likely than someone with a nonviolent conviction to commit a violent offense in the future ( 71 ). Substance abuse in community samples has been linked to violence at odds ratios of 20 for men and 32 for women ( 72 ).

The scale of the odds ratios associated with criminal behavior and drug use may explain why past violence alone, taken in conjunction with sociodemographic variables, is capable of predicting a majority of violent acts that can be predicted using more exhaustive methods ( 73 ). It seems likely that criminal history variables, or others acting as proxies for these, contribute much of the predictive power of actuarial instruments. The predictive accuracy of even the simplest behavioral measures seems to exceed the predictive power of diagnosis. Patients who are violent in the week before admission are nine times more likely to be violent in the two weeks after discharge ( 48 ).

These two types of variable, the one derived from the patient's diagnosis or overall clinical condition and the other derived from his or her behavior, may in the future be seen as serving different roles. Behaviorally derived variables have proved better able to distinguish those patients who will go on to act violently, albeit not yet at levels that can prevent high numbers needed to detain at the base rates of violence seen in clinical practice. Clinically derived variables, on the other hand, seem better placed to offer insights into the mechanisms by which a minority of people with mental disorders act violently.

Insights into the mechanism of violence

Risk management requires some understanding of the "causal nexus" ( 74 , 75 ) linking mental disorder to violence. Research has controlled for demographic characteristics, including age, sex, race, and marital status. But other variables linked to violence in the general population, such as an unstable family environment ( 76 ), unemployment ( 77 ), and television viewing ( 78 ), have received less attention. At least two of these, victimization ( 79 ) and community disorganization ( 80 ), correlate with mental disorder ( 81 ). When included in epidemiological analyses, they may go some way to explain previously observed associations. Threat control override symptoms, for instance, may be more strongly associated with violence in particular domestic settings.

It is also the case that potential confounders, such as substance abuse, can be present to a greater or lesser extent. An association between mental disorder and violence that persists after controlling for drug dependence does not exclude the possibility that a disproportionate number of individuals with mental disorders were using substances at a nondependent level. Violence is associated with substance use at levels that fail to meet diagnostic criteria ( 82 ). Residual confounding ( 83 ) of this kind may explain why Scandinavian studies that show the highest odds ratios for substance abuse also report high odds ratios with respect to mental disorder.

Further epidemiological research in these areas may assist clinicians in three ways. One is by offering a better understanding of the temporal relationship between psychotic symptoms and violence. The causal implications of symptoms occurring before and after a violent act are obviously different, but methodologies have not always distinguished the two ( 27 ). A second concerns the roles of delusions, command-type hallucinations, and flattening of affect ( 84 , 85 ) and their interaction with protective factors, such as treatment adherence ( 86 , 87 ). Of particular importance is the nature of the association between violence and substance use among persons with mental illness. It is still unclear whether the association stems from the effects of intoxication, from exacerbation of symptoms, or from participation in a social milieu where violence is common.

Practice-based research

The value to clinical management of behaviorally derived variables that have been linked to violence should be investigated by practice-based research. This could validate the use of actuarial instruments in clinical settings. Clinicians are more accurate when they base their judgments on test scores and not on the data that generate those scores ( 88 ), perhaps because tests are less likely than people to be overwhelmed by large amounts of information ( 89 ).

These validation studies should select the most relevant outcome variables for each area of clinical practice. If an instrument is to be used in civil commitment determinations, for instance, its predictive validity needs to be established not in relation to all violent acts but in relation to the kind of behavior occurring sufficiently close to the point of discharge that, had it been foreseen, would have justified continued hospitalization ( 90 ). Research should examine also what types of information are most useful. There is evidence, for instance, that clinicians' judgments have predictive power even when they are made without knowledge of the patient's history of violence ( 52 ).

Conclusions

The accuracy of clinical and actuarial predictions of violence among patients consistently exceeds chance, but the number of errors is related to the base rate of violence in the population. At the rates seen in outpatient settings, current approaches, if used to screen and detain within hospitals those predicted to be violent, would result in the detention of many people to prevent each violent act. In these circumstances predictions of violence have to remain only some of the many considerations guiding clinical judgment. The relation between prevalence of violence and the number needed to detain means that this state of affairs is unlikely to change even if the psychometric qualities of prediction methods show further substantial improvement.

This is different from saying that research in this area does not benefit clinical practice ( 91 , 92 ). Unless a risk of harm to others ceases to be a criterion for voluntary and compulsory admission to a hospital, that risk will continue to require to be assessed using the best methods available. There is a need to improve the quality of those assessments and to address the implications of what research has already shown—for instance, that the same approaches achieve different levels of accuracy in different circumstances.

Risk management that means more than detaining people requires the ability to anticipate dangerous situations and prevent the circumstances from recurring. By describing the correlates of violence and the roles of confounding variables, epidemiological methods offer the prospect of clarifying the causal nexus behind some violent acts. Some correlates of violence, such as drug use, seem to offer the opportunity to intervene regardless of the nature of the causal link ( 28 , 93 ). Practice-based research can test these opportunities and, by concentrating on behavioral variables, may be able to offer greater predictive accuracy to clinicians.

Practice-based research should investigate also how actuarial and clinical approaches can best be combined. It is likely that this will vary according to the circumstances in which a prediction is required. Actuarial schemes will never detect most instances where someone acts violently for the first time because the most powerful predictor in actuarial schemes is past behavior. The same actuarial approaches may nevertheless be all that is feasible when no clinical assessment has been conducted ( 94 ). To be widely adopted, any approach will have to take into account the limitations that apply to staff time and resources in most community psychiatric settings.

The question of whether, as a result of their improved accuracy, actuarial instruments now warrant more widespread clinical use is an empirical one. The authors of an equivalent tool for use by criminal justice agencies have recommended that the score it generates be used as a benchmark against which a clinical assessment and recommendation can be reviewed ( 95 ). In the present state of knowledge, a similar approach seems a reasonable response to clinical uncertainty in psychiatry. In the future, practice-based research may show that actuarial instruments should have a more extensive role and what that role might be.

Acknowledgments and disclosures

The author reports no competing interests.