Assessments by Patients With Schizophrenia and Psychiatrists of Relative Risk of Research Procedures

Schizophrenia is a severe and persistent mental illness present in nearly 1 percent of the world's population ( 1 ). Many people living with schizophrenia experience significant changes in their thoughts and feelings, cognitive abilities, everyday activities, capacity for fulfilling relationships, and ability to work ( 2 ). The consequences of this serious neuropsychiatric disorder extend far beyond people directly affected to include their families, their communities, and society as a whole ( 3 , 4 ). Currently available treatments offer substantial efficacy as well as the potential for adverse effects ( 3 , 5 ). For these reasons, the need to develop and test new treatments for schizophrenia—primarily through clinical trials—is increasingly recognized ( 6 , 7 ).

The conduct of clinical research involving people with schizophrenia poses many ethical challenges, however ( 8 , 9 , 10 , 11 , 12 ). As thousands of volunteers enter privately and publicly funded clinical trials, and as the job of institutional review boards (IRBs) has become more complex, attention to these ethical issues remains paramount in the national dialogue on human research ethics ( 13 ). A critical ethical concern in schizophrenia research specifically pertains to the way in which people with serious mental illness understand the risks associated with research participation ( 8 , 9 , 10 , 11 , 12 , 14 , 15 ). This is important for two reasons. First, to be ethically acceptable, foreseeable risks associated with certain protocol procedures must be justified by potential benefits, and prospective volunteers must understand and appreciate these risks if the safeguard of informed consent is to be meaningful ( 16 , 17 ). Second, there is an obligation to recognize the concerns and perspectives of those involved in and affected by research—namely, the participants themselves as well as their families and communities ( 13 ). These commitments—to appropriate scientific procedures with acceptable risk-to-benefit ratios, to informed consent, and to valuing participant perspectives—are an expression of the fundamental ethical principle of respect for people, which strives to ensure that the rights and dignity of research participants are regarded fully ( 18 ).

In designing clinical research studies ( 19 ), psychiatric investigators thus must seek to minimize the potential risks to volunteers. In reviewing and approving clinical research studies, IRBs must prospectively evaluate the risk-to-benefit ratio associated with a study's procedures and the adequacy of efforts by investigators to protect volunteers from unnecessary harm ( 20 ). Moreover, IRBs typically evaluate risk relative to everyday risks faced by the relevant study population. This required implicit risk calculus has important implications, because protocols deemed "minimal risk" (that is, those in which participants do not face risks above what they encounter in everyday life or in routine clinical care) may be subject to quicker review and less intense scrutiny. Finally, participants themselves, as part of the recruitment and informed consent process, must make judgments regarding the value of participation in light of their understanding, their personal preferences, and their willingness to risk potential harm ( 21 , 22 ).

Despite this complicated, multidimensional process that occurs before any participant enrolls, little is known about how stakeholders in schizophrenia research (for example, participants and psychiatrists) actually evaluate risks associated with various clinical research procedures ( 23 , 24 , 25 ). Moreover, controversy exists about how best to benchmark risk assessments—for example, in relation to healthy versus ill individuals ( 26 ). Greater understanding of these issues is valuable, especially for certain kinds of studies that have engendered serious concerns and controversy in psychiatric research, such as protocols that involve genetic testing, medication washouts, or symptom provocation ( 23 , 27 , 28 , 29 , 30 , 31 ).

To address this gap, we examined perspectives of people with schizophrenia and psychiatrists regarding risks of different research procedures. On the basis of prior literature examining patients' versus physicians' views of ethically important features of research ( 8 ) and on the basis of literature suggesting that patients overestimate the risks associated with certain medical or surgical procedures ( 32 ), we hypothesized that compared with psychiatrists, individuals with schizophrenia would rate the described procedures as posing greater relative risk.

Methods

For this IRB-approved, preliminary study funded by the National Institute of Mental Health, we developed a questionnaire for people with schizophrenia with 298 quantitative scaled questions and six qualitative items. The survey assessed views of ethically important considerations in mental illness research and related areas—for example, research procedure risks, protocol safeguards, and attitudes toward research participation. A total of 43 people with schizophrenia were recruited by community outreach or physician referral. Diagnosis was confirmed by chart review. Data were collected from this group in 2001 and 2002.

After obtaining and documenting informed consent, a trained interviewer administered the survey by reading each question and recording responses. Most participants completed the survey in 2.5 to 3.0 hours, which sometimes occurred over more than one session to minimize fatigue. For the portion of the study presented in this report, participants were asked to compare the risk of the procedure with the risks in their usual daily lives.

A version of the questionnaire with fewer questions and less detail was developed for psychiatrists. In the survey given to participants with schizophrenia, for instance, we gave procedure descriptions (for example, for "having a spinal tap," we described the procedure as "putting a needle in the back, between the bones of the spine, and removing a tube of fluid"); this level of specificity was not seen as necessary for the survey given to psychiatrists. All 105 faculty and resident psychiatrists at the University of New Mexico School of Medicine were invited to participate. Sixty-eight responded, giving a response rate of 65 percent. Data were collected from this group in 2003. Psychiatrists were asked to compare the risks of the procedures with the risks faced daily by people with schizophrenia. Psychiatrists and patients were not matched, but some psychiatrists had patients who participated in the survey.

Data were confidentially encoded. After complete description of the study to the participants with schizophrenia, documentation of informed consent was obtained. All participants with schizophrenia received $30 as compensation for time and effort. Clinicians received no compensation for their participation.

Five-point scaled rating responses were subjected to a repeated measures procedure (within subjects) × rater (patient versus psychiatrist) (between subjects) × gender (between subjects) multivariate analysis of variance (MANOVA). This design allowed us to examine the degree to which different protocols were seen to have different risks and the degree to which patients and psychiatrists agreed or disagreed on the degree of risk of individual protocols.

Results

Participant characteristics

Participants with schizophrenia were mostly men (79 percent). Participants were mostly single (88 percent) and white (58 percent) and had a mean age of 44 years ( Table 1 ). One in five (21 percent) were Hispanic, 14 percent were black, and 9 percent were Native American. Participants who were psychiatrists were mostly men (54 percent). Most were married (72 percent) and white (74 percent), and the sample had a mean age of 42 years. One in five (19 percent) were Hispanic, and 13 percent were Asian. Forty-one percent of the psychiatrists were residents (postgraduate year 1 through 5), and 59 percent were attending physicians; 13 residents (46 percent) and 26 attending physicians (65 percent) were involved in research. Ratings of psychiatrists did not differ by resident or attending physician status or by whether they were involved in research.

Enlarge table

Procedure risks

The 12 procedures were rated on a 5-point scale. Possible scores range from 1, much less risk, to 3, same risk, to 5, much more risk, relative to the usual risks that people with schizophrenia live with every day—that is, relative risk. Ratings were subjected to a repeated-measures procedure (12 procedures) (within subjects) × rater (patient versus psychiatrist) (between subjects) × gender of rater (between subjects) MANOVA. The procedure main effect showed that all participants perceived the 12 procedures to have widely ranging relative risks (F=26.18, df=11 and 97, p<.001; maximum between-procedure effect size, Cohen's d=1.70). However, the overall means of risks of procedures combine patient and psychiatrist views and thus are not relevant for the purposes of this study and are not described here.

Patients' versus psychiatrists' views of procedure risks

As shown in Table 2 , the procedure × rater interaction revealed that the pattern of risks as rated by people with schizophrenia was different from that rated by psychiatrists (F=4.30, df=11 and 97, p<.001). Although no differences in mean ratings for individual procedures were detected when we compared ratings of patients to those of psychiatrists, four procedures showed effect size differences between the groups that were of moderate magnitude, although these results were not significant (all p<.20). Specifically, patients saw drawing a tube of blood for a genetic test as being less risky than usual daily risks; psychiatrists also saw it as being less risky than usual daily risks, although they tended to see it as being more risky than patients did (d=-.44). And patients tended to see having a spinal tap; showing a word or picture that is upsetting; and giving a medication that causes temporary symptoms of schizophrenia for a few hours, such as hearing voices, as being more risky than psychiatrists did (d values of .53, .48, and .42, respectively).

Enlarge table

The analyses of simple effects of procedures for patients and psychiatrists separately showed that people with schizophrenia viewed filling out questionnaires, drawing a tube of blood, drawing a tube of blood for a genetic test, and having X-rays of the head as being significantly less risky than usual daily risks (all p<.05; comparing mean ratings to the scale midpoint of 3, same risk). However, psychiatrists saw showing a word or picture that is upsetting as having less than normal risks, whereas people with schizophrenia saw it as equivalent to usual risks. In contrast, patients saw having a spinal tap, giving a new experimental medication, giving a medication that causes temporary symptoms of schizophrenia for a few hours (such as hearing voices), and stopping usual medications for two weeks as having significantly greater risk than usual (all p<.05); however, psychiatrists rated only stopping usual medications for two weeks as having greater than usual risk (p<.05). None of the research procedures were seen by either group as having "much more risk" (that is, ratings did not reach 4 on the 5-point scale) than what people with schizophrenia experience in their everyday lives.

Comparisons among conceptually related procedures

We performed secondary analyses to study relationships among conceptually related subsets of the 12 procedures with four separate post hoc MANOVAs. A procedure × rater MANOVA on the two psychiatric evaluative procedures (X-rays of the head and a spinal tap) revealed a procedure × rater interaction (F= 7.42, df=1 and 107, p<.01), which showed that people with schizophrenia and psychiatrists both saw a spinal tap as being significantly more risky than a head X-ray and that people with schizophrenia saw a spinal tap as being much more risky than did psychiatrists (d=.54).

In a second MANOVA contrasting two other kinds of evaluative procedures (drawing a tube of blood and drawing a tube of blood for a genetic test), a procedure × rater interaction was again detected (F=9.70, df=1 and 107, p<.01), revealing that people with schizophrenia did not see a blood draw for a genetic test as being more risky than a simple blood draw (d=-.02); however, psychiatrists saw blood draws for genetic tests as riskier than simple blood draws (d=.42). That is, drawing a tube of blood was seen as having the same level of risk by both patients and psychiatrists, but psychiatrists saw more risk for the genetic test than did patients.

A procedure × rater MANOVA on the two intervention medication procedures (stopping usual medications for two days and stopping usual medications for two weeks) found a procedure main effect (F=23.48, df=1 and 107, p<.001) but no rater effect or interaction. Both people with schizophrenia and psychiatrists saw the longer washout as more risky than the shorter one (d=.32).

A final MANOVA on the two provocation interventions (showing a word or picture that is upsetting and giving a medication that causes temporary symptoms of schizophrenia) revealed a procedure main effect (F=28.21, df=1 and 107, p<.001), indicating that both psychiatrists and people with schizophrenia saw symptom induction as being more risky than showing an upsetting word or picture (d=.57), and a rater main effect (F=7.11, df=1 and 107, p<.01), indicating that people with schizophrenia viewed both interventions as riskier than did psychiatrists (d=.54).

Discussion

Exposing volunteers to research-related risk is arguably the most serious ethical concern posed in human studies. Authentic, voluntary acceptance of appropriate risks in the quest for valuable new scientific knowledge is the remarkable gift that study volunteers offer, sometimes at significant personal sacrifice, to our society. Ethical inclusion of these study volunteers is predicated in part on their being well informed and their possessing a true understanding of the nature of risks that they may undertake in the context of research protocols. It is for this reason that evidence regarding the perceptions of risk expressed by candidates for biomedical research participation is so very important.

This is the first empirical project to our knowledge that has focused attention on a key conceptual and practical issue in the federal regulatory guidelines separating "minimal risk" from other, riskier forms of research involving human volunteers. We asked two stakeholder groups, people living with schizophrenia and psychiatrists, to assess risk associated with 12 research procedures relative to everyday risks experienced by people with schizophrenia. To help ensure the relevance of this work, we selected procedures that are vital to the conduct of biomedically oriented schizophrenia research trials, present and future ( 28 , 33 ). We highlight our major findings below.

First, patients with schizophrenia and psychiatrists both viewed the 12 procedures as having varying risk levels. Four of the 12 procedures (psychotic symptom induction, medication discontinuation for two weeks, giving a new experimental medication, and spinal tap) were rated by both patients with schizophrenia and psychiatrists as posing risks greater than those usually encountered in the daily lives of people with schizophrenia. This set of findings suggests that greater care should be used when designing, evaluating, referring patients to, conducting, and overseeing protocols involving these procedures.

Second, our results indicate that all 12 of the research procedures were seen by people with schizophrenia and psychiatrists alike as not differing greatly from the usual risks of living with schizophrenia. These results are important because—if these assessments are used as a benchmark—the procedures thus may properly fall under "minimal risk" standards used in an IRB review. Nevertheless, there remains significant controversy over whose daily life should in fact serve as the point of comparison for research procedures ( 26 , 34 ). Interestingly, in our entire study, no participant used the "much more risk" rating (that is, a score of 5 on a 5-point scale) for any of the procedures that we assessed. Further, none of the mean ratings provided by schizophrenia participants and by psychiatrists reached even a score of 4 on the rating scale. This observation suggests that our participants did not see any of the 12 procedures as posing especially high risk—or even much above usual risk. We hasten to add that these findings do not speak to the issue of the objective risk level of such procedures. Instead, the data reflect the subjective perception of risk as expressed by individuals who derive from two different and unique stakeholder groups—that is, one with relevant clinical expertise and one with personal illness experience.

Third, the relative risk ratings between patients and psychiatrists showed substantial congruence on a majority of the items. Patients and psychiatrists agreed, for example, that four of the research procedures posed less than usual daily risk and that another was greater than usual risk. Comparison of mean ratings on individual items by the respondent groups, moreover, did not reveal significant differences. In another illustration, in post hoc analyses comparing responses to the two medication-washout items (that is, washout for two days versus two weeks), we found that patients with schizophrenia and psychiatrists alike found the longer medication washout period to be relatively riskier than the shorter washout period. This striking level of agreement between patients' and psychiatrists' (that is, personal and expert) risk ratings replicates prior work ( 25 ). Although open to interpretation, our data suggest that, with appropriate information and supportive efforts, many people with schizophrenia may be well able to understand the risks associated with procedures frequently encountered in research participation, despite concerns to the contrary ( 14 , 15 , 35 , 36 , 37 , 38 ).

Fourth, it is of interest that risk ratings diverged between the patients with schizophrenia and psychiatrists for four of the 12 procedures. Patients saw three procedures as having greater than usual risk (whereas psychiatrists did not), and psychiatrists saw one procedure as having lower than usual risk (whereas patients with schizophrenia thought it posed "usual" risk). The differences in the mean values were not large, however. More specifically, people with schizophrenia saw the conceptually related comparisons involving spinal tap, viewing an upsetting word or picture, and psychotic symptom induction in the same direction as psychiatrists did, but they viewed these procedures as slightly riskier than did the psychiatrists. The disparate views on the item pertaining to psychotic symptom emergence are consistent with a previous study in which we found that psychiatrists underestimated the degree of risk that people with schizophrenia assigned to a hypothetical protocol in which a volunteer developed severe symptoms during a medication-washout phase ( 39 ).

Finally, our results are the first, to our knowledge, to present data regarding the views of patients with schizophrenia toward genetic testing, a field with great promise and growing importance in psychiatry ( 28 , 40 , 41 , 42 ). People with schizophrenia in our study appeared to miss the greater psychosocial risks associated with "drawing a tube of blood for a genetic test" as opposed to "drawing a tube of blood." Psychiatrists did, however, view the former as a riskier procedure. Importantly, lay individuals without schizophrenia might also not appreciate the additional risks associated with genetic studies. Existing literature raises concerns about the adequacy of understanding of genetic susceptibility testing in other clinical populations ( 43 ) and about whether and how testing for genetic risks should be treated differently from other medical tests ( 44 , 45 , 46 , 47 ). Given the complex genetics of mental disorders, the future possibility of presymptomatic risk determination, the continuing stigma of mental illness, and the potential ramifications of genetic risk information (for example, psychological distress, effects on health or life insurance coverage, the decision to inform one's family, and potentially for childbearing decisions) ( 48 ), the academic community ideally will seek input from stakeholders early in the development of psychiatric genetics research. Furthermore, evidence-based ethics projects are needed to help clarify how best to inform potential study volunteers about their rights, the limitations on their privacy in such studies, and related issues ( 28 , 49 ).

Several limitations of our study should be noted. We did not evaluate whether the participants in our study had undergone the procedures that we asked about. In addition, we only briefly described each research procedure, rather than providing a full research protocol to our participants. We also were unable to assess the contributions of a variety of respondent characteristics (for example, personal risk tolerance and history of research experience) to ratings of relative risk. It is possible that the volunteers in this study were more willing to participate in research (as shown, perhaps, by their enrollment in our study) than people with schizophrenia in general. Another limitation of this study is the relatively small sample of both psychiatrists and patients from a single geographic area.

For these reasons, our project should serve to stimulate further inquiry into this complex area. Our work addresses the previously unstudied question of how to evaluate the risks associated with research procedures, a task routinely performed by IRB members but without the benefit of evidence to help guide their decisions. Rigorous research is urgently needed about how studies are designed, vetted, approved (that is, by IRBs or other regulatory authorities), conducted, and monitored, so that we can meaningfully protect the rights and welfare of the participants ( 13 ). Future research may also determine the generalizability of our results and hence the potential usefulness of our data to investigators and protocol reviewers. Data are needed to explore how healthy people and people with illnesses other than schizophrenia view research risks in relation to the ordinary risks of their lives. It is entirely possible that other stakeholders will assess research procedure risks differently. Future research may also fruitfully explore the views of alternative decision makers (for example, legally authorized representatives) in assessing protocol risks. Finally, new studies are needed to determine optimal ways of helping health professionals to address the ethically important aspects of their work with patients and participants, a topic area of great interest and concern ( 50 , 51 , 52 , 53 , 54 , 55 ).

Conclusions

Our study is unique in that we asked two key stakeholder groups—one with clinical expertise and one with personal illness experience—to rate the risks of diverse evaluative and intervention research procedures relative to everyday risks faced by people with schizophrenia—that is, one potential benchmark for defining what is a "minimal risk" study. We compared the perspectives of ill individuals, who represent potential research participants, to those of psychiatrists, who may care for them clinically, refer them to research studies, conduct research, or serve as IRB members evaluating risk.

The concordance in viewpoints offered by the two stakeholder groups is encouraging. Nevertheless, it is clear from our findings that there may be some areas in which psychiatrists may not fully appreciate the degree of potential harm seen by ill individuals who may be candidates for research participation. Moreover, there may be areas where ill individuals assess risks as being greater than what psychiatrists view as the actual potential for harm. Such gaps are important in efforts to ensure understanding and appreciation of risks as one component of the informed consent process ( 8 , 12 , 13 , 14 , 15 , 22 , 25 , 35 , 36 , 38 , 39 , 43 , 48 ).

Beyond instigating additional inquiry, this work on stakeholder risk assessment represents an invitation to develop and provide more educational materials for people with schizophrenia regarding procedures in research, as well as in clinical practice—particularly given recent evidence that people with schizophrenia wish to engage in a shared decision-making model of care ( 56 ). On the other hand, it implores psychiatrists to listen and remain attuned to the perspectives of potential participants who so generously give of themselves in the pursuit of valuable new knowledge to help understand serious mental illness.

Acknowledgments

This study was supported by grant 1-R01-DA-13139 from the National Institute of Drug Abuse. The authors also acknowledge the National Institute for Mental Health for support in the form of a Career Scientist Development Award (1-K02-MH-01918) for Dr. Roberts and a Mentored Career Development Award (K23-MH-66062) for Dr. Dunn.