Press Releases by Academic Medical Centers: Not So Academic?

Context News reports often exaggerate the importance of medical research. Contribution The researchers reviewed press releases issued by academic medical centers. They found that many press releases overstated the importance of study findings while underemphasizing cautions that limited the findings' clinical relevance. Caution The researchers did not attempt to see how the press releases influenced actual news stories. Implication Academic center press releases often promote research with uncertain clinical relevance without emphasizing important cautions or limitations. The Editors Medical journalism is often criticized for what reporters cover (for example, preliminary work) and how they cover it (for example, turning modest findings into miracles) (14). Critics often place blame squarely on the media, pointing out that few journalists are trained to critically read medical research or suggesting that sensationalism is deliberate: Whereas scientists want to promote the truth, the media just want to sell newspapers. But exaggeration may begin with the journalists' sources. Researchers and their funders, and even medical journals, often court media attention through press releases. The strategy works: Press releases increase the chance of getting media coverage (5, 6) and shape subsequent reporting (7). An independent medical news rating organization found that more than one third of U.S. health news stories seemed to rely solely or largely on press releases (1). Academic medical centers produce large volumes of research and attract press coverage through press releases. Because these centers set the standard for research and education in U.S. medicine, one might assume that their press releases are measured and unexaggerated. To test this assumption, we examined press releases from academic medical centers in a systematic manner. Methods We selected the 10 highest-ranked and 10 lowest-ranked of the academic medical centers covered in U.S. News & World Report's medical school research rankings (8) that issued at least 10 releases in 2005. In addition, we identified each medical school's affiliates by using an Association of American Medical Colleges database. The Appendix Table lists the centers and their affiliated press offices. We initially intended to compare press releases by research ranking, but because we found few differences, we report findings across the entire study sample, highlighting the few differences by rank where they exist. Appendix Table. Highest- and Lower-Ranked Medical Schools (and Their Affiliated Press Offices) for Research That Issued at Least 10 Press Releases in 2005 Press Release Process During 2006, a former medical school press officer conducted semistructured (15-minute) telephone interviews with the person in charge of media relations at the 20 centers. The interview script (Appendix) covered release policy (how is research chosen?), production (writing, review, researcher's role), and an overall assessment (perceived pressure for media results, praise, or backlash). Supplement. Appendix Press Release Content We searched Eureka Alert (a press release database) for all medical and health releases issued by the 20 centers and their affiliates in 2005. The Figure summarizes the search results. Figure. Study flow diagram. *Of the medical schools that issued at least 10 press releases in 2005. Science Promoted After excluding duplicate or nonresearch releases (such as those announcing grants), we determined study focus (animal or human) and publication status; if the study was published, we characterized the journal's academic prominence by using the Thompson Scientific Journal Citation Reports impact factor. Content Analysis We randomly selected 200 press releases (10 per center) and assessed presentation of study facts, cautions, and presence of exaggeration by using separate coding schemes for human and nonhuman studies (the Appendix includes both schemes). The schemes included 32 unique items (10 for human studies only, 4 for nonhuman studies only, and 18 common to both). Sixteen items involved simply extracting facts from the release (for example, study size); the other 16 items required subjective judgments (for example, were there cautions about confounding for observational studies?). To confirm key study details (such as population, design, and size), we obtained the research reports (journal article or meeting abstract) referenced in the releases. Coding Reliability and Analysis Two research assistants who were blinded to the study's purpose independently coded releases. To measure reliability, the coders and investigators reviewed each code's definition and then reread the release to confirm (or change) their code. Errors due to definition or data entry problems were corrected before agreement was calculated. Intercoder agreement was nearly perfect (9) for both sets of items: for factual items, was 1.0 (range, 0.98 to 1.0), and for subjective items, was 0.97 (range, 0.79 to 1.0). Disagreements were resolved by 4 of the investigators. We used STATA, version 10 (StataCorp, College Station, Texas) for all analyses. Role of the Funding Source The project was funded by the National Cancer Institute and the Robert Wood Johnson Generalist Faculty Scholars Program. Neither source had any role in study design, conduct, or analysis or in the decision to seek publication. Results Press Release Process All centers said that investigators routinely request press releases and are regularly involved in editing and approving them (Table 1). Only 2 centers routinely involve independent reviewers. On average, centers employed 5 press release writers (the highest-ranked centers had more writers than lower-ranked centers [mean, 6.6 vs. 3.7]). Three centers said that they trained writers in research methods and results presentation, but most expected writers to already have these skills and hone them on the job. All 20 centers said that media coverage is an important measure of their success, and most report the number of media hits garnered to the administration. Table 1. Press Release Process and Press Releases Issued by the 20 Academic Medical Centers Press Releases Issued Table 1 shows that the centers issued 989 medical research-related releases in 2005. The centers averaged 49 releases per year; the range was 13 (Brown Medical School) to 186 (Johns Hopkins University School of Medicine). Twelve percent of the releases promoted unpublished research from scientific meetings. Higher-ranked centers issued more releases than lower-ranked centers (743 vs. 246) and were less likely to promote unpublished research (9% vs. 20%). Press Release Quality Table 2 summarizes the measures of press release quality. Table 2. Type of Research Promoted in and Quality of the 200 Press Releases Analyzed in Detail Study Details and Cautions Of the 95 releases about primary human research (excluding unstructured reviews and decision models), 77% provided study size and most (66%) quantified the main finding in some way; 47% used at least 1 absolute number, the most transparent way to represent results (10, 11). Few releases (12%) provided access to the full scientific report. Two thirds of the 200 randomly selected releases reported study funding sources; 4% noted conflicts of interest (either that none [3 releases] or some existed [4 releases]). Of all 113 releases about human studies, 17% promoted published studies with the strongest designs (randomized trials or meta-analyses). Forty percent reported on inherently limited studies (for example, sample size <30, uncontrolled interventions, primary surrogate outcomes, or unpublished meeting reports). Fewer than half (42%) provided any relevant caveats. For example, a release titled Lung-sparing treatment for cancer proving effective (which concluded that treatment was a safe and effective way to treat early stage lung cancer in medically inoperable patients) lacked cautions about this uncontrolled study of 70 patients. Among the 87 releases about animal or laboratory studies, most (64 of 87) explicitly claimed relevance to human health, yet 90% lacked caveats about extrapolating results to people. For example, a release about a study of ultrasonography reducing tumors in mice, titled Researchers study the use of ultrasound for treatment of cancer, claimed (without caveats) that in the future, treatments with ultrasound either alone or with chemotherapeutic and antivascular agents could be used to treat cancers. Exaggeration Twenty-nine percent of releases (58 of 200) were rated as exaggerating the finding's importance. Exaggeration was found more often in releases about animal studies than human studies (41% vs. 18%). Almost all releases (195 of 200) included investigator quotes, 26% of which were judged to overstate research importance. For example, a release for a study of mice with skin cancer, titled Scientists inhibit cancer gene. Potential therapy for up to 30 percent of human tumors, quoted the investigator as saying that the implication is that a drug therapy could be developed to reduce tumors caused by Ras without significant side effects. Coders thought that the implication exaggerated the study findings, because neither treatment efficacy nor tolerability in humans was assessed. Although 24% (47 of 200) of releases used the word significant, only 1 clearly distinguished statistical from clincial significance. All other cases were ambiguous, creating an opportunity for overinterpretation: for example, Not-for-profit hospitals consistently had significantly higher scores than for-profit hospitals. Discussion Press releases issued by 20 academic medical centers frequently promoted preliminary research or inherently limited human studies without providing basic details or cautions needed to judge the meaning, relevance, or validity of the science. Our findings are consistent with those of other analyses of pharmaceutical industry (12) and medical jo

M edical journalism is often criticized for what reporters cover (for example, preliminary work) and how they cover it (for example, turning modest findings into miracles) (1)(2)(3)(4). Critics often place blame squarely on the media, pointing out that few journalists are trained to critically read medical research or suggesting that sensationalism is deliberate: Whereas scientists want to promote the truth, the media just want to sell newspapers.
But exaggeration may begin with the journalists' sources. Researchers and their funders, and even medical journals, often court media attention through press releases. The strategy works: Press releases increase the chance of getting media coverage (5,6) and shape subsequent reporting (7). An independent medical news rating organization found that more than one third of U.S. health news stories seemed to rely solely or largely on press releases (1).
Academic medical centers produce large volumes of research and attract press coverage through press releases. Because these centers set the standard for research and education in U.S. medicine, one might assume that their press releases are measured and unexaggerated. To test this assumption, we examined press releases from academic medical centers in a systematic manner.

METHODS
We selected the 10 highest-ranked and 10 lowestranked of the academic medical centers covered in U.S. News & World Report's medical school research rankings (8) that issued at least 10 releases in 2005. In addition, we identified each medical school's affiliates by using an Association of American Medical Colleges database. The Appendix Table (available at www.annals.org) lists the centers and their affiliated press offices. We initially intended to compare press releases by research ranking, but because we found few differences, we report findings across the entire study sample, highlighting the few differences by rank where they exist.

Press Release Process
During 2006, a former medical school press officer conducted semistructured (15-minute) telephone interviews with "the person in charge" of media relations at the 20 centers. The interview script (Appendix, available at www.annals.org) covered release policy (how is research chosen?), production (writing, review, researcher's role), and an overall assessment (perceived pressure for media results, praise, or backlash).

Press Release Content
We searched Eureka Alert (a press release database) for all "medical and health" releases issued by the 20 centers and their affiliates in 2005. The Figure summarizes the search results.

Science Promoted
After excluding duplicate or nonresearch releases (such as those announcing grants), we determined study focus (animal or human) and publication status; if the study was published, we characterized the journal's academic prominence by using the Thompson Scientific Journal Citation Reports "impact factor."

Content Analysis
We randomly selected 200 press releases (10 per center) and assessed presentation of study facts, cautions, and presence of exaggeration by using separate coding schemes for human and nonhuman studies (the Appendix, available at www.annals.org, includes both schemes). The schemes included 32 unique items (10 for human studies only, 4 for nonhuman studies only, and 18 common to both). Sixteen items involved simply extracting facts from the release (for example, study size); the other 16 items required subjective judgments (for example, were there cautions about confounding for observational studies?). To confirm key study details (such as population, design, and size), we obtained the research reports (journal article or meeting abstract) referenced in the releases.

Coding Reliability and Analysis
Two research assistants who were blinded to the study's purpose independently coded releases. To measure reliability, the coders and investigators reviewed each code's definition and then reread the release to confirm (or change) their code. Errors due to definition or data entry problems were corrected before agreement was calculated. Intercoder agreement was "nearly perfect" (9) for both sets of items: for factual items, was 1.0 (range, 0.98 to 1.0), and for subjective items, was 0.97 (range, 0.79 to 1.0). Disagreements were resolved by 4 of the investigators. We used STATA, version 10 (StataCorp, College Station, Texas) for all analyses.

Role of the Funding Source
The project was funded by the National Cancer Institute and the Robert Wood Johnson Generalist Faculty Scholars Program. Neither source had any role in study design, conduct, or analysis or in the decision to seek publication.

Press Release Process
All centers said that investigators routinely request press releases and are regularly involved in editing and approving them ( Table 1). Only 2 centers routinely involve independent reviewers. On average, centers employed 5 press release writers (the highest-ranked centers had more writers than lower-ranked centers [mean, 6.6 vs. 3.7]). Three centers said that they trained writers in research methods and results presentation, but most expected writers to already have these skills and hone them on the job. All 20 centers said that media coverage is an important measure of their success, and most report the number of "media hits" garnered to the administration. Table 1 shows that the centers issued 989 medical research-related releases in 2005. The centers averaged 49 releases per year; the range was 13 (Brown Medical School) to 186 (Johns Hopkins University School of Medicine). Twelve percent of the releases promoted unpublished research from scientific meetings. Higher-ranked centers issued more releases than lower-ranked centers (743 vs. 246) and were less likely to promote unpublished research (9% vs. 20%). Table 2 summarizes the measures of press release quality.

Study Details and Cautions
Of the 95 releases about primary human research (excluding unstructured reviews and decision models), 77% provided study size and most (66%) quantified the main finding in some way; 47% used at least 1 absolute number, the most transparent way to represent results (10,11). Few releases (12%) provided access to the full scientific report.
Two thirds of the 200 randomly selected releases reported study funding sources; 4% noted conflicts of interest (either that none [3 releases]

or some existed [4 releases]).
Of all 113 releases about human studies, 17% promoted published studies with the strongest designs (randomized trials or meta-analyses). Forty percent reported on inherently limited studies (for example, sample size Ͻ30, uncontrolled interventions, primary surrogate outcomes, or Context News reports often exaggerate the importance of medical research.

Contribution
The researchers reviewed press releases issued by academic medical centers. They found that many press releases overstated the importance of study findings while underemphasizing cautions that limited the findings' clinical relevance.

Caution
The researchers did not attempt to see how the press releases influenced actual news stories.

Implication
Academic center press releases often promote research with uncertain clinical relevance without emphasizing important cautions or limitations.

-The Editors
Article Press Releases by Academic Medical Centers unpublished meeting reports). Fewer than half (42%) provided any relevant caveats. For example, a release titled "Lung-sparing treatment for cancer proving effective" (which concluded that treatment was "a safe and effective way to treat early stage lung cancer in medically inoperable patients") lacked cautions about this uncontrolled study of 70 patients.
Among the 87 releases about animal or laboratory studies, most (64 of 87) explicitly claimed relevance to human health, yet 90% lacked caveats about extrapolating results to people. For example, a release about a study of ultrasonography reducing tumors in mice, titled "Researchers study the use of ultrasound for treatment of cancer," claimed (without caveats) that "in the future, treatments with ultrasound either alone or with chemotherapeutic and antivascular agents could be used to treat cancers."

Exaggeration
Twenty-nine percent of releases (58 of 200) were rated as exaggerating the finding's importance. Exaggeration was found more often in releases about animal studies than human studies (41% vs. 18%).
Almost all releases (195 of 200) included investigator quotes, 26% of which were judged to overstate research importance. For example, a release for a study of mice with skin cancer, titled "Scientists inhibit cancer gene. Potential therapy for up to 30 percent of human tumors," quoted the investigator as saying that "the implication is that a drug therapy could be developed to reduce tumors caused by Ras without significant side effects." Coders thought that the "implication" exaggerated the study findings, because neither treatment efficacy nor tolerability in humans was assessed. Although 24% (47 of 200) of releases used the word "significant," only 1 clearly distinguished statistical from clincial significance. All other cases were ambiguous, creating an opportunity for overinterpretation: for example, "Not-for-profit hospitals consistently had significantly higher scores than for-profit hospitals."

DISCUSSION
Press releases issued by 20 academic medical centers frequently promoted preliminary research or inherently limited human studies without providing basic details or cautions needed to judge the meaning, relevance, or validity of the science. Our findings are consistent with those of other analyses of pharmaceutical industry (12) and medical journal (13) press releases, which also revealed a tendency to overstate the importance and downplay (or ignore) the limitations of research.
Our study has several limitations. First, content analysis coding always involves subjectivity. The high level of agreement observed among coders, however, is reassuring. Second, our findings are based on 20 centers, which may raise concern about generalizability. Because only the top 52 (of 125) centers receive a U.S. News & World Report ranking, we believe that our findings represent a best-case scenario. Third, our coding scheme was not exhaustive. We focused on study details and cautions we thought were Releases with the word "significant" 47 Clarified that "significant" refers to "statistical significance" 1 (2) * 1 randomized trial was unpublished. † Individual press releases can appear in more than 1 category. ‡ Includes only primary reports of human studies (e.g., excludes unstructured reviews, decision models). § Either a statement of conflict of interest or a statement that there were no conflicts of interest. * Established from interviews with academic center press offices and description of all press releases issued. † Data are the number of academic medical centers, unless otherwise indicated. ‡ Journal with an impact factor greater than the fifth-ranked journal for that Thompson Scientific category.
Article Press Releases by Academic Medical Centers of highest priority because press releases are typically 1 page or fewer. Most important, because we did not analyze subsequent news coverage of press-released research, we cannot directly link problems with press releases (such as lack of cautions or numbers, or exaggeration) with those in news reports. Our findings would be stronger if we had shown integration of exaggerated information from releases into news stories. Nevertheless, press releases matter: They attract journalists' attention, and although the practice is generally discouraged, many health news stories-perhaps as many as one third-seem to rely largely or solely on the press release (1). Journalists worry that such reliance will increase, given newsroom cutbacks and greater demand for online news (7). The problems that we document are very similar to those seen in analyses of medical news: no quantification of main results (14 -16) and no mention of intervention side effects (14 -16), conflicts of interest (16), or study limitations (1)(2)(3)(4). We believe that academic centers contribute to poor media coverage and are forgoing an opportunity to help journalists do better.
The quickest strategy for improvement would be for centers to issue fewer releases about preliminary research, especially unpublished scientific meeting presentations, because findings often change substantially-or fail to hold up-as studies mature (17). Forty percent of meeting abstracts and 25% of abstracts that garner media attention (18) are never subsequently published as full reports in medical journals (19). Similarly, centers should limit releases about animal or laboratory research. Although such research is important, institutions should not imply clinical benefit when it does not exist (and may not for years, if ever): Two thirds of even highly cited animal studies fail to translate into successful human treatments (20).
When press releases are issued, they should include basic study facts and explicit cautions. For example, press releases should remind journalists that strong inferences cannot be drawn from uncontrolled studies, or that surrogate outcomes do not always translate into clinical outcomes. Although good press releases will probably help, quality reporting also requires good critical evaluation skills. Fortunately, journalists have opportunities to acquire these skills, through such programs as the Association of Health Care Journalists seminars; the Knight Science Journalism Medical Evidence Boot Camp at MIT; and "Medicine in the Media: The Challenge of Reporting on Medical Research," a workshop sponsored by the National Institutes of Health, the Dartmouth Institute for Health Policy and Clinical Practice, and the Department of Veterans Affairs.
Investigators can also do better. They could forgo requesting releases for studies with obvious limitations and review releases before dissemination, taking care to temper their tone (particularly their own quotes, which we often found overly enthusiastic).
By issuing fewer but better press releases, academic centers could help reduce the chance that journalists and the public are misled about the importance or implications of medical research. Centers might get less press coverage, but they would better serve their mission: to improve the health of their communities and the larger society in which they reside.
Research will be presented on the following topics: