Five year survival rates can mislead

While running for president of the United States the former New York mayor Rudy Giuliani announced in a 2007 campaign advertisement, “I had prostate cancer, 5, 6 years ago. My chance of surviving prostate cancer—and thank God, I was cured of it—in the United States? Eighty-two percent. My chance of surviving prostate cancer in England? Only 44 percent under socialized medicine.”1

To Giuliani this meant that he was lucky to be living in New York and not in York, because his chances of surviving prostate cancer seemed to be twice as high in New York. Yet despite this impressive difference in the five year survival rate, the mortality rate was about the same in the US and the UK.

Why is an increase in survival from 44% to 82% not evidence that screening saves lives? For two reasons. The first is lead time bias. Earlier detection implies that the time of diagnosis is earlier; this alone leads to higher survival at five years even when patients do not live any longer. The second is overdiagnosis. Screening detects abnormalities that meet the pathological definition of cancer but that will never progress to cause symptoms or death (non-progressive or slow growing cancers). The higher the number of overdiagnosed patients, the higher the survival rate. In the US a larger proportion of men are screened by prostate specific antigen testing than in the UK, contributing to the US’s higher survival rate.

The important thing to understand is that the correlation between differences in survival rates and mortality rates is zero (r = 0.0 for the 20 most common solid tumours over the past 50 years).2 Thus the message is clear: the benefit of screening needs to be communicated in mortality rates, not survival rates.

This point has been made many times before. But the message has not yet reached every doctor. In a convenience sample of 65 German internal medicine specialists only two could explain what lead time bias was, and none could explain overdiagnosis.3 When given further information on the benefit of screening in the form of mortality rates, only three (5%) of the physicians concluded that screening was effective; when given the same information in the form of survival rates, 51 (78%) believed it to be effective, as Giuliani had. Are US doctors just as easily misled? A national random sample of 412 US primary care physicians was asked, “Which of the following prove that a screening test ‘saves lives’ from cancer?”4

• a) More cancers are detected in screened populations than in unscreened populations (47% of the doctors agreed).

• b) Screen detected cancers have better five year survival rates than cancers detected because of symptoms (76% agreed).

• c) Mortality rates are lower among screened people than unscreened people in a randomised trial (81% agreed).

Only reduced mortality rates (statement c) can prove that screening saves lives. Yet three quarters of the doctors mistook five year survival rates as such a proof. In addition, almost half erroneously believed that the detection of more cancers proved that screening saved lives.

Some reviewers and editors of major journals still allow authors to present the benefit of screening in terms of survival rates. Three illustrations:

• An article in the Journal of Clinical Oncology concluded that regular mammography may be beneficial for women aged 80 or over because “breast cancer-specific survival among nonusers was 82%, that among irregular users was 88%, and that among regular users was 94%.”5

• An article in the New England Journal of Medicine concluded that “annual spiral CT [computed tomography] screening can detect lung cancer that is curable,” because the estimated 10 year survival among patients with screen detected stage I lung cancer was 88%, much higher than what clinicians see in practice.6

• An article in Lancet Oncology emphasised in its abstract that in the case of most solid tumours “survival in patients diagnosed in 2000-02 was higher in the US SEER [surveillance, epidemiology, and end results] register than for the European mean.”7

Similarly, cancer centres, charities, and health pamphlets misuse five year survival data to impress people with apparently large benefits of screening, even if there is no mortality reduction.8 A variant of this tactic is for an institution to compare its own survival rate with the national mortality figure.9

What to do? We have three suggestions.

Firstly, risk communication needs to become a central skill in medical education. For decades medical schools have failed to teach students statistical thinking (biostatistics does not seem to help much). The basic structure for such a teaching programme already exists.9 10

Secondly, organisations responsible for continuing medical education and recertification programmes should ensure that doctors are trained in understanding evidence and in risk communication.

Finally, journal editors and reviewers should no longer allow misleading statistics such as five year survival to be reported as evidence for screening. Editors should enforce transparent reporting of evidence, for the benefit of their readers and of healthcare in general.