Is There a Scientific Foundation for Vaccine Recommendations at This Time?

will prevent sarcoma formation.

countries of Europe, together with the large number of owned cats, there are now more than 20 years of experience managing afflicted patients, providing a plethora of information about current standards of practice as well as emerging state-of-theart therapies. The veterinary medical professional manifestly benefits from such reflection, as do owners and their feline companions.
I am less sanguine, however, that these authors' recommendations for prevention share the same evidence-based scientific standing that their management recommendations have. For there to be standing to justify recommendations there must be foundation. For there to be foundation there must be evidence; for there to be evidence there must be research. The latter presents in many forms, and I have become increasingly concerned that the findings from preliminary or tenuous research have, over time, taken on a quasi-mythical standing through a disciplinary support network that places more weight on belief than on the weight of the evidence itself. Opinion is, of course, the natural evolution of the assimilation of information, and is the provenance of assertions by decision makers occupying positions of leadership, influence, and change. In the proper setting, and in the appropriate context, such expressions contribute to a healthy exchange and dialogue (eg, the Vaccine-Associated Feline Sarcoma Task Force). 3 For an article focusing on prevention of this disease in a peer-reviewed scientific journal, far more circumspection is not only warranted, but arguably essential. In this article, I hope to underscore this contention by illustrating that not only do I judge that such recommendations are premature (although not necessarily incorrect), but that others absorbing the same body of evidence could be impelled to reach entirely different conclusions.
The key statement in that article, and hence the most provocative, is the following from the abstract: "Non-adjuvanted, modified-live or recombinant vaccines should be selected in preference to adjuvanted vaccines." This is manifestly similar to a principle expressed in the World Small Animal Veterinary Association's (WSAVA) Guidelines for the Vaccination of Dog and Cats 4 : "Non-adjuvanted vaccines should be administered to cats wherever possible." Indeed, the WSAVA 4 and Hartmann and colleagues 2 articles share authors in common. However, these prescriptions go well beyond the recommendations of the 2013 American Association of Feline Practitioners Advisory Panel Report, which judiciously exercised considerably more restraint in writing: "Overall, however, the Advisory Panel concluded that, at the current time, there is insufficient information to make definitive recommendations to use particular vaccine types to reduce the risk of FISS [feline injection-site sarcomas]." 5 What is the evidence to support the Hartmann and colleagues 2 recommendation, as indicated in the abstract and on page 611: "Vaccines without adjuvants should be used rather than adjuvant-containing vaccines, which means that MLV or recombinant vaccines (eg, canarypox-vectored vaccine) without adjuvant are preferred over inactivated vaccines with adjuvants?" The section "Recommendations for reducing inflammatory reactions" (pages 610-611) provides some guidance. Three articles cited found that recombinant canarypox-vectored vaccines caused less inflammation when injected into rats and cats. [6][7][8] The use of such experimental studies to measure postvaccinal tissue inflammation is enigmatic and can be faulted on several grounds. Using rodents as models of adjuvant-induced inflammation or carcinogenesis in the cat remains notional, and its validity has previously been called into question. 9 Given the near-certain differences between species in immunologic and tissue-based responses to vaccine adjuvants, it should be difficult to ascribe more than a passing interest in these results. As for the use of cats in experimental studies, the goal should not be to measure relative inflammatory responses, which would inevitably be expected under different vaccine formulations, but rather to estimate neoplastic incidence. None of these experimental studies, however, had anything close to the statistical power necessary to detect differences in vaccine risk. Suppose, for example, that the incidence of sarcomas following vaccination is 5 cases in 10,000 cat-doses, and the incidence in the absence of vaccines is 1 case in 10,000 cat-doses (ie, the relative risk is 5). A prospective 2-armed randomized study analyzed with a 2-sided Pearson's chi-square test, with equal allocation between arms and Type I and Type II error proportions of 5%, would require almost 100,000 cats. In contrast, the experimental studies in rats and cats cited previously had sample sizes in the double-digits.
Although they employed different methods to arrive at their respective conclusions, the experimental studies, not unexpectedly, shared a key critical feature: none of the study subjects developed injection site sarcomas. Although their findings may have implications for the study of postvaccinal inflammatory responses, they fail to provide a rational basis for making causal inferences about vaccine propensity to induce tumorigenesis. Such a causal connection between postvaccinal inflammation and tumorigenesis remains to this day one entirely of conjecture, speculation, and hypothesis, and until that connection can be firmly established, such articles may be useful in understanding vaccine-specific inflammation, but have unproven and hence questionable value in understanding vaccine formulation-specific risk of sarcoma development. They and others (eg, the vaccine manufacturer-funded experimental Grosenbaugh and colleagues 10 study) emphatically do not rise to the level of research upon which policy supported by science about reducing the incidence of injection-site sarcomas should be promulgated and distributed.
Nevertheless, this has not prevented several authors from doing exactly that, [11][12][13][14] a practice that at this time I consider to be imprudent. Of considerable concern is that these articles include unpublished data, personal communications, citations of review articles, reliance on science by authority, or engagement in associational speculation. It is also sometimes the case that authors have financial ties, as collaborators, consultants, or employees, to the very industries that are impacted by their recommendations. It must be incumbent on all authors (and presenters) in the field of FISS to fully disclose such relationships to their readers (and audiences) to further transparency and scientific integrity.
The final article the authors invoked to support their recommendation is from my own research group at the University of California, Davis. 15 And although this study did include cats with injection-site sarcomas, and indeed found supportive evidence for differential tumorigenic propensity between vaccine types, I would firmly contend that it alone (discounting the previously mentioned articles about inflammation but not sarcomas) remains insufficient to this day to be the basis for the recommendations in Hartmann and colleagues 2 In fact, we attempted to temper our findings within the article itself by citing the study's shortcomings that could pose a threat to validity: A low response rate from veterinarians, which could have been differential with respect to the types of vaccines administered A small sample size, which makes a single study far more susceptible to biased and imprecise estimates Missing data The use of multiple vaccines at the same site either at one or multiple times The tendency to report a single number or conclusion from a single study is unfortunately all too common, and I have witnessed individual odds ratios (ORs) from this study presented outside of their proper statistical context. For example, Srivastav Prevention of Feline Injection-Site Sarcomas and colleagues 15 reported that ". there was evidence of a significantly lower frequency of use of recombinant rabies vaccines in case cats than controls. Using cats with nonvaccine site sarcomas as controls, in years 1, 2, and 3, the ORs were 0.1 (95% CI, 0.0-0.7; P 5 .014), 0.1 (95% CI, 0.0-0.4; P 5 .001), and 0.1 (95% CI, 0.0-0.6; P 5 .005), respectively." At a recent international meeting, I heard these findings communicated as: the odds of cases receiving a nonrecombinant vaccine was tenfold greater than receiving a recombinant vaccine (ie, 1/0.1 5 10). Although literally correct in an algebraic sense, such statements entirely ignore a key purpose of statistical inference in the first place: the analysis of variance. Focusing solely on point estimates fails to convey important information (eg, an OR of 10 from a sample size of 50 should naturally be given far less credibility than an odds ratio of 10 from a sample size of 500). A more recent presentation at least pointed out that the CI around the communicated value of 10 would have been (using, for example, the year 2 value) 2.5 to infinity. 16 But the real story about the quantitative relationship between vaccine type and sarcoma incidence, which from this article is profoundly imprecise, is how little we still really understand even after this study's publication. Moreover, it is often unappreciated that the P-values associated with such tests are only correct insofar as the assumptions underlying them are accurate, including the absence of bias, under the probability distribution model utilized in the analysis. In other words, if any of the biases noted previously were present, then regardless of the statistical significance, the P-values would be incorrect, as would the point estimates (ORs) and CIs. It is a scientific disservice to recapitulate potentially headline-grabbing findings from articles without concomitantly and fully assessing and disclosing those features that could adversely impact study accuracy. Far too often excessive credence is placed on statistical significance, and far too little weight on the myriad subtleties of observational study design and analysis that can lead to invalid inferences:

Errors of comparisons (confounding bias) Errors in selection of study subjects (selection bias) Errors in (often historical) measurements (information bias) Errors in statistical modeling (specification bias)
In the case of the Srivastav and colleagues article, 15 such threats to validity could include (but are not limited to): The low veterinarian participation rate (eg, participation could be related to veterinarian preference for vaccine type) Diagnostic work-up being related to type of vaccine administered Missing data that could have been related to type of vaccine administered I contend that the authors' avidity for their prevention recommendations 2 exceeds the weight of foundational scientific evidence to support them at this time. Although they apparently consider them to be accurate, and they may be correct, the insertion of such recommendations into such an authoritative report strikes me as premature, fails to convey the paucity of evidence supporting them, and omits a critical analysis of research upon which they are based. It is evocative of the admonitions of author/ journalist Christopher Hitchens: "Forgotten were the elementary rules of logic, that extraordinary claims require extraordinary evidence and that what can be asserted without evidence can also be dismissed without evidence." 17 A similar forewarning about the shortcomings of published research was forcefully made by Ioannides, 18 who wrote: "Several methodologists have pointed out that the high rate of nonreplication (lack of confirmation) of research discoveries is a consequence of the convenient, yet ill-founded strategy of claiming conclusive research findings solely based on a single study assessed by formal statistical significance, typically for a P-value less than 0.05." All too often, human medicine has been forced to disavow widely disseminated public health recommendations founded on nonexperimental studies when later, more robust evidence failed to support their establishment. 19 I only hope that history does not repeat itself here.