Enhancing Credibility of Chemical Safety Studies: No Consensus

I commend Conrad and Becker (2011) for frankly discussing the criteria by which to judge data quality in chemical risk assessment. Those criteria revolve tightly around financial conflict of interests. I agree with Conrad and Becker (2011) that industry seems likely to continue to perform the toxicity tests in risk assessment. That is due to what I call the “GLP [Good Laboratory Practices] shield.” Industry’s compliance with GLP has caused many tens of thousands of published independent chronic toxicity studies—any of which might have determined the allowable “safe” daily dose of a chemical—to be excluded from premarket approval risk assessments and substituted with financially conflicted, yet GLP-compliant, data from the chemical’s manufacturer. 
 
It is incomprehensible to me that Conrad and Becker (2011) assert that a financial conflict of interests should not be a criterion in determining the financial independence of researchers (and therefore the reliability of results). A financial conflict of interests exists as soon as there is a link between a researcher and a monetary value. It does not signify unethical behavior, but it does warn of that possibility. Scientists should be reassured—not upset, as Conrad and Becker (2011) claim—if financial conflict of interests was the lead criteria to assess data quality. Conrad and Becker’s substitute criterion—disclosure of financial conflict of interests—becomes useless with their other recommendation to accept the data of financially conflicted scientists. 
 
Conrad and Becker (2011) failed to mention the independent and consistent reviews that all but prove that sponsorship of science by the pharmaceutical industry produces results more financially favorable to them than those of financially independent science, and several reviews of toxicity studies of petrochemicals reach the same conclusion (Bekelman et al. 2003; Fagin et al. 1999; Swaen and Meijers 1988; vom Saal and Hughes 2005). 
 
Repeatedly Conrad and Becker (2011) urge regulators to accept GLP as a key criterion determining data reliability (the ability to predict actual toxicities). The Organisation for Economic Co-operation and Development (OECD) creates toxicity study guidelines (OECD 2011) featuring GLP, which are adopted worldwide by regulators. These OECD regulatory test protocols are stuck in the age of the light microscope, test a narrow and unrealistic portion of the dose–response curve and relatively few end points, mostly fail to test toxicity during vulnerable development, and kill the animals being tested before most diseases develop (a human equivalent of ~ 60 years). Society should not accept that the OECD GLP protocols are better than those developed by independent, curious academics. Therefore, for any common petrochemical, readers should compare in depth the independent toxicity findings via PubMed and the OECD GLP alleged safe exposure level, which Conrad and Becker (2011) promote. 
 
Conrad and Becker (2011) proposed that industry be allowed to continue to influence research, although they would discount studies for which a sponsor owns the results. Journals seem to prefer the simplicity and finality of forbidding outsider control of a researcher’s data. 
 
Finally, Conrad and Becker (2011) (compared with Becker et al. 2009) gave lukewarm support to traditional journal peer review and publication, but they continue to question its value, claiming instead that peer review by government regulatory agencies is of better quality. However, such a criterion would simply reinforce these agencies’ current use of these financially conflicted data in determining risk assessment outcome. It would be better if risk assessment relied on traditional peer review, which is science’s most fundamental tool for ensuring reliable data. False-negative error is more consequential than false-positive error.


Enhancing Credibility of Chemical Safety Studies: No Consensus
http: //dx.doi.org/10.1289/ehp.1104130 I commend Conrad and Becker (2011) for frankly discussing the criteria by which to judge data quality in chemical risk assessment. Those criteria revolve tightly around financial conflict of interests. I agree with Conrad and Becker (2011) that industry seems likely to continue to perform the toxicity tests in risk assessment. That is due to what I call the "GLP [Good Laboratory Practices] shield." Industry's compliance with GLP has caused many tens of thousands of published independent chronic toxicity studies-any of which might have determined the allowable "safe" daily dose of a chemical-to be excluded from pre market approval risk assessments and substituted with financially conflicted, yet GLP-compliant, data from the chemical's manufacturer.
It is incomprehensible to me that Conrad and Becker (2011) assert that a financial conflict of interests should not be a criterion in determining the financial independence of researchers (and therefore the reliability of results). A financial conflict of interests exists as soon as there is a link between a researcher and a monetary value. It does not signify unethical behavior, but it does warn of that possibility. Scientists should be reassurednot upset, as Conrad and Becker (2011) claim-if financial conflict of interests was the lead criteria to assess data quality. Conrad and Becker's substitute criterion-disclosure of financial conflict of interests-becomes useless with their other recom mendation to accept the data of financially conflicted scientists. Conrad and Becker (2011) failed to mention the independent and consistent reviews that all but prove that sponsor ship of science by the pharmaceutical industry produces results more financially favorable to them than those of financially independent science, and several reviews of toxicity studies of petrochemicals reach the same conclusion (Bekelman et al. 2003;Fagin et al. 1999;Swaen and Meijers 1988;vom Saal and Hughes 2005).
Repeatedly Conrad and Becker (2011) urge regulators to accept GLP as a key criterion determining data reliability (the ability to predict actual toxicities). The Organisation for Economic Co-operation and Development (OECD) creates toxicity study guidelines (OECD 2011) featuring GLP, which are adopted worldwide by regulators. These OECD regulatory test protocols are stuck in the age of the light microscope, test a narrow and unrealistic portion of the doseresponse curve and relatively few end points, mostly fail to test toxicity during vulnerable develop ment, and kill the animals being tested before most diseases develop (a human equivalent of ~ 60 years). Society should not accept that the OECD GLP protocols are better than those developed by independent, curious academics. Therefore, for any common petro chemical, readers should compare in depth the independent toxicity findings via PubMed and the OECD GLP alleged safe exposure level, which Conrad and Becker (2011) promote. Conrad and Becker (2011) proposed that industry be allowed to continue to influence research, although they would discount studies for which a sponsor owns the results. Journals seem to prefer the simplicity and finality of forbidding outsider control of a researcher's data.
Finally, Conrad and Becker (2011) (compared with Becker et al. 2009) gave lukewarm support to traditional journal peer review and publication, but they continue to question its value, claiming instead that peer review by government regulatory agencies is of better quality. However, such a criterion would simply reinforce these agencies' current use of these financially conflicted data in determining risk assessment outcome. It would be better if risk assessment relied on traditional peer review, which is science's most fundamental tool for ensuring reliable data. False-negative error is more consequential than false-positive error.
The author receives fees for advising nonprofit public health organizations.  (Conrad and Becker 2011) because we did not include industry funding of a study in our proposed set of criteria. Fundamentally, we rejected "industry funding" as a credibility criterion "because questions can also arise about the credibility of research by scientists funded by government agencies or nonprofit organizations" (Conrad and Becker 2011). We accept as a given that someone has always paid for scientific work and that funding has inherent potential to influence results, whether it comes from industry, environmental groups, or government. It appears that at least one member of Sutton et al. agrees with us to some degree on this point, because they repeatedly cite Bero (1999), who endorsed "establishing restrictions on sponsorship, regardless of its source." We agree that funding bias has been docu mented, at least in clinical trials for pharmaceuticals and medical procedures, although as Sutton et al. note, the published literature does not yet appear to have systematically studied the issue in the field of toxicology or epidemiology across a broad spectrum of substances. (Tobacco is a unique and extreme case and should be recognized as such, not cited tendentiously as indicative of all industry support of research.) The guidelines for routine toxicity studies are publically available and incorporate end points reflecting both input from a broad spectrum of experts and approval by government regulatory authorities [U.S. Environmental Protection Agency (EPA) 2011], and all such studies employ an entirely independent quality assurance program documenting that facilities, equipment, personnel, methods, practices, records, and controls are in conformance with Good Laboratory Practice (GLP) requirements. The extent to which these features govern the design and conduct of toxicity studies mitigates funding bias to a large degree.
In our review (Conrad and Becker 2011), we also noted that source of funding is often considered a "less significant" cause of publication bias than other causes [e.g., career advancement (publish or perish), personal advancement, status in one's professional field, interest in obtaining positive results].
Sutton et al. attempt to show that the documents from ostensibly non industryfunded sources that we cited in our review (Conrad and Becker 2011) in fact diverge from the other source documents, but their effort is unconvincing. First, they quote the Bipartisan Policy Center's statement that one should "consider sources of funding and any conflicts of interest" associated with a study (Bipartisan Policy Center 2009). But that is consistent with our review. We did not urge readers to dismiss questions of funding; rather, we stated that such questions "trigger application of the criteria" (Conrad and Becker 2011 (2006) pointed out that human-subjects research is a case of special concern, and they also addressed who should be permitted to participate in conducting research. But their larger goal was as we characterized it: given that "academia-industry collaborations can benefit society," they proposed "voluntary measures that guard against research bias and foster transparency and accountability" (Brockway and Furcht 2006).
It is unclear what Sutton et al.'s solution-"application of systematic and transparent methodologies to vet the science"-would mean in practice when one is confronted with a toxicological or epidemiological study. We suspect that their solution would end up looking a lot like our criteria; for example, Bero (1999) noted the danger of suppression by sponsors, but our criteria 2, 3, and 7 (Conrad and Becker 2011) all militate against that possibility. Bero (1999) also argued that "sponsored investigators should retain control over the publication of results, regardless of their outcome"-our criterion 2. Tweedale likewise endorses "the simplicity and finality of forbidding outsider control of a researcher's data," but again, that is what a sponsor has to accept to satisfy criterion 2.
Sutton et al. complain that our criteria would not "eliminate" bias, but we do not claim to do that. We claim that each of our criteria "either a) increases confidence that the sponsor or experimenter did not shape or skew the results or interpretation of an experiment; or b) enables others to assess independently whether such shaping or skewing occurred." Our criteria allow the scientific evidence to speak for itself.
Tweedale criticizes our review (Conrad and Becker 2011) for not proposing "financial conflict of interests [as] the lead criteria to assess data quality" and for "fail[ing] to mention" the funding bias issue. As to the former, the purpose of our review was to address credibility, not reliability; this is important because generally accepted methods for determining data reliability have already been adopted and implemented by regulatory agencies (European Chemicals Agency 2009;U.S. EPA 1999). As to the latter, in our review (Conrad and Becker 2011) we stated that "critics have argued that industrysupported work has employed methods, animal strains, or other test features that tend to miss or under estimate adverse effects," so we clearly acknowledged the under lying concern, even if we did not cite Tweedale's references.
Beyond conflict of interest, Tweedale additionally mis charac terizes our review (Conrad and Becker 2011) regarding the topic of GLP. We did not propose excluding any relevant study simply because it did not follow GLP. Consistent with established best practices of systematic evidence-based reviews, we support use of transparent, objective criteria for determining data quality and study reliability. Such criteria allow data from laboratory experiments, epidemiological investigations, and cutting-edge mechanistic research from all relevant studies, GLP and non-GLP, and from all investigators, regardless of affiliation or funding source, to be comprehensively and systematically reviewed, given appropriate weight, and integrated in a manner that provides a robust understanding of the mode of action and the potential hazards and risks that exposures to a substance could pose. These basic principles of causal inference are widely endorsed and practiced (e.g., National Research Council 2011), and such analysis will reveal the strengths and flaws of a study, independent of study authorship or funding.
Tweedale ignores or misunderstands a) previous discourse (Becker et al. 2009(Becker et al. , 2010Tyl 2009) explaining how and why the elements of GLP often result in greater weight being given to such studies, and b) the processes by which the Organisation for Economic Co-operation and Development (OECD) develops test guidelines, in which experts around the world collaborate to formulate, validate, update, and independently peer review OECD test guidelines (e.g., OECD 2008). When new end points or