Judging the Data: Peer Review versus Good Laboratory Practice Standards

When it comes to assessing the quality of toxicologic data to develop policy decisions, should regulators rely on journal peer review or on Good Laboratory Practice (GLP) standards? Authors of a review comparing the two conclude that the answer is neither. Instead, they propose that regulators need a well-defined scheme in which the best elements of both processes enable data to be weighed and evaluated in a consistent fashion [EHP 120(7):927–934; McCarty et al.]. 
 
The authors point out that peer review and GLP standards serve different but complementary needs. Traditional peer review aims to supply scientists with published articles worthy of consideration and debate; GLP standards—which are specific protocols for conducting and reporting experiments—aim to provide regulators with high-quality data that are acceptable across jurisdictions. 
 
The authors go on to expose the shortcomings of each framework for policy use. According to their analysis, peer review suffers from reviewer bias, inconsistency in methods between journals, variable ability to identify fraud or falsified data, and a reluctance to publish results that show little or no effects. GLP standards, on the other hand, don’t address broad issues of scientific validity, although they supply consensual formats for gathering and analyzing data. 
 
But instead of arguing over which of the two approaches best suits policy making, the authors propose that stakeholders should focus instead on the approaches’ growing convergence. That convergence, they propose, comes mainly from the peer-review side, which increasingly requires additional data reporting and supplementary methodological information for online publication—a trend they say serves public needs for data transparency and better communication of scientific concepts. 
 
But what regulators in toxicology need above all else, the authors emphasize, is reliable, adequate, relevant data. Toward that end, the authors propose a weight-of-evidence scheme for data evaluation that comprises six steps: 1) Define the uses and goals of the regulatory action, and identify testable hypotheses; 2) define priorities for data weighting and support them with references; 3) gather the relevant data in a systematic way; 4) evaluate how well each selected study fulfilled its initial intent; 5) combine all data weightings in a predefined manner to achieve a score for each study; and 6) integrate the scores into a narrative that addresses judgments and conclusions derived from the entire evaluation process. Although neither peer review nor GLP standards can fully meet the needs for data quality and relevant science on their own, the authors write, a properly devised weight-of-evidence scheme can fill those overarching requirements.

supply consensual formats for gathering and analyzing data.
But instead of arguing over which of the two approaches best suits policy making, the authors propose that stakeholders should focus instead on the approaches' growing convergence. That convergence, they propose, comes mainly from the peer-review side, which increasingly requires additional data reporting and supplementary methodological information for online publication-a trend they say serves public needs for data transparency and better communication of scientific concepts.
But what regulators in toxicology need above all else, the authors emphasize, is reliable, adequate, relevant data. Toward that end, the authors propose a weight-ofevidence scheme for data evaluation that comprises six steps: 1) Define the uses and goals of the regulatory action, and identify testable hypotheses; 2) define priorities for data weighting and support them with references; 3) gather the relevant data in a systematic way; 4) evaluate how well each selected study fulfilled its initial intent; 5) combine all data weightings in a predefined manner to achieve a score for each study; and 6) integrate the scores into a narrative that addresses judgments and conclusions derived from the entire evaluation process. Although neither peer review nor GLP standards can fully meet the needs for data quality and relevant science on their own, the authors write, a properly devised weight-of-evidence scheme can fill those overarching requirements.
Charles W. Schmidt, MS, an award-winning science writer from Portland, ME, has written for Discover Magazine, Science, and Nature Medicine.

Teasing Out Relationships between Autism and Maternal Smoking
Both genetic and environmental factors have been implicated in autism spectrum disorders (ASDs), which affect an estimated 1 in 88 children. One such environmental factor, prenatal exposure to tobacco smoke via maternal smoking, has been associated with ASDs in some studies but not others. A new study reports evidence of a positive association between maternal smoking during pregnancy and higher-functioning ASD subtypes [EHP 120 ( The authors collected information on maternal smoking and other factors from the birth certificates for 633,989 children born in 1992, 1994, 1996, and 1998 in 11 U.S. states. They linked these data with surveillance data from the U.S. Centers for Disease Control and Prevention's Autism and Developmental Disabilities Monitoring network and identified 3,315 of the children who were subsequently diagnosed with an ASD by age 8 years. About 13% of all the mothers smoked during pregnancy, compared with about 11% of mothers with children diagnosed with an ASD. Maternal smoking has been associated with both lower education and reduced access to health care, factors that might increase the likelihood that ASDs go undiagnosed among children of women who smoked during pregnancy. When the authors corrected for this potential bias using outcome misclassification sensitivity analyses, a weak positive association emerged between maternal smoking and cases classified as "ASD not otherwise specified," which were assumed to be higherfunctioning ASDs such as Asperger's disorder. The association was not found for lower-functioning (that is, more severe) ASDs.
The authors write that their findings concerning ASD subgroups should be interpreted with caution because the accuracy of subgroup classification may have varied depending upon mothers' access to evaluation services, and because it was not bas ed on direct clinical observation. They also note that positive associations may reflect the presence in higher-functioning subgroups of children with co-occurring disorders (such as attention deficit/hyperactivity disorder) that can be affected by nicotine exposure.
Strengths of the study include the large sample size, the population-based design with standardized identification of ASD cases, and the use of sensitivity analyses to evaluate potential sources of bias. The authors conclude that the observed association between maternal smoking during pregnancy and higher-functioning ASDs warrants further research.