Evaluation of the Benchmark Dose Method for Dichotomous Data: Model Dependence and Model Selection

doi:10.1006/rtph.2002.1578

Regulatory Toxicology and Pharmacology

Volume 36, Issue 2, October 2002, Pages 184-197

https://doi.org/10.1006/rtph.2002.1578 Get rights and content

Abstract

The benchmark dose (BMD) method was evaluated using the USEPA BMD software. Dose–response data on cleft palate and hydronephrosis for a number of related polyhalogenated aromatic compounds were obtained from the literature. According to χ² test statistics, each dichotomous USEPA model failed to adequately describe only 1 of 12 cleft palate data sets. For hydronephrosis, the models were discriminated to a higher extent according to global goodness-of-fit. NOAELs for cleft palate corresponded to BMDLs (the approximate lower confidence limit on the BMD) for extra risks in the range of 5% or below. Model dependence of the BMDL estimate was more pronounced at lower levels of benchmark response (BMR). A BMR of 5% (extra risk) is recommended for cleft palate since model differences at this level were limited for all data. In addition, at BMRs of 5–10% the BMDL for all models was little affected by the specified confidence limit size (in the 90–99% range). For BMDL determination a conservative model selection approach was applied. At the suggested level of BMR (5%) this procedure resulted in use of the same model (multistage model) for the cleft palate endpoint in general. Akaike's information criterion (AIC) was considered for comparison between models. Determination of appropriateness of use of such methods in dose–response applications requires further analysis.

References (35)

Akaike, H. 1973, Information theory and an extension of the maximum likelihood principle. In, Proceedings of the Second...
B.C. Allen et al.
Dose–response assessment for developmental toxicity. III. Statistical models
Fundam. Appl. Toxicol
(1994)
B.C. Allen et al.
Calculation of benchmark doses for reproductive and developmental toxicity observed after exposure to isopropanol
Regul. Toxicol. Pharamcol
(1998)
T.R. Auton
Calculation of benchmark doses from teratology data
Regul. Toxicol. Pharmacol
(1994)
L.S. Birnbaum et al.
Teratogenicity of three polychlorinated dibenzofurans in c57bl/6n mice
Toxicol. Appl. Pharmacol
(1987)
L.S. Birnbaum et al.
Teratogenic effects of polychlorinated dibenzofurans in combination c57bl/6n mice
Toxicol. Appl. Pharmacol
(1987)
L.S. Birnbaum et al.
Retinoic acid and 2,3,7,8-tetra- chlorodibenzo-p-dioxin selectively enhance teratogenesis in c57bl/6n mice
Toxicol. Appl. Pharmacol
(1989)
L.S. Birnbaum et al.
Teratogenic effects of 2,3,7,8-terabromodibemzo-p-dioxin and three polybrominated dibenzofurans in c57bl/6n mice
Toxicol. Appl. Pharmacol
(1991)
H. Bozdogan
Akaike's information criterion and recent developments in information complexity
J. Math. Psychol
(2000)
K.P. Burnham et al.
Model Selection and Inference: A Practical Information-Theoretic Approach
(1998)

J.R. Busemeyer et al.

Model comparison and model selection based on generalisation criterion methodology

J. Math. Psychol

(2000)

L.J. Casarett et al.

D. Cox et al.

Theoretical Statistics

(1974)

K. Crump

A new method for determining allowable daily intakes

Fundam. Appl. Toxicol

(1984)

Crump, K, and, Howe, R. 1985, A review of methods for calculating confidence limits in low dose extrapolation. In,...

L. Edler et al.

Statistical models for low dose exposure

Mutat. Res

(1998)

M.R. Forster

Key concepts in model selection: Performance and generalizability

J. Math. Psychol

(2000)

Cited by (35)

Endocrine, metabolic and apical effects of in utero and lactational exposure to non-dioxin-like 2,2´,3,4,4´,5,5´-heptachlorobiphenyl (PCB 180): A postnatal follow-up study in rats
2021, Reproductive Toxicology
Citation Excerpt :
BMD calculations for neonatal deaths, a quantal endpoint, were carried out using the Proast 70.0 package (RIVM, Bilthoven, The Netherlands) in the R software version 4.0.3 (R Development Core Team, R Foundation for Statistical Computing, Vienna, Austria). The recommended BMR default for quantal variables was used, i.e. an extra risk of 10 % [40]. Model averaging was applied by using two.stage, log.logist, Weibull, log.prob, gamma and latent variable models of exponential and Hill families [41].
PCB 180 is a persistent and abundant non-dioxin-like PCB (NDL-PCB). We determined the developmental toxicity profile of ultrapure PCB 180 in developing offspring following in utero and lactational exposure with the focus on endocrine, metabolic and retinoid system alterations. Pregnant rats were given total doses of 0, 10, 30, 100, 300 or 1000 mg PCB 180/kg bw on gestational days 7−10 by oral gavage, and the offspring were sampled on postnatal days (PND) 7, 35 and 84. Decreased serum testosterone and triiodothyronine concentrations on PND 84, altered liver retinoid levels, increased liver weights and induced 7-pentoxyresorufin O-dealkylase (PROD) activity were the sensitive effects used for margin of exposure (MoE) calculations. Liver weights were increased together with induction of the metabolizing enzymes cytochrome P450 (CYP) 2B1, CYP3A1, and CYP1A1. Less sensitive effects included decreased serum estradiol and increased luteinizing hormone levels in females, decreased prostate and seminal vesicle weight and increased pituitary weight in males, increased cortical bone area and thickness of tibial diaphysis in females and decreased cortical bone mineral density in males. Developmental toxicity profiles were partly different in male and female offspring, males being more sensitive to increased liver weight, PROD induction and decreased thyroxine concentrations. MoE assessment indicated that the 95^th percentile of current maternal PCB 180 concentrations do not exceed the estimated tolerable human lipid-based PCB 180 concentration. Although PCB 180 is much less potent than dioxin-like compounds, it shares several toxicological targets suggesting a potential for interactions.
Comparing probabilistic and descriptive analyses of time-dose-toxicity relationship for determining no-observed-adverse-effect level in drug development
2015, Toxicology and Applied Pharmacology
Citation Excerpt :
A safety margin (such as 10 fold) can then be applied to account for the uncertainty in cross-species translation. The concept of benchmark dose (BMD) has been proposed in environmental safety as an alternative to the NOAEL approach (Crump, 1984; Sand et al., 2002; Filipsson et al., 2003). The BMD approach involves fitting a dose–response model to all the toxicity data within a study.
The no-observed-adverse-effect level (NOAEL) of a drug defined from animal studies is important for inferring a maximal safe dose in human. However, several issues are associated with its concept, determination and application. It is confined to the actual doses used in the study; becomes lower with increasing sample size or dose levels; and reflects the risk level seen in the experiment rather than what may be relevant for human. We explored a pharmacometric approach in an attempt to address these issues.
We first used simulation to examine the behaviour of the NOAEL values as determined by current common practice; and then fitted the probability of toxicity as a function of treatment duration and dose to data collected from all applicable toxicology studies of a test compound. Our investigation was in the context of an irreversible toxicity that is detected at the end of the study.
Simulations illustrated NOAEL's dependency on experimental factors such as dose and sample size, as well as the underlying uncertainty. Modelling the probability as a continuous function of treatment duration and dose simultaneously to data from multiple studies allowed the estimation of the dose, along with its confidence interval, for a maximal risk level that might be deemed as acceptable for human. The model-based data integration also reconciled between-study inconsistency and explicitly provided maximised estimation confidence.
Such alternative NOAEL determination method should be explored for its more efficient data use, more quantifiable insight to toxic doses, and the potential for more relevant animal-to-human translation.
On use of the multistage dose-response model for assessing laboratory animal carcinogenicity
2007, Regulatory Toxicology and Pharmacology
Citation Excerpt :
As greater scientific study has centered attention on the multistage paradigm, the model’s form has evolved in both complexity and biological validity, and now includes multiple birth–death processes and elaborate stochastic pathways/stages for tumor onset and progression (Sherman and Portier, 1998; Moolgavkar, 1999). While debate continues on the full applicability such a model can achieve for explaining the intricacies of mammalian cancer induction, its use in cancer dose–response modeling remains quite common (Bailer and Smith, 1994; Luebeck and Moolgavkar, 1996; Sand et al., 2002). Unavailable in the literature, however, is any formal assessment of how well the chosen value of k, and hence the multistage predictor, fits quantal data from carcinogenicity studies.
We explore how well a statistical multistage model describes dose–response patterns in laboratory animal carcinogenicity experiments from a large database of quantal response data. The data are collected from the US EPA’s publicly available IRIS data warehouse and examined statistically to determine how often higher-order values in the multistage predictor yield significant improvements in explanatory power over lower-order values. Our results suggest that the addition of a second-order parameter to the model only improves the fit about 20% of the time, while adding even higher-order terms apparently does not contribute to the fit at all, at least with the study designs we captured in the IRIS database. Also included is an examination of statistical tests for assessing significance of higher-order terms in a multistage dose–response model. It is noted that bootstrap testing methodology appears to offer greater stability for performing the hypothesis tests than a more-common, but possibly unstable, “Wald” test.
Assessment factors-Applications in health risk assessment of chemicals
2007, Environmental Research
Citation Excerpt :
This last statement is a major advantage, but also represents one of the challenges associated with the benchmark dose method, i.e., which level of response, or risk, should the BMD correspond to? A number of suggestions have been made for how to define the BMD (Crump, 1984, 1995; Kimmel and Gaylor, 1988; Gaylor and Slikker, 1990; Allen et al., 1994; Kodell and West, 1993; Gaylor et al., 1998; Slob, 2002; Falk Filipsson et al., 2003; Gaylor and Aylward, 2004; Sand et al., 2002, 2003; Sand, 2005). An additional advantage in the context of assessment factors is that the use of distributions of ratios of benchmark doses results in more accurate and less conservative distributions of assessment factors in general compared to the distributions based on NOAEL ratios (Bokkers and Slob, 2005).
We review the scientific basis for default assessment factors used in risk assessment of nongenotoxic chemicals including the use of chemical- and pathways specific assessment factors, and extrapolation approaches relevant to species differences, age and gender. One main conclusion is that the conventionally used default factor of 100 does not cover all inter-species and inter-individual differences. We suggest that a species-specific default factor based on allometric scaling should be used for inter-species extrapolation (basal metabolic rate). Regarding toxicodynamic and remaining toxicokinetic differences we suggest that a percentile from a probabilistic distribution is chosen to derive the assessment factor. Based on the scarce information concerning the human-to-human variability it is more difficult to suggest a specific assessment factor. However, extra emphasis should be put on sensitive populations such as neonates and genetically sensitive subgroups, and also fetuses and children which may be particularly vulnerable during development and maturation. Factors that also need to be allowed for are possible gender differences in sensitivity, deficiencies in the databases, nature of the effect, duration of exposure, and route-to-route extrapolation. Since assessment factors are used to compensate for lack of knowledge we feel that it is prudent to adopt a “conservative” approach, erring on the side of protectiveness.
A benchmark dose analysis for sodium monofluoroacetate (1080) using dichotomous toxicity data
2007, Regulatory Toxicology and Pharmacology
The use of a benchmark dose (BMD) as an alternative to a no-observed-adverse-effect-level (NOAEL) approach was investigated as a means to improve current risk assessment values of sodium monofluoroacetate (1080). The feasibility of implementing the two approaches was investigated for three critical toxicological end points, namely cardiomyopathy, testicular toxicity and teratogenic effects identified from the few available critical studies. The BMD provides better representation of the dose–response relationship, offering an advantage over the current NOAEL approach. The calculated BMDs and lower-bound confidence limits (BMDLs) for the three end points were estimated using the Weibull, probit and quantal linear models for each end point. All models passed the χ² test statistics (p ⩾ 0.1) for all three toxicity endpoints tested. A benchmark response (BMR) of 10% (extra risk) was chosen and the Akaike’s information criterion (AIC) was used in selecting the appropriate model. The BMDL estimates derived were found to be generally slightly higher but comparable to the NOAEL for those same endpoints. The BMD₁₀ and BMDL₁₀ for cardiomyopathy and testicular effects were 0.21 mg kg⁻¹ bw and 0.10 mg kg⁻¹ bw, respectively. These values are proposed for use in the eventual determination of the tolerable daily intake (TDI) for 1080.
Using benchmark dose modeling for the quantitative risk assessment: Carbon nanotubes, asbestos, glyphosate
2021, Journal of Applied Toxicology

View all citing articles on Scopus

¹: To whom correspondence should be addressed. Fax: +468343849. E-mail: [email protected].

View full text

Regular ArticleEvaluation of the Benchmark Dose Method for Dichotomous Data: Model Dependence and Model Selection

Abstract

Dose–response assessment for developmental toxicity. III. Statistical models

Fundam. Appl. Toxicol

Calculation of benchmark doses for reproductive and developmental toxicity observed after exposure to isopropanol

Regul. Toxicol. Pharamcol

Calculation of benchmark doses from teratology data

Regul. Toxicol. Pharmacol

Teratogenicity of three polychlorinated dibenzofurans in c57bl/6n mice

Toxicol. Appl. Pharmacol

Teratogenic effects of polychlorinated dibenzofurans in combination c57bl/6n mice

Toxicol. Appl. Pharmacol

Retinoic acid and 2,3,7,8-tetra- chlorodibenzo-p-dioxin selectively enhance teratogenesis in c57bl/6n mice

Toxicol. Appl. Pharmacol

Teratogenic effects of 2,3,7,8-terabromodibemzo-p-dioxin and three polybrominated dibenzofurans in c57bl/6n mice

Toxicol. Appl. Pharmacol

Akaike's information criterion and recent developments in information complexity

J. Math. Psychol

Model Selection and Inference: A Practical Information-Theoretic Approach

Model comparison and model selection based on generalisation criterion methodology

J. Math. Psychol

Theoretical Statistics

A new method for determining allowable daily intakes

Fundam. Appl. Toxicol

Statistical models for low dose exposure

Mutat. Res

Key concepts in model selection: Performance and generalizability

J. Math. Psychol

Regular Article
Evaluation of the Benchmark Dose Method for Dichotomous Data: Model Dependence and Model Selection