Uncertainty in the estimation of benzene risks: application of an uncertainty taxonomy to risk assessments based on an epidemiology study of rubber hydrochloride workers.

This paper reviews 14 risk assessments that use the data from descriptions by Rinsky, Young, and co-workers of benzene-associated leukemias among a group of rubber hydrochloride workers in Ohio. The leukemogenic risks of benzene estimated in these assessments differ. The assessors use different assumptions (parameters, confounding factors, or formulas), which account for the differences in risk. The purpose of the review is to determine whether the major source of uncertainty in assessments of benzene risk arises from data, method, or concept. The results show that methodological differences dominate the other two potential sources with respect to impact on risk magnitude.


Introduction
Benzene has received intense attention because it causes a dreaded human health effect, leukemia, but it also has an essential economic role. Benzene serves as a benchmark for the toxicology community, both as a case study to compare effects between man and laboratory animals and as a challenge to understand its mechanism of action. Perhaps because of regulatory scrutiny, prediction of leukemic risk from benzene exposure has served as an important case study for the application of risk assessment techniques.
The taxonomy in this paper allocates uncertainty in risk estimations to one of three possible sources: measure-ment, model, or concept. Measurement error in a parameter results from variation in the data. Model error arises from application of an erroneous method, procedure, or principle. Conceptual error comes from study of the wrong problem or use of the wrong decision rules.

Methods
The risk assessments appear either in the scientific literature or in the Occupational Safety and Health Administration Docket. The relative magnitude of uncertainty from each source was ranked, using different methods for each source, as described in the next sections.

Measurement Uncertainty
Overall data uncertainty derives from variation in the value for each parameter. Data comparisons can be normalized by assuming a common methodology and treating each unique value assigned by a different assessor to a component as if it represents an independent sample from a distribution. Because replications of the same value for a parameter often derive from a single source, the values are not weighted for frequency of occurrence. This combinatorial procedure permits an estimate of uncertainty according to the formula ir ai,bj,ck ... hp _ rn, where ai, bj, Ck ... hp represents one value from the set for each factor (or its reciprocal), taken in all possible combinations, and r. represents a risk estimate. The resulting log-normal probability distribution describes an aggregate belief of the assessors, given a common methodology. When the authors of an assessment did not state a value for a component, it was inferred by back calculation. These "as-if" values are not necessarily the same values the authors would give and should be attributed only to the current paper. Rinsky et al. published their data progressively in 1977Rinsky et al. published their data progressively in , 1981Rinsky et al. published their data progressively in , and 1987. Different assessors used data from descriptions at different times, so the method described above was applied separately for assessments using data from each of the three publications. Data for acute myeloid leukemia (AML) were addressed separately from total leukemias. Although Rinsky et al., as well as most of the 14 assessments, evaluated all diagnoses of leukemia and lymphoma, AML is specifically associated with benzene exposure, whereas association with other kinds of leukemia and lymphoma appears in doubt (20).
The following formula for retrospective case control evaluation was used, which assumes absence of a threshold and Haber's rule (that effect is proportional to the product of concentration and time): where: R = risk of excess deaths; 0 = observed leukemic deaths; E = exposed population; M = risk of leukemia in the control population; D = duration of exposure in years; and EX =exposure concentration, ppm.
This equation is consistent with the observation of no latency by Rinsky et al. (3). Calculations were carried out using a compiled version of dBase III, which is available on request from the authors. The program currently displays the results in both tabular and graphical form and gives geometric average and variance. Geometric mean and variance were calculated form the formulas given by Aitchison and Brown (21).

Methodological Uncertainty
The assumptions used in each of the 14 risk assessments were cataloged. The impact of the alternative assumptions were estimated as lifetime risk at 500 ppmyear, 50 ppm-year, and 5 ppm-year of occupational exposure.

Conceptual Uncertainty
A checklist of potential alternative explanations for the risk estimates was developed and each estimate was analyzed for the number of logical steps between the observations of Rinsky et al. and the phenomenon of interest to the assessor.

Measurement Uncertainty
Geometric mean (and variance in brackets) for AML cases per million persons exposed to 1 ppm for 1 year with the 1977 data was 192 [35], with the 1981 data was 56 [5] and for the 1987 data was 168 [3], given the valuation of parameters by the 14 risk assessors (17). The distributions for each of the 3 years are illustrated for total leukemias (Fig. 1) and AML cases (Fig. 2). Introduction of latency into the formula for risk estimation could alter the values for mean and variance but would have little effect on relative relationships between aggregate risks based on the three papers.

Methodological Uncertainty
The impact of uncertainty about the structure of a risk model will vary with exposure level. For example, simple substitution of a quadratic term for cumulative exposure in the retrospective case-control formula (above) results in a difference of two orders of magnitude in risk at 5 ppm-year instead of 500 ppm-year (0.0004 instead of 0.03), using the 1981 data. This range is of approximately the same two orders of magnitude as the range of risk estimates that uncertainty in data yields for the 1981 data (17). However, as Figures 1 and 2 illustrate, the 1987 data of Rinsky et al. improves measurement uncertainty by about an order of magnitude, whereas the effect of a quadratic term on risk magnitude remains the same. Further, extrapolation to environmental levels of benzene (5-0.5 ppb) results in five to six orders of magnitude difference in risk if a quadratic term is used. Use of a quadratic term instead of a linear term constitutes a minimal change in methodological assumptions. It was not necessary to make more radical assumptions about structure of the risk model to show that methodological uncertainty dominates measurement uncertainty under all but a few, constricted conditions.

Conceptual Uncertainty
Rinsky et al. directly observed the end point of interest, cases of human leukemia in relation to exposure. No variables intervene between the physical characteristics of risk and the observations. Even for dose, extrapolation is not necessary for some occupational situations. Confounding factors (such as sources of benzene exposure other than inhalation, systematic errors in exposure measurement, joint effects of other substances with benzene, indirect effects of benzene exposure through other effects on hematologic status, correlated exposure of benzene with other substances, skew in population age or chance occurrence) could hypothetically explain the association seen among the Ohio workers. However, an explanation of AML causation that is completely independent of benzene exposure is very unlikely, based on observations of the same association in other studies (20).

Discussion
Analysis of risk, based on an epidemiology study, potentially supports a wide range of risk estimates. If the probability of a conceptual error is high, then risk estimates will be subject to catastrophic change. However, AML in the studies of Ohio workers probably was caused by benzene exposure. The same association has been observed in studies of different populations. Most potential confounding factors would only modify the magnitude of benzene potency, as Rinsky, et al. observed it, but not negate the association. For example, AML incidence increases in rate with age, and AML cases occurred at an average age of 63.5 years in this study. As the cohort ages, the number of AML cases should rise, and incidence should change. In this context the effect of benzene can be viewed as an incremental change in risk above background.
The traditional exhortation to describe assumptions in a risk assessment is not adequate to address the conceptual source of uncertainty. Most assumptions relate to structure of the risk model, and choice of an appropriate model traditionally has preoccupied risk assessors (22). Disagreements about appropriate values for parameters derived from a study for a risk model (for example, the number of observed cases, population at risk, exposure levels, duration of exposure, latency of effect and/or ap-propriate control group for comparison) usually do not obtain much attention. This paper suggests that for benzene the preoccupation is justified. Methodological uncertainty constitutes the major source of uncertainty in benzene risk estimates.
Where assessors have studied the relationships between benzene exposure and risk in the Ohio cohort, the dose-response relationship was nonlinear (3,6,14). In animal studies, intermittent exposure to high concentrations of benzene also creates greater risk than continuous exposure, and current understanding of the effect of benzene on synchronization of hematopoeitic cells does not suggest a linear dose-response relationship (23). Yet, merely substituting a quadratic term for exposure into the risk formula results in changes in risk much larger than changes seen with data variation in the same cohort. However, as this paper demonstrates, the contribution of measurement uncertainty to risk uncertainty can be described.
We previously proposed that risk assessors state each component of an epidemiology study as a probability distribution, instead of as a point value, to estimate measurement uncertainty better (17). To  to the extent that fit to the data leads to model acceptance or rejection. Another advantage of estimating measurement uncertainty is to demonstrate some of the value of research. In the studies by Rinsky, et al., sustained effort from 1979 to 1987 led to a decrease of about two orders of magnitude in the range of risk estimates consistent with the data (a decrease of one order of magnitude in variance of both risk increase and decrease). A focus on methodological uncertainty also might impact research. The assumption of a linear dose-response relationship can influence data gathering, whereas concern about nonlinearity may improve the evaluation of critical parameters.