Letter Report - Issues Regarding the Hanford Environmental Dose Reconstruction (HEDR) Atmospheric I-131 Pathway Models (1999)

COMMISSION ON LIFE SCIENCES

2101 Constitution Avenue Washington, D.C. 20418

January 25, 1999

Dear Dr. Smith:

On December 18th, 1997, at the request of the Centers for Disease Control and Prevention and the National Research Council, the Committee on an Assessment of CDC Radiation Studies met in Washington to address a number of specific issues related to the validation of the Hanford Environmental Dose Reconstruction (HEDR) atmospheric I-131 pathway models. In particular, the committee was asked to respond to the following six questions:

1. How does one evaluate the validity of a set of stochastic models using measured data that contain uncertainties?

2. Should a model be calibrated on the basis of a limited number of empirical data sets, and if so, how?

3. How does one use and interpret intermediate results, e.g., vegetation concentration, when the goal of a model is to calculate doses to people? How does the answer to this question depend on the vegetation being considered, e.g., is sagebrush an appropriate surrogate for grass in a cow-milk-person pathway validation exercise?

4. How does one judge the validity of a model that is designed to estimate the impact of events that have already taken place and for which little, if any, measurement data exist?

5. How can model validation results effectively be communicated to the public?

6. Are the HEDR models “valid” for calculating a range of thyroid doses to representative people and real individuals for I-131 released to the atmosphere?

Although the National Research Council committee has been reviewing various components of the HEDR study for some time, its previous charges have not been those specifically identified above. Accordingly, prior to the meeting, the committee was provided

The National Research Council is the principal operating agency of the National Academy of Sciences and the National Academy of Engineering to serve government and other organizations

with materials to inform the committee and to focus discussion on current concerns. Those materials included the following:

1. Report PNWD-2221, HEDR, Battelle Pacific Northwest Laboratories, Richland, Washington, entitled “Validation of HEDR Models;”

2. A Transcript of Breakout Session III, CDC Technical Workshop, “Issues Relating to Estimating Doses Due to I-131 Releases to the Atmosphere from the Hanford Nuclear Reservation,” Tacoma, Washington, August 14, 1997; and

3. A letter report from B. A. Napier to the Centers for Disease Control and Prevention including a copy of the updated BIOMASS Hanford Scenario and a report relating to a BIOMASS meeting in Mol, Belgium.

During the December 18^th meeting of the committee, Dr. Bruce Napier of the Battelle Northwest Laboratory briefly reviewed the history of the HEDR Project and then addressed the steps that have been taken to validate the models employed in the project.

Subsequent to its December meeting, the committee was provided with several additional documents by representatives of litigants in a class action law suit on behalf of persons who lived or have lived near the Hanford site, specifically the following:

1. A preliminary report prepared by B. Hermann and F. J. Hermann on behalf of the plaintiffs in the Hanford litigation entitled “Iodine-131 Release to Atmosphere from Hanford Separations Plants for the period May 1948 through December 1960” (dated March 10, 1996);

2. A report (and a supplement thereto) prepared by Robert Goble on behalf of the plaintiffs in the Hanford litigation entitled “Estimating Exposures from Releases of Radioactive Iodine at Hanford and Implications for Assessing the Significance of these Exposures in Causing Disease ” (dated November 14, 1995, and March 31, 1996);

3. A report by Thomas A. Cochran entitled “Errors in the Source Term of the Hanford Environmental Dose Reconstruction ” (dated November 12, 1995, and revised on March 19, 1996); and

4. A second report prepared by Cochran entitled “Calibration of the Hanford Environmental Dose Reconstruction using Vegetation Data: A Revised Report” (dated March 28, 1996).

Those reports challenge specific aspects of the Hanford Environmental Dose Reconstruction, with particular criticism of the discrepancy between the vegetation measurements (principally sagebrush) and the predictions of the HEDR model.

Since the meaning of the expression “model validation” can be differently construed, in the paragraphs to follow the committee sets out its use of this term to avoid ambiguity:

“Model validation” is the procedure used to assess the appropriateness of applying a given model to the problem under consideration. This implies determining if the processes described, the parameters chosen, and the stage-by-stage sequence of events are fully representative of the particular scenario and target conditions under consideration. All of the input data, parameters employed and possible feedback conditions introduce uncertainties that affect the numerical outcome of all model calculations. Validation of the model, therefore, involves three stages:

• Inspections of the program to ensure that the assumptions incorporated in the model and the parameters selected are appropriate to the process or operation being modeled.

• Comparison of calculated values for various intermediate stages for specific conditions with measurements of the same quantities. For a valid comparison, the conditions of measurement should approximate as closely as possible the assumed conditions in the model. Validation of the model then implies acceptable agreement between predicted and measured values within the uncertainty ranges appropriate to both.

• Reassessment of the model in the event of a disagreement between observed and predicted values to identify any structural element or parameter that may account for the discrepancy. This may require some sensitivity analysis.

The HEDR dose calculations consist of a number of sequential stages (source release, environmental dispersion, deposition, uptake and conversion, incorporation in man, etc.). Each of these stages incorporates uncertainties and, ideally, should be validated independently. The concentration of I-131 in vegetation, for instance, tests only the stages of the model that simulate processes that result in those concentrations, such as sources, dispersion, deposition, and vegetative uptake; dose to people would require reliance on satisfactory performance of the subsequent portions of the model.

Issue 1: How does one evaluate the validity of a set of stochastic models using measured data that contain uncertainties?

Statistical methods have been proposed to provide validation of models when there is no definitive outcome measure, but rather only outcome measures with uncertainty. However, these statistical methods generally require multiple outcome measures to be available in the same experimental units (in the present context, this would mean having multiple types of exposure measures collected at the same times and locations). This does not correspond to the validation measurement data available in the HEDR study, which were collected at a variety of times and locations and which may reference different parts of the model (e.g., stack measurements to validate the source term vs. vegetation measurements that also incorporate environmental dispersion, and deposition). Hence, it would appear difficult or impossible to develop a formal, comprehensive statistical validation of the set of HEDR models. The primary recourse,

therefore, is to assemble the various validation substudies that were conducted and form a scientific judgment as to whether there seems to be a systematic trend toward under- or overestimation of exposure levels by the models. If no systematic trends are observed, this constitutes reasonable albeit not very precise evidence for the validity of the models—reasonable because the measurement data come from several independent sources, but imprecise because of the small numbers of measurements. Sometimes stages of a model can be validated with data from other studies provided the processes being described are comparable. For example, data on the uptake of I-131 by plants might be used to validate the part of the model that is used to calculate plant uptake of I-131 from Hanford if such data were available. Several of the model components used in the environmental pathway models have been at least partially validated by data from studies of fallout from weapons tests, Chernobyl, and other dose reconstructions.

Issue 2: Should a model be calibrated on the basis of a limited number of empirical data sets, and if so, how?

To avoid confusion, the committee observes that validation and calibration are two different things. In a model validation, model results are compared with experimental results (which were not used in the development of the model), in order merely to judge the quality of the model. Preferably these experimental results will encompass the whole range of exposures and dose levels, weather conditions, population groups, etc., that the model will be used to represent. In practice the number of empirical data sets available will be severely limited and any reviewer has to use his or her judgment as to the extent the model has been validated. Model calibration, on the other hand, means that the model or its parameter values are changed to better represent the experimental data available for the construction of a part of the model. If a model preponderantly over predicts or under predicts, it should obviously be improved, preferably by modification of the parameter values as a result of further research. However, calibration should not be simply curve-fitting such that the model components and parameters are no longer consistent with the underlying science.

Issue 3: How does one use and interpret intermediate results, e.g., vegetation concentration, when the goal of the model is to calculate doses to people? How does the answer to this question depend on the vegetation being considered, e.g., is sagebrush an appropriate surrogate for grass in a cow-milk-person pathway validation exercise?

The important deposition site for I-131 in this study is ground-cover used for grazing by milk-producing cattle. Sagebrush does not generally fall into this category and use of deposition data obtained from this source could be misleading. Thus, data obtained from pasture grass, even though very limited, are more suitable for model validation than a larger amount of data from inappropriate deposition sites, such as sagebrush. Estimates of radiation dose to the individual based upon his or her lifestyle (diet, milk and milk-product consumption, age at exposure, etc.) are derived from the amount (concentration) of deposition on grass-lands. However, the uncertainties attached to the estimation of some of those “lifestyle” factors might be much greater than those involved in I-131 deposition.

The committee finds the discussion of the validation study difficult to follow. The Battelle investigators have noted in a personal communication to Robert Goble in 1996 that in their validation report (Napier 1994) dry-weight predictions had been erroneously compared with measurements made on a wet-weight basis. If this error is corrected, which the Battelle workers did not do, the discrepancy between the predictions and the measurements becomes substantial.

In a letter to Joe Wayman of the Hanford Litigation Office dated April 14, 1997, Napier notes a further error in the original calculations, namely, in the conversion factor to obtain concentrations from counts. Correction of this error tends to increase the measured concentrations by a factor of about 2.5. However, in this letter, the error presented pertains only to the data on I-131 in vegetation for the Green Run, but not for the earlier data when measurements were of gross beta. Thus, the committee finds the account of the validation study murky and difficult to follow, which precludes drawing reliable conclusions from its findings.

Issue 4: How does one judge the validity of a model that is designed to estimate the impact of events that have already taken place and for which little, if any, measurement data exist?

Estimating the impact of events such as the Hanford iodine releases that occurred decades ago is a daunting task, one that is very dependent on the historical record and the memory of individuals. Such estimates are dependent on knowledge of the release, the environmental pathway through which contamination occurred, and biological factors that ultimately lead to a radiation dose to humans (biological factors are widely variable among individuals and greatly influenced by age, sex, habits, etc.). One of the more important parameters in an environmental assessment is the source term, a value that is (in the case of Hanford) thought to be well known (within 10%) even for the 1940s and 1950s, because it is based on measured data that are presumably reliable since the number of reactors and their operating histories, and the amount of plutonium they produced, are matters of public record. Information on the amount of iodine released (source term) from Hanford is well known because of the detailed records on the number of fissions that occurred for each fuel batch (as established by inlet and outlet temperature measurements on each fuel channel). These data combined with fuel cooling times yielded reliable determinations of the releases of I-131 because, at least during the early days, all of the iodine was released due to its volatility during the fuel processing step. It appears that the correct source term amounts were actually used in the calculations implemented by the HEDR staff but that the published report on the inventory (Heeb and Bates 1994)¹ contains flaws. If the committee's view is correct, it would be helpful if the HEDR staff would acknowledge publicly the problem with the source-term report and publish a corrected version.

Issue 5: How can model validation results effectively be communicated to the public?

Model validation is a technical concept that is difficult to communicate effectively to the public. This stems, in part, from the fact that the definition of validity varies greatly among scientists as well as among members of the public. As previously noted, one definition of validation would be an acceptable agreement between predicted and measured values within the uncertainty ranges appropriate to both. Another, broader definition of validity is that the data produced by the model are useful for the purposes for which the model was developed.

Irrespective of the definition of validity, model validation results must be communicated to the public in a manner that is non-mathematical and easily understandable. Demonstrations of the HEDR model's ability to predict the measured results of known releases, e.g., the Green Run, could help members of the public appreciate the strengths and weaknesses of the model. If there are limited data available to validate a model, adequate model validation may be impossible, and if this is likely, this fact needs to be openly communicated to the public. Similarly, if adequate model validation is impossible, it should be clearly communicated that the dose determinations are at best approximations and could be seriously misleading.

Theoretically, model predictions should be calculated with confidence intervals, but even when this is possible the determination of validity is apt to remain subjective. When the quality and quantity of actual measurements are limited, a model can never be fully validated, only invalidated. Extreme disagreement is obvious enough, but validity is a matter of judgment when a model is right some of the time and wrong at other times. The reasons for such deviations are seldom obvious, and their importance depends on the resulting degree of agreement or disagreement. A model might be accepted by some individuals, despite considerable uncertainty, if all reasonable steps have been taken to reduce uncertainty. However, other individuals may be unwilling to accept the large uncertainty that is inevitably associated with complex dose reconstruction and will reject any model.

Issue 6: Are the HEDR models “valid” for calculating a range of thyroid doses to representative people and real individuals for I-131 released to the atmosphere?

At this stage the HEDR model can be considered satisfactory for the purpose of setting bounds on the potential health impact of Hanford releases on the surrounding population. It has been tested against the rather limited sets of environmental measurements that are available and, given the diverse nature of these measurements, the agreement can be considered adequate. Validation with more measurements of a more consistent type over a period of years would have been desirable, but under the circumstances the committee is satisfied with the structure of the model as it stands. Nonetheless, the HEDR models have been shown to yield reasonable estimates of a range of thyroid doses to representative persons having a certain lifestyle, age, sex or physiology in exposed populations. However, these doses do not necessarily accurately represent the actual dose to a specific individual. Calculation of individual doses is important to support the studies of thyroid effects in the Hanford Thyroid Disease Study—an epidemiologic study—and can be useful in helping individuals to understand better the magnitude of their potential risk.

It is important to recognize that the models incorporate a range of uncertainties in establishing the representative thyroid dose estimates. In translating these representative doses to individual dose estimates, further uncertainties are incorporated (e.g., milk and milk-product consumption, source of milk, time of processing and distribution, etc.) that can substantially affect thyroid dose. Thus, interpretation of dose estimates for selected individuals must be tempered with the recognition that these doses are more imprecise than the representative doses and will provide only a very general idea of a specific individual's risk rather than the exact value. As a result, it should be noted that the inherent uncertainty associated with the individual doses will decrease the likelihood of determining a meaningful risk coefficient for the effects of radioiodine on the target population.

The committee notes that the public wants to know what their past, present and future somatic risks are. Specifically, they want to know what their general health risks may be, and what specific cancers they may have to confront. As they are aware that these risks are functionally dependent upon the doses received, they would like to have the means to calculate their doses if so motivated.

Another factor that must be included in estimating specific individual risk is the uncertainty associated with thyroid cancer risk itself. The public should be made aware that even though the relative risk of developing thyroid cancer might be quite large, the absolute risk is small. The implications of individual thyroid doses should be interpreted in comparison to a “background” thyroid cancer cumulative incidence of about 0.5% per lifetime. This incidence is low on a per year basis, about 10 cases per 100,000 persons per year. If the uncertainty around this value is such as to give it limits of 20% (8-12 cases per 100,000 persons per year), then the individual doses and the associated increase in their related cancer incidences over background should be interpreted in terms of the known variability in background rates. Although there is variability in the HEDR doses, there is also considerable variability in the incidence factors that needs to be considered by individuals. Thus, the large combined uncertainty in individual doses decreases the usefulness of any attempt to screen the population for thyroid disease since it is difficult to credibly identify high-risk individuals.

Conclusion: At this stage, while the discrepancies between predictions of the HEDR model and certain environmental measurements might affect the accuracy of the HEDR model and the doses derived therefrom, the committee believes the HEDR model is structurally sound. However, validation studies are not as reliable as desired. Accordingly, the committee recommends that the HEDR investigators supplement their description of the model with an account of the origin of the errors made with regard to the estimation of the I-131 concentrations in pasture grass on the basis of measurements, the impact on the predicted values when the errors are corrected, and a preliminary assessment of the effect of a reparametrization on estimates of absorbed dose to the thyroid. Until such a reassessment has been made it is difficult to know

whether the current estimates of dose to representative individuals need recalculation. The committee further suggests that in this reassessment the issue of uncertainty be revisited to clarify sources of potential error that may have been missed. In this regard, a better defense of the plume trajectories, the interface between the plume and milk-producing vegetation, and the dietary patterns of milk and milk-product consumers would be most useful.

What is needed is a serious effort by the HEDR team to resolve the issues that have been raised and to issue a formal report that documents the findings and recommendations. As a minimum, the HEDR team should publish a corrected source-term report, publish a document that explains the errors made in the validation study and presents the corrected data, and produce the uncertainty report in a more serious and scientifically justified manner.

I trust that you will not hesitate to contact me or Dr. Evan Douple if you have questions regarding any of the committee's views or recommendations.