II The Committee’s Process
To undertake this study, the National Research Council established a committee to review the dose reconstruction program of the Defense Threat Reduction Agency (DTRA). To fulfill its charge, the committee reviewed 99 randomly selected dose reconstructions in some detail and supporting material in the files. The committee supplemented that set with 12 randomly selected dose assessments for occupation forces in Japan after the bombings of Hiroshima and Nagasaki. The committee also interacted with various atomic veterans’ representatives, federal agency representatives, and other interested parties.
II.A DISCUSSION OF THE COMMITTEE’S CHARGE
The scope of work of this review, as stated in Section I.A, has from the beginning been somewhat troubling to the committee. The charge required the committee to select a random sample of reconstructed doses and to determine whether they were “accurate” and were “accurately reported.” The committee was also to determine whether the historical source data upon which doses are based are, in themselves, “accurate.”
Dose reconstruction, because of its historical nature and the large gaps that often exist in available data, cannot be an “accurate” science. The best one can hope for in dose reconstruction is to specify a range of doses that is likely to encompass the true dose. In most cases, when all uncertainties have been taken into account, the dose can be represented by a distribution of possible values, which may spread out over a wide range. Therefore, in responding to its charge, the committee had to work within its understanding of the limits inherent in any
dose reconstruction process. It is important for readers of this report to understand that dose reconstruction generally is an exercise in applying subjective scientific judgment. In much the same way, the committee’s conclusions on the adequacy of the program of dose reconstruction for atomic veterans rely to a significant extent on judgment, which is founded on the nature and extent of information on dose reconstructions that was available to the committee.
Moreover, in the process of adjudicating claims, the Department of Veterans Affairs (VA), in recognition of the unavoidable uncertainty that attends historical dose reconstructions (and in the spirit of regulations that specify that the veteran should be given the benefit of the doubt), relies on specified upper bounds of doses rather than best or central estimates of dose. The committee was more concerned with whether uncertainties in estimated doses had been appropriately addressed than with exactly where on the distribution of possible values a particular actual dose lay. That was the committee’s approach to interpreting and addressing the “accuracy” issue with respect to dose reconstructions.
The question about whether doses were “accurately reported” to the veterans is much more straightforward to address, but the unavoidable uncertainties inherent in dose reconstruction are important here as well. The committee believes that uncertainties in assigned doses should be carefully explained and reported to VA for compensation purposes and also to the veterans.
With regard to whether the source data on nuclear-weapons tests used by DTRA are accurate, we interpreted this question as asking whether the historical data that have been comprehensively compiled are sufficiently accurate and complete for use in dose reconstruction. Such original source data would include the instrument-based measurements made at the time of the tests, weapons-debris data, film-badge records, and historical records of activities and movements of personnel participating in the tests. The data that dose reconstructions are based on and the uncertainties in them clearly are important in the estimation of doses and upper bounds provided by the Nuclear Test Personnel Review (NTPR) program to VA.
The committee recognizes that the intent of the scope of work was to focus its attention on the technical methods and assumptions being used in dose reconstructions, the historical data on which dose reconstructions are based, and how the results are being reported to VA and communicated to the veterans. Those questions encompass a range of issues, including specific scientific methods, judgments about scenarios of exposure, and effective communication with veterans. Therefore, the committee has taken a broad view of its scope out of necessity and its desire to do a thorough, defensible, and enduring job. The committee hopes that the process it has used will help to answer questions that have lingered for many years regarding dose reconstructions performed for the atomic veterans compensation program. The committee also hopes that its process will provide guidance for making improvements in the program as a whole.
In its statement of task, the committee was asked to issue an interim letter report with recommendations if after a year it concluded that its findings differed
from previously published and congressionally directed studies or if it found that substantial changes were required in the dose reconstruction procedures and methods. Because of the large number of existing dose reconstructions, the committee elected to use a sampling process (described below) for reviewing the dose reconstruction files. In addition, the committee established a protocol to provide consistent review of the files, and this took time to develop after the preliminary examinations of several files. At the one-year point in the study, the committee had identified specific examples of issues and concerns on the basis of reviews of a limited number of selected cases. However, the committee recognized that it needed to examine many more cases before it could determine whether the specific examples were symptomatic of pervasive problems in dose reconstruction procedures. For those reasons, the committee was not prepared to recommend changes in the program at that time.
II.B SAMPLING PROCEDURES
The first issue in sampling of dose reconstruction files was what to use as the sampling frame. Ideally, one might consider all veterans who had applied for compensation under any of the laws that cover service-connected disability related to radiation exposure in the nuclear-weapons testing program. However, the relevant records are not accessible at VA through any unified database that can be readily queried, as would have been required to construct a list to serve as a comprehensive sampling frame. A database available through the research contractor, currently JAYCOR, contains records for all veterans who requested information about their participation in the program and all veterans for whom documentation was requested by VA in connection with a compensation claim. However, most of the veterans in the JAYCOR database have not filed a claim for compensation, so it was not feasible to use that database as our sampling frame. We elected instead to use the database at Science Applications International Corporation (SAIC), which is the contractor responsible for individualized dose reconstructions and upper-bound dose determinations, as the most appropriate and feasible sampling frame. Most veterans with a record in the SAIC database were there because they had filed a claim for service-connected disability based on radiation exposure in the testing program, although the SAIC list also includes additional assessments for veterans who had not developed a radiogenic disease but simply wanted information about their radiation exposure.
Appendix B describes characteristics of each selected veteran’s case file including: the branch of service; the weapons test series (or presence in Japan); the assigned external, internal, and organ doses; the assigned upper bound for the total dose; the date of the most recent dose reconstruction; and whether the veteran filed a claim for compensation, and if so whether it was under the presumptive or the nonpresumptive regulation (see Sections III.B and III.C).
Only 72 of the 99 sampled cases involved claims for alleged radiogenic service-connected diseases.
It should be noted that some veterans who filed claims for compensation would not have been entered into SAIC’s database. There are two reasons for this. First, if a claim was filed for a disease that the VA adjudicator considered not to be radiogenic, it might be denied as having no merit, without consideration of dose. Second, if the unit-based dose of record at the time the claim was considered was determined to be very low and there was no indication in the record that the veteran had engaged in any unusual activities that might have increased the potential for radiation exposure, VA might have denied the claim on the basis of a low unit-based dose without requesting an individualized dose reconstruction.
At the time of sampling, in October 2001, the database that was provided to the committee by SAIC contained 3,725 veterans and their assigned doses. To ensure that we had adequate numbers of veterans with a high potential for significant radiation exposure, we carried out a stratified random sampling, sampling at random 66 veterans from the subset of those with an assigned dose of at least 1 rem and 33 from the larger group with a lower assigned dose. The committee thus oversampled veterans whose dose reconstructions may have required a relatively complex approach, and this offered a diverse set of examples for the committee to learn about how scenarios with potential for significant exposure were handled by the dose reconstruction analysts. The weighting also provided greater numbers of veterans who would have had a relatively high potential for radiation exposure and whose exposure may therefore have been high enough for errors in dose assignment to have influenced the compensation-adjudication process. Within each of the two dose-assignment-based strata, selection was based on computer generation of random numbers.
II.C INTERACTION WITH ATOMIC VETERANS
Throughout the course of its work, the committee interacted with DTRA, VA, and atomic veterans. That aspect of our work was important in seeking to answer questions raised and to understand the issues involved. Although interactions with DTRA and VA were essential to obtaining the information needed to fulfill its charge, it was also important to understand the concerns of the veterans, as summarized in Section I.D, and to seek information from them regarding such matters as communication related to the dose reconstruction program and the overall disposition of claims. We found the veterans eager to assist us. In particular, they seemed to be supportive of our study, interested in learning more about the dose reconstruction process, and curious about a number of questions they hoped the committee would be able to answer.
Veterans were invited on several occasions to speak to the committee at its meetings. Such interaction provided a formal, yet open exchange of ideas, ques-
tions, and responses with veterans, and in some cases spouses of deceased veterans, and proved useful to us. Early on in the project, we took advantage of an opportunity to expand the interaction.
In 2001, several members of the committee were invited to attend the annual meeting of the National Association of Atomic Veterans (NAAV) in Las Vegas, Nevada. The committee made a brief presentation to the veterans regarding the scope and status of the project, after which the veterans asked a number of questions. We carefully explained our approach in selecting records to review; selection was required under the terms of our charge to be a random sample. Nevertheless, many veterans at the meeting volunteered their records for the committee’s use and provided specific concerns that they wished to see addressed. Several members of the committee also briefly visited the NAAV headquarters in Albuquerque, New Mexico, for additional discussions with NAAV officers and to determine which of the various records maintained by NAAV might be of use for the committee’s study. NAAV also provided information about our study to veterans through its newsletter and informed veterans about how they could submit their files for the committee’s use or express a concern. The committee agreed to accept records that the veterans wished to send to the National Research Council, but only with written permission to use them. If a veteran was deceased, access to records was allowed after verification of death. Furthermore, the committee emphasized that it would continue to use its randomly selected records, and information provided by veterans would be used to supplement its work. Records sent to the committee by the veterans have been helpful in illustrating some key points, such as communication of dose reconstruction results and scenario development, that might not have been available in records maintained by the agencies and their contractors.
Appendix C lists the names of invited speakers and veterans (or spouses) who have interacted with the committee through correspondence, by providing information at meetings, or by providing their files for our use. All together, about 50 veterans and spouses of deceased veterans have interacted with the committee. The type of interaction—for example, e-mail, letter, and attendance at meetings—is also noted. We appreciate and are impressed by the efforts of NAAV and other veterans to work with us during the project, and it has been important to our efforts.
II.D INFORMATION GATHERING
The committee was initially charged with the tasks described in Section I.A and earlier in this chapter. The committee met five times in 2001 and six times in 2002. Between formal meetings, individual committee members were sent copies of files (with names and other identifiers blacked out to protect privacy) to permit detailed review of the 99 sampled dose reconstruction cases.
The committee review of the 99 selected dose reconstructions sometimes led it to records outside a veteran’s file, including supporting documentation in the SAIC files. We also examined a number of other dose reconstructions provided by veterans. Committee members reviewed some of the source data and many of the reports on which reported doses were based, including many published reports describing the generic unit dose reconstructions or methods used in dose reconstructions and many unpublished SAIC internal memoranda. Additional information was obtained in formal DTRA and VA replies to written questions from the committee (see Appendix D) and in presentations by DTRA, SAIC, JAYCOR, and VA staff at open committee meetings.
The committee had access to all the information it required during the course of the project. Most information related to the reconstruction of doses to atomic veterans is not classified, but some data on atmospheric nuclear tests require a security clearance for review. The committee had members with appropriate clearances to review the classified records, but access to classified information was not necessary, and we were able to complete our study by using the unclassified records available.
To be consistent with the policies of the National Academies, the committee conducted fact-finding activities involving outside parties in public information-gathering meetings and met in closed session only to develop committee procedures, review documents, and consider findings and recommendations. The information-gathering meetings were structured to solicit information from technical experts, the study sponsor and contractors, and veterans on topics related to the dose reconstruction program. Atomic veterans were also invited to make oral or written statements or to provide written comments to the committee.
Four committee meetings included data-gathering sessions not open to the public, and five committee meetings included information-gathering sessions open to the public. At open meetings, members of the public and veterans were given the opportunity to provide statements, ask questions, or make comments about the dose reconstruction program. A description of meetings at which the committee gathered information is included below.
On the first day of its first meeting, in open session, the committee and observers were briefed by a representative of DTRA. He introduced the charge to the committee and gave background on the NTPR dose reconstruction program. A spokesperson from JAYCOR and another from SAIC were present to address committee questions related to DTRA-contractor support of dose reconstruction. On the second day, the committee visited JAYCOR and SAIC sites to examine sample dose reconstruction records and to identify which aspects of the records would be our focus for auditing the dose reconstruction process.
The second meeting included a full-day open session. The committee heard from a congressional staff person, a representative of the General Accounting Office, the medical doctor responsible for providing medical advice related to
claims adjudication for VA, the chief of judicial and advisory review of the VA Compensation and Pension Service, the widow of an atomic veteran who also was an official of NAAV, and an atomic veteran who represented NAAV.
The third meeting included a half-day data-gathering session not open to the public. The committee visited the SAIC site to examine sample dose reconstruction records.
The fourth meeting included a full-day data-gathering session not open to the public. The committee visited the SAIC site to examine the 99 randomly selected dose reconstruction records.
The sixth meeting included a half-day information-gathering session. The committee heard presentations by a retired Navy captain (former manager of the JAYCOR-NTPR effort and chief of the Navy NTPR team) and representatives of VA and DTRA.
The ninth meeting included a full-day data-gathering session not open to the public. Three committee members visited the SAIC site to examine sample dose reconstruction records. The meeting also included a half-day information-gathering session. The committee heard presentations related to the possible use of tooth enamel for radiation dosimetry (see Section I.C.3.3). Representatives of DTRA and VA were present to answer questions related to dose reconstruction and the claims process.
All the information gathered at open meetings is part of the National Research Council’s public-access file and is available, on request, to anyone interested.