Review of the US Navy's Human Health Risk Assessment of the Naval Air Facility at Atsugi, Japan (2001)

In conclusion, NEHC has collected a large amount of sampling data at NAF Atsugi. If analyzed and interpreted appropriately, those data might be adequate to determine whether air pollution at NAF Atsugi poses a health risk and how much the incinerator facility contributes to that pollution. However, the analyses of the data by NEHC and its contractors are inadequate to draw conclusions about the health risks for persons residing at NAF Atsugi and about the contribution of the incinerator to those risks. In addition, NEHC interpreted some of the results of the risk assessment without taking into account the meaning and limitations of the risk-assessment process. Aspects of the analyses and interpretation of the data, not the underlying data themselves, constitute the main limitation of the risk assessment. The subcommittee has provided recommendations throughout this document to improve the NEHC risk assessment. Recommendations are given for the planning of risk assessments, determination of attributable risk, analysis of air-monitoring data, interpretation of the risk assessment, treatment of uncertainty, information gaps that should be filled, and inprovements in the presentation and organization of the NEHC draft summary report itself.

Given the aforementioned limitations of the Navy's risk assessment draft summary report, the subcommittee found that the analyses presented do not determine reliably whether military personnel and their families incur increased health risks by living at NAF Atsugi. Nor do the analyses presented reliably determine the contribution of the incinerator facility to health risks.

Some problems in the NEHC risk assessment seem to stem from a lack of planning. The problems include multiple, and sometimes conflicting, objectives and an inadequate basis for the air-sampling plan. In future risk assessments, NEHC should consider the use of independent peer reviewers—beyond the management group mentioned in Appendix B —throughout the project (including the planning stages) to evaluate objectives and proposed methods, to ensure that the project remains focused on the objectives, and to critique the final document. That would allow an overall plan to be assessed, provide feedback during project implementation, provide extensive scrutiny of the final draft summary report, and permit an evaluation of how well the objectives were met.

When planning a risk assessment, NEHC should follow the basic framework for making risk-management decisions that has been laid out for risk assessments. That framework is outlined in Risk Assessment in the Federal Government: Managing the Process (NRC 1983), Science and Judgment in Risk Assessment (NRC 1994), and Understanding Risk: Informing Decisions in a Democratic Society (NRC 1996). In Framework for Environmental Health Risk Management and Risk Assessment and Risk Management in Regulatory Decision-Making , the Presidential/Congressional Commission on Risk Assessment and Risk Management (1997a,b) discusses those concepts further. In a statement especially relevant to the planning stages of a risk assessment, the Presidential/Congressional Commission notes in Volume 1 (1997a) that the first stage of the framework for environmental-health risk management is to “define the problem and put it in context”. The recently released book Waste Incineration & Public Health (NRC 2000) discusses specifically the possible human health effects of incinerator releases, various risk-assessment approaches used for incinerators, and the requirements for such risk assessments. NEHC is directed to those reports for general guidance and frameworks for design of risk assessments like the one at NAF Atsugi.

There is no well-defined statement of objectives and no planning document that lays out the requirements for the entire risk assessment and a reasoned basis for such requirements. The lack of such items has resulted in different and sometimes inconsistent objectives among the parts of the program and a failure, in

some instances, to collect the correct data or to correctly evaluate the data collected. The objectives of some data-collection protocols seem to be different from the overall risk-assessment project, and some were changed during the course of the project. The exposure scenarios examined might also be inconsistent with the data collected. Even the overall objective of the risk assessment is not clear; NEHC should be clear that it was trying to determine whether there is a problem at NAF Atsugi, not that it was trying to show that there is a problem.

The statement of objectives in the planning stage should include a discussion of the target population for which the risk assessment is being done. Identification of the population at risk is needed to determine the appropriate exposure scenarios for the risk assessment. The main text of the NEHC report should identify the population at risk, define sensitive subpopulations that are of special concern, and describe who or what is meant by “sensitive receptors” (see Section 3.2.1; Radian 1998a). For example, are Americans living at NAF Atsugi the only relevant population, or are non-American civilians and other personnel not living at NAF Atsugi included? Some information on the target population is provided in Appendix B of the NEHC draft summary report and in the exposure-pathways analysis report (Radian 2000a), but it should be included in the main section of the NEHC report. The report should provide information on the size of the military population at NAF Atsugi, on the composition of that population (including age and percentage of women, children, and infants), and on the number of retired military and nonmilitary personnel employed on the facility. Information on the average duration of a tour at NAF Atsugi and the frequency with which the standard tour is extended would also be helpful. The NEHC risk-assessment draft summary report assumes, for some analyses, an average exposure duration of 30 years. In Appendix B, NEHC states that a 30-year timeframe was considered because the Environmental Protection Agency estimates the risk of a 30-year period for residence in one location. That justification, however, might not be applicable to the population of interest at Atsugi; for example, it is possible that no one in the target population of the risk assessment lives at Atsugi for 30 years.

Specific examples of differing objectives in the various documents and the potential effects of those sometimes conflicting objectives on the risk assessment are presented in Appendix A of this report. For its report, the subcommittee has assumed that the overall objectives of the risk assessment are the following as stated in the Pioneer (2000) draft report:

1. Estimate the potential human health risks to U.S. Navy personnel and their families and other individuals living and working on NAF Atsugi, Japan resulting from exposure to constituents of concern (COCs) in soil, ambient air, indoor air, and indoor dust.

2. Estimate the contribution of the risk attributable to emissions from the SIC.

Two screening risk assessments (NEHC 1995, 1998) were conducted for NAF Atsugi, and the air-monitoring data from them should have been used in the planning of this study to provide a better basis for risk-assessment design (for example, to determine the number of samples, type of samples, method of sampling, and analytic approach). In the current risk-assessment draft summary report, concentrations of pollutants of interest were seldom above the Fourier-transform infrared (FTIR) monitoring detection limit (NEHC 2000; p. 32). Provided that the target pollutant concentrations have not decreased, it should have been apparent from a review of the data from the two previous studies (NEHC 1995, 1998) that they would rarely exceed the FTIR detection limit. If FTIR monitoring was expected to detect pollutants more often, the question arises of whether a change in the incinerator complex resulted in pollutant concentrations lower than in the past. If that is the case (the supporting, documentation seems to indicate that contaminant concentrations were lower than expected), the report should discuss the decreases.

The NEHC draft summary report (p. 11) states that a statistical method indicated that outdoor air

sampling should be conducted every 6 days, but the subcommittee was unable to locate a discussion of any such statistical method or its application. The relevant section of the planning document (Radian 1998a; pp. 3-11 to 3-13) states only this:

Air samples will be collected approximately every six days. For each sampling event, analytical results will be obtained for VOCs [volatile organic compounds], mercury, acid gases, and PCDDs/PCDFs (dioxins/furans). Aldehydes and ketones, heavy metals, PCBs, pesticides, PM₁₀, PM_2.5, and SVOCs [semivolatile organic compounds] will be analyzed every other sampling event, or every 12 days.

[The air-sampling plan] will yield approximately 60 outdoor air samples at the elementary school for those chemicals measured during each sampling event and approximately 30 for the remaining chemicals. Such sample sizes are likely to lead to a great deal of precision in the estimated RMEs [reasonable maximum exposure].

For this comparison, either 30 or 60 upwind and either 90 or 180 downwind samples will be available. This is considered to be an adequate number of samples to make a valid comparison.

Statistical analyses, which should have been conducted using air measurements from previous years, were needed during the planning stage to determine the number and frequency of samples required if conclusions regarding the health risks and contributions of the incinerator were to be drawn. Such analyses were conducted for the soil monitoring, but not for the air monitoring.

One of the objectives of the risk assessment was to determine the health risks to people living at NAF Atsugi. Although most people are exposed primarily to indoor air, the indoor-air sampling was much less extensive than the outdoor-air sampling. For the collection of indoor-air samples, the quality-assurance and quality-control (QA-QC) plan stated the following (Radian 1998a; p. 3-9):

For this investigation, eight samples (two per quarter for one year) will be collected for each AOC [area of concern], except for the ground electronics maintenance building. Eight samples is a reasonable number to obtain an RME which is not likely to overinflate the risk estimate (see the discussion in Section 3.1.2.1 for soil sampling). The exact precision of the estimate, however, is a function of the variability among analytical results, and cannot be determined prior to data collection.

It is not clear how the decision to collect only eight samples was made. It is also not clear how collecting eight indoor-air samples could answer the question “What is the inhalation exposure risk for sensitive receptors in buildings likely to be impacted by the Jinkampo Incineration Complex?” (Radian 1998a; Section 3.2.1) or could determine the impact of the incinerator complex.

Furthermore, indoor-air samples were collected only for an 8-h period (NEHC 2000; p.17), which is not long enough to fulfill the study objectives. The only rationale provided for that sampling duration was a limitation in homes because the sampling pumps were noisy, but that limitation is not discussed in the planning documents, nor is the possibility of using quieter pumps. It was pointed out that further sampling would have been pointless for the high-volume samplers in the low-infiltration-rate locations because the samplers already sample all the available air several times over, but a rationale for the limitation in high-infiltration-rate locations is needed. In addition, it is not clear why a particular apartment was chosen for sampling and whether any consideration was given to other factors, such as smoking in the apartment. The possible effects of smoking on the indoor-air samples should be discussed. NEHC should also discuss other behavioral patterns that could affect the risk assessment, such as the proportion of time residents at NAF Atsugi spend indoors versus outdoors.