Between 1963 and 1969, the U.S. military carried out a series of tests, termed Project SHAD (Shipboard Hazard and Defense), to evaluate the vulnerabilities of U.S. Navy ships to chemical and biological warfare agents. These tests involved use of active chemical and biological agents, simulants, tracers, and decontaminants. Approximately 5,900 military personnel, primarily from the Navy and Marine Corps, are reported to have been included in Project SHAD testing. At the time they were conducted, virtually all aspects of the Project 112/SHAD tests were assigned a security classification of Secret or Top Secret.
In the 1990s some veterans who participated in the SHAD tests expressed concerns to the Department of Veterans Affairs (VA) that their health problems might be the result of exposures in the testing. Congress and VA requested information from the Department of Defense (DoD) to clarify what substances veterans may have been exposed to and when the tests had taken place. In 2002 and 2003 DoD publicly released fact sheets that described each test, identified the participating military units, and named the chemical and biological agents, simulants, decontaminants, and tracers used, but many details about the tests remain classified.
Studies by the Institute of Medicine (IOM) and VA researchers, published in 2007-2009, produced complex results. The IOM study found no difference in all-cause mortality between SHAD veterans and the comparison group, but there was an increased risk of death from heart disease among some SHAD veterans and responses to a 2004 health survey indicated poorer overall physical and mental health among the SHAD participants. The VA study found an increase in all-cause mortality among the SHAD veterans that was due primarily to heart disease.
In response to continuing concerns, Congress in 2010 requested an additional IOM study (SHAD II). The SHAD II study, reported on here, expanded on the previous IOM work by making use of additional years of follow up and some analysis of diagnostic data from Medicare and the VA health care system. In finding no overall differences in all-cause mortality between SHAD participants and comparison groups, the SHAD II analysis agreed with the results of the previous IOM study. However, it differed in that, with an additional 7 years of follow up, it did not find an elevation in heart disease mortality. Indications of an increased risk of heart disease in the crew of the USS George Eastman, one of the exposure groups examined, did not attain statistical significance after adjustment for the multiple comparisons carried out in the analysis. The SHAD II analysis found no clear differences in degree of illness between SHAD participants and the comparison group. This result was consistent with the absence of differences in self-reported hospitalizations noted in SHAD I, but not in accord with the SHAD I survey responses that indicated overall worse health among SHAD veterans.
THE SHAD II STUDY
The Caregivers and Veterans Omnibus Health Services Act of 2010 (PL 111-163) requested that the IOM conduct a new epidemiological study of the potential long-term health effects of participation in the SHAD testing. The charge to the Committee on Shipboard Hazard and Defense (SHAD) II appears in Box S-1.
The intent of Project SHAD was to evaluate operational characteristics of ships and protective and dissemination equipment as well as the behavior of test agents in marine environments. Although exposure samples (e.g., gargle samples or chemical dosimeters on clothing) were obtained from the men present during a few tests, the purpose of the SHAD tests was not study the health of the participants.
The SHAD tests involved use of active chemical and biological agents, simulants, tracers, and decontaminants. The active chemical and biological agents were potential weapons and known to be debilitating or even fatal in the short term to unprotected personnel. The simulants were substances that shared certain chemical or biological characteristics of the active agents but were considered at the time to be relatively safe to use. For example, Bacillus globigii (BG) has been used for many years as a simulant for the aerosol behavior of Bacillus anthracis, which causes the disease anthrax. Tracers were substances used to track the movement of test agents or simulants.
Approach to the Study
The committee’s interest was in evaluating whether the long-term health of SHAD veterans has been adversely affected by their exposures during SHAD testing. Specifically, the committee sought to determine (1) whether participation in SHAD tests that used certain agents, simulants, tracers, or decontaminants is associated with differences in specific long-term health effects, as manifested in greater risk of mortality or morbidity (hypothesis-based analysis); and
An ad hoc committee of experts in conjunction with the Institute of Medicine’s (IOM’s) Medical Follow-up Agency (MFUA) staff will conduct an epidemiological study comparing the health status of the SHAD veterans with a comparison population. This study will build on knowledge gained from a prior study (Long-Term Health Effects of Participation in Project SHAD) conducted by MFUA staff between 2003 and 2007. The SHAD II study will use the established SHAD participants list and the comparison population list of individuals determined from the prior study.
As part of the data collection process, the committee will plan and conduct an initial workshop to receive suggestions and input from SHAD veterans about their experiences so that the study can be informed by new insights from this group.
Following the completion of the study, the committee will author an IOM report summarizing the approach and findings of this study.
If the results of this SHAD II study indicate a difference in outcome between the SHAD participants and the comparison population, or if the study finds new information that would warrant new research avenues, MFUA will be prepared to conduct additional research as appropriate.
(2) whether Project SHAD participants were at increased risk of mortality or morbidity across a broad range of standard diagnostic categories (exploratory analysis) compared with veterans who served on similar ships or in similar units that did not participate in the SHAD tests. “Long-term” was taken to mean persisting or recurring conditions as well as those appearing several years or more after exposure.
The SHAD II study was a retrospective cohort design. The study population consisted of men identified by DoD from ship logs and personnel diaries as having been present during some portion of at least one SHAD test. To assemble the comparison population, ships and Marine Corps units that participated in SHAD tests were matched, to the extent possible, with a ship (or unit) of a similar class, crew complement, operating area, and home port at the time of the SHAD test. The crew members of the five light tug boats (9-12 men per boat) could be identified for only one of the several tests that they participated in, and no comparison vessels could be named. Identification of the study population is described in greater detail in Chapters 2 and 3, and a list of all ships and units from which the study population was drawn is provided in Appendix E.
Generating Hypotheses Concerning SHAD Exposures
Using existing summaries of older literature and a review of the scientific literature published in 2000 or later, the committee examined information on the potential health effects from exposure to the substances used in the SHAD tests. On the basis of this review, the committee generated hypotheses about health outcomes that might be associated with six of the test substances (see Table S-1) and used them as a way to focus portions of its analysis.
TABLE S-1 Hypotheses to Be Tested Concerning Certain Substances Used in SHAD Testing and Adverse Health Outcomes
|Substance||Use in SHAD Test||Health Outcome Hypothesized|
|Coxiella burnetii||Biological agent||Chronic hepatitis|
|Escherichia coli||Biological simulant||Irritable bowel syndrome|
Staphylococcal enterotoxin type Ba
|Sarin||Chemical agent||Neurological effects (central nervous system)|
|Neurological effects (peripheral nervous system)|
|Neurological effects (hearing loss)|
|Betapropiolactone||Decontaminant||Cancer (any type)|
|Zinc cadmium sulfide||Tracer||Chronic kidney disease|
a Because the individuals who served on the vessels involved in testing staphylococcal enterotoxin type B were not identified, the committee could not test the hypotheses generated for this agent.
The nature of SHAD veterans’ potential exposures vary. The SHAD tests involved at least 19 different vessels and seven Marine Corps units in 21 tests that used at least 16 different test substances (alone or in combination) over a period of 7 years. Each test included multiple trials (ranging from as few as 5 to as many as 31), and the participating ships and test substances could vary from trial to trial within a test. For ships that participated in multiple tests, crew members could change from test to test. Approximately 69 percent of the SHAD participants (4,050 men) were in only one test. The remainder participated in from two to seven tests.
Substance-specific exposure groups were created where possible for the substances for which hypotheses regarding health outcomes were formulated. Also created were three broad exposure groups based on the substances used in the SHAD tests: (1) any biological substance; (2) any chemical substance, except trioctyl phosphate (TOF); and (3) any decontaminant. These exposure groups are not mutually exclusive; a member of the study population might be included in any or all of them.
Analyses were also conducted of three subgroups with distinctive experiences: (1) the crew of the USS George Eastman, a specially outfitted ship involved in multiple tests, including some with live agents; (2) members of the Project SHAD technical staff, including persons with service on the light tugs, who were identified in a roster from January 1965; and (3) participants in Test 69-10, who received unique exposure to TOF. The membership of these groups was largely mutually exclusive.
Special consideration was also given to the crew of the USS Granville S. Hall; which served as the laboratory ship for many SHAD tests. The ship’s regular crew was supplemented by identified members of the Project SHAD technical staff. For overall analyses, the committee included all Granville S. Hall crew members as part of the Project SHAD population. For analyses that addressed specific forms of exposure, the Project SHAD technical staff was considered as potentially exposed and the members of the regular crew were excluded as unexposed.
The comparison group for each of the exposure groups was determined on the basis of the tests in which the exposures occurred, the units in the test that received the specified exposure, and the designated comparison unit for that test.
Data were compiled on cumulative numbers of exposure events for each of the test substances for each man identified as being present during the test period. For example, a person present on a ship for nine trials in one test and for three trials in another test that used the same substance would be considered to have 12 exposures (9 + 3 = 12) to that substance. Where the size of the exposed group and the variation in the number of potential exposures were considered sufficient, the committee established subgroups to reflect the range of exposure experience (e.g., low, medium, high) among the SHAD participants. Appendix D includes a description of the process used to evaluate the distribution of exposure opportunities and determine the exposure subgroups.
Data on Health Outcomes
The committee evaluated the causes of deaths in the study population since the 1960s and the diagnoses assigned at hospital and outpatient visits in data from Medicare from 1999 through 2011 and from the VA health system from late 1997 through 2011.
Mortality follow-up for SHAD participants began at the date of their first test exposure, and ended on the date of death or December 31, 2011, whichever came first. In addition to the members of the study population known to be dead at the time of the first IOM study, additional deaths were determined through automated matching with records of the Veterans Benefits Administration and records available through Lexis/Nexis. Data on cause of death were obtained from the National Death Index or from death certificates obtained from state vital statistics offices for 91 percent of the members of the study population who were identified as deceased. The underlying cause of death was assigned a code corresponding to the International Classification of Diseases (ICD, 9th or 10th revisions).
After leaving military service, some veterans receive medical care from VA facilities, many receive care from clinicians and facilities in the community, and some may receive care from both sources. On reaching age 65, almost all veterans become eligible for Medicare, but some may continue to receive some or all of their care through VA or private insurance. The SHAD II study assessed morbidity through records collected into automated databases as a byproduct of medical care received through VA or under the auspices of Medicare billing. As of December 31, 2011, 83 percent of the study participants assumed alive were 65 years of age or older.
The VA data were derived from multiple datasets held by the Veterans Health Administration (VHA), including Inpatient Encounters, Bed section acute care dataset, Main acute care dataset, Procedure acute care dataset, Surgery acute care dataset, and Event dataset. These files captured overlapping information on inpatient and outpatient encounters.
Medicare enrollment information is provided in the Master Beneficiary Summary File (formerly the Denominator File). Data on diagnoses were derived from the Medicare Provider Analysis and Review (MedPAR) file (hospital stays) and the Outpatient and Carrier files (services provided in non-inpatient facilities and care by physicians). The Medicare data do not include information about care provided to Medicare Managed Care enrollees or care provided outside of the Medicare program. Diagnostic data are available only for conditions for which diagnoses have been recorded as part of a health care claim, and some conditions that could have been diagnosed are not recorded.
THE STUDY POPULATION
The population for this study was made up of 5,868 Project SHAD participants and 6,753 other veterans who served as the comparison group. Approximately 60 percent of both groups were born in 1937-1946, and about 18 to 19 percent born earlier and 15 to 16 percent born later. Date of birth was unknown for 5.4 percent of SHAD veterans and 8.4 percent of the comparison
population. The proportion of personnel who were white was 87 to 88 percent of those whose race was known in participant and comparison groups. In both groups, the majority served in the Navy (86 percent of SHAD veterans and 91 percent of the comparison group) and were enlisted personnel (92 and 93 percent, respectively). Similar proportions of the two groups had evidence from a VA disability application of service in Vietnam or presumed or documented exposure to Agent Orange. The similarity of the two groups on these factors offered some assurance that the two populations were reasonably comparable.
FINDINGS ON MORTALITY
As of December 31, 2011, approximately 30 percent of both the SHAD veterans and the comparison population had died. Kaplan-Meier survival analysis showed no evidence of statistically significant difference between the participant and comparison groups in the overall number or timing of deaths. When cause of death, grouped by broad ICD categories, was examined using a Cox proportional hazards model that adjusted for age, Project SHAD veterans showed no statistically significant increases in hazard over the comparison population.
Testing Hypotheses on Exposures
In analyses limited to the SHAD veterans exposed to the five test substances for which it was possible to test the committee’s hypotheses regarding health outcomes, no statistically significant differences were seen in all-cause mortality. This held true as well when the enlisted and officer groups were examined separately.
In analyses that examined the specific causes of death identified in the committee’s hypotheses, proportional hazards analysis that included adjustment for age and rank found no statistically significant excess risk among SHAD participants for the two exposures (betapropriolactone and zinc cadmium sulfide) for which there were sufficient numbers of cases to pursue the analysis. No significant differences emerged when the enlisted group was examined separately or when the analysis took into account whether the SHAD veterans had a greater or lesser number of exposures to these substances.
Exploratory Analysis in Subgroups of Special Interest
All-cause mortality among the members of the groups of special interest was not significantly different from that among their comparison groups, and stratifying the groups on the basis of rank (officer and enlisted) resulted in no statistically significant differences. Members of the tug and Project SHAD technical staff subgroup had higher crude mortality when compared to the other members of the crew of the USS Granville S. Hall or to the crew of the comparison vessel for the Granville S. Hall (external control), but this difference was accounted for by the older age of this subgroup of SHAD veterans.
Number of Potential Exposures
For three groups—those exposed to any biological substance, to any chemical substance except TOF, or to any decontaminant—it was possible to assess whether higher or lower
numbers of potential exposures influenced mortality risks. Using two or three levels of exposure, depending upon the numbers of people exposed and the distribution of the exposure counts across the group, no significant differences emerged at any level of exposure. This type of dose–response analysis was not feasible for the other exposure groups because of the limited variation within the group in the number of exposure opportunities.
When mortality attributed to specific causes was examined, an increase in risk among SHAD participants was seen only in the crew of the USS George Eastman, for heart disease mortality. However, when the committee applied a statistical adjustment to take into account the multiple comparisons in the analysis, the result no longer attained statistical significance.
FINDINGS ON MORBIDITY
Diagnoses noted in records generated as a result of medical care received under the auspices of the Medicare program or through VA were the basis for the assessment of morbidity in the study population. Approximately 55 percent of the SHAD veterans and 52 percent of the comparison population had ever been enrolled in the fee-for-service form of Medicare. Although comparable “enrollment” data are not available for VA health care, the committee found that 40 percent of SHAD veterans and 33 percent of the comparison population had indications of use of inpatient or outpatient care through VA, a statistically significant difference. An increase in use of VA health care among Project SHAD veterans relative to the comparison population was observed beginning in 2002, which is when VA began notifying SHAD veterans of their participation in the previously secret SHAD testing. By 2005, usage of VA health services among SHAD veterans remained higher than that of the comparison population, but the rate of increase in use from year to year became similar in the two groups.
Overall morbidity was evaluated using median hospital days per person-year of enrollment among Medicare enrollees hospitalized during the study period. Roughly 42 percent of both Project SHAD participant and comparison groups who had been enrolled in Medicare during the study period had a hospitalization during that time. Median hospital days per person-year of enrollment among those hospitalized was slightly lower among Project SHAD participants than the comparison group, but the difference was not statistically significant. Looking at broad categories of disease showed that circulatory disease was the most common diagnosis in the Medicare data in both groups, with approximately 51 percent in each group having such a diagnosis. The proportions receiving diagnoses of other types were also comparable in the two groups. Again in data from VA, the percentages of diagnoses in various categories were not higher in the SHAD veteran group than in the comparison group.
Testing Hypotheses on Exposures
The committee focused its morbidity analysis on the members of the study population with exposure to the substances for which hypotheses regarding health outcomes were developed. Of those enrolled in Medicare, 41 to 53 percent were hospitalized during the study
period. Among those hospitalized, median hospital days per person-year of enrollment were not statistically higher in the SHAD veterans in the exposure groups than in their comparison groups. The results of cause-specific analysis also showed no statistically significant increases in the specific causes of morbidity identified in the committee’s hypotheses.
Testimony and other information from SHAD veterans was essential in helping the committee gain a better understanding of the context of the SHAD tests and the experience of those who had been present. The study also benefited from the efforts during the previous IOM study to establish and validate the cohort of Project SHAD test participants from ship logs and diaries. The previous IOM study also provided the unique benefit of a comparison population of military personnel with service more comparable in time and character to that of the SHAD veterans than would typically be found. With nearly 50 years of follow-up since the first SHAD test, the study had an opportunity to assess the mortality experience of the study population across a broad range of causes of death, including those that appear later in life. Similarly, the long follow-up period made it feasible to turn to Medicare data to examine morbidity.
But the study also faced important challenges. The time that has elapsed since the tests took place in the 1960s makes it difficult to account for all the factors beyond the SHAD tests that will have influenced the health of the veterans who participated in the tests. The evidence from VA data that similar proportions of the SHAD veterans and the comparison population appeared to have had service in Vietnam and possible exposure to Agent Orange provides some reassurance that those factors did not have a marked influence on the analysis.
The study was also constrained by the limits of the available documentation. Information on the concentrations of test substances and results from sampling stations largely remains classified. Having access to the classified material in Project SHAD reports might have informed the committee regarding the range of potential exposures for a given unit in a test. However, because the location of individual crew members during tests was also not available, the committee still could not have estimated doses at the individual level, and the statistical analyses would remain those of comparing exposure groups. As a result, the committee does not expect that having the classified information would alter the findings of the analysis. Rosters listing the crews of the tugs were not available except for one early test, preventing testing of one of the a priori hypotheses developed by the committee. The committee also lacked data for explicit exploration of the concerns of certain SHAD veterans about exposure to vaccines against biological warfare agents or exposure to residual radiation from nuclear tests previously conducted in areas of the Pacific Ocean where SHAD tests were also conducted.
Analytical challenges arose from the complexity of the data. While the Medicare data for enrollees over age 65 can be interpreted as providing population-based measures, data from VA are harder to interpret because of the variable criteria for eligibility for and decisions to use these services. But at no time was the entire Project SHAD participant cohort eligible for Medicare. Thus the committee’s analysis had to focus on the morbidity of the oldest veterans, who were also the oldest at the time of the testing.
The committee’s use of hypotheses generated from the literature provided some focus for the analyses, but in assessing the health status of SHAD participants and the comparison groups for various categories of exposures the committee carried out many statistical tests, increasing the likelihood that statistically significant associations could arise by chance. In some cases the
exposure groups were of modest size, which could make it difficult to ascertain any true subtle differences in either mortality or morbidity.
Although the committee was cognizant of the questions raised by veterans and others about the ethics and legality of the SHAD testing, it was not charged with reviewing the merits of the program. Its focus had to be on the scientific question of whether test participation is associated with adverse long-term health effects.
Epidemiological studies such as this investigation of health outcomes among veterans of the SHAD tests are complex undertakings requiring substantial time and resources. The committee invested considerable effort in learning about the SHAD tests and in formulating its approaches to data analysis. In the numerous analyses of both the full study population and of several subgroups, the only finding of a seemingly higher risk—of heart disease mortality among the 356 men who served on the USS George Eastman—did not attain statistical significance after adjustments for the multiple tests carried out on this group. The vast majority of the analyses showed no evidence of different health outcomes among SHAD veterans relative to the comparison group. The committee recognizes that with the limitations of epidemiological studies these negative findings cannot unequivocally rule out some potential effect from the SHAD testing. However, within the limits of the data available to the committee, the results of the analyses provide no evidence that the health of SHAD veterans overall or those in the exposure groups is significantly different from that of similar veterans who did not participate in these tests.