3
The Air Force Health Study Database

This chapter addresses matters concerning the medical records and other study data collected as part of the Air Force Health Study (AFHS). It details the number of study subjects in the Ranch Hand and comparison cohorts and how those figures have changed over the course of the study. The chapter also catalogs the electronic and hard copy information collected and addresses the question of whether obstacles exist to retaining and maintaining these data.

Some of the content in this chapter was previously presented in the committee’s interim letter report (IOM, 2005).

DATA COLLECTION AND DATABASE CHARACTERISTICS

Collection

Chapter 2 provides in-depth information on the collection of AFHS data over the course of the study. The information in that chapter related to medical records and other study data is summarized here, and some additional details on changes in the study population over time are presented.

The original cohort of Ranch Hand personnel totaled 1,2421 subjects who had served in the C-123 crews in Operation Ranch Hand herbicide spray missions and included pilots, copilots, navigators, spray operators, and ground support crews. This study was designed as a matched cohort study; it used a complex

1  

Twenty-two Ranch Hands had been killed in action (AFHS, 1984a).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 68
Disposition of the Air Force Health Study 3 The Air Force Health Study Database This chapter addresses matters concerning the medical records and other study data collected as part of the Air Force Health Study (AFHS). It details the number of study subjects in the Ranch Hand and comparison cohorts and how those figures have changed over the course of the study. The chapter also catalogs the electronic and hard copy information collected and addresses the question of whether obstacles exist to retaining and maintaining these data. Some of the content in this chapter was previously presented in the committee’s interim letter report (IOM, 2005). DATA COLLECTION AND DATABASE CHARACTERISTICS Collection Chapter 2 provides in-depth information on the collection of AFHS data over the course of the study. The information in that chapter related to medical records and other study data is summarized here, and some additional details on changes in the study population over time are presented. The original cohort of Ranch Hand personnel totaled 1,2421 subjects who had served in the C-123 crews in Operation Ranch Hand herbicide spray missions and included pilots, copilots, navigators, spray operators, and ground support crews. This study was designed as a matched cohort study; it used a complex 1   Twenty-two Ranch Hands had been killed in action (AFHS, 1984a).

OCR for page 68
Disposition of the Air Force Health Study strategy to enroll comparison subjects. Specifically, a pool of 23,978 subjects comprising C-130 crewmembers who had served in the Southeast Asia theater but who were not involved in spray missions were identified and deemed to be an appropriate comparison group. The exposure period for the Ranch Hand subjects spanned from 1962 to 1971, and the pool subjects served during the same period. From the comparison pool, 1,241 subjects were chosen to be part of the original comparison group. Comparison subjects were matched to Ranch Hand subjects by age, race, and military occupation. Most of the AFHS research is focused on the morbidity component of the study, where exceptionally detailed data gathering took place. The initial physical examination and surveying of these AFHS subjects (Cycle 1) was conducted in 1982 and subsequent cycles were conducted in 1985, 1987, 1992, 1997, and 2002. The AFHS uses a rather complex terminology in its reports to designate the status of its subjects. Comparison subjects enrolled in the study at the beginning of Cycle 1 evaluations are referred to as original comparison subjects. Because some of the original comparison subjects were deemed to be ineligible subsequent to enrollment (when more detailed investigation showed that they did not meet the selection criteria),2 a second category of comparison subjects was established to indicate those comparison subjects that were shifted from the pool to replace ineligible subjects during Cycle 1. A third category is called replacement comparison subjects. Replacement comparison subjects are those subjects available for study inclusion from the individual-level pools of up to 10 comparison subjects. This replacement strategy was implemented to reduce the effect on study power of loss to follow-up on comparison subjects. If an original comparison subject did not participate in a particular cycle visit but participated in a later cycle, he was still designated as an original comparison. Cohort enumeration is discussed in detail in Chapter 2. Subjects who participated in Cycle 1 were termed either partially or fully compliant, depending on whether they completed only the questionnaire3 required of all subjects or both the questionnaire and physical exam. Cycle 1 questionnaires were administered by Louis Harris and Associates, Inc. The National Opinion Research Center (NORC), under contract with Science Applications International Corporation (SAIC), administered questionnaires during Cycles 2–6. The physical examination was conducted by the Kelsey-Seybold Clinic in Houston, 2   During Cycle 1 it was discovered that 18 percent of the comparison population was ineligible for study inclusion (AFHS, 1984a). 3   A short telephone questionnaire was offered to those subjects that refused to participate in the questionnaire. Refusal subjects who completed this short telephone survey are another “compliance” class reported in the Cycle 1 report (AFHS, 1984a). If a Ranch Hand survived the war but died prior to Cycle 1, next of kin were contacted and—if they consented—were administered a proxy version of the participants’ baseline questionnaire.

OCR for page 68
Disposition of the Air Force Health Study TABLE 3-1 Participation in Physical Examinations by Cycle Examination Ranch Hand Subjects Comparison Subjects* Eligible Participated Eligible Participated Cycle 1—1982 1,209 1,046 (87%) 1,666 1,223 (73%) Cycle 2—1985 1,199 1,017 (85%) 1,713 1,292 (75%) Cycle 3—1987 1,188 996 (84%) 1,730 1,298 (75%) Cycle 4—1992 1,149 953 (83%) 1,761 1,280 (73%) Cycle 5—1997 1,102 870 (79%) 1,920 1,251 (65%) Cycle 6—2002 1,043 777 (74%) 2,044 1,174 (57%) NOTE: These numbers reflect corrections made to earlier cycles due to the identification of misclassified study subjects in later cycles and therefore vary slightly from the numbers published in early AFHS reports. *Totals include original, shifted, and replacement comparison subjects. SOURCE: Table C-1 (Compliance of Ranch Hands by Examination Year) and Table C-2 (Compliance of All Comparisons by Examination Year) (AFHS, 2005b). Texas, in Cycle 1 and at the Scripps Clinic and Research Foundation in La Jolla, California, for Cycles 2–6. Summaries of participation by cycle are presented in Table 3-14 and Figure 3-1. Demographic characteristics of the Cycle 6 participants are presented in Table 3-2. Figure 3-2 details the participation of both Ranch Hand and comparison subjects throughout the study and provides summary counts by cycle for the following subject subclasses: new to the cohort (N),5 deceased (D), unlocatable (U), refused to take part in the exam or were noncompliant (F), and found to be ineligible (I). The population numbers presented in Figure 3-2 represent the best available information. However, as is sometimes the case with long-term longitudinal studies, there are some inconsistencies in the study population numbers reported by the AFHS at various times, and some data are missing altogether. Among the 1,241 comparison subjects first identified for participation in the morbidity study, 212 individuals were deemed ineligible (primarily due to the determination that their tour of duty did not occur in the designated territory) 4   After the first (1982) physical examination cycle, “eligible” subjects include those new to the study and exclude those who died after the conclusion of the previous cycle. Subjects were deemed to have “participated” in a cycle if they were fully compliant with the physical examination. The numbers reported in the table reflect corrections made to earlier cycles due to the identification of misclassified study subjects in later cycles and therefore vary slightly from the numbers published in early AFHS reports and shown in Figure 3-2. 5   A subject would be classified as new to the cohort if his status as a Ranch Hand or comparison subject was being corrected from the initial classification, or if he was moved from the pool of potential comparison subjects to the status of an active study participant.

OCR for page 68
Disposition of the Air Force Health Study FIGURE 3-1 AFHS physical examination participation by year and cycle. TABLE 3-2 Selected Demographic Characteristics of Cycle 6 (2002) Participants   Ranch Hand Subjects Comparison Subjects Number of subjects 777 1,174 Mean age 63.1 63.0 Race (%) Black 6.3 6.5 Nonblack 93.7 93.5 Military occupation (%) Officer 39.5 39.4 Enlisted flyer 17.1 15.8 Enlisted groundcrew 43.4 44.9   SOURCE: Table 8-1. Associations between Matching Demographic Variables (Age, Race, and Military Occupation) and Estimates of Herbicide or Dioxin Exposure (AFHS, 2005b).

OCR for page 68
Disposition of the Air Force Health Study

OCR for page 68
Disposition of the Air Force Health Study FIGURE 3-2 AFHS-Morbidity Study Population. SOURCE: AFHS, 1984a, 1987, 1990, 1995, 2000, 2005c.

OCR for page 68
Disposition of the Air Force Health Study during the Cycle 1 examinations and were removed, resulting in 1,029 original comparison subjects. Subsequently, 212 new comparison subjects were added or shifted6 into the original comparison group. In addition, five new Ranch Hand subjects were identified toward the end of the Cycle 1 assessment and enrolled. Additional, or new (N), Ranch Hand subjects were added at subsequent cycles as well when further investigation revealed that they met the criteria for Ranch Hand designation. Between Cycles 1 and 2, four additional Ranch Hand subjects were identified in this manner. Original comparison subjects who did not participate in an earlier cycle but did participate in a later cycle are designated in Figure 3-2 with an “N:O” or as new original comparisons. An intricate strategy was used to add new subjects to the comparison group from the individually matched pools of up to 10 potential comparison subjects for each Ranch Hand subject. These additional comparison subjects are identified in Figure 3-2 as replacement comparison subjects (R). Because the investigators expected greater non-compliance from comparison subjects than from Ranch Hand subjects, 431 subjects were added as replacements before Cycle 1. Between Cycles 4 and 5, 234 replacement subjects were added. Replacements were already matched at the individual pool level on age, race, and military occupation; in Cycles 2–6, they were also matched on the (self-reported) perception of overall health of the Ranch Hand subject. Where an exact health perception match was not available, the closest match was made. The rationale for the actual number of replacements added at each cycle—which exceeded the 1:1 matched ratio of the original study design—was not provided in AFHS documentation. Data on the factors influencing study participation were collected, analyzed, and the results presented in the Study Selection and Participation chapters7 of the final reports for each of the cycle examinations. Some potential participants gave investigators specific reasons why they chose not to be a part of the data gathering exercises (lack of interest or time, or job commitments, for example) while others did not respond to contacts, failed to honor appointments, or communicated a desire not to have any contact with or from the AFHS under any circumstances. Among potential participants for whom data were available, veterans who refused to participate were more likely to self-report fair or poor health than those who were fully compliant. The Cycle 6 report noted that “[t]his pattern of … reporting poorer health has been observed since the baseline examination” (AFHS, 2005b). Comprehensive information was collected on the reproductive history of current and former spouse(s) and partner(s) of subjects in Cycles 1–4. This was done via a questionnaire that was administered to that person either onsite (if the per- 6   Distinctions are not made between original and shifted comparison subjects in reports after Cycle 1; both are referred to as original comparison subjects. 7   These chapters are called Study Selection and Participation Bias in Cycles 1 and 2.

OCR for page 68
Disposition of the Air Force Health Study son accompanied the subject to the cycle exam), over the phone, or by postal questionnaire. Information was collected in private to facilitate obtaining independent histories. Consent was obtained for medical records review to verify some data. Information on fertility, miscarriages, stillbirths, induced abortions, intrauterine growth retardation, and live births (pre- and full-term) were collected and coded. For live births, birth defects data were obtained along with general information on the health and well-being of the child through age 18.8 For the first five mortality studies (AFHS, 1983, 1984b, 1985, 1986, 1989) the original matched cohort of one Ranch Hand to five comparison subjects was used for internal comparisons as well as several external comparison populations, as detailed in Chapter 2. The entire comparison pool population was used for additional analyses in the 1989 mortality follow-up. These numbered 20,340 veterans at the 20-year follow-up: 1,262 Ranch Hand subjects and 19,078 comparison subjects (Ketchum and Michalek, 2005). Date of death was obtained using U.S. Air Force (USAF) Military Personnel Center records, the Department of Veterans Affairs Death Beneficiary Identification and Record Location System, and the Internal Revenue Service database of active Social Security numbers (SSNs). Death certificates provided information on the underlying cause(s) of death (Michalek et al., 1990). These data were verified and coded and are a part of the electronic database. They were combined with basic demographic information (age, race, military occupation, and the like) for all-subject analyses and with the data collected in the cycle exams for more detailed examinations with the smaller cohort of morbidity study participants. Database Characteristics The AFHS database is vast. It comprises electronically stored data and programs and a number of materials originally collected in hard copy form, which also exist as PDFs. Electronically Stored Data and Programs The AFHS’s electronically stored data consists of information collected from subjects during the six cycles of in-person physical examinations and in computer-aided telephone interviewing, computer-assisted personal interviewing, and machine and hand coding of questionnaires and other data intake instruments. In addition, the SAS and Fortran programs and Excel spreadsheets used to analyze these data have been retained. The AFHS investigators maintain what are referred to in this report as master data files for each of the cycles. These master data files are used as the starting 8   The most recent paper on this component of the study was published in 1998 (Michalek et al., 1998a). It contains a citation to the study’s earlier reproductive outcomes work.

OCR for page 68
Disposition of the Air Force Health Study TABLE 3-3 AFHS Electronically Stored Data Assets Type Description Number *.dat Flat files 11,140 *.sd2, *.sas7bdat SAS databases 5,694 *.sas SAS programs 35,444 *.for Fortran programs 8,844 *.xls Excel spreadsheets 2,184   SOURCE: Michalek, 2005. point for all analyses.9 Within a cycle, there are separate master files for various components of the cycle’s data gathering effort: responses to questionnaires, results of particular physical exams, and so on. Data in these files are stored by the subject’s study identification number, which the AFHS refers to as a case number. The separate files relate case numbers to participant names, military records, and other information that does not vary between cycles. Working files are created from these master files and used for specific analyses. Raw and working files are saved for potential future reference (for example, if it was necessary to verify a seemingly anomalous value or reexamine the steps that led to a particular result) but are not otherwise used. Table 3-3 lists the number of files contained in the database. The database presently resides on two servers—an IBM 580 (Kiowa) and a Compaq ProLiant ML370—housed in the AFHS research facility at Brooks City-Base, San Antonio, Texas. The data10 take up ~525 GB of space on the machines, including operating system and backup files (J. Robinson, Air Force Health Study, personal communication, September 21, 2005). The committee does not have a comprehensive listing of all of the data collected during the course of the AFHS. However, it has compiled lists of the general classes of endpoints analyzed in the six cycle examinations as cited in AFHS technical reports. These are presented in 16 tables contained in Appendix B. The tables are organized to parallel the manner in which the data are addressed in AFHS technical reports. Their contents are indexed in Table 3-4 below. The tables in Appendix B indicate that not all questionnaire components, physical examination parameters, or other information gathered was analyzed during the cycle in which it was collected. Most standard physical examination assessments and laboratory tests were performed at all cycles. However, other 9   The original form in which data are delivered to the AFHS is referred to as a raw data file. A raw data file is quality-control checked against paper copy or other alternative documentation and corrected where necessary before a master data file is created. Summary variables derived from collected information—such as body mass index—are also incorporated into master data files. 10   This includes PDFs of the hard copy materials (discussed below).

OCR for page 68
Disposition of the Air Force Health Study TABLE 3-4 Index of Appendix B Tables Table Category B-1 General health assessment B-2 Neoplasia assessment B-3 Neurologic assessment B-4 Psychological assessment B-5 Gastrointestinal assessment B-6 Dermatologic assessment B-7 Cardiovascular assessment B-8 Hematologic assessment B-9 Renal assessment B-10 Endocrine assessment B-11 Immunologic assessment B-12 Pulmonary assessment B-13 Reproductive assessment B-14 Laboratory data B-15 Covariable data B-16 Questionnaire data tests were deleted or added as the investigators had the opportunity to consider early results and form new hypotheses, or take account of changes in technology or standards of practice. Some variables—such as information on sleeping difficulties and dietary habits—were only assessed at one cycle as part of a focused investigation. Although these tables are not a comprehensive or definitive list of the information contained in the AFHS database(s), they illustrate the scope of material stored. The committee’s interim letter report (IOM, 2005) offered recommendations for several database documentation activities that will—if carried out—produce authoritative lists of what variables were collected and when. These recommendations are also listed at the end of this chapter. For a time in the early 2000s, deidentified11 data from Cycles 2–5 were available to the public through the AFHS website and the Government Printing Office (AFRL, 2000a,b,c; 2001). The data were in both SAS and flat file formats and comprised the clinical datasets for the various component of the physical exam: general health, dermatology, cancer, heart disease, diabetes, endocrinology, and 11   Deidentification is the process of removing or altering data in a record that could be used to identify a subject. Under the HIPAA Privacy Rule (45 CFR §164.514 (a–c)), data are considered to be deidentified if either a qualified expert determines that the risk that it could be used to identify a subject is “very small” or if they do not include any of eighteen specified potentially identifying pieces of information.

OCR for page 68
Disposition of the Air Force Health Study the like (RHAC, 2000). Laboratory, reproductive outcomes, and mortality datasets were also available. These data were made accessible, at least in part, in response to a 1999 General Accounting Office12 report that criticized the lack of public access and recommended that AFHS “establish and publicize a timetable for the release of all study data and provisions to release the data in a format (such as compact disc or the Internet) that is easily accessible to the general public” (GAO, 1999).13 However, AFHS later withdrew the data from their website, owing to questions over whether the study subjects had given consent for the posting (RHAC, 2003). Nonelectronic Materials The AFHS data collection also comprises a number of materials originally collected in hard copy or other nonelectronic form. These include paper originals of cycle physical exam reports and completed questionnaires; medical records from the subjects’ physicians, dentists, and other health providers; X rays and other diagnostic imagery; lists of medications taken; military administrative records such as duty station orders, flight records, performance reports, awards and decorations, and discharge documents; vital status records such as birth and death certificates; limited information on the subjects’ spouse(s) and children; research reference materials; and the study’s reports and papers. These materials are currently stored in a secure building at the study’s Brooks City-Base facility in Texas. They take up ~5,350 cubic feet of records storage systems space (J. Robinson, Air Force Health Study, personal communication, September 21, 2005). All have been scanned and the images stored in electronic files in Portable Document Format (PDF). The PDF files containing a particular subject’s materials are cataloged in a directory labeled with the subject’s name and case number. There are over 8.8 million such PDF files (Michalek, 2005). Electrocardiogram (ECG) strips, X rays, and Super-VHS video tapes collected during the cycle exams are also stored at the AFHS facilities. Twelve-lead scalar resting ECGs were obtained from subjects during all six exam cycles. The ECGs were evaluated and information regarding various parameters were coded and placed into the electronic database, along with variables characterizing any prior heart problems in the subject.14 The X-ray films—primarily, chest X rays—were taken during Cycles 2–6 and fill ~50 linear feet of storage space.15 12   Now called the Government Accountability Office. 13   A follow-up publication of the study director’s testimony before a congressional committee reiterated the criticisms and recommendation (Kwai-Cheung, 2000). 14   Depending on the cycle, these parameters included ST-T, P, U, and Q-wave morphology; QRS and QT interval; axis deviation; and evidence of bradycardia, tachycardia, arrhythmia, abnormal sinus rhythm, and myocardial infarction. 15   Source: J. Robinson, Air Force Health Study, personal communication, September 21, 2005.

OCR for page 68
Disposition of the Air Force Health Study TABLE 3-5 AFHS Hard Copy Data Assets Type Description Number Files *.pdf Image files† 8,812,945 Hard copy materials ECGs Paper tracings [not enumerated] X ray films Images on film 15,390 SVHS tapes Dental videos (1994) 1,166   Dental videos (1997) 1,908 †Text in these files is not searchable. SOURCE: Michalek, 2005. The SVHS format video tapes in the data collection store high-resolution images of subjects’ teeth taken during the Cycle 4 and 5 exams. The tapes were made as part of a National Institute of Dental and Craniofacial Research (NIDCR) adjunct to the study examining the association between exposure to elemental mercury via dental amalgams and measures of neurologic function (Kingman et al., 2005). They were used to document the examinations, supply images of subjects’ dental restorations, and provide a means of resolving postexam discrepancies in collected data. NIDCR funded the examination and conducted the data analysis; collected data were given to the AFHS for integration into the database. A separate consent was obtained from the subjects who participated in the project. The tapes take up ~160 linear feet of storage space.16 Table 3-5 shows the quantities of these materials. OBSTACLES AND LIMITATIONS TO FURTHER USE OF AFHS DATA The committee considered three primary categories of obstacles and limitations to further use of the AFHS medical records and other study data: those related to the design and execution of the study; logistical—that is, those relating to the management and analysis of the data; and procedural—those addressing how personal information on study subjects is handled. These are discussed below. 16   Source: J. Robinson, Air Force Health Study, personal communication, September 21, 2005.

OCR for page 68
Disposition of the Air Force Health Study Limitations Related to the Design and Execution of the Study The AFHS—like all epidemiologic studies—suffers from limitations related to factors intrinsic to its design and resulting from implementation decisions made by the investigators. Many of these are specific to the study of the health effects of wartime exposure to herbicides and would carry into future research on this topic, although some of the limitations can be addressed by making different assumptions in analyses. However, the limitations would not necessarily extend to more general studies using the data assets. Study limitations were a central topic of the 1999 GAO report on the AFHS (GAO, 1999). The GAO study director, Kwai-Cheung Chan (2000), summarized that report’s findings as follows: The [AFHS] has two major limitations: it has difficulty in detecting low to moderate increases in risks of rare diseases because of the relatively small size of the Ranch Hand population, and its findings cannot be generalized to all Vietnam veterans because Ranch Hands and ground troops were exposed to different levels of herbicides in different ways. Blood measurements of dioxin … suggest that the Ranch Hands’ exposure levels were significantly higher than those of many ground troops. But ground troops may have been exposed in ways (such as through contaminated food and water) that Ranch Hands were not, and little is known about the potential effects of such differences. GAO asserted that “the Air Force has not clearly or effectively communicated these limitations to the public” (GAO, 1999) and suggested that lack of knowledge of these issues was leading to misunderstanding of the study’s results. In congressional testimony concerning the GAO report in 2000, Dr. Linda Spoonster Schwartz—a Yale University researcher and retired Major USAF nurse—offered additional observations (Schwartz, 2000). Among her comments were that the AFHS protocol (AFHS, 1982) stated that data collected from active duty personnel17 were not confidential because information that indicated a risk to “public safety or national defense” would be made known to the USAF. The fact that a subject’s information could affect his career could, she said, have had an influence on the subject’s responses and willingness to submit to certain tests. Dr. Schwartz also indicated that, since all of the AFHS participants were in Vietnam at one time, it could not be assumed that the comparison subjects had no significant exposure to herbicides,18 and that this called into question the validity of the comparison group for studies of the health effects of herbicides. 17   At the time of the Cycle 1 exam,185 Ranch Hands and 184 comparison subjects were on active duty; in addition, 210 Ranch Hand subjects and 234 comparison subjects held current military or civilian flying certificates, which have rigorous physical and mental fitness requirements (AFHS, 1984a). 18   Serum dioxin levels in study subjects are not a reliable proxy for exposure because these levels decrease over time in the absence of exposure, blood draws were not taken until several years after the end of US military involvement in Vietnam, and not all herbicides were contaminated with dioxin.

OCR for page 68
Disposition of the Air Force Health Study Dr. Joel Michalek, then principal investigator of the AFHS, spoke in a January 2005 presentation before the committee about how the study had dealt with obstacles (Michalek, 2005). He noted four limitations of the study related to herbicide health effects research: the inherently small size of the cohort; lack of any biomarkers of herbicide exposure other than dioxin; little information on participants’ locations in the theater of operations; and unavailability of a detailed exposure history. Michalek also indicated that AFHS investigators had confronted several exposure-related design and analysis issues. Lack of a good herbicide exposure metric led to concerns over exposure misclassification and bias that were recognized in the study’s original protocol (AFHS, 1982).19 After CDC developed an assay for measuring serum TCDD levels in the late 1980s that AFHS adopted as a proxy, more issues arose. One of these was the effect of measurement error in the estimation of TCDD half-life, an issue because this value was used to estimate a common baseline serum dioxin level for each study participant. Papers by Caudill et al. (1992) and Michalek et al. (1992) discuss this in greater detail. Later papers addressed the validity of dioxin body burden as an exposure index (Michalek et al., 1995), reliability of the dioxin assay (Michalek et al., 1996), and the correction of bias in half-life calculations (Michalek et al., 1998b). The AFHS web site notes a weakness specific to the examination of questions outside of the study’s stated mission to evaluate the health effects of wartime exposure to herbicides: “[b]ecause all of our study subjects served in Vietnam or Southeast Asia, contrasting Ranch Hands with comparisons may not fully reveal health differences associated with service in Vietnam” (AFHS, 2005a). An additional obstacle identified by this committee is related to study design. As described above, the design allowed the addition of replacement comparisons at each cycle. The integration of replacements in statistical analyses cannot be handled using standard statistical techniques. Subjects who were found to have been misclassified (designated as a comparison subject when in fact they were a Ranch Hand subject and vice versa) were in turn reassigned to the other group and followed under this new group assignment. Such a design, coupled with the usual issues of missing data and losses to follow-up, complicates the reanalysis of results presented in AFHS reports and papers. Logistical Limitations Logistical limitations are those related to documentation and organization of the data assets. These issues were the focus of the committee’s interim letter report (IOM, 2005). 19   The protocol also addresses a number of other recognized study difficulties and planned correction measures.

OCR for page 68
Disposition of the Air Force Health Study As noted in that report, the amount, detail, and quality of documentation of the data vary by the cycle in which the data were collected. Data from the most recent cycle (Cycle 6, collected in 2002) appear to be well documented, and the data dictionary for this cycle (SAIC, 2003) contains many desirable features: it exists as a searchable PDF, and it contains thorough descriptions of the variables and how information gathering changed from Cycle 5. Documentation for earlier cycles is less complete and is not necessarily in printed or electronically stored form. For example, committee members noted that investigators referred to handwritten annotations in their data dictionary to determine the meaning of coded responses to a questionnaire item during a May 2005 visit to the AFHS research facility. AFHS reports and papers focus on analyses of a particular cycle’s data; little longitudinal (across cycles) analysis has been done to date. Data from early cycles are in different file formats than later data, requiring the analyst to be familiar with this fact and be able to write code that accommodates it. The data location for a particular piece of information—for example, the questionnaire or the master data file where a variable can be found—may change between cycles. Variable names for the same piece of information sometimes change between cycles. Data formatting—whether responses are coded as a numeric value versus an alphanumeric character—are not always consistent between cycles. Differences sometimes exist in how data are coded in identically labeled variables: for example, the year of the subject’s birth (DOBYY) is given as the last two digits of the year in the Cycle 4 database, but it is given by all four digits in the Cycle 6 database. Technology (sensitivity or limit of detection or quantitation in laboratory tests, for example) has changed over time; it is therefore possible that an observation coded as below the limit in one cycle may have a value associated with it in a later cycle. And, missing data codes, error codes, and codes for specific outcomes are not uniformly documented or necessarily consistent between cycles. Because the data may vary in so many and subtle ways, it is necessary to carefully consult the variable name and data dictionaries for each cycle where such documents exist—and to know where such information may be found in their absence—in order to carry out analyses. The interim letter report offered several specific recommendations intended to create a more complete and uniform accounting of the AFHS medical records and other study data. The final section of this chapter recapitulates these recommendations. The other electronically stored materials—the SAS and Fortran programs and Excel spreadsheets used to analyze these data—are not necessarily documented in a form that would allow them to be easily understood. This is not an obstacle to retaining and maintaining these files, but it might present a challenge to someone who wishes to review the steps taken to generate the results of an

OCR for page 68
Disposition of the Air Force Health Study analysis published by the AFHS investigators. However, any future need to access these thousands of programs is uncertain at best. The study’s hard copy materials present an obstacle unrelated to their documentation and organization. As already noted, the AFHS currently is storing ECG strips, X-ray films, SVHS tapes, and large volumes of paper records. There are costs to store and maintain these assets in a manner that minimizes the risk that the subjects’ privacy and confidentiality may be unintentionally compromised. The AFHS has been in existence for approximately 25 years. The committee found that data and specimens have been maintained at a level typical of most long-term epidemiologic research. Over the course of the study, the best practices in epidemiologic investigations and specimen storage have changed, and the technology for managing and analyzing data and samples has advanced. The AFHS has evolved in response to these advances. Study personnel were not tasked with updating and harmonizing the system over time or rendering it accessible to outside researchers, and there was no particular incentive to expend time and funds on efforts to do so. Thus, the observations offered here on the state of the data should therefore not be viewed as a criticism of the work of the AFHS staff. Procedural Limitations: Confidentiality and Other Ethical, Legal, and Social Issues (ELSI) The medical records and other AFHS study data have features that present challenges to the preservation of the privacy of the study subjects and the confidentiality of their personal information. Two prominent features related to the electronic records files and materials originally collected in hard copy but not digitized are discussed below. Data in AFHS electronic records files are stored by case number. However, such coding does not guarantee the privacy of study subjects. The subjects, the Ranch Hands in particular, are not a randomly chosen cohort, picked from a vast, anonymous population. They are a relatively small and well-defined group, many of whom are either known or can be found through publicly available sources. With names as a starting point, other information on the subjects (service rank, tours of duty, and the like) could be gathered and hypothetically, matched against variables in the electronic database to determine which person was designated by which case number. The exceptionally large amount of data available on each subject makes this task easier than would be the case in a less extensive epidemiologic study. And, were such an effort successful, great amounts of personal information would be revealed. A second feature relates to the nature of certain supporting documentation in the study’s collection. Some data assets such as medical records from subjects’ health providers, military administrative records, and vital status records were collected in hard copy form. Information varies from subject to subject, and the

OCR for page 68
Disposition of the Air Force Health Study majority of it has not been coded and placed in electronic records files. Instead, it has served as a supplemental information source that could be consulted when uncertainties or inconsistencies arose in the interpretation of the electronic records. All of these materials have been scanned into PDF image files. The privacy and confidentiality concern with both the hard copy and PDF materials is that almost all contain personal information (names, addresses, SSNs), and it is not practical to redact or otherwise deidentify them. Although destroying these materials is an option, that step would impair the ability of any future investigators to check and correct data. Issues such as these are inherent to large-scale epidemiologic studies. The committee does not believe that the AFHS data assets present any extraordinary or insurmountable issues. The AFHS has successfully managed these in the past, as have investigators on other epidemiologic studies. Additional ELSI, including consent, are discussed in Chapter 6. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS On the basis of its review of AFHS reports, its site visit to the study facility, the scientific literature, and other information presented in this chapter, the committee has reached the following findings regarding the medical records and other study data collected in the course of the Air Force Health Study. As noted in the interim letter report, the committee finds that: The medical records and other study data collected in the course of the AFHS have been properly maintained. However, they are not currently organized and documented in a manner that allows them to be easily understood, evaluated, managed, or analyzed by persons outside of the AFHS. The committee also finds that: Obstacles, including privacy concerns, exist to retaining and maintaining the medical records and other study data. A database as large and complex as AFHS’s requires proper documentation in order to maximize its effective use, especially when the data represent information accumulated over decades of time. Without such documentation, accessing and analyzing even the most basic information could prove to be a tedious undertaking. Such documentation is necessary whether the database is only to be used by a dedicated staff or is made available to a wider audience. The committee thus believes that there is merit in creating a more complete and uniform accounting of the AFHS medical records and other study data whatever policy decision may be taken in response to its recommendations. The data assets may be subject to records retention statutes or regulations and may therefore need to be appraised for possible preservation. It thus makes sense that they be in a form that is comprehensible to people who are not already familiar with them so that reasoned decisions can be made. Any future uses of the data—should

OCR for page 68
Disposition of the Air Force Health Study such options be pursued—would be facilitated by having them documented completely and in a uniform manner. Much of the knowledge of these data files is currently stored within the collective minds and personal notes of the AFHS staff charged with maintaining it. Unless the salient information for future access and analysis is documented, it will inevitably be lost. The committee recognizes that resources are required to produce such documentation but believe the investment is necessary and, in the end, worth the effort. The committee’s interim letter report offered several recommendations for addressing the obstacles to retaining and maintaining the AFHS medical records and other study data related to the organization and documentation of these materials. These are reiterated below. Approaches to addressing the other obstacles identified here are presented in Chapter 6 in the context of the committee’s recommendations for further study. Recommendations Contained in the Interim Letter Report The interim letter report (IOM, 2005) describes the committee’s conclusion that the present state of the documentation and organization of AFHS medical records and other study data was an obstacle to retaining and maintaining these materials after the currently scheduled termination date of the study. The committee therefore recommended that: Action should be taken prior to the currently scheduled termination date of the AFHS to reorganize and document the study’s medical records and other study data in a form and format that allows them to be easily understood, evaluated, managed, or analyzed by persons outside of the AFHS. The following actions were recommended: Create a comprehensive inventory of master data files, organized by examination cycle. This inventory should include the file name and type (flat file,20 SAS database, and so on); a brief description of its contents; and the name, column location, and length; variable type (character or numeric); data codes; and description of each variable stored in the file. The Variable Name Dictionary for the Air Force Health Study, 1992 Questionnaire and Analyses (Michalek, 2000c) is an example of such a document.21 20   A flat file contains records that are stored without structured relationships or formatting. This simplified form allows data to be used by a variety of applications and minimizes the possibility of data loss due to software obsolescence. 21   Variable name dictionaries also exist for the Cycle 2 (Michalek, 2000a) and Cycle 3 (Michalek, 2000b) physical exams.

OCR for page 68
Disposition of the Air Force Health Study Create a comprehensive inventory of the variables contained in the master data files, organized by examination cycle and by questionnaire, physical examination report, or other data intake instrument. The Data Dictionary for Physical and Psychological Examination and Laboratory Data, Air Force Health Study, Cycle 6 (SAIC, 2003) is an example of such a document.22 The contents of this Cycle 6 data dictionary include an annotated version of the data collection forms, comprising the variables and codes used in the associated database, a synopsis of the variable names and their descriptions, the summary variables created and codes used, the number of study subjects examined for each test, changes in the database structure from the previous examination, and notes on data comparability between cycles. In addition, the laboratory results section of the dictionary contains descriptions of assays, units of measurement and normal ranges, and data codes. The committee recommends that such information be compiled for all variables in all examination cycles and that it include notation of whether any attributes of a variable have changed over the course of the study. Create a master data codebook containing the name of every data variable represented anywhere in the AFHS database—that is, at any examination cycle—along with a brief description of the variable, the master data file(s) in which it was stored, and its pertinent attributes. This codebook would be derived from the documents outlined above and would constitute a comprehensive distillation of database contents. It should make clear during which examination cycles a particular piece of information was gathered and the variable name(s) associated with that information over the course of the study. A master identification table with rows containing variable descriptions, columns representing each of the six exam cycles, and the intersections listing the variable name used in that cycle, would be a useful adjunct to this effort. Create a document/database describing the contents, format, and location of the AFHS collection of materials that have been scanned into PDF image files—subjects’ medical records, diagnostic imagery, military and vital status records, and the like—and explaining the collection’s organizational structure. This document/database will serve as a directory to these data. 22   There is a separate data dictionary for the Cycle 6 Health Interval and 1982 baseline questionnaires (NORC, 2003).

OCR for page 68
Disposition of the Air Force Health Study The committee recommended that these documents/databases be in a form and format that facilitates easy access to their contents; searchable electronic files with paper backup for archival purposes would accomplish this. In addition, the committee recommended that an overall plan be developed and implemented for archiving the medical records and other study data of the AFHS.23 Ease of accessibility to the data should be a primary consideration in this effort. The committee noted that federal regulations addressing the preparation of electronic records for transmittal to the National Archives (36 C.F.R. § 1228.270) contain specific information about file formats and media that are appropriate for long-term storage. The committee believes that it is incumbent on the USAF, as the custodian of the AFHS research materials, to ensure their proper documentation and organization. It therefore recommends that: If available AFHS program funds are not sufficient to accomplish the actions elucidated above, supplemental funding should be provided to carry out such work in a complete and timely manner. REFERENCES AFHS (Air Force Health Study). 1982. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides: Study Protocol, Initial Report. Brooks AFB, TX: USAF School of Aerospace Medicine. SAM-TR-82-44. AFHS. 1983. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. Baseline Mortality Study Results. Brooks AFB, TX: USAF School of Aerospace Medicine. NTIS AD-A130 793. AFHS. 1984a. Air Force Health Study (Project Ranch Hand II). An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. Baseline Morbidity Study Results. Brooks AFB, TX: USAF School of Aerospace Medicine. NTIS AD-A138-340. AFHS. 1984b. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. Mortality Update: 1984. Brooks AFB, TX: USAF School of Aerospace Medicine. NTIS-AD-A-162-687. AFHS. 1985. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. Mortality Update: 1985. Brooks AFB, TX: USAF School of Aerospace Medicine. NTIS-AD-A-163-237. AFHS. 1986. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. Mortality Update: 1986. Brooks AFB, TX: USAF School of Aerospace Medicine. USAFSAM-TR-86-43. AFHS. 1987. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. First Follow-up Examination Results. 2 vols. Brooks AFB, TX: USAF School of Aerospace Medicine. USAFSAM-TR-87-27. AFHS. 1989. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. Mortality Update: 1989. Brooks AFB, TX: USAF School of Aerospace Medicine. USAFSAM-TR-89-9. 23   The committee’s recommendations constitute elements of such a plan.

OCR for page 68
Disposition of the Air Force Health Study AFHS. 1990. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. 2 vols. Brooks AFB, TX: USAF School of Aerospace Medicine. USAFSAM-TR-90-2. AFHS. 1995. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. 1992 Follow-up Examination Results. Brooks AFB, TX: Epidemiologic Research Division, Armstrong Laboratory. AL-TR-920107. AFHS. 2000. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. 1997 Follow-up Examination Results. Brooks AFB, TX: Epidemiologic Research Division, Armstrong Laboratory. AFRL-HE-BR-TR-2000-02. AFHS. 2005a. Frequently Asked Questions. [Online]. Available: www.brooks.af.mil/AFRL/HED/hedb/afhs/FAQ.html [accessed October 30, 2005]. AFHS. 2005b. An Epidemiologic Investigation of Health Effects in Air Force Personnel Following Exposure to Herbicides. 2002 Follow-up Examination Results. Brooks AFB, TX: Epidemiologic Research Division, Armstrong Laboratory. AFRL-HE-BR-SR-2005-0003. AFRL (Air Force Research Laboratory). 2000a. Air Force Health Study 1987 Follow-up, Cycle 3. Project Ranch Hand II. [computer files]. Washington, DC: Air Force Research Laboratory and Government Printing Office (GPO). AFRL. 2000b. Air Force Health Study 1992 Follow-up, Cycle 4. Project Ranch Hand II. [computer files]. Washington, DC: AFRL and GPO. AFRL. 2000c. Air Force Health Study 1997 Follow-up, Cycle 5. Project Ranch Hand II. [computer files]. Washington, DC: AFRL and GPO. AFRL. 2001. Air Force Health Study 1985 Follow-up, Cycle 2. Project Ranch Hand II. [computer files]. Washington, DC: AFRL and GPO. Caudill SP, Pirkle JL, Michalek JE. 1992. Effects of measurement error on estimating biological half-life. Journal of Exposure Analysis and Environmental Epidemiology 2(4):463–476. GAO (General Accounting Office). 1999, December 17. Agent Orange—Actions Needed to Improve Communications of Air Force Ranch Hand Study Data and Results. GAO/NSIAD-00-31. Washington, DC: GAO. IOM (Institute of Medicine). 2005. Disposition of the Air Force Health Study—Interim Letter Report. Washington, DC: The National Academies Press. [Online]. Available: http://www.nap.edu/catalog/11483.html [accessed December 1, 2005]. Ketchum NS, Michalek JE. 2005. Postservice mortality of Air Force veterans occupationally exposed to herbicides during the Vietnam War: 20-year follow-up results. Military Medicine 170(5): 406–413. Kingman A, Albers JW, Arezzo JC, Garabrant DH, Michalek JE. 2005. Amalgam exposure and neurological function. Neurotoxicology 26(2):241–255. Kwai-Cheung Chan, Director, Special Studies and Evaluations, National Security and International Affairs Division, GAO. 2000. Persisting Problems With Communication of Ranch Hand Study Data and Results. Statement at the March 15, 2000, hearing of the Subcommittee on National Security, Veterans Affairs, and International Relations, Committee on Government Reform, House of Representatives. GAO/T-NSIAD-00-117. Michalek JE. 2000a. Variable Name Dictionary for the Air Force Health Study. 1985 Physical Exam Questionnaire and Analyses. Brooks AFB, TX: Air Force Research Laboratory. AFRL-HE-BR-SR-2001-0003. [Online]. Available: http://handle.dtic.mil/100.2/ADA387644 [accessed October 30, 2005]. Michalek JE. 2000b. Variable Name Dictionary for the Air Force Health Study. 1987 Analyses Data Sets for 1987 Report and Dioxin Report. Brooks AFB, TX: Air Force Research Laboratory. AFRL-HE-BR-SR-2000-0003. [Online]. Available: http://handle.dtic.mil/100.2/ADA380762 [accessed October 30, 2005].

OCR for page 68
Disposition of the Air Force Health Study Michalek JE. 2000c. Variable Name Dictionary for the Air Force Health Study. 1992 Questionnaire and Analyses. Brooks AFB, TX: Human Effectiveness Directorate, Human Systems Wing. AFRL-HE-BR-SR-2000-0008. [Online]. Available: http://handle.dtic.mil/100.2/ADA387728 [accessed August 31, 2005]. Michalek, JE. 2005 (January 5). The Air Force Health Study: An Overview and Update. Washington, DC. [PowerPoint presentation] The Committee on the Disposition of the Air Force Health Study. Michalek JE, Wolfe WH, Miner JC. 1990. Health status of Air Force veterans occupationally exposed to herbicides in Vietnam. II. Mortality. Journal of the American Medical Association 264(14): 1832–1836. Michalek JE, Tripathi RC, Caudill SP, Pirkle JL. 1992. Investigation of TCDD half-life heterogeneity in veterans of Operation Ranch Hand. Journal of Toxicology and Environmental Health 35(1):29–38. Erratum in: Journal of Toxicology and Environmental Health 36(4):431. Michalek JE, Wolfe WH, Miner JC, Papa TM, Pirkle JL. 1995. Indices of TCDD exposure and TCDD body burden in veterans of Operation Ranch Hand. Journal of Exposure Analysis and Environmental Epidemiology 5(2):209–223. Michalek JE, Tripathi RC, Kulkarni PM, Pirkle JL. 1996. The reliability of the serum dioxin measurement in veterans of Operation Ranch Hand. Journal of Exposure Analysis and Environmental Epidemiology 6(3):327–338. Michalek JE, Rahe AJ, Boyle CA. 1998a. Paternal dioxin, preterm birth, intrauterine growth retardation, and infant death. Epidemiology 9(2):161–167. Michalek JE, Tripathi RC, Kulkarni PM, Gupta PL, Selvavel K. 1998b. Correction for bias introduced by truncation in pharmacokinetic studies of environmental contaminants. Environmetrics 9(2):165–174. NORC (National Opinion Research Center). 2003. Air Force Health Study 2002–2003 Follow-Up Examinations. Data Dictionary: Health Interval Questionnaire; 1982 Baseline Questionnaire. Chicago, IL: NORC, University of Chicago. RHAC (Ranch Hand Advisory Committee). 2000. Transcript: Ranch Hand Advisory Committee Meeting, October 19–20, 2000, Day One. [Online]. Available: www.fda.gov/ohrms/dockets/ac/00/transcripts/3654t1.pdf [accessed October 17, 2005]. RHAC. 2003. Transcript: Ranch Hand Advisory Committee Meeting, March 13, 2003. [Online]. Available: http://www.fda.gov/ohrms/dockets/ac/03/transcripts/3937T1.doc [accessed October 17, 2005]. SAIC (Science Applications International Corporation). 2003. Data Dictionary for Physical and Psychological Examination and Laboratory Data Air Force Health Study, Cycle 6. Brooks AFB, TX: Human Systems Wing/YASP. USAF contract: F41624-01-C-1012. SAIC project: 01-0813-04-2273. Schwartz LS. 2000. Agent Orange—Actions Needed to Improve Communications of Air Force Ranch Hand Study Data and Results (GAO/NSIAD-00-31). Statement at the March 15, 2000, hearing of the United States House of Representatives’ Subcommittee on National Security, Veterans Affairs, and Intergovernmental Relations, Subcommittee, United States House of Representatives. Regarding GAO’s December 1999 report: Agent Orange—Actions Needed to Improve Communications of Air Force Ranch Hand Study Data and Results (GAO/NSIAD-00-31). March 15, 2000. [Online]. Available: http://www.vva.org/legiss/31500tes.htm [accessed January 4, 2006].