The Air Force Health Study Assets Research Program (2015)

5


Findings, Conclusions, and Recommendations

This chapter builds on the 2006 Disposition of the Air Force Health Study report and the foundation laid in Chapters 1–4 to provide findings, conclusions, and recommendations in response to Congress’s directive of “assessing the feasibility and advisability of conducting additional research on the assets transferred to the Agency from the Air Force Health Study (AFHS).”

OVERVIEW OF THE COMMITTEE’S APPROACH AND FINDINGS

To carry out its charge, the committee responsible for this report drew upon their professional background, training, and expertise, along with the experience garnered by the advisory committee that facilitated and oversaw research on the assets over the previous 3 years. Its findings fall into two overarching categories:

1. the scientific value of the AFHS data and biospecimens, and
2. the lessons learned in managing access to the assets.

Specifics of these are delineated below.

Scientific Value

The committee responsible for the 2006 Disposition of the Air Force Health Study (hereafter, Disposition) report faced several uncertainties when they offered their observations regarding future research prospects for the AFHS data and biospecimens. The available evidence allowed them to conclude that the assets

had been well maintained and had been successfully used by the research team that gathered and analyzed them. It was less clear, however, whether the study participants would be willing to allow their materials to be used once the original AFHS concluded, whether the data and biospecimens collected could be successfully managed by another custodian, and whether the research community would be interested in and able to use them in future studies. These uncertainties have now been largely resolved.

The committee finds:

As Chapter 3 notes, approximately 80%¹ of cohort members consented to both have their data and biospecimens transferred to the Institute of Medicine’s (IOM’s) custodianship and allow for these materials to be used in future research studies. The relatively high degree of continued participation improves the potential use of the assets because a larger study population allows more confident estimation of its statistical characteristics.

A 2006 small-scale analysis (Fox and Pavuk, 2006; Pavuk, 2006) found that the AFHS serum samples stored 9–24 years provided detectable results in a high-density immunoassay, suggesting that it might be possible to use them in present-day studies. Subsequent work by investigators who were granted access to these assets under the IOM research program has shown this is still the case, with assays performed successfully with high levels of sensitivity in multiple pilot studies. Investigators were able to successfully isolate genetic material from multiple tissues and use it for their research. Because the volumes of biospecimens needed for such work are modest related to the total volume in the biorepository, the AFHS collection should be usable for several years to come.

The intent of the original Air Force Health Study was to determine whether exposure to the herbicides used during the Vietnam War was associated with adverse health outcomes. The Disposition report committee concluded, though, that its potential was much greater. It speculated that a broad range of researchers—

_______________

¹ 2,210 of the 2,758 persons who participated in at least one exam cycle.

including academic, government, private sector, and foreign investigators—might be interested in performing a variety of studies using the AFHS assets, including

• reanalysis of outcomes examined by the AFHS using different assumptions and approaches than have been applied to date,
• new analyses of the medical records and other study data that examine questions that were not addressed in the AFHS,
• new studies of the collected biospecimens that take advantage of advances in technology and science to conduct analyses that were not contemplated in the AFHS protocol, and
• expansion of the study’s period of analysis through follow-up of the cohort using publicly available information (IOM, 2006, p. 6).²

All of these possibilities have been realized in the time since the IOM has made the assets available.

The Disposition report did not speculate on the number and type of studies that might be proposed for and conducted with the assets, something that would in turn influence the amount of staff time required to support those studies. It instead simply noted that the amount of support required to fund a data management and access operation would depend on the demand for the assets. The fact that the number of studies that have been proposed to date is relatively small has been a mixed blessing. While a higher demand would have been evidence of greater interest among researchers it is unclear whether it would have been possible to meet it, given the initial difficulties in rendering the data into a form amenable to analysis by investigators other than the original study team. The committee believes that the breadth of the interest demonstrated over the past 2.5 years in both research topics and researcher institutions (academic, government, and industry) is indicative of untapped potential but it remains to be seen whether higher visibility in the research community will result in greater demand. Expanding the number of users of the AFHS assets will be necessary to more fully evaluate the usefulness and value of these materials for new and cutting-edge research, and thereby determine whether the cost and effort of maintaining the data and biospecimens is justified.

_______________

² The report also suggested that additional follow-up of health outcomes in the AFHS participants was possible, but a study of this type has not yet been proposed. If such a study were proposed and approved, it would be limited to only those participants who gave their consent to be recontacted and the questions or information requested would have to be approved by the advisory committee. The IOM staff would contact the participants and remove any personally identifiable information before providing it to the investigators.

Assets Management

The lessons learned by the advisory committee and the IOM staff involved in managing the AFHS assets will facilitate their future dissemination. These are summarized in the findings below.

The committee finds:

The AFHS began in the early 1980s and used the technology and practices that were available to store information. While some of this was migrated to more modern forms, the data transferred to the IOM were in a format that was not amenable to current-day data management and analysis software, and were not documented in a way that facilitated their ready use. As a result, considerable time and effort had to be devoted to transforming this material into a form that could be disseminated to investigators.

The current committee feels strongly that if the organization and documentation activities for both the electronic data and biospecimens had been implemented as was recommended in the Disposition report (IOM, 2006, pp. 85–87), the IOM’s research program would have been better equipped to begin enabling research on these assets sooner, and would have had fewer data quality problems to resolve. The IOM data staff has focused its efforts on data processing and integration, which has been a complex and time-consuming process. Although the current practice of managing the electronic data and responding to investigator requests is not an ideal model in terms of efficiency, it was the most appropriate model in terms of staff time and the condition of the data and resulted in successful completion of investigator data requests. As the number of data requests was relatively small, the current model for distributing data to investigators was effective, but reevaluating this model may be necessary as the program moves forward and the number of investigators increases.

However, additional value could be derived from the assets by rendering some records—for example, personal medical records provided in hand-coded form by participants and images of X-rays stored in a proprietary software format—into forms that would be usable by investigators, if demand for the information justified the investment.

The Disposition report’s review of the Medical Follow-Up Agency (MFUA) as a candidate custodian of the AFHS assets noted that it is “the nongovernmental

body with the longest history of veterans’ health research [and] has proven track records of protecting participants’ privacy and confidentiality” (IOM, 2006, p. 157). This is the result of a number of steps taken to protect such assets. The National Academy of Sciences’ institutional review board (IRB) oversees and approves all data uses. The AFHS records with personally identifiable information are stored on a stand-alone hard drive that is accessed through a computer that has no connection to the internet and that is located in a room secured with a cyberlock. Files containing personally identifiable information are coded and are located separately from those containing other information. The biospecimens are maintained in a secured space in a building that is within a military base. Data are only released to investigators in de-identified form, and the variables provided are limited to those absolutely necessary to perform the proposed work. The identifiers used to distinguish participants’ data and biospecimens are different for every investigator and study to minimize the ability to combine information. Data are transmitted in coded form and can only be decoded with a separately provided key. Investigators are subject to data use agreements—and material use agreements if they are provided with biospecimens—that govern their use of the assets. Provisions include prohibitions against transferring the data to other investigators or using it for other purposes, and requirements to return or destroy the data at the completion of the research project. Taken together, these practices help ensure the security of these assets and the privacy of cohort members’ information. To the committee’s and the IOM’s knowledge, there have been no data breaches while the assets have been under the custodianship of MFUA.

The serum, whole blood, urine, semen, and adipose tissue specimens collected in the course of the AFHS have been maintained by the Air Force Research Laboratory (AFRL) at Wright-Patterson Air Force Base since they were transported there at the conclusion of the original study. While AFRL personnel are responsible for the day-to-day upkeep of the biospecimens, they are under the administrative control of MFUA and can only be disseminated at MFUA’s direction. A cooperative research and development agreement governs the working relationship between the parties and includes training requirements for AFRL staff and requirements for periodic reports that catalog all accessions of the samples and any difficulties with the ultralow temperature freezers that hold them. Public Law 110-389 includes a provision that dedicates funding for their upkeep that is provided by the Department of Veterans Affairs (VA). The mandate for this funding, like that which supports the committee’s operation and MFUA’s assets management effort, ends on June 30, 2015.

The storage, monitoring, and emergency response guidelines recommended by the Disposition report (IOM, 2006, pp. 134–135) have been implemented

by AFRL. The biospecimen repository has been well maintained and has not experienced any equipment failures or any loss of biospecimens. AFRL staff have worked with MFUA staff to establish and implement biospecimen dissemination procedures and have provided timely distribution of biospecimens. Some problems have been encountered in locating specific samples in freezers owing to inaccuracies in the manifest provided by the original study. While these are corrected when found, the committee observes that a prospective, thorough inventory of biospecimens would allow quicker access and more accurate sample selection for studies.

The committee responsible for the Disposition report expressed the belief that pilot funding was “needed to establish the AFHS data assets as a resource for independent researchers” and that it would “stimulate prospective researchers to also seek external funding from other existing sources for further or more in-depth projects” (IOM, 2006, p. 168). It recommended that Congress allocate a minimum of $250,000 per year of direct costs for 3 years for small grants for secondary data analysis or pilot projects using the data assets of the AFHS, a recommendation that Congress adopted as part of Public Law 110-389.

As noted above and discussed in Chapter 3, considerable effort was required to render the AFHS data into a form that was amenable to distribution and analysis. This, combined with the other work needed to build the research program from scratch, prevented any data, biospecimens, or funds from being disseminated in the first year of the program. The remaining 2 years attracted interest from researchers: six of the seven studies that were approved for use of data and biospecimens in 2012–2014 were awarded pilot funding; one data-only study also obtained support. This and the feedback received from investigators suggest that pilot funding was an influential factor in stimulating research on the AFHS assets.

While the committee believes that the support offered and provided to some investigators did inspire interest in the AFHS assets, the sample size (seven funded studies) is too small to draw firm conclusions on this issue. It is also too soon to know whether the results of this effort will stimulate interest in larger and more in-depth research projects in the future, something that will depend at least in part on the results of the pilot studies. In any event, it will be necessary for future researchers to obtain funding from other, outside sources if they are to pursue work on the data or the biospecimens.

In the 3 years that the IOM has exercised management responsibility for the AFHS assets, it has, with the guidance of the advisory committee and cooperation of AFRL,

• improved the accessibility of the data to outside researchers by transforming it into a format that is compatible with present-day software systems and built data dictionaries that more thoroughly document it;
• corrected errors in the manifest that lists the location of stored biospecimens;
• created protocols for managing the secure storage and dissemination of the AFHS data and biospecimens; and
• developed and implemented procedures for soliciting and reviewing proposals to perform research on the assets, and facilitated the work of investigators who submit successful applications.

The experience preparing and delivering assets to successful applicants has also allowed the IOM staff to form a more realistic view of the level of effort needed to perform this work. The lessons learned have been applied to refine and streamline the operation over time and exist as a resource for whoever manages them in the future.

CONCLUSIONS

Based on the findings articulated above, the committee draws four conclusions, highlighted below.

The committee that wrote the Disposition report presumed that a successful AFHS management program could be implemented but had no way of knowing whether it would work as they envisioned. That presumption has now been confirmed and the once-speculative potential of the program has been largely realized. The assets have generated continuous interest from researchers, as evidence by the more than 80 inquiries and requests between 2012 and 2014, something that is especially encouraging considering that it has not been possible to actively promote their availability until recently. While few research results had been produced to date because investigators have had relatively little time to perform their work, the initial results appear promising and the studies in progress have the potential to advance medical and scientific understanding.

The research performed using the AFHS assets in the course of the original study has benefited veterans through findings from studies of dioxin and herbicide exposure that were unique risks in this population and of their service in Operation Ranch Hand. It is too soon to know whether and which studies of the assets under the IOM’s AFHS Assets Research Program will yield information that will promote health and wellness in veterans or in the greater population, but the research performed to date has the potential to contribute to the body of medical knowledge, especially the results of studies that are examining potential biomarkers of disease. This is an area of research that has greatly advanced in recent years, allowing studies that were not possible when the original AFHS was conducted. The combination of longitudinally collected data and biospecimens are well suited to biomarker research, which could advance the science regarding disease diagnosis and treatment.

These conclusions and the findings that underlie them lead the committee to the following response to the question posed by the Congress in Public Law 110-389:

While the committee is pleased that a broad spectrum of investigators is carrying out a variety of studies using the AFHS assets, it believes that the full potential of the assets has not come close to being realized. Despite the fact that the AFHS is among the few long-term longitudinal prospective studies that included longitudinally collected biospecimens, anecdotal experience suggests that few outside of the veterans health community are aware of it and its availability to outside researchers. This is likely due to both the limited marketing of the assets in professional journals, conferences, and other forums frequented by the scientific community and a general perception that data derived from military populations is only applicable to those populations. Publication of the results of currently ongoing research should help to address this problem but a more concerted effort to raise the asset’s visibility to the broader research community is also needed to maximize the use of the resource. Any attempts to expand the number of researchers, though, will have to be accompanied by plans for how to fulfill requests for assets and for how to fund the work.

The congressionally directed transfer of the AFHS assets to the custodianship of the IOM has resulted in a promising research program that is yielding further benefits from the government’s approximately $143 million investment in the original study. Many of the assets have now been transformed and documented in a manner that facilitates research by outside investigators, and sustaining access to them will advance knowledge regarding the determinants of health and wellness.

The committee responsible for the 2014 IOM report Veterans and Agent Orange: Update 2012³ also supported the idea that continued research use of the assets was advisable, recommending that “[a]vailable information should be gleaned from existing cohort studies” (p. 942). The report went on to specifically cite the AFHS and stated that

The value of the AFHS assets lie in their combination of longitudinal epidemiologic data and associated biospecimens, and would be greatly diminished if the participants’ serum, whole blood, urine, semen, and adipose tissue samples are not maintained in addition to their health data and other personal information. Both resources should thus be preserved, and the funding to maintain the resources must include support for both if the assets are to remain valuable for research. A multiyear commitment to funding is highly desirable as this would permit the long-term planning and sustaining of the infrastructure needed to conduct an effective assets dissemination program. Because the AFHS followed a cohort of active duty military personnel and veterans, and because the legislation that authorized the transfer was initiated in the Senate Committee on Veterans’ Affairs, it is not unreasonable that VA would be tasked with providing the initial support to maintain the assets and fund research using them. However, as the AFHS assets have also been found to be suitable for research addressing a broader variety of topics addressing the health of the general public, another entity may be equally appropriate to continue to provide all of the ongoing funding to support this resource. The National Institutes of Health (NIH), for example, is an appropriate candidate to consider because several of its component institutes and centers

_______________

³ The Veterans and Agent Orange series of reports is a comprehensive review of the literature addressing exposure to the herbicides used during the Vietnam War and their contaminants and adverse health outcomes.

address topics—cancer, heart and lung disease, aging, environmental exposures, and the like—that are amenable to study with the assets.⁴

RECOMMENDATION

Based on its findings and conclusions,

This must be done in a manner that continues to preserve the privacy of study participants and the security of their data.

The committee identified two options regarding the continued maintenance and management of the AFHS assets:

1. Retain custody of the assets within MFUA of the IOM, either maintaining the current management structure—where MFUA curates the assets—or forming a partnership between the IOM and another organization to manage and distribute the data to researchers (as is presently done with the biospecimens); or
2. Transfer custody of the AFHS data and biospecimens to another organization.

The Disposition report identified several characteristics of a good custodian for the AFHS assets. Those characteristics included a demonstrated ability to properly house, secure, and manage both very large and complex data and the accompanying biospecimens (IOM, 2006). The committee finds that the IOM has met each of these criteria and has maintained and managed the AFHS assets in accordance with best practices.

MFUA has much experience in storing, administering, and disseminating epidemiologic data, and maintaining the current structure of the assets management program is a viable option. However, joining forces with an organization that administers databases or funds research as its primary mission may allow for better promotion and dissemination of the AFHS data and biospecimens for new and original research than could be accomplished by the IOM alone. This is not meant to cast doubt on the ability of the IOM to manage this program, but instead to suggest a possible alternative approach that combines the strengths of

_______________

⁴ Two NIH institutes collaborated with the AFHS over the course of the original study: the National Institute of Environmental Health Sciences (Longnecker and Michalek, 2000; Michalek et al., 2001a,b) and National Institute of Dental and Craniofacial Research (Kingman et al., 2005).

the IOM in scientific review with the institutional capabilities of a dedicated data archive. In this scenario, the IOM would retain custodianship and partner with an organization that has the infrastructure for storing, curating, and maintaining the electronic data, and experience in promoting the use of large datasets. Such a partnership would be similar to the one currently in place for the storage and maintenance of the biospecimens between the IOM and AFRL. Under such an arrangement, the IOM would retain custody of the assets and approve the type and amount of information given to researchers through an advisory committee. The partnering organization would physically hold the assets; have responsibility for cleaning and applying quality control, managing, and maintaining them; and preparing and distributing data (and selecting applicable biospecimen samples) to researchers as directed by the IOM. Affiliating with a clearinghouse for data and biospecimens such as these could significantly raise the visibility of the AFHS assets and spur their greater use.

The committee does not have any recommendations concerning partnering organizations, but can offer some examples for consideration should this option be pursued. The National Institute on Aging sponsors the Chicago Core for Biomarkers in Population-Based Aging Research (NIA, 2014), which—among other activities—maintains a directory of longitudinal studies that include biomarker and personal health data that are made available to investigators. The National Heart, Lung, and Blood Institute’s Biologic Specimen and Data Repositories Information Coordinating Center serves a similar function for studies that hold assets relevant to the institute’s mission (NHLBI, 2014). And the Inter-university Consortium for Political and Social Research “stores, curates, and provides access to scientific data so others can reuse the data and validate research findings” (ICPSR, 2014). The organization’s vast collection of databases, which are primarily focused on health (including aging) and social science issues, includes some that address health questions and a small number with associated biospecimens. All are made available to interested investigators according to the rules set out by the data depositor. A thorough evaluation of partnering options would require discussions with candidate organizations to determine their willingness and capacity to take on curation of the AFHS data and a determination of the logistics and costs of preparing them for the relocation—tasks that are beyond the scope of this committee.

If Congress chooses to designate an alternate custodian to manage the AFHS data and biospecimens and to administer a research program, such a decision should be carefully considered and based on thorough comparison of the attributes and characteristics between the IOM and the alternatives it evaluates.⁵ Many issues would have to be resolved before such a transfer could take place. A new custodian would need to exercise the same commitment to protecting the confi-

_______________

⁵ Chapter 6 of the Disposition report contains a thorough analysis of potential alternative custodians (pp. 149–158) that the committee believes is still valid.

dentiality of the assets as has been practiced to date; and to adhere to best practices, as well as legal and regulatory requirements for the storage, maintenance, and dissemination of data and biospecimens. The cohort would have to consent to having responsibility for their study materials transferred to a new custodian, which based on previous experience would likely result in a further diminution of the cohort. Loss of additional cohort members will reduce the value of these assets for use in additional research because the sample size may not be adequate for many types of studies. Loss of additional participants may introduce further selection bias, which has not been problematic to date, as well as weaken the scientific validity of the analyses. Additionally, if the AFHS assets were transferred to a different custodian, their availability for continued use in research would likely suffer a setback because the new custodian would have to set up a new research program or integrate the AFHS into an existing one. It is unclear in what format the data would be transferred and whether the custodian would choose to create a new database using the original text files or would try to incorporate the work MUFA has done (if the systems are compatible). Finally, a new custodian would need to find an appropriate and secure location to house the freezers in which the biospecimens are stored, or to transfer them to a different ultralow temperature storage facility. Funding would need to be provided to support the staff needed to prepare assets for the transfer and assure that it was successfully completed. Given the issues that a new custodian would have to navigate, the committee observes that the feasibility and advisability of maintaining the AFHS assets for research and maximizing their use for continued research may be best met if the IOM’s MFUA remains custodian but partners with an agency, organization, or institution knowledgeable in and with the infrastructure to support curation of the electronic data.

No matter who serves as custodian of the AFHS assets, a multiyear commitment to funding this organization is highly desirable as this would permit the long-term planning and sustaining of the infrastructure needed to conduct an effective assets dissemination program. The committee believes that a full 5 years of sustained access is appropriate in order to allow enough time for prospective investigators to be made better aware of the assets, develop research hypotheses, propose studies, receive approval for analyzing them, obtain funding, perform work, and disseminate results through publication in peer-reviewed journals and other venues. A reassessment of the support for maintaining and disseminating the assets should be carried out after this time period to evaluate whether the level of interest in them justifies continuing the investment of time and money needed to support their maintenance.

To derive the maximum value from research conducted on the assets, steps should be taken to make sure that the knowledge developed from them is widely disseminated through peer-reviewed publications and other professional venues and integrated into the database for subsequent use by other investigators. Data use agreements with investigators should continue to require that newly derived data

be returned to the custodian, with a reasonable amount of time allotted for publications to first be generated. After these data are integrated, their availability should be promoted. When investigators establish that the biospecimens are amenable to analysis by particular assays, this information should be made generally available so other potential researchers can factor it into their plans.

Finally, the committee believes it important to increase the breadth of scientific disciplines that are aware of and use the AFHS assets. These data and biospecimens have been used to make important contributions to the understanding of the toxicologic mechanisms and health impacts of exposure to herbicides and their contaminants. However, there are many additional opportunities to learn from these assets beyond the purpose of their initial collection or use technology and science that had yet to be developed when the original study ended. Given the rapid development of new analysis techniques and instrument technologies; assay methods with improved analytical sensitivity and specificity; and an abundance of existing proteomic, metabolomic, and genetic methods that can be applied to biospecimens and data from the AFHS, the potential value and utility of the AFHS biospecimens is far greater than what was originally imagined at the beginning of the study. It will be up to the custodian to identify new areas of research and to find ways to reach the people who conduct it so they can start taking advantage of this rare and valuable resource.

REFERENCES

Fox, K. A., and M. Pavuk. 2006. Testing the viability of stored frozen serum samples from the Air Force Health Study using human multi-analyte profiles (MAP). Organohalogen Compounds 68:1390-1393.

ICPSR (Inter-university Consortium for Political and Social Research). 2014. ICPSR data management and curation. https://www.icpsr.umich.edu/icpsrweb/content/datamanagement/index.html (accessed November 24, 2014).

IOM (Institute of Medicine). 2006. Disposition of the Air Force Health Study. Washington, DC: The National Academies Press.

IOM. 2014. Veterans and Agent Orange: Update 2012. Washington, DC: The National Academies Press.

Kingman, A., J. W. Albers, J. C. Arezzo, D. H. Garabrant, and J. E. Michalek. 2005. Amalgam exposure and neurological function. Neurotoxicology 26(2):241-255.

Longnecker, M. P., and J. E. Michalek. 2000. Serum dioxin level in relation to diabetes mellitus among Air Force veterans with background levels of exposure. Epidemiology 11(1):44-48.

Michalek, J. E., F. Z. Akhtar, M. P. Longnecker, and J. E. Burton. 2001a. Relation of serum 2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD) level to hematological examination results in veterans of Operation Ranch Hand. Archives of Environmental Health 56(5):396-405.

Michalek, J. E., N. S. Ketchum, M. P. Longnecker. 2001b. Serum dioxin and hepatic abnormalities in veterans of Operation Ranch Hand. Annals of Epidemiology 11(5):304-311.

NHLBI (National Heart, Lung, and Blood Institute). 2014. Welcome to the Biologic Specimen and Data Repository Information Coordinating Center (BioLINCC) website. https://biolincc.nhlbi.nih.gov/home (accessed November 24, 2014).

NIA (National Institute on Aging). 2014. Search population-based studies collecting biomarkers. http://biomarkers.health-studies.org/studydemo.php (accessed November 24, 2014).

Pavuk, M. 2006. Testing the viability of stored frozen serum samples from the Air Force Health Study using human multi-analyte profiles (MAP). AFRL-HE-BR-TE-2007-0015. Brooks-City-Base, TX: Air Force Research Laboratory Human Effectiveness Directorate, Information Operations and Special Program Division.