National Academies Press: OpenBook

Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects (2018)

Chapter: 4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects

« Previous: 3 Evaluation of the Army's Causation Methodology
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×

4

Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects

This chapter presents the committee’s proposed strategy for conducting assessments of potential long-term health effects from biological and chemical agents, drugs, medications, and substances to which human subjects were exposed in Army testing programs. The results of the assessments will be “general causation” conclusions that could be used by a Benefits Application Panel to adjudicate applications for medical care. The strategy includes an approach to prioritize the agents for evaluation and to conduct a streamlined, scientifically rigorous approach to categorize potential health hazards. It may be applied to any of the test agents of concern regardless of the individual submitting a medical claim, the adverse health condition, or other information. The strategy is designed to ensure transparency and consistency in the approaches used to make causality determinations, to ensure the scientific rigor of the evaluations and decisions, and to preclude bias in the individual adjudication decisions (see Chapter 3 for recommendation about keeping assessment of general causation separate from the process of adjudicating of medical applications). This strategy is designed as a guide for the Army and other stakeholders who may be under time and resource constraints that preclude implementing a full-fledged systematic review process. Implementation of this strategy by the Army and other stakeholders will require a combination of scientific and policy decisions.

The committee’s proposed strategy includes six steps: (1) agent prioritization, (2) problem formulation, (3) literature search and screening, (4) data analysis and synthesis, (5) evidence integration, and (6) drawing hazard identification conclusions. Figure 4-1 illustrates the steps of the strategy, and the elements and potential options in each step are shown in Figure 4-2.

STEP 1: AGENT PRIORITIZATION

In the report Assessment of Potential Long-Term Health Effects on Army Human Test Subjects of Relevant Biological and Chemical Agents, Drugs, Medications and Substances: Literature Review and Analysis (the Report) (Ho-Chunk Technical Solutions 2016), a list of more than 100 agents is included. The list was

Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×

provided to the contractor by the Army and is presumably a compilation obtained from Army records of all the agents used in testing programs to which subjects may have been exposed. Given the number of agents and the court-imposed deadlines the Army must meet, the agents and substances need to be prioritized for hazard evaluation. Although the applications received by the Army will likely dictate the priorities, the Army could also try to anticipate agents and health effects to consider on the basis of exposure, hazard, or risk information; Army policy; or some combination of criteria. These may include, but are not limited to, the number of subjects that were likely exposed to a particular agent and previous reviews of the literature. For example, the agents identified in the Report as having potential long-term health effects could also be used as a basis for setting priorities.

images
FIGURE 4-1 Proposed steps of a strategy to evaluate and categorize potential long-term health effects of biological and chemical agents, drugs, medications, and substances.
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×
images
FIGURE 4-2 Proposed steps of a strategy to evaluate and categorize potential long-term health effects of biological and chemical agents, drugs, medications, and substances. Abbreviations: ACGIH, American Conference of Governmental Industrial Hygienists; ATSDR, Agency for Toxic Substances and Disease Registry; EPA, U.S. Environmental Protection Agency; IARC, International Agency for Research on Cancer; IOM, Institute of Medicine; National Academies, National Academies of Sciences, Engineering, and Medicine; NTP, National Toxicology Program.
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×

STEP 2: PROBLEM FORMULATION

Problem formulation is the process of defining the scope of a problem, formulating a question about it, and establishing the assessment parameters by which the question will be answered.

Several reports by the National Academies of Sciences, Engineering, and Medicine have provided guidance on the problem formulation step in decision making (NRC, 2009, 2014; NASEM, 2017). In the context of the proposed strategy, important practices in problem formulation include specifying the agent, the relevant route of exposure, the health end point(s) to be considered, and the types of information that will be considered. The type of information to be considered will be influenced by the evidence integration approach the Army will use to make causality determinations (see section below, Step 5: Evidence Integration).

Defining the scope of effects the Army will consider to be “long-term health effects” will also be important. The committee noted in Chapter 2 that long-term health effects (or “sequelae”) were defined differently in the Report with respect to the timeframe in which the effects are manifested. It might be helpful for the Army to identify the health effects by the most recent International Classification of Diseases (ICD-10) codes, as described by the National Center for Health Statistics.

For some of the agents, relevant health hazard evaluations might be available from an authoritative body and could be used in lieu of conducting an independent assessment. By way of example, the State of California has regulations in place for such a process. The state defines an authoritative body for such purposes as “an agency or formally organized program or group which utilizes one of the methods . . . for the identification of chemicals, and which the [State of California] has identified as having expertise in the identification of chemicals as causing cancer . . . or reproductive toxicity” (27 California Code of Regulations § 25306). The following national and international agencies have been identified by California’s Office of Environmental Health Hazard Assessment as authoritative bodies for the identification of chemicals as causing cancer or reproductive toxicity: the International Agency for Research on Cancer, the National Institute for Occupational Safety and Health, the National Toxicology Program (NTP), the U.S. Environmental Protection Agency, and the U.S. Food and Drug Administration. Other relevant groups the committee suggests be considered are the National Academies, Institute of Medicine, the Agency for Toxic Substances and Disease Registry, and the American Conference of Governmental Industrial Hygienists.

Systematic reviews of potential human health risks from exposure to environmental chemicals are another resource for the Army to consider. For example, NTP is currently conducting several systematic reviews, including a review of long-term neurological effects from acute exposure to sarin (NTP, 2017). However, there are numerous examples of publications that claim to be systematic reviews but are not, so the Army should be cautious about adopting the conclusions of any publication that touts being a systematic review unless it can be verified

Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×

that appropriate methods were followed. A recent National Academies (2017) report has a case study on polybrominated diphenyl ethers where a search for systematic reviews was conducted and the citations evaluated to determine whether they met the criteria of being systematic reviews. The report specified that the minimum criteria for verification was the “conduct of an explicit and adequate literature search, application of predefined eligibility criteria, consideration of the quality of included studies or risk of bias assessment, and synthesis (or attempt at synthesis) of the findings, either qualitatively or quantitatively” (NASEM, 2017, pp. 389-390). Other important elements are whether the systematic review question is consistent with the question the Army is trying to answer and whether the date range of studies included in the review is appropriate.

If the Army determines that a hazard or risk assessment for a particular agent or substance from an authoritative body is available and relevant to answering its question about general causality, the proposed strategy would allow the Army to adopt and use the causality conclusion without having to undertake Steps 3-5 to reach a hazard identification conclusion (see Figure 4-1). If an appropriate assessment is not available, the remaining steps of the strategy should be followed in a sequential manner.

STEP 3: LITERATURE SEARCH AND SCREENING

Best practices in performing literature reviews involve following systematic review procedures to the extent possible (see Chapter 2). Systematic review involves the development of a review protocol to establish the methods by which literature will be searched, the criteria by which references will be screened for relevance, and the criteria for evaluating study quality. Specifying the methods before conducting the review helps to ensure transparency and objectivity in identifying the relevant scientific literature.

For literature searches, a search strategy is developed that typically identifies the databases to be searched, the time period for the searches, and the search terms (e.g., CAS numbers and synonyms for the agents, keywords for capturing the health effects, and route of exposure) (IOM, 2011; Woodruff and Sutton, 2014; NTP, 2015). Searches of animal and human literature are performed separately. The results of the literature searches are then screened for relevance by determining which studies meet the criteria for inclusion or exclusion, such as whether the study involves a relevant target population (e.g., adults versus children), route of exposure, or even the relevant agent. The results of the literature searches and screening activities should be documented. A number of software tools are available to facilitate literature screening and documentation using standardized workflows, such as the Health Assessment Workspace Collaboration (https://hawcproject.org), Distiller SR (https://www.evidencepartners.com), and Sciome Workbench for Interactive computer-Facilitated Text-mining, more commonly referred to as SWIFT (Howard et al., 2016).

Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×

STEP 4: DATA ANALYSIS AND EVIDENCE SYNTHESIS

After the relevant studies are identified, the data are analyzed and synthesized. The different lines of evidence are evaluated separately. For animal and human evidence, each of the relevant studies is evaluated for quality on the basis of a specified set of criteria. Such criteria could include confidence in how exposures were characterized, confidence in the outcome assessment, and whether the study design accounted for confounding variables. The evidence is synthesized by considering such factors as the consistency across study designs, species, and populations; dose response; and magnitude of the effect. Examples of data analysis and evidence synthesis are available from the Institute of Medicine (IOM, 2000), NASEM (2016), and NTP (2015). Mechanistic evidence is often more diverse and voluminous and the practices for how to incorporate it into cancer and noncancer evaluations are still evolving (NASEM, 2017; Chiu et al., 2018; Guyton et al., 2018).

STEP 5: EVIDENCE INTEGRATION

Evidence integration is the most critical step in the proposed strategy because it involves determining a causality conclusion on the basis of the strength of association between an agent and an adverse health effect. In this step, the different lines of evidence that were analyzed separately in Steps 3 and 4 are integrated. Expert judgment is used during evidence integration in all human health assessments and the transparency and documentation of this step is critical for the credibility and confidence in the conclusions drawn from the available evidence.

As noted in the Army’s Memorandum in Appendix C, there are many methods for determining causation that have been developed since the seminal ones developed by the Surgeon General (DHEW, 1964) and Hill (1965). Chapter 3 describes some of the more relevant ones for the Army to consider. The evidence integration approach(es) the Army selects will determine the types of information that will be considered and how various lines of evidence will be weighed. For example, the evidence integration approaches used by the Institute of Medicine (IOM, 2000, 2012; NASEM, 2016) to draw causality conclusions rely primarily on epidemiological evidence whereas the NTP (2015) approach includes explicit consideration of animal data and epidemiological data, as well as mechanistic data. See Chapter 3 for further comparison of evidence integration approaches. Many of the established approaches require more time and resources than the Army is likely to have available, so adaptation of an existing approach or a combination of approaches might be necessary. It will be critical for the Army to specify the approach(es) to evidence integration it decides on, and to provide supporting documentation on how conclusions were reached using those approach(es).

Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×

STEP 6: DRAWING HAZARD IDENTIFICATION CONCLUSIONS

Depending on the Army’s chosen method(s) for evidence integration or its adoption of a hazard conclusion from an authoritative body, the classification terminology for the causality conclusions could vary; however, there are several important types of information that should be incorporated in the final statement about causality. The test agent, the exposure scenario, the adverse health effect, and the strength of the association between them should be clearly stated.

REFERENCES

Chiu, W.A., K.Z. Guyton, M.T. Martin, D.M. Reif, and I. Rusyn. 2018. Use of high-throughput in vitro toxicity screening data in cancer hazard evaluations by IARC Monograph Working Groups. ALTEX 35(1):51-64.

DHEW (U.S. Department of Health, Education, and Welfare). 1964. Smoking and Health: Report of the Advisory Committee to the Surgeon General of the Public Health Service. PHS Publication No. 1103. U.S. Department of Health, Education, and Welfare, Public Health Service, U.S. Department of Health, Education, and Welfare [online]. Available: https://profiles.nlm.nih.gov/ps/access/nnbbmq.pdf [accessed February 13, 2018].

Guyton, K.Z., I. Rusyn, W.A. Chiu, D.E. Corpet, M. van den Berg, M.K. Ross, D.C. Christiani, F.A. Beland, and M.T. Smith. 2018. Application of the key characteristics of carcinogens in cancer hazard identification. Carcinogenesis doi: 10.1093/carcin/bgy031.

Hill, A.B. 1965. The environment and disease: Association or causation? Proc. R. Soc. Med. 58:295-300.

Ho-Chunk Technical Solutions. 2016. Assessment of Potential Long-Term Health Effects on Army Human Test Subjects of Relevant Biological and Chemical Agents, Drugs, Medications and Substances: Literature Review and Analysis. Contract no. W81XWH-14-R-0102. February 4 [online]. Available: http://www.dtic.mil/dtic/tr/fulltext/u2/1009505.pdf [accessed July 24, 2017].

Howard, B.E., J. Phillipos, K. Miller, A. Tandon, D. Mav, M.R. Shah, S. Holmgren, K.E. Pelch, V. Walker, A.A. Rooney, M. Macleod, R.R. Shah, and K. Thayer. 2016. WEIFT-Review: A text-mining workbench for systematic review. Syst. Rev. 5:87.

IOM (Institute of Medicine). 2000. Gulf War and Health, Volume 1: Depleted Uranium, Pyridostigmine Bromide, Sarin, Vaccines. Washington, DC: National Academy Press.

IOM. 2011. Finding What Works in Health Care: Standards for Systematic Review. Washington, DC: The National Academies Press.

IOM. 2012. Veterans and Agent Orange: Update 2010. Washington, DC: The National Academies Press.

NASEM (National Academies of Sciences, Engineering, and Medicine). 2016. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press.

NASEM. 2017. Application of Systematic Review Methods in an Overall Strategy for Evaluating Low-Dose Toxicity from Endocrine Active Chemicals. Washington, DC: The National Academies Press.

NRC (National Research Council). 2009. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press.

Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×

NRC. 2014. Review of EPA’s Integrated Risk Information System (IRIS) Process. Washington, DC: The National Academies Press.

NTP (National Toxicology Program). 2015. Handbook for Conducting a Literature-Base Health Assessment Using OHAT Approach for Systematic Review and Evidence Integration, January 9. Office of Health Assessment and Translation, Division of the National Toxicology Program, National Institute of Environmental Health Sciences [online]. Available: https://ntp.niehs.nih.gov/ntp/ohat/pubs/handbookjan2015_508.pdf [accessed December 4, 2017].

NTP. 2017. Protocol for Systematic Review of Long-Term Neurological Effects Following Acute Exposure to the Organophosphorus Nerve Agent Sarin. April. Office of Health Assessment and Translation, Division of the National Toxicology Program, National Institute of Environmental Health Sciences [online]. Available: https://ntp.niehs.nih.gov/ntp/ohat/sarin/sr_protocol_508.pdf [accessed February 6, 2018].

Woodruff, T.J., and P. Sutton. 2014. The Navigation Guide systematic review methodology: A rigorous and transparent method for translating environmental health science into better health outcomes. Environ. Health Perspect. 122(10):1007-1014.

Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×
Page 35
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×
Page 36
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×
Page 37
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×
Page 38
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×
Page 39
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×
Page 40
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×
Page 41
Suggested Citation:"4 Strategy for Evaluating Potential Long-Term Health Effects in Army Test Subjects." National Academies of Sciences, Engineering, and Medicine. 2018. Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects. Washington, DC: The National Academies Press. doi: 10.17226/25065.
×
Page 42
Next: Appendix A: Biographical Information on the Members of the Committee to Review Report on Long-Term Health Effects on Army Test Subjects »
Review of Report and Approach to Evaluating Long-Term Health Effects in Army Test Subjects Get This Book
×
Buy Ebook | $14.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Between 1942 and 1975, the U.S. Army conducted tests on human subjects to study the effects of a variety of agents, including chemical warfare agents, biological agents, medications, vaccines, and other substances. The tests investigated the immediate or short-term health effects from acute exposure to understand vulnerabilities to attack. Whether the exposures could have resulted in long-term health consequences to the test subjects has been assessed periodically, and the Army is required to notify subjects of information relating to potential health effects associated with exposure to the test agents. Most recently, a 2016 court injunction directed the Army to provide test subjects with new information about potential long-term health effects associated with their exposures, and to provide medical care if an injury or illness could be attributed to their participation in an Army chemical or biological testing program. In support of the first requirement, the Army contracted a report, Assessment of Potential Long-Term Health Effects on Army Human Test Subjects of Relevant Biological and Chemical Agents, Drugs, Medications and Substances: Literature Review and Analysis (the Report), to determine whether new information published since 2006 should be provided to the veterans.

At the request of the Army, the National Academies of Sciences, Engineering, and Medicine formed an ad hoc committee that was tasked with conducting an independent review of the Report. The committee assessed whether the Report appropriately identified potential long-term health effects that could have resulted from test exposures using an adequate weight-of-evidence approach. The general approach for evaluating agent- and outcome-specific associations as outlined in the Army Memorandum was also reviewed. An interim report of its overarching findings and their supporting evidence was prepared in February 2018. This final report provides additional detail about the basis of the committee’s findings and recommendations. No new findings or recommendations have been added to this report.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!