Application of Modern Toxicology Approaches for Predicting Acute Toxicity for Chemical Defense (2015)

6

Lessons Learned and Next Steps

The objective of the committee’s conceptual framework as presented in Chapter 2 is to predict the potential of a chemical to cause acute toxicity to organ systems that could result in debilitating or lethal effects.¹ In developing its framework, the committee considered how to characterize the inherent toxicity of a chemical, evaluate metabolic and pharmacokinetic attributes that can modify chemical toxicity, and integrate information over different domains. As discussed in Chapter 3, the conceptual framework includes models that use physicochemical and biological data to make predictions about potential acute toxicity. The predictive models can be qualitative (such as structural alerts) or quantitative (such as quantitative structure–activity relationship [QSAR] models), and the resulting outputs themselves might be qualitative (for example, a toxic or nontoxic determination) or quantitative (for example, a potency estimate). Medium-throughput and high-throughput assays will also be needed to predict acute mammalian toxicity as reviewed in Chapter 4. The toxicity predictions ultimately will depend on the collection and integration of the physicochemical and biological data that might be indicators of potential acute toxicity as discussed in Chapter 5.

One primary goal of the committee’s conceptual framework is to develop data sufficient to categorize chemicals on the basis of their predicted acute toxicity. The committee considered existing toxicity-based chemical classification schemes developed for industrial chemicals, agrochemicals, biocides, and pharmaceuticals that often use lethality data to estimate toxicity, such as the dose required to kill 50% of a population of test animals (LD₅₀) (Seidle et al. 2010). Developing LD₅₀s was considered a useful benchmarking approach for predicting acute toxicity of chemicals of interest to the Department of Defense (DOD) because it allows DOD to exclude low-toxicity chemicals and to focus its resources on more toxic chemicals of concern. The committee considered the need to develop mechanistically based assays and to use well-characterized chemicals as positive controls to improve the predictive validity of LD₅₀s and to identify potential organ toxicities.

In the present chapter, the committee briefly discusses some current programs that are evaluating or developing modern testing strategies, reviews its suggested tiered testing strategy, and highlights important lessons learned from current testing programs that should be considered by DOD in developing its future testing strategy. The chapter also provides the committee’s overall conclusions and identifies several steps that DOD could take in the short term to medium term (3–10 years) to implement a program that uses modern approaches to identify chemicals that have the potential to induce life-threatening acute toxicity in deployed personnel. ²

__________________

¹As defined in Chapters 1 and 2, organ systems included the cardiovascular, respiratory, hepatic, renal, skeletomuscular, immune, and nervous systems, including special senses (vision and hearing).

²The committee recognizes that a more detailed research plan is needed and that development of such a plan is noted in the statement of task as a potential Phase 2 of this project.

MODERN APPROACHES FOR THE ASSESSMENT OF ACUTE CHEMICAL TOXICITY

The committee explored whether DOD could adopt a modern testing strategy for the prediction of acute toxicity.³ In particular, the committee’s statement of task required a focus on the assessment of existing high-throughput screening (HTS) methods to identify acutely (and likely highly) toxic chemicals with greater predictivity. The committee considered whether several projects were relevant for DOD’s purposes. For example, it examined the European Centre for the Validation of Alternative Methods (ECVAM) ACuteTox project⁴ whose stated aim is to develop a strategy to replace all in vivo tests of acute oral toxicity. The ACuteTox effort considers in vitro methods that address specific mechanisms of action relevant to acute systemic toxicity (such as assays for neurotoxicity) and includes such computational methods as QSAR modeling and physiologically based biokinetic modeling.

Initially, the ACuteTox project selected and tested 97 reference chemicals with six basal cytotoxicity assays and compared the results with published human and animal in vivo data (Clothier et al. 2008; Sjöström et al. 2008). Later, 57 reference chemicals were tested in a number of functional tests that covered absorption, distribution, metabolism, excretion, and specific organ and system toxicity, such as hematotoxicity, neurotoxicity, nephrotoxicity, and hepatotoxicity (Kinsner-Ovaskainen et al. 2009). Standardized experimental design and data acquisition were used in the second phase of ACuteTox program (Kopp-Schneider et al. 2013). Concentration–response data were routinely collected (for example, IC₂₀, IC₅₀, EC₂₀, or EC₅₀ values) and served as the statistical basis of the development of testing strategies (Kinsner-Ovaskainen et al. 2013; Prieto et al. 2013b). The oral acute-toxicity category was predicted using a chemical’s physicochemical properties, in silico modeling results, and values (such as IC₅₀) obtained from the in vitro studies (Kopp-Schneider et al. 2013). In general, the predictive validity seen in the efforts has been moderate to low. In addition, there has been little effort to assess the predictive validity for highly toxic chemicals that would be of concern to DOD.

Examples of large-scale US projects include the Toxicology Testing in the 21st Century (Tox21) partnership and the US Environmental Protection Agency (EPA) ToxCast program. The ToxCast program uses a large suite of in vitro biochemical (cell-free) and cell-based assays to evaluate chemicals and to analyze their bioactivity profiles computationally (Kavlock et al. 2012; Judson et al. 2014; Kleinstreuer et al. 2014). Phase I of the ToxCast program included about 300 conventional pesticide active ingredients that were tested in a battery of cell-free and cell-based assays to evaluate the ability of the assays to predict potential human toxicity (Judson et al. 2010). In Phase II, the chemical space was broadened to include chemicals that are used in consumer products and industrial processes and unmarketed drugs that were donated by pharmaceutical companies (Sipes et al. 2013). However, few ToxCast assays were designed specifically to assess acute toxicity. Furthermore, there have been few efforts to use HTS approaches to evaluate acute toxicity. Regardless, the ToxCast program has identified a number of important technical issues that could be considered in the design of a program relevant for DOD (Tice et al. 2013).

__________________

³Modern approaches are ones that do not rely primarily on traditional toxicology testing and include computational and in vitro or nontraditional in vivo assays.

⁴See http://www.acutetox.eu/. The ACuteTox consortium was initially funded with €8 million by the European Commission’s 6th Framework Programme and included 35 academic, industrial, and government research institutes in 13 European countries (Clemedson 2008). The project was divided into 10 work packages that included selection of reference chemicals, generation of animal and human in vivo databases, development of an Internet-based database for central management of all project data, adaptation of promising in vitro methods to robotic screening platforms, statistical analysis (identification of outliers and design of the preliminary algorithm or prediction model), development of in vitro assays for neurotoxicity, and construction and optimization of the in vitro testing strategy.

The committee also explored modern toxicity-testing strategies that are under development in the pharmaceutical industry. For example, the pharmaceutical industry is exploring ways to predict drug-induced liver injury (DILI). DILI is a low-incidence but important idiosyncratic cause of drug toxicity and a major reason for attrition during drug development or withdrawal from the market (Stephens et al. 2014). Traditional toxicity-testing strategies do not predict DILI reliably in patients: fewer than 55% and 25% of DILI drugs are predicted on the basis of the regulatory animal-toxicity studies and simple in vitro tests, respectively (Olson et al. 2000; Xu et al. 2004). Box 6-1 describes the recent development of predictive models that can be used in preclinical studies to detect DILI risk in humans.

The committee was unable to find a robust modern testing program that DOD could readily adopt for its purposes. Lessons learned from the ToxCast, ACuteTox, and industry programs do, however, provide a great deal of guidance for DOD in designing a system that uses HTS approaches and predictive models to interpret the performance of cell-free and cell-based assays in predicting acute toxicity.

IMPLEMENTATION OF THE COMMITTEE’S CONCEPTUAL MODEL FOR ASSESSMENT OF ACUTE CHEMICAL TOXICITY

As discussed in Chapter 2, the committee developed a conceptual framework that links chemical structure, physicochemical properties, biochemical properties, and biological activity to acute toxicity. Implementation of the conceptual framework will require DOD to support development of a suite of databases, assays, models, and tools that are based on in vitro, nonmammalian in vivo, and in silico approaches for predicting acute toxicity. As presented in Chapter 2, these databases, assays, models, and tools would be used as part of a tiered prioritization strategy to predict acute toxicity (see Figure 6-1). Relatively easily obtained nontesting data (such as existing toxicity data or physicochemical properties; see Chapter 3 for more details) could allow initial evaluation of

a large number of chemicals in the initial tiers. Higher testing tiers incorporate a variety of assays (in vitro or nontraditional in vivo) whose biological complexity increases as a chemical moves from one tier to the next. At each tier, a decision is made as to whether further assessment of toxicity is needed. The ultimate goal of the effort is to sort chemicals into categories that would indicate the predicted acute-toxicity hazard (for example, low toxicity, high toxicity, or uncertain toxicity because of inadequate data) and help to prioritize chemicals for follow-up evaluation. DOD can also apply its expertise to use other factors that were not considered by the committee (such as chemical availability, ease of chemical synthesis, and weaponizability) to exclude chemicals from further testing. One goal of the overall approach is to reduce the number of chemicals that progress through each tier in an efficient and cost-effective manner.

DEVELOPMENT OF A MODERN TIERED APPROACH FOR PREDICTING ACUTE TOXICITY: INITIAL STEPS

There are some initial steps that DOD could take to implement the committee’s tiered testing strategy. This section describes in greater detail some approaches that might be useful for DOD to pursue in implementing modern testing approaches for predicting acute toxicity.

FIGURE 6-1 Prioritization strategy based on a tiered approach for using predictive-toxicology models and tools to evaluate agents for acute toxicity.

Computational Approaches

Chapter 3 discusses a variety of nontesting approaches that can be used to predict acute toxicity. Physicochemical data can be used to predict physical hazards or reactivity, such pharmacokinetic characteristics as metabolism and distribution, and likely routes of exposure. Those data can help to group chemicals by using chemical descriptors of various physicochemical or topological properties and to identify the biologically relevant chemical space (Lipinski and Hopkins 2004). Physicochemical data can also help to fill data gaps and support read-across approaches that apply data on a particular property or effect of a tested chemical to a similar untested chemical. A variety of in silico methods have been developed to predict the molecular sites where metabolism could occur and the types of metabolites that could be formed.⁵ Several available QSAR models use structural properties or physicochemical properties to predict acute oral LD₅₀s. Few such models are available for predicting inhalation LC₅₀s, and the committee was unable to identify models for predicting dermal LD₅₀s.

A few QSAR models predict neurotoxicity and cytotoxicity but not other end points that are relevant for acute, debilitating toxicity. Box 6-2 describes a QSAR model for the prediction of acetylcholinesterase (AChE) inhibition. Quantitative structure–toxicity relationship models have also been developed to evaluate the role of lipophilicity, polarity, molecular geometry, and quantum chemical descriptors for molecular orbital energy in the toxicity of organophosphate insecticides (Can 2014). Inhibition of AChE and other cholinesterases is an important step in the toxicity of nerve-gas agents (such as VX and soman) that are highly potent organophosphates; thus, the computational models that have been developed could be useful in a DOD testing strategy.

A large number of chemicals have reported LD₅₀ data, which cover a large portion of chemical space. Although LD₅₀ data are available on many chemicals, they are not informative about a chemical’s mechanism of action. Such knowledge is important because acute toxicity might involve multiple biochemical mechanisms; this highlights the need for improved mechanistic insights about structure–toxicity relationships. Efforts to derive acute-toxicity QSAR models that have high predictive accuracy have fallen short because of mechanistic complexities.

High-Throughput Screens

How the Tox21 and ToxCast data have been used to predict mammalian toxicity was demonstrated recently by the National Toxicology Program Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM), which evaluated whether HTS data could predict the results of an in vivo uterotrophic assay that screens for estrogen activity in rodents (see Box 6-3). The project is part of a larger NICEATM and EPA effort to develop a robust in vitro screen for chemicals with estrogenic or androgenic activity. The committee recognizes that estrogenic and androgenic end points are not relevant for the assessment of acute toxicities of concern to DOD, but the NICEATM and EPA efforts provide important insights. The committee found the targeted toxicity-prediction effort to be relatively successful (see Box 6-3); this finding is promising for the DOD goal of acute-toxicity prediction. However, estrogenic activity is, in principle, an easier target for prediction because it depends on specific gene-expression patterns for which sensitive and specific HTS assays can be designed. Nevertheless, the effort identified several needed features that can help to inform future DOD efforts, including the following:

• Performance standards for new assays that consider validated test methods, reference chemicals, and standards for assay accuracy and reliability (Wind and Stokes 2010; Stokes et al. 2012).

__________________

⁵Metabolites could also be identified through additional experimentation, for example, by using chemical analyses of biological test systems or by comparing results obtained from cell systems that use different levels of metabolic competence.

• Suitable in vivo end points and study results that can be used to assess HTS assay performance (Rotroff et al. 2013); that is, there is a need for phenotypic anchoring of the in vitro results.
• Reference chemicals that have known biological activity (such as ERα in the cited NICEATM project) for evaluating the sensitivity, specificity, and predictivity of assays to identify agonists and antagonists for a biological target of interest (Huang et al. 2014).
• Appropriate data-integration models that can pool information from multiple assays (Rotroff et al. 2013); model performance can be evaluated by calculating sensitivity, specificity, and balanced accuracy for a specific set of criteria across chemical space (Rotroff et al. 2013; Cox et al. 2014).
• Recognition that misclassification of chemicals might occur (Rotroff et al. 2013; Cox et al. 2014). Possible explanations for misclassification of a chemical include a failure to test it at a high enough concentration to exhibit a response in ToxCast assays, inadequate metabolic competence of the test system, and species, tissue, or cell-type differences in response between the ToxCast assay and in vivo models that are used to assess HTS assay performance (Rotroff et al. 2013). Although not addressed directly by the ToxCast program, an additional source of misclassification of a chemical could be the variability in the in vivo data that are used to assess the ability of an assay to predict acute toxicity.

The example shown in Box 6-3 illustrates a current trend in the application of HTS testing as a replacement for some animal-based assays. In particular, the use of HTS assay data as part of an endocrine screening program has received some traction in the scientific community (Dix et al. 2007; Thomas et al. 2012; Rotroff et al. 2013; Thomas et al. 2013; Cox et al. 2014; Becker et al. 2015). In some cases (such as chemical interactions with the estrogen or androgen receptor system), adoption of HTS assays in the context of a tiered testing approach as a replacement for in vivo assays appears likely (van der Burg et al. 2015).

The predictiveness of many HTS assays, however, remains low (< 50%) (Thomas et al. 2012; Cox et al. 2014; Patlewicz et al. 2015). Many in vivo end points cannot be predicted any better by using HTS assay data than by using chemical descriptor information and QSAR models. Thomas et al. (2012) identified several factors that could account for the relatively poor ability of in vitro assays to predict in vitro responses, including (1) the failure of current in vitro assays to capture biochemical and cellular processes or properties in the in vivo tissues with adequate fidelity; (2) the possibility that in vitro assays do not capture biological context-specific outcomes reliably; (3) inadequate coverage of pathways, protein targets, and cell types; (4) substantial species differences between the in vitro assays and in vivo end points that are being predicted; and (5) the inadequate number of positive chemicals for each end point to capture the broad array of mechanisms that lead to in vivo toxicity.

Cytotoxicity Assays as General Indicators of Acute Toxicity

As discussed in Chapter 4, one approach to predicting acute toxicity relies on in vitro cyto-

toxicity assays that use human cells in culture (Seibert et al. 1996; Ekwall et al. 1998; Halle 2003; NICEATM 2006; Xia et al. 2008; Halwachs et al. 2013; Prieto et al. 2013a). That approach uses relatively simple assays and presumes that in vivo toxicity does not result primarily from impairment of specific functions of differentiated cells (Prieto et al. 2013a). In many cases, validation of the assays rests on an examination of the regression equation from the correlation of experimental IC₅₀⁶ cytotoxicity values with published LD₅₀s,⁷ an approach that can be used to estimate unknown LD₅₀ values from IC₅₀ cytotoxicity values measured in vitro (Seibert et al. 1996). The committee examined the NICEATM and ECVAM validation study that assessed the predictive capacity of the BALB/3T3 neutral-red uptake cytotoxicity assay (see Box 6-4); that assay has been evaluated for its ability to identify chemicals that would be labeled as toxic or hazardous (LD₅₀ < 2,000 mg/kg) (NTP 2014). The overall results of the studies have shown that there is a relatively good correlation of around 60–70% between in vitro cytotoxic concentrations (IC₅₀s) and rat oral LD₅₀s (JRC 2013).

Although cytotoxicity assays hold some promise for the prediction of acute toxicity, several important caveats and assay limitations need to be considered, including the following:

• Little evidence of assay performance exists for highly toxic chemicals. DOD is faced with identifying agents that have extremely low LD₅₀s or LC₅₀s. For example, acute oral LD₅₀ values reported in mice for some agents of current concern to DOD are well below 1 mg/kg: botulinum toxin (0.001 μg/kg), ricin (3 μg/kg), VX (15 μg/kg), anatoxin–a(s) (50 μg/kg), soman (64 μg/kg), and sarin (100 μg/kg).⁸ In addition, those examples of highly toxic chemicals have different mechanisms of action, including inhibition of cholinesterase activity (for example, anatoxin-a(s), VX, soman, and sarin), ribosome inactivation (ricin), and blockade of acetylcholine secretion (botulinum toxin). Applying the BALB/3T3 NRU cytotoxicity assay to a set of 67 chemicals and using the rat LD₅₀ data from the Registry of Cytotoxicity showed that predictions for highly toxic chemicals were generally poor—0/6 for chemicals with an LD₅₀ below 5 mg/kg and 1/11 chemicals with an LD₅₀ of 5–50 mg/kg (NICEATM 2006). Substances with LD₅₀s of 300–2,000 mg/kg were predicted better by this assay—about 81% accuracy (NICEATM 2006).
• Prediction of acute toxicity with cytotoxicity assays remains highly variable. There are several possible reasons why acute systemic toxicity of some chemicals is poorly predicted by basal cytotoxicity assays. First, toxicity might be due to tissue- or organ-specific effects caused by the chemical of interest. For example, substances with a mechanism of action on molecular targets (receptors, channels, and enzymes) are not modeled in most cell lines that are used for cytotoxicity assays, and this probably accounts for poor prediction of the toxicity of anatoxin, botulinum toxin, soman, and sarin, which perturb neuronal synapses. Second, even though a generally toxic mechanism is modeled by a cell line, the cell line could be much less sensitive to this mechanism than is some sensitive cell type in the human body. For example, ricin targets protein synthesis, which is needed by all cells, but in the human body, only specific organs are highly sensitive. Third, chemicals have restricted accessibility to target cells or tissues. Fourth, chemical toxicity might depend on bioactivation pathways that are absent in the test system. Fifth, a chemical might be quickly eliminated or detoxified through metabolism; thus, in vitro results might overpredict toxicity seen in vivo.
• Assay limitations can also contribute to reduced predictive power of in vitro cytotoxicity assays. For example, actual concentrations available to cells or to intracellular targets in in vitro tests might be much lower than the nominal concentrations.

__________________

⁶IC₅₀ is the concentration of a substance that causes 50% inhibition in vitro.

⁷In some cases, LD₅₀ values are available from the Registry of Cytotoxicity (Halle 2003).

⁸LD₅₀ data available from Franz (1997).

The ACuteTox project identified several important technical considerations in the design of an in vitro testing strategy (Kopp-Schneider et al. 2013). Some of the more important considerations were the following:

• Select chemicals to span a wide range of outcomes of interest.
• Perform all assays with all chemicals.
• Be aware that chemical solubility might limit the concentrations that can be used in the test system.
• Design experiments carefully to guarantee reliable and meaningful estimates.
• Avoid overfitting of the prediction model, which occurs when a model is fitted exactly to the training data; this hinders performance in future screening efforts or applications.
• Use cross-validation and bootstrapping to estimate error rates.

A QSAR model based on in vitro cytotoxicity data and oral LD₅₀ values from the Registry of Cytotoxicity has been developed for use in predicting acute toxicity in rodents (Freidig et al. 2007). The predictions from that model tend to overestimate toxicity; thus, substances that were predicted to have no or low toxicity with that model could be eliminated from additional in vivo testing.

Mechanistically Based Assays

Another approach that has been used to predict acute chemical toxicity is based on the development of assays that evaluate a mechanism of action known to be critical for a chemical’s toxic response. One mechanism-based example explored by the committee uses HTS assays and computational approaches to predict chemical-induced inhibition of AChE activity. Inhibition of AChE results in an accumulation of acetylcholine (ACh) at cholinergic synapses and associated clinical signs. A variety of agents, including nerve agents (such as VX and soman) and other toxic organophosphorus (OP) chemicals and some carbamates, inhibit AChE activity. Several molecular models have been developed that use reference chemicals to describe inhibitor binding with human (and other) AChE (Barril et al. 2001; Kua et al. 2002; Guo et al. 2004; Akula et al. 2006; Sopkova de Oliveira Santos et al. 2010; Gupta and Mohan 2011; Deb et al. 2012). The models evaluate inhibitor interactions at the two principal binding sites—catalytic anionic site (CAS) and a peripheral anionic site (PAS)—in the AChE enzyme. The nerve gases and other classical AChE inhibitors bind a phosphoryl group on a serine residue in the CAS (Deb et al. 2012). Another approach for assessing chemical–AChE interactions is to evaluate how the chemical of interest docks with the active site and other portions of the protein. Several structure–activity relationships (SARs) based on pose⁹ predictions for the interactions have been developed (Huang et al. 2010; Samadi et al. 2010). Results obtained with those computational approaches often depend on the type of ligand, the protein conformation, and the presence of water (Berg et al. 2011). Box 6-5 illustrates several examples of how an HTS approach can be used to screen chemicals as AChE inhibitors.

Similar mechanism-based screening efforts have been developed by the pharmaceutical industry and others to develop in vitro models that can predict in vivo biology in support of drug-safety assessment or drug discovery. For example, some investigators have examined the relationship between cytotoxicity data, basal gene-expression measurements, and a chemical’s structure to identify putative molecular targets and the role of gene expression in cytotoxicity (Covell et al. 2005). Another example involves screening chemicals for their ability to inhibit mitochondrial complex I of the electron transport chain (Glover et al. 2007). Berg et al. (2006) successfully grouped chemicals by mechanism of action (for example, modulators of NFkB signaling or the phosphatidylinositol 3-kinase/Akt signal-transduction pathway) by using data from human cell-based HTS assays. Such approaches are consistent with the development of an adverse-outcome pathway (AOP), which is a linear conceptual model that links a molecular initiating event, key events, and an adverse outcome (Figure 3-1) (Ankley et al. 2010; Garcia-Reyero 2015).

__________________

⁹The term pose refers to the conformation and alignment of a molecule (Coats 2002).

An AOP begins with a well-defined molecular initiating event and then describes the key events along a biological pathway that ultimately lead to an adverse outcome associated with chemical exposure. One has been developed for lethality associated with AChE-inhibiting organophosphate and carbamate pesticides (Russom et al. 2014). That AOP considers the role of metabolism (such as metabolic activation by cytochrome P450s to form oxon metabolites of some organophosphate pesticides) and links the main initiating event (cholinesterase inhibition) with various physiological cholinergic responses. Whether that or other AOPs in development (such as the ones for nonpolar narcosis and mitochondrial toxicity¹⁰) will be relevant for the highly toxic chemicals of concern to DOD is unknown but deserves further study.

OTHER CONSIDERATIONS

The committee has identified several broad technical considerations that are important, such as the use of reference chemicals and development of appropriate data-integration models.

__________________

¹⁰OECD provides additional information about developing AOPs at http://www.oecd.org/chemicalsafety/testing/listsofprojectsontheaopdevelopmentprogrammeworkplan.htm.

DOD is faced with several additional challenges that will need to be considered as it develops a tiered testing program. They include technical challenges associated with the assessment of chemicals for direct debilitating portal-of-entry effects (such as skin blistering and pulmonary edema) because there are few in vitro systems for assessing the acute toxicity of inhaled chemicals or agents that produce skin blistering. Likewise, it is unknown whether toxicity screens developed to predict oral toxicity could also predict dermal LD₅₀s or inhalation LC₅₀s. Another important challenge that DOD faces is related to the broad array of chemicals (or chemical classes) that could require assessment. That issue might require DOD to develop distinct tiered approaches for different chemical classes, such as metals. DOD will also need to support the development of HTS assays for mechanisms of actions that are known or suspected to be involved in acute chemical toxicity. In the future, AOPs and QSARs could be developed to strengthen DOD’s ability to predict acute chemical toxicity (Tollefsen et al. 2014). Finally, DOD will need to develop methods for integrating data across physicochemical and biological domains.

DOD has a history of using alternative test methods for the assessment of chemical-warfare agents. For example, the US Army’s Medical Chemical Defense Research Program used a variety of assays to elucidate mechanisms of action and identify countermeasures for many of the classical chemical-warfare agents, such as sulfur mustard, lewisite, nerve agents, and hydrogen cyanide (see Siddell et al. 1997; Somani and Romano 2001). Considerable DOD effort went into being able to conduct the assays safely when working with agents, such as soman and VX, which have human dermal LD₅₀ estimates of 0.14 and 5 μg /kg of body weight, respectively. If the overt toxicity of a new or novel chemical is totally unknown and within an order of magnitude or so of a classical chemical-warfare agent’s potency, serious consideration of the engineering controls necessary to conduct the assays will be required. Because few research facilities are equipped to work with known highly toxic chemical-warfare agents as reference chemicals, DOD might be required initially to use surrogate chemicals that have a shared chemical mechanism or clinical effect of more militarily relevant agents.

The committee expects that in the next 3–10 years any tiered testing approach will not be able to replace completely the need for follow-up targeted in vivo studies to confirm the toxicity of a chemical of interest. Indeed, the state of the science suggests that development of a predictive acute-toxicity program will require extensive DOD investment in development of fit-for-purpose assays, (Q)SAR and other computational modeling approaches, and in vitro–in vivo extrapolation and data-integration methods for the prediction of acute toxicity. To begin the investment, the committee recommends that DOD initiate pilot studies that evaluate chemical classes of highest concern with well-characterized reference chemicals. The pilot studies will allow it to develop the assays and tools that are needed to predict acute chemical toxicity efficiently and accurately and to evaluate the rate of false negatives and false positives. The pilot studies could also examine how generalizable the results of various assays and tools are from one chemical class¹¹ to another. That research would allow DOD to begin to address the size of the chemical space needed to make predictions about unknown chemicals. DOD could benefit from leveraging its efforts with existing federal activities, such as the ToxCast program. Collaboration would allow DOD to complete pilot studies more rapidly.

FINDINGS AND RECOMMENDATIONS

• Finding: On the basis of its review of the current state of the science, the committee concludes that development of a screening program to predict acute toxicity that incorporates such information as existing toxicity data, physicochemical properties, and results from in silico

__________________

¹¹In this context, chemical class is defined broadly to include structurally related chemicals, chemicals that have different mechanisms of action, and chemicals that have different toxic end points, such as hepatotoxicity and neurotoxicity.

• modeling and in vitro testing is consistent with and supported by other testing frameworks that use modern toxicology data.

• Finding: The current state of the science for prediction of acute toxicity with computational approaches is seeing steady growth. An investment by DOD in computational approaches could yield benefits in characterizing the toxicity of chemicals on which toxicity data are sparse or lacking.
• Finding: Prediction of acute toxicity with HTS assays is in its infancy, and DOD will need to evaluate what assays or approaches might be applicable for evaluating acute toxicity for its system and consider the lessons learned from current large-scale HTS programs. Regardless, an investment by DOD in HTS approaches could yield benefits in characterizing the toxicity of chemicals on which few or no toxicity data are available. HTS approaches might prove useful in excluding chemicals that have low toxic potential (for example, rodent LD₅₀ greater than 1,000 mg/kg) from further testing and might also help to identify more toxic chemicals of concern for further testing.
• Finding: There are sufficient data to suggest that DOD could use simple cytotoxicity assays to identify chemicals that have low acute-toxicity potential; this would allow it to focus its attention on more toxic chemicals. Additional effort is needed to determine whether the assays are relevant for the identification of highly toxic chemicals that could be used against deployed troops. Furthermore, because simple cytotoxicity assays fail to predict toxicity of highly toxic chemicals that act on specific molecular targets, such as neuronal synapses, they would need to be supplemented with assays designed specifically to detect such effects.
• Finding: On the basis of scientific advances, the committtee concludes that the development of appropriate cellular models and targeted mechanistically based assays could provide DOD with a useful resource for understanding and predicting potential toxicity of chemicals. In particular, having more explicit knowledge available on the numerous mechanisms of action that lead to acute systemic toxicity would be valuable in the design and validation of integrated prediction methods.
• Recommendation: DOD should initiate pilot studies that evaluate chemical classes of highest concern with well-characterized reference chemicals. The pilot studies will allow it to develop the assays and tools that are needed to predict acute chemical toxicity efficiently and accurately and to evaluate the rate of false negatives and false positives.

REFERENCES

Akula, N., L. Lecanu, J. Greeson, and V. Papadopoulos. 2006. 3D QSAR studies of AChE inhibitors based on molecular docking scores and CoMFA. Bioorg. Med. Chem. Lett. 16(24):6277-6280.

Ankley, G.T., R.S. Bennett, R.J. Erickson, D.J. Hoff, M.W. Hornung, R.D. Johnson, D.R. Mount, J.W. Nichols, C.L. Russom, P.K. Schmieder, J.A. Serrrano, J.E. Tietge, and D.L. Villeneuve. 2010. Adverse outcome pathways: A conceptual framework to support ecotoxicology research and risk assessment. Environ. Toxicol. Chem. 29(3):730-741.

Barril, X., M. Orozco, and F.J. Luque. 2001. Towards improved acetylcholinesterase inhibitors: A structural and computational approach. Mini Rev. Med. Chem. 1(3):255-266.

Becker, R.A., K.P. Friedman, T.W. Simon, M.S. Marty, G. Patlewicz, and J.C. Rowlands. 2015. An exposure: Activity profiling method for interpreting high-throughput screening data for estrogenic activity-Proof of concept. Regul. Toxicol. Pharmacol. 71(3):398-408.

Berg, E.L., E.J. Kunkel, E. Hytopoulos, and I. Plavec. 2006. Characterization of compound mechanisms and secondary activities by BioMAP analysis. J. Pharmacol. Toxicol. Meth. 53(1):67-74.

Berg, L., C.D. Andersson, E. Artursson, A. Hörnberg, A.K. Tunemalm, A. Linusson, and F. Ekström. 2011. Targeting acetylcholinesterase: Identification of chemical leads by high throughput screening, structure determination and molecular modeling. PLoS One 6(11):e26039.

Can, A. 2014. Quantitative structure-toxicity relationship (QSTR) studies on the organophosphate insecticides. Toxicol. Lett. 230(3):434-443.

Casey, W. 2014. Assessing the Biological Relevance of In Vitro Data: A Case Study Using Estrogen Pathway Signaling. Presentation at NTP Board of Scientific Counselors (BSC) Meeting, December 9-10, 2014 [online]. Available: http://ntp.niehs.nih.gov/ntp/about_ntp/bsc/2014/dec/presentations/04casey_bscdec2014_508.pdf [accessed April 6, 2015].

Chen, M., C.W. Tung, Q. Shi, L. Guo, L. Shi, H. Fang, J. Borlak, and W. Tong. 2014a. A testing strategy to predict risk for drug-induced liver injury in humans using high-content screen assays and the ‘rule-of-two’ model. Arch. Toxicol. 88(7):1439-1449.

Chen, M., H. Bisgin, L. Tong, H. Hong, H. Fang, J. Borlak, and W. Tong. 2014b. Toward predictive models for drug-induced liver injury in humans: Are we there yet? Biomark. Med. 8(2):201-213.

Clemedson, C. 2008. The European ACuteTox project: A modern integrative in vitro approach to better prediction of acute toxicity. Clin. Pharmacol. Ther. 84(2):200-202.

Clothier, R., P. Dierickx, T. Lakhanisky, M. Fabre, M. Betanzos, R. Curren, M. Sjöström, H. Raabe, N. Bourne, V. Hernandez, J. Mainez, M. Owen, S. Watts, and R. Anthonissen. 2008. A database of IC50 values and principal component analysis of results from six basal cytotoxicity assays, for use in the modelling of the in vivo and in vitro data of the EU ACuteTox project. ATLA 36(5):503-519.

Coats, E.A. 2002. Part III. 3D QSAR applications: The CoMFA steroids as a benchmark dataset for development of 3D QSAR methods . Pp. 197-213 in 3D QSAR in Drug Design, Vol. 3. Recent Advances, H. Kubinyi, G. Folkers, and Y.C. Martin, eds. New York: Kluwer.

Covell, D.G., A. Wallqvist, R. Huang, N. Thanki, A.A. Rabow, and X.J. Lu. 2005. Linking tumor cell cytotoxicity to mechanism of drug action: An integrated analysis of gene expression, small-molecule screening and structural databases. Proteins 59(3):403-433.

Cox, L.A., D. Popken, M.S. Marty, J.C. Rowlands, G. Patlewicz, K.O. Goyak, and R.A. Becker. 2014. Developing scientific confidence in HTS-derived prediction models: Lessons learned from an endocrine case study. Regul. Toxicol. Pharmacol. 69(3):443-450.

Deb, P.K., A. Sharma, P. Piplani, and R.R. Akkinepally. 2012. Molecular docking and receptor-specific 3D-QSAR studies of acetylcholinesterase inhibitors. Mol. Divers. 16(4):803-823.

Dix, D.J., K.A. Houck, M.T. Martin, A.M. Richard, R.W. Setzer, and R.J. Kavlock. 2007. The ToxCast program for prioritizing toxicity testing of environmental chemicals. Toxicol. Sci. 95(1):5-12.

Ekwall, B., F.A. Barile, A. Castaño, C. Clemedson, R.H. Clothier, P. Dierickx, B. Ekwall, M. Ferro, G. Fiskesjö, L. Garza-Ocañas, M.J. Gómez-Lechón, M. Gülden, T. Hall, B. Isomaa, A. Kahru, G. Kerszman, U. Kristen, M. Kunimoto, S. Kärenlampi, L. Lewan, A. Loukianov, T. Ohno, G. Persoone, L. Romert, T.W. Sawyer, R. Shrivastava, H. Segner, A. Stammati, N. Tanaka, M. Valentino, E. Walum, and F. Zucco. 1998. MEIC evaluation of acute systemic toxicity. Part VI. The prediction of human toxicity by rodent LD50 values and results from 61 in vitro methods. ATLA 26(Suppl. 2):617-658.

Franz, D.R. 1997. Defense against Toxin Weapons. US Army Medical Research and Materiel Command [online]. Available: http://www.usamriid.army.mil/education/defensetox/toxdefbook.pdf [accessed December 24, 2014].

Freidig, A.P., S. Dekkers, M. Verwei, E. Zvinavashe, J.G. Bessems, and J.J. van de Sandt. 2007. Development of a QSAR for worst case estimates of acute toxicity of chemically reactive compounds. Toxicol. Lett. 170(3):214-222.

Garcia-Reyero, N. 2015. Are adverse outcome pathways here to stay? Environ. Sci. Technol. 49(1):3-9.

Gennari, A., C. van den Berghe, S. Casati, J. Castell, C. Clemedson, S. Coecke, A. Colombo, R. Curren, G. Dal Negro, A. Goldberg, C. Gosmore, T. Hartung, I. Langezaal, I. Lessigiarska, W. Maas, I. Mangelsdorf, R. Parchment, P. Prieto, J. Riego Sintes, M. Ryan, G. Schmuck, K. Stitzel, W. Stokes, J.A. Vericat, and L. Gribaldo. 2004. Strategies to replace in vivo acute systemic toxicity testing. The report and recommendations of ECVAM Workshop 50. ATLA 32(4):437-459.

Glover, C.J., A.A. Rabow, Y.G. Isgor, R.H. Shoemaker, and D.G. Covell. 2007. Data mining of NCI’s anticancer screening database reveals mitochondrial complex I inhibitors cytotoxic to leukemia cell lines. Biochem. Pharmacol. 73(3):331-340.

Guo, J., M.M. Hurley, J.B. Wright, and G.H. Lushington. 2004. A docking score function for estimating lig- and-protein interactions: Application to acetylcholinesterase inhibition. J. Med. Chem. 47(22):5492-5500.

Gupta, S., and C.G. Mohan. 2011. 3D-pharmacophore model based virtual screening to identify dual-binding site and selective acetylcholinesterase inhibitors. Med. Chem. Res. 20(9):1422-1430.

Halle, W. 2003. The registry of cytotoxicity: Toxicity testing in cell cultures to predict acute toxicity (LD50) and to reduce testing in animals. ATLA 31(2):89-198.

Halwachs, S., C. Lakoma, and W. Honscha. 2013. Validation of the hepatocyte-like HPCT-1E3 cell line as an in vitro model for the prediction of acute in vivo toxicity. ATLA 41(5):369-383.

Huang, L., A.D. Shi, F. He, and X.S. Li. 2010. Synthesis, biological evaluation, and molecular modeling of berberine derivatives as potent acetylcholinesterase inhibitors. Bioorg. Med. Chem. 18(3):1244-1251.

Huang, R., S. Sakamuru, M.T. Martin, D.M. Reif, R.S. Judson, K.A. Houck, W. Casey, J.H. Hsieh, K.R. Shockley, P. Ceger, J. Fostel, K.L. Witt, W. Tong, D.M. Rotroff, T. Zhao, P. Shinn, A. Simeonov, D.J. Dix, C.P. Austin, R.J. Kavlock, R.R. Tice, and M. Xia. 2014. Profiling of the Tox21 10K compound library for agonists and antagonists of the estrogen receptor alpha signaling pathway. Sci. Rep. 4:5664.

JRC (Joint Research Centre). 2013. EURL ECVAM Recommendation on the 3T3 Neutral Red Uptake Cytotoxicity Assay for Acute Oral Toxicity Testing. Report EUR 25946. Luxembourg: Office of European Union [online]. Available: https://eurl-ecvam.jrc.ec.europa.eu/eurl-ecvam-recommendations/files-3t3/ReqNo_JRC79556_lbna25946enn.pdf [accessed December 24, 2014].

Judson, R.S., K.A. Houck, R.J. Kavlock, T.B. Knudsen, M.T. Martin, H.M. Mortensen, D.M. Reif, D.M.Rotroff, I. Shah, A.M. Richard, and D.J. Dix. 2010. In vitro screening of environmental chemicals for targeted testing prioritization: The ToxCast project. Environ. Health Perspect. 118(4):485-492.

Judson, R., K. Houck, M. Martin, T. Knudsen, R.S. Thomas, N. Sipes, I. Shah, J. Wambaugh, and K. Crofton. 2014. In vitro and modelling approaches to risk assessment from the US Environmental Protection Agency ToxCast programme. Basic Clin. Pharmacol. Toxicol. 115(1):69-76.

Kavlock, R., K. Chandler, K. Houck, S. Hunter, R. Judson, N. Kleinstreuer, T. Knudsen, M. Martin, S. Padilla, D. Reif, A. Richard, D. Rotroff, N. Sipes, and D. Dix. 2012. Update on EPA’s ToxCast program: Providing high throughput decision support tools for chemical risk management. Chem. Res. Toxicol. 25(7):1287-1302.

Kinsner-Ovaskainen, A., R. Rzepka, R. Rudowski, S. Coecke, T. Cole, and P. Prieto. 2009. Acutoxbase, an innovative database for in vitro acute toxicity studies. Toxicol In Vitro 23(3):476-485.

Kinsner-Ovaskainen, A., P. Prieto, S. Stanzel, and Kopp-Schneider. 2013. Selection of test methods to be included in a testing strategy to predict acute oral toxicity: An approach based on statistical analysis of data collected in phase 1 of the ACuteTox project. Toxical. In Vitro 27(4):1377-1394.

Kleinstreuer, N.C., ,J. Yang,, E.L Berg, T.B. Knudsen, A.M Richard, M.T Martin, D.M Reif, R.S Judson, M. Polokoff, D.J Dix, R.J Kavlock, and K.A Houck. 2014. Phenotypic screening of the ToxCast chemical library to classify toxic and therapeutic mechanisms. Nat. Biotechnol. 32(6):583-591.

Kopp-Schneider, A., P. Prieto, A. Kinsner-Ovaskainen, and S. Stanzel. 2013. Design of a testing strategy using non-animal based test methods: Lessons learnt from the ACuteTox project. Toxicol. In Vitro 27(4):1395-1401.

Kua, J., Y. Zhang, and J.A. Mccammon. 2002. Studying enzyme binding specificity in acetylcholinesterase using a combined molecular dynamics and multiple docking approach. J. Am. Chem. Soc. 124(28): 8260-8267.

Lipinski, C., and A. Hopkins. 2004. Navigating chemical space for biology and medicine. Nature 432 (7019):855-861.

Low, Y., T. Uehara, Y. Minowa, H. Yamada, Y. Ohno, T. Urushidani, A. Sedykh, E. Muratov, D. Fourches, H. Zhu, I. Rusyn, and A. Tropsha. 2011. Predicting drug-induced hepatotoxicity using QSAR and toxicogenomics approaches. Chem. Res. Toxicol. 24(8):1251-1262.

NICEATM (National Toxicology Program Interagency Center for the Evaluation of Alternative Toxicological Methods), 2006. In Vitro Cytotoxicity Test Methods for Estimating Acute Oral Systemic Toxicity. Background Review Document (BRD). NIH Publication No. 07-4518. National Institute of Health, National Institute of Environmental Health Sciences, Research Triangle Park, NC. November 2006 [online]. Available: http://ntp.niehs.nih.gov/pubhealth/evalatm/test-method-evaluations/acutesystemic-tox/in-vitro-validation/brd/index.html [accessed March 26, 2015].

NIEHS (National Institute of Environmental Health Sciences). 2009. Report on the ICCVAMICEATM/ECVAM/JaCVAM Scientific Workshop on Acute Chemical Safety Testing: Advancing In Vitro Approaches and Humane Endpoints for Systemic Toxicity Evaluations. Interagency Coordinating Committee on the Validation of Alternative Methods (ICCVAM), National Toxicology Program Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM), National Institute of Environmental Health Sciences, National Toxicology Program, Research Triangle Park, NC [online]. Available: http://ntp.niehs.nih.gov/iccvam/docs/acutetox_docs/workshop-feb09/acutewksprpt.pdf [accessed April 6, 2015].

NTP (National Toxicology Program). 2014. Validation Study of In Vitro Cytotoxicity Test Methods [online]. Available: http://ntp.niehs.nih.gov/pubhealth/evalatm/test-method-evaluations/acute-systemictox/in-vitro-validation/ [accessed April 6, 2015].

Olson, H., G. Betton, D. Robinson, K. Thomas, A. Monro, G. Kolaja, P. Lilly, J. Sanders, G. Sipes, W. Bracken, M. Dorato, K. Van Deun, P. Smith, B. Berger, and A. Heller. 2000. Concordance of the toxicity of pharmaceuticals in humans and in animals. Regul. Toxicol. Pharmacol. 32(1):56-67.

Patlewicz, G., T.W. Simon, J.C. Rowlands, R.A. Budinsky, and R.A. Becker. 2015. Proposing a scientific confidence framework to help support the application of adverse outcome pathways for regulatory purposes. Regul. Toxicol. Pharmacol. 71(3):463-477.

Prieto, P., T. Cole, R. Curren, R.M. Gibson, M. Liebsch, H. Raabe, A.M. Tuomainen, M. Whelan, and A. Kinsner-Ovaskainen. 2013a. Assessment of the predictive capacity of the 3T3 Neutral Red Uptake cytotoxicity test method to identify substances not classified for acute oral toxicity (LD50>2000 mg/kg): Results of an ECVAM validation study. Regul. Toxicol. Pharmacol. 65(3):344-365.

Prieto, P., A. Kinsner-Ovaskainen, S. Stanzel, B. Albella, P. Artursson, N. Campillo, R. Cecchelli, L. Cerrato, L. Díaz, E. Di Consiglio, A. Guerra, L. Gombau, G. Herrera, P. Honegger, C. Landry, J.E. O’Connor, J.A. Páez, G. Quintas, R. Svensson, L. Turco, M.G. Zurich, M.J. Zurbano, and A. Kopp-Schneider. 2013b. The value of selected in vitro and in silico methods to predict acute oral toxicity in a regulatory context: Results from the European Project ACuteTox. Toxical. In Vitro 27(4):1357-1376.

Rotroff, D.M., D.J. Dix, K.A. Houck, T.B. Knudsen, M.T. Martin, K.W. McLaurin, D.M. Reif, M. Crofton, A.V. Singh, M. Xia, R. Huang, and R.S. Judson. 2013. Using in vitro high throughput screening assays to identify potential endocrine disrupting chemicals. Environ. Health Perspect. 121(1):7-14.

Rotroff, D.M., D.J. Dix, K.A. Houck, T.B. Knudsen, M.T. Martin, K.W. McLaurin, D.M. Reif, K.M. Crofton, A.V. Singh, M. Xia, R. Huang, and R.S. Judson. 2014. Using in vitro high throughput screening assays to identify potential endocrine-disrupting chemicals. Environ. Sci. Technol. 48(15):8706-8716.

Russom, C.L., C.A. LaLone, D.L. Villeneuve, and G.T. Ankley. 2014. Development of an adverse outcome pathway for acetylcholinesterase inhibition leading to acute mortality. Environ. Toxicol. Chem. 33(10):2157-2169.

Rusyn, I., A. Sedykh, Y. Low, K.Z. Guyton, and A. Tropsha. 2012. Predictive modeling of chemical hazard by integrating numerical descriptors of chemical structures and short-term toxicity assay data. Toxicol. Sci. 127(1):1-9.

Samadi, A., J. Marco-Contelles, E. Soriano, M. Álvarez-Pérez, M. Chioua, A. Romero, L. González-Lafuente, L. Gandía, J.M. Roda, M.G. López, M. Villarroya, A.G. García, and C. de los Ríos. 2010. Multipotent drugs with cholinergic and neuroprotective properties for the treatment of Alzheimer and neuronal vascular diseases. I. Synthesis, biological assessment, and molecular modeling of simple and readily available 2-aminopyridine-, and 2-chloropyridine-3,5-dicarbonitriles. Bioorg. Med. Chem. 18(16):5861-5872.

Seibert, H., M. Balls, J.H. Fentem, V. Bianchi, R.H. Clothier, P.J. Dierickx, M. Garle, M.J. Gomez-Lechon, L. Gribaldo, M. Gulden, M. Liebsch, E. Rasmussen, R. Roguet, R. Shrivastana, and E. Walum, 1996. Acute toxicity testing in vitro and the classification and labelling of chemicals. The report and recommendations of ECVAM Workshop 16. ATLA 24:499-510.

Seidle, T., S. Robinson, T. Holmes, S. Creton, P. Prieto, J. Scheel, and M. Chlebus. 2010. Cross-sector review of drivers and available 3Rs approaches for acute systemic toxicity testing. Toxicol. Sci. 116(2):382-396.

Siddell, F.R., E.T. Takafuji, and D.R. Franz, eds. 1997. Medical Aspects of Chemical and Biological Warfare. Textbook of Medicine, Part 1. Warfare, Weaponry and the Casualty, Vol. 3. Office of Surgeon General, Borden Institute, Walter Reed Army Medical center, Washington, DC.

Sipes, N.S., M.T. Martin, P. Kothiya, D.M. Reif, R.S. Judson, A.M. Richard, K.A. Houck, D.J. Dix, R.J. Kavlock, and T.B. Knudsen. 2013. Profiling 976 ToxCast chemicals across 331 enzymatic and receptor signaling assays. Chem. Res. Toxicol. 26(6):878-895.

Sjöström, M., A. Kolman, C. Clemedson, and R. Clothier. 2008. Estimation of human blood LC50 values for use in modeling of in vitro-in vivo data of the ACuteTox project. Toxicol. In Vitro 22(5):1405-1411.

Somani, S.M., and J.A. Romano, eds. 2001. Chemical Warfare Agents: Toxicity at Low Levels. Boca Ra-ton, FL: CRC Press.

Sopkova de Oliveira Santos, J., A. Lesnard, J.H. Agondanou, N. Dupont, A.M. Godard, S. Stiebing, C. Rochais, F. Fabis, P. Dallemagne, R. Bureau, and S. Rault. 2010. Virtual screening discovery of new acetylcholinesterase inhibitors issued from CERMN chemical library. J. Chem. Inf. Model. 50(3):422-428..

Stephens, C., R.J. Andrade, and M.I. Lucena. 2014. Mechanisms of drug-induced liver injury. Curr. Opin. Allergy Clin. Immunol. 14(4):286-292.

Stokes, W.S., S. Casati, J. Strickland, and M. Paris. 2008. Neutral red uptake cytotoxicity tests for estimating starting doses for acute oral toxicity tests. Curr. Protoc. Toxicol. (Suppl. 36):Unit 20.4.

Stokes, W.S., J. Strickland, and W. Casey. 2012. Validation of the 21st century toxicology toolbox: Challenges, opportunities, and the way forward. Pp. 323-328 in Proceedings of the 8th World Congress on Alternatives and Animal Use in the Life Sciences, Montreal 2011. ALTEX Proceedings Vol. 1 [online]. Available: http://www.altex.ch/resources/WC8proceedings_full_issue.pdf [accessed March 27, 2015].

Thomas, R.S., M.B. Black, L. Li, E. Healy, T.M. Chu, W. Bao, M.E. Andersen, and R.D. Wolfinger. 2012. A comprehensive statistical analysis of predicting in vivo hazard using high-throughput in vitro screening. Toxicol. Sci. 128(2):398-417.

Thomas, R.S., M.A. Philbert, S.S. Auerbach, B.A. Wetmore, M.J. Devito, I. Cote, J.C. Rowlands, M.P. Whelan, S.M. Hays, M.E. Andersen, M.E. Meek, L.W. Reiter, J.C. Lambert, H.J. Clewell, III, M.L. Stephens, Q.J. Zhao, S.C. Wesselkamper, L. Flowers, E.W. Carney, T.P. Pastoor, D.D. Petersen, C.L. Yauk, and A. Nong. 2013. Incorporating new technologies into toxicity testing and risk assessment: Moving from 21st century vision to a data-driven framework. Toxicol. Sci. 136(1):4-18.

Tice, R.R., C.P. Austin, R.J. Kavlock, and J.R. Bucher. 2013. Improving the human hazard characterization of chemicals: a Tox21 update. Environ. Health Perspect. 121(7):756-765.

Tollefsen, K.E., S. Scholz, M.T. Cronin, S.W. Edwards, J. de Knecht, K. Crofton, N. Garcia-Reyero, T. Hartung, A. Worth, and G. Patlewicz. 2014. Applying Adverse Outcome Pathways (AOPs) to support Integrated Approaches to Testing and Assessment (IATA). Regul. Toxicol. Pharmacol. 70(3):629-640.

van der Burg, B., E.B. Wedebye, D.R. Dietrich, J. Jaworska, I. Mangelsdorf, E. Paune, M. Schwarz, A.H. Piersma, and E.D. Kroese. 2015. The ChemScreen project to design a pragmatic alternative approach to predict reproductive toxicity of chemicals. Reprod. Toxicol 55(1):114-123.

Wind. M., and W.S. Stokes. 2010. Developing performance standards to expedite validation of innovative and improved test methods. ALTEX 27(Special Issue):97-102.

Xia, M., R. Huang, K.L. Witt, N. Southall, J. Fostel, M.H. Cho, A. Jadhav, C.S. Smith, J. Inglese, C.J. Portier, R.R. Tice, and C.P. Austin. 2008. Compound cytotoxicity profiling using quantitative high-throughput screening. Environ. Health Perspect. 116(3):284-291.

Xu, J.J., D. Diaz, and P.J. O’Brien. 2004. Applications of cytotoxicity assays and pre-lethal mechanistic assays for assessment of human hepatotoxicity potential. Chem. Biol. Interact. 150(1):115-128.

Zhu, X, A. Sedykh, and S. Liu. 2014. Hybrid in silico models for drug-induced liver injury using chemical descriptors and in vitro cell-imaging information J. Appl. Toxicol. 34(3):281-288.