2

Committee’s Approach to Its Task

In response to the U.S. Department of Defense’s (DoD’s) request, the National Academies assembled a committee of eight members (biographical sketches of the members are presented in Appendix B). In the course of preparing its report, the committee held three meetings. Its June 2019 meeting included a public information-gathering session to hear presentations from representatives of DoD, the U.S. Environmental Protection Agency (EPA), and the California Environmental Protection Agency on issues relevant to DoD’s efforts for developing an occupational exposure limit (OEL) and the use of the DoD-O’Flaherty model to support that activity. In addition, the committee considered relevant written material it received from DoD and other organizations.

It is important to note that the committee was not asked to, nor did it attempt to carry out, a comparative assessment of various modeling approaches to determine if the DoD-O’Flaherty model is the best model that could be used for developing an OEL for DoD workers. Consistent with its Statement of Task, the committee focused on evaluating whether DoD’s selected modeling approach and the application of the approach were appropriate for deriving candidate lead OELs.

The committee did not perform a comprehensive evaluation of the DoD-O’Flaherty model framework, including the model code, equations, and model variables that define the model. In addition, the committee did not attempt to verify the ability of the model to reproduce results reported in Sweeney (2019), which presents model estimates of the workplace airborne lead concentrations that would correspond to maintaining personnel blood lead levels (BLLs) below various specified concentrations. While such activities are important aspects of developing a biokinetic model for regulatory application, the committee determined that such activities were not necessary to carry out its Statement of Task.

The DoD-O’Flaherty model is intended to derive TWA candidate OELs that would be used to maintain the BLL of the 95th percentile DoD employee below a specified BLL for a working lifetime. The committee did not consider how the model might be used for assessing short-term exposures (e.g., over 1 day or week), because that would be a use of the model that is separate from supporting the establishment of an OEL.

CONSIDERATIONS FOR DETERMINING MODEL APPROPRIATENESS

In its evaluation of model appropriateness, the committee considered questions posed in its Statement of Task and additional questions it selected from those commonly considered in reviews of physiologically-based pharmacokinetic models (see, e.g., Barton et al. 2007; Chiu et al. 2007; Clark et al. 2004; McLanahan et al. 2012). Selection of additional aspects for review supported a fuller assessment of model appropriateness by providing focus and detail to the broad questions posed by DoD. Additionally, the committee focused its evaluation on the elements of biokinetic modeling that may have the greatest impact on the predicted relationship between exposure concentrations of airborne lead and BLLs of adults. The questions addressed by the committee were organized into four broad categories, which were ordered to reflect DoD’s model selection and development process. Presented in the order they are addressed in this report, the questions are:

• Was an appropriate model chosen?
• Were structural modifications appropriately justified?

• Were model assumptions and inputs reasonable?
• Was the model application appropriate?

The committee’s final summary conclusions regarding the approach and application of the approach are built from the analyses of those four broad questions. In assessing whether specific aspects of DoD’s modeling approach and its application were appropriate or reasonable, the committee often relied on its professional judgment, as it was not feasible to apply specific written standards.

COMMITTEE’S USE OF TERMS

In Sweeney (2015, 2019), the term “model parameter” is used to refer to the components of the model structure and the term “inputs” is used to refer to the parameter values, where some inputs are described as probability distributions for sets of parameters. The committee elected to clarify its charge to evaluate “parameterization” of the model by adopting definitions consistent with EPA’s guidance (EPA 2001a; see Box 2-1). The committee used the term “model variable” when referring to a factor used in an equation, whereas “model parameter” is the value assigned to that variable. The committee focused its review on the appropriateness of the parameter values selected to represent point estimates and probability distributions.

The terms validation, verification, calibration, and evaluation are commonly used in reporting the outcomes from an assessment of biokinetic models, such as the DoD-O’Flaherty model. While terms such as verification and validation are routinely and even interchangeably used, these terms have specific meanings (Oreskes et al. 1994; see Box 2-1) and typically do not apply to the development and assessment of most biokinetic models. The committee found that DoD’s efforts involved activities best described as model calibration and model confirmation, rather than verification and validation, which can rarely be established for models of open systems (EPA 2009a; Oreskes et al. 1994).