Research Priorities for Airborne Particulate Matter: III. Early Research Progress (2001)

In its first two reports (NRC 1998, 1999), the committee offered criteria for evaluating various qualities of research and its policy relevance for informing the review of the national ambient air quality standards (NAAQS) for airborne particulate matter (PM). In preparing this, its third report, the committee began the task of applying the evaluation criteria to determine whether progress has been made in implementing the PM research program and to assess the rate of progress, in each priority subject. The evaluation effort should help to ensure that the PM research program provides quantitative guidance on key issues, such as the dose-response relationships of PM and the association of PM characteristics with health effects. It should also help to ensure that the PM research program provides qualitative guidance on some topics, such as physiological mechanisms of PM toxicity and atmospheric processes that lead to airborne-particle formation. And the committee's evaluation has the goal of helping to ensure the effective implementation of the research program.

The committee arrayed the PM research projects sponsored by the Environmental Protection Agency (EPA) and other institutions against

the committee's recommended research portfolio to assess the extent to which research is addressing the major issues that decisionmakers need to consider as they review the scientific evidence relevant to the PM NAAQS. It then evaluated current and planned research activities with six criteria, which are described below. The first three—scientific value, decisionmaking value, and timing and feasibility ¹ —are research-outcome criteria, and they were applied to research in each of the committee's 10 topics (see Chapter 3). The last three—multidisciplinary interaction, integration and planning, and accessibility of information—are research-planning criteria, and they were applied more generally than the other three (see Chapter 4).

Where possible and appropriate, the committee applied quantitative indicators to evaluate progress in implementing PM research, but most of the indicators were qualitative and were based on professional judgment.

The first three evaluation criteria guided the development of the committee's research portfolio, including recommended research topics, estimated budgets, and approximate periods for conducting research (see Table 1.3). The committee concluded that these criteria serve equally well as outcome measures for evaluating progress in obtaining research results that decisionmakers will use in determining whether to revise the PM NAAQS in 2002.

How well does the research build on previous scientific findings to address key scientific uncertainties in a source-to-effect framework (see Figure 1.1)? The research should contribute to the development

of a scientifically credible and integrated understanding of the health effects of PM and gaseous copollutants. The investigations should address testable hypotheses and provide reproducible results that can be generalized, to some degree, beyond the immediate study being performed. Specific indicators of progress for this criterion include the following:

• Number and distribution of research projects that address key uncertainties within each research-priority topic. There should not be overemphasis on a few kinds of uncertainty while others, which could be addressed, are not covered.

• Adequacy of studies in considering the heterogeneous nature of particles.

• Adequacy of addressing specific potential characteristics of PM and other factors that appear to be involved in PM-induced health effects.

• Usefulness of study results for forming hypotheses for future studies and for helping to build new research capacity and skills that might be valuable for addressing future questions about PM and other air pollutants.

• Extent to which new results of studies strengthen the bases of existing conclusions that lack a clear interpretation.

• Extent to which collective judgment of expert groups, such as EPA 's Clean Air Scientific Advisory Committee and other scientific bodies, indicates a substantial increase in the scientific value of completed research.

How well does the research contribute to reducing key uncertainties associated with regulatory standard-setting and risk-management decisions for the next scheduled review of the NAAQS in 2002 and for later reviews? New research should provide more-accurate and more-substantial knowledge to address uncertainties about sources of PM, biologically important PM constituents and mechanisms, ambient

concentrations, personal exposures, dose-response relationships, and the extent of short-term and long-term risk to human health. Performance indicators for this criterion include the following:

• Usefulness of results in identifying the magnitude of uncertainty in the scientific information provided to decisionmakers. Studies should be evaluated on the basis of the extent to which results provide a means of testing the reliability of analytical tools, such as newly developed air-quality models.

• Usefulness of results in defining adverse effects and susceptible subpopulations.

• Usefulness of results in informing decisions concerning the setting and implementation of standards, including all four elements of a standard:

  • Indicator—the specific measurement (for example, mass, chemical form, or size fraction, such as PM_2.5) of airborne particles that is important to control for protection of public health.

  • Concentration—the amount per unit volume of air.

  • Averaging time—the period over which measurements are made or averaged (for example, annual or 24-hour periods).

  • Form—the statistical nature of the standard, used for determining the allowable number of exceedences per averaging time (for example, the 98th percentile).

• Usefulness of results for implementation of strategies for achieving the NAAQS for PM and other pollutants, including identification of emission sources and development of emission-control strategies.

Is the research technically, operationally, and financially feasible? The technical methods needed to conduct the research should be available. There should be sufficient research capacity and expertise to achieve the research objectives, and the research objectives should

be achievable within reasonable budgets. Indicators of performance include the following:

• Sequencing of research projects in such a way that the results of early investigations can be used advantageously in the planning and conduct of later research and regulatory reviews.

• Conduct of research in a timeframe responsive to decisionmakers' needs.

• Availability of new scientific information for the air-quality criteria document, staff paper, and standards development.

• When appropriate, conduct of research to develop the capability of addressing a key uncertainty that had not been technically feasible to address.

• Adequacy of funding for high-priority research and adequate agreement of allocations of research funding with resource estimates across the research portfolio.

The final three criteria are relevant to evaluating the planning, management, and implementation of the recommended research to ensure that criteria 1-3 are achieved. The first three criteria address the question, Is the needed information being obtained? The final three ask, Is the information being effectively sought and made available?

How well do scientists involved in PM research collaborate across scientific disciplines? This criterion is highly pertinent to the effective involvement of science managers during the planning and funding of scientific research. The success of the PM research program depends heavily on the ability of government and extramural scientists in a variety of institutions to collaborate across disciplines and organiza-

tions and to communicate the status and progress of their research in an effective and accessible manner. Success in meeting this criterion will enhance the timing and feasibility of the planned research. Specific performance indicators include the following:

• Specification of disciplines and skills needed to achieve explicit research objectives.

• Use of resources in a cooperative manner or, where appropriate, for mutually funded research.

• Extent to which hypotheses based on data in one domain are tested in another, complementary domain (for example, evidence obtained in the laboratory is tested by observing the population).

• Consistent use of terminology and measures, for example names for the size fractions of PM characteristics, so that study comparisons are possible not only between disciplines, but also between control studies.

• Encouragement of investigators to be broadly aware of the information that their research can provide to researchers in other disciplines.

• Proportion of completed projects and peer-reviewed publications that have multidisciplinary participation.

How well are research planning, budgeting, and management integrated to optimize the use of financial resources, scientific talent, and infrastructure across governmental and private organizations? The diversity of sponsors and the many kinds of scientific investigations under way increase the need for federal interagency coordination and management of the effort. This criterion represents both preliminary planning and postproject analyses to fit the various pieces of the research puzzle together to provide timely guidance and feedback for achieving scientific and decisionmaking value. Performance indicators include the following:

• Effective use of available financial and human resources and, where appropriate, leveraging of resources with other scientific studies (leveraging is the degree to which the results of a research activity can increase the value of other research results).

• Formal statement of research objectives and timeframes for each of the major research activities that address PM.

• Differentiation between the kinds of research that can be conducted intramurally and extramurally, including research best sponsored jointly with the private sector, other federal agencies, or nonprofit research institutions.

• Data-sharing among scientific investigators, when feasible, to aid research planning.

• Planning to design and coordinate research projects within EPA and between EPA and other institutions and to include relevant activities from other research programs, and use of planning to address the process by which needed results will be interpreted and used as they become available.

• Speed and effectiveness of evaluation of new information and its incorporation into decisionmaking.

How effectively is information concerning research planning, budgeting, progress, and results communicated and shared among research organizations and other interested parties? This criterion represents a measure of the continuing ability of research sponsors, scientists, decisionmakers, and other interested parties to gain access to and use information appropriate to their needs. Performance indicators for this criterion include the following:

• Availability of research plans to interested parties.

• Maintenance of an inventory of current PM research projects, which (as described later in this chapter) should be continually updated to serve as a status report of research in progress.

• Publication of periodic reports that synthesize the status of research activities and the progress being made to reduce key uncertainties; such reports should be integrated summaries that enable PM investigators, the broader scientific community, regulatory decisionmakers, Congress, and the public to understand major accomplishments and identify key challenges still to be met.

In conducting its evaluation, the committee applied its criteria to research activities in the 10 priority areas. The committee recommended adjustments in the implementation of the research program with respect to key uncertainties, rather than specific studies.

The committee has applied the six criteria in a flexible, commonsense fashion that relies on professional judgment and recognizes the practicalities of conducting a large research program in support of public-policy decisions. In addition, the committee is aware that not every research project conducted through this program should be expected to meet the criteria to the same degree. For example, different timeframes for applying specific criteria must be taken into account.

The committee is applying the criteria with a near-term perspective, such as the rapid startup of a large number of exposure-assessment studies, and with a long-term perspective, such as the effectiveness of the interagency Committee on Environment and Natural Resources (CENR) as a planning mechanism for PM research. However, the preliminary nature of many of the results emanating from application of the committee 's research portfolio by EPA and other agencies places constraints on the present application of the evaluation criteria. Thus, the committee's evaluation represents an interim review of the PM research program and is subject to update and modification as additional data become available. Also, because of such constraints, the committee did not attempt to determine if any changes should be made to the resource estimates of the research portfolio.

The committee used various sources of information to assess progress in PM research, including published papers and abstracts from scientific meetings, such as the Health Effects Institute (HEI) annual conferences of 1999 and 2000 (HEI 1999; 2000), the Third Colloquium on Particulate Air Pollution and Human Health held in June 1999 (Phalen and Bell 1999), and the PM2000 Conference held in January 2000 (AWMA 2000). EPA representatives provided briefings, documents, and responses to committee questions. The committee examined published reports and written descriptions of ongoing research projects to sort relevant studies into appropriate research topics within the committee's research portfolio. Some studies were found to be relevant to more than one topic. The committee did not attempt to use its evaluation criteria as means of selecting studies to be in this report.

The principal source of information on research begun in response to the committee's research portfolio is the PM research inventory database. Beginning with its first report, the committee has recommended, as an essential element of the PM research program, the development and maintenance of an “evergreen” inventory of all PM research. That inventory now exists at an Internet site (www.pmra.org) developed and maintained by HEI in collaboration with EPA. The inventory contains a searchable database of descriptions of nearly 500 current PM research projects in a variety of disciplines throughout the world; it is being used by about 1,200 visitors per month. Most recently, the entire federal government program of PM research was added as a result of efforts of the CENR Air Quality Subcommittee. In addition to the current projects accessible through the Web site, all completed projects are maintained in an archive that will allow a complete evaluation of the PM research effort.

The establishment of the inventory has been an important step in implementing the current PM research effort. At the same time, the need to draw on disparate reports from various research-funding agencies to compile the inventory has, of necessity, resulted in some duplication and potential omission. Thus, although the inventory

clearly gives a sound overall picture of the current PM research program, there will undoubtedly be some inaccuracies in the descriptions of individual projects. Among the tasks set out by HEI and EPA for the next steps are development of procedures for regularly updating and improving the accuracy of the entries and development of a mechanism for tracking research results that emerge from the projects under way.