3
COMMENTS ON THE ASSESSMENT’S EXECUTIVE SUMMARY

The NARSTO PM assessment draft begins with a 53-page “Executive Summary.” The assessment cochairs indicated to the committee that the summary would be published with the full document and also separately in English, Spanish, and French (Hales 2002; Shepherd 2002; Vickery 2002). The executive summary is currently structured with several distinct sections:

  • A short introduction.

  • A list of 19 “key insights” considered to be the most important scientific results that decision-makers should know.

  • Answers to eight policy questions (PQs) with some figures and tables drawn from the chapters and indications of where in the chapters the supporting evidence could be found.

  • Five major recommendations, which pertain primarily to research needs.

  • A narrative titled “Highlights of the Assessment.”

The assessment cochairs indicated that they chose to structure the executive summary in this way so that it would be capable of speaking to three audiences: senior decision-makers, scientific advisers to the senior decision-makers, and the science and science-user community (M. Shepherd, Meteorological Service of Canada, personal communication, April 26, 2002).

RESTRUCTURING OF THE EXECUTIVE SUMMARY

The committee finds that the executive summary as written does not conform to standard expectations of executive summaries in that it is too long and too technical for the intended decision-maker audience. A separate, very brief executive summary that discusses in a terse manner the important points presented in the document should be written. Nevertheless, a longer section that is similar to the current executive summary, that treats the issues more fully, and that responds in more detail to the PQs is valuable, particularly if it is aimed at the scientific advisers to decision-makers. This longer section could be called “Synthesis of Key Issues” or have some other title of NARSTO’s choice. Both the brief executive summary and the longer synthesis need



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter 3 COMMENTS ON THE ASSESSMENT’S EXECUTIVE SUMMARY The NARSTO PM assessment draft begins with a 53-page “Executive Summary.” The assessment cochairs indicated to the committee that the summary would be published with the full document and also separately in English, Spanish, and French (Hales 2002; Shepherd 2002; Vickery 2002). The executive summary is currently structured with several distinct sections: A short introduction. A list of 19 “key insights” considered to be the most important scientific results that decision-makers should know. Answers to eight policy questions (PQs) with some figures and tables drawn from the chapters and indications of where in the chapters the supporting evidence could be found. Five major recommendations, which pertain primarily to research needs. A narrative titled “Highlights of the Assessment.” The assessment cochairs indicated that they chose to structure the executive summary in this way so that it would be capable of speaking to three audiences: senior decision-makers, scientific advisers to the senior decision-makers, and the science and science-user community (M. Shepherd, Meteorological Service of Canada, personal communication, April 26, 2002). RESTRUCTURING OF THE EXECUTIVE SUMMARY The committee finds that the executive summary as written does not conform to standard expectations of executive summaries in that it is too long and too technical for the intended decision-maker audience. A separate, very brief executive summary that discusses in a terse manner the important points presented in the document should be written. Nevertheless, a longer section that is similar to the current executive summary, that treats the issues more fully, and that responds in more detail to the PQs is valuable, particularly if it is aimed at the scientific advisers to decision-makers. This longer section could be called “Synthesis of Key Issues” or have some other title of NARSTO’s choice. Both the brief executive summary and the longer synthesis need

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter to be written in plain language, avoiding atmospheric-science jargon as much as possible. In the synthesis of key issues, the objectives of the assessment and its limitations and assumptions need to be clearly described. The committee recommends that the charge, goals and specific objectives, and assumptions set forth by NARSTO for the assessment (see Box 1–1) be included in their entirety in the synthesis. At present, that information appears nowhere in the document. Without such material, the reader is uncertain why this assessment was written and what its intended audience is. Of particular importance is an explicit definition of the “PM problem” in the context of NARSTO’s first assumption for the assessment (see Box 3–1 in the present report). At no point in the current executive summary is it clearly and explicitly stated how the PM problem is defined. When the committee queried the assessment cochairs, they indicated that they considered the PM problem to be exceedance of existing or proposed mass-based standards. The committee appreciates why the authors focused on providing scientific and technical guidance toward meeting the standards, but this substantial limitation of the scope needs to be made clear. In addition to clearly stating the objectives of the assessment, the synthesis of key issues should provide a crosswalk between the objectives and the main body of the assessment. One way to accomplish that would be to align the discussion in the synthesis with the objectives. The committee has developed an outline that shows how this could be done (see Box 3–1). According to the suggested outline, the synthesis of key issues would begin with a discussion of why the assessment was written and a presentation of the charge, goal, objectives, and assumptions that guided and limited the preparation of the document. The second section of the synthesis would explain the PM problem in terms of a framework for informing airborne-PM management, thereby addressing NARSTO objectives 2 and 3. As discussed in more detail previously (in Chapter 2 of this report), the framework should enable a reader to understand what PM is, how the various processes that influence it interact, and what is involved in assessing and managing the PM problem. The third section would describe how the task was approached, with a focus on explaining the interactions with the policy community, which is NARSTO objective 1. The fourth section would step through the eight PQs, providing answers that would be useful to research managers (NARSTO objective 4) and linking the answers to specific atmospheric-science research accomplishments and needs (NARSTO objective 5). The last section of the synthesis would address NARSTO objective 6 by providing a context for researchers in related fields. OTHER COMMENTS ON THE EXECUTIVE SUMMARY Elicitation of the Policy Questions The committee commends NARSTO for taking the time to query decision-makers about their scientific and technical needs. Efforts to identify what information would be useful to the intended audience before writing a scientific assessment of this sort are rare. A document providing the information that decision-makers expressly request is certainly valuable, and presenting this information as responses to specific PQs is a useful format.

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter BOX 3–1 Suggested outline for a revised NARSTO PM Assessment Preface: What NARSTO is and why it is an appropriate group to do an assessment of airborne PM Executive Summary: A few pages of the most important messages for policy-makers Synthesis of Key Issues: Discussion of why the report was written, highlighting the need to assist the policy community in interpreting complex and new scientific information associated with airborne PM Motivation Charge (see Box 1–1 of this report) Goals and the six specific objectives (see Box 1–1 of this report) Assumptions (see Box 1–1 of this report) Framing of the problem in the context of the air-quality standards (introduced here) Explanation of the PM problem via a framework for informing airborne-PM management accompanied by plain-language discussion and multiple figures (NARSTO objectives 2 and 3) Rationale of the framework Factors that affect PM, including anthropogenic and natural sources, seasonally, and regional and local contributions (use existing text from the response to PQ3) Description of the exercise and of how the task was approached Interaction with policy community (NARSTO objective 1) Approach to eliciting input from decision-makers Scope of decision-makers polled Approach to reviewing scientific information, selecting authors, and choosing citations Outline of the full assessment document Review of issues framed by the PQs, addressing all the following topics for each question (NARSTO objectives 4 and 5) Current answer to each question, stated clearly and succinctly Key knowledge that supports the answer, highlighting recent findings and advances (use existing text from “Key Insights” and “Highlights of the Assessment”) Additional atmospheric-science knowledge needed to answer the question better (use existing text from “Recommendations”) How the recommended research may affect air-quality management decisions In the overall context of the eight PQs, discussion of how new atmospheric-science knowledge would inform research in other fields (including health, ecosystems, climate, acid deposition, and visibility) and how research in those fields can help to guide atmospheric-science research (NARSTO objective 6) Body of the Report Appendixes

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter The committee is concerned, however, that the method for selecting and formulating the PQs is not clearly articulated and thus lacks transparency in the assessment document. To assist in the committee’s deliberations, the assessment cochairs provided a summary of how the eight PQs were identified (M.Shepherd, Meteorological Service of Canada, personal communication, April 26, 2002). The committee’s understanding of the process is that the assessment cochairs distilled the PQs from a larger set of questions developed at a number of NARSTO workshops. The set of eight PQs was refined by the assessment cochairs with input from NARSTO members and finalized at the first meeting of the assessment authors. Later, the assessment cochairs conducted interviews with senior decision-makers in federal, state, and provincial environment departments and in private industry in Canada, Mexico, and the United States. The interviews included questions about policy goals for PM management, gaps in science, how science fits into decision-making, and how to present scientific information. As part of the interview, decision-makers were asked to comment on whether the eight PQs “capture the issues that need to be resolved.” On the basis of the comments, the assessment cochairs decided that the decision-makers had confirmed the usefulness of the eight PQs previously developed. The committee finds the method used to select and formulate the eight PQs unsatisfactory. The approach does not conform to current social-science methods. The potential stakeholders assembled for interviews were not representative. Only government and industry representatives were selected, and only five of the 45 interviews were with decision-makers from Mexico. The resulting understanding of what information decision-makers need from the atmospheric-science community may be inadequate. Another deficiency is the apparent lack of a well-constructed interview protocol. Several of the questions posed to the decision-makers called for a mix of prompted and unprompted responses. For example, one question asked of decision-makers was “What issues need to be resolved to achieve these [policy] goals?” In providing some context for the question, the interviewers listed the eight PQs and asked whether they “capture the issues that need to be resolved.” A respondent would probably be confused about how to answer the initial question, particularly about whether to contribute a separate list of issues or to simply affirm that the list provided was sufficient. As a result of that deficiency, the eight questions do not necessarily represent the highest-priority questions about PM identified by policy-makers themselves but, rather the policy-relevant questions about PM that the assessment authors thought were the most important, which could be substantially differentLikewise, as far as the committee can tell, no rigorous quantitative analysis was used to assess the responses. In their presentation to the committee, the assessment cochairs showed only bar graphs tabulating responses by decision-makers to various questions (Vickery 2002). The total number of responses varied greatly for different questions. Indeed, it appears difficult to tabulate answers to questions like the example above because some respondents might contribute their own list of issues and others might just affirm that the provided list was sufficient. If a rigorous quantitative analysis was conducted, the committee recommends that the method be transparent in the assessment document and that the results be presented in the synthesis of key issues. Despite the problems with how the PQs were obtained and characterized, the committee recognizes that the exercise of generating them cannot be redone at this point. Furthermore, despite their limitations, the PQs appear generally appropriate, with some exceptions that are noted below. The committee recommends that the PQs be retained as long as the method by

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter which they were developed, including all its limitations, is clearly explained (Part 3a in the revised outline). It is particularly important to state that the PQs were developed by the assessment team, which later sought confirmation from decision-makers about their policy relevance. The committee strongly recommends that social scientists with expertise in elicitation of information be engaged in the process of developing policy guidance for future assessments conducted by NARSTO. Introduction of the PM Standards Despite defining the PM problem in terms of exceeding existing or expected standards for ambient PM concentrations, the executive summary does not clearly introduce the standards in the three nations. The discussion in response to PQ1 does mention where observed concentrations of PM exceed the national standards; however, this discussion is confusing because there are many different standards. For example, the U.S. 24-h PM2.5 standard of 65 µg/m3 is higher than the Canadian 24-h PM2.5 standard of 30 µg/m3, and Mexico does not have a standard for PM2.5. The committee recommends that the standards, when they became or will become effective, and whether or when they will be reviewed be presented in the synthesis of key issues. The discussion should clarify that different sizes of PM are regulated, that the standards have different averaging times and statistical forms, and that each nation has set standards to address multiple regulatory goals (such as, protecting human health and improving visibility). A table comparing the different standards may be effective for summarizing the information. It should also be mentioned that although the PM problem is defined in the draft assessment in the context of the standards, health problems associated with PM have been detected at concentrations below those of the standards (Samet et al. 2000). In addition, health problems associated with PM likely exist in countries where standards have not been set, and the document should not be written in a way that may imply the contrary. Integration of “Key Insights” and “Highlights of the Assessment” into Discussion of Policy Questions The executive summary starts with a list of 19 “key insights” that are intended to provide the “most important scientific aspects of current knowledge relevant to the atmospheric-science underpinnings of multinational PM-management strategies” (M.Shepherd, Meteorological Service of Canada, personal communication, April 26, 2002). Although the key insights are interesting and useful, the committee finds that having them as the first section of the executive summary is not effective for communicating them to decision-makers, who may not at that point in their reading of the document have a basic conceptual understanding of PM. The insights should be presented in such a way as to clearly indicate what scientists do and do not know, to be connected to the PQs and to the framework for airborne-PM management, and to provide a rationale for research recommendations. The final section of the current executive summary is a narrative titled “Highlights of the Assessment.” The committee finds that this section is generally

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter well written but that it contains much of the information presented elsewhere in the executive summary. Some of the text in the highlights section could be used in section 2 of the synthesis of key issues, where the framework for informing airborne-PM management is described. As in the case of the key insights, the ties between the information presented in the highlights section and the PQs, the general framework for informing airborne-PM management, and the research recommendations could be clarified. To address those concerns, the committee recommends that the key insights and some of the material in the highlights section be incorporated into the responses to the eight PQs, as indicated in the suggested outline of the synthesis of key issues presented in Box 3–1. The response to each PQ needs to include the scientific knowledge available to answer the question, which is basically the information provided in the current key insights. Presenting this scientific information with the PQ will clearly demonstrate that recent scientific advances relevant to policy needs have been made. Describing the scientific knowledge that is available to answer the PQs could easily lead into a discussion of gaps in the understanding of PM and the need for additional research. Indeed, the recommendations listed on the current pages ES-32 to ES-33 could be presented in text boxes accompanying the discussion, thereby highlighting the connection between the PQs and the research needed to answer them. Use of Figures and Tables A number of figures and tables are presented in the executive summary to illustrate points made in the text. Most of the figures are brought forward from the body of the draft assessment, where they are discussed in more detail, and many are reproductions of figures used in other publications. Although the figures and tables provide useful additional information and enhance communication of many issues, the committee finds that they are not adequately integrated into the executive summary. The reader is left to determine the relevance of most of them and to identify the important information they convey. For the more technical figures, it is also important to provide more complete explanations in the captions, as was done well for Figure 6.4 (used to address PQ4). COMMENTS ON THE POLICY QUESTIONS The eight PQs are the primary mechanism by which the draft executive summary communicates to decision-makers. The committee directed substantial energy to assessing the questions and the responses to them. As discussed in the previous section, the committee has some reservations about how the PQs were developed and used, but it finds that they are a useful set of questions nonetheless. Some suggestions for rewording the questions are given below, but the committee recommends that the general intent of each question be retained. The committee recognizes that rewording the questions presents some difficulty for NARSTO in explaining its prior elicitation activities, but feels that the improvement in communication gained by clarifying the questions outweighs this concern. In general, the committee finds that presenting the policy-

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter relevant information in a question-answer format is effective for communicating to decision-makers. Many of the responses to the questions should be revised. In general, each response should address the four topics described in Part 4 of the suggested outline for the synthesis of key issues (see Box 3–1). A response should begin with a clear and succinct answer to the question and follow it with a description of important scientific knowledge that supports that answer. The explanation of scientific knowledge should highlight recent findings and advances and identify gaps in knowledge that call for additional research. Finally, the response should explain how additional recommended research would help to answer the PQ and would affect air-quality management decisions. Another general concern about the presentation of the PQs pertains to the subquestions listed immediately below most of the questions. The committee is uncertain about the intent of the subquestions. They appear to be intended to clarify the question or define the scope of the response. But the subquestions are not always addressed, as in the case of PQ3. The committee recommends that the intent of the subquestions be made clear if they are to be retained. Other options are to use the ideas expressed in the subquestions to improve the questions or to remove the subquestions. The committee also cautions the authors to be careful in their use of first-person pronouns in the PQ discussion. The current text uses “we” in the phrasing of the questions and in some of the responses. Employing the first-person pronoun contributes to the confusion because it is unclear whether it refers to NARSTO, the research community more generally, environmental regulatory agencies, decision-makers, all these, or some other entity. A final concern about the overall presentation of the PQs involves how PQ2, PQ3, and PQ4 fit together. Those three questions characterize the PM problem, its sources, and possible options for addressing it. The current responses do not align exactly with the questions, indicating that perhaps the intent of the questions is not clear. The committee can conceive of at least two ways to interpret how the three questions are supposed to interconnect. PQ2 could be intended to describe the composition of PM at various locations, PQ3 to link sources to the composition of PM to its likely sources, and PQ4 to identify possible control options for the sources. Or PQ2 could be intended to describe the composition of PM and its likely sources, PQ3 to discuss broad control strategies that could be implemented nationally or over large regions, and PQ4 to identify sources that need to be controlled locally. The committee recommends that the three questions be clarified and that the responses given address them directly. PQ1: “Do we have a significant PM problem, and how confident are we?” This question is framed correctly and is policy-relevant. Assuming that the “PM problem” is more clearly defined earlier in the document, the question should not need to be rephrased except to clarify to whom “we” refers. The current response to PQ1 does not provide a clear answer. The committee recommends that the answer be “yes; very confident” for the following reasons: Persistent exceedances of the existing and proposed standards have been observed.

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter Some information about observed ambient PM concentrations in all three nations should be presented here so that the reader can compare these values to the standards. Large portions of the population in North America are exposed to PM concentrations above the current or proposed standards. To emphasize the point and to provide context, it would be useful to include statements such as, “There are X million people in Canada, X million people in Mexico, and X million people in the United States that live with average PM levels above Y µg/m3.” There is considerable evidence suggesting health effects are associated with exposure to PM. A brief summary of the health effects should be provided, including discussion of some of the subtleties associated with the standards, such as these: Thresholds for population health effects have not been observed (Schwartz et al. 2001; Dominici et al. 2002). There may be regional differences in the potency of PM (Samet et al. 2000) and these may be associated with compositional differences. There is evidence of health effects of acute, short-term exposure and long-term exposure to PM (Dockery et al. 1993; Pope et al. 1995; Abbey et al. 1999; Samet et al. 2000; Pope et al. 2002). There are a few examples of relationships between reduced PM exposure and improved health (Pope 1989; Chay and Greenstone 1999; Friedman et al. 2001; Avol et al. 2001). In addition to health effects, high concentrations of ambient PM are associated with acid deposition and the resulting deterioration of ecosystems and structures, visibility impairment, and effects of climate change. All those impose societal costs, so controlling PM will yield economic and other benefits beyond improved health. Of course, it should be pointed out that the assessment focuses on human health effects and visibility impairment. A better discussion of the standards in the synthesis of key issues will clarify the response to PQ1. Because the PM problem is defined for the assessment as exceedance of the existing or proposed standards, it is important that the standards be clearly presented to answer this question. The current response does not provide any information about ambient PM2.5 concentrations in Mexico. Some data are available, including those discussed in Chapter 10 of the draft and additional studies (e.g., Mejia-Velazquez and Rodriguez-Gallegos 1997; Mukerjee 2001; Molina and Molina 2002; SCERP 2002). PQ2: “Where we have a PM problem, what is the source of the concentrations we observe?” This question is important for policy and could be expressed more precisely. The current response is more consistent with the question “Where we have identified a PM problem, what is the composition and what are the sources of the elevated concentrations that are observed?” A general response to this rephrased question is given in the second bullet, which qualitatively describes the composition and broad source categories for various locations. That discussion is

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter accompanied by two figures, one that shows the composition of PM2.5 mass at 16 locations and a second that shows the contribution of local and regional sources of PM2.5 at seven locations. Overall, the committee finds that the discussion and figures are a good response to the rephrased question. One problem with the response is that it is not sufficiently specific about the types of sources. For example, it is stated that 95% of SO2 emissions, the source of particulate sulfate, comes from fossil-fuel combustion, but the proportional contribution of electricity generation to such emissions is not described. From the perspective of PM management, more information about the sources of specific emission sources could help decision-makers to target the control strategies more effectively. A second problem pertains to the lack of discussion about Mexico, aside from the inclusion of one city in Figure 5.12. The assessment should make it clearer where a PM problem has been identified, but observational data are insufficient to determine the source of the problem. Given the high concentration of PM reported in Mexico, the committee finds that more emphasis on cities other than Mexico City, especially the border region between Mexico and the United States, is warranted (e.g., Mukerjee 2001; Mejia-Velazquez and Rodriguez-Gallegos 1997). PQ3: “What broad approaches might we take to fix the problem?” This question is appropriate and policy-relevant, but it could be rephrased to clarify the distinction between the “broad approaches” discussed here and the “specific options” discussed in the response to PQ4. Furthermore, the committee notes an inconsistency between how PQ3 is phrased in the draft assessment and in the presentation given by the cochairs at the first meeting with the committee (Vickery 2002), in which the question was phrased as “What broad (pollutant based) approaches might we take to fix the problem?” (italics added). The confusion over the definition of a “broad approach” is amplified by the lack of a definitive answer to the question. The first bullet of the response discusses generally how one might develop a strategy to control PM. The second bullet provides a partial description of the conceptual model for airborne-PM burden coupled with some tentative suggestions about emission-reduction strategies; much of this text belongs in Section 2b in the synthesis of key issues, where the conceptual model is described. The last bullet gives specific recommendations for the individual case-study areas and should instead be used to answer PQ4. The sub-bullets have too much detail to be included as broad approaches. The committee recommends that the response be rewritten to elaborate that a large portion of the PM2.5 mass loading is anthropogenic. In particular, the response should identify the major anthropogenic sources. Figure 6.7 and the table from slide 12 of Shepherd (2002) may be appropriate for this purpose. In addition, the committee recommends that the response try to identify source categories that might be good targets for broad (nationwide or at least regional) emission control. The committee also finds that it may be appropriate to discuss some issues of nonlinearity here, in particular how emission reductions are not necessarily linearly related to changes in ambient concentrations. This subject is described well in Chapter 2 of the draft and should be

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter brought forward into the synthesis of key issues with some specific examples of linear and nonlinear relationships between sources and ambient concentrations. PQ4: “What specific options do we have for fixing the problem? Given the broad control approaches above, what source control alternatives do we have and where can we get the biggest reductions?” PQ4 is an important question and of great interest to decision-makers. The committee is concerned, however, about NARSTO’s ability to provide a comprehensive answer to it. Indeed, the first sentence of the response to PQ4, “Answers to this question go beyond the scope of this assessment,” is indicative of NARSTO’s limitations. Questions that NARSTO is unable to answer may not belong in this section, although they may be important for pointing out research needs. With that in mind, the committee finds that NARSTO is able to answer the first part of the question for the nine case-study areas described in Chapter 10. The second part of the question, however, is not addressed in the executive summary or in the body of the report, and therefore it may be appropriate to omit it. If NARSTO chooses to answer the second part of PQ4 in this assessment, the case studies should discuss source-control options and provide estimates of the emission reductions expected from each. Such quantitative information is needed for the decision-makers to choose the right policy. Another problem with the question asked in PQ4 is that it is unclear what is meant by “specific options,” as was the case for the term “broad approaches” in PQ3. In a presentation to the committee, Vickery (2002) phrased PQ4 as “What specific source options are there?” (italics added). One might expect the answer to this question to include the type of discussion that is found in the third bullet in the response to PQ3. However, the response seems to answer the question “How do we better understand the source contributions?” by giving examples of modeling tools. To alleviate the confusion engendered by the first part of PQ4, the committee recommends that it be reworded to clarify its intent. Because the current response does not give a clear answer to either part of PQ4, the committee recommends that the response be rewritten to address at least the first part of the question, which NARSTO has already addressed in the body of the report via the nine case studies. A clear indication of specific options available for the case-study areas should be provided by using the text from bullet 3 of PQ3 with recent findings from measurements and models that support the findings. Following the suggested format for responding to all the PQs, the response could then transition into a discussion of the research that would be needed to answer the question better. Recommendations that may be appropriate here include improving the understanding of carbonaceous aerosols, improving emission inventories and models, and conducting case studies similar to the ones discussed in this report for other areas. A shorter version of the wordy and somewhat unclear description of models that currently forms the response to PQ4 could be retained if it included a discussion of how models can be used to reduce uncertainties.

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter PQ5: “What are the relationships between the PM problem and other problems we are working on, particularly considering sources?” The committee finds that PQ5, which addresses the interaction between the PM problem and other problems, is appropriate and policy-relevant. However, the question could be rephrased to clarify what types of other problems are of concern and to make the question more consistent with the response. The use of the word “we” contributes to the confusion for this PQ; knowing who is working on the problems could help to clarify what problems are being considered. It appears to the committee that the current response answers the rephrased question “What is the relationship between the PM problem and other problems that the atmospheric-science community is working on?” The subquestions introduce another level of confusion. Because the PM problem is defined for this assessment as exceedance of health-based standards, the questions concerning its relationship to nonhealth effects seem poorly posed. A more appropriate question would be “What is the relationship between PM-management strategies for health reasons and the problems of acid rain and visibility?” Furthermore, the accompanying table addresses the relationship between the PM problem and climate, although climate is not mentioned in the subquestions. Once the question is clarified, a clear answer should be provided, even if the answer is not yet known for some of the interactions. In fact, the text in Chapters 2 and 5 (referred to in the response to PQ5) indicates that more research is needed to quantify the synergistic and nonlinear effects of copollutants. In the synthesis of key issues discussion, it is important to be clear about what aspects of these relationships are and are not understood. The current response relies heavily on Table 2.2 to illustrate typical relationships between PM precursors or components and other atmospheric issues. The committee, however, identified a number of problems with the table: It is unclear how the indicators were derived, and the level of confidence in each effect is not consistently quantified or communicated. Because no column is dedicated to PM mass or to improvements in health, the table does not give sufficient weight to the benefits of reducing each precursor. The effect on ultrafine particles is not included as a column under “PM composition,” so the table does not indicate the important effect that reductions in black carbon could have on particle size range and thus on visibility and climate change. Many of the effects are qualified with possible or are indicated as being able to go either way; this makes it difficult to ascertain any definitive understanding of them. The color scheme intended to indicate significant expected changes in red, moderate changes in blue, and negligible or unknown effects in black is redundant with the qualifications of many of the effects with words like possible and small. Without careful reading of the footnotes, a reader may misinterpret the color scheme to indicate desired versus undesired effects, an added piece of information that may be more useful in any case. Many of the effects under climate are misleading. For example, saying that a “reduction in NH3” results in an “increased warming” may be taken to equate reduction of pollutants with increased greenhouse-gas emissions. It would be better to indicate a result of “decreased aerosol induced cooling.” In addition, because precipitation and other climate feedback effects are

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter unknown and could be much greater than the temperature changes, the climate effects could be substantially different from what is noted in the table. The committee finds that Table 2.2 may be useful as a first step for decision-makers who want to consider tradeoffs associated with particular control strategies. The text should mention that this table is only a first step toward evaluating the risks, and ultimately the costs, of air-quality management decisions. However, it may be prudent to replace the table with a narrative discussing the relationships between PM and the problems of haze, acid deposition, and ozone. The discussion of potential interactions between PM and ozone strategies could draw more from NARSTO’s ozone assessment (NARSTO 2000). If the table is retained, the committee recommends that the “Climate Impact” column be removed for the reasons listed above and because NARSTO has not focused on climate science. The discussion in this document should summarize the current Intergovernmental Panel on Climate Change (IPCC) assessment findings pertaining to PM (IPCC 2002). PQ6: “How can we measure our progress? How can we determine the effectiveness of our actions in bringing about emissions reductions, air quality improvements, and corresponding health improvements?” The question of how to measure progress is appropriate for this assessment and useful for decision-makers. The question is framed correctly, except that the current response addresses how to measure progress in exposure, a topic that is useful to discuss but one that is not mentioned in the question and is distinct from health improvements. The response should begin with a clear and succinct summary statement of the present ability to measure progress in emission reductions, ambient concentrations, exposure, human health, and visibility. The summary statement should provide the reader with a general sense of the overall capability of the measurement techniques, monitoring network, and analytic tools available to measure progress. The details of each indicator of progress can be fleshed out in the bullets. The committee finds that this section does not give a fair treatment to the ability to measure progress in health effects. The current discussion makes the prospect of ever measuring progress in PM-induced human health effects sound futile (for example, “Tracking changes in human health effects tied to ambient PM changes is much more difficult, at present is in doubt, and may only be inferred”). Furthermore, the discussion of health in this section should acknowledge that methods are available to relate ambient-air quality to health effects (e.g., Samet et al. 2000) and mention some examples of how reducing exposures has resulted in improved health (e.g., Pope 1989; Chay and Greenstone 1999; Friedman et al. 2001; Avol et al. 2001). The committee recommends that the response emphasize ways to improve the understanding of how emission reductions lead to changes in ambient concentrations, including any nonlinearities. This subject draws on NARSTO’s expertise and is of primary interest to air-quality managers. The current response does not provide much information beyond what decision-makers may already know and what air-quality managers are already implementing. The response could be more helpful to those intended audiences if it commented on what NARSTO sees as the best way to measure progress and key deficiencies in the current system.

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter PQ7: “When and how should we reassess and update our implementation programs to adjust for any weaknesses in our plan, and take advantage of advances in science and technology?” The question is policy-relevant and well posed. The aspect of the question pertaining to how often implementation programs should be reassessed is answered directly in the first and fourth bullets, and the other aspects are addressed in the second and third bullets. In general, the committee finds that the response is satisfactory. The discussion emphasizes how an adaptive approach to PM management that takes advantage of advances in science and technology is essential for building robust policies. Given that premise, the committee concurs with the need for periodic updates of scientific assessments, such as the NARSTO PM assessment. Furthermore, if important new information materializes, such as the identification of particulate species that are most damaging to human health, a reassessment should be conducted sooner than the 6–8 years suggested. The committee recommends that a figure illustrating the iterative science-policy process be used instead of or in addition to Figure 1.4. Although Figure 1.4 effectively shows the integrative process required for assessing the linkages between pollutant emissions and health effects, it is limited to a scientific framework. Thus, it does not address the question posed in PQ7 and does not support the current discussion of the need for an iterative science-policy process. PQ8: “What further atmospheric sciences information will be needed in the periodic reviews of our national standards?” PQ8 is a question that is useful to decision-makers and atmospheric scientists. The response addresses many of the important issues associated with the question. The only large omission is the important role in assessing personal exposure played by study design, in addition to measurement techniques. A schematic showing how the different disciplines interrelate, perhaps building on the framework for informing airborne-PM management and on Figure 1.4, may enhance the presentation. Such a schematic would be especially useful if it identified the gaps in the understanding of each discipline that need to be filled by advances in the others. Table 8.1, included in the response to PQ8, provides a list of possible harmful components of PM and NARSTO’s opinion of the current capabilities for measuring ambient concentrations and personal exposure to those components. Although Table 8.1 is useful for presenting hypotheses that explain why PM is toxic, columns A and B are too abbreviated and therefore at risk of being misinterpreted. For example, a reader might conclude from this table that an objective should be to measure all these aspects of PM routinely. The role of study design in assessing personal exposure is not reflected in Table 8.1. Likewise, the capabilities to measure quantities associated with each hypothesis cannot always be characterized as discretely as they are presented here. An example is in the designation of “research” techniques available to measure ambient concentrations of organic compounds; techniques are available to measure only some of the organic compounds. Indeed, discussion elsewhere in the draft assessment indicates that only a small fraction of the organic mass has been speciated even by the most advanced research methods (e.g., Figure 2.9).

OCR for page 17
Review of the Narsto Draft Report: Narsto Assessment of the Atmospheric Science on Particulate Matter The committee recommends that column B indicating the capabilities of personal-exposure measurements, be eliminated, largely because it does not draw on NARSTO’s expertise. Furthermore, the committee finds that the material presented in column A may be better handled in the text, where some of the subtleties could be discussed. Indeed, the only measurements that can be considered fully routine are those of particle mass associated with a particular size cutoff and those of number of particles. Discussing this material in the text could provide an opportunity to indicate what the priorities should be for improvements in measurement technology. There are two additional ways in which the response could be improved. First, the response calls for improvements in monitoring, emission-inventory development, and modeling, but it does not identify a need for laboratory research. Experiments in the laboratory help to improve the understanding of the chemical and physical mechanisms by which PM evolves in the atmosphere and provide critical input to models; this piece of a comprehensive research program should be mentioned. Second, there could be more discussion of why a mass-based standard may not lead to a reduction in the most toxic components of PM and how atmospheric scientists can assist the health-science community in addressing this topic.