Review of the NARSTO Draft Report: NARSTO Assessment of the Atmospheric Science on Particulate Matter (2002)

Understanding the processes responsible for high concentrations of participate matter (PM)¹ in the atmosphere is critical for air-quality managers. NARSTO², a public-private partnership with members from government, utilities, industry, and academe in Canada, Mexico, and the United States, was founded in 1995 to coordinate and enhance policy-relevant atmospheric science research and assessment on tropospheric ozone. The scope of NARSTO was expanded to include ambient PM in 1998. To inform policy-makers on how emissions of PM and its gaseous precursors are related to the distribution of ambient PM, NARSTO has recently completed a draft of the NARSTO Assessment of the Atmospheric Science on Particulate Matter. The assessment is intended to be a concise, scientifically credible, comprehensive discussion of the atmospheric-science issues associated with management of airborne PM to achieve air quality standards. The assessment strives to present the issues from a policy-relevant perspective and to develop a common scientific understanding that can be used in implementing new PM ambient air-quality standards.

The National Research Council (NRC), in consultation with the Royal Society of Canada (RSC) and the United States-Mexico Foundation for Science (Fundacion Mexico Estados Unidos para la Ciencia, FUMEC), was asked to review the draft NARSTO assessment, dated December 31, 2001. In response to that request, the NRC established the trinational Committee to Review NARSTO’s Assessment of Airborne Particulate Matter. The committee was formed with input and concurrence from the RSC and FUMEC in accordance with a memorandum of understanding that was approved by the two organizations and the NRC.³ In conducting its review, the committee was tasked to consider the stated overall goals and objectives of the assessment, the

scientific and technical analysis provided, and how effectively policy-relevant questions were addressed.

The draft NARSTO assessment consists of a comprehensive review of the science pertaining to PM and an executive summary intended to synthesize the scientific information for a decision-maker audience. Within the body of the assessment are technical chapters addressing formation and transport of PM, emission inventories, measurement techniques, ambient PM concentrations, source apportionment and air quality-models, health effects, and visibility. Some analysis of the technical information is provided via recommendations for future research needs and “conceptual descriptions” of the factors that affect observed ambient PM concentrations in nine regions in North America. The executive summary of the assessment contains a summary of the information presented in the body of the document and responses to eight policy questions, which are used to synthesize the atmospheric-science information necessary for implementing ambient PM standards. The assessment cochairs informed the committee that the executive summary will be published with the full document and also separately in English, Spanish, and French. Here, the committee summarizes the highest-priority recommendations for the chapters of the draft assessment and its executive summary.

The committee commends NARSTO for undertaking this assessment, particularly in light of the challenge it posed. The committee finds that the draft NARSTO PM assessment is a good representation of the state of atmospheric science, has an appropriate level of detail in most chapters, and provides the best currently available information on the formation and distribution of PM for North America. In particular, the draft clearly identifies the widespread existence and variable nature of the PM problem in different regions of North America. The draft provides an unmatched compilation of chemical measurements of both inorganic and organic fractions of PM, with preliminary indications of the sources and fates of the components. A comprehensive discussion of computational models shows how the weight of evidence defines a clear path for linking emission sources with air-quality outcomes. The committee finds the descriptions of PM characteristics and source contributions in nine regions to constitute an original scientific contribution and to provide information of great use to decision-makers. With suitable revisions, the document will be useful to the air-quality management community, to atmospheric-science researchers, and potentially to researchers in related fields. Indeed, the committee is unaware of any other assessments of the atmospheric science of PM that are this comprehensive; this NARSTO assessment is potentially of great use.

The draft assessment in its current form needs substantial improvements to meet its goal of providing useful information in a way that is accessible to decision-makers, especially in terms of consistently presenting a framework for informing airborne-PM management and stating clearly the policy implications of scientific findings. The committee’s highest-priority recommendations for improving the main body of the assessment follow. Additional comments are provided in the chapters of the present report and its Attachment B.

• A framework for informing airborne-PM management, which includes a general conceptual model for understanding PM formation, is not explicitly presented and consistently used in the draft assessment. In Chapter 2 of this review, the committee offers a diagram that illustrates such a framework. The committee recommends that a framework of this sort be explicitly defined and introduced in Chapter 1 of the assessment and used consistently throughout it. Each technical chapter should begin by positioning its subject matter in the framework. The descriptions of PM characteristics and emission sources in nine regions should be presented by inserting regionally specific information into the same basic figure.

• The assessment authors do not appear to have made a clear and consistent distinction between public policy itself and policy-relevant science, and sometimes they unnecessarily shy away from interpretive statements, perhaps out of concern that they could be misconstrued as recommending policy. The committee recommends that each technical chapter (Chapters 2–9) conclude with a synthesis of how the scientific information presented could be used in managing ambient PM and what additional scientific information would be useful for making policy decisions in the future.

• The draft assessment focused much less on PM in Mexico than on that in Canada and the United States. For example, only one Mexican location, the Mexico City area, was included in the regional descriptions. The committee recommends that the assessment authors expand the representation of Mexico in the assessment, particularly by bringing in several recently completed and current studies in key Mexican cities, to fulfill NARSTO’s goal of developing an inclusive North American perspective on PM.

• The assessment authors clearly recognize the importance of health effects associated with airborne PM, but they have failed to write a clear and readable summary of current PM-related health findings in the chapter devoted to the topic. The committee believes that a succinct summary of health effects would be useful for atmospheric scientists and recommends that the chapter be redirected to provide a framework for interactions between health scientists and atmospheric scientists and rewritten to reflect the numerous comments provided by the committee in Attachment B.

• Much attention is paid to the organic component of PM throughout the draft assessment, accurately reflecting recent research advances and the substantial remaining uncertainty on this topic. However, the committee found that the introductory material of the assessment does not call sufficient attention to this key foundation point. The committee recommends that a focused discussion of the importance of the organic component of PM be provided early in the assessment.

• The assessment authors gave inadequate attention to editorial details and the presentation of the text, particularly with respect to unnecessary repetition, inconsistent use of technical terms and concepts, and the quality of figures and tables. The committee recommends that there be a thorough editing to address those problems, as itemized in Attachment B. The authors should also strongly consider using a terminology box near the beginning of each chapter to define critical terms and abbreviations, which should also be consistently defined in a complete glossary and when introduced in the text.

The draft NARSTO assessment begins with a long (53-page) executive summary intended for senior decision-makers and their advisers and the science community. As a preliminary step in the assessment process, NARSTO conducted interviews with senior decision-makers in Canada, Mexico, and the United States to help to elucidate policy goals for PM management, gaps in science, how science fits into decision-making, and how to present scientific information. On the basis of those interviews, NARSTO chose to use eight policy questions as the primary mechanism by which the draft executive summary would communicate the state of PM atmospheric science to decision-makers. The committee commends NARSTO for taking the time to query decision-makers; this step is often omitted in assessments. However, the committee has strong reservations about how the policy questions were developed. Furthermore, the responses to many of the policy questions and more generally the executive summary fall short of effectively communicating to decision-makers a clear description of the current understanding of PM atmospheric science and its policy implications. Specific recommendations for improving the executive summary include the following:

• The executive summary does not conform to standard expectations of executive summaries. It is too long and too technical for the intended decision-maker audience. A separate, brief executive summary that discusses in a terse manner the important points presented in the document should be written.

• A longer summary that more fully addresses the scientific issues and responds in detail to the policy questions is valuable. The current executive summary could meet this objective if renamed and extensively revised. In the body of the present review, the committee offers a suggested outline for revising the assessment’s summary. The revised summary should introduce the charge, goal and specific objectives, and assumptions set forward by NARSTO for the assessment and then provide a crosswalk between the objectives and the main body of the assessment. The revised summary should also present a framework for informing airborne-PM management (as discussed above) and the adopted or proposed air-quality standards for PM for the three countries, which together form the foundation of the assessment.

• The interview process and method for developing the policy questions is not clearly articulated in the report and do not conform to current social-science methods. The policy questions were apparently developed before the interviews with decision-makers, who were then asked to confirm that the questions were appropriate. Thus, the policy questions are not necessarily the highest-priority questions about PM identified by policy-makers themselves but are the policy-relevant questions about PM that the assessment authors thought were the most important. In addition, only five of the 45 interviews were with Mexican decision-makers. Despite these problems, the policy questions are a useful organizing framework for communicating to policy-makers. The committee recommends that the eight policy questions be retained, with minor rewording to improve clarity, and that the method by which they were developed, including all its limitations, be clearly explained.

• The responses to the eight policy questions need to be revised. The committee recommends that each response begins with a clear and succinct reply to the question and follow it with a description of important scientific knowledge that supports it. Each response should also identify additional research needed to answer the policy question and to make better air-quality management decisions.

The committee recommends that NARSTO, in preparing future assessments, enhance its interaction with the policy community. The committee finds that the draft assessment falls short especially in terms of the method for eliciting information needs from decision-makers and communicating the policy implications of the science to them. The committee strongly recommends that social scientists with expertise in elicitation of information be engaged in the process of developing policy guidance for future assessments. In providing policy-relevant atmospheric-science information, NARSTO should strive to discuss tradeoffs, options, and priorities more explicitly. By articulating policy implications less ambiguously, NARSTO will be more effective in informing policy development.

The committee also recommends that NARSTO focus on better placing the atmospheric-science information in the context of impacts on health, visibility, ecosystems, and global climate and with regard to implications for economics and other social sciences. To do that effectively, NARSTO needs to engage the relevant communities more fully and foster interaction with them. For example, NARSTO should consider establishing or increasing the prominence of leadership positions in its organization for persons whose responsibilities include linking to the other communities via standing committees, workshops, briefings, and similar endeavors. Failure of NARSTO to strengthen its cross-discipline interactions will limit the value of chapters of a future assessment, devoted to other related fields, as is the case with the chapter on health effects in the current draft assessment. What is more serious, the relevance of atmospheric-science research to the larger air-quality management activity could be compromised.