4

CONCLUDING COMMENTS

The previous chapters in this report have provided a detailed review of the scientific and technical information in the technical basis report (TBR) related to surface characteristics, erosion, and preclosure hydrology. In this final chapter of its review, the committee wishes to address a broader question: How effective is the TBR at meeting its stated objective as “ . . . a synthesis of currently available site characterization data, analyses, and technical interpretations . . .” (TBR, Preface, p. v)?

In addressing the question of effectiveness, it is important to consider the context for the TBR. This TBR is the product of an effort of great national importance: to assess the suitability of Yucca Mountain as a facility for the safe permanent disposal of high-level radioactive waste. For this effort to succeed, the Department of Energy (DOE) will need to demonstrate that it has a good understanding of the scientific and technical issues that would affect the construction and long-term performance of such a facility. Given the importance of this undertaking to the health and safety of present and future generations of this nation's citizens, it is this committee's opinion that the scientific and technical analyses presented in the TBR should meet the highest standards of scientific quality. Considered in this context, it is the committee 's judgment that this TBR is not an effective synthesis of data,



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion 4 CONCLUDING COMMENTS The previous chapters in this report have provided a detailed review of the scientific and technical information in the technical basis report (TBR) related to surface characteristics, erosion, and preclosure hydrology. In this final chapter of its review, the committee wishes to address a broader question: How effective is the TBR at meeting its stated objective as “ . . . a synthesis of currently available site characterization data, analyses, and technical interpretations . . .” (TBR, Preface, p. v)? In addressing the question of effectiveness, it is important to consider the context for the TBR. This TBR is the product of an effort of great national importance: to assess the suitability of Yucca Mountain as a facility for the safe permanent disposal of high-level radioactive waste. For this effort to succeed, the Department of Energy (DOE) will need to demonstrate that it has a good understanding of the scientific and technical issues that would affect the construction and long-term performance of such a facility. Given the importance of this undertaking to the health and safety of present and future generations of this nation's citizens, it is this committee's opinion that the scientific and technical analyses presented in the TBR should meet the highest standards of scientific quality. Considered in this context, it is the committee 's judgment that this TBR is not an effective synthesis of data,

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion analyses, and interpretations related to surface characteristics, erosion, and preclosure hydrology. The following sections provide the committee's reasons for this conclusion. In the spirit of peer review as a method to improve scientific quality, these sections also offer constructive suggestions of ways to increase the effectiveness of this and future TBRs. AUDIENCE FOR THE TBR The audience for this TBR is never defined clearly in the report. During the course of this review, particularly during the public information-gathering sessions, it became clear to the committee that a number of different audiences were interested in this TBR. Foremost among these interested parties are regulators and policymakers who will be responsible for ultimately licensing and providing funds for the construction and operation of the proposed repository; the scientific community, whose members are knowledgeable or concerned about the issues addressed in this TBR; and the public, particularly the state and local institutions and individuals potentially affected by the decision to site a facility at Yucca Mountain. In the committee's judgment, the TBR is not written effectively for any of these audiences for a variety of reasons, many of which are noted in the previous chapters: The technical questions addressed in each of the sections of the TBR are not stated clearly; the presentation of data and analyses is incomplete; and the lack of effective graphics makes for difficult reading and comprehension. The report is to a significant extent inaccessible to nonscientific readers.

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion In the committee's opinion, the credibility of the TBR and the underlying site characterization program would be increased if the report were written to be comprehensible to a broad audience. The committee recognizes the significant challenges involved in producing a report that is broadly comprehensible—particularly when that report is highly technical in nature and contains extensive technical documentation. The committee believes, however, that it is possible to produce such a document by utilizing a variety of stylistic devices. These include the use of additional graphics (e.g., detailed maps and cross sections) to illustrate technical points, the use of appendixes for technical documentation, and the use of extended, “nontechnical” summaries for managers, legislators, and the public. Recommendation: The audiences for the TBR should be stated explicitly in the report, and the TBR should be written to be comprehensible by these groups. The designation of target audiences is a DOE policy issue. The committee recommends that DOE policymakers carefully consider the advantages of writing for a broad audience to build scientific credibility and public acceptance for its site characterization program. QUESTIONS ADDRESSED BY THE TBR The scientific and technical questions to be addressed by the TBR also were not stated clearly in the report. The TBR does provide the qualifying and disqualifying conditions from 10 CFR

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion 9601 relavant to surface characteristics, erosion, and preclosure hydrology —but there is little attempt to discuss the associated technical questions that need to be answered or hypotheses that need to be tested in order to assess compliance with these 10 CFR 960 conditions. The concern here is not whether the TBR demonstrates compliance with 10 CFR 960. Indeed, consideration of compliance issues is outside the committee's statement of task (Appendix C). Rather, the concern is the context in which the scientific and technical content of the TBR is to be assessed. Without a clear statement of the questions to be addressed, the report reads as more of a narrative rather than a focused technical analysis. A list of questions or hypotheses of concern at the beginning of each major section of the TBR would have served to focus the supporting technical analysis —and would have made it easier for the committee and other concerned readers to assess whether that technical analysis adequately addressed those questions. The committee's review of water resource potential (Chapter 3 of this report) perhaps best illustrates the problem with assessing technical analysis in the absence of a clear problem statement. It was unclear to the committee whether the point of the analysis was to determine (1) total available water supply or (2) some minimum, or bounding, estimate of supply. The differences between these points may seem to be minor, but they are significant for assessing whether the supporting technical analyses are adequate. The committee has provided examples of the kinds of focus questions that should have appeared in the TBR in Chapter 2 and Chapter 3 of this review. 1   Title 10, Part 960 of the Code of Federal Regulations, General Guidelines for the Recommendation of Sites for Nuclear Waste Repositories.

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion Recommendation: The TBR should contain a clear statement of the questions to be addressed and hypotheses to be tested for each technical topic. These questions will serve to focus the supporting technical analyses and make it easier for the reader to assess their adequacy. SYNTHESIS OF AVAILABLE DATA Another problem noted by the committee in its review is the failure of the TBR to explicitly address all available data in the scientific and technical analyses. This problem may not be apparent to nonexpert readers without knowledge of the scientific literature related to the site characterization program. However, this deficiency was recognized by scientists and other technically oriented individuals identified by DOE as “stakeholders” in the site characterization process, and a number of these individuals pointed out this problem to the committee during its information-gathering sessions. This deficiency also became very clear to the committee during its review of the scientific literature cited in the TBR and made it difficult for the committee to perform its review. There are two aspects of the data synthesis issue that deserve comment. The first is a failure to reference and discuss all data relevant to the issues addressed by the TBR, as noted elsewhere in this review. For example, Chapter 2 of the TBR does not synthesize the surficial work done by DOE and its contractors on Jackass Flats (Lundstrom et al., 1995), nor does it cite or discuss the surficial work done by State of Nevada scientists on Crater Flat (Faulds et al., 1994)—even though these results are generally consistent. The committee believes that incorporation of the Crater

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion Flat work into the TBR would have provided a more complete understanding of the surficial geology of the Yucca Mountain region. The failure of the TBR to explicitly address all available data calls into question the thoroughness with which the scientific and technical issues were evaluated. The second aspect of the data synthesis problem relates to tracing information used in the TBR back to primary references. Many of the data and models cited in the TBR were taken from previously published papers, and references to those papers are provided throughout the report. However, in some cases, the committee found it difficult to trace information and conclusions cited in the TBR back to those references. For instance, as noted in the review of surface flooding (Chapter 3 of this report), the TBR incorporates technical information from three published papers (Bullard, 1991; Blanton, 1992; Glancy, 1994) to estimate probable maximum flooding (PMF). However, it is not clear from the discussion in the TBR how the information from these three sources was used to make the PMF estimates; nor are reasons given for the selection of parameter values used in the analyses. Similarly, the TBR does not provide details of the model used by Czarnecki (1991) to estimate the effects of ground water withdrawals. The TBR cites an earlier paper by Czarnecki (1985) as the source of details on this model, but in fact, much of the basis for the model is not found in that paper but in Waddell (1982) and Czarnecki and Waddell (1984). The ability to trace information back to primary sources is important for two reasons: (1) it allows readers to access related background and explanatory information to increase understanding; and (2) it also allows them to check that the information from these sources has been used correctly. The ability to trace information back to primary references is one of many essential re-

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion quirements for establishing the scientific credibility of any scientific or technical document. Recommendation: All available scientific and technical information related to the issues addressed in the TBR should be cited and discussed. In addition, essential information (e.g., data, equations, and model parameters) used in analyses should be provided in the TBR, and primary sources should be referenced. SYNTHESIS OF ANALYSES In the committee's judgment, the TBR does an inadequate job of compiling and synthesizing available data and analyses to explain the scientific and technical issues related to surface characteristics, erosion, and preclosure hydrology. Indeed, the TBR presents the results of analyses with relatively little synthesis or documentation. As a consequence, it was difficult and in some cases impossible for the committee to check these analyses. Nor was it always possible to determine whether alternative analyses had been attempted and, if so, to assess the results of those efforts. The TBR could be improved significantly by including a more complete discussion of analyses supporting the technical interpretations; a discussion of alternative hypotheses and the methods used to test them; and a discussion of remaining uncertainties and additional data needed to address them. The TBR is a scientific and technical document and should follow conventional practice with respect to presentation of data, testing of hypotheses, and acknowledgment of uncertainties.

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion Recommendation: The TBR should provide a complete discussion of analyses supporting the technical interpretations; a discussion of alternative hypotheses and the methods used to test them; and a discussion of remaining uncertainties and additional data needed to address them. During its first information-gathering session, the committee learned that the TBR was compiled by a single individual who was working under extremely tight deadlines. The committee understands that this person was broadly knowledgeable about Yucca Mountain and site characterization efforts, but was not involved directly in the scientific studies addressed in the TBR. These criticisms are not directed at the individual who compiled the TBR, but rather at the management process that led to preparation of the TBR by someone other than the scientists whose work was used in the report. The integration of data and analyses from multiple sources into a coherent document is a complex and difficult process. The committee believes that this process is carried out most effectively with the direct involvement of those doing the science—in this case the scientists who are involved directly in site characterization efforts—because they have the most complete understanding of the data and methodologies and are trained in analytical synthesis. In order to assess the quality of this integration, it is also necessary to understand how the integration was carried out and to know who was involved in the process.

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion Recommendation: The TBR should be prepared with the direct involvement of the scientists who did the site characterization studies, and these scientists should be identified in the report. The TBR should also provide a discussion of how data and analyses were selected and integrated. A recurring theme in the committee's detailed review of the TBR (Chapter 2 and Chapter 3 of this report) is the importance of using multiple techniques of analysis to improve understanding, reduce uncertainties, and thereby build confidence in the interpretations and conclusions. Multiple-methodology approaches seem particularly well suited to many of the problems addressed in the TBR. As noted in Chapter 2 of this report, for example, the use of multiple geochronologic methods for measuring surface exposure ages might have significantly reduced uncertainties and increased confidence in the validity of the results obtained from the cation ratio dating method. Similarly, the use of multiple techniques to estimate rates of erosion would have provided information on the spatial distribution of rates at Yucca Mountain and could have generated more reasonable ranges of estimates in the vicinity of the proposed repository. A valuable tool in the arsenal of multiple methodologies is the bounding calculation, which can be used to determine minimum or maximum values of processes or phenomena. Bounding calculations could have been used to address many of the questions in the TBR with little or no additional data collection. As noted in Chapter 3 of this report, for instance, such calculations could probably have been used to demonstrate that there is an adequate supply of water for construction and operation of the proposed repository. Other examples of bounding calculations are given in the summary and conclusions section on erosion in Chapter 2.

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion In the committee's judgment, the TBR placed too much emphasis on producing single numbers that could be compared to some specified standard, perhaps to demonstrate regulatory compliance. The effort to produce a single estimate of erosion (Chapter 4 of the TBR) for the Yucca Mountain region is perhaps the best example of this tendency. Science is a process of logical inquiry into the nature of the world. The committee believes that the management of science is not achieved most effectively when the results of such inquiry (e.g., geochronological measurements in laboratories and field measurements of processes) are generated in separate investigations, transformed into numerical form, and subsequently combined by formula to yield a prediction (e.g., of erosion rate) that is envisioned primarily for comparison to a regulatory standard. The committee's concern with this approach is threefold: (1) there is little room for formulating or testing alternative hypotheses of processes and applying multiple methodologies; (2) there is little opportunity to assess the “hidden” uncertainties that exist because all processes are not measured; and (3) the spatial and temporal variabilities in rates and processes are obscured through the use of average values. The focus on generating average values or single explanations is contrary to science as an open process of inquiry in which investigators are encouraged to invent alternative explanations for phenomena, explore the whole range of natural causative processes, and make discoveries of previously unrecognized facts—even if those facts seem inconsistent with current theories. The types and numbers of analyses presented in this TBR appear to reflect, to a great extent, decisions made by DOE on the kinds of studies to undertake during the site characterization phase of its program. It is beyond the charge of this committee to com

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion ment on those decisions specifically or to evaluate DOE's site characterization program. However, it appears that the failure of the TBR to employ multiple methodologies may be traced to management decisions that discouraged the use of such methodologies in an iterative and integrative manner as site characterization progressed. Recommendation: Multiple methods of analysis should be employed in the TBR to improve understanding, reduce uncertainties, and thereby build confidence in its interpretations and conclusions. Bounding calculations are particularly well suited to many of the issues addressed in the TBR and should be applied as part of a multiple-methodology approach to place minimum or maximum bounds on processes and phenomena. PRESENTATION OF DATA AND ANALYSES Aside from the fact that necessary data are not always provided in the TBR, as noted previously, the data, analyses, and conclusions generally are not presented effectively. Of particular concern to the committee is the absence of basic geologic and topographic maps and cross sections to orient the reader and to illustrate spatial relationships among the various elements of the site: the land surface, surface drainages, rock units, surficial deposits, the proposed repository, the ground water table, perched water zones, and important data collection sites. Many of the graphics used in the TBR are inadequate; they lack even rudimentary information such as accurate topography and scale. The lack of informative graphics and tables makes the report less under

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion standable to readers and, therefore, discourages readers from developing their own understanding of the underlying scientific and technical issues. The committee also discovered several minor, yet nagging, production-related errors in its review of the TBR. For example, Section 4.5 in the report is missing—most likely an error in the table of contents; there are citation errors (e.g., Figure 3.3-4 is from Czarnecki, 1991, not 1992 as given in the figure caption); pages 3.6 and 3.5 are reversed; and there are simple mistakes of fact. Such errors detract from the readability of the TBR and inevitably call into question the thoroughness with which it was reviewed before its release. Recommendation: The TBR should be illustrated with informative graphics to orient the reader to the region and to illustrate spatial relationships among the various elements of the site. Of particular importance are geologic and topographic maps and cross sections that illustrate the locations of land surfaces, drainages, rock units, surficial deposits, perched and ground water, the proposed repository, and selected data collection sites. PEER REVIEW Many of the problems noted in this report could have been discovered and corrected had an effective peer review mechanism been in place during preparation of the TBR. The committee learned during its first information-gathering session that the scientists working in the site characterization program—whose data and analyses are cited in the TBR—had an inadequate opportunity

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion to review the TBR before it was released. During its information-gathering sessions, the committee also observed an apparent lack of communication and research integration among scientists involved in site characterization and between project scientists and other researchers, not supported by the DOE, who are working in related areas. Peer review is an integral part of science, and such review is most effective when it is integrated throughout the scientific process —starting with decisions on what problems to address and how to address them; as a check on the quality of the investigations themselves; and finally, as a check on the quality of the presentation of results. The DOE has demonstrated its commitment to peer review at several steps in the site characterization process—for example, in its quality assurance program to ensure that the scientific and engineering data used in site characterization efforts are of high quality, and in its commitment to obtaining external peer review of the TBRs as demonstrated by the work of this committee. Yet quality cannot just be added on at the end of a project; it has to be built in to every step of the process. The committee also learned at its first information-gathering session that the issue of erosion had been reviewed by the U.S. Nuclear Regulatory Commission (USNRC, 1994) prior to the release of this TBR. There is no reference made to this review in the TBR; nor does the TBR address the issues raised in the USNRC review related to extreme erosion and the cation ratio dating method. To be useful, peer review must have an influence on the scientific process; that is, peer review must be allowed to feed back into scientific investigations. There is no indication that such feedback occurred in the preparation of this TBR.

OCR for page 93
Review of U.S. Department of Energy Technical Basis Report for Surface Characteristics, Preclosure Hydrology, and Erosion Recommendation: Preparation of the TBR should include provision for a rigorous program of peer review by scientists whose work is used in the report. This should occur in addition to external peer review, which serves as an independent check on scientific and technical quality. The peer review process should also include provisions to ensure that the results of internal and external peer review effectively feed back into this and future TBRs and, when appropriate, into the associated scientific and technical programs.