APPENDIX A-1
SCOPE OF WORK

A Review of the Department of Energy's National Energy Modeling System (NEMS) by the National Research Council (NRC)

The Department of Energy (DOE) uses a variety of energy and economic models to forecast energy supply and demand. It also uses a variety of more narrowly focussed analytical tools to examine energy policy options. For the purpose of this scope of work, this set of models and analytical tools is called the National Energy Modeling System (NEMS).

THE PROBLEM.

The NEMS is the result of many years of development of energy modeling and analysis tools, many of which were developed for different applications and under different assumptions. As such, NEMS is believed to be less than satisfactory in certain areas. For example, NEMS is difficult to keep updated and expensive to use. Various outputs are often difficult to reconcile. Products were not required to interface, but were designed to stand alone. Because different developers were involved, the inner workings of the NEMS are often not easily or fully understood.

Even with these difficulties, however, NEMS comprises the best tools currently identified to deal with our global, national and regional energy modeling and energy analysis needs.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 97
The National Energy Modeling System APPENDIX A-1 SCOPE OF WORK A Review of the Department of Energy's National Energy Modeling System (NEMS) by the National Research Council (NRC) The Department of Energy (DOE) uses a variety of energy and economic models to forecast energy supply and demand. It also uses a variety of more narrowly focussed analytical tools to examine energy policy options. For the purpose of this scope of work, this set of models and analytical tools is called the National Energy Modeling System (NEMS). THE PROBLEM. The NEMS is the result of many years of development of energy modeling and analysis tools, many of which were developed for different applications and under different assumptions. As such, NEMS is believed to be less than satisfactory in certain areas. For example, NEMS is difficult to keep updated and expensive to use. Various outputs are often difficult to reconcile. Products were not required to interface, but were designed to stand alone. Because different developers were involved, the inner workings of the NEMS are often not easily or fully understood. Even with these difficulties, however, NEMS comprises the best tools currently identified to deal with our global, national and regional energy modeling and energy analysis needs.

OCR for page 97
The National Energy Modeling System ANALYSIS OF THE PROBLEM With the goal of evaluating and improving the NEMS, the National Academy of Sciences, National Research Council is requested to review NEMS and advise DOE in the following areas regarding energy modeling and policy analysis tools: Energy Data: Are the quality and types of data that are currently gathered and available to DOE adequate for NEMS? If not, what additional data should be gathered? If some critical data are not obtainable, what alternatives are available? Modeling Assumptions: Are the assumptions embodied in models internally consistent and valid? Is it possible to reduce the number of assumptions currently built into the models and the scenario runs? Energy and Economic Models: What is the availability of other models or data, not currently being utilized in NEMS, that might be useful in improving the NEMS? Are there any gaps in the NEMS in terms of supply/demand feedbacks, consideration of international factors, weaknesses in analyzing certain policy options or other shortcomings? Role of NEMS in Energy Policymaking: In general, what are both the capabilities and practical limitations of energy forecasting tools? Taking such limitations into account, what are the appropriate uses of forecasting tools in developing the National Energy Strategy (NES)? What other alternatives are available for developing long-range energy policy? How might they be incorporated into future NEMS/NES development? PROPOSED EFFORT A 21-month study by NRC is proposed for accomplishing a review of the NEMS modeling and analysis needs, to be conducted in two parts: First Part of Review The first part of the review will be completed six months from contract award. NRC will examine the composition of the NEMS and its initial application with respect to a representative slate of analytical tasks. NRC will define and apply criteria to evaluate the capabilities and adequacy of the NEMS for its intended purposes of forecasting and analyzing a variety of energy supply, end-use and environmental issues. In this context, NRC will review, primarily through briefings and presentations from EIA and other DOE sources, NEMS assumptions and methodologies. Particular emphasis will be given to the review of significant components of NEMS as they concern the cost and performance (output) curves for energy supply and end use technologies, environmental releases. NRC will also review the Department's plans for further development of NEMS. NRC will, within six months of contract award, prepare and issue an initial report of its findings. This report, following NRC peer review procedures, will be forwarded to DOE for use in support of the development of the National Energy Strategy.

OCR for page 97
The National Energy Modeling System The report will present findings and conclusions regarding the efficacy of the NEMS in its early configuration, the reasonableness and adequacy of the underlying data, assumptions and methodology, and make recommendations, as appropriate, for near term action by DOE. The report may also outline the steps NRC will take in conducting the second part of the review. Second Part of Review In the second part of the review, to be completed 21 months after contract award, NRC will evaluate the NEMS and plans for further development in greater detail. On the strength of this evaluation, the NRC will provide recommendations for improving the models, data and associated analytical tools comprising the system, and for future model development and data collection in order to enhance DOE capabilities for strategic planning. In its review, NRC will address relevant long-range issues and will appraise the robustness and the flexibility of the NEMS to support the DOE policy making process. NRC will examine the adequacy of the models and analytical tools and the quality of data to support relevant policy analyses and development, in all supply and demand sectors, including renewables and in environmental and new technologies areas. In particular, NRC will examine DOE needs and priorities for the NEMS over the long term. NRC will then identify what system improvements can be made cost effectively and new directions that could be undertaken within the state of the art, given emerging policy issues and other requirements and constraints that will confront DOE in the early 1990's and beyond. An important part of this review is for NRC to assess the adequacy of current data collection for NEMS and related analyses in terms of its scope, frequency, sample sizes, and other statistical characteristics. An assessment will be made of the role that new data might service in enhancing the capabilities of the NEMS to meet its intended objectives. NRC will make recommendations on data requirements needed to more effectively support the DOE policy-making process and suggest priorities for new data collection and ways that data needs can be met cost effectively. In its assessment of the modeling activities in DOE, NRC will define and address issues it considers critical in contexts such as the following: (1) applications history of models in the NEMS; (2) completeness and adequacy of model documentation both for those running the models and those using them; (3) validity of the models in terms of how well the theoretical concepts, data bases, and operations approximate relevant aspects of the real world, including behavioral effects that impact on energy supply and demand; (4) treatment of uncertainties in both model construction and application; (5) integrity of the models in their representation in the computer codes which make them operational; (6) quality of model forecasts, and extent of out-of-system adjustments to outputs needed to account for conceptual and practical limitations of the models; (7) interfaces within the system and the ease with which the models can be maintained and updated; and (8) usability of the system including costs associated with use and upkeep of the system.

OCR for page 97
The National Energy Modeling System In executing the foregoing assessment, it is expected that NRC will rely on existing information and documentation and will not undertake any original model or data validation efforts. In order to assure timely advice enabling effective allocation of DOE resources to the continuing development of the NEMS, NRC may, as appropriate, issue brief advisory reports on findings and recommendations considered significant. The advisory reports will be subject to peer review. NRC will issue, within 21 months of contract award, a peer reviewed final report to the DOE covering their findings and recommendations during the duration of the study. ANTICIPATED RESULTS NRC's report, after the first part of the review is completed, will provide informed judgements on the capabilities and adequacy of NEMS for its intended purposes, taking account of its initial applications and will advise the DOE with regard to further NEMS development. NRC's findings and recommendations at the end of the second part of the review will provide DOE an assessment of NEMS, suggesting the level of confidence appropriate for its analytical products, and guiding its continued development and application. It is expected that NRC's interim advisory reports and final recommendations will enhance DOE's capabilities for strategic planning over the long term. REPORTING NRC's findings and recommendations will take the form of the reports described under “Proposed Effort.” All reports will be subject to regular NRC report review procedures. Reports will be provided to DOE and to other appropriate individuals and agencies. Briefings will be conducted as necessary.