National Academies Press: OpenBook

The National Energy Modeling System (1992)

Chapter: Appendix B: First Advisory Report

« Previous: Appendix A-2: Committee Charge
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×

APPENDIX B
FIRST ADVISORY REPORT

DEVELOPMENT OF THE NATIONAL ENERGY MODELING SYSTEM

COMMITTEE ON THE NATIONAL ENERGY MODELING SYSTEM

Energy Engineering Board

Commission on Engineering and Technical Systems

in cooperation with the

Committee on National Statistics

Commission on Behavioral and Social Sciences and Education

January 30, 1991

NATIONAL RESEARCH COUNCIL

WASHINGTON, D.C. 1991

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×

COMMITTEE ON NATIONAL ENERGY MODELING SYSTEM.

PETER T. JOHNSON, (Chairman). Former Administrator, Bonneville Power Administration, McCall, Idaho

DENNIS J. AIGNER, Dean, Graduate School of Management, University of California, Irvine

DOUGLAS R. BOHI, Director, Energy and Natural Resources Division, Resources for the Future, Washington, D.C.

JAMES H. CALDWELL, Jr., Washington, DC 20016

ESTELLE B. DAGUM, Director, Time Series Research and Analysis Division, STATISTICS CANADA, Ottawa, Ontario, Canada

DANIEL A. DREYFUS, Vice President, Strategic Planning and Analysis, Gas Research Institute, Washington, D.C.

EDWARD L. FLOM, Manager, Industry Analysis & Forecasts, Amoco Corporation, Chicago, IL

DAVID B. GOLDSTEIN, Senior Staff Scientist, Natural Resources Defense Council, San Francisco, CA

LOUIS GORDON, Professor, Department of Mathematics, University of Southern California, Los Angeles, CA

VELLO A. KUUSKRAA, Chairman, ICF Resources Inc., Fairfax, VA

JAMES W. LITCHFIELD, Director of Power Planning, Northwest Power Planning Council, Portland, Oregon 97204

STEPHEN C. PECK, Director, Environment Division, Electric Power Research Institute, Palo Alto, CA

MARC H. ROSS, Professor, Department of Physics, University of Michigan, Ann Arbor, MI

EDWARD S. RUBIN, Professor, Department of Engineering & Public Policy, Carnegie-Mellon University, Pittsburgh, Pennsylvania

JAMES L. SWEENEY, Professor, Department of Engineering-Economic Systems, Terman Engineering Center, Stanford University, Stanford, California

DAVID O. WOOD, Director, Center for Energy Policy Research, Massachusetts Institute of Technology, Sloan School of Management, Cambridge, MA

Staff

MAHADEVAN (DEV) MANI, Study Director, and Acting Director, Energy Engineering Board

ARCHIE L. WOOD, Acting Executive Director, Commission on Engineering and Technical Systems

JUDY AMRI, Administrative Associate

PHILOMINA MAMMEN, Study Assistant

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×

I. INTRODUCTION

The National Research Council's Committee on the National Energy Modeling System was established at the request of the Department of Energy (DOE) primarily to provide guidance to DOE and the Energy Information Administration (EIA) on the long-term development of a modeling system to support national energy analysis and strategic planning. This report is an interim advisory report that has been prepared by the Committee at the request of the Secretary of Energy.

The Committee held its first meeting in Washington, D.C, on July 31 and August 1, 1990, and has since met again on September 20 and 21, 1990 and November 8 and 9, 1990. At the first two meetings, the Committee was briefed on a preliminary system of existing models configured and applied by DOE and EIA to the analysis underway of National Energy Strategy (NES). That system comprised the 2-tier set of models shown in Figure 1.

The Committee has conducted its work to-date using information provided through presentations from DOE and EIA (See Appendix 1) and from reviews of selected documents (See Bibliography). The Chairman and committee members also met with the Secretary of Energy and his staff on November 6 and 9, 1990 to exchange views on the use of existing models and data in on-going energy strategy analysis at the department and on needs for future capabilities of a National Energy Modeling System (NEMS). Largely on the strength of these presentations and exchanges, the committee has made observations on DOE's use of existing models that is the subject of this advisory report.

The report addresses, in broad terms and at a high level of aggregation, the efficacy of existing models configured by DOE to support the NES activities and the adequacy of the underlying data, assumptions and methodology. It also outlines the Committee's goal in focusing on the long-term development of modeling capabilities at DOE. The committee expects to complete its work and publish a final report in the Fall of 1991.

II. FINDINGS

A. Background

The analytical staff of the Department of Energy (including EIA) has been called upon to support the effort leading to the formulation of a National Energy Strategy (NES). A part of that effort involved the generation of one or more “Reference Case” projections of the U.S. energy situation to the year 2030 along with a set of “excursions” (scenarios) off the reference cases.

The existing modeling capability of the department was used in this effort. The predominant group of models used resided in the Energy Information Administration (EIA) and the DOE Office of Policy, Planning and Analysis (OPPA), although some use

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×

was made of other models, such as the ARGUS (Argonne Utility Simulation) model of the Argonne National Laboratory.

Ad hoc interagency staff organizations, particularly the NES Modeling Subgroups, were created to assist in developing assumptions and off-line parameters for the modeling effort (See Figure 1). The cooperation involving the DOE policy and program offices, the EIA, the national laboratories and ad hoc inter-agency staff organizations in developing assumptions and conducting off-line estimations for the NES exercise is to be commended.

In the Committee's view, the foregoing process was constructive. The working group format served to pull together and synthesize information. The interactions and the pooling of expertise were commendable uses of personnel resources.

B.General Observations

There is no comprehensive model or group of integrated models existing within DOE, or probably within the federal government, that has the analytical capability to respond to the long time horizon needs imposed by the NES effort. In the Committee's view, the approach taken by DOE in using available models as appropriate, along with off-line supplemental analysis as necessary, was a rational response to the department's need for expedient support of the NES process. The rough integration of the modeled and off-line intermediate analyses that DOE accomplished through the calibration of the FOSSIL2 model has been a useful way to maintain consistent accounting and reporting of results. (See Note in Figure 1 on the calibration of FOSSIL2.)

The aggregate structure of the models used, however, has significant limitations relative to the analytical results reported by DOE and presented to the Committee. Thus, it is important for the decisionmakers who will be using the results of the NES analyses to appreciate the limited power of the existing set of models used for evaluating policy choices. It would be misleading to assign too much quantitative precision to the results of the model runs or to presume that the models incorporate a great deal of relevant detailed information that can enhance judgments about the future impact of policy choices, particularly impacts beyond a decade or two. In the presentations that were made to the committee, little reference was made per se to the validation of the models used in the current national energy strategy analysis exercise. Policymakers should appreciate the important role of the a priori assumptions and simplifications, and the off-line contributions made by the NES Modeling Subgroups in shaping the excursions or scenarios, which to a great extent dictated the results of model runs. In instances where such working groups play such an important role, DOE ought to consider enhancing the working group format with greater outside participation.

As the Committee understands it, the NES analysis and the development of a National Energy Modeling System were initiated by the DOE Office of Policy, Planning and Analysis, which was also responsible for the choice of excursions or scenarios to be analyzed and for the selection and application of the FOSSIL2 integrating model. EIA has provided extensive analytical support in these activities, but not in its customary role

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×

of generating, independently, energy information and forecasts such as the Annual Energy Outlook.

C. Specific Comments

1. Long-Run Modeling and Forecasting

None of the current models used in the NES analysis was developed to generate 40 year projections. It would be misleading to presume that the models incorporate a great deal of information relevant to the long-term implications of the excursions or scenarios analyzed.

The models are deterministic, and are based either on historical data and relationships that may or may not have relevance to situations anticipated in the future, or on external judgments about the course of technological and institutional changes. The variances can be high in the relevant parameters as well as in the structure of relationships within the models. It may well be that uncertainties are so great for forecasts over a 30 to 40 year period that deterministic forecasts cease to be useful over such time frames.

There are issues still to be resolved concerning the internal consistency of the models within the two-tier hierarchy of which they are part. (See The FOSSIL2 Model, below.)

2. The FOSSIL2 Model

In the current NES exercise, the FOSSIL2 model is the integrating vehicle used by DOE to yield long-term forecasts. While the calibration and application of FOSSIL2 may be a useful way to (a) achieve rough integration of the analysis performed with other models, and (b) maintain consistent accounting and reporting of results, the aggregate modeling structure (as depicted in Figure 1) has major limitations relative to the analytical results being obtained from the models.

The FOSSIL2 model itself, according to DOE, is not well understood in the department except by a few people in the Office of Policy, Planning and Analysis. And, the staff at EIA indicated that they too did not have in-depth knowledge of the characteristics and workings of the FOSSIL2 model.

The compatibility of the FOSSIL2 model with the other models is also not well understood by DOE. The influence of off-line inputs to the FOSSIL2 model does not seem to have been adequately studied. For example, the influence of the coal-dominated ARGUS model in determining future fuel choices for electric power production in the FOSSIL2 model could be important and perhaps overly restrictive.

Many of the excursions (from the reference cases) analyzed by DOE with the FOSSIL2 model involved substantial changes in the use of energy technologies, and assumptions on technology availabilities, efficiencies and costs. While the FOSSIL2

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×

model does solve for an energy market equilibrium (both prices and quantities), the excursion analyses entailed no reexamination by DOE of the macroeconomic consequences of the technology changes and other changes assumed. Under these circumstances, the Committee questions the outcomes of the excursions (especially for instance those involving high levels of energy conservation) as being potentially inconsistent with reference case economic assumptions. (See also Adequacy of Data, below.)

The Committee sought but did not obtain clear justification from DOE for the selective use of the FOSSIL2 model for virtually all time horizons considered in the NES analysis i.e, to the exclusion of other available integrating models such as the Intermediate Future Forecasting System (IFFS) used by EIA to generate mid-term forecasts.

3. Adequacy of Data

Much of the energy data pertaining to the NES modeling and analysis are on the supply side of the energy markets. To-date, much less effort has been expended on obtaining demand side data. As a consequence, analyses of the demand side are weak.

Parameter values used in the models are based on very limited data and aggregated information about energy consumption and demand side management. For example, there is little information (at DOE and elsewhere) of the potentials for energy efficiency improvements in the industrial sector. The feasibility of new or alternative energy supply and some end-use technologies appear to rely on estimates made by DOE that are based on limited experience and have a documented history of over-optimism. For instance, specifications made by DOE of technology-derived gains associated with clean coal technologies and methanol are important influences in the analysis, but appear to be speculative.

Energy efficiency improvements were invoked by DOE in some cases and ignored in others. Many efficiency measures seem to have been inadequately studied preparatory to the analysis, and may lead to undue reliance being placed on modeling results.

4. Dealing with Uncertainty.

In the analyses to date, it is the Committee's view that little attention has been given to dealing with uncertainties regarding assumptions, input data and results, and to risks posed by an uncertain future. More direct treatment of uncertainties will strengthen the NES analyses.

Ignoring uncertainties could lead to undue reliance on “model outputs” and a tendency to overlook the speculative nature of the projections, particularly over long-term time horizons. In context it is worth emphasizing that, for the models considered in the NES analysis, the Committee is not aware of any basis to assign a greater level of certainty to the differences (“deltas”) in outcomes among various excursions than to the outcomes of the individual excursions themselves.

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×

III. Concluding Remarks

Despite the general limitations of forecasting models and aside from the specific limitations of the models used by DOE in the NES analysis, the Committee believes that models can play a crucial role in enabling informed judgments and decisions to be made in matters of national energy policy. Thus, the committee considers it vital that DOE continue to develop and sustain capabilities for analyzing national energy issues using resources from within the department and from appropriate organizations in both the public and private sectors.

But the models used for energy forecasting and policy analysis have inherent limitations that must be clearly recognized. Indeed, if the future were not malleable there would be little interest and less controversy over the validity of forecasting models in general. In terms of the particular models used in the NES analysis, DOE should take care not to overemphasize the models' capabilities in light of the types of deficiencies observed by the committee. Perhaps, in documenting its current analytical efforts with regard to national energy strategy and communicating findings to the public, DOE should emphasize that the analysis of policy options and combinations thereof were defined and developed drawing on many resources, only one of which was DOE's modeling capabilities.

The decision taken by the Department of Energy to develop and maintain an effective National Energy Modeling System (NEMS) is an important step. The Committee is pleased to participate in that process and to assist DOE in achieving that objective.

The Committee views its charge as follows:

The Committee will develop a statement of the desirable analytical, modeling, and supporting database capabilities necessary to support the evaluation of U.S. energy policy options. The committee will examine the correspondence between the current and desirable capabilities and will recommend priorities for development of the National Energy Modeling System (NEMS). Special attention will be given to the regional and international context for U.S. energy policy development, and to the interaction between energy and environmental policy formulation and analysis.

The more specific objectives of the committee include:

  1. Recommend a process and priorities for on-going modifications to the existing EIA/DOE modeling capabilities to better serve the purposes of planning, forecasting and policy formulation as exemplified by the ongoing NES process, considering data needs, model enhancements and integration. However, detailed technical assessments will not be undertaken of specific models currently in use at DOE and EIA.

  2. Examine whether improved information, both new data series and focused case studies on how energy systems function, would be critical to achieving the recommended modifications and improvements.

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
  1. Discuss the appropriate role for models in the policy formulation process, the inherent limitations of models, and the specific practical limitations and tradeoffs of data collection and integration in developing more effective modeling capabilities.

  2. Examine broadly the capabilities of relevant public, academic, industrial, regional, state and international interests and make general recommendations of how the activities of these parties can complement and enhance the capabilities of the DOE and EIA.

The execution of this charge will require significant effort by the Committee to better understand the prospective applications for a National Energy Modeling System within DOE and possibly by other users; to evaluate the existing modeling resources that are available for use by DOE either internal to the department or elsewhere; and finally, to propose an approach which will enhance the modeling support for future policy analysis and strategic planning given available resources. This effort will require further collection of information from DOE and other sources, and time for evaluation and deliberation by the Committee.

BIBLIOGRAPHY

Energy Information Administration (EIA). July 2, 1990. Requirements Analysis For a National Energy Modeling System. Paper prepared by an Energy Information Administration Working Group. Washington, D.C. U.S. Department of Energy (USDOE).

Energy Information Administration. September 19, 1990. A Comparison of Requirements with Current Capabilities and Issues in the Design of a New System. Prepared by an EIA Working Group. Washington, D.C. USDOE.

The AES Corporation, Arlington, VA. July 6, 1990. An Overview of the FOSSIL2 Model: A Dynamic Long-term Policy Simulation Model of U.S. Energy Supply and Demand . Prepared for the United States Department of Energy, Office of Policy and Evaluation. DOE Contract No. DE-ACO1-89PE79041. Washington, D.C., USDOE.

Energy Information Administration. October 1990. Improving Technology: Modeling Energy Futures for the National Energy Strategy (Draft). EIA. Washington, D.C. U.S. DOE.

U.S. Department of Energy. April 1990. Interim Report. National Energy Strategy. A Compilation of Public Comments. DOE/S-0066P. Washington, D.C.: U.S. DOE.

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×

U.S. Department of Energy. July 26, 1989. Statement of Admiral James D. Watkins, Secretary of Energy, before Committee on Energy and Natural Resources, United States Senate, Washington, D.C.: U.S. DOE

Energy Information Administration (EIA). 1989. Directory of Energy Data Collection Forms: Forms in Use as of October 1989. DOE/EIA-0249(89). Washington, D.C. U.S. DOE.

Energy Information Administration (EIA). 1990. Annual Energy Outlook with Long-Term Projections. DOE/EIA-0383(90). Washington, D.C.: U.S. Government Printing Office.

APPENDIX

DOE and EIA PRESENTATIONS TO THE COMMITTEE ON THE NATIONAL ENERGY MODELING SYSTEM (NEMS)

July 31 & August 1, 1990

The National Energy Strategy and the National Energy Modeling System”

Linda G. Stuntz, Deputy Under Secretary, Policy, Planning and Analysis, U.S.

Department of Energy (DOE)

The Development and Operation of the NEMS: An EIA Perspective”

Calvin A. Kent, Administrator, Energy Information Administration (EIA)

Reference Case for the National Energy Strategy”

Eric Petersen, DOE Office of Policy, Planning and Analysis

Current Configuration and Applications of the NEMS”

W. Calvin Kilgore, Director, EIA Office of Energy Markets and End Use

Use of Energy Models and Data Systems at EIA”

Lawrence A. Pettis, Deputy Administrator, EIA

Requirements Analysis for the NEMS”

C. William Skinner, Technical Assistant to the Administrator, EIA

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×

September 20 & 21, 1990

Update on NES and Key Policy Issues”

Linda G. Stuntz, Deputy Under Secretary, Policy, Planning and Analysis, DOE

Context for the Analysis of Key Policy Issues”

--Descriptions of Options

--Definition of the “Reference Case”

--Conceptual basis for integrated analysis

--Choice of issues to “demonstrate” characteristics & capabilities of models applied to the current NES effort

Eric Petersen, DOE Office of Policy, Planning and Analysis

Sectoral Energy Demand: PC-AEO Models”

John D. Pearson, Director, Energy Analysis & Forecasting Division, EIA Office of Energy Markets and End Use

Energy Supply-Coal & Electricity: NCM & ARGUS”

Mary J. Hutzler, Director, Electric Power Division, EIA Office of Coal, Nuclear, Electric and Alternate Fuels

Energy Supply-Oil & Gas” GAMS & PROLOG”

Susan Shaw, Analysis and Forecasting Branch, Reserves and Natural Gas Division, EIA Office of Oil and Gas.

Integration Model: FOSSIL2”

Roger Nail, AES Corporation, Arlington, Virginia

Oil Market Simulation Model”

Erik Kreil, International/Contingency Information Division,

EIA Office of Energy Markets and End Use

DRI Macroeconomic Model”

Ronald Earley, Economics and Statistics Division, EIA Office of Energy Markets and End Use

Wrap-up on Current NES Analysis Effort”

Robert C. Marlay, Acting Director, DOE Office of Program Review and Analysis

Update on NEMS Development/Look-Ahead”

Calvin A. Kent, Administrator, EIA

Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 103
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 104
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 105
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 106
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 107
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 108
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 109
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 110
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 111
Suggested Citation:"Appendix B: First Advisory Report." National Research Council. 1992. The National Energy Modeling System. Washington, DC: The National Academies Press. doi: 10.17226/1997.
×
Page 112
Next: Appendix C: The Mission and Functions of the Department of Energy »
The National Energy Modeling System Get This Book
×
Buy Paperback | $50.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This book addresses the process and actions for developing enhanced capabilities to analyze energy policy issues and perform strategic planning activities at the U.S. Department of Energy (DOE) on an ongoing basis.

Within the broader context of useful analytical and modeling capabilities within and outside the DOE, this volume examines the requirements that a National Energy Modeling System (NEMS) should fulfill, presents an overall architecture for a NEMS, identifies data needs, and outlines priority actions for timely implementation of the system.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!