National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

THINKING STRATEGICALLY

THE APPROPRIATE USE OF METRICS FOR THE CLIMATE CHANGE SCIENCE PROGRAM

Committee on Metrics for Global Change Research

Climate Research Committee

Board on Atmospheric Sciences and Climate

Division on Earth and Life Studies

NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS
Washington, D.C. www.nap.edu

Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

THE NATIONAL ACADEMIES PRESS
500 Fifth Street, N.W. Washington, DC 20001

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This study was supported by the federal agencies of the U.S. Climate Change Science Program through the National Aeronautics and Space Administration under Contract No. NASW-01008. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project.

International Standard Book Number (ISBN) 0-309-09659-6 (Book)

Library of Congress Control Number 2005929740

Additional copies of this report are available from the
National Academies Press,
500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313; Internet http://www.nap.edu

Cover design by Van Nguyen, the National Academies Press.

Copyright 2005 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America.

Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

THE NATIONAL ACADEMIES

Advisers to the Nation on Science, Engineering, and Medicine


The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.


The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Wm. A. Wulf is president of the National Academy of Engineering.


The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.


The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Wm. A. Wulf are chair and vice chair, respectively, of the National Research Council.

www.national-academies.org

Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

COMMITTEE ON METRICS FOR GLOBAL CHANGE RESEARCH

ERIC J. BARRON, Chair,

Pennsylvania State University, University Park

ROGER C. BALES,

University of California, Merced

JOHN B. CARBERRY,

E.I. duPont de Nemours & Company, Newark, Delaware

DAVID J.C. CONSTABLE,

GlaxoSmithKline, King of Prussia, Pennsylvania

PAUL V. DESANKER,

Pennsylvania State University, University Park

MARVIN A. GELLER,

State University of New York, Stonybrook

EILEEN E. HOFMANN,

Old Dominion University, Norfolk, Virginia

HENRY D. JACOBY,

Massachusetts Institute of Technology, Cambridge

JOYCE E. PENNER,

University of Michigan, Ann Arbor

EUGENE A. ROSA,

Washington State University, Pullman

SUSAN E. TRUMBORE,

University of California, Irvine

KARL K. TUREKIAN,

Yale University, New Haven, Connecticut

CARL WUNSCH,

Massachusetts Institute of Technology, Cambridge

NRC Staff

ANNE LINN, Study Director

Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

CLIMATE RESEARCH COMMITTEE

ANTONIO J. BUSALACCHI, JR., Chair,

University of Maryland, College Park

LEE E. BRANSCOME,

Climatological Consulting Cooperation, Palm Beach Gardens, Florida

JAMES A. COAKLEY, JR.,

Oregon State University, Corvallis

JULIA E. COLE,

University of Arizona, Tucson

JUDITH A. CURRY,

Georgia Institute of Technology, Atlanta

CLARA DESER,

National Center for Atmospheric Research, Boulder, Colorado

DAVID KAROLY,

University of Oklahoma, Norman

ROBERT J. LEMPERT,

RAND Corporation, Santa Monica, California

LINDA O. MEARNS,

National Center for Atmospheric Research, Boulder, Colorado

GERALD A. MEEHL,

National Center for Atmospheric Research, Boulder, Colorado

PETER B. RHINES,

University of Washington, Seattle

W. JAMES SHUTTLEWORTH,

University of Arizona, Tucson

LYNNE D. TALLEY,

Scripps Institution of Oceanography, La Jolla, California

HASSAN VIRJI,

International START Secretariat, Washington, D.C.

YUK YUNG,

California Institute of Technology, Pasadena

NRC Staff

AMANDA STAUDT, Senior Program Officer

ROB GREENWAY, Senior Program Assistant

Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

BOARD ON ATMOSPHERIC SCIENCES AND CLIMATE

ROBERT J. SERAFIN, Chair,

National Center for Atmospheric Research, Boulder, Colorado

M. JOAN ALEXANDER,

Colorado Research Associates, Boulder

FREDERICK R. ANDERSON,

McKenna Long & Aldridge LLP, Washington, D.C.

MICHAEL L. BENDER,

Princeton University, New Jersey

ROSINA M. BIERBAUM,

University of Michigan, Ann Arbor

MARY ANNE CARROLL,

University of Michigan, Ann Arbor

CAROL ANNE CLAYSON,

Florida State University, Tallahassee

WALTER F. DABBERDT,

Vaisala Inc., Boulder, Colorado

KERRY A. EMANUEL,

Massachusetts Institute of Technology, Cambridge

DENNIS L. HARTMANN,

University of Washington, Seattle

PETER R. LEAVITT,

Weather Information, Inc., Newton, Massachusetts

JENNIFER A. LOGAN,

Harvard University, Cambridge, Massachusetts

VERNON R. MORRIS,

Howard University, Washington, D.C.

WILLIAM J. RANDEL,

National Center for Atmospheric Research, Boulder, Colorado

THOMAS H. VONDER HAAR,

Colorado State University/CIRA, Fort Collins

ROGER M. WAKIMOTO,

University of California, Los Angeles Ex Officio Members

ANTONIO J. BUSALACCHI, JR.,

University of Maryland, College Park

ERIC F. WOOD,

Princeton University, New Jersey

NRC Staff

CHRIS ELFRING, Director

PAUL CUTLER, Senior Program Officer

AMANDA STAUDT, Senior Program Officer

MARIA UHLE, Program Officer

ELIZABETH A. GALINIS, Senior Program Assistant

ROB GREENWAY, Senior Program Assistant

DIANE GUSTAFSON, Administrative Coordinator

ANDREAS SOHRE, Financial Associate

Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

Acknowledgments

This report has been reviewed by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:

Jack Azar, XEROX Corporation, Webster, New York

Rosina Bierbaum, University of Michigan, Ann Arbor

Susan Cozzens, Georgia Institute of Technology, Atlanta

Jack Fellows, University Corporation for Atmospheric Research, Boulder, Colorado

Debra Knopman, RAND Corporation, Arlington, Virginia

Roger Lukas, University of Hawaii, Honolulu

Michael Mann, University of Virginia, Charlottsville

Philip Marcus, University of California, Berkeley

Linda Mearns, National Center for Atmospheric Research, Boulder, Colorado

Elinor Ostrom, Indiana University, Bloomington

David Skole, Michigan State University, East Lansing

Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

Although the individuals listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions and recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Robert Frosch, Harvard University, and Thomas Graedel, Yale University. Appointed by the National Research Council, they were responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

Preface

Federal agencies are increasingly being asked to document progress and measure performance to improve their accountability to Congress and the public and to provide information useful for making budget decisions. This task can be difficult to accomplish, especially in a program as complex as the Climate Change Science Program (CCSP), which spans all of the environmental and related social science disciplines and includes activities ranging from basic research to decision making in 13 federal agencies. Current approaches to evaluate progress (e.g., peer review of basic research, reduction of uncertainty) focus on particular aspects of the CCSP and/or have other limitations. For example, gaining improved understanding of the climate system can lead to increased uncertainties about some aspects of the system, yet progress has clearly been made. So, the question remains: How can progress in climate science be demonstrated after nearly 15 years of sponsored research and observations?

At the request of Dr. James Mahoney, director of the U.S. Climate Change Science Program and chair of the Subcommittee on Global Change Research, the National Research Council established a committee to develop quantitative metrics and performance measures for documenting progress and evaluating future performance for selected areas of global change and climate change research. Committee membership included researchers drawn from a wide range of global change disciplines and experts from industry and government with practical experience in developing and using metrics.

Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×

The Committee on Metrics for Global Change Research held three meetings from December 2003 to June 2004 to discuss the issues and to gather input in three major areas:

  1. the different types of metrics (e.g., input, outcome) and the different scales of programs that can be evaluated usefully by such measures;

  2. the experience of industry, academia, and federal government agencies in measuring performance; and

  3. lessons learned from retrospective analysis of science programs.

A fourth meeting (September-October 2004) was devoted to writing this report. In preparing its report the committee strove to provide practical advice on the applicability of performance measures across the full range of CCSP goals and approaches—from discovery science, to modeling and assessment, to communicating results and managing risk.

The committee thanks the following individuals for making presentations or providing other input: David Bader, Susan Cozzens, James Hack, Richard Hallgren, Jack Kaye, Charles Kennel, Mike MacCracken, James Mahoney, Richard Moss, Franklin Nutter, Cheryl Oros, John Parascandola, Craig Robinson, Jason Rothenberg, Sherwood Rowland, and Spencer Weart. Thanks also go to members of the Climate Research Committee and Board on Atmospheric Sciences and Climate—particularly Anthony Busalacchi, James Coakley, David Karoly, Robert Serafin, and Lynne Talley—for their input and guidance throughout the study. Finally, the committee extends its appreciation to the NRC staff, particularly study director Anne Linn, for their highly professional contributions to this report.


Eric Barron

Chair

Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R1
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R2
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R3
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R4
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R5
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R6
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R8
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R9
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R10
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R11
Suggested Citation:"Front Matter." National Research Council. 2005. Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. Washington, DC: The National Academies Press. doi: 10.17226/11292.
×
Page R12
Next: Executive Summary »
Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program Get This Book
×
Buy Paperback | $52.00 Buy Ebook | $41.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Climate Change Science Program (CCSP) and its predecessor U.S. Global Change Research Program have sponsored climate research and observations for nearly 15 years, yet the overall progress of the program has not been measured systematically. Metrics—a system of measurement that includes the item being measured, the unit of measurement, and the value of the unit—offer a tool for measuring such progress; improving program performance; and demonstrating program successes to Congress, the Office of Management and Budget, and the public. This report lays out a framework for creating and implementing metrics for the CCSP. A general set of metrics provides a starting point for identifying the most important measures, and the principles provide guidance for refining the metrics and avoiding unintended consequences.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!