and divisions into three core programs: Research, Research Training and Career Development, and Facilities.

The GPRA allows an agency to either aggregate, desegregate, or consolidate its program activities in the way most appropriate to performance reporting. We consider it essential that the activities of our 24 different institutes, centers, and divisions be aggregated. Our rationale for this with respect to NIH's research programs is simple. Disease is typically systemic. It is influenced by multiple factors and affects more than one organ or body system. Therefore, expertise across a wide range of disciplines is necessary to establish the mechanisms and etiology of disease and to develop strategies for diagnosis, treatment, and prevention. Multidisciplinary expertise necessarily cuts across institute boundaries; therefore, we have to combine the efforts of many of our institutes to make progress on any particular disease or disability. Following the same line of reasoning, training and facility needs are also not unique to any one particular institute or center. Therefore, we must coordinate and collaborate these efforts across NIH to ensure that we can meet our long-term goals.

Establishment of Program Outcomes and Means

To assess the performance of the three aggregated programs, we have established for each a framework of expected outcomes and program means. As the name implies, the expected program outcomes are the expected, tangible results of NIH programs. The program means reflect the process and management activities that we undertake to support the conduct of our programs and to enable us to achieve our goals. For each of the expected program outcomes and program means, we have set performance goals that identify what we expect to accomplish. Each goal is accompanied by a set of performance indicators, which will be used to measure our success in achieving it. I will discuss a few examples of the expected program outcomes, the program means, and the performance goals and indicators that we are considering for our research program. I use the term "considering" because everything we are doing for the GPRA is very much in a state of development. What I tell you today may not necessarily be what will soon be forwarded to Congress.

We have defined two broad expected program outcomes that are expected. The first is increased understanding of normal and abnormal biological functions and behavior. The second is improved prevention, diagnosis, and treatment of diseases and disabilities. For each expected program outcome in the research program, we have defined performance goals, many of which are not assessable by quantitative measures.

As previous speakers have pointed out, agencies whose missions encompass fundamental science face unique challenges in implementing the quantitative evaluations that seem to be preferred under the GPRA. This is an issue to which we have devoted much time and effort. We have had extensive internal discussions and have also sought substantial input from outside groups. In particular, the National Science Foundation has been very helpful to us. They have generally been somewhat ahead of NIH in the GPRA process, and we have benefited from their insights, experiences, and lessons learned as they, too, have grappled with GPRA.

First and foremost, we concluded that just because something can be counted does not mean it should be reported. We decided that a combination of both qualitative and quantitative performance goals and indicators is the most meaningful response for NIH. We think that strictly numerical results are neither feasible nor sufficient to capture the breadth and impact of NIH's research activities. Although conventional scientific and research metrics can be relevant, they measure only some of the dimensions of output. These metrics provide important data, but they alone cannot assess the full scope of the quality of our work, its relevance, and the impact of our research program on human health that



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement