DR. CEHELSKY: One of the most interesting comments that was made this morning by several people is that we have to be careful about the use of quantitative measures. That is rather refreshing. I think that it states a true precept. It is something that responds to a lot of pressure that we are feeling within the federal government, certainly at NSF, to be much more quantitative about assessing what we do, evaluating our performance, and justifying our existence. I found it interesting that this is a concept that industry seems to adopt for itself, in other words, that you cannot quantify everything that you do, and you cannot assess everything quantitatively. But I think it poses a dilemma for us. Industry ultimately works on the basis of profit, and there are some very clear goals. Industry works in terms of profit and clear objectives—products and processes that are productive by fairly common-sense measures, for the most part. How do we translate that to the government arena, and particularly to NSF, where the product is so open ended and where the process itself, even when we measure it, does not satisfy us in terms of yielding results? For instance, citations give us an indirect measure, an artifact perhaps, but they do not really tell us what the taxpayer gets out of this investment. This occurs because the ultimate investment is an investment in productivity, the economy, and social returns. So we get back to the very difficult question of how we measure, or alternatively, how we convince the public, that in fact we have made a wise investment of those monies that have come into the federal treasury, even if we cannot really measure the ultimate impact to any level of specificity.

DR. McTAGUE: I think there are a couple of points. One of them has to do with quantification. If there is a number, people will misuse it. The most fundamental function of government probably is national security. I do not see anyone attempting to quantify national security in some sense. It represents a qualitative judgment. You do not ask how many nonwars you get per dollar, for example, or how many non-body bags you get per dollar. No one would suggest putting a quantitative measure on national security as an outcome. You might put a quantitative measure on it as a process. Are we being efficient about procurement? The answer is no, et cetera. So here is an area in which people make qualitative judgments. Is the world more secure because of what we are spending, for example? Is what we are spending on national security getting in the way of other things that we want, in terms of quality of life? People make qualitative judgments on the big issues. So there is no reason why you cannot convince people that qualitative judgments apply equally to other fundamental roles of government, but you have to decide for yourself what your role really is. Is your main role educating very good technical people? Is your main role educating the general population on the nature of technology in a democratic society? Is your main role strengthening the university system? Is your main role developing more and better researchers in universities? Is your main role improving the impact on the economy? Is it having an impact on other social issues, such as health, environment, and public safety? Is it expanding the human vision? Is it all of the above? The first thing you must do is decide what your own role is.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement