Finally, there is an educational and training aspect to spillovers: R&D, particularly when conducted jointly among companies and universities, builds the internal capacity of firms to be innovative. In the arena of human capital, these spillovers are huge and have fueled a great deal of this country's economic growth.
The program has taken on a tough set of problems, Dr. Kelly continued, and although the problems are difficult, the administration believes that it is well worth the effort to have the ATP address them. For instance, some research problems cannot be assigned easily to mission agencies, such as the National Institutes of Health or the Department of Agriculture. Some technology initiatives extend across a number or fields, and the ATP has played an indispensable role in addressing such R&D problems.
Moreover, in achieving a balanced portfolio of risks in R&D projects—from relatively risky university research at the frontiers of new knowledge to less risky advances in industrial process—the presence of the ATP has helped to spread that risk across agencies.
The challenge, continued Dr. Kelly, is to take this complex set of objectives and make them operational. It has been said today that the ATP is an experiment and, as with any experiment, it is important to make adjustments along the way. From the perspective of an official at the White House Office of Technology Policy, Dr. Kelly said that he was very gratified that ATP management has made adjustments in the program in the past several years. Last year, for example, an evaluation yielded changes in how the ATP treats the participation of small businesses and states in the program.
Evaluation of public R&D programs faces a unique challenge, in that federal support for research involves provision of public funds for projects whose outcomes are inherently uncertain. Dr. Kelly pointed to the Academy's recent publication Evaluating Federal Research Programs as a thoughtful examination of the issue. Plainly, you do not want to hand out public money without accountability, but specifying an evaluation program to ensure accountability when research outcomes are uncertain is a difficult task. Dr. Kelly noted that, in the Academy's study, a problem in the evaluation of basic research was identified: If you set the objectives too narrowly, you wind up characterizing an outcome very conservatively. That is, the evaluation criteria discourage the risk taking that the R&D program is supposed to foster. Dr. Kelly said that society must be prepared to tolerate some failure in R&D programs.
In concluding, Dr. Kelly said that the panelists could provide "the gift of common sense" in providing their perspectives on how the ATP has operated and adapted to change over the past 10 years.