overseas, however, should not block steady progress toward opening U.S. R&D ventures to the participation of firms that contribute in a substantial manner to the economic vitality of the United States.

Therefore, expanded programs to finance pre-commercial R&D should not be closed to the participation of foreign firms that are characterized by substantial contributions to the U.S. GDP. In many cases, participation in these projects would require eligible firms to have extensive R&D, manufacturing, and processing operations within U.S. borders, even when they are neither under the direction of U.S. citizens nor incorporated as independent businesses in the United States. Finally, U.S. government funds provided for pre-commercial R&D would be expended on projects undertaken predominantly in the United States.

Principle 6: Program Evaluation

Rigorous technical and economic evaluation is an essential part of any technology program, especially of efforts to extend federal support for pre-commercial R&D. An independent evaluation of each project should be undertaken approximately five years after an R&D project is initiated. A review of program performance conducted for Congress and the President, under the following options, should be undertaken by a qualified group of experts, including those with technical, managerial, and economic experience. An overall evaluation of the mechanism chosen for any further substantial federal investment in pre-commercial R&D would be appropriate after 10 years.

Current efforts to review government R&D programs have suffered, in some instances, from the fact that annual reports to Congress or the executive branch have been conducted by mission agency employees with an direct interest in having projects they evaluate continue. Technical evaluations of the R&D work and of the contributions to national economic welfare of pre-commercial R&D programs should be conducted by nongovernmental groups that do not have a direct role in program management or funding decisions.

The review proposed for an extended federal program in pre-commercial R&D should be conducted by an independent panel of experts, nominated by the President and confirmed by the Senate. The panel should include a wide range of individuals with both technical and economic expertise to ensure that it represents, to the extent practical, knowledgeable and disinterested reviewers. Admittedly, it is difficult to identify review panels that are composed entirely of disinterested and knowledgeable experts. Such panels can, however, be formed with a careful, rigorous adherence to balance and the transparency of potential conflict and bias.

This type of review would not only evaluate the efficacy of the R&D



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement