necessity to obtain clearance from the Office of Management and Budget (OMB). Part of the Paperwork Reduction Act of 1995 is designed to ensure that government agencies do not burden the public with needless paperwork. Consequently, with a few exceptions, any survey sent to the public must go through a review by OMB that demonstrates that the data are not already collected and the collection effort is not burdensome. The bad part of OMB clearance is that it takes a long time, and this time must be built into any evaluation plans. The good part is that NSF has staff that is very talented at writing OMB clearance packages. Consequently, surveys can be conducted as part of evaluation activities, but they can be expensive and time-consuming.
The third constraint to NSF’s ability to conduct evaluations derives from the Government Performance Results Act. GPRA requires a great deal of input from program officers who review all the annual reports they receive from grantees and synthesize highlights into nuggets. These are written up as short paragraphs and submitted into the NSF database. Each January this large number of nuggets is transmitted to the Congress as part of a large package of materials to meet GPRA reporting requirements. At the same time, GPRA reporting is tied into NSF’s strategic planning exercises. The NSF budget cycle is always three years in advance. Consequently, planning always looks three years in advance, but the GRPR reports focus on how funds were allocated during the current fiscal year. This disconnect between planning and reporting cycles hampers NSF’s ability to conduct evaluations because there is not enough time for outcomes to occur.
The final constraint to NSF’s ability to conduct evaluations is the privacy act. The privacy act limits the collection of data, limits the use of Social Security numbers as an identifier, and limits access to data collected. While the privacy act does permit disclosure of personal information for program evaluation, interpretation of the act is not consistent across government agencies. Many agencies restrict the disclosure of personal data for nearly all reasons.
This paper describes one of the evaluations conducted by the Directorate for Biological Sciences through a contract to SRI International to examine outcomes and impacts of the Minority Postdoctoral Research Fellowship (MPRF) program. This program was started in 1990 and is still in operation. It is a postdoctoral fellowship program in (1) biological sciences and (2) social and behavioral sciences and economics. The goal of the program is to increase the number of underrepresented minorities in leadership positions in the United States in academia, industry, and government.
The number of applicants and awards in this program has been relatively small, averaging 26 applications and 13 awards each year. By 2002