committee believes this is a matter of policy that is best addressed by NIH once legitimate program evaluation becomes feasible.
In the course of assessing NIH’s minority research training programs, the study committee engaged in a lengthy and detailed information-gathering process. This effort to collect information quickly became a diagnostic test of the NIH trainee data systems. The test found these data systems and the information available to be uncoordinated and inadequate for the task the committee was assigned. The committee identified the following concerns:
Numerous NIH trainee data sets are distributed across NIH institutes and centers (ICs) in both hard-copy (i.e., annual progress reports, grant supplements) and digital formats (i.e., in-house Excel spreadsheets). Furthermore, archival trainee data sets are housed in off-site storage facilities. The committee determined that some essential trainee tracking data are already being collected on a regular basis by the ICs, but since these data are not stored centrally and electronically, and since the data do not use common definitions, are in different formats, and include quite different forms of information, the task of assessing trainee outcomes across all NIH research training programs is not currently feasible.55
Although NIH-wide trainee data sets such as the Trainee Fellow File (TFF) and the Information for Management, Planning, Analysis, and Coordination (IMPAC) system are centralized and electronic, they do not contain adequate trainee tracking data. The data sets are similarly inadequate to identify minority trainees who participated in targeted programs, because no data element contained within these data sets specifically denotes a program as targeted to minorities. NIH may wish to consider incorporating such a data element into these systems in the future. To that end, NIH-wide consensus on trainee data collection practices in general will ultimately enhance the ability of future evaluators to assess these programs.
Pursuant to a Congressional mandate requiring federal agencies to migrate from paper-based to electronic systems, NIH has undertaken the “electronic Research Administration” (eRA) project to lower costs and administrative effort, expedite extramural grants processing, and provide better-quality information to NIH and the external grantee community. A functional component of eRA is the electronic Streamlined Noncompeting Award Process (eSNAP). When fully implemented, eSNAP will allow researchers and grantee institutions to submit progress reports electronically for their noncompeting awards. The pilot phase for eSNAP began in 2003 and ended in 2004. Currently, grantees are required to submit all hard-copy progress reports to a central NIH mailing address, so that they may be electronically scanned into the eSNAP database. Scanning of progress reports is an interim measure that will eventually be phased out, as full-scale electronic functionality of eSNAP is achieved.