AIDS and Behavior: An Integrated Approach (1994)

This appendix provides more detail about the methodology employed by the committee in conducting the analysis of the AIDS grants presented in Chapter 6. Specifically, it reviews sources of information, development of the database, and general limitations of the data.

The primary source of grant information for the three institutes came from the CRISP (Computer Retrieval of Information on Scientific Projects) system. CRISP is an on-line computer-based system that contains information on funded extramural and intramural research in the Public Health Service.

Initial requests for grant information were made to the AIDS coordinators at NIAAA, NIDA, and NIMH. Specifically, IOM staff requested information for all extramural AIDS research grants (all mechanisms), cooperative agreements, contracts, and intramural research projects since the institutes' first funded AIDS research (FY 1983). The institutes responded by compiling and submitting CRISP files for all extramural AIDS-related research grants (including cooperative agreements) and contracts. NIMH was the only institute to submit CRISP files for all intramural AIDS-related research projects.

NIDA and NIMH also submitted comprehensive lists of AIDS grants (titles, investigators, ID numbers, and total funding) for FY

1989 through FY 1992. IOM staff cross-checked these lists against all CRISP abstracts received and both of these sets of information against the institutes' historical budget tables for each fiscal year. Although minor discrepancies were found, IOM staff worked with the AIDS offices and the budget offices of each institute to address them and to ensure that the committee ultimately had access to a complete set of AIDS-related grant abstracts with accurate information. Remaining discrepancies are described in the database section, below.

The CRISP files for each AIDS research grant funded by NIAAA, NIDA, and NIMH contained detailed information identifying the principal investigator, institution, project title, project summary abstract, IRG code, program class code, years of funding, and amounts paid in each year. The identification number for each grant (which also appears on the CRISP file) includes: an activity type code to identify the type of application received and processed (e.g., new and competing); an activity code (e.g., funding mechanism) to identify the category of research (e.g., R01); the institute, center, or division (ICD) code to identify the institute (AA = NIAAA; DA = NIDA; MH = NIMH); a five-digit serial number; and suffixes to identify other relevant information such as the grant year. CRISP files for contracts provide the total funding commitment without a breakdown of spending for each fiscal year, whereas CRISP files for intramural research do not provide funding information.

To guide the analysis of the grants data, the IOM staff and committee also reviewed and referred to a number of documents, including formal and informal budget documents, "Moyer Reports" (FY 1989 through FY 1993), AIDS Expenditure Reports from HHS, the NIMH Strategic Plan for AIDS Research (1993–1998), the NIMH AIDS Research Blueprint for the Second Decade, the NIMH HIV Prevention Highlights, the NIMH Handbook on Developing a Successful Research Application, the NIDA Five-Year Strategic Plan, a variety of documents describing program initiatives at all three institutes, and all program announcements (PAs), requests for applications (RFAs), and requests for proposals (RFPs) issued by NIAAA, NIDA, and NIMH since FY 1983.

To facilitate the grants analysis, IOM staff created a database using the Paradox database management system. A form was developed to record pertinent information for each AIDS grant and contract (Box A.1). In addition to information tracked by the

CRISP system (investigator, title, summary abstract, identification number, etc.), the IOM database includes fields to identify domains of science using the committee's matrix codes. Other fields identify whether a project included or targeted homosexuals, women, racial/ethnic minorities, and people of different economic classes in the study sample, whether it included an analysis of gender, race/ethnicity, and class factors, and whether it included qualitative, evaluation, and health services research. Although the committee intended to analyze the grants on these characteristics, the information available from the CRISP abstracts and other data sources was insufficient for this task. Many abstracts did not specify the composition of the study sample, and, even if they did, most did not mention whether an analysis of gender, race/ethnicity, or class factors was part of the project design. Similarly, many did not make clear if an evaluation component was included.

Three IOM staff members were involved in reading the CRISP abstracts and entering key information into the IOM database. Two staff members coded each grant for all new variables—including science categories. One of these staff members checked each entry to ensure consistency.

Efforts were made to ensure a high level of coding reliability. Two IOM staff members reviewed the science categories for each record in the database. In addition, several committee members and additional IOM staff with relevant science expertise reviewed samples of grants from the database to verify appropriate coding.

As mentioned above, the basic grant information that was entered into the IOM database was originally compiled by the AIDS coordinators at each institute. Every effort was made to ensure consistency and comparability between the grant information in the IOM database and institute budget information. Although institute staff from the AIDS offices and budget offices were extremely helpful in providing information when requested, IOM staff could not obtain explanations for all discrepancies that were found.

Specifically, IOM staff was informed that complete and/or accurate information about grants and grant funding was unavailable for the first several years of AIDS funding at the institutes. For example, summary abstracts were not available for grants funded in FY 1983 and FY 1984. Likewise, the institutes could not explicitly

state which grants were categorized as AIDS research during the first several years of AIDS funding before systematic record-keeping for AIDS research began. Despite these limitations, the database total dollar amounts were comparable to the amounts reported in the institutes' historical budget tables for FY 1987 through FY 1992. Therefore, the committee's analysis primarily focuses on these years.

Another major limitation of the data was the absence of detailed information on study procedures or outcomes. Although this information was not critical to the basic analysis of balance within the institutes' AIDS programs, it is critical to other aspects of the report, in particular the committee's discussions of research findings from institute-supported AIDS intervention projects. Where possible, the committee turned to the published literature for information.