4
Program Execution

The TriService Nursing Research Program (TSNR Program) is established at the Uniformed Services University of the Health Sciences (USUHS), as recently mandated by Congress in the fiscal year (FY) 1996 Department of Defense Authorization Act (Chapter 104, title 10, U.S. Code as amended, see Chapter 1). The legislation further mandates that a TriService Nursing Research Group (TSNR Group) administer the program. The purpose of the TSNR Program is to facilitate research on the furnishing of care and services by nurses in the armed forces by expanding the body of scientific knowledge upon which military nursing practice is based. Background information about the TSNR Program was provided by the program or is based on data provided by the program. Supplementary information was provided by a small sample of funded and unfunded applicants.

In 1996, the chief of the Army Nurse Corps and the directors of the Navy and Air Force Nurse Corps agreed on the following goals for the TSNR Program (TriService Nursing Brief to the Institute of Medicine Committee on Military Nursing Research, January 27, 1996):

  • continue its growth and improvement;
  • identify, reduce, and eliminate barriers to research process and progress;
  • strengthen existing bonds and create new bonds with civilian universities;
  • maximize collaboration with triservice nurses who have research, doctoral backgrounds; and


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 51
--> 4 Program Execution The TriService Nursing Research Program (TSNR Program) is established at the Uniformed Services University of the Health Sciences (USUHS), as recently mandated by Congress in the fiscal year (FY) 1996 Department of Defense Authorization Act (Chapter 104, title 10, U.S. Code as amended, see Chapter 1). The legislation further mandates that a TriService Nursing Research Group (TSNR Group) administer the program. The purpose of the TSNR Program is to facilitate research on the furnishing of care and services by nurses in the armed forces by expanding the body of scientific knowledge upon which military nursing practice is based. Background information about the TSNR Program was provided by the program or is based on data provided by the program. Supplementary information was provided by a small sample of funded and unfunded applicants. In 1996, the chief of the Army Nurse Corps and the directors of the Navy and Air Force Nurse Corps agreed on the following goals for the TSNR Program (TriService Nursing Brief to the Institute of Medicine Committee on Military Nursing Research, January 27, 1996): continue its growth and improvement; identify, reduce, and eliminate barriers to research process and progress; strengthen existing bonds and create new bonds with civilian universities; maximize collaboration with triservice nurses who have research, doctoral backgrounds; and

OCR for page 51
--> ensure ongoing, predictable levels of resource support (dollars and staff) for its researchers and for the TSNR Program itself. The full text of these joint goals appears in Appendix C. General Program Administration The TSNR Group consists of six doctorally prepared military nurses, two each from the Army, Navy, and Air Force, representing both the active and the reserve components. Members are appointed by the chief and directors of the respective Nurse Corps. Length of experience as a TSNR Group member varies; however, continuity has been achieved through staggered terms. The position of chair of the TSNR Group rotates among the three services, and the intent is to have it rotate across the active and reserve components as well. To serve as chair, an individual must have served on the TSNR Group for at least 1 year prior to assuming leadership. The TSNR Program currently fulfills its mandate to expand the body of knowledge upon which military nursing practice is based in two ways: first, by funding research proposals of active, reserve, and National Guard Nurse Corps officers, and second, by sponsoring grants-writing workshops for military nurse researchers. Administration of the TSNR Program at Uniformed Services University of the Health Sciences Daily administration of the program is handled by the director of the USUHS Office of Research Administration. This is consistent with the mission of USUHS, which ''. . . serve[s] the uniformed services and the Nation as an outstanding academic health sciences center with a worldwide perspective for education, research, service, and consultation. . . .'' The university is ". . . unique in relating these activities to military medicine, disaster medicine, and military medical readiness" (USUHS, 1996). The university is authorized to grant appropriate advanced degrees, to establish post-doctoral and postgraduate programs and technological institutes, to conduct medical readiness training and continuing education for uniformed members of the health professions, to prepare individuals for careers in the health professions in the uniformed services, and to award grants. USUHS serves all three military services as does the TSNR Program. Administrative support of the TSNR Program within the Office of Research Administration currently consists of one full-time equivalent (FTE) clerk typist/administrative assistant, 0.3 FTE GS-11 grants management specialist,

OCR for page 51
--> and 0.4 FTE overall program administrator. Administration costs were reported to be approximately $120 thousand annually for the first 4 years of operation, exclusive of the salary of the program administrator, who is a military officer. Space and certain other indirect costs were contributed by USUHS. The director of the Office of Research Administration handles overall administration of the program, directs and oversees all grant actions, acts as executive secretary at meetings and teleconferences of the Scientific Review Panel, and prepares and disseminates executive summaries. At the end of FY 1995, the Office of USUHS Research Administration subcontracted specific tasks in support of the TSNR Program. This $305,000 subcontract provides for a number of activities (see Box 4-1), including some not previously implemented by the TSNR Program. These new activities appear to be directed mainly toward improved program monitoring and dissemination. Administration of Grants The USUHS Office of Research Administration receives congressionally mandated funds for the TSNR Program from the Department of Defense (DOD) and channels them to an institution with which the principal investigator is affiliated or, at the principal investigator's request, to a not-for-profit foundation that provides grants administration. Many TSNR Program grants have been administered through nonprofit foundations, principally the Henry M. Jackson Foundation for the Advancement of Military Medicine—a private, not-for-profit organization chartered by Congress in 1983 to support medical research and education at USUHS and throughout the military medical community (Henry M. Jackson Foundation, 1996). The indirect costs of using the foundations have been approximately 14 to 16 percent. The use of a foundation for grant administration by active-duty military personnel makes it possible for the investigator to spend grant funds for the study beyond the end date of the award. According to several grant recipients, it also allows the investigator to complete hiring actions and acquisition of materials in a timely manner and facilitates the hiring of the most qualified candidates for conducting research. The ability to hire personnel to aid in the research may be especially valuable for active-duty military nurse researchers for whom primary work responsibilities are clinical or administrative rather than research.

OCR for page 51
--> Box 4-1 Subcontractor Services for the TriService Nursing Research Program, FY 1996 Support for Peer Review of Proposals Assists in the selection of members of the peer review panel to provide scientific review of all proposals. Assists selection of primary and secondary reviewers for all proposals. Categorizes proposals submitted for funding consideration by subject area to facilitate identification of scientific expertise required for peer review. Provides two notetakers at each peer review panel discussion (approximately 3 days) to record comments. Using written reviews from primary and secondary reviewers, and the salient features of the written comments of the panel discussions from the notetakers, prepares summary reviews of proposals to be returned to each respective applicant. Composes positive and negative award letters and prepares them for mailing through the USUHS office. Develops a database log to track the status of all submitted proposals. Develops a file/database of all active and reserve nurses who are master's or doctorally prepared or who are currently in an advanced degree program; lists clinical specialty/practice and area of research interest. Support for Program Management Develops and prepares presentation slides and viewgraphs for program briefings. Prepares program reports and information papers. Prepares program budgets and summaries. Provides written review of semiannual and annual reports of funded research. Provides support for site visits to funded institutions. Develops, prepares, publishes, and disseminates a newsletter to provide program status and updates to program staff and awardees. Support for Grants-Writing Workshops Schedules and reserves meeting sites. Prepares packets of read-ahead materials for attendees. Prepares travel orders for program staff and attendees. Most are government travel requests executed by staff at USUHS. Provides on-site support in the form of two staff for meetings. NOTE: Activities that are new to the TSNR Program are shown in bold.

OCR for page 51
--> Requests for Proposals Dissemination Requests for Proposals (RFPs) are disseminated by the USUHS Office of Research Administration. Deadlines for submission of applications have varied, depending on the availability of congressionally appropriated funds. In FY 1995, the $5 million appropriated and mandated by Congress for the TSNR Program was withheld by the DOD comptroller until May 4, 1995—seven months into the fiscal year—and the money had to be obligated by September 30, 1995, the end of the fiscal year. According to the 1995 chair of the TSNR Group, the late call for proposals severely limited the time for potential applicants to learn of the availability of funding and to prepare meritorious proposals that met deadlines mandated by the need to encumber funds. It also hampered the TSNR Program's ability to execute the mechanisms for peer review and to obligate the money most efficiently. In FY 1996, the TSNR Program issued its call for proposals in December 1995, with the caveat: "It is anticipated that five million dollars will be available for military nursing research in FY 96." The Institute of Medicine (IOM) committee commends the TSNR Program for making the announcement well before funds were made available. Current dissemination activity consists of distribution of the RFP to the offices of the Nurse Corps chief and directors, all commanders of military medical units, reserve nurse units, and all deans of graduate programs in civilian schools of nursing that have graduate programs. Anecdotal evidence from a small sample of grantees and unfunded applicants suggests that information about the TSNR Program funds has been received late. Proposal Format In this young competitive grants program, the format of proposal applications has evolved steadily, as shown in Table 4-1. In 1994, for the first time, the RFP stated the primary goal of the program as "to improve nursing care for DOD beneficiaries by expanding the body of scientific knowledge upon which military nursing practice is based." The change in the award categories introduced stricter eligibility requirements. The RFP strongly encouraged new investigators to have a sponsor with research experience assist them with proposal development. Contact information for TSNR Group members was provided to assist new investigators in locating sponsors. The RFP for FY 1996 introduces a notable change: funding support for up to 2 years for new investigators and up to 3 years for experienced investigators.

OCR for page 51
--> TABLE 4-1 Evolution of TriService Nursing Research Program Eligibilities and Requirements, FY 1992–1996   Requirements Eligibility 1992 1993 1994 1995 1996 Military status         Minimum education R.N. R.N., Baccalaureate degree, PHS Form 398 R.N., Baccalaureate degree (depends on award category) R.N., Baccalaureate degree (depends on award category) R.N., Baccalaureate degree (depends on award category) Application form No form required PHS Form 398 PHS Form 398 PHS Form 398, plus triservice application cover page PHS Form 398, plus cover page with key words, and an additional pagea IRB approval required         American Association for Accreditation of Lab Animal Control facilityb Not specified       Disbursement of funds Military installations and universities Same as 1992 Use of a nonprofit foundation for grants administration or project work orders allowed for military installations Same as 1994 Same as 1994 and 1995, plus universities and other nonprofit entities may receive grants directly from USUHSc

OCR for page 51
-->   Requirements Eligibility 1992 1993 1994 1995 1996 Award categories and funding caps, if any   Junior investigators,d cap of $75,000; senior investigatorse New investigators,f experienced investigatorsg New investigators, experienced investigators New investigators, $40,000/year; experienced investigators, $100,000/year Period of award 1 year 1 year 1 year 1 year 2 years for new investigators; 3 years for experienced investigators NOTE: PHS = Public Health Service a Page requires a statement of relevance of the proposed research to military nursing and the way in which the research will expand the body of scientific knowledge upon which military nursing practice in based. b Applicable only for animal studies. c Uniformed Services University of the Health Sciences. d Junior investigator: an applicant who has never been a designated principal investigator on any federal or privately supported research project for more than 3 years. e Senior investigator: not defined. f New investigator (FY 1994–1995): a licensed registered nurse with a minimum of a baccalaureate degree in nursing or a baccalaureate degree in another discipline and a graduate degree or candidacy in nursing. g Experienced investigator: a licensed registered nurse who has previously received competitive research grants of at least $5,000 (excluding recipients of a National Research Service Award).

OCR for page 51
--> Budget proposals are required on a per annum basis for the number of years of support requested. Grant Review Process The TSNR Program uses a three-level grant review process: In level 1, the Scientific Review Panel reviews proposals for scientific and technical merit. In level 2, the TSNR Group reviews recommendations of the Scientific Review Panel and considers programmatic issues, and in level 3, the corps chief and directors approve or disapprove recommendations from the TSNR Group based on service-specific priorities and relevance to military nursing. More specific information about each level follows. Scientific Review Panel (Peer Review, Level 1) Peer review is defined as the making of substantive evaluative judgments about scientific merit and the quality of scholarship and scholarly productivity. Peer review helps ensure that the conduct and reporting of research meet accepted scientific standards. The approaches to such assurance include the practice of peer review by study sections and advisory councils, the work of journal editors and referees, mock reviews by colleagues, and the process of collegial discourse. Because peer review is the primary mechanism for evaluating science and scientists, the committee closely examined the detailed information received about the peer review process used by the TSNR Program. The TSNR Program provided the following information about 1995 scientific peer review practices and procedures to the committee. Proposals and Review Materials The RFP instructs researchers to send a proposal plus a specified number of copies of the proposal to the USUHS Office of Research Administration by a specified date. Proposal packages are reviewed for format, checked for Institutional Review Board (IRB) submission, and processed for distribution to appropriate members of the Scientific Review Panel. This is an ad hoc committee of 18 persons, distributed among the three armed services with representation from the active and reserve components. Each member of the Scientific Review Panel receives a set of materials including the following: copies of all proposals;

OCR for page 51
--> master list indicating the proposal number, title of the proposal, name and service affiliation of the principal investigator, and amount of funding requested; review criteria; outline of "review summaries" to be prepared by each primary and secondary reviewer for each proposal; listing of expected contents of review summaries (a brief description of the study, strengths of the proposal, weaknesses of the proposal, relevance to military nursing research, recommendations [including approval, modification, or disapproval and a priority score); a copy of the current RFP; and a list of proposals funded in previous years. Selection of Scientific Review Panel Members Each TSNR Group member identifies 6 candidates for the Scientific Review Panel from his or her respective service and component to form a pool of 36 triservice candidates. To be eligible for consideration, the individual must be an active member of the service and component represented by the TSNR Group member, have research knowledge demonstrated by an earned doctorate and research experience, and be available to review grant applications in advance and at the time of the deliberations. Current members of the TSNR Group and individuals who are submitting grants during the current funding cycle are excluded from consideration. An attempt is made to achieve diversity in terms of subject matter, knowledge of research methods, and position (administrative or clinical). Those candidates who can be available are asked for a curriculum vita. Names, contact information, and curriculum vitae are provided to the program administrator who makes the final decisions regarding the match of proposal content and method with reviewer qualifications and with the stipulation that primary reviewers may not be from the same service as the individual submitting the proposal. The IOM committee notes that the pool of available reviewers is limited by the number of military nurses who meet the qualifications (see Chapter 3, Table 3–1). In 1992, the decision was made to select only military active-duty and reserve nurses to serve on the annual Scientific Review Panel. Chair of the Scientific Review Panel The chair of the Scientific Review Panel is selected by the members of the TSNR Group, based upon service, rank, availability, and capability to manage

OCR for page 51
--> the panel. The chair reviews a smaller number of proposals than other panel members because of additional responsibilities that include ensuring adequate discussion of each proposal; accepting motions; counting votes; and assisting with the preparation of the final report, the vote tally, the "After Action Report" (an analysis of the process with recommendations for future improvements), and summaries for presentation to the corps chief and directors. Orientation of the Scientific Review Panel An orientation is held to facilitate group process since the panel members have never before worked as a group. As an ad hoc group, some panelists are first-time reviewers; a few have served on the panel for several years. Some will have attended one or more TSNR Group grants-writing workshops; some will not. Deliberations and Priority Scoring System In 1995, the stated purpose of the TSNR Program Scientific Review Panel was to review new and competitive renewal applications requesting funding. Under the direction of the chair, the Scientific Review Panel is expected to judge scientific merit in a manner whereby the integrity of individuals is preserved, proposals are rated, and summary statements cover the assessment of each proposal's scientific and technical merit. All meetings are held at USUHS. Since 1993, reviews have been completed in 3 days. Meeting content includes discussion regarding conflict of interest and procedures to avoid conflict of interest, a review of the scoring system (see below), and closed-door deliberation of submitted proposals. During deliberations, the primary and secondary reviewers present an oral review of the proposal. Members are expected to have read and become familiar with each application so that all can enter into the general discussion after written reviews have been presented. Discussion by all members of the Scientific Review Panel is followed by one of three motions (approval, approval with modifications, disapproval), and a voice vote by the entire Scientific Review Panel is recorded by the chair. The category approval indicates that the application is of sufficient merit to be worthy of support. A vote for approval recommends that an award be made if sufficient funds are available. Reviewers may recommend accepting the

OCR for page 51
--> application as submitted or with adjustments in time, funding, or scope. Budgetary considerations are not be used in determining if an application is to be approved. The category disapproval is to be used for applications that are judged of insufficient merit to be worthy of support. In 1995, the category approval with modification was used for applications that the Scientific Review Panel considered worthy of funding but judged to require either a minor change, clarification, or more information. These applications were to be managed in one of two ways: While the Scientific Review Panel was in session, the principal investigator was contacted for the additional information. The panel postponed discussion until the information was received. The Scientific Review Panel deferred the application because the principal investigator could not be reached or because it needed specific written information. Three members of the Scientific Review Panel, at least one of whom was the primary or secondary reviewer, were assigned to a rereview panel for each application that was approved with modification. Each rereview panel was convened by teleconference for the discussion of and vote on the approval status of the application. For approved proposals, each Scientific Review Panel member individually, and privately, records a numerical rating that reflects his or her own opinion of the scientific merit relative to the quality of the proposed research. The numerical rating is based on a scale from 1.0 (best) to 5.0 (the least acceptable rating) with increments of 0.1. If two or more members of the Scientific Review Panel vote against the majority recommendation, a minority critique must be provided. The executive secretary gives each reviewed proposal a final three-digit score by averaging all reviewer ratings for each recommended application and multiplying each result by 100. After the Scientific Review Panel meeting, the executive secretary prepares a summary statement of each application by combining written comments, minority critiques, and general Scientific Review Panel discussion notes. These statements become the official documents describing the deliberations of the panel. Observations, comments, and recommendations are incorporated into After Action Reports after each Scientific Review Panel. These reports are intended to provide a basis for the maturing of the process of review. Generally, findings reported to the committee indicate that the critiques provided by assigned reviewers were through and complete; panel discussions were full, unbiased, and uninhibited; and the Scientific Review Panel had the requisite expertise.

OCR for page 51
--> The committee recommends that the TSNR Program develop comprehensive mechanisms for research training. This recommendation presumes continued support by the military Nurse Corps for doctoral education and the addition of post-doctoral fellowships for research training. Suggested mechanisms include the following. Mechanisms for formal and informal mentoring of junior researchers. Several of the grant award categories recommended in Chapter 5 would foster mentoring, but additional mechanisms should be pursued as well. Workshops on scientific writing. This focus would help military nurse researchers to publish their work in peer-reviewed journals and could also be beneficial in the preparation of future proposals. Addition of the solicitation of advice and criticism for proposals and the use of mock reviews during the grants-writing workshops. Mock reviews are used widely to assist investigators in improving the quality of their proposals. They are critical analyses patterned after the National Institutes of Health review process but provided by local colleagues and other experts from inside and outside the military. Reviewers serving as knowledgeable skeptics offer commentary, support, coaching, and encouragement. There is precedent in this approach in that some schools of nursing use mock reviews for all extramural research applications. Mock reviews by colleagues help improve research design and interpretation and thus enhance the researcher's ability to develop a high-quality proposal. Workshops in research methods. The purpose of these workshops would be to help refine the various research skills of junior nurse researchers. Program Monitoring and Evaluation Adequate and useful evaluation of programs allows those administering them to know the status of all activities at any point in time and to forecast the program's accomplishments and needs over reasonable time periods. Such capability requires systematic assembly of data and information concerning four aspects of the program: input, process, outcome, and impact. As is common with many new programs, systems for monitoring and evaluating the TSNR Program appear to be in the early stages of development.

OCR for page 51
--> Input Evaluation Input evaluation requires knowledge of the monetary costs of administering various aspects of the program, personnel time commitments, space and equipment costs, time taken from other activities, etc. Detailed information of this type was not provided to the committee for review. Process Evaluation According to information provided to the committee, the TSNR Program has a number of mechanisms in place for evaluating the processes involved in program implementation. Peer Review Process Evaluation As described earlier in the discussion of scientific peer review, After Action Reports are written to describe the functioning of the panel and make recommendations to improve the process. Records indicate that many of the recommendations from 1994 and earlier were implemented in 1995. Additional recommendations were made after the 1995 review, but procedures for the 1996 Scientific Review Panel have not yet been announced. Relevance of Past Grant Awards to Priority Areas Beginning in FY 1995, the TSNR Program prepared a summary of the number of funded proposals in each of the stated research priorities for that year; the dollar amounts allocated to each priority area were not identified. Chapter 3 presents the committee's examination of this aspect of the program. Relevance of Past Grant Awards to Military Nursing Statements of the relevance of the research to military nursing have been required in awardees' final reports and are now being required in the application. Statements from the final reports appear in research monographs that summarize the portfolio of completed projects; however, the committee received no other materials that summarize the results. Review of the statements revealed that the following types of relevance to military nursing were cited frequently by the grantees: increased mission readiness and deployability; improved job satisfaction, reduced turnover;

OCR for page 51
--> improved productivity; reduced manpower costs and health care costs; improved patient outcomes; reduced stress for active-duty personnel resulting from improved care of their dependents; and reduced loss of time from work resulting from pregnancy or health problems. Documentation was not available to describe the production and dissemination of such benefits. Distribution of Funds In FY 1992–1995, $9.7 million (88 percent) of the $11 million appropriated for the TSNR Program was distributed to applicants. In 1995, approximately 80 percent of funds were awarded to applicants. The remainder was allocated for conduct of the scientific review panels, grant-writing workshops, and associated travel; subcontracted administrative support (see Box 4-1); this Institute of Medicine study; equipment and supplies; and civilian personnel (<1 percent). Tables provided by the program suggest that close attention was paid to distribution of awards by service and component. Slightly revised presentations of the available data on the distribution of awards and funds appear in Tables 4-2 and 4-3, respectively. The committee tabulated the following descriptive statistics about funding levels for the 1-year grants: For the 77 grants funded, the funding level ranged from $3,680 to $750,000. The mean level of funding was $125,654; the median was $74,232. The funding level for 23 of the 77 awards was less than $50,000. The funding level for 21 of the 77 awards was greater than the $125,000 mean. The committee notes that the 1996 Request for Proposals gives upper limits on funding for experienced investigators as well as for new investigators, which would be expected to reduce the range of the funding levels.

OCR for page 51
--> TABLE 4-2 Number of Proposals Funded by and Submitted to the TriService Nursing Research Program, FY 1992–1995, by Service and Component     Number     1992 1993 1994 1995 Total Service Component F S F S F S F S F S Army Active 4 19 11 20 13 19 9 36 37 94   Reserve 1 9 1 3 5 6 3 15 10 33   Guard 0 1 1 2 1 2 0 0 2 5   Subtotal 5 29 13 25 19 27 12 51 49 132 Navy Active 2 15 3 8 3 8 2 4 10 35   Reserve 0 2 1 8 0 2 5 11 6 23   Subtotal 2 17 4 16 3 10 7 15 16 58 Air Force Active 0 14 3 12 0 0 1 6 4 32   Reserve 1 6 2 6 2 3 3 6 8 21   Guard 0 0 0 0 0 0 0 0 0 0   Subtotal 1 20 5 18 2 3 4 12 12 53   Total 8 66 22 59 24 40 23 78 77 243 Percentage funded 12   37   60   29   32   NOTE: F = funded; S = submitted. TABLE 4-3 Proposals Submitted and Awarded, and Funding Requested and Awarded, for the TriService Nursing Research Program, FY 1992–1995   Proposals   Funds ($ million) Year Submitted Awarded Requested Available Awarded 1992 66 8 4.49 1.0 0.97 1993 59 22 4.17 2.0 1.70 1994 40 24 4.86 3.0 3.07a 1995 78 23 13.71 5.0 3.93 Total 243 77 27.23 11.0 9.67 a Funds had been carried over from FY 1993.

OCR for page 51
--> Academic Preparation of Awardees The committee examined the academic preparation of the grant recipients across the 4 years of funding (Table 4-4). More than half of the successful applicants were prepared at the master's level only. Although available data were not conclusive, it appeared that a minimum of 9 and up to 11 of these master's-prepared grantees were enrolled in doctoral programs. One of the three awardees with only baccalaureate-level preparation was a predoctoral candidate, and the other two were students, but their course of study was not clearly identified. A majority of the awardees who were students were active-duty service members. Reporting Requirements To monitor the progress of research, the TSNR Program has instituted a number of reporting requirements, which include the following: The recipient must report any proposed changes in the principal investigator or continuation of the project in excess of 3 months without the participation of the principal investigator. A progress report must be submitted within 6 months of receipt of award contract. The progress report must include a summary of scientific and technical progress; a copy of the original budget; and information on disbursements, obligations, and commitments relative to personnel, supplies, and equipment. TABLE 4-4 Academic Achievement of Grant Recipients in the TriService Nursing Research Program, FY 1992–1995a   Doctoral Degree Master's Degree Bachelor's/Other Year Ph.D. D.N.Sc. Ed.D. M.S.N. M.S. M.A. M.B.A. B.S. Unspecified 1992 1 2 0 3 1 0 0 0 1 1993 7 2 0 12 0 0 0 1 0 1994 8 2 1 10 0 1 0 2 0 1995 6 4 1 9 0 0 1 1 1 Total 22 10 2 34 1 1 1 4 2 a Table indicates the highest degree level achieved by the grant recipients at the time of application.

OCR for page 51
--> The final report must be submitted within 90 days of contract expiration. The final report must include a summary of scientific and technical results; any publications or presentations resulting from the grant; specific aims accomplished and their significance to military nursing; a financial status report (form SF 269); a Grantee's Release Form; and a Grantee's Assignment of Refunds, Rebates, and Credits Form. Institute of Medicine staff had access to selected information in TSNR Program files at USUHS. Using standardized data collection forms, project staff abstracted data from files of 76 funded applicants (one was missing at the time of data collection). Data abstraction was conducted to obtain documented evidence about transfers of the principal investigator, requests for no-cost extensions, completed studies, self-reports of publications and presentations, and other matters relevant to process and outcome evaluation. Data were compiled, and results covering transfers, no-cost extensions, interim reports, and study completion for FY 1992–1994 are summarized in Table 4-5. Data in Table 4-5 clearly indicate the common use of no-cost extensions by grantees. The grantees' files contain documentation of the need for additional time to complete projects. In at least one instance, the study was interrupted while the principal investigator was deployed to another country. Transfers and deployment are factors that may hamper study completion by military nurse TABLE 4-5 Summary of Key Parameters of Grant Recipients and Their Studies for the TriService Nursing Research Groupa   PIb Transferred During Study No-Cost Extension Interim Report Filed Study Completed   Year Yes No Yes No Yes No Yesc Nod Unknowne 1992 0 8 1 7 8 0 8 0 0 1993 3 18 17 5 11 11 6 5 11 1994 3 21 25 9 19 5 1 13 10 a At the time of this writing, 1995 data are not yet available. The total number of grant recipients for FY 1992–1994 was 54. b PI = principal investigator. c Verifiable completion of the grant agreement, including submission of a final report. d All 18 in this category have been granted extensions and are pending at the time of this writing. e The termination date for theses studies has passed, but there is no record of closure.

OCR for page 51
--> researchers. Principal investigators responded to transfers by making arrangements for continuation of the projects by a different principal investigator or, if feasible, requesting no-cost extensions to allow completion of the project when time could be made available. The committee concluded that TSNR Program records showed evidence of a need for additional monitoring. Interim reports were missing for 30 percent of the completed studies although reminder letters had been sent. The available documentation made it impossible to determine whether or not 21 of the studies had been completed even though their termination dates had passed. The newly subcontracted services (see Box 4-1) appear to be intended to improve program monitoring. Outcome Evaluation Publications and Presentations Listings of publications and presentations are included in monographs compiled by staff of the TSNR Program. Although the results of many studies have been reported in poster sessions and research symposia, program files provided the title of only one journal publication that appears to have resulted from TSNR Program grants. However, because of the need for no-cost extensions, few studies had been completed before the end of FY 1994, and several were said to be in preparation or submitted. A small amount of anecdotal evidence of outcomes was provided by a randomly selected group of nine grantees, only three of whom had completed their studies before 1995 (see Appendix A). Value of Past Efforts in Mentoring New Investigators Since the only information available about the mentoring of new investigators was provided in anecdotal form by grantees and by the chief of the Army Nurse Corps, the committee concludes that additional efforts are needed to develop relationships between new investigators and experienced investigators in nursing research and related disciplines. Impact Evaluation No impact evaluation has been conducted. This part of the evaluation process must evolve as part of the overall implementation of a comprehensive evaluation plan.

OCR for page 51
--> Conclusions and Recommendations Concerning Program Monitoring and Evaluation The committee recognizes that the TSNR Program has implemented several forms of process evaluation, the most highly developed of which is the After Action Report that follows the Scientific Review Panel. However, it is clear that a comprehensive evaluation plan has not yet been implemented. The committee recommends the early development and implementation of a plan for ongoing monitoring and evaluation of the TriService Nursing Research Program. Recommended basic elements of such a plan are presented in Chapter 5. Data files should be established to facilitate analysis and reporting. In addition, the committee strongly urges the TSNR Group to monitor and report on the progress and outcomes of the program within 3 years and at regular intervals thereafter, as documented by systematic criteria that include evidence of peer-reviewed research publications and applications to practice. Summary The committee recognizes that substantial progress has been made in managing the TSNR Program under very tight deadlines with limited resources. This chapter describes the administration of the program, the dissemination and evolution of Requests for Proposals, the grant review process, research training offered by the program, and program monitoring and evaluation. It also includes recommendations for improving scientific peer review, research training, and program evaluation and monitoring. In addition, this chapter provides the basis for the following recommendations. Appoint a full-time director for the TSNR Program, preferably a doctorally prepared military nurse researcher, to be housed at USUHS with adequate support staff. To be done properly, research program administration requires the commitment of time and resources, scientific oversight, and the creation of partnerships that foster an environment that is conducive to maintaining and nurturing productive interactions among colleagues and the use of information and communication technologies. Ideally, the director would continue conducting research on a part-time basis. Establish a consistent date for grant submission. The committee believes that military nursing research would be best served if the TSNR

OCR for page 51
--> Program were treated as an ongoing program. It applauds early announcement of the possible availability of TSNR Program grant funds in FY 1996. It is easier to postpone starting dates than to postpone dates for submission and review of proposals. Continue to enhance the distribution and timeliness of information about the program, the availability of program money for grants, and the schedule of grants-writing workshops or other research training mechanisms. The use of electronic means for more widespread distribution should be considered, along with the preparation of notices for selected nursing and other journals. Explore ways for the three services to standardize, to the fullest extent possible, the Institutional Review Board process across military installations to facilitate multisite nursing research and proposal submission. With regard to the delayed completion of many of the funded studies, the committee makes the following observations: 1.   Negotiated assignments for active-duty funded researchers would be a positive step toward supporting a productive military nursing research effort. With some exceptions, the work of nursing research in the military is done in conjunction with other job responsibilities. It appears that the commitment of military nurses to research has been a key factor in conducting research while also fulfilling other demanding job expectations. Some grantees stated that they had been able to negotiate time reserved for research activities. Some have used grant money to support a staff that could complete the study under the grantee's supervision. The chief of the Army Nurse Corps states that Army nurse investigators typically have approximately a 10 percent release time from their official jobs to conduct research (B.H. Simmons, Army Nurse Corps, Falls Church, Va., memorandum dated February 15, 1996). Stabilization of assignments is not a new issue in the military, and protocols have been developed to address this matter. Therefore, the needs of specific researchers should be addressed under such protocols. 2.   A firmly established multiyear funding base would encourage and enhance focused research and development efforts that transcend any one corps. Such a base is essential for effective forward planning, efficient operation, and highest return on DOD investment. The following chapter presents the committee's conclusions and recommendations.

OCR for page 51
--> References Henry M. Jackson Foundation. 1996. Homepage at http://www.hjf.org/, 24 April. USUHS (Uniformed Services University of the Health Sciences). 1996. USUHS Mission Statement at http://www.usuhs.mil/, 8 April.

OCR for page 51
This page in the original is blank.