Review of Disability and Rehabilitation Research NIDRR Grantmaking Processes and Products (2012) / Chapter Skim
Currently Skimming:
Summary
Summary
Pages 1-16
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
NIDRR takes pride in proactively establishing program performance measures focused on the quality of its grantee outputs, developing account 1
|
From page 2... ...
In addition, the committee was charged with assessing the methods it developed for conducting the summative evaluation and making recommendations for the conduct of future evaluations.1 The requested study was the most recent effort in a series of NIDRR-funded activities aimed at assessing and improving the agency's performance. PROCESS EVALUATION NIDRR posed three questions specific to the process evaluation aimed at assessing the process used for priority writing, practices for peer review of grant applications, and the planning and budgetary processes used by grantees.
|
From page 3... ...
Second, priority setting not only reflects the agency's intent to influence the advancement of research in targeted areas but also offers specific funding opportunities for potential grant applications to the agency. The announced priorities should therefore be developed and communicated in a manner that attracts the best researchers to participate in disability and rehabilitation research.
|
From page 4... ...
Strategic Planning NIDRR's multiple stakeholders include persons with disabilities and their families, the scientific community, professional associations, and advocacy organizations representing a variety of disability groups. In the face of this diversity, it is important for the agency to have a consistent mechanism for gathering information and input to inform the strategic planning process beyond the input that will be possible through an advisory council.
|
From page 5... ...
For multiyear grants, the committee recommends that NIDRR establish a long-range operational plan listing projected future grant application submission dates, pending funding availability in that fiscal year. Soliciting Applications Like other federal agencies, NIDRR makes its NIAs available at Grants.
|
From page 6... ...
Enhancements to the Peer Review Process The committee concluded that NIDRR's peer review process is hampered by a limited pool of potential reviewers. NIDRR's competition managers take great care to assemble and facilitate qualified review panels and spend considerable time recruiting and screening potential reviewers.
|
From page 7... ...
In addition, reviewers surveyed by the committee reported that the quality of the training they received was inconsistent; enhancing this training would be a simple and effective way to improve the quality of the review process. Finally, considerable variation exists among competitions in the way NIDRR staff facilitate panel discussions.
|
From page 8... ...
While grantees generally commented that NIDRR's grant management processes were effective in facilitating their own grant management processes, they offered some suggestions for improvement that would help them further. NIDRR staff also offered suggestions for improvement, focused on strengthening their capacity to monitor grants and help grantees stay on course in implementing their grants and meeting performance expectations.
|
From page 9... ...
The committee concluded that research and development outputs are used to generate new projects by grantees to a great extent and lead to substantial numbers of new collaborations with other researchers and organizations, as well as transfers of data, instruments, or models to other projects, and commercialization of technology products. SUMMATIVE EVALUATION The summative evaluation, designed to inform NIDRR's performance monitoring and reporting, involved assessing the quality of outputs pro
|
From page 10... ...
First, the quality of outputs is the product of multiple complex factors that involve the priority-setting process, the funding level, the peer review process, the quality of the proposed science/research and the grantees, and ultimately the quality of the research findings. For grantees that are not performing optimally, NIDRR may conduct ongoing formative reviews with experts to identify strategies for improvement, increase its grant monitoring activities, and require additional grantee reporting.
|
From page 11... ...
For outputs other than publications, NIDRR should establish standards for quality to be achieved and adopt appropriate metrics to assess adherence to these standards. One way of setting the quality bar higher would be to encourage grantees to use standardized reporting forms and checklists5 for reviewing the technical quality of their own work before subjecting it to external review.
|
From page 12... ...
Defining Future Evaluation Objectives The primary focus of the committee's summative evaluation was on assessing the quality of outputs produced by grantees; the evaluation did not include in-depth examination or comparison of the larger contexts of the funding programs, grants, or projects within which the outputs were produced. However, the committee was asked to formulate an overall rating for each grant based on the outputs reviewed and the information available about the grant from the Annual Performance Report (APR)
|
From page 13... ...
Using the four criteria outlined above, the reviewers were able to identify and describe varying levels of quality and the characteristics associated with each. However, the committee's quality criteria were not as easily applied to outputs such as websites, conferences, and interventions; these outputs require more individualized criteria for assessing specialized technical elements, and sometimes more in-depth evaluation methods.
|
From page 14... ...
Reviewer Expertise The committee was directed to assess the quality of four types of prespecified outputs. While the most common output type was publications, NIDRR grants produce a variety of other outputs, including tools and measures, technology devices and standards, and informational products.
|
From page 15... ...
CONCLUSION In summary, the committee concluded that NIDRR grants have produced valuable research, tools, and other outputs for advancing the field of disability and rehabilitation research in line with the agency's mandate. Improvements to NIDRR's priority-setting, peer review, and grant management processes, as well as consideration of alternative evaluation goals and strategies, would further enhance the quality of these processes, their results, and the agency's efforts to improve the lives of individuals with disabilities.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.