National Academies Press: OpenBook
« Previous: Attachment B Acknowledgment of Reviewers
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

Attachment C

This attachment comprises five parts: C-1, the grantee invitation letter; C-2, an informed consent form; C-3, the grantee questionnaire for the summative evaluation; C-4, the committee review procedures for the summative evaluation; and C-5, the rating sheet for committee members for the quality of outputs.

Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

C-1 GRANTEE INVITATION LETTER

THE NATIONAL ACADEMIES

Advisers to the Nation on Science, Engineering, and Medicine

Division of Behavioral and Social Sciences and Education
Committee on Human-Systems Integration
500 Fifth Street, NW
Washington, DC 20001
Phone: 202 334 2678
Fax:     202 334 2210
Email:  cohsi@nas.edu
www.nationalacademies.org

October 6, 2010

Grantee Address

Dear Dr. _______:

Last week we sent you an email to inform you that you are being invited to participate in the External Evaluation of the National Institute on Disability and Rehabilitation Research (NIDRR) and Its Grantees that is being conducted by an expert committee of the National Research Council of the National Academies. This independent evaluation is being sponsored by NIDRR for the purpose of: (1) assessing NIDRR's priority setting and peer review processes and (2) reviewing the quality of grantee outputs for a sample of grants that represent the NIDRR portfolio.

Your NIDRR-funded grant (Grant #__________, Grant Title:____________________________) was selected to be reviewed as part of the evaluation of grantee outputs. However, your participation is completely voluntary. If you do agree to be part of the evaluation, your participation will involve the following activities.

  1. We will ask you to nominate two outputs that were produced under each project that was funded by the grant. These will be outputs that best reflect your grant’s achievements. We are using the four NIDRR categories of outputs as defined in NIDRR's Annual Performance Report (APR), which include (a) Publications; (b) Tools, Measures, and Intervention Protocols; (c) Technology Products and Devices; and (d) Informational Products. The committee will assess the quality of the outputs which you identify using criteria of technical quality, the extent to which they advance knowledge, their potential impact, and their dissemination.
  2. To conduct the review, we would like to examine the actual outputs and to review any documentation that you may have about the outputs.
  3. We will ask you to complete a questionnaire about each output that asks you to briefly summarize evidence of their technical quality, how they advance knowledge, their potential impact, and how they were disseminated.
  4. The questionnaire will also ask you to respond to a brief set of questions at the grant level about approaches you used in managing your grant, how the grant may have generated new research and projects, and about your perspectives of key NIDRR processes which may influence grant results.
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
  1. We may also ask you to participate in a follow-up telephone or videoconference interview. If so, we would like to audio-record the discussion between you and the committee.

The attachments that follow this letter include, first, an informed consent form that explains what we will do with the results of the evaluation and what steps we will take to protect the confidentiality of the evaluation results specific to your grant. The second attachment is the Grantee Questionnaire referred to above that provides instructions for:

  • identifying outputs to be reviewed,
  • completing the supplemental questions for each output, and
  • sending us your signed consent forms, your outputs, and your completed questionnaire.

Please note that we are sending this package to you in electronic and hard copy form.

We hope that you will decide to participate. If you have any questions please don’t hesitate to contact one of us using the information below.

Sincerely,

Jeanne Rivard, Ph.D., Co-Study Director Mary Ellen O’Connell, Co-Study Director
The National Academies The National Academies
National Research Council National Research Council
500 Fifth Street, NW 500 Fifth Street, NW
Washington, DC 20001 Washington, DC 20001
Phone: 202-334-2967 Phone: 202-334-2607
Email: jrivard@nas.edu Email: moconnell@nas.edu
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

C-2 INFORMED CONSENT FORM
(For NIDRR Grantees)

What the study is about: An expert committee of the National Research Council of the National Academies, in Washington, DC is developing an evaluation framework that will be used to: 1) review NIDRR’s priority setting and peer review processes; and 2) review the quality of grantee “outputs” for a sample of grants that represent the NIDRR portfolio. [“Peer review” refers to a process in which experts review the merits of a grant application in considering whether it should be funded. “Outputs” are publications, measures, intervention protocols, devices, or information resources that are produced as part of a grant.] The committee will also assess the design and implementation of the evaluation process and make recommendations for additional evaluation cycles that may be performed subsequent to this effort.

What we will ask you to do: We would like to invite you to participate in the evaluation.

Your participation will involve:

  1. Having outputs produced under your grant peer reviewed through a quality assessment by an expert panel.
  2. To conduct the review, we would like to examine a copy of the actual outputs and to review any documentation that you may have about the outputs.
  3. We will ask you to complete a questionnaire about the outputs to assist in assessing their technical quality, the extent to which they advance knowledge, their potential impact, and if applicable, their dissemination.
  4. We may ask you to participate in a follow-up telephone or videoconference interview where we would audio-record the discussion between you and the committee.
  5. We will also ask you to respond to a brief set of questions at the grant level about your grant management, the generation of new research and projects, and about key NIDRR processes which may influence grant results.

Taking part is voluntary. Your participation is completely voluntary. You can chose not to answer some of the questions, and there will be no consequences.

Benefits and risks:

Benefits: By taking part in the evaluation you will provide information that may help NIDRR improve its research portfolio for the benefit of persons with disabilities.

Risks: Because NIDRR has funded some of your research and development activities, you may feel uncomfortable having your grant’s outputs formally rated in the study, or in sharing your opinions and perspectives on NIDRR’s key management processes. You might feel that this could be a risk to your future grant funding. We want you to know, however, that we will take every step necessary to protect your confidentiality and minimize this risk.

Your answers will be confidential. In the final evaluation report, we will briefly describe your grant and the outputs that were reviewed, by their titles and grantee institutions. Your name as the Principal Investigator, or other investigators on your team, will not be used. In most cases we expect that you have already placed information about the outputs in the public domain through publications, presentations at conferences, and through NIDRR’s National Rehabilitation Information Center (NARIC) website.

However, distinct from these descriptions will be your responses on the Grantee Questionnaire and the committee’s quality ratings of your outputs. For analysis and reporting, these narrative and quantitative

Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

data will be de-identified and aggregated across all outputs and all grants. A research identification number will be used to track grants and their specific outputs. Outputs may be analyzed by categories, such as output type (e.g., publications, tools, technology, information products); quality criteria assessed (e.g., technical quality, knowledge advancement, potential impact); or program funding type (e.g., center grant, field initiated grant, training grant, etc.). If your grant or your output represents one of a kind and there is a risk of identifying you because of this, your data will be aggregated with another larger group where identification will not be a risk.

Every effort will be made to protect the confidentiality of the information that you provide. The Study Director will keep a list linking the grant and output research ID numbers with that output’s identifying information (institution, grant title). This list, along with the data collected, will be stored securely at the National Research Council, and will be accessible only by the Study personnel. If a telephone or videoconference interview is convened to gather additional follow-up information, the transcription of audio-recorded interviews will be combined in a dataset with the interviews of all of the other respondents, then analyzed for common themes across the interviews. The audiotapes, transcriptions, grantee questionnaires, committee ratings, and other raw data collected will be destroyed at the end of the study when the report is released.

Compensation: There is no compensation for participating in the evaluation.

If you have any questions: The Co-Study Director of the evaluation and contact for questions is Jeanne Rivard, Ph.D. If you have any questions about this consent form or the study, she can be contacted by phone at: 202-334-2697, or by email at: jrivard@nas.edu.

If you have any questions, comments, or concerns about taking part in this study, first talk to Dr. Rivard above. If for any reason you do not want to do this, or you still have concerns after doing so, you may contact the Institutional Review Board (IRB) of the National Academy of Sciences (NAS), which reviewed and approved the study plans and this consent form. You can reach the chair of the IRB by contacting Ronald D. Taylor, Human Protections Administrator, by telephone at 202-334-1659 or you may write to him at the National Academy of Sciences; Room 1026; 500 Fifth Street, NW; Washington, DC 20001.

Statement of Consent: I have read the above information, and have received answers to any questions I asked. I consent to take parting in the study.

Your Signature_______________________________________ Date__-__-__

Your Name (printed) _____________________________________________

In addition to agreeing to participate, if there is a follow-up interview, I also consent to having it tape-recorded.

Your Signature_______________________________________ Date__-__-__

Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

C-3 SUMMATIVE EVALUATION: GRANTEE QUESTIONNAIRE

Grant Award Number:

Grant Title:

Grantee:

Program Mechanism:

Grant End Date:

INSTRUCTIONS TO GRANTEES

This questionnaire has been designed to obtain information to assist the Committee in assessing the quality of your grant’s outputs. NIDRR has provided to the Committee and the National Research Council (NRC) staff copies of its Annual Performance Report (APR) database and your last APR and your final APR. As you will see in certain places on the questionnaire we have inserted information from your APR to facilitate your completion of the questionnaire (e.g., Table 1 lists your research and development projects, and Table 2 lists your outputs reported in the APR). Where this information is in error, we would appreciate your pointing the errors out to us and correcting it; and/or updating the information as needed.

The questionnaire is divided into the following three parts:

Part A. Nominating Outputs for Review. This section asks you to nominate, for the Committee’s review, the “top 2” outputs for each of your projects that best reflect your grant's achievements (Table 1 below). The Committee would prefer to review one publication and one other type of output for each project. However if you only have publications, please nominate these as your “top 2”.

To make this process easier, the NRC staff has populated a list of the outputs (Table 2 below) that were reported for your grant in the APR. You could select the top 2 outputs from this table. However you are not constrained to select from this list if there are other outputs that you think better reflect your grant’s achievements.

For Committee review we are requesting materials and information regarding the actual outputs selected as the top 2 for each project.

  • For publications, the material for review would be pdf copies of each article.
  • For the other outputs, materials for review would include:
    • Electronic or hard copies of the measures, tools, intervention protocols, manuals; or links to websites, pictures or other graphic representations of tools or devices that have been produced.
    • An abstract or summary of each output, which briefly describes:
  • o what the output is,
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
  • o its purpose,
  • o target audience,
  • o methods, and
  • o how the output fits into the overall goals and objectives of the project and grant

Part B. Additional Questions about Outputs. For each of the outputs you nominated for review, the Committee has a series of questions related to their technical quality, how they may have advanced knowledge, their potential impact, and their dissemination. We ask that you complete the Part B section for each output. If the answers to certain questions would be the same across different outputs, you can note this and cut and paste responses from earlier output forms to other ones. Please make your responses brief, but as specific and quantitative as possible.

Part C. Grant-level Questions. The questionnaire will also contain a few other items asking about how you managed your grants to produce the highest quality outputs, how your grant’s results may have generated new projects, and how key NIDRR processes influence results.

Your complete package of materials will contain:

  • Your signed informed consent form
  • Copies of your publications and other outputs (e.g., measures, tools, intervention protocols, manuals, links to websites, pictures or other graphic representations of devices that have been produced)
  • Your completed Grantee Questionnaire

Please send these materials by ___DATE___ to:

Matt McDonough

The National Academies

National Research Council

500 Fifth Street, NW

WS 1134

Washington, DC 20001

We are enclosing an addressed Fedex form that can be used when mailing your package of materials to us. We estimated a shipment cost that would cover a weight up to 10 lbs. (e.g., for large center grants or devices). If your package weighs more than this, Fedex will charge us the correct amount.

If your package is light and you want to send it electronically, you could email it to Matt at mmcdonough@nas.edu. However, you would need to scan your signed consent form, and send that in a pdf document.

Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

Part A. Nominating Outputs for Review

When referring to “outputs”, we are using the four NIDRR categories of outputs as defined in NIDRR's Annual Performance Report, which include: a) Publications; b) Tools, Measures, and Intervention Protocols; c) Technology Products and Devices; and d) Informational Products.

Per the instructions for nominating outputs for review, please record your nominations for your “top 2” outputs for each of your projects in Table 1 below. (Reminder “top 2” refers to those that best reflect your grant's achievements). As you can see the NRC staff has already populated Table 1 with the names of your research and development projects from data in the APR. Table 2, which follows, contains a list of outputs from which you can cut and paste into Table 1 below. Please identify any errors in this information that we have provided from your APR and correct it as needed.

Table 1. Projects and Nominated Outputs

# Names of R & D Projects in Grant Names of Top 2 Outputs for each Project
Outputs to be inserted by grantee from Table 2 below or add others as needed
Research Projects
R1 1.
2.
Development Projects
D1 Dissemination 1.
2.

INSERT GRANTEES' PROJECTS TABLE HERE


The table below lists all of the publications and other outputs that were listed in the APR data provided by NIDRR. Please use this table below in selecting your top 2 outputs for each project. (You can cut and paste from Table 2 into Table 1.) However you are not constrained to select only from this list if there are other outputs that you think better reflect your grant’s achievements.

Table 2. List of Outputs from APR

Type of Output Title of Output
publications (title)
tool (title)
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

Part B. Additional Questions about Outputs

Please use one copy of this form for each publication and each other output for the “top 2” outputs that you selected for each project in Part A above, and provide the following information. Please make your responses brief, but as specific and quantitative as possible. If you consider the criterion not to be applicable to your output, please explain. (Please note that an electronic copy of the questionnaire was included in the email version of this package.)

Name of Output:____________________________________________

B1. Technical Quality of Output

In the space below, please describe examples of the technical quality of your output, such as:

 

  • The particular approach or methodology used in developing your output
  • Relevant peer recognition such as peer reviews or evaluations, peer endorsements, invitations to present at professional forums or conferences, invitations to present testimony, receipt of awards or honors, etc.
  • Receipt of a patent, FDA approval, or use of your output in standards development
  • Evidence of the usability and accessibility of the output

 

 

B2. Advancement of Knowledge

Please use the space below to describe how this output has advanced knowledge. To structure your response, include points such as:

 

  • What the importance of your original question or issue was
  • How the output has advanced knowledge in arenas, such as:

o making discoveries

o providing new information

o establishing theories, measures, and methods

o closing gaps in the knowledge base

o developing new interventions, products, technology, and environmental adaptations

 

 

Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

B3. Potential Impact

In the space below, please briefly describe evidence of your outputs’ potential (or actual) impact on the following audiences, as relevant to your output:

 

  • Science (e.g., new areas of inquiry, methodology, etc.)
  • People with disabilities: health, quality of life, participation
  • Provider practice
  • Health and social systems
  • Social and health policy
  • Private sector/commercialization
  • Capacity building in the field of rehabilitation and disability research and development (e.g., scientists, graduate students, etc.)
  • Other

Include information about how this potential impact was tested, and what the results were.

 

 

B4. Dissemination of Outputs

In the space below please provide evidence of your dissemination efforts for this output. Describe this for publications if you have made any effort beyond those of the sponsor of the publication (journal, book, proceedings, etc.). Please include important aspects of dissemination such as:

 

  • Stage and scope (e.g., local, regional, national) of dissemination
  • Dissemination activities
  • Identification and tailoring of materials for reaching different audience/user types
  • Collaboration with audience/users in identifying content and medium needs/preferences
  • Delivery of information through multiple media types and sources for optimal reach and accessibility
  • Evaluation of your dissemination efforts and impacts

 

 

Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

Part C. Grant-level Questions

Please respond to these final questions for your overall grant, not by each output specifically as in Section B.

C1. In the space below please describe what types of planning, project management, and budgetary processes were used to promote high quality outputs. In your statement consider the following types of questions:

 

  • Which processes were useful and how? How could they be improved?
  • Did you dedicate funds for quality assurance activities?
  • How did you track progress and spending against your original plans for the grant?
  • If grants or projects were jointly funded by NIDRR and other extramural or intramural sources, how did you ensure that NIDRR resources were used exclusively for NIDRR-funded activities?

 

 

 

C2. Have the results of the research and development outputs from this grant, or prior NIDRR grants, been used to inform the development of new grant applications or other kinds of projects?

No ____

Yes____

If yes, please use the space below to briefly describe what new grant applications, other projects, funding opportunities, or collaborations have emerged.

 

 

 

C3. Please share any perspectives you may have about how NIDRR’s key processes (e.g., priority setting, peer review, and/or grants management) influence results, such as successful grants and high quality outputs.

 

 

 

Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

C-4 COMMITTEE MEMBER REVIEW PROCEDURES FOR SUMMATIVE EVALUATION

(Revised 11-08-10)

A. Review Subgroups: Each subgroup that will be reviewing outputs will be composed of five Committee members. For each output one committee member will be assigned as the primary reviewer; the remaining four committee members will be secondary reviewers.

B. Output Rating Procedures:

1. All reviewers will independently rate outputs using the following quality criteria (Dimensions of these criteria are shown on the attached rating sheet.):

  • Technical quality of output
  • Advancement of knowledge or the field (research, practice, or policy)
  • Likely impact
  • Dissemination

The following scale will be used for rating the outputs:

image

2. The rating will be based on review of hard copy and electronic materials (i.e., articles/descriptive information about output and questionnaire responses) prior to the subcommittee meeting.

3. The grantee’s final summary APR, and a list of all outputs reported over the course of the project, is provided for contextual purposes. The APR also will be used to inform an overall, qualitative grant-level assessment.

4. Multiple outputs of one grant will generally be rated independently of each other. However, in some cases outputs may be rated as a pair with one score applied. This could occur when one output is a derivative or different expression of another output, and when the PI responses to criterion questions are basically the same. Examples of these include:

 

  • A manual describing a device (1) and a patent of the device (2)
  • A publication describing how a new technology for assessing a condition can be applied in disability rehabilitation (1) and a description of the technology itself (2)
  • A software application (e.g., map reader for persons with visual impairments) (1) and web-based method for individualizing the software for users (2)

5. The meeting will be structured as follows.

  • The primary reviewer will open discussion of each output by presenting a brief summary of the output and then his/her rationale for rating each relevant criterion (up to four) plus the overall score,
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
  • Secondary reviewers will then present their ratings for each output and a brief rationale.
  • Using the same criteria, the subgroup will then develop consensus group ratings for each output. Discussion will be facilitated by the subgroup chair. If there is a subgroup member with a significantly divergent view, his/her score and rationale will be captured separately.
  • Staff will document discussion points that lead to the consensus group ratings and will record the subgroup’s rationale for each criterion, the overall rating, and the grant performance rating in a brief narrative.
  • At the end of the review of each output, the individual subgroup members’ rating sheets will be gathered.

C. Grant Assessment

Once all outputs of an individual grant are reviewed, the subgroup will consider and rate the grant’s overall performance. The outputs reviewed had been identified by the grant’s Principal Investigator as the “top” two outputs per project, which best reflected the grant’s achievements. Taking into consideration this designation, the consensus group ratings of the entire set of outputs, and the grant’s overall purpose and objectives (using the grant’s APR), the subgroup will assign a grant performance rating using the same 7-point scale. These grant-level ratings and their rationale will also be documented by staff.

Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

C-5 COMMITTEE MEMBER RATING SHEET OF THE QUALITY OF OUTPUTS

Grantee ID: 

Date of Review: 

Output to be Reviewed:

To be completed by NRC staff

Output Title: _______________________________________________________

Research Output: ______ Development Output: ________

Type and Subtype of Output (marked below):

Type of Publication   Type of Tool, Measure, or Intervention Protocol   Type of Technology Product or Device   Type of Informational Product  
1. abstract   1. checklist   1. industry standards/guidelines   1. training manuals/curricula  
2. book   2. survey or interview schedule   2. software or netware   2. fact sheets  
3. book chapter   3. diagnostic or assessment instrument   3. invention   3. newsletters  
4. journal article   4. outcome measure   4. patent, license, or disclosures   4. audiovisual materials  
5. proceedings   5. intervention protocol or program   5. working prototype   5. marketing tools  
6. technical   6. statistical technique   6. product evaluated or field tested   6. educational aids  
7. web journal   7. database   7. product transferred to industry for potential commercialization   7. websites or other internet sites  
8. other   8. other   8. product in marketplace   8. other  
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×

Quality Criteria and Dimensions

For each criterion provide one rating using the scale below:

image

Criteria and Dimensions Score

Technical Quality of Output

  • Applying standards of science and technology
  • Appropriate methodology (quantitative or qualitative design and statistics)
  • Accessibility, usability, etc.

Score Rationale:


 

Advancement of Knowledge or the Field (research, practice, or policy as relevant)

  • Science: Establishment of methods, tools, theory
  • New information
  • Closing an identified gap
  • New technology
  • Innovative or novel

Score Rationale:


Likely or Demonstrated Impact On:

  • Science (impact factor, citations)
  • Consumers (people with disabilities: health, quality of life, participation)
  • Provider practice
  • Health and social system
  • Social and health policy
  • Private sector/commercialization
  • Other

Score Rationale:


Dissemination

  • Identification and tailoring of materials for reaching different audience/user types
  • Collaboration with audience/users in identifying content and medium needs/preferences
  • Delivery of information through multiple media types and sources for optimal reach and accessibility
  • Evaluation of dissemination efforts and impacts
  • Commercialization/patenting of devices, if applicable

Score Rationale:

Overall Score


Score Rationale:

Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 25
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 26
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 27
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 28
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 29
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 30
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 31
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 32
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 33
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 34
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 35
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 36
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 37
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 38
Suggested Citation:"Attachment C." National Research Council. 2011. External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/13203.
×
Page 39
External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees: Letter Report Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The National Institute on Disability and Rehabilitation Research funds applied research and development to improve the lives and functioning of persons with disabilities. At the request of the National Institute on Disability and Rehabilitation Research (NIDRR) within the Office of Special Education and Rehabilitative Services, U.S. Department of Education, the Board on Human-Systems Integration of the National Research Council (NRC) convened a committee to conduct an evaluation of aspects of NIDRR's program. Specifically, the committee was charged to assess the NIDRR priority-setting, peer review, and grant management processes, and to develop an overall framework and evaluation design for the review of grantee outputs for a sample of 30 grantees.

External Evaluation of the National Institute on Disability and Rehabilitation Research and Its Grantees Letter Report discusses the procedures the committee used in its output evaluation, its assessment of those procedures, and recommendations for future evaluations. This report works to improve future evaluation practices and ensure that evaluation results optimally inform NIDRR's efforts to maximize the impact of its research grants. Additionally, it offers conclusions, recommendations, and suggestions on defining evaluation objectives, strengthening the output assessment, and using NIDRR's Annual Performance Reports system to capture data for future evaluations.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!