National Academies Press: OpenBook

Strengthening the National Institute of Justice (2010)

Chapter: Appendix B: Survey of Researchers and Practitioners

« Previous: Appendix A: Unavailable or Incomplete Information Requested by the Committee
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×

Appendix B
Survey of Researchers and Practitioners

The Committee to Assess the Research Program of the National Institute of Justice (NIJ) asked HCM Marketing Research to conduct a survey in order to learn about the views of criminal justice researchers and practitioners. The committee wanted to know how familiar they were with NIJ’s activities and what they thought about the quality and impact of these activities. The committee was also interested in overall perceptions of NIJ as an independent science agency.

METHODOLOGY

Between November 20 and December 4, 2008, a total of 509 self-administered questionnaires were completed using an online survey technique. There were 347 questionnaires completed by researchers and 162 completed by practitioners. The sample, provided by the committee, included 2,603 e-mail addresses supplied by professional organizations.

The target researcher sample consisted of members of the American Society of Criminology. The target practitioner sample consisted of leaders and key staff in well-known organizations with an interest in criminal justice issues: the American Academy of Forensic Scientists, the American Correctional Association, the American Probation and Parole Association, the International Association of Chiefs of Police, the Justice Research Statistics Association, the National Association of Drug Court Professionals, the National Association of Sentencing Commissions, the National Center for State Courts, the National Criminal Justice Association, the National District Attorneys Association, and the Police Executive Research Forum. The

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×

sample pool was such that a small percentage of “practitioner” respondents (14 percent) would have been considered researchers if it had been possible to separate them out. They were affiliated with the target practitioner organizations but work as researchers in a government agency.

Quotas for the study were set proportional to the number of researchers and practitioners provided in the sample. A quota of 500 completed interviews had been set in advance, but a light excess was due to multiple respondents completing the survey at the same time. An initial email invitation was sent on November 19, 2008, and one reminder email was sent during the course of the data collection period. Excluding 212 email addresses that bounced back as undeliverable, the overall response rate for the Internet survey is 21 percent, with nearly equal response among researchers (21 percent) and practitioners (22 percent). In order to qualify for the survey, respondents had to have some level of familiarity with NIJ. Only 13 respondents did not qualify and were terminated from continuing the survey. In addition, 73 respondents dropped out of the survey midway; had these surveys been completed, the response rate would have been 23 percent.

Data from the online interviews were captured using Kinesis interviewing software. The data were cleaned and responses to open-ended questions were coded and classified by similar responses. Cross-tabulations of the data were prepared showing replies to all questions for the entire sample as well as by multiple subgroups, including researchers, practitioners, years in field, current position, type of contact with NIJ, years of association with NIJ, field of training, respondent age, and gender. Since the cross-tabulations contain more information than can be readily assimilated, it should be regarded as the database for the study, of which this appendix is a summary.

When a question was answered by a small number of respondents (30 or less), results are not as statistically reliable as those answered by larger groups, and results should be viewed with caution or for directional purposes only.

RESULTS

Familiarity with NIJ

NIJ is well known to criminal justice researchers and practitioners. Most of those interviewed (70 percent) said that they are very familiar with NIJ; only 13 of the potential respondents were dropped because they were unfamiliar with NIJ. This high level of familiarity underscores the importance of NIJ to the field and lends credibility to the survey findings.

Familiarity with NIJ was higher among researchers than practitioners:

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×

74 percent of the researchers said they were very familiar with NIJ compared with 61 percent of the practitioners. Many researchers actively use NIJ information resources, cite NIJ research findings in their work, and participate in NIJ’s grant process.

  • Nearly all researchers (98 percent) have made use of or have cited NIJ-sponsored research in their own work, including 87 percent who have done so three or more times.

  • Among available sources of research funding, researchers most frequently named state or local governments (43 percent), followed by NIJ (31 percent), as their source of funding within the past 5 years.

The types of contact with NIJ varied among respondents. A majority (75 percent) have used NIJ’s products and services. Over half said they had attended a NIJ conference or workshop or applied for a research grant (53 percent each). Roughly two in five have applied for and received a grant (43 percent). A third (33 percent) have been a peer reviewer, and over a quarter (26 percent) have been a participant in an advisory group. Only 8 percent said they have not been associated with NIJ in any of these types of roles. Of the 92 percent of respondents who have been associated with NIJ in some type of role, nearly all (91 percent) reported contact with NIJ since 2001.

Image and Perceptions of NIJ

Respondents believe that NIJ is an important government agency dedicated to the funding and dissemination of research on crime control and criminal justice issues, and many feel the need for NIJ has become more critical over the past decade. Furthermore, they believe that NIJ has impacted the policy and practices in many areas of criminal justice.

  • Nearly all (99 percent) of respondents believe it is important to have a government agency, such as NIJ, dedicated to funding and disseminating research on crime control and criminal justice issues.

  • More researchers than practitioners believe such an agency is very important (94 versus 85 percent).

  • Two-thirds (67 percent) of respondents feel the need for NIJ has become more critical over the past 10 years. Nearly a quarter (23 percent) feel the need has remained the same, and 5 percent feel the need has become less critical over the past 10 years. Another 5 percent were unsure.

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×

While deemed important and influential, some respondents questioned whether the agency has the necessary independence required to operate objectively, consistently, and fairly to best meet field needs.

  • Respondents were equally likely to say that NIJ does not have the independence necessary to be a research agency as to say that it does (37 percent doesn’t have independence versus 36 percent has independence). Over a quarter are not sure if NIJ does or not (27 percent).

    • More practitioners than researchers believe NIJ has the necessary independence (44 versus 32 percent).

  • Areas in which researchers believe political considerations have impacted NIJ are setting research priorities (85 percent), selecting proposals for funding (73 percent), and disseminating research findings (60 percent). Open-ended comments indicated that respondents believe that political considerations, whether external or internal to the agency, are too influential in establishing research priorities and in selecting proposals for funding.

  • NIJ commitment to fairness and openness in disseminating research findings was rated positively by 65 percent of the researchers, but only half as many researchers gave positive ratings to NIJ commitment to fairness and openness in establishing research priorities (37 percent) and selecting proposals for funding (36 percent).

One measure of NIJ influence is the percentage of respondents saying that NIJ has an impact on policies and practices. Ratings of NIJ influence range from moderate to low for each of 15 areas listed in the questionnaire, as described below.

  • Areas in NIJ seen as having the greatest impact are crime mapping (75 percent), law enforcement (66 percent), forensics and investigative science (66 percent), forensic laboratory enhancement (62 percent), and program evaluation (60 percent).

  • Other areas of impact reported by more that half of the respondents include technology research and development (59 percent), courts (59 percent), violence against women and family violence (59 percent), corrections (59 percent), crime prevention (53 percent), victims (53 percent), juvenile justice (51 percent), and drugs and crime (51 percent).

  • Over a quarter (28 percent) indicated that NIJ has impacted policies and practices in areas other than these 15.

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×

Performance on Key Measures

Overall satisfaction is a key measure of how respondents feel about their experience with NIJ. Satisfaction was rated on a five-point scale from very positive to very negative with the midpoint of three indicating neither satisfied nor dissatisfied (answer wording varied by question). These scores were reclassified to indicate the percentage with positive or very positive satisfaction ratings. Based on other studies of performance using similar measures, scores in the 90 percent or higher range are considered outstanding. Organizations with scores at this level are usually growing and have a high level of retention. Scores in the 70 to 80 percent range indicate some good points and some areas for improvement. Scores below 70 percent are indicative of more serious problems.

The results reveal opportunities for improvements to increase satisfaction with NIJ. Fewer than 60 percent of all respondents (57 percent) rated NIJ’s overall performance positively. This includes 19 percent who rated NIJ overall performance as excellent. About a quarter (23 percent) gave a neutral rating and 20 percent gave a low rating, including 7 percent who rated it poor. Practitioners are more satisfied than researchers with NIJ’s performance (69 versus 50 percent).

Respondents were asked to rate five key aspects on their importance to NIJ performance. Staff qualifications, adequacy of resources, and consultation with the researcher community were rated as important by over 90 percent of the respondents. A large majority also rated leadership (89 percent) and consultation with the practitioner community (87 percent) as important.

However, respondent satisfaction with NIJ performance for the same five key aspects is low. Over half of respondents are satisfied with the qualifications of the staff (57 percent) and consultation with the researcher and practitioner communities (51 percent each). And 40 percent are satisfied with NIJ leadership and 27 percent with adequate resources. Researchers are less satisfied than practitioners in several areas: qualifications of the staff (53 versus 64 percent), NIJ leadership (33 versus 56 percent), adequate resources (22 versus 37 percent), and consultation with the researcher community (47 versus 60 percent).

The open-ended question elicited comments from over a third (34 percent) of the 509 respondents. Areas of concern described in the mostly negative comments include inappropriate political influence on NIJ (6 percent), lack of continued research funding (4 percent), the need for NIJ to operate independently (3 percent), the desire for NIJ to develop an unbiased grant process (3 percent), and an interest in diversifying the research to include topics other than DNA, technology, and terrorism (3 percent).

Separate questions for researchers and practitioners further probed for

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×

opinions on NIJ performance related to their areas of interest, as described below.

Researcher Satisfaction with Key Measures

Researchers reported low to moderate levels of satisfaction with NIJ performance.

  • Dissemination of findings to the research community was rated positively by nearly two-thirds (64 percent) of researchers, while over half (56 percent) are satisfied with the dissemination of research findings to policy makers and practitioners.

  • Levels of researchers’ satisfaction are lower in funding high-quality research (44 percent), establishing research priorities that are policy relevant (44 percent), encouraging top flight researchers to apply for funding (37 percent), communicating research priorities to the field (37 percent), developing affordable and effective tools and technologies (36 percent), establishing research priorities that are coherent, important, and cumulative (33 percent), and developing and financially supporting future researchers (25 percent).

Recipients of NIJ grants (44 percent of the researchers interviewed) expressed low levels of satisfaction with the grant process and project monitoring, with the exception of some moderate levels of satisfaction with NIJ staff.

  • Just over half (53 percent) of grantees are satisfied with NIJ’s monitoring of research activities.

  • Less than half are satisfied with the review of research products (46 percent), ease of applying (44 percent), dissemination of research products (43 percent), and the quality of feedback/reviews (41 percent).

  • Fewer grantees are satisfied with the quality of funding decisions (38 percent) and transparency of the award process (28 percent).

  • Levels of researchers’ satisfaction are moderate for the staff’s responsiveness (70 percent), fairness (69 percent), and competence (66 percent) but lower for the staff’s scientific knowledge (56 percent).

Unsuccessful grant applicants (56 percent of researchers interviewed who had ever been denied a grant after applying) rated NIJ somewhat lower than grantees. The percentage satisfied with NIJ was quite low in the areas of ease of applying (44 percent), quality of feedback/reviews (33 percent),

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×

quality of funding decisions (24 percent), and transparency of the award process (22 percent).

Over a third (35 percent) of researchers had never applied to NIJ for a grant for a variety of reasons. The most frequently given reasons were that they thought it was unlikely that they would get funding (55 percent), research opportunities were not in their field of expertise or interest (45 percent), and they had insufficient notice to prepare an application (24 percent).

Practitioner Satisfaction with Key Measures

Practitioner satisfaction with NIJ performance in four key areas is moderate:

  1. Dissemination of relevant research knowledge to practitioners and policy makers (72 percent).

  2. Excellence and integrity in the conduct of NIJ activities (68 percent).

  3. Identification of research and technology needs of criminal justice agencies and practitioners (66 percent).

  4. Commitment to fairness and openness in practices (61 percent).

Practitioner satisfaction with NIJ performance for six other areas is low:

  1. Development of developing affordable and effective tools and technologies (55 percent).

  2. Improvement of forensic laboratories (53 percent).

  3. Technical assistance (51 percent).

  4. Testing of existing and new technologies (51 percent).

  5. Development of equipment standards (48 percent).

  6. Training of new scientists (28 percent).

NIJ Information Resources

NIJ supports a number of activities designed to communicate information on criminal justice research to researchers and practitioners. NIJ data resources are used more widely by researchers, while NIJ sponsored workshops and conferences are attended more widely by practitioners. Many of those who use these resources find them to be useful.

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
  • The NIJ website has been used by nearly all (98 percent) and is useful to three-quarters of them (73 percent).

  • The National Criminal Justice Reference Service has been accessed by 90 percent of the researchers and is useful to most (89 percent) of them.

  • The University of Michigan data archive has been used by 75 percent of the researchers and 90 percent of them find it useful.

  • NIJ-sponsored national conferences have been attended by about half of the researchers (51 percent) and about two-thirds (62 percent) of attendees found them useful.

  • NIJ-sponsored workshops have been attended by more practitioners (65 percent) than researchers (43 percent), and more practitioner participants (83 percent) than researcher participants (60 percent) found them useful.

  • NIJ Criminal Justice Research and Evaluation Annual Conferences have been attended by more than a third (37 percent) of the practitioners, and over three-quarters (77 percent) of attendees found the conference useful.

  • NIJ Annual Technology Conferences have been attended by 27 percent of practitioners, and nearly two-thirds (62 percent) of attendees found them useful.

  • The NIJ Critical Incidents Conference was attended by 5 percent of practitioners, and half (50 percent) of the attendees found it useful.

Researcher and Practitioner Profiles and Demographics

The practitioner sample consisted of leaders and key staff in wellknown organizations with an interest in criminal justice issues. Most of these respondents work full-time (89 percent) and are very experienced, with most (70 percent) having more than 20 years in the field. While nearly half (44 percent) were trained in social sciences, a number were trained in law (23 percent) and science and technology (17 percent). Nearly a quarter are currently criminal justice practitioners (24 percent) or government officials (24 percent), and 1 in 7 is a researcher in a government agency (14 percent). Demographically, practitioners are middle-aged with a median age of 52, more likely to be male than female (73 versus 27 percent), and to have an advanced degree (40 percent master’s, 21 percent M.D./J.D./D.D.S., 21 percent doctorate).

The researcher sample consists of members of the American Society of Criminology and includes a broader range of age groups and experience levels. Nearly all researchers work full-time (97 percent) and are quite experienced, with half (47 percent) having more than 20 years in

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×

the field. Nearly all (92 percent) were trained in social sciences, and most (80 percent) currently work as an academic. Compared with practitioners, researchers are younger, with a median age of 48, more likely to be female (37 percent), and more likely to have a doctorate degree (86 percent).

Corrections and program evaluation were the topics most often named by researchers and practitioners as an area of interest. The primary area of interest was corrections and law enforcement for both researchers and practitioners. In addition, a fair number of practitioners focus on forensics and investigative sciences and courts.

Respondent Segment Differences

Differences in satisfaction found in the examination of the detailed tabulations are summarized below.

  • Respondents with longer experience in criminal justice had more different kinds of contact with NIJ and are more satisfied with its performance than respondents who have worked in the field for shorter periods. Research applicants with over 10 years in the field who had ever been denied a grant were significantly less satisfied with the grant process than denied applicants who are less experienced.

  • Opinions and perceptions of NIJ of those who have been in contact with NIJ since 2001 were compared with those who have not been in contact as recently. Both groups rated overall satisfaction with NIJ and the importance of NIJ similarly; however, respondents with recent contact were less satisfied with the leadership and having adequate resources.

  • Compared with researchers with only earlier contact, researchers with recent contact since 2001 are more satisfied with the NIJ grant process, but are less satisfied with some other service aspects of NIJ.

    • Researcher satisfaction with the quality of funding decisions during the grant process and the quality of feedback and reviews if denied a grant is higher among those with recent contact than among those with only earlier contact.

    • Researcher satisfaction with NIJ’s leadership and resources is lower among those with recent contact than among those with only earlier contact.

    • Researcher satisfaction with the coherence, importance, and cumulativeness of research priorities is lower among those with recent contact than among those with only earlier contact.

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
  • Practitioners with recent contact since 2001 appear less satisfied with NIJ than those who have not been in contact as recently.

    • Practitioner satisfaction with excellence and integrity in the conduct of NIJ activities, the dissemination of research findings, and the utility of NIJ-sponsored activities is lower among those with more recent contact with the agency than those with only earlier contact.

  • There are some differences in NIJ ratings between men and women. Female respondents place greater importance on NIJ and feel its impact on policy and practice has been greater in some areas than do men. Ratings for overall performance are similar among men and women, but women are more likely than men to believe the NIJ staff is qualified. Female grantees gave much higher ratings to the grant process than male grantees, but ratings among those denied a grant were similar between the two genders.

Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 263
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 264
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 265
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 266
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 267
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 268
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 269
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 270
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 271
Suggested Citation:"Appendix B: Survey of Researchers and Practitioners." National Research Council. 2010. Strengthening the National Institute of Justice. Washington, DC: The National Academies Press. doi: 10.17226/12929.
×
Page 272
Next: Appendix C: Legislation Impacting NIJ »
Strengthening the National Institute of Justice Get This Book
×
Buy Paperback | $92.00 Buy Ebook | $74.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The National Institute of Justice (NIJ) is the nation's primary resource for advancing scientific research, development, and evaluation on crime and crime control and the administration of justice in the United States. Headed by a presidentially appointed director, it is one of the major units in the Office of Justice Programs (OJP) of the U.S. Department of Justice. Under its authorizing legislation, NIJ awards grants and contracts to a variety of public and private organizations and individuals.

At the request of NIJ, Strengthening the National Institute of Justice assesses the operations and quality of the full range of its programs. These include social science research, science and technology research and development, capacity building, and technology assistance.

The book concludes that a federal research institute such as NIJ is vital to the nation's continuing efforts to control crime and administer justice. No other federal, state, local, or private organization can do what NIJ was created to do. Forty years ago, Congress envisioned a science agency dedicated to building knowledge to support crime prevention and control by developing a wide range of techniques for dealing with individual offenders, identifying injustices and biases in the administration of justice, and supporting more basic and operational research on crime and the criminal justice system and the involvement of the community in crime control efforts. As the embodiment of that vision, NIJ has accomplished a great deal. It has succeeded in developing a body of knowledge on such important topics as hot spots policing, violence against women, the role of firearms and drugs in crime, drug courts, and forensic DNA analysis. It has helped build the crime and justice research infrastructure. It has also widely disseminated the results of its research programs to help guide practice and policy. But its efforts have been severely hampered by a lack of independence, authority, and discretionary resources to carry out its mission.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!