5
Building a Research Infrastructure and Guiding Policy and Practice

In addition to funding individual research projects, research agencies typically are also involved in building the infrastructure for research in their areas of interest and enhancing dissemination and utilization of research results. In this chapter we review and assess the efforts of the National Institute of Justice (NIJ) to improve the research infrastructure—efforts that are especially important for the development of crime and justice research—and to enhance the use of research by criminal justice practitioners and policy makers.

In 1968, when NIJ was established, the systems supporting crime and justice research were not well developed, and research efforts were underfunded and driven primarily by the interests of researchers. Few university-based researchers worked in this area, there were few doctoral programs that excelled in research training, and not-for-profit and other private-sector research capabilities devoted to these topics were almost nonexistent. In fact, included in the functions assigned to NIJ was the charge to support the growth of crime and criminal justice research and to take steps to encourage research utilization. The 1977 National Research Council (NRC) committee noted the absence of a strong crime and justice research community as one of the barriers to the success of NIJ (National Research Council, 1977). It also found few systematic efforts to translate research into practice or (where appropriate) to adopt research-based standards for practice and policy.

Building a research field and enhancing research utilization include the following types of efforts: (1) building research capacities, (2) using research to guide policy and practice, and (3) disseminating research findings



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 145
5 Building a Research Infrastructure and Guiding Policy and Practice I n addition to funding individual research projects, research agencies typically are also involved in building the infrastructure for research in their areas of interest and enhancing dissemination and utilization of research results. In this chapter we review and assess the efforts of the National Institute of Justice (NIJ) to improve the research infrastructure— efforts that are especially important for the development of crime and justice research—and to enhance the use of research by criminal justice practitioners and policy makers. In 1968, when NIJ was established, the systems supporting crime and justice research were not well developed, and research efforts were underfunded and driven primarily by the interests of researchers. Few university-based researchers worked in this area, there were few doctoral programs that excelled in research training, and not-for-profit and other private-sector research capabilities devoted to these topics were almost non- existent. In fact, included in the functions assigned to NIJ was the charge to support the growth of crime and criminal justice research and to take steps to encourage research utilization. The 1977 National Research Council (NRC) committee noted the absence of a strong crime and justice research community as one of the barriers to the success of NIJ (National Research Council, 1977). It also found few systematic efforts to translate research into practice or (where appropriate) to adopt research-based standards for practice and policy. Building a research field and enhancing research utilization include the following types of efforts: (1) building research capacities, (2) using re- search to guide policy and practice, and (3) disseminating research findings 

OCR for page 145
 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE to the research and practitioner communities. Throughout its history, NIJ has supported programs aimed at addressing each of these areas. In fact, as we show, in some of these areas NIJ has been a leader at the federal level. The efforts of NIJ described in this chapter include fellowship programs, the data archive and training programs for researchers at the National Ar- chive of Criminal Justice Data (NACJD), the equipment compliance stan- dards program, technology assistance1 provided to practitioners, and the communications office and its dissemination activities. In addition, we look at the function recently assigned to NIJ of administering grant programs to improve crime laboratories and reduce the backlog of DNA processing in these laboratories. We examine these forensic capacity-building grant programs as well as NIJ’s technology assistance activities to determine what, if any, impact they have had on its ability to fulfill its mission as a research agency. We conclude the chapter with a description and appraisal of NIJ’s dissemination efforts to provide information to its state and local constituents as well as the criminal justice research field. buILDINg RESEARCH CAPACITIES The field of crime and justice research has greatly expanded since the creation of NIJ. While this growth can be attributed mostly to changes in higher education, increased public interest in crime and justice, and the growth of the criminal justice system, the federal government, including NIJ, has played an important role. The Omnibus Crime Control and Safe Streets Act of 1968 that established NIJ also enacted the Law Enforcement Education Program, which provided almost $200 million in grants and loans to support the education of law enforcement officers returning to col- lege as well as to the development of new doctoral programs as models for higher education (Jacobs and Magdovitz, 1977). The increased availability of federal funding for programmatic improvements inevitably attracted researchers to assist in program development and evaluations. NIJ’s efforts to provide direction to the growing number of researchers working on crime and justice topics included strategies to reach and appeal to a wide range of disciplines in solicitations for research studies as well as support for researchers through fellowship programs and the creation of a publicly available data archive for criminal justice research. This is most notable in the growth of criminology as a scientific dis- cipline. Prior to 1966 there were 2 doctoral programs in the field; today 1 “Technology assistance” is an elusive phrase intended to indicate support in many forms related to the operation of technology devices, such as radios or lab equipment. The commit- tee develops a definition of technology assistance as it applies to NIJ’s activities and services later in the chapter.

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE there are over 30 (Association of Doctoral Programs in Criminology and Criminal Justice, 2009). Masters programs have increased from a handful to over 1,100. Membership in the American Society of Criminology has grown from under 500 in 1970 (Morris, 1975) to over 3,000 today. In 1968 there was only a single reputable peer-reviewed journal focusing on crime and justice issues; today there are over 100. One commentator on the growth of criminology concluded that the Law Enforcement Assistance Agency (most importantly the Law Enforcement Education Program) was central to the establishment of these programs and the development of the discipline (Akers, 1992). Savelsberg, Cleveland, and King (2004) have docu- mented how federal funding, including support from NIJ, has influenced the growth and direction of criminology as a science. Fellowship Programs The Omnibus Crime Control Act of 1968 calls for the institute to “carry out programs of instructional assistance consisting of research fel- lowships.” From the beginning, it has provided support to doctoral students in their research through the Graduate Research Fellowship Program (Law Enforcement Assistance Administration, 1969) and has since expanded efforts to encourage young minority scholars to develop their research programs through the W.E.B. Du Bois Fellowship Program and to attract senior researchers and practitioners to NIJ through the Visiting Fellows Program. These programs have had as their goals increasing the number of young scholars in crime and justice research, addressing the gross underrep- resentation of minority researchers in this field, and augmenting NIJ staff with short-term senior researchers and practitioners. In order to examine how successful they have been in achieving their goals, the committee had a review conducted of these programs.2 The following is a summary of that review and the committee’s assessment of the data that were gathered. Graduate Research Fellowship Program The Graduate Research Fellowship Program provides dissertation re- search support to outstanding doctoral students undertaking independent research on issues in crime and justice (Crossland, 2008). The goal of the program is to expand the pool of research talent by attracting doctoral stu- 2 The committee notes with appreciation the assistance of Nicola Smith, who prepared a re- port that was helpful in the preparation of this chapter. She contacted current and former NIJ staff and W.E.B. Du Bois fellowship and Graduate Research Fellowship recipients to gather information on the fellowship programs with regard to history and current management, as well as information on recipients’ research achievements.

OCR for page 145
 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE dents who can contribute critical and innovative thinking to pressing crime and justice problems. Students from any relevant academic discipline may apply through their university. NIJ asks dissertation advisers to bring this program to the attention of their most promising students early in their dis- sertation pursuit. There is no predetermined number of awards each year. The monetary value of the award is a fixed amount ($20,000), although some applicants in the past have applied for less. Awardees are expected to complete their research in a 1-year period. Individual award information is maintained through the OJP grant management system and budget files in the Planning, Budget, Management, and Administration Division. The program has been managed by four dif- ferent staff members since 1999. This is not an unusual situation in gov- ernment agencies. However, at no time have there been formal transition procedures from one program manager to another. As a result, the program history is not well documented, since records are missing or incomplete. Fellows are monitored like other NIJ grantees, contacted on a monthly or quarterly basis via phone or e-mail, and required to make progress reports every six months using the categorical assistance progress report (Office of Justice Programs, 2006a). They are also encouraged to present their preliminary and final findings at appropriate academic meetings and conferences. Many fellows present their dissertation work at “job talks” when they enter the job market. No final report is required; the disserta- tion as approved by the grantee’s university is the final report, along with a final progress report. Fellows are invited to present at colloquia at NIJ in special cases. According to NIJ, there was a total of 68 recipients in the period 1998- 2006. The recipients, at the time of their award, represented a variety of disciplines, but most were in criminology or criminal justice (55 percent) or sociology (18 percent) departments. Other disciplines include psychol- ogy, mental health, geography, engineering, political science, social wel- fare, social work, anthropology, economics, and biology. According to our review,3 all but the recent awardees had completed their doctoral degrees and more than half are now in full-time teaching positions. More than half have published4 since receipt of their fellowship. Of those who have 3 Report prepared by Nicola Smith for the committee meeting January 8-9, 2009, on NIJ Fellowship Programs. 4 On the basis of information on 51 of the recipients, 425 publications were authored, which included 287 articles in peer-reviewed journals, 69 technical reports, 61 chapters, and 8 books. And 42 of the journal articles are available in Tier 1 journals and 62 in Tier 2 journals. Less than half (48 percent) of the subsequent publications were deemed unrelated to dissertation topics of the respective authors. Journal publications were classified in tiers based on the clas- sification of the University of Maryland College of Behavioral and Social Sciences. This tier classification is used to rank research publications: typically, Tier 1 (premier), Tier 2 (leading),

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE published, the average number of their publications was six. The strategy employed by the fellowship program, suggesting a list of broad topic areas of research, has had success in generating research publications on a wide range of topics. A list of fellows provided by NIJ for our assessment suggests that ad- equate management and monitoring of the fellowship program were not in place. The original list included individuals as having received funding when in fact they had not. It also included administrative personnel from the universities as recipients of the fellowship but no information regarding the actual recipient. Most awards are for a 1-year period; however, time of completion is not recorded by personnel in charge of the fellowship programs. Ac- cording to our review,5 more than half of the Graduate Research Fellow recipients contacted did not complete their research in 1 year—the range for completion was 1-7 years. This is not surprising for dissertation work. Although there are models for successfully completing and defending one’s dissertation within a year of proposing the research,6 the average criminol- ogy Ph.D. candidate takes 2-4 years to complete the written dissertation in a full-time program and 3-6 years in a part-time program.7 The University of Maryland allows up to 4 years for completion of the dissertation once admitted to candidacy,8 and other Ph.D. programs in criminology have similar requirements. In summary, after making these awards, NIJ does not appear to have spent much effort to see that work was completed in a timely manner, to assess the career impacts of the award, or to continue a relationship with the awardees. The programs, while well intentioned and contributing to the growth of doctoral graduates in the criminal justice field, have not been integrated into the program plan or operations of the agency. Tier 3 (reputable), and Tier 4 (local/others). Tier 1 journals represent the top publications in individual disciplines and for this analysis include the following publications: Criminology, Journal of Research in Crime and Delinquency, and Journal of Quantitatie Criminology. The Tier 2 journals include Criminology and Public Policy, Justice Quarterly, Social Science Quarterly, Law and Social Inquiry, Psicotherma, Journal of Health and Social Behaior, and Aggression and Violent Behaior. 5 Report prepared by Nicola Smith for the committee meeting January 8-9, 2009, on NIJ Fellowship Programs. 6 See for example the Ph.D. timeline at the University of Missouri, St. Louis, available at http://www.umsl.edu/~ccj/pdfs/program_time_line.pdf [accessed July 16, 2009]. 7 See information on pursuing a Ph.D. program in criminology at http://www.topcriminal justicecolleges.com/phd-program-in-criminology.html [accessed July 16, 2009]. 8 See degree program information at http://www.ccjs.umd.edu/Graduate/phd.htm [accessed July 16, 2009].

OCR for page 145
0 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE W.E.B. Du Bois Fellowship Program The W.E.B. Du Bois Fellowship Program was undertaken to address a critical need to build the ranks of minority researchers interested in crime and justice issues. Planning of the program began in 1998-1999, with the goal of providing an opportunity for minority researchers to interact with NIJ staff and to provide their perspective on NIJ programs. Seeking the guidance of the U.S. Department of Justice (DOJ) on how to initiate such a program, NIJ staff were told the department’s view that any federal pro- gram explicitly for minorities would be in violation of federal laws against discrimination and, if a suit arose challenging federal programs that were exclusively for minorities, the department would not support them.9 Despite this setback, plans for the program moved ahead. However, to address the department’s concern, the requirement that all applicants be minority researchers was revised. New guidelines were written in such a way to appeal as much as possible to minority researchers, and race as a selection criterion was dropped. The original guidelines also included a re- quirement that grantees spend at least a portion of their time onsite at NIJ, where they would participate in its various activities.10 In more recent years, terms of residence in the fellowship program have become more flexible. The first W.E.B. Du Bois fellowship was awarded in 2000. The Du Bois fellowship complements NIJ’s other fellowship programs and provides tal- ented researchers early in their professional career with an opportunity to elevate independently generated research and ideas to the level of national discussion (Crossland, 2008). Fellowship awards have the goal of support- ing projects with direct implications for criminal justice policy and practice in the United States (National Institute of Justice, 2007c). The W.E.B. Du Bois fellows are monitored like other grant recipients. Recipients are required to submit progress reports every six months for the duration of the grant. A final report is also required. Du Bois fellows are encouraged to present at professional meetings, but NIJ has neither facili- tated this nor tracked when it happens. W.E.B. Du Bois individual award information is maintained in the grant management system of the Office of Justice Programs (OJP) and budget files in the Planning, Budget, Manage- ment, and Administration Division. Since the beginning of the program there have been four different man- agers. Again like the Graduate Research Fellowship Program, there are no formal procedures in place for transition of the program and the program history has not been well documented. The committee’s assessment of this program is therefore solely depen- 9 Conversations with NIJ Staff, September 9, 2009. 10 Conversations with NIJ Staff, September 9, 2009.

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE dent on what we learned about past recipients of the fellowship awards. There have been a total of nine recipients in the period 2000-2007, and the total amount awarded was $563,078. The awards ranged from $45,343 to $78,767, with an average of $70,385, being awarded per year. Recipients were selected from across the United States, with three selected from the South, two from the Midwest, two from the mid Atlantic, and one from the Pacific. The review of the current resumes of seven of the nine awardees, identified 67 publications11 since completion of their fellowships, approxi- mately half of which were related to their fellowships; 16 of 37 published journal articles are available in Tier 1 and Tier 2 journals.12 All the award recipients were located in established educational institu- tions and committed to the field of criminology. All have stayed in the field, becoming productive scholars and continuing work that began during the fellowship. Although there is variation in the productivity of the fellows, our review of their professional careers suggests that the fellowship stimu- lated their work and resulted in numerous publications. Of course we can- not determine how these individuals would have done without the award, but we note that the Du Bois fellows have contributed to the breadth of crime and justice research in terms of the range of topics related to the goals of the program. In addition, those supported by this program have become productive contributors to the crime and justice research literature. However, NIJ currently does not have a very effective way to monitor and track research produced by fellowship recipients. Without such follow- up, there is no way to assess the extent to which the goal of increasing the number of young scholars, particularly underrepresented minorities, is being met. Visiting Fellows Program The goal of the Visiting Fellows Program, begun in 1969, was to bring criminal justice professionals and researchers to NIJ for a period of 6-18 months to work on research projects or the development of research-based action programs. For the most part, the fellows were established leaders in their field. The award included salary, benefits, reasonable relocation expenses, travel and supplementary costs for their project, and office ex- penses. Fellows were required to spend at least 80 percent of their time at NIJ. Often they were assigned to work with a particular program or organizational unit. Over time, there may have been some deviation from 11 Publicationsinclude articles in peer-reviewed journals, books, chapters, and technical reports. 12 Report prepared by Nicola Smith for the committee meeting January 8-9, 2009, on NIJ Fellowship Programs.

OCR for page 145
 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE the original guidelines, depending on the research tasks undertaken and the program offices involved. The committee could not find any systematic records on the fellows program or the contributions of the fellows. Without better documentation, we cannot determine how many applications were received, how the fel- lows were selected, what role they played at NIJ, or how their final reports were used by the agency. We were verbally informed that the program was discontinued in 2004 because of resource constraints. The agency did not issue a solicitation for the program for fiscal year (FY) 2005-2009. As the writing of this report, the solicitation for NIJ’s Visiting Fellowship Program was released for FY 2010. Our very impressionistic observation is that the program attracted many well-known criminal justice researchers and practitioners, many of the projects undertaken by the fellows have had influence on criminal justice practice and policy, and NIJ staff thought that many of the fellows played important roles in developing the agency’s research plans and pro- grams. The absence of better records on this program left the committee in a position of recognizing the potential value of this type of effort but with no way to assess that value. Data Archiving Program NIJ has been a leader at the federal level in recognizing the importance of collecting, archiving, and making available to other researchers the data collected as a part of the projects it funds. In 1981 NIJ began requiring research award recipients to submit the data collected in their projects. This was in part a response to the recommendation of the 1977 NRC com- mittee that NIJ “collect and maintain a data archive that retains various general criminal justice data . . . collected by individual investigators after the original investigator has completed use of the data” (National Research Council, 1977, p. 113). This recommendation led directly to the establish- ment of the Data Resources Program (DRP). In 1984, NIJ established DRP to ensure that data collected with NIJ funding are permanently preserved and are available for use by others in the research community. NIJ-sponsored researchers are required to submit their data to the DRP at the conclusion of their projects. Since 1991, NIJ data, codebooks, and other supporting documentation are deposited with the National Archive of Criminal Justice Data (NACJD) at the University of Michigan’s Inter-university Consortium for Political and Social Research (ICPSR). Scholars, practitioners, policy makers, students, community activ- ists, and any other interested parties throughout the world can freely use all NIJ-sponsored data collections for the purposes of conducting scientific research. This research can include verifying, refining, or refuting the origi-

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE nal investigator’s findings or pursuing inquiries not addressed by the other investigators by using alternative theoretical and analytical models or by combining with data collected at other sites or time periods. With the support of NIJ, ICPSR is able to acquire and to facilitate the use of a wide variety of data in the fields of criminology, law, and justice. NIJ benefits from a number of important ICPSR roles, including (1) the perpetual preservation of its research investments and products, (2) an infrastructure on the Internet that provides worldwide dissemination of its data and related documentation to the research community, (3) added value by documenting and describing data files that aid the users of data, (4) a team of justice data specialists who have unique skills in data management and computerized mapping technical assistance needed by archive sponsors and data users, and (5) training programs to communicate quality research methods and analysis and promote the use of existing data. Funding for the archive has come from NIJ and the Bureau of Justice Statistics. NIJ has awarded approximately $6 million to ICPSR for the archive since 2001. As with other aspects of NIJ, no systematic review has been conducted of DRP or the data archive at NACJD. In September 2003, NACJD pro- vided a memorandum to NIJ indicating ways the program could be evalu- ated, but no action was taken on any of its recommendations. NACJD, however, regularly monitors submissions to the archive, uses of the archive, and feedback from its training programs. We were able to review the data assembled over the years in a report prepared for us by NACJD.13 A sum- mary of some of the information and analyses is provided in the subsequent sections. Monitored Data Submissions In 1997 web access to the data from NACJD was made available. Based on web logs from 1997 to October 2008, the number of NIJ-sponsored studies in NACJD grew from 256 to 776, a 200 percent increase. In 1997, NIJ-sponsored data comprised 43.3 percent of the studies in NACJD; in Oc- tober 2008, they were 50.4 percent (see Figure 5-1). The database tracking NIJ-funded research was developed in December 1999 and therefore con- tains complete records from the year 2000. According to NACJD records, from January 2000 to October 2008, there was data “activity” (defined as data acquired, released, or preserved only, or designated as “no data to 13 Our review of the data archive program included conversations with current and former staff at NIJ and the University of Michigan, responses to questions posed by the committee to archive staff, a site visit to the archive, and consideration of a report on the archives prepared by Kaye Marz and Christopher Maxwell. The committee notes with appreciation the contribu- tion of the report prepared by Marz and Maxwell made to this section.

OCR for page 145
 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE Number of Studies Year FIguRE 5-1 Total studies submitted to NACJD by year. SOURCE: Report prepared for the committee by Kaye Marz and Christopher Maxwell, Inter-university Consortium for Political and Social Research. archive”) on 395 studies, an average of 43.9 NIJ-sponsored studies per year, ranging from a low of 28 in 2004 to a high of 80 in 2007. During the same period, NACJD released data from 320 studies, an average of 35.6 studies per year, ranging from a low of 16 in 2005 to a high of 63 in 2003; and designated 28 studies as no data to archive or storage only. Data from 47 studies remained to be processed. Note that this number of studies released Figure 5-1 or preserved does not include theR01756 174 studies from the Project on Human Development in Chicago Neighborhoods (PHDCN) released between 2005 uneditable bitmapped image and 2007, for which a dedicated processor was assigned, jointly supported by NIJ and the MacArthur Foundation until March 2007. There have been improvements in the quality of files deposited in the archive. Staff and grantee efforts to raise quality have resulted in larger numbers of data sets in the archive being judged as having higher quality and therefore ease of use, according to NACJD’s data-quality measurement system, put into place in 2005. Even though data sets submitted to the ar- chive have increased in size and complexity, their usefulness has improved as more attention has been paid to data quality. In our survey of researchers and practitioners, 75 percent of the respondents indicated that they had used the archive and 90 percent of those judged it useful to their work.

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE Over the years, NIJ and NACJD have sought to encourage submission of data by NIJ grantees. These efforts include special award conditions, letters to those who had not submitted data indicating the possibility of negative consequences for future funding if they did not submit their data, and new special conditions in 2008 that require awardees to deposit data and data documentation 90 days before award closeout or risk nonpay- ment of grant funds. These efforts combined with technological changes at the archives that make submission easier have resulted in increases in the number of data sets deposited. What is not known is the current extent of data archiving that is occur- ring. One obvious measure is the proportion of completed awards that de- posit complete and usable data in NACJD. This calculation is complicated by the difficulty of determining the number of awards made by NIJ and the number of those awards that produce data sets. Neither NIJ nor NACJD was able to provide us with the total number of awards that should have their data archived. However, records from NACJD as well as our own ef- forts to identify principal investigators in the archive database (see citation analysis in Chapter 6) indicate that a substantial number of projects that should have archived data have not done so—the estimates range from half to upward of 90 percent of all research awards since 1995. Even though we cannot precisely gauge the problem, we think that efforts must be under- taken to improve the archiving of data. An important first step to achieving this will be for NIJ to substantially improve its administrative records. We also urge NIJ to screen former NIJ grantees who are applying for grants as to whether they have complied with data archiving requirements and to withhold support until such requirements are met. Monitored Uses of Data and Online Resources MyData is the ICPSR user registration and authentication system for the NACJD website that collects user affiliation, institutional department, and statistical package preferences. In the years 2006-2008, roughly half of NACJD users were graduate students. Undergraduate students, faculty, and, to a lesser extent, nonacademic researchers constituted the rest of the users. Nearly half of the total users were from criminal justice or sociology departments. The remainder of users represented other disciplines. In 1997, 4,544 data users accessed any file related to an NIJ-sponsored study on the NACJD website, representing 33 percent of the NACJD down- loads for the year. By 2007, the number of similar accesses was 18,274, representing 35 percent of NACJD downloaded studies (see Figure 5-2). NACJD also keeps track of literature that cites any of the data it archives. This bibliography is publicly available online and contains links to the respective data sources and their descriptions (see Chapter 6). Over the

OCR for page 145
 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE In reviewing the documents from the period 1995-2002, one is struck not only by the volume of reports but also by the degree to which various formats were tailored to specific audiences (see Appendix D). With the exception of some of the reports specifically labeled “research reports” and some equipment-related materials, such as guides, test protocols, and testing results, most were written in nontechnical language and empha- sized research findings that had implications for policy and practice as appropriate. After 2002, however, there was a sharp decline in the total number of publications and products. In the NIJ FY 2002 annual report, 20 pub- lications and reports are listed. The numbers have increased only slightly since then, and, through 2007, NIJ produced on average about 30 prod- ucts and publications exclusive of issues of TechBeat, NIJ Journal, and the newly developed Geography and Public Safety Bulletin, a quarterly publication produced jointly by COPS and NIJ. There also appears to be a shift from targeting a readership made up of both practitioners and researchers to one aimed primarily at practitioners and policy makers. In 2002, NIJ abandoned several of its former report formats, including Research Previews (two-page summaries), Research in Action, and Issues and Practices. The currently named Research in Brief, a document of 16-20 pages, is the only publication specifically intended for researchers and scholars as well high-level practitioners and policy makers (National Institute of Justice, 2005) (see Table 5-1). By producing fewer documents for the research community, NIJ appears to have abandoned an important strategy for encouraging interest in criminal justice research and for keeping researchers informed about its research activities. The committee also notes that, in 2003, NIJ discontinued its long- term support for Crime and Justice, A Reiew of Research. Begun in 1978, Crime and Justice was intended to be an annual volume of essays on crime and justice. The rationale behind the series was to bring together the very best thinkers to address a particular topic and purposefully look beyond their own specialty to develop a broader understanding of crime and justice. NIJ provided support for an editorial board, editors, and commissioned papers. The average cost of a volume was approximately $175,000. The staple essays presented a summary of the state-of-the-art with authors also presenting their views on the policy and research implications of what was known at that time. Crime and Justice has continued to be funded, most often with funding from European sources, and its content consists primar- ily of European topics. The committee did not discuss whether funding for such a publication should be reinstated but offers it as an example of a

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE TAbLE 5-1 Types of NIJ Grant Products Type Audience Length Practitioners and their 3,000-5,000 words Research for Practice staff 20-30 manuscript pages 8-16 printed pages Policy makers and their 500-1,000 words Research for Policy staff 4-8 manuscript pages 16-20 printed pages Researchers and scholars; 4,000-6,000 words Research in Brief high-level policy makers, 32-40 manuscript pages practitioners, and their 16-20 printed pages staffs High-level policy makers, 500-5,000 words NIJ Journal practitioners, and their 3-35 manuscript pages staffs 1-5 printed pages Science and Technology Practitioners and policy Varies with content (20-500 Report makers printed pages) Special Report Varies with content Varies with content (20-500 printed pages) Web-only Document Varies with content Varies with content (20-500 printed pages) research tool that was of value to the scholarly community and contributed to NIJ’s stature as a research institute.32 With the shift in the NIJ budget to one in which technology-related research activities receive heavy support, it is not surprising that there has also been a marked shift in the number of reports that focus on technology and, in particular, forensic science. This is most dramatically reflected by the most requested documents compiled annually by the National Criminal Justice Clearinghouse. In FY 2000, of the top 24 most requested documents, 10 dealt with technology topics. For each subsequent year, the number of technology-related reports has increased. In FY 2007, 19 reports dealt with technology issues, 3 reports dealt with social science research, and the 2 remaining reports were issues of NIJ Journal, the semiannual periodical. The shift is also reflected in NIJ’s FY 2006 and FY 2007 annual reports. In the 2006 annual report, only two of its six chapters describe social science research on victims and victimization and improving corrections programs 32 I n 2005 ISI citation analyses showed Crime and Justice had the highest “impact” (fre- quency of citation per article published) of any “criminology, criminal justice, or penology” journal. No later statistics are available because the Crime and Justice data were inadvertently dropped in 2006-2008 but reinstated in 2009 (Tonry, 2009).

OCR for page 145
 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE and practices. The remaining four chapters are devoted to technology ac- tivities. The 2007 annual report devotes even fewer pages to social science research. In its five chapters, only one chapter on courts and corrections describes past and ongoing social science research activities.33 In addition to its own sponsored publications, NIJ also relies on other venues to disseminate useful information to the practitioner and policy maker. An example that was touted by NIJ was the regular appearance of articles authored by NIJ staff or others writing about NIJ research activities in Corrections Today, a monthly publication produced by the American Correctional Association with a monthly distribution of 36,000. The committee reviewed these articles because of criticism from national corrections leaders regarding NIJ’s failure “to do a good job of marketing to constituents” and a failure to disseminate executive summaries of NIJ research as well as other information of use to corrections but not neces- sarily produced by NIJ. Since 2003, NIJ has submitted 33 articles to Corrections Today for inclusion under its “NIJ Update” or NIJ “Technology Update” bylines or as a Corrections Today feature. (The latter publication is produced approxi- mately 4-5 times a year.) Three topics have been the subject of multiple articles: stress reduction, reentry, and federal benefits for disabled offend- ers. Other articles highlight a specific research project (e.g., boot camps); describe several studies on a similar topic (e.g., drug courts); identify new technologies being developed and adapted for corrections use (e.g., duress systems in correctional facilities); and approaches to improving security (e.g., cell phone smuggling); one describes the difficulties in undertaking cost–benefit analyses. Interestingly, one article reports on an unsuccessful program, Project Greenlight, a reentry program in New York, and the pos- sible reasons for its failure. In almost all cases, the articles provide citations and web addresses. The committee notes the usefulness of this approach for disseminating information about NIJ activities but makes no judgment regarding the qual- ity or usefulness of the information itself. Given the criticism we heard and the relatively low satisfaction ratings with NIJ, there do seem to be ques- tions as to whether NIJ is providing topics of interest to the corrections field and whether the current format and content lend themselves to increased familiarity with NIJ and its activities. A systematic assessment might shed some light on these questions. 33 Singled out in this chapter are activities that involve adoption of a mental health screening tool developed by an NIJ-funded researcher, an evaluation of a domestic violence field experi- ment, and an analysis of California’s parole policy.

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE National Criminal Justice Reference Serice NIJ relies heavily on NCJRS, the federally supported information clear- inghouse, to disseminate information about its research program and re- lated activities. It also serves as the archive for all grantee final reports. Initiated in the mid-1970s through a competitive contract, NCJRS now serves as the information clearinghouse for 13 other federal offices and bureaus who contribute funding support. It has approximately 14,600 registered users, who can tailor the information they want to receive.34 Throughout 2008, NIJ sought to attract more users for its information by offering direct subscription from its own website. In 5 months since e-mail alerts were offered, almost 3,000 subscribers have signed on.35 The activities associated with the clearinghouse include a call center, a library and abstract service, a website (see http://www.ncjrs.gov [accessed March 17, 2010]), a warehouse and fulfillment function, conference ex- hibits, and associated management/administration expenses. NCJRS also assists NIJ with editorial and graphics work on their publications. Numerous changes occurred with NCJRS in the late 1990s. A major shift to electronic technologies occurred, and the warehouse was emptied out. After 2003, documents could be downloaded. This had a significant impact, with major changes in the basic operation and a shift in costs. Mail- ing lists needed to be developed, and priorities for other OJP offices (that were contributing to NCJRS) needed to be redefined. This process took a couple of years. In 2002, responsibility for NCJRS was transferred to the Office of Communication in OJP. The NIJ staff member responsible for the contract was moved to that office, but, by 2007 when that person was reassigned, for the first time in NCJRS history, no one with former or current ties to NIJ was responsible for the oversight and management of NCJRS. It is difficult to ascertain whether this change has impacted the quality or efficiency of the NIJ dissemination activities conducted by NCJRS. No records are available to compare the costs or services prior to and after OJP assumed responsibility. Former and current NIJ staff members who were interviewed reported that from time to time there are disagreements with OJP management over NCJRS policies and practices, but no details were offered. One observation was made that OJP centralized management of NCJRS has appeared to impact NIJ in the degree to which NIJ as an organization is visible at various conferences and meetings. Since OJP assumed responsibil- ity for the NCJRS contract, there has been an OJP policy that prohibited an 34 E-mail communication from Lesley C. Flaim, NCJRS, May 22, 2009. 35 E-mail communication from Lee Mockensturm, NIJ, May 22, 2009.

OCR for page 145
0 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE individual agency from displaying its own banner at the NCJRS conference booth if two or more OJP bureaus wanted to have a staff person there. Attendance by multiple OJP bureaus occurs frequently because many of them view attendance there as an effective way to share information about their office’s activities and to communicate with the broader criminal justice community. Also, the bureaus often want to reap the benefits of sharing the costs of the NCJRS booth. Without individual banners, OJP is the focus of attention and is viewed as the developer and provider of the informational materials and activities rather than the individual bureaus, such as NIJ. As a result, NIJ’s visibility as well as those of other OJP bureaus is significantly reduced. NIJ-Sponsored Conferences Conferences constitute a major dissemination strategy. Some of the regular conferences that highlight research sponsored by NIJ include its Annual Conference on Research and Evaluation, the Applied Technology Conference, Technologies for Critical Incident Preparedness Conference (cosponsored with the Department of Defense and the Department of Homeland Security), and a Crime Mapping Conference. In addition, NIJ sponsors electronic seminars (webinars) and symposia. NIJ’s Assessment of Its Dissemination Activities NIJ relies on several methods to assess the quality and utility of its dissemination activities. These include direct feedback from users from a regular 800 number, flyers attached to periodicals, and occasional NCJRS user surveys and focus groups. The most recent survey of NCJRS users by an outside organization was completed by the Gallup Organization in 2000. Gallup conducted a series of three focus groups, drawing on attend- ees at an OJP-sponsored conference on corrections and a national meeting of forensics science directors. The survey of NCJRS users reported a high familiarity with NIJ products and services (95 percent); high satisfaction with publications and videos (89 percent); and a high rating of NIJ as an excellent or very good source of criminal justice information (81 percent). The Gallup report of the focus groups also reported great familiarity with NIJ products and services, a finding not surprising since the focus group members were drawn from the ranks of policy makers attending OJP- and NIJ-sponsored conferences. As might be expected in any survey of us- ers, some respondents reported that the NIJ information was not concise enough and that they were more likely to read and use quick summaries. NIJ has not conducted a similar survey since 2000, instead relying on online surveys distributed randomly to its website users.

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE Other kinds of data compiled and used to assess the use of NCJRS and the quality of NIJ products include statistics on requests for reprints; the use of the website (characteristics of users accessing documents; numbers of documents downloaded; and customer satisfaction index ratings); and citations of NIJ documents as well as anecdotal information and commu- nications from the field through “ASK NIJ.” Finally, the NIJ considers the more than half a dozen awards for content and visual display received in 2007 and 2008 from government and private communication organizations as evidence of the high caliber of its publications. Although NIJ conducts some assessment of its dissemination activities, its efforts are limited and are not designed to improve its targeting of the information or to assess its quality or utility. For example, data being com- piled by NIJ on web usage and the distribution and accessing of documents will not reveal who is using NIJ-produced materials and to what use they are being put. It is this kind of critical look that needs to be undertaken by an outside entity to ensure proper distancing and validity. Our survey of researchers and practitioners (see Appendix B) found very high levels of use of the major dissemination modalities supported by NIJ: 98 percent had used the NIJ website and 73 percent of those found it useful; 90 percent had used NCJRS and 89 percent of those found it use- ful. Practitioners (83 percent) tended to find the NIJ-sponsored workshops more useful than researchers (60 percent). Of the three NIJ conferences, 77 percent of practitioners found the NIJ Research and Evaluation Confer- ence useful; 62 percent found the NIJ annual technology conferences useful; and 50 percent found the NIJ critical incidents conferences useful. However, only 56 percent of respondents thought NIJ did a good job of disseminating information to policy makers, and only 64 percent thought that NIJ did a good job of disseminating information to researchers. CONCLuSIONS The 1977 NRC committee concluded that NIJ was doing little to es- tablish a foundation for crime and justice research and its dissemination efforts were not well developed. That group found little if anything being done by NIJ to attract new researchers to the field, make data available for secondary analysis, provide continuity to research efforts, or promote effec- tive dissemination of research results. Over 30 years later, we have found substantial efforts and accomplishments by NIJ in each of these areas and commend it for them. In the past three decades, NIJ has developed and sustained fellowship programs at three levels: (1) doctoral candidates, (2) young faculty, and (3) visiting fellows. The recipients of NIJ fellowships have made scholarly contributions to the criminal justice literature or to NIJ’s research programs

OCR for page 145
 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE and many have remained in the criminal justice field. NIJ has supported the development and operation of a publicly available criminal justice data archive, which serves to extend research capabilities and opportunities to a broad pool of researchers. NIJ’s DRP and the resulting data archive at the National Archive of Criminal Justice Data are unparalleled and have pro- vided the community with valuable research and information resources. NIJ has carried out its mission to guide policy and practice by pur- suing a broad range of activities that include testing and evaluation and standards, technology assistance, and forensic capacity building. Since their inception, these activities have helped criminal justice agencies make in- formed procurement, deployment, operating, and training decisions and have enabled them to improve and expand their facilities and equipment and to process growing backlogs of DNA samples. The police body ar- mor is the best known of the technologies that NIJ has pioneered and run through its standards process. To its credit, NIJ has continued to refine the standards development process and to expand its testing to include com- parative evaluations. Finally, NIJ has expended considerable efforts and resources to provide useful information and assistance to the field. The agency has developed and supported a series of publications, websites, workshops, and confer- ences aimed at different audiences. Our survey of criminal justice research- ers and practitioners indicated a high level of use of NIJ resources. The committee’s review of NIJ’s efforts to build a research infrastructure strongly suggests that many of its programs have been successful enough that they warrant significant continuation and expansion, although with important modifications as described below. Some functions that have been recently assigned to or taken on by NIJ (e.g., providing direct technology assistance to individual agencies and building forensic laboratory capaci- ties), though successful to some degrees, have been shown to consume its operations. As a result, they detract from the agency’s research mission. In our review, the committee identified the following circumstances that limit NIJ’s ability to fulfill its research mission. • he NIJ budget contains large segments designated by earmark T or congressional mandate for functions that are at best minimally related to research. These mandates include funds for the NLECTC system and associated technology assistance activities and the more recent congressional appropriations for the DNA backlog reduc- tion and crime laboratory improvement programs. Funds for fo- rensic laboratory capacity-building and forensic training activities, such as the Paul Coverdell, DNA Backlog Reduction, Solving Cold Cases, and Forensic DNA Unit Efficiency Improvement programs, swamp the NIJ program and now represent more than half of the

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE OST budget. NIJ bears the burden of administering these funds, a drain on resources that might otherwise be directed toward ad- vancing its research mission. • ome of NIJ’s efforts are directed toward programs that are better S suited for other agencies with clearly defined technical assistance missions. For example, management of the forensic capacity- building programs diverts a considerable fraction of NIJ’s time and resources away from its research mission. Although there is po- tential to incorporate research questions into the capacity-building programs (asking, for example, about the impact of backlog clear- ance on police operations, prosecutions, and crime rates), both the legislative intent and the grantee perception is that these programs are vehicles for the distribution of funds to state and local agen- cies. This situation is unlikely to change, particularly in the current economic climate, as cash-strapped local forensic laboratories can be expected to look increasingly to the federal treasury for relief. Moreover, the administrative burden of these programs is likely to increase as OJP implements the recommendations encouraged by reports of the Office of the Inspector General. These considerations lead us to question whether the DNA backlog reduction and the forensic laboratory capacity-building programs properly belong in NIJ. Relocation of these programs to bureaus more accustomed to managing assistance programs to state and local agencies such as the Bureau of Justice Assistance or the Community Policing Ori- ented Services office would remove a major distraction from NIJ’s ability to focus on its research mission. • IJ’s outreach activities to the research community have declined. N As a federal research agency, NIJ has a duty to transfer knowledge from research in a way that helps its constituencies improve skills and practices. NIJ’s audiences include both the nation’s criminal justice practitioners and its researchers. Historically, NIJ has played an important role in helping build the research field by reaching out to researchers in a wide range of disciplines, communicating useful information, and providing a variety of venues in which researchers can learn from each other. In the past 5 to 10 years, NIJ has reduced its activities and products intended specifically for the research community. Examples of such activities and products include research forums for discussing research in progress or com- pleted research and written and electronic products that present syntheses of research or research reviews across broad topics, as was done in the Crime and Justice series. NIJ does not make public

OCR for page 145
 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE available information on ongoing research, such as the names of principal investigators and abstracts useful for scientific pursuit. NIJ appears to have shifted dramatically the balance of its dissemi- nation activities to an almost exclusive focus on the information needs of practitioners and policy makers. The committee does not intend to imply that the shift in NIJ’s dis- semination strategy conflicts with the goal of creating appropriate and effective mechanisms for translating research into policy and practice. The committee does not see a conflict between a strategy that is aimed at get- ting research findings into the hands of policy makers and practitioners and a strategy aimed at informing and communicating with the research community about ongoing research and the current state of knowledge. NIJ needs to pursue both strategies and its products and services should be tailored to each group. • he practitioner community appears less satisfied. A common T thread running through the presentations and interviews was the valuable contribution NIJ has made to improving criminal justice practices. NIJ was cited as being helpful in providing services for specific activities, such as those in connection with the 2002 Washington, DC, sniper investigation, Hurricane Katrina, and the 2009 presidential inauguration and in supporting crime lab im- provements. But these types of anecdotal comments were offset by numerous criticisms, which the committee also heard from prac- titioners who had a long relationship and were very familiar with NIJ. Some commented that NIJ was no longer the go-to place for information and guidance. Leaders of the largest police and correc- tions agencies said that they had limited contact in the past 5 years with NIJ and that they had not been consulted on important crime- related issues despite their positions and expertise. In our review, the committee identified numerous failings in NIJ’s programs. They are highlighted below. If these failings are addressed, NIJ would be in a better position to document the successes of its programs, to determine if program goals have been achieved, and to set a program agenda that is more appropriate to its role of building a research infra- structure and enhancing dissemination and utilization of research results. Addressing these failings will lay the groundwork for more effective expan- sion of important programs. • IJ lacks proper procedures for staff transitions. Management of N the fellowship programs has not been as efficient as one would

OCR for page 145
 BUILDING A RESEARCH INFRASTRUCTURE expect. This has had an impact on NIJ’s ability to effectively carry out the goals of these programs. Over the years, procedures have not been in place for a smooth transition of information and docu- mentation regarding the fellowship programs for the NIJ personnel in charge of the programs. This has led to inaccurate documenta- tion of basic and relevant information regarding the fellowship recipients. • IJ lacks a culture of self-assessment. NIJ does not maintain proper N records for assessment. The committee notes the lack of documen- tation that was available to us on the activities reported in this chapter and their impact. Assessing NIJ programs was challeng- ing, furthermore, because of the amalgamation of different types of activities in available records. NIJ has not conducted formal assessments of the fellowship programs, DRP, nor its standards program. Given NIJ’s sustained support for them, they should be fully assessed and the results of the assessment made publicly available. There were extensive external assessments of NLECTC and the federal role in technology assistance, but these were both conducted at about the same time in early 2000. Very little trans- parent assessment is done periodically and systematically (this is discussed further in Chapter 6). This is also true of its dissemina- tion activities. The limited efforts undertaken cannot reveal who is using NIJ products and, equally important, to what use are they being put. The regular assessments NIJ does support are conducted in conjunction with its forensic capacity-building programs and not for the programs that better reflect the research mission. In part this is due to the absence of a plan to guide these efforts, to fund- ing problems, and to the absence of an effective advisory structure. More generally, it reflects an agency more focused on processing awards than in building a research infrastructure and enhancing research utilization. • IJ’s status quo management of its fellowship programs has limited N its ability to build the field. Programs aimed at supporting new and more diverse scholars working on crime and justice issues have been modest in size, not well managed, and poorly integrated into the operations of NIJ. The need for new and diverse research- ers studying crime and justice issues is as critical today as it was when NIJ was founded. Expanded fellowship programs, especially ones that more effectively reach out to new disciplines, could be an important component of a federal research agency for crime and justice studies. Moreover, in being targeted to provide only

OCR for page 145
 STRENGTHENING THE NATIONAL INSTITUTE OF JUSTICE 1 year of support for graduate students, typically the final year, the graduate fellowship programs offered by NIJ are compromised in their capacity to attract new scholars to criminal justice stud- ies. Specifically, the NIJ fellowship programs do not compare well against other graduate fellowship programs, such as the multiyear programs offered by the National Institutes of Health and the Na- tional Science Foundation and the training grant programs offered by NIH. • IJ has not lived up to its potential in contributing to the de- N velopment of a resource database for use by the criminal justice research community. Although a significant fraction of the data in the National Archive of Criminal Justice Data originates from NIJ grant activities, too many grantees continue to ignore requirements for submitting data generated by their grants to the data archive. Contributing to the problem, NIJ has not mounted a successful strategy to obtain compliance from research grantees or to provide the necessary support to produce quality data sets for NACJD. For example, NIJ could require applicants to designate funds in their budgets for this purpose. Our review suggests that the data archive program is effective, well run, and making significant contributions to research. To maximize the contributions of NIJ’s data archiving program, we encourage NIJ to adopt practices as outlined to NIJ in a 2003 memorandum from ICPSR/NACJD. These efforts include (1) being more diligent in making sure that all grant- ees with data submit it to the archive in user-friendly formats, refusing to continue or make new awards to any university or research organization whose projects have failed to submit data is a reasonable step to enforce the data submission requirement; (2) requiring grantees who generate data to include in their proposals funding to cover the costs of preparing data for archiving; and (3) expanding the summer program to cover more topics and to include researchers who are in criminal justice policy and operational agencies. These steps, combined with careful monitoring by NIJ of compli- ance with its own requirements and efforts already under way at ICPSR, will greatly improve this important aspect of NIJ.