Advisers to the Nation on Science, Engineering, and Medicine

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.

www.national-academies.org

This page intentionally left blank.

Experience strongly suggests that few people or organizations seek public assessments of their performance. This is especially true for government agencies, for which the consequences of such reviews can be painful and seldom seem to result in agency enhancements. It was therefore both surprising and exciting when the National Institute of Justice (NIJ) asked the National Research Council (NRC) to conduct an assessment of its operations, research, and impact. Although NIJ was prompted to seek this review by the requirements of the Office of Management and Budget’s Program Assessment Rating Tool (PART) process and the results of some assessments by the Government Accountability Office, the agency’s genuine interest in having a comprehensive and objective assessment impressed me as well as the other members of the committee. Having worked at NIJ for a year following completion of my graduate work and in a number of different capacities over the years since then, I knew how important the decision of the NIJ leadership to seek this review was to the committee’s ability to conduct the assessment.

While the committee received admirable cooperation from NIJ and other components of the U.S. Department of Justice, the reader will see that our assessment was at times limited by the absence of basic data describing NIJ’s work and accomplishments. Time and again, information that any effective agency would be expected to maintain as part of its review of its operations was difficult to access or not available. We detail these problems in the report and call for changes in management and record-keeping that, if implemented, will mean that future assessments will not face this problem.

A vibrant and effective federal research agency addressing issues of crime and justice continues to be vital to an improved ability to reduce crime and increase justice. The committee concluded this after a careful review of NIJ and a consideration of the role that other federal, state, and nonprofit organizations can and do play in advancing knowledge about crime and justice. Nothing in this report should be construed to suggest otherwise. We expect there will be debate about some of our recommendations, but there should be no debate on whether an NIJ is important to facing the “challenge of crime in a free society.”

This report is the product of collective contributions. We could not have completed our work without the assistance of the NRC staff who provided wise counsel as well as invaluable support in drafting our report, and of numerous scholars, practitioners, policy officials, and program mangers who met with the committee and provided the information, data, and research necessary for our assessment.

We are grateful for the involvement of staff from NIJ. The director, deputy directors, and division chiefs briefed the committee on their programs and fielded numerous questions. These staff included David Hagy, director; Marc Caplan, chief, Operational Technologies Division; Christine Crossland, acting chief, Violence and Victimization Research Division; Thomas Feucht, executive science advisor and previous director of the Office of Research and Evaluation; William Ford, acting chief, Information and Sensor Technology Division; Nancy Merritt, chief, Justice Systems Research Division; John Morgan, director, Office of Science and Technology; Winifred Reed, chief, Crime Control and Prevention Research Division; Mike Sheppo, chief, Investigative and Forensics Sciences Division; Cindy Smith, chief, International Center, Office of the Director; and Edwin Zedlewski, senior science advisor. We also recognize the NIJ staff that assisted the committee in assembling documents and data on the agency and clarifying information on their programs. These included Portia Graham, associate director, Office of Operations; Jolene Hernon, director, Office of Communications; Angela Moore Parmley, acting director, Office of Research and Evaluation; John Picarelli, social science analyst; and George Tillery, associate director, Office of Science and Technology. A special note of thanks goes to Patrick Clark, senior social science analyst, and Karen Stern, social science analyst, who served as the NIJ liaisons to the committee.

We also thank the many individuals who served as presenters and discussants at our meetings and provided perspective on NIJ from the field as well as inside the Department of Justice. These included Richard Thornburgh, former U.S. Attorney General (1988-1991); Janet Reno, former U.S. Attorney General (1993-2001); Alfred Blumstein, J. Erik Jonsson university professor, urban systems and operations research, Carnegie Mellon

University; Charles Bostian, alumni distinguished professor, electrical and computer engineering, Virginia Polytechnic Institute and University; Ronnie Earle, district attorney, Travis County, Texas; Michael J. Farrell, deputy commissioner, New York City Police Department; Jeff Frazier, global justice and public safety director, Cisco Systems, Inc.; Bruce Goldberger, professor, toxicology, University of Florida College of Medicine and ex officio trustee of the American Academy of Forensic Sciences; Martin F. Horn, commissioner, Corrections and Probation, New York City; Gary LaFree, professor, criminology and criminal justice and director of the National Consortium for the Study of Terrorism and Responses to Terrorism, University of Maryland; John M. Pellegrino, director, Sensors and Electron Devices Directorate, U.S. Army Research Laboratory; Richard Rosenfeld, professor, criminology and criminal justice, University of Missouri; David G. Ross, former circuit court judge, Seventh Judicial Circuit, Maryland; and Howard Silver, executive director, Consortium of Social Science Associations.

We are particularly grateful that several former Department of Justice officials took time out of their schedules to meet with committee members. We thank former NIJ directors Sarah Hart, James K. Stewart, and Jeremy Travis for their perspectives on challenges facing NIJ as well as NIJ’s role in priority setting and dissemination, and former assistant attorneys general, Office of Justice Programs (OJP), Deborah Daniels and Laurie Robinson for their perspectives, respectively, on the need for a criminal justice research institute and on transition activities of the new administration and their effect on NIJ. Laurie Robinson, at the time she briefed the committee, was director, Master of Science in Criminology Program at the Jerry Lee Center of Criminology, University of Pennsylvania, and has since been reappointed as assistant attorney general, OJP.

We also received briefings from agency directors and program division directors of several federal research agencies. We thank those individuals who provided perspective on the roles and responsibilities of a research agency: Wilson Compton, director, Division of Epidemiology, Services and Prevention Research, National Institute of Drug Abuse; Rolf Dietrich, deputy director, Research Division, U.S. Department of Homeland Security; Christopher Doyle, director, Infrastructure and Geophysical Division, U.S. Department of Homeland Security; Naomi Goldstein, director, Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services; Wayne Goodman, director, Division of Adult Translational Research, National Institute of Mental Health; Patricia Gruber, director of research, Office of Naval Research; Susan Haire, project officer, Law and Science Program, National Science Foundation; Ralph Hingson, acting director, and Vivian Faden, deputy director, Division of Epidemiology and Prevention, National Institute on Alcohol Abuse and Alcoholism; David Lightfoot, assistant director, Social,

Behavioral, and Economic Sciences Directorate, National Science Foundation; Richard Nakamura, deputy director, National Institute of Mental Health; Kevin Neary, deputy assistant secretary for research, Office of Research, Evaluation, and Monitoring, U.S. Department of Housing and Urban Development; Lynn Okagaki, commissioner, National Center for Education Research, U.S. Department of Education; Georgeanne Patmios, assistant director, Division of Behavioral and Social Research, National Institute on Aging; Norka Ruiz Bravo, director, Office of Extramural Research, National Institutes of Health; Eric Steel, director, Program Office, National Institute of Standards and Technology; and Grover (Russ) Whitehurst, director, Institute of Education Sciences, U.S. Department of Education.

In addition to these public briefings, we reviewed published literature and legislation, documents assembled by NIJ, as well as reports prepared for the committee. We were thankful to have the opportunity to review a report on the National Archive of Criminal Justice Data submitted by Kaye Marz, archive manager, and Christopher D. Maxwell, associate research scientist, from the Inter-university Consortium for Political and Social Research at the University of Michigan. We are also grateful to Nicola Smith, University of Maryland, who prepared a report on NIJ’s graduate research and W.E.B. Du Bois fellowship programs and assisted the committee in our citation analyses. We thank Scott McBride and Donna Kenly from Hollander Cohen & McBride Marketing Research for conducting a web-based survey of criminal justice researchers and practitioners and assembling and summarizing the data in a report for the committee.

On behalf of the committee, staff conducted site visits to some of NIJ’s technology centers and interviewed 26 current and former NIJ staff to learn more about its processes, programs, and achievements. We are grateful to those who helped make the site visits informative including Troy Krenning, director, National Law Enforcement and Corrections Technology Center, Rocky Mountain; Kevin Lothridge, director, Forensic Science Center of Excellence; Andy Mazzara, director, Weapons and Protective Systems Technology Center of Excellence; Raj Nanavati, director, Sensors, Surveillance, and Biometric Technologies Center of Excellence; Joe Peters, director, Border Research and Technology Center; and Brian Regli, former director, Communications Technologies Center of Excellence.

We are also thankful to the current and former NIJ staff that agreed to be interviewed. The interviews included staff from every division within the Office of Research and Evaluation, two of the three divisions within the Office of Science and Technology, and every function (administration, special advisors, international crime, and communications) within the Office of the Director and as a whole covered the scope of NIJ’s operations from the 1970s to present. Each interview consisted of three parts: (1) employee history

and professional background; (2) questions related to grant processes and roles; and (3) questions on NIJ’s mission and impact. Since our invitation to the interviewees indicated that responses would remain anonymous and all names would be kept confidential, we do not recognize them by name here. However, we acknowledge that without their candor and insight into agency processes we would not have gained as complete an understanding of NIJ.

This study and its report have also benefited from the valuable assistance of many NRC staff within the Division of Behavioral and Social Sciences and Education. Betty Chemers was the study director. As such, she organized meetings, identified sources of information and conducted analyses, and worked tirelessly with the committee to integrate their ideas, writings, and conclusions into a sound report. Julie Schuck, as research associate, assembled background documents, created databases of award histories when none was available, and assisted in drafting and editing this report. Carol Petrie, director, Committee on Law and Justice, provided perspective on NIJ’s history and helped us integrate our work with prior NRC studies. This study also benefited from the counsel and experience of Barney Cohen, Anne-Marie Mazza, and Daniel Cork, NRC staff who oversaw relevant assessments of other federal agencies and research programs. Jacqui Sovde, program associate, made sure meetings were organized and conducted in a professional manner and assisted in the editing and formatting of this report. Several others provided administrative support as needed including Barbara Boyd, Linda DePugh, and Anthony Mann. We greatly appreciate the efforts undertaken by Eugenia Grohman, Christine McShane, Jane Ross, Kirsten Sampson Snyder, and Yvonne Wise to complete the review and editing processes and bring this report to fruition.

This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their review of this report: Todd R. Clear, Department of Law, Police Science and Criminal Justice Administration, John Jay College; Max M. Houck, Forensic Science Initiative, West Virginia University; Rick Kern, Office of the Director, Virginia Criminal Sentencing Commission; Janet Lauritsen, Department of Criminology and Criminal Justice, University of Missouri, St. Louis; Michael D. Maltz, Department of Sociology, Criminal Justice Research Center, Ohio State University; Stan Orchowsky, Office of the Research Director, Justice Research and Statistics Association, Washington,

DC; Robert J. Sampson, Department of Sociology, Harvard University; Robert Santos, Statistical Methods Group, The Urban Institute, Washington, DC; Joan C. Weiss, Office of the Executive Director, Justice Research and Statistics Association, Washington, DC; Chuck Wexler, Office of the Executive Director, Police Executive Research Forum, Washington, DC; Tara Wildes, Office of the Chief, Jacksonville Sheriff’s Office, Jacksonville, Florida; and Paul Wormeli, Office of the Executive Director, IJIS Institute, Ashburn, Virginia.

Although reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before it was released. The review of this report was overseen by John C. Bailar III, Department of Health Studies (emeritus), University of Chicago, and Gary LaFree, National Consortium for the Study of Terrorism and Responses to Terrorism, University of Maryland. Appointed by the NRC, they were responsible for making certain that an independent examination of this report was carried out in accordance with the institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution.

Charles F. Wellford, Chair

Committee on Assessing the Research Program of the National Institute of Justice

This page intentionally left blank.

AAG Assistant Attorney General

ABA American Bar Association

ACA American Correctional Association

ADAM Arrestee Drug Abuse Monitoring system

AG Attorney General

AHRQ Agency for Healthcare Research and Quality

APM Analysis, Planning and Management

ASA American Sociological Association

ASC American Society of Criminology

ASCLD American Society of Crime Lab Directors

ATF Bureau of Alcohol, Tobacco and Firearms and Explosives

BIS-WDS Brijot Imaging Systems–Weapons Detection System

BJA Bureau of Justice Assistance

BJS Bureau of Justice Statistics

BLS Bureau of Labor Statistics

BPI OJP Business Process Improvement

BRTC Border Research and Technology Center

BRTD U.S. Border Patrol Technology Demonstrations

BSR Behavioral and Social Research

BTC NIJ’s Breaking the Cycle Program

CAPRAD Computer Assisted Pre-Coordination Resource and Database System

CAPS Chicago Alternative Policing Strategy

CBO Congressional Budget Office

CBRN Chemical, biological, radioactive, nuclear

CBRNE Chemical, biological, radioactive, nuclear, and high-yield explosives

CDC Centers for Disease Control and Prevention

CLAJ Committee on Law and Justice

CLIP Crime Laboratory Improvement Program

CMS Community Management Staff

CNSTAT Committee on National Statistics

CODIS Combined DNA Index System

COE Center of Excellence

COMPSTAT the name given to a strategic problem-solving process or “strategic control system” first implemented by the New York City Police Department; short for computer statistics or comparative statistics

COPS Community Oriented Policing Services

COSSA Consortium of Social Science Associations

COTR OJP Contracting Officer’s Technical Representative

COTS commercial-off-the-shelf

COV Committee of Visitors

CPTED Crime Prevention Through Environmental Design

CRISP Computer Retrieval of Information on Scientific Projects

CRS Congressional Research Service

CSR Center for Scientific Review

DARE Drug Abuse Resistance Education

DARPA Defense Advanced Research Projects Agency

DEA Drug Enforcement Administration

DEPR Division of Epidemiology and Prevention Research

DHS U.S. Department of Homeland Security

DMA Drug Market Analysis Program

DOC U.S. Department of Commerce

DoD U.S. Department of Defense

DoEd U.S. Department of Education

DOJ U.S. Department of Justice

DOL U.S. Department of Labor

DPCPSI Division of Program Coordination, Planning, and Strategic Initiatives

DRP Data Resources Program

DUF Drug Use Forecasting system

EPA U.S. Environmental Protection Agency

ESIP Equipment Systems Improvement Program

FBI Federal Bureau of Investigation

FDA U.S. Food and Drug Administration

FOIA Freedom of Information Act

GAO U.S. Government Accountability Office

GIS geographic information systems

GMS Grant Management System

GPA Grant Progress Assessment Program

GPRA Government Performance and Results Act

GRF Graduate Research Fellowship

GS general schedule or grade service (refers to pay level)

HHS U.S. Department of Health and Human Services

HRSA Health Resources and Services Administration

HUD U.S. Department of Housing and Urban Development

IAA Interagency Agreement

IACP International Association of Chiefs of Police

ICAM Information Collection for Automated Mapping project (Chicago PD)

ICP-OES inductively coupled plasma-optical emission spectroscopy

ICPSR Inter-university Consortium for Political and Social Research

IES Institute of Education Sciences

IPT Integrated Product Team

IRB Institutional Review Board

JAG Justice Assistance Grants

JD Juris Doctor

JRSA Justice Research and Statistics Association

JUSTNET Justice Technology Information Network

LAPD Los Angeles Police Department

LEAA Law Enforcement Assistance Administration

LECTAC Law Enforcement and Corrections Technology Advisory Council

LEEP Law Enforcement Education Program

LESL Law Enforcement Standards Laboratory

MAPS Mapping and Analysis for Public Safety Program

MD Maryland

MMW millimeter wave

MOA memorandum of agreement

MOU memorandum of understanding

MPD District of Columbia Metropolitan Police Department

NACJD National Archive of Criminal Justice Data

NAS National Academy of Sciences

NCJRS National Criminal Justice Reference Service

NDIS National DNA Index System

NFC National Finance Center

NFSIA Paul Coverdell National Forensic Sciences Improvement Act

NFSTC National Forensic Sciences Technology Center

NIA National Institute on Aging

NIAAA National Institute on Alcohol Abuse and Alcoholism

NIBIN National Integrated Ballistic Information Network

NIDA National Institute of Drug Abuse

NIH National Institutes of Health

NIJ National Institute of Justice

NILECJ National Institute of Law Enforcement and Criminal Justice

NIMH National Institute of Mental Health

NIST National Institute of Standards and Technology

NLECTC National Law Enforcement and Corrections Technology Center

NPR National Performance Review

NRC National Research Council

NSB National Science Board

NSF National Science Foundation

NYC New York City

NYPD New York City Police Department

OASH Office of the Assistant Secretary of Health

OCFO-BD Office of the Chief Financial Officer-Budget Division

OCOM Office of Communications

OERI Office of Educational Research and Improvement

OGC Office of the General Counsel

OIG DOJ Office of the Inspector General

OJARS Office of Justice Assistance, Research and Statistics

OJJDP Office of Juvenile Justice and Delinquency Prevention

OJP Office of Justice Programs

OLES Office of Law Enforcement Standards

OLETC Office of Law Enforcement Technology Commercialization

OLP Office of Legal Policy

OMB Office of Management and Budget

ONDCP Office of National Drug Control Policy

ONR Office of Naval Research

ORE Office of Research and Evaluation

OST Office of Science and Technology

OVC Office for Victims of Crime

OVW Office on Violence Against Women

PAR Performance and Accountability Report

PART Program Assessment Rating Tool

PBMA Planning, Budget, Management and Administration Office

PD police department

PHDCN Project on Human Development in Chicago Neighborhoods

PI principal investigator

POSC Program Office Solicitation Coordinator

PREA Prison Rape Elimination Act

PSN Project Safe Neighborhoods

RAC Regional Advisory Council

R&D research and development

RDT&E research, development, testing, and evaluation

RFP request for proposal

RSAT Residential Substance Treatment Programs

SACSI Strategic Approaches to Community Safety Initiative

SAMHSA Office of Substance Abuse and Mental Health Services

SBE Directorate of Social, Behavioral & Economic Sciences

SES Senior Executive Service

SES Social and Economic Sciences

SETA Systems Engineering and Technical Assistance

SME subject-matter expert

SRG Scientific Review Group

SSCI Social Science Citation Index

S&T science and technology

STOP (Services, Training, Officers, and Prosecutors) Violence Against Women Formula Grants

SVORI Serious Violent Offender Re-entry Initiative

TAPAC Technology Assessment Program Advisory Council

TAPIC Technology Assessment Program Information Center

TATP triacetone rriperoxide

TSWG Technical Support Working Group

TWG technical working group

USAFRL U.S. Air Force Research Laboratory

VAWA Violence Against Women Act