As discussed in Chapter 2, a variety of federal entities engage in forensic science research and practice. However, the National Institute of Justice (NIJ) is the principal federal agency that supports research and development (R&D) for forensic science, having done so since the 1970s.
In 2009 and 2010, two reports of the National Research Council (NRC) critically reviewed the needs of the forensic science field and the performance of NIJ as a science agency, respectively (National Research Council, 2009b, 2010). The recommendations in those reports include some directed at NIJ, some directed at policy makers, and others directed at the broader forensic science field (see Appendix B). Both reports recognized that NIJ was operating under significant challenges and lacked strategic direction at the time. This chapter reviews the agency’s current research operations and its progress toward advancing forensic science research since the previous two NRC reports. The committee also examines and assesses the adequacy of the resources NIJ has available to support its research operations.
As discussed below, NIJ has made productive organizational and process changes in its operations since the release of Forensic Science: Path Forward in 2009 and Strengthening NIJ in 2010.1 However, the committee believes the agency still lacks a clear strategy for its research portfolio and thus falls short in its ability to advance the field.
1As noted in Chapter 1, this report uses the abbreviation Forensic Science: Path Forward for the 2009 NRC report and Strengthening NIJ for the 2010 report.
NIJ’s forensic science R&D portfolio is divided among its three science offices (see Figure 1-2 in Chapter 1): The Office of Investigative and Forensic Sciences (OIFS) manages R&D to support forensic disciplines that are based on physical and life sciences (anthropology, biology/DNA, drug chemistry, pathology, toxicology, trace evidence analysis, etc.). The Office of Science and Technology (OST) manages technology development associated with digital and multimedia forensic evidence. The Office of Research and Evaluation (ORE) manages social science research related to forensic science.2
The work of OIFS is entirely focused on forensic science, whereas OST and ORE also manage research in other areas of crime, crime control, and the administration of justice. Prior to 2009, a division of investigative and forensic sciences was housed within OST (see Figure 1-1 in Chapter 1); many of that division’s activities were subsequently assumed by OIFS when the office was created. The motivation for the change, according to documents provided by NIJ,3 was to focus efforts on building a body of research in forensic science and to explicitly highlight NIJ activity in this critical area to audiences within the Department of Justice and across the forensic science field. The committee believes this reorganization is appropriate to elevate the stature of forensic science and improve access to resources within the agency.
Grants awarded by OIFS comprise most of the forensic science portfolio at NIJ; this portfolio includes forensic science research cooperative agreements as well as assistance grants for casework backlog reduction and forensic laboratory improvements. The mission of OIFS and its goals for R&D are shown in Box 3-1.
Much of NIJ’s research portfolio in forensic science has focused on developing technologies and tools for forensic science practitioners. The portfolio and its size and areas of research are discussed later in the chapter. NIJ follows a research, development, testing, and evaluation (RDT&E) process that is designed to align its R&D portfolio with the expressed needs of the forensic science community.4 The process has five phases: (1) determine technology needs; (2) develop a program plan to address these
2ORE has historically provided oversight to the agency’s social science research and evaluation studies. OST manages science and technology research and development, creates technical standards and equipment testing for certain technologies, and provides technology assistance to state and local law enforcement and corrections agencies (National Research Council, 2010, p. 16).
3Written response to Question 1, provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C.
4The RDT&E process is also used in OST to develop other technology-based research and development portfolios.
Mission: improve the quality and practice of forensic science through innovative solutions that support research and development, testing and evaluation, technology, information exchange, and the development of training resources for the criminal justice community.
Research and development goals: (1) expand the information that can be extracted from forensic evidence, including DNA, and quantify its evidentiary value; (2) develop reliable and widely applicable tools and technologies that allow faster, cheaper, and less labor-intensive identification, collection, preservation, and analysis of forensic evidence of all kinds and reduce existing case backlogs; and (3) strengthen the scientific basis of the forensic science disciplines.
SOURCE: NIJ’s Office of Investigative and Forensic Sciences, see http://www.nij.gov/about/pages/oifs.aspx [June 2015].
needs; (3) develop solutions; (4) demonstrate, test, evaluate, and adopt into practice; and (5) build capacity and inform the forensic science community. These phases are similar to steps taken by other federal agencies or other entities to gather needs or requirements and define and develop research priorities that will address policy and practice issues (Lenaway et al., 2006; Campbell, 2010). Notably, such steps include identifying needs, setting research priorities, communicating research priorities, and building an infrastructure for research and for the translation of research to practice. In accordance with its charge (see Chapter 1), the committee reviews NIJ’s operations in terms of these general research-setting strategies in order to determine how NIJ develops forensic science research priorities and communicates these priorities, as well as research findings, to the scientific and practitioner communities. Strengths in its process as well as areas for improvement are identified.
NIJ has a process for identifying the forensic science needs of practitioners, and it uses these needs as a significant input to the development of its research solicitations. Both OST and OIFS use Technology Working Groups
(TWGs) to develop and categorize a list of operational and technology needs and practical problems affecting the day-to-day work of practitioners; until recently, this list was only used internally by staff.5 This process is very similar to that used by OST prior to 2009. At that time, there were two TWGs related to forensic science (of 17 TWGs across OST): one focused on DNA and another on general forensics. Currently, there is one forensic science TWG that is overseen by OIFS; it has 5 subgroups, each with 8 to 14 members: (1) Standard DNA, (2) Non-Standard DNA, (3) Crime Scene and Medicolegal Death Investigations, (4) Pattern and Trace Evidence, and (5) Drugs and Toxicology. These subgroups cover a greater breadth of forensic science disciplines than before and are able to identify at least a few needs in each of the disciplines listed in Box 2-1 (see Chapter 2). However, the resulting TWG review is by no means a comprehensive analysis of all the research gaps in each discipline.
The TWG, made up of forensic science practitioners, is not intended to be a research advisory group but rather to be a source of input into the research priorities ultimately chosen by NIJ. According to OIFS, the current TWG process is designed to avoid bias by researcher input and avoid conflating current practices with technologies in development or research findings not yet published.6 For those reasons, individuals who are primarily researchers and not practicing forensic scientists have not been included in the TWG membership. The committee finds that the current process of gathering and assessing needs is entirely focused on the needs within the practice of forensic science and does not reflect the views or address the needs as seen by researchers trying to advance the scientific basis of forensic science.
For fiscal 2015, OIFS made its full list of TWG-generated forensic science needs more transparent to the research community by posting it on the NIJ website and providing a link within the R&D solicitation (National Institute of Justice, 2015b, p. 4). Providing the full list of identified needs is intended to inform researchers of practitioners’ priorities and assist in proposal development (National Institute of Justice, 2015b). However, the list is made available solely for researchers’ consideration; there is no requirement or incentive to develop proposals that specifically address the identified needs. NIJ notes that it writes its solicitation broadly enough so that researchers are not restricted to ideas proposed by practitioners and to allow them to propose other solutions designed to move the state of the science forward.7
5The current list of needs can be found at http://www.nij.gov/topics/forensics/Documents/fy15-forensic-twg-table.pdf [April 2015].
6Written response to Question 6, provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C.
7Written response to Question 6, provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C.
The committee heard from both practitioners and researchers who found the compilation and broader availability of the TWG-generated list useful.8 However, the committee found that the list of needs is not comprehensive across all the forensic science disciplines (e.g., there are over 40 needs identified for forensic biology/DNA and only 4 needs for crime scene analysis); it also lacks description of critical research gaps and promising new approaches. Moreover, it is a long list with little structure or hierarchy of priorities and without any connection to the priorities (e.g., microbiome) identified in the most recent NIJ R&D solicitation (National Institute of Justice, 2015b, pp. 9-10).
Strengthening NIJ recommended that NIJ “strengthen its science mission” and “revise its research operations to allow greater transparency, consistency, timeliness, and appropriate involvement of the research and practitioner communities” (National Research Council, 2010, pp. 4, 7; also reproduced in Appendix B of this report). The committee observed that NIJ has taken steps in the last 5 years to respond to these recommendations, including a renewed focus on science and research (see, for example, Box 3-1 and recent NIJ solicitations [National Institute of Justice, 2010, 2015b]) and efforts to make its processes more transparent. However, the agency has yet to appropriately involve the research community in identifying research priorities.
The committee commends OIFS for engaging with the practitioner community. Forensic science work is necessarily practical; assessing the needs of practitioners is therefore critical to carrying out OIFS’s mission. But including researchers more actively in the process of identifying research gaps and articulating priorities would be useful, since relevant disciplinary researchers (anthropologists, biologists, chemists, physicists, statisticians, etc.) can bring their specialized knowledge to bear on possible uses and limitations of forensic evidence as well as promising areas for advancing research and knowledge. Currently, NIJ uses its broad solicitation for forensic science R&D as the avenue for researchers to direct priorities (i.e., by proposing research projects in response to the extensive statement of needs), as opposed
8Panel presentations to the committee. PI Panel, by Ann Bunch, SUNY-Brockport; Cedric Nueman, South Dakota State University; and Hanlee Ji, Stanford University (April 1, 2015) and Technology Working Groups panel, by George Herrin, Georgia Bureau of Investigation; Mike Gorn, Sarasota County Sheriff; and Steve Renteria, Los Angeles County Sheriff’s Department, to the Committee on Strengthening Forensic Science at the National Institute of Justice, May 7, 2015. The practicing forensic scientists from the Technology Working Groups panel acknowledged that because of large caseloads there is little to no time to conduct research in the laboratories, so they appreciate the larger research community’s awareness of needs in their field. A 2009 census of public crime laboratories found that approximately 7 percent of respondents reported the ability to dedicate any resources to research activities (Bureau of Justice Statistics, 2012).
to asking researchers for scientific advice on framing research opportunities before the solicitation is created. Including researchers at an earlier stage in solicitation development would enhance NIJ’s ability to prioritize research areas and develop short- and long-term research agendas. This practice is used by many other funding agencies (e.g., National Science Foundation [NSF], National Institutes of Health) and should not pose a significant conflict of interest for the researchers, since their role is purely advisory and the final decisions are made by the agency staff.
Conclusion 3-1: The National Institute of Justice has an established mechanism for identifying the needs of forensic science in practice through a Technology Working Group of practitioners. However, this group does not adequately represent the needs or perspectives of the broad range of forensic science disciplines. In addition, there is no mechanism for integrating the perspective of research scientists to help identify scientific gaps and opportunities and develop an overarching strategic research agenda.
Currently, NIJ does not have a strategic plan that lays out its short-term and long-term forensic science research priorities or how it will go about achieving these priorities. NIJ has not changed how it communicates research priorities to researchers. The committee heard that NIJ still communicates its forensic science research priorities to the research community only through its solicitations.9 The committee believes that this approach is not as effective as having a public statement of research priorities, which is available before the release of solicitations and can direct researchers to critical areas in forensic science for longer-term funding opportunities. However, NIJ has made other changes to its procedures for awarding research grants. The following sections review recent changes in its solicitations, peer review, and award decisions and assess whether these changes have positively affected NIJ’s stature as a research agency.
According to NIJ, reviewing the priority areas for forensic science research occurs on a routine and continuing basis. Within OIFS, program managers hold weekly meetings to discuss current R&D investments, potential new funding opportunities, recent scientific or legal events that may
9Written response to Questions 6, provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C.
impact the forensic community, and dissemination of key research findings. NIJ representatives informed the committee that changes to the solicitation process reflected in the fiscal 2011 and 2014 forensic science R&D solicitations were the result of such internal planning.10
Recent solicitations have focused on building the scientific basis supporting forensic science practice, most notably the 2009 and 2010 solicitation, Fundamental Research to Improve Understanding of the Accuracy, Reliability, and Measurement Validity of Forensic Science Disciplines, in direct response to recommendations in Forensic Science: Path Forward (see, for example, National Institute of Justice, 2009, p. 3; 2010, p. 5). NIJ has increased efforts to promote its forensic science R&D funding solicitations to the academic community via presentations and booths at annual meetings of scientific organizations, as well as through social media and email notices.11
Solicitations encourage new investigators (i.e., junior faculty who have never received NIJ funding for research projects or established researchers who have not received NIJ funding in the past 10 years) in an attempt to expand research interest in forensic science issues. In addition, proposals in NIJ-specified innovative areas of research are encouraged by giving them special consideration in the selection process. The three designated innovative areas for fiscal 2015 were nanotechnology, the microbiome, and fatal head trauma (National Institute of Justice, 2015b, pp. 8-10).
These innovative areas reflect national research initiatives throughout the federal government, but the level of strategic planning involved in designating them as special-consideration areas for forensic research is unclear. OIFS does not articulate how these innovative areas may advance the stated goals of the R&D program more than other areas of forensic research. Nor is it clear how long these areas will remain priority areas for NIJ funding.
Conclusion 3-2: Through its solicitations, the National Institute of Justice (NIJ) has taken several steps to advance forensic science research and attract new investigators. However, the priority issues emphasized in the agency’s solicitations appear to be reactive to short-term political
10Written response to Questions 6 and 15, provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C. After 2009, the solicitations by OIFS became less targeted and were generalized into one (fiscal 2014) or two (fiscal 2011) solicitations to cover basic and applied research across a broad range of disciplines (National Institute of Justice, 2011c, 2011d, 2014b). NIJ’s expired solicitations are available at http://www.nij.gov/funding/Pages/expired.aspx [June 2015].
11Presentation on Research, Development, Testing, and Evaluation Process, by Gerry LaPorte and Danielle McLeod-Henning, OIFS, NIJ, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015.
and public concerns, and it is not clear how the priorities announced by NIJ relate to an overall long-term research agenda for forensic science.
Proposals to NIJ in response to solicitations have undergone peer review since the late 1970s. In 2004, the Office of Justice Programs (OJP), the parent agency of NIJ and other agencies, centralized the peer-review process for all its bureaus, including NIJ. Strengthening NIJ found that this centralized process limited NIJ’s ability to tailor peer review to its own needs and mission—that is, applications proposing to conduct research require a different kind of expertise and review than those requesting programmatic support. Strengthening NIJ noted that “. . . the use of practitioners on review panels, the scoring and ranking system, and the process for making individual peer reviews available . . .” deserved reconsideration (National Research Council, 2010, p. 123).
NIJ and OIFS have made changes to their peer-review process since 2009. As a pilot program, NIJ has instituted Standing Review Panels (SRPs) for a number of its solicitations, including those for forensic science. Proposals related to R&D on impression/pattern evidence and trace evidence analysis are reviewed through SRPs.12 Proposals submitted in other forensic areas, such as those for biology/DNA, controlled substances, crime scene investigation, forensic anthropology, forensic pathology, and toxicology, are reviewed by what are known as ad hoc review panels.
NIJ oversees its own peer-review panels following OJP standard operating procedures or procedures for SRPs developed in consultation with OJP’s Office of Audit, Assessment, and Management. But, notably since 2009, NIJ as well as the Bureau of Justice Statistics and Office of Juvenile Justice and Delinquency Prevention have been exempt from certain procedures/criteria that apply to other OJP bureaus. With these exemptions, NIJ does not have to follow OJP’s normalization process for peer-review scores and has the ability to vary peer-review formats under one solicitation. The latter approach has been used for its forensic science R&D solicitation, which receives proposals across a broad array of disciplines (Office of Justice Programs, 2013).
As a result, instructions to proposal reviewers currently reflect a greater focus on science; recently, the weight assigned to the research idea in the scoring system has increased, so that 50 percent of the proposal score is now based on the quality of the idea, the project design, and the implementation. The rest of the score is based on the quality of the statement
12NIJ’s OIFS also uses a SRP for proposals received under its Paul Coverdell Forensic Science Improvement Grants Program.
of the problem (10%), the potential impact of the research (20%), and capabilities/competencies of the investigators (20%). The budget and plan for dissemination are reviewed but not scored (Office of Investigative and Forensic Sciences, 2015). Additionally, in recent years considerable effort has gone into enhancing the scientific credentials of individuals on the review panels. For example, NIJ has reached out to the American Statistical Association for help in identifying statisticians to serve on the review panels.
In remarks to the committee, a number of NIJ peer reviewers described their experiences with NIJ as comparable to other federal agencies, including the National Institutes of Health, NSF, and the Department of Defense. Those with experience as members of SRPs and ad hoc review panels expressed their belief that both capably assess the science underlying the proposals.13
Some previous concerns about NIJ’s lack of independence (National Research Council, 2010, pp. 3-4) have been addressed in the award process by designating final sign-off authority to the NIJ director. Previously, the Assistant Attorney General, as the head of OJP, had to sign off on all of NIJ’s awards. Now the Department of Justice Scientific and Integrity Policy14 states (p. 7): “[t]he Director of the National Institute of Justice shall have final authority over all grants, cooperative agreements and contracts awarded by the institute. . . .” From the committee’s perspective, this is an important improvement.
OIFS staff recommend forensic science research proposals for funding to the NIJ director based on a number of factors, including the strength of the research proposals as evaluated through their peer-review rankings, the needs of the forensic science practitioner community, and the quality and state of existing R&D initiatives.15 In addition, special consideration may also be given to proposals that involve investigators not previously funded by NIJ or that address one of the designated innovative areas of research (discussed above).16 Peer-reviewer recommendations are used as guidance in
13Panel presentations on NIJ Peer Review Panels, by Phillip Danielson, University of Denver, and Eric Bartelink, California State University, Chico, and Standing Review Panels at NIJ, by Eric Buel, State of Vermont Forensic Laboratory; Brooke Weinger Kammrath, University of New Haven; and Martin Wells, Cornell University, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015.
15Presentation on Research, Development, Testing, and Evaluation Process, by Gerry LaPorte and Danielle McLeod-Henning, OIFS, NIJ, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015.
16Presentation on Research, Development, Testing, and Evaluation Process, by Gerry LaPorte and Danielle McLeod-Henning, OIFS, NIJ, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015.
selecting awardees (Office of Justice Programs, 2013) but are not the only consideration. In the determination of awards, it is unclear to the committee what weight is given to peer-review rankings in relation to the TWG-generated list of needs discussed earlier and to other staff considerations, such as current investments and recent events.
Conclusion 3-3: As recommended in previous National Research Council reports, the National Institute of Justice has obtained an increased level of autonomy and independence for its scientific peer-review process.
Conclusion 3-4: As recommended in previous National Research Council reports, the Director of the National Institute of Justice was given final sign-off authority for research awards.
OIFS has been able to expand the size of its forensic science research portfolio. In the period from fiscal 2004 through 2008, there were 13 to 39 research grants awarded annually, with total annual research funding in the range of $6 to $15 million. In the period from fiscal 2009 through 2013, there were 33 to 65 research grants awarded annually, with total annual research funding in the range of $15 to $33 million. The budget for NIJ’s forensic science research awards is discussed later in this chapter. Here the general nature of its research awards is examined.
Forensic science research spans a number of disciplines. Figure 3-1 shows the distribution by forensic science discipline of research awards administered by OIFS in the periods fiscal 2004-2008 and fiscal 2009-2013. Forensic DNA research still accounts for a large proportion of NIJ’s forensic science portfolio in number of awards (as well as funding). Since fiscal 2009, the number of grant awards in the areas of medicolegal death investigations (forensic pathology), impression and pattern evidence, and trace evidence has significantly increased. There have also been more awards in the areas of DNA and forensic biology, fire debris analysis and arson, questioned documents, forensic crime scene analysis, and forensic toxicology/controlled substances.
Conclusion 3-5: From fiscal 2009 to fiscal 2013, the National Institute of Justice has increased the number of awards for forensic science research across disciplines in comparison with prior years, and it has expanded research attention in areas recommended in previous National Research Council reports.
FIGURE 3-1 Distribution of research awards by discipline, fiscal 2004-2008 and 2009-2013.
SOURCE: Committee generated using data provided by NIJ. Disciplines are those outlined in NIJ’s forensic science R&D solicitation (National Institute of Justice, 2015b), shown in Box 2-1 in Chapter 2, except that Controlled Substances and Forensic Toxicology have been combined; firearms and toolmark identification, latent print, and shoeprint/tire tread examination were all considered under Impression & Pattern Evidence; and award data were unavailable for the digital evidence discipline.
NIJ’s R&D portfolio has covered a wide range of technology development and process validation, as suggested by the number of disciplines and distribution of awards in Figure 3-1. The agency’s programs and studies around DNA evidence are most often recognized (Lovrich et al., 2004; National Research Council, 2010). NIJ continues to support research in this area that could advance techniques and technologies, including automated systems, or that could allow laboratories to work more efficiently and at lower cost. It also supports research to improve the collection and processing of other types of forensic evidence. Recent projects, for example, are developing imaging technologies for detection of fingerprints, body fluids, and other residues at crime scenes; developing hand-held analyzers for controlled substances; and tackling new challenges such as the identification of synthetic cannabinoids. Other studies have focused on validating the accuracy of expert forensic examinations (e.g., blood stain patterns, fingerprints, and firearms) and investigating any sources of human error and bias (National Institute of Justice, 2015a).
Findings and developments from NIJ-funded studies can and have been used in the protocols and practices of forensic laboratories.17 According to NIJ-involved researchers and practitioners, the forensic science community makes use of the knowledge and products produced through NIJ and values the informational resources available.18 Similar support from the community for a research agency focused on criminal justice challenges was also documented in Strengthening NIJ (see National Research Council, 2010, p. 12).
The advances in practice observed today were influenced by research conducted years ago, some notably funded by NIJ prior to 2009. OIFS
17NIJ-funded research was used to benchmark how DNA should be quantified and served as a guide for industry to develop analysis kits, which ultimately streamlined the process from 2.5 hours to a semi-automated 15-minute procedure (presentation on Standing Review Panels at NIJ, by Eric Buel, State of Vermont Forensic Laboratory, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015). Findings from an NIJ-funded study on applying signal-to-noise ratios in forensic glass analysis (Ernst et al., 2012) have been adopted into the laboratory protocol of Johnson County, KS (presentation on Perspectives of Forensic Labs, by Kristine Olsson, Johnson County Sheriff’s Office Crime Laboratory, to the Committee on Strengthening Forensic Science at the National Institute of Justice, May 7, 2015). NIJ and the National Institute of Standards and Technology (NIST) have collaborated on publications for the community, notably handbooks related to biological evidence preservation, laboratory construction, and crime scene investigation. They also recently sponsored a working group (Expert Working Group on Human Factors in Latent Print Analysis, 2012) to recommend ways to improve the practice of latent print analysis and reduce the risk of human error (presentation on Standards and Practices of Forensic Science from an NIST Perspective, by John Butler, NIST, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015). Researchers recognize that initial work was done through NIJ funding, notably in the introduction of automation, such as the implementation of capillary electrophoresis in laboratories, and development of better kits for processing Y-STR (short tandem repeat on the Y-chromosome) casework (presentation on PI Panel on Impact of Research, by Bruce McCord, Florida International University, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015).
18Techniques initiated by NIJ-funded work have transformed laboratory work, such as advances in mass spectrometric methods for toxicology and real-time polymerase chain reaction quantification. NIJ support for databases is also useful. Other techniques and tools are starting to appear, such as the use of messenger RNA (ribonucleic acid) to identify body fluids and the handheld Raman infrared devices. In addition, the workshops conducted and archived by the NIJ-funded Forensic Technology Center of Excellence as well as other resources and training materials have reached many forensic scientists and repeat customers (presentations on PI Panel on Impact of Research, by Bruce McCord, Florida International University, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015, and on Technology Working Groups, by George Herrin, Georgia Bureau of Investigation, to the Committee on Strengthening Forensic Science at the National Institute of Justice, May 7, 2015). The Green Mountain DNA conference has become a prominent forum on improving research and practice and encouraging collaboration in the field of forensic science and was launched 8 years ago with funding from NIJ (presentation on Standing Review Panels at NIJ, by Eric Buel, State of Vermont Forensic Laboratory, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015).
provided the committee with an example of how an applied R&D project can begin to make its way into forensic practice. The study of a portable, tandem mass spectrometer for onsite identification of forensic evidence (O’Leary et al., 2015) was funded in 2011 with promising research results published in 2015. The potential commercialization and adoption of such a tool is likely still years away. This research lag makes determining the impact of research on the practice of forensic science challenging.
The committee finds that NIJ is just starting to collect information useful for assessing its research program more systematically. Beginning in fiscal 2011, NIJ-funded researchers are required to respond to non-budgetary components of the Research Performance Progress Report19 as part of their progress reporting to the agency. This allows the agency to capture information that can help assess the impact of its research portfolio, such as dissemination activities, patent applications, licenses, and qualitative interpretations of the impact of research awards.20 NIJ collects a list of publications, presentations, and deliverables through semi-annual reports from research grant recipients and now keeps track of publications and presentations by its research grantees and of downloads of technical reports posted to the National Criminal Justice Reference Service.21 The reference service recently has begun assembling citation information on all NIJ-supported research reports to date.
Although the recordkeeping is useful and an improvement over the types of indicators collected before 2009, tracking citations and scholarly products can only suggest whether or not research studies are reaching the research community. Measuring impact—that is, influencing or changing the field of research and ultimately practice—is much more difficult to gauge. Program officers catalog grantee products from final summary reports, but these are not consistently compared across different time periods or analyzed across program offices or the forensic science portfolio as a whole to identify overarching conclusions or cumulative impact.
In 2015, NIJ released a report on The Impact of Forensic Science Research and Development (National Institute of Justice, 2015a). This
20Written response to Question 6, provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C.
21As noted in NIJ’s recent report on The Impact of Forensic Science Research and Development, for awards made from fiscal 2009 through 2013, there had been 77 final technical reports submitted, 255 refereed journal publications, and more than 600 conference presentations (through July 2014) (National Institute of Justice, 2015a). In a written statement to Question 7, provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C, NIJ reported annual downloads of all its available forensic science R&D final technical reports in the range of about 275,000 to 1,600,000 for the years 2009-2014, with several technical reports exceeding 40,000 downloads in those 5 years.
report recognizes the importance of R&D underlying forensic science and highlights several successful research investments and research initiatives under way (i.e., case studies). However, it falls short of analyzing trends and improvements and demonstrating how the agency’s investments contribute to progress toward agency goals. Box 3-1 (see above) identifies NIJ’s three current goals for forensic science: (1) quantifying the evidentiary value of forensic evidence, (2) developing tools and technologies, and (3) strengthening the scientific basis of forensic science disciplines. The quantitative information assembled by NIJ and presented in the 2015 Impact report—which includes funding distribution by discipline and total number of publications/presentations lumped together for a 6-year period (see National Institute of Justice, 2015a, p. 9)—is not easily interpreted into outcomes related to these three goals identified by NIJ for its forensic science program.
Strengthening NIJ concluded that NIJ did “not have any mechanisms in place for monitoring on a regular basis the impact of the research it funds” nor had it “adopted an assessment approach by qualified staff that integrates quantitative metrics . . . and qualitative reviews . . .” (National Research Council, 2010, p. 208). Since then, NIJ has taken steps to develop mechanisms and integrate them into its operations for the collection of information useful for assessing impact. However, much information and data have yet to be assembled. And NIJ has yet to demonstrate how it will use these data to advance its goals.
Conclusion 3-6: The National Institute of Justice has made some progress in accumulating measures for assessing the productivity of its funded researchers, but there is no evidence that measures are being collected to assess the impact on the practice of forensic science.
NIJ has the mandated roles to build and sustain a research infrastructure around forensic science and to foster improvement in the criminal justice field, working toward efficient and reliable collection and interpretation of forensic evidence. In this section the committee will review the changes NIJ has made to encourage the development of forensic science research, expand the number of people engaged in that research, and facilitate transition of new technologies and solutions to forensic science practitioners.
Strengthening NIJ observed that support and outreach from NIJ to researchers declined in the mid- to late 2000s. During this period, research funding decreased and research projects became a significantly smaller por-
tion of NIJ’s overall portfolio of activities. In addition, dissemination and outreach to the academic community declined; notably, the NIJ research briefs and the annual NIJ research conference were discontinued, and few efforts were made to ensure the accuracy and completeness of scholarly databases (e.g., the National Archive of Criminal Justice Data) (National Research Council, 2010, pp. 182-186).
More recently, NIJ has placed a greater emphasis on developing the infrastructure for research in forensic science. Some activities supporting this emphasis include the calls for new investigators in solicitations (discussed earlier), stronger encouragement for funded researchers to disseminate their results through journal publications and conference presentations, increased support for future researchers (e.g., graduate students), and an emphasis on improving access to databases and datasets for research purposes.
Some previous concerns about the lack of independence of NIJ (National Research Council, 2010) in the dissemination of research findings have been addressed by returning control of its publications to NIJ. Now, the Department of Justice Scientific and Integrity Policy states (pp. 9-10):
In keeping with National Research Council’s Principles and Practices for a Federal Statistical Agency [National Research Council, 2009a], the Bureau of Justice Statistics and the National Institute of Justice retain control over the timing and content of statistical and research reports and the press releases associated with them.22
OIFS has hired a forensic science writer to help disseminate results. The writer is contributing articles to the NIJ Journal and to the trade press, writing content for the NIJ website and social media, and producing a brochure for NIJ’s forensic science R&D program that can be distributed at events (e.g., major academic conferences). In addition, OIFS encourages research grantees to publish research findings in peer-reviewed journals and to present at conferences and meetings.23
23Presentation on OIFS Dissemination/Strategic Plan, by Gerry LaPorte and Danielle McLeod-Henning, OIFS, NIJ, to the Committee on Strengthening Forensic Science at the National Institute of Justice, May 7, 2015.
Before 2009, NIJ did not adequately support future researchers in areas relevant to criminal justice research. For example, just an annual average of three 1-year fellowships were awarded to graduate students (National Research Council, 2010, pp. 185-186). NIJ has increased its commitment to building the field through the Graduate Research Fellowship (GRF) Program. The number of GRF awards made across NIJ has increased considerably from an average of three awards per year to almost four times that number (11 awards were made in 2014). There are now two different programs: one focused on science, technology, engineering, and mathematics (STEM) and the other focused on the social and behavioral sciences. In 2014, three STEM awards (all with forensic science projects) and eight social and behavioral science awards were made. Each fellowship now provides larger allowances: up to 3 years of support over a 5-year period, pending satisfactory progress toward the doctoral degree and the availability of funds.24 Increased efforts to promote the GRF Program in order to attract a broader set of qualified candidates have paid off. From fiscal 2012 to 2014, OIFS received between five and seven applications for each year’s awards. For fiscal 2015, the number of GRF applications received increased over six-fold.25
One area of continued need is the creation of a broader set of databases across the forensic disciplines for practitioners and academics conducting research to use in developing and evaluating methods. Existing databases are narrowly focused on biometrics. The absence of sufficient research data is a significant barrier to advancing technologies. Recently, NIJ sponsored NIST to catalog available datasets and host a symposium (January 2015) that discussed the adequacy of existing publicly available datasets and future needs.26
Conclusion 3-7: As recommended in previous National Research Council reports, the National Institute of Justice has taken positive steps to expand outreach and dissemination to the research community and has significantly increased the number of graduate student fellowships.
24For each year of support, NIJ now provides an allowance of $35,000 to cover salary/stipend and related costs and up to $15,000 for tuition and fees, research expenses, and related costs, compared with about $20,000 of total annual support in the past. NIJ anticipates that up to $1 million will be awarded to those applicants chosen in fiscal 2015. For more information, see http://nij.gov/funding/fellowships/graduate-research-fellowship/Pages/grf-stem.aspx [June 2015].
25Presentation on OIFS Budget, by Gerry LaPorte, OIFS, NIJ, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015.
26See more information at http://www.nist.gov/forensics/biometric-research-datasets-event.cfm [June 2015].
This shows promise for broadening the field of researchers engaged in forensic science. However, the agency has not yet developed a set of metrics to assess the impact of its outreach efforts.
Technology transfer and transition of research findings into practice at forensic laboratories is a primary objective of NIJ’s forensic science research portfolio. In the past, NIJ made significant contributions to the technology needs of the criminal justice community in areas where there were national attention and dedicated sources of funding, such as with DNA technologies. However, a broader set of forensic science disciplines and needs were neglected (National Research Council, 2009b, 2010).
The Forensic Technology Center of Excellence (FTCoE) has been a key component of NIJ outreach in this area. Currently, the center is hosted by Research Triangle Institute and several academic partners. It received approximately $11 million from NIJ over 4 years from fiscal 2011 through 2014.27 FTCoE developed a technology transition management process that triages research projects to assess which merit further dissemination efforts. As a result of implementing this process, FTCoE has been able to (1) interact with principal investigators on the status and potential impact of transitioning their research to practice, (2) maintain a database on NIJ’s R&D awards with performance metrics, and (3) identify projects that could benefit from additional support. An initial review of fiscal 2009-2011 research grants (about 160 awards) led to 22 projects with “go” potential (i.e., proof of user interest and identification of logical next steps for broader dissemination), with 9 of those projects being identified to receive additional planning and support toward technology transition (National Institute of Justice, 2014a). FTCoE provides additional support in several ways:
- by facilitating stakeholder round tables to discuss commercial paths and the barriers to and plans for validation and development;
- by enabling technology assistance and validation to help test emerging technologies;
- by linking users to data from NIJ-funded studies; and
- by communicating, educating, and disseminating research results and technology improvements (e.g., webinars, training, publications, case studies).
27For more information, see http://www.nij.gov/funding/awards/pages/award-detail.aspx?award=2011-DN-BX-K564 [June 2015].
For example, FTCoE has developed a variety of publications and workshops, both for onsite delivery and web-based, to provide a range of options for its practitioner audience. The center has disseminated NIJ-funded research and delivered technology assistance and web-based technology-transfer workshops to over 25,000 registered practitioners during the period from October 1, 2011, through December 31, 2014. It also sends out a weekly newsletter to over 12,000 subscribers and has presented and exhibited at 53 national meetings and released 11 reports (National Institute of Justice Forensic Technology Center of Excellence, 2015).
Conclusion 3-8: The Forensic Technology Center for Excellence, funded by the National Institute of Justice (NIJ), has served an important role in (1) creating a database of the agency’s research projects to determine technologies in development that merit further investment, (2) delivering information and workshops to practitioners, and (3) maintaining a connection between the agency’s applied research portfolio and the practice of forensic science. However, the center’s efforts are reflected in neither a NIJ strategic research plan nor a NIJ strategic communications plan.
This section discusses the changes to NIJ’s resources—its budget, federal partnerships, and staffing for forensic science—in support of its research mission.
Currently NIJ has no budget appropriated specifically for forensic science research.28 NIJ supports its forensic science research portfolio by drawing funds from NIJ’s appropriated budget in three ways: (1) from NIJ’s base funding, (2) from funding set aside from OJP’s assistance programs for research and statistics in NIJ and the Bureau of Justice Statistics, and (3) from funding for the DNA backlog reduction and sexual assault forensic examination programs that can be used for related purposes. In fiscal 2009, appropriations language was adjusted to acknowledge that funds under the DNA backlog reduction program could be used for reducing the backlogs and for other forensic activities. This clause has provided NIJ the authority
28However, in fiscal 2009, 2010, and 2011, appropriators recommended that amounts of $2.5, $5, and $5 million from NIJ’s base funds, respectively, be directed to assist forensic and DNA activities. According to written response to Question 2 provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C, these funds supported R&D, primarily in the area of computer forensics.
Figure 3-2 shows the amounts appropriated for each of these funding streams from fiscal 2009 through 2014. Figure 3-3 shows the amounts used to fund forensic science research. Most of the funding to support forensic science R&D has been carved out of funds appropriated to support the DNA backlog reduction program. Funding for this program has declined since fiscal 2010; as a consequence, funding for forensic science research within NIJ has also declined. In theory, NIJ could choose to pull more funds for R&D, but the agency notes that is has to continually balance the priorities of policy makers and needs of the forensic laboratories with the research mission of the institute.30
Although amounts used to fund forensic science research have increased in recent years relative to what they were prior to 2009, these funds are still quite limited when compared with the needs of the forensic science community, and they are relatively small compared with the research dollars available for other fields. For example, funding for drug abuse research by the National Institute on Drug Abuse or for transportation research, development, and technology by the U.S. Department of Transportation is about $1 billion annually.31
In addition to funding external research awards, NIJ has worked closely with a variety of federal agencies over the past 5-7 years to support the field of forensic science as envisioned by Forensic Science: Path Forward (see recommendations in Appendix B of this report). These partners include NIST, NSF, the Federal Bureau of Investigation (FBI) Laboratory, the Defense Forensic Science Center, and the Bureau of Alcohol, Tobacco, Firearms and Explosives. In almost all of its interagency agreements or collaborations, NIJ has provided the funding for activities. Among these activities are the following (NIJ partners shown in parentheses):
- development of standard reference materials for a number of types of forensic evidence (NIST);
- training in novel DNA mixture analysis software (Defense Forensic Science Center);
29Presentation on OIFS Budget, by Gerry LaPorte, OIFS, NIJ, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015.
30Presentation on OIFS Budget, by Gerry LaPorte, OIFS, NIJ, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015. See also U.S. Government Accountability Office (2013).
31See http://www.drugabuse.gov/about-nida/legislative-activities/budget-information/fiscal-year-2014-budget-information-congressional-justification-national-institute-drug-abuse for the drug abuse research funding and http://www.rita.dot.gov/sites/rita.dot.gov.rdt/files/rdt_strategic_plan_2013.pdf for transportation research funding [June 2015].
|Expenditures Funding Sources||Funding (in millions) by Fiscal Year|
|Forensic Science R&D (OIFS)||DNA analysis and capacity enhancement||23.3||31.4||15.7||13.0||12.2||18.4|
|Post-conviction DNA testing||0||0||0||0||0||0|
|Sexual Assault Forensic Exam Program||0||0||0.5||0||0.3||0|
|NIJ base funds||0||0||0||0||0.3||2.5|
|NIJ base forensics carve-out||0||1.0||0||0||0||0|
|Research and statistics set-aside||0||0||0||1.9||2.4||2.3|
|Incoming reimbursable agreements/earmarks||0||0||0||0||0||0|
|Computer Forensics (OST)||DNA analysis and capacity enhancement||0.5||1.2||0||0||0||0|
|NIJ base forensics carve-out||2.9||3.9||3.9||0||0||0|
|Forensic-Related Social Science Research (ORE)||DNA analysis and capacity enhancement||2.4||0.7||2.5||0.8||0||0|
|Sexual Assault Forensic Exam Program||0||0||0||0||1.3||2.6|
|Technical Assistance (including FTCoE)||DNA analysis and capacity enhancement||0.2||6.8||8.5||2.7||6.5||6.9|
|NIJ base funds||0||0||0||0||0.9||0|
|NIJ base forensics carve-out||0||0||0.1||0||1.9||0|
|Incoming reimbursable agreements/earmarks||12.4||10.0||0||0||0||0|
|Training Development & Delivery||DNA analysis and capacity enhancement||13.8||9.4||4.6||0||0.5||0|
|Sexual Assault Forensic Exam Program||0||0||0.7||0||0||0|
|Incoming reimbursable agreements/earmarks||1.0||0||0||0||0||0|
|DNA Backlog and Other Backlog Casework||DNA analysis and capacity enhancement||89.5||84.9||93.3||85.2||76.3||72.3|
|Post-conviction DNA testing||10.2||2.3||7.4||3.5||3.3||3.6|
|Sexual Assault Forensic Exam Program||0||2.4||0||3.6||0.6||1.0|
|NIJ base funds||0||0||0||0||0||0|
|Incoming reimbursable agreements/earmarks||2.8||2.6||0||0||0||0|
|Laboratory Improvements||DNA analysis and capacity enhancement||3.7||0.2||0||0||0||0|
|NIJ base funds||0||0||0||0||0||0|
SOURCE: Committee created with data provided by NIJ.
NOTE: Funding sources are those related to NIJ’s forensic science portfolio in its OIFS; these represent most of the types of funding appropriated to NIJ but not all. Funding from these sources was also used toward dissemination, peer review, and other program support, which are not reflected in the table.
FIGURE 3-2 Funding for NIJ applicable to forensic science, fiscal 2009-2014, in millions of dollars. NIJ base = appropriated amounts for NIJ’s total base budget; R&S set-aside = total funds set aside for research and statistics across OJP’s programs; Lab programs = funds for programs to assist forensic laboratories, including DNA backlog reduction (DNA related and other forensics) and sexual assault forensic exam program.
SOURCE: Committee generated from data supplied by NIJ.
- expansion of DNA-related databases (Defense Forensic Science Center);
- a dual solicitation with NSF for Industry/University Cooperative Research Centers, bringing industry, university, and government organizations together to support the development of technology and research advances in forensic science (NSF);
- training at the National Firearms Examiner Academy and briefings on firearm- and toolmark-related technologies and techniques (Bureau of Alcohol, Tobacco, Firearms and Explosives); and
- an NIJ-FBI collaboration to process sexual assault kits through the FBI Laboratory with NIJ funding evaluations to gather knowledge to inform and improve training practices and testing protocols (FBI).
NIJ’s funding for some of these activities has derived from the appropriated funding for the DNA backlog reduction program, which can be used for DNA analysis and other forensic activities including research, and from
FIGURE 3-3 NIJ forensic science R&D funding by source, fiscal 2009-2014, in millions of dollars. Sources of funds include amounts from NIJ’s base budget (NIJ base); funds set aside for research and statistics across OJP’s programs (R&S set-aside); and funds for some programs to assist forensic laboratories, including DNA backlog reduction (DNA related and other forensics) and sexual assault forensic exam program (laboratory programs).
SOURCE: Committee generated from data supplied by NIJ.
the sexual assault forensic exam program. NIJ gave a total of $11 million to its federal partners (through interagency agreements) from fiscal 2009 to 2014 (National Institute of Justice, 2015a). In addition to those awards, NIJ was directed by Congress to transfer $16 million to NIST in fiscal 2012 through 2015.32 In fiscal 2012 and 2013, $5 million per year was directly transferred from NIJ’s base budget to NIST’s Office of Law Enforcement Standards. In fiscal 2014 and 2015, Congress appropriated $3 million to NIJ to transfer to NIST for support of the Organization of Scientific Area Committees (see Chapter 2), and an additional $1 million was given to NIJ for support of the National Commission on Forensic Sciences. From fiscal 2009 through 2014, NIJ has not been the recipient of funds transferred from other agencies in support of its forensic science R&D program except as a byproduct of nondiscretionary earmarks before fiscal 2011.33 Given NIJ’s limited funding, future partnerships (initiated by the agency or legisla-
32See Public Laws 112-55, 113-76, and 113-235.
33Written response to Question 2, provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C.
tors) that augment rather than further deplete NIJ resources would enhance the agency’s ability to advance forensic science R&D.
Conclusion 3-9: The lack of adequate and stable funding for forensic science at the National Institute of Justice from year to year contributes to the difficulty of establishing a long-term research agenda for forensic science.
Conclusion 3-10: As recommended in previous National Research Council reports, the National Institute of Justice (NIJ) has formalized partnerships with other federal agencies. Although these partnerships are commendable as they are currently executed, many rely solely on NIJ for funding and therefore further deplete the agency’s limited resources for funding its own projects.
Since OIFS oversees most of the forensic science research supported by NIJ, the committee reviewed the staffing in this office. OIFS oversees both forensic science research and the programs to enhance capacity in forensic laboratories (e.g., DNA backlog reduction, Paul Coverdell Forensic Science Improvement Grants Program). With the inclusion of the latter, OIFS manages more than half of the funds allocated to NIJ (see Table 3-2). Unlike ORE and OST, OIFS oversees both research and assistance grants.
According to recent organization charts, OIFS has fewer assigned staffing positions (physical scientists or analysts) than the other two program offices. In addition, the number of assigned staff positions does not appear to have changed from 2009, when there was a division of investigative and forensic science within OST, to the current organization in which forensic sciences has its own program office parallel to OST. Currently, OIFS uses a R&D team of four physical scientists to develop and manage one solicitation for forensic science R&D (the other offices usually have a number of R&D solicitations managed by individual physical or social scientists). This team approach was implemented in order to facilitate a R&D program of foundational science, applied science, method development, and technology development across a range of disciplines in forensic science (refer to Box 2-1 in Chapter 2) within the existing limitations on staff resources.34
34Written response to Question 1, provided by the NIJ’s OIFS, in response to committee’s questions found in Appendix C.
|Type of Activity||Subcategory||Percentage|
|Social Science (ORE)||
Science and Technology (OST)
Research and development
Technology assistance/test and evaluation
Investigative and Forensic Science (OIFS)
Forensic laboratory analysis and capacity enhancement
Research and development
Training and technical assistance
National missing and unidentified persons system
Dissemination, Outreach, and Program Support
Carve-Out for Section 215 Set-Aside for Research and Statistics
OJP Management and Administration
Direct Transfer to NIST Pursuant to Public Law 112-55
SOURCE: National Institute of Justice (2013).
Conclusion 3-11: The programmatic staffing for forensic science research has not changed commensurate with the increasing scope of responsibilities for the National Institute of Justice in this area.
Since 2009, as recommended by Strengthening NIJ, NIJ has made changes to its process for soliciting and awarding research grants, thereby better positioning the agency in developing and implementing a forensic science research agenda. Actions taken to (1) increase the level of autonomy and independence for its scientific peer-review process, (2) return grant sign-off authority to the NIJ director, and (3) return control to NIJ for the timing and content of its publications have restored authority and independence that is appropriate for a science agency. Recent activities and programs intended to support the work of graduate students in forensic science and to attract new investigators from a broader set of scientific disciplines to the forensic science field show promise toward building a research infrastructure necessary to develop and sustain research that advances forensic
science methods. The committee believes these efforts are well worth continuing, but that NIJ also needs to explore ways to evaluate and document the impact of its activities and programs, as an integral part of promoting a robust research infrastructure.
NIJ continues to involve its established Technology Working Groups in identifying needs of practicing forensic scientists. Gathering input from the practice community is important, especially considering that NIJ’s applied research portfolio is directed toward improving forensic science methods and analytic techniques at crime scenes or in forensic laboratories. In the past year, NIJ has made practitioner-identified needs more transparent to the research community through its forensic science R&D solicitation. However, the agency has yet to develop mechanisms for integrating the perspective of researchers into the process for identifying needs and scientific gaps and opportunities. Including researchers in an advisory capacity will enhance NIJ’s ability to prioritize research areas and develop short- and long-term research agendas.
Forensic Science: Path Forward and Strengthening NIJ both concluded that there was neither a long-term research agenda to help direct researchers nor a clear overarching strategy for forensic science research across the federal agencies. Today, there is still no publicly available strategic plan for forensic science research, and there needs to be one. The committee believes that great progress can be made in the field of forensic science if NIJ develops a strategic plan that includes short-and long-term research agendas and communication goals.
Most of the funding for forensic science research is drawn from and managed by NIJ, including support to other federal agencies for their research activities. Funds for forensic science research currently come from appropriated funding streams that have been unstable from year to year. In addition, this funding has been inadequate to meet the needs facing the forensic science field. Predictable and stable funding, as well as staffing commensurate with increasing responsibilities for forensic science research, would improve NIJ’s ability to establish appropriate short- and long-term research agendas for forensic science.
In conclusion, NIJ has made productive changes to the organization and operation of its R&D program since 2009. However, the committee believes the agency can do more to set priorities for forensic science research and advance the type of work that is needed to strengthen the scientific basis of forensic science.