4
Forensic Science Research at NIJ: A Blueprint for the Future
The previous chapters have established that the forensic science research and development (R&D) program of the National Institute of Justice (NIJ) has improved since 2009, but additional improvements are needed. This chapter contains the committee’s recommendations to NIJ and to other policy makers who influence criminal justice and forensic science policy. The recommendations are intended to provide a blueprint for the future to substantially improve the amount and quality of forensic science research supported by NIJ. Implementing them has the potential to produce tremendous improvement in the ability of the criminal justice system to collect, identify, preserve, and interpret the forensic evidence that is so central to a fair and evidence-based criminal justice system.
When these recommendations are fully implemented, the forensic science R&D program at NIJ will improve, which the committee believes is critically important for the future of the field. The committee also believes, consistent with findings in Forensic Science: Path Forward (National Research Council, 2009b), that there is a need to develop standards and protocols across forensic disciplines and establish accreditation and certification requirements for forensic laboratories and practitioners, among others. However, it is important to acknowledge that because this report focuses only on strengthening forensic science research at NIJ, the committee’s recommendations are not a panacea for the systemic problems facing forensic science (mentioned above; see also Chapter 1 and Forensic Science: Path Forward), many of which go beyond the need for a stronger scientific basis and are, ultimately, outside the scope of NIJ’s R&D mission.
SUPPORT FOR NIJ BY POLICY MAKERS
In order to implement the recommendations contained in this report, commitment and support will need to be expressed throughout the executive and legislative branches and particularly at all levels of the Department of Justice. Research from the field of implementation science suggests that implementing and sustaining change within an organization is dependent upon the support of influential organization leaders and commitment from external organizational partners, such as legislators and policy makers (Durlak and DuPre, 2008; van Achterberg et al., 2008; James Bell Associates, 2013). Given this, policy makers across the government will play an important role in building a supportive infrastructure that enables NIJ to improve its forensic science R&D program.
In addition, within NIJ, an identified internal champion for implementation of this report’s recommendations may need to be identified. Internal champions can keep improvements on track and build internal support for sustained change (Gold and Taylor, 2007; Kilbourne et al., 2007).
The development of a formal, comprehensive, and long-term strategic plan with articulated goals and timelines for achieving those goals is critical to any federal agency’s effectiveness. Unfortunately, the current priority-setting process for forensic science R&D at NIJ is opaque and ultimately insufficient to the task. While year-to-year flexibility allows NIJ to be responsive to the field as issues arise, it provides little predictability and stability for forensic science researchers and does not allow for the development of strategies to generate meaningful change in the field, which are necessarily long term. The establishment and management of a strategic plan for the forensic science R&D program with short- and long-term objectives would help create stability for the research community (and ultimately the practice and policy associated with forensic science) and provide a roadmap for critical advancements, while still allowing for creativity and innovation. Elements generally included in a R&D strategic plan are described in Box 4-1.
A strategic plan should outline specific short-term (1-2 years), mid-term (3-5 years), and long-term (6-10 years) goals and should be made publicly available in a timely fashion. These goals should guide all internal decision making regarding NIJ’s forensic science research agenda (including requests for funding, the development of research solicitations, and grant awards). Each goal identified in the strategic plan needs to be linked to outcomes. Specific elements of a research agenda designed to achieve those outcomes should be identified, and, at a minimum, the outcomes would need to in-
clude foundational research outcomes, technology transfer outcomes, laboratory efficiency outcomes, and justice system outcomes. The strategic plan can also include an “emergency response” contingency, so that the forensic science R&D program can remain responsive to new, special concerns that arise unexpectedly. Targets for this emergency response would need to be special projects that complement, rather than drive, the long-term research agenda.
The Technology Working Groups (TWGs) are currently the only mechanism through which NIJ identifies the needs of the forensic science field, and those needs appear to have a major influence on NIJ’s forensic science research priorities. Although the TWGs serve an important function, they do not adequately represent the needs of the broad range of forensic science disciplines and should not be the sole mechanism NIJ uses to gather input from the forensic field to establish its research agenda. In addition, it would be useful at this time to integrate the perspectives of researchers into the needs-gathering and priority-setting process in order to more efficiently and effectively identify scientific gaps and opportunities and develop an overarching forensic science research agenda. Establishing and maintaining a research advisory board comprising a broad array of scientists, including forensic science researchers and an expanded range of practitioners, would assist NIJ leadership and qualified forensic science staff in identifying and prioritizing the research needs to be used in developing a strategic plan for its forensic science program. Such a group should report to the NIJ director and provide oversight by monitoring progress toward achieving the goals identified in the strategic plan.
The committee did not develop, nor was it tasked to develop, a forensic science research agenda for NIJ, but it is aware of activities currently being undertaken at the American Association for the Advancement of Science and at the National Institute of Standards and Technology through the Organization of Scientific Area Committees that may identify gaps and priority areas for research in a number of the forensic disciplines.1 These recommendations could be vetted by the research advisory board recommended here and, if appropriate, integrated into its research agenda and strategic plan. It may also be appropriate for the research advisory board to consider additional inputs from other forensic science stakeholders.
The development and promulgation of a strategic plan for forensic
______________
1Personal communication between Dr. Tania Simoncelli, Assistant Director for Forensic Science, White House Office of Science and Technology Policy, and Dr. Alan Leshner, chair, Committee on Strengthening Forensic Science at the National Institute of Justice, on May 19, 2015. See also http://www.aaas.org/page/forensic-science-assessments-quality-and-gap-analysis [June 2015]; and the presentation on Standards and Practices of Forensic Science from a NIST Perspective, by John Butler and Mark Stolorow, NIST, to the Committee on Strengthening Forensic Science at the National Institute of Justice, April 1, 2015.
BOX 4-1
Elements of a Research and Development (R&D) Strategic Plan
A meaningful and effective strategic plan for a R&D program must be consistent with the mission statement of the organization. Such a plan will outline what the program does, for whom the program executes its work, and the path by which the program will attain its goals. It needs to describe the primary purposes, topics, expected outcomes, and anticipated resources available for carrying out the plan, including R&D funding (National Research Council, 2012; Transportation Research Board, 2013).
In general, it should
- 1. Articulate the role and value of R&D. The plan should measure the value of the research program and detail how the program supports the broad organization’s priorities, highlighting past successes and future prospects. A plan may explain how research supports a congressionally mandated mission as well as how research helps ensure cost-effective investment of federal resources in order to make a strong case about the value of a particular program’s activities.
- 2. Highlight and promote ways to overcome constraints to strategic R&D investment. The plan needs to identify administrative and institutional hurdles to strategic research investment and suggest ways to overcome these constraints.
science R&D at NIJ will signify to stakeholders—researchers, practitioners, policy makers, and the public—the agency’s dedication to advancing forensic science.
Recommendation 4-1: The National Institute of Justice (NIJ) should take immediate steps to develop a formal and comprehensive strategic plan for its forensic science research and development program. The strategic plan should be based on a thorough understanding of the state of the science, an analysis of NIJ’s past and current research portfolios, and extensive consultation with both the research and practice communities.
Recommendation 4-2: The National Institute of Justice should establish a research advisory board that includes a broad array of scientists,
- 3. Describe the R&D program in various dimensions to inform decisions. The plan should inform decisions about priority topics that research should address and about the nature of the research that is needed to address these topics. It may provide information about allocation of past and current R&D funding toward the outlined goals, underscoring the program’s priorities.
- 4. Identify gaps in research. The plan needs to identify gaps in research, by topic and type, and determine important areas for future research. It may use these identified gaps to inform policy makers about important areas that are being neglected for want of resources. The plan should also provide substantive details regarding the nature of collaborations with other research organizations that perform similar work and may be relied upon to fill these gaps.
- 5. Promote efficient and effective research processes. The plan should provide information about the development of research processes that would improve the performance of the research program. This may include information about stakeholder input and financial information. It should designate performance measures specific to the research objectives and may also include an outcome-based assessment of previous strategic plans that describes how well the program has performed against its goals (Transportation Research Board, 2013).
including forensic science researchers and practitioners, in order to better integrate their perspectives into its processes for identifying and prioritizing research needs. The research advisory board should also monitor progress in achieving the strategic plan’s designated goals.
BUILDING A RESEARCH INFRASTRUCTURE
NIJ needs to be critically concerned with building the research infrastructure for forensic science, including broadening the forensic science research community and supporting the next generation of forensic science researchers. Bolstering the research infrastructure could also include internal efforts, such as building institutional knowledge of emerging, relevant technologies in related fields that may have forensic uses. In addition, NIJ
should integrate research and evaluation into all of its forensic science investments.
NIJ has conducted outreach to the broader scientific community (whose work may have a nexus with forensic science) through the dissemination of proposal solicitations, and historically it has made appearances at adjacent disciplines’ scientific meetings. However, budget and travel constraints have limited NIJ’s ability to engage prospective cross-field collaborators directly. There is a need for novel approaches to recruit collaborators from both the typical, current disciplines and from new areas.
NIJ should attend or have a presence at professional conferences and meetings that attract audiences from disciplines that have typically been outside the forensic disciplines. It can enlist funded researchers to participate in local and national outreach events at meetings, universities, and public forums where forensic science needs, opportunities, and challenges are shared. To facilitate these efforts, the agency could provide materials (e.g., PowerPoint slides) to NIJ-funded scientists to use in their own presentations that briefly outline NIJ’s forensic science R&D interests and portfolio.
In addition, NIJ’s efforts to support the next generation of forensic science researchers could include not only fellowships for doctoral students but also dissertation grant programs or funding for the inclusion of forensic science graduate students in projects led by senior scientists.
In order to build the research infrastructure, NIJ—in accordance with its scientific mission—needs to incorporate research and evaluation into all of its forensic science activities. Currently, the financial investment in programs to reduce casework backlogs and fund improvements in forensic laboratories significantly outweighs the investment in research (see Table 3-1). Historically, these programs have directly supported the purchase of equipment, training, and additional staffing (Nelson, 2010). Especially in light of shrinking resources and increased demand for services, NIJ needs a capability to invest in innovative scientific research that promises to produce increases in capability by orders of magnitude through both technology and method.
NIJ needs to structure these programs in a way designed to learn from these efforts and promote efficiency in laboratories on a large scale. NIJ is designated as the agency to distribute funding for these programs, and given its science mission, it could require these and other assistance grants to include a research component with the potential to bring marked increases in casework processing and accuracy and/or an evaluation component that will help provide an evidence base that could be used to improve the outcomes of future efforts. Though these added components may require additional resources per grant, they are critical to improving outcomes and may, in the long run, reduce the funds required to support laboratory capacity
and improvement. Future funding distributions could be contingent upon laboratories’ use of evidence-based practices.2 This requirement would better integrate NIJ’s R&D program with its assistance efforts.
Recommendation 4-3: The National Institute of Justice (NIJ) should increase efforts to expand forensic science research by recruiting researchers from the broader scientific community whose work may have a nexus with forensic science. At a minimum, NIJ should promote greater cross-field collaboration, conduct more outreach to research communities in adjacent disciplines that do not currently focus on forensic science applications, and increase the institutional knowledge within NIJ of relevant technology developments in other fields that might have forensic uses.
Recommendation 4-4: In keeping with its scientific mission, all of the National Institute of Justice’s (NIJ’s) forensic science funding, including capacity-building investments such as backlog reduction, should include a research component and/or an evaluation component. NIJ should create a clear translation pipeline from research to implementation for promising approaches, and future capacity-building funding should be tied to the use of evidence-based practices.
As stated in Chapter 1 and in prior National Research Council reports, given the challenges currently facing forensic science laboratories and practitioners, there is a national need for a sustained and extensive program of forensic science research to improve the validity, reliability, and breadth of forensic tools available to the criminal justice system. Regrettably, the committee believes that the current level of federal funding available to NIJ to support forensic science R&D is sorely inadequate to the task. The varying and unpredictable funding levels from year to year are also counterproductive to the agency’s goals.
One of the hallmarks of the success of other federal research funding agencies is stability in their budgets and consistency in the kinds of research they support, even if specific priorities do vary over time. Stable and predictable funding enables agencies to build and sustain a research infrastructure: it encourages talented students to pursue scientific careers,
______________
2Programs and practices are generally considered to be evidence based when their effectiveness has been demonstrated by causal evidence, generally obtained through high-quality outcome evaluations. See http://ojp.gov/funding/Explore/SolicitationRequirements/EvidenceResearchEvaluationRequirements.htm [September 2015].
keeps established researchers engaged over a career, and attracts and retains talent (National Research Council, 2014, p. 2). Therefore, funding stability, at least at some core level, would be a critical element of any effective strategy going forward.
The funds available to NIJ to support forensic science R&D should be sufficient to enable the agency to strengthen forensic science research and practice at the local, state, and federal levels. To achieve this end, policy makers will need to designate a dedicated funding stream for R&D that is of sufficient magnitude. This dedicated funding stream would have to be adequate, and stable appropriations would have to be coupled with funding flexibility to help support both short- and long-term research strategies. The addition of a dedicated funding stream for R&D would demonstrate the importance of research to strengthening forensic science and convey policy makers’ prioritization and recognition of the urgent needs facing the forensic science field.
To complement these necessary budgetary actions from federal policy makers, NIJ also could ensure that some of its formalized partnerships with other federal agencies augment its limited resources. NIJ can explore ways to bolster its resources, in both size and stability, by strategically leveraging funds through partnerships with other federal agencies.
Recommendation 4-5: Federal policy makers should ensure the ability of the National Institute of Justice to advance forensic science research and development through dedicated, adequate, and stable appropriations coupled with funding flexibility to help support both short- and long-term research strategies. In order to ensure funding stability from year to year, policy makers should designate a dedicated funding stream for research and development that is of sufficient magnitude to address the challenges facing forensic science.
Federal science agencies can use strategic communication as a tool to promote their investment in R&D to policy makers and the general public in order to ensure that their work is valued and supported. NIJ needs to develop a strategic communication plan for its forensic science R&D program that proactively communicates the importance of forensic science research to the advancement of a fair and effective justice system, demonstrates successes and future objectives, and estimates future savings from the creation and adoption of innovative tools and techniques. A strategic communication plan can also help the agency achieve its goal of advancing forensic science by (1) encouraging the uptake of innovative evidence-based practices by forensic science practitioners and other criminal justice stake-
holders and (2) more actively recruiting researchers from related disciplines to engage with the forensic science community of practice.
NIJ has made substantial progress in its ability to disseminate research findings to the forensic science field. It encourages its grantees to communicate their research findings by publishing in peer-reviewed forensic science journals and presenting at conferences, and the agency disseminates promising findings to practitioners and researchers through online communications. However, strategic communication needs to involve more than simply disseminating research findings to the field through traditional mechanisms such as those described above.
NIJ will need to strategically communicate its forensic science mission, vision, and strategic plan to a broad audience, including researchers, practitioners, policy makers, and the public. To do so, it can engage forensic science professional organizations, such as the American Academy of Forensic Sciences and the Society of Forensic Toxicologists, and other prominent stakeholders to distribute research findings and promote the agency’s goals, activities, and accomplishments. In addition, federal officials need to recognize the value of in-person meetings and allow NIJ to convene its stakeholders through national conferences to promote the agency’s goals and investments in forensic science R&D.
NIJ should serve as a clearinghouse for evidence-based solutions to persistent forensic science problems and facilitate knowledge transfer between field leaders and new adopters. The Forensic Technology Center of Excellence has served an important role in this area. However, the center’s efforts have not been aligned with a NIJ strategic R&D plan or a NIJ strategic communication plan. NIJ’s communication efforts will need to encourage and facilitate adoption of evidence-based practices, identified and validated through research, to advance the field of forensic science. The agency may look to the Department of Defense’s Multidisciplinary University Research Initiatives or the National Science Foundation’s Small Business Innovation Research/Small Business Technology Transfer programs for additional models to consider.3
Recommendation 4-6: In concert with the development of a strategic plan, the National Institute of Justice should develop and implement a strategic communication plan that directs its messages in ways appropriate to its various constituencies. This plan should include valuable in-person activities, such as hosting national conferences and workshops.
______________
3See http://www.nsf.gov/eng/iip/sbir/home.jsp and http://www.onr.navy.mil/ScienceTechnology/Directorates/office-research-discovery-invention/Sponsored-Research/UniversityResearch-Initiatives/MURI.aspx [September 2015].
Recommendation 4-7: As part of its strategic plan, the National Institute of Justice should support transfer of technologies developed in its research and development portfolio to end users.
Evaluating impact and subsequently communicating that impact is necessary to encourage support for an agency’s activities and to establish and communicate the value of research investments. An assessment of impact should aim to inform by demonstrating the benefits of supported research (giving consideration to what would not have happened if the supported research had not been conducted), show that resources have been used efficiently and effectively, and provide direction for future research (Guthrie et al., 2013).
NIJ currently does not have adequate mechanisms to evaluate and communicate the impact of its forensic science research portfolio. It has recently made strides in this area: NIJ now routinely tracks publications and presentations, dissemination activities, and technology-transition activities that result from its supported research. However, in order to fully assess the impact of its forensic science R&D program, NIJ will need to develop a set of metrics that go beyond primarily tracking outputs to a process that also measures the outcomes that result from the activities it supports (e.g., increased accuracy of particular forensic methods, the use of NIJ-sponsored research to set legal precedent, and the implementation of new methods and techniques in laboratories).4 The metrics, measuring both outputs and outcomes, must then be used to routinely evaluate NIJ’s impact. A 2012 National Research Council report provides examples of some outputs and outcomes that a R&D program could measure; these are included in Box 4-2.
Given NIJ’s mission, it would be beneficial to measure impact on criminal justice system outcomes like suspect identification, arrest, case clearance, and convictions, especially in those places that have demonstrated gains in competencies, productivity, and/or efficiency. Outcome measures that show trends, such as increases in the reproducibility, reliability, and accuracy of forensic analyses, would also be critical indicators.
The committee recognizes that measuring impact will be a challenging task. Attempts to trace major innovations back to their original supporting research rarely reveal a direct flow from original investment to final impact
______________
4The committee defines outputs as activities or accomplishments delivered by research programs, such as papers published or methods validated. It defines outcomes as the benefits resulting from a research program (both short and long term); see also National Research Council (2008b, p. 4).
BOX 4-2
Examples of Output and Outcome Measures
Outputs
- Publications, patents, reports, and the citations garnered
- Technical assistance provided to end users, customers, and stakeholders
- Invited presentations (e.g., conferences and workshops)
- Training and mentoring of personnel
- New and improved products, materials, and processes
- Patents leading to new products
- Development of test and evaluation protocols, codes, and standards
- Technology transfer
- Maintained competencies
- New competencies
Outcomes
- Cost savings (e.g., in materials or processes)
- Increased productivity
- Increased efficiency
- Safety practices and culture
- Effectiveness of management structure and strategy
- Recognition of the R&D organization as best, among the best, or unique
SOURCE: National Research Council (2012, p. 34).
but rather a complex and tangled path (Martin and Tang, 2006; National Research Council, 2014). Before considering the best metrics, methods, and indicators to use and how to use them, NIJ will first need to determine what is to be measured and why. This determination (that is, which outputs and outcomes to measure) should flow from the strategic plan recommended above and should allow for comparison of the portfolio to the agency’s mission and the community’s expressed needs in terms of both substance and priorities (National Research Council, 2012). These decisions need to be made in consultation with those who have formal evaluation research expertise both internal and external to NIJ.
In general, measures should include leading and lagging indicators that can be measured beyond the award period, as there can be a considerable time lag between completed research and impact. They should be both quantitative and qualitative, consider failures and successes, and enable analysis of researchers and clusters of researchers, so that intangible factors such as opportunities and relationships that spur innovation and discovery may be considered (Lane and Bertuzzi, 2011; National Research Council, 2011, 2012, 2014).
In order to improve its ability to measure the impact of its R&D program, NIJ may also consider requiring laboratory assistance grantees to capture and report specific outcomes as part of its new performance management system for the DNA Capacity Enhancement and Backlog Reduction Program and to structure implementation so that necessary control situations exist.5 NIJ could evaluate those measures to identify the impact of previous research (i.e., if research findings—such as more efficient processes—are implemented into practice).
Recommendation 4-8: The National Institute of Justice should develop an appropriate set of procedures and metrics to measure outcomes regularly and evaluate the impact of its forensic science research and capacity-building portfolio.
These recommendations are designed to strengthen the role, capacity, and commitment of NIJ as a major science agency whose role is to improve forensic science research. However, NIJ will only be successful if it is given broad support by both the executive and legislative branches of government. This is particularly important throughout the Department of Justice, given that NIJ’s placement within the department has been perceived as a potential source of conflict of interest in the past (National Research Council, 2009b, pp. 79-80).
If these recommendations are fully implemented and any potential barriers overcome, the committee believes there will be tremendous improvement in the criminal justice system’s ability to collect, identify, preserve, and interpret forensic evidence. As a result, the public’s confidence in a fairer and more credible criminal justice system will be enhanced.
______________
5A counterfactual situation could be experimental (a randomly assigned control group) or quasi-experimental (a comparison group). For examples, see http://betterevaluation.org/plan/understandcauses/compare_results_to_counterfactual [September 2015].
