7
Strengthening Oversight of Forensic Science Practice

Several commentators appearing before the committee noted that nearly anyone with a garage and some capital theoretically could open a forensics laboratory and start offering services. Although this might be a bit hyperbolic, the fact is that there are no requirements, except in a few states (New York, Oklahoma, and Texas), for forensics laboratories to meet specific standards for quality assurance or for practitioners to be certified according to an agreed set of standards.1 Well-publicized problems in large crime laboratories have uncovered systematic deficiencies in quality control. For example, in 2002, the Houston Police Department Crime Laboratory and Property Room came under scrutiny because of a range of quality concerns that created “profound doubts about the integrity of important aspects of the criminal justice system in Harris County.”2 Problems included poor documentation, serious analytical and interpretive errors, the absence of quality assurance programs, inadequately trained personnel, erroneous reporting, the use of inaccurate and misleading statistics, and even “drylabbing” (the falsification of scientific results).3 In most cases, existing efforts

1

See N.Y. EXEC. § 995-b (McKinney 1996); (accreditation by Forensic Science Commission); OKLA. STAT. ANN. tit. 74 § 150.37 (requiring accreditation by the American Society of Crime Laboratory Directors/Laboratory Accreditation Board or the American Board of Forensic Toxicology); TEX. CRIM. PROC. CODE art. 38.35 (accreditation by the Department of Public Safety).

2

M.R. Bromwich. 2007. Final Report of the Independent Investigator for the Houston Police Department Crime Laboratory and Property Room. June 13. Available at www.hpdlabinvestigation.org, p. 1.

3

Ibid.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 193
7 Strengthening Oversight of Forensic Science Practice Several commentators appearing before the committee noted that nearly anyone with a garage and some capital theoretically could open a forensics laboratory and start offering services. Although this might be a bit hyperbolic, the fact is that there are no requirements, except in a few states (New York, Oklahoma, and Texas), for forensics laboratories to meet specific standards for quality assurance or for practitioners to be certified according to an agreed set of standards.1 Well-publicized problems in large crime laboratories have uncovered systematic deficiencies in quality control. For example, in 2002, the Houston Police Department Crime Laboratory and Property Room came under scrutiny because of a range of quality concerns that created “profound doubts about the integrity of important aspects of the criminal justice system in Harris County.”2 Problems included poor documentation, serious analytical and interpretive errors, the absence of quality assurance programs, inadequately trained personnel, erroneous reporting, the use of inaccurate and misleading statistics, and even “drylab- bing” (the falsification of scientific results).3 In most cases, existing efforts 1 See n.Y. exeC. § 995-b (McKinney 1996); (accreditation by Forensic Science Commis- sion); okLa. stat. ann. tit. 74 § 150.37 (requiring accreditation by the American Society of Crime Laboratory Directors/Laboratory Accreditation Board or the American Board of Forensic Toxicology); tex. Crim. ProC. Code art. 38.35 (accreditation by the Department of Public Safety). 2 M.R. Bromwich. 2007. Final Report of the Independent Inestigator for the Houston Police Department Crime Laboratory and Property Room. June 13. Available at www. hpdlabinvestigation.org, p. 1. 3 Ibid. 

OCR for page 193
 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES to impose standards and best practices in forensic science practice rely on the voluntary participation of some members of the forensic science com- munity working diligently to improve overall quality in the field. Despite important movement in recent years toward developing and implementing quality control measures in the forensic science disciplines, a lack of uniform and mandatory quality assurance procedures, combined with some highly publicized problems involving large crime laboratories, has led to heightened attention to efforts to remedy uneven quality among laboratories through the imposition of standards and best practices. The American Bar Association has recommended that, “Crime laboratories and medical examiner officers should be accredited, examiners should be certi- fied, and procedures should be standardized and published to ensure the validity, reliability, and timely analysis of forensic evidence.”4 In Daubert . Merrell Dow Pharmaceuticals,5 the Supreme Court cited as a relevant factor in assessing expert testimony the “existence and main- tenance of standards controlling the technique’s operation.” Standards and best practices create a professional environment that allows organizations and professions to create quality systems, policies, and procedures and maintain autonomy from vested interest groups. Standards ensure desir- able characteristics of services and techniques such as quality, reliability, efficiency, and consistency among practitioners. Typically standards are enforced through systems of accreditation and certification, wherein inde- pendent examiners and auditors test and audit the performance, policies, and procedures of both laboratories and service providers. In addition, re- quirements for quality control can be imposed on entities receiving federal funds, and professional groups can develop codes of ethics and conduct to serve as measures against which performance can be assessed. This chapter addresses some of the traditional approaches used by technical professions to enhance the quality of performance—accreditation, certification (including proficiency testing), and oversight—tied to federal funding. In each approach, standards are used to measure the quality of institutions or organizations, either in terms of their policies and proce- dures or in terms of the proficiency and skills of an individual practicing the discipline. However, as mentioned above, with the exception of three states mandating accreditation (New York, Oklahoma, and Texas), the ac- creditation of laboratories and certification of forensic examiners remains voluntary. 4 American Bar Association. 2006. Report of the ABA Criminal Justice Section’s Ad Hoc Innocence Committee to Ensure the Integrity of the Criminal Process. Achieing Justice: Freeing the Innocent, Conicting the Guilty. P.C. Giannelli and M. Raeder (eds.). Chicago: American Bar Association. 5 509 U.S. 579 (1993).

OCR for page 193
 STRENGTHENING OVERSIGHT ACCREDITATION Accreditation is just one aspect of an organization’s quality assurance program, which also should include proficiency testing where relevant, continuing education, and other programs to help the organization provide better overall services. In the case of laboratories, accreditation does not mean that accredited laboratories do not make mistakes, nor does it mean that a laboratory utilizes best practices in every case, but rather, it means that the laboratory adheres to an established set of standards of quality and relies on acceptable practices within these requirements. An accredited labo- ratory has in place a management system that defines the various processes by which it operates on a daily basis, monitors that activity, and responds to deviations from the acceptable practices using a routine and thoughtful method. This cannot be a self-assessing program. Oversight must come from outside the participating laboratory to ensure that standards are not self-serving and superficial and to remove the option of taking shortcuts when other demands compete with quality assurance. In addition, accredi- tation serves as a mechanism to strengthen professional community ties, transmit best practices, and expose laboratory employees directly to the perspectives and expectations of other leaders in the profession. An example of a strong accreditation system is that required through the Clinical Laboratory Improvement Amendments of 1988 (CLIA).6 Through this legislation, the Centers for Medicare & Medicaid Services (CMS) regu- lates all clinical laboratory testing (except research) performed on humans in the United States. In total, CLIA covers approximately 189,000 labora- tory entities (see Box 7-1). Some key elements of CLIA and of other accreditation programs that might be incorporated into a mandatory accreditation system for forensic science include: • national organization that can mediate the accreditation process; a • n application process with criteria by which organizations are a eligible to apply; • process of self-evaluation; a • n external evaluation process, including site visits by external a evaluators; • an appeals process; • a repeat cycle of evaluation and external evaluation, and; • a set of standards by which entities can be evaluated.7 6 42 U.S.C. § 263a. 7 Institute of Medicine. 2001. Presering Public Trust: Accreditation and Human Research Participation Protection Programs. Washington, DC: National Academy Press.

OCR for page 193
 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES Box 7-1 Clinical Laboratory Improvement Amendments of 1988 (CLIA) The objective of the CLIA program is to ensure quality laboratory testing. All clinical laboratories must be properly certified to receive Medicare or Medicaid payments. CLIA requires all entities that perform even one test using “materials derived from the human body for the purpose of providing information for the diagnosis, prevention or treatment of any disease or impairment of, or the assess- ment of the health of, human beings” to meet certain federal requirements. If an entity performs tests for these purposes, it is considered to be covered by CLIA and must register with the CLIA program. CMS and CDC develop standards for laboratory certification (it is actually a certificate of accreditation). In addition, CDC conducts studies and convenes con- ferences to help determine when changes in regulatory requirements are needed. Oversight is conducted through onsite inspections of laboratories conducted every two years using federal surveyors or surveyors of deemed organizations or state- operated CLIA programs approved for this purpose. Oversight includes a compre- hensive evaluation of the laboratory’s operating environment and personnel, as well as its proficiency testing, quality control, and quality assurance procedures. The laboratory director plays a critical role in assuring the safe and appropriate use of laboratory tests—he or she must meet required qualifications and must ensure that the test methodologies selected are capable of providing the quality of results required for patient care. Laboratory directors are required to take specific actions to establish a comprehensive quality assurance program. Six organizations are deemed to offer accreditation of laboratories for CLIA. An accreditation organization that applies or reapplies to CMS for deeming author- ity, or a state licensure program that applies or reapplies to CMS for exemption from CLIA program requirements of licensed or approved laboratories within the state, must provide extensive documentation of its process. This includes a detailed description of the inspection process, a description of the steps taken to monitor the correction of deficiencies, a description of the process for monitoring performance, procedures for responding to and for the investigation of complaints against its laboratories, and a list of all its current laboratories and the expiration dates of their certification. CLIA also provides for sanctions that may be imposed on laboratories found to be out of compliance with one or more of the conditions of accreditation (e.g., unsuccessful participation in proficiency testing). These include suspension, limi- tation, or revocation of the certificate; civil suit to enjoin any laboratory activity that constitutes a significant hazard to the public health; and imprisonment or fine for any person convicted of the intentional violation of CLIA requirements. The regula- tions also require that the Department of Health and Human Services Secretary annually publish a list of all laboratories that have been sanctioned during the preceding year. Sanctions can be appealed. SOURCE: www.cms.hhs.gov/clia/.

OCR for page 193
 STRENGTHENING OVERSIGHT In addition, accrediting organizations typically offer education and training programs to help the participating entities comply with the stan- dards. Accreditation cannot guarantee high quality—that is, it cannot guard against those who intentionally disobey or ignore requirements. However, over time it can reduce the likelihood that violations will occur, and reports of infractions should trigger increased scrutiny by an accrediting body. And, by requiring that education be a standard that must be met as a condition of accreditation, incremental change and quality improvement can be achieved individual by individual. Development of Current Forensic Laboratory Accrediting Organizations In the 1970s, FBI Director Clarence Kelley and FBI Laboratory Director Briggs White organized a group of crime laboratory directors that eventu- ally became known as the American Society of Crime Laboratory Directors, or ASCLD. ASCLD’s Committee on Laboratory Evaluation and Standards was focused on developing quality assurance standards, and in 1981 the ASCLD/Laboratory Accreditation Board (ASCLD/LAB) was formed. In 1988, it was officially incorporated as a not-for-profit organization. In 1994, the passage of the DNA Identification Act established a DNA Advisory Board (DAB) to develop and enforce quality assurance standards for crime laboratories seeking access to the FBI’s national database of DNA profiles (see below). The DAB recommended that crime laboratories seek accreditation as quickly as possible. According to the Crime Lab Report, “Because ASCLD/LAB policies and procedures would not allow accredita- tion to be awarded to a single work unit, laboratories that were not pre- pared to undergo a full ASCLD/LAB accreditation assessment seemed to have no other alternative but to forfeit access to the DNA database until they were ready for a full accreditation audit.”8 In 1995, the private not-for-profit corporation National Forensic Sci- ence Technology Center (NFSTC) was formed by the ASCLD executive board for training, education, and support of accreditation.9 NFSTC could support and assist crime laboratories preparing for a full ASCLD/LAB accreditation as well as audit and temporarily certify DNA units that complied with DNA-specific quality assurance standards.10,11 NFSTC sub- sequently formed a new independent accreditation corporation, Forensic Quality Services (FQS), with the idea that its program would be based on 8 Crime Lab Report. December 20, 2007. Available at www.crimelabreport.com/monthly_ report/12-2007.htm. 9 See http://nfstc.org/aboutus/history/history.htm. 10 Ibid. 11 DNA procedures are regulated under the DNA Identification Act of 1994. DNA Identi- fication Act of 1994, 42 U.S.C. § 14132 (1994).

OCR for page 193
 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES the new ISO/IEC 17025 international standard for testing and calibration laboratories.12 In 2003, the ASCLD/LAB Delegate Assembly approved the implemen- tation of an ISO/IEC 17025 program, and ASCLD/LAB began offering these accreditations in April 2004. Accreditations for forensic science labo- ratories are now conducted using General requirements for the competence of testing and calibration laboratories 0 ISO/IEC (2005),13 the same requirements under which private and public laboratories are accredited. The international standards are developed through technical committees to deal with particular fields of technical activity. In order for sector spe- cific requirements for forensic laboratories to be addressed, ISO allows for the amplification of requirements or supplemental requirements, such as ASCLD/LAB-International Supplemental requirements for the accredita- tion of forensic science testing laboratories (2006). ASCLD/LAB’s areas of focus are laboratory management and opera- tions, personnel qualifications, and the physical plant. The following must be in place for accreditation: • rocedures to protect evidence from loss, cross-transfer, contamina- p tion, and/or deleterious change; • validated and documented technical procedures; • the use of appropriate controls and standards; • calibration procedures; • complete documentation of all evidence examination; • documented training programs that include competency testing; • technical review of a portion of each examiner’s work product; • testimony monitoring of all who testify; and • a comprehensive proficiency testing program.14 The ASCLD/LAB accreditation cycle is five years, with annual reports required from each accredited laboratory that consist of any changes in management, staff, facilities, methodologies, proficiency testing, and testi- mony monitoring. All accredited laboratories must maintain written cop- ies of appropriate technical procedures, including descriptions of sample preparation methods, controls, standards, and calibration procedures, as well as a discussion of precautions, sources of possible error, and literature references. In addition, ASCLD/LAB has a policy regarding the reporting of noncompliance with requirements, a portion of which is excerpted below: 12 See www.forquality.org. 13 See www.iso.org/iso/catalogue_detail?csnumber=39883. 14 R. Stacey, President, ASCLD/LAB. Presentation to the committee. January 25, 2007.

OCR for page 193
 STRENGTHENING OVERSIGHT In keeping with the stated objective of ‘identifying those laboratories which meet established standards,’ the ASCLD/LAB Board has determined that, as an accrediting body, we must be timelier in reviewing instances of significant non-compliance. To further this objective, all accredited laboratories must disclose to ASCLD/LAB all substantive occurrences of non-compliance within 30 calendar days of determining that the non- compliance has occurred.15 In addition to this particular requirement, the ISO program has a re- quirement for an annual surveillance visit. During this site visit, any issues that may have come to the attention of ASCLD/LAB and/or requirements selected by ASCLD/LAB are reviewed. The accreditation programs are managed by a paid staff member working under the direction of a board of directors, which is elected by the Delegate Assembly. The Delegate As- sembly is composed of the directors of all accredited laboratories and labo- ratory systems. Inspectors must complete a training program and must be employed in an accredited laboratory. At any time, if an issue is brought to the attention of ASCLD/LAB, the board of directors can, after determining that the claim is substantive, implement an interim inspection of that par- ticular issue and the entire laboratory. The program also includes a system of sanctions and an appeal process. Status of Accreditation ASCLD/LAB’s international program has accredited 60 laboratories as of April 2008, in addition to 337 laboratories accredited under the origi- nal Legacy program.16 FQS-International (FQS-I) has accredited just over 50 laboratories in one or more disciplines; however, FQS-I allows forensic laboratories to customize their accreditation by phasing in one discipline at a time.17 A survey of International Association for Identification (IAI) mem- bers, who tend to work in settings other than traditional crime laboratories, revealed that only 15 percent of respondents are accredited.18 Only a few jurisdictions require that their forensics laboratories be accredited. According to the 2005 census of 351 publicly funded crime laboratories, more than three-quarters of laboratories (78 percent) were 15 2008 version of the ASCLD/LAB Legacy Accreditation Manual. 16 See www.ascld-lab.org/legacy/aslablegacylaboratories.html. 17 See www.forquality.org/fqs_I_Labs.htm. 18 T.S. Witt. Director, Bureau of Business and Economic Research, West Virginia University. Presentation to the committee. December 6, 2007.

OCR for page 193
00 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES accredited by ASCLD/ LAB.19 Another 3 percent were accredited by some other professional organization, such as the ISO. State-operated laborato- ries (91 percent) were more likely to be accredited than laboratories serving county (67 percent) or municipal (62 percent) jurisdictions. Among the 230 laboratories providing accreditation information in both the 200220 and 2005 censuses, the accreditation rate increased during the three years from 75 to 87 percent. However, identification units—that is, those forensic entities outside crime laboratories—do not participate in accreditation systems and are not required to do so. Given that some disciplines are practiced largely outside the laboratory environment (e.g., 66 percent of fingerprint analyses are not conducted in crime laboratories), there is a substantial gap in the number of programs participating in accreditation.21,22 As mentioned previously, DNA analysis is regulated under the DNA Identification Act of 1994, which created an advisory board on quality assurance, tasked with promulgating standards for proficiency testing of laboratories and analysts. The terms of the original advisory board expired, and now the FBI Quality Assurance Standards apply to DNA laboratories receiving federal funds. The standards require periodic (every other year) audits using the FBI Quality Assurance Standards to ensure compliance. The FBI guidelines require that two proficiency tests be completed annu- ally by DNA examiners as well as by technical support personnel perform- ing relevant analytical techniques. The tests must be administered by a source external to the laboratory. The FBI is responsible for developing and maintaining a DNA audit document for assessing compliance with DNA standards and also provides DNA auditor instruction to all ASCLD/LAB inspectors, in addition to the forensic DNA community, on how to inter- pret the DNA standards. The FBI also reviews audit findings and remedial action, if any. Once all standards are met, it notifies the laboratory of full compliance. 19 M.R. Durose. 2008. Census of Publicly Funded Forensic Crime Laboratories, 00. U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics. Available at www.ojp.usdoj.gov/bjs/pub/pdf/cpffcl05.pdf. 20 J.L. Peterson and M. J. Hickman. 2005. Census of Publicly Funded Forensic Crime Laboratories, 00. U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics. Available at www.ojp.usdoj.gov/bjs/pub/pdf/cpffcl02.pdf. 21 Witt, op. cit. 22 Accreditation is also available for other more specific forensic science disciplines. For example, the National Association of Medical Examiners (NAME) operates an accreditation program for coroners and medical examiners offices (see Chapter 9). The American Board of Forensic Toxicology accredits toxicology laboratories.

OCR for page 193
0 STRENGTHENING OVERSIGHT STANDARDS AND guIDELINES FOR QuALITy CONTROL Standards provide the foundation against which performance, reli- ability, and validity can be assessed. Adherence to standards reduces bias, improves consistency, and enhances the validity and reliability of results. Standards reduce variability resulting from the idiosyncratic tendencies of the individual examiner—for example, setting conditions under which one can declare a “match” in forensic identifications. They make it possible to replicate and empirically test procedures and help disentangle method er- rors from practitioner errors. Importantly, standards not only guide practice but also can serve as guideposts in accreditation and certification programs. Many forensic science disciplines have developed standards, but others have not, which contributes to questions about the validity of conclusions. Several groups produce standards for use in the forensic science disci- plines. For example, ASTM International (ASTM), originally known as the American Society for Testing and Materials, is an international standards organization that develops and publishes voluntary technical standards for a wide range of materials, products, systems, and services. In the area of forensic science it offers, for example: • tandard Guide for Minimum Training Requirements for Forensic S Document Examiners • Standard Guide for Forensic Paint Analysis and Comparison • Standard Guide for Nondestructive Examination of Paper • tandard Guide for Forensic Analysis of Fibers by Infrared S Spectroscopy • tandard Terminology for Expressing Conclusions of Forensic S Document Examiners At the federal level, the National Institute of Standards and Technology (NIST) conducts research to establish standards in a limited number of fo- rensic areas, for example, organic gunshot residue analysis, trace explosives detectors, and improvised explosive devices.23 Its laboratories develop tests, test methods, produce reference data, conduct proof-of-concept implemen- tations, and perform technical analyses. They also develop guides to help forensic organizations formulate appropriate policies and procedures, such as those concerning mobile phone forensic examinations. These guides are not all-inclusive and they do not prescribe how law enforcement and 23 B. MacCrehan. National Institute of Standards and Technology. Analytical Chemistry Division. Presentation to the committee. September 21, 2007.

OCR for page 193
0 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES incident response communities should handle investigations. Instead, they provide principles for establishing policies and procedures.24 In accordance with ISO/IEC 17025, which states that all technical pro- cedures used by a science laboratory should be fully validated before they are used in casework, the European Network of Forensic Science Institutes has developed a guidance document for its member laboratories to use in validating techniques employed in forensic casework.25 The FBI initiated the first Scientific Working Groups (SWGs) in the early 1990s to facilitate consensus around forensic science operations among federal, state, and local agencies.26 Each SWG has a formal struc- ture and functions in accordance with its bylaws. Membership is at the discretion of the chair of the working group. Most SWGs include members from both public and private organizations. Meetings held at least once a year allow SWG members to discuss issues of concern and reach consensus on documents drafted throughout the year. The SWGs create, prepare, and publish standards and guidelines for their constituents in the forensic science community. These documents provide crime laboratories a basis for operational requirements, although the committee found that some standards and guidelines lack the level of specificity needed to ensure con- sistency. However, enforcement of the guidelines is left to the appropriate governing agency and each group’s internal policies. The SWGs generate voluntary guidelines and protocols, which carry no force of law. Nonethe- less, the SWGs have been a source of improved standards for the forensic science disciplines and represent the results of a profession that is working to strengthen its professional services with only limited resources. The FBI Laboratory currently sponsors the following groups: • cientific Working Group for Firearms and Toolmarks (SWGGUN) S • cientific Working Group for Forensic Document Examination S (SWGDOC) • Scientific Working Group for Materials Analysis (SWGMAT) • cientific Working Group on Bloodstain Pattern Analysis (SWGSTAIN) S • cientific Working Group on DNA Analysis Methods (SWGDAM) S • cientific Working Group on Dog and Orthogonal Detector Guide- S lines (SWGDOG) • cientific Working Group on the Forensic Analysis of Chemical S Terrorism (SWGFACT) 24 B. Guttman. National Institute of Standards and Technology National Software Reference Library. Presentation to the committee. September 21, 2007. 25 European Network of Forensic Science Institutes Standing Committee for Quality and Competence (QCC). 2006. Validation and Implementation of (New) Methods. 26 Federal Bureau of Investigation. 2000. Scientific Working Groups. Available at www.fbi. gov/hq/lab/fsc/backissu/july2000/swgroups.htm.

OCR for page 193
0 STRENGTHENING OVERSIGHT • cientific Working Group on the Forensic Analysis of Radiological S Materials (SWGFARM) • cientific Working Group on Friction Ridge Analysis, Study and S Technology (SWGFAST) • cientific Working Group on Microbial Genetics and Forensics S (SWGMGF) • cientific Working Group on Shoeprint and Tire Tread Evidence S (SWGTREAD) Additional SWGs may be sponsored by other FBI divisions or other agencies. For example, the U.S. Drug Enforcement Administration supports the Scientific Working Group for the Analysis of Seized Drugs (SWGDRUG) (see Box 7-2). Despite the proliferation of standards in many of the forensic science disciplines, their voluntary nature and inconsistent application make it difficult to assess their impact. Ideally, standards should be consistently applicable and measurable. In addition, mechanisms should be in place Box 7-2 A Sampling of SWGs SWGDRUGa In 1997, the Drug Enforcement Agency and the Office of National Drug Control Policy created and sponsored a Technical Working Group for the Analysis of Seized Drugs (TWGDRUG), which was renamed a Scientific Working Group (SWGDRUG) in 1999. The stated objectives of SWGDRUG include the specifica- tion of requirements for forensic drug practitioners, the promotion of professional development, the exchange of information within the forensic science community, the promotion of ethical standards of practitioners, the provision of minimum standards for drug examinations and reporting, the establishment of quality as- surance requirements, the consideration of relevant international standards, and the promotion of international acceptance of SWGDRUG recommendations. In- dividual subcommittees currently are devoted to evaluating analytical methods, setting standards for quality assurance, estimating uncertainty, formatting draft and final recommendations, and maintaining a glossary. The subcommittee de- velops recommendations, which the core committee votes to accept or reject. If accepted, draft documents are released for public comment for at least 60 days. Following public comment and possible revision, the core committee holds a final vote. Three-quarters of the core committee must be present, and two-thirds of those present must vote affirmatively in order to confer official status to a proposed recommendation.

OCR for page 193
0 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES for their enforcement, with sanctions imposed against those who fail to comply. As such, standards should be developed with a consideration of the relevant measures that will be used to provide a meaningful evaluation of an organization’s or individual’s level of compliance. Appropriate standards must be coupled with effective systems of accreditation and/or certification that include strong enforcement mechanisms and sanctions. Individual laboratories undergoing accreditation develop their own laboratory protocols. Whether these protocols adhere to the SWG stan- dards depends on the individual examiners in the discipline in the labora- tory in question. Accrediting bodies require that the methods meet a level of acceptable practice. Currently, most of these practices are slight variations of the SWG guidelines, with adjustments to accommodate differences in equipment. PROFICIENCy TESTINg Although many forensic science disciplines have engaged in proficiency testing for the past several decades, several courts have noted that profi- ciency testing in some disciplines is not sufficiently rigorous.27 ASCLD/LAB’s Web site states that “Proficiency testing is an integral part of an effective quality assurance program. It is one of many measures used by laboratories to monitor performance and to identify areas where improvement may be needed. A proficiency testing program is a reliable method of verifying that the laboratory’s technical procedures are valid and that the quality of work is being maintained.” 28 Similarly, ISO/IEC 17025 policies state: Proficiency testing is one of the important tools used by laboratories and Accreditation Bodies for monitoring test and calibration results and for verifying the effectiveness of the accreditation process. As such, it is an im- portant element in establishing confidence in the competence of Signatories and their accredited laboratories covered by this Arrangement.29 27 See United States . Crisp, 324 F.3d 261, 274 (4th Cir. 2003); United States . Llera Plaza, 188 F. Supp. 2d 549, 565, 558 (E.D. Pa. 2002); United States . Lewis, 220 F. Supp. 2d 548, 554 (S.D. W.Va. 2002). 28 See www.ascld-lab.org/legacy/pdf/aslabinternproficiencyreviewprogram.pdf. It is worth noting that several studies have assessed or published crime laboratory proficiency testing results, which generally reveal the need for improvement; J.L. Peterson, E.L. Fabricant, K.S. Field, and J.I. Thornton. 1978. Crime Laboratory Proficiency Testing Research Program. Washington, DC: U.S. Government Printing Office; J.L. Peterson and P. Markham. 1995. Crime laboratory proficiency testing results, 1978-1991, I: Identification and classification of physical evidence. Journal of Forensic Sciences 40(6):994-1008; J.L. Peterson and P. Markham, 1995. Crime laboratory proficiency testing results, 1978-1991, II: Resolving questions of com- mon origin. Journal of Forensic Sciences 40(6):1009-1029. 29 See www.iso.org/iso/catalogue_detail?csnumber=39883.

OCR for page 193
0 STRENGTHENING OVERSIGHT There are several types of proficiency tests, with the primary distinc- tion among them being whether the examiner is aware that he or she is being tested (an open or declared test) or does not realize that the sample presented for analysis is a test sample and not a real case (a blind test). Tests can be generated externally, by another laboratory (sometimes called an interlaboratory test), or internally. Another type of testing involves ran- dom case reanalysis, in which an examiner’s completed prior casework is randomly selected for reanalysis by a supervisor or another examiner.30 Interlaboratory testing can be conducted for a number of purposes: (1) to determine the performance of individual laboratories for specific tests or measurements and to monitor laboratories’ continuing performance; (2) to identify problems in laboratories and initiate remedial actions, which may be related to, for example, individual staff performance or the calibration of instrumentation; (3) to determine the performance characteristics of a method and to establish the effectiveness and comparability of new tests or mea- surement methods; or (4) to assign values to reference materials and assess their suitability for use in specific tests or measurement procedures.31 Blind proficiency testing is recommended, but not required, by ASCLD/ LAB—not as a way to determine error rates, but as a more precise test of a worker’s accuracy. Initially, mandatory blind testing was proposed as part of the federal DNA Identification Act. A Department of Justice (DOJ) panel designed blind tests, evaluated them, and estimated it would cost $500,000 to $1 million annually for one test per laboratory.32 In appropriate circum- stances, proficiency testing should include blind testing. ASCLD/LAB has a detailed proficiency testing program that requires all active examiners to take at least one proficiency test per year (two tests per year in DNA), that each discipline within the laboratory participate in an external proficiency test that is reviewed by a proficiency test review 30 Refer to ISO/IEC Guide 43-1:1997(E) Section 4 for a list of proficiency testing schemes. Refer to ASTM E 1301 Section 6 for an overview of organization and design of proficiency tests. SWGs also provide guidelines for proficiency testing in the relevant discipline. 31 European Network of Forensic Science Institutes. 2005. Guidance on the Conduct of Proficiency Tests and Collaboratie Exercises Within ENFSI. Available at www.enfsi.eu/ uploads/files/QCC-PT-001-003.pdf. 32 J.L. Peterson, G. Lin, M. Ho, Y. Chen, and R.E. Gaensslen. 2003. The feasibility of external blind DNA proficiency testing. Available at www.astm.org/JOURNALS/FORENSIC/ PAGES/4241.htm.

OCR for page 193
0 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES panel, and that any proficiency test that is not successfully completed be im- mediately reported to ASCLD/LAB along with a corrective action plan. To retain accredited status for a full five-year term, a laboratory must continue to meet the standards under which it was accredited. One of the means by which ASCLD/LAB monitors compliance is by reviewing proficiency testing reports submitted by approved test providers. According to the 2002 BJS census,33 274 of the 351 publicly funded laboratories were engaged in proficiency testing. Proficiency testing was slightly less common among smaller laboratories and those serving munici- pal jurisdictions (8 laboratories did not engage in such testing, and 69 did not answer the survey question). Among the laboratories engaged in profi- ciency testing, almost all use declared tests. Slightly more than half engaged in proficiency testing use random case reanalysis. Twenty-six percent of the laboratories engaged in proficiency testing use blind tests. In addition, the BJS survey reported that almost all laboratories engaged in proficiency testing used tests that were generated externally (thus allowing comparative analysis). In addition to external tests, 74 percent of laboratories engaged in proficiency testing also used internally generated tests. Data on proficiency testing were not collected for the 2005 census. CERTIFICATION The certification of individuals complements the accreditation of labo- ratories for a total quality assurance program. In other realms of science and technology, professionals, including nurses, physicians, professional engineers, and some laboratorians, typically must be certified before they can practice.34 The same should be true for forensic scientists who practice and testify. Although the accreditation process primarily addresses the management system, technical methods, and quality of the work of a labo- ratory (which includes the education and training of staff), certification is a process specifically designed to ensure the competency of the individual examiner. The American Bar Association has recommended that certification stan- dards be required of examiners, including “demanding written examina- tions, proficiency testing, continuing education, recertification procedures, 33 Peterson and Hickman, op. cit. 34 T. Ortelli. 2008. Characteristics of candidates who have taken the Certified Nurse Edu- cator: CNE examination: A two-year review. Nursing Education Perspecties 29(2):120; P. Nowak. 2008. Get IT-certified: Having employees with the right certifications can help deal- ers and integrators qualify for business and gain access to IT networks. Network Technology 38(3):123; S. Space. 2007. Investigator certification. Issues in Clinical Trials Management 8(2):73.

OCR for page 193
0 STRENGTHENING OVERSIGHT an ethical code, and effective disciplinary procedures.”35 In addition to improving quality, certification programs can enhance the credibility of certificate holders. An excellent description of the certification process is contained in the following excerpt from the National Association of Medi- cal Examiners (NAME) Web site: In general, certification boards consist of respected professionals in a particular area of professional practice who develop standards for educa- tion, training, and experience that are required before one can become ‘certified’ in a particular professional discipline. Successful completion of a written and/or practical examination is also usually required. In essence, ‘certification’ usually means that a particular individual has completed a defined course of education, training, and experience, and has passed an examination prepared by peers which demonstrates that the individual has obtained at least the minimum level of competence required to practice the specific discipline. A number of ‘Certification Boards’ exist for people in various scientific disciplines. . . .36 The professional forensic science community supports the concept of certification. ASCLD recommends that laboratory managers support peer certification programs that promote professionalism and provide objective standards. In 2002, the Technical Working Group on Forensic Science Education recommended certification of an individual’s competency by an independent peer-based organization, if available, from a certifying body with appropriate credentials. In addition, IAI supports certification of fo- rensic science practitioners.37 Some organizations, such as the American Board of Criminalists (ABC), offer examiner certification programs, but some certification organizations appear to lack stringent requirements.38 In response, the American Academy of Forensic Sciences has formed a Forensic Specialties Accreditation Board to accredit certifying organizations. Organizations are invited to participate if they meet established requirements, such as periodic recertification, a suf- ficient knowledge base for certification, a process for providing credentials, and a code of ethics.39 Currently accredited boards include: • American Board of Criminalistics 35 American Bar Association, op. cit., p. 7. 36 See http://thename.org/index.php?option=com_content&task=view&id=80&Itemid=41. 37 K.F. Martin, President, IAI. Presentation to the committee. September 19, 2007. 38 See M. Hansen. 2000. Expertise to go. ABA J. 86:44-45; E. MacDonald. 1999. “The Making of an Expert Witness: It’s in the Credentials.” Wall Street Journal. February 8, p. B1. 39 See FABS Standards for Accrediting Forensic Specialty Certification Boards at www. thefsab.org/standards_20070218.pdf.

OCR for page 193
0 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES • American Board of Forensic Document Examiners • American Board of Forensic Toxicology • American Board of Medicolegal Death Investigators • Board of Forensic Document Examiners • International Institute of Forensic Engineering Sciences IAI also has established certification programs in: • Bloodstain Pattern Analysis • Crime Scene Investigation • Footwear • Forensic Art • Forensic Photography/Imaging • Latent Print • Tenprint Fingerprint40 Other certification programs exist for (but are not limited to) the fol- lowing forensic science disciplines: • ocument Examination (The American Board of Forensic Docu- D ment Examiners [ABFDE]) • rug Analysis, Fire Debris Analysis, Molecular Biology, Trace D Analysis, and General Criminalistics (ABC) • irearms and ToolMark Identification (Association of Firearm and F ToolMark Examiners [AFTE]) • orensic Odontology (The American Board of Forensic Odontol- F ogy [ABFO]) • Forensic Pathology (The American Board of Pathology [ABP]) • Toxicology (American Board or Forensic Toxicology [ABFT]) Each of these entities has specific educational, training, and experience requirements, including a series of competency tests—both written and practical—and participation in proficiency testing, and provide continuing education/active participation by means of publication, presentation, and membership in professional organizations. OVERSIgHT AS A REQuIREMENT OF PAuL COVERDELL FORENSIC SCIENCE IMPROVEMENT gRANTS One way of enforcing quality control is through the conditional fund- ing of programs. The Justice for All Act of 00 (P.L. 108-405) that created 40 K.F. Martin, President, IAI. Presentation to the committee. September 19, 2007.

OCR for page 193
 STRENGTHENING OVERSIGHT the Coverdell Forensic Science Improvement Grants required that grant recipients certify that they have a process in place for independent, exter- nal investigations if allegations arise of “serious negligence or misconduct substantially affecting the integrity of the forensic results.”41 In December 2005, the Office of the Inspector General (OIG) of DOJ issued a report of an audit that found that the Office of Justice Programs (OJP), which administers the program, “had not enforced or exercised ef- fective oversight over the external investigation requirement for the Fiscal Year (FY) 2005 Coverdell Program.”42 OJP did not require grant applicants to identify the government entities that they certified could perform inde- pendent external investigations: Our review found that NIJ did not enforce the Act’s certification require- ment. NIJ’s FY 2005 Coverdell Grant Program Announcement did not give applicants necessary guidance on what constitutes an independent external investigation or how to make the required certification. In addition, the announcement did not provide examples of external investigation certifi- cations and did not require an applicant to name the government entity responsible for conducting independent, external investigations. NIJ was aware of the shortcomings in the announcement because of questions it received from potential applicants and concerns expressed by the OIG, but failed to correct them.43 The OIG made three recommendations to improve the program announce- ment and application process (see Box 7-3). A second audit of the program was released in January 2008.44 Again, it reported that not all forensic laboratories that had received FY 2006 grant funds were covered by a government entity with the authority and capability to independently investigate allegations of serious negligence or misconduct. “Further, OJP’s guidance does not require grantees and sub- grantees (forensic laboratories) to refer allegations of serious negligence and misconduct to entities for investigation.”45 The OIG found that 78 of the 231 entities contacted did not meet the external investigation certifica- tion requirement. It also found that “OJP did not adequately review the information it did obtain to ascertain that the certifications submitted by 41 42 U.S.C. § 3797k(4). 42 U.S. Department of Justice, Office of the Inspector General. 2005. Reiew of the Office of Justice Programs’ Forensic Science Improement Grant Program, Evaluation and Inspections Report I-2006-002. Available at www.usdoj.gov/oig/semiannual/0605/ojp.htm. 43 Ibid. 44 U.S. Department of Justice, Office of the Inspector General. 2008. Reiew of the Office of Justice Programs’ Forensic Science Improement Grant Program, Evaluation and Inspections Report I-2008-001. 45 Ibid., p, ii.

OCR for page 193
 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES the grantees were properly completed.”46 The OIG made three recommen- dations to OJP to correct its certification process (see Box 7-3). CODES OF ETHICS A code of ethics is another mechanism for encouraging the development and use of professional standards of conduct. However, there is disagree- ment about how effective such codes are in achieving that goal.47 In 1991, Ladd argued that codes of ethics serve no good purpose and that reliance on such codes confuses ethics with law.48 Some authors have noted that although practicing professionals rarely turn to their codes of ethics for guidance, the adoption of a code of ethics is critical to the professionaliza- tion of a group, because it indicates that the group recognizes an obligation to society that transcends its own self-interest.49 However, codes of ethics can serve to provide rational bases for punishments, such as exiling viola- tors from the community. In the field of engineering, Davis asserts that codes of ethics should be understood as conventions among professionals: The code is to protect each professional from certain pressures (for ex- ample, the pressure to cut corners to save money) by making it reason- ably likely . . . that most other members of the profession will not take advantage of her good conduct. A code protects members of a profession from certain consequences of competition. A code is a solution to a coor- dination problem.50 Also in the field of engineering, Harris et al. argue that codes can serve as a collective recognition by members of a profession of its responsibilities, creating an environment in which ethical behavior is the norm.51 Moreover, a code of ethics can serve as an educational tool, providing a starting point for discussion in coursework and professional meetings. 46 Ibid., p. iii. 47 A series of articles published in the Journal of Forensic Sciences 34(3) (May 1989) ad- dressed a range of ethical dilemmas facing individuals practicing science in the criminal justice system. 48 J. Ladd. 1991. The quest for a code of professional ethics: An intellectual and moral confusion. In: D.G. Johnson (ed.). Ethical Issues in Engineering. Englewood Cliffs, NJ: Prentice-Hall, pp. 130-136. 49 H.C. Luegenbiehl. 1983. Codes of ethics and the moral education of engineers. Business and Professional Ethics Journal 2:41-61; D.G. Johnson (ed.). 1991. Ethical Issues in Engineer- ing. 1991. Englewood Cliffs, NJ: Prentice-Hall, pp. 137-154. 50 M. Davis. 1991. Thinking like an engineer: The place of a code of ethics in the practice of a profession. Philosophy and Public Affairs 20(2):150-167, p. 154. 51 C.E. Harris, M.S. Pritchard, and M.J. Rabins. 1995. Engineering Ethics: Concepts and Cases. Belmont, CA: Wadsworth Publishing.

OCR for page 193
 STRENGTHENING OVERSIGHT Box 7-3 Recommendations from Two Reviews of the Coverdell Grant Program 2005 - We believe that Coverdell Grant Program Announcements must provide necessary guidance to applicants and request the information required for NIJ to evaluate the external investigation certifications and conduct effective oversight of the grants. To meet the requirements of the Justice for All Act of 2004, we recom- mend that OJP, as part of its oversight of NIJ: 1. Require that all Coverdell Grant Program Announcements contain guid- ance on what constitutes an independent external investigation and examples of government entities and processes that could satisfy the certification requirement. 2. Require that each Coverdell Grant applicant, prior to receiving funds, provide the name of the government entity with a process in place to conduct independent external investigations into allegations of serious negligence or misconduct. 3. Consider requiring each Coverdell Grant applicant, prior to receiving funds, to submit a letter from the government entity that will conduct in- dependent external investigations acknowledging that the entity has the authority and process to investigate allegations of serious negligence or misconduct. 2006 - To improve OJP’s administration of the Coverdell Program and better ensure that allegations of negligence or misconduct are subject to independent external investigation, the OIG recommends that OJP take the following actions: 1. Revise the certification template to require that applicants name the government entities and confirm that the government entities have: a. the authority, b. the independence, c. a process in place that excludes laboratory management, and d. the resources to conduct independent external investigations into allegations of serious negligence or misconduct by labs that will received Coverdell funds. 2. Provide applicants with guidance that allegations of serious negligence or misconduct substantially affecting the integrity of forensic results are to be referred to the certified government entities. 3. Revise and document the Coverdell Program application review process so that only applicants that submit complete external investigation cer- tifications are awarded grants. SOURCE: U.S.DOJ Office of the Inspector General. 2005. Review of the Office of Justice Programs’ Forensic Science Improvement Grant Program, Evaluation and Inspections Report I-2006-002. Available at www.usdoj.gov/oig/semiannual/0605/ojp.htm; U.S. DOJ OFFICE of Inspector General. 2008. Review of the Office of Justice Programs’ Forensic Science Improve- ment Grant Program, Evaluation and Inspections Report I-2008-001.

OCR for page 193
 STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES Many forensic science organizations—such as the American Acad- emy of Forensic Sciences, the California Association of Criminalists, and ASCLD—have codes of ethics or codes of professional practice imploring members to act with honesty, integrity, and objectivity; to work within the bounds of their professional competence; to present testimony and reports in a clear and objective manner; and to avoid conflicts of interest and potential bias, among other things. The codes that do exist are generally comprehensive, but they vary in content. As a consequence, there is no single code of ethics to which all members of the forensic science profession subscribe. As the committee concluded its work, it learned of an effort by ASCLD/LAB to develop a uniform code of ethics. CONCLuSIONS AND RECOMMENDATIONS Although some areas of the forensic science disciplines have made no- table efforts to achieve standardization and best practices, most disciplines still lack any consistent structure for the enforcement of “better practices,” operating standards, and certification and accreditation programs. Accredi- tation is required in only three states—New York, Oklahoma, and Texas. In other states, accreditation is voluntary, as is individual certification. Certification, while broadly accepted by the forensic science community, is not uniformly offered or required. Although many forensic science organizations have codes of ethics, these codes can be enforced to regulate only the practices of persons who belong to a given organization. A uniform code of ethics should be in place across all forensic organizations to which all forensic practitioners and laboratories should adhere. Recommendation 6: To facilitate the work of the National Institute of Forensic Science (NIFS), Congress should authorize and appropriate funds to NIFS to work with the National Institute of Standards and Technology (NIST), in conjunction with government laboratories, universi- ties, and private laboratories, and in consultation with Scientific Working groups, to develop tools for advancing measurement, validation, reliability, information sharing, and proficiency testing in forensic science and to establish protocols for forensic examina- tions, methods, and practices. Standards should reflect best prac- tices and serve as accreditation tools for laboratories and as guides for the education, training, and certification of professionals. upon completion of its work, NIST and its partners should report find-

OCR for page 193
 STRENGTHENING OVERSIGHT ings and recommendations to NIFS for further dissemination and implementation. Recommendation 7: Laboratory accreditation and individual certification of forensic science professionals should be mandatory, and all forensic science professionals should have access to a certification process. In de- termining appropriate standards for accreditation and certification, the National Institute of Forensic Science (NIFS) should take into account established and recognized international standards, such as those published by the International Organization for Standard- ization (ISO). No person (public or private) should be allowed to practice in a forensic science discipline or testify as a forensic sci- ence professional without certification. Certification requirements should include, at a minimum, written examinations, supervised practice, proficiency testing, continuing education, recertification procedures, adherence to a code of ethics, and effective disciplinary procedures. All laboratories and facilities (public or private) should be accredited, and all forensic science professionals should be certi- fied, when eligible, within a time period established by NIFS. Recommendation 8: Forensic laboratories should establish routine quality assurance and quality control procedures to ensure the accuracy of forensic analyses and the work of forensic practitioners. Quality control procedures should be designed to identify mistakes, fraud, and bias; confirm the continued validity and reliability of standard operating procedures and protocols; ensure that best practices are being followed; and correct procedures and protocols that are found to need improvement. Recommendation 9: The National Institute of Forensic Science (NIFS), in consultation with its advisory board, should establish a national code of ethics for all forensic science disciplines and encourage individual societies to incorporate this national code as part of their professional code of ethics. Additionally, NIFS should explore mechanisms of enforce- ment for those forensic scientists who commit serious ethical viola- tions. Such a code could be enforced through a certification process for forensic scientists.

OCR for page 193