National Calibration Facility for Retroreflective Traffic Control Materials (2005)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

CHAPTER 7 CALIBRATION SERVICES AND TRACEABILITY TO NIST The mission of NIST is to develop and promote measurement, standards, and technology to enhance productivity, facilitate trade, and improve the quality of life. To help meet the measurement and standards needs of retroreflective materials manufacturers and users, NIST provides calibrations, test methods, proficiency evaluation materials, measurement quality assurance programs, and laboratory accreditation services that assist a customer in establishing traceability of results of measurements or values of standards describe in the next sections. TRACEABILITY TO NIST Traceability requires the establishment of an unbroken chain of comparisons to stated references. NIST assures the traceability of results of measurements or values of standards that NIST itself provides, either directly or through an official NIST program or collaboration. Other organizations are responsible for establishing the traceability of their own results or values to those of NIST or other stated references. The specific NIST statement of policy follows. To support the conduct of its mission and to ensure that the use of its name, products, and services is consistent with its authority and responsibility, NIST: 1. Adopts for its own use and recommends for use by others the definition of traceability provided in the most recent version of the International Vocabulary of Basic and General Terms in Metrology: “property of the result of a measurement or the value of a standard whereby it can be related to stated references, usually national or international standards, through an unbroken chain of comparisons all having stated uncertainties.” 2. Establishes traceability of the results of its own measurements and values of its own standards and of results and values provided to customers in NIST calibration and measurement certificates, operating in accordance with the NIST System for Assuring Quality in the Results of Measurements Delivered to Customers in Calibration and Measurement Certificates. 3. Asserts that providing support for a claim of traceability of the result of a measurement or value of a standard is the responsibility of the provider of that result or value, whether that provider is NIST or another organization; and that assessing the validity of such a claim is the responsibility of the user of that result or value. 47

4. Communicates, especially where claims expressing or implying the contrary are made, that NIST does not define, specify, assure, or certify traceability of the results of measurements or values of standards except those that NIST itself provides, either directly or through an official NIST program or collaboration. 5. Collaborates on development of standard definitions, interpretations, and recommended practices with organizations that have authority and responsibility for variously defining, specifying, assuring, or certifying traceability. 6. Develops and disseminates technical information on traceability and conducts coordinated outreach programs on issues of traceability and related requirements. Assigns responsibility for oversight of implementation of the NIST policy on traceability to the NIST Measurement Services Advisory Group. Traceability is established through “an unbroken chain of comparisons all having stated uncertainties.” In practical terms, for each link in the chain the following must be in place, • A clear definition of the particular quantity that has been measured, in this case examples are coefficient of retroreflection or coefficient of luminous intensity • A complete description of the measurement system used to perform the measurement • A stated measurement result or value, with a documented uncertainty • A complete specification of the stated reference at the time the measurement system was compared to it, whether the calibrated artifact came directly from NIST or was supplied by another vendor that is traceable to NIST • An ‘internal measurement assurance’ program for establishing the status of the measurement system at all times pertinent to the claim of traceability • An ‘internal measurement assurance’ program for establishing the status of the stated reference at the time that the measurement system was compared to it. Therefore, as an example, a NIST calibration report gives a brief description of the quantity measured, the description of the instrument used to measure the quantity, the value of the quantity measured along with a stated uncertainty and uncertainty budget, and a time frame of when the measurement device was last calibrated or the scale realized that it transfers. The full descriptions are published in a journal and are referenced in the calibration report. Also, behind the report is the NIST System for Assuring Quality in the Results of Measurements Delivered to Customers in Calibration and Measurement Certificates. The quality system provides a written 48

set of documents to ensure a consistent methodology of calibration, which must exist and be reviewed regularly by NIST management. The user of the result of a measurement or value of a standard is responsible for assessing the validity of a claim of traceability. The provider of the result of a measurement or value of a standard is responsible for supporting its claim of the traceability of that result or value. Thus, if a customer asks a provider for a complete chain of traceability, it is the provider who is responsible for providing the necessary information that the user assesses. With the project NIST has developed a calibration program where submitted artifacts are calibrated for requested quantities, such as coefficient of retroreflection, coefficient of luminous intensity, or coefficient of retroreflected luminance. This calibration program provides documentation of calibration, which includes the measurement value and the uncertainty associated with the measurement. The uncertainty is determined by an uncertainty budget, which is derived from characterizing the artifact. This calibration report begins the chain of traceability to NIST. It is important to note that ‘NIST traceable,’ in and of itself, does not guarantee the lowest uncertainty. If the calibration is too long or the intermediate calibrations are not of high quality, the uncertainty may become significantly large, much larger than that of the NIST scale. However, as long as an unbroken chain of comparisons with stated uncertainties is made, the measurement is considered traceable to NIST. For example, a calibration facility has a piece of white non-exposed glass bead retroreflective sheeting calibrated by NIST at four sets of angles, (α = 0.2° and 0.5°, β = -4° and 30°, specifications from ASTM D4956).(1) The uncertainty for this artifact is 1.0 % (k=2), which is the basis for the calibration facilities scale. The calibration facility uses this artifact to calibrate the instrument, which requires measuring the artifact introducing uncertainty. The calibrated instrument has an uncertainty of 1.4 % (k=2), half attributed to the calibration standard and half to the transfer of the calibration standard value to the instrument. When the calibration facility measures a test sample that is white glass bead sheeting at one of these four sets of angles, the total uncertainty is 1.7 % (k=2). However, if a test sample is measured that is red or prismatic material, the uncertainty budget must account for these contributions. An important aspect is if a second facility follows these same procedures but the equipment is not as good and the uncertainty budget is 5.0 % (k=2), both calibrations are still traceable to NIST. 49

CALIBRATION SERVICES The specific calibration service procedures at NIST are expanded upon in a series of documents labeled Special Publications. A Special Publication in the 250 series is to be published in the Spring of 2005 on the calibration of retroreflective material. The publication provides a general description and then describes the scale realization, appropriate artifacts for calibration, equipment for calibration, calibration procedures, and the uncertainty budget of the calibration. The scale realization describes internal calibrations done at NIST that provide internal traceability to fundamental units. Included in this section of the SP250 is a verification and maintenance schedule. On a regular daily, monthly and annual schedule certain components need to be revalidated to maintain the level of quoted uncertainty. This list is always expanding and changing as time passes and data is collected on the stability of items. For example, on a daily basis the alignment tool is calibrated and the reference check standard is validated. The color temperature of the light source is validated every 50 lamp hours, the illuminance uniformity of the source is scanned every 100 lamp hours, the spectrometer is calibrated for spectral responsivity on a monthly basis, and the photometer response is measured against a calibrated FEL using a 0/45 measurement of a diffuse reflectance plaque on a monthly basis. On an annual basis, the photometer is calibrated for absolute spectral responsivity to monitor changes in the V(λ) filter, the goniometer magnetic encoder correction curve, the detection stage correction curves and the observation angle is re-calibrated. SP250 provides a discussion on what samples are appropriate for standards. Suggestions on tests that can be performed before submission are stated. The equipment for calibration and uncertainty budget of the calibration are very brief statements of the material presented in this report. The calibration procedures are based on ASTM standards and CIE recommendations. This section in the SP250 provides more specific details to enable the customer to reproduce the measurement. For example the following procedure is provided. Before calibration the light source is turned on and set to the current which provides 2856 K at the sample, typically 17.6 A. A 30-minute warm-up period is required. The goniometer is positioned at the specified distance from the source aperture. The absolute entrance angles are set to 0° using the reflective alignment plate. The goniometer yaw axis is rotated 90°. The detector package is mounted on 50

the goniometer sample plate and the aperture is centered on the illumination axis. The illuminance at the sample is measured at nine points. The mean and standard deviation are calculated and recorded. The detector is removed and position of the sample plate restored to the observation positioner. A reference check standard is mounted and the system is validated. The test sample is mounted and the absolute rotation angle is set by viewing through the theodolite. The angles to be measured are entered into the controlling program. The retroreflected light is measured at each required angle and the coefficient of retroreflectance is calculated and recorded. At each angle, sensitivity curves are automatically measured and used to calculate the uncertainty of the measurement. Additionally, at each angle a retroreflection spectral power distribution is measured, which is used to calculate the spectral mismatch correction factor. The measurements are repeated two more times to determine the repeatability of the sample. The ambient temperature at the sample and the relative humidity are recorded. After the measurements are completed, the results are analyzed and a calibration report is generated. The cost of the calibration service changes with time depending on factors determine by the NIST management. All of these factors are based on the number of hours required to perform the calibration. A time budget for the calibration of retroreflective samples has been prepared. With the basic automation of the equipment and the complete uncertainty analysis, an estimate of the time required for calibration has been determined: Number of hours = [n * ( m + 0.3) + 2] (18) where n is the number of test samples and m is number of angle combinations per sample. At current hourly rates and factors 1 test sample with 1 angle combination would have a price of $921. A test sample with 4 angle combinations would have a price of $1758. Two test samples with 6 angle combinations would have a price of $4075. History has shown that the factors determine by the NIST management will continue to increase; however, as the system becomes more automated, the required hours of measurement for multiple angle sets will decrease the price. Another service under investigation is a test. A test is different from a calibration because the determine quantity does not have an uncertainty associated with it. The test sample is tested to determine if it meets a specification significantly reducing the number of hours measurement required. A testing service is for determining compliance only. 51

MEASUREMENT ASSURANCE PROGRAM The purpose of the Measurement Assurance Program (MAP) is to determine the capabilities of a given laboratory and tie it to the National Standardization Program. This is achieved in a method that will simultaneously check measurements and procedures of the given laboratory by means of a package of various items to be measured. NIST in the past has provided a MAP service document in NBS Special Publication 671, “Retroreflectance MAP Service for Coefficient of Luminous Intensity.”(2) The original MAP package contained several types of retroreflectors that are chosen as diagnostic tools for certain aspects of a retroreflectometer, shown in Figure 42. The aspects range from geometrical capabilities to detection limits. Included were a white bead sheeting retroreflector, a second type of white bead sheeting retrorelfector, a white prismatic retroreflector and a flat black mat. Also, included were seven colored glass filters shown in Figure 43. The luminous transmittance of the seven colored glass filters were measured by two methods, a direct line of sight and an indirect line of sight using a white diffusing plaque. The luminous transmittance provides spectral information on the source and detector capabilities. As part of the measurement the test laboratory completed a questionnaire that asks about specific details of the measurement such as the distance and uncertainty characteristics of the measurement. The final product of the MAP comparison is a report of calibration that is property of the given laboratory. Included in Appendix D is a sample MAP service report from NBS SP671. The report contains: the Purpose of the report, the Materials used in the comparison, the Measurement Procedures of NIST and Instrument Description of the NIST reference retroreflectometer, the Results which show the measurements at NIST before and after the laboratory made its measurements, and the Conclusions derived from these measurements. Through the MAP service a given laboratory can receive more assistance and interaction than by obtaining a calibrated artifact from NIST. As part of this project, the artifacts from the original MAP service are being tested for stability. Additional artifacts are under development to provide the test laboratory more diagnostic tools. Once a few sets of artifacts are characterized and have passed, SP671 will be rewritten to include the updated information due to the capabilities of the new facility. The cost for this service has not been determined. 52

NATIONAL VOLUNTARY LABORATORY ACCREDITATION PROGRAM The National Voluntary Laboratory Accreditation Program (NVLAP) is an accreditation body operated by NIST. Laboratory accreditation is a finding of a laboratory’s competence and capability to provide scientifically sound and appropriate measurement services within their scope of accreditation. Embedded in the process is an evaluation of the lab’s ability to achieve and maintain traceability for the accredited services. Accreditation to ISO/IEC 17025: General requirements for the competence of testing and calibration laboratories (3) determines that a laboratory has all of the necessary facilities, equipment, standards, procedures, uncertainty analyses, personnel, etc., which make it capable of providing traceable measurement results. Laboratory accreditation does not speak to the specifics of any individual measurement result but to the overall capability of a lab to provide the service. As part of this project, NIST investigated and began to develop an accreditation program with NVLAP. With the development of the CIPM Mutual Recognition Arrangement (MRA) that was drawn up by the International Committee of Weights and Measures (CIPM) under the authority given to it in the Metre Convention, the need for a NVLAP program will become necessary for any international trade. The principal objectives of the MRA are to establish through measurement comparisons the degree of equivalence of national measurement standards maintained by National Metrology Institutes (NMIs), to provide for the mutual recognition of calibration and measurement certificates issued by NMIs, and to provide a secure technical foundation for wider agreements related to international trade, commerce, and regulatory affairs. In developing a program the following items must be accomplished: Identify laboratory community The laboratory community consists of federal and state department of transportation laboratories, manufacturers, and secondary laboratories that typically are instrument producers. The number of potential laboratories that would apply for NVLAP accreditation is not large, but it is not required to be a certain size. Identify laboratory user community The laboratory user community consists of federal and state department of transportation laboratories and a significant number of manufacturers including manufacturers of materials and instruments. 53

Identify standards writing bodies The standards writing bodies have been identified in this report, ASTM International and Commission Internationale De L’Éclairage. Soon the International Organization for Standardization will be included in this list. Identify test methods and standards Many of the documents are provided in the Bibliography of this report. Identify product certification/validation programs This aspect needs to be identified. Identify regulatory bodies both domestic and foreign This aspect needs to be identified. Identify other stakeholders This aspect needs to be identified. Identify sources of technical expertise and assistance – domestic and foreign The product of this report has established technical expertise at NIST. Through interactions with the CIE Technical Committee 2-56 on developing an ISO/CIE standard for retroreflection measurement, information on foreign technical expertise and assistance is being collected. Identify number of labs and amount of calibrations This aspect needs to be identified. Select the “units” of the Scope of Accreditation The units for Scope of Accreditation will depend on the requests from customers. Most likely the units will be limited to Coefficient of Luminous Intensity, Coefficient of Retroreflection, and Coefficient of Retroreflected Luminance. Establish specific technical requirements The specific technical requirements are currently be developed as part of working in CIE Technical Committee 2-56. Establish laboratory staff qualifications and certifications This aspect needs to be established. 54

Establish components of a laboratory: equipment, facilities, … The components of the laboratory need to be established and will depend on the determination of the specific technical requirements. Establish accreditation process sequence This aspect needs to be identified. Develop the assessment techniques for on-sites, demonstrations, … The assessment techniques need to be developed and listed. Design proficiency testing programs The proficiency testing programs will be almost exactly like the programs finalized in the MAP service. Set on-site team size and skill set This aspect needs to be determined, but mostly like will include two individuals: a quality assessor and a technical assessor. Create NVLAP program-specific checklists This aspect needs to be identified. Establish evaluation criteria for granting of accreditation The evaluation criteria are based on ISO 17025 and the ISO and ASTM standards associated with the measurement of the identified units for scope of accreditation. Establish fee schedule A fee schedule needs to be determined. The NVLAP headquarters can be contact to provide rough estimates of the initial assessment and then continuing assessments in years following the initial assessment. Determine contents of the NVLAP application package This aspect needs to be identified. Publish Federal Register announcement This aspect needs to be completed. Set time window for accepting initial applications, then no restrictions. This aspect needs to be identified. 55

The accreditation program with NVLAP will be completed and instituted once a few customers provide written requests of interest for this service to the NVLAP headquarters. (1) “Standard Specification for Retroreflective Sheeting for Traffic Control.” ASTM D4956-04, ASTM International, (2003) 12 pp. (2) Eckerle, K.L., and Hsia, J.J., “Retroreflectance MAP Service for Coefficient of Luminous Intensity.” NBS Special Publication 671, U.S. Department of Commerce/National Bureau of Standards, Washington D.C. (1984) 49 pp. (3) “General requirements for the competence of testing and calibration laboratories,” ISO/IEC 17025, International Organization for Standardization, Geneva, Switzerland, First Edition, (1999) 34 pp. 56