Science-Based Assessment and Experimental Design
SCIENCE-BASED TESTING FOR COMBINATION DEVICES
Aric Kaiser
Center for Devices and Radiological Health
U.S. Food and Drug Administration
Before a new medical device can be marketed, the Center for Devices and Radiological Health at the U.S. Food and Drug Administration (FDA) must review its risks and effectiveness. A device is defined as an apparatus or implant intended for the diagnosis, mitigation, treatment, or prevention of disease that does not achieve its primary intended purposes through chemical action and that is not dependent on being metabolized. The review process for traditional devices, familiar to most device manufacturers, requires that a manufacturer demonstrate one of two sets of device classification criteria:
-
There must be reasonable assurance of the safety and effectiveness of the device based on preclinical and clinical evaluations; or
-
The device must have the same intended use as a predicate device, along with either the same technological characteristics or different technological characteristics that do not raise different questions of safety and effectiveness.
Combination devices combine with drugs and/or biologic components to effect a treatment. The review process for these products may be different from that for traditional devices, but generally, although some additional regulatory requirements must be fulfilled, the questions that a manufacturer must address as part of a preclinical or clinical evaluation of a combination product are essentially the same. For example, in the evaluation of any device that is implanted or that has contact with tissues, the materials used in the device must be assessed to determine whether they can safely be used or implanted in the human body (biocompatibility). When a biologic component is combined with a device, an additional determination must be made regarding whether the materials used are free from potentially infectious agents.
SCIENCE-BASED TESTING FOR BIOLOGICS
Darin J. Weber
Center for Biologics Evaluation and Research
U.S. Food and Drug Administration
In general, biological products are complex mixtures of multiple components. Because they are prepared from living sources, special consideration must be given to preventing the transmission or introduction of infectious agents while preserving product identity, purity, and potency. In order to ensure that no infectious agents are present, U.S. Food and Drug Administration (FDA) regulations for biological products include specific science-based testing of the final product.
FDA’s General Biological Product Standards (21 CFR 610) include standards for manufacturing safety (sterility, mycoplasma, purity, adventitious viral agents) and for the assessment of product characteristics such as identity, viability, and potency. In addition to testing the final product, safety testing and other assessments are performed throughout manufacturing in order to evaluate the manufacturing process itself and to ensure that the quality and consistency of the product lots are maintained.
Standard test methods are of limited use, however, when testing biological products consisting of living cells. Many of the prescribed tests, such as sterility tests, take days or weeks to complete, and may thus limit the development of products that cannot be stored for such long periods of time. FDA has therefore adopted a flexible approach that allows some products to be used clinically even if the final test results are not available. FDA’s Center for Biologics Evaluation and Research also supports and encourages the development of alternative test methods (as described in FDA standard 21 CFR 610.9) to meet the need for fast, sensitive, and reliable test methods for these products.
USING DESIGN OF EXPERIMENT METHODS IN THE INNOVATION PROCESS
James Rutledge
DataVision Statistical Consulting and Training, LLC
It is often said that “knowledge is power,” and gaining knowledge is what design of experiments (DOE) methods are all about. Effectively researching, developing, and maintaining a product requires all the knowledge that can
reasonably be obtained about the process used to make that product. A process is defined here as a series of inputs and outputs. Inputs include the various aspects of the process that might influence the resulting product, such as reaction time and temperature, material thickness, vendor source, and lot variations. Knowing the impact on the end product of variance in the inputs is important. Outputs are measurable quantities that describe product characteristics or performance, such as the size of an extruded part or the yield of a chemical process. A mathematical understanding of how process inputs relate to outputs results in profound process knowledge and the power to control and improve processes.
It is very important to have quantifiable measures for inputs and outputs, and the systems used for making those measurements must be repeatable and reproducible. For example, in a measurement system, an engineer would perform a gauge repeatability and reproducibility study. In a chemical system, a scientist would validate the analytical method used for detection to ensure that it is sensitive, repeatable, and reproducible. There are industry standards for many of these tests, although new quantitative or semi-quantitative methods might have to be developed for histological evaluations of inflammation, for example.
The typical alternatives to DOE methods are “best-guess” and “one-factor-at-a-time” (OFAT) methods, both of which are based on the researcher’s strong understanding of the system to be evaluated. However, these intuitive approaches may not result in the correct answer or may not be understood and believed by others. They may miss important events, such as the interaction of one input variable with another and how that affects the product output. DOE methods, by contrast, allow the importance of the various inputs to be summarized quantitatively and allow the development of a mathematical model to run simulation experiments of the expected product when the process is run under various conditions.
Modeling is important because it allows us to understand the factors that influence the robustness of the process. This understanding is critical to initial process validation as well as ongoing manufacturing. It is also important in the up-front development process, where it can be used to refine decisions about how to operate the process before experiments are undertaken. This mathematical process, or in silico experimentation, is helpful in identifying process specifications that can eventually be verified in confirmation studies.
The modeling process will be most valuable if it is undertaken both to make the product to the desired specifications (targeting the process center) and to manage product variability within acceptable limits (reducing process variation). Controlling product variability is crucial for keeping the process under control and ensuring that every unit produced will fall within acceptable limits of the desired target.
One attribute of DOE experiments that is often overlooked is their efficiency in determining the importance of the inputs and developing the process models with a minimum number of experiments. In fact, DOE techniques require fewer resources than traditional methods because of the efficiency of the design. This is counterintuitive because designing and executing DOE studies seems to take longer. While the up-front planning stages of DOE are often more involved than those of traditional techniques, the efficient design used to analyze information results in the need for fewer experiments, thus saving time and resources. In addition, because more time is invested in planning the experiments, they are more likely to be definitive in their results. DOE experiments can also be used to identify those factors that have no impact on product target or variability. Decisions can then be made about eliminating such factors, or reducing control of those variables in order to save process costs.
In summary, DOE methods improve understanding of what influences the product process, both in hitting the target and in controlling variability; help inform decisions about how to run the process, through the modeling approach; and provide in-depth knowledge about interactions and the quantitative influence of inputs on product performance and characteristics. While computer software can simplify the analysis component of DOE studies, the experimental discipline and creative thinking of the engineer or scientist are what make DOE methods successful.
MAKING IT FLY: CURRENT BOEING CERTIFICATION PROCESSES
Stephen G. LaRiviere
Boeing Commercial Airplanes
When Boeing Commercial Airplanes develops a large transport aircraft, numerous certification processes must be successfully completed. The Federal Aviation Administration (FAA) of the Department of Transportation is the agency with oversight responsibility for these certification processes; Boeing Commercial Airplanes interacts primarily with the FAA Aircraft Certification Service, Transport Airplane Directorate, Manufacturing Inspection Office, Seattle Aircraft Certification Office, and Seattle Manufacturing Inspection District Office. A designated engineering representative (DER) is a Boeing employee who represents the FAA with the agency’s concurrence and who facilitates the certification process. The DER is generally a senior engineer who has both technical and communication skills and who is well respected by both the FAA and Boeing colleagues.
Federal aviation requirements are incorporated into all Boeing policies and procedures. Boeing must obtain type certificates (design approval for each airplane to be manufactured), production certificates (approval to build airplanes and airplane parts in accordance with the type design), and airworthiness certificates (approval to deliver and operate an airplane that has been built and tested in compliance with the type and production certification). Airplane type certification covers four areas: structures, materials, systems, and propulsion. For Boeing structural certification, a building block approach is used, with testing on the coupon, element, detail, subcomponent, and component levels. For materials certification, efforts are currently being made to achieve faster qualification through the use of critical-chain project management and, to a limited extent, design of experiments.