Session V:
What Is the Future of Materials State Awareness?

AN OVERVIEW OF DATA FUSION METHODS AND APPLICATIONS

R. Joseph Stanley, Missouri Institute of Science and Technology


Fusion of information sources or data fusion is becoming increasingly employed to merge information obtained from individual or multiple sensors and associated databases to improve decision-making, analytical, or inference-making capability. Humans often use multiple information sources such as sight, smell, touch, taste, and personal experiences to evaluate the quality of their dining experiences. From a system development perspective, data fusion involves the usage of tools, techniques, and/or methods for information merging. Techniques for individual, multisensor, and/or database-related data fusion are obtained from a wide range of areas including image and signal processing, control theory, numerical methods, artificial intelligence, fuzzy systems, neural networks, evolutionary computation, pattern recognition, statistical estimation, and other areas. The selection of techniques depends on the type, availability, and dynamic nature of data available for the system or application. Fusing multiple information sources is ideally intended to improve the accuracy with which an entity or entities of interest can be observed and characterized over a single information source.

There are numerous applications to which data fusion has been applied. Some military application examples of data fusion are (1) detection and tracking of targets of interest using air-, ground-, or ocean-based surveillance; (2) land mine or minefield detection using individual or multiple sensors in airborne-, vehicle-, or handheld-based systems; and (3) nondestructive evaluation (NDE) for assessing the structural health of aircraft and ships. A few examples of



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 37
Session V: What Is the Future of Materials State Awareness? AN OVERVIEW OF DATA FUSION METHODS AND APPLICATIONS R. Joseph Stanley, Missouri Institute of Science and Technology Fusion of information sources or data fusion is becoming increasingly employed to merge information obtained from individual or multiple sensors and associated databases to improve decision-making, analytical, or inference-making capability. Humans often use multiple information sources such as sight, smell, touch, taste, and personal experiences to evaluate the quality of their dining experiences. From a system development perspective, data fusion involves the usage of tools, techniques, and/or methods for information merging. Techniques for individual, multisensor, and/or database-related data fusion are obtained from a wide range of areas including image and signal processing, control theory, numerical methods, artificial intelligence, fuzzy systems, neural networks, evolutionary computation, pattern recognition, statistical estimation, and other areas. The selection of techniques depends on the type, availability, and dynamic nature of data available for the system or application. Fusing multiple information sources is ideally intended to improve the accuracy with which an entity or entities of interest can be observed and characterized over a single information source. There are numerous applications to which data fusion has been applied. Some military application examples of data fusion are (1) detection and tracking of targets of interest using air-, ground-, or ocean-based surveillance; (2) land mine or minefield detection using individual or multiple sensors in airborne-, vehicle-, or handheld-based systems; and (3) nondestructive evaluation (NDE) for assessing the structural health of aircraft and ships. A few examples of 37

OCR for page 37
38 Proceedings of a Workshop on Materials State Awareness nonmilitary applications include (1) medical patient diagnosis or assessment based on integrating information from diagnostic tests, vital signs, patient symptoms, and so forth; (2) weather forecasting using ground-, aircraft-, subsurface-, and satellite-based sensors; and (3) robot navigation using map and visualization sources. There are several issues that need to be addressed in the design and development of a data fusion-based system, including: (1) the number and type of information sources, (2) what information from each source is to be used to get the most from the data fusion process, (3) the operating conditions of the system for which data fusion will foster enhanced system operation, (4) the system architecture to provide for where and how the data from different sources are to be fused, (5) the algorithm type and choice that are appropriate for the data sources and the application, (6) the target level of performance of the data fusion process, and (7) the ability of the system to adapt in a dynamic environment. Data fusion can be used to translate between observed values from one or more sources for an entity and a decision or inference related to the entity at different levels of complexity. Data fusion can be used for data alignment or registration from multiple sensors such as a robotic unit determining positional information from a mounted camera and the Global Positioning System. Data fusion can also be used for enhancing classification tasks, where individual or multiple modalities and/or sensors can be merged at the raw data, feature, or decision level for contributing to classification decisions. For performing raw data fusion, there is typically some common domain to integrate raw data from multiple sources. For example, ultrasound and radiographic images of an aircraft panel may be fused at the pixel level based on mapping or registering the corresponding pixel locations from the two image sources. Approaches for performing raw data fusion commonly involve standard detection, estimation, and registration methods. Feature-level fusion typically involves extracting representative attributes from objects or entities of interest from the source data. Using the aircraft example, if the ultrasound and radiographic images for a panel are used separately, attributes related to size and shape may be determined from objects that may be potential areas of corrosion. The attributes or features from the ultrasound and radiographic images for each object may be combined into a single-feature vector for describing the object. The generated feature vectors are input into classification algorithms such as template matching, clustering methods, or neural networks. Decision-level fusion can involve combining source information after a preliminary decision or confidence has been determined for each source related to an entity’s detection, position, presence of a specific feature, or identity. Decision-level fusion may also involve combining single-source information based on multiple approaches for generating preliminary decisions or confidence values related to the entity. Techniques for performing decision-level fusion include voting and weighting schemes and other computational intelligence methods. An important consideration for performing raw data, feature, or decision-level fusion is data normalization. Data normalization is often necessary to allow data collected from different sources to be directly compared or combined. For example, in collecting an ultrasound and a radiograph image of the same aircraft panel, the sensor values will typically have different value ranges. In order to perform raw data fusion, part of the fusion process is to standardize the values from the ultrasound and radiograph images for combining the values to generate a single fused image. The utilization of data fusion-based systems is becoming more prevalent as the need increases for systems with enhanced decision- and inference-making capability. Accordingly, future considerations in the development of data fusion-based systems would appear to include the development of technology that will allow data fusion capability to be transparently integrated into the system development and manufacturing processes for a wide range of

OCR for page 37
Session V: What Is the Future of Material State Awareness? 39 applications. Some potential future directions for data fusion-based systems may include (1) the development and large-scale manufacturing of specialized hardware integrated circuits that implement standard multisensor data fusion methods, (2) parallel architectures for real-time implementation of multisensor or information source fusion systems, (3) methods for dynamic source selection for adaptive system design and development for source and modality integration, (4) validation methods for source uncertainty models and the development of robust approaches to estimate model parameters, (5) improved sensor design for usage in real-time systems, and (6) the development of multisensor systems providing the capability to acquired co- registered data. MODEL-BASED SYSTEM DESIGN AND SIGNAL PROCESSING FOR MATERIALS STATE AWARENESS Wm. Garth Frazier, Miltec Corporation In the field of systems health monitoring, there are many researchers working on various aspects of the problem: for example, sensor development, mathematical modeling, and signal- processing algorithms. However, there is still no overarching philosophy to help focus research contributions. As the field evolves away from merely detecting damage to estimating the entire material state, this overarching philosophy should address how systems health monitoring and traditional nondestructive evaluation are different and how they will complement each other in the new goal of materials state awareness. There is also a need for systematic methods for designing these systems to meet the new end user goals. A “systematic design method” is defined here to mean a method that can be applied to a very broad class of problems by changing the particulars of the application of interest and the desired outcome without changing the steps of the method. An attempt is made to identify a materials state awareness philosophy along with some of the scientific knowledge components and technological capabilities that are needed to develop systematic design methods and how they can be used by these methods to achieve specified outcomes. This naturally leads to the question of how these desired outcomes can be specified in a quantitative way to achieve materials state awareness. As defined here, material state is that set of quantitative information about a material system that, when known at an instant of time, along with the value of all current and future independent influences on its behavior, is sufficient to know that same set of quantitative information for all future time. A simple example is that knowing the velocity and density distributions (the state) in a flow field along with the nonsteady (time-varying) boundary conditions is sufficient to know these distributions for all future time. This is so because we know from mechanics that the equation of continuity and the equation of momentum balance must hold. In addition, we have models of the material constitutive relations. Drawing on the now-well-developed (and successful) fields of automatic control system design and signal processing as examples, it is proposed that the most essential scientific knowledge component required to perform successful designs using a systematic methodology is the availability of mathematical models (usually nonsteady, nonlinear differential equations), of sufficient fidelity, that are able to predict the evolution of the states that are desired to be estimated over the range of anticipated conditions. To achieve materials state awareness as a goal, in addition to mechanical (continuum) state awareness, an aggressive approach to the

OCR for page 37
40 Proceedings of a Workshop on Materials State Awareness identification of the appropriate material states and development of quantitative models of their evolution in their operating environments is needed. Moreover, it is anticipated that these models will need to include a stochastic component in order to provide a satisfactory description. For most industrial-scale problems, these models will have to be analyzed and simulated using numerical methods, for example, dynamic finite-element techniques. For practical systems there will be a need for adequate monitoring (passive sensing) and interrogation (active sensing) capabilities in order reduce the effect of imperfect modeling and uncontrolled, unmeasured influences on the system’s state evolution. In this context, adequate monitoring or interrogation does not imply that direct or instantaneous measurement of the state is required. It does mean, however, that the ability to obtain stable estimates of the state as time passes by using the available data is required. This can be achieved even when the system is nonsteady and nonlinear by using the well-established signal processing technique traditionally referred to as a model-based observer. Therefore, the essential technological capability that is needed is a sufficient variety of actuating and sensing technologies from which a designer can choose, or better yet, from which a systematic design methodology can choose. Needless to say, models of how these sensors and actuators interact with the structure and material system are required. The need for satisfactory models, as well as adequate actuating and sensing technologies, is not a surprise to anyone involved in the field of NDE. But a method for formally quantifying and achieving what is meant by adequate and sufficient when trying to achieve complex design goals for a wide variety of materials state awareness applications might be less obvious, especially when the goals are not limited to a single traditional measure of performance. When trying to achieve several figures-of-merit such as false call rate, cost per inspection, and so forth for a design, in addition to the commonly used one-time interrogation probability of detection, a systematic, mathematically based design optimization method is likely to be the only feasible way to achieve consistent design results that are not prejudiced by a human designer’s preconceived notions. This is not because human designers are always biased, but it is because in order to make a highly complex problem with many decision variables tractable, the human designer needs to fix some decision variables in advance to keep the size of the decision space manageable. This can easily lead to a suboptimal solution. Therefore, the synthesizing element for materials state awareness system design is the use of model-based engineering design algorithms, of which there are many good ones such as the classical methods of mathematical programming, pattern search algorithms, genetic algorithms, simulated annealing, and particle swarm optimization (a recent area of research interest in the field of design algorithms). The most appropriate algorithm to use will depend on the mathematical structure of the particular problem, but frequently this choice does not affect the outcome, but instead only the time it takes to find a solution. In addition, it is very important to realize at this stage of the development of these research areas that the resulting optimal solution to a particular problem may prescribe the use of integrated sensors and signal processing, which provide only course estimates of the material state that trigger the use of higher-fidelity interrogation methods to reduce the state uncertainty to a satisfactory level. In other words, a mix of global and local methods for obtaining state awareness should not be disregarded in advance as a good design solution. In summary, it is proposed that the problem of designing a materials state awareness system for a component, multiple components, or as part of a larger system, in a systematic way is more closely related to the problem of designing a quantitative state estimation system than a damage detection system. It requires at a minimum (1) the availability of mathematical models

OCR for page 37
Session V: What Is the Future of Material State Awareness? 41 (likely stochastic) that describe the evolution of the states under all expected environments and a method to analyze these models numerically; (2) monitoring and interrogation technologies that provide a means to periodically estimate either directly, or indirectly by using model-based observers, the values of the states as time passes; (3) a mathematical description of design criteria for the particular application, that is, objectives, constraints, and decision variables; and (4) an algorithm that uses items 1 through 3 to manipulate the decision variables to find a solution or multiple solutions that achieve the design criteria. In light of this, it is proposed that the weakest link is the availability of the definition and models for the evolution of material state—that is, our understanding of the relevant physics. This is followed by the potential lack of availability of appropriate sensors and actuators for particular applications—for example, environmentally extreme environments. SYSTEM STATE AWARENESS: AN INTEGRATED PERSPECTIVE Thomas Cruse, Vanderbilt University, Emeritus Materials state awareness (MSA) goes beyond traditional NDE in its challenge to characterize the current state of material damage long before the onset of macro-damage such as cracks. MSA must link nontraditional and innovative NDE with advances in microstructurally based damage progression modeling. Such modeling is tied to the variability in material microstructure, microstress, and processing history. System state awareness refers to the global application of this proposed integrated damage modeling for the entire life cycle of any structural system. System state awareness requires the integration of three critical technologies: (1) high- fidelity life-prediction models that include processing and usage history, (2) high-fidelity characterization of the mechanical and environmental “loading” history, and (3) the ability to provide real-time MSA. Such an approach to system state awareness requires advances in all three technologies, but the primary advance has to be in new approaches and capabilities in NDE for MSA.

OCR for page 37

OCR for page 37
Appendixes

OCR for page 37