nonmilitary applications include (1) medical patient diagnosis or assessment based on integrating information from diagnostic tests, vital signs, patient symptoms, and so forth; (2) weather forecasting using ground-, aircraft-, subsurface-, and satellite-based sensors; and (3) robot navigation using map and visualization sources. There are several issues that need to be addressed in the design and development of a data fusion-based system, including: (1) the number and type of information sources, (2) what information from each source is to be used to get the most from the data fusion process, (3) the operating conditions of the system for which data fusion will foster enhanced system operation, (4) the system architecture to provide for where and how the data from different sources are to be fused, (5) the algorithm type and choice that are appropriate for the data sources and the application, (6) the target level of performance of the data fusion process, and (7) the ability of the system to adapt in a dynamic environment.
Data fusion can be used to translate between observed values from one or more sources for an entity and a decision or inference related to the entity at different levels of complexity. Data fusion can be used for data alignment or registration from multiple sensors such as a robotic unit determining positional information from a mounted camera and the Global Positioning System. Data fusion can also be used for enhancing classification tasks, where individual or multiple modalities and/or sensors can be merged at the raw data, feature, or decision level for contributing to classification decisions. For performing raw data fusion, there is typically some common domain to integrate raw data from multiple sources. For example, ultrasound and radiographic images of an aircraft panel may be fused at the pixel level based on mapping or registering the corresponding pixel locations from the two image sources. Approaches for performing raw data fusion commonly involve standard detection, estimation, and registration methods. Feature-level fusion typically involves extracting representative attributes from objects or entities of interest from the source data. Using the aircraft example, if the ultrasound and radiographic images for a panel are used separately, attributes related to size and shape may be determined from objects that may be potential areas of corrosion. The attributes or features from the ultrasound and radiographic images for each object may be combined into a single-feature vector for describing the object. The generated feature vectors are input into classification algorithms such as template matching, clustering methods, or neural networks. Decision-level fusion can involve combining source information after a preliminary decision or confidence has been determined for each source related to an entity’s detection, position, presence of a specific feature, or identity. Decision-level fusion may also involve combining single-source information based on multiple approaches for generating preliminary decisions or confidence values related to the entity. Techniques for performing decision-level fusion include voting and weighting schemes and other computational intelligence methods. An important consideration for performing raw data, feature, or decision-level fusion is data normalization. Data normalization is often necessary to allow data collected from different sources to be directly compared or combined. For example, in collecting an ultrasound and a radiograph image of the same aircraft panel, the sensor values will typically have different value ranges. In order to perform raw data fusion, part of the fusion process is to standardize the values from the ultrasound and radiograph images for combining the values to generate a single fused image.
The utilization of data fusion-based systems is becoming more prevalent as the need increases for systems with enhanced decision- and inference-making capability. Accordingly, future considerations in the development of data fusion-based systems would appear to include the development of technology that will allow data fusion capability to be transparently integrated into the system development and manufacturing processes for a wide range of