Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
A STREAM PROCESSOR FOR EXTRACTING USAGE INTELLIGENCE FROM HIGH-MOMENTUM INTERNET DATA 315 Figure 4. The events of the stream are converted into Normalized Metered Events and passed directly to a rule engine. Once collected, the data of an ME are normalized into a common data structure called a Normalized Metered Event (NME, see Figure 4), which is an array of attributes that contain different data types similar to a DB record. The only attribute that is required is an end-time stamp. All the other attributes can be configured to suit the processing needs of the application. Unlike DB record schemas the number and type of attributes can change dynamically as it undergoes processing. In this stream processing context new attributes can be dynamically âadornedâ to the NME and be used as intermediate variables, which travel along with the NME in the stream, and then disposed of when no longer needed. (The word âtravelâ is only a metaphor. The NME object doesn't actually move; its position in the stream is tracked by passing a small reference or âpointerâ to the NME object from rule to rule.) 7. STREAM RULE PROCESSING Once normalized, the NMEs move directly into a Rule Engine, which has been specifically designed for merging, processing, and splitting streams. The input streams can be independent or related (such as a usage stream and a session stream). In the rule engine (Figure 5) there can be multiple rule chains that operate on the input streams. It is possible to have a single stream processed by multiple rule chains each producing distinctly different output streams, or multiple similar streams processed by a single rule chain, or combinations of the above. There are several forms that the output streams can take: (1) During processing of an NME, attributes within the NME are added, modified, or deleted and the result NME is forwarded immediately to a downstream Collector for further processing. (2) A cyclic aggregation time interval is configured into the Collector and rule chains are configured to
