16R have proposed a policy for IP
traffic precedence that can enable a graceful transition into a
more competitively supplied Internet market that might reduce the
need for federal interest in regulation. Their paper discusses how
taking advantage of existing IP functionality to use multiple
levels of service precedence can begin to address disparities
between the requirements of conventional and newer and more highly
demanding applications.
Our approach thus far been based on the belief that at least so
far, the less regulation of the Internet—and in fact the more
progress in removing regulatory barriers for existing
telecommunications companies so they can participate more
effectively in the Internet market—the better. However,
regulation may be necessary in order to foster a healthy
competitive network environment where consumers can make educated
decisions regarding which network service providers to patronize. A
key requirement in analyzing the relative performance of and
customer satisfaction with network service providers is public
availability of statistics that measure their capacity,
reliability, security, integrity, and performance.
Recommendations
In this section we highlight several examples of recommended
statistics collection objects and tools.
Reaching consensus on the definition of a community-wide
statistics collection architecture will require cooperation between
the private and public sectors. We hope that key federally
sponsored networks such as the NSF very high speed backbone network
service (vBNS) 17R and the federally
sponsored NAPs can serve as a role model in providing an initial
set of statistics to the community and interacting with the
research community to refine metrics as research reveals the
relative utility of various ones.
Existing Work
The last community-wide document of which we are aware was
Stockman's RFC 1404 18R, "A Model
for Common Operational Statistics." Since that time the Internet
environment has changed considerably, as have the underlying
technologies for many service providers such as the NAPs. As a
result these specific metrics are not wholly applicable to every
service provider, but they serve as a valuable starting point. We
emphasize that the exact metrics used are not a critical decision
at this point, since refinements are inevitable as we benefit from
experience with engineering the technologies; what is essential is
that we start with some baseline and create a community facility
for access and development in the future.
OCR for page 92
Page 92
From Stockman 19R:
The metrics used in evaluating network traffic could be
classified into (at least) four major categories:
•
Utilization metrics (input and output packets and
bytes, peak metrics, per protocol and per application volume
metrics)
•
Performance metrics (round trip time [rtt] on
different protocol layers, collision count on a bus network, ICMP
Source Quench message count, count of packets dropped)
•
Availability metrics (longer term) (line, route,
or application availability)
•
Stability metrics (short-term fluctuations that
degrade performance) (line status transitions, fast route changes
(flapping), routes per interface in the tables, next hop count
stability, short-term ICMP anomalous behavior).
Some of these objects are part of standard simple network
management protocol (SNMP) MIBs; others, of private MIBs. Others
are not possible to retrieve at all due to technical limitations;
i.e., measurement of a short term problematic network situation
only exacerbates it or takes longer to perform than the problem
persists. For example, counts of packets and bytes, for non-unicast
and unicast, for both input and output are fairly standard SNMP
variables. Less standard but still often supported in private MIBs
are counts of packet discards, congestion events, interface resets,
or other errors. Technically difficult variables to collect, due to
the high resolution polling required, include queue lengths and
route changes. Although such variables would be useful for many
research topics in Internet traffic characterization, operationally
collected statistics will likely not be able to support them. For
example, one characteristic of network workload is "burstiness,"
which reflects variance in traffic rate. Network behavioral
patterns of burstiness are important for defining, evaluating, and
verifying service specifications, but there is not yet agreement in
the Internet community on the best metrics to define burstiness.
Several researchers 20R have
explored the failure of Poisson models to adequately characterize
the burstiness of both local and wide-area Internet traffic. This
task relies critically on accurate packet arrival timestamps, and
thus on tools adequate for packet tracing of the arrivals of
packets at high rates with accurate (microsecond) time
granularities. Vendors may still find incentive in providing
products that can perform such statistics collection, for customers
that need fine-grained examination of workloads.
The minimal set of metrics recommended for IP providers in
Stockman 21R were packets and bytes
in and out (unicast and non-unicast) of each interface, discards in
and out of each interface, interface status, IP forwards per node,
IP input discards per node, and system uptime. All of the
recommended metrics were available in the Internet standard MIB.
The suggested polling frequency was 60 seconds for unicast packet
and byte counters, and an unspecified multiple of 60 seconds for
the others. Stockman also suggested aggregation periods for
presenting the data by interval: over 24-hour, 1-month, and 1-year
periods, aggregate by 15 minutes, 1 hour, and 1 day, respectively.
Aggregation includes calculating and storting the average and
maximum values for each period.
Switched Environments
In switched environments, where there is no IP layer
information, the above statistics are not completely applicable.
Without demand from their customer base, many switch vendors have
put collection of statistics at a second priority since it tends to
detract from forwarding performance anyway. Some ATM switches can
collect per-VC (virtual circuit) statistics such as those described
in the ATM MIB 22R.
One alternative for providers that support IP over an internal
ATM infrastructure is to collect the IP statistics described above,
and in fact use objects such as host-host matrices to plan what
number and quality of ATM virtual circuits might be necessary to
support the IP traffic workload. For switched FDDI environments,
the provider could collect statistics on each LAN coming into the
switch, and collapse it during analysis to determine possible
compound effects within the switch, in addition segmenting traffic
by interface, customer, and perhaps by protocol/application. If
there is no access to network layer information, such as at NSF
NAPs or certain ATM switches, the network service provider will
still have an interest in these statistics, since sorting the
OCR for page 93
Page 93
resulting arrays would give the NSP an indication of what
fraction of traffic comes from what number of users, which may be
critical for planning switched or permanent virtual circuit
configuration, and by the same token for accounting and capacity
planning purposes. However, converting virtual circuits to end
customers, most likely IP customers for the near future, requires
maintaining mappings to higher layers.
Dedicated Studies of IP Workloads
Even with a solid set of operational statistics there are times
when one wants dedicated collection to gain greater insight into
short-term dynamics of workloads. For example, there are
limitations of the operationally collected statistics for the
NSFNET for describing flows in terms of their impact on an
aggregate Internet workload 23R. We
have developed tools for supporting operational flow assessment, to
gain insight into both individual traffic signatures as well as
heavy aggregations of end users. We have tested our methodology
using packet header traces from a variety of Internet locations,
yielding insight far beyond a simple aggregated utilization
assessment, into details of the composition of traffic aggregation,
e.g., what components of the traffic dominate in terms of flow
frequency, durations, or volumes. We have shown that shifts in
traffic signatures as a result of evolving technologies, e.g.,
toward multimedia applications, will require a different approach
in network architectures and operational procedures. In particular,
the much higher demands of some of these new applications will
interfere with the ability of the network to aggregate the
thousands of simultaneous but relatively short and low-volume flows
that we observe in current environments.
The methodology 24R defines a
flow based on actual traffic activity from, to, or between
entities, rather than using the explicit setup and teardown
mechanisms of transport protocols such as TCP. Our flow metrics
fall into two categories: metrics of individual flows and metrics
of the aggregate traffic flow. Metrics of individual flows include
flow type, packet and byte volume, and duration. Metrics of the
aggregate flow, or workload characteristics seen from the network
perspective, include counts of the number of active, new, and timed
out flows per time interval; flow interarrival and arrival
processes; and flow locality metrics. Understanding how individual
flows and the aggregate flow profile influence each other is
essential to securing Internet stability, and requires ongoing flow
assessment to track changes in Internet workload in a given
environment.
Because flow assessment requires comprehensive and detailed
statistics collection, we recognize that the NAPs and other service
providers may not be able to afford to continuously monitor flow
characteristics on an operational basis. Nonetheless we imagine
that NAP operators will find it useful to undertake traffic flow
assessment at least periodically to obtain a more accurate picture
of the workload their infrastructure must support. The methodology
and tools that implement it4 will be
increasingly applicable, even on a continuous basis, for NAP tasks
such as ATM circuit management, usage-based accounting, routing
table management, establishing benchmarks by which to shop for
equipment from vendors, and load balancing in future Internet
components.
The methodology can form a complementary component to other
existing operational statistics collection, yielding insights into
larger issues of Internet evolution, e.g., how environments of
different aggregation can cope with contention for resources by an
ever-changing composition and volume of flows. Internet traffic
cross-section and flow characteristics are a moving target, and we
intend that our methodology serve as a tool for those who wish to
track and keep pace with its trajectory. For example, as video and
audio flows and even single streams combining voice and audio
become more popular, Internet service providers will need to
parametrize them to determine how many such end user streams they
will be able to support and how many more resources each new such
stream would require. Multicast flows will also likely constitute
an increasingly significant component of Internet traffic, and
applying our methodology to multicast flows would be an important
step toward coping with their impact on the infrastructure.
OCR for page 94
Page 94
Community Exchange
A key requirement in analyzing the relative performance of and
customer satisfaction with network service providers is public
availability of statistics that measure their capacity,
reliability, security, integrity, and performance. One vehicle
conducive to the development of an effective marketplace is a
client/server-based architecture for providing access to statistics
of various NSPs. Each NSP would support a server that provides
query support for collected statistics for its clients or potential
customers. MCI provides this functionality for statistics collected
on the NSF-sponsored vBNS network. Community-wide participation in
such an open forum would foster a healthy competitive network
environment where consumers could make educated decisions regarding
which network service providers to patronize.
Conclusion
When we get through we won't be done.
—Steve Wolff, then director, DNCRI, NSF on the NSFNET
transition (one among many since 1983) at 1994 Farnet meeting
We conclude by mentioning an issue of recent popular interest as
well as symbolic of the larger problem: Internet security and
prevention of criminal behavior. Ironically, workload and
performance characterization issues are inextricably intertwined
with security and privacy. Much of the talk about the Internet's
inherent insecurity due to the inability to track traffic at the
required granularity is misleading. It is not an inability
to examine traffic operationally that has prevented it thus far,
whether for security or workload characterization (and the same
tools could do both), but rather its priority relative to the rest
of the community research agenda.
As a result, the gap has grown large between the unambiguous
results of confined experiments that target isolated environments,
and the largely unknown characteristics of the extensive Internet
infrastructure that is heading toward global ubiquity.
Empirical investigation and improved methodology for doing so
can improve current operational statistics collection
architectures, allowing sercice providers to prepare for more
demanding use of the infrastructure and allowing network analysts
to develop more accurate Internet models. In short, we can
contribute to a greater understanding of real computer networks of
pervasive scale by reducing the gaps among (1) what network service
providers need; (2) what statistics service providers can provide;
and (3) what in-depth network analysis requires.
We encourage discussion as soon as possible within the community
on developing a policy on statistics collection/exchange/posting of
available NAP/Internet service provider statistics, with supporting
tools to allow greater understanding of customer requirements and
service models, equitable cost allocation models for Internet
services, verification that a given level of service was actually
rendered, and evolution toward a level of Internet performance that
matches or surpasses that of most telecommunication systems
today.
Author Information
Hans-Werner Braun is a principal scientist at the San Diego
Supercomputer Center. Current research interests include network
performance and traffic characterization, and working with NSF on
NREN engineering issues. He also participates in activities
fostering the evolution of the national and international
networking agenda. San Diego Supercomputer Center, P.O. Box 85608,
San Diego, CA 92186-9784; email address: hwb@sdsc.edu.
Kimberly Claffy received her doctoral degree from the Department
of Computer Science and Engineering at the University of
California, San Diego in June 1994, and is currently an associate
staff scientist at the San Diego Supercomputer Center. Her research
focuses on establishing and improving the efficacy of traffic and
performance characterization methodologies on wide-area
communication networks, in particular to cope with the
OCR for page 95
Page 95
changing traffic workload, financial structure, and underlying
technologies of the Internet. San Diego Supercomputer Center, P.O.
Box 85608, San Diego, CA 92186-9784; email address:
kc@sdsc.edu.
References
[1] Braun, H.-W., C.E. Catlett, and K.
Claffy. 1995. "http://vedana.sdsc.edu/," tech. rep., SDSC and NCSA,
March. San Diego Supercomputer Center.
[2] Braun, H.-W., C.E. Catlett, and K.
Claffy. 1995. "National Laboratory for Applied Network Research,"
tech. rep., SDSC and NCSA, March. San Diego Supercomputer
Center.
[3] Caceres, R., P. Danzig, S. Jamin, and
D. Mitzel. 1991. "Characterictics of Wide-area TCP/IP
Conversations," Proceedings of ACM SIGCOMM '91, pp. 101–112,
September.
[4] Paxson, V. 1994. "Empirically-Derived
Analytic Models of Wide Area TCP Connections," IEEE/ACM
Transactions on Networking 2(4), August.
[5] Leland, W., M. Taqqu, W. Willinger,
and D. Wilson. 1994. "On the Self-similar Nature of Ethernet
Traffic (extended version)," IEEE/ACM Transactions on Networking,
February.
[6] Paxson, V., and S. Floyd. 1994.
"Wide-area Traffic: The Failure of Poisson Modeling," Proceedings
of ACM SIGCOMM '94, February.
[7] Claffy, K., H.-W. Braun, and G.C.
Polyzos. 1993. "Long-term Traffic Aspects of the NSFNET,"
Proceedings of INET '93, pp. CBA—1:10, August.
[8] Rekhter, Y. 1995. "Routing in a
Multi-provider Internet," Internet Request for Comments Series RFC
1787, April.
[9] See Claffy, K., H.-W. Braun, and G.C.
Polyzos. 1993. "Long-term Traffic Aspects of the NSFNET,"
Proceedings of INET '93, pp. CBA—1:10, August; Davis, M. 1988.
"Analysis and Optimization of Computer Network Routing,"
unpublished Master's thesis, University of Delaware; Heimlich, S.
1988. "Traffic Characterization of the NSFNET National Backbone,"
Proceedings of the 1990 Winter USENIX Conference, December; Claffy,
K., G.C. Polyzos, and H.-W. Braun. 1993. "Traffic Characteristics
of the T1 NSFNET Backbone," Proceedings of IEEE Infocom 93, pp.
885–892, March; Claffy, K. 1994. ''Internet Workload
Characterization," Ph.D. thesis, University of California, San
Diego, June; and Claffy, K., H.-W. Braun, and G.C. Polyzos. 1994.
"Tracking Long-term Growth of the NSFNET Backbone," Communications
of the ACM, August, pp. 34–45.
[10] Caceres, R., P. Danzig, S. Jamin, and
D. Mitzel. 1991. "Charactericstics of Wide-area TCP/IP
Conversations," Proceedings of ACM SIGCOMM '91, pp. 101–112,
September.
[11] Schmidt, A., and R. Campbell. 1993.
"Internet Protocol Traffic Analysis with Applications for ATM
Switch Design," Computer Communications Review, April, pp.
39–52.
[12] Danzig, P.B., K. Obraczka, and A.
Kumar. 1992. "An Analysis of Wide-area Name Server Traffic,"
Proceedings of ACM SIGCOMM '92, August.
[13] Wolff, H., and the IETF Opstat
Working Group. 1995. "The Opstat Client-server Model for Statistics
Retrieval," April, draft-ietf-opstat-client-server-03.txt.
[14] See Caceres, R., P. Danzig, S. Jamin,
and D. Mitzel. 1991. "Characteristics of Wide-area TCP/IP
Conversations," Proceedings of ACM SIGCOMM '91, pp. 101–112,
September; and Jain, R., and S.A. Routhier. 1986. "Packet
Trains—Measurement and a New Model for Computer Network
Traffic," IEEE Journal on Selected Areas in Communications, Vol. 4,
September, pp. 986–995.
[15] Braun, H.-W., and K. Claffy. 1994.
"Web Traffic Characterization: An Assessment of the Impact of
Caching Documents from NCSA's Web Server," Second International
World Wide Web Conference, October.
[16] Bohn, R., H.-W. Braun, K. Claffy, and
S. Wolff. 1994. "Mitigating the Coming Internet Crunch: Multiple
Service Levels via Precedence," Journal of High Speed Networks,
forthcoming. Available by anonymous ftp from
ftp.sdsc.edu:pub/sdsc/anr/papers/ and
http://www.sdsc.edu/0/SDSC/Research/ANR/kc/precedence/precedence.html.
[17] Braun, H.-W., C.E. Catlett, and K.
Claffy. 1995. "http://vedana.sdsc.edu/," tech. rep., SDSC and NCSA,
March. San Diego Supercomputer Center.
[18] Stockman, B. 1993. "A Model for
Common Operational Statistics," RFC 1404, January.
[19] Stockman, B. 1993. "A Model for
Common Operational Statistics," RFC 1404, January.
[20] See Willinger, W. 1994.
"Self-similarity in High-speed Packet Traffic: Analysis and
Modeling of Ethernet Traffic Measurements," Statistical Science;
Garrett, M.W., and W. Willinger. 1994. "Analysis, Modeling and
Generation of Self-Similar VBR Video Traffic," Proceedings of ACM
SIGCOMM '94, September; and Paxson, V., and S. Floyd. 1994.
"Wide-area Traffic: The Failure of Poisson Modeling," Proceedings
of ACM SIGCOMM '94, February.
[21] Stockman, B. 1993. "A Model for
Common Operational Statistics," RFC 1404, January.
OCR for page 96
Page 96
[22] Ahmed, M., and K. Tesink. 1994.
"Definitions of Managed Objects for ATM Management, Version 7.0,"
Internet Request for Comments Series RFC 1695, August.
[23] Claffy, K., G.C. Polyzos, and H.-W.
Braun. 1995. "Internet Traffic Flow Profiling," IEEE Journal on
Selected Areas in Communications, forthcoming.
[24] Claffy, K., G.C. Polyzos, and H.-W.
Braun. 1995. "Internet Traffic Flow Profiling," IEEE Journal on
Selected Areas in Communications, forthcoming.
Notes
1. See http://www.merit.edu for more
information on the NSFNET transition.
2. Specifically, NSF-sponsored regional
providers, i.e., those that received funding from the NSF
throughout the life of the NSFNET, will only continue to receive
funding if they connect to all three NSF NAPs. Even so, the funding
will gradually diminish within 4 years, an interval of time in
which regional providers will have to become fully self-sustaining
within the marketplace.
3. North American Network Operators Group,
International Engineering Planning Group, Federal Engineering and
Planning Group, Engineering and Operations Working Group, Federal
Association of Regional Networks.
4. The software we used is available at
ftp://ftp.sdsc.edu/pub/sdsc/anr/software/Flows.tar.Z.
Representative terms from entire chapter:
statistics collection
Items in cart [0]
