draft-ietf-tewg-measure-04.txt   draft-ietf-tewg-measure-05.txt 
Traffic Engineering Working Group Wai Sum Lai Traffic Engineering Working Group Wai Sum Lai
Internet Draft AT&T Labs Internet Draft AT&T Labs
Document: <draft-ietf-tewg-measure-04.txt> Document: <draft-ietf-tewg-measure-05.txt>
Category: Informational Richard W. Tibbs Category: Informational Richard W. Tibbs
Oak City Networks & Oak City Networks &
Solutions Solutions
Steven Van den Berghe Steven Van den Berghe
Ghent University/IMEC Ghent University/IMEC
January 2003 Febuary 2003
A Framework for Internet Traffic Engineering Measurement A Framework for Internet Traffic Engineering Measurement
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
skipping to change at page 1, line 40 skipping to change at page 1, line 40
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
1. Abstract 1. Abstract
In this document, a measurement framework for supporting the traffic In this document, a measurement framework for supporting the traffic
engineering of IP-based networks is presented. Uses of traffic engineering of IP networks is presented. Uses of traffic
measurement in service provider environments are described, and measurement in service provider environments are described, and
issues related to time scale and read-out period are discussed. issues related to time scale and read-out period are discussed.
Different measurement types are classified, with each being Different measurement types are classified, with each being
specified as a meaningful combination of a measurement entity and a specified as a meaningful combination of a measurement entity and a
measurement basis. measurement basis.
For interoperable compatibility, uniform definitions across vendors For interoperable compatibility, uniform definitions across vendors
and operators must be ensured, e.g., in the distinction between and operators must be ensured, e.g., in the distinction between
offered load and achieved throughput. To aid network dimensioning, offered load and achieved throughput. To aid network dimensioning,
mechanisms to collect node-pair-based traffic data should be mechanisms to collect node-pair-based traffic data should be
skipping to change at page 2, line 13 skipping to change at page 2, line 13
detail at manageable sample volumes, there is a need for packet- detail at manageable sample volumes, there is a need for packet-
sampled measurements. To manage large volume of measured data, use sampled measurements. To manage large volume of measured data, use
of bulk transfer and filtering/aggregation mechanisms may be of bulk transfer and filtering/aggregation mechanisms may be
appropriate. appropriate.
Table of Contents Table of Contents
Status of this Memo................................................1 Status of this Memo................................................1
1. Abstract........................................................1 1. Abstract........................................................1
2. Conventions used in this document...............................2 2. Conventions used in this document...............................2
3. Introduction....................................................2 3. Introduction....................................................3
4. Terminology.....................................................4 4. Terminology.....................................................4
4.1 Route, path....................................................4 4.1 Active, passive measurements...................................4
4.2 Throughput, traffic volume.....................................4 4.2 Route, path....................................................5
5. Uses of Traffic Measurement.....................................5 4.3 Throughput, traffic volume.....................................5
5.1 Traffic characterization.......................................5 5. Uses of Traffic Measurement.....................................6
5.1 Traffic characterization.......................................6
5.2 Network monitoring.............................................6 5.2 Network monitoring.............................................6
5.3 Traffic control................................................6 5.3 Traffic control................................................7
6. Time Scales for Network Operations..............................7 6. Time Scales for Network Operations..............................7
7. Read-Out Periods................................................7 7. Read-Out Periods................................................8
8. Measurement Bases...............................................9 7.1 Data Reduction.................................................8
8.1 Flow-based....................................................10 7.2 Measurement Interval...........................................9
8.2 Interface-based, link-based, node-based.......................10 7.3 Summarization..................................................9
8.3 Node-pair-based...............................................11 7.4 Sampling Issues................................................9
8.4 Path-based....................................................11 8. Measurement Bases..............................................10
9. Measurement Entities...........................................12 8.1 Flow-based....................................................11
9.1 Entities related to traffic and performance...................12 8.2 Interface-based, link-based, node-based.......................12
9.2 Entities related to establishment of connection or path.......14 8.3 Node-pair-based...............................................12
10. Measurement Types.............................................15 8.4 Path-based....................................................13
10.1 Measurement types related to traffic or performance..........15 9. Measurement Entities...........................................13
10.2 Measurement types related to resource usage..................16 9.1 Entities related to traffic and performance...................13
11. Traffic Matrix Statistics.....................................16 9.2 Entities related to establishment of connection or path.......16
12. Performance Monitoring........................................17 10. Measurement Types.............................................16
13. Packet Sampling...............................................18 10.1 Measurement types related to traffic or performance..........16
14. Statistical Estimation and Information Modeling...............19 10.2 Measurement types related to resource usage..................17
14.1 Engineering methods for statistical estimation of measures...19 11. Traffic Matrix Statistics.....................................18
14.2 TE Measure Information Modeling..............................20 12. Performance Monitoring........................................19
15. Conclusions and Recommendations...............................22 13. Packet Sampling...............................................20
16. Security Considerations.......................................22 14. Statistical Estimation and Information Modeling...............20
17. References....................................................22 14.1 Engineering methods for statistical estimation of measures...21
18. Intellectual Property Statement...............................25 14.2 TE Measure Information Modeling..............................21
19. Acknowledgments...............................................25 15. Conclusions and Recommendations...............................24
20. Author's Addresses............................................25 16. Security Considerations.......................................24
Full Copyright Statement..........................................25 17. References....................................................24
18. Intellectual Property Statement...............................27
19. Acknowledgments...............................................27
20. Author's Addresses............................................27
Full Copyright Statement..........................................28
2. Conventions used in this document 2. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in RFC-2119. this document are to be interpreted as described in RFC-2119.
3. Introduction 3. Introduction
This document describes a framework for Internet traffic engineering This document describes a framework for Internet traffic engineering
measurement, with the objective of providing principles for the measurement, with the objective of providing principles for the
development of a set of measurement systems to support the traffic development of a set of measurement systems to support the traffic
engineering of IP-based networks [1]. A major goal is to provide engineering of IP networks [1]. A major goal is to provide the
the scope of measurements involved and outline a context for scope of measurements involved and outline a context for
establishing operator-, platform-, protocol-, and vendor-neutral establishing operator-, platform-, protocol-, and vendor-neutral
traffic measurement standards. To achieve multi-vendor inter- traffic measurement standards. To achieve multi-vendor inter-
operability, it is critical to minimize the possibilities of operability, it is critical to minimize the possibilities of
inconsistencies arising from, e.g., differing statistical inconsistencies arising from, e.g., differing statistical
definitions, overlapping data collection, processing at different definitions, overlapping data collection, processing at different
protocol levels, and similar inconsistencies by different vendors or protocol levels, and similar inconsistencies by different vendors or
network operators. network operators.
The need for a common framework, including identification, The need for a common framework, including identification,
principles and scope of measurements, is motivated by the needs for principles and scope of measurements, is motivated by the needs for
skipping to change at page 3, line 36 skipping to change at page 3, line 42
performance measurement process at a high level. We point to performance measurement process at a high level. We point to
objectives that a comprehensive set of measurement standards should objectives that a comprehensive set of measurement standards should
achieve. We also list brief informal definitions for most achieve. We also list brief informal definitions for most
measurements of interest, but leaving exhaustive and precise measurements of interest, but leaving exhaustive and precise
definitions and standards to the documents cited or subsequent definitions and standards to the documents cited or subsequent
documents to be developed by other working groups. documents to be developed by other working groups.
The scope of this document is limited to those aspects of The scope of this document is limited to those aspects of
measurement pertaining to intra-domain operations, i.e., within a measurement pertaining to intra-domain operations, i.e., within a
given autonomous system. However, measurements on its boundary with given autonomous system. However, measurements on its boundary with
other domains are included as well. The focus is primarily on other domains are taken into consideration as well. The focus is
traffic engineering in Internet service provider environments. primarily on traffic engineering in Internet service provider
environments.
In this document, uses of traffic measurement in traffic In this document, uses of traffic measurement in traffic
characterization, network monitoring, and traffic control are first characterization, network monitoring, and traffic control are first
described. Depending on the network operations to be performed in described. Depending on the network operations to be performed in
these tasks, three different time scales can be identified, ranging these tasks, three different time scales can be identified, ranging
from months, through days or hours, to minutes or less. To support from months, through days or hours, to minutes or less. To support
these operations, traffic measurement must be able to capture these operations, traffic measurement must be able to capture
accurately, within a given confidence interval, the traffic accurately, within a given confidence interval, the traffic
variations and peaks without degrading network performance and variations and peaks without degrading network performance and
without generating an immense amount of data. As one consequence of without generating an immense amount of data. As one consequence of
skipping to change at page 4, line 16 skipping to change at page 4, line 23
data such as topological data and router configuration data, traffic data such as topological data and router configuration data, traffic
matrix and other relevant statistics can be derived for traffic matrix and other relevant statistics can be derived for traffic
engineering purposes. Traffic load measurement also plays a key engineering purposes. Traffic load measurement also plays a key
role in network performance management. role in network performance management.
In addition to these capabilities, functions of a measurement system In addition to these capabilities, functions of a measurement system
should also include data storage, data processing, statistics should also include data storage, data processing, statistics
generation and reporting. However, these aspects are outside the generation and reporting. However, these aspects are outside the
scope of this document. scope of this document.
Also, IP multicast traffic measurement is not explicitly addressed
in this document. Nonetheless, given additional elaboration on
tree-based measurement principles, most of the considerations for
different measurement types (to be discussed in Sections 8 and 9)
could be applied to IP multicast traffic. Such elaboration may be
dealt with in a subsequent document for specific IP multicast-
inferred Internet traffic measurement.
As a framework, this document is mainly concerned with a discussion As a framework, this document is mainly concerned with a discussion
of various technical issues surrounding traffic measurement, of various technical issues surrounding traffic measurement,
particularly in the area of statistical traffic load estimation for particularly in the area of statistical traffic load estimation for
traffic engineering purposes. As far as possible and to avoid traffic engineering purposes. As far as possible and to avoid
duplication of effort, relevant work done in measurements by other duplication of effort, relevant work done in measurements by other
standards organizations will be applied or adapted, and references standards organizations will be applied or adapted, and references
to them will be made. These include, in particular, to them will be made. These include, in particular,
. IP Performance Metrics (IPPM) Working Group of the IETF: its . IP Performance Metrics (IPPM) Working Group of the IETF: its
framework document [2] and the associated documents on individual framework document [2] and the associated documents on individual
skipping to change at page 4, line 37 skipping to change at page 4, line 52
. ITU-T: Recommendation I.380/Y.1540 [11] and Recommendation Y.1541 . ITU-T: Recommendation I.380/Y.1540 [11] and Recommendation Y.1541
[12] [12]
4. Terminology 4. Terminology
The intent of this section is not to provide definition or The intent of this section is not to provide definition or
description of terms used in this document. Rather, it is to description of terms used in this document. Rather, it is to
highlight the difference in usage of closely related terms. highlight the difference in usage of closely related terms.
4.1 Route, path 4.1 Active, passive measurements
These terms are used in the sense of [2]. In an active measurement,
test packets, or probes, are injected into the network. Data
collected about these packets are taken as representative of the
behavior of the network. Passive measurements are in-service, non-
intrusive, and so can be performed directly on the user traffic.
4.2 Route, path
A route is any unidirectional sequence of nodes and links, for A route is any unidirectional sequence of nodes and links, for
sending packets from a source node to a destination node. A path sending packets from a source node to a destination node. A path
refers to an MPLS tunnel, i.e., a label-switched path [13]. refers to an MPLS tunnel, i.e., a label-switched path (LSP) [13],
this LSP possibly being a traffic-engineered LSP. Measurements on
non-traffic-engineered LSPs may be collected to support the possible
future traffic-engineering of those LSPs. (Note: What is defined
as a route here is referred to as a path in [2]. The route/path
distinction is made here to facilitate applications to MPLS.)
It should be pointed out that there are also methods for creating It should be pointed out that there are also methods for creating
paths with other technologies such as frame relay or ATM. The paths with other technologies such as frame relay or ATM. The
measurement described in this document may apply to these measurement described in this document may apply to these
technologies with suitable adaptation. To simplify description, technologies with suitable adaptation. To simplify description,
reference is made to MPLS only in what follows. reference is made to MPLS only in what follows.
4.2 Throughput, traffic volume 4.3 Throughput, traffic volume
Both quantities can be applied to a network, a network segment, or Both quantities can be applied to a network, a network segment, or
an individual network element. an individual network element. Thus, measurement points need to be
appropriately defined when a specific measurement is to be performed
(e.g., from a given ingress node to another egress or a set of
egress nodes).
Throughput of a network, as a measure of delivered performance, Throughput of a network, as a measure of delivered performance,
refers to the maximum sustainable rate of transferring packets refers to the maximum sustainable rate of transferring packets
successfully across the network, under given network conditions, successfully across the network, under given network conditions,
e.g., a given traffic mix, while meeting quality of service (QoS) e.g., a given traffic mix, while meeting quality of service (QoS)
objectives. This usage is consistent with the definition of objectives. This usage is consistent with the definition of
throughput for a network interconnect device as specified in [14]. throughput for a network interconnect device as specified in [14].
For real-time network control, active measurement of throughput by For real-time network control, active measurement of throughput by
probing may be used to determine the currently available capacity of probing may be used to determine the currently available capacity of
a network to carry additional traffic. (In an active measurement, a network to carry additional traffic. (Note: Goodput is a related
test packets are injected into the network. Data collected about term referring to a proportion of the traffic successfully
these packets are taken as representative of the behavior of the transmitted; similarly, badput refers to a proportion of the traffic
network.) lost or being corrupted.)
Traffic volume, as a measure of the traffic carried, characterizes Traffic volume, reflecting the traffic carried, is the amount of
the level of traffic that a network is designed to support. traffic measured during a given period of time. Passive measurement
Passive, i.e., in-service non-intrusive, measurement of the traffic of the traffic volume is usually used to estimate the long-term
volume is usually used to estimate the long-term offered traffic for offered traffic for the purposes of network dimensioning in the
the purposes of network dimensioning in the capacity-management and capacity-management and network-planning processes (see the Section
network-planning processes (see the Section on Time Scales for on Time Scales for Network Operations). A network should be
Network Operations). A network should be properly dimensioned so properly dimensioned so that its throughput is adequate to handle
that its throughput is adequate to handle the expected traffic the expected traffic volume. Hence, traffic volume measurement
volume. should be performed on a regular basis.
Throughput at a cross-section, or specific point in the network, is Throughput at a cross-section, or specific point in the network, is
expressed in terms of number of data units per time unit. Traffic expressed in terms of number of data units per time unit. Traffic
volume is expressed in data units with reference to a read-out volume is expressed in data units with reference to a read-out
period (see the Section on Read-Out Periods). For transmission period (see the Section on Read-Out Periods). For transmission
systems, the data unit is usually a multiple of either bits or systems, the data unit is usually a multiple of either bits or
bytes. For processing systems, the data unit is usually a multiple bytes. For processing systems, the data unit is usually a multiple
of packets. of packets.
5. Uses of Traffic Measurement 5. Uses of Traffic Measurement
skipping to change at page 6, line 14 skipping to change at page 6, line 45
parametric stochastic model based on the mean and variance of the parametric stochastic model based on the mean and variance of the
time series can be constructed. This traffic model is then used in time series can be constructed. This traffic model is then used in
the ensuing phases of traffic engineering, such as link dimensioning the ensuing phases of traffic engineering, such as link dimensioning
to meet service objectives. to meet service objectives.
5.2 Network monitoring 5.2 Network monitoring
. Determining the operational state of the network, including fault . Determining the operational state of the network, including fault
detection. detection.
. Monitoring the continuity and quality of network services, to . Monitoring the continuity and quality of network services, to
ensure that QoS/GoS objectives are met for various classes of ensure that QoS/CoS objectives are met for various classes of
traffic, to verify the performance of delivered services, or to traffic, to verify the performance of delivered services, or to
serve as a means of sectionalizing performance issues seen by a serve as a means of sectionalizing performance issues seen by a
customer. [QoS reflects the performance perceivable by a user of customer. [Note 1. QoS reflects the performance perceivable by a
a service, while GoS (grade of service) is used by a service user of a service, while CoS (class of service) is used by a
provider for internal design and operation of a network.] service provider for internal design and operation of a network.]
[Note 2. Mechanisms for monitoring service continuity may be
service-specific and are not discussed here.]
. Evaluating the effectiveness of traffic engineering policies, or . Evaluating the effectiveness of traffic engineering policies, or
triggering certain policy-based actions (such as alarm generation, triggering certain policy-based actions (such as alarm generation,
or path preemption) upon threshold crossing; this may be based on or path preemption) upon threshold crossing; this may be based on
the use of performance history data. the use of performance history data.
. Verifying peering agreements between service providers by . Verifying peering agreements between service providers by
monitoring/measuring the traffic flows over interconnecting links monitoring/measuring the traffic flows over interconnecting links
at border routers (note that peers are in general not willing to at border routers (note that peers are in general not willing to
divulge detailed traffic picture inside their autonomous systems); divulge detailed traffic picture inside their autonomous systems);
this includes the estimation of inter- and intra-network traffic, this includes the estimation of inter- and intra-domain traffic,
as well as originating, terminating, and transit traffic that are as well as originating, terminating, and transit traffic that are
being exchanged between peers. being exchanged between peers.
An example of using traffic measurements in this area might be An example of using traffic measurements in this area might be
monitoring packet loss rates at various points in a network to monitoring packet loss rates at various points in a network to
detect apparent link failure. Another example is monitoring the QoS detect apparent link failure. Another example is monitoring the QoS
delivered to external peers by an autonomous system to ensure that at peering points to ensure that peering agreements are met.
peering agreements are met.
5.3 Traffic control 5.3 Traffic control
. Adaptively optimizing network performance in response to network . Adaptively optimizing network performance in response to network
events, e.g., rerouting to work around congestion or failures. events, e.g., rerouting to work around congestion or failures.
. Providing a feedback mechanism in the reverse flow messaging of . Providing a feedback mechanism in the reverse flow messaging of
RSVP-TE or CR-LDP signaling in MPLS to report on actual topology RSVP-TE [16] or CR-LDP [17] signaling in MPLS to report on actual
state information such as link bandwidth availability. topology state information such as link bandwidth availability.
(An example of such a feedback mechanism is described in [18]. As
described therein, care should be exercised to ensure network
stability and consistency for any mechanism that makes direct
operational use of measurement (e.g., to use as feedback into path
computation). However, such issues will not be dealt with here as
this framework document is mainly concerned with the definitions
and principles of measurements, rather than their usage to
subsequently ensure other network features such as accurate
bandwidth allocation.)
. Support of measurement-based admission control, i.e., by . Support of measurement-based admission control, i.e., by
predicting the future demands of the aggregate of existing flows predicting the future demands of the aggregate of existing flows
so that admission decisions can be made on new flows. so that admission decisions can be made on new flows.
An example of traffic engineering measurements used to effect a An example of traffic engineering measurements used to effect a
traffic control mechanism is to configure policing mechanisms in traffic control mechanism is to configure policing mechanisms in
response to traffic load and performance measurements. A network response to traffic load and performance measurements. A network
operator could selectively throttle low-priority flows to improve operator could selectively throttle low-priority flows to improve
near-real-time performance of higher-priority flows, and maintain near-real-time performance of higher-priority flows, and maintain
tighter QoS envelopes. Another example would be to use measurement tighter QoS envelopes. Another example would be to use measurement
skipping to change at page 7, line 37 skipping to change at page 8, line 27
vendors, in alignment with financial forecasts and market vendors, in alignment with financial forecasts and market
assessments. assessments.
Capacity management Capacity management
Intermediate-scale (e.g., six months or less) capacity planning Intermediate-scale (e.g., six months or less) capacity planning
deals with detailed implementation of the build plan, short-lead- deals with detailed implementation of the build plan, short-lead-
time activities and out-of-plan events. It typically uses a time activities and out-of-plan events. It typically uses a
rolling-month forecast of traffic and demand. Information that rolling-month forecast of traffic and demand. Information that
changes on the order of days or hours is used to manage the deployed changes on the order of days or hours is used to manage the deployed
facilities, by taking appropriate maintenance or engineering actions facilities, by taking appropriate maintenance or engineering actions
to optimize utilization. For example, new MPLS tunnels may be set to optimize utilization. For example, new MPLS paths may be set up
up or existing tunnels modified while meeting service level or existing paths modified while meeting service level agreements.
agreements. Also, load balancing may be performed, or traffic may Also, load balancing may be performed, or traffic may be rerouted
be rerouted for re-optimization after a failure. for re-optimization after a failure.
Real-time network control Real-time network control
Information that changes on the order of minutes or less is used to Information that changes on the order of minutes or less is used to
adapt to the current network conditions in near real time. Thus, to adapt to the current network conditions in near real time. Thus, to
combat localized congestion, traffic management actions may perform combat localized congestion, traffic management actions may perform
temporary rerouting to redistribute the load. Upon detecting a temporary rerouting to redistribute the load. Upon detecting a
failure, traffic may be diverted to pre-established, secondary failure, traffic may be diverted to pre-established, secondary
routes until more optimized routes can be arranged. routes until more optimized routes can be arranged.
7. Read-Out Periods 7. Read-Out Periods
A measurement infrastructure must be able to scale with the size and A measurement infrastructure must be able to scale with the size and
the speed of a network as it evolves. Hence, it is important to the speed of a network as it evolves. Hence, it is important to
minimize the amount of data to be collected, and to condense the minimize the amount of data to be collected, and to condense the
collected data by periodic summarization. This is to prevent collected data by periodic summarization over read-out periods.
network performance from being adversely affected by the
7.1 Data Reduction
Techniques to manage large volumes of measured data are needed to
prevent network performance from being adversely affected by the
unnecessarily excessive loading of router control processors, router unnecessarily excessive loading of router control processors, router
memories, transmission facilities, and the administrative support memories, transmission facilities, and the administrative support
systems. For example, offline bulk file transfer may be used as a systems.
method to manage large volumes of measured traffic data. Bulk
transfer from routers to collection devices can help reduce the For example, offline bulk file transfer may be used as a method to
packet processing overhead experienced by using other management manage large volumes of measured traffic data. Bulk transfer from
interfaces. Also, data correlation or filtering rules may be set up routers to collection devices can help reduce the packet processing
to suppress redundant data, or to aggregate flows into suitable overhead experienced by using other management interfaces. Also,
classes with the corresponding aggregation of statistics. These data correlation or filtering rules may be set up to suppress
types of data reduction may be used as an appropriate or acceptable redundant data, or to aggregate flows into suitable classes with the
means for pruning down the overall volume of traffic data that a TE corresponding aggregation of statistics. These types of data
system may ultimately have to store, maintain, and process. reduction may be used as an appropriate or acceptable means for
pruning down the overall volume of traffic data that a TE system may
ultimately have to store, maintain, and process.
7.2 Measurement Interval
A measurement interval is the time interval over which measurements A measurement interval is the time interval over which measurements
are taken. Some traffic data must be collected continuously, while are taken. Some traffic data must be collected continuously, while
others by sampling, or on a scheduled basis. For example, peak others by sampling, or on a scheduled basis. For example, peak
loads and peak periods can be identified only by continuous loads and peak periods can be identified only by continuous
measurement as traffic typically fluctuates irregularly during the measurement as traffic typically fluctuates irregularly during the
whole day. If traffic variations are regular and predictable, it whole day. If traffic variations are regular and predictable, it
may be possible to measure the expected normal load on pre- may be possible to measure the expected normal load on pre-
determined portions of the day. Such duration of peak traffic is determined portions of the day. Such duration of peak traffic is
referred to as a busy period. Special studies on selected segments referred to as a busy period. Special studies on selected segments
of the network may be conducted on a scheduled basis. Occasionally of the network may be conducted on a scheduled basis. Occasionally
unexpected events or other decision support needs may arise that unexpected events or other decision support needs may arise that
require ad-hoc, unscheduled measurement, with the involvement of require ad-hoc, unscheduled measurement, with the involvement of the
network operator, and in such a case measurements may be activated network operator, and in such a case measurements may be activated
manually. For instance, active throughput measurement may be used manually. For instance, active throughput measurement may be used
to identify alternate routes for spreading traffic during periods of to identify alternate routes for spreading traffic to avoid future
network congestion. periods of network congestion, based on observations of current
local congestion events.
7.3 Summarization
A measurement interval consists of a sequence of consecutive read- A measurement interval consists of a sequence of consecutive read-
out periods. Summarization is usually done by integrating the raw out periods. Summarization is usually done by integrating the raw
data over a pre-specified read-out period. The granularity of this data over a pre-specified read-out period. The granularity of this
period must be suitably chosen. It should be short enough to period must be suitably chosen. It should be short enough to
capture, with acceptable accuracy, the bursty nature of the traffic, capture, with acceptable accuracy, the bursty nature of the traffic,
i.e., the traffic variations and peaks. Since measurements i.e., the traffic variations and peaks. Since measurements
represent a load for the router (if third-party measurement devices represent a load for the router (if third-party measurement devices
are not employed), the read-out period should not be so short that are not employed), the read-out period should not be so short that
router performance is degraded while a voluminous quantity of data router performance is degraded while a voluminous quantity of data
is produced. Also, read-out may be started when the measured data is produced. Also, read-out may be started when the measured data
exceeds a preset threshold, or when the space allocated for exceeds a preset threshold, or when the space allocated for
temporarily holding the data in a router is exhausted. temporarily holding the data in a router is exhausted.
For a multi-service IP-based network, each service typically has its For a multi-service IP network, each service typically has its own
own traffic characteristics and performance objectives. To ensure traffic characteristics and performance objectives. To ensure that
that service-specific features are reflected in the measurement CoS-specific features are reflected in the measurement process,
process, different read-out periods may be needed for different different read-out periods may be needed for different classes of
classes of service. service.
7.4 Sampling Issues
The concept of read-out periods applies to both active and passive The concept of read-out periods applies to both active and passive
measurements. This concept is consistent with the sampling issues measurements. This concept is consistent with the sampling issues
for a series of measurements as developed in [2], for example. See for a series of measurements as developed in [2], for example. See
sections 10 and 11 of that document for important distinctions sections 10 and 11 of that document for important distinctions
between "singletons, samples, and statistics." The procedure of between "singletons, samples, and statistics." The procedure of
Poisson sampling, for example, may be used within a read-out period Poisson sampling, for example, may be used within a read-out period
to select a subset of total packet events that are chosen as the to select a subset of total packet events that are chosen as the
sample. Then a statistic (e.g., mean or variance) can be computed sample. Then a statistic (e.g., mean or variance) can be computed
over that sample and associated with the read-out period. Although over that sample and associated with the read-out period. Although
skipping to change at page 9, line 22 skipping to change at page 10, line 24
8. Measurement Bases 8. Measurement Bases
Measurements can be classified on the basis of where, and at which Measurements can be classified on the basis of where, and at which
level of aggregation the traffic data are gathered. This is similar level of aggregation the traffic data are gathered. This is similar
to the concept of a *population of interest* as specified in ITU-T to the concept of a *population of interest* as specified in ITU-T
Recommendation I.380/Y.1540. As defined therein, this refers to a Recommendation I.380/Y.1540. As defined therein, this refers to a
set of packets, possibly relative to a particular pair of source and set of packets, possibly relative to a particular pair of source and
destination hosts, for the purposes of defining performance destination hosts, for the purposes of defining performance
parameters. However, measurement bases as used here may not have parameters. However, measurement bases as used here may not have
any association with a source-destination pair. any association with a source-destination pair. This is to be
described in more details below. Currently, the different
measurement bases to be defined below have not been explicitly
specified in the IPPM Framework [2].
In this document, the focus is on service providers as organizations In this document, the focus is on service providers as organizations
requiring traffic and performance measurements. (However, customer- requiring traffic and performance measurements. (However, customer-
based measurements of enterprise networks may have similar issues.) based measurements of enterprise networks may have similar issues.)
Service providers will make decisions on how to perform the Service providers will make decisions on how to perform the
measurements needed, and there are various tradeoffs involved. One measurements needed, and there are various tradeoffs involved. One
option is to obtain the measurements directly from the network option is to obtain the measurements directly from the network
elements themselves, e.g., via SNMP (Simple Network Management elements themselves, e.g., via SNMP (Simple Network Management
Protocol). Collecting the measurements on the operational network Protocol). Collecting the measurements on the operational network
elements such as routers is sometimes a performance concern. elements such as routers is sometimes a performance concern.
Currently, there are a number of third-party measurement/monitoring Currently, there is a number of third-party measurement/monitoring
products available. Hence, another option is to deploy such products available. Hence, another option is to deploy such
equipment, which might have performance advantages but also equipment, which might have performance advantages but also
introduces additional cost. introduces additional cost.
Regardless of the type of measurement source, either a network Regardless of the type of measurement source, either a network
element or a third-party product, measurements should be collected, element or a third-party product, measurements should be collected,
as far as possible, by a measurement source without requiring as far as possible, by a measurement source without requiring
coordination with other measurement sources. Thus, it is desirable coordination with other measurement sources. Thus, it is desirable
to perform those measurements that do not require the use of to perform those measurements that do not require the use of
specialized monitoring equipment connected to the network at specialized monitoring equipment connected to the network at
multiple locations. While each measurement source may act multiple locations. While each measurement source may act
autonomously with regard to taking measurements, a network operator autonomously with regard to taking measurements, a network operator
may specify some network-wide policy regarding measurement may specify some network-wide policy regarding measurement
scheduling. Such policy may be, say, the use of the same time of scheduling. Such policy may be, say, the use of the same time of
day, the same measurement interval, or measurement intervals that day, the same measurement interval, or measurement intervals that
are multiples of each other (e.g., nested intervals with are multiples of each other (e.g., nested intervals with
synchronized boundaries). A schedule therefore should include such synchronized boundaries). A schedule therefore should include such
time information as the start, the duration, and periodicity of a time information as the start, the duration, and periodicity of a
certain measurement. certain measurement. Also note that the accuracy of traffic
measurement is highly dependent on the synchronization capabilities
of the measurement devices that will be involved in the measurement
procedures. While synchronization issues are out of the scope of
this document, they should be explicitly addressed whenever a
measurement campaign is to be launched, whatever its scope and its
frequency.
The following measurement bases are considered in this document: The following measurement bases are considered in this document:
. Flow-based . Flow-based
. Interface-based, link-based, node-based . Interface-based, link-based, node-based
. Node-pair-based . Node-pair-based
. Path-based . Path-based
Generally speaking, for traffic engineering purposes, passive Generally speaking, for traffic engineering purposes, passive
measurements are mostly used. However, as to be described later in measurements are mostly used. However, as to be described later in
the "Measurement Types" section, the above measurement bases may the "Measurement Types" section, the above measurement bases may
result in active or passive measurements. For example, an active result in active or passive measurements. For example, an active
measurement may be a two-point delay metric such as type-P-one-way- measurement may be a two-point delay metric such as type-P-one-way-
delay defined in [4], and obtained by time-stamping probe packets at delay defined in [4], and obtained by time-stamping probe packets at
selected ingress and egress points; a passive measurement may be to selected ingress and egress points; a passive measurement may be to
obtain packet inter-arrival times by time-stamping at a selected obtain packet inter-arrival times by time-stamping successive
point in the network successive packets of the offered traffic. packets of the traffic at a selected point in the network. Note
Note that both active and passive measurements are subject to the that both active and passive measurements are subject to the same
same sampling and time-source accuracy concerns. sampling and time-source accuracy concerns.
MPLS has certain advantages when compared with conventional IP MPLS has certain advantages when compared with conventional IP
networks, from the perspective of the difficulty involved in networks, from the perspective of the difficulty involved in
obtaining unambiguous measurements. As different service providers obtaining unambiguous measurements. As different service providers
will adopt different technologies, technology-neutral solutions to will adopt different technologies, technology-neutral solutions to
the problem of obtaining measurements are presented as far as the problem of obtaining measurements are presented as far as
possible. possible.
Applicability of traffic measurements to the derivation of traffic Applicability of traffic measurements to the derivation of traffic
matrix statistics and performance monitoring are to be described in matrix statistics and performance monitoring are to be described in
later sections. later sections.
8.1 Flow-based 8.1 Flow-based
This is conceptually similar to the call detail record (CDR) in This is conceptually similar to the call detail record (CDR) in
circuit-switched telecommunications networks. It is primarily used circuit-switched telecommunications networks. It is primarily used
on interfaces at access routers, edge routers, or aggregation on interfaces at access routers, edge routers, or aggregation
routers where traffic originates or terminates, rather than on routers, rather than on backbone routers in the core network. Like
backbone routers in the core network. Like CDR measurements, flow- CDR measurements, flow-based records are used to collect detailed
based records are used to collect detailed information about a flow. information about a flow. This includes such information as source
This includes such information as source and destination IP and destination IP addresses/port numbers, protocol, type of
addresses/port numbers, protocol, type of service, timestamps for service, timestamps for the start and end of a flow, packet count,
the start and end of a flow, packet count, octet count, etc. octet count, etc.
As flow is a fine-grained object, measuring every flow that passes As flow is a fine-grained object, measuring every flow that passes
through all the edge devices may not be scalable or feasible. through all the edge devices may not be scalable or feasible.
Hence, per-flow data are usually used in a special study conducted Hence, per-flow data are usually used in a special study conducted
on a non-continuous schedule and on selected routers only. Sampling on a non-continuous schedule and on selected routers only. Sampling
of flow-based measurements may also be needed to reduce both the of flow-based measurements may also be needed to reduce both the
amount of data collected and the associated overhead. amount of data collected and the associated overhead.
8.2 Interface-based, link-based, node-based 8.2 Interface-based, link-based, node-based
skipping to change at page 11, line 9 skipping to change at page 12, line 20
passive measurements can be taken at each network element. For passive measurements can be taken at each network element. For
example, SNMP uses passive monitoring to collect raw data on an example, SNMP uses passive monitoring to collect raw data on an
interface at an edge or backbone router. These data are stored in interface at an edge or backbone router. These data are stored in
MIBs (Management Information Bases) and include counts on packets MIBs (Management Information Bases) and include counts on packets
and octets sent/received, packet discards, errored packets. Such and octets sent/received, packet discards, errored packets. Such
measurements may have the disadvantage that the identity of each measurements may have the disadvantage that the identity of each
flow is lost. flow is lost.
To reduce the overhead in managing multiple links between the same To reduce the overhead in managing multiple links between the same
ingress and egress points, there is proposal to aggregate links for ingress and egress points, there is proposal to aggregate links for
network optimization [16]. Component links in such a *bundled link* network optimization [19]. Component links in such a *bundled link*
will have the same routing constraints, resource classes, and will have the same routing constraints, resource classes, and
attributes. Multiple links are treated as a single IP link. attributes. Multiple links are treated as a single IP link.
Traffic measurements, such as bandwidth availability, throughput, Traffic measurements, such as bandwidth availability, throughput,
etc., should consider the measurement implications for bundled etc., should consider the measurement implications for bundled
links, and should not inhibit link bundling. Also, such links, and should not inhibit link bundling. (For example, a single
measurements should be protocol independent and media independent to IP link may presumably be referenced as a pair of IP addresses that
ensure portability and commonality in the measurements. are assigned to both extremities of the link. An implicit issue
that may need to be resolved relates to the exact characterization
of the traffic that will be conveyed in each component link, since a
couple of IP addresses may not be sufficient for such link-based
measurement.) Also, such measurements should be protocol
independent and media independent to ensure portability and
commonality in the measurements.
8.3 Node-pair-based 8.3 Node-pair-based
Active measurements by probing, as specified in the IPPM framework Active measurements by probing, as specified in the IPPM framework
for example, can be conducted between each pair of major routing for example, can be conducted between each pair of (major) routing
hubs for determining edge-to-edge performance of a core network. hubs for determining edge-to-edge performance of a core network.
This complements the passive measurements of the previous sub- This complements the passive measurements of the previous sub-
section, which provide local views of the performance of individual section, which provide local views of the performance of individual
network elements. network elements.
In contrast to performance statistics, traffic loading statistics In contrast to performance statistics, traffic loading statistics
require passive measurements of the actual traffic. In circuit- require passive measurements of the actual traffic. In circuit-
switched telecommunications networks, each established call has an switched telecommunications networks, each established call has an
associated source/destination node-pair. By maintaining a set of associated source/destination node-pair. By maintaining a set of
node-pair data registers (usage, peg count, overflow, etc) in each node-pair data registers [usage, call attempts (so-called "peg
switch, node-pair-based measurements for traffic statistics such as count" in telephony operation and management), overflow, etc.] in
the load between a given node pair are taken directly. In IP-based each switch, node-pair-based measurements for traffic statistics
networks, currently such node-pair-based measurements are difficult such as the load between a given node pair are taken directly. In
to establish due to the dynamic and asymmetric properties of IP IP networks, currently such node-pair-based measurements are
routing. However, it is possible to infer them from flow-based difficult to establish due to the dynamic and asymmetric properties
passive measurements and other network information, such as routing of IP routing. However, it is possible to infer them from flow-
table snapshots. A problem with this approach is that flow-based based passive measurements and other network information, such as
measurement data are voluminous. Also, another problem that must be routing table snapshots. A problem with this approach is that flow-
accounted for is the routing changes among the multiple routes due based measurement data are voluminous. Also, another problem that
to, e.g., a change in the configuration of intradomain routing, or a must be accounted for is the routing changes among the multiple
change in interdomain policies made by another autonomous system. routes due to, e.g., a change in the configuration of intra-domain
This is further discussed in the Section on Traffic Matrix routing, or a change in inter-domain policies made by another
Statistics. autonomous system. This is further discussed in the Section on
Traffic Matrix Statistics.
8.4 Path-based 8.4 Path-based
The ability of MPLS to use fixed preferred paths for routing The ability of MPLS to use fixed preferred paths for routing
traffic, so-called route pinning, gives the means to develop path- traffic, so-called "route pinning" (or "path pinning", using the
based measurements. This may enable the development of definitions of Section 4.2), gives the means to develop path-based
methodologies for such functions as admission control and measurements. This may enable the development of methodologies for
performance verification of delivered service. such functions as admission control and performance verification of
delivered service.
Like a flow, a path is associated with a pair of nodes. However, Like a flow, a path is associated with a pair of nodes. However,
path is a more coarse-grained object than flow, as paths are usually path is a more coarse-grained object than flow, as paths are usually
used to carry aggregated traffic. In addition, when routing changes used to carry aggregated traffic (from different flows). In
occur, the amount of traffic to be carried by a path will either not addition, when routing changes occur, the amount of traffic to be
be affected or be merged with that of another path. Because of carried by a path will either not be affected or be merged with that
these properties, path-based measurements are more scalable and may of another path. Because of these properties, path-based
be used to provide more readily an accurate, network-wide, view of measurements are more scalable and may be used to provide more
the traffic demands. For example, the traffic between a given pair readily an accurate, network-wide, view of the traffic demands. For
of nodes may be inferred from the aggregate of the traffic carried example, the traffic between a given pair of nodes may be inferred
by all paths either terminated by or passed through the same node- from the aggregate of the traffic carried by all paths either
pair. terminated by or passed through the same node-pair.
9. Measurement Entities 9. Measurement Entities
A measurement entity defines what is measured: it is a quantity for A measurement entity defines what is measured: it is a quantity for
which data collection must be performed with a certain measurement. which data collection must be performed with a certain measurement.
A measurement type can be specified by a (meaningful) combination of A measurement type can be specified by a (meaningful) combination of
a measurement entity with the measurement basis described in the a measurement entity with the measurement basis described in the
previous section. previous section.
An important issue with any measurement is measurement precision
and/or accuracy. However, this issue is not dealt with here since
each measurement type will potentially have its own unique
requirements. For example, see [4], Section 3.7, for a discussion
on error issues for one-way delay.
9.1 Entities related to traffic and performance 9.1 Entities related to traffic and performance
Some of the measurement entities listed below, such as throughput, Some of the measurement entities listed below, such as throughput,
delay, delay variation, and packet loss, are related to the delay, delay variation, and packet loss, are related to the
respective IPPM performance metrics or the I.380/Y.1540 performance respective IPPM performance metrics or the I.380/Y.1540 performance
parameters. parameters.
. Traffic volume (mean and variance, in number of bits, bytes, or . Traffic volume (mean and variance, in number of bits, bytes, or
packets transferred, as counted over a given time interval), on a packets transferred, as counted over a given time interval), on a
per service class basis, at various aggregation levels (IP address per service class basis, at various aggregation levels (IP address
prefix, interface, link, node, node-pair, path, network edge, prefix, interface, link, node, node-pair, path, network edge,
customer, or autonomous system) customer, or autonomous system)
Note: (1) This is a measurement for the traffic carried by a Note: (1) This is a measurement for the traffic carried by a
network, a network segment, or an individual network element; it network, a network segment, or an individual network element; it
is used to derive the carried load or carried traffic intensity is used to derive the carried load or carried traffic intensity
[17]. When measured during the busy period, this entity is [20]. When measured during the busy period, this entity is
normally used to estimate the traffic offered. However, the normally used to estimate the traffic offered. However, the
estimation procedure should take into account such factors as estimation procedure should take into account such factors as
congestion, which may result in decreased carried traffic. In congestion, which may result in a decreased volume of carried
addition, congestion may lead to user behavior such as reattempt traffic. In addition, congestion may lead to user behavior such
or abandonment, which may affect the actual traffic offered. (2) as reattempt or abandonment, which may affect the actual traffic
To reduce uncertainty in traffic estimation, second-order measures offered. (2) To reduce uncertainty in traffic estimation, second-
may need to be developed. Beyond the use of variance as in order measures may need to be developed. Beyond the use of
current practice, further study is needed for the feasibility of variance as in current practice, further study is needed for the
other second-order techniques. (3) Measurement of traffic volumes feasibility of other second-order techniques. (3) Measurement of
over interconnecting links at border routers can be used to traffic volumes over interconnecting links at border routers can
estimate the traffic exchange between peers for contract be used to estimate the traffic exchange between peers for
verification. contract verification.
. Average holding time (e.g., flow duration or lifetime, duration of . Average holding time (e.g., flow duration or lifetime, duration of
an MPLS path), on a per service class basis an MPLS path), on a per service class basis
Note: (1) When MPLS traffic engineering is used, this is similar Note: (1) When MPLS traffic engineering is used, this is similar
to call holding time in telecommunications networks. Peg count, to call holding time in telecommunications networks. Call
usage, and call holding time are three busy-hour entities that attempts, usage, and call holding time are three busy-hour
should be independently measured for both call-dependent and load- entities that should be independently measured for both call-
dependent engineering. This is important especially when the call dependent and load-dependent engineering. This is important
busy hour and the load busy hour during a day are non-coincident, especially when the call busy hour and the load busy hour during a
due to the hour-to-hour variation of call holding times. (2) The day are non-coincident, due to the hour-to-hour variation of call
holding time statistics of long-living static paths reflect the holding times. (2) The holding time statistics of long-living
effect of network equipment failures, link outages, or scheduled static paths reflect the effect of network equipment failures,
maintenance, and hence may to used to derive information about up- link outages, or scheduled maintenance, and hence may be used to
time or service availability. derive information about up-time or service availability. (3) It
is desirable to be able to gather, by passive means, the up-time
durations for each pair of label bindings in the label-forwarding
information base for labels distributed by different protocols
(such as LDP, RSVP-TE, MP-BGP, or BGP). Then, the derivation of
LSP average holding time does not need to be finely correlated
with network events such as link/node failures.
. Available bandwidth of a link or path - useful for load balancing, . Available bandwidth of a link or path - useful for load balancing,
measurement-based admission control to determine the feasibility measurement-based admission control to determine the feasibility
of creating a new MPLS tunnel (real-time information can be used of creating a new MPLS tunnel (real-time information can be used
for dynamic establishment) for dynamic establishment)
. Throughput (in bits per second, bytes per second, or packets per . Throughput (in bits per second, bytes per second, or packets per
second) second)
Note: (1) This is a measure of the "goodput." That is, the rate Note: (1) This is the rate at which a given amount of traffic
at which a given amount of traffic excluding lost, misdelivered, excluding lost, misdelivered, or errored packets, that passes
or errored packets, that passes between a set of end points, where between a set of end points, where end points can be logically or
end points can be logically or physically defined. The condition physically defined. The condition of the network, e.g., normal or
of the network, e.g., normal or high load, under which the high load, under which the measurement is taken should be noted.
measurement is taken should be noted. (2) The protocol level at (2) The protocol level at which a throughput measurement is taken
which a throughput measurement is taken must be specified, as the must be specified, as the packet payload and packet overheads are
packet payload and packet overheads are protocol dependent. (3) protocol dependent. (3) The average packet size may be inferred
The average packet size may be inferred from the bit rate and from the bit rate and packet rate measurements, when performed on
packet rate measurements, when performed on the basis of an the basis of an individual router. This quantity is useful to
individual router. This quantity is useful to gauge router gauge router performance, since router operations are typically
performance, since router operations are typically packet-oriented packet-oriented and small packets are more processing-intensive.
and small packets are more processing-intensive.
. Delay (e.g., cross-router delay from node-based measurement may be . Delay (e.g., cross-router delay from node-based measurement may be
used to measure queueing delay within a router; end-to-end one-way used to measure queueing delay within a router; end-to-end one-way
or round-trip packet delay can be obtained by node-pair-based or round-trip packet delay can be obtained by node-pair-based
measurement) measurement)
Note: The condition of the network, e.g., normal or high load, Note: The condition of the network, e.g., normal or high load,
under which the measurement is taken should be noted. This is under which the measurement is taken should be noted. This is
useful to determine if delay objectives are met. useful to determine if delay objectives are met.
. Delay variation . Delay variation
Note: There are several methods to measure this quantity as Note: There are several methods to measure this quantity as
specified in ITU-T and IPPM. (1) In Appendix II of I.380/Y.1540, specified in ITU-T and IPPM. (1) In Y.1540, measurements are
IP packet delay variation is defined via four alternative methods. defined for both 2-point and 1-point IP packet delay variation.
The first two methods define an end-to-end two-point delay However, 2-points methods are being specified as normative. (2)
variation of a given packet, measured between two measurement In IPPM [9], the concept of a selection function is introduced
points (such as ingress and egress), as the difference between the that allows for the explicit designation of selected packets whose
one-way delay of the given packet and some nominal delay. This one-way delay values are compared to compute one-way delay
nominal delay is chosen to be the first packet delay in the first variation. For example, to define a method of measurement, a
method and the average delay of the population of packets in the selection function can be specified to select the consecutive
second method. The third alternative, interval-based method,
measures the percentage of packets with delay variations that fall
outside some pre-specified delay variation interval. Finally, the
quantile-based method measures the distance (in time units)
between pre-selected quantiles, e.g., 99.5 percentile and 0.5
percentile, of the delay variation distribution. This method is
tighter than the interval-based method since it bounds the tail of
the delay variation distribution. In Y.1541, additional
considerations and more alternatives of delay variations are
described. (2) In IPPM [9], the concept of a selection function
is introduced that allows for the explicit designation of selected
packets whose one-way delay values are compared to compute one-way
delay variation. For example, to define a method of measurement,
a selection function can be specified to select the consecutive
packets within a specified interval, or to select the maximum and packets within a specified interval, or to select the maximum and
minimum one-way delays within a specified interval. minimum one-way delays within a specified interval.
. Packet loss . Packet loss
Note: (1) While packet losses due to transmission and/or protocol Note: (1) While packet losses due to transmission and/or protocol
errors may not be traffic related, unexpected excessive loss may errors may not be traffic related, unexpected excessive loss may
be used as a means of fault detection. (2) Packet losses due to be used as a means of fault detection. (2) In most active
policing or network congestion should be distinguished. The measurements, the cause of packet loss is not distinguished.
However, it may be desirable to distinguish (e.g., by passive
means) packet losses due to policing or network congestion. The
former is a result of user violation of service contract and the former is a result of user violation of service contract and the
network operator should not be penalized for it. The latter, network operator should not be penalized for it. The latter,
whether intentional or unintentional, is caused by network whether intentional or unintentional, is caused by network
conditions such as buffer overflow, router forwarding process conditions such as buffer overflow, router forwarding process
busy, and may not be the user's fault. When policing is done by a busy, and may not be the user's fault. When policing is done by a
network, measurement of non-conforming packets at the edge network, measurement of non-conforming packets at the edge
provides an indication on the extent to which the network is provides an indication on the extent to which the network is
carrying this type of packets (which can potentially be dropped if carrying this type of packets (which can potentially be dropped if
network gets congested). Loss due to congestion of any packets, network gets congested). Loss due to congestion of any packets,
including loss of non-conforming packets, is a useful measure in including loss of non-conforming packets, is a useful measure in
skipping to change at page 15, line 6 skipping to change at page 16, line 26
Where connection admission control is used, a measurement entity for Where connection admission control is used, a measurement entity for
monitoring network performance may be the proportion of connections monitoring network performance may be the proportion of connections
denied admission. Also, it may be useful to score the requested denied admission. Also, it may be useful to score the requested
bandwidth within the traffic parameters for the setup request. bandwidth within the traffic parameters for the setup request.
Corresponding to the number of call attempts (i.e., peg count) in Corresponding to the number of call attempts (i.e., peg count) in
telecommunications networks, the number of connection requests, the telecommunications networks, the number of connection requests, the
number of flows, etc., may be measured in given read-out periods to number of flows, etc., may be measured in given read-out periods to
characterize the traffic. characterize the traffic.
To characterize paths for MPLS traffic engineering, the following To characterize paths for MPLS traffic engineering, the following
measurement entities may be defined: path setup delay, path setup measurement entities may possibly be defined: path setup delay, path
error probability, path setup denial (blocking) probability, path setup error probability, path setup denial (blocking) probability,
release delay, path disconnect probability, path restoration time. path release delay, path disconnect probability, path restoration
time. However, note that path establishment may be dependent on
routing and signaling protocols, in particular, whether preemption
or fast-reroute restoration capability is used or not. Hence,
further investigation is needed to determine if and how these
measurement entities are to be defined.
10. Measurement Types 10. Measurement Types
A measurement matrix can be defined wherein each column represents a A measurement matrix can be defined wherein each column represents a
measurement basis and each row represents a measurement entity. An measurement basis and each row represents a measurement entity. An
entry in this measurement matrix, corresponding to a meaningful and entry in this measurement matrix, corresponding to a meaningful and
measurable combination of an entity and a basis, defines a measurable combination of an entity and a basis, defines a
particular measurement type. For each measurement type, there particular measurement type. For each measurement type, there
should be a set of measurement points specified to bound the network should be a set of measurement points specified to bound the network
segment for the purposes of taking measurement. A measurement point segment for the purposes of taking measurement. A measurement point
skipping to change at page 15, line 45 skipping to change at page 17, line 17
Avg. Hold. Time x x(3) Avg. Hold. Time x x(3)
Avail. Bandwidth x x(3) Avail. Bandwidth x x(3)
Throughput x(4) x(4) Throughput x(4) x(4)
Delay x(2) x(4) x(4) Delay x(2) x(4) x(4)
Delay Variation x(2) x(4) x(4) Delay Variation x(2) x(4) x(4)
Packet Loss x x(5) x(5) Packet Loss x x(5) x(5)
Notes: Notes:
(1) This measurement type can be used to derive flow size (1) This measurement type can be used to derive flow size
statistics. statistics.
(2) These are 1-point measurements. (2) These are 1-point measurements. For a discussion on 1-point
packet delay variation, see [11], Appendix II.
(3) As a starting point, statistics collected by passive measurement (3) As a starting point, statistics collected by passive measurement
through the MIBs useful for traffic engineering [18, 19, 20] may be through the MIBs useful for traffic engineering [21, 22, 23] may be
used. used.
(4) Active measurements based on IPPM metrics are currently in use (4) Active measurements based on IPPM metrics are currently in use
for node-pairs; they may be developed for paths. for node-pairs; they may be developed for paths.
(5) Besides active measurements based on IPPM, path loss may (5) Besides active measurements based on IPPM, path loss may
possibly be inferred from the difference between ingress and egress possibly be inferred from the difference between ingress and egress
traffic statistics at the two endpoints of a path. However, such traffic statistics at the two endpoints of a path. However, such
inference for the cumulative losses between a given node pair over inference for the cumulative losses between a given node pair over
multiple routes may be less useful, since different routes may have multiple routes may be less useful, since different routes may have
different loss characteristics. different loss characteristics.
skipping to change at page 16, line 28 skipping to change at page 17, line 54
Entities: Entities:
Control Util. x x x Control Util. x x x
Signaling Util. x x x Signaling Util. x x x
Service Class Util. x x x Service Class Util. x x x
The amount of control and signaling traffic carried by a network is The amount of control and signaling traffic carried by a network is
a function of many factors. To name a few, they include the size a function of many factors. To name a few, they include the size
and topology of the network, the control and signaling protocols and topology of the network, the control and signaling protocols
used, the amount of user traffic carried, the number of failure used, the amount of user traffic carried, the number of failure
events, etc. Also, flooding of link-state advertisement (LSA) events, etc. Also, flooding of link-state advertisement (LSA)
messages in Interior Gateway Protocol (IGP, such as OSPF or IS-IS) messages in Interior Gateway Protocols (IGP, such as OSPF or IS-IS)
may cause significant routing control traffic during events such as may cause significant routing control traffic during events such as
an LSA storm as a result of failures due to fiber cuts or failed an LSA storm as a result of failures due to fiber cuts or failed
power supply. The above utilization measurements for control and power supply. The above utilization measurements for control and
signaling traffic are intended to help develop guidelines for the signaling traffic are intended to help develop guidelines for the
proper dimensioning and apportionment of network resources so that a proper dimensioning and apportionment of network resources so that a
given level of user traffic can be adequately supported. As the given level of user traffic can be adequately supported. As the
primary focus here is on user traffic measurements, the additional primary focus here is on user traffic measurements, the additional
needs and properties of control and signaling traffic measurements needs and properties of control and signaling traffic measurements
are beyond the scope of this document. are beyond the scope of this document.
11. Traffic Matrix Statistics 11. Traffic Matrix Statistics
An important set of data for traffic engineering is point-to-point An important set of data for traffic engineering is point-to-point
or point-to-multipoint demands. This data is needed in the or point-to-multipoint demands. This data may be of significant use
provisioning of intradomain routes and external peering in the in the provisioning of traffic-engineered intra-domain paths and
existing network, as well as planning for the placement and sizing external peering in the existing network, as well as planning for
of new links, routers, or peers. the placement and sizing of new links, routers, or peers.
In current practice, estimates for traffic demands are usually In current practice, estimates for traffic demands are usually
determined from a combination of traffic projections, customer determined from a combination of traffic projections, customer
prescriptions, and service level agreements. Under existing mode of prescriptions, and service level agreements. Under existing mode of
operation, it is not easy to obtain network-wide traffic demands operation, it is not easy to obtain network-wide traffic demands
from the local interface measurements taken by different IP routers. from the local interface measurements taken by different IP routers.
As explained in [21, 22], information from diverse network As explained in [24, 25], information from diverse network
measurements and various configuration files are needed to infer the measurements and various configuration files are needed to infer the
traffic volume. Besides raw measurement data, additional traffic volume. Besides raw measurement data, additional
information such as topological data and router configuration data information such as topological data and router configuration data
are required to obtain a network view. Furthermore, destination- are required to obtain a network view. Furthermore, destination-
based routing/forwarding in IGP provides a network operator with based IP routing and forwarding provides a network operator with
primitive and limited control over the routing of traffic flows. primitive and limited control over the routing of traffic flows.
This necessitates the association of a time sequence of forwarding This necessitates the association of a time sequence of forwarding
tables from different routers to reconstruct the different routes tables from different routers to reconstruct the different routes
used by the network over time. By using this auxiliary information, used by the network over time. By using this auxiliary information,
together with flow-based measurements, the above-cited references together with flow-based measurements, the above-cited references
describe how to determine the traffic volume from an ingress link to describe how to determine the traffic volume from an ingress link to
a set of egress links by validating and joining various data sets a set of egress links by validating and joining various data sets
together. together.
As described in Section 8.3, some shortcomings in today's method to As described in Section 8.3, some shortcomings in today's method to
derive traffic matrix statistics as above include the volume of data derive traffic matrix statistics as above include the volume of data
from flow-based measurement, the lack of sufficient routing control from flow-based measurement, the lack of sufficient routing control
information, and the need to correlate data from a variety of information, and the need to correlate data from a variety of
sources. The routing control offered by MPLS can be used to avoid sources. The routing control offered by MPLS can be used to avoid
some of these deficiencies. To take advantage of this capability, some of these deficiencies. To take advantage of this capability,
path-based passive measurement should be developed. Furthermore, as path-based passive measurement should be developed. Furthermore, as
explained in the Section on Path-based Measurement Bases, by explained in Section 8.4 (Path-based Measurement Bases), by
aggregating the appropriate set of path-based traffic data, the aggregating the appropriate set of path-based traffic data, the
corresponding node-pair-based traffic data can be obtained. This corresponding node-pair-based traffic data can be obtained. This
will facilitate the derivation of traffic matrix statistics, will facilitate the derivation of traffic matrix statistics,
possibly on a per service class basis. Note that in the case of possibly on a per service class basis. Note that in the case of
hop-by-hop routed label-switched paths that are established by Label hop-by-hop routed label-switched paths that are established by Label
Distribution Protocol (LDP) signaling, there is no explicit binding Distribution Protocol (LDP) signaling, there is no explicit binding
between path end points. This will result in the use of different between path end points. This will result in the use of different
label bindings at both the ingress and egress nodes over time as label bindings at both the ingress and egress nodes over time as
network topology changes. Although the forwarding equivalence class network topology changes. Although the forwarding equivalence class
(FEC) to label binding information already exists in the MPLS FTN (FEC) to label binding information already exists in the MPLS FTN
and LSR MIBs [23, 18], a mechanism is needed to keep track of and LSR MIBs [26, 21], a mechanism is needed to keep track of
binding changes. An example of such a mechanism may be the periodic binding changes. An example of such a mechanism may be the periodic
exchange of FEC to label binding information for each ingress-egress exchange of FEC to label binding information for each ingress-egress
pair. pair.
Besides traffic engineering, a major application of MPLS is the Besides traffic engineering, a major application of MPLS is the
support of network-based virtual private networks (VPNs). A VPN can support of network-based virtual private networks (VPNs). A VPN can
be an enterprise network or a carrier's carrier network. It is not be an enterprise network or a carrier's carrier network. It is not
the purpose of this document to discuss VPNs. However, it is the purpose of this document to discuss VPNs. However, it is
relevant to highlight the use of traffic measurements to maintain relevant to highlight the use of traffic measurements to maintain
proper engineering and performance of MPLS tunnels in the support of proper engineering and performance of MPLS tunnels in the support of
VPNs between VPN sites. This would include also the support for VPNs between VPN sites. This would include also the support for
MPLS-based pseudo-wire connections as developed by the PWE3 Working MPLS-based pseudo-wire connections as developed by the PWE3 Working
Group [24]. For example, path-based measurement by a network Group [27]. For example, path-based measurement by a network
operator on behalf of the VPN customers facilitates the estimation operator on behalf of the VPN customers facilitates the estimation
of the traffic offered by these VPNs. of the traffic offered by these VPNs.
12. Performance Monitoring 12. Performance Monitoring
General aspects of measurements required to support the operation, General aspects of measurements required to support the operation,
administration, and maintenance of a network are outside the scope administration, and maintenance of a network are outside the scope
of this document (see [25, 26, 27] for a discussion of MPLS OAM). of this document (see [28, 29, 30] for a discussion of MPLS OAM).
The focus of the measurements here is only on operations related to The focus of the measurements here is only on operations related to
traffic engineering and network performance management. traffic engineering and network performance management.
A major component of performance management is performance A major component of performance management is performance
monitoring, i.e., continuous real-time monitoring of the quality or monitoring, i.e., continuous real-time monitoring of the quality or
health of the network and its various elements to ensure a health of the network and its various elements to ensure a
sustained, uninterrupted delivery of quality service. This requires sustained, uninterrupted delivery of quality service. This requires
the use of measurement, either passively or actively, to collect the use of measurement, either passively or actively, to collect
information about the operational state of the network and to track information about the operational state of the network and to track
its performance. For a discussion of passive monitoring and the use its performance. For a discussion of passive monitoring and the use
of synthetic traffic sources in active probing, see [28]. Alarms of synthetic traffic sources in active probing, see [31]. Alarms
may be generated when the state of a network element exceeds may be generated when the state of a network element exceeds
prescribed thresholds. prescribed thresholds.
Performance degradation can occur as a result of routing Performance degradation can occur as a result of routing
instability, congestion, or failure of network components. Periods instability, congestion, or failure of network components. Periods
of congestion may be detected when the resource usage of a network of congestion may be detected when the resource usage of a network
segment consistently exceeds a certain threshold, or when the cross- segment consistently exceeds a certain threshold, or when the cross-
router delay is unexpectedly high. Unexpected excessive loss of router delay is unexpectedly high. Unexpected excessive loss of
packets or throughput drops may be used as a means of fault packets or throughput drops may be used as a means of fault
detection, and may result in restoration activities. detection, and may result in restoration activities.
Internet utilities such as ping and traceroute have been useful to Internet utilities such as ping and traceroute have been useful to
help diagnose network problems and performance debugging. Utilities help diagnose network problems and performance debugging. Utilities
with similar functions would be essential for path-oriented with similar functions would be essential for path-oriented
operations like in MPLS. This would include the capability to list, operations like in MPLS. This would include the capability to list,
at any time, (1) for a given path, all the nodes traversed by it, at any time, (1) for a given path, all the nodes traversed by it,
and (2) for a given node, all the paths originating from it, and (2) for a given node, all the paths originating from it,
transiting through it, and/or terminating on it. A proposal for transiting through it, and/or terminating on it. A proposal for
route tracing is described in [29]. path tracing is described in [32]. A proposal to establish basic
MPLS data plane liveness is described in [33].
13. Packet Sampling 13. Packet Sampling
A wide spectrum of operational applications can be built on traffic A wide spectrum of operational applications can be built on traffic
measurement. However, different applications usually require measurement. However, different applications usually require
traffic measurements at different levels of temporal and spatial traffic measurements at different levels of temporal and spatial
granularity. To achieve an effective tradeoff between granularity. To achieve an effective tradeoff between
implementation complexity and the range of operational tasks to be implementation complexity and the range of operational tasks to be
enabled, a passive measurement framework based on packet sampling is enabled, a passive measurement framework based on packet sampling is
proposed in [30]. proposed in [34].
The use of packet sampling has two motivations. First, the enormous The use of packet sampling has two motivations. First, the enormous
volumes of traffic require that some form of data reduction to be volumes of traffic require that some form of data reduction to be
used. Second, simple data reduction by aggregation at the used. Second, simple data reduction by aggregation at the
measurement point will not provide sufficiently detailed views for measurement point will not provide sufficiently detailed views for
all network management applications or exploratory studies. For all network management applications or exploratory studies. For
this reason, packet sampling is proposed as a means to reduce data this reason, packet sampling is proposed as a means to reduce data
volume while still retaining representative detail. volume while still retaining representative detail.
The primary aim of the proposal [30] is to define a minimal set of The primary aim of the proposal [34] is to define a minimal set of
primitive packet selection operations out of which all sampling primitive packet selection operations out of which all sampling
operations that are necessary to support measurement-based operations that are necessary to support measurement-based
applications can be composed. Operations currently under applications can be composed. Operations currently under
consideration include filtering and statistical sampling, and also consideration include filtering and statistical sampling, and also
hash-based packet selection, a method that can be used to support hash-based packet selection, a method that can be used to support
the determination of spatial traffic flows across a domain [31]. the determination of spatial traffic flows across a domain [35].
Whichever method is used, the interpretation of the stream of Whichever method is used, the interpretation of the stream of
measurements arising from sampled packets must be both transparent measurements arising from sampled packets must be both transparent
and standard. Other goals are to specify a means to format and and standard. Other goals are to specify a means to format and
export measurements, and a means to manage the configuration of the export measurements, and a means to manage the configuration of the
sampling and export operations. sampling and export operations.
The proposal positions these function to provide a basic packet The proposal positions these functions to provide a basic packet
sampled measurement service to higher level "consumers." A typical sampled measurement service to higher level "consumers." A typical
consumer is a network management application that sits behind a consumer is a network management application that sits behind a
remote measurement collector. Such measurements can support remote measurement collector. Such measurements can support
applications for a number of tasks: troubleshooting, demand applications for a number of tasks: troubleshooting, demand
characterization, scenario evaluation and what-ifs. Another type of characterization, scenario evaluation and what-ifs. Another type of
consumer is a higher level on-router measurement application. consumer is a higher level on-router measurement application. One
One potential class of examples is composite measurements (e.g., potential class of examples is composite measurements (e.g., inter-
interpacket delay statistics) formed from a number of individual packet delay statistics) formed from a number of individual packet
packet measurements. Another class is network security measurements. Another class is network security applications, e.g.,
applications, e.g., IP traceback [32]. For some applications, the IP traceback [36]. For some applications, the ability to have low
ability to have low latency between packet measurement and reporting latency between packet measurement and reporting will be
will be particularly useful. particularly useful.
14. Statistical Estimation and Information Modeling 14. Statistical Estimation and Information Modeling
This section deals with engineering methods in statistical This section deals with engineering methods in statistical
estimation, as well as the need for an information model and estimation, as well as the need for an information model and
associated repository schema for the measurements. associated repository schema for the measurements.
14.1 Engineering methods for statistical estimation of measures 14.1 Engineering methods for statistical estimation of measures
The use of the well-established methods of optimal estimation [33, The use of the well-established methods of optimal estimation [37,
34, 35, 36] to obtain estimates of the measures for TE is 38, 39, 40] to obtain estimates of the measures for TE is
recommended. This draws upon several facts: recommended. This draws upon several facts:
. Internet traffic is inherently band-limited, but non-stationary; . Internet traffic is inherently band-limited, but non-stationary;
. Internet traffic may be heavy-tailed and possess strong short-term . Internet traffic may be heavy-tailed and possess strong short-term
correlations; correlations;
. A stationary, band-limited process can be approximated arbitrarily . A stationary, band-limited process can be approximated arbitrarily
closely by optimal estimation methods based on a finite number of closely by optimal estimation methods based on a finite number of
past samples. past samples.
Standard procedures for de-trending the raw data to provide "trend + Standard procedures for de-trending the raw data to provide "trend +
skipping to change at page 20, line 9 skipping to change at page 21, line 35
estimates of the measures defined herein. As the original raw estimates of the measures defined herein. As the original raw
process may be any of the measurements discussed in this document, process may be any of the measurements discussed in this document,
the above procedure may be applied without loss of generality to the above procedure may be applied without loss of generality to
measures of delay, loss, or complex measures of network state such as measures of delay, loss, or complex measures of network state such as
path characteristics, etc. path characteristics, etc.
In addition, these methods need to be applied across multiple time- In addition, these methods need to be applied across multiple time-
scales, so that TE applications can work with measures related to: scales, so that TE applications can work with measures related to:
. long-term trends over days, weeks, and months; . long-term trends over days, weeks, and months;
. busy-hour characterizations; and . busy-hour characterizations; and
. statistics and correlation properties on the order of seconds [37]. . statistics and correlation properties on the order of seconds [41].
The above estimation procedures apply equally to traffic workload, The above estimation procedures apply equally to traffic workload,
traffic performance, or other estimates of network state, such as the traffic performance, or other estimates of network state, such as the
state of routes. state of routes.
14.2 TE Measure Information Modeling 14.2 TE Measure Information Modeling
An information model is valuable for organizing data generated An information model is valuable for organizing data generated
through the estimation process. An information model is needed for through the estimation process. Measures must be associated with a
TE measures because a complete model does not exist for these large, and sometimes complicated set of attributes (e.g., as simple
measures. Measures must be associated with a large, and sometimes as an IP address of a measurement point, or as complex as the path of
complicated set of attributes (e.g., as simple as an IP address of a a round-trip measurement). Information models exist that richly
measurement point, or as complex as the path of a round-trip describe network elements and their configuration [42]. These models
measurement). have been extended to include policy mechanisms [43]. Specifications
for flows have been developed for network resource allocation
Information models exist that richly describe network elements and purposes [44]. No centralized information model exists that can
their configuration [38]. These models have been extended to include completely describe many of the TE measures defined herein.
policy mechanisms [39]. Specifications for flows have been developed Therefore, necessary integrating information models that make maximal
for network resource allocation purposes [40]. No centralized reuse of pre-existing work may need to be developed for TE measures.
information model exists that can completely describe many of the TE
measures defined herein. Therefore, necessary integrating
information models that make maximal reuse of pre-existing work may
need to be developed for TE measures.
As a brief example of the limitations of existing information models, As a brief example of the limitations of existing information models,
consider RFC 1363 [40] as a model for a traffic flow. It can be consider RFC 1363 [44] as a model for a traffic flow. It can be
described as collection of attributes defining traffic offered load, described as collection of attributes defining traffic offered load,
performance to be delivered (a goal), and the assurance level (risk) performance to be delivered (a goal), and the assurance level (risk)
associated with the actual performance obtained. The traffic offered associated with the actual performance obtained. The traffic offered
load is specified via an envelope described by a token bucket concept load is specified via an envelope described by a token bucket concept
(token bucket rate, bucket size) and a maximum transmission rate. (token bucket rate, bucket size) and a maximum transmission rate.
This model, while clearly intended for description of what a network This model, while clearly intended for description of what a network
will tolerate of a flow, could also be used to describe a flow in a will tolerate of a flow, could also be used to describe a flow in a
TE measure sense, e.g., "a flow that lives within the token rate x TE measure sense, e.g., "a flow that lives within the token rate x
and size y with probability 0.999." Note that a probability and size y with probability 0.999." Note that a probability
skipping to change at page 21, line 7 skipping to change at page 22, line 32
Similar limitations exist with respect to delivered performance Similar limitations exist with respect to delivered performance
specification in RFC1363, and the text of the RFC is quick to point specification in RFC1363, and the text of the RFC is quick to point
out, for example, that the "loss model is crude." For these reasons, out, for example, that the "loss model is crude." For these reasons,
and others, an appropriate information model is needed for TE and others, an appropriate information model is needed for TE
measures that can support uniformity of data definition in subsequent measures that can support uniformity of data definition in subsequent
TE applications. TE applications.
Several approaches and options for repository technology are now Several approaches and options for repository technology are now
broadly discussed. Relationships between TE measure information broadly discussed. Relationships between TE measure information
models on other information models (e.g., COPS) that drive network models on other information models (e.g., the COPS Policy Information
outcomes are of particular importance. Base, PIB) that drive network outcomes are of particular importance.
For an example of a PIB, see [45].
Linkages may need to be considered between policy mechanisms and TE Linkages may need to be considered between policy mechanisms and TE
measures. This is useful because, while policy-driven networking is measures. This is useful because, while policy-driven networking is
well-developed between the policy repositories, policy control points well-developed between the policy repositories, policy decision
and policy enforcement, policy content is very likely the output of points and policy enforcement points, policy content is very likely
TE applications. Since TE applications are dependent upon TE the output of TE applications [46]. Since TE applications are
measures, it is advantageous to provide traceability between the dependent upon TE measures, it is advantageous to provide
measures and the engineering changes made as a consequence of them. traceability between the measures and the engineering changes made as
a consequence of them. An example of a client application that might
be driven by TE measures through a PIB is found in [47, 48].
Measures (represented by their estimates) should be centrally stored Measures (represented by their estimates) should be centrally stored
and collected in a logical sense. This does not preclude distributed and collected in a logical sense. This does not preclude distributed
storage for purposes of volume management or security/survivability, storage for purposes of volume management or security/survivability,
but alludes to the need for a consistent retrieval mechanism (e.g., but alludes to the need for a consistent retrieval mechanism (e.g.,
NFS). Two methods are: (1) extend MIBs with new definitions for TE NFS). Two methods are: (1) extend MIBs with new definitions for TE
measure estimates, and (2) create data depositories through more measure estimates, and (2) create data depositories through more
centralized facilities, such as LDAP repositories. Both methods have centralized facilities, such as PIB repositories that can be accessed
merits as collection processes for TE measures, and are simple via LDAP (see [45]). Both methods have merits as collection
examples spanning a wide spectrum of solutions. These two methods processes for TE measures, and are simple examples spanning a wide
are discussed here for expository purposes, not to exclude other spectrum of solutions. These two methods are discussed here for
solutions. expository purposes, not to exclude other solutions.
Using MIBs allows well-established SNMP protocol and related Using MIBs allows well-established SNMP protocol and related
applications to retrieve data from the network elements being applications to retrieve data from the network elements being
measured. This is inherently "vendor-neutral," allowing commonly measured. This is inherently "vendor-neutral," allowing commonly
defined TE measurements to be stored for retrieval in a common MIB defined TE measurements to be stored for retrieval in a common MIB
definition, regardless of network element vendor, technology or other definition, regardless of network element vendor, technology or other
differences. Measurements from individual network elements differences. Measurements from individual network elements
(interfaces, routers, etc.) can be obtained "locally," if measures (interfaces, routers, etc.) can be obtained "locally," if measures
from a single network element are sufficient for a given TE from a single network element are sufficient for a given TE
application. However, if a network-wide view of the measurements is application. However, if a network-wide view of the measurements is
skipping to change at page 22, line 10 skipping to change at page 23, line 37
the advantage that TE applications (such as offline and online TE, or the advantage that TE applications (such as offline and online TE, or
measurement-based admission control) can be performed, and policy measurement-based admission control) can be performed, and policy
database content can be updated without invasive retrieval of data database content can be updated without invasive retrieval of data
from network-wide MIBs. Further, traceability can be established from network-wide MIBs. Further, traceability can be established
between the TE measurements in an LDAP repository, and the associated between the TE measurements in an LDAP repository, and the associated
policy content derived from them. policy content derived from them.
It is possible that both the MIB-based and LDAP-based (or another It is possible that both the MIB-based and LDAP-based (or another
approach altogether) should be considered jointly. approach altogether) should be considered jointly.
Although this document focuses on the motivation for providing
traffic measurement information, it is assumed that this information
should be provided to the participating devices by means of a
communication protocol that would be used between the aforementioned
participating devices and a presumably centralized entity that would
aim at storing, maintaining and updating this information, as well
as making appropriate decisions at the right time and under various
conditions.
This communication protocol should have the following
characteristics:
1. The protocol should make use of a reliable transport mode, given
the importance of configuration information.
2. The protocol architecture should provide a means for dynamically
provisioning the configuration information to the participating
devices, so that it may introduce/contribute to a high level of
automation in the actual traffic measurement operation.
3. The protocol should support a reporting mechanism that may be
used for statistical information retrieval.
4. The protocol should support the appropriate security mechanisms
to provide some guarantees as far as the preservation of the
confidentiality of the configuration information is concerned.
15. Conclusions and Recommendations 15. Conclusions and Recommendations
This document is intended as a framework for traffic metrics needed This document is intended as a framework for traffic metrics needed
for successful TE. Principles of best practice in traffic for successful TE. Principles of best practice in traffic
characterization and performance characterization are described. characterization and performance characterization are described.
For interoperable compatibility, basic areas of traffic measurement For interoperable compatibility, basic areas of traffic measurement
recommended for standardization include: recommended for standardization include:
(1) specific TE measurements (1) Specific TE measurements
. Use of node-pair-based traffic data to derive per-service-class . Use of node-pair-based traffic data to derive per-service-class
traffic matrix statistics traffic matrix statistics
. Statistics of carried load versus performance . Statistics of carried load versus performance
. A standardized mechanism to detect and record label binding . A standardized mechanism to detect and record label binding
changes for LDP-signaled label-switched paths, to facilitate the changes for LDP-signaled label-switched paths, to facilitate the
collection of node-pair-based traffic data collection of node-pair-based traffic data
(2) traffic data collection methods (2) Traffic data collection methods
. Need for uniform measurement definitions across vendors and . Need for uniform measurement definitions across vendors and
operators operators
. Distinction between traffic offered load versus achieved . Distinction between traffic offered load versus achieved
throughput throughput
. Need for higher-order statistics for service assurance . Need for higher-order statistics for service assurance
. Need for packet-sampled measurements that preserve representative . Need for packet-sampled measurements that preserve representative
traffic detail at manageable sample volumes traffic detail at manageable sample volumes
. Need for offline bulk file transfer and standardized . Need for offline bulk file transfer and standardized
filtering/aggregation mechanisms to manage large volumes of filtering/aggregation mechanisms to manage large volumes of
skipping to change at page 23, line 4 skipping to change at page 24, line 55
provider are responsible for providing sufficient data integrity provider are responsible for providing sufficient data integrity
(e.g., to prevent forgery of measurement records) and (e.g., to prevent forgery of measurement records) and
confidentiality (e.g., by restricting attention only to the packet confidentiality (e.g., by restricting attention only to the packet
headers of interest). It is also assumed that a service provider headers of interest). It is also assumed that a service provider
will take proper precautions to ensure that access to its will take proper precautions to ensure that access to its
measurement systems and all associated data is secure by using measurement systems and all associated data is secure by using
appropriate authentication techniques. Methods to achieve these appropriate authentication techniques. Methods to achieve these
security considerations are not addressed in this document. security considerations are not addressed in this document.
17. References 17. References
Normative References
Normative References
References 1, 2, and 13 below are considered normative. References 1, 2, and 13 below are considered normative.
Informative References Informative References
1 D.O. Awduche, A. Chiu, A. Elwalid, I. Widjaja, and X. Xiao, 1 D.O. Awduche, A. Chiu, A. Elwalid, I. Widjaja, and X. Xiao,
"Overview and Principles of Internet Traffic Engineering," RFC "Overview and Principles of Internet Traffic Engineering," RFC
3272, May 2002. 3272, May 2002.
2 V. Paxson, G. Almes, J. Mahdavi, and M. Mathis, "Framework for IP 2 V. Paxson, G. Almes, J. Mahdavi, and M. Mathis, "Framework for IP
Performance Metrics," RFC 2330, May 1998. Performance Metrics," RFC 2330, May 1998.
3 J. Mahdavi and V. Paxson, "IPPM Metrics for Measuring 3 J. Mahdavi and V. Paxson, "IPPM Metrics for Measuring
skipping to change at page 23, line 33 skipping to change at page 25, line 31
7 M. Mathis and M. Allman, "A Framework for Defining Empirical Bulk 7 M. Mathis and M. Allman, "A Framework for Defining Empirical Bulk
Transfer Capacity Metrics," RFC 3148, July 2001. Transfer Capacity Metrics," RFC 3148, July 2001.
8 R. Koodli and R. Ravikanth, "One-way Loss Pattern Sample 8 R. Koodli and R. Ravikanth, "One-way Loss Pattern Sample
Metrics," RFC 3357, August 2002. Metrics," RFC 3357, August 2002.
9 C. Demichelis and P. Chimento, "IP Packet Delay Variation Metric 9 C. Demichelis and P. Chimento, "IP Packet Delay Variation Metric
for IP Performance Metrics (IPPM)," RFC 3393, November 2002. for IP Performance Metrics (IPPM)," RFC 3393, November 2002.
10 V. Raisanen, G. Grotefeld, and A. Morton, "Network performance 10 V. Raisanen, G. Grotefeld, and A. Morton, "Network performance
measurement with periodic streams," RFC 3432, November 2002. measurement with periodic streams," RFC 3432, November 2002.
11 ITU-T Recommendation I.380/Y.1540, "Internet Protocol Data 11 ITU-T Recommendation I.380/Y.1540, "Internet Protocol Data
Communication Service -- IP Packet Transfer and Availability Communication Service -- IP Packet Transfer and Availability
Performance Parameters," February 1999. Performance Parameters," First Issued February 1999, Revised
December 2002.
12 ITU-T Recommendation Y.1541, "Network Performance Objectives for 12 ITU-T Recommendation Y.1541, "Network Performance Objectives for
IP-Based Services," May 2002. IP-Based Services," May 2002.
13 E. Rosen, A. Viswanathan, and R. Callon, "Multiprotocol Label 13 E. Rosen, A. Viswanathan, and R. Callon, "Multiprotocol Label
Switching Architecture," RFC 3031, January 2001. Switching Architecture," RFC 3031, January 2001.
14 S. Bradner (Editor), "Benchmarking Terminology for Network 14 S. Bradner (Editor), "Benchmarking Terminology for Network
Interconnection Devices," RFC 1242, July 1991. Interconnection Devices," RFC 1242, July 1991.
15 G. Ash, "Traffic Engineering & QoS Methods for IP-, ATM-, & TDM- 15 G. Ash, "Traffic Engineering & QoS Methods for IP-, ATM-, & TDM-
Based Multiservice Networks," Internet-Draft, Work in Progress, Based Multiservice Networks," Internet-Draft, Work in Progress,
October 2001. October 2001.
16 K. Kompella, Y. Rekhter, and L. Berger, "Link Bundling in MPLS 16 D. Awduche, L. Berger, D. Gan, T. Li, V. Srinivasan, and G.
Swallow, "RSVP-TE: Extensions to RSVP for LSP Tunnels," RFC 3209,
December 2001.
17 B. Jamoussi (Editor), "Constraint-Based LSP Setup using LDP," RFC
3212, January 2002.
18 P. Ashwood-Smith, B. Jamoussi, D. Fedyk, and D. Skalecki,
"Improving Topology Data Base Accuracy with Label Switched Path
Feedback in Constraint Based Label Distribution Protocol,"
Internet-Draft, Work in Progress, November 2002.
19 K. Kompella, Y. Rekhter, and L. Berger, "Link Bundling in MPLS
Traffic Engineering," Internet-Draft, Work in Progress, February Traffic Engineering," Internet-Draft, Work in Progress, February
2001. 2001.
17 W.S. Lai, "Traffic Measurement for Dimensioning and Control of IP 20 W.S. Lai, "Traffic Measurement for Dimensioning and Control of IP
Networks," Internet Performance and Control of Network Systems II Networks," Internet Performance and Control of Network Systems II
Conference, SPIE Proceedings, Vol. 4523, Denver, Colorado, 21-22 Conference, SPIE Proceedings, Vol. 4523, Denver, Colorado, 21-22
August 2001, pp. 359-367. August 2001, pp. 359-367.
18 C. Srinivasan, A. Viswanathan, and T.D. Nadeau, "Multiprotocol 21 C. Srinivasan, A. Viswanathan, and T.D. Nadeau, "Multiprotocol
Label Switching (MPLS) Label Switch Router (LSR) Management Label Switching (MPLS) Label Switch Router (LSR) Management
Information Base," Internet-Draft, Work in Progress, January Information Base," Internet-Draft, Work in Progress, January
2002. 2002.
22 C. Srinivasan, A. Viswanathan, and T.D. Nadeau, "Multiprotocol
19 C. Srinivasan, A. Viswanathan, and T.D. Nadeau, "Multiprotocol
Label Switching (MPLS) Traffic Engineering Management Information Label Switching (MPLS) Traffic Engineering Management Information
Base," Internet-Draft, Work in Progress, January 2002. Base," Internet-Draft, Work in Progress, January 2002.
20 K. Kompella, "A Traffic Engineering MIB," Internet-Draft, Work in 23 K. Kompella, "A Traffic Engineering MIB," Internet-Draft, Work in
Progress, September 2002. Progress, September 2002.
21 A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford, and 24 A. Feldmann, A. Greenberg, C. Lund, N. Reingold, J. Rexford, and
F. True, "Deriving Traffic Demands for Operational IP Networks: F. True, "Deriving Traffic Demands for Operational IP Networks:
Methodology and Experience," Proc. ACM SIGCOMM 2000, Stockholm, Methodology and Experience," Proc. ACM SIGCOMM 2000, Stockholm,
Swedan. Swedan.
22 A. Feldmann, A. Greenberg, C. Lund, N. Reingold, and J. Rexford, 25 A. Feldmann, A. Greenberg, C. Lund, N. Reingold, and J. Rexford,
"NetScope: Traffic Engineering for IP Networks," IEEE Network, "NetScope: Traffic Engineering for IP Networks," IEEE Network,
March/April 2000. March/April 2000.
23 T.D. Nadeau, C. Srinivasan, and A. Viswanathan, "Multiprotocol 26 T.D. Nadeau, C. Srinivasan, and A. Viswanathan, "Multiprotocol
Label Switching (MPLS) FEC-To-NHLFE (FTN) Management Information Label Switching (MPLS) FEC-To-NHLFE (FTN) Management Information
Base," Internet-Draft, Work in Progress, January 2002. Base," Internet-Draft, Work in Progress, January 2002.
24 P. Pate, X. Xiao, T. So, A. Malis, T. Nadeau, S. Bryant, C. 27 P. Pate, X. Xiao, T. So, A. Malis, T. Nadeau, S. Bryant, C.
White, K. Kompella, and T. Johnson, "Framework for Pseudo Wire White, K. Kompella, and T. Johnson, "Framework for Pseudo Wire
Emulation Edge-to-Edge (PWE3)," Internet-Draft, Work in Progress, Emulation Edge-to-Edge (PWE3)," Internet-Draft, Work in Progress,
June 2002. June 2002.
25 N. Harrison, P. Willis, S. Davari, E. Cuevas, B. Mack-Crane, E. 28 N. Harrison, P. Willis, S. Davari, E. Cuevas, B. Mack-Crane, E.
Franze, H. Ohta, T. So, S. Goldfless, and F. Chen, "Requirements Franze, H. Ohta, T. So, S. Goldfless, and F. Chen, "Requirements
for OAM in MPLS Networks," Internet-Draft, Work in Progress, May for OAM in MPLS Networks," Internet-Draft, Work in Progress, May
2001. 2001.
26 ITU-T Draft Recommendation Y.1710, "Requirements for OAM 29 ITU-T Draft Recommendation Y.1710, "Requirements for OAM
Functionality for MPLS Networks," May 2001. Functionality for MPLS Networks," May 2001.
27 ITU-T Draft Recommendation Y.1711, "OAM Mechanisms for MPLS 30 ITU-T Draft Recommendation Y.1711, "OAM Mechanisms for MPLS
Networks," May 2001. Networks," May 2001.
28 R.G. Cole, R. Dietz, C. Kalbfleisch, and D. Romascanu, "A 31 R.G. Cole, R. Dietz, C. Kalbfleisch, and D. Romascanu, "A
Framework for Synthetic Sources for Performance Monitoring," Framework for Synthetic Sources for Performance Monitoring,"
Internet-Draft, Work in Progress, May 2001. Internet-Draft, Work in Progress, May 2001.
29 R. Bonica, K. Kompella, and D. Meyer, "Tracing Requirements for 32 R. Bonica, K. Kompella, and D. Meyer, "Tracing Requirements for
Generic Tunnels," Internet-Draft, Work in Progress, August 2002. Generic Tunnels," Internet-Draft, Work in Progress, August 2002.
30 N.G. Duffield (Editor), "A Framework for Passive Packet 33 K. Kompella, P. Pan, N. Sheth, D. Cooper, G. Swallow, S. Wadhwa,
and R. Bonica, "Detecting MPLS Data Plane Liveness," Internet-
Draft, Work in Progress, October 2002.
34 N.G. Duffield (Editor), "A Framework for Passive Packet
Measurement," Internet-Draft, Work in Progress, September 2002. Measurement," Internet-Draft, Work in Progress, September 2002.
31 N.G. Duffield and M. Grossglauser, "Trajectory Sampling for 35 N.G. Duffield and M. Grossglauser, "Trajectory Sampling for
Direct Traffic Observation," IEEE/ACM Trans. on Networking, 9(3), Direct Traffic Observation," IEEE/ACM Trans. on Networking, 9(3),
pp. 280-292, June 2001. pp. 280-292, June 2001.
32 C. Partridge, C. Jones, D. Waitzman, and A. Snoeren, "New 36 C. Partridge, C. Jones, D. Waitzman, and A. Snoeren, "New
Protocols to Support Internet Traceback," Internet-Draft, Work in Protocols to Support Internet Traceback," Internet-Draft, Work in
Progress, November 2001. Progress, November 2001.
33 S. Haykin, Ed., "Kalman Filtering and Neural Networks," Wiley 37 S. Haykin, Ed., "Kalman Filtering and Neural Networks," Wiley
Interscience, 2001. Interscience, 2001.
34 A. Papoulis, "Probability, Random Variables and Stochastic
38 A. Papoulis, "Probability, Random Variables and Stochastic
Processes," 3rd Ed., McGraw-Hill, 1991. Processes," 3rd Ed., McGraw-Hill, 1991.
35 A. Gelb, Ed., "Applied Optimal Estimation," MIT Press, 1974. 39 A. Gelb, Ed., "Applied Optimal Estimation," MIT Press, 1974.
36 I. R. Petersen, V. A. Ugrinovskii, A. V. Savkin, "Robust Control 40 I. R. Petersen, V. A. Ugrinovskii, A. V. Savkin, "Robust Control
Design Using H<\infinity> Methods," Springer, 2000. Design Using H<\infinity> Methods," Springer, 2000.
37 V. Bolotin, J. Coombs-Reyes, D. Heyman, Y. Levy, and D. Liu, "IP 41 V. Bolotin, J. Coombs-Reyes, D. Heyman, Y. Levy, and D. Liu, "IP
Traffic Characterization for Planning and Control," Proc. ITC16, Traffic Characterization for Planning and Control," Proc. ITC16,
Edinburgh, Scotland, June 1999. Edinburgh, Scotland, June 1999.
42 Distributed Management Task Force (DMTF) Common Information Model
38 Distributed Management Task Force (DMTF) Common Information Model
(CIM), www.dmtf.org (CIM), www.dmtf.org
39 B. Moore, E. Ellesson, and J. Strassner, "Policy Core Information 43 B. Moore, E. Ellesson, and J. Strassner, "Policy Core Information
Model -- Version 1 Specification," RFC 3060, February 2001. Model -- Version 1 Specification," RFC 3060, February 2001.
40 C. Partridge, "A Proposed Flow Specification," RFC 1363, 44 C. Partridge, "A Proposed Flow Specification," RFC 1363,
September 1992. September 1992.
45 R. Yavatkar, D. Pendarakis, and R. Guerin, "A Framework for
Policy-based Admission Control," RFC 2753, January 2000.
46 D. Rawlins, A. Kulkarni, M. Bokaemper, and K.H. Chan, "Framework
for Policy Usage Feedback for Common Open Policy Service with
Policy Provisioning (COPS-PR)," Internet-Draft, Work in Progress,
December 2002.
47 C. Jacquenet, "An IP Forwarding Policy Information Base,"
Internet-Draft, Work in Progress, January 2003.
48 C. Jacquenet, "A COPS client-type for IP traffic engineering,"
Internet-Draft, Work in Progress, January 2003.
18. Intellectual Property Statement 18. Intellectual Property Statement
AT&T Corp. may own intellectual property applicable to packet AT&T Corp. may own intellectual property applicable to packet
sampling as presented in references [30, 31] and summarized in sampling as presented in references [34, 35] and summarized in
Section 13. AT&T is currently reviewing its licensing intent Section 13. AT&T is currently reviewing its licensing intent
relative to the Intellectual Property and will notify the IETF when relative to the Intellectual Property and will notify the IETF when
AT&T has made a determination of that intent. AT&T has made a determination of that intent.
19. Acknowledgments 19. Acknowledgments
Thanks to the inputs from Gerald Ash, Jim Boyle, Robert Cole, Thanks to the inputs from Gerald Ash, Jim Boyle, Robert Cole,
Enrique Cuevas, Merike Kaeo, Ed Kern, Spyros Kontogiorgis, Alfred Enrique Cuevas, Christian Jacquenet, Merike Kaeo, Ed Kern, Spyros
Morton, Thomas Nadeau, Moshe Segal, Bert Wijnen, and the Tequila Kontogiorgis, Alfred Morton, Thomas Nadeau, Dimitri Papadimitriou,
Moshe Segal, Jing Shen, Bert Wijnen, Raymond Zhang, and the Tequila
project. Special thanks to Blaine Christian for starting this work project. Special thanks to Blaine Christian for starting this work
and contributing to the initial versions. Nick Duffield provided and contributing to the initial versions. Nick Duffield provided
section 13 on packet sampling. section 13 on packet sampling.
20. Author's Addresses 20. Author's Addresses
Wai Sum Lai Wai Sum Lai
AT&T Labs AT&T Labs
Room D5-3D18 Room D5-3D18
200 Laurel Avenue 200 Laurel Avenue
skipping to change at page 26, line 4 skipping to change at page 28, line 19
Email: drtibbs@oakcitysolutions.com Email: drtibbs@oakcitysolutions.com
Steven Van den Berghe Steven Van den Berghe
Ghent University/IMEC Ghent University/IMEC
St. Pietersnieuwsstraat 41 St. Pietersnieuwsstraat 41
B-9000 Ghent, Belgium B-9000 Ghent, Belgium
Phone: ++32 9 267 35 86 Phone: ++32 9 267 35 86
E-mail: steven.vandenberghe@intec.rug.ac.be E-mail: steven.vandenberghe@intec.rug.ac.be
Full Copyright Statement Full Copyright Statement
"Copyright (C) The Internet Society (date). All Rights Reserved. "Copyright (C) The Internet Society (date). All Rights Reserved.
This document and translations of it may be copied and furnished to This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it others, and derivative works that comment on or otherwise explain it
or assist in its implmentation may be prepared, copied, published or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than followed, or as required to translate it into languages other than
English. English.
 End of changes. 

This html diff was produced by rfcdiff 1.25, available from http://www.levkowetz.com/ietf/tools/rfcdiff/