draft-ietf-ippm-framework-compagg-09.txt   rfc5835.txt 
Network Working Group A. Morton, Ed. Internet Engineering Task Force (IETF) A. Morton, Ed.
Internet-Draft AT&T Labs Request for Comments: 5835 AT&T Labs
Intended status: Informational S. Van den Berghe, Ed. Category: Informational S. Van den Berghe, Ed.
Expires: June 23, 2010 Alcatel-Lucent ISSN: 2070-1721 Alcatel-Lucent
December 20, 2009 April 2010
Framework for Metric Composition Framework for Metric Composition
draft-ietf-ippm-framework-compagg-09
Abstract Abstract
This memo describes a detailed framework for composing and This memo describes a detailed framework for composing and
aggregating metrics (both in time and in space) originally defined by aggregating metrics (both in time and in space) originally defined by
the IP Performance Metrics (IPPM) RFC 2330 and developed by the IETF. the IP Performance Metrics (IPPM), RFC 2330, and developed by the
This new framework memo describes the generic composition and IETF. This new framework memo describes the generic composition and
aggregation mechanisms. The memo provides a basis for additional aggregation mechanisms. The memo provides a basis for additional
documents that implement the framework to define detailed documents that implement the framework to define detailed
compositions and aggregations of metrics which are useful in compositions and aggregations of metrics that are useful in practice.
practice.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Status of This Memo
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at This document is not an Internet Standards Track specification; it is
http://www.ietf.org/ietf/1id-abstracts.txt. published for informational purposes.
The list of Internet-Draft Shadow Directories can be accessed at This document is a product of the Internet Engineering Task Force
http://www.ietf.org/shadow.html. (IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are a candidate for any level of Internet
Standard; see Section 2 of RFC 5741.
This Internet-Draft will expire on June 23, 2010. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc5835.
Copyright Notice Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the Copyright (c) 2010 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the BSD License. described in the Simplified BSD License.
This document may contain material from IETF Documents or IETF This document may contain material from IETF Documents or IETF
Contributions published or made publicly available before November Contributions published or made publicly available before November
10, 2008. The person(s) controlling the copyright in some of this 10, 2008. The person(s) controlling the copyright in some of this
material may not have granted the IETF Trust the right to allow material may not have granted the IETF Trust the right to allow
modifications of such material outside the IETF Standards Process. modifications of such material outside the IETF Standards Process.
Without obtaining an adequate license from the person(s) controlling Without obtaining an adequate license from the person(s) controlling
the copyright in such materials, this document may not be modified the copyright in such materials, this document may not be modified
outside the IETF Standards Process, and derivative works of it may outside the IETF Standards Process, and derivative works of it may
not be created outside the IETF Standards Process, except to format not be created outside the IETF Standards Process, except to format
it for publication as an RFC or to translate it into languages other it for publication as an RFC or to translate it into languages other
than English. than English.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Introduction ....................................................4
1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1. Motivation .................................................4
1.1.1. Reducing Measurement Overhead . . . . . . . . . . . . 4 1.1.1. Reducing Measurement Overhead .......................4
1.1.2. Measurement Re-use . . . . . . . . . . . . . . . . . . 5 1.1.2. Measurement Re-Use ..................................5
1.1.3. Data Reduction and Consolidation . . . . . . . . . . . 5 1.1.3. Data Reduction and Consolidation ....................5
1.1.4. Implications on Measurement Design and Reporting . . . 6 1.1.4. Implications on Measurement Design and Reporting ....6
2. Purpose and Scope . . . . . . . . . . . . . . . . . . . . . . 6 2. Requirements Language ...........................................6
3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 3. Purpose and Scope ...............................................6
3.1. Measurement Point . . . . . . . . . . . . . . . . . . . . 6 4. Terminology .....................................................7
3.2. Complete Path . . . . . . . . . . . . . . . . . . . . . . 7 4.1. Measurement Point ..........................................7
3.3. Complete Path Metric . . . . . . . . . . . . . . . . . . . 7 4.2. Complete Path ..............................................7
3.4. Complete Time Interval . . . . . . . . . . . . . . . . . . 7 4.3. Complete Path Metric .......................................7
3.5. Composed Metric . . . . . . . . . . . . . . . . . . . . . 7 4.4. Complete Time Interval .....................................7
3.6. Composition Function . . . . . . . . . . . . . . . . . . . 7 4.5. Composed Metric ............................................7
3.7. Ground Truth . . . . . . . . . . . . . . . . . . . . . . . 7 4.6. Composition Function .......................................7
3.8. Interval . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.7. Ground Truth ...............................................8
3.9. Sub-interval . . . . . . . . . . . . . . . . . . . . . . . 8 4.8. Interval ...................................................8
3.10. Sub-path . . . . . . . . . . . . . . . . . . . . . . . . . 8 4.9. Sub-Interval ...............................................8
3.11. Sub-path Metrics . . . . . . . . . . . . . . . . . . . . . 8 4.10. Sub-Path ..................................................8
4. Description of Metric Types . . . . . . . . . . . . . . . . . 8 4.11. Sub-Path Metrics ..........................................8
4.1. Temporal Aggregation Description . . . . . . . . . . . . . 8 5. Description of Metric Types .....................................9
4.2. Spatial Aggregation Description . . . . . . . . . . . . . 9 5.1. Temporal Aggregation Description ...........................9
4.3. Spatial Composition Description . . . . . . . . . . . . . 10 5.2. Spatial Aggregation Description ............................9
4.4. Help Metrics . . . . . . . . . . . . . . . . . . . . . . . 10 5.3. Spatial Composition Description ...........................10
4.5. Higher Order Composition . . . . . . . . . . . . . . . . . 10 5.4. Help Metrics ..............................................10
5. Requirements for Composed Metrics . . . . . . . . . . . . . . 11 5.5. Higher-Order Composition ..................................11
5.1. Note on IPR . . . . . . . . . . . . . . . . . . . . . . . 12 6. Requirements for Composed Metrics ..............................11
6. Guidelines for Defining Composed Metrics . . . . . . . . . . . 12 6.1. Note on Intellectual Property Rights (IPR) ................12
6.1. Ground Truth: Comparison with other IPPM Metrics . . . . . 12 7. Guidelines for Defining Composed Metrics .......................12
6.1.1. Ground Truth for Temporal Aggregation . . . . . . . . 14 7.1. Ground Truth: Comparison with Other IPPM Metrics ..........12
6.1.2. Ground Truth for Spatial Aggregation . . . . . . . . . 15 7.1.1. Ground Truth for Temporal Aggregation ..............14
6.2. Deviation from the Ground Truth . . . . . . . . . . . . . 15 7.1.2. Ground Truth for Spatial Aggregation ...............15
6.3. Incomplete Information . . . . . . . . . . . . . . . . . . 15 7.2. Deviation from the Ground Truth ...........................15
6.4. Time Varying Metrics . . . . . . . . . . . . . . . . . . . 15 7.3. Incomplete Information ....................................15
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 7.4. Time-Varying Metrics ......................................15
8. Security Considerations . . . . . . . . . . . . . . . . . . . 16 8. Security Considerations ........................................16
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 16 9. Acknowledgements ...............................................16
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 16 10. References ....................................................16
10.1. Normative References . . . . . . . . . . . . . . . . . . . 16 10.1. Normative References .....................................16
10.2. Informative References . . . . . . . . . . . . . . . . . . 17 10.2. Informative References ...................................17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 18
1. Introduction 1. Introduction
The IPPM framework [RFC2330] describes two forms of metric The IP Performance Metrics (IPPM) framework [RFC2330] describes two
composition, spatial and temporal. The text also suggests that the forms of metric composition, spatial and temporal. The text also
concepts of the analytical framework (or A-frame) would help to suggests that the concepts of the analytical framework (or A-frame)
develop useful relationships to derive the composed metrics from real would help to develop useful relationships to derive the composed
metrics. The effectiveness of composed metrics is dependent on their metrics from real metrics. The effectiveness of composed metrics is
usefulness in analysis and applicability to practical measurement dependent on their usefulness in analysis and applicability to
circumstances. practical measurement circumstances.
This memo expands on the notion of composition, and provides a This memo expands on the notion of composition, and provides a
detailed framework for several classes of metrics that were described detailed framework for several classes of metrics that were described
in the original IPPM framework. The classes include temporal in the original IPPM framework. The classes include temporal
aggregation, spatial aggregation, and spatial composition. aggregation, spatial aggregation, and spatial composition.
1.1. Motivation 1.1. Motivation
Network operators have deployed measurement systems to serve many Network operators have deployed measurement systems to serve many
purposes, including performance monitoring, maintenance support, purposes, including performance monitoring, maintenance support,
skipping to change at page 5, line 13 skipping to change at page 5, line 16
spans multiple networks, and where each operator conducts their own spans multiple networks, and where each operator conducts their own
measurements. Here, the complete path performance may be estimated measurements. Here, the complete path performance may be estimated
from measurements on the component parts. from measurements on the component parts.
Operators that take advantage of the composition and aggregation Operators that take advantage of the composition and aggregation
methods recognize that the estimates may exhibit some additional methods recognize that the estimates may exhibit some additional
error beyond that inherent in the measurements themselves, and so error beyond that inherent in the measurements themselves, and so
they are making a trade-off to achieve reasonable measurement system they are making a trade-off to achieve reasonable measurement system
overhead. overhead.
1.1.2. Measurement Re-use 1.1.2. Measurement Re-Use
There are many different measurement users, each bringing specific There are many different measurement users, each bringing specific
requirements for the reporting timescale. Network managers and requirements for the reporting timescale. Network managers and
maintenance forces prefer to see results presented very rapidly, to maintenance forces prefer to see results presented very rapidly, to
detect problems quickly or see if their action has corrected a detect problems quickly or see if their action has corrected a
problem. On the other hand, network capacity planners and even problem. On the other hand, network capacity planners and even
network users sometimes prefer a long-term view of performance, for network users sometimes prefer a long-term view of performance, for
example to check trends. How can one set of measurements serve both example to check trends. How can one set of measurements serve both
needs? needs?
skipping to change at page 5, line 39 skipping to change at page 5, line 42
long-term measurement if the problem was undetected. long-term measurement if the problem was undetected.
1.1.3. Data Reduction and Consolidation 1.1.3. Data Reduction and Consolidation
Another motivation is data reduction. Assume there is a network in Another motivation is data reduction. Assume there is a network in
which delay measurements are performed among a subset of its nodes. which delay measurements are performed among a subset of its nodes.
A network manager might ask whether there is a problem with the A network manager might ask whether there is a problem with the
network delay in general. It would be desirable to obtain a single network delay in general. It would be desirable to obtain a single
value that gives an indication of the overall network delay. Spatial value that gives an indication of the overall network delay. Spatial
aggregation methods would address this need, and can produce the aggregation methods would address this need, and can produce the
desired "single figure of merit" asked for, one that may also be desired "single figure of merit" asked for, which may also be useful
useful in trend analysis. in trend analysis.
The overall value would be calculated from the elementary delay The overall value would be calculated from the elementary delay
measurements, but it not obvious how: for example, it may not to be measurements, but it is not obvious how: for example, it may not be
reasonable to average all delay measurements, as some paths (e.g. reasonable to average all delay measurements, as some paths (e.g.,
having a higher bandwidth or more important customers) might be those having a higher bandwidth or more important customers) might be
considered more critical than others. considered more critical than others.
Metric composition can help to provide, from raw measurement data, Metric composition can help to provide, from raw measurement data,
some tangible, well-understood and agreed upon information about the some tangible, well-understood and agreed-upon information about the
service guarantees provided by a network. Such information can be service guarantees provided by a network. Such information can be
used in the Service Level Agreement/Service Level Specification (SLA/ used in the Service Level Agreement/Service Level Specification
SLS) contracts between a service provider and its customers. (SLA/SLS) contracts between a service provider and its customers.
1.1.4. Implications on Measurement Design and Reporting 1.1.4. Implications on Measurement Design and Reporting
If a network measurement system operator anticipates needing to If a network measurement system operator anticipates needing to
produce overall metrics by composition, then it is prudent to keep produce overall metrics by composition, then it is prudent to keep
that requirement in mind when considering the measurement design and that requirement in mind when considering the measurement design and
sampling plan. Also, certain summary statistics are more conducive sampling plan. Also, certain summary statistics are more conducive
to composition than others, and this figures prominently in the to composition than others, and this figures prominently in the
design of measurements and when reporting the results. design of measurements and when reporting the results.
2. Purpose and Scope 2. Requirements Language
The purpose of this memo is provide a common framework for the The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
3. Purpose and Scope
The purpose of this memo is to provide a common framework for the
various classes of metrics that are composed from primary metrics. various classes of metrics that are composed from primary metrics.
The scope is limited to the definitions of metrics that are composed The scope is limited to the definitions of metrics that are composed
from primary metrics using a deterministic function. Key information from primary metrics using a deterministic function. Key information
about each composed metric, such as the assumptions under which the about each composed metric is included, such as the assumptions under
relationship holds and possible sources of error/circumstances where which the relationship holds and possible sources of
the composition may fail, are included. error/circumstances where the composition may fail.
At this time, the scope of effort is limited to composed metrics for At this time, the scope of effort is limited to composed metrics for
packet loss, delay, and delay variation, as defined in [RFC2679], packet loss, delay, and delay variation, as defined in [RFC2679],
[RFC2680], [RFC2681], [RFC3393], [RFC5481], and the comparable [RFC2680], [RFC2681], [RFC3393], [RFC5481], and the comparable
metrics in [Y.1540] . Composition of packet reordering metrics metrics in [Y.1540]. Composition of packet reordering metrics
[RFC4737] and duplication metrics [RFC5560] are considered research [RFC4737] and duplication metrics [RFC5560] are considered research
topics at the time this memo was prepared, and beyond its scope. topics at the time this memo was prepared, and are beyond the scope
of this document.
This memo will retain the terminology of the IPPM Framework This memo will retain the terminology of the IPPM Framework [RFC2330]
[RFC2330]as much as possible, but will extend the terminology when as much as possible, but will extend the terminology when necessary.
necessary. It is assumed that the reader is familiar with the It is assumed that the reader is familiar with the concepts
concepts introduced in [RFC2330], as they will not be repeated here. introduced in [RFC2330], as they will not be repeated here.
3. Terminology 4. Terminology
This section defines the terminology applicable to the processes of This section defines the terminology applicable to the processes of
Metric Composition and Aggregation. metric composition and aggregation.
3.1. Measurement Point 4.1. Measurement Point
The logical or physical location where packet observations are made. A measurement point is the logical or physical location where packet
The term Measurement Point is synonymous with the term "observation observations are made. The term "measurement point" is synonymous
position" used in [RFC2330] when describing the notion of wire time. with the term "observation position" used in [RFC2330] when
A measurement point may be at the boundary between a host and an describing the notion of wire time. A measurement point may be at
adjacent link (physical), or it may be within a host (logical) that the boundary between a host and an adjacent link (physical), or it
performs measurements where the difference between host time and wire may be within a host (logical) that performs measurements where the
time is understood. difference between host time and wire time is understood.
3.2. Complete Path 4.2. Complete Path
The complete path is the actual path that a packet would follow as it The complete path is the actual path that a packet would follow as it
travels from the packet's Source to its Destination. A Complete path travels from the packet's Source to its Destination. A complete path
may span the administrative boundaries of one or more networks. may span the administrative boundaries of one or more networks.
3.3. Complete Path Metric 4.3. Complete Path Metric
The complete path metric is the Source to Destination metric that a The complete path metric is the Source-to-Destination metric that a
composed metric attempts to estimate. A complete path metric composed metric attempts to estimate. A complete path metric
represents the ground-truth for a composed metric. represents the ground-truth for a composed metric.
3.4. Complete Time Interval 4.4. Complete Time Interval
The complete time interval is comprised of two or more contiguous The complete time interval is comprised of two or more contiguous
sub-intervals, and is the interval whose performance will be sub-intervals, and is the interval whose performance will be
estimated through temporal aggregation. estimated through temporal aggregation.
3.5. Composed Metric 4.5. Composed Metric
A composed metric is an estimate of an actual metric describing the A composed metric is an estimate of an actual metric describing the
performance of a path over some time interval. A composed metric is performance of a path over some time interval. A composed metric is
derived from other metrics by applying a deterministic process or derived from other metrics by applying a deterministic process or
function (e.g., a composition function). The process may use metrics function (e.g., a composition function). The process may use metrics
that are identical to the metric being composed, or metrics that are that are identical to the metric being composed, or metrics that are
dissimilar, or some combination of both types. dissimilar, or some combination of both types.
3.6. Composition Function 4.6. Composition Function
A composition function is a deterministic process applied to A composition function is a deterministic process applied to
individual metrics to derive another metric (such as a Composed individual metrics to derive another metric (such as a composed
metric). metric).
3.7. Ground Truth 4.7. Ground Truth
As applied here, the notion of ground truth is defined as the actual As applied here, the notion of "ground truth" is defined as the
performance of a network path over some time interval. The ground actual performance of a network path over some time interval. The
truth is a metric on the (unavailable) packet transfer information ground truth is a metric on the (unavailable) packet transfer
for the desired path and time interval that a composed metric seeks information for the desired path and time interval that a composed
to estimate. metric seeks to estimate.
3.8. Interval 4.8. Interval
A span of time. An interval refers to a span of time.
3.9. Sub-interval 4.9. Sub-Interval
A Sub-interval is a time interval that is included in another A sub-interval is a time interval that is included in another
interval. interval.
3.10. Sub-path 4.10. Sub-Path
A Sub-path is a portion of the complete path where at least the Sub- A sub-path is a portion of the complete path where at least the
path Source and Destination hosts are constituents of the complete sub-path Source and Destination hosts are constituents of the
path. We say that such a sub-path is "involved" in the complete complete path. We say that such a sub-path is "involved" in the
path. complete path.
Since sub-paths terminate on hosts, it is important to describe how Since sub-paths terminate on hosts, it is important to describe how
sub-paths are considered to be joined. In practice, the Source and sub-paths are considered to be joined. In practice, the Source and
Destination hosts may perform the function of measurement points. Destination hosts may perform the function of measurement points.
If the Destination and Source hosts of two adjoining paths are co- If the Destination and Source hosts of two adjoining paths are
located and the link between them would contribute negligible co-located and the link between them would contribute negligible
performance, then these hosts can be considered equivalent (even if performance, then these hosts can be considered equivalent (even if
there is no physical link between them, this is a practical there is no physical link between them, this is a practical
concession). concession).
If the Destination and Source hosts of two adjoining paths have a If the Destination and Source hosts of two adjoining paths have a
link between them that contributes to the complete path performance, link between them that contributes to the complete path performance,
then the link and hosts constitutes another sub-path that is involved then the link and hosts constitute another sub-path that is involved
in the complete path, and should be characterized and included in the in the complete path, and should be characterized and included in the
composed metric. composed metric.
3.11. Sub-path Metrics 4.11. Sub-Path Metrics
A sub-path path metric is an element of the process to derive a A sub-path path metric is an element of the process to derive a
Composite metric, quantifying some aspect of the performance a composed metric, quantifying some aspect of the performance of a
particular sub-path from its Source to Destination. particular sub-path from its Source to Destination.
4. Description of Metric Types 5. Description of Metric Types
This section defines the various classes of Composition. There are This section defines the various classes of composition. There are
two classes more accurately described as aggregation over time and two classes more accurately described as aggregation over time and
space, and the third involves concatenation in space. space, and the third involves concatenation in space.
4.1. Temporal Aggregation Description 5.1. Temporal Aggregation Description
Aggregation in time is defined as the composition of metrics with the Aggregation in time is defined as the composition of metrics with the
same type and scope obtained in different time instants or time same type and scope obtained in different time instants or time
windows. For example, starting from a time series of the windows. For example, starting from a time series of the
measurements of maximum and minimum One-Way Delay on a certain measurements of maximum and minimum one-way delay (OWD) on a certain
network path obtained over 5-minute intervals, we obtain a time network path obtained over 5-minute intervals, we obtain a time
series measurement with a coarser resolution (60 minutes) by taking series measurement with a coarser resolution (60 minutes) by taking
the maximum of 12 consecutive 5-minute maxima and the minimum of 12 the maximum of 12 consecutive 5-minute maxima and the minimum of 12
consecutive 5-minute minima. consecutive 5-minute minima.
The main reason for doing time aggregation is to reduce the amount of The main reason for doing time aggregation is to reduce the amount of
data that has to be stored, and make the visualization/spotting of data that has to be stored, and make the visualization/spotting of
regular cycles and/or growing or decreasing trends easier. Another regular cycles and/or growing or decreasing trends easier. Another
useful application is to detect anomalies or abnormal changes in the useful application is to detect anomalies or abnormal changes in the
network characteristics. network characteristics.
In RFC 2330, the term "temporal composition" is introduced and In RFC 2330, the term "temporal composition" is introduced and
differs from temporal aggregation in that it refers to methodologies differs from temporal aggregation in that it refers to methodologies
to predict future metrics on the basis of past observations (of the to predict future metrics on the basis of past observations (of the
same metrics), exploiting the time correlation that certain metrics same metrics), exploiting the time correlation that certain metrics
can exhibit. We do not consider this type of composition here. can exhibit. We do not consider this type of composition here.
4.2. Spatial Aggregation Description 5.2. Spatial Aggregation Description
Aggregation in space is defined as the combination of metrics of the Aggregation in space is defined as the combination of metrics of the
same type and different scope, in order to estimate the overall same type and different scope, in order to estimate the overall
performance of a larger network. This combination may involve performance of a larger network. This combination may involve
weighing the contributions of the input metrics. weighing the contributions of the input metrics.
Suppose we want to compose the average One-Way-Delay (OWD) Suppose we want to compose the average one-way delay (OWD)
experienced by flows traversing all the Origin-Destination (OD) pairs experienced by flows traversing all the origin-destination (OD) pairs
of a network (where the inputs are already metric "statistics"). of a network (where the inputs are already metric "statistics").
Since we wish to include the effect of the traffic matrix on the Since we wish to include the effect of the traffic matrix on the
result, it makes sense to weight each metric according to the traffic result, it makes sense to weight each metric according to the traffic
carried on the corresponding OD pair: carried on the corresponding OD pair:
OWD_sum=f1*OWD_1+f2*OWD_2+...+fn*OWD_n OWD_sum=f1*OWD_1+f2*OWD_2+...+fn*OWD_n
where fi=load_OD_i/total_load. where fi=load_OD_i/total_load.
A simple average OWD across all network OD pairs would not use the A simple average OWD across all network OD pairs would not use the
traffic weighting. traffic weighting.
Another example metric that is "aggregated in space", is the maximum Another example metric that is "aggregated in space" is the maximum
edge-to-edge delay across a single network. Assume that a Service edge-to-edge delay across a single network. Assume that a Service
Provider wants to advertise the maximum delay that transit traffic Provider wants to advertise the maximum delay that transit traffic
will experience while passing through his/her network. There can be will experience while passing through his/her network. There can be
multiple edge-to-edge paths across a network, and the Service multiple edge-to-edge paths across a network, and the Service
Provider chooses either to publish a list of delays (each Provider chooses either to publish a list of delays (each
corresponding to a specific edge-to-edge path), or publish a single corresponding to a specific edge-to-edge path), or publish a single
maximum value. The latter approach simplifies the publication of maximum value. The latter approach simplifies the publication of
measurement information, and may be sufficient for some purposes. measurement information, and may be sufficient for some purposes.
Similar operations can be provided to other metrics, e.g. "maximum Similar operations can be provided to other metrics, e.g., "maximum
edge-to-edge packet loss", etc. edge-to-edge packet loss", etc.
We suggest that space aggregation is generally useful to obtain a We suggest that space aggregation is generally useful to obtain a
summary view of the behaviour of large network portions, or in summary view of the behaviour of large network portions, or of
general of coarser aggregates. The metric collection time instant, coarser aggregates in general. The metric collection time instant,
i.e. the metric collection time window of measured metrics is not i.e., the metric collection time window of measured metrics, is not
considered in space aggregation. We assume that either it is considered in space aggregation. We assume that either it is
consistent for all the composed metrics, e.g. compose a set of consistent for all the composed metrics, e.g., compose a set of
average delays all referred to the same time window, or the time average delays all referring to the same time window, or the time
window of each composed metric does not affect aggregated metric. window of each composed metric does not affect the aggregated metric.
4.3. Spatial Composition Description 5.3. Spatial Composition Description
Concatenation in space is defined as the composition of metrics of Concatenation in space is defined as the composition of metrics of
same type and (ideally) different spatial scope, so that the same type with (ideally) different spatial scope, so that the
resulting metric is representative of what the metric would be if resulting metric is representative of what the metric would be if
obtained with a direct measurement over the sequence of the several obtained with a direct measurement over the sequence of the several
spatial scopes. An example is the sum of OWDs of different edge-to- spatial scopes. An example is the sum of mean OWDs of adjacent edge-
edge network's delays, where the intermediate edge points are close to-edge networks, where the intermediate edge points are close to
to each other or happen to be the same. In this way, we can for each other or happen to be the same. In this way, we can for example
example estimate OWD_AC starting from the knowledge of OWD_AB and estimate OWD_AC starting from the knowledge of OWD_AB and OWD_BC.
OWD_BC. Note that there may be small gaps in measurement coverage, Note that there may be small gaps in measurement coverage; likewise,
likewise there may be small overlaps (e.g., the link where test there may be small overlaps (e.g., the link where test equipment
equipment connects to the network). connects to the network).
One key difference from examples of aggregation in space is that all One key difference from examples of aggregation in space is that all
sub-paths contribute equally to the composed metric, independent of sub-paths contribute equally to the composed metric, independent of
the traffic load present. the traffic load present.
4.4. Help Metrics 5.4. Help Metrics
In practice there is often the need to compute a new metric using one In practice, there is often the need to compute a new metric using
or more metrics with the same spatial and time scope. For example, one or more metrics with the same spatial and time scope. For
the metric rtt_sample_variance may be computed from two different example, the metric rtt_sample_variance may be computed from two
metrics: the help metrics rtt_square_sum and the rtt_sum. The different metrics: the help metrics rtt_square_sum and the rtt_sum.
process of using help metrics is a simple calculation and not an
The process of using help metrics is a simple calculation and not an
aggregation or a concatenation, and will not be investigated further aggregation or a concatenation, and will not be investigated further
in this memo. in this memo.
4.5. Higher Order Composition 5.5. Higher-Order Composition
Composed metrics might themselves be subject to further steps of Composed metrics might themselves be subject to further steps of
composition or aggregation. An example would be the delay of a composition or aggregation. An example would be the delay of a
maximal path obtained through the spatial composition of several maximal path obtained through the spatial composition of several
composed delays for each Complete Path in the maximal path (obtained composed delays for each complete path in the maximal path (obtained
through spatial composition). All requirements for first order through spatial composition). All requirements for first-order
composition metrics apply to higher order composition. composition metrics apply to higher-order composition.
An example using temporal aggregation: twelve 5-minute metrics are An example using temporal aggregation: twelve 5-minute metrics are
aggregated to estimate the performance over an hour. The second step aggregated to estimate the performance over an hour. The second step
of aggregation would take 24 hourly metrics and estimate the of aggregation would take 24 hourly metrics and estimate the
performance over a day. performance over a day.
5. Requirements for Composed Metrics 6. Requirements for Composed Metrics
The definitions for all composed metrics MUST include sections to The definitions for all composed metrics MUST include sections to
treat the following topics. treat the following topics.
The description of each metric will clearly state: The description of each metric will clearly state:
1. the definition (and statistic, where appropriate); 1. the definition (and statistic, where appropriate);
2. the composition or aggregation relationship; 2. the composition or aggregation relationship;
3. the specific conjecture on which the relationship is based and 3. the specific conjecture on which the relationship is based and
assumptions of the statistical model of the process being assumptions of the statistical model of the process being
measured, if any (see [RFC2330] section 12); measured, if any (see [RFC2330], Section 12);
4. a justification of practical utility or usefulness for analysis 4. a justification of practical utility or usefulness for analysis
using the A-frame concepts; using the A-frame concepts;
5. one or more examples of how the conjecture could be incorrect and 5. one or more examples of how the conjecture could be incorrect and
lead to inaccuracy; lead to inaccuracy;
6. the information to be reported. 6. the information to be reported.
For each metric, the applicable circumstances will be defined, in For each metric, the applicable circumstances will be defined, in
terms of whether the composition or aggregation: terms of whether the composition or aggregation:
o Requires homogeneity of measurement methodologies, or can allow a o Requires homogeneity of measurement methodologies, or can allow a
degree of flexibility (e.g., active or passive methods produce the degree of flexibility (e.g., active or passive methods produce the
"same" metric). Also, the applicable sending streams will be "same" metric). Also, the applicable sending streams will be
specified, such as Poisson, Periodic, or both. specified, such as Poisson, Periodic, or both.
o Needs information or access that will only be available within an o Needs information or access that will only be available within an
operator's network, or is applicable to Inter-network composition. operator's network, or is applicable to inter-network composition.
o Requires precisely synchronized measurement time intervals in all o Requires precisely synchronized measurement time intervals in all
component metrics, or loosely synchronized, or no timing component metrics, or perhaps only loosely synchronized time
requirements. intervals, or has no timing requirements at all.
o Requires assumption of component metric independence w.r.t. the o Requires assumption of component metric independence with regard
metric being defined/composed, or other assumptions. to the metric being defined/composed, or other assumptions.
o Has known sources of inaccuracy/error, and identifies the sources. o Has known sources of inaccuracy/error and identifies the sources.
5.1. Note on IPR 6.1. Note on Intellectual Property Rights (IPR)
If one or more components of the composition process are encumbered If one or more components of the composition process are encumbered
by Intellectual Property Rights (IPR), then the resulting Composed by Intellectual Property Rights (IPR), then the resulting composed
Metrics may be encumbered as well. See BCP 79 [RFC3979] for IETF metrics may be encumbered as well. See BCP 79 [RFC3979] for IETF
policies on IPR disclosure. policies on IPR disclosure.
6. Guidelines for Defining Composed Metrics 7. Guidelines for Defining Composed Metrics
6.1. Ground Truth: Comparison with other IPPM Metrics 7.1. Ground Truth: Comparison with Other IPPM Metrics
Figure 1 illustrates the process to derive a metric using spatial Figure 1 illustrates the process to derive a metric using spatial
composition, and compares the composed metric to other IPPM metrics. composition, and compares the composed metric to other IPPM metrics.
Metrics <M1, M2, M3> describe the performance of sub-paths between Metrics <M1, M2, M3> describe the performance of sub-paths between
the Source and Destination of interest during time interval <T, Tf>. the Source and Destination of interest during time interval <T, Tf>.
These metrics are the inputs for a Composition Function that produces These metrics are the inputs for a composition function that produces
a Composed Metric. a composed metric.
Sub-Path Metrics Sub-Path Metrics
++ M1 ++ ++ M2 ++ ++ M3 ++ ++ M1 ++ ++ M2 ++ ++ M3 ++
Src ||.......|| ||.......|| ||.......|| Dst Src ||.......|| ||.......|| ||.......|| Dst
++ `. ++ ++ | ++ ++ .' ++ ++ `. ++ ++ | ++ ++ .' ++
`. | .-' `. | .-'
`-. | .' `-. | .'
`._..|.._.' `._..|.._.'
,-' `-. ,-' `-.
,' `. ,' `.
| Composition | | Composition |
\ Function ' \ Function '
`._ _,' `._ _,'
`--.....--' `--.....--'
| |
++ | ++ ++ | ++
Src ||...............................|| Dst Src ||...............................|| Dst
++ Composed Metric ++ ++ Composed Metric ++
++ Complete Path Metric ++ ++ Complete Path Metric ++
Src ||...............................|| Dst Src ||...............................|| Dst
++ ++ ++ ++
Spatial Metric Spatial Metric
++ S1 ++ S2 ++ S3 ++ ++ S1 ++ S2 ++ S3 ++
Src ||........||.........||..........|| Dst Src ||........||.........||..........|| Dst
++ ++ ++ ++ ++ ++ ++ ++
Figure 1: Comparison with other IPPM metrics Figure 1: Comparison with Other IPPM Metrics
The Composed Metric is an estimate of an actual metric collected over The composed metric is an estimate of an actual metric collected over
the complete Source to Destination path. We say that the Complete the complete Source-to-Destination path. We say that the complete
Path Metric represents the "Ground Truth" for the Composed Metric. path metric represents the ground truth for the composed metric. In
In other words, Composed Metrics seek to minimize error w.r.t. the other words, composed metrics seek to minimize error with regard to
Complete Path Metric. the complete path metric.
Further, we observe that a Spatial Metric [RFC5644] collected for Further, we observe that a spatial metric [RFC5644] collected for
packets traveling over the same set of sub-paths provide a basis for packets traveling over the same set of sub-paths provides a basis for
the Ground Truth of the individual Sub-Path metrics. We note that the ground truth of the individual sub-path metrics. We note that
mathematical operations may be necessary to isolate the performance mathematical operations may be necessary to isolate the performance
of each sub-path. of each sub-path.
Next, we consider multiparty metrics as defined in [RFC5644], and Next, we consider multiparty metrics (as defined in [RFC5644]) and
their spatial composition. Measurements to each of the Receivers their spatial composition. Measurements to each of the receivers
produce an element of the one-to-group metric. These elements can be produce an element of the one-to-group metric. These elements can be
composed from sub-path metrics and the composed metrics can be composed from sub-path metrics, and the composed metrics can be
combined to create a composed one-to-group metric. Figure 2 combined to create a composed one-to-group metric. Figure 2
illustrates this process. illustrates this process.
Sub-Path Metrics Sub-Path Metrics
++ M1 ++ ++ M2 ++ ++ M3 ++ ++ M1 ++ ++ M2 ++ ++ M3 ++
Src ||.......|| ||.......|| ||.......||Rcvr1 Src ||.......|| ||.......|| ||.......||Rcvr1
++ ++ ++`. ++ ++ ++ ++ ++ ++`. ++ ++ ++
`-. `-.
M4`.++ ++ M5 ++ M4`.++ ++ M5 ++
|| ||.......||Rcvr2 || ||.......||Rcvr2
skipping to change at page 14, line 33 skipping to change at page 14, line 33
`-. ++ ++ `-. ++ ++
`-||..........||Rcvr2 `-||..........||Rcvr2
++. ++ ++. ++
`-. `-.
`-. ++ `-. ++
`-.||Rcvr3 `-.||Rcvr3
++ ++
Figure 2: Composition of One-to-Group Metrics Figure 2: Composition of One-to-Group Metrics
Here, Sub-path Metrics M1, M2, and M3 are combined using a Here, sub-path metrics M1, M2, and M3 are combined using a
relationship to compose the metric applicable to the Src-Rcvr1 path. relationship to compose the metric applicable to the Src-Rcvr1 path.
Similarly, M1, M4, and M5 are used to compose the Src-Rcvr2 metric Similarly, M1, M4, and M5 are used to compose the Src-Rcvr2 metric
and M1, M4, and M6 compose the Src-Rcvr3 metric. and M1, M4, and M6 compose the Src-Rcvr3 metric.
The Composed One-to-Group Metric would list the Src-Rcvr metrics for The composed one-to-group metric would list the Src-Rcvr metrics for
each Receiver in the Group: each receiver in the group:
(Composed-Rcvr1, Composed-Rcvr2, Composed-Rcvr3) (Composed-Rcvr1, Composed-Rcvr2, Composed-Rcvr3)
The "Ground Truth" for this composed metric is of course an actual The ground truth for this composed metric is of course an actual one-
One-to-Group metric, where a single source packet has been measured to-group metric, where a single Source packet has been measured after
after traversing the Complete Paths to the various receivers. traversing the complete paths to the various receivers.
6.1.1. Ground Truth for Temporal Aggregation 7.1.1. Ground Truth for Temporal Aggregation
Temporal Aggregation involves measurements made over sub-intervals of Temporal aggregation involves measurements made over sub-intervals of
the complete time interval between the same Source and Destination. the complete time interval between the same Source and Destination.
Therefore, the "Ground Truth" is the metric measured over the desired Therefore, the ground truth is the metric measured over the desired
interval. interval.
6.1.2. Ground Truth for Spatial Aggregation 7.1.2. Ground Truth for Spatial Aggregation
Spatial Aggregation combines many measurements using a weighting Spatial aggregation combines many measurements using a weighting
function to provide the same emphasis as though the measurements were function to provide the same emphasis as though the measurements were
based on actual traffic, with inherent weights. Therefore, the based on actual traffic, with inherent weights. Therefore, the
"Ground Truth" is the metric measured on the actual traffic instead ground truth is the metric measured on the actual traffic instead of
of the active streams that sample the performance. the active streams that sample the performance.
6.2. Deviation from the Ground Truth 7.2. Deviation from the Ground Truth
A metric composition can deviate from the ground truth for several A metric composition can deviate from the ground truth for several
reasons. Two main aspects are: reasons. Two main aspects are:
o The propagation of the inaccuracies of the underlying measurements o The propagation of the inaccuracies of the underlying measurements
when composing the metric. As part of the composition function, when composing the metric. As part of the composition function,
errors of measurements might propagate. Where possible, this errors of measurements might propagate. Where possible, this
analysis should be made and included with the description of each analysis should be made and included with the description of each
metric. metric.
o A difference in scope. When concatenating many active measurement o A difference in scope. When concatenating many active measurement
results (from two or more sub-paths) to obtain the complete path results (from two or more sub-paths) to obtain the complete path
metric, the actual measured path will not be identical to the metric, the actual measured path will not be identical to the
complete path. It is in general difficult to quantify this complete path. It is in general difficult to quantify this
deviation with exactness, but a metric definition might identify deviation with exactness, but a metric definition might identify
guidelines for keeping the deviation as small as possible. guidelines for keeping the deviation as small as possible.
The description of the metric composition MUST include an section The description of the metric composition MUST include a section
identifying the deviation from the ground truth. identifying the deviation from the ground truth.
6.3. Incomplete Information 7.3. Incomplete Information
In practice, when measurements cannot be initiated on a sub-path or In practice, when measurements cannot be initiated on a sub-path or
during a particular measurement interval (and perhaps the measurement during a particular measurement interval (and perhaps the measurement
system gives up during the test interval), then there will not be a system gives up during the test interval), then there will not be a
value for the sub-path reported, and the result SHOULD be recorded as value for the sub-path reported, and the result SHOULD be recorded as
"undefined". "undefined".
6.4. Time Varying Metrics 7.4. Time-Varying Metrics
The measured values of many metrics depend on time-variant factors, The measured values of many metrics depend on time-variant factors,
such as the level of network traffic on the source to destination such as the level of network traffic on the Source-to-Destination
path. Traffic levels often exhibit diurnal (or daily) variation, but path. Traffic levels often exhibit diurnal (or daily) variation, but
a 24 hour measurement interval would obscure it. Temporal a 24-hour measurement interval would obscure it. Temporal
Aggregation of hourly results to estimate the daily metric would have aggregation of hourly results to estimate the daily metric would have
the same effect, and so the same cautions are warranted. the same effect, and so the same cautions are warranted.
Some metrics are predominantly* time-invariant, such as the actual Some metrics are predominantly* time-invariant, such as the actual
minimum one-way delay of fixed path, and therefore temporal minimum one-way delay of a fixed path, and therefore temporal
aggregation does not obscure the results as long as the path is aggregation does not obscure the results as long as the path is
stable. However, paths do vary, and sometimes on less predictable stable. However, paths do vary, and sometimes on less predictable
time intervals than traffic variations. (* Note - It is recognized time intervals than traffic variations. (* Note: It is recognized
that propagation delay on transmission facilities may have diurnal, that propagation delay on transmission facilities may have diurnal,
seasonal, and even longer-term variations.) seasonal, and even longer-term variations.)
7. IANA Considerations
This document makes no request of IANA.
Note to RFC Editor: this section may be removed on publication as an
RFC.
8. Security Considerations 8. Security Considerations
The security considerations that apply to any active measurement of The security considerations that apply to any active measurement of
live networks are relevant here as well. See [RFC4656], and live networks are relevant here as well. See [RFC4656] and
[RFC5357]. [RFC5357].
The exchange of sub-path measurements among network providers may be The exchange of sub-path measurements among network providers may be
a source of concern, and the information should be sufficiently a source of concern, and the information should be sufficiently
anonymized to avoid revealing unnecessary operational details (e.g., anonymized to avoid revealing unnecessary operational details (e.g.,
the network addresses of measurement devices). However, the schema the network addresses of measurement devices). However, the schema
for measurement exchange is beyond the scope of this memo, and likely for measurement exchange is beyond the scope of this memo and likely
to be covered by bilateral agreements for some time to come. to be covered by bilateral agreements for some time to come.
9. Acknowledgements 9. Acknowledgements
The authors would like to thank Maurizio Molina, Andy Van Maele, The authors would like to thank Maurizio Molina, Andy Van Maele,
Andreas Haneman, Igor Velimirovic, Andreas Solberg, Athanassios Andreas Haneman, Igor Velimirovic, Andreas Solberg, Athanassios
Liakopulos, David Schitz, Nicolas Simar and the Geant2 Project. We Liakopulos, David Schitz, Nicolas Simar, and the Geant2 Project. We
also acknowledge comments and suggestions from Phil Chimento, Emile also acknowledge comments and suggestions from Phil Chimento, Emile
Stephan, Lei Liang, Stephen Wolff, Reza Fardid, Loki Jorgenson, and Stephan, Lei Liang, Stephen Wolff, Reza Fardid, Loki Jorgenson, and
Alan Clark. Alan Clark.
10. References 10. References
10.1. Normative References 10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330, "Framework for IP Performance Metrics", RFC 2330,
May 1998. May 1998.
[RFC3979] Bradner, S., "Intellectual Property Rights in IETF [RFC3979] Bradner, S., Ed., "Intellectual Property Rights in IETF
Technology", BCP 79, RFC 3979, March 2005. Technology", BCP 79, RFC 3979, March 2005.
[RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and
Zekauskas, "A One-way Active Measurement Protocol M. Zekauskas, "A One-way Active Measurement Protocol
(OWAMP)", RFC 4656, September 2006. (OWAMP)", RFC 4656, September 2006.
[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and
Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", J. Babiarz, "A Two-Way Active Measurement Protocol
RFC 5357, October 2008. (TWAMP)", RFC 5357, October 2008.
10.2. Informative References 10.2. Informative References
[RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
Delay Metric for IPPM", RFC 2679, September 1999. Delay Metric for IPPM", RFC 2679, September 1999.
[RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
Packet Loss Metric for IPPM", RFC 2680, September 1999. Packet Loss Metric for IPPM", RFC 2680, September 1999.
[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip
Delay Metric for IPPM", RFC 2681, September 1999. Delay Metric for IPPM", RFC 2681, September 1999.
[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay
Metric for IP Performance Metrics (IPPM)", RFC 3393, Variation Metric for IP Performance Metrics (IPPM)",
November 2002. RFC 3393, November 2002.
[RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, [RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov,
S., and J. Perser, "Packet Reordering Metrics", RFC 4737, S., and J. Perser, "Packet Reordering Metrics", RFC 4737,
November 2006. November 2006.
[RFC5481] Morton, A. and B. Claise, "Packet Delay Variation [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation
Applicability Statement", RFC 5481, March 2009. Applicability Statement", RFC 5481, March 2009.
[RFC5560] Uijterwaal, H., "A One-Way Packet Duplication Metric", [RFC5560] Uijterwaal, H., "A One-Way Packet Duplication Metric",
RFC 5560, May 2009. RFC 5560, May 2009.
[RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance [RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance
Metrics (IPPM): Spatial and Multicast", RFC 5644, Metrics (IPPM): Spatial and Multicast", RFC 5644,
October 2009. October 2009.
[Y.1540] ITU-T Recommendation Y.1540, "Internet protocol data [Y.1540] ITU-T Recommendation Y.1540, "Internet protocol data
communication service - IP packet transfer and communication service - IP packet transfer and
availability performance parameters", December 2007. availability performance parameters", November 2007.
Authors' Addresses Authors' Addresses
Al Morton (editor) Al Morton (editor)
AT&T Labs AT&T Labs
200 Laurel Avenue South 200 Laurel Avenue South
Middletown,, NJ 07748 Middletown, NJ 07748
USA USA
Phone: +1 732 420 1571 Phone: +1 732 420 1571
Fax: +1 732 368 1192 Fax: +1 732 368 1192
Email: acmorton@att.com EMail: acmorton@att.com
URI: http://home.comcast.net/~acmacm/ URI: http://home.comcast.net/~acmacm/
Steven Van den Berghe (editor) Steven Van den Berghe (editor)
Alcatel-Lucent Alcatel-Lucent
Copernicuslaan 50 Copernicuslaan 50
Antwerp 2018 Antwerp 2018
Belgium Belgium
Phone: +32 3 240 3983 Phone: +32 3 240 3983
Email: steven.van_den_berghe@alcatel-lucent.com EMail: steven.van_den_berghe@alcatel-lucent.com
URI:
 End of changes. 125 change blocks. 
306 lines changed or deleted 291 lines changed or added

This html diff was produced by rfcdiff 1.38. The latest version is available from http://tools.ietf.org/tools/rfcdiff/