--- 1/draft-ietf-ippm-framework-compagg-01.txt 2006-10-24 22:12:41.000000000 +0200 +++ 2/draft-ietf-ippm-framework-compagg-02.txt 2006-10-24 22:12:41.000000000 +0200 @@ -1,16 +1,16 @@ Network Working Group A. Morton, Ed. Internet-Draft AT&T Labs -Expires: December 26, 2006 S. Van den Berghe, Ed. - Ghent University - IBBT - June 24, 2006 +Intended status: Informational S. Van den Berghe, Ed. +Expires: April 25, 2007 Ghent University - IBBT + October 22, 2006 Framework for Metric Composition draft-ietf-ippm-framework-compagg-01 Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. @@ -24,21 +24,21 @@ and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. - This Internet-Draft will expire on December 26, 2006. + This Internet-Draft will expire on April 25, 2007. Copyright Notice Copyright (C) The Internet Society (2006). Abstract This memo describes a framework for composing and aggregating metrics (both in time and in space) defined by RFC 2330 and developed by the IPPM working group. The framework describes the generic composition @@ -54,42 +54,54 @@ Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.1. Reducing Measurement Overhead . . . . . . . . . . . . 3 1.1.2. Measurement Re-use . . . . . . . . . . . . . . . . . . 4 1.1.3. Data Reduction and Consolidation . . . . . . . . . . . 4 1.1.4. Implications on Measurement Design and Reporting . . . 5 2. Purpose and Scope . . . . . . . . . . . . . . . . . . . . . . 5 - 3. Description of Metric Types . . . . . . . . . . . . . . . . . 5 - 3.1. Temporal Aggregation Description . . . . . . . . . . . . . 5 - 3.2. Spatial Aggregation Description . . . . . . . . . . . . . 6 - 3.3. Spatial Composition Description . . . . . . . . . . . . . 7 - 3.4. Help Metrics . . . . . . . . . . . . . . . . . . . . . . . 7 - 3.5. Higher Order Composition . . . . . . . . . . . . . . . . . 7 - 4. Requirements for Composed Metrics . . . . . . . . . . . . . . 7 - 5. Guidelines for Defining Composed Metrics . . . . . . . . . . . 9 - 5.1. Ground Truth: Comparison with other IPPM Metrics . . . . . 9 - 5.2. Deviation from the Ground Truth . . . . . . . . . . . . . 11 - 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 - 7. Security Considerations . . . . . . . . . . . . . . . . . . . 12 - 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12 - 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 12 - 9.1. Normative References . . . . . . . . . . . . . . . . . . . 12 - 9.2. Informative References . . . . . . . . . . . . . . . . . . 13 + 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5 + 3.1. Measurement Point . . . . . . . . . . . . . . . . . . . . 5 + 3.2. Complete path . . . . . . . . . . . . . . . . . . . . . . 6 + 3.3. Complete path metric . . . . . . . . . . . . . . . . . . . 6 + 3.4. Composed Metric . . . . . . . . . . . . . . . . . . . . . 6 + 3.5. Composition Function . . . . . . . . . . . . . . . . . . . 6 + 3.6. Ground Truth . . . . . . . . . . . . . . . . . . . . . . . 6 + 3.7. Sub-interval . . . . . . . . . . . . . . . . . . . . . . . 6 + 3.8. Sub-path . . . . . . . . . . . . . . . . . . . . . . . . . 6 + 3.9. Sub-path metrics . . . . . . . . . . . . . . . . . . . . . 6 + 4. Description of Metric Types . . . . . . . . . . . . . . . . . 7 + 4.1. Temporal Aggregation Description . . . . . . . . . . . . . 7 + 4.2. Spatial Aggregation Description . . . . . . . . . . . . . 7 + 4.3. Spatial Composition Description . . . . . . . . . . . . . 8 + 4.4. Help Metrics . . . . . . . . . . . . . . . . . . . . . . . 8 + 4.5. Higher Order Composition . . . . . . . . . . . . . . . . . 9 + 5. Requirements for Composed Metrics . . . . . . . . . . . . . . 9 + 6. Guidelines for Defining Composed Metrics . . . . . . . . . . . 10 + 6.1. Ground Truth: Comparison with other IPPM Metrics . . . . . 10 + 6.1.1. Ground Truth for Temporal Aggregation . . . . . . . . 12 + 6.1.2. Ground Truth for Spatial Aggregation . . . . . . . . . 13 + 6.2. Deviation from the Ground Truth . . . . . . . . . . . . . 13 + 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 + 8. Security Considerations . . . . . . . . . . . . . . . . . . . 13 + 9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 13 + 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 14 + 10.1. Normative References . . . . . . . . . . . . . . . . . . . 14 + 10.2. Informative References . . . . . . . . . . . . . . . . . . 14 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 14 - Intellectual Property and Copyright Statements . . . . . . . . . . 15 + Intellectual Property and Copyright Statements . . . . . . . . . . 16 1. Introduction - The IPPM framework RFC 2330 [RFC2330] describes two forms of metric + The IPPM framework [RFC2330] describes two forms of metric composition, spatial and temporal. Also, the text suggests that the concepts of the analytical framework (or A-frame) would help to develop useful relationships to derive the composed metrics from real metrics. The effectiveness of composed metrics is dependent on their usefulness in analysis and applicability to practical measurement circumstances. This memo expands on the notion of composition, and provides a detailed framework for several classes of metrics that were mentioned in the original IPPM framework. The classes include temporal @@ -98,21 +110,21 @@ 1.1. Motivation Network operators have deployed measurement systems to serve many purposes, including performance monitoring, maintenance support, network engineering, and customer reporting. The collection of elementary measurements alone is not enough to understand a network's behaviour. In general, measurements need to be post-processed to present the most relevant information for each purpose. The first step is often a process of "composition" of single measurements or measurement sets into other forms. Composition and aggregation - present several more post-processing opportunites to the network + present several more post-processing opportunities to the network operator, and we describe the key motivations below. 1.1.1. Reducing Measurement Overhead A network's measurement possibilities scale upward with the square of the number of nodes. But each measurement implies overhead, in terms of the storage for the results, the traffic on the network (assuming active methods), and the OA&M for the measurement system itself. In a large network, it is impossible to perform measurements from each node to all others. @@ -195,30 +207,92 @@ from primary metrics using a deterministic function. Key information about each metric, such as its assumptions under which the relationship holds, and possible sources of error/circumstances where the composition may fail, are included. At this time, the scope of effort is limited to the metrics for packet loss, delay, and delay variation. Composition of packet reordering metrics is considered a research topic, and beyond the scope at the time this memo was prepared. - This memo will retain the terminology of the IPPM Framework as much - as possible, but will extend the terminology when necessary. + This memo will retain the terminology of the IPPM Framework + [RFC2330]as much as possible, but will extend the terminology when + necessary. It is assumed that the reader is familiar with the + concepts introduced in [RFC2330], as they will not be repeated here. -3. Description of Metric Types +3. Terminology + + This section defines the terminology applicable to the processes of + Metric Composition and Aggregation. + +3.1. Measurement Point + + The logical or physical location where packet observations are made. + The term Measurement Point is synonymous with the term "observation + position" used in [RFC2330] when describing the notion of wire time. + A measurement point may be at the boundary between a host and an + adjacent link (physical), or it may be within a host (logical) that + performs measurements where the difference between host time and wire + time is understood. + +3.2. Complete path + + The complete path is the true path that a packet would follow as it + traverses from the packet's Source to its Destination. + +3.3. Complete path metric + + The complete path metric is the Source to Destination metric that a + composed metric is estimating. A complete path metric represents the + ground-truth for a composed metric. + +3.4. Composed Metric + + A composed metric is derived from other metrics principally by + applying a composition function. + +3.5. Composition Function + + A composition function is a deterministic process applied to Sub-path + metrics to derive another metric (such as a Composed metric). + +3.6. Ground Truth + + As applied here, the notion of ground truth is defined as the actual + performance of a network entity over some time interval. The ground + truth is the (unavailable) measurement that a composed metric seeks + to estimate. + +3.7. Sub-interval + + A Sub-interval is a time interval that is included in another + interval. + +3.8. Sub-path + + A Sub-path is a portion of the complete path where at least the Sub- + path Source and Destination hosts are constituents of the complete + path. We say that this sub-path is "involved" in the complete path. + +3.9. Sub-path metrics + + A sub-path path metric is an element of the process to derive a + Composite metric, quantifying some aspect of the performance a + particular sub-path from its Source to Destination. + +4. Description of Metric Types This section defines the various classes of Composition. There are two classes more accurately referred to as aggregation over time and space, and the third is simply composition in space. -3.1. Temporal Aggregation Description +4.1. Temporal Aggregation Description Aggregation in time is defined as the composition of metrics with the same type and scope obtained in different time instants or time windows. For example, starting from a time series of One-Way Delay measurements on a certain network path obtained in 5-minute periods and averaging groups of 12 consecutive values, we obtain a time series measurement with a coarser resolution (60 minutes). The main reason for doing time aggregation is to reduce the amount of data that has to be stored, and make the visualization/spotting of regular cycles and/or growing or decreasing trends easier. Another useful @@ -227,21 +301,21 @@ In RFC 2330, the term "temporal composition" is introduced and differs from temporal aggregation in that it refers to methodologies to predict future metrics on the basis of past observations, exploiting the time correlation that certain metrics can exhibit. We do not consider this type of composition here. >>>>>>>>Comment: Why no forecasting? This was apparently a limit on the Geant2 project, but may not apply here. -3.2. Spatial Aggregation Description +4.2. Spatial Aggregation Description Aggregation in space is defined as the combination of metrics of the same type and different scope, in order to estimate the overall performance of a larger domain. This combination may involve weighing the contributions of the input metrics. Suppose we want to compose the average One-Way-Delay (OWD) experienced by flows traversing all the Origin-Destination (OD) pairs of a network domain (where the inputs are already metric "statistics"). Since we wish to include the effect of the traffic @@ -269,61 +343,62 @@ We suggest that space aggregation is generally useful to obtain a summary view of the behaviour of large network portions, or in general of coarser aggregates. The metric collection time instant, i.e. the metric collection time window of measured metrics is not considered in space aggregation. We assume that either it is consistent for all the composed metrics, e.g. compose a set of average delays all referred to the same time window, or the time window of each composed metric does not affect aggregated metric. -3.3. Spatial Composition Description +4.3. Spatial Composition Description Concatenation in space is defined as the composition of metrics of same type and (ideally) different spatial scope, so that the resulting metric is representative of what the metric would be if obtained with a direct measurement over the sequence of the several spatial scopes. An example is the sum of OWDs of different edge-to- edge domain's delays, where the intermediate edge points are close to each other or happen to be the same. In this way, we can for example estimate OWD_AC starting from the knowledge of OWD_AB and OWD_BC. Note that there may be small gaps in measurement coverage, likewise there may be small overlaps (e.g., the link where test equipment connects to the network). One key difference from examples of aggregation in space is that all sub-paths contribute equally to the composed metric, independent of the traffic load present. -3.4. Help Metrics +4.4. Help Metrics Finally, note that in practice there is often the need of extracting a new metric making some computation over one or more metrics with the same spatial and time scope. For example, the composed metric rtt_sample_variance may be composed from two different metrics: the help metric rtt_square_sum and the statistical metric rtt_sum. This operation is however more a simple calculation and not an aggregation or a concatenation, and we'll not investigate it further in this memo. -3.5. Higher Order Composition +4.5. Higher Order Composition Composed metrics might themselves be subject to further steps of composition or aggregation. An example would be a the delay of a maximal domain obtained through the spatial composition of several composed end-to-end delays (obtained through spatial composition). All requirements for first order composition metrics apply to higher order composition. >>>>> Comment Response: are more examples needed here? -4. Requirements for Composed Metrics +5. Requirements for Composed Metrics + The definitions for all composed metrics MUST include sections to treat the following topics. The description of each metric will clearly state: 1. the definition (and statistic, where appropriate); 2. the composition or aggregation relationship; 3. the specific conjecture on which the relationship is based; @@ -365,23 +439,23 @@ o Requires precisely synchronized measurement time intervals in all component metrics, or loosely synchronized, or no timing requirements. o Requires assumption of component metric independence w.r.t. the metric being defined/composed, or other assumptions. o Has known sources of inaccuracy/error, and identifies the sources. -5. Guidelines for Defining Composed Metrics +6. Guidelines for Defining Composed Metrics -5.1. Ground Truth: Comparison with other IPPM Metrics +6.1. Ground Truth: Comparison with other IPPM Metrics Figure 1 illustrates the process to derive a metric using spatial composition, and compares the composed metric to other IPPM metrics. Metrics describe the performance of sub-paths between the Source and Destination of interest during time interval . These metrics are the inputs for a Composition Function that produces a Composed Metric. Sub-Path Metrics @@ -402,21 +476,22 @@ Src ||...............................|| Dst ++ Composed Metric ++ ++ Complete Path Metric ++ Src ||...............................|| Dst ++ ++ Spatial Metric ++ S1 ++ S2 ++ S3 ++ Src ||........||.........||..........|| Dst ++ ++ ++ ++ - Figure 1 Comparison with other IPPM metrics + + Figure 1: Comparison with other IPPM metrics The Composed Metric is an estimate of an actual metric collected over the complete Source to Destination path. We say that the Complete Path Metric represents the "Ground Truth" for the Composed Metric. In other words, Composed Metrics seek to minimize error w.r.t. the Complete Path Metric. Further, we observe that a Spatial Metric I-D.ietf-ippm-multimetrics [I-D.ietf-ippm-multimetrics]collected for packets traveling over the same set of sub-paths provide a basis for the Ground Truth of the @@ -448,91 +523,114 @@ Src ||........||.........||..........||Rcvr1 ++ ++. ++ ++ `-. `-. ++ ++ `-||..........||Rcvr2 ++. ++ `-. `-. ++ `-.||Rcvr3 ++ - Figure 2 Composition of One-to-Group Metrics + + Figure 2: Composition of One-to-Group Metrics Here, Sub-path Metrics M1, M2, and M3 are combined using a relationship to compose the metric applicable to the Src-Rcvr1 path. Similarly, M1, M4, and M5 are used to compose the Src-Rcvr2 metric and M1, M4, and M6 compose the Src-Rcvr3 metric. The Composed One-to-Group Metric would list the Src-Rcvr metrics for each Receiver in the Group: (Composed-Rcvr1, Composed-Rcvr2, Composed-Rcvr3) The "Ground Truth" for this composed metric is of course an actual One-to-Group metric, where a single source packet has been measured after traversing the Complete Paths to the various receivers. -5.2. Deviation from the Ground Truth +6.1.1. Ground Truth for Temporal Aggregation + + Temporal Aggregation involves measurements made over sub-intervals of + the desired test interval between the same Source and Destination. + Therefore, the "Ground Truth" is the metric measured over the desired + interval. + +6.1.2. Ground Truth for Spatial Aggregation + + Spatial Aggregation combines many measurements using a weighting + function to provide the same emphasis as though the measurements were + based on actual traffic, with inherent weights. Therefore, the + "Ground Truth" is the metric measured on the actual traffic instead + of the active streams that sample the performance. + +6.2. Deviation from the Ground Truth A metric composition can deviate from the ground truth for several reasons. Two main aspects are: o The propagation of the inaccuracies of the underlying measurements when composing the metric. As part of the composition function, errors of measurements might propagate. Where possible, this analysis should be made and included with the description of each metric. o A difference in scope. When concatenating hop-by-hop active measurement results to obtain the end-to-end metric, the actual measured path will not be identical to the end-to-end path. It is in general difficult to quantify this deviation, but a metric definition might identify guidelines for keeping the deviation as small as possible. The description of the metric composition MUST include an section identifying the deviation from the ground truth. -6. IANA Considerations +7. IANA Considerations This document makes no request of IANA. Note to RFC Editor: this section may be removed on publication as an RFC. -7. Security Considerations +8. Security Considerations -8. Acknowledgements + The security considerations that apply to any active measurement of + live networks are relevant here as well. See [RFC4656]. + +9. Acknowledgements The authors would like to thank Maurizio Molina, Andy Van Maele, Andreas Haneman, Igor Velimirovic, Andreas Solberg, Athanassios Liakopulos, David Schitz, Nicolas Simar and the Geant2 Project. We also acknowledge comments and suggestions from Phil Chimento, Emile Stephan and Lei Liang. -9. References +10. References -9.1. Normative References +10.1. Normative References [I-D.ietf-ippm-multimetrics] Stephan, E., "IP Performance Metrics (IPPM) for spatial - and multicast", draft-ietf-ippm-multimetrics-00 (work in - progress), January 2006. + and multicast", draft-ietf-ippm-multimetrics-01 (work in + progress), July 2006. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, May 1998. -9.2. Informative References + [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. + Zekauskas, "A One-way Active Measurement Protocol + (OWAMP)", RFC 4656, September 2006. + +10.2. Informative References Authors' Addresses Al Morton (editor) AT&T Labs 200 Laurel Avenue South Middletown,, NJ 07748 USA Phone: +1 732 420 1571 @@ -543,21 +640,37 @@ Steven Van den Berghe (editor) Ghent University - IBBT G. Crommenlaan 8 bus 201 Gent 9050 Belgium Phone: +32 9 331 49 73 Email: steven.vandenberghe@intec.ugent.be URI: http://www.ibcn.intec.ugent.be -Intellectual Property Statement +Full Copyright Statement + + Copyright (C) The Internet Society (2006). + + This document is subject to the rights, licenses and restrictions + contained in BCP 78, and except as set forth therein, the authors + retain all their rights. + + This document and the information contained herein are provided on an + "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS + OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET + ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, + INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE + INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED + WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. + +Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. @@ -567,30 +680,14 @@ such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. -Disclaimer of Validity - - This document and the information contained herein are provided on an - "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS - OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET - ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, - INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE - INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED - WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. - -Copyright Statement - - Copyright (C) The Internet Society (2006). This document is subject - to the rights, licenses and restrictions contained in BCP 78, and - except as set forth therein, the authors retain all their rights. - Acknowledgment - Funding for the RFC Editor function is currently provided by the - Internet Society. + Funding for the RFC Editor function is provided by the IETF + Administrative Support Activity (IASA).