--- 1/draft-ietf-ippm-reporting-02.txt 2009-03-09 22:12:10.000000000 +0100 +++ 2/draft-ietf-ippm-reporting-03.txt 2009-03-09 22:12:10.000000000 +0100 @@ -1,124 +1,144 @@ Network Working Group S. Shalunov Internet-Draft Intended status: Standards Track M. Swany -Expires: January 15, 2009 University of Delaware - July 14, 2008 +Expires: September 10, 2009 University of Delaware + March 9, 2009 Reporting IP Performance Metrics to Users - draft-ietf-ippm-reporting-02.txt + draft-ietf-ippm-reporting-03.txt Status of this Memo - By submitting this Internet-Draft, each author represents that any - applicable patent or other IPR claims of which he or she is aware - have been or will be disclosed, and any of which he or she becomes - aware will be disclosed, in accordance with Section 6 of BCP 79. + This Internet-Draft is submitted to IETF in full conformance with the + provisions of BCP 78 and BCP 79. This document may contain material + from IETF Documents or IETF Contributions published or made publicly + available before November 10, 2008. The person(s) controlling the + copyright in some of this material may not have granted the IETF + Trust the right to allow modifications of such material outside the + IETF Standards Process. Without obtaining an adequate license from + the person(s) controlling the copyright in such materials, this + document may not be modified outside the IETF Standards Process, and + derivative works of it may not be created outside the IETF Standards + Process, except to format it for publication as an RFC or to + translate it into languages other than English. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. - This Internet-Draft will expire on January 15, 2009. + This Internet-Draft will expire on September 10, 2009. + +Copyright Notice + + Copyright (c) 2009 IETF Trust and the persons identified as the + document authors. All rights reserved. + + This document is subject to BCP 78 and the IETF Trust's Legal + Provisions Relating to IETF Documents in effect on the date of + publication of this document (http://trustee.ietf.org/license-info). + Please review these documents carefully, as they describe your rights + and restrictions with respect to this document. Abstract The aim of this document is to define a small set of metrics that are robust, easy to understand, orthogonal, relevant, and easy to compute. The IPPM WG has defined a large number of richly parameterized metrics because network measurement has many purposes. Often, the ultimate purpose is to report a concise set of metrics describing a network's state to an end user. It is for this purpose that the present set of metrics is defined. Table of Contents - 1. Requirements Notation . . . . . . . . . . . . . . . . . . . . 3 - 2. Goals and Motivation . . . . . . . . . . . . . . . . . . . . . 4 - 3. Applicability for Long-Term Measurements . . . . . . . . . . . 6 - 4. Reportable Metrics Set . . . . . . . . . . . . . . . . . . . . 7 - 4.1. Median Delay . . . . . . . . . . . . . . . . . . . . . . . 7 - 4.2. Loss Ratio . . . . . . . . . . . . . . . . . . . . . . . . 7 - 4.3. Delay Spread . . . . . . . . . . . . . . . . . . . . . . . 7 - 4.4. Duplication . . . . . . . . . . . . . . . . . . . . . . . 7 - 4.5. Reordering . . . . . . . . . . . . . . . . . . . . . . . . 8 - 5. Sample Source . . . . . . . . . . . . . . . . . . . . . . . . 9 - 5.1. One-Way Active Measurement . . . . . . . . . . . . . . . . 9 - 5.2. Round-Trip Active Measurement . . . . . . . . . . . . . . 10 - 5.3. Passive Measurement . . . . . . . . . . . . . . . . . . . 10 - 6. Security Considerations . . . . . . . . . . . . . . . . . . . 11 - 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 12 - 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 - 9. Internationalization Considerations . . . . . . . . . . . . . 14 - 10. Normative References . . . . . . . . . . . . . . . . . . . . . 15 - Appendix A. Sample Source Code for Computing the Metrics . . . . 16 - Appendix B. Example Report . . . . . . . . . . . . . . . . . . . 40 - Appendix C. TODO . . . . . . . . . . . . . . . . . . . . . . . . 41 - Appendix D. Revision History . . . . . . . . . . . . . . . . . . 42 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 43 - Intellectual Property and Copyright Statements . . . . . . . . . . 44 + 1. Requirements Notation . . . . . . . . . . . . . . . . . . . . 4 + 2. Goals and Motivation . . . . . . . . . . . . . . . . . . . . . 5 + 3. Applicability for Long-Term Measurements . . . . . . . . . . . 7 + 4. Reportable Metrics Set . . . . . . . . . . . . . . . . . . . . 8 + 4.1. Median Delay . . . . . . . . . . . . . . . . . . . . . . . 8 + 4.2. Loss Ratio . . . . . . . . . . . . . . . . . . . . . . . . 8 + 4.3. Delay Spread . . . . . . . . . . . . . . . . . . . . . . . 8 + 4.4. Duplication . . . . . . . . . . . . . . . . . . . . . . . 9 + 4.5. Reordering . . . . . . . . . . . . . . . . . . . . . . . . 9 + 5. Sample Source . . . . . . . . . . . . . . . . . . . . . . . . 10 + 5.1. One-Way Active Measurement . . . . . . . . . . . . . . . . 10 + 5.2. Round-Trip Active Measurement . . . . . . . . . . . . . . 11 + 5.3. Passive Measurement . . . . . . . . . . . . . . . . . . . 11 + 6. Security Considerations . . . . . . . . . . . . . . . . . . . 12 + 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 13 + 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 + 9. Internationalization Considerations . . . . . . . . . . . . . 15 + 10. Normative References . . . . . . . . . . . . . . . . . . . . . 16 + Appendix A. Sample Source Code for Computing the Metrics . . . . 17 + Appendix B. Example Report . . . . . . . . . . . . . . . . . . . 41 + Appendix C. TODO . . . . . . . . . . . . . . . . . . . . . . . . 42 + Appendix D. Revision History . . . . . . . . . . . . . . . . . . 43 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 44 1. Requirements Notation The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 2. Goals and Motivation The IPPM working group has defined many richly parameterized performance metrics with a number of variants (one-way delay, one-way loss, delay variation, reordering, etc.) and a protocol for obtaining the measurement data needed to compute these metrics (OWAMP). It would be beneficial to define a standard way to report a set of performance metrics to end users. Parameterization might be acceptable in such a set, but there must still be defaults for everything. It is especially important to get these defaults right. Such a set would enable different tools to produce results that can be compared against each other. - Existing tools already report statistic about the network. This is + Existing tools already report statistics about the network. This is done for varying reasons: network testing tools, such as the ping program available in UNIX-derived operating systems as well as in Microsoft Windows, report statistics with no knowledge of why the user is running the program; networked games might report statistics of the network connection to the server so users can better understand why they get the results they get (e.g., if something is slow, is this because of the network or the CPU?), so they can compare their statistics to those of others ("you're not lagged any more than I am") or perhaps so that users can decide whether they need to upgrade the connection to their home; IP telephony hardware and software might report the statistics for similar reasons. While - existing tools report statistics all right, the particular set of - metrics they choose is ad hoc; some metrics are not statistically - robust, some are not relevant, and some are not easy to understand; - more important than specific shortcomings, however, is the - incompatibility: even if the sets of metrics were perfect, they would - still be all different, and, therefore, metrics reported by different - tools would not be comparable. + existing tools report statistics, the particular set of metrics they + choose is ad hoc; some metrics are not statistically robust, some are + not relevant, and some are not easy to understand; more important + than specific shortcomings, however, is the incompatibility: even if + the sets of metrics were perfect, they would still be all different, + and, therefore, metrics reported by different tools would not be + comparable. The set of metrics of this document is meant for human consumption. - It must therefore be small. Anything greater than half-dozen numbers - is certainly too confusing. + It must therefore be small. A screen full of numbers is likely to be + too confusing. Restricting output to a single number would likewise + not be descriptive enough. (Would you pick loss? delay? throughput? + something else?) In this document we aim for a "handful" of numbers. Each of the metrics must be statistically robust. Intuitively, this means that having a small number of bad data points and a small amount of noise must not dramatically change the metric. Each metric in the set must have, qualitatively, an immediate intuitive meaning that has to be obvious for an advanced end user without consulting documentation (that is, it has to be clear what rough meaning, intuitively, the larger values of a given metric have). @@ -170,50 +190,68 @@ Each of the above is represented by a single numeric quantity, computed as described below. 4.1. Median Delay The reported median delay is the median of all delays in the sample. When a packet is lost, its delay is to be considered +infinity for the purposes of this computation; therefore, if more than half of all packets are lost, the delay is +infinity. + For more information, refer to section 5.2 (Type-P-One-way-Delay- + Median) of RFC 2679 [RFC2679] (A One-way Delay Metric for IPPM), and + supporting text. + 4.2. Loss Ratio Loss Ratio is the fraction, expressed as a percentile, of packets that did not arrive intact within a given number of seconds (the timeout value) after being sent. Since this set of metrics often has to be reported to a waiting human user, the default timeout value should be small. By default, 2 seconds MUST be the timeout value. Use of a different timeout value MUST be reported. + For more information, refer to Section 4.1 (Type-P-One-way-Packet- + Loss-Average) of RFC 2680 [RFC2680] (A One-way Packet Loss Metric for + IPPM). The Loss Ratio is 100*Type-P-One-way-Packet-Loss-Average. + 4.3. Delay Spread Delay spread is the interquartile spread of observed delays. This is a metric to represent what is commonly referred to as "jitter". Delay spread is reported as the difference of the 75th and 25th percentiles of delay. When both percentiles are +infinity, the value of delay spread is undefined. Therefore, if less than 25% of packets are lost, it is defined and finite; if between 75% and 25% of packets are lost, it is +infinity; finally, if more than 75% of packets are lost, it is undefined. + For more information, refer to section 5.1 (Type-P-One-way-Delay- + Percentile) of RFC 2679 [RFC2679] (A One-way Delay Metric for IPPM), + and supporting text. The Delay Spread is the 75th Type-P-One-way- + Delay-Percentile minus the 25th Type-P-One-way-Delay-Percentile. + 4.4. Duplication Duplication is the fraction of transmitted packets for which more than a single copy of the packet was received within the timeout period (ideally using the same timeout as in the definition of loss), expressed in percentage points. Note: while most received packets can be ones previously seen, duplication can never exceed 100%. + For more information, see section 5.2 (Type-P-one-way-replicated- + packet-rate) of RFC TBD-duplicate [I-D.ietf-ippm-duplicate] (A One- + Way Packet Duplication Metric). Duplication is 100*Type-P-one-way- + replicated-packet-rate. + 4.5. Reordering Reordering is the fraction of sent packets for which the sequence number of the packet received immediately before the first copy of the given packet is not the decrement of the sequence number of the given packet. For the purposes of determining the sequence number of the preceding packet in this definition, assuming sequence numbers starting with 1, an extra packet at the start of the stream of received packets, with a sequence number of 0, is considered to be present (this extra packet, of course, is not counted for the @@ -320,23 +358,34 @@ 9. Internationalization Considerations The reported metrics, while they might occasionally be parsed by machine, are primarily meant for human consumption. As such, they MAY be reported in the language preferred by the user, using an encoding suitable for the purpose, such as UTF-8. 10. Normative References + [I-D.ietf-ippm-duplicate] + Uijterwaal, H., "A One-Way Packet Duplication Metric", + draft-ietf-ippm-duplicate-07 (work in progress), + January 2009. + [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. + [RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way + Delay Metric for IPPM", RFC 2679, September 1999. + + [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way + Packet Loss Metric for IPPM", RFC 2680, September 1999. + Appendix A. Sample Source Code for Computing the Metrics This appendix only serves for illustrative purposes. /* * reporting.c -- performance metrics reporting as in Internet Draft * draft-ietf-ippm-reporting. * * Written by Stanislav Shalunov, http://www.internet2.edu/~shalunov/ * Bernhard Lutzmann, belu@users.sf.net @@ -1473,20 +1522,27 @@ Appendix C. TODO FIXME: This section needs to be removed before publication. o Add references Appendix D. Revision History FIXME: This section needs to be removed before publication. + version -03: + Add most references; some small wording changes for clarity. + + version -02: + Update terminology. + + Log leading up to -01: $Log: draft-ietf-ippm-reporting.xml,v $ Revision 1.8 2006/10/23 21:45:54 shalunov draft-ietf-ippm-reporting-01.txt Revision 1.7 2006/10/23 21:45:13 shalunov Add sample source code and output. Revision 1.6 2006/06/02 21:21:57 shalunov draft-ietf-ippm-reporting-00: Include a ``Scope'' section. Change tags from draft-shalunov-ippm-reporting. @@ -1519,50 +1575,10 @@ URI: http://shlang.com/ Martin Swany University of Delaware Department of Computer and Information Sciences Newark, DE 19716 US Email: swany@cis.udel.edu URI: http://www.cis.udel.edu/~swany/ - -Full Copyright Statement - - Copyright (C) The IETF Trust (2008). - - This document is subject to the rights, licenses and restrictions - contained in BCP 78, and except as set forth therein, the authors - retain all their rights. - - This document and the information contained herein are provided on an - "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS - OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND - THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS - OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF - THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED - WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. - -Intellectual Property - - The IETF takes no position regarding the validity or scope of any - Intellectual Property Rights or other rights that might be claimed to - pertain to the implementation or use of the technology described in - this document or the extent to which any license under such rights - might or might not be available; nor does it represent that it has - made any independent effort to identify any such rights. Information - on the procedures with respect to rights in RFC documents can be - found in BCP 78 and BCP 79. - - Copies of IPR disclosures made to the IETF Secretariat and any - assurances of licenses to be made available, or the result of an - attempt made to obtain a general license or permission for the use of - such proprietary rights by implementers or users of this - specification can be obtained from the IETF on-line IPR repository at - http://www.ietf.org/ipr. - - The IETF invites any interested party to bring to its attention any - copyrights, patents or patent applications, or other proprietary - rights that may cover technology that may be required to implement - this standard. Please address the information to the IETF at - ietf-ipr@ietf.org.