[Docs] [txt|pdf] [Tracker] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02 03 draft-ietf-ippm-reporting

Network Working Group                                        S. Shalunov
Internet-Draft                                                 Internet2
Expires: October 3, 2006                                      April 2006


               Reporting IP Performance Metrics to Users
                  draft-shalunov-ippm-reporting-03.txt

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on October 3, 2006.

Copyright Notice

   Copyright (C) The Internet Society (2006).

Abstract

   The aim of this document is to define a small set of metrics that are
   robust, easy to understand, orthogonal, relevant, and easy to
   compute.  The IPPM WG has defined a large number of richly
   parameterized metrics because network measurement has many purposes.
   Often, the ultimate purpose is to report a concise set of metrics
   describing a network's state to an end user.  It is for this purpose
   that the present set of metrics is defined.





Shalunov                 Expires October 3, 2006                [Page 1]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


Table of Contents

   1.  Requirements Notation  . . . . . . . . . . . . . . . . . . . .  3
   2.  Goals and Motivation . . . . . . . . . . . . . . . . . . . . .  4
   3.  Reportable Metrics Set . . . . . . . . . . . . . . . . . . . .  6
     3.1.  Delay  . . . . . . . . . . . . . . . . . . . . . . . . . .  6
     3.2.  Loss . . . . . . . . . . . . . . . . . . . . . . . . . . .  6
     3.3.  Jitter . . . . . . . . . . . . . . . . . . . . . . . . . .  6
     3.4.  Duplication  . . . . . . . . . . . . . . . . . . . . . . .  7
     3.5.  Reordering . . . . . . . . . . . . . . . . . . . . . . . .  7
   4.  Sample Source  . . . . . . . . . . . . . . . . . . . . . . . .  8
     4.1.  One-Way Active Measurement . . . . . . . . . . . . . . . .  8
     4.2.  Round-Trip Active Measurement  . . . . . . . . . . . . . .  9
     4.3.  Passive Measurement  . . . . . . . . . . . . . . . . . . .  9
   5.  Security Considerations  . . . . . . . . . . . . . . . . . . . 10
   6.  Internationalization Considerations  . . . . . . . . . . . . . 11
   7.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 12
   8.  Normative References . . . . . . . . . . . . . . . . . . . . . 12
   Appendix A.  Acknowledgments . . . . . . . . . . . . . . . . . . . 13
   Appendix B.  TODO  . . . . . . . . . . . . . . . . . . . . . . . . 14
   Appendix C.  Revision History  . . . . . . . . . . . . . . . . . . 15
   Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 16
   Intellectual Property and Copyright Statements . . . . . . . . . . 17




























Shalunov                 Expires October 3, 2006                [Page 2]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


1.  Requirements Notation

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].














































Shalunov                 Expires October 3, 2006                [Page 3]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


2.  Goals and Motivation

   The IPPM working group has defined many richly parameterized
   performance metrics with a number of variants (one-way delay, one-way
   loss, delay variation, reordering, etc.) and a protocol for obtaining
   the measurement data needed to compute these metrics (OWAMP).  It
   would be beneficial to define a standard way to report a set of
   performance metrics to end users.  Parameterization might be
   acceptable in such a set, but there must still be defaults for
   everything.  It is especially important to get these defaults right.
   Such a set would enable different tools to produce results that can
   be compared against each other.

   Existing tools already report statistic about the network.  This is
   done for varying reasons: network testing tools, such as the ping
   program available in UNIX-derived operating systems as well as in
   Microsoft Windows, report statistics with no knowledge of why the
   user is running the program; networked games might report statistics
   of the network connection to the server so users can better
   understand why they get the results they get (e.g., if something is
   slow, is this because of the network or the CPU?), so they can
   compare their statistics to those of others (``you're not lagged any
   more than I am'') or perhaps so that users can decide whether they
   need to upgrade the connection to their home; IP telephony hardware
   and software might report the statistics for similar reasons.  While
   existing tools report statistics all right, the particular set of
   metrics they choose is ad hoc; some metrics are not statistically
   robust, some are not relevant, and some are not easy to understand;
   more important than specific shortcomings, however, is the
   incompatibility: even if the sets of metrics were perfect, they would
   still be all different, and, therefore, metrics reported by different
   tools would not be comparable.

   The set of metrics of this document is meant for human consumption.
   It must therefore be small.  Anything greater than half-dozen numbers
   is certainly too confusing.

   Each of the metrics must be statistically robust.  Intuitively, this
   means that having a small number of bad data points and a small
   amount of noise must not dramatically change the metric.

   Each metric in the set must have, qualitatively, an immediate
   intuitive meaning that has to be obvious for an advanced end user
   without consulting documentation (that is, it has to be clear what
   rough meaning, intuitively, the larger values of a given metric
   have).

   To be small, the set has to be orthogonal: each of the metrics has to



Shalunov                 Expires October 3, 2006                [Page 4]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


   express a property not covered by other metrics (otherwise, there's
   redundancy).

   The metrics in the set must be relevant.  Partly, being easy to
   understand will help achieve this, but additional constraint may be
   placed by relevancy.

   Finally, while this set will most frequently be computed for small
   data sets, where efficiency is not a serious consideration, it must
   be possible to compute for large data sets, too.  In particular, it
   must be possible to compute the metrics in a single pass over the
   data using a limited amount of memory (i.e., it must be possible to
   take a source of measurement data with a high data rate and compute
   the metrics on the fly, discarding each data point after it is
   processed).




































Shalunov                 Expires October 3, 2006                [Page 5]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


3.  Reportable Metrics Set

   The following metrics comprise the set:

   1.  delay;

   2.  loss;

   3.  jitter;

   4.  duplication;

   5.  reordering.

   Each of the above is represented by a single numeric quantity,
   computed as described below.

3.1.  Delay

   The reported delay is the median of all delays in the sample.  When a
   packet is lost, its delay is to be considered +infinity for the
   purposes of this computation; therefore, if more than half of all
   packets are lost, the delay is +infinity.

   FIXME: References.

3.2.  Loss

   Loss is the fraction, expressed as a percentage, of packets that did
   not arrive intact within a given number of seconds (timeout value)
   after being sent.  Since this set of metrics often has to be reported
   to a waiting human user, the default timeout value has to be small.
   By default, 2 seconds MUST be the timeout value.

   FIXME: References.

3.3.  Jitter

   Jitter is the interquartile spread of delay.  In other words, jitter
   is equal to the difference of the 75th and 25th percentiles of delay.
   When both percentiles are +infinity, the value of jitter is
   undefined.  Therefore, if less than 25% of packets are lost, jitter
   is defined and finite; if between 75% and 25% of packets are lost,
   jitter is +infinity; finally, if more than 75% of packets are lost,
   jitter is undefined.

   FIXME: References.




Shalunov                 Expires October 3, 2006                [Page 6]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


3.4.  Duplication

   Duplication is the fraction of packets for which more than a single
   copy of the packet was received within the timeout period (same
   timeout as in the definition of loss), expressed in percentage
   points.

   Note: while most received packets can be ones previously seen,
   duplication can never exceed 100%.

   FIXME: References (tough one---IPPM hasn't defined duplication).

3.5.  Reordering

   Reordering is the fraction of sent packets for which the sequence
   number of the packet received immediately before the first copy of
   the given packet is not the decrement of the sequence number of the
   given packet.  For the purposes of determining the sequence number of
   the preceding packet in this definition, assuming sequence numbers
   starting with 1, an extra packet at the start of the stream of
   received packets, with a sequence number of 0, is considered to be
   present (this extra packet, of course, is not counted for the
   purposes of computing the fraction).

   FIXME: References.


























Shalunov                 Expires October 3, 2006                [Page 7]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


4.  Sample Source

   Section 3 describes the metrics to compute on a sample of
   measurements.  The source of the sample in not discussed there, and,
   indeed, the metrics discussed (delay, loss, etc.) are simply
   estimators that could be applied to any sample whatsoever.  For the
   names of the estimators to be applicable, of course, the measurements
   need to come from a packet delivery network.

   The data in the samples for the set of metrics discussed in this
   document can come from the following sources: one-way active
   measurement, round-trip measurement, and passive measurement.  There
   infrequently is a choice between active and passive measurement, as,
   typically, only one is available; consequently, no preference is
   given to one over the other.  In cases where clocks can be expected
   to be synchronized, in general, one-way measurements are preferred
   over round-trip measurements (as one-way measurements are more
   informative).  When one-way measurements cannot be obtained, or when
   clocks cannot be expected to be synchronized, round-trip measurement
   MAY be used.

4.1.  One-Way Active Measurement

   The default duration of the measurement interval is 10 seconds.

   The default sending schedule is a Poisson stream.

   The default sending rate is 10 packets/second on average.  The
   default sending schedule is a Poisson stream.  When randomized
   schedules, such as a Poisson stream, are used, the rate MUST be set
   with the distribution parameter(s).  With a randomized schedule, the
   default sample size is 100 packets and the measurement window
   duration can vary to some extent depending on the values of the
   (pseudo-)random deviates.

   The default packet size is the minimum necessary for the measurement.

   Values other than the default ones MAY be used; if they are used,
   their use, and specific values used, MUST be reported.

   A one-way active measurement is characterized by the source IP
   address, the destination IP address, the time when measurement was
   taken, and the type of packets (e.g., UDP with given port numbers and
   a given DSCP) used in the measurement.  For the time, the middle of
   the measurement interval MUST be reported.






Shalunov                 Expires October 3, 2006                [Page 8]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


4.2.  Round-Trip Active Measurement

   The same default parameters and characterization apply to round-trip
   measurement as to one-way measurement (Section 4.1).

4.3.  Passive Measurement

   Passive measurement use whatever data it is natural to use.  For
   example, an IP telephony application or a networked game would use
   the data that it sends.  An analysis of performance of a link might
   use all the packets that traversed the link in the measurement
   interval.  An analysis of performance of an Internet service
   provider's network might use all the packets that traversed the
   network in the measurement interval.  An analysis of performance of a
   specific service from the point of view of a given site might use an
   appropriate filter to select only the relevant packets.  In any case,
   the source needs to be reported.

   The same default duration applies to passive measurement as to one-
   way active measurement (Section 4.1).

   When the passive measurement data is reported in real time, a sliding
   window SHOULD be used as a measurement period, so that recent data
   become more quickly reflected.



























Shalunov                 Expires October 3, 2006                [Page 9]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


5.  Security Considerations

   The reporting per se, not being a protocol, does not raise security
   considerations.

   An aspect of reporting relevant to security is how the reported
   metrics are used and how they are collected.  If it is important that
   the metrics satisfy certain conditions (e.g., that the ISP whose
   network is being measured be unable to make the metrics appear better
   than they are), the collection mechanism MUST ensure that this is,
   indeed, so.  The exact mechanisms to do so our outside of scope of
   this document and belong with discussion of particular measurement
   data collection protocols.






































Shalunov                 Expires October 3, 2006               [Page 10]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


6.  Internationalization Considerations

   The reported metrics, while they might occasionally be parsed by
   machine, are primarily meant for human consumption.  As such, they
   MAY be reported in the language preferred by the user, using an
   encoding suitable for the purpose, such as UTF-8.













































Shalunov                 Expires October 3, 2006               [Page 11]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


7.  IANA Considerations

   This document requires no action from the IANA.

8.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.











































Shalunov                 Expires October 3, 2006               [Page 12]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


Appendix A.  Acknowledgments

   The author gratefully acknowledges discussion with, encouragement
   from, and contributions of Lawrence D. Dunn, Reza Fardid,
   Ruediger Geib, Matt Mathis, Al Morton, Carsten Schmoll,
   Henk Uijterwaal, and Matthew J. Zekauskas.













































Shalunov                 Expires October 3, 2006               [Page 13]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


Appendix B.  TODO

   FIXME: This section needs to be removed before publication.

   o  Add references

   o  Add non-normative code for illustration

   o  Add examples (code output)










































Shalunov                 Expires October 3, 2006               [Page 14]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


Appendix C.  Revision History

   FIXME: This section needs to be removed before publication.


   $Log: draft-shalunov-ippm-reporting.xml,v $
   Revision 1.5  2006/05/02 20:25:44  shalunov
   Version 03: Various refinements and clarifications based on feedback
   from Reza Fardid, Ruediger Geib, and Al Morton.

   Revision 1.4  2006/04/25 22:38:58  shalunov
   Version 02: Address comments from Carsten Schmoll, sent in message
   70524A4436C03E43A293958B505008B61B9CFB@EXCHSRV.fokus.fraunhofer.de.
   My response, with clarifications and diffs, is in message
   8664kxwazk.fsf@abel.internet2.edu.

   Revision 1.3  2006/04/11 22:09:47  shalunov
   Version 01: Wording changes based on discussion with Matt Zekauskas
   (reordering, loss).  Rewrite abstract a bit.  Add TODO list.

   Revision 1.2  2006/04/04 21:39:20  shalunov
   Convert to xml2rfc 1.30 and RFC 3978 IPR statement.

   Revision 1.1.1.1  2006/04/02 17:07:36  shalunov
   Initial import into CVS.


























Shalunov                 Expires October 3, 2006               [Page 15]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


Author's Address

   Stanislav Shalunov
   Internet2
   1000 Oakbrook Drive, Suite 300
   Ann Arbor, MI  48104
   US

   Phone: +1-734-913-4260
   Email: shalunov@internet2.edu
   URI:   http://www.internet2.edu/~shalunov/








































Shalunov                 Expires October 3, 2006               [Page 16]


Internet-Draft  Reporting IP Performance Metrics to Users     April 2006


Intellectual Property Statement

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.


Disclaimer of Validity

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
   ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
   INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Copyright Statement

   Copyright (C) The Internet Society (2006).  This document is subject
   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights.


Acknowledgment

   Funding for the RFC Editor function is currently provided by the
   Internet Society.




Shalunov                 Expires October 3, 2006               [Page 17]


Html markup produced by rfcmarkup 1.124, available from https://tools.ietf.org/tools/rfcmarkup/