draft-ietf-ippm-delay-03.txt | draft-ietf-ippm-delay-04.txt | |||
---|---|---|---|---|
Network Working Group G. Almes | Network Working Group G. Almes | |||
Internet Draft S. Kalidindi | Internet Draft S. Kalidindi | |||
Expiration Date: December 1998 M. Zekauskas | Expiration Date: March 1999 M. Zekauskas | |||
Advanced Network & Services | Advanced Network & Services | |||
June 1998 | August 1998 | |||
A One-way Delay Metric for IPPM | A One-way Delay Metric for IPPM | |||
<draft-ietf-ippm-delay-03.txt> | <draft-ietf-ippm-delay-04.txt> | |||
1. Status of this Memo | 1. Status of this Memo | |||
This document is an Internet-Draft. Internet-Drafts are working | This document is an Internet-Draft. Internet-Drafts are working | |||
documents of the Internet Engineering Task Force (IETF), its areas, | documents of the Internet Engineering Task Force (IETF), its areas, | |||
and its working groups. Note that other groups may also distribute | and its working groups. Note that other groups may also distribute | |||
working documents as Internet Drafts. | working documents as Internet Drafts. | |||
Internet-Drafts are draft documents valid for a maximum of six | Internet-Drafts are draft documents valid for a maximum of six | |||
months, and may be updated, replaced, or obsoleted by other documents | months, and may be updated, replaced, or obsoleted by other documents | |||
skipping to change at page 1, line 38 | skipping to change at page 1, line 38 | |||
Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). | Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). | |||
This memo provides information for the Internet community. This memo | This memo provides information for the Internet community. This memo | |||
does not specify an Internet standard of any kind. Distribution of | does not specify an Internet standard of any kind. Distribution of | |||
this memo is unlimited. | this memo is unlimited. | |||
2. Introduction | 2. Introduction | |||
This memo defines a metric for one-way delay of packets across | This memo defines a metric for one-way delay of packets across | |||
Internet paths. It builds on notions introduced and discussed in the | Internet paths. It builds on notions introduced and discussed in the | |||
IPPM Framework document, RFC 2223 [1]; the reader is assumed to be | IPPM Framework document, RFC 2330 [1]; the reader is assumed to be | |||
familiar with that document. | familiar with that document. | |||
This memo is intended to be parallel in structure to a companion | This memo is intended to be parallel in structure to a companion | |||
document for Packet Loss ("A Packet Loss Metric for IPPM" | document for Packet Loss ("A Packet Loss Metric for IPPM" | |||
<draft-ietf-ippm-loss-02.txt>) [2]. | <draft-ietf-ippm-loss-04.txt>) [2]. | |||
The structure of the memo is as follows: | The structure of the memo is as follows: | |||
+ A 'singleton' analytic metric, called Type-P-One-way-Delay, will | + A 'singleton' analytic metric, called Type-P-One-way-Delay, will | |||
be introduced to measure a single observation of one-way delay. | be introduced to measure a single observation of one-way delay. | |||
+ Using this singleton metric, a 'sample', called Type-P-One-way- | + Using this singleton metric, a 'sample', called Type-P-One-way- | |||
Delay-Poisson-Stream, will be introduced to measure a sequence of | Delay-Poisson-Stream, will be introduced to measure a sequence of | |||
singleton delays measured at times taken from a Poisson process. | singleton delays measured at times taken from a Poisson process. | |||
skipping to change at page 2, line 24 | skipping to change at page 2, line 24 | |||
This progression from singleton to sample to statistics, with clear | This progression from singleton to sample to statistics, with clear | |||
separation among them, is important. | separation among them, is important. | |||
Whenever a technical term from the IPPM Framework document is first | Whenever a technical term from the IPPM Framework document is first | |||
used in this memo, it will be tagged with a trailing asterisk. For | used in this memo, it will be tagged with a trailing asterisk. For | |||
example, "term*" indicates that "term" is defined in the Framework. | example, "term*" indicates that "term" is defined in the Framework. | |||
2.1. Motivation: | 2.1. Motivation: | |||
One-way delay of a type-P packet from a source host* to a destination | One-way delay of a Type-P* packet from a source host* to a | |||
host is useful for several reasons: | destination host is useful for several reasons: | |||
+ Some applications do not perform well (or at all) if end-to-end | + Some applications do not perform well (or at all) if end-to-end | |||
delay between hosts is large relative to some threshold value. | delay between hosts is large relative to some threshold value. | |||
+ Erratic variation in delay makes it difficult (or impossible) to | + Erratic variation in delay makes it difficult (or impossible) to | |||
support many real-time applications. | support many real-time applications. | |||
+ The larger the value of delay, the more difficult it is for | + The larger the value of delay, the more difficult it is for | |||
transport-layer protocols to sustain high bandwidths. | transport-layer protocols to sustain high bandwidths. | |||
skipping to change at page 4, line 15 | skipping to change at page 4, line 15 | |||
3.2. Metric Parameters: | 3.2. Metric Parameters: | |||
+ Src, the IP address of a host | + Src, the IP address of a host | |||
+ Dst, the IP address of a host | + Dst, the IP address of a host | |||
+ T, a time | + T, a time | |||
3.3. Metric Units: | 3.3. Metric Units: | |||
The value of a type-P-One-way-Delay is either a non-negative real | The value of a Type-P-One-way-Delay is either a non-negative real | |||
number or an undefined (informally, infinite) number of seconds. | number, or an undefined (informally, infinite) number of seconds. | |||
3.4. Definition: | 3.4. Definition: | |||
For a non-negative real number dT, >>the *Type-P-One-way-Delay* from | For a non-negative real number dT, >>the *Type-P-One-way-Delay* from | |||
Src to Dst at T is dT<< means that Src sent the first bit of a type-P | Src to Dst at T is dT<< means that Src sent the first bit of a Type-P | |||
packet to Dst at wire-time* T and that Dst received the last bit of | packet to Dst at wire-time* T and that Dst received the last bit of | |||
that packet at wire-time T+dT. | that packet at wire-time T+dT. | |||
>>The *Type-P-One-way-Delay* from Src to Dst at T is undefined | >>The *Type-P-One-way-Delay* from Src to Dst at T is undefined | |||
(informally, infinite)<< means that Src sent the first bit of a type- | (informally, infinite)<< means that Src sent the first bit of a Type- | |||
P packet to Dst at wire-time T and that Dst did not receive that | P packet to Dst at wire-time T and that Dst did not receive that | |||
packet. | packet. | |||
Suggestions for what to report along with metric values appear in | ||||
Section 3.8 after a discussion of the metric, methodologies for | ||||
measuring the metric, and error analysis. | ||||
3.5. Discussion: | 3.5. Discussion: | |||
Type-P-One-way-Delay is a relatively simple analytic metric, and one | Type-P-One-way-Delay is a relatively simple analytic metric, and one | |||
that we believe will afford effective methods of measurement. | that we believe will afford effective methods of measurement. | |||
The following issues are likely to come up in practice: | The following issues are likely to come up in practice: | |||
+ Since delay values will often be as low as the 100 usec to 10 msec | + Since delay values will often be as low as the 100 usec to 10 msec | |||
range, it will be important for Src and Dst to synchronize very | range, it will be important for Src and Dst to synchronize very | |||
closely. GPS systems afford one way to achieve synchronization to | closely. GPS systems afford one way to achieve synchronization to | |||
skipping to change at page 5, line 17 | skipping to change at page 5, line 19 | |||
large (and the packet is yet to arrive at Dst). As noted by | large (and the packet is yet to arrive at Dst). As noted by | |||
Mahdavi and Paxson [4], simple upper bounds (such as the 255 | Mahdavi and Paxson [4], simple upper bounds (such as the 255 | |||
seconds theoretical upper bound on the lifetimes of IP | seconds theoretical upper bound on the lifetimes of IP | |||
packets [5]) could be used, but good engineering, including an | packets [5]) could be used, but good engineering, including an | |||
understanding of packet lifetimes, will be needed in practice. | understanding of packet lifetimes, will be needed in practice. | |||
{Comment: Note that, for many applications of these metrics, the | {Comment: Note that, for many applications of these metrics, the | |||
harm in treating a large delay as infinite might be zero or very | harm in treating a large delay as infinite might be zero or very | |||
small. A TCP data packet, for example, that arrives only after | small. A TCP data packet, for example, that arrives only after | |||
several multiples of the RTT may as well have been lost.} | several multiples of the RTT may as well have been lost.} | |||
+ The context in which the metric is measured must be carefully | ||||
considered, and should always be reported along with metric | ||||
results. | ||||
As noted in the Framework document [1], the value of the metric | ||||
may depend on the type of IP packets used to make the measurement, | ||||
or "type-P". The value of Type-P-One-way-Delay could change if | ||||
the protocol (UDP or TCP), port number, size, or arrangement for | ||||
special treatment (e.g., IP precedence or RSVP) changes. The | ||||
exact Type-P used to make the measurements must be accurately | ||||
reported. | ||||
In addition, the threshold (or methodology to distinguish) between | ||||
a large finite delay and loss should be reported. | ||||
Finally, the path traversed by the packet should be reported, if | ||||
possible. In general it is impractical to know the precise path a | ||||
given packet takes through the network. The precise path may be | ||||
known for certain Type-P on short or stable paths. If Type-P | ||||
includes the record route (or loose-source route) option in the IP | ||||
header, and the path is short enough, and all routers* on the path | ||||
support record (or loose-source) route, then the path will be | ||||
precisely recorded. This is impractical because the route must be | ||||
short enough, many routers do not support (or are not configured | ||||
for) record route, and use of this feature would often | ||||
artificially worsen the performance observed by removing the | ||||
packet from common-case processing. However, partial information | ||||
is still valuable context. For example, if a host can choose | ||||
between two links* (and hence two separate routes from src to | ||||
dst), then the initial link used is valuable context. {Comment: | ||||
For example, with Merit's NetNow setup, a Src on one NAP can reach | ||||
a Dst on another NAP by either of several different backbone | ||||
networks.} | ||||
The above list is not exhaustive; any additional information that | ||||
could be useful in interpreting applications of the metrics should | ||||
be reported. | ||||
+ If the packet is duplicated along the path (or paths) so that | + If the packet is duplicated along the path (or paths) so that | |||
multiple non-corrupt copies arrive at the destination, then the | multiple non-corrupt copies arrive at the destination, then the | |||
packet is counted as received, and the first copy to arrive | packet is counted as received, and the first copy to arrive | |||
determines the packet's one-way delay. | determines the packet's one-way delay. | |||
+ If the packet is fragmented and if, for whatever reason, | + If the packet is fragmented and if, for whatever reason, | |||
reassembly does not occur, then the packet will be deemed lost. | reassembly does not occur, then the packet will be deemed lost. | |||
3.6. Methodologies: | 3.6. Methodologies: | |||
skipping to change at page 7, line 5 | skipping to change at page 6, line 20 | |||
subtracting the two timestamps, an estimate of one-way delay can | subtracting the two timestamps, an estimate of one-way delay can | |||
be computed. Error analysis of a given implementation of the | be computed. Error analysis of a given implementation of the | |||
method must take into account the closeness of synchronization | method must take into account the closeness of synchronization | |||
between Src and Dst. If the delay between Src's timestamp and the | between Src and Dst. If the delay between Src's timestamp and the | |||
actual sending of the packet is known, then the estimate could be | actual sending of the packet is known, then the estimate could be | |||
adjusted by subtracting this amount; uncertainty in this value | adjusted by subtracting this amount; uncertainty in this value | |||
must be taken into account in error analysis. Similarly, if the | must be taken into account in error analysis. Similarly, if the | |||
delay between the actual receipt of the packet and Dst's timestamp | delay between the actual receipt of the packet and Dst's timestamp | |||
is known, then the estimate could be adjusted by subtracting this | is known, then the estimate could be adjusted by subtracting this | |||
amount; uncertainty in this value must be taken into account in | amount; uncertainty in this value must be taken into account in | |||
error analysis. | error analysis. See the next section, "Errors and Uncertainties", | |||
for a more detailed discussion. | ||||
+ If the packet fails to arrive within a reasonable period of time, | + If the packet fails to arrive within a reasonable period of time, | |||
the one-way delay is taken to be undefined (informally, infinite). | the one-way delay is taken to be undefined (informally, infinite). | |||
Note that the threshold of 'reasonable' here is a parameter of the | Note that the threshold of 'reasonable' is a parameter of the | |||
methodology. | methodology. | |||
Issues such as the packet format, the means by which Dst knows when | Issues such as the packet format, the means by which Dst knows when | |||
to expect the test packet, and the means by which Src and Dst are | to expect the test packet, and the means by which Src and Dst are | |||
synchronized are outside the scope of this document. {Comment: We | synchronized are outside the scope of this document. {Comment: We | |||
plan to document elsewhere our own work in describing such more | plan to document elsewhere our own work in describing such more | |||
detailed implementation techniques and we encourage others to as | detailed implementation techniques and we encourage others to as | |||
well.} | well.} | |||
3.7. Errors and Uncertainties: | 3.7. Errors and Uncertainties: | |||
The description of any specific measurement method should include an | The description of any specific measurement method should include an | |||
accounting and analysis of various sources of error/uncertainty. The | accounting and analysis of various sources of error or uncertainty. | |||
Framework document provides general guidence on this point, but we | The Framework document provides general guidence on this point, but | |||
note here the following specifics related to delay metrics: | we note here the following specifics related to delay metrics: | |||
+ Errors/uncertainties due to uncertainties in the clocks of the Src | + Errors or uncertainties due to uncertainties in the clocks of the | |||
and Dst hosts. | Src and Dst hosts. | |||
+ Errors/uncertainties due to the difference between 'wire time' and | + Errors or uncertainties due to the difference between 'wire time' | |||
'host time'. | and 'host time'. | |||
Each of these are discussed in more detail below. | In addition, the loss threshold may affect the results. Each of | |||
these are discussed in more detail below, along with a section | ||||
("Calibration") on accounting for these errors and uncertainties. | ||||
3.7.1. Errors/uncertainties related to Clocks | 3.7.1. Errors or uncertainties related to Clocks | |||
The uncertainty in a measurement of one-way delay is related, in | The uncertainty in a measurement of one-way delay is related, in | |||
part, to uncertainties in the clocks of the Src and Dst hosts. In | part, to uncertainties in the clocks of the Src and Dst hosts. In | |||
the following, we refer to the clock used to measure when the packet | the following, we refer to the clock used to measure when the packet | |||
was sent from Src as the source clock, we refer to the clock used to | was sent from Src as the source clock, we refer to the clock used to | |||
measure when the packet was received by Dst as the dest clock, we | measure when the packet was received by Dst as the dest clock, we | |||
refer to the observed time when the packet was sent by the source | refer to the observed time when the packet was sent by the source | |||
clock as Tsource, and the observed time when the packet was received | clock as Tsource, and the observed time when the packet was received | |||
by the dest clock as Tdest. Alluding to the notions of | by the dest clock as Tdest. Alluding to the notions of | |||
synchronization, accuracy, resolution, and skew mentioned in the | synchronization, accuracy, resolution, and skew mentioned in the | |||
skipping to change at page 8, line 11 | skipping to change at page 7, line 28 | |||
+ Any error in the synchronization between the source clock and the | + Any error in the synchronization between the source clock and the | |||
dest clock will contribute to error in the delay measurement. We | dest clock will contribute to error in the delay measurement. We | |||
say that the source clock and the dest clock have a | say that the source clock and the dest clock have a | |||
synchronization error of Tsynch if the source clock is Tsynch | synchronization error of Tsynch if the source clock is Tsynch | |||
ahead of the dest clock. Thus, if we know the value of Tsynch | ahead of the dest clock. Thus, if we know the value of Tsynch | |||
exactly, we could correct for clock synchronization by adding | exactly, we could correct for clock synchronization by adding | |||
Tsynch to the uncorrected value of Tdest-Tsource. | Tsynch to the uncorrected value of Tdest-Tsource. | |||
+ The accuracy of a clock is important only in identifying the time | + The accuracy of a clock is important only in identifying the time | |||
at which a given delay was measured. Accuracy, per se, has no | at which a given delay was measured. Accuracy, per se, has no | |||
importance to the accuracy of the measurement of delay. This is | importance to the accuracy of the measurement of delay. When | |||
because, when computing delays, we are interested only in the | computing delays, we are interested only in the differences | |||
differences between clock values. | between clock values, not the values themselves. | |||
+ The resolution of a clock adds to uncertainty about any time | + The resolution of a clock adds to uncertainty about any time | |||
measured with it. Thus, if the source clock has a resolution of | measured with it. Thus, if the source clock has a resolution of | |||
10 msec, then this adds 10 msec of uncertainty to any time value | 10 msec, then this adds 10 msec of uncertainty to any time value | |||
measured with it. We will denote the resolution of the source | measured with it. We will denote the resolution of the source | |||
clock and the dest clock as Rsource and Rdest, respectively. | clock and the dest clock as Rsource and Rdest, respectively. | |||
+ The skew of a clock is not so much an additional issue as it is a | + The skew of a clock is not so much an additional issue as it is a | |||
realization of the fact that Tsynch is itself a function of time. | realization of the fact that Tsynch is itself a function of time. | |||
Thus, if we attempt to measure or to bound Tsynch, this needs to | Thus, if we attempt to measure or to bound Tsynch, this needs to | |||
skipping to change at page 8, line 40 | skipping to change at page 8, line 8 | |||
Esynch(t) to denote an upper bound on the uncertainty in | Esynch(t) to denote an upper bound on the uncertainty in | |||
synchronization. Thus, |Tsynch(t)| <= Esynch(t). | synchronization. Thus, |Tsynch(t)| <= Esynch(t). | |||
Taking these items together, we note that naive computation Tdest- | Taking these items together, we note that naive computation Tdest- | |||
Tsource will be off by Tsynch(t) +/- (|Rsource|+|Rdest|). Using the | Tsource will be off by Tsynch(t) +/- (|Rsource|+|Rdest|). Using the | |||
notion of Esynch(t), we note that these clock-related problems | notion of Esynch(t), we note that these clock-related problems | |||
introduce a total uncertainty of Esynch(t)+|Rsource|+|Rdest|. This | introduce a total uncertainty of Esynch(t)+|Rsource|+|Rdest|. This | |||
estimate of total clock-related uncertainty should be included in the | estimate of total clock-related uncertainty should be included in the | |||
error/uncertainty analysis of any measurement implementation. | error/uncertainty analysis of any measurement implementation. | |||
3.7.2. Errors/uncertainties related to Wire-time vs Host-time | 3.7.2. Errors or uncertainties related to Wire-time vs Host-time | |||
As we've defined one-way delay, we'd like to measure the time between | As we have defined one-way delay, we would like to measure the time | |||
when the test packet leaves the network interface of Src and when it | between when the test packet leaves the network interface of Src and | |||
(completely) arrives at the network interface of Dst, and we refer to | when it (completely) arrives at the network interface of Dst, and we | |||
this as 'wire time'. If the timings are themselves performed by | refer to this as 'wire time'. If the timings are themselves | |||
software on Src and Dst, however, then this software can only | performed by software on Src and Dst, however, then this software can | |||
directly measure the time between when Src grabs a timestamp just | only directly measure the time between when Src grabs a timestamp | |||
prior to sending the test packet and when Dst grabs a timestamp just | just prior to sending the test packet and when Dst grabs a timestamp | |||
after having received the test packet, and we refer to this as 'host | just after having received the test packet, and we refer to this as | |||
time'. | 'host time'. | |||
To the extent that the difference between wire time and host time is | To the extent that the difference between wire time and host time is | |||
accurately known, this knowledge can be used to correct for host time | accurately known, this knowledge can be used to correct for host time | |||
measurements and the corrected value more accurately estimates the | measurements and the corrected value more accurately estimates the | |||
desired (wire time) metric. | desired (wire time) metric. | |||
To the extent, however, that the difference between wire time and | To the extent, however, that the difference between wire time and | |||
host time is uncertain, this uncertainty must be accounted for in an | host time is uncertain, this uncertainty must be accounted for in an | |||
analysis of a given measurement method. We denote by Hsource an | analysis of a given measurement method. We denote by Hsource an | |||
upper bound on the uncertainty in the difference between wire time | upper bound on the uncertainty in the difference between wire time | |||
and host time on the Src host, and similarly define Hdest for the Dst | and host time on the Src host, and similarly define Hdest for the Dst | |||
host. We then note that these problems introduce a total uncertainty | host. We then note that these problems introduce a total uncertainty | |||
of Hsource+Hdest. This estimate of total wire-vs-host uncertainty | of Hsource+Hdest. This estimate of total wire-vs-host uncertainty | |||
should be included in the error/uncertainty analysis of any | should be included in the error/uncertainty analysis of any | |||
measurement implementation. | measurement implementation. | |||
3.7.3. Calibration | ||||
Generally, the measured values can be decomposed as follows: | ||||
measured value = true value + systematic error + random error | ||||
If the systematic error (the constant bias in measured values) can be | ||||
determined, it can be compensated for in the reported results. | ||||
reported value = measured value - systematic error | ||||
therefore | ||||
reported value = true value + random error | ||||
The goal of calibration is to determine the systematic and random | ||||
error in as much detail as possible. At a minimum, a bound ("e") | ||||
should be found such that the reported value is in the range (true | ||||
value - e) to (true value + e) at least 95 percent of the time. We | ||||
call "e" the error bar for the measurements. {Comment: 95 percent | ||||
was chosen because (1) some confidence level is desirable to be able | ||||
to remove outliers which will be found in measuring any physical | ||||
property; (2) a particular confidence level should be specified so | ||||
that the results of independent implementations can be compared; and | ||||
(3) even with a prototype user-level implementation, 95% was loose | ||||
enough to exclude outliers.} | ||||
From the discussion in the previous two sections, the error in | ||||
measurements could be bounded by determining all the individual | ||||
uncertainties, and adding them together to form | ||||
Esynch(t) + |Rsource| + |Rdest| + Hsource + Hdest. | ||||
However, reasonable bounds on both the clock-related uncertainty | ||||
captured by the first three terms and the host-related uncertainty | ||||
captured by the last two terms should be possible by careful design | ||||
techniques and calibrating the instruments using a known, isolated, | ||||
network in a lab. | ||||
For example, the clock-related uncertainties are greatly reduced | ||||
through the use of a GPS time source. The sum of Esynch(t) + | ||||
|Rsource| + |Rdest| is small, and is also bounded for the duration of | ||||
the measurement because of the global time source. | ||||
The host-related uncertainties, Hsource + Hdest, could be bounded by | ||||
connecting two instruments back-to-back with a high-speed serial link | ||||
or isolated LAN (depending on the intended network connection for | ||||
actual measurement), and performing repeated measurements. In this | ||||
case, unlike measuring live networks, repeated measurements are | ||||
measuring the same wire time. (When measuring live networks, the | ||||
wire time is what you are measuring, and varies with the load | ||||
encountered on the path traversed by the test packets.) | ||||
If the test packets are small, such a network connection has a | ||||
minimal wire time that may be approximated by zero. The measured | ||||
delay therefore contains only systematic and random error in the | ||||
instrumentation. The "average value" of repeated measurements is the | ||||
systematic error, and the variation is the random error. | ||||
One way to compute the systematic error, and the random error to a | ||||
95% confidence is to repeat the experiment many times - at least | ||||
hundreds of tests. The systematic error would then be the median, | ||||
and likely the mode (the most frequently occuring value). {Comment: | ||||
It's likely the systematic error is represented by the minimum value | ||||
(which is also the median and the mode); with unloaded instruments on | ||||
a single test path all the random error will tend to be increased | ||||
time due to host processing. The only error resulting an a delay | ||||
less than the systematic error would be due to clock-related | ||||
uncertainties (resolution and relative skew).} The random error could | ||||
then be found by removing the systematic error from the measured | ||||
values. The 95% confidence interval would be the range from the 2nd | ||||
percentile to the 97th percentile of these deviations from the true | ||||
value. The error bar "e" could then be taken to be the largest | ||||
absolute value of these two numbers, plus the clock-related | ||||
uncertainty. If all of the deviations are positive, then the 95% | ||||
confidence interval is simply the 95th percentile, and that value | ||||
should be used instead of the larger of the 2nd and 97th percentiles. | ||||
{Comment: as described, this bound is relatively loose since the | ||||
uncertainties are added, and the absolute value of the largest | ||||
deviation is used. As long as the resulting value is not a | ||||
significant fraction of the measured values, it is a reasonable | ||||
bound. If the resulting value is a significant fraction of the | ||||
measured values, then more exact methods will be needed to compute an | ||||
error bar.} | ||||
Note that random error is a function of measurement load. For | ||||
example, if many paths will be measured by one instrument, this might | ||||
increase interrupts, process scheduling, and disk I/O (for example, | ||||
recording the measurements), all of which may increase the random | ||||
error in measured singletons. Therefore, in addition to minimal load | ||||
measurements to find the systematic error, calibration measurements | ||||
should be performed with the same measurement load that the | ||||
instruments will see in the field. | ||||
In addition to calibrating the instruments for finite one-way delay, | ||||
two checks should be made to ensure that packets reported as losses | ||||
were really lost. First, the threshold for loss should be verified. | ||||
In particular, ensure the "reasonable" threshold is reasonable: that | ||||
it is very unlikely a packet will arrive after the threshold value, | ||||
and therefore the number of packets lost over an interval is not | ||||
sensitive to the error bound on measurements. Second, consider the | ||||
probability that a packet arrives at the network interface, but is | ||||
lost due to congestion on that interface or to other resource | ||||
exhaustion (e.g. buffers) in the instrument. | ||||
3.8. Reporting the metric: | ||||
The calibration and context in which the metric is measured must be | ||||
carefully considered, and should always be reported along with metric | ||||
results. We now present four items to consider: the Type-P of test | ||||
packets, the threshold of infinite delay (if any), error calibration, | ||||
and the path traversed by the test packets. This list is not | ||||
exhaustive; any additional information that could be useful in | ||||
interpreting applications of the metrics should also be reported. | ||||
3.8.1. Type-P | ||||
As noted in the Framework document [1], the value of the metric may | ||||
depend on the type of IP packets used to make the measurement, or | ||||
"type-P". The value of Type-P-One-way-Delay could change if the | ||||
protocol (UDP or TCP), port number, size, or arrangement for special | ||||
treatment (e.g., IP precedence or RSVP) changes. The exact Type-P | ||||
used to make the measurements must be accurately reported. | ||||
3.8.2. Loss threshold | ||||
In addition, the threshold (or methodology to distinguish) between a | ||||
large finite delay and loss should be reported. | ||||
3.8.3. Calibration results | ||||
+ If the systematic error can be determined, it should be removed | ||||
from the measured values. | ||||
+ Report an error bar, e, such that the true value is the reported | ||||
value plus or minus e, with 95% confidence. | ||||
+ If possible, report the probability that a test packet with finite | ||||
delay is reported as lost due to resource exhaustion on the | ||||
measurement instrument. | ||||
3.8.4. Path | ||||
Finally, the path traversed by the packet should be reported, if | ||||
possible. In general it is impractical to know the precise path a | ||||
given packet takes through the network. The precise path may be | ||||
known for certain Type-P on short or stable paths. If Type-P | ||||
includes the record route (or loose-source route) option in the IP | ||||
header, and the path is short enough, and all routers* on the path | ||||
support record (or loose-source) route, then the path will be | ||||
precisely recorded. This is impractical because the route must be | ||||
short enough, many routers do not support (or are not configured for) | ||||
record route, and use of this feature would often artificially worsen | ||||
the performance observed by removing the packet from common-case | ||||
processing. However, partial information is still valuable context. | ||||
For example, if a host can choose between two links* (and hence two | ||||
separate routes from src to dst), then the initial link used is | ||||
valuable context. {Comment: For example, with Merit's NetNow setup, | ||||
a Src on one NAP can reach a Dst on another NAP by either of several | ||||
different backbone networks.} | ||||
4. A Definition for Samples of One-way Delay | 4. A Definition for Samples of One-way Delay | |||
Given the singleton metric Type-P-One-way-Delay, we now define one | Given the singleton metric Type-P-One-way-Delay, we now define one | |||
particular sample of such singletons. The idea of the sample is to | particular sample of such singletons. The idea of the sample is to | |||
select a particular binding of the parameters Src, Dst, and Type-P, | select a particular binding of the parameters Src, Dst, and Type-P, | |||
then define a sample of values of parameter T. The means for | then define a sample of values of parameter T. The means for | |||
defining the values of T is to select a beginning time T0, a final | defining the values of T is to select a beginning time T0, a final | |||
time Tf, and an average rate lambda, then define a pseudo-random | time Tf, and an average rate lambda, then define a pseudo-random | |||
Poisson arrival process of rate lambda, whose values fall between T0 | Poisson arrival process of rate lambda, whose values fall between T0 | |||
and Tf. The time interval between successive values of T will then | and Tf. The time interval between successive values of T will then | |||
skipping to change at page 11, line 27 | skipping to change at page 14, line 27 | |||
test packet at TS[i], then send a second one (later) at TS[i+1], | test packet at TS[i], then send a second one (later) at TS[i+1], | |||
while the Dst could receive the second test packet at TR[i+1], and | while the Dst could receive the second test packet at TR[i+1], and | |||
then receive the first one (later) at TR[i]. | then receive the first one (later) at TR[i]. | |||
4.7. Errors and Uncertainties: | 4.7. Errors and Uncertainties: | |||
In addition to sources of errors and uncertainties associated with | In addition to sources of errors and uncertainties associated with | |||
methods employed to measure the singleton values that make up the | methods employed to measure the singleton values that make up the | |||
sample, care must be given to analyze the accuracy of the Poisson | sample, care must be given to analyze the accuracy of the Poisson | |||
arrival process of the wire-time of the sending of the test packets. | arrival process of the wire-time of the sending of the test packets. | |||
Problems with this process could be caused by either of several | Problems with this process could be caused by several things, | |||
things, including problems with the pseudo-random number techniques | including problems with the pseudo-random number techniques used to | |||
used to generate the Poisson arrival process, or with jitter in the | generate the Poisson arrival process, or with jitter in the value of | |||
value of Hsource (mentioned above as uncertainty in the singleton | Hsource (mentioned above as uncertainty in the singleton delay | |||
delay metric). The Framework document shows how to use an Anderson- | metric). The Framework document shows how to use the Anderson- | |||
Darling test for this. | Darling test to verify the accuracy of the Poisson process. | |||
4.8. Reporting the metric: | ||||
You should report the calibration and context for the underlying | ||||
singletons along with the stream. (See "Reporting the metric" for | ||||
Type-P-One-way-Delay.) | ||||
5. Some Statistics Definitions for One-way Delay | 5. Some Statistics Definitions for One-way Delay | |||
Given the sample metric Type-P-One-way-Delay-Poisson-Stream, we now | Given the sample metric Type-P-One-way-Delay-Poisson-Stream, we now | |||
offer several statistics of that sample. These statistics are | offer several statistics of that sample. These statistics are | |||
offered mostly to be illustrative of what could be done. | offered mostly to be illustrative of what could be done. | |||
5.1. Type-P-One-way-Delay-Percentile | 5.1. Type-P-One-way-Delay-Percentile | |||
Given a Type-P-One-way-Delay-Poisson-Stream and a percent X between | Given a Type-P-One-way-Delay-Poisson-Stream and a percent X between | |||
skipping to change at page 12, line 16 | skipping to change at page 15, line 25 | |||
Stream1 = < | Stream1 = < | |||
<T1, 100 msec> | <T1, 100 msec> | |||
<T2, 110 msec> | <T2, 110 msec> | |||
<T3, undefined> | <T3, undefined> | |||
<T4, 90 msec> | <T4, 90 msec> | |||
<T5, 500 msec> | <T5, 500 msec> | |||
> | > | |||
Then the 50th percentile would be 110 msec, since 90 msec and 100 | Then the 50th percentile would be 110 msec, since 90 msec and 100 | |||
msec are smaller and 110 msec and 'undefined' are larger. | msec are smaller and 110 msec and 'undefined' are larger. | |||
Note that if the probability that a finite packet is reported as lost | ||||
is significant, then a high percentile (90th or 95th) might be | ||||
reported as infinite instead of finite. | ||||
5.2. Type-P-One-way-Delay-Median | 5.2. Type-P-One-way-Delay-Median | |||
Given a Type-P-One-way-Delay-Poisson-Stream, the median of all the dT | Given a Type-P-One-way-Delay-Poisson-Stream, the median of all the dT | |||
values in the Stream. In computing the median, undefined values are | values in the Stream. In computing the median, undefined values are | |||
treated as infinitely large. | treated as infinitely large. | |||
As noted in the Framework document, the median differs from the 50th | As noted in the Framework document, the median differs from the 50th | |||
percentile only when the sample contains an even number of values, in | percentile only when the sample contains an even number of values, in | |||
which case the mean of the two central values is used. | which case the mean of the two central values is used. | |||
skipping to change at page 14, line 9 | skipping to change at page 17, line 17 | |||
be used where appropriate to guard against injected traffic attacks. | be used where appropriate to guard against injected traffic attacks. | |||
The privacy concerns of network measurement are limited by the active | The privacy concerns of network measurement are limited by the active | |||
measurements described in this memo. Unlike passive measurements, | measurements described in this memo. Unlike passive measurements, | |||
there can be no release of existing user data. | there can be no release of existing user data. | |||
7. Acknowledgements | 7. Acknowledgements | |||
Special thanks are due to Vern Paxson of Lawrence Berkeley Labs for | Special thanks are due to Vern Paxson of Lawrence Berkeley Labs for | |||
his helpful comments on issues of clock uncertainty and statistics. | his helpful comments on issues of clock uncertainty and statistics. | |||
Thanks also to Sean Shapira and to Roland Wittig for several useful | Thanks also to Will Leland, Sean Shapira, and Roland Wittig for | |||
suggestions. | several useful suggestions. | |||
8. References | 8. References | |||
[1] V. Paxson, G. Almes, J. Mahdavi, and M. Mathis, "Framework for | [1] V. Paxson, G. Almes, J. Mahdavi, and M. Mathis, "Framework for | |||
IP Performance Metrics", RFC 2330, May 1998. | IP Performance Metrics", RFC 2330, May 1998. | |||
[2] G. Almes, S. Kalidindi, and M. Zekauskas, "A One-way Delay | [2] G. Almes, S. Kalidindi, and M. Zekauskas, "A Packet Loss Metric | |||
Metric for IPPM", Internet-Draft <draft-ietf-ippm-delay-02.txt>, | for IPPM", Internet-Draft <draft-ietf-ippm-loss-04.txt>, August | |||
June 1998. | 1998. | |||
[3] D. Mills, "Network Time Protocol (v3)", RFC 1305, April 1992. | [3] D. Mills, "Network Time Protocol (v3)", RFC 1305, April 1992. | |||
[4] J. Mahdavi and V. Paxson, "Connectivity", Work in Progress, | [4] J. Mahdavi and V. Paxson, "IPPM Metrics for Measuring | |||
November 1997. | Connectivity", Internet-Draft <draft-ietf-ippm- | |||
connectivity-02.txt>, August 1998. | ||||
[5] J. Postel, "Internet Protocol", RFC 791, September 1981. | [5] J. Postel, "Internet Protocol", RFC 791, September 1981. | |||
9. Authors' Addresses | 9. Authors' Addresses | |||
Guy Almes | Guy Almes | |||
Advanced Network & Services, Inc. | Advanced Network & Services, Inc. | |||
200 Business Park Drive | 200 Business Park Drive | |||
Armonk, NY 10504 | Armonk, NY 10504 | |||
USA | USA | |||
Phone: +1 914 765 1120 | Phone: +1 914 765 1120 | |||
EMail: almes@advanced.org | EMail: almes@advanced.org | |||
Sunil Kalidindi | Sunil Kalidindi | |||
skipping to change at page 15, line 4 | skipping to change at page 18, line 21 | |||
EMail: almes@advanced.org | EMail: almes@advanced.org | |||
Sunil Kalidindi | Sunil Kalidindi | |||
Advanced Network & Services, Inc. | Advanced Network & Services, Inc. | |||
200 Business Park Drive | 200 Business Park Drive | |||
Armonk, NY 10504 | Armonk, NY 10504 | |||
USA | USA | |||
Phone: +1 914 765 1128 | Phone: +1 914 765 1128 | |||
EMail: kalidindi@advanced.org | EMail: kalidindi@advanced.org | |||
Matthew J. Zekauskas | Matthew J. Zekauskas | |||
Advanced Network & Services, Inc. | Advanced Network & Services, Inc. | |||
200 Buisiness Park Drive | 200 Buisiness Park Drive | |||
Armonk, NY 10504 | Armonk, NY 10504 | |||
USA | USA | |||
Phone: +1 914 765 1112 | Phone: +1 914 765 1112 | |||
EMail: matt@advanced.org | EMail: matt@advanced.org | |||
Expiration date: December, 1998 | Expiration date: March, 1999 | |||
End of changes. | ||||
This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/ |