draft-ietf-ippm-reordering-04.txt   draft-ietf-ippm-reordering-05.txt 
Network Working Group A.Morton Network Working Group A.Morton
Internet Draft L.Ciavattone Internet Draft L.Ciavattone
Document: <draft-ietf-ippm-reordering-04.txt> G.Ramachandran Document: <draft-ietf-ippm-reordering-05.txt> G.Ramachandran
Category: Standards Track AT&T Labs Category: Standards Track AT&T Labs
S.Shalunov S.Shalunov
Internet2 Internet2
J.Perser J.Perser
Spirent Consultant
Packet Reordering Metric for IPPM Packet Reordering Metric for IPPM
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026 [1]. all provisions of Section 10 of RFC2026 [1].
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
skipping to change at line 43 skipping to change at line 43
This memo defines metrics to evaluate if a network has maintained This memo defines metrics to evaluate if a network has maintained
packet order on a packet-by-packet basis. It provides motivations packet order on a packet-by-packet basis. It provides motivations
for the new metrics and discusses the measurement issues. The memo for the new metrics and discusses the measurement issues. The memo
first defines a reordered singleton, and then uses it as the basis first defines a reordered singleton, and then uses it as the basis
for sample metrics to quantify the extent of reordering in several for sample metrics to quantify the extent of reordering in several
useful dimensions. Additional metrics quantify the frequency of useful dimensions. Additional metrics quantify the frequency of
reordering and the distance between separate occurrences. We then reordering and the distance between separate occurrences. We then
define metrics with a receiver analysis orientation. Several define metrics with a receiver analysis orientation. Several
examples of evaluation using the various sample metrics are examples of evaluation using the various sample metrics are
included. included. An Appendix gives extended definitions for evaluating
order with packet fragmentation.
1. Conventions used in this document 1. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [2]. document are to be interpreted as described in RFC 2119 [2].
Although RFC 2119 was written with protocols in mind, the key words Although RFC 2119 was written with protocols in mind, the key words
are used in this document for similar reasons. They are used to are used in this document for similar reasons. They are used to
Packet Reordering Metric for IPPM October 2003
ensure the results of measurements from two different ensure the results of measurements from two different
implementations are comparable, and to note instances when an implementations are comparable, and to note instances when an
implementation could perturb the network. implementation could perturb the network.
2. Introduction 2. Introduction
Ordered delivery is a property of successful packet transfer Ordered delivery is a property of successful packet transfer
attempts, where the packet sequence ascends for each arriving packet attempts, where the packet sequence ascends for each arriving packet
and there are no backward steps. and there are no backward steps.
skipping to change at line 84 skipping to change at line 82
out-of-order, or reordered. For example, if sequentially numbered out-of-order, or reordered. For example, if sequentially numbered
packets arrive 1,2,4,5,3, then packet 3 is reordered. This is packets arrive 1,2,4,5,3, then packet 3 is reordered. This is
equivalent to Paxon's reordering definition in [4], where "late" equivalent to Paxon's reordering definition in [4], where "late"
packets were declared reordered. The alternative is to emphasize packets were declared reordered. The alternative is to emphasize
"premature" packets instead (4 and 5 in the example), but only the "premature" packets instead (4 and 5 in the example), but only the
arrival of packet 3 distinguishes this circumstance from packet arrival of packet 3 distinguishes this circumstance from packet
loss. Focusing attention on late packets allows us to maintain loss. Focusing attention on late packets allows us to maintain
orthogonality with the packet loss metric. The metric's construction orthogonality with the packet loss metric. The metric's construction
is very similar to the sequence space validation for received is very similar to the sequence space validation for received
segments in RFC793 [5]. Earlier work to define ordered delivery segments in RFC793 [5]. Earlier work to define ordered delivery
includes [6], [7], [8], [9], [10] and more ???. includes [6], [7], [8], [9], [10] and [11.
2.1 Motivation 2.1 Motivation
A reordering metric is relevant for most applications, especially A reordering metric is relevant for most applications, especially
when assessing network support for Real-Time media streams. The when assessing network support for Real-Time media streams. The
extent of reordering may be sufficient to cause a received packet to extent of reordering may be sufficient to cause a received packet to
be discarded by functions above the IP layer. be discarded by functions above the IP layer.
Packet order is not expected to change during transfer, but several Packet order is not expected to change during transfer, but several
specific path characteristics can cause order to change. specific path characteristics can cause order to change.
skipping to change at line 106 skipping to change at line 104
Examples are: Examples are:
* When two paths, one with slightly longer transfer time, support a * When two paths, one with slightly longer transfer time, support a
single packet stream or flow, then packets traversing the longer single packet stream or flow, then packets traversing the longer
path may arrive out-of-order. Multiple paths may be used to path may arrive out-of-order. Multiple paths may be used to
achieve load balancing, or may arise from route instability. achieve load balancing, or may arise from route instability.
* To increase capacity, a network device designed with multiple * To increase capacity, a network device designed with multiple
processors serving a single port may reorder as a byproduct. processors serving a single port may reorder as a byproduct.
* A layer 2 retransmission protocol that compensates for an error- * A layer 2 retransmission protocol that compensates for an error-
prone link may cause packet reordering. prone link may cause packet reordering.
Packet Reordering Metric for IPPM October 2003
* If for any reason, the packets in a buffer are not serviced in the * If for any reason, the packets in a buffer are not serviced in the
order of their arrival, their order will change. order of their arrival, their order will change.
* If packets in a flow are assigned to multiple buffers (following * If packets in a flow are assigned to multiple buffers (following
evaluation of traffic characteristics, for example), and the evaluation of traffic characteristics, for example), and the
buffers have different occupations and/or service rates, then buffers have different occupations and/or service rates, then
order will likely change. order will likely change.
When one or more of the above path characteristics are present When one or more of the above path characteristics are present
continuously, then reordering may be present on a steady-state continuously, then reordering may be present on a steady-state
basis. Measurements most easily detect this form of reordering when basis. Measurements most easily detect this form of reordering when
skipping to change at line 159 skipping to change at line 155
+ have relevance to Real-time application performance + have relevance to Real-time application performance
3. A Reordered Packet Singleton Metric 3. A Reordered Packet Singleton Metric
The IPPM framework RFC 2330 [3] describes the notions of singletons, The IPPM framework RFC 2330 [3] describes the notions of singletons,
samples, and statistics. For easy reference: samples, and statistics. For easy reference:
By a 'singleton' metric, we refer to metrics that are, By a 'singleton' metric, we refer to metrics that are,
in a sense, atomic. For example, a single instance of "bulk in a sense, atomic. For example, a single instance of "bulk
throughput capacity" from one host to another might be defined throughput capacity" from one host to another might be defined
Packet Reordering Metric for IPPM October 2003
as a singleton metric, even though the instance involves as a singleton metric, even though the instance involves
measuring the timing of a number of Internet packets. measuring the timing of a number of Internet packets.
The evaluation of packet order requires several supporting concepts. The evaluation of packet order requires several supporting concepts.
The first is a sequence number applied to packets at the source to The first is a sequence number applied to packets at the source to
uniquely identify the order of packet transmission. The sequence uniquely identify the order of packet transmission. The sequence
number may be established by a simple message number, a byte stream number may be established by a simple message number, a byte stream
number, or it may be the actual time when each packet departs from number, or it may be the actual time when each packet departs from
the Source. the Source.
skipping to change at line 214 skipping to change at line 207
transfer. transfer.
3.3 Definition: 3.3 Definition:
The value of Type-P-Reordered is defined as false if s >= NextExp The value of Type-P-Reordered is defined as false if s >= NextExp
(the packet is in-order). In this case, NextExp is set to s+1. (the packet is in-order). In this case, NextExp is set to s+1.
The value of Type-P-Reordered is defined as true if s < NextExp (the The value of Type-P-Reordered is defined as true if s < NextExp (the
packet is reordered). In this case, NextExp value does not change. packet is reordered). In this case, NextExp value does not change.
Packet Reordering Metric for IPPM October 2003
Since the Next Expected value cannot decrease, it provides a non- Since the Next Expected value cannot decrease, it provides a non-
reversing order criterion to identify reordered packets. reversing order criterion to identify reordered packets.
This definition can also be specified in pseudo-code. This definition can also be specified in pseudo-code.
On successful arrival of a packet with sequence number s: On successful arrival of a packet with sequence number s:
if s >= NextExp, /* s is in-order */ if s >= NextExp, /* s is in-order */
then then
NextExp = s + 1; NextExp = s + 1;
Type-P-Reordered = False; Type-P-Reordered = False;
skipping to change at line 261 skipping to change at line 252
establishes the Next Expected value can be evaluated to determine establishes the Next Expected value can be evaluated to determine
whether it is in-order or reordered, based on a previous packet's whether it is in-order or reordered, based on a previous packet's
arrival. In the case where Next Expected is Undefined (because the arrival. In the case where Next Expected is Undefined (because the
arriving packet is the first successful transfer), the packet is arriving packet is the first successful transfer), the packet is
designated in-order. designated in-order.
This metric assumes re-assembly of packet fragments before This metric assumes re-assembly of packet fragments before
evaluation. In principle, it is possible to use the Type-P-Reordered evaluation. In principle, it is possible to use the Type-P-Reordered
metric to evaluate reordering among packet fragments, but each metric to evaluate reordering among packet fragments, but each
fragment must contain source sequence information. fragment must contain source sequence information.
<<<<<<<< Note: Additional considerations will be needed here. See the Appendix on fragment order evaluation for more detail.
If duplicate packets (multiple non-corrupt copies) arrive at the If duplicate packets (multiple non-corrupt copies) arrive at the
destination, they MUST be noted and only the first to arrive is destination, they MUST be noted and only the first to arrive is
considered for further analysis (copies would be declared reordered considered for further analysis (copies would be declared reordered
according to the definition above). This requirement has the same according to the definition above). This requirement has the same
Packet Reordering Metric for IPPM October 2003
storage implications as earlier IPPM metrics, and follows the storage implications as earlier IPPM metrics, and follows the
precedent of RFC 2679. precedent of RFC 2679.
Packets with s > NextExp are a special case of in-order delivery. Packets with s > NextExp are a special case of in-order delivery.
This condition indicates a sequence discontinuity, either because of This condition indicates a sequence discontinuity, either because of
packet loss or reordering. Reordered packets must arrive for the packet loss or reordering. Reordered packets must arrive for the
sequence discontinuity to be defined as a reordering discontinuity sequence discontinuity to be defined as a reordering discontinuity
(see next section). Discontinuities are easiest to detect with (see next section). Discontinuities are easiest to detect with
message numbering or payload byte numbering where payload size is message numbering or payload byte numbering where payload size is
constant (and retransmissions are distinguished), and may be constant (and retransmissions are distinguished), and may be
skipping to change at line 320 skipping to change at line 308
and RFC3432), plus the following: and RFC3432), plus the following:
+ T0, a start time + T0, a start time
+ Tf, an end time + Tf, an end time
+ dT, a waiting time for each packet to arrive + dT, a waiting time for each packet to arrive
4.1.3 Definition: 4.1.3 Definition:
Packet Reordering Metric for IPPM October 2003
For the packets arriving successfully between T0 and Tf+dT, the For the packets arriving successfully between T0 and Tf+dT, the
ratio of reordered packets in the sample is ratio of reordered packets in the sample is
(Total of Reordered packets) / (Total packets received) (Total of Reordered packets) / (Total packets received)
This fraction may be expressed as a percentage (multiply by 100%). This fraction may be expressed as a percentage (multiply by 100%).
Note that in the case of duplicate packets, only the first copy is Note that in the case of duplicate packets, only the first copy is
used. used.
4.2 Reordering Extent 4.2 Reordering Extent
skipping to change at line 367 skipping to change at line 353
arrival index i and source sequence number s[i]. There exists a set arrival index i and source sequence number s[i]. There exists a set
of indexes j (1<=j<i) such that s[j] > s[i]. of indexes j (1<=j<i) such that s[j] > s[i].
4.2.3 Definition: 4.2.3 Definition:
The reordering extent, e, of packet s[i] is defined to be The reordering extent, e, of packet s[i] is defined to be
i-j for the smallest value of j. i-j for the smallest value of j.
Informally, the reordering extent is the maximum distance, in Informally, the reordering extent is the maximum distance, in
packets, from a reordered packet to the earliest packet received packets, from a reordered packet to the earliest packet received
that has a larger sequence number. If a packet is in-order, it's that has a larger sequence number. If a packet is in-order, its
reordering extent is undefined. The first packet to arrive is in- reordering extent is undefined. The first packet to arrive is in-
order by definition, and has undefined reordering extent. order by definition, and has undefined reordering extent.
>>>>>>> Comment on this definition of extent: For some arrival >>>>>>> Comment on this definition of extent: For some arrival
orders, the assignment of a simple position/distance as the orders, the assignment of a simple position/distance as the
reordering extent tends to overestimate the receiver storage needed reordering extent tends to overestimate the receiver storage needed
Packet Reordering Metric for IPPM October 2003
to restore order. We need to weigh the value of adding more to restore order. We need to weigh the value of adding more
complexity in this definition against the accuracy it would provide. complexity in this definition against the accuracy it would provide.
A more accurate and complex procedure to calculate packet storage A more accurate and complex procedure to calculate packet storage
would be to subtract any earlier reordered packets that the receiver would be to subtract any earlier reordered packets that the receiver
could pass on to the upper layers. could pass on to the upper layers.
Those who desire "on-the-fly" calculation must assess whether such a Those who desire "on-the-fly" calculation must assess whether such a
procedure is feasible. procedure is feasible.
4.2.4 Discussion: 4.2.4 Discussion:
skipping to change at line 429 skipping to change at line 412
4.3.1 Metric Name: Type-P-packet-Late-Time-Stream 4.3.1 Metric Name: Type-P-packet-Late-Time-Stream
Metric Parameters: In addition to the parameters defined for Type-P- Metric Parameters: In addition to the parameters defined for Type-P-
Reordered, we specify: Reordered, we specify:
+ DstTime, the time that each packet in the stream arrives at Dst + DstTime, the time that each packet in the stream arrives at Dst
Definition: Lateness in time is calculated using Dst times. When Definition: Lateness in time is calculated using Dst times. When
received packet i is reordered, and has a reordering extent e, then: received packet i is reordered, and has a reordering extent e, then:
Packet Reordering Metric for IPPM October 2003
LateTime(i) = DstTime(i)-DstTime(i-e) LateTime(i) = DstTime(i)-DstTime(i-e)
Alternatively, using similar notation to that of section 4.2, an Alternatively, using similar notation to that of section 4.2, an
equivalent definition is: equivalent definition is:
LateTime(i) = DstTime(i)-DstTime(j), for min{j|1<=j<i} that LateTime(i) = DstTime(i)-DstTime(j), for min{j|1<=j<i} that
satisfies s[j]>s[i], or SrcTime[j]>SrcTime[i]. satisfies s[j]>s[i], or SrcTime[j]>SrcTime[i].
4.3.2 Metric Name: Type-P-packet-Byte-Offset-Stream 4.3.2 Metric Name: Type-P-packet-Byte-Offset-Stream
Metric Parameters: We use the same parameters defined above. Metric Parameters: We use the same parameters defined above.
skipping to change at line 460 skipping to change at line 441
PayloadSize(packet at i-e if in-order)] PayloadSize(packet at i-e if in-order)]
4.3.3 Discussion 4.3.3 Discussion
The offset metrics can help predict whether reordered packets will The offset metrics can help predict whether reordered packets will
be useful in a general receiver buffer system with finite limits. be useful in a general receiver buffer system with finite limits.
The limit may be the number of bytes or packets the buffer can The limit may be the number of bytes or packets the buffer can
store, or the time of storage prior to a cyclic play-out instant (as store, or the time of storage prior to a cyclic play-out instant (as
with de-jitter buffers). with de-jitter buffers).
Note that the One-way IPDV [11] gives the delay variation for a Note that the One-way IPDV [12] gives the delay variation for a
packet w.r.t. the preceding packet in the source sequence. Lateness packet w.r.t. the preceding packet in the source sequence. Lateness
and IPDV give an indication of whether a buffer at Dst has and IPDV give an indication of whether a buffer at Dst has
sufficient storage to accommodate the network's behavior and restore sufficient storage to accommodate the network's behavior and restore
order. When an earlier packet in the Src sequence is lost, IPDV will order. When an earlier packet in the Src sequence is lost, IPDV will
necessarily be undefined for adjacent packets, and Late Time may necessarily be undefined for adjacent packets, and Late Time may
provide the only way to evaluate the usefulness of a packet. provide the only way to evaluate the usefulness of a packet.
In the case of de-jitter buffers, there are circumstances where the In the case of de-jitter buffers, there are circumstances where the
receiver employs loss concealment at the intended play-out time of a receiver employs loss concealment at the intended play-out time of a
late packet. However, if this packet arrives out of order, the Late late packet. However, if this packet arrives out of order, the Late
Time determines whether the packet is still useful. IPDV no longer Time determines whether the packet is still useful. IPDV no longer
applies, because the receiver establishes a new play-out schedule applies, because the receiver establishes a new play-out schedule
with additional buffer delay to accommodate similar events in the with additional buffer delay to accommodate similar events in the
future - this requires very minimal processing. future - this requires very minimal processing.
When packets in the stream have variable sizes, it may be most When packets in the stream have variable sizes, it may be most
useful to characterize Offset in terms of the payload size(s) of useful to characterize Offset in terms of the payload size(s) of
stored packets (using byte stream numbering). stored packets (using byte stream numbering).
Packet Reordering Metric for IPPM October 2003
4.4 Gaps between multiple Reordering Discontinuities 4.4 Gaps between multiple Reordering Discontinuities
4.4.1 Metric Name: 4.4.1 Metric Name:
Type-P-packet-Reordering-Gap-Stream Type-P-packet-Reordering-Gap-Stream
4.4.2 Parameters: 4.4.2 Parameters:
No new parameters. No new parameters.
skipping to change at line 533 skipping to change at line 512
below: below:
Gap(j') = (j') - (j) Gap(j') = (j') - (j)
Otherwise: Otherwise:
The Type-P-packet-Reordering-Gap-Stream for the packet is 0. The Type-P-packet-Reordering-Gap-Stream for the packet is 0.
Gaps may also be expressed in time: Gaps may also be expressed in time:
Packet Reordering Metric for IPPM October 2003
GapTime(j') = DstTime(j') - DstTime(j) GapTime(j') = DstTime(j') - DstTime(j)
4.4.5 Discussion 4.4.5 Discussion
When separate reordering discontinuities can be distinguished, then When separate reordering discontinuities can be distinguished, then
a count may also be reported (along with the discontinuity a count may also be reported (along with the discontinuity
description, such as the number of reordered packets associated with description, such as the number of reordered packets associated with
that discontinuity and their extents and offsets). The Gaps between that discontinuity and their extents and offsets). The Gaps between
a sample's reordering discontinuities may be expressed as a a sample's reordering discontinuities may be expressed as a
histogram, to easily summarize the frequency of various gaps. histogram, to easily summarize the frequency of various gaps.
skipping to change at line 586 skipping to change at line 563
while(packets arrive with sequence number s) while(packets arrive with sequence number s)
{ {
P++; P++;
if (s >= NextExp) /* s is in-order */ if (s >= NextExp) /* s is in-order */
then r++; then r++;
a++; a++;
else /* s is reordered */ else /* s is reordered */
q+= r*r; q+= r*r;
r = 1; r = 1;
Packet Reordering Metric for IPPM October 2003
n++; n++;
} }
4.5.4 Discussion:
Each arrival of a reordered packet yields a new count in the Run Each arrival of a reordered packet yields a new count in the Run
vector. Long runs accompany periods where order was maintained, vector. Long runs accompany periods where order was maintained,
while short runs indicate frequent, or multi-packet reordering. while short runs indicate frequent, or multi-packet reordering.
4.5.4 Discussion:
Each in-order arrival increments the run counter and the accumulator
of in order packets, each reordered packet resets the run counter
after adding it to the accumulator.
The percent of packets in order is 100*a/p
The average in order run length is a/n
The q counter gives an indication of variation of the in order runs
from the average by comparing q/a to a/n ((q/a)/(a/n))
For example for 36 packets with 3 runs of 11 in-order packets we
have:
p = 36
n = 3
a = 33
q = 3 * (11*11) = 363
ave io = 11
q/a = 11
(q/a)/ave = 1.0
For 36 packets with 3 runs, 2 of length 1 and one of length 31
p = 36
n = 3
a = 33
q = 1 + 1 + 961 = 963
ave io = 11
q/a = 29.18
(q/a)/ave = 2.65
5. Metric Related to Receiver Assessment 5. Metric Related to Receiver Assessment
5.1 A TCP-Relevant Metric 5.1 A TCP-Relevant Metric
5.1.1 Metric Name: 5.1.1 Metric Name:
Type-P-packet-n-Reordering-Stream Type-P-packet-n-Reordering-Stream
5.1.2 Parameter Notation: 5.1.2 Parameter Notation:
skipping to change at line 642 skipping to change at line 646
probably report it for several values of n, such as 1, 2, 3, 4 -- probably report it for several values of n, such as 1, 2, 3, 4 --
and maybe a few more consecutive values). and maybe a few more consecutive values).
This definition does not assign an n to all reordered packets as This definition does not assign an n to all reordered packets as
defined by the singleton metric, in particular when blocks of defined by the singleton metric, in particular when blocks of
successive packets are reordered. (In the arrival sequence successive packets are reordered. (In the arrival sequence
s={1,2,3,7,8,9,4,5,6}, packets 4, 5, and 6 are reordered, but only 4 s={1,2,3,7,8,9,4,5,6}, packets 4, 5, and 6 are reordered, but only 4
is n-reordered, with n=3.) is n-reordered, with n=3.)
Definition 2: The degree of n-reordering of the sample is m/l. Definition 2: The degree of n-reordering of the sample is m/l.
Packet Reordering Metric for IPPM October 2003
Definition 3: The degree of "monotonic reordering" of the sample is Definition 3: The degree of "monotonic reordering" of the sample is
its degree of 1-reordering. its degree of 1-reordering.
Definition 4: A sample is said to have no reordering if its degree Definition 4: A sample is said to have no reordering if its degree
of n-reordering is 0. of n-reordering is 0.
5.1.4 Discussion: 5.1.4 Discussion:
The degree of n-reordering may be expressed as a percentage, in The degree of n-reordering may be expressed as a percentage, in
which case the number from definition 2 is multiplied by 100. which case the number from definition 2 is multiplied by 100.
skipping to change at line 683 skipping to change at line 685
A sample's n-reordering may be expressed as a histogram, to A sample's n-reordering may be expressed as a histogram, to
summarize the frequency for each value of n. summarize the frequency for each value of n.
We note that the definition of n-reordering cannot predict the exact We note that the definition of n-reordering cannot predict the exact
number of packets unnecessarily retransmitted by a TCP sender under number of packets unnecessarily retransmitted by a TCP sender under
some circumstances, such as cases with closely-spaced reordered some circumstances, such as cases with closely-spaced reordered
singletons. The definition is less complicated than a TCP singletons. The definition is less complicated than a TCP
implementation where both time and position influence the sender's implementation where both time and position influence the sender's
behavior. behavior.
A packet's n-reordering is sometimes equal to it's reordering A packet's n-reordering is sometimes equal to its reordering extent,
extent, e. n-reordering is different in the following ways: e. n-reordering is different in the following ways:
1. n is a count of *adjacent* early packets. 1. n is a count of *adjacent* early packets.
2. Some reordered packets may not be n-reordered, but will have e 2. Some reordered packets may not be n-reordered, but will have e
(see the examples). (see the examples).
6. Measurement Issues 6. Measurement Issues
The results of tests will be dependent on the time interval between The results of tests will be dependent on the time interval between
measurement packets (both at the Src, and during transport where measurement packets (both at the Src, and during transport where
Packet Reordering Metric for IPPM October 2003
spacing may change). Clearly, packets launched infrequently (e.g., spacing may change). Clearly, packets launched infrequently (e.g.,
1 per 10 seconds) are unlikely to be reordered. 1 per 10 seconds) are unlikely to be reordered.
Test streams may prefer to use a periodic sending interval so that a Test streams may prefer to use a periodic sending interval so that a
known temporal bias is maintained, also bringing simplified results known temporal bias is maintained, also bringing simplified results
analysis (as described in RFC 3432 [12]). In this case, the periodic analysis (as described in RFC 3432 [13]). In this case, the periodic
sending interval should be chosen to reproduce the closest Src sending interval should be chosen to reproduce the closest Src
packet spacing expected. Of course, packet spacing is likely to vary packet spacing expected. Of course, packet spacing is likely to vary
as the stream traverses the test path. as the stream traverses the test path.
<<<<Ed.Note: Is this sufficient? It is a very important <<<<Ed.Note: Is this sufficient? It is a very important
consideration. consideration.
The non-reversing order criterion and all metrics described above The non-reversing order criterion and all metrics described above
remain valid and useful when a stream of packets experiences packet remain valid and useful when a stream of packets experiences packet
loss, or both loss and reordering. In other words, losses alone do loss, or both loss and reordering. In other words, losses alone do
not cause subsequent packets to be declared reordered. not cause subsequent packets to be declared reordered.
skipping to change at line 750 skipping to change at line 749
1. There must be a test to detect if a roll-over has occurred. It 1. There must be a test to detect if a roll-over has occurred. It
would be nearly impossible for the sequence numbers of successive would be nearly impossible for the sequence numbers of successive
packets to jump by more than half the total range, so these large packets to jump by more than half the total range, so these large
discontinuities are designated as roll-over. discontinuities are designated as roll-over.
2. All sequence numbers used in computations are represented in a 2. All sequence numbers used in computations are represented in a
sufficiently large precision. The numbers have a correction applied sufficiently large precision. The numbers have a correction applied
(equivalent to adding a significant digit) whenever roll-over is (equivalent to adding a significant digit) whenever roll-over is
detected. detected.
Packet Reordering Metric for IPPM October 2003
3. Reordered packets coincident with sequence numbers reaching end- 3. Reordered packets coincident with sequence numbers reaching end-
of-range must also be detected for proper application of correction of-range must also be detected for proper application of correction
factor. factor.
In practice, there may be limited ability to determine reordering In practice, there may be limited ability to determine reordering
extent, because the storage for previous packets may be limited. extent, because the storage for previous packets may be limited.
Saving only packets that indicate discontinuities (and their arrival Saving only packets that indicate discontinuities (and their arrival
positions) will reduce storage volume. When discarding all stream positions) will reduce storage volume. When discarding all stream
information beyond a threshold packet count, the reordering extent information beyond a threshold packet count, the reordering extent
or degree of n-reordering may need to be expressed as greater than or degree of n-reordering may need to be expressed as greater than
skipping to change at line 803 skipping to change at line 800
5 4 80 148 68 -82 4 5 4 80 148 68 -82 4
6 6 100 168 68 0 5 6 6 100 168 68 0 5
7 7 120 188 68 0 6 7 7 120 188 68 0 6
8 8 140 208 68 0 7 8 8 140 208 68 0 7
4 9 60 210 150 82 8 400 62 4 9 60 210 150 82 8 400 62
9 9 160 228 68 0 9 9 9 160 228 68 0 9
10 10 180 248 68 0 10 10 10 180 248 68 0 10
Each column gives the following information: Each column gives the following information:
Packet Reordering Metric for IPPM October 2003
s Packet sequence number at the Source. s Packet sequence number at the Source.
NextExp The value of NextExp when the packet arrived(before NextExp The value of NextExp when the packet arrived(before
update). update).
SrcTime Packet time stamp at the Source, ms. SrcTime Packet time stamp at the Source, ms.
DstTime Packet time stamp at the Destination, ms. DstTime Packet time stamp at the Destination, ms.
Delay 1-way delay of the packet, ms. Delay 1-way delay of the packet, ms.
IPDV IP Packet Delay Variation, ms IPDV IP Packet Delay Variation, ms
IPDV = Delay(SrcNum)-Delay(SrcNum-1) IPDV = Delay(SrcNum)-Delay(SrcNum-1)
DstOrder Order in which the packet arrived at the Destination. DstOrder Order in which the packet arrived at the Destination.
Byte Offset The Byte Offset of a reordered packet, in bytes. Byte Offset The Byte Offset of a reordered packet, in bytes.
skipping to change at line 856 skipping to change at line 851
Table 2 Example with Packets 5 and 6 Reordered, Table 2 Example with Packets 5 and 6 Reordered,
Sending order(s @Src): 1,2,3,4,5,6,7,8,9,10 Sending order(s @Src): 1,2,3,4,5,6,7,8,9,10
s Src Dst Dst Byte Late s Src Dst Dst Byte Late
@Dst NextExp Time Time Delay IPDV Order Offset Time @Dst NextExp Time Time Delay IPDV Order Offset Time
1 1 0 68 68 1 1 1 0 68 68 1
2 2 20 88 68 0 2 2 2 20 88 68 0 2
3 3 40 108 68 0 3 3 3 40 108 68 0 3
4 4 60 128 68 0 4 4 4 60 128 68 0 4
7 5 120 188 68 -22 5 7 5 120 188 68 -22 5
5 8 80 189 109 41 6 100 1 5 8 80 189 109 41 6 100 1
Packet Reordering Metric for IPPM October 2003
6 8 100 190 90 -19 7 100 2 6 8 100 190 90 -19 7 100 2
8 8 140 208 68 0 8 8 8 140 208 68 0 8
9 9 160 228 68 0 9 9 9 160 228 68 0 9
10 10 180 248 68 0 10 10 10 180 248 68 0 10
Table 2 shows a case where Packets 5 and 6 arrive just behind Packet Table 2 shows a case where Packets 5 and 6 arrive just behind Packet
7, so both 5 and 6 are reordered. The Late times (189-188=1, 190- 7, so both 5 and 6 are reordered. The Late times (189-188=1, 190-
188=2) are small. 188=2) are small.
Using the notation <s[1], ..., s[i], ..., s[l]>, the received Using the notation <s[1], ..., s[i], ..., s[l]>, the received
skipping to change at line 909 skipping to change at line 902
Table 3 Example with Packets 4, 5, and 6 reordered Table 3 Example with Packets 4, 5, and 6 reordered
Sending order(s @Src): 1,2,3,4,5,6,7,8,9,10,11 Sending order(s @Src): 1,2,3,4,5,6,7,8,9,10,11
s Src Dst Dst Byte Late s Src Dst Dst Byte Late
@Dst NextExp Time Time Delay IPDV Order Offset Time @Dst NextExp Time Time Delay IPDV Order Offset Time
1 1 0 68 68 1 1 1 0 68 68 1
2 2 20 88 68 0 2 2 2 20 88 68 0 2
3 3 40 108 68 0 3 3 3 40 108 68 0 3
7 4 120 188 68 -88 4 7 4 120 188 68 -88 4
8 8 140 208 68 0 5 8 8 140 208 68 0 5
Packet Reordering Metric for IPPM October 2003
9 9 160 228 68 0 6 9 9 160 228 68 0 6
10 10 180 248 68 0 7 10 10 180 248 68 0 7
4 11 60 250 190 122 8 400 62 4 11 60 250 190 122 8 400 62
5 11 80 252 172 -18 9 400 64 5 11 80 252 172 -18 9 400 64
6 11 100 256 156 -16 10 400 68 6 11 100 256 156 -16 10 400 68
11 11 200 268 68 0 11 11 11 200 268 68 0 11
The case in Table 3 is where three packets in sequence have long The case in Table 3 is where three packets in sequence have long
transit times (Packets with s = 4,5,and 6). Delay, Late time, and transit times (Packets with s = 4,5,and 6). Delay, Late time, and
Byte Offset capture this very well, and indicate variation in Byte Offset capture this very well, and indicate variation in
skipping to change at line 964 skipping to change at line 953
a single Packet (4) with a sufficient degree of reordering to result a single Packet (4) with a sufficient degree of reordering to result
in one unnecessary packet retransmission by the New Reno TCP sender. in one unnecessary packet retransmission by the New Reno TCP sender.
Also, the reordered arrival of Packets 5 and 6 will allow the Also, the reordered arrival of Packets 5 and 6 will allow the
receiver process to pass Packets 7 through 10 up the protocol stack receiver process to pass Packets 7 through 10 up the protocol stack
(the singleton metric indicates 5 and 6 are reordered, and they are (the singleton metric indicates 5 and 6 are reordered, and they are
all associated with a single reordering discontinuity). all associated with a single reordering discontinuity).
Table 4 Example with Packets Multiple Reordering Discontinuities Table 4 Example with Packets Multiple Reordering Discontinuities
Sending order(s @Src): 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16 Sending order(s @Src): 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16
Packet Reordering Metric for IPPM October 2003
Discontinuity Discontinuity Discontinuity Discontinuity
|---------Gap---------| |---------Gap---------|
s = 1, 2, 3, 6, 7, 4, 5, 8, 9, 10, 12, 13, 11, 14, 15, 16 s = 1, 2, 3, 6, 7, 4, 5, 8, 9, 10, 12, 13, 11, 14, 15, 16
i = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 i = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
r = 1, 2, 3, 4, 5, 1, 1, 2, 3, 4, 5, 6, 1, 2, 3, 4, ... r = 1, 2, 3, 4, 5, 1, 1, 2, 3, 4, 5, 6, 1, 2, 3, 4, ...
n = 1 2 3 n = 1 2 3
end r counts = 5 1 6 end r counts = 5 1 6
Packet 4 has extent e=2, Packet 5 has extent e=3, and Packet 11 has Packet 4 has extent e=2, Packet 5 has extent e=3, and Packet 11 has
skipping to change at line 1016 skipping to change at line 1002
8.2 User data confidentiality 8.2 User data confidentiality
Active use of this method generates packets for a sample, rather Active use of this method generates packets for a sample, rather
than taking samples based on user data, and does not threaten user than taking samples based on user data, and does not threaten user
data confidentiality. Passive measurement must restrict attention to data confidentiality. Passive measurement must restrict attention to
the headers of interest. Since user payloads may be temporarily the headers of interest. Since user payloads may be temporarily
stored for length analysis, suitable precautions MUST be taken to stored for length analysis, suitable precautions MUST be taken to
keep this information safe and confidential. keep this information safe and confidential.
Packet Reordering Metric for IPPM October 2003
8.3 Interference with the metric 8.3 Interference with the metric
It may be possible to identify that a certain packet or stream of It may be possible to identify that a certain packet or stream of
packets is part of a sample. With that knowledge at Dst and/or the packets is part of a sample. With that knowledge at Dst and/or the
intervening networks, it is possible to change the processing of the intervening networks, it is possible to change the processing of the
packets (e.g. increasing or decreasing delay) that may distort the packets (e.g. increasing or decreasing delay) that may distort the
measured performance. It may also be possible to generate measured performance. It may also be possible to generate
additional packets that appear to be part of the sample metric. additional packets that appear to be part of the sample metric.
These additional packets are likely to perturb the results of the These additional packets are likely to perturb the results of the
sample measurement. sample measurement.
skipping to change at line 1062 skipping to change at line 1046
5 Postel, J., "Transmission Control Protocol", STD 7, RFC 793, 5 Postel, J., "Transmission Control Protocol", STD 7, RFC 793,
September 1981. September 1981.
Obtain via: http://www.rfc-editor.org/rfc/rfc793.txt Obtain via: http://www.rfc-editor.org/rfc/rfc793.txt
6 L.Ciavattone and A.Morton, "Out-of-Sequence Packet Parameter 6 L.Ciavattone and A.Morton, "Out-of-Sequence Packet Parameter
Definition (for Y.1540)", Contribution number T1A1.3/2000-047, Definition (for Y.1540)", Contribution number T1A1.3/2000-047,
October 30, 2000. ftp://ftp.t1.org/pub/t1a1/2000-A13/0a130470.doc October 30, 2000. ftp://ftp.t1.org/pub/t1a1/2000-A13/0a130470.doc
7 J.C.R.Bennett, C.Partridge, and N.Shectman, "Packet Reordering is 7 J.C.R.Bennett, C.Partridge, and N.Shectman, "Packet Reordering is
Not Pathological Network Behavior," IEEE/ACM Transactions on Not Pathological Network Behavior," IEEE/ACM Transactions on
Neteworking, vol.7, no.6, pp.789-798, December 1999. Networking, vol.7, no.6, pp.789-798, December 1999.
8 D.Loguinov and H.Radha, "Measurement Study of Low-bitrate 8 D.Loguinov and H.Radha, "Measurement Study of Low-bitrate
Internet Video Streaming", Proceedings of the ACM SIGCOMM Internet Video Streaming", Proceedings of the ACM SIGCOMM
Internet Measurement Workshop 2001 November 1-2, 2001, San Internet Measurement Workshop 2001 November 1-2, 2001, San
Francisco, USA. Francisco, USA.
Packet Reordering Metric for IPPM October 2003
9 J.Bellardo and S.Savage, "Measuring Packet Reordering," 9 J.Bellardo and S.Savage, "Measuring Packet Reordering,"
Proceedings of the ACM SIGCOMM Internet Measurement Workshop Proceedings of the ACM SIGCOMM Internet Measurement Workshop
2002, November 6-8, Marseille, France. 2002, November 6-8, Marseille, France.
10 S.Jaiswal et al., "Measurement and Classification of Out-of- 10 S.Jaiswal et al., "Measurement and Classification of Out-of-
Sequence Packets in a Tier-1 IP Backbone," Proceedings of the ACM Sequence Packets in a Tier-1 IP Backbone," Proceedings of the ACM
SIGCOMM Internet Measurement Workshop 2002, November 6-8, SIGCOMM Internet Measurement Workshop 2002, November 6-8,
Marseille, France. Marseille, France.
11 Demichelis, C., and Chimento, P., "IP Packet Delay Variation 11 L.Ciavattone, A.Morton, and G.Ramachandran, "Standardized Active
Measurements on a Tier 1 IP Backbone," IEEE Communications Mag.,
pp 90-97, June 2003.
12 Demichelis, C., and Chimento, P., "IP Packet Delay Variation
Metric for IP Performance Metrics (IPPM)", RFC 3393, November Metric for IP Performance Metrics (IPPM)", RFC 3393, November
2002. 2002.
12 Raisanen, V., Grotefeld, G., and Morton, A., "Network performance 13 Raisanen, V., Grotefeld, G., and Morton, A., "Network performance
measurement with periodic streams", RFC 3432, November 2002. measurement with periodic streams", RFC 3432, November 2002.
12. Acknowledgments 12. Acknowledgments
The authors would like to acknowledge many helpful discussions with The authors would like to acknowledge many helpful discussions with
Matt Mathis, Jon Bennett, and Matt Zekauskas. We gratefully Matt Mathis, Jon Bennett, and Matt Zekauskas. We gratefully
acknowledge the foundation laid by the authors of the IP performance acknowledge the foundation laid by the authors of the IP performance
Framework [3]. Framework [3].
13. Appendix A (informative) 13. Appendix A (informative)
skipping to change at line 1122 skipping to change at line 1107
* the sequence numbers come from stdin; in an actual test, they * the sequence numbers come from stdin; in an actual test, they
would come would come
* from the network. * from the network.
*/ */
int int
read_sequence_number() read_sequence_number()
{ {
int res, rc; int res, rc;
rc = scanf("%d\n", &res); rc = scanf("%d\n", &res);
if (rc == 1) return res; if (rc == 1) return res;
Packet Reordering Metric for IPPM October 2003
else return EOF; else return EOF;
} }
int int
main() main()
{ {
int m[MAX_N]; /* We have m[j-1] == number int m[MAX_N]; /* We have m[j-1] == number
of of
* j-reordered packets. */ * j-reordered packets. */
int ring[MAX_N]; /* Last sequence numbers int ring[MAX_N]; /* Last sequence numbers
skipping to change at line 1175 skipping to change at line 1157
#define min(a, b) ((a) < (b)? (a): (b)) #define min(a, b) ((a) < (b)? (a): (b))
#define loop(x) ((x) >= 0? x: x + MAX_N) #define loop(x) ((x) >= 0? x: x + MAX_N)
/* Global counters */ /* Global counters */
int receive_packets=0; /* number of recieved */ int receive_packets=0; /* number of recieved */
int reorder_packets=0; /* number of reordered packets */ int reorder_packets=0; /* number of reordered packets */
/* function to test if current packet has been reordered /* function to test if current packet has been reordered
* returns 0 = not reordered * returns 0 = not reordered
* 1 = reordered * 1 = reordered
Packet Reordering Metric for IPPM October 2003
*/ */
int testorder1(int seqnum) // Al int testorder1(int seqnum) // Al
{ {
static int NextExp = 1; static int NextExp = 1;
int iReturn = 0; int iReturn = 0;
if (seqnum >= NextExp) { if (seqnum >= NextExp) {
NextExp = seqnum+1; NextExp = seqnum+1;
} else { } else {
iReturn = 1; iReturn = 1;
skipping to change at line 1224 skipping to change at line 1203
for (i=1; i< argc; i++) { for (i=1; i< argc; i++) {
receive_packets++; receive_packets++;
packet = atoi(argv[i]); packet = atoi(argv[i]);
reorder_packets += testorder2(packet); reorder_packets += testorder2(packet);
} }
printf("Received packets = %d, Reordered packets = %d\n", printf("Received packets = %d, Reordered packets = %d\n",
receive_packets, reorder_packets); receive_packets, reorder_packets);
exit(0); exit(0);
} }
13. Author's Addresses 13. Appendix on fragment order evaluation
Section 3 stated that fragment re-assembly is assumed prior to order
evaluation, but that similar procedures could be applied prior to
re-assembly. This appendix gives definitions and procedures to
identify reordering in a packet stream that includes fragmentation.
The Metric retains the same name, Type-P-Reordered, but additional
parameters are required.
This Appendix assumes that the device that divides a packet into
fragments send them according to ascending fragment offset. Early
Linux OS sent fragments in reverse order, so this possibility is
worth checking.
13.1 Additional Metric Parameters:
+ MoreFrag, the state of the More Fragments Flag in the IP header
+ FragOffset, the offset from the beginning of a fragmented packet,
in 8 octet units (also from the IP header).
+ FragSeq#, the sequence number from the IP header of a fragmented
packet currently under evaluation for reordering. When set to
zero, fragment evaluation is not in progress.
+ NextExpFrag, the Next Expected Fragment Offset at the
Destination, in 8 octet units. Set to zero when fragment
evaluation is not in progress.
The packet sequence number, s, is assumed to be the same as the IP
header sequence number. Also, the value of NextExp does not change
with the in-order arrival of fragments. NextExp is only updated when
a last fragment or a complete packet arrives.
Note that packets with missing fragments MUST be declared lost, and
the Reordering status of any fragments that do arrive MUST be
excluded from sample metrics.
13.2 Definition:
The value of Type-P-Reordered is typically false (the packet is in-
order) when
* the sequence number s >= NextExp,
* AND the fragment offset FragOffset >= NextExpFrag
However, it more efficient to define reordered conditions exactly,
and designate Type-P-Reordered as False otherwise.
The value of Type-P-Reordered is defined as True (the packet is
reordered) under the conditions below. In these cases, the NextExp
value does not change.
Case 1: if s < NextExp
Case 2: if s < FragSeq#
Case 3: if s>= NextExp AND s = FragSeq# AND FragOffset < NextExpFrag
This definition can also be illustrated in pseudo-code. A draft
version of the code follows, and some simplification may be
possible. A challenging aspect surrounds the housekeeping for the
new parameters.
NextExp=0;
NextExpFrag=0;
FragSeq#=0;
while(packets arrive with s, MoreFrag, FragOffset)
{
if (s>=NextExp AND MoreFrag==0 AND s>=FragSeq#){
/* a normal packet or last frag of an in-order packet arrived
*/
NextExp = s+1;
FragSeq# = 0;
NextExpFrag = 0;
Reordering = False;
}
if (s>=NextExp AND MoreFrag==1 AND s>FragSeq#>=0){
/* a fragment of a new packet arrived, possibly with a
higher sequence number than the current fragmented packet */
FragSeq# = s;
NextExpFrag = FragOffset+1;
Reordering = False;
}
if (s>=NextExp AND MoreFrag==1 AND s==FragSeq#){
/* a fragment of the "current packet s" arrived */
if (FragOffset >= NextExpFrag){
NextExpFrag = FragOffset+1;
Reordering = False;
}
else{
Reordering = True; /* fragment reordered */
}
}
if (s>=NextExp AND MoreFrag==1 AND s < FragSeq#){
/* case where a late fragment arrived */
Reordering = True;
}
else { /* when s < NextExp, or MoreFrag==0 AND s < FragSeq# */
Reordering = True;
}
}
A working version of the code would include a check to ensure that
all fragments of a packet arrive before using the Reordered status
further, such as in sample metrics.
13.3 Notes on Sample Metrics
All fragments with the same Source Sequence Number are assigned the
same Source Time.
Evaluation with byte stream numbering may be simplified if the
fragment offset is simply added to the SourceByte of the first
packet (with fragment offset = 0), keeping the 8 octet units of the
offset in mind.
14. Author's Addresses
Al Morton Al Morton
AT&T Labs AT&T Labs
Room D3 - 3C06 Room D3 - 3C06
200 Laurel Ave. South 200 Laurel Ave. South
Packet Reordering Metric for IPPM October 2003
Middletown, NJ 07748 USA Middletown, NJ 07748 USA
Phone +1 732 420 1571 Phone +1 732 420 1571
<acmorton@att.com> EMail: <acmorton@att.com>
Len Ciavattone Len Ciavattone
AT&T Labs AT&T Labs
Room C4 - 2B29 Room C4 - 2B29
200 Laurel Ave. South 200 Laurel Ave. South
Middletown, NJ 07748 USA Middletown, NJ 07748 USA
Phone +1 732 420 1239 Phone +1 732 420 1239
<lencia@att.com> EMail: <lencia@att.com>
Gomathi Ramachandran Gomathi Ramachandran
AT&T Labs AT&T Labs
Room C4 - 3D22 Room C4 - 3D22
200 Laurel Ave. South 200 Laurel Ave. South
Middletown, NJ 07748 USA Middletown, NJ 07748 USA
Phone +1 732 420 2353 Phone +1 732 420 2353
<gomathi@att.com> EMail: <gomathi@att.com>
Stanislav Shalunov Stanislav Shalunov
Internet2 Internet2
200 Business Park Drive, Suite 307 200 Business Park Drive, Suite 307
Armonk, NY 10504 Armonk, NY 10504
Phone: + 1 914 765 1182 Phone: + 1 914 765 1182
EMail: <shalunov@internet2.edu> EMail: <shalunov@internet2.edu>
Jerry Perser Jerry Perser
Spirent Communications Consultant
26750 Agoura Road
Calabasas, CA 91302 USA Calabasas, CA 91302 USA
Phone: + 1 818 676 2300 Phone: + 1
EMail: <jerry.perser@spirentcom.com> EMail: <jerry@perser.org>
Full Copyright Statement Full Copyright Statement
"Copyright (C) The Internet Society (date). All Rights Reserved. "Copyright (C) The Internet Society (date). All Rights Reserved.
This document and translations of it may be copied and furnished to This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it others, and derivative works that comment on or otherwise explain it
or assist in its implmentation may be prepared, copied, published or assist in its implmentation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than followed, or as required to translate it into languages other than
English. English.
Packet Reordering Metric for IPPM October 2003
The limited permissions granted above are perpetual and will not be The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns. revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/