draft-ietf-rmcat-sbd-05.txt   draft-ietf-rmcat-sbd-06.txt 
RTP Media Congestion Avoidance Techniques D. Hayes, Ed. RTP Media Congestion Avoidance Techniques D. Hayes, Ed.
Internet-Draft S. Ferlin Internet-Draft S. Ferlin
Intended status: Experimental Simula Research Laboratory Intended status: Experimental Simula Research Laboratory
Expires: March 21, 2017 M. Welzl Expires: August 19, 2017 M. Welzl
K. Hiorth K. Hiorth
University of Oslo University of Oslo
September 17, 2016 February 15, 2017
Shared Bottleneck Detection for Coupled Congestion Control for RTP Shared Bottleneck Detection for Coupled Congestion Control for RTP
Media. Media.
draft-ietf-rmcat-sbd-05 draft-ietf-rmcat-sbd-06
Abstract Abstract
This document describes a mechanism to detect whether end-to-end data This document describes a mechanism to detect whether end-to-end data
flows share a common bottleneck. It relies on summary statistics flows share a common bottleneck. It relies on summary statistics
that are calculated based on continuous measurements and used as that are calculated based on continuous measurements and used as
input to a grouping algorithm that runs wherever the knowledge is input to a grouping algorithm that runs wherever the knowledge is
needed. This mechanism complements the coupled congestion control needed. This mechanism complements the coupled congestion control
mechanism in draft-ietf-rmcat-coupled-cc. mechanism in draft-ietf-rmcat-coupled-cc.
skipping to change at page 1, line 39 skipping to change at page 1, line 39
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on March 21, 2017. This Internet-Draft will expire on August 19, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. The signals . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. The basic mechanism . . . . . . . . . . . . . . . . . . . 3
1.1.1. Packet Loss . . . . . . . . . . . . . . . . . . . . . 3 1.2. The signals . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.2. Packet Delay . . . . . . . . . . . . . . . . . . . . 3 1.2.1. Packet loss . . . . . . . . . . . . . . . . . . . . . 3
1.1.3. Path Lag . . . . . . . . . . . . . . . . . . . . . . 4 1.2.2. Packet delay . . . . . . . . . . . . . . . . . . . . 3
1.2.3. Path lag . . . . . . . . . . . . . . . . . . . . . . 4
2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1. Parameters and their Effect . . . . . . . . . . . . . . . 7 2.1. Parameters and their effect . . . . . . . . . . . . . . . 7
2.2. Recommended Parameter Values . . . . . . . . . . . . . . 8 2.2. Recommended parameter values . . . . . . . . . . . . . . 8
3. Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3. Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.1. SBD feedback requirements . . . . . . . . . . . . . . . . 9 3.1. SBD feedback requirements . . . . . . . . . . . . . . . . 9
3.1.1. Feedback when all the logic is placed at the sender . 9 3.1.1. Feedback when all the logic is placed at the sender . 9
3.1.2. Feedback when the statistics are calculated at the 3.1.2. Feedback when the statistics are calculated at the
receiver and SBD performed at the sender . . . . . . 10 receiver and SBD performed at the sender . . . . . . 10
3.1.3. Feedback when bottlenecks can be determined at both 3.1.3. Feedback when bottlenecks can be determined at both
senders and receivers . . . . . . . . . . . . . . . . 10 senders and receivers . . . . . . . . . . . . . . . . 10
3.2. Key metrics and their calculation . . . . . . . . . . . . 11 3.2. Key metrics and their calculation . . . . . . . . . . . . 11
3.2.1. Mean delay . . . . . . . . . . . . . . . . . . . . . 11 3.2.1. Mean delay . . . . . . . . . . . . . . . . . . . . . 11
3.2.2. Skewness Estimate . . . . . . . . . . . . . . . . . . 11 3.2.2. Skewness estimate . . . . . . . . . . . . . . . . . . 11
3.2.3. Variability Estimate . . . . . . . . . . . . . . . . 12 3.2.3. Variability estimate . . . . . . . . . . . . . . . . 12
3.2.4. Oscillation Estimate . . . . . . . . . . . . . . . . 12 3.2.4. Oscillation estimate . . . . . . . . . . . . . . . . 12
3.2.5. Packet loss . . . . . . . . . . . . . . . . . . . . . 13 3.2.5. Packet loss . . . . . . . . . . . . . . . . . . . . . 13
3.3. Flow Grouping . . . . . . . . . . . . . . . . . . . . . . 13 3.3. Flow Grouping . . . . . . . . . . . . . . . . . . . . . . 13
3.3.1. Flow Grouping Algorithm . . . . . . . . . . . . . . . 13 3.3.1. Flow grouping algorithm . . . . . . . . . . . . . . . 13
3.3.2. Using the flow group signal . . . . . . . . . . . . . 15 3.3.2. Using the flow group signal . . . . . . . . . . . . . 15
3.4. Removing Noise from the Estimates . . . . . . . . . . . . 15 4. Enhancements to the basic SBD algorithm . . . . . . . . . . . 15
3.4.1. Oscillation noise . . . . . . . . . . . . . . . . . . 15 4.1. Reducing lag and improving responsiveness . . . . . . . . 15
3.4.2. Clock skew . . . . . . . . . . . . . . . . . . . . . 15 4.1.1. Improving the response of the skewness estimate . . . 16
3.5. Reducing lag and Improving Responsiveness . . . . . . . . 16 4.1.2. Improving the response of the variability estimate . 18
3.5.1. Improving the response of the skewness estimate . . . 16 4.2. Removing oscillation noise . . . . . . . . . . . . . . . 18
3.5.2. Improving the response of the variability estimate . 19 5. Measuring OWD . . . . . . . . . . . . . . . . . . . . . . . . 19
4. Measuring OWD . . . . . . . . . . . . . . . . . . . . . . . . 19 5.1. Time stamp resolution . . . . . . . . . . . . . . . . . . 19
4.1. Time stamp resolution . . . . . . . . . . . . . . . . . . 19 5.2. Clock skew . . . . . . . . . . . . . . . . . . . . . . . 19
5. Expected feedback from experiments . . . . . . . . . . . . . 20 6. Expected feedback from experiments . . . . . . . . . . . . . 19
6. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 20 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 20
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20
8. Security Considerations . . . . . . . . . . . . . . . . . . . 20 9. Security Considerations . . . . . . . . . . . . . . . . . . . 20
9. Change history . . . . . . . . . . . . . . . . . . . . . . . 20 10. Change history . . . . . . . . . . . . . . . . . . . . . . . 20
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 21
10.1. Normative References . . . . . . . . . . . . . . . . . . 21 11.1. Normative References . . . . . . . . . . . . . . . . . . 21
10.2. Informative References . . . . . . . . . . . . . . . . . 21 11.2. Informative References . . . . . . . . . . . . . . . . . 21
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22
1. Introduction 1. Introduction
In the Internet, it is not normally known if flows (e.g., TCP In the Internet, it is not normally known if flows (e.g., TCP
connections or UDP data streams) traverse the same bottlenecks. Even connections or UDP data streams) traverse the same bottlenecks. Even
flows that have the same sender and receiver may take different paths flows that have the same sender and receiver may take different paths
and may or may not share a bottleneck. Flows that share a bottleneck and may or may not share a bottleneck. Flows that share a bottleneck
link usually compete with one another for their share of the link usually compete with one another for their share of the
capacity. This competition has the potential to increase packet loss capacity. This competition has the potential to increase packet loss
and delays. This is especially relevant for interactive applications and delays. This is especially relevant for interactive applications
that communicate simultaneously with multiple peers (such as multi- that communicate simultaneously with multiple peers (such as multi-
party video). For RTP media applications such as RTCWEB, party video). For RTP media applications such as RTCWEB,
[I-D.ietf-rmcat-coupled-cc] describes a scheme that combines the [I-D.ietf-rmcat-coupled-cc] describes a scheme that combines the
congestion controllers of flows in order to honor their priorities congestion controllers of flows in order to honor their priorities
and avoid unnecessary packet loss as well as delay. This mechanism and avoid unnecessary packet loss as well as delay. This mechanism
relies on some form of Shared Bottleneck Detection (SBD); here, a relies on some form of Shared Bottleneck Detection (SBD); here, a
measurement-based SBD approach is described. measurement-based SBD approach is described.
1.1. The signals 1.1. The basic mechanism
The mechanism groups flows that have similar statistical
characteristics together. Section 3.3.1 describes a simple method
for achieving this, however, a major part of this draft is concerned
with collecting suitable statistics for this purpose.
1.2. The signals
The current Internet is unable to explicitly inform endpoints as to The current Internet is unable to explicitly inform endpoints as to
which flows share bottlenecks, so endpoints need to infer this from which flows share bottlenecks, so endpoints need to infer this from
whatever information is available to them. The mechanism described whatever information is available to them. The mechanism described
here currently utilizes packet loss and packet delay, but is not here currently utilizes packet loss and packet delay, but is not
restricted to these. As ECN becomes more prevalent it too will restricted to these. As ECN becomes more prevalent it too will
become a valuable base signal. become a valuable base signal.
1.1.1. Packet Loss 1.2.1. Packet loss
Packet loss is often a relatively rare signal. Therefore, on its own Packet loss is often a relatively rare signal. Therefore, on its own
it is of limited use for SBD, however, it is a valuable supplementary it is of limited use for SBD, however, it is a valuable supplementary
measure when it is more prevalent. measure when it is more prevalent.
1.1.2. Packet Delay 1.2.2. Packet delay
End-to-end delay measurements include noise from every device along End-to-end delay measurements include noise from every device along
the path in addition to the delay perturbation at the bottleneck the path in addition to the delay perturbation at the bottleneck
device. The noise is often significantly increased if the round-trip device. The noise is often significantly increased if the round-trip
time is used. The cleanest signal is obtained by using One-Way-Delay time is used. The cleanest signal is obtained by using One-Way-Delay
(OWD). (OWD).
Measuring absolute OWD is difficult since it requires both the sender Measuring absolute OWD is difficult since it requires both the sender
and receiver clocks to be synchronized. However, since the and receiver clocks to be synchronized. However, since the
statistics being collected are relative to the mean OWD, a relative statistics being collected are relative to the mean OWD, a relative
OWD measurement is sufficient. Clock skew is not usually significant OWD measurement is sufficient. Clock skew is not usually significant
over the time intervals used by this SBD mechanism (see [RFC6817] A.2 over the time intervals used by this SBD mechanism (see [RFC6817] A.2
for a discussion on clock skew and OWD measurements). However, in for a discussion on clock skew and OWD measurements). However, in
circumstances where it is significant, Section 3.4.2 outlines a way circumstances where it is significant, Section 5.2 outlines a way of
of adjusting the calculations to cater for it. adjusting the calculations to cater for it.
Each packet arriving at the bottleneck buffer may experience very Each packet arriving at the bottleneck buffer may experience very
different queue lengths, and therefore different waiting times. A different queue lengths, and therefore different waiting times. A
single OWD sample does not, therefore, characterize the path well. single OWD sample does not, therefore, characterize the path well.
However, multiple OWD measurements do reflect the distribution of However, multiple OWD measurements do reflect the distribution of
delays experienced at the bottleneck. delays experienced at the bottleneck.
1.1.3. Path Lag 1.2.3. Path lag
Flows that share a common bottleneck may traverse different paths, Flows that share a common bottleneck may traverse different paths,
and these paths will often have different base delays. This makes it and these paths will often have different base delays. This makes it
difficult to correlate changes in delay or loss. This technique uses difficult to correlate changes in delay or loss. This technique uses
the long term shape of the delay distribution as a base for the long term shape of the delay distribution as a base for
comparison to counter this. comparison to counter this.
2. Definitions 2. Definitions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
skipping to change at page 5, line 4 skipping to change at page 5, line 15
N -- the number of base time, T, intervals used in some N -- the number of base time, T, intervals used in some
calculations. calculations.
M -- the number of base time, T, intervals used in some M -- the number of base time, T, intervals used in some
calculations. calculations.
sum_T(...) -- summation of all the measurements of the variable sum_T(...) -- summation of all the measurements of the variable
in parentheses taken over the interval T in parentheses taken over the interval T
sum(...) -- summation of terms of the variable in parentheses sum(...) -- summation of terms of the variable in parentheses
sum_N(...) -- summation of N terms of the variable in parentheses sum_N(...) -- summation of N terms of the variable in parentheses
sum_NT(...) -- summation of all measurements taken over the sum_NT(...) -- summation of all measurements taken over the
interval N*T interval N*T
sum_MT(...) -- summation of all measurements taken over the
interval M*T
E_T(...) -- the expectation or mean of the measurements of the E_T(...) -- the expectation or mean of the measurements of the
variable in parentheses over T variable in parentheses over T
E_N(...) -- the expectation or mean of the last N values of the E_N(...) -- the expectation or mean of the last N values of the
variable in parentheses variable in parentheses
E_M(...) -- the expectation or mean of the last M values of the E_M(...) -- the expectation or mean of the last M values of the
variable in parentheses, where M <= N. variable in parentheses, where M <= N.
max_T(...) -- the maximum recorded measurement of the variable in max_T(...) -- the maximum recorded measurement of the variable in
skipping to change at page 7, line 5 skipping to change at page 7, line 5
freq_est -- a measure of low frequency oscillation in the OWD freq_est -- a measure of low frequency oscillation in the OWD
measurements. measurements.
p_l, p_f, p_mad, c_s, c_h, p_s, p_d, p_v -- various thresholds p_l, p_f, p_mad, c_s, c_h, p_s, p_d, p_v -- various thresholds
used in the mechanism used in the mechanism
M and F -- number of values related to N M and F -- number of values related to N
. .
2.1. Parameters and their Effect 2.1. Parameters and their effect
T T should be long enough so that there are enough packets T T should be long enough so that there are enough packets
received during T for a useful estimate of short term mean received during T for a useful estimate of short term mean
OWD and variation statistics. Making T too large can limit OWD and variation statistics. Making T too large can limit
the efficacy of freq_est. It will also increase the response the efficacy of freq_est. It will also increase the response
time of the mechanism. Making T too small will make the time of the mechanism. Making T too small will make the
metrics noisier. metrics noisier.
N & M N should be large enough to provide a stable estimate of N & M N should be large enough to provide a stable estimate of
oscillations in OWD. Usually M=N, though having M<N may be oscillations in OWD. Usually M=N, though having M<N may be
skipping to change at page 8, line 5 skipping to change at page 8, line 5
determining groups. determining groups.
p_* Flows are separated when the skew_est|var_est|freq_est p_* Flows are separated when the skew_est|var_est|freq_est
measure is greater than p_s|p_f|p_d|p_mad. Adjusting these measure is greater than p_s|p_f|p_d|p_mad. Adjusting these
is a compromise between false grouping of flows that do not is a compromise between false grouping of flows that do not
share a bottleneck and false splitting of flows that do. share a bottleneck and false splitting of flows that do.
Making them larger can help if the measures are very noisy, Making them larger can help if the measures are very noisy,
but reducing the noise in the statistical measures by but reducing the noise in the statistical measures by
adjusting T and N|M may be a better solution. adjusting T and N|M may be a better solution.
2.2. Recommended Parameter Values 2.2. Recommended parameter values
Reference [Hayes-LCN14] uses T=350ms, N=50, p_l=0.1. The other Reference [Hayes-LCN14] uses T=350ms, N=50, p_l=0.1. The other
parameters have been tightened to reflect minor enhancements to the parameters have been tightened to reflect minor enhancements to the
algorithm outlined in Section 3.4: c_s=-0.01, p_f=p_d=0.1, p_s=0.15, algorithm outlined in Section 4: c_s=-0.01, p_f=p_d=0.1, p_s=0.15,
p_mad=0.1, p_v=0.7. M=30, F=20, and c_h = 0.3 are additional p_mad=0.1, p_v=0.7. M=30, F=20, and c_h = 0.3 are additional
parameters defined in the document. These are values that seem to parameters defined in the document. These are values that seem to
work well over a wide range of practical Internet conditions. work well over a wide range of practical Internet conditions.
3. Mechanism 3. Mechanism
The mechanism described in this document is based on the observation The mechanism described in this document is based on the observation
that the distribution of delay measurements of packets that traverse that the distribution of delay measurements of packets that traverse
a common bottleneck have similar shape characteristics. These shape a common bottleneck have similar shape characteristics. These shape
characteristics are described using 3 key summary statistics: characteristics are described using 3 key summary statistics:
skipping to change at page 9, line 36 skipping to change at page 9, line 36
1. Both summary statistic calculations and SBD are performed at 1. Both summary statistic calculations and SBD are performed at
senders only. senders only.
2. Summary statistics calculated on the receivers and SBD at the 2. Summary statistics calculated on the receivers and SBD at the
senders. senders.
3. Summary statistic calculations on receivers, and SBD performed at 3. Summary statistic calculations on receivers, and SBD performed at
both senders and receivers (beyond the current scope, but allows both senders and receivers (beyond the current scope, but allows
cooperative detection of bottlenecks). cooperative detection of bottlenecks).
Note that the mechanism bases its calculations on the interval T. It
does not require T to be the feedback interval, only that
calculations can be performed over measurements made in that
interval.
3.1.1. Feedback when all the logic is placed at the sender 3.1.1. Feedback when all the logic is placed at the sender
Having the sender calculate the summary statistics and determine the Having the sender calculate the summary statistics and determine the
shared bottlenecks based on them has the advantage of placing most of shared bottlenecks based on them has the advantage of placing most of
the functionality in one place -- the sender. the functionality in one place -- the sender.
The sender requires precise accurate OWD measurements for every The sender requires precise accurate OWD measurements for every
packet, along with an indication of lost packets (or the proportion packet, along with an indication of lost packets (or the proportion
of packets lost over the interval T). The mechanism performs its of packets lost over the interval T). The mechanism performs its
calculations every T and requires measurements to be available for calculations every T and requires measurements to be available for
this. this.
An initialization message may be required to agree on the feedback It is expected that the draft-ietf-rmcat-feedback-message will
interval. provide the necessary feedback for both SBD and congestion
controllers.
3.1.2. Feedback when the statistics are calculated at the receiver and 3.1.2. Feedback when the statistics are calculated at the receiver and
SBD performed at the sender SBD performed at the sender
This scenario minimizes feedback, but requires receivers to send This scenario minimizes feedback, but requires receivers to send
selected summary statistics at an agreed regular interval. We selected summary statistics at an agreed regular interval. We
envisage the following exchange of information to initialize the envisage the following exchange of information to initialize the
system: system:
o An initialization message from the sender to the receiver will o An initialization message from the sender to the receiver will
skipping to change at page 10, line 41 skipping to change at page 10, line 45
precision of the relayed statistics. precision of the relayed statistics.
o A response message from the receiver acknowledges this message o A response message from the receiver acknowledges this message
with a list of key metrics it supports (subset of the senders with a list of key metrics it supports (subset of the senders
list) and is able to relay back to the sender. list) and is able to relay back to the sender.
This initialization exchange may be repeated to finalize the agreed This initialization exchange may be repeated to finalize the agreed
metrics should not all be supported by all receivers. metrics should not all be supported by all receivers.
After initialization the agreed summary statistics will be fed back After initialization the agreed summary statistics will be fed back
to the sender every T. to the sender (nominally every T).
3.1.3. Feedback when bottlenecks can be determined at both senders and 3.1.3. Feedback when bottlenecks can be determined at both senders and
receivers receivers
This type of mechanism is currently beyond the scope of SBD in RMCAT. This type of mechanism is currently beyond the scope of SBD in RMCAT.
It is mentioned here to ensure more advanced sender/receiver It is mentioned here to ensure more advanced sender/receiver
cooperative shared bottleneck determination mechanisms remain cooperative shared bottleneck determination mechanisms remain
possible in the future. possible in the future.
It is envisaged that such a mechanism would be initialized in a It is envisaged that such a mechanism would be initialized in a
similar manner to that described in Section 3.1.2. similar manner to that described in Section 3.1.2.
After initialization both summary statistics and shared bottleneck After initialization both summary statistics and shared bottleneck
determinations should be exchanged every T. determinations should be exchanged, nominally every T.
3.2. Key metrics and their calculation 3.2. Key metrics and their calculation
Measurements are calculated over a base interval, T and summarized Measurements are calculated over a base interval, T and summarized
over N or M such intervals. All summary statistics can be calculated over N or M such intervals. All summary statistics can be calculated
incrementally. incrementally.
3.2.1. Mean delay 3.2.1. Mean delay
The mean delay is not a useful signal for comparisons between flows The mean delay is not a useful signal for comparisons between flows
skipping to change at page 11, line 30 skipping to change at page 11, line 35
delay measured over T. delay measured over T.
To facilitate the other calculations, the last N E_T(OWD) values will To facilitate the other calculations, the last N E_T(OWD) values will
need to be stored in a cyclic buffer along with the moving average of need to be stored in a cyclic buffer along with the moving average of
E_T(OWD): E_T(OWD):
mean_delay = E_M(E_T(OWD)) = sum_M(E_T(OWD)) / M mean_delay = E_M(E_T(OWD)) = sum_M(E_T(OWD)) / M
where M <= N. Setting M to be less than N allows the mechanism to be where M <= N. Setting M to be less than N allows the mechanism to be
more responsive to changes, but potentially at the expense of a more responsive to changes, but potentially at the expense of a
higher error rate (see Section 3.5 for a discussion on improving the higher error rate (see Section 4.1 for a discussion on improving the
responsiveness of the mechanism.) responsiveness of the mechanism.)
3.2.2. Skewness Estimate 3.2.2. Skewness estimate
Skewness is difficult to calculate efficiently and accurately. Skewness is difficult to calculate efficiently and accurately.
Ideally it should be calculated over the entire period (M * T) from Ideally it should be calculated over the entire period (M * T) from
the mean OWD over that period. However this would require storing the mean OWD over that period. However this would require storing
every delay measurement over the period. Instead, an estimate is every delay measurement over the period. Instead, an estimate is
made over M * T based on a calculation every T using the previous T's made over M * T based on a calculation every T using the previous T's
calculation of mean_delay. calculation of mean_delay.
The base for the skewness calculation is estimated using a counter The base for the skewness calculation is estimated using a counter
initialized every T. It increments for one way delay samples (OWD) initialized every T. It increments for one way delay samples (OWD)
skipping to change at page 12, line 13 skipping to change at page 12, line 20
enable it to be calculated iteratively. enable it to be calculated iteratively.
skew_est = sum_MT(skew_base_T)/num_MT(OWD) skew_est = sum_MT(skew_base_T)/num_MT(OWD)
where skew_est is a number between -1 and 1 where skew_est is a number between -1 and 1
Note: Care must be taken when implementing the comparisons to ensure Note: Care must be taken when implementing the comparisons to ensure
that rounding does not bias skew_est. It is important that the mean that rounding does not bias skew_est. It is important that the mean
is calculated with a higher precision than the samples. is calculated with a higher precision than the samples.
3.2.3. Variability Estimate 3.2.3. Variability estimate
Mean Absolute Deviation (MAD) delay is a robust variability measure Mean Absolute Deviation (MAD) delay is a robust variability measure
that copes well with different send rates. It can be implemented in that copes well with different send rates. It can be implemented in
an online manner as follows: an online manner as follows:
var_base_T = sum_T(|OWD - E_T(OWD)|) var_base_T = sum_T(|OWD - E_T(OWD)|)
where where
|x| is the absolute value of x |x| is the absolute value of x
E_T(OWD) is the mean OWD calculated in the previous T E_T(OWD) is the mean OWD calculated in the previous T
var_est = MAD_MT = sum_MT(var_base_T)/num_MT(OWD) var_est = MAD_MT = sum_MT(var_base_T)/num_MT(OWD)
For calculation of freq_est p_v=0.7 For calculation of freq_est p_v=0.7
For the grouping threshold p_mad=0.1 For the grouping threshold p_mad=0.1
3.2.4. Oscillation Estimate 3.2.4. Oscillation estimate
An estimate of the low frequency oscillation of the delay signal is An estimate of the low frequency oscillation of the delay signal is
calculated by counting and normalizing the significant mean, calculated by counting and normalizing the significant mean,
E_T(OWD), crossings of mean_delay: E_T(OWD), crossings of mean_delay:
freq_est = number_of_crossings / N freq_est = number_of_crossings / N
where we define a significant mean crossing as a crossing that where we define a significant mean crossing as a crossing that
extends p_v * var_est from mean_delay. In our experiments we extends p_v * var_est from mean_delay. In our experiments we
have found that p_v = 0.7 is a good value. have found that p_v = 0.7 is a good value.
skipping to change at page 13, line 31 skipping to change at page 13, line 39
supplementary measure: supplementary measure:
pkt_loss = sum_NT(lost packets) / sum_NT(total packets) pkt_loss = sum_NT(lost packets) / sum_NT(total packets)
Note: When pkt_loss is small it is very variable, however, when Note: When pkt_loss is small it is very variable, however, when
pkt_loss is high it becomes a stable measure for making grouping pkt_loss is high it becomes a stable measure for making grouping
decisions. decisions.
3.3. Flow Grouping 3.3. Flow Grouping
3.3.1. Flow Grouping Algorithm 3.3.1. Flow grouping algorithm
The following grouping algorithm is RECOMMENDED for SBD in the RMCAT The following grouping algorithm is RECOMMENDED for SBD in the RMCAT
context and is sufficient and efficient for small to moderate numbers context and is sufficient and efficient for small to moderate numbers
of flows. For very large numbers of flows (e.g. hundreds), a more of flows. For very large numbers of flows (e.g. hundreds), a more
complex clustering algorithm may be substituted. complex clustering algorithm may be substituted.
Since no single metric is precise enough to group flows (due to Since no single metric is precise enough to group flows (due to
noise), the algorithm uses multiple metrics. Each metric offers a noise), the algorithm uses multiple metrics. Each metric offers a
different "view" of the bottleneck link characteristics, and used different "view" of the bottleneck link characteristics, and used
together they enable a more precise grouping of flows than would together they enable a more precise grouping of flows than would
skipping to change at page 14, line 29 skipping to change at page 14, line 39
These flows, flows transiting a bottleneck, are then progressively These flows, flows transiting a bottleneck, are then progressively
divided into groups based on the freq_est, var_est, and skew_est divided into groups based on the freq_est, var_est, and skew_est
summary statistics. The process proceeds according to the following summary statistics. The process proceeds according to the following
steps: steps:
2. Group flows whose difference in sorted freq_est is less than a 2. Group flows whose difference in sorted freq_est is less than a
threshold: threshold:
diff(freq_est) < p_f diff(freq_est) < p_f
3. Group flows whose difference in sorted E_M(var_est) (highest to 3. Subdivide freq_est groups by grouping flows whose difference in
lowest) is less than a threshold: sorted E_M(var_est) (highest to lowest) is less than a threshold:
diff(var_est) < (p_mad * var_est) diff(var_est) < (p_mad * var_est)
The threshold, (p_mad * var_est), is with respect to the highest The threshold, (p_mad * var_est), is with respect to the highest
value in the difference. value in the difference.
4. Group flows whose difference in sorted skew_est is less than a 4. Subdivide var_est groups by grouping flows whose difference in
threshold: sorted skew_est is less than a threshold:
diff(skew_est) < p_s diff(skew_est) < p_s
5. When packet loss is high enough to be reliable (pkt_loss > p_l), 5. When packet loss is high enough to be reliable (pkt_loss > p_l),
group flows whose difference is less than a threshold Subdivide skew_est groups by grouping flows whose difference is
less than a threshold
diff(pkt_loss) < (p_d * pkt_loss) diff(pkt_loss) < (p_d * pkt_loss)
The threshold, (p_d * pkt_loss), is with respect to the highest The threshold, (p_d * pkt_loss), is with respect to the highest
value in the difference. value in the difference.
This procedure involves sorting estimates from highest to lowest. It This procedure involves sorting estimates from highest to lowest. It
is simple to implement, and efficient for small numbers of flows (up is simple to implement, and efficient for small numbers of flows (up
to 10-20). to 10-20).
skipping to change at page 15, line 19 skipping to change at page 15, line 32
2*N'th T interval. We recommend that grouping decisions are not made 2*N'th T interval. We recommend that grouping decisions are not made
until 2*M T intervals. until 2*M T intervals.
Network conditions, and even the congestion controllers, can cause Network conditions, and even the congestion controllers, can cause
bottlenecks to fluctuate. A coupled congestion controller MAY decide bottlenecks to fluctuate. A coupled congestion controller MAY decide
only to couple groups that remain stable, say grouped together 90% of only to couple groups that remain stable, say grouped together 90% of
the time, depending on its objectives. Recommendations concerning the time, depending on its objectives. Recommendations concerning
this are beyond the scope of this document and will be specific to this are beyond the scope of this document and will be specific to
the coupled congestion controllers objectives. the coupled congestion controllers objectives.
3.4. Removing Noise from the Estimates 4. Enhancements to the basic SBD algorithm
The following describe small changes to the calculation of the key
metrics that help remove noise from them. These "tweaks" are
described separately to keep the main description succinct.
3.4.1. Oscillation noise
When a path has no bottleneck, var_est will be very small and the
recorded significant mean crossings will be the result of path noise.
Thus up to N-1 meaningless mean crossings can be a source of error at
the point a link becomes a bottleneck and flows traversing it begin
to be grouped.
To remove this source of noise from freq_est:
1. Set the current var_base_T = NaN (a value representing an invalid
record, i.e. Not a Number) for flows that are deemed to not be
transiting a bottleneck by the first skew_est based grouping test
(see Section 3.3.1).
2. Then var_est = sum_MT(var_base_T != NaN) / num_MT(OWD)
3. For freq_est, only record a significant mean crossing if flow
deemed to be transiting a bottleneck.
These three changes can help to remove the non-bottleneck noise from
freq_est.
3.4.2. Clock skew
Generally sender and receiver clock skew will be too small to cause The SBD algorithm as specified in Section 3 was found to work well
significant errors in the estimators. Skew_est and freq_est are the for a broad variety of conditions. The following enhancements to the
most sensitive to this type of noise due to their use of a mean OWD basic mechanisms have been found to significantly improve the
calculated over a longer interval. In circumstances where clock skew algorithm's performance under some circumstances and SHOULD be
is high, basing skew_est only on the previous T's mean and ignoring implemented. These "tweaks" are described separately to keep the
freq_est provides a noisier but reliable signal. main description succinct.
A more sophisticated method is to estimate the effect the clock skew 4.1. Reducing lag and improving responsiveness
is having on the summary statistics, and then adjust statistics
accordingly. There are a number of techniques in the literature,
including [Zhang-Infocom02].
3.5. Reducing lag and Improving Responsiveness This section describes how to improve the responsiveness of the basic
algorithm.
Measurement based shared bottleneck detection makes decisions in the Measurement based shared bottleneck detection makes decisions in the
present based on what has been measured in the past. This means that present based on what has been measured in the past. This means that
there is always a lag in responding to changing conditions. This there is always a lag in responding to changing conditions. This
mechanism is based on summary statistics taken over (N*T) seconds. mechanism is based on summary statistics taken over (N*T) seconds.
This mechanism can be made more responsive to changing conditions by: This mechanism can be made more responsive to changing conditions by:
1. Reducing N and/or M -- but at the expense of having less accurate 1. Reducing N and/or M -- but at the expense of having less accurate
metrics, and/or metrics, and/or
skipping to change at page 16, line 40 skipping to change at page 16, line 24
exponentially weighted moving average weights drop off too quickly exponentially weighted moving average weights drop off too quickly
for our requirements and have an infinite tail. A simple linearly for our requirements and have an infinite tail. A simple linearly
declining weighted moving average also does not provide enough weight declining weighted moving average also does not provide enough weight
to the most recent measurements. We propose a piecewise linear to the most recent measurements. We propose a piecewise linear
distribution of weights, such that the first section (samples 1:F) is distribution of weights, such that the first section (samples 1:F) is
flat as in a simple moving average, and the second section (samples flat as in a simple moving average, and the second section (samples
F+1:M) is linearly declining weights to the end of the averaging F+1:M) is linearly declining weights to the end of the averaging
window. We choose integer weights, which allows incremental window. We choose integer weights, which allows incremental
calculation without introducing rounding errors. calculation without introducing rounding errors.
3.5.1. Improving the response of the skewness estimate 4.1.1. Improving the response of the skewness estimate
The weighted moving average for skew_est, based on skew_est in The weighted moving average for skew_est, based on skew_est in
Section 3.2.2, can be calculated as follows: Section 3.2.2, can be calculated as follows:
skew_est = ((M-F+1)*sum(skew_base_T(1:F)) skew_est = ((M-F+1)*sum(skew_base_T(1:F))
+ sum([(M-F):1].*skew_base_T(F+1:M))) + sum([(M-F):1].*skew_base_T(F+1:M)))
/ ((M-F+1)*sum(numsampT(1:F)) / ((M-F+1)*sum(numsampT(1:F))
+ sum([(M-F):1].*numsampT(F+1:M))) + sum([(M-F):1].*numsampT(F+1:M)))
where numsampT is an array of the number of OWD samples in each T where numsampT is an array of the number of OWD samples in each T
(i.e. num_T(OWD)), and numsampT(1) is the most recent; skew_base_T(1) (i.e. num_T(OWD)), and numsampT(1) is the most recent; skew_base_T(1)
is the most recent calculation of skew_base_T; 1:F refers to the is the most recent calculation of skew_base_T; 1:F refers to the
integer values 1 through to F, and [(M-F):1] refers to an array of integer values 1 through to F, and [(M-F):1] refers to an array of
the integer values (M-F) declining through to 1; and ".*" is the the integer values (M-F) declining through to 1; and ".*" is the
array scalar dot product operator. array scalar dot product operator.
To calculate this weighted skew_est incrementally: To calculate this weighted skew_est incrementally:
skipping to change at page 18, line 10 skipping to change at page 17, line 10
is the most recent calculation of skew_base_T; 1:F refers to the is the most recent calculation of skew_base_T; 1:F refers to the
integer values 1 through to F, and [(M-F):1] refers to an array of integer values 1 through to F, and [(M-F):1] refers to an array of
the integer values (M-F) declining through to 1; and ".*" is the the integer values (M-F) declining through to 1; and ".*" is the
array scalar dot product operator. array scalar dot product operator.
To calculate this weighted skew_est incrementally: To calculate this weighted skew_est incrementally:
Notation: F_ - flat portion, D_ - declining portion, W_ - weighted Notation: F_ - flat portion, D_ - declining portion, W_ - weighted
component component
Initialise: sum_skewbase = 0, F_skewbase=0, W_D_skewbase=0 Initialize: sum_skewbase = 0, F_skewbase=0, W_D_skewbase=0
skewbase_hist = buffer length M initialize to 0 skewbase_hist = buffer length M initialize to 0
numsampT = buffer length M initialized to 0 numsampT = buffer length M initialized to 0
Steps per iteration: Steps per iteration:
1. old_skewbase = skewbase_hist(M) 1. old_skewbase = skewbase_hist(M)
2. old_numsampT = numsampT(M) 2. old_numsampT = numsampT(M)
skipping to change at page 19, line 5 skipping to change at page 18, line 5
11. sum_skewbase = sum_skewbase + skewbase_hist(F+1) - old_skewbase 11. sum_skewbase = sum_skewbase + skewbase_hist(F+1) - old_skewbase
12. sum_numsamp = sum_numsamp + numsampT(1) - old_numsampT 12. sum_numsamp = sum_numsamp + numsampT(1) - old_numsampT
13. skew_est = ((M-F+1)*F_skewbase + W_D_skewbase) / 13. skew_est = ((M-F+1)*F_skewbase + W_D_skewbase) /
((M-F+1)*F_numsamp+W_D_numsamp) ((M-F+1)*F_numsamp+W_D_numsamp)
Where cycle(....) refers to the operation on a cyclic buffer where Where cycle(....) refers to the operation on a cyclic buffer where
the start of the buffer is now the next element in the buffer. the start of the buffer is now the next element in the buffer.
3.5.2. Improving the response of the variability estimate 4.1.2. Improving the response of the variability estimate
Similarly the weighted moving average for var_est can be calculated Similarly the weighted moving average for var_est can be calculated
as follows: as follows:
var_est = ((M-F+1)*sum(var_base_T(1:F)) var_est = ((M-F+1)*sum(var_base_T(1:F))
+ sum([(M-F):1].*var_base_T(F+1:M))) + sum([(M-F):1].*var_base_T(F+1:M)))
/ ((M-F+1)*sum(numsampT(1:F)) / ((M-F+1)*sum(numsampT(1:F))
+ sum([(M-F):1].*numsampT(F+1:M))) + sum([(M-F):1].*numsampT(F+1:M)))
where numsampT is an array of the number of OWD samples in each T where numsampT is an array of the number of OWD samples in each T
(i.e. num_T(OWD)), and numsampT(1) is the most recent; skew_base_T(1) (i.e. num_T(OWD)), and numsampT(1) is the most recent; skew_base_T(1)
is the most recent calculation of skew_base_T; 1:F refers to the is the most recent calculation of skew_base_T; 1:F refers to the
integer values 1 through to F, and [(M-F):1] refers to an array of integer values 1 through to F, and [(M-F):1] refers to an array of
the integer values (M-F) declining through to 1; and ".*" is the the integer values (M-F) declining through to 1; and ".*" is the
array scalar dot product operator. When removing oscillation noise array scalar dot product operator. When removing oscillation noise
(see Section 3.4.1) this calculation must be adjusted to allow for (see Section 4.2) this calculation must be adjusted to allow for
invalid var_base_T records. invalid var_base_T records.
Var_est can be calculated incrementally in the same way as skew_est Var_est can be calculated incrementally in the same way as skew_est
in Section 3.5.1. However, note that the buffer numsampT is used for in Section 4.1.1. However, note that the buffer numsampT is used for
both calculations so the operations on it should not be repeated. both calculations so the operations on it should not be repeated.
4. Measuring OWD 4.2. Removing oscillation noise
When a path has no bottleneck, var_est will be very small and the
recorded significant mean crossings will be the result of path noise.
Thus up to N-1 meaningless mean crossings can be a source of error at
the point a link becomes a bottleneck and flows traversing it begin
to be grouped.
To remove this source of noise from freq_est:
1. Set the current var_base_T = NaN (a value representing an invalid
record, i.e. Not a Number) for flows that are deemed to not be
transiting a bottleneck by the first skew_est based grouping test
(see Section 3.3.1).
2. Then var_est = sum_MT(var_base_T != NaN) / num_MT(OWD)
3. For freq_est, only record a significant mean crossing if flow
deemed to be transiting a bottleneck.
These three changes can help to remove the non-bottleneck noise from
freq_est.
5. Measuring OWD
This section discusses the OWD measurements required for this This section discusses the OWD measurements required for this
algorithm to detect shared bottlenecks. algorithm to detect shared bottlenecks.
The SBD mechanism described in this document relies on differences The SBD mechanism described in this document relies on differences
between OWD measurements to avoid the practical problems with between OWD measurements to avoid the practical problems with
measuring absolute OWD (see [Hayes-LCN14] section IIIC). Since all measuring absolute OWD (see [Hayes-LCN14] section IIIC). Since all
summary statistics are relative to the mean OWD and sender/receiver summary statistics are relative to the mean OWD and sender/receiver
clock offsets should be approximately constant over the measurement clock offsets should be approximately constant over the measurement
periods, the offset is subtracted out in the calculation. periods, the offset is subtracted out in the calculation.
4.1. Time stamp resolution 5.1. Time stamp resolution
The SBD mechanism requires timing information precise enough to be The SBD mechanism requires timing information precise enough to be
able to make comparisons. As a rule of thumb, the time resolution able to make comparisons. As a rule of thumb, the time resolution
should be less than one hundredth of a typical path's range of should be less than one hundredth of a typical path's range of
delays. In general, the lower the time resolution, the more care delays. In general, the coarser the time resolution, the more care
that needs to be taken to ensure rounding errors do not bias the that needs to be taken to ensure rounding errors do not bias the
skewness calculation. skewness calculation.
Typical RTP media flows use sub-millisecond timers, which should be Typical RTP media flows use sub-millisecond timers, which should be
adequate in most situations. adequate in most situations.
5. Expected feedback from experiments 5.2. Clock skew
Generally sender and receiver clock skew will be too small to cause
significant errors in the estimators. Skew_est and freq_est are the
most sensitive to this type of noise due to their use of a mean OWD
calculated over a longer interval. In circumstances where clock skew
is high, basing skew_est only on the previous T's mean and ignoring
freq_est provides a noisier but reliable signal.
A more sophisticated method is to estimate the effect the clock skew
is having on the summary statistics, and then adjust statistics
accordingly. There are a number of techniques in the literature,
including [Zhang-Infocom02].
6. Expected feedback from experiments
The algorithm described in this memo has so far been evaluated using The algorithm described in this memo has so far been evaluated using
simulations. Real network tests using the proposed congestion simulations. Real network tests using the proposed congestion
control algorithms will help confirm the default parameter choice. control algorithms will help confirm the default parameter choice.
For example, the time interval T may need to be made longer if the For example, the time interval T may need to be made longer if the
packet rate is very low. Implementers and testers are invited to packet rate is very low. Implementers and testers are invited to
document their findings in an Internet draft. document their findings in an Internet draft.
6. Acknowledgments 7. Acknowledgments
This work was part-funded by the European Community under its Seventh This work was part-funded by the European Community under its Seventh
Framework Programme through the Reducing Internet Transport Latency Framework Programme through the Reducing Internet Transport Latency
(RITE) project (ICT-317700). The views expressed are solely those of (RITE) project (ICT-317700). The views expressed are solely those of
the authors. the authors.
7. IANA Considerations 8. IANA Considerations
This memo includes no request to IANA. This memo includes no request to IANA.
8. Security Considerations 9. Security Considerations
The security considerations of RFC 3550 [RFC3550], RFC 4585 The security considerations of RFC 3550 [RFC3550], RFC 4585
[RFC4585], and RFC 5124 [RFC5124] are expected to apply. [RFC4585], and RFC 5124 [RFC5124] are expected to apply.
Non-authenticated RTCP packets carrying shared bottleneck indications Non-authenticated RTCP packets carrying OWD measurements, shared
and summary statistics could allow attackers to alter the bottleneck bottleneck indications, and/or summary statistics could allow
sharing characteristics for private gain or disruption of other attackers to alter the bottleneck sharing characteristics for private
parties communication. gain or disruption of other parties communication.
9. Change history 10. Change history
Changes made to this document: Changes made to this document:
WG-05->WG-06 : Updates addressing WG reviews
https://mailarchive.ietf.org/arch/msg/rmcat/-
1JdrTMq1Y5T6ZNlOkrQJQ27TzE and
https://mailarchive.ietf.org/arch/msg/rmcat/
eI2Q1f8NL2SxbJgjFLR4_rEmJ_g. This has mainly
involved minor clarifications, including the moving
of 3.4.1 and 3.5 into the new Section 4, and 3.4.1
into Section 5
WG-04->WG-05 : Fix ToC formatting. Add section on expected WG-04->WG-05 : Fix ToC formatting. Add section on expected
feedback from experiments replacing short section feedback from experiments replacing short section
on implementation status. Added comment on ECN as on implementation status. Added comment on ECN as
a signal. Clarification of lost packet signaling. a signal. Clarification of lost packet signaling.
Change term "draft" to "document" where Change term "draft" to "document" where
appropriate. American spelling. Some tightening appropriate. American spelling. Some tightening
of the text. of the text.
WG-03->WG-04 : Add M to terminology table, suggest skew_est based WG-03->WG-04 : Add M to terminology table, suggest skew_est based
on previous T and no freq_est in clock skew on previous T and no freq_est in clock skew
section, feedback requirements as a separate sub section, feedback requirements as a separate sub
section. section.
WG-02->WG-03 : Correct misspelled author WG-02->WG-03 : Correct misspelled author
WG-01->WG-02 : Removed ambiguity associated with the term WG-01->WG-02 : Removed ambiguity associated with the term
"congestion". Expanded the description of "congestion". Expanded the description of
initialisation messages. Removed PDV metric. initialization messages. Removed PDV metric.
Added description of incremental weighted metric Added description of incremental weighted metric
calculations for skew_est. Various clarifications calculations for skew_est. Various clarifications
based on implementation work. Fixed typos and based on implementation work. Fixed typos and
tuned parameters. tuned parameters.
WG-00->WG-01 : Moved unbiased skew section to replace skew WG-00->WG-01 : Moved unbiased skew section to replace skew
estimate, more robust variability estimator, the estimate, more robust variability estimator, the
term variance replaced with variability, clock term variance replaced with variability, clock
drift term corrected to clock skew, revision to drift term corrected to clock skew, revision to
clock skew section with a place holder, description clock skew section with a place holder, description
skipping to change at page 21, line 35 skipping to change at page 21, line 35
3.3.3 3.3.3
01->02 : New section describing improvements to the key 01->02 : New section describing improvements to the key
metric calculations that help to remove noise, metric calculations that help to remove noise,
bias, and reduce lag. Some revisions to the bias, and reduce lag. Some revisions to the
notation to make it clearer. Some tightening of notation to make it clearer. Some tightening of
the thresholds. the thresholds.
00->01 : Revisions to terminology for clarity 00->01 : Revisions to terminology for clarity
10. References 11. References
10.1. Normative References 11.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>. <http://www.rfc-editor.org/info/rfc2119>.
10.2. Informative References 11.2. Informative References
[Hayes-LCN14] [Hayes-LCN14]
Hayes, D., Ferlin, S., and M. Welzl, "Practical Passive Hayes, D., Ferlin, S., and M. Welzl, "Practical Passive
Shared Bottleneck Detection using Shape Summary Shared Bottleneck Detection using Shape Summary
Statistics", Proc. the IEEE Local Computer Networks Statistics", Proc. the IEEE Local Computer Networks
(LCN) pp150-158, September 2014, (LCN) pp150-158, September 2014,
<http://heim.ifi.uio.no/davihay/ <http://heim.ifi.uio.no/davihay/
hayes14__pract_passiv_shared_bottl_detec-abstract.html>. hayes14__pract_passiv_shared_bottl_detec-abstract.html>.
[I-D.ietf-rmcat-coupled-cc] [I-D.ietf-rmcat-coupled-cc]
 End of changes. 56 change blocks. 
117 lines changed or deleted 151 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/