draft-ietf-tcpm-generalized-ecn-04.txt   draft-ietf-tcpm-generalized-ecn-05.txt 
Network Working Group M. Bagnulo Network Working Group M. Bagnulo
Internet-Draft UC3M Internet-Draft UC3M
Obsoletes: 5562 (if approved) B. Briscoe Obsoletes: 5562 (if approved) B. Briscoe
Intended status: Experimental CableLabs Intended status: Experimental Independent
Expires: January 9, 2020 July 8, 2019 Expires: May 6, 2020 November 3, 2019
ECN++: Adding Explicit Congestion Notification (ECN) to TCP Control ECN++: Adding Explicit Congestion Notification (ECN) to TCP Control
Packets Packets
draft-ietf-tcpm-generalized-ecn-04 draft-ietf-tcpm-generalized-ecn-05
Abstract Abstract
This document describes an experimental modification to ECN when used This document describes an experimental modification to ECN when used
with TCP. It allows the use of ECN on the following TCP packets: with TCP. It allows the use of ECN on the following TCP packets:
SYNs, pure ACKs, Window probes, FINs, RSTs and retransmissions. SYNs, pure ACKs, Window probes, FINs, RSTs and retransmissions.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
skipping to change at page 1, line 34 skipping to change at page 1, line 34
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 9, 2020. This Internet-Draft will expire on May 6, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 28 skipping to change at page 2, line 28
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 4 1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . 4
1.2. Experiment Goals . . . . . . . . . . . . . . . . . . . . 5 1.2. Experiment Goals . . . . . . . . . . . . . . . . . . . . 5
1.3. Document Structure . . . . . . . . . . . . . . . . . . . 6 1.3. Document Structure . . . . . . . . . . . . . . . . . . . 6
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6
3. Specification . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Specification . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Network (e.g. Firewall) Behaviour . . . . . . . . . . . . 7 3.1. Network (e.g. Firewall) Behaviour . . . . . . . . . . . . 7
3.2. Sender Behaviour . . . . . . . . . . . . . . . . . . . . 8 3.2. Sender Behaviour . . . . . . . . . . . . . . . . . . . . 8
3.2.1. SYN (Send) . . . . . . . . . . . . . . . . . . . . . 9 3.2.1. SYN (Send) . . . . . . . . . . . . . . . . . . . . . 9
3.2.2. SYN-ACK (Send) . . . . . . . . . . . . . . . . . . . 12 3.2.2. SYN-ACK (Send) . . . . . . . . . . . . . . . . . . . 13
3.2.3. Pure ACK (Send) . . . . . . . . . . . . . . . . . . . 13 3.2.3. Pure ACK (Send) . . . . . . . . . . . . . . . . . . . 14
3.2.4. Window Probe (Send) . . . . . . . . . . . . . . . . . 14 3.2.4. Window Probe (Send) . . . . . . . . . . . . . . . . . 15
3.2.5. FIN (Send) . . . . . . . . . . . . . . . . . . . . . 15 3.2.5. FIN (Send) . . . . . . . . . . . . . . . . . . . . . 16
3.2.6. RST (Send) . . . . . . . . . . . . . . . . . . . . . 15 3.2.6. RST (Send) . . . . . . . . . . . . . . . . . . . . . 16
3.2.7. Retransmissions (Send) . . . . . . . . . . . . . . . 16 3.2.7. Retransmissions (Send) . . . . . . . . . . . . . . . 17
3.2.8. General Fall-back for any Control Packet or 3.2.8. General Fall-back for any Control Packet or
Retransmission . . . . . . . . . . . . . . . . . . . 16
3.3. Receiver Behaviour . . . . . . . . . . . . . . . . . . . 16
3.3.1. Receiver Behaviour for Any TCP Control Packet or
Retransmission . . . . . . . . . . . . . . . . . . . 17 Retransmission . . . . . . . . . . . . . . . . . . . 17
3.3.2. SYN (Receive) . . . . . . . . . . . . . . . . . . . . 17 3.3. Receiver Behaviour . . . . . . . . . . . . . . . . . . . 17
3.3.3. Pure ACK (Receive) . . . . . . . . . . . . . . . . . 18 3.3.1. Receiver Behaviour for Any TCP Control Packet or
3.3.4. FIN (Receive) . . . . . . . . . . . . . . . . . . . . 18 Retransmission . . . . . . . . . . . . . . . . . . . 18
3.3.5. RST (Receive) . . . . . . . . . . . . . . . . . . . . 18 3.3.2. SYN (Receive) . . . . . . . . . . . . . . . . . . . . 18
3.3.6. Retransmissions (Receive) . . . . . . . . . . . . . . 19 3.3.3. Pure ACK (Receive) . . . . . . . . . . . . . . . . . 19
4. Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3.4. FIN (Receive) . . . . . . . . . . . . . . . . . . . . 19
4.1. The Reliability Argument . . . . . . . . . . . . . . . . 19 3.3.5. RST (Receive) . . . . . . . . . . . . . . . . . . . . 20
4.2. SYNs . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.3.6. Retransmissions (Receive) . . . . . . . . . . . . . . 20
4.2.1. Argument 1a: Unrecognized CE on the SYN . . . . . . . 20 4. Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.2.2. Argument 1b: ECT Considered Invalid on the SYN . . . 21 4.1. The Reliability Argument . . . . . . . . . . . . . . . . 20
4.2.3. Caching Strategies for ECT on SYNs . . . . . . . . . 23 4.2. SYNs . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2.4. Argument 2: DoS Attacks . . . . . . . . . . . . . . . 25 4.2.1. Argument 1a: Unrecognized CE on the SYN . . . . . . . 21
4.3. SYN-ACKs . . . . . . . . . . . . . . . . . . . . . . . . 26 4.2.2. Argument 1b: ECT Considered Invalid on the SYN . . . 22
4.3.1. Possibility of Unrecognized CE on the SYN-ACK . . . . 26 4.2.3. Caching Strategies for ECT on SYNs . . . . . . . . . 24
4.3.2. Response to Congestion on a SYN-ACK . . . . . . . . . 26 4.2.4. Argument 2: DoS Attacks . . . . . . . . . . . . . . . 26
4.3.3. Fall-Back if ECT SYN-ACK Fails . . . . . . . . . . . 28 4.3. SYN-ACKs . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4. Pure ACKs . . . . . . . . . . . . . . . . . . . . . . . . 28 4.3.1. Possibility of Unrecognized CE on the SYN-ACK . . . . 27
4.4.1. Mechanisms to Respond to CE-Marked Pure ACKs . . . . 29 4.3.2. Response to Congestion on a SYN-ACK . . . . . . . . . 28
4.4.2. Summary: Enabling ECN on Pure ACKs . . . . . . . . . 32 4.3.3. Fall-Back if ECT SYN-ACK Fails . . . . . . . . . . . 29
4.5. Window Probes . . . . . . . . . . . . . . . . . . . . . . 33 4.4. Pure ACKs . . . . . . . . . . . . . . . . . . . . . . . . 29
4.6. FINs . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.4.1. Mechanisms to Respond to CE-Marked Pure ACKs . . . . 31
4.7. RSTs . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.4.2. Summary: Enabling ECN on Pure ACKs . . . . . . . . . 34
4.8. Retransmitted Packets. . . . . . . . . . . . . . . . . . 35 4.5. Window Probes . . . . . . . . . . . . . . . . . . . . . . 34
4.9. General Fall-back for any Control Packet . . . . . . . . 36 4.6. FINs . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5. Interaction with popular variants or derivatives of TCP . . . 37 4.7. RSTs . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.1. IW10 . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.8. Retransmitted Packets. . . . . . . . . . . . . . . . . . 37
5.2. TFO . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.9. General Fall-back for any Control Packet . . . . . . . . 38
5.3. TCP Derivatives . . . . . . . . . . . . . . . . . . . . . 39 5. Interaction with popular variants or derivatives of TCP . . . 38
6. Security Considerations . . . . . . . . . . . . . . . . . . . 39 5.1. IW10 . . . . . . . . . . . . . . . . . . . . . . . . . . 39
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 39 5.2. TFO . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 39 5.3. L4S . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.4. Other transport protocols . . . . . . . . . . . . . . . . 41
9.1. Normative References . . . . . . . . . . . . . . . . . . 40 6. Security Considerations . . . . . . . . . . . . . . . . . . . 41
9.2. Informative References . . . . . . . . . . . . . . . . . 40 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 41
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 43 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 42
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 42
9.1. Normative References . . . . . . . . . . . . . . . . . . 42
9.2. Informative References . . . . . . . . . . . . . . . . . 43
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 46
1. Introduction 1. Introduction
RFC 3168 [RFC3168] specifies support of Explicit Congestion RFC 3168 [RFC3168] specifies support of Explicit Congestion
Notification (ECN) in IP (v4 and v6). By using the ECN capability, Notification (ECN) in IP (v4 and v6). By using the ECN capability,
network elements (e.g. routers, switches) performing Active Queue network elements (e.g. routers, switches) performing Active Queue
Management (AQM) can use ECN marks instead of packet drops to signal Management (AQM) can use ECN marks instead of packet drops to signal
congestion to the endpoints of a communication. This results in congestion to the endpoints of a communication. This results in
lower packet loss and increased performance. RFC 3168 also specifies lower packet loss and increased performance. RFC 3168 also specifies
support for ECN in TCP, but solely on data packets. For various support for ECN in TCP, but solely on data packets. For various
reasons it precludes the use of ECN on TCP control packets (TCP SYN, reasons it precludes the use of ECN on TCP control packets (TCP SYN,
TCP SYN-ACK, pure ACKs, Window probes) and on retransmitted packets. TCP SYN-ACK, pure ACKs, Window probes) and on retransmitted packets.
RFC 3168 is silent about the use of ECN on RST and FIN packets. RFC RFC 3168 is silent about the use of ECN on RST and FIN packets. RFC
5562 [RFC5562] is an experimental modification to ECN that enables 5562 [RFC5562] is an experimental modification to ECN that enables
ECN support for TCP SYN-ACK packets. ECN support for TCP SYN-ACK packets.
This document defines an experimental modification to ECN [RFC3168] This document defines an experimental modification to ECN [RFC3168]
that shall be called ECN++. It enables ECN support on all the that shall be called ECN++. It enables ECN support on all the
aforementioned types of TCP packet. aforementioned types of TCP packet. The mechanisms proposed in this
document have been defined conservatively and with safety in mind,
possibly in some cases at the expense of performance.
ECN++ uses a sender-only deployment model. It works whether the two ECN++ uses a sender-only deployment model. It works whether the two
ends of the TCP connection use classic ECN feedback [RFC3168] or ends of the TCP connection use classic ECN feedback [RFC3168] or
experimental Accurate ECN feedback (AccECN experimental Accurate ECN feedback (AccECN
[I-D.ietf-tcpm-accurate-ecn]). Nonetheless, if the client does not
implement AccECN, it cannot use ECN++ on the one packet that offers [I-D.ietf-tcpm-accurate-ecn]), the two ECN feedback mechanisms for
most benefit from it - the initial SYN. Therefore, implementers of TCP being standardized at the time of writing.
ECN++ are RECOMMENDED to also implement AccECN.
Using ECN on initial SYN packets provides significant benefits, as we
describe in the next subsection. However, only AccECN provides a way
to feed back whether the SYN was CE marked, and RFC 3168 does not.
Therefore, implementers of ECN++ are RECOMMENDED to also implement
AccECN. Conversely, if AccECN (or an equivalent safety mechanism) is
not implemented with ECN++, this specification rules out ECN on the
SYN.
ECN++ is designed for compatibility with a number of latency ECN++ is designed for compatibility with a number of latency
improvements to TCP such as TCP Fast Open (TFO [RFC7413]), initial improvements to TCP such as TCP Fast Open (TFO [RFC7413]), initial
window of 10 SMSS (IW10 [RFC6928]) and Low latency Low Loss Scalable window of 10 SMSS (IW10 [RFC6928]) and Low latency Low Loss Scalable
Transport (L4S [I-D.ietf-tsvwg-l4s-arch]), but they can all be Transport (L4S [I-D.ietf-tsvwg-l4s-arch]), but they can all be
implemented and deployed independently. [RFC8311] is a standards implemented and deployed independently. [RFC8311] is a standards
track procedural device that relaxes requirements in RFC 3168 and track procedural device that relaxes requirements in RFC 3168 and
other standards track RFCs that would otherwise preclude the other standards track RFCs that would otherwise preclude the
experimental modifications needed for ECN++ and other ECN experimental modifications needed for ECN++ and other ECN
experiments. experiments.
skipping to change at page 4, line 27 skipping to change at page 4, line 38
The absence of ECN support on TCP control packets and retransmissions The absence of ECN support on TCP control packets and retransmissions
has a potential harmful effect. In any ECN deployment, non-ECN- has a potential harmful effect. In any ECN deployment, non-ECN-
capable packets suffer a penalty when they traverse a congested capable packets suffer a penalty when they traverse a congested
bottleneck. For instance, with a drop probability of 1%, 1% of bottleneck. For instance, with a drop probability of 1%, 1% of
connection attempts suffer a timeout of about 1 second before the SYN connection attempts suffer a timeout of about 1 second before the SYN
is retransmitted, which is highly detrimental to the performance of is retransmitted, which is highly detrimental to the performance of
short flows. TCP control packets, particularly TCP SYNs and SYN- short flows. TCP control packets, particularly TCP SYNs and SYN-
ACKs, are important for performance, so dropping them is best ACKs, are important for performance, so dropping them is best
avoided. avoided.
Non-ECN control packets particularly harm performance in environments Not using ECN on control packets can be particularly detrimental to
where the ECN marking level is high. For example, [judd-nsdi] shows performance in environments where the ECN marking level is high. For
that in a controlled private data centre (DC) environment where ECN example, [judd-nsdi] shows that in a controlled private data centre
is used (in conjunction with DCTCP [RFC8257]), the probability of (DC) environment where ECN is used (in conjunction with DCTCP
being able to establish a new connection using a non-ECN SYN packet [RFC8257]), the probability of being able to establish a new
drops to close to zero even when there are only 16 ongoing TCP flows connection using a non-ECN SYN packet drops to close to zero even
transmitting at full speed. The issue is that DCTCP exhibits a much when there are only 16 ongoing TCP flows transmitting at full speed.
more aggressive response to packet marking (which is why it is only The issue is that DCTCP exhibits a much more aggressive response to
applicable in controlled environments). This leads to a high marking packet marking (which is why it is only applicable in controlled
probability for ECN-capable packets, and in turn a high drop environments). This leads to a high marking probability for ECN-
probability for non-ECN packets. Therefore non-ECN SYNs are dropped capable packets, and in turn a high drop probability for non-ECN
aggressively, rendering it nearly impossible to establish a new packets. Therefore non-ECN SYNs are dropped aggressively, rendering
connection in the presence of even mild traffic load. it nearly impossible to establish a new connection in the presence of
even mild traffic load.
Finally, there are ongoing experimental efforts to promote the Finally, there are ongoing experimental efforts to promote the
adoption of a slightly modified variant of DCTCP (and similar adoption of a slightly modified variant of DCTCP (and similar
congestion controls) over the Internet to achieve low latency, low congestion controls) over the Internet to achieve low latency, low
loss and scalable throughput (L4S) for all communications loss and scalable throughput (L4S) for all communications
[I-D.ietf-tsvwg-l4s-arch]. In such an approach, L4S packets identify [I-D.ietf-tsvwg-l4s-arch]. In such an approach, L4S packets identify
themselves using an ECN codepoint [I-D.ietf-tsvwg-ecn-l4s-id]. With themselves using an ECN codepoint [I-D.ietf-tsvwg-ecn-l4s-id]. With
L4S, preventing TCP control packets from obtaining the benefits of L4S, preventing TCP control packets from obtaining the benefits of
ECN would not only expose them to the prevailing level of congestion ECN would not only expose them to the prevailing level of congestion
loss, but it would also classify them into a different queue. Then loss, but it would also classify them into a different queue. Then
only L4S data packets would enjoy ultra-low latency forwarding, while only L4S data packets would be classified into the L4S queue that is
the packets controlling and retransmitting these data packets would expected to have lower latency, while the packets controlling and
still get stuck behind the queue induced by legacy ('Classic') TCP retransmitting these data packets would still get stuck behind the
traffic. queue induced by non-L4S-enabled TCP traffic.
1.2. Experiment Goals 1.2. Experiment Goals
The goal of the experimental modifications defined in this document The goal of the experimental modifications defined in this document
is to allow the use of ECN on all TCP packets. Experiments are is to allow the use of ECN on all TCP packets. Experiments are
expected in the public Internet as well as in controlled environments expected in the public Internet as well as in controlled environments
to understand the following issues: to understand the following issues:
o How SYNs, Window probes, pure ACKs, FINs, RSTs and retransmissions o How SYNs, Window probes, pure ACKs, FINs, RSTs and retransmissions
that carry the ECT(0), ECT(1) or CE codepoints are processed by that carry the ECT(0), ECT(1) or CE codepoints are processed by
skipping to change at page 5, line 31 skipping to change at page 5, line 43
o The scale of deployment of the different flavours of ECN, o The scale of deployment of the different flavours of ECN,
including [RFC3168], [RFC5562], [RFC3540] and including [RFC3168], [RFC5562], [RFC3540] and
[I-D.ietf-tcpm-accurate-ecn]. [I-D.ietf-tcpm-accurate-ecn].
o How much the performance of TCP communications is improved by o How much the performance of TCP communications is improved by
allowing ECN marking of each packet type. allowing ECN marking of each packet type.
o To identify any issues (including security issues) raised by o To identify any issues (including security issues) raised by
enabling ECN marking of these packets. enabling ECN marking of these packets.
o To conduct the specific experiments identified in the text by the
strings "EXPERIMENTATION NEEDED" or "MEASUREMENTS NEEDED".
The data gathered through the experiments described in this document, The data gathered through the experiments described in this document,
particularly under the first 2 bullets above, will help in the particularly under the first 2 bullets above, will help in the
redesign of the final mechanism (if needed) for adding ECN support to redesign of the final mechanism (if needed) for adding ECN support to
the different packet types considered in this document. Whenever the different packet types considered in this document.
data input is needed to assist in a design choice, it is spelled out
throughout the document.
Success criteria: The experiment will be a success if we obtain Success criteria: The experiment will be a success if we obtain
enough data to have a clearer view of the deployability and benefits enough data to have a clearer view of the deployability and benefits
of enabling ECN on all TCP packets, as well as any issues. If the of enabling ECN on all TCP packets, as well as any issues. If the
results of the experiment show that it is feasible to deploy such results of the experiment show that it is feasible to deploy such
changes; that there are gains to be achieved through the changes changes; that there are gains to be achieved through the changes
described in this specification; and that no other major issues may described in this specification; and that no other major issues may
interfere with the deployment of the proposed changes; then it would interfere with the deployment of the proposed changes; then it would
be reasonable to adopt the proposed changes in a standards track be reasonable to adopt the proposed changes in a standards track
specification that would update RFC 3168. specification that would update RFC 3168.
skipping to change at page 6, line 25 skipping to change at page 6, line 32
of specific reasons why ECN support is not appropriate for each of specific reasons why ECN support is not appropriate for each
packet type. In Section 4, we revisit each of these arguments for packet type. In Section 4, we revisit each of these arguments for
each packet type to justify why it is reasonable to conduct this each packet type to justify why it is reasonable to conduct this
experiment. experiment.
2. Terminology 2. Terminology
The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD,
SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL in this SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL in this
document, are to be interpreted as described in BCP 14 [RFC2119] when document, are to be interpreted as described in BCP 14 [RFC2119] when
and only when they appear in all capitals. and only when they appear in all capitals [RFC8174].
Pure ACK: A TCP segment with the ACK flag set and no data payload. Pure ACK: A TCP segment with the ACK flag set and no data payload.
SYN: A TCP segment with the SYN (synchronize) flag set. SYN: A TCP segment with the SYN (synchronize) flag set.
Window probe: Defined in [RFC0793], a window probe is a TCP segment Window probe: Defined in [RFC0793], a window probe is a TCP segment
with only one byte of data sent to learn if the receive window is with only one byte of data sent to learn if the receive window is
still zero. still zero.
FIN: A TCP segment with the FIN (finish) flag set. FIN: A TCP segment with the FIN (finish) flag set.
skipping to change at page 8, line 48 skipping to change at page 9, line 34
| | | | | | | | | |
| Re-XMT | ECT | ECT | Usual cwnd response | | Re-XMT | ECT | ECT | Usual cwnd response |
+---------+----------------+-----------------+----------------------+ +---------+----------------+-----------------+----------------------+
Window probe and retransmission are abbreviated to W Probe an Re-XMT. Window probe and retransmission are abbreviated to W Probe an Re-XMT.
* For a SYN, "negotiated" means "requested". * For a SYN, "negotiated" means "requested".
Table 1: Summary of sender behaviour. In each case the relevant Table 1: Summary of sender behaviour. In each case the relevant
section below should be referred to for the detailed behaviour section below should be referred to for the detailed behaviour
It can be seen that the sender cannot set ECT on the SYN if it is not It can be seen that we recommend against the sender setting ECT on
requesting AccECN feedback. Therefore it is RECOMMENDED that the the SYN if it is not requesting AccECN feedback. Therefore it is
experimental AccECN specification [I-D.ietf-tcpm-accurate-ecn] is RECOMMENDED that the experimental AccECN specification
implemented (as well as the present specification), because it is [I-D.ietf-tcpm-accurate-ecn] is implemented, along with the ECN++
expected that ECT on the SYN will give the most significant experiment, because it is expected that ECT on the SYN will give the
performance gain, particularly for short flows. most significant performance gain, particularly for short flows.
Nonetheless, this specification also caters for the case where an Nonetheless, this specification also caters for the case where an
ECN++ TCP sender is not using AccECN. This could be because it does ECN++ TCP sender is not using AccECN. This could be because it does
not support AccECN or because the other end of the TCP connection not support AccECN or because the other end of the TCP connection
does not (AccECN can only be used for a connection if both ends does not (AccECN can only be used for a connection if both ends
support it). support it).
3.2.1. SYN (Send) 3.2.1. SYN (Send)
3.2.1.1. Setting ECT on the SYN 3.2.1.1. Setting ECT on the SYN
With classic [RFC3168] ECN feedback, the SYN was not expected to be With classic [RFC3168] ECN feedback, the SYN was not expected to be
ECN-capable, so the flag provided to feed back congestion was put to ECN-capable, so the flag provided to feed back congestion was put to
another use (it is used in combination with other flags to indicate another use (it is used in combination with other flags to indicate
that the responder supports ECN). In contrast, Accurate ECN (AccECN) that the responder supports ECN). In contrast, Accurate ECN (AccECN)
feedback [I-D.ietf-tcpm-accurate-ecn] provides a codepoint in the feedback [I-D.ietf-tcpm-accurate-ecn] provides a codepoint in the
SYN-ACK for the responder to feed back whether the SYN arrived marked SYN-ACK for the responder to feed back whether the SYN arrived marked
CE. Therefore the setting of the IP/ECN field on the SYN is CE. Therefore the setting of the IP/ECN field on the SYN is
specified separately for each case in the following two subsections. specified separately for each case in the following two subsections.
skipping to change at page 9, line 35 skipping to change at page 10, line 24
3.2.1.1.1. ECN++ TCP Client also Supports AccECN 3.2.1.1.1. ECN++ TCP Client also Supports AccECN
For the ECN++ experiment, if the SYN is requesting AccECN feedback, For the ECN++ experiment, if the SYN is requesting AccECN feedback,
the TCP sender will also set ECT on the SYN. It can ignore the the TCP sender will also set ECT on the SYN. It can ignore the
prohibition in section 6.1.1 of RFC 3168 against setting ECT on such prohibition in section 6.1.1 of RFC 3168 against setting ECT on such
a SYN, as per Section 4.3 of [RFC8311]. a SYN, as per Section 4.3 of [RFC8311].
3.2.1.1.2. ECN++ TCP Client does not Support AccECN 3.2.1.1.2. ECN++ TCP Client does not Support AccECN
A TCP initiator MUST NOT set ECT on a SYN if it does not also attempt If the SYN sent by a TCP initiator does not attempt to negotiate
to negotiate Accurate ECN feedback in the same SYN. Accurate ECN feedback, or does not use an equivalent safety
mechanism, it MUST still comply with RFC 3168, which says that a TCP
initiator "MUST NOT set ECT on a SYN".
If the TCP initiator does not support AccECN, the rest of The only envisaged examples of "equivalent safety mechanisms" are: a)
Section 3.2.1 does not apply. It solely applies to the case where some future TCP ECN feedback protocol, perhaps evolved from AccECN,
the TCP initiator supports AccECN as well as ECN++. that feeds back CE marking on a SYN; b) setting the initial window to
1 SMSS. IW=1 is NOT RECOMMENDED because it could degrade
performance, but might be appropriate for certain lightweight TCP
implementations.
See Section 4.2 for discussion and rationale.
If the TCP initiator does not set ECT on the SYN, the rest of
Section 3.2.1 does not apply.
3.2.1.2. Caching where to use ECT on SYNs 3.2.1.2. Caching where to use ECT on SYNs
As explained above, this subsection only applies if the ECN++ TCP This subsection only applies if the ECN++ TCP client set ECTs on the
client also supports AccECN. SYN and supports AccECN.
Until AccECN servers become widely deployed, a TCP initiator that Until AccECN servers become widely deployed, a TCP initiator that
sets ECT on a SYN (which implies the same SYN also requests AccECN, sets ECT on a SYN (which typically implies the same SYN also requests
as required above) SHOULD also maintain a cache entry per server to AccECN, as above) SHOULD also maintain a cache entry per server to
record servers that it is not worth sending an ECT SYN to, e.g. record servers that it is not worth sending an ECT SYN to, e.g.
because they do not support AccECN and therefore have no logic for because they do not support AccECN and therefore have no logic for
congestion markings on the SYN. Mobile hosts MAY maintain a cache congestion markings on the SYN. Mobile hosts MAY maintain a cache
entry per access network to record 'non-ECT SYN' entries against entry per access network to record 'non-ECT SYN' entries against
proxies (see Section 4.2.1). proxies (see Section 4.2.3). This cache can be implemented as part
of the shared state across multiple TCP connections, following
[RFC2140].
Subsequently the initiator will not set ECT on a SYN to such a server Subsequently the initiator will not set ECT on a SYN to such a server
or proxy, but it can still always request AccECN support (because the or proxy, but it can still always request AccECN support (because the
response will state any earlier stage of ECN evolution that the response will state any earlier stage of ECN evolution that the
server supports with no performance penalty). The initiator will server supports with no performance penalty). If a server
discover a server that has upgraded to support AccECN as soon as it subsequently upgrades to support AccECN, the initiator will discover
next connects, then it can remove the server from its cache and this as soon as it next connects, then it can remove the server from
subsequently always set ECT for that server. its cache and subsequently always set ECT for that server.
The client can limit the size of its cache of 'non-ECT SYN' servers. The client can limit the size of its cache of 'non-ECT SYN' servers.
Then, while AccECN is not widely deployed, it will only cache the Then, while AccECN is not widely deployed, it will only cache the
'non-ECT SYN' servers that are most used and most recently used by 'non-ECT SYN' servers that are most used and most recently used by
the client. As the client accesses servers that have been expelled the client. As the client accesses servers that have been expelled
from its cache, it will simply use ECT on the SYN by default. from its cache, it will simply use ECT on the SYN by default.
Servers that do not support ECN as a whole do not need to be recorded Servers that do not support ECN as a whole do not need to be recorded
separately from non-support of AccECN because the response to a separately from non-support of AccECN because the response to a
request for AccECN immediately states which stage in the evolution of request for AccECN immediately states which stage in the evolution of
ECN the server supports (AccECN [I-D.ietf-tcpm-accurate-ecn], classic ECN the server supports (AccECN [I-D.ietf-tcpm-accurate-ecn], classic
ECN [RFC3168] or no ECN). ECN [RFC3168] or no ECN).
The above strategy is named "optimistic ECT and cache failures". It The above strategy is named "optimistic ECT and cache failures". It
is believed to be sufficient based on three measurement studies and is believed to be sufficient based on three measurement studies and
assumptions detailed in Section 4.2.1. However, Section 4.2.1 gives assumptions detailed in Section 4.2.3. However, Section 4.2.3 gives
two other strategies and the choice between them depends on the two other strategies and the choice between them depends on the
implementer's goals and the deployment prevalence of ECN variants in implementer's goals and the deployment prevalence of ECN variants in
the network and on servers, not to mention the prevalence of some the network and on servers, not to mention the prevalence of some
significant bugs. significant bugs.
If the initiator times out without seeing a SYN-ACK, it will If the initiator times out without seeing a SYN-ACK, it will
separately cache this fact (see fall-back in Section 3.2.1.4 for separately cache this fact (see fall-back in Section 3.2.1.4 for
details). details).
3.2.1.3. SYN Congestion Response 3.2.1.3. SYN Congestion Response
As explained above, this subsection only applies if the ECN++ TCP As explained above, this subsection only applies if the ECN++ TCP
client also supports AccECN. client sets ECT on the initial SYN.
If the SYN-ACK returned to the TCP initiator confirms that the server If the SYN-ACK returned to the TCP initiator confirms that the server
supports AccECN, it will also indicate whether or not the SYN was CE- supports AccECN, it will also be able to indicate whether or not the
marked. If the SYN was CE-marked, the initiator MUST reduce its SYN was CE-marked. If the SYN was CE-marked, and if the initial
window is greater than 1 MSS, then, the initiator MUST reduce its
Initial Window (IW) and SHOULD reduce it to 1 SMSS (sender maximum Initial Window (IW) and SHOULD reduce it to 1 SMSS (sender maximum
segment size). segment size). The rationale is the same as that for the response to
CE on a SYN-ACK (Section 4.3.2).
If the initiator has set ECT on the SYN and if the SYN-ACK shows that If the initiator has set ECT on the SYN and if the SYN-ACK shows that
the server does not support AccECN, the TCP initiator MUST the server does not support feedback of a CE on the SYN (e.g. it does
not support AccECN) and if the initial congestion window of the
initiator is greater than 1 MSS, then the TCP initiator MUST
conservatively reduce its Initial Window and SHOULD reduce it to 1 conservatively reduce its Initial Window and SHOULD reduce it to 1
SMSS. A reduction to greater than 1 SMSS MAY be appropriate (see SMSS. A reduction to greater than 1 SMSS MAY be appropriate (see
Section 4.2.1). Conservatism is necessary because a non-AccECN SYN- Section 4.2.1). Conservatism is necessary because the SYN-ACK cannot
ACK cannot show whether the SYN was CE-marked. show whether the SYN was CE-marked.
If the TCP initiator (host A) receives a SYN from the remote end If the TCP initiator (host A) receives a SYN from the remote end
(host B) after it has sent a SYN to B, it indicates the (unusual) (host B) after it has sent a SYN to B, it indicates the (unusual)
case of a simultaneous open. Host A will respond with a SYN-ACK. case of a simultaneous open. Host A will respond with a SYN-ACK.
Host A will probably then receive a SYN-ACK in response to its own Host A will probably then receive a SYN-ACK in response to its own
SYN, after which it can follow the appropriate one of the two SYN, after which it can follow the appropriate one of the two
paragraphs above. paragraphs above.
In all the above cases, the initiator does not have to back off its In all the above cases, the initiator does not have to back off its
retransmission timer as it would in response to a timeout following retransmission timer as it would in response to a timeout following
no response to its SYN [RFC6298], because both the SYN and the SYN- no response to its SYN [RFC6298], because both the SYN and the SYN-
ACK have been successfully delivered through the network. Also, the ACK have been successfully delivered through the network. Also, the
initiator does not need to exit slow start or reduce ssthresh, which initiator does not need to exit slow start or reduce ssthresh, which
is not even required when a SYN is lost [RFC5681]. is not even required when a SYN is lost [RFC5681].
If an initial window of 10 (IW10 [RFC6928]) is implemented, Section 5 If an initial window of more than 3 segments is implemented (e.g.
gives additional recommendations. IW10 [RFC6928]), Section 5 gives additional recommendations.
3.2.1.4. Fall-Back Following No Response to an ECT SYN 3.2.1.4. Fall-Back Following No Response to an ECT SYN
As explained above, this subsection only applies if the ECN++ TCP As explained above, this subsection only applies if the ECN++ TCP
client also supports AccECN. client also sets ECT on the initial SYN.
An ECT SYN might be lost due to an over-zealous path element (or An ECT SYN might be lost due to an over-zealous path element (or
server) blocking ECT packets that do not conform to RFC 3168. Some server) blocking ECT packets that do not conform to RFC 3168. Some
evidence of this was found in a 2014 study [ecn-pam], but in a more evidence of this was found in a 2014 study [ecn-pam], but in a more
recent study using 2017 data [Mandalari18] extensive measurements recent study using 2017 data [Mandalari18] extensive measurements
found no case where ECT on TCP control packets was treated any found no case where ECT on TCP control packets was treated any
differently from ECT on TCP data packets. Loss is commonplace for differently from ECT on TCP data packets. Loss is commonplace for
numerous other reasons, e.g. congestion loss at a non-ECN queue on numerous other reasons, e.g. congestion loss at a non-ECN queue on
the forward or reverse path, transmission errors, etc. the forward or reverse path, transmission errors, etc.
Alternatively, the cause of the loss might be the attempt to Alternatively, the cause of the loss might be the associated attempt
negotiate AccECN, or possibly other unrelated options on the SYN. to negotiate AccECN, or possibly other unrelated options on the SYN.
Therefore, if the timer expires after the TCP initiator has sent the Therefore, if the timer expires after the TCP initiator has sent the
first ECT SYN, it SHOULD make one more attempt to retransmit the SYN first ECT SYN, it SHOULD make one more attempt to retransmit the SYN
with ECT set (backing off the timer as usual). If the retransmission with ECT set (backing off the timer as usual). If the retransmission
timer expires again, it SHOULD retransmit the SYN with the not-ECT timer expires again, it SHOULD retransmit the SYN with the not-ECT
codepoint in the IP header, to expedite connection set-up. If other codepoint in the IP header, to expedite connection set-up. If other
experimental fields or options were on the SYN, it will also be experimental fields or options were on the SYN, it will also be
necessary to follow their specifications for fall-back too. It would necessary to follow their specifications for fall-back too. It would
make sense to coordinate all the strategies for fall-back in order to make sense to coordinate all the strategies for fall-back in order to
isolate the specific cause of the problem. isolate the specific cause of the problem.
skipping to change at page 12, line 35 skipping to change at page 13, line 39
For the ECN++ experiment, the TCP implementation will set ECT on SYN- For the ECN++ experiment, the TCP implementation will set ECT on SYN-
ACKs. It can ignore the requirement in section 6.1.1 of RFC 3168 to ACKs. It can ignore the requirement in section 6.1.1 of RFC 3168 to
set not-ECT on a SYN-ACK, as per Section 4.3 of [RFC8311]. set not-ECT on a SYN-ACK, as per Section 4.3 of [RFC8311].
3.2.2.2. SYN-ACK Congestion Response 3.2.2.2. SYN-ACK Congestion Response
A host that sets ECT on SYN-ACKs MUST reduce its initial window in A host that sets ECT on SYN-ACKs MUST reduce its initial window in
response to any congestion feedback, whether using classic ECN or response to any congestion feedback, whether using classic ECN or
AccECN (see Section 4.3.1). It SHOULD reduce it to 1 SMSS. This is AccECN (see Section 4.3.1). It SHOULD reduce it to 1 SMSS. This is
different to the behaviour specified in an earlier experiment that different to the behaviour specified in an earlier experiment that
set ECT on the SYN-ACK [RFC5562]. This is justified in Section 4.3. set ECT on the SYN-ACK [RFC5562]. This is justified in
Section 4.3.2.
The responder does not have to back off its retransmission timer The responder does not have to back off its retransmission timer
because the ECN feedback proves that the network is delivering because the ECN feedback proves that the network is delivering
packets successfully and is not severely overloaded. Also the packets successfully and is not severely overloaded. Also the
responder does not have to leave slow start or reduce ssthresh, which responder does not have to leave slow start or reduce ssthresh, which
is not even required when a SYN-ACK has been lost. is not even required when a SYN-ACK has been lost.
The congestion response to CE-marking on a SYN-ACK for a server that The congestion response to CE-marking on a SYN-ACK for a server that
implements either the TCP Fast Open experiment (TFO [RFC7413]) or the implements either the TCP Fast Open experiment (TFO [RFC7413]) or
initial window of 10 experiment (IW10 [RFC6928]) is discussed in experimentation with an initial window of more than 3 segments (e.g.
Section 5. IW10 [RFC6928]) is discussed in Section 5.
3.2.2.3. Fall-Back Following No Response to an ECT SYN-ACK 3.2.2.3. Fall-Back Following No Response to an ECT SYN-ACK
After the responder sends a SYN-ACK with ECT set, if its After the responder sends a SYN-ACK with ECT set, if its
retransmission timer expires it SHOULD retransmit one more SYN-ACK retransmission timer expires it SHOULD retransmit one more SYN-ACK
with ECT set (and back-off its timer as usual). If the timer expires with ECT set (and back-off its timer as usual). If the timer expires
again, it SHOULD retransmit the SYN-ACK with not-ECT in the IP again, it SHOULD retransmit the SYN-ACK with not-ECT in the IP
header. If other experimental fields or options were on the initial header. If other experimental fields or options were on the initial
SYN-ACK, it will also be necessary to follow their specifications for SYN-ACK, it will also be necessary to follow their specifications for
fall-back. It would make sense to co-ordinate all the strategies for fall-back. It would make sense to co-ordinate all the strategies for
skipping to change at page 17, line 43 skipping to change at page 18, line 43
TCP packet. TCP packet.
3.3.2. SYN (Receive) 3.3.2. SYN (Receive)
RFC 3168 negotiates the use of ECN for the connection end-to-end RFC 3168 negotiates the use of ECN for the connection end-to-end
using the ECN flags in the TCP header. When RFC3168 says that "A using the ECN flags in the TCP header. When RFC3168 says that "A
host MUST NOT set ECT on SYN ... packets." it is silent as to what a host MUST NOT set ECT on SYN ... packets." it is silent as to what a
TCP server ought to do if it receives a SYN packet with a non-zero TCP server ought to do if it receives a SYN packet with a non-zero
IP/ECN field. IP/ECN field.
Some implementations of TCP servers (e.g. current Linux) assume that, As the time of the writing, some implementations of TCP servers (see
if a host receives a SYN with a non-zero IP/ECN field, it must be due Section 4.2.2.2) assume that, if a host receives a SYN with a non-
to network mangling, and they disable ECN for the rest of the zero IP/ECN field, it must be due to network mangling, and they
connection. Section 4.2.2.2 finds that this type of network mangling disable ECN for the rest of the connection. Section 4.2.2.2 also
seems to be virtually non-existent so it would be preferable to finds that this type of network mangling seems to be virtually non-
report any such mangling so it can be fixed. existent so it would be preferable to report any such mangling so it
can be fixed.
For the avoidance of doubt, the normative statements for all TCP For the avoidance of doubt, the normative statements for all TCP
control packets in Section 3.3.1 are interpreted for the case when a control packets in Section 3.3.1 are interpreted for the case when a
SYN is received as follows: SYN is received as follows:
o Any TCP server implementation SHOULD accept receipt of a valid SYN o Any TCP server implementation SHOULD accept receipt of a valid SYN
that requests ECN support for the connection, irrespective of the that requests ECN support for the connection, irrespective of the
IP/ECN field of the SYN. If any existing implementation does not, IP/ECN field of the SYN. If any existing implementation does not,
it SHOULD be updated to do so. it SHOULD be updated to do so.
skipping to change at page 18, line 34 skipping to change at page 19, line 38
o Any TCP implementation SHOULD accept receipt of a pure ACK with a o Any TCP implementation SHOULD accept receipt of a pure ACK with a
non-zero ECN field, despite current RFCs precluding the sending of non-zero ECN field, despite current RFCs precluding the sending of
such packets. such packets.
o A TCP implementation taking part in the ECN++ experiment MUST o A TCP implementation taking part in the ECN++ experiment MUST
accept receipt of a pure ACK with a non-zero ECN field. accept receipt of a pure ACK with a non-zero ECN field.
The question of whether and how the receiver of pure ACKs is required The question of whether and how the receiver of pure ACKs is required
to feed back any CE marks on them is outside the scope of the present to feed back any CE marks on them is outside the scope of the present
specification because it is a matter for the relevant feedback specification because it is a matter for the relevant feedback
specification ([RFC3168] or [I-D.ietf-tcpm-accurate-ecn]). Currently specification ([RFC3168] or [I-D.ietf-tcpm-accurate-ecn]). AccECN
AccECN feedback is required to count CE marking of any control packet feedback is required to count CE marking of any control packet
including pure ACKs. Whereas RFC 3168 is silent on this point, so including pure ACKs. Whereas RFC 3168 is silent on this point, so
feedback of CE-markings might be implementation specific (see feedback of CE-markings might be implementation specific (see
Section 4.4.1.1). Section 4.4.1.1).
3.3.4. FIN (Receive) 3.3.4. FIN (Receive)
The TCP data receiver MUST ignore the CE codepoint on incoming FINs The TCP data receiver MUST ignore the CE codepoint on incoming FINs
that fail any validity check. The validity check in section 5.2 of that fail any validity check. The validity check in section 5.2 of
[RFC5961] is RECOMMENDED. [RFC5961] is RECOMMENDED.
skipping to change at page 21, line 9 skipping to change at page 22, line 15
1. An AccECN server uses the 3 'ECN' bits in the TCP header of the 1. An AccECN server uses the 3 'ECN' bits in the TCP header of the
SYN-ACK to respond to the client. 4 of the possible 8 codepoints SYN-ACK to respond to the client. 4 of the possible 8 codepoints
provide enough space for the server to feed back which of the 4 provide enough space for the server to feed back which of the 4
IP/ECN codepoints was on the incoming SYN (including CE of IP/ECN codepoints was on the incoming SYN (including CE of
course). course).
2. If any of these 4 codepoints are in the SYN-ACK, it confirms that 2. If any of these 4 codepoints are in the SYN-ACK, it confirms that
the server supports AccECN and, if another codepoint is returned, the server supports AccECN and, if another codepoint is returned,
it confirms that the server doesn't support AccECN. it confirms that the server doesn't support AccECN.
This still does not seem to allow a client set ECT on a SYN, it only This still does not seem to allow a client to set ECT on a SYN, it
finds out whether the server would have supported it afterwards. The only finds out whether the server would have supported it afterwards.
trick the client uses for ECN++ is to set ECT on the SYN The trick the client uses for ECN++ is to set ECT on the SYN
optimistically then, if the SYN-ACK reveals that the server wouldn't optimistically then, if the SYN-ACK reveals that the server wouldn't
have understood CE on the SYN, the client responds conservatively as have understood CE on the SYN, the client responds conservatively as
if the SYN was marked with CE. if the SYN was marked with CE.
Happily, the appropriate conservative congestion response is to The recommended conservative congestion response is to reduce the
reduce the initial window, and it is extremely rare for a TCP client initial window, which does not affect the performance of very popular
to send more than one packet as its initial request anyway. Any protocols such as HTTP, since it is extremely rare for an HTTP client
to send more than one packet as its initial request anyway (for data
on HTTP/1 & HTTP/2 request sizes see Fig 3 in [Manzoor17]). Any
clients that do frequently use a larger initial window for their clients that do frequently use a larger initial window for their
first message to the server can cache which servers will not first message to the server can cache which servers will not
understand ECT on a SYN (see Section 4.2.3 below). understand ECT on a SYN (see Section 4.2.3 below). If caching is not
practical, such clients could reduce the initial window to say IW2 or
IW3.
EXPERIMENTATION NEEDED: Experiments will be needed to determine
any better strategy for reducing IW in response to congestion on a
SYN, when the server does not support congestion feedback on the
SYN-ACK (whether cached or discovered explicitly).
4.2.2. Argument 1b: ECT Considered Invalid on the SYN 4.2.2. Argument 1b: ECT Considered Invalid on the SYN
Given, until now, ECT-marked SYN packets have been prohibited, it Given, until now, ECT-marked SYN packets have been prohibited, it
cannot be assumed they will be accepted, by TCP middleboxes or cannot be assumed they will be accepted, by TCP middleboxes or
servers. servers.
4.2.2.1. ECT on SYN Considered Invalid by Middleboxes 4.2.2.1. ECT on SYN Considered Invalid by Middleboxes
According to a study using 2014 data [ecn-pam] from a limited range According to a study using 2014 data [ecn-pam] from a limited range
skipping to change at page 24, line 25 skipping to change at page 25, line 37
administrator can make sure that servers support ECN-capable administrator can make sure that servers support ECN-capable
SYN packets. Examples of controlled environments are single- SYN packets. Examples of controlled environments are single-
tenant DCs, and possibly multi-tenant DCs if it is assumed tenant DCs, and possibly multi-tenant DCs if it is assumed
that each tenant mostly communicates with its own VMs. that each tenant mostly communicates with its own VMs.
For unmanaged environments like the public Internet, pragmatically For unmanaged environments like the public Internet, pragmatically
the choice is between strategies (S1), (S2A) and (S2B). The the choice is between strategies (S1), (S2A) and (S2B). The
normative specification for ECT on a SYN in Section 3.2.1 recommends normative specification for ECT on a SYN in Section 3.2.1 recommends
the "optimistic ECT and cache failures" strategy (S2B) but the choice the "optimistic ECT and cache failures" strategy (S2B) but the choice
depends on the implementer's motivation for using ECN++, and the depends on the implementer's motivation for using ECN++, and the
deployment prevalence of different technologies and bug-fixes. For deployment prevalence of different technologies and bug-fixes.
instance, if a user's Internet access bottleneck supported L4S ECN
but not Classic ECN, strategy (S2A) would make most sense and there
would be no point trying to avoid the 'over-strict' test and
negotiate Classic ECN.
o The "pessimistic ECT and cache successes" strategy (S1) suffers o The "pessimistic ECT and cache successes" strategy (S1) suffers
from exposing the initial SYN to the prevailing loss level, even from exposing the initial SYN to the prevailing loss level, even
if the server supports ECT on SYNs, but only on the first if the server supports ECT on SYNs, but only on the first
connection to each AccECN server. If AccECN becomes widely connection to each AccECN server. If AccECN becomes widely
deployed on servers, SYNs to those AccECN servers that are less deployed on servers, SYNs to those AccECN servers that are less
frequently used by the client and therefore don't fit in the cache frequently used by the client and therefore don't fit in the cache
will not benefit from ECN protection at all. will not benefit from ECN protection at all.
o The "optimistic ECT without a cache" strategy (S2A) is the o The "optimistic ECT without a cache" strategy (S2A) is the
simplest. It would satisfy the goal of an implementer who is simplest. It would satisfy the goal of an implementer who is
solely interested in ultra-low latency using AccECN and ECN++ solely interested in low latency using AccECN and ECN++ and is not
(e.g. accessing L4S servers) and is not concerned about fall-back concerned about fall-back to Classic ECN.
to Classic ECN (e.g. when accessing other servers).
o The "optimistic ECT and cache failures" strategy (S2B) exploits o The "optimistic ECT and cache failures" strategy (S2B) exploits
ECT on SYNs from the very first attempt. But if the server turns ECT on SYNs from the very first attempt. But if the server turns
out to be 'over-strict' it will disable ECN for the connection, out to be 'over-strict' it will disable ECN for the connection,
but only for the first connection if it's one of the client's more but only for the first connection if it's one of the client's more
popular servers that fits in the cache. If the server turns out popular servers that fits in the cache. If the server turns out
not to support AccECN, the initiator has to conservatively limit not to support AccECN, the initiator has to conservatively limit
its initial window, but again only for the first connection if its initial window, but again only for the first connection if
it's one of the client's more popular servers (and anyway this it's one of the client's more popular servers (and anyway this
rarely makes any difference when most client requests fit in a rarely makes any difference when most client requests fit in a
skipping to change at page 27, line 41 skipping to change at page 28, line 49
response when it experiences a CE mark having sent only one (small) response when it experiences a CE mark having sent only one (small)
packet. If TCP had been adding one segment per RTT, it would have packet. If TCP had been adding one segment per RTT, it would have
halved its congestion window, but it hasn't established a congestion halved its congestion window, but it hasn't established a congestion
window yet. If it had been exponentially increasing it would have window yet. If it had been exponentially increasing it would have
exited slow start, but it hasn't started exponentially increasing yet exited slow start, but it hasn't started exponentially increasing yet
so it hasn't established a slow-start threshold. so it hasn't established a slow-start threshold.
Therefore, we have to work out a reasoned argument for what to do. Therefore, we have to work out a reasoned argument for what to do.
If an AQM is CE-marking packets, it implies there is already a queue If an AQM is CE-marking packets, it implies there is already a queue
and it is probably already somewhere around the AQM's operating point and it is probably already somewhere around the AQM's operating point
- it is unlikely to be well below and it might be well above. So, it - it is unlikely to be well below and it might be well above. So,
does not seem sensible to add a number of packets at once. On the the more data packets that the client sends in its IW, the more
other hand, it is highly unlikely that the SYN-ACK itself pushed the likely at least one will be CE marked, leading it to exit slow-start
AQM into congestion, so it will be safe to introduce another single early. On the other hand, it is highly unlikely that the SYN-ACK
segment immediately (1 RTT after the SYN-ACK). Therefore, starting itself pushed the AQM into congestion, so it will be safe to
to probe for capacity with a slow start from an initial window of 1 introduce another single segment immediately (1 RTT after the SYN-
segment seems appropriate to the circumstances. This is the approach ACK). Therefore, starting to probe for capacity with a slow start
adopted in Section 3.2.2. from an initial window of 1 segment seems appropriate to the
circumstances. This is the approach adopted in Section 3.2.2.
EXPERIMENTATION NEEDED: Experiments will be needed to check the
above reasoning and determine any better strategy for reducing IW
in response to congestion on a SYN-ACK (or a SYN).
4.3.3. Fall-Back if ECT SYN-ACK Fails 4.3.3. Fall-Back if ECT SYN-ACK Fails
An alternative to the server caching failed connection attempts would An alternative to the server caching failed connection attempts would
be for the server to rely on the client caching failed attempts (on be for the server to rely on the client caching failed attempts (on
the basis that the client would cache a failure whether ECT was the basis that the client would cache a failure whether ECT was
blocked on the SYN or the SYN-ACK). This strategy cannot be used if blocked on the SYN or the SYN-ACK). This strategy cannot be used if
the SYN does not request AccECN support. It works as follows: if the the SYN does not request AccECN support. It works as follows: if the
server receives a SYN that requests AccECN support but is set to not- server receives a SYN that requests AccECN support but is set to not-
ECT, it replies with a SYN-ACK also set to not-ECT. If a middlebox ECT, it replies with a SYN-ACK also set to not-ECT. If a middlebox
skipping to change at page 29, line 24 skipping to change at page 30, line 36
Mechanism feasibility: If ECN is enabled on Pure ACKs, are there, or Mechanism feasibility: If ECN is enabled on Pure ACKs, are there, or
could there be, suitable mechanisms to detect, feed back and could there be, suitable mechanisms to detect, feed back and
respond to ECN-marked Pure ACKs? respond to ECN-marked Pure ACKs?
Do no extra harm: There has never been a mechanism to respond to Do no extra harm: There has never been a mechanism to respond to
loss of non-ECN Pure ACKs. So it seems that adding ECN without a loss of non-ECN Pure ACKs. So it seems that adding ECN without a
response mechanism will do no extra harm to others, while response mechanism will do no extra harm to others, while
improving a connection's own performance (because loss of an ACK improving a connection's own performance (because loss of an ACK
holds back new data). However, if the end systems have no holds back new data). However, if the end systems have no
response mechanism, ECN Pure ACKS do slightly more harm than non- response mechanism, ECN Pure ACKs do slightly more harm than non-
ECN, because the AQM doesn't immediately clear ECT packets from ECN, because the AQM doesn't immediately clear ECT packets from
the queue until it reaches overload and disables ECN. the queue until it reaches overload and disables ECN.
Standards policy: Even if there were no harm to others, does it set Standards policy: Even if there were no harm to others, does it set
an undesirable precedent to allow a flow to use ECN to protect its an undesirable precedent to allow a flow to use ECN to protect its
Pure ACKs from loss, when there is no mechanism to respond to ECN- Pure ACKs from loss, when there is no mechanism to respond to ECN-
marking? marking?
The last two arguments involve value judgements, but they both depend The last two arguments involve value judgements, but they both depend
on the concrete technical question of mechanism feasibility, which on the concrete technical question of mechanism feasibility, which
skipping to change at page 33, line 22 skipping to change at page 34, line 33
RFC 3168 implementations might happen to (unintentionally) provide a RFC 3168 implementations might happen to (unintentionally) provide a
feedback mechanism to support a cwnd response. For those that did feedback mechanism to support a cwnd response. For those that did
not, setting ECT on pure ACKs would be better for the flow's own not, setting ECT on pure ACKs would be better for the flow's own
performance than not setting it. However, where there was no performance than not setting it. However, where there was no
feedback mechanism, setting ECT could do slightly more harm than not feedback mechanism, setting ECT could do slightly more harm than not
setting it. AckCC could provide a complementary response mechanism, setting it. AckCC could provide a complementary response mechanism,
because it is designed to work with RFC 3168 ECN, but it has because it is designed to work with RFC 3168 ECN, but it has
deployment challenges. In summary, a congestion response mechanism deployment challenges. In summary, a congestion response mechanism
is unlikely to be feasible with the installed base of classic ECN. is unlikely to be feasible with the installed base of classic ECN.
During review of this specification, it was decided that allowing This specification uses a safe approach. Allowing hosts to set ECT
hosts to set ECT on Pure ACKs without a feasible response mechanism on Pure ACKs without a feasible response mechanism could result in
would set an undesirable precedent. It would certainly improve the risk. It would certainly improve the flow's own performance, but it
flow's own performance, but it would slightly increase potential harm would slightly increase potential harm to others. Morevoer, if would
to others. Therefore, Section 3.2.3 allows ECT on Pure ACKs if set an undesirable precedent for setting ECT on packets with no
AccECN feedback has been negotiated, but not with classic RFC 3168 mechanism to respond to any resulting congestion signals. Therefore,
ECN feedback. Section 3.2.3 allows ECT on Pure ACKs if AccECN feedback has been
negotiated, but not with classic RFC 3168 ECN feedback.
4.5. Window Probes 4.5. Window Probes
Section 6.1.6 of RFC 3168 presents only the reliability argument for Section 6.1.6 of RFC 3168 presents only the reliability argument for
prohibiting ECT on Window probes: prohibiting ECT on Window probes:
"If a window probe packet is dropped in the network, this loss is "If a window probe packet is dropped in the network, this loss is
not detected by the receiver. Therefore, the TCP data sender MUST not detected by the receiver. Therefore, the TCP data sender MUST
NOT set either an ECT codepoint or the CWR bit on window probe NOT set either an ECT codepoint or the CWR bit on window probe
packets. packets.
skipping to change at page 37, line 7 skipping to change at page 38, line 24
something's broken, don't fix it. something's broken, don't fix it.
If an implementation has had to disable ECT to ensure the first If an implementation has had to disable ECT to ensure the first
packet of a flow (SYN or SYN-ACK) gets through, the question arises packet of a flow (SYN or SYN-ACK) gets through, the question arises
whether it ought to disable ECT on all subsequent control packets whether it ought to disable ECT on all subsequent control packets
within the same TCP connection. Without evidence of any such within the same TCP connection. Without evidence of any such
problems, this seems unnecessarily cautious. Particularly given it problems, this seems unnecessarily cautious. Particularly given it
would be hard to detect loss of most other types of TCP control would be hard to detect loss of most other types of TCP control
packets that are not ACK'd. And particularly given that packets that are not ACK'd. And particularly given that
unnecessarily removing ECT from other control packets could lead to unnecessarily removing ECT from other control packets could lead to
performance problems, e.g. by directing them into an inferior queue performance problems, e.g. by directing them into another queue
[I-D.ietf-tsvwg-ecn-l4s-id] or over a different path, because some [I-D.ietf-tsvwg-ecn-l4s-id] or over a different path, because some
broken multipath equipment (erroneously) routes based on all 8 bits broken multipath equipment (erroneously) routes based on all 8 bits
of the Diffserv field. of the Diffserv field.
In the case where a connection starts without ECT on the SYN (perhaps In the case where a connection starts without ECT on the SYN (perhaps
because problems with previous connections had been cached), there because problems with previous connections had been cached), there
will have been no test for ECT traversal in the client-server will have been no test for ECT traversal in the client-server
direction until the pure ACK that completes the handshake. It is direction until the pure ACK that completes the handshake. It is
possible that some middlebox might block ECT on this pure ACK or on possible that some middlebox might block ECT on this pure ACK or on
later retransmissions of lost packets. Similarly, after a route later retransmissions of lost packets. Similarly, after a route
skipping to change at page 38, line 11 skipping to change at page 39, line 25
IW10 is an experiment to determine whether it is safe for TCP to use IW10 is an experiment to determine whether it is safe for TCP to use
an initial window of 10 SMSS [RFC6928]. an initial window of 10 SMSS [RFC6928].
This subsection does not recommend any additions to the present This subsection does not recommend any additions to the present
specification in order to interwork with IW10. The specifications as specification in order to interwork with IW10. The specifications as
they stand are safe, and there is only a corner-case with ECT on the they stand are safe, and there is only a corner-case with ECT on the
SYN where performance could be occasionally improved, as explained SYN where performance could be occasionally improved, as explained
below. below.
As specified in Section 3.2.1.1, a TCP initiator can only set ECT on As specified in Section 3.2.1.1, a TCP initiator will typically only
the SYN if it requests AccECN support. If, however, the SYN-ACK set ECT on the SYN if it requests AccECN support. If, however, the
tells the initiator that the responder does not support AccECN, SYN-ACK tells the initiator that the responder does not support
Section 3.2.1.1 advises the initiator to conservatively reduce its AccECN, Section 3.2.1.1 advises the initiator to conservatively
initial window to 1 SMSS because, if the SYN was CE-marked, the SYN- reduce its initial window, preferably to 1 SMSS because, if the SYN
ACK has no way to feed that back. was CE-marked, the SYN-ACK has no way to feed that back.
If the initiator implements IW10, it seems rather over-conservative If the initiator implements IW10, it seems rather over-conservative
to reduce IW from 10 to 1 just in case a congestion marking was to reduce IW from 10 to 1 just in case a congestion marking was
missed. Nonetheless, the reduction to 1 SMSS will rarely harm missed. Nonetheless, a reduction to 1 SMSS will rarely harm
performance, because: performance, because:
o as long as the initiator is caching failures to negotiate AccECN, o as long as the initiator is caching failures to negotiate AccECN,
subsequent attempts to access the same server will not use ECT on subsequent attempts to access the same server will not use ECT on
the SYN anyway, so there will no longer be any need to the SYN anyway, so there will no longer be any need to
conservatively reduce IW; conservatively reduce IW;
o currently, at least for web sessions, it is extremely rare for a o currently, at least for web sessions, it is extremely rare for a
TCP initiator (client) to have more than one data segment to send TCP initiator (client) to have more than one data segment to send
at the start of a TCP connection [28; Fig 3] - IW10 is primarily at the start of a TCP connection (see Fig 3 in [Manzoor17]) - IW10
exploited by TCP servers. is primarily exploited by TCP servers.
If a responder receives feedback that the SYN-ACK was CE-marked, If a responder receives feedback that the SYN-ACK was CE-marked,
Section 3.2.2.2 mandates that it reduces its initial window to 1 Section 3.2.2.2 recommends that it reduces its initial window,
SMSS. When the responder also implements IW10, it is particularly preferably to 1 SMSS. When the responder also implements IW10, it
important to adhere to this requirement in order to avoid overflowing might again seem rather over-conservative to reduce IW from 10 to 1.
a queue that is clearly already congested. But in this case the rationale is somewhat different:
o Feedback that the SYN-ACK was CE-marked is an explicit indication
that the queue has been building, not just uncertainty due to
absence of feedback;
o Given it is now likely that a queue already exists, the more data
packets that the server sends in its IW, the more likely at least
one will be CE marked, leading it to exit slow-start early.
Experimentation will be needed to determine the best strategy. It
should be noted that experience from recent congestion avoidance
experiments where the window is reduced by less than half is not
necessarily applicable to a flow start scenario. Reducing cwnd by
less is one thing. Reducing an increase in cwnd by less is another.
5.2. TFO 5.2. TFO
TCP Fast Open (TFO [RFC7413]) is an experiment to remove the round TCP Fast Open (TFO [RFC7413]) is an experiment to remove the round
trip delay of TCP's 3-way hand-shake (3WHS). A TFO initiator caches trip delay of TCP's 3-way hand-shake (3WHS). A TFO initiator caches
a cookie from a previous connection with a TFO-enabled server. Then, a cookie from a previous connection with a TFO-enabled server. Then,
for subsequent connections to the same server, any data included on for subsequent connections to the same server, any data included on
the SYN can be passed directly to the server application, which can the SYN can be passed directly to the server application, which can
then return up to an initial window of response data on the SYN-ACK then return up to an initial window of response data on the SYN-ACK
and on data segments straight after it, without waiting for the ACK and on data segments straight after it, without waiting for the ACK
skipping to change at page 39, line 12 skipping to change at page 40, line 41
TCP control packets can be combined without altering either TCP control packets can be combined without altering either
specification, which is justified as follows: specification, which is justified as follows:
o The handling of ECN marking on a SYN is no different whether or o The handling of ECN marking on a SYN is no different whether or
not it carries data. not it carries data.
o In response to any CE-marking on the SYN-ACK, the responder adopts o In response to any CE-marking on the SYN-ACK, the responder adopts
the normal response to congestion, as discussed in Section 7.2 of the normal response to congestion, as discussed in Section 7.2 of
[RFC7413]. [RFC7413].
5.3. TCP Derivatives 5.3. L4S
A Low Latency Low Loss Scalable throughput (L4S) variant of TCP such
as TCP Prague [PragueLinux] is mandated to negotiate AccECN feedback,
and strongly recommended to use ECN++ [I-D.ietf-tsvwg-ecn-l4s-id].
The L4S experiment and the present ECN++ experiment can be combined
without altering any of the specifications. The only difference
would be in the recommendation of the best SYN cache strategy.
The normative specification for ECT on a SYN in Section 3.2.1
recommends the "optimistic ECT and cache failures" strategy (S2B
defined in Section 4.2.3) for the general Internet. However, if a
user's Internet access bottleneck supported L4S ECN but not Classic
ECN, the "optimistic ECT without a cache" strategy (S2A) would make
most sense, because there would be little point trying to avoid the
'over-strict' test and negotiate Classic ECN, if L4S ECN but not
Classic ECN was available on that user's access link (as is the case
with Low Latency DOCSIS [DOCSIS3.1]).
Strategy (S2A) is the simplest, because it requires no cache. It
would satisfy the goal of an implementer who is solely interested in
ultra-low latency using AccECN and ECN++ (e.g. accessing L4S servers)
and is not concerned about fall-back to Classic ECN (e.g. when
accessing other servers).
5.4. Other transport protocols
Experience from experiments on adding ECN support to all TCP packets Experience from experiments on adding ECN support to all TCP packets
ought to be directly transferable between TCP and derivatives of TCP, ought to be directly transferable between TCP and other transport
like SCTP or QUIC. protocols, like SCTP or QUIC.
Stream Control Transmission Protocol (SCTP [RFC4960]) is a standards Stream Control Transmission Protocol (SCTP [RFC4960]) is a standards
track transport protocol derived from TCP. SCTP currently does not track transport protocol derived from TCP. SCTP currently does not
include ECN support, but Appendix A of RFC 4960 broadly describes how include ECN support, but Appendix A of RFC 4960 broadly describes how
it would be supported and a (long-expired) draft on the addition of it would be supported and a (long-expired) draft on the addition of
ECN to SCTP has been produced [I-D.stewart-tsvwg-sctpecn]. This ECN to SCTP has been produced [I-D.stewart-tsvwg-sctpecn]. This
draft avoided setting ECT on control packets and retransmissions, draft avoided setting ECT on control packets and retransmissions,
closely following the arguments in RFC 3168. closely following the arguments in RFC 3168.
QUIC [I-D.ietf-quic-transport] is another standards track transport QUIC [I-D.ietf-quic-transport] is another standards track transport
skipping to change at page 40, line 8 skipping to change at page 42, line 17
Thanks to Mirja Kuehlewind, David Black, Padma Bhooma, Gorry Thanks to Mirja Kuehlewind, David Black, Padma Bhooma, Gorry
Fairhurst, Michael Scharf, Yuchung Cheng and Christophe Paasch for Fairhurst, Michael Scharf, Yuchung Cheng and Christophe Paasch for
their useful reviews. their useful reviews.
The work of Marcelo Bagnulo has been performed in the framework of The work of Marcelo Bagnulo has been performed in the framework of
the H2020-ICT-2014-2 project 5G NORMA. His contribution reflects the the H2020-ICT-2014-2 project 5G NORMA. His contribution reflects the
consortium's view, but the consortium is not liable for any use that consortium's view, but the consortium is not liable for any use that
may be made of any of the information contained therein. may be made of any of the information contained therein.
Bob Briscoe's contribution was partly funded by the Research Council Bob Briscoe's contribution was partly funded by the Research Council
of Norway through the TimeIn project. The views expressed here are of Norway through the TimeIn project, partly by CableLabs and partly
solely those of the authors. by the Comcast Innovation Fund. The views expressed here are solely
those of the authors.
9. References 9. References
9.1. Normative References 9.1. Normative References
[I-D.ietf-tcpm-accurate-ecn] [I-D.ietf-tcpm-accurate-ecn]
Briscoe, B., Kuehlewind, M., and R. Scheffenegger, "More Briscoe, B., Kuehlewind, M., and R. Scheffenegger, "More
Accurate ECN Feedback in TCP", draft-ietf-tcpm-accurate- Accurate ECN Feedback in TCP", draft-ietf-tcpm-accurate-
ecn-08 (work in progress), March 2019. ecn-09 (work in progress), July 2019.
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, DOI 10.17487/RFC0793, September 1981,
<https://www.rfc-editor.org/info/rfc793>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
of Explicit Congestion Notification (ECN) to IP", of Explicit Congestion Notification (ECN) to IP",
RFC 3168, DOI 10.17487/RFC3168, September 2001, RFC 3168, DOI 10.17487/RFC3168, September 2001,
<https://www.rfc-editor.org/info/rfc3168>. <https://www.rfc-editor.org/info/rfc3168>.
[RFC5961] Ramaiah, A., Stewart, R., and M. Dalal, "Improving TCP's [RFC5961] Ramaiah, A., Stewart, R., and M. Dalal, "Improving TCP's
Robustness to Blind In-Window Attacks", RFC 5961, Robustness to Blind In-Window Attacks", RFC 5961,
DOI 10.17487/RFC5961, August 2010, DOI 10.17487/RFC5961, August 2010,
<https://www.rfc-editor.org/info/rfc5961>. <https://www.rfc-editor.org/info/rfc5961>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8311] Black, D., "Relaxing Restrictions on Explicit Congestion [RFC8311] Black, D., "Relaxing Restrictions on Explicit Congestion
Notification (ECN) Experimentation", RFC 8311, Notification (ECN) Experimentation", RFC 8311,
DOI 10.17487/RFC8311, January 2018, DOI 10.17487/RFC8311, January 2018,
<https://www.rfc-editor.org/info/rfc8311>. <https://www.rfc-editor.org/info/rfc8311>.
9.2. Informative References 9.2. Informative References
[DOCSIS3.1]
CableLabs, "MAC and Upper Layer Protocols Interface
(MULPI) Specification, CM-SP-MULPIv3.1", Data-Over-Cable
Service Interface Specifications DOCSIS(R) 3.1 Version i17
or later, January 2019, <https://specification-
search.cablelabs.com/CM-SP-MULPIv3.1>.
[ecn-overload] [ecn-overload]
Steen, H., "Destruction Testing: Ultra-Low Delay using Steen, H., "Destruction Testing: Ultra-Low Delay using
Dual Queue Coupled Active Queue Management", Masters Dual Queue Coupled Active Queue Management", Masters
Thesis, Uni Oslo , May 2017, Thesis, Uni Oslo , May 2017,
<https://www.duo.uio.no/bitstream/handle/10852/57424/ <https://www.duo.uio.no/bitstream/handle/10852/57424/
thesis-henrste.pdf?sequence=1>. thesis-henrste.pdf?sequence=1>.
[ecn-pam] Trammell, B., Kuehlewind, M., Boppart, D., Learmonth, I., [ecn-pam] Trammell, B., Kuehlewind, M., Boppart, D., Learmonth, I.,
Fairhurst, G., and R. Scheffenegger, "Enabling Internet- Fairhurst, G., and R. Scheffenegger, "Enabling Internet-
Wide Deployment of Explicit Congestion Notification", Wide Deployment of Explicit Congestion Notification",
skipping to change at page 41, line 19 skipping to change at page 43, line 40
(PAM'15) pp193-205, 2015, <https://link.springer.com/ (PAM'15) pp193-205, 2015, <https://link.springer.com/
chapter/10.1007/978-3-319-15509-8_15>. chapter/10.1007/978-3-319-15509-8_15>.
[ECN-PLUS] [ECN-PLUS]
Kuzmanovic, A., "The Power of Explicit Congestion Kuzmanovic, A., "The Power of Explicit Congestion
Notification", ACM SIGCOMM 35(4):61--72, 2005, Notification", ACM SIGCOMM 35(4):61--72, 2005,
<http://dl.acm.org/citation.cfm?id=1080100>. <http://dl.acm.org/citation.cfm?id=1080100>.
[I-D.ietf-quic-transport] [I-D.ietf-quic-transport]
Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed Iyengar, J. and M. Thomson, "QUIC: A UDP-Based Multiplexed
and Secure Transport", draft-ietf-quic-transport-20 (work and Secure Transport", draft-ietf-quic-transport-23 (work
in progress), April 2019. in progress), September 2019.
[I-D.ietf-tsvwg-ecn-l4s-id] [I-D.ietf-tsvwg-ecn-l4s-id]
Schepper, K. and B. Briscoe, "Identifying Modified Schepper, K. and B. Briscoe, "Identifying Modified
Explicit Congestion Notification (ECN) Semantics for Explicit Congestion Notification (ECN) Semantics for
Ultra-Low Queuing Delay (L4S)", draft-ietf-tsvwg-ecn-l4s- Ultra-Low Queuing Delay (L4S)", draft-ietf-tsvwg-ecn-l4s-
id-06 (work in progress), March 2019. id-07 (work in progress), July 2019.
[I-D.ietf-tsvwg-l4s-arch] [I-D.ietf-tsvwg-l4s-arch]
Briscoe, B., Schepper, K., and M. Bagnulo, "Low Latency, Briscoe, B., Schepper, K., Bagnulo, M., and G. White, "Low
Low Loss, Scalable Throughput (L4S) Internet Service: Latency, Low Loss, Scalable Throughput (L4S) Internet
Architecture", draft-ietf-tsvwg-l4s-arch-03 (work in Service: Architecture", draft-ietf-tsvwg-l4s-arch-04 (work
progress), October 2018. in progress), July 2019.
[I-D.stewart-tsvwg-sctpecn] [I-D.stewart-tsvwg-sctpecn]
Stewart, R., Tuexen, M., and X. Dong, "ECN for Stream Stewart, R., Tuexen, M., and X. Dong, "ECN for Stream
Control Transmission Protocol (SCTP)", draft-stewart- Control Transmission Protocol (SCTP)", draft-stewart-
tsvwg-sctpecn-05 (work in progress), January 2014. tsvwg-sctpecn-05 (work in progress), January 2014.
[judd-nsdi] [judd-nsdi]
Judd, G., "Attaining the promise and avoiding the pitfalls Judd, G., "Attaining the promise and avoiding the pitfalls
of TCP in the Datacenter", USENIX Symposium on Networked of TCP in the Datacenter", USENIX Symposium on Networked
Systems Design and Implementation (NSDI'15) pp.145-157, Systems Design and Implementation (NSDI'15) pp.145-157,
skipping to change at page 42, line 18 skipping to change at page 44, line 42
over Mobile", IEEE Communications Magazine , March 2018, over Mobile", IEEE Communications Magazine , March 2018,
<https://ieeexplore.ieee.org/document/8316790>. <https://ieeexplore.ieee.org/document/8316790>.
[Manzoor17] [Manzoor17]
Manzoor, J., Drago, I., and R. Sadre, "How HTTP/2 is Manzoor, J., Drago, I., and R. Sadre, "How HTTP/2 is
changing Web traffic and how to detect it", In Proc: changing Web traffic and how to detect it", In Proc:
Network Traffic Measurement and Analysis Conference (TMA) Network Traffic Measurement and Analysis Conference (TMA)
2017 pp.1-9, June 2017, 2017 pp.1-9, June 2017,
<https://ieeexplore.ieee.org/document/8002899>. <https://ieeexplore.ieee.org/document/8002899>.
[PragueLinux]
Briscoe, B., De Schepper, K., Albisser, O., Misund, J.,
Tilmans, O., Kuehlewind, M., and A. Ahmed, "Implementing
the `TCP Prague' Requirements for Low Latency Low Loss
Scalable Throughput (L4S)", Proc. Linux Netdev 0x13 ,
March 2019, <https://www.netdevconf.org/0x13/
session.html?talk-tcp-prague-l4s>.
[relax-strict-ecn] [relax-strict-ecn]
Tilmans, O., "tcp: Accept ECT on SYN in the presence of Tilmans, O., "tcp: Accept ECT on SYN in the presence of
RFC8311", Linux netdev patch list , April 2019, RFC8311", Linux netdev patch list , April 2019,
<https://lore.kernel.org/patchwork/patch/1057812/>. <https://lore.kernel.org/patchwork/patch/1057812/>.
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, DOI 10.17487/RFC0793, September 1981,
<https://www.rfc-editor.org/info/rfc793>.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, Communication Layers", STD 3, RFC 1122,
DOI 10.17487/RFC1122, October 1989, DOI 10.17487/RFC1122, October 1989,
<https://www.rfc-editor.org/info/rfc1122>. <https://www.rfc-editor.org/info/rfc1122>.
[RFC2140] Touch, J., "TCP Control Block Interdependence", RFC 2140,
DOI 10.17487/RFC2140, April 1997,
<https://www.rfc-editor.org/info/rfc2140>.
[RFC3540] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit [RFC3540] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit
Congestion Notification (ECN) Signaling with Nonces", Congestion Notification (ECN) Signaling with Nonces",
RFC 3540, DOI 10.17487/RFC3540, June 2003, RFC 3540, DOI 10.17487/RFC3540, June 2003,
<https://www.rfc-editor.org/info/rfc3540>. <https://www.rfc-editor.org/info/rfc3540>.
[RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol", [RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol",
RFC 4960, DOI 10.17487/RFC4960, September 2007, RFC 4960, DOI 10.17487/RFC4960, September 2007,
<https://www.rfc-editor.org/info/rfc4960>. <https://www.rfc-editor.org/info/rfc4960>.
[RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common
skipping to change at page 44, line 15 skipping to change at page 46, line 47
Universidad Carlos III de Madrid Universidad Carlos III de Madrid
Av. Universidad 30 Av. Universidad 30
Leganes, Madrid 28911 Leganes, Madrid 28911
SPAIN SPAIN
Phone: 34 91 6249500 Phone: 34 91 6249500
Email: marcelo@it.uc3m.es Email: marcelo@it.uc3m.es
URI: http://www.it.uc3m.es URI: http://www.it.uc3m.es
Bob Briscoe Bob Briscoe
CableLabs Independent
UK UK
Email: ietf@bobbriscoe.net Email: ietf@bobbriscoe.net
URI: http://bobbriscoe.net/ URI: http://bobbriscoe.net/
 End of changes. 60 change blocks. 
183 lines changed or deleted 286 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/