--- 1/draft-ietf-tcpm-ecnsyn-06.txt 2008-11-03 22:12:10.000000000 +0100 +++ 2/draft-ietf-tcpm-ecnsyn-07.txt 2008-11-03 22:12:10.000000000 +0100 @@ -1,22 +1,22 @@ Internet Engineering Task Force A. Kuzmanovic INTERNET-DRAFT A. Mondal Intended status: Proposed Standard Northwestern University -Expires: 22 February 2009 S. Floyd +Expires: 3 May 2009 S. Floyd Updates: 3168 ICIR K.K. Ramakrishnan AT&T - 22 August 2008 + 3 November 2008 Adding Explicit Congestion Notification (ECN) Capability to TCP's SYN/ACK Packets - draft-ietf-tcpm-ecnsyn-06.txt + draft-ietf-tcpm-ecnsyn-07.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that @@ -27,80 +27,93 @@ and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. - This Internet-Draft will expire on August 2008. + This Internet-Draft will expire on May 2009. Copyright Notice Copyright (C) The IETF Trust (2008). Abstract This draft specifies a modification to RFC 3168 to allow TCP SYN/ACK packets to be ECN-Capable. For TCP, RFC 3168 only specifies setting an ECN-Capable codepoint on data packets, and not on SYN and SYN/ACK packets. However, because of the high cost to the TCP transfer of having a SYN/ACK packet dropped, with the resulting retransmit timeout, this document specifies the use of ECN for the SYN/ACK packet itself, when sent in response to a SYN packet with the two ECN flags set in the TCP header, indicating a willingness to use ECN. - Setting TCP SYN/ACK packets as ECN-Capable can be of great benefit to - the TCP connection, avoiding the severe penalty of a retransmit - timeout for a connection that has not yet started placing a load on - the network. The sender of the SYN/ACK packet must respond to a - report of an ECN-marked SYN/ACK packet by reducing its initial - congestion window from two, three, or four segments to one segment, - thereby reducing the subsequent load from that connection on the - network. This document updates RFC 3168. + Setting the initial TCP SYN/ACK packet as ECN-Capable can be of great + benefit to the TCP connection, avoiding the severe penalty of a + retransmit timeout for a connection that has not yet started placing + a load on the network. The TCP responder (the sender of the SYN/ACK + packet) must reply to a report of an ECN-marked SYN/ACK packet by + resending a SYN/ACK packet that is not ECN-Capable. If the resent + SYN/ACK packet is acknowledged, then the TCP responder reduces its + initial congestion window from two, three, or four segments to one + segment, thereby reducing the subsequent load from that connection on + the network. If instead the SYN/ACK packet is dropped, or for some + other reason the TCP responder does not receive an acknowledgement in + the specified time, the TCP responder follows TCP standards for a + dropped SYN/ACK packet (setting the retransmit timer). This document + updates RFC 3168. Table of Contents 1. Introduction ....................................................5 - 2. Conventions and Terminology .....................................6 + 2. Conventions and Terminology .....................................7 3. Specification ...................................................7 - 3.1. SYN/ACK Packets Dropped in the Network .....................7 - 3.2. SYN/ACK Packets ECN-Marked in the Network ..................8 - 3.3. Management Interface ......................................10 - 4. Discussion .....................................................10 - 4.1. Flooding Attacks ..........................................10 - 4.2. The TCP SYN Packet ........................................11 - 4.3. SYN/ACK Packets and Packet Size ...........................11 - 4.4. Response to ECN-marking of SYN/ACK Packets ................12 - 5. Related Work ...................................................13 - 6. Performance Evaluation .........................................14 - 6.1. The Costs and Benefit of Adding ECN-Capability ............14 + 3.1. SYN/ACK Packets Dropped in the Network .....................8 + 3.2. SYN/ACK Packets ECN-Marked in the Network ..................9 + 3.3. Management Interface ......................................11 + 4. Discussion .....................................................12 + 4.1. Flooding Attacks ..........................................12 + 4.2. The TCP SYN Packet ........................................12 + 4.3. SYN/ACK Packets and Packet Size ...........................13 + 4.4. Response to ECN-marking of SYN/ACK Packets ................13 + 5. Related Work ...................................................15 + 6. Performance Evaluation .........................................16 + 6.1. The Costs and Benefit of Adding ECN-Capability ............16 6.2. An Evaluation of Different Responses to ECN-Marked SYN/ACK - Packets ........................................................15 - 7. Security Considerations ........................................16 - 7.1. 'Bad' Routers or Middleboxes ..............................16 - 7.2. Congestion Collapse .......................................16 - 8. Conclusions ....................................................17 - 9. Acknowledgements ...............................................18 - A. Report on Simulations ..........................................18 - A.1. Simulations with RED in Packet Mode .......................19 - A.2. Simulations with RED in Byte Mode .........................21 - B. Issues of Incremental Deployment ...............................23 - Normative References ..............................................26 - Informative References ............................................26 - IANA Considerations ...............................................27 - Full Copyright Statement ..........................................28 - Intellectual Property .............................................28 + Packets ........................................................17 + 7. Security Considerations ........................................18 + 7.1. 'Bad' Routers or Middleboxes ..............................18 + 7.2. Congestion Collapse .......................................19 + 8. Conclusions ....................................................19 + 9. Acknowledgements ...............................................20 + A. Report on Simulations ..........................................20 + A.1. Simulations with RED in Packet Mode .......................21 + A.2. Simulations with RED in Byte Mode .........................25 + B. Issues of Incremental Deployment ...............................27 + Normative References ..............................................30 + Informative References ............................................30 + IANA Considerations ...............................................31 + Full Copyright Statement ..........................................32 + Intellectual Property .............................................32 NOTE TO RFC EDITOR: PLEASE DELETE THIS NOTE UPON PUBLICATION. + Changes from draft-ietf-tcpm-ecnsyn-06: + + * Updated text and simulation results to specify ECN+/TryOnce + instead of ECN+. Added tables on CDFs. + + * Acknowledged Adam's Linux implementation of ECN+/TryOnce. + Changes from draft-ietf-tcpm-ecnsyn-05: * Added "Updates: 3168" to the header. Added a reference to RFC 4987. Mild editing. Feedback from Lars's Area Director review. * Updated simulation results with new simulation scripts that don't require any modifications to the ns simulator, and that all use the same seed for generating traffic. The results are somewhat different for the very-high-congestion scenarios @@ -215,68 +228,67 @@ the router detects congestion before buffer overflow, the router can provide a congestion indication either by dropping a packet, or by setting the Congestion Experienced (CE) codepoint in the Explicit Congestion Notification (ECN) field in the IP header [RFC3168]. The IETF has standardized the use of the Congestion Experienced (CE) codepoint in the IP header for routers to indicate congestion. For incremental deployment and backwards compatibility, the RFC on ECN [RFC3168] specifies that routers may mark ECN-capable packets that would otherwise have been dropped, using the Congestion Experienced codepoint in the ECN field. The use of ECN allows TCP to react to - congestion while avoiding unnecessary retransmissions and, in some - cases, unnecessary retransmit timeouts. Thus, using ECN has several - benefits: + congestion while avoiding unnecessary retransmit timeouts. Thus, + using ECN has several benefits: 1) For short transfers, a TCP connection's congestion window may be small. For example, if the current window contains only one packet, and that packet is dropped, TCP will have to wait for a retransmit timeout to recover, reducing its overall throughput. Similarly, if the current window contains only a few packets and one of those packets is dropped, there might not be enough duplicate acknowledgements for a fast retransmission, and the sender of the data packet might have to wait for a delay of several round-trip times using Limited Transmit [RFC3042]. With the use of ECN, short flows are less likely to have packets dropped, sometimes avoiding unnecessary delays or costly retransmit timeouts. 2) While longer flows may not see substantially improved throughput - with the use of ECN, they experience lower loss. This may benefit TCP - applications that are latency- and loss-sensitive, because of the + with the use of ECN, they may experience lower loss. This may benefit + TCP applications that are latency- and loss-sensitive, because of the avoidance of retransmissions. RFC 3168 only specifies marking the Congestion Experienced codepoint on TCP's data packets, and not on SYN and SYN/ACK packets. RFC 3168 specifies the negotiation of the use of ECN between the two TCP end- points in the TCP SYN and SYN-ACK exchange, using flags in the TCP header. Erring on the side of being conservative, RFC 3168 does not - specify the use of ECN for the SYN/ACK packet itself. However, + specify the use of ECN for the first SYN/ACK packet itself. However, because of the high cost to the TCP transfer of having a SYN/ACK packet dropped, with the resulting retransmit timeout, this document specifies the use of ECN for the SYN/ACK packet itself. This can be of great benefit to the TCP connection, avoiding the severe penalty of a retransmit timeout for a connection that has not yet started placing a load on the network. The sender of the SYN/ACK packet must - respond to a report of an ECN-marked SYN/ACK packet by reducing its - initial congestion window from two, three, or four segments to one - segment, reducing the subsequent load from that connection on the - network. + respond to a report of an ECN-marked SYN/ACK packet by sending a non- + ECN-Capable SYN/ACK packet, and by reducing its initial congestion + window from two, three, or four segments to one segment, reducing the + subsequent load from that connection on the network. The use of ECN for SYN/ACK packets has the following potential benefits: 1) Avoidance of a retransmit timeout; 2) Improvement in the throughput of short connections. - This draft specifies ECN+, a modification to RFC 3168 to allow TCP - SYN/ACK packets to be ECN-Capable. Section 3 contains the - specification of the change, while Section 4 discusses some of the - issues, and Section 5 discusses related work. Section 6 contains an - evaluation of the specified change. + This draft specifies a modification to RFC 3168 to allow TCP SYN/ACK + packets to be ECN-Capable. Section 3 contains the specification of + the change, while Section 4 discusses some of the issues, and Section + 5 discusses related work. Section 6 contains an evaluation of the + specified change. 2. Conventions and Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119]. We use the following terminology from RFC 3168: The ECN field in the IP header: @@ -295,39 +307,40 @@ refer to the sender of the SYN packet and of the SYN-ACK packet, respectively. 3. Specification This section specifies the modification to RFC 3168 to allow TCP SYN/ACK packets to be ECN-Capable. RFC 3168 in Section 6.1.1. states that "A host MUST NOT set ECT on SYN or SYN-ACK packets." In this section, we specify that a TCP node - MAY respond to an ECN-setup SYN packet by setting ECT in the + MAY respond to an initial ECN-setup SYN packet by setting ECT in the responding ECN-setup SYN/ACK packet, indicating to routers that the SYN/ACK packet is ECN-Capable. This allows a congested router along the path to mark the packet instead of dropping the packet as an indication of congestion. Assume that TCP node A transmits to TCP node B an ECN-setup SYN packet, indicating willingness to use ECN for this connection. As specified by RFC 3168, if TCP node B is willing to use ECN, node B responds with an ECN-setup SYN-ACK packet. 3.1. SYN/ACK Packets Dropped in the Network Figure 1 shows an interchange with the SYN/ACK packet dropped by a congested router. Node B waits for a retransmit timeout, and then retransmits the SYN/ACK packet. --------------------------------------------------------------- TCP Node A Router TCP Node B + (initiator) (responder) ---------- ------ ---------- ECN-setup SYN packet ---> ECN-setup SYN packet ---> <--- ECN-setup SYN/ACK, possibly ECT 3-second timer set SYN/ACK dropped . . . @@ -354,67 +367,133 @@ the exchange in Figure 1, the responder wouldn't set a timer upon transmission of the SYN/ACK packet [SYN-COOK] [RFC4987]. In this case, if the SYN/ACK packet was lost, the initiator (Node A) would have to timeout and retransmit the SYN packet in order to trigger another SYN-ACK. 3.2. SYN/ACK Packets ECN-Marked in the Network Figure 2 shows an interchange with the SYN/ACK packet sent as ECN- Capable, and ECN-marked instead of dropped at the congested router. + This document specifies ECN+/TryOnce, which differs from the original + proposal for ECN+ in [ECN+]; with ECN+/TryOnce, if the TCP responder + is informed that the SYN/ACK was ECN-marked, the TCP responder + immediately sends a SYN/ACK packet that is not ECN-Capable. The TCP + responder is only allowed to send data packets after the TCP + initiator reports the receipt of a SYN/ACK packet that is neither + marked nor dropped. --------------------------------------------------------------- TCP Node A Router TCP Node B + (initiator) (responder) ---------- ------ ---------- ECN-setup SYN packet ---> ECN-setup SYN packet ---> <--- ECN-setup SYN/ACK, ECT + 3-second timer set <--- Sets CE on SYN/ACK <--- ECN-setup SYN/ACK, CE Data/ACK, ECN-Echo ---> Data/ACK, ECN-Echo ---> Window reduced to one segment. - <--- Data, CWR (one segment only) + <--- ECN-setup SYN/ACK, CWR, not ECT + <--- ECN-setup SYN/ACK, CWR + + Data/ACK ---> + Data/ACK ---> + <--- Data (one segment only) --------------------------------------------------------------- Figure 2: SYN exchange with the SYN/ACK packet marked. + ECN+/TryOnce. If the initiator (node A) receives a SYN/ACK packet that has been marked by the congested router, with the CE codepoint set, the initiator MUST respond by setting the ECN-Echo flag in the TCP header - of the responding ACK packet. As specified in RFC 3168, the - initiator continues to set the ECN-Echo flag in packets until it - receives a packet with the CWR flag set. + of the responding ACK packet. However, with ECN+/TryOnce the + initiator does not advance from the "SYN-Sent" to the "SYN-Received" + state until it receives a SYN/ACK packet that is not ECN-marked. As + specified in RFC 3168, the initiator continues to set the ECN-Echo + flag in packets until it receives a packet with the CWR flag set. When the responder (node B) receives the ECN-Echo packet reporting the Congestion Experienced indication in the SYN/ACK packet, the responder MUST set the initial congestion window to one segment, instead of two segments as allowed by [RFC2581], or three or four - segments allowed by [RFC3390]. If the responder (node B) was going - to use an initial window of one segment, and receives an ECN-Echo - packet informing it of a Congestion Experienced indication on its - SYN/ACK packet, the responder MAY continue to send with an initial - window of one segment, without waiting for a retransmit timeout. We - note that this updates RFC 3168, which specifies that "the sending - TCP MUST reset the retransmit timer on receiving the ECN-Echo packet - when the congestion window is one." As specified by RFC 3168, the - responder (node B) also sets the CWR flag in the TCP header of the - next data packet sent, to acknowledge its receipt of and reaction to - the ECN-Echo flag. + segments allowed by [RFC3390]. In the original proposal for ECN+, if + the responder (node B) received an ECN-Echo packet informing it of a + Congestion Experienced indication on its SYN/ACK packet, the + responder would been able to send data packets using an initial + window of one segment, without waiting for a retransmit timeout. In + contrast, this document specifies ECN+/TryOnce, illustrated in Figure + 2; if the responder (node B) receives an ECN-Echo packet informing it + of a Congestion Experienced indication on its SYN/ACK packet, the + responder sends a SYN/ACK packet that is not ECN-Capable, in addition + to setting the initial window to one segment. - If the data transfer in Figure 2 is entirely from Node A to Node B, - then data packets from Node A continue to set the ECN-Echo flag in - data packets, waiting for the CWR flag from Node B acknowledging a - response to the ECN-Echo flag. + We note that this document updates RFC 3168, which specified that + "the sending TCP MUST reset the retransmit timer on receiving the + ECN-Echo packet when the congestion window is one." As an update, + this document specifies the response of a TCP host to receiving an + ECN-Echo packet acknowledging the receipt of an ECN-Capable SYN/ACK + packet. + + RFC 3168 specifies that in response to an ECN-Echo packet, the TCP + responder also sets the CWR flag in the TCP header of the next data + packet sent, to acknowledge its receipt of and reaction to the ECN- + Echo flag. This document updates RFC 3168 by specifying that in + response to an ECN-Echo packet acknowledging the receipt of an ECN- + Capable SYN/ACK packet, the responder sets the CWR flag in the TCP + header of the non-ECN-Capable SYN/ACK packet. + + --------------------------------------------------------------- + TCP Node A Router TCP Node B + (initiator) (responder) + ---------- ------ ---------- + + ECN-setup SYN packet ---> + ECN-setup SYN packet ---> + + <--- ECN-setup SYN/ACK, ECT + <--- Sets CE on SYN/ACK + <--- ECN-setup SYN/ACK, CE + + Data/ACK, ECN-Echo ---> + Data/ACK, ECN-Echo ---> + Window reduced to one segment. + + <--- ECN-setup SYN/ACK, CWR, not ECT + 3-second timer set + SYN/ACK dropped . + . + . + 3-second timer expires + <--- ECN-setup SYN/ACK, CWR, not ECT + <--- ECN-setup SYN/ACK, CWR, not ECT + Data/ACK ---> + Data/ACK ---> + <--- Data (one segment only) + --------------------------------------------------------------- + + Figure 3: SYN exchange with the first SYN/ACK packet marked, + and the second SYN/ACK packet dropped. ECN+/TryOnce. + + In contrast to Figure 2, Figure 3 shows an interchange where the + first SYN/ACK packet is ECN-marked and the second SYN/ACK packet is + dropped in the network. As in Figure 2, the TCP responder sets a + timer when the second SYN/ACK packet is sent. Figure 3 shows that if + the timer expires before the TCP responder receives an + acknowledgement for the other end, the TCP responder resends the + SYN/ACK packet, following the TCP standards. 3.3. Management Interface The TCP implementation using ECN-Capable SYN/ACK packets SHOULD include a management interface to allow the use of ECN to be turned off for SYN/ACK packets. This is to deal with possible backwards compatibility problems such as those discussed in Appendix B. 4. Discussion @@ -429,30 +508,30 @@ SYN/ACK packets, as is discussed further in [ECN+], and reported briefly in Section 5 below. Congestion is most likely to occur in the server-to-client direction. As a result, setting an ECN-capable codepoint in SYN/ACK packets can reduce the occurrence of three- second retransmit timeouts resulting from the drop of SYN/ACK packets. 4.1. Flooding Attacks Setting an ECN-Capable codepoint in the responding TCP SYN/ACK - packets does not raise any novel security vulnerabilities. For - example, provoking servers or hosts to send SYN/ACK packets to a - third party in order to perform a "SYN/ACK flood" attack would be - highly inefficient. Third parties would immediately drop such - packets, since they would know that they didn't generate the TCP SYN - packets in the first place. Moreover, such SYN/ACK attacks would - have the same signatures as the existing TCP SYN attacks. Provoking - servers or hosts to reply with SYN/ACK packets in order to congest a - certain link would also be highly inefficient because SYN/ACK packets - are small in size. + packets does not raise any new or additional security + vulnerabilities. For example, provoking servers or hosts to send + SYN/ACK packets to a third party in order to perform a "SYN/ACK + flood" attack would be highly inefficient. Third parties would + immediately drop such packets, since they would know that they didn't + generate the TCP SYN packets in the first place. Moreover, such + SYN/ACK attacks would have the same signatures as the existing TCP + SYN attacks. Provoking servers or hosts to reply with SYN/ACK packets + in order to congest a certain link would also be highly inefficient + because SYN/ACK packets are small in size. However, the addition of ECN-Capability to SYN/ACK packets could allow SYN/ACK packets to persist for more hops along a network path before being dropped, thus adding somewhat to the ability of a SYN/ACK attack to flood a network link. 4.2. The TCP SYN Packet There are several reasons why an ECN-Capable codepoint MUST NOT be set in the IP header of the initiating TCP SYN packet. First, when @@ -520,71 +599,69 @@ rate below one packet per round-trip time, by waiting for one RTO before sending another packet. If the RTO was set to the average round-trip time, this would result in halving the sending rate; because the RTO is in fact larger than the average round-trip time, the sending rate is reduced to less than half of its previous value. TCP's congestion control response to the *dropping* of a SYN/ACK packet is to wait a default time before sending another packet. This document argues that ECN gives end-systems a wider range of possible responses to the *marking* of a SYN/ACK packet, and that waiting a - default time before sending a data packet is not the desired + default time before sending another packet is not the desired response. On the conservative end, one could assume an effective congestion window of one packet for the SYN/ACK packet, and respond to an ECN- marked SYN/ACK packet by reducing the sending rate to one packet every two round-trip times. As an approximation, the TCP end-node could measure the round-trip time T between the sending of the SYN/ACK packet and the receipt of the acknowledgement, and reply to the acknowledgement of the ECN-marked SYN/ACK packet by waiting T seconds before sending a data packet. However, we note that for an ECN-marked SYN/ACK packet, halving the *congestion window* is not the same as halving the *sending rate*; there is no `sending rate' associated with an ECN-Capable SYN/ACK packet, as such packets are only sent as the first packet in a connection from that host. Further, a router's marking of a SYN/ACK packet is not affected by any past history of that connection. - Adding ECN-Capability to SYN/ACK packets allows the simple response - of the responder setting the initial congestion window to one packet, - instead of its allowed default value of two, three, or four packets, - with the responder proceeding with a cautious sending rate of one - packet per round-trip time. If that data packet is ECN-marked or - dropped, then the responder will wait an RTO before sending another - packet. This document argues that this approach is useful to users, - with no dangers of congestion collapse or of starvation of competing - traffic. This is discussed in more detail below in Section 6.2. In - particular, Section 6.2 discusses simulation results that support the - responder's specified behavior of setting the initial congestion - window to one packet in response to an ECN-marked SYN/ACK packet. + Adding ECN-Capability to SYN/ACK packets allows the response of the + responder setting the initial congestion window to one packet, + instead of its allowed default value of two, three, or four packets. + The responder sends a non-ECN-Capable SYN/ACK packet, and proceeds + with a cautious sending rate of one data packet per round-trip time + after that SYN/ACK packet is acknowledged. This document argues that + this approach is useful to users, with no dangers of congestion + collapse or of starvation of competing traffic. This is discussed in + more detail below in Section 6.2. We note that if the data transfer is entirely from Node A to Node B, - then there is no effective difference between the two possible - responses to an ECN-marked SYN/ACK packet outlined above. In either - case, Node B sends no data packets, only sending acknowledgement - packets in response to received data packets. + there is still a difference in performance between the original + mechanism ECN+ and the mechanism ECN+/TryOnce specified in this + document. In particular, with ECN+/TryOnce the TCP originator does + not send data packets until it has received a non-ECN-marked SYN/ACK + packet from the other end. 5. Related Work - The addition of ECN-capability to TCP's SYN/ACK packets was proposed - in [ECN+]. The paper includes an extensive set of simulation and - testbed experiments to evaluate the effects of the proposal, using - several Active Queue Management (AQM) mechanisms, including Random - Early Detection (RED) [RED], Random Exponential Marking (REM) [REM], - and Proportional Integrator (PI) [PI]. The performance measures were - the end-to-end response times for each request/response pair, and the - aggregate throughput on the bottleneck link. The end-to-end response - time was computed as the time from the moment when the request for - the file is sent to the server, until that file is successfully - downloaded by the client. + The addition of ECN-capability to TCP's SYN/ACK packets was initially + proposed in [ECN+]. The paper includes an extensive set of + simulation and testbed experiments to evaluate the effects of the + proposal, using several Active Queue Management (AQM) mechanisms, + including Random Early Detection (RED) [RED], Random Exponential + Marking (REM) [REM], and Proportional Integrator (PI) [PI]. The + performance measures were the end-to-end response times for each + request/response pair, and the aggregate throughput on the bottleneck + link. The end-to-end response time was computed as the time from the + moment when the request for the file is sent to the server, until + that file is successfully downloaded by the client. The measurements from [ECN+] show that setting an ECN-Capable codepoint in the IP packet header in TCP SYN/ACK packets systematically improves performance with all evaluated AQM schemes. When SYN/ACK packets at a congested router are ECN-marked instead of dropped, this can avoid a long initial retransmit timeout, improving the response time for the affected flow dramatically. [ECN+] shows that the impact on aggregate throughput can also be quite significant, because marking SYN ACK packets can prevent larger @@ -672,35 +749,56 @@ Thus, the degree of benefit of adding ECN-Capability to SYN/ACK packets depends not only on the overall packet drop rate in the network, but also on the queue management architecture at the congested link. 6.2. An Evaluation of Different Responses to ECN-Marked SYN/ACK Packets This document specifies that the end-node responds to the report of an ECN-marked SYN/ACK packet by setting the initial congestion window to one segment, instead of its possible default value of two to four - segments. We call this ECN+ with NoWaiting. However, Section 4 - discussed another possible response to an ECN-marked SYN/ACK packet, - of the end-node waiting an RTT before sending a data packet. We call - this approach ECN+ with Waiting. + segments, and resending a SYN/ACK packet that is not ECN-Capable. We + call this ECN+/TryOnce. + + However, Section 4 discussed two other possible responses to an ECN- + marked SYN/ACK packet. In ECN+, the original proposal from [ECN+], + the end node responds to the report of an ECN-marked SYN/ACK packet + by setting the initial congestion window to one segment and + immediately sending a data packet, if it has one to send. In + ECN+/Wait, the end node responds to the report of an ECN-marked + SYN/ACK packet by setting the initial congestion window to one + segment and waiting an RTT before sending a data packet. Simulations comparing the performance with Standard ECN (without ECN- - marked SYN/ACK packets), ECN+ with NoWaiting, and ECN+ with Waiting - show little difference, in terms of aggregate congestion, between - ECN+ with NoWaiting and ECN+ with Waiting. The details are given in - Appendix A below. Our conclusions are that ECN+ with NoWaiting is - perfectly safe, and there are no congestion-related reasons for - preferring ECN+ with Waiting over ECN+ with NoWaiting. That is, - there is no need for the TCP end-node to wait a round-trip time - before sending a data packet after receiving an acknowledgement of an - ECN-marked SYN/ACK packet. + marked SYN/ACK packets), ECN+, and ECN+/Wait, and ECN/TryOnce show + little difference, in terms of aggregate congestion, between ECN+ and + ECN+/Wait. However, for some scenarios with queues that are packet- + based rather than byte-based, and with packet drop rates above 25% + without ECN+, the use of ECN+ or of ECN+/Wait can more than double + the packet drop rates, to greater than 50%. The details are given in + Tables 1 and 3 of Appendix A below. ECN+/TryOnce does not increase + the packet drop rate in scenarios of high congestion. Therefore, + ECN+/TryOnce is superior to ECN+ or to ECN+/Wait, which both + significantly increase the packet drop rate in scenarios of high + congestion. At the same time, ECN+/TryOnce gives a performance + improvement similar to that of ECN+ or ECN+/Wait (Tables 2 and 4 of + Appendix A). + + Our conclusions are that ECN+/TryOnce is safe, and has significant + benefits to the user, and avoids the problems of ECN+ or ECN+/Wait + under extreme levels of congestion. As a consequence, this document + specifies the use of ECN+/TryOnce. + + [Note: We only discovered the occasional congestion-related problems + of ECN+ and of ECN+/Wait when re-running the simulations with an + updated version of the ns-2 simulator, after the internet-draft had + almost completed the standardization process.] 7. Security Considerations TCP packets carrying the ECT codepoint in IP headers can be marked rather than dropped by ECN-capable routers. This raises several security concerns that we discuss below. 7.1. 'Bad' Routers or Middleboxes There are a number of known deployment problems from using ECN with @@ -727,341 +825,436 @@ already known to crash when a data packet arrives with either ECT(0) or ECT(1)), but we have not conducted any measurement studies of this [F07]. 7.2. Congestion Collapse Because TCP SYN/ACK packets carrying an ECT codepoint could be ECN- marked instead of dropped at an ECN-capable router, the concern is whether this can either invoke congestion, or worsen performance in highly congested scenarios. However, after learning that a SYN/ACK - packet was ECN-marked, the responder will only send one data packet; - if this data packet is ECN-marked, the responder will then wait for a - retransmission timeout. In addition, routers are free to drop rather - than mark arriving packets in times of high congestion, regardless of - whether the packets are ECN-capable. When congestion is very high - and a router's buffer is full, the router has no choice but to drop - rather than to mark an arriving packet. + packet was ECN-marked, the responder sends a SYN/ACK packet that is + not ECN-Capable; if this SYN/ACK packet is dropped, the responder + then waits for a retransmission timeout, as specified in the TCP + standards. In addition, routers are free to drop rather than mark + arriving packets in times of high congestion, regardless of whether + the packets are ECN-capable. When congestion is very high and a + router's buffer is full, the router has no choice but to drop rather + than to mark an arriving packet. The simulations reported in Appendix A show that even with demanding traffic mixes dominated by short flows and high levels of congestion, the aggregate packet dropping rates are not significantly different - with Standard ECN, ECN+ with NoWaiting, or ECN+ with Waiting. In - particular, the simulations show that in periods of very high - congestion the packet-marking rate is low with or without ECN+, and - the use of ECN+ does not significantly increase the number of dropped - or marked packets. - - The simulations show that ECN+ is most effective in times of moderate - congestion. In these moderate-congested scenarios, the use of ECN+ - increases the number of ECN-marked packets, because ECN+ allows - SYN/ACK packets to be ECN-marked. At the same time, in these times - of moderate congestion, the use of ECN+ instead of Standard ECN does - not significantly affect the overall levels of congestion. - - The simulations show that the use of ECN+ is less effective in times - of high congestion; the simulations show that in times of high - congestion more packets are dropped instead of marked, both with - Standard ECN and with ECN+. In times of high congestion, the buffer - can overflow, even with Active Queue Management and ECN; when the - buffer is full arriving packets are dropped rather than marked, - whether the packets are ECN-capable or not. Thus while ECN+ is less - effective in times of high congestion, it still doesn't result in a - significant increase in the level of congestion. More details are - given in the appendix. + with Standard ECN or with ECN+/TryOnce. However, in our simulations, + we have one scenario where ECN+ or ECN+/Wait results in a + significantly higher packet drop rate than ECN or ECN+/TryOnce + (Tables 1 and 3 in Appendix A below). 8. Conclusions This draft specifies a modification to RFC 3168 to allow TCP nodes to send SYN/ACK packets as being ECN-Capable. Making the SYN/ACK packet ECN-Capable avoids the high cost to a TCP transfer when a SYN/ACK packet is dropped by a congested router, by avoiding the resulting retransmit timeout. This improves the throughput of short - connections. The sender of the SYN/ACK packet responds to an ECN - mark by reducing its initial congestion window from two, three, or - four segments to one segment, reducing the subsequent load from that - connection on the network. The addition of ECN-capability to SYN/ACK - packets is particularly beneficial in the server-to-client direction, - where congestion is more likely to occur. In this case, the initial - information provided by the ECN marking in the SYN/ACK packet enables - the server to more appropriately adjust the initial load it places on - the network. + connections. This document specifies the ECN+/TryOnce mechanism for + ECN-Capability for SYN/ACK packets, where the sender of the SYN/ACK + packet responds to an ECN mark by reducing its initial congestion + window from two, three, or four segments to one segment, and sending + a SYN/ACK packet that is not ECN-Capable. The addition of ECN- + capability to SYN/ACK packets is particularly beneficial in the + server-to-client direction, where congestion is more likely to occur. + In this case, the initial information provided by the ECN marking in + the SYN/ACK packet enables the server to appropriately adjust the + initial load it places on the network, while avoiding the delay of a + retransmit timeout. 9. Acknowledgements We thank Anil Agarwal, Mark Allman, Remi Denis-Courmont, Wesley Eddy, Lars Eggert, Alfred Hoenes, Janardhan Iyengar, and Pasi Sarolahti for - feedback on earlier versions of this draft. + feedback on earlier versions of this draft. We thank Adam Langley + [L08] for contributing a patch for ECN+/TryOnce for the Linux + development tree. A. Report on Simulations This section reports on simulations showing the costs of adding ECN+ in highly-congested scenarios. This section also reports on - simulations for a comparative evaluation between ECN+ with NoWaiting - and ECN+ with Waiting. + simulations for a comparative evaluation between ECN, ECN+, + ECN+/Wait, and ECN+/TryOnce. The simulations are run with a range of file-size distributions, using the PackMime traffic generator in the ns-2 simulator. They all use a heavy-tailed distribution of file sizes. The simulations reported in the tables below use a mean file size of 3 KBypes, to show the results with a traffic mix with a large number of small - transfers. Othe simulations were run with mean file sizes of 5 + transfers. Other simulations were run with mean file sizes of 5 KBytes, 7 Kbytes, 14 KBytes, and 17 Kbytes. The title of each chart gives the targeted average load from the traffic generator. Because the simulations use a heavy-tailed distribution of file sizes, and run for only 85 seconds (including ten seconds of warm-up time), the actual load is often much smaller than the targeted load. The congested link is 100 Mbps. RED is run in gentle mode, and arriving ECN-Capable packets are only dropped instead of marked if the buffer is full (and the router has no choice). - We explore two alternatives for a TCP node's response to a report of - an ECN-marked SYN/ACK packet. With ECN+ with NoWaiting, the TCP node + We explore three possible mechanisms for a TCP node's response to a + report of an ECN-marked SYN/ACK packet. With ECN+, the TCP node sends a data packet immediately (with an initial congestion window of - one segment). With the alternative ECN+ with Waiting, the TCP node - waits a round-trip time before sending a data packet; the responder - already has one measurement of the round-trip time when the - acknowledgement for the SYN/ACK packet is received. - - In the tables below, ECN+ refers to ECN+ with NoWaiting, where the - responder starts transmitting immediately, and ECN+/wait refers to - ECN+ with Waiting, where the responder waits a round-trip time before - sending a data packet into the network. + one segment). With ECN+/Wait, the TCP node waits a round-trip time + before sending a data packet; the responder already has one + measurement of the round-trip time when the acknowledgement for the + SYN/ACK packet is received. With ECN+/TryOnce, the mechanism + standardized in this document, the TCP responder replies to a report + of an ECN-marked SYN/ACK packet by sending a SYN/ACK packet that is + not ECN-Capable, and reducing the initial congestion window to one + segment. - The simulation scripts are available on [ECN-SYN]. + The simulation scripts are available on [ECN-SYN]. along with graphs + showing the distribution of response times for the TCP connections. A.1. Simulations with RED in Packet Mode The simulations with RED in packet mode and with the queue in packets - show that ECN+ is useful in times of moderate congestion, though it - adds little benefit in times of high congestion. The simulations - show a minimal increase in levels of congestion with either ECN+ with - Waiting or ECN+ with NoWaiting, either in terms of packet dropping or - marking rates or in terms of the distribution of responses times. - Thus, the simulations show no problems with ECN+ in times of high - congestion, and no reason to use ECN+ with Waiting instead of ECN+ - with NoWaiting. + show that ECN+ is useful in times of moderate or of high congestion. + However, for the simulations with a target load of 125%, with a + packet loss rate of over 25% for ECN, ECN+ and ECN+/Wait both result + in a packet loss rate of over 50%. (In contrast, the packet loss + rate with ECN+/TryOnce is less than that of ECN alone.) For the + distribution of response times, the simulations show that ECN+, + ECN+/Wait, and ECN+/TryOnce all significantly improve the response + times, compared to the response times with plain ECN. Table 1 shows the congestion levels for simulations with RED in packet mode, with a queue in packets. To explore a worst-case scenario, these simulations use a traffic mix with an unrealistically small flow size distribution, with a mean flow size of 3 Kbytes. For - each table showing a particular traffic load, the three rows show the - number of packets dropped, the number of packets ECN-marked, and the - aggregate packet drop rate, and the three columns show the - simulations with Standard ECN, ECN+ (NoWaiting) and ECN+/wait. + each table showing a particular traffic load, the four rows show the + number of packets dropped, the number of packets ECN-marked, the + aggregate packet drop rate, and the aggregate throughput, and the + four columns show the simulations with Standard ECN, ECN+, ECN+/Wait, + and ECN+/TryOnce. These simulations were run with RED set to mark instead of drop - packets any time that the queue is not full. For the default - implementation of RED in the ns-2 simulator, the router drops instead - of marks arriving packets when the average queue size exceeds a + packets any time that the queue is not full. This is a worst-case + scenario for ECN+ and its variants. For the default implementation + of RED in the ns-2 simulator, when the average queue size exceeds a + configured threshold. the router drops all arriving packets. For + scenarios with this RED mechanisms, it is less likely that ECN+ or + one of its variants would increase the average queue size above the configured threshold. The usefulness of ECN+: The first thing to observe is that for all of - the simulations, the use of ECN+ or ECN+/wait significantly increased - the number of packets marked. This indicates that with ECN+ or - ECN+/wait, many SYN/ACK packets are marked instead of dropped. + the simulations, the use of ECN+ or ECN+/Wait significantly increases + the number of packets marked. In contrast, the use of ECN+/TryOnce + significantly increases the number of packets marked in the + simulations with moderate congestion, and gives a more moderate + increase in the number of packets marked for the simulations with + higher levels of congestion. However, the cumulative distribution + function (CDF) in Table 2 shows that ECN+, ECN+/Wait, and + ECN+/TryOnce all improve response times for all of the simulations, + with moderate or with larger levels of congestion. Little increase in congestion, sometimes: The second thing to observe is that for the simulations with low or moderate levels of congestion - (that is, with packet drop rates less than 10%), the use of ECN+ or - ECN+/wait decreases the aggregate packet drop rate, relative to the - simulations with ECN. This makes sense, since with low or moderate - levels of congestion, ECN+ allows SYN/ACK packets to be marked - instead of dropped, and the use of ECN+ doesn't add to the aggregate - congestion. However, for the simulations with packet drop rates of - 15% or higher with ECN, the use of ECN+ or ECN+/wait increases the - aggregate packet drop rate, sometimes even doubling it. + (that is, with packet drop rates less than 10%), the use of ECN+, + ECN+/Wait, and ECN+/TryOnce all decrease the aggregate packet drop + rate, relative to the simulations with ECN. This makes sense, since + with low or moderate levels of congestion, ECN+ allows SYN/ACK + packets to be marked instead of dropped, and the use of ECN+ doesn't + add to the aggregate congestion. However, for the simulations with + packet drop rates of 15% or higher with ECN, the use of ECN+ or + ECN+/Wait increases the aggregate packet drop rate, sometimes even + doubling it. - Comparing ECN+ and ECN+/wait: The third thing to observe is that the - aggregate packet drop rate is generally higher with ECN+/wait than - with ECN+. Thus, there is no congestion-related reason to prefer - ECN+/wait over ECN+. + Comparing ECN+, ECN+/Wait, and ECN+/TryOnce: The aggregate packet + drop rate is generally higher with ECN+/Wait than with ECN+. Thus, + there is no congestion-related reason to prefer ECN+/Wait over ECN+. + In contrast, the aggregate packet drop rate with ECN+/TryOnce is + often significantly lower than the aggregate packet drop rate with + either ECN, ECN+, ECN+/Wait. Target Load = 95%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 18,512 11,244 12,135 - Marked 27,026 36,977 38,743 - Loss rate 1.27% 0.78% 0.84% - Throughput 81% 81% 81% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 20,516 11,226 11,735 16,446` + Marked 30,586 37,741 37,425 40,530 + Loss rate 1.41% 0.78% 0.81% 1.01% + Throughput 81% 81% 81% 81% Target Load = 110%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 165,866 110,525 144,821 - Marked 180,714 290,629 311,233 - Loss rate 9.04% 6.36% 7.94% - Throughput 92% 92% 92% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 165,566 106,083 147,180 218,594 + Marked 179,735 281,306 308,473 242,969 + Loss rate 9.01% 6.12% 8.02% 7.14% + Throughput 92% 92% 92% 94% Target Load = 125%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 574,114 1,764,677 2,229,280 - Marked 409,441 1,172,550 1,181,209 - Loss rate 24.55% 52.00% 57.64% - Throughput 94% 98% 97% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 600,628 1,746,768 2,176,530 650,781 + Marked 418,433 1,166,450 1,164,932 440,432 + Loss rate 25.45% 51.73% 56.87% 18.22% + Throughput 94% 98% 97% 95% + + Target Load = 1.50% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 1,449,945 1,565,0517 1,563,0801 1,372,067 + Marked 669,840 583,378 591,315 675,290 + Loss rate 46.7% 59.0% 59.0% 32.3% + Throughput 88% 94% 94% 93% Table 1: Simulations with an average flow size of 3 Kbytes, a 100 Mbps link, RED in packet mode, queue in packets. Target Load = 95%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 8,754 6,719 7,269 - Marked 10,376 17,637 16,956 - Loss rate 5.68% 4.50% 4.75% - Throughput 78% 78% 78% + + TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000 + ------------------------------------------------------ + ECN: 0.00 0.07 0.26 0.51 0.82 0.96 0.97 0.97 0.97 1.00 1.00 + ECN+: 0.00 0.07 0.27 0.53 0.85 0.99 1.00 1.00 1.00 1.00 1.00 + Wait: 0.00 0.07 0.26 0.51 0.83 0.97 1.00 1.00 1.00 1.00 1.00 + Once: 0.00 0.07 0.24 0.49 0.83 0.97 1.00 1.00 1.00 1.00 1.00 Target Load = 110%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 32,110 32,014 48,838 - Marked 28,476 56,550 62,252 - Loss rate 15.68% 16.11% 21.92% - Throughput 96% 96% 96% + + TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000 + ------------------------------------------------------ + ECN: 0.00 0.05 0.19 0.41 0.67 0.79 0.80 0.80 0.80 0.96 0.96 + ECN+: 0.00 0.07 0.22 0.48 0.81 0.96 1.00 1.00 1.00 1.00 1.00 + Wait: 0.00 0.05 0.18 0.38 0.64 0.77 0.95 1.00 1.00 1.00 1.00 + Once: 0.00 0.06 0.19 0.41 0.70 0.86 0.95 0.96 0.96 0.99 0.99 Target Load = 125%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 60,710 174,920 215,001 - Marked 43,497 119,620 118,172 - Loss rate 25.08% 51.59% 56.27% - Throughput 98% 98% 98% - Target Load = 150%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 133,128 250,762 327,584 - Marked 63,306 146,581 147,307 - Loss rate 43.34% 61.11% 67.33% - Throughput 93% 100% 100% + TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000 + ------------------------------------------------------ + ECN: 0.00 0.04 0.13 0.27 0.46 0.56 0.58 0.59 0.59 0.82 0.82 + ECN+: 0.00 0.06 0.18 0.33 0.58 0.76 0.97 0.99 0.99 1.00 1.00 + Wait: 0.00 0.01 0.06 0.13 0.21 0.27 0.68 0.98 0.99 1.00 1.00 + Once: 0.00 0.05 0.16 0.34 0.58 0.73 0.85 0.87 0.87 0.95 0.96 - Table 2: Simulations with an average flow size of 3 Kbytes, a 10 Mbps + TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000 + ------------------------------------------------------ + ECN: 0.00 0.03 0.08 0.18 0.31 0.39 0.42 0.42 0.43 0.68 0.68 + ECN+: 0.00 0.06 0.18 0.39 0.67 0.81 0.83 0.84 0.84 0.93 0.93 + Wait: 0.00 0.06 0.18 0.39 0.67 0.81 0.83 0.84 0.84 0.93 0.94 + Once: 0.00 0.04 0.13 0.28 0.47 0.60 0.72 0.75 0.76 0.88 0.89 + + Table 2: The cumulative distribution function (CDF) for transfer + times, for simulations with an average flow size of 3 Kbytes, a + 100 Mbps link, RED in packet mode, queue in packets. (The graphs are + available from "http://www.icir.org/floyd/ecn-syn/".) + Target Load = 0.95% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 8,448 6,362 7,740 16,323 + Marked 9,891 16,787 17,456 17,186 + Loss rate 5.5% 4.3% 5.0% 5.4% + Throughput 78% 78% 78% 82% + + Target Load = 1.10% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 31,284 29,773 49,297 42,201 + Marked 28,429 54,729 60,383 33,672 + Loss rate 15.3% 15.2% 21.9% 13.5% + Throughput 97% 96% 96% 95% + + Target Load = 1.25% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 61,433 176,682 214,096 79,463 + Marked 44,408 119,728 117,301 48,991 + Loss rate 25.4% 51.9% 56.0% 22.5% + Throughput 97% 98% 98% 95% + + Target Load = 1.50% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 130,007 251,856 326,845 141,418 + Marked 63,066 146,757 147,239 67,772 + Loss rate 42.5% 61.3% 67.3% 33.3% + Throughput 93% 99% 99% 94% + + Table 3: Simulations with an average flow size of 3 Kbytes, a 10 Mbps link, RED in packet mode, queue in packets. + Target Load = 95%: + + TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000 + ------------------------------------------------------ + ECN: 0.00 0.05 0.18 0.42 0.70 0.86 0.88 0.88 0.88 0.98 0.98 + ECN+: 0.00 0.06 0.20 0.45 0.78 0.96 1.00 1.00 1.00 1.00 1.00 + Wait: 0.00 0.05 0.18 0.40 0.68 0.84 0.96 1.00 1.00 1.00 1.00 + Once: 0.00 0.05 0.18 0.39 0.69 0.87 0.96 0.96 0.96 0.99 0.99 + + Target Load = 110%: + + TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000 + ------------------------------------------------------ + ECN: 0.00 0.03 0.13 0.29 0.52 0.66 0.69 0.69 0.69 0.91 0.91 + ECN+: 0.00 0.05 0.17 0.36 0.66 0.88 0.98 0.99 1.00 1.00 1.00 + Wait: 0.00 0.02 0.08 0.20 0.35 0.47 0.76 0.98 1.00 1.00 1.00 + Once: 0.00 0.04 0.15 0.33 0.59 0.76 0.89 0.91 0.91 0.98 0.98 + + Target Load = 125%: + + TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000 + ------------------------------------------------------ + ECN: 0.00 0.03 0.10 0.22 0.40 0.52 0.56 0.56 0.57 0.82 0.82 + ECN+: 0.00 0.03 0.14 0.27 0.49 0.70 0.96 0.99 0.99 0.99 1.00 + Wait: 0.00 0.00 0.03 0.07 0.12 0.18 0.50 0.94 0.99 0.99 1.00 + Once: 0.00 0.04 0.13 0.29 0.51 0.66 0.81 0.84 0.84 0.94 0.94 + + Target Load = 150%: + + TIME: 10 100 200 300 400 500 1000 2000 3000 4000 5000 + ------------------------------------------------------ + ECN: 0.00 0.02 0.07 0.15 0.28 0.38 0.42 0.42 0.43 0.67 0.68 + ECN+: 0.00 0.00 0.00 0.00 0.01 0.05 0.68 0.83 0.95 0.97 0.98 + Wait: 0.00 0.00 0.00 0.00 0.00 0.00 0.10 0.62 0.83 0.93 0.97 + Once: 0.00 0.03 0.11 0.23 0.42 0.56 0.71 0.74 0.74 0.87 0.88 + + Table 4: The cumulative distribution function (CDF) for transfer + times, for simulations with an average flow size of 3 Kbytes, a + 10 Mbps link, RED in packet mode, queue in packets. (The graphs are + available from "http://www.icir.org/floyd/ecn-syn/".) + A.2. Simulations with RED in Byte Mode - Table 3 below shows simulations with RED in byte mode and the queue + Table 5 below shows simulations with RED in byte mode and the queue in bytes. There is no significant increase in aggregate congestion - with the use of ECN+ or ECN+/wait, and no congestion-related reason - to prefer ECN+/wait over ECN+. + with the use of ECN+, ECN+/Wait, or ECN+/TryOnce. However, unlike the simulations with RED in packet mode, the simulations with RED in byte mode show little benefit from the use of - ECN+ or ECN+/wait, in that the packet marking rate with ECN+ or - ECN+/wait is not much different than the packet marking rate with + ECN+ or ECN+/Wait, in that the packet marking rate with ECN+ or + ECN+/Wait is not much different than the packet marking rate with Standard ECN. This is because with RED in byte mode, small packets like SYN/ACK packets are rarely dropped or marked - that is, there is no drawback from the use of ECN+ in these scenarios, but not much need for ECN+ either, in a scenario where small packets are unlikely to be dropped or marked. - Target Load = 95%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 739 438 442 - Marked 32,405 34,357 34,000 - Loss rate 0.05% 0.03% 0.03% - Throughput 81% 81% 81% + Target Load = 95% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 766 446 427 408 + Marked 32,683 34,289 33,412 31,892 + Loss rate 0.05% 0.03% 0.03% 0.03% + Throughput 81% 81% 81% 81% - Target Load = 110%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 2,473 1,679 3,020 - Marked 226,971 222,234 327,608 - Loss rate 0.15% 0.10% 0.18% - Throughput 92% 92% 91% + Target Load = 110% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 2,496 2,110 1,733 2,024 + Marked 220,573 258,696 230,955 224,338 + Loss rate 0.15% 0.13% 0.11% 0.11% + Throughput 92% 91% 92% 92% - Target Load = 125%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 19,358 14,057 14,064 - Marked 717,123 728,513 729,001 - Loss rate 1.07% 0.78% 0.78% - Throughput 95% 95% 95% + Target Load = 125% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 20,032 13,555 13,979 19,544 + Marked 725,165 726,992 726,823 627,088 + Loss rate 1.11% 0.76% 0.78% 0.72% + Throughput 95% 95% 95% 95% - Table 3: Simulations with an average flow size of 3 Kbytes, a + Target Load = 150% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 484,251 483,847 507,727 572,373 + Marked 865,905 872,254 873,317 816,841 + Loss rate 19.09% 19.13% 19.71% 12.28% + Throughput 99% 98% 99% 99% + + Table 5: Simulations with an average flow size of 3 Kbytes, a 100 Mbps link, RED in byte mode, queue in bytes. - Target Load = 95%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 142 81 78 - Marked 11,694 11,812 11,964 - Loss rate 0.01% 0.06% 0.05% - Throughput 78% 78% 78% + Target Load = 0.95% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 142 77 103 99 + Marked 11,694 11,387 11,604 12,129 + Loss rate 0.1% 0.1% 0.1% 0.1% + Throughput 78% 78% 78% 78% - Target Load = 110%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 314 215 188 - Marked 39,697 42,388 40,229 - Loss rate 0.19% 0.13% 0.11% - Throughput 95% 94% 95% + Target Load = 1.10% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 338 210 247 292 + Marked 41,676 40,412 44,173 37,527 + Loss rate 0.2% 0.1% 0.1% 0.1% + Throughput 94% 94% 94% 95% - Target Load = 125%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 1,599 1,011 985 - Marked 74,567 75,782 75,528 - Loss rate 0.87% 0.56% 0.54% - Throughput 98% 98% 98% + Target Load = 1.25% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 1,559 951 978 1,490 + Marked 74,933 75,499 75,481 57,721 + Loss rate 0.8% 0.5% 0.5% 0.5% + Throughput 99% 99% 99% 96% - Target Load = 150%: - ECN ECN+ ECN+/wait - ------- ------- ------- - Dropped 2,429 1,538 1,571 - Marked 85,312 86,481 86,476 - Loss rate 1.22% 0.78% 0.79% - Throughput 98% 98% 98% + Target Load = 1.50% + ECN ECN+ ECN+/Wait ECN+/TryOnce + ------- ------- ------- ---------- + Dropped 2,374 1,528 1,515 4,517 + Marked 85,739 86,428 86,144 81,695 + Loss rate 1.2% 0.8% 0.8% 1.3% + Throughput 99% 98% 98% 98% - Table 4: Simulations with an average flow size of 3 Kbytes, a 10 Mbps + Table 6: Simulations with an average flow size of 3 Kbytes, a 10 Mbps link, RED in byte mode, queue in bytes. B. Issues of Incremental Deployment In order for TCP node B to send a SYN/ACK packet as ECN-Capable, node B must have received an ECN-setup SYN packet from node A. However, it is possible that node A supports ECN, but either ignores the CE codepoint on received SYN/ACK packets, or ignores SYN/ACK packets with the ECT or CE codepoint set. If the TCP initiator ignores the CE codepoint on received SYN/ACK packets, this would mean that the TCP responder would not respond to this congestion indication. However, this seems to us an acceptable cost to pay in the incremental deployment of ECN-Capability for TCP's SYN/ACK packets. It would mean that the responder would not reduce the initial congestion window from two, three, or four segments down to one - segment, as it should. However, the TCP end nodes would still - respond correctly to any subsequent CE indications on data packets - later on in the connection. + segment, as it should. and would not sent a non-ECN-Capable SYN/ACK + packet to complete the SYN exchange. However, the TCP end nodes + would still respond correctly to any subsequent CE indications on + data packets later on in the connection. - Figure 3 shows an interchange with the SYN/ACK packet ECN-marked, but + Figure 4 shows an interchange with the SYN/ACK packet ECN-marked, but with the ECN mark ignored by the TCP originator. --------------------------------------------------------------- TCP Node A Router TCP Node B + (initiator) (responder) ---------- ------ ---------- ECN-setup SYN packet ---> ECN-setup SYN packet ---> <--- ECN-setup SYN/ACK, ECT <--- Sets CE on SYN/ACK <--- ECN-setup SYN/ACK, CE Data/ACK, No ECN-Echo ---> Data/ACK ---> <--- Data (up to four packets) --------------------------------------------------------------- - Figure 3: SYN exchange with the SYN/ACK packet marked, + Figure 4: SYN exchange with the SYN/ACK packet marked, but with the ECN mark ignored by the TCP initiator. Thus, to be explicit, when a TCP connection includes an initiator that supports ECN but *does not* support ECN-Capability for SYN/ACK packets, in combination with a responder that *does* support ECN- Capability for SYN/ACK packets, it is possible that the ECN-Capable SYN/ACK packets will be marked rather than dropped in the network, and that the responder will not learn about the ECN mark on the SYN/ACK packet. This would not be a problem if most packets from the responder supporting ECN for SYN/ACK packets were in long-lived TCP @@ -1146,20 +1339,23 @@ "http://www.icir.org/floyd/ecn-syn". [F07] S. Floyd, "[BEHAVE] Response of firewalls and middleboxes to TCP SYN packets that are ECN-Capable?", August 2, 2007, email sent to the BEHAVE mailing list, URL "http://www1.ietf.org/mail- archive/web/behave/current/msg02644.html". [Kelson00] Dax Kelson, note sent to the Linux kernel mailing list, September 10, 2000. + [L08] A. Landley, "Re: [tcpm] I-D Action:draft-ietf-tcpm- + ecnsyn-06.txt", Email to the tcpm mailing list, August 24, 2008. + [MAF05] A. Medina, M. Allman, and S. Floyd. Measuring the Evolution of Transport Protocols in the Internet, ACM CCR, April 2005. [PI] C. Hollot, V. Misra, W. Gong, and D. Towsley, On Designing Improved Controllers for AQM Routers Supporting TCP Flows, April 1998. [RED] Floyd, S., and Jacobson, V. Random Early Detection gateways for Congestion Avoidance . IEEE/ACM Transactions on Networking, V.1 N.4, August 1993.