Network Working Group T. Henderson Internet-Draft Boeing Obsoletes: 3782 (if approved) S. Floyd Intended status: Standards Track ICSI Expires:June 24,September 15, 2011 A. Gurtov HIIT Y. Nishida WIDE ProjectJanuary 24,March 14, 2011 The NewReno Modification to TCP's Fast Recovery Algorithmdraft-ietf-tcpm-rfc3782-bis-00.txtdraft-ietf-tcpm-rfc3782-bis-01.txt Abstract RFC 5681 [RFC5681] documents the following four intertwined TCP congestion control algorithms: Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery. RFC 5681 explicitly allows certain modifications of these algorithms, including modifications that use the TCP Selective Acknowledgement (SACK) option [RFC2883], and modifications that respond to "partial acknowledgments" (ACKs which cover new data, but not all the data outstanding when loss was detected) in the absence of SACK. This document describes a specific algorithm for responding to partial acknowledgments, referred to as NewReno. This response to partial acknowledgments was first proposed by Janey Hoe in [Hoe95]. The purpose of this revision from [RFC3782] is to make errata changes and to adopt a proposal from Yoshifumi Nishida to slightly increase the minimum window size after Fast Recovery from one to two segments, to improve performance when the receiver uses delayed acknowledgments. Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire onJune 24,September 15, 2011. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. 1. Introduction For the typical implementation of the TCP Fast Recovery algorithm described in [RFC5681] (first implemented in the 1990 BSD Reno release, and referred to as the Reno algorithm in [FF96]), the TCP data sender only retransmits a packet after a retransmit timeout has occurred, or after three duplicateacknowledgementsacknowledgments have arrived triggering the Fast Retransmit algorithm. A single retransmit timeout might result in the retransmission of several data packets, but each invocation of the Fast Retransmit algorithm in RFC 5681 leads to the retransmission of only a single data packet.Problems can arise, therefore,Two problems arise with Reno TCP when multiplepackets are dropped frompacket losses occur in a singlewindow of data and the Fast Retransmitwindow. First, Reno will often take a timeout, as has been documented in [Hoe95]. Second, even if a retransmission timeout is avoided, multiple fast retransmits andFast Recovery algorithms are invoked. In this case,window reductions can occur, as documented in [F94]. When multiple packet losses occur, if the SACK option [RFC2883] is available, the TCP sender has the information to make intelligent decisions about which packets to retransmit and which packets not to retransmit during Fast Recovery. This document appliesonly forto TCP connections that are unable to use the TCP Selective Acknowledgement (SACK) option, either because the option is not locally supported or because the TCP peer did not indicate a willingness to use SACK. In the absence of SACK, there is little information available to the TCP sender in making retransmission decisions during Fast Recovery. From the three duplicateacknowledgements,acknowledgments, the sender infers a packet loss, and retransmits the indicated packet. After this, the data sender could receive additional duplicateacknowledgements,acknowledgments, as the data receiver acknowledges additional data packets that were already in flight when the sender entered Fast Retransmit. In the case of multiple packets dropped from a single window of data, the first new information available to the sender comes when the sender receives anacknowledgementacknowledgment for the retransmitted packet (that is, the packet retransmitted when Fast Retransmit was first entered). If there is a single packet drop and no reordering, then theacknowledgementacknowledgment for this packet will acknowledge all of the packets transmitted before Fast Retransmit was entered. However, if there are multiple packet drops, then theacknowledgementacknowledgment for the retransmitted packet will acknowledge some but not all of the packets transmitted before the Fast Retransmit. We call thisacknowledgementacknowledgment a partial acknowledgment. Along with several other suggestions, [Hoe95] suggested that during Fast Recovery the TCP data sender responds to a partial acknowledgment by inferring that the next in-sequence packet has been lost, and retransmitting that packet. This document describes a modification to the Fast Recovery algorithm in RFC 5681 that incorporates a response to partialacknowledgementsacknowledgments received during Fast Recovery. We call this modified Fast Recovery algorithm NewReno, because it is a slight but significant variation of the basic Reno algorithm in RFC 5681. This document does not discuss the other suggestions in [Hoe95] and [Hoe96], such as a change to the ssthresh parameter during Slow-Start, or the proposal to send a new packet for every two duplicateacknowledgementsacknowledgments during Fast Recovery. The version of NewReno in this document also draws on other discussions of NewReno in the literature [LM97, Hen98]. We do not claim that the NewReno version of Fast Recovery described here is an optimal modification of Fast Recovery for responding to partialacknowledgements,acknowledgments, for TCP connections that are unable to use SACK. Based on our experiences with the NewReno modification in the NS simulator [NS] and with numerous implementations of NewReno, we believe that this modification improves the performance of the Fast Retransmit and Fast Recovery algorithms in a wide variety of scenarios. 2. Terminology and Definitions In this document, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as described in BCP 14, RFC 2119 [RFC2119]. This RFC indicates requirement levels for compliant TCP implementations implementing the NewReno Fast Retransmit and Fast Recovery algorithms described in this document. This document assumes that the reader is familiar with the terms SENDER MAXIMUM SEGMENT SIZE (SMSS), CONGESTION WINDOW (cwnd), and FLIGHT SIZE (FlightSize) defined in [RFC5681]. FLIGHT SIZE is defined as in [RFC5681] as follows: FLIGHT SIZE: The amount of data that has been sent but not yet cumulatively acknowledged. 3. The Fast Retransmit and Fast Recovery Algorithms in NewReno The basic idea of these extensions to the Fast Retransmit and Fast Recovery algorithms described in [RFC5681] is as follows. The TCP sender can infer, from the arrival of duplicate acknowledgments, whether multiple losses in the same window of data have most likely occurred, and avoid taking a retransmit timeout or making multiple congestion window reductions due to such an event. The standard implementation of the Fast Retransmit and Fast Recovery algorithms is given in [RFC5681]. This section specifies the basic NewReno algorithm.SectionsSection 4throughdescribes heuristics for processing duplicate acknowledgments after a retransmission timeout. Sections 5 and 6describeprovide someoptional variants,guidance to implementors based on experience with NewReno implementations. Several appendices provide more background information andthe motivations behind them,describe variations that an implementor may want to consider when tuning performance for certain network scenarios.Sections 7 and 8 provide some guidance to implementors based on experience with NewReno implementations.The NewReno modificationconcernsapplies to the Fast Recovery procedure that begins when three duplicate ACKs are received and ends when either a retransmission timeout occurs or an ACK arrives that acknowledges all of the data up to and including the data that was outstanding when the Fast Recovery procedure began. The NewReno algorithm specified in this documentdiffers fromextends the implementation in [RFC5681]inby introducing a variable specified as "recover" whose initial value is theintroduction ofinitial send sequence number. This new variable is used by the sender to record the send sequence number that must be acknowledged before the Fast Recovery procedure is declared to be over. This variable"recover"is used below in step 1, in the response to a partial or newacknowledgementacknowledgment in step 5, and in modifications to step 1 and the addition of step 6 for avoiding multiple Fast Retransmits caused by the retransmission of packets already received by the receiver.The algorithm specified in this document uses a variable "recover", whose initial value is the initial send sequence number.1) Three duplicate ACKs: When the third duplicate ACK is received and the sender is not already in the Fast Recovery procedure, check to see if the CumulativeAcknowledgementAcknowledgment field covers more than "recover". If so, go to Step 1A. Otherwise, go to Step 1B. 1A) Invoking Fast Retransmit: If so, then set ssthresh to no more than the value given in equation 1 below. (This is equation 4 from [RFC5681]). ssthresh = max (FlightSize / 2, 2*SMSS) (1) In addition, record the highest sequence number transmitted in the variable "recover", and go to Step 2. 1B) Not invoking Fast Retransmit: Do not enter the Fast Retransmit and Fast Recovery procedure. In particular, do not change ssthresh, do not go to Step 2 to retransmit the "lost" segment, and do not execute Step 3 upon subsequent duplicate ACKs. 2) Entering Fast Retransmit: Retransmit the lost segment and set cwnd to ssthresh plus 3*SMSS. This artificially "inflates" the congestion window by the number of segments (three) that have left the network and the receiver has buffered. 3) Fast Recovery: For each additional duplicate ACK received while in Fast Recovery, increment cwnd by SMSS. This artificially inflates the congestion window in order to reflect the additional segment that has left the network. 4) Fast Recovery, continued: Transmit a segment, if allowed by the new value of cwnd and the receiver's advertised window. 5) When an ACK arrives that acknowledges new data, this ACK could be the acknowledgment elicited by the retransmission from step 2, or elicited by a later retransmission. Fullacknowledgements:acknowledgments: If this ACK acknowledges all of the data up to and including "recover", then the ACK acknowledges all the intermediate segments sent between the original transmission of the lost segment and the receipt of the third duplicate ACK. Set cwnd to either (1) min (ssthresh, max(FlightSize, SMSS) + SMSS) or (2) ssthresh, where ssthresh is the value set in step 1; this is termed "deflating" the window. (We note that "FlightSize" in step 1 referred to the amount of data outstanding in step 1, when Fast Recovery was entered, while "FlightSize" in step 5 refers to the amount of data outstanding in step 5, when Fast Recovery is exited.) If the second option is selected, the implementation is encouraged to take measures to avoid a possible burst of data, in case the amount of data outstanding in the network is much less than the new congestion window allows. A simple mechanism is to limit the number of data packets that can be sent in response to a singleacknowledgement; this is known as "maxburst_" in the NS simulator.acknowledgment. Exit the Fast Recovery procedure. Partialacknowledgements:acknowledgments: If this ACK does *not* acknowledge all of the data up to and including "recover", then this is a partial ACK. In this case, retransmit the first unacknowledged segment. Deflate the congestion window by the amount of new data acknowledged by the cumulativeacknowledgementacknowledgment field. If the partial ACK acknowledges at least one SMSS of new data, then add back SMSS bytes to the congestion window. As in Step 3, this artificially inflates the congestion window in order to reflect the additional segment that has left the network. Send a new segment if permitted by the new value of cwnd. This "partial window deflation" attempts to ensure that, when Fast Recovery eventually ends, approximately ssthresh amount of data will be outstanding in the network. Do not exit the Fast Recovery procedure (i.e., if any duplicate ACKs subsequently arrive, execute Steps 3 and 4 above). For the first partial ACK that arrives during Fast Recovery, also reset the retransmit timer. Timer management is discussed in more detail in Section 4. 6) Retransmit timeouts: After a retransmit timeout, record the highest sequence number transmitted in the variable "recover" and exit the Fast Recovery procedure if applicable. Step 1 specifies a check that the CumulativeAcknowledgementAcknowledgment field covers more than "recover". Because theacknowledgementacknowledgment field contains the sequence number that the sender next expects to receive, theacknowledgementacknowledgment "ack_number" covers more than "recover" when: ack_number - 1 > recover; i.e., at least one byte more of data is acknowledged beyond the highest byte that was outstanding when Fast Retransmit was last entered. Note that in Step 5, the congestion window is deflated after a partialacknowledgementacknowledgment is received. The congestion window was likely to have been inflated considerably when the partialacknowledgementacknowledgment was received. In addition, depending on the original pattern of packet losses, the partialacknowledgementacknowledgment might acknowledge nearly a window of data. In this case, if the congestion window was not deflated, the data sender might be able to send nearly a window of data back-to-back. This document does not specify the sender's response to duplicate ACKs when the Fast Retransmit/Fast Recovery algorithm is not invoked. This is addressed in other documents, such as those describing the Limited Transmit procedure [RFC3042]. This document also does not address issues of adjusting the duplicateacknowledgementacknowledgment threshold, but assumes the threshold specified in the IETF standards; the current standard is RFC 5681, which specifies a threshold of three duplicateacknowledgements.acknowledgments. As a final note, we would observe that in the absence of the SACK option, the data sender is working from limited information. When the issue of recovery from multiple dropped packets from a single window of data is of particular importance, the best alternative would be to use the SACK option. 4.Resetting the Retransmit Timer in Response to Partial Acknowledgements One possible variant toHandling Duplicate Acknowledgments After A Timeout After each retransmit timeout, theresponse to partial acknowledgements specifiedhighest sequence number transmitted so far is recorded inSection 3 concerns when to resettheretransmit timervariable "recover". If, after apartial acknowledgement. The algorithm in Section 3, Step 5, resets theretransmittimer only aftertimeout, thefirst partial ACK. In this case, if a large number ofTCP data sender retransmits three consecutive packetswere dropped from a window of data,that have already been received by theTCPdatasender's retransmit timer will ultimately expire, andreceiver, then the TCP data sender willinvoke Slow-Start. (This is illustrated on page 12 of [F98].) We callreceive three duplicate acknowledgments that do not cover more than "recover". In this case, theImpatient variantduplicate acknowledgments are not an indication ofNewReno. We notea new instance of congestion. They are simply an indication that theImpatient variantsender has unnecessarily retransmitted at least three packets. However, when a retransmitted packet is itself dropped, the sender can also receive three duplicate acknowledgments that do not cover more than "recover". In this case, the sender would have been better off if it had initiated Fast Retransmit. For a TCP that implements the algorithm specified in Section 3doesn't followof this document, therecommended algorithmsender does not infer a packet drop from duplicate acknowledgments inRFC 2988 of restartingthis scenario. As always, the retransmit timerafter every packet transmission or retransmission (step 5.1 of [RFC2988]). In contrast,is theNewReno simulationsbackup mechanism for inferring packet loss in[FF96] illustrate the algorithm described above with the modification that the retransmit timer is reset after each partial acknowledgement. We callthis case. There are several heuristics, based on timestamps or on theSlow-but-Steady variantamount ofNewReno. In this case, for a window with a large numberadvancement ofpacket drops,theTCP data sender retransmits at most one packet per roundtrip time. (This behavior is illustrated incumulative acknowledgment field, that allow theNew-Reno TCP simulation of Figure 5sender to distinguish, in[FF96],some cases, between three duplicate acknowledgments following a retransmitted packet that was dropped, andon page 11three duplicate acknowledgments from the unnecessary retransmission of[F98]). When Nthree packetshave been dropped from[Gur03, GF04]. The TCP sender MAY use such awindow of data forheuristic to decide to invoke alarge value of N, the Slow-but-Steady variant can remain inFastRecovery for N round-trip times, retransmitting one more dropped packet each round-trip time; for these scenarios, the Impatient variant gives a faster recovery and better performance. The tests "ns test-suite-newreno.tcl impatient1" and "ns test-suite-newreno.tcl slow1"Retransmit in some cases, even when theNS simulator illustrate such a scenario, where the Impatient variant performs betterthree duplicate acknowledgments do not cover more than "recover". For example, when three duplicate acknowledgments are caused by theSlow-but-Steady variant. The Impatient variant canunnecessary retransmission of three packets, this is likely to beparticularly important for TCP connections with large congestion windows, as illustratedaccompanied by thetests "ns test-suite-newreno.tcl impatient4" and "ns test-suite-newreno.tcl slow4" in the NS simulator. One can also construct scenarios where the Slow-but-Steady variant gives better performance thancumulative acknowledgment field advancing by at least four segments. Similarly, a heuristic based on timestamps uses theImpatient variant. As an example, this occursfact that whenonlythere is asmall number of packets are dropped,hole in theRTOsequence space, the timestamp echoed in the duplicate acknowledgment issufficiently smallthe timestamp of the most recent data packet that advanced theretransmit timer expires, and performance would have been better without a retransmit timeout. The tests "ns test-suite-newreno.tcl impatient2"cumulative acknowledgment field [RFC1323]. If timestamps are used, and"ns test-suite-newreno.tcl slow2" intheNS simulator illustrate such a scenario. The Slow-but-Steady variant can also achieve higher goodput thansender stores theImpatient variant, by avoiding unnecessary retransmissions. This could betimestamp ofspecial interest for cellular links, where every transmission costs battery power and money. The tests "ns test-suite-newreno.tcl impatient3" and "ns test-suite-newreno.tcl slow3" intheNS simulator illustrate such a scenario. The Slow-but-Steady variantlast acknowledged segment, then the timestamp echoed by duplicate acknowledgments canalsobemore robustused todelay variation in the network, where a delay spike might force the Impatient variant intodistinguish between atimeoutretransmitted packet that was dropped andgo-back-N recovery. Neitherthree duplicate acknowledgments from the unnecessary retransmission of three packets. 4.1. ACK Heuristic If thetwo variants discussed above are optimal. Our recommendationACK-based heuristic isforused, then following theImpatient variant, as specified in Section 3 of this document, becauseadvancement of thepoor performancecumulative acknowledgment field, the sender stores the value of theSlow-but-Steady variant for TCP connections with large congestion windows. One possibility for a more optimal algorithm would be one that recovered from multiple packet drops as quicklyprevious cumulative acknowledgment asdoes slow-start, while resettingprev_highest_ack, and stores theretransmit timers after each partial acknowledgement,latest cumulative ACK asdescribed inhighest_ack. In addition, thesection below. We note, however, that therefollowing step isa limitation to the potential performance in this case in the absence of the SACK option. 5. Retransmissions after a Partial Acknowledgement One possible variant to the response to partial acknowledgements specifiedperformed if Step 1 in Section 3would befails, before proceeding toretransmitStep 1B. 1*) If the Cumulative Acknowledgment field didn't cover more thanone packet after each partial acknowledgement, and"recover", check toresetsee if theretransmit timer after each retransmission. The algorithm specified in Section 3 retransmits a single packet after each partial acknowledgement. Thiscongestion window is greater than SMSS bytes and the difference between highest_ack and prev_highest_ack is at mostconservative alternative,4*SMSS bytes. If true, duplicate ACKs indicate a lost segment (proceed to Step 1A inthat it is the leastSection 3). Otherwise, duplicate ACKs likelytoresultin an unnecessarily-retransmitted packet. A variant that would recover fasterfromaunnecessary retransmissions (proceed to Step 1B in Section 3). The congestion windowwith many packet drops would becheck serves toeffectively Slow-Start, retransmitting two packetsprotect against fast retransmit immediately aftereach partial acknowledgement. Such an approach would take less than N roundtrip times to recover from N losses [Hoe96]. However,a retransmit timeout. If several ACKs are lost, the sender can see a jump in theabsencecumulative ACK ofSACK, recovering as quickly as slow-start introducesmore than three segments, and thelikelihood of unnecessarily retransmitting packets, and this could significantly complicate the recovery mechanisms. We note that the response to partial acknowledgements specified in Section 3 of this document and inheuristic can fail. RFC2582 differs from the response in [FF96], even though both approaches only retransmit one packet in response to a partial acknowledgement. Step 5 of Section 3 specifies5681 recommends thatthe TCP sender responds toapartial ACK by deflating the congestion window by the amount of newreceiver should send duplicate ACKs for every out-of-order dataacknowledged, adding back SMSS bytespacket, such as a data packet received during Fast Recovery. The ACK heuristic is more likely to fail if thepartial ACK acknowledges at least SMSS bytesreceiver does not follow this advice, because then a smaller number ofnew data, and sendingACK losses are needed to produce anew segment if permitted bysufficient jump in thenew value of cwnd. Thus, only one previously-sent packetcumulative ACK. 4.2. Timestamp Heuristic If this heuristic isretransmitted in response to each partial acknowledgement, but additional new packets might be transmitted as well, depending onused, theamountsender stores the timestamp ofnew data acknowledged bythepartial acknowledgement.last acknowledged segment. Incontrast,addition, thevariantsecond paragraph ofNewReno illustrated in [FF96] simply set the congestion window to ssthresh when a partial acknowledgement was received. The approachstep 1 in[FF96]Section 3 is replaced as follows: 1**) If the Cumulative Acknowledgment field didn't cover moreconservative, and does not attemptthan "recover", check toaccurately tracksee if theactual number of outstanding packets after a partial acknowledgement is received. While either of these approaches gives acceptable performance,echoed timestamp in thevariant specifiedlast non-duplicate acknowledgment equals the stored timestamp. If true, duplicate ACKs indicate a lost segment (proceed to Step 1A in Section3 recovers more smoothly when multiple packets are dropped3). Otherwise, duplicate ACKs likely result froma window of data. (The [FF96] behavior can be seen in the NS simulator by setting the variable "partial_window_deflation_" for "Agent/TCP/Newreno"unnecessary retransmissions (proceed to0; the behavior specifiedStep 1B in Section3 is achieved3). The timestamp heuristic works correctly, both when the receiver echoes timestamps as specified bysetting "partial_window_deflation_" to 1.) 6. Avoiding Multiple Fast Retransmits This section describes[RFC1323], and by its revision attempts. However, if themotivation forreceiver arbitrarily echoes timestamps, thesender's state variable "recover", and discusses possible heuristics for distinguishing betweenheuristic can fail. The heuristic can also fail if aretransmitted packet thattimeout wasdropped,spurious andthree duplicate acknowledgementsreturning ACKs are not from retransmitted segments. This can be prevented by detection algorithms such as [RFC3522]. 5. Implementation Issues for theunnecessary retransmission of three packets. InData Receiver [RFC5681] specifies that "Out-of-order data segments SHOULD be acknowledged immediately, in order to accelerate loss recovery." Neal Cardwell has noted that some data receivers do not send an immediate acknowledgment when they send a partial acknowledgment, but instead wait first for their delayed acknowledgment timer to expire [C98]. As [C98] notes, this severely limits theabsencepotential benefit of NewReno by delaying theSACK option or timestamps, a duplicate acknowledgement carries no information to identifyreceipt of thedata packet or packetspartial acknowledgment at theTCPdatareceiver that triggered that duplicate acknowledgement. In this case,sender. Echoing RFC 5681, our recommendation is that theTCPdatasender is unable to distinguish betweenreceiver send an immediate acknowledgment for an out-of-order segment, even when that out-of-order segment fills aduplicate acknowledgementhole in the buffer. 6. Implementation Issues for the Data Sender In Section 3, Step 5 above, it is noted thatresults fromimplementations should take measures to avoid alost or delayedpossible burst of datapacket, and a duplicate acknowledgement that results fromwhen leaving Fast Recovery, in case thesender's unnecessary retransmissionamount ofanew datapacketthathad already been received attheTCP data receiver. Becausesender is eligible to send due to the new value ofthis, withtheRetransmit and Fast Recovery algorithms in Reno TCP, multiple segment losses from a singlecongestion windowof datais large. This cansometimes result in unnecessary multiple Fast Retransmits (and multiple reductions ofarise during NewReno when ACKs are lost or treated as pure window updates, thereby causing thecongestion window) [F94]. Withsender to underestimate theFast Retransmit and Fast Recovery algorithms in Reno TCP,number of new segments that can be sent during theperformance problems caused by multiple Fast Retransmits are relatively minor compared torecovery procedure. Specifically, bursts can occur when thepotential problems with Tahoe TCP, which does not implementFlightSize is much less than the new congestion window when exiting from Fast Recovery.Nevertheless, unnecessary Fast Retransmits can occur with Reno TCP unless some explicitOne simple mechanismis addedto avoidthis, such as the usea burst ofthe "recover" variable. (This modificationdata when leaving Fast Recovery iscalled "bugfix" in [F98], and is illustrated on pages 7 and 9 of that document. Unnecessary Fast Retransmits for Reno without "bugfix" is illustrated on page 6 of [F98].) Section 3 of [RFC2582] defined a default variant of NewReno TCP that did not use the variable "recover", and did not check if duplicate ACKs cover the variable "recover" before invoking Fast Retransmit. With this default variant from RFC 2582,to limit theproblem of multiple Fast Retransmits from a single windownumber of data packets that canoccur afterbe sent in response to aRetransmit Timeout (assingle acknowledgment. (This is known as "maxburst_" inpage 8 of [F98])the ns simulator.) Other possible mechanisms for avoiding bursts include rate-based pacing, orin scenarios with reordering (as insetting thevalidation test "./test-all-newreno newreno5_noBF" in directory "tcl/test" ofslow-start threshold to theNS simulator. This gives performance similarresultant congestion window and then resetting the congestion window tothatFlightSize. A recommendation onpage 8 of [F03].) RFC 2582 also defined Careful and Less Careful variants oftheNewReno algorithm, and recommendedgeneral mechanism to avoid excessively bursty sending patterns is outside theCareful variant. The algorithm specified in Section 3scope of thisdocument correspondsdocument. An implementation may want to use a separate flag to record whether or not it is presently in theCareful variantFast Recovery procedure. The use ofNewReno TCP from RFC 2582,the value of the duplicate acknowledgment counter for this purpose is not reliable because it can be reset upon window updates andeliminatesout-of-order acknowledgments. When updating theproblemCumulative Acknowledgment field outside ofmultipleFastRetransmits. This algorithm usesRecovery, the "recover" state variable"recover", whose initial value ismay also need to be updated in order to continue to permit possible entry into Fast Recovery (Section 3, step 1). This issue arises when an update of theinitial sendCumulative Acknowledgment field results in a sequencenumber. After each retransmit timeout,wraparound that affects thehighest sequence number transmitted so farordering between the Cumulative Acknowledgment field and the "recover" state variable. Entry into Fast Recovery isrecorded inonly possible when thevariable "recover". If, after a retransmit timeout,Cumulative Acknowledgment field covers more than the "recover" state variable. It is important for theTCP datasenderretransmits three consecutive packets that have already beento respond correctly to duplicate ACKs receivedby the data receiver, thenwhen theTCP datasenderwill receiveis no longer in Fast Recovery (e.g., because of a Retransmit Timeout). The Limited Transmit procedure [RFC3042] describes possible responses to the first and second duplicate acknowledgments. When three or more duplicateacknowledgements that do notacknowledgments are received, the Cumulative Acknowledgment field doesn't cover more than"recover". In this case, the duplicate acknowledgements are not an indication of"recover", and a newinstance of congestion. They are simply an indication that the sender has unnecessarily retransmitted at least three packets. However, when a retransmitted packetFast Recovery isitself dropped,not invoked, it is important that the sendercan also receive three duplicate acknowledgements that donotcover more than "recover". In this case,execute thesender would have been better off if it had initiatedFastRetransmit. For a TCP that implements the algorithm specifiedRecovery steps (3) and (4) in Section3 of this document,3. Otherwise, the senderdoes not infer a packet drop from duplicate acknowledgements in this scenario. As always, the retransmit timer is the backup mechanism for inferring packet losscould end up in a chain of spurious timeouts. We mention thiscase. There areonly because severalheuristics, based on timestamps or onNewReno implementations had this bug, including theamount of advancement ofimplementation in thecumulative acknowledgement field,NS simulator. It has been observed thatallow the sender to distinguish, insomecases, between three duplicate acknowledgements following a retransmitted packet that was dropped, and three duplicate acknowledgements from the unnecessary retransmission of three packets [Gur03, GF04]. TheTCPsender MAY use such a heuristic to decide to invokeimplementations enter aFast Retransmit in some cases, even when the three duplicate acknowledgements do not cover more than "recover". For example, when three duplicate acknowledgements are caused byslow start or congestion avoidance window updating algorithm immediately after theunnecessary retransmission of three packets, thiscwnd islikely to be accompaniedset by thecumulative acknowledgement field advancing by at least four segments. Similarly,equation found in (Section 3, step 5), even without aheuristic based on timestamps usesnew external event generating thefactcwnd change. Note thatwhen there is a hole in the sequence space, the timestamp echoed in the duplicate acknowledgementafter cwnd is set based on thetimestamp of the most recent data packet that advanced the cumulative acknowledgement field [RFC1323]. If timestamps are used, and the sender stores the timestamp of the last acknowledged segment, then the timestamp echoed by duplicate acknowledgements canprocedure for exiting Fast Recovery (Section 3, step 5), cwnd SHOULD NOT beused to distinguish betweenupdated until aretransmitted packetfurther event occurs (e.g., arrival of an ack, or timeout) after this adjustment. 7. Security Considerations RFC 5681 discusses general security considerations concerning TCP congestion control. This document describes a specific algorithm thatwas dropped and three duplicate acknowledgements fromconforms with theunnecessary retransmissioncongestion control requirements ofthree packets. The heuristicsRFC 5681, and so those considerations apply to this algorithm, too. There areillustrated inno known additional security concerns for this specific algorithm. 8. IANA Considerations This document has no actions for IANA. 9. Conclusions This document specifies theNS simulator inNewReno Fast Retransmit and Fast Recovery algorithms for TCP. This NewReno modification to TCP can even be important for TCP implementations that support thevalidation test "./test-all-newreno". 6.1. ACK Heuristic IfSACK option, because theACK-based heuristic is used, then followingSACK option can only be used for TCP connections when both TCP end-nodes support theadvancementSACK option. NewReno performs better than Reno (RFC 5681) in a number ofthe cumulative acknowledgement field, the sender stores the valuescenarios discussed herein. A number of options to theprevious cumulative acknowledgement as prev_highest_ack, and stores the latest cumulative ACK as highest_ack. In addition, the following step is performed if Step 1basic algorithm presented in Section 3fails, before proceedingare also described in appendices toStep 1B. 1*) Ifthis document. These include theCumulative Acknowledgement field didn't cover more than "recover", checkhandling of the retransmission timer (Appendix A), the response tosee ifpartial acknowledgments (Appendix B), and whether or not thecongestion windowsender maintains a state variable called "recover" (Appendix C). Our belief isgreater than SMSS bytes andthat thedifferencedifferences betweenhighest_ack and prev_highest_ack is at most 4*SMSS bytes. If true, duplicate ACKs indicate a lost segment (proceed to Step 1A in Section 3). Otherwise, duplicate ACKs likely result from unnecessary retransmissions (proceedthese variants of NewReno are small compared toStep 1B in Section 3). The congestion window check servesthe differences between Reno and NewReno. That is, the important thing is toprotect against fast retransmit immediately afterimplement NewReno instead of Reno, for aretransmit timeout, similarTCP connection without SACK; it is less important exactly which of the variants of NewReno is implemented. 10. Acknowledgments Many thanks to Anil Agarwal, Mark Allman, Armando Caro, Jeffrey Hsu, Vern Paxson, Kacheong Poon, Keyur Shah, and Bernie Volz for detailed feedback on this document or on its precursor, RFC 2582. Jeffrey Hsu provided clarifications on the"exitFastRetrans_" variable in NS. Exampleshandling ofapplyingtheACK heuristic are in validation tests "./test-all-newreno newreno_rto_loss_ack"recover variable that were applied to RFC 3782 as errata, and"./test-all-newreno newreno_rto_dup_ack"now are indirectory "tcl/test"Section 8 of this document. Yoshifumi Nishida contributed a modification to theNS simulator. If several ACKs are lost,fast recovery algorithm to account for thesender can see a jumpcase in which flightsize is 0 when thecumulative ACK of more than three segments,TCP sender leaves fast recovery, and theheuristic can fail. A validation test for this scenario is "./test-all-newreno newreno_rto_loss_ackf". RFC 5681 recommends that aTCP receivershould send duplicate ACKs for every out-of-order data packet, such as a data packet received during Fast Recovery. The ACK heuristic is more likelyuses delayed acknowledgments. Alexander Zimmermann provided several suggestions tofail ifimprove thereceiver does notclarity of the document. 11. References 11.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2988] Paxson, V. and M. Allman, "Computing TCP's Retransmission Timer", RFC 2988, November 2000. [RFC5681] Allman, M., Paxson, V. and E. Blanton, "TCP Congestion Control", RFC 5681, September 2009. 11.2. Informative References [C98] Cardwell, N., "delayed ACKs for retransmitted packets: ouch!". November 1998, Email to the tcpimpl mailing list, Message-ID "Pine.LNX.4.02A.9811021421340.26785-100000@sake.cs.washington.edu", archived at "http://tcp-impl.lerc.nasa.gov/tcp-impl". [F98] Floyd, S., Revisions to RFC 2001, "Presentation to the TCPIMPL Working Group", August 1998. URLs "ftp://ftp.ee.lbl.gov/talks/sf-tcpimpl-aug98.ps" and "ftp://ftp.ee.lbl.gov/talks/sf-tcpimpl-aug98.pdf". [F03] Floyd, S., "Moving NewReno from Experimental to Proposed Standard? Presentation to the TSVWG Working Group", March 2003. URLs "http://www.icir.org/floyd/talks/newreno-Mar03.ps" and "http://www.icir.org/floyd/talks/newreno-Mar03.pdf". [FF96] Fall, K. and S. Floyd, "Simulation-based Comparisons of Tahoe, Reno and SACK TCP", Computer Communication Review, July 1996. URL "ftp://ftp.ee.lbl.gov/papers/sacks.ps.Z". [F94] Floyd, S., "TCP and Successive Fast Retransmits", Technical report, October 1994. URL "ftp://ftp.ee.lbl.gov/papers/fastretrans.ps". [GF04] Gurtov, A. and S. Floyd, "Resolving Acknowledgment Ambiguity in non-SACK TCP", Next Generation Teletraffic and Wired/Wireless Advanced Networking (NEW2AN'04), February 2004. URL "http://www.cs.helsinki.fi/u/gurtov/papers/ heuristics.html". [Gur03] Gurtov, A., "[Tsvwg] resolving the problem of unnecessary fast retransmits in go-back-N", email to the tsvwg mailing list, message ID <3F25B467.9020609@cs.helsinki.fi>, July 28, 2003. URL "http://www1.ietf.org/mail-archive/working-groups/tsvwg/current/msg04334.html". [Hen98] Henderson, T., Re: NewReno and the 2001 Revision. September 1998. Email to the tcpimpl mailing list, Message ID "Pine.BSI.3.95.980923224136.26134A-100000@raptor.CS.Berkeley.EDU", archived at "http://tcp-impl.lerc.nasa.gov/tcp-impl". [Hoe95] Hoe, J., "Startup Dynamics of TCP's Congestion Control and Avoidance Schemes", Master's Thesis, MIT, 1995. [Hoe96] Hoe, J., "Improving the Start-up Behavior of a Congestion Control Scheme for TCP", ACM SIGCOMM, August 1996. URL "http://www.acm.org/sigcomm/sigcomm96/program.html". [LM97] Lin, D. and R. Morris, "Dynamics of Random Early Detection", SIGCOMM 97, September 1997. URL "http://www.acm.org/sigcomm/sigcomm97/program.html". [NS] The Network Simulator (NS). URL "http://www.isi.edu/nsnam/ns/". [PF01] Padhye, J. and S. Floyd, "Identifying the TCP Behavior of Web Servers", June 2001, SIGCOMM 2001. [RFC1323] Jacobson, V., Braden, R. and D. Borman, "TCP Extensions for High Performance", RFC 1323, May 1992. [RFC2582] Floyd, S. and T. Henderson, "The NewReno Modification to TCP's Fast Recovery Algorithm", RFC 2582, April 1999. [RFC2883] Floyd, S., J. Mahdavi, M. Mathis, and M. Podolsky, "The Selective Acknowledgment (SACK) Option for TCP, RFC 2883, July 2000. [RFC3042] Allman, M., Balakrishnan, H. and S. Floyd, "Enhancing TCP's Loss Recovery Using Limited Transmit", RFC 3042, January 2001. [RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm for TCP", RFC 3522, April 2003. [RFC3782] Floyd, S., T. Henderson, and A. Gurtov, "The NewReno Modification to TCP's Fast Recovery Algorithm", RFC 3782, April 2004. Appendix A. Resetting the Retransmit Timer in Response to Partial Acknowledgments One possible variant to the response to partial acknowledgments specified in Section 3 concerns when to reset the retransmit timer after a partial acknowledgment. The algorithm in Section 3, Step 5, resets the retransmit timer only after the first partial ACK. In this case, if a large number of packets were dropped from a window of data, the TCP data sender's retransmit timer will ultimately expire, and the TCP data sender will invoke Slow-Start. (This is illustrated on page 12 of [F98].) We call this the Impatient variant of NewReno. We note that the Impatient variant in Section 3 doesn't follow the recommended algorithm in RFC 2988 of restarting the retransmit timer after every packet transmission or retransmission (step 5.1 of [RFC2988]). In contrast, the NewReno simulations in [FF96] illustrate the algorithm described above with the modification that the retransmit timer is reset after each partial acknowledgment. We call thisadvice, because thenthe Slow-but-Steady variant of NewReno. In this case, for asmallerwindow with a large number ofACK losses are needed to producepacket drops, the TCP data sender retransmits at most one packet per roundtrip time. (This behavior is illustrated in the New-Reno TCP simulation of Figure 5 in [FF96], and on page 11 of [F98]). When N packets have been dropped from asufficient jumpwindow of data for a large value of N, the Slow-but-Steady variant can remain in Fast Recovery for N round-trip times, retransmitting one more dropped packet each round-trip time; for these scenarios, thecumulative ACK. 6.2. Timestamp Heuristic IfImpatient variant gives a faster recovery and better performance. The Impatient variant can be particularly important for TCP connections with large congestion windows. One can also construct scenarios where the Slow-but-Steady variant gives better performance than the Impatient variant. As an example, thisheuristicoccurs when only a small number of packets are dropped, the RTO isused,sufficiently small that thesender storesretransmit timer expires, and performance would have been better without a retransmit timeout. The Slow-but-Steady variant can also achieve higher goodput than thetimestampImpatient variant, by avoiding unnecessary retransmissions. This could be of special interest for cellular links, where every transmission costs battery power and money. The Slow-but-Steady variant can also be more robust to delay variation in thelast acknowledged segment. In addition,network, where a delay spike might force the Impatient variant into a timeout and go-back-N recovery. Neither of the two variants discussed above are optimal. Our recommendation is for thesecond paragraph of step 1Impatient variant, as specified in Section 3is replaced as follows: 1**) Ifof this document, because of theCumulative Acknowledgement field didn't coverpoor performance of the Slow-but-Steady variant for TCP connections with large congestion windows. One possibility for a morethan "recover", check to see ifoptimal algorithm would be one that recovered from multiple packet drops as quickly as does slow-start, while resetting theechoed timestampretransmit timers after each partial acknowledgment, as described in thelast non-duplicate acknowledgment equals the stored timestamp. If true, duplicate ACKs indicatesection below. We note, however, that there is alost segment (proceedlimitation toStep 1Athe potential performance inSection 3). Otherwise, duplicate ACKs likely result from unnecessary retransmissions (proceed to Step 1Bthis case inSection 3). Examplesthe absence ofapplyingthetimestamp heuristic are in validation tests "./test-all-newreno newreno_rto_loss_tsh" and "./test-all-newreno newreno_rto_dup_tsh". The timestamp heuristic works correctly, both whenSACK option. Appendix B. Retransmissions after a Partial Acknowledgment One possible variant to thereceiver echoes timestamps asresponse to partial acknowledgments specifiedby [RFC1323],in Section 3 would be to retransmit more than one packet after each partial acknowledgment, andby its revision attempts. However, if the receiver arbitrarily echoes timestamps,to reset theheuristic can fail.retransmit timer after each retransmission. Theheuristic can also fail ifalgorithm specified in Section 3 retransmits atimeout was spurious and returning ACKs are not from retransmitted segments. This can be prevented by detection algorithms such as [RFC3522]. 7. Implementation Issues for the Data Receiver [RFC5681] specifies that "Out-of-order data segments SHOULD be acknowledged immediately,single packet after each partial acknowledgment. This is the most conservative alternative, inorder to accelerate loss recovery." Neal Cardwell has notedthatsome data receivers do not sendit is the least likely to result in animmediate acknowledgement when they sendunnecessarily-retransmitted packet. A variant that would recover faster from a window with many packet drops would be to effectively Slow-Start, retransmitting two packets after each partialacknowledgment, but instead wait first for their delayed acknowledgement timeracknowledgment. Such an approach would take less than N roundtrip times toexpire [C98]. As [C98] notes, this severely limitsrecover from N losses [Hoe96]. However, in thepotential benefitabsence ofNewReno by delayingSACK, recovering as quickly as slow-start introduces thereceiptlikelihood of unnecessarily retransmitting packets, and this could significantly complicate thepartial acknowledgement atrecovery mechanisms. We note that thedata sender. Echoingresponse to partial acknowledgments specified in Section 3 of this document and in RFC5681, our recommendation is that2582 differs from thedata receiver send an immediate acknowledgement for an out-of-order segment,response in [FF96], evenwhen that out-of-order segment fills a holethough both approaches only retransmit one packet inthe buffer. 8. Implementation Issues for the Data Sender In Section 3,response to a partial acknowledgment. Step 5above, it is notedof Section 3 specifies thatimplementations should take measuresthe TCP sender responds toavoidapossible burst of data when leaving Fast Recovery, in casepartial ACK by deflating the congestion window by the amount of new datathatacknowledged, adding back SMSS bytes if thesender is eligible to send due topartial ACK acknowledges at least SMSS bytes of new data, and sending a new segment if permitted by the new value ofthe congestion windowcwnd. Thus, only one previously-sent packet islarge. This can arise during NewReno when ACKs are lost or treated as pure window updates, thereby causing the senderretransmitted in response tounderestimateeach partial acknowledgment, but additional new packets might be transmitted as well, depending on thenumberamount of newsegments that can be sent duringdata acknowledged by therecovery procedure. Specifically, bursts can occur whenpartial acknowledgment. In contrast, theFlightSize is much less thanvariant of NewReno illustrated in [FF96] simply set thenewcongestion windowwhen exiting from Fast Recovery. One simple mechanismtoavoid a burst of datassthresh whenleaving Fast Recoverya partial acknowledgment was received. The approach in [FF96] is more conservative, and does not attempt tolimitaccurately track the actual number ofdataoutstanding packetsthat can be sent in response toafter asingle acknowledgment. (Thispartial acknowledgment isknown as "maxburst_"received. While either of these approaches gives acceptable performance, the variant specified in Section 3 recovers more smoothly when multiple packets are dropped from a window of data. Appendix C. Avoiding Multiple Fast Retransmits This appendix describes thens simulator.) Other possible mechanismsmotivation foravoiding bursts include rate-based pacing, or settingtheslow-start threshold tosender's state variable "recover". In theresultant congestion window and then resettingabsence of thecongestion windowSACK option or timestamps, a duplicate acknowledgment carries no information toFlightSize. A recommendation onidentify thegeneral mechanism to avoid excessively bursty sending patterns is outsidedata packet or packets at thescope ofTCP data receiver that triggered that duplicate acknowledgment. In thisdocument. An implementation may wantcase, the TCP data sender is unable tousedistinguish between aseparate flag to record whetherduplicate acknowledgment that results from a lost ornot it is presently indelayed data packet, and a duplicate acknowledgment that results from theFast Recovery procedure. The usesender's unnecessary retransmission of a data packet that had already been received at thevalueTCP data receiver. Because of this, with theduplicate acknowledgment counter for this purpose is not reliable because it can be reset upon window updatesRetransmit andout-of-order acknowledgments. When updating the Cumulative Acknowledgement field outside of Fast Recovery, the "recover" state variable may also need to be updated in order to continue to permit possible entry intoFast Recovery(Section 3, step 1). This issue arises when an update of the Cumulative Acknowledgement field resultsalgorithms in Reno TCP, multiple segment losses from asequence wraparound that affectssingle window of data can sometimes result in unnecessary multiple Fast Retransmits (and multiple reductions of theordering betweencongestion window) [F94]. With theCumulative Acknowledgement fieldFast Retransmit andthe "recover" state variable. Entry intoFast Recoveryis only possible when the Cumulative Acknowledgment field covers more than the "recover" state variable. It is important foralgorithms in Reno TCP, thesender to respond correctlyperformance problems caused by multiple Fast Retransmits are relatively minor compared toduplicate ACKs received whenthesender is no longer inpotential problems with Tahoe TCP, which does not implement FastRecovery (e.g., because of a Retransmit Timeout). The Limited Transmit procedure [RFC3042] describes possible responsesRecovery. Nevertheless, unnecessary Fast Retransmits can occur with Reno TCP unless some explicit mechanism is added to avoid this, such as thefirst and second duplicate acknowledgements. When three or more duplicate acknowledgements are received,use of theCumulative Acknowledgement field doesn't cover more than "recover","recover" variable. (This modification is called "bugfix" in [F98], anda new Fast Recoveryisnot invoked, itillustrated on pages 7 and 9 of that document. Unnecessary Fast Retransmits for Reno without "bugfix" isimportantillustrated on page 6 of [F98].) Section 3 of [RFC2582] defined a default variant of NewReno TCP that did not use thesendervariable "recover", and did notexecutecheck if duplicate ACKs cover the variable "recover" before invoking FastRecovery steps (3) and (4) in Section 3. Otherwise,Retransmit. With this default variant from RFC 2582, thesender could end up inproblem of multiple Fast Retransmits from achainsingle window ofspurious timeouts. We mention this only because several NewReno implementations had this bug, including the implementationdata can occur after a Retransmit Timeout (as inthe NS simulator. (This bugpage 8 of [F98]) or in scenarios with reordering. RFC 2582 also defined Careful and Less Careful variants of theNS simulator was fixedNewReno algorithm, and recommended the Careful variant. The algorithm specified inJuly 2003, withSection 3 of this document corresponds to thevariable "exitFastRetrans_".) It has been observed that someCareful variant of NewReno TCPimplementations enter a slow start or congestion avoidance window updatingfrom RFC 2582, and eliminates the problem of multiple Fast Retransmits. This algorithmimmediately afteruses thecwndvariable "recover", whose initial value isset bytheequation foundinitial send sequence number. After each retransmit timeout, the highest sequence number transmitted so far is recorded in the variable "recover". Appendix D. Simulations This section provides pointers to simulation scripts available in(Section 3, step 5), even without a new external event generatingthecwnd change. NoteNS simulator thatafter cwndreproduce behavior described above. In Section 3, a simple mechanism isset based ondescribed to limit theprocedure for exiting Fast Recovery (Section 3, step 5), cwnd SHOULD NOTnumber of data packets that can beupdated untilsent in response to afurther event occurs (e.g., arrival of an ack, or timeout) after this adjustment. 9. Simulationssingle acknowledgment. This is known as "maxburst_" in the NS simulator. Simulations with NewReno are illustrated with the validation test "tcl/test/test-all-newreno" in the NS simulator. The command "../../ns test-suite-newreno.tcl reno" shows a simulation with Reno TCP, illustrating the data sender's lack of response to a partialacknowledgement.acknowledgment. In contrast, the command "../../ns test-suite-newreno.tclnewreno_B" shows a simulation with the same scenario using the NewReno algorithms described in this paper. 10. Comparisons between Reno and NewReno TCP As we stated in the introduction, we believe that the NewReno modification described in this document improves the performance of the Fast Retransmit and Fast Recovery algorithms of Reno TCP in a wide variety of scenarios. This has been discussed in some depth in [FF96], which illustrates Reno TCP's poor performance when multiple packets are dropped from a window of data and also illustrates NewReno TCP's good performance in that scenario. We do, however, know of one scenario where Reno TCP gives better performance than NewReno TCP, that we describe here for the sake of completeness. Consider a scenario with no packet loss, but with sufficient reordering so that the TCP sender receives three duplicate acknowledgements. This will trigger the Fast Retransmit and Fast Recovery algorithms. With Reno TCP or with Sack TCP, this will result in the unnecessary retransmission ofnewreno_B" shows asingle packet, combinedsimulation witha halving ofthecongestion window (shown on pages 4 and 6 of [F03]). Withsame scenario using the NewRenoTCP, however, this reordering will also resultalgorithms described in this paper. Regarding theunnecessary retransmissionhandling ofan entireduplicate acknowledgments after a timeout, the congestion windowof data (shown on page 5 of [F03]). While Reno TCP performs better than NewReno TCP incheck serves to protect against fast retransmit immediately after a retransmit timeout, similar to thepresence of reordering, NewReno's superior performance"exitFastRetrans_" variable in NS. Examples of applying thepresenceACK heuristic (Section 4) are in validation tests "./test-all-newreno newreno_rto_loss_ack" and "./test-all-newreno newreno_rto_dup_ack" in directory "tcl/test" ofmultiple packet drops generally outweighs its less optimal performancethe NS simulator. If several ACKs are lost, the sender can see a jump in thepresencecumulative ACK ofreordering. (Sack TCPmore than three segments, and the heuristic can fail. A validation test for this scenario is "./test-all-newreno newreno_rto_loss_ackf". Examples of applying thepreferred solution, with good performancetimestamp heuristic (Section 4) are inboth scenarios.) This document recommends the Fast Retransmitvalidation tests "./test-all-newreno newreno_rto_loss_tsh" andFast Recovery algorithms of NewReno TCP instead of those of Reno TCP for those TCP connections that do not support SACK. We would also note that NewReno's Fast Retransmit"./test-all-newreno newreno_rto_dup_tsh". Section 6 described a problem involving possible spurious timeouts, andFast Recovery mechanisms are widely deployedmentions that this bug existed inTCP implementationsthe NS simulator. This bug in theInternet today, as documentedNS simulator was fixed in[PF01]. For example,July 2003, with the variable "exitFastRetrans_". Regarding the Slow-but-Steady and Impatient variants described in Appendix A, The testsof TCP implementations"ns test-suite-newreno.tcl impatient1" and "ns test-suite-newreno.tcl slow1" inseveral thousand web serversthe NS simulator illustrate a scenario in2001 showed thatwhich the Impatient variant performs better than the Slow-but-Steady variant. The Impatient variant can be particularly important forthoseTCP connectionswhere the web browser was not SACK-capable, more web servers usedwith large congestion windows, as illustrated by theFast Retransmittests "ns test-suite-newreno.tcl impatient4" andFast Recovery algorithms of NewReno than those of Reno or Tahoe TCP [PF01]. 11. Changes Relative to RFC 2582 The purpose of this document is to advance"ns test-suite-newreno.tcl slow4" in theNewReno's Fast RetransmitNS simulator. The tests "ns test-suite-newreno.tcl impatient2" andFast Recovery algorithms"ns test-suite-newreno.tcl slow2" inRFC 2582 to Standards Track. The main changethe NS simulator illustrate scenarios inthis document relative to RFC 2582 is to specifywhich theCarefulSlow-but-Steady variantof NewReno's Fast Retransmit and Fast Recovery algorithms.outperforms the Impatient variant. Thebase algorithm described in RFC 2582 did not attempt to avoid unnecessary multiple Fast Retransmits that can occur after a timeout (describedtests "ns test-suite-newreno.tcl impatient3" and "ns test-suite-newreno.tcl slow3" inmore detailthe NS simulator illustrate scenarios in which thesection above). However, RFC 2582 also defined "Careful" and "Less Careful"Slow-but-Steady variantsthatavoidtheseunnecessaryFast Retransmits, and recommended the Careful variant. This document specifiesretransmissions. Appendix B describes different policies for partial window deflation. The [FF96] behavior can be seen in thepreviously-named "Careful" variant asNS simulator by setting thebasic version of NewReno. As described below, this algorithm uses avariable"recover", whose initial value is the send sequence number. The algorithm"partial_window_deflation_" for "Agent/TCP/Newreno" to 0; the behavior specified in Section 3checks whether the acknowledgement fieldis achieved by setting "partial_window_deflation_" to 1. Section 3 ofa partial acknowledgement covers *more* than "recover", as[RFC2582] definedin Section 3. Another possiblea default variantwould be to simply requireof NewReno TCP that did not use theacknowledgement field covers *more than or equal to*variable "recover", and did not check if duplicate ACKs cover the variable "recover" beforeinitiating anotherinvoking Fast Retransmit.We calledWith thisthe Less Carefuldefault variantinfrom RFC2582. There are two separate scenarios in which the TCP sender could receive three duplicate acknowledgements acknowledging "recover" but no more than "recover". One scenario would be that2582, the problem of multiple Fast Retransmits from a single window of datasender transmitted four packetscan occur after a Retransmit Timeout (as in page 8 of [F98]) or in scenarios withsequence numbers higher than "recover", that the first packet was droppedreordering (as An NS validation test "./test-all-newreno newreno5_noBF" inthe network, and the following three packets triggered three duplicate acknowledgements acknowledging "recover". The second scenario would be that the sender unnecessarily retransmitted three packets below "recover", and that these three packets triggered three duplicate acknowledgements acknowledging "recover". In the absencedirectory "tcl/test" ofSACK,theTCP sender is unable to distinguish between these two scenarios. ForNS simulator illustartes theCarefuldefault variant ofFast Retransmit,NewReno TCP that doesn't use thedata sender would havevariable "recover"; this gives performance similar towait for a retransmit timeout in the first scenario, but would not have an unnecessary Fast Retransmitthat on page 8 of [F03]. Appendix E. Comparisons between Reno and NewReno TCP As we stated in thesecond scenario. Forintroduction, we believe that theLess Careful variant to Fast Retransmit,NewReno modification described in this document improves the performance of thedata sender wouldFast Retransmitas desired in the first scenario,andwould unnecessarilyFastRetransmitRecovery algorithms of Reno TCP inthe second scenario.a wide variety of scenarios. Thisdocument only specifies the Careful varianthas been discussed inSection 3. Unnecessary Fast Retransmits with the Less Careful variantsome depth inscenarios with reordering[FF96], which illustrates Reno TCP's poor performance when multiple packets areillustrated in page 8dropped from a window of[F03]. The documentdata and alsospecifies two heuristicsillustrates NewReno TCP's good performance in that scenario. We do, however, know of one scenario where Reno TCP gives better performance than NewReno TCP, that we describe here for the sake of completeness. Consider a scenario with no packet loss, but with sufficient reordering so that the TCP senderMAY use to decide to invoke Fast Retransmit even when thereceives three duplicateacknowledgements do not cover more than "recover". These heuristics, an ACK-based heuristicacknowledgments. This will trigger the Fast Retransmit anda timestamp heuristic, are describedFast Recovery algorithms. With Reno TCP or with Sack TCP, this will result inSections 6.1 and 6.2 respectively. 12. Changes Relative to RFC 3782 In [RFC3782],thecwnd after Full ACK reception will be set to (1) min (ssthresh, FlightSize + SMSS) or (2) ssthresh. However, there isunnecessary retransmission of arisk insingle packet, combined with a halving of thefirst logic which resultscongestion window (shown on pages 4 and 6 of [F03]). With NewReno TCP, however, this reordering will also result inperformance degradation. Withthefirst logic, if FlightSize is zero, the result will be 1 SMSS. This meansunnecessary retransmission of an entire window of data (shown on page 5 of [F03]). While Reno TCP performs better than NewReno TCPcan transmit only 1 segment at this moment, which can cause delayinACK transmission at receiver due to delayed ACK algorithm. The FlightSize on Full ACK reception can be zerothe presence of reordering, NewReno's superior performance insome situations. A typical example is where sending window size during fast recovery is small. In this case,theretransmittedpresence of multiple packetand new data packets can be transmitted within a short interval. If all these packets successfully arrive,drops generally outweighs its less optimal performance in thereceiver may generate a Full ACK that acknowledges all outstanding data. Even if window size is not small, losspresence ofACK packets or receive buffer shortage during fast recovery can also increasereordering. (Sack TCP is thepossibility to fall into this situation. The proposed fixpreferred solution, with good performance inthis document ensures that sender TCP transmits at least two segments on Full ACK reception. In addition, errata for RFC3782 (editorial clarification to Section 8) has been applied. 13. Conclusionsboth scenarios.) This documentspecifiesrecommends theNewRenoFast Retransmit and Fast Recovery algorithmsfor TCP. Thisof NewRenomodification toTCPcan even be importantinstead of those of Reno TCP for those TCP connections that do not support SACK. We would also note that NewReno's Fast Retransmit and Fast Recovery mechanisms are widely deployed in TCP implementations in the Internet today, as documented in [PF01]. For example, tests of TCP implementations in several thousand web servers in 2001 showed thatsupport the SACK option, because the SACK option can only be usedfor those TCP connectionswhen both TCP end-nodes supportwhere theSACK option.web browser was not SACK-capable, more web servers used the Fast Retransmit and Fast Recovery algorithms of NewRenoperforms betterthanReno (RFC 5681) in a number of scenarios discussed herein. A numberthose ofoptionsReno or Tahoe TCP [PF01]. Appendix F. Changes Relative tothe basic algorithm presented in Section 3 are also described. These include the handlingRFC 2582 The purpose ofthe retransmission timer (Section 4), the responsethis document is topartial acknowledgments (Section 5), and the value ofadvance thecongestion window when leavingNewReno's Fast Retransmit and Fast Recovery(section 3, step 5). Our beliefalgorithms in RFC 2582 to Standards Track. The main change in this document relative to RFC 2582 isthat the differences between these variants of NewReno are small comparedto specify thedifferences between RenoCareful variant of NewReno's Fast Retransmit andNewReno. That is, the important thing isFast Recovery algorithms. The base algorithm described in RFC 2582 did not attempt toimplement NewReno instead of Reno, foravoid unnecessary multiple Fast Retransmits that can occur after aTCP connection without SACK; it is less important exactly which oftimeout (described in more detail in thevariants of NewReno is implemented. 14. Security Considerationssection above). However, RFC5681 discusses general security considerations concerning TCP congestion control.2582 also defined "Careful" and "Less Careful" variants that avoid these unnecessary Fast Retransmits, and recommended the Careful variant. This documentdescribesspecifies the previously-named "Careful" variant as the basic version of NewReno. As described below, this algorithm uses aspecificvariable "recover", whose initial value is the send sequence number. The algorithmthat conforms withspecified in Section 3 checks whether thecongestion control requirementsacknowledgment field ofRFC 5681, and so those considerations applya partial acknowledgment covers *more* than "recover", as defined in Section 3. Another possible variant would be to simply require that the acknowledgment field covers *more than or equal to* "recover" before initiating another Fast Retransmit. We called thisalgorithm, too.the Less Careful variant in RFC 2582. There are two separate scenarios in which the TCP sender could receive three duplicate acknowledgments acknowledging "recover" but noknown additional security concerns for this specific algorithm. 15. IANA Considerations This document has no actions for IANA. 16. Acknowledgements Many thanks to Anil Agarwal, Mark Allman, Armando Caro, Jeffrey Hsu, Vern Paxson, Kacheong Poon, Keyur Shah, and Bernie Volz for detailed feedback on this document or on its precursor, RFC 2582. Jeffrey Hsu provided clarifications onmore than "recover". One scenario would be that thehandling ofdata sender transmitted four packets with sequence numbers higher than "recover", that therecover variablefirst packet was dropped in the network, and the following three packets triggered three duplicate acknowledgments acknowledging "recover". The second scenario would be thatwere applied to RFC 3782 as errata,the sender unnecessarily retransmitted three packets below "recover", andnow are in Section 8that these three packets triggered three duplicate acknowledgments acknowledging "recover". In the absence ofthis document. Yoshifumi Nishida contributed a modificationSACK, the TCP sender is unable to distinguish between these two scenarios. For thefast recovery algorithmCareful variant of Fast Retransmit, the data sender would have toaccountwait for a retransmit timeout in thecasefirst scenario, but would not have an unnecessary Fast Retransmit inwhich flightsize is 0 whentheTCPsecond scenario. For the Less Careful variant to Fast Retransmit, the data senderleaves fast recovery,would Fast Retransmit as desired in the first scenario, and would unnecessarily Fast Retransmit in theTCP receiver uses delayed acknowledgments. 17. References 17.1. Normative References [RFC2119] Bradner, S., "Key words for usesecond scenario. This document only specifies the Careful variant inRFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2988] Paxson, V. and M. Allman, "Computing TCP's Retransmission Timer", RFC 2988, November 2000. [RFC5681] Allman, M., Paxson, V. and E. Blanton, "TCP Congestion Control", RFC 5681, September 2009. 17.2. Informative References [C98] Cardwell, N., "delayed ACKs for retransmitted packets: ouch!". November 1998, Email toSection 3. Unnecessary Fast Retransmits with thetcpimpl mailing list, Message-ID "Pine.LNX.4.02A.9811021421340.26785-100000@sake.cs.washington.edu", archived at "http://tcp-impl.lerc.nasa.gov/tcp-impl". [F98] Floyd, S., Revisions to RFC 2001, "Presentation toLess Careful variant in scenarios with reordering are illustrated in page 8 of [F03]. The document also specifies two heuristics that theTCPIMPL Working Group", August 1998. URLs "ftp://ftp.ee.lbl.gov/talks/sf-tcpimpl-aug98.ps" and "ftp://ftp.ee.lbl.gov/talks/sf-tcpimpl-aug98.pdf". [F03] Floyd, S., "Moving NewReno from ExperimentalTCP sender MAY use toProposed Standard? Presentationdecide tothe TSVWG Working Group", March 2003. URLs "http://www.icir.org/floyd/talks/newreno-Mar03.ps" and "http://www.icir.org/floyd/talks/newreno-Mar03.pdf". [FF96] Fall, K. and S. Floyd, "Simulation-based Comparisons of Tahoe, Reno and SACK TCP", Computer Communication Review, July 1996. URL "ftp://ftp.ee.lbl.gov/papers/sacks.ps.Z". [F94] Floyd, S., "TCP and Successiveinvoke FastRetransmits", Technical report, October 1994. URL "ftp://ftp.ee.lbl.gov/papers/fastretrans.ps". [GF04] Gurtov, A.Retransmit even when the three duplicate acknowledgments do not cover more than "recover". These heuristics, an ACK-based heuristic andS. Floyd, "Resolving Acknowledgment Ambiguitya timestamp heuristic, are described innon-SACK TCP", Next Generation TeletrafficSections 6.1 andWired/Wireless Advanced Networking (NEW2AN'04), February 2004. URL "http://www.cs.helsinki.fi/u/gurtov/papers/ heuristics.html". [Gur03] Gurtov, A., "[Tsvwg] resolving6.2 respectively. Appendix G. Changes Relative to RFC 3782 In [RFC3782], theproblem of unnecessary fast retransmits in go-back-N", emailcwnd after Full ACK reception will be set to (1) min (ssthresh, FlightSize + SMSS) or (2) ssthresh. However, there is a risk in thetsvwg mailing list, message ID <3F25B467.9020609@cs.helsinki.fi>, July 28, 2003. URL "http://www1.ietf.org/mail-archive/working-groups/tsvwg/current/msg04334.html". [Hen98] Henderson, T., Re: NewReno andfirst logic which results in performance degradation. With the2001 Revision. September 1998. Email tofirst logic, if FlightSize is zero, thetcpimpl mailing list, Message ID "Pine.BSI.3.95.980923224136.26134A-100000@raptor.CS.Berkeley.EDU", archivedresult will be 1 SMSS. This means TCP can transmit only 1 segment at"http://tcp-impl.lerc.nasa.gov/tcp-impl". [Hoe95] Hoe, J., "Startup Dynamics of TCP's Congestion Control and Avoidance Schemes", Master's Thesis, MIT, 1995. [Hoe96] Hoe, J., "Improving the Start-up Behavior of a Congestion Control Scheme for TCP", ACM SIGCOMM, August 1996. URL "http://www.acm.org/sigcomm/sigcomm96/program.html". [LM97] Lin, D. and R. Morris, "Dynamics of Random Early Detection", SIGCOMM 97, September 1997. URL "http://www.acm.org/sigcomm/sigcomm97/program.html". [NS]this moment, which can cause delay in ACK transmission at receiver due to delayed ACK algorithm. TheNetwork Simulator (NS). URL "http://www.isi.edu/nsnam/ns/". [PF01] Padhye, J.FlightSize on Full ACK reception can be zero in some situations. A typical example is where sending window size during fast recovery is small. In this case, the retransmitted packet andS. Floyd, "Identifyingnew data packets can be transmitted within a short interval. If all these packets successfully arrive, theTCP Behaviorreceiver may generate a Full ACK that acknowledges all outstanding data. Even if window size is not small, loss ofWeb Servers", June 2001, SIGCOMM 2001. [RFC1323] Jacobson, V., Braden, R. and D. Borman, "TCP Extensions for High Performance", RFC 1323, May 1992. [RFC2582] Floyd, S. and T. Henderson, "The NewReno ModificationACK packets or receive buffer shortage during fast recovery can also increase the possibility toTCP's Fast Recovery Algorithm", RFC 2582, April 1999. [RFC2883] Floyd, S., J. Mahdavi, M. Mathis, and M. Podolsky, "The Selective Acknowledgment (SACK) Optionfall into this situation. The proposed fix in this document ensures that sender TCP transmits at least two segments on Full ACK reception. In addition, errata forTCP, RFC 2883, July 2000. [RFC3042] Allman, M., Balakrishnan, H.RFC3782 (editorial clarification to Section 8 of RFC2582, which is now Section 6 of this document) has been applied. Sections 4, 5, andS. Floyd, "Enhancing TCP's Loss Recovery Using Limited Transmit", RFC 3042, January 2001. [RFC3522] Ludwig, R.9-11 of RFC2582 were relocated to appendices of this document since they are non-normative andM. Meyer, "The Eifel Detection Algorithm for TCP", RFC 3522, April 2003. [RFC3782] Floyd, S., T. Henderson,provide background information andA. Gurtov, "The NewReno Modificationreferences toTCP's Fast Recovery Algorithm", RFC 3782, April 2004.simulation results. AppendixA.H. Document Revision History To be removed upon publication +----------+--------------------------------------------------+ | Revision | Comments | +----------+--------------------------------------------------+ | draft-00 | RFC3782 errata applied, and changes applied from | | | draft-nishida-newreno-modification-02 | +----------+--------------------------------------------------+ | draft-01 | Non-normative sections moved to appendices, | | | editorial clarifications applied as suggested | | | by Alexander Zimmermann. | +----------+--------------------------------------------------+ Authors' AddressesThomas R.Tom Henderson The Boeing CompanyP.O. Box 3707 Seattle, WA 98124EMail: thomas.r.henderson@boeing.com Sally Floyd International Computer Science Institute Phone: +1 (510) 666-2989 EMail: floyd@acm.org URL: http://www.icir.org/floyd/ Andrei Gurtov HIIT Helsinki Institute for Information TechnologyAalto UniversityP.O. Box19800 Helsinki FIN-00076 FINLAND19215 00076 Aalto Finland EMail: gurtov@hiit.fi Yoshifumi Nishida WIDE Project Endo 5322 Fujisawa, Kanagawa 252-8520 Japan Email: nishida@wide.ad.jp