draft-ietf-tcpm-fastopen-10.txt   rfc7413.txt 
Internet Draft Y. Cheng Internet Engineering Task Force (IETF) Y. Cheng
draft-ietf-tcpm-fastopen-10.txt J. Chu Request for Comments: 7413 J. Chu
Intended status: Experimental S. Radhakrishnan Category: Experimental S. Radhakrishnan
Expiration date: April, 2015 A. Jain ISSN: 2070-1721 A. Jain
Google Google
September 29, 2014 December 2014
TCP Fast Open TCP Fast Open
Abstract Abstract
This document describes an experimental TCP mechanism TCP Fast Open This document describes an experimental TCP mechanism called TCP Fast
(TFO). TFO allows data to be carried in the SYN and SYN-ACK packets Open (TFO). TFO allows data to be carried in the SYN and SYN-ACK
and consumed by the receiving end during the initial connection packets and consumed by the receiving end during the initial
handshake, and saves up to one full round trip time (RTT) compared to connection handshake, and saves up to one full round-trip time (RTT)
the standard TCP, which requires a three-way handshake (3WHS) to compared to the standard TCP, which requires a three-way handshake
complete before data can be exchanged. However TFO deviates from the (3WHS) to complete before data can be exchanged. However, TFO
standard TCP semantics since the data in the SYN could be replayed to deviates from the standard TCP semantics, since the data in the SYN
an application in some rare circumstances. Applications should not could be replayed to an application in some rare circumstances.
use TFO unless they can tolerate this issue detailed in the Applications should not use TFO unless they can tolerate this issue,
Applicability section. as detailed in the Applicability section.
Status of this Memo
Distribution of this memo is unlimited.
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Status of This Memo
Task Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months This document is not an Internet Standards Track specification; it is
and may be updated, replaced, or obsoleted by other documents at any published for examination, experimental implementation, and
time. It is inappropriate to use Internet-Drafts as reference evaluation.
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at This document defines an Experimental Protocol for the Internet
http://www.ietf.org/1id-abstracts.html community. This document is a product of the Internet Engineering
Task Force (IETF). It represents the consensus of the IETF
community. It has received public review and has been approved for
publication by the Internet Engineering Steering Group (IESG). Not
all documents approved by the IESG are a candidate for any level of
Internet Standard; see Section 2 of RFC 5741.
The list of Internet-Draft Shadow Directories can be accessed at Information about the current status of this document, any errata,
http://www.ietf.org/shadow.html and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc7413.
Copyright Notice Copyright Notice
Copyright (c) 2014 IETF Trust and the persons identified as the Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction ....................................................3
1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Terminology ................................................4
2. Data In SYN . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Data in SYN .....................................................4
2.1 Relaxing TCP Semantics on Duplicated SYNs . . . . . . . . . 4 2.1. Relaxing TCP Semantics on Duplicated SYNs ..................4
2.2. SYNs with Spoofed IP Addresses . . . . . . . . . . . . . . 4 2.2. SYNs with Spoofed IP Addresses .............................5
3. Protocol Overview . . . . . . . . . . . . . . . . . . . . . . . 5 3. Protocol Overview ...............................................5
4. Protocol Details . . . . . . . . . . . . . . . . . . . . . . . 6 4. Protocol Details ................................................7
4.1. Fast Open Cookie . . . . . . . . . . . . . . . . . . . . . 6 4.1. Fast Open Cookie ...........................................7
4.1.1. Fast Open option . . . . . . . . . . . . . . . . . . . 7 4.1.1. Fast Open Option ....................................8
4.1.2. Server Cookie Handling . . . . . . . . . . . . . . . . 7 4.1.2. Server Cookie Handling ..............................8
4.1.3. Client Cookie Handling . . . . . . . . . . . . . . . . 8 4.1.3. Client Cookie Handling ..............................9
4.1.3.1 Client Caching Negative Responses . . . . . . . . . 9 4.1.3.1. Client Caching Negative Responses .........10
4.2. Fast Open Protocol . . . . . . . . . . . . . . . . . . . . 9 4.2. Fast Open Protocol ........................................11
4.2.1. Fast Open Cookie Request . . . . . . . . . . . . . . . 10 4.2.1. Fast Open Cookie Request ...........................11
4.2.2. TCP Fast Open . . . . . . . . . . . . . . . . . . . . . 11 4.2.2. TCP Fast Open ......................................12
5. Security Considerations . . . . . . . . . . . . . . . . . . . . 13 5. Security Considerations ........................................14
5.1. Resource Exhaustion Attack by SYN Flood with Valid 5.1. Resource Exhaustion Attack by SYN Flood with Valid
Cookies . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Cookies ...................................................14
5.1.1 Attacks from behind Shared Public IPs (NATs) . . . . . . 14 5.1.1. Attacks from behind Shared Public IPs (NATs) .......15
5.2. Amplified Reflection Attack to Random Host . . . . . . . . 14 5.2. Amplified Reflection Attack to Random Host ................16
6. TFO's Applicability . . . . . . . . . . . . . . . . . . . . . . 15 6. TFO Applicability ..............................................17
6.1 Duplicate Data in SYNs . . . . . . . . . . . . . . . . . . . 15 6.1. Duplicate Data in SYNs ....................................17
6.2 Potential Performance Improvement . . . . . . . . . . . . . 16 6.2. Potential Performance Improvement .........................17
6.3. Example: Web Clients and Servers . . . . . . . . . . . . . 16 6.3. Example: Web Clients and Servers ..........................18
6.3.1. HTTP Request Replay . . . . . . . . . . . . . . . . . . 16 6.3.1. HTTP Request Replay ................................18
6.3.2. HTTP over TLS (HTTPS) . . . . . . . . . . . . . . . . . 16 6.3.2. HTTP over TLS (HTTPS) ..............................18
6.3.3. Comparison with HTTP Persistent Connections . . . . . . 17 6.3.3. Comparison with HTTP Persistent Connections ........18
6.3.4. Load Balancers and Server farms . . . . . . . . . . . . 17 6.3.4. Load Balancers and Server Farms ....................19
7. Open Areas for Experimentation . . . . . . . . . . . . . . . . 17
7.1. Performance impact due to middle-boxes and NAT . . . . . . 18
7.2. Impact on congestion control . . . . . . . . . . . . . . . 18
7.3. Cookie-less Fast Open . . . . . . . . . . . . . . . . . . . 18
8. Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 19
8.1. T/TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
8.2. Common Defenses Against SYN Flood Attacks . . . . . . . . . 19
8.3. Speculative Connections by the Applications . . . . . . . . 19
8.4. Fast Open Cookie in FIN . . . . . . . . . . . . . . . . . . 19
8.5. TCP Cookie Transaction (TCPCT) . . . . . . . . . . . . . . 20
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 20
10. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 20
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 20
11.1. Normative References . . . . . . . . . . . . . . . . . . . 20
11.2. Informative References . . . . . . . . . . . . . . . . . . 21
Appendix A. Example Socket API Changes to support TFO . . . . . . 22
A.1 Active Open . . . . . . . . . . . . . . . . . . . . . . . . 22
A.2 Passive Open . . . . . . . . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 23
1. Introduction 7. Open Areas for Experimentation .................................19
7.1. Performance Impact Due to Middleboxes and NAT .............19
7.2. Impact on Congestion Control ..............................20
7.3. Cookie-less Fast Open .....................................20
8. Related Work ...................................................20
8.1. T/TCP .....................................................20
8.2. Common Defenses against SYN Flood Attacks .................21
8.3. Speculative Connections by the Applications ...............21
8.4. Fast Open Cookie-in-FIN ...................................21
8.5. TCP Cookie Transaction (TCPCT) ............................21
9. IANA Considerations ............................................22
10. References ....................................................22
10.1. Normative References .....................................22
10.2. Informative References ...................................23
Appendix A. Example Socket API Changes to Support TFO .............25
A.1. Active Open .................................................25
A.2. Passive Open ................................................25
Acknowledgments ...................................................26
Authors' Addresses ................................................26
1. Introduction
TCP Fast Open (TFO) is an experimental update to TCP that enables TCP Fast Open (TFO) is an experimental update to TCP that enables
data to be exchanged safely during TCP's connection handshake. This data to be exchanged safely during TCP's connection handshake. This
document describes a design that enables applications to save a round document describes a design that enables applications to save a round
trip while avoiding severe security ramifications. At the core of TFO trip while avoiding severe security ramifications. At the core of
is a security cookie used by the server side to authenticate a client TFO is a security cookie used by the server side to authenticate a
initiating a TFO connection. This document covers the details of client initiating a TFO connection. This document covers the details
exchanging data during TCP's initial handshake, the protocol for TFO of exchanging data during TCP's initial handshake, the protocol for
cookies, potential new security vulnerabilities and their mitigation, TFO cookies, potential new security vulnerabilities and their
and the new socket API. mitigation, and the new socket API.
TFO is motivated by the performance needs of today's Web TFO is motivated by the performance needs of today's Web
applications. Current TCP only permits data exchange after the 3-way applications. Current TCP only permits data exchange after the
handshake (3WHS)[RFC793], which adds one RTT to network latency. For three-way handshake (3WHS) [RFC793], which adds one RTT to network
short Web transfers this additional RTT is a significant portion of latency. For short Web transfers this additional RTT is a
overall network latency, even when HTTP persistent connection is significant portion of overall network latency, even when HTTP
widely used. For example, the Chrome browser [Chrome] keeps TCP persistent connection is widely used. For example, the Chrome
connections idle for up to 5 minutes but 35% of HTTP requests are browser [Chrome] keeps TCP connections idle for up to 5 minutes, but
made on new TCP connections [RCCJR11]. For such Web and Web-like 35% of HTTP requests are made on new TCP connections [RCCJR11]. For
applications placing data in the SYN can yield significant latency such Web and Web-like applications, placing data in the SYN can yield
improvements. Next we describe how we resolve the challenges that significant latency improvements. Next we describe how we resolve
arise upon doing so. the challenges that arise upon doing so.
1.1. Terminology 1.1. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
TFO refers to TCP Fast Open. Client refers to the TCP's active open "TFO" refers to TCP Fast Open. "Client" refers to TCP's active open
side and server refers to the TCP's passive open side. side, and "server" refers to TCP's passive open side.
2. Data in SYN
2. Data In SYN
Standard TCP already allows data to be carried in SYN packets Standard TCP already allows data to be carried in SYN packets
([RFC793], section 3.4) but forbids the receiver from delivering it ([RFC793], Section 3.4) but forbids the receiver from delivering it
to the application until 3WHS is completed. This is because TCP's to the application until the 3WHS is completed. This is because
initial handshake serves to capture old or duplicate SYNs. TCP's initial handshake serves to capture old or duplicate SYNs.
To enable applications exchange data in TCP handshake, TFO removes To enable applications to exchange data in a TCP handshake, TFO
the constraint and allows data in SYN packets to be delivered to the removes the constraint and allows data in SYN packets to be delivered
application. This change of TCP semantic raises two issues discussed to the application. This change to TCP semantic raises two issues
in the following subsections, making TFO unsuitable for certain (discussed in the following subsections) that make TFO unsuitable for
applications. certain applications.
Therefore TCP implementations MUST NOT use TFO by default, but only Therefore, TCP implementations MUST NOT use TFO by default, but only
use TFO if requested explicitly by the application on a per service use TFO if requested explicitly by the application on a per-service-
port basis. Applications need to evaluate TFO applicability described port basis. Applications need to evaluate TFO applicability as
in Section 6 before using TFO. described in Section 6 before using TFO.
2.1 Relaxing TCP Semantics on Duplicated SYNs 2.1. Relaxing TCP Semantics on Duplicated SYNs
TFO allows data to be delivered to the application before the 3WHS TFO allows data to be delivered to the application before the 3WHS is
is completed, thus opening itself to a data integrity issue in either completed, thus opening itself to a data integrity issue in either of
of the two cases below: the two cases below:
a) the receiver host receives data in a duplicate SYN after it has a) the receiver host receives data in a duplicate SYN after it has
forgotten it received the original SYN (e.g. due to a reboot); forgotten it received the original SYN (e.g., due to a reboot);
b) the duplicate is received after the connection created by the b) the duplicate is received after the connection created by the
original SYN has been closed and the close was initiated by the original SYN has been closed and the close was initiated by the
sender (so the receiver will not be protected by the 2MSL TIMEWAIT sender (so the receiver will not be protected by the TIME-WAIT 2
state). MSL state).
The now obsoleted T/TCP [RFC1644] attempted to address these issues. The now obsoleted T/TCP [RFC1644] (obsoleted by [RFC6247]) attempted
It was not successful and not deployed due to various vulnerabilities to address these issues. It was not successful and not deployed due
as described in the Related Work section. Rather than trying to to various vulnerabilities as described in Section 8, "Related Work".
capture all dubious SYN packets to make TFO 100% compatible with TCP Rather than trying to capture all dubious SYN packets to make TFO
semantics, we made a design decision early on to accept old SYN 100% compatible with TCP semantics, we made a design decision early
packets with data, i.e., to restrict TFO use to a class of on to accept old SYN packets with data, i.e., to restrict TFO use to
applications (Section 6) that are tolerant of duplicate SYN packets a class of applications (Section 6) that are tolerant of duplicate
with data. We believe this is the right design trade-off balancing SYN packets with data. We believe this is the right design trade-
complexity with usefulness. off: balancing complexity with usefulness.
2.2. SYNs with Spoofed IP Addresses 2.2. SYNs with Spoofed IP Addresses
Standard TCP suffers from the SYN flood attack [RFC4987] because SYN Standard TCP suffers from the SYN flood attack [RFC4987] because SYN
packets with spoofed source IP addresses can easily fill up a packets with spoofed source IP addresses can easily fill up a
listener's small queue, causing a service port to be blocked listener's small queue, causing a service port to be blocked
completely until timeouts. completely.
TFO goes one step further to allow server-side TCP to send up data to TFO goes one step further to allow server-side TCP to send up data to
the application layer before 3WHS is completed. This opens up serious the application layer before the 3WHS is completed. This opens up
new vulnerabilities. Applications serving ports that have TFO enabled serious new vulnerabilities. Applications serving ports that have
may waste lots of CPU and memory resources processing the requests TFO enabled may waste lots of CPU and memory resources processing the
and producing the responses. If the response is much larger than the requests and producing the responses. If the response is much larger
request, the attacker can further mount an amplified reflection than the request, the attacker can further mount an amplified
attack against victims of choice beyond the TFO server itself. reflection attack against victims of choice beyond the TFO server
itself.
Numerous mitigation techniques against regular SYN flood attacks Numerous mitigation techniques against regular SYN flood attacks
exist and have been well documented [RFC4987]. Unfortunately none are exist and have been well documented [RFC4987]. Unfortunately, none
applicable to TFO. We propose a server-supplied cookie to mitigate are applicable to TFO. We propose a server-supplied cookie to
these new vulnerabilities in Section 3 and evaluate the effectiveness mitigate these new vulnerabilities in Section 3 and evaluate the
of the defense in Section 7. effectiveness of the defense in Section 7.
3. Protocol Overview 3. Protocol Overview
The key component of TFO is the Fast Open Cookie (cookie), a message The key component of TFO is the Fast Open Cookie (cookie), a message
authentication code (MAC) tag generated by the server. The client authentication code (MAC) tag generated by the server. The client
requests a cookie in one regular TCP connection, then uses it for requests a cookie in one regular TCP connection, then uses it for
future TCP connections to exchange data during 3WHS: future TCP connections to exchange data during the 3WHS:
Requesting a Fast Open Cookie: Requesting a Fast Open Cookie:
1. The client sends a SYN with a Fast Open option with an empty 1. The client sends a SYN with a Fast Open option with an empty
cookie field to request a cookie. cookie field to request a cookie.
2. The server generates a cookie and sends it through the Fast Open 2. The server generates a cookie and sends it through the Fast Open
option of a SYN-ACK packet. option of a SYN-ACK packet.
3. The client caches the cookie for future TCP Fast Open connections 3. The client caches the cookie for future TCP Fast Open connections
skipping to change at page 5, line 41 skipping to change at page 6, line 11
3. The client caches the cookie for future TCP Fast Open connections 3. The client caches the cookie for future TCP Fast Open connections
(see below). (see below).
Performing TCP Fast Open: Performing TCP Fast Open:
1. The client sends a SYN with data and the cookie in the Fast Open 1. The client sends a SYN with data and the cookie in the Fast Open
option. option.
2. The server validates the cookie: 2. The server validates the cookie:
a. If the cookie is valid, the server sends a SYN-ACK a. If the cookie is valid, the server sends a SYN-ACK
acknowledging both the SYN and the data. The server then acknowledging both the SYN and the data. The server then
delivers the data to the application. delivers the data to the application.
b. Otherwise, the server drops the data and sends a SYN-ACK b. Otherwise, the server drops the data and sends a SYN-ACK
acknowledging only the SYN sequence number. acknowledging only the SYN sequence number.
3. If the server accepts the data in the SYN packet, it may send the 3. If the server accepts the data in the SYN packet, it may send the
response data before the handshake finishes. The maximum amount is response data before the handshake finishes. The maximum amount
governed by the TCP's congestion control [RFC5681]. is governed by TCP's congestion control [RFC5681].
4. The client sends an ACK acknowledging the SYN and the server data. 4. The client sends an ACK acknowledging the SYN and the server data.
If the client's data is not acknowledged, the client retransmits If the client's data is not acknowledged, the client retransmits
the data in the ACK packet. the data in the ACK packet.
5. The rest of the connection proceeds like a normal TCP connection. 5. The rest of the connection proceeds like a normal TCP connection.
The client can repeat many Fast Open operations once it acquires a The client can repeat many Fast Open operations once it acquires a
cookie (until the cookie is expired by the server). Thus TFO is cookie (until the cookie is expired by the server). Thus, TFO is
useful for applications that have temporal locality on client and useful for applications that have temporal locality on client and
server connections. server connections.
Requesting Fast Open Cookie in connection 1: Requesting Fast Open Cookie in connection 1:
TCP A (Client) TCP B(Server) TCP A (Client) TCP B (Server)
______________ _____________ ______________ ______________
CLOSED LISTEN CLOSED LISTEN
#1 SYN-SENT ----- <SYN,CookieOpt=NIL> ----------> SYN-RCVD #1 SYN-SENT ----- <SYN,CookieOpt=NIL> ----------> SYN-RCVD
#2 ESTABLISHED <---- <SYN,ACK,CookieOpt=C> ---------- SYN-RCVD #2 ESTABLISHED <---- <SYN,ACK,CookieOpt=C> ---------- SYN-RCVD
(caches cookie C) (caches cookie C)
Performing TCP Fast Open in connection 2: Performing TCP Fast Open in connection 2:
TCP A (Client) TCP B(Server) TCP A (Client) TCP B (Server)
______________ _____________ ______________ ______________
CLOSED LISTEN CLOSED LISTEN
#1 SYN-SENT ----- <SYN=x,CookieOpt=C,DATA_A> ----> SYN-RCVD #1 SYN-SENT ----- <SYN=x,CookieOpt=C,DATA_A> ----> SYN-RCVD
#2 ESTABLISHED <---- <SYN=y,ACK=x+len(DATA_A)+1> ---- SYN-RCVD #2 ESTABLISHED <---- <SYN=y,ACK=x+len(DATA_A)+1> ---- SYN-RCVD
#3 ESTABLISHED <---- <ACK=x+len(DATA_A)+1,DATA_B>---- SYN-RCVD #3 ESTABLISHED <---- <ACK=x+len(DATA_A)+1,DATA_B>---- SYN-RCVD
#4 ESTABLISHED ----- <ACK=y+1>--------------------> ESTABLISHED #4 ESTABLISHED ----- <ACK=y+1>--------------------> ESTABLISHED
#5 ESTABLISHED --- <ACK=y+len(DATA_B)+1>----------> ESTABLISHED #5 ESTABLISHED --- <ACK=y+len(DATA_B)+1>----------> ESTABLISHED
4. Protocol Details 4. Protocol Details
4.1. Fast Open Cookie 4.1. Fast Open Cookie
The Fast Open Cookie is designed to mitigate new security The Fast Open Cookie is designed to mitigate new security
vulnerabilities in order to enable data exchange during handshake. vulnerabilities in order to enable data exchange during a handshake.
The cookie is a message authentication code tag generated by the The cookie is a MAC tag generated by the server and is opaque to the
server and is opaque to the client; the client simply caches the client; the client simply caches the cookie and passes it back on
cookie and passes it back on subsequent SYN packets to open new subsequent SYN packets to open new connections. The server can
connections. The server can expire the cookie at any time to enhance expire the cookie at any time to enhance security.
security.
4.1.1. Fast Open option 4.1.1. Fast Open Option
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Kind | Length | | Kind | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | | |
~ Cookie ~ ~ Cookie ~
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Kind 1 byte: constant-TBD (to be assigned by IANA) Kind 1 byte: value = 34
Length 1 byte: range 6 to 18 (bytes); limited by Length 1 byte: range 6 to 18 (bytes); limited by
remaining space in the options field. remaining space in the options field.
The number MUST be even. The number MUST be even.
Cookie 0, or 4 to 16 bytes (Length - 2) Cookie 0, or 4 to 16 bytes (Length - 2)
The Fast Open option is used to request or to send a Fast Open The Fast Open option is used to request or to send a Fast Open
Cookie. When cookie is not present or empty, the option is used by Cookie. When a cookie is not present or is empty, the option is used
the client to request a cookie from the server. When the cookie is by the client to request a cookie from the server. When the cookie
present, the option is used to pass the cookie from the server to the is present, the option is used to pass the cookie from the server to
client or from the client back to the server (to perform a Fast the client or from the client back to the server (to perform a Fast
Open). Open).
The minimum Cookie size is 4 bytes. Although the diagram shows a The minimum cookie size is 4 bytes. Although the diagram shows a
cookie aligned on 32-bit boundaries, alignment is not required. cookie aligned on 32-bit boundaries, alignment is not required.
Options with invalid Length values or without SYN flag set MUST be Options with invalid Length values or without the SYN flag set MUST
ignored. be ignored.
4.1.2. Server Cookie Handling 4.1.2. Server Cookie Handling
The server is in charge of cookie generation and authentication. The The server is in charge of cookie generation and authentication. The
cookie SHOULD be a message authentication code tag with the following cookie SHOULD be a MAC tag with the following properties. We use
properties. We use SHOULD because in some cases the cookie may be "SHOULD" because, in some cases, the cookie may be trivially
trivially generated as discussed in Section 7.3. generated as discussed in Section 7.3.
1. The cookie authenticates the client's (source) IP address of the 1. The cookie authenticates the client's (source) IP address of the
SYN packet. The IP address may be an IPv4 or IPv6 address. SYN packet. The IP address may be an IPv4 or IPv6 address.
2. The cookie can only be generated by the server and can not be 2. The cookie can only be generated by the server and cannot be
fabricated by any other parties including the client. fabricated by any other parties, including the client.
3. The generation and verification are fast relative to the rest of 3. The generation and verification are fast relative to the rest of
SYN and SYN-ACK processing. SYN and SYN-ACK processing.
4. A server may encode other information in the cookie, and accept 4. A server may encode other information in the cookie and accept
more than one valid cookie per client at any given time. But this more than one valid cookie per client at any given time. But this
is server implementation dependent and transparent to the is server-implementation dependent and transparent to the client.
client.
5. The cookie expires after a certain amount of time. The reason for 5. The cookie expires after a certain amount of time. The reason for
cookie expiration is detailed in the "Security Consideration" cookie expiration is detailed in the "Security Considerations"
section. This can be done by either periodically changing the section (Section 5). This can be done by either periodically
server key used to generate cookies or including a timestamp when changing the server key used to generate cookies or including a
generating the cookie. timestamp when generating the cookie.
To gradually invalidate cookies over time, the server can To gradually invalidate cookies over time, the server can
implement key rotation to generate and verify cookies using implement key rotation to generate and verify cookies using
multiple keys. This approach is useful for large-scale servers to multiple keys. This approach is useful for large-scale servers to
retain Fast Open rolling key updates. We do not specify a retain Fast Open rolling key updates. We do not specify a
particular mechanism because the implementation is server particular mechanism because the implementation is server
specific. specific.
The server supports the cookie generation and verification The server supports the cookie-generation and verification
operations: operations:
- GetCookie(IP_Address): returns a (new) cookie - GetCookie(IP_Address): returns a (new) cookie.
- IsCookieValid(IP_Address, Cookie): checks if the cookie is valid, - IsCookieValid(IP_Address, Cookie): checks if the cookie is valid,
i.e., it has not expired and it authenticates the client IP address. i.e., it has not expired and the cookie authenticates the client
IP address.
Example Implementation: a simple implementation is to use AES_128 to Example Implementation: a simple implementation is to use AES_128 to
encrypt the IPv4 (with padding) or IPv6 address and truncate to 64 encrypt the IPv4 (with padding) or IPv6 address and truncate to 64
bits. The server can periodically update the key to expire the bits. The server can periodically update the key to expire the
cookies. AES encryption on recent processors is fast and takes only a cookies. AES encryption on recent processors is fast and takes only
few hundred nanoseconds [RCCJR11]. a few hundred nanoseconds [RCCJR11].
If only one valid cookie is allowed per-IP and the server can If only one valid cookie is allowed per IP, and the server can
regenerate the cookie independently, the best validation process is regenerate the cookie independently, the best validation process is
to simply regenerate a valid cookie and compare it against the to simply regenerate a valid cookie and compare it against the
incoming cookie. In that case if the incoming cookie fails the check, incoming cookie. In that case, if the incoming cookie fails the
a valid cookie is readily available to be sent to the client. check, a valid cookie is readily available to be sent to the client.
4.1.3. Client Cookie Handling 4.1.3. Client Cookie Handling
The client MUST cache cookies from servers for later Fast Open The client MUST cache cookies from servers for later Fast Open
connections. For a multi-homed client, the cookies are dependent on connections. For a multihomed client, the cookies are dependent on
the client and server IP addresses. Hence the client should cache at the client and server IP addresses. Hence, the client should cache
most one (most recently received) cookie per client and server IP at most one (most recently received) cookie per client and server IP
addresses pair. address pair.
When caching cookies, we recommend that the client also cache the When caching cookies, we recommend that the client also cache the
Maximum Segment Size (MSS) advertised by the server. The client can Maximum Segment Size (MSS) advertised by the server. The client can
cache the MSS advertised by the server in order to determine the cache the MSS advertised by the server in order to determine the
maximum amount of data that the client can fit in the SYN packet in maximum amount of data that the client can fit in the SYN packet in
subsequent TFO connections. Caching the server MSS is useful because subsequent TFO connections. Caching the server MSS is useful
with Fast Open a client sends data in the SYN packet before the because, with Fast Open, a client sends data in the SYN packet before
server announces its MSS in the SYN-ACK packet. If the client sends the server announces its MSS in the SYN-ACK packet. If the client
more data in the SYN packet than the server will accept, this will sends more data in the SYN packet than the server will accept, this
likely require the client to retransmit some or all of the data. will likely require the client to retransmit some or all of the data.
Hence caching the server MSS can enhance performance. Hence, caching the server MSS can enhance performance.
Without a cached server MSS, the amount of data in the SYN packet is Without a cached server MSS, the amount of data in the SYN packet is
limited to the default MSS of 536 bytes for IPv4 [RFC1122] and 1240 limited to the default MSS of 536 bytes for IPv4 [RFC1122] and 1220
bytes for IPv6 [RFC2460]. Even if the client complies with this limit bytes for IPv6 [RFC2460]. Even if the client complies with this
when sending the SYN, it is known that an IPv4 receiver advertising limit when sending the SYN, it is known that an IPv4 receiver
an MSS less than 536 bytes can receive a segment larger than it is advertising an MSS less than 536 bytes can receive a segment larger
expecting. than it is expecting.
If the cached MSS is larger than the typical size (1460 bytes for If the cached MSS is larger than the typical size (1460 bytes for
IPv4, or 1440 bytes for IPv6), then the excess data in the SYN packet IPv4 or 1440 bytes for IPv6), then the excess data in the SYN packet
may cause problems that offset the performance benefit of Fast Open. may cause problems that offset the performance benefit of Fast Open.
For example, the unusually large SYN may trigger IP fragmentation and For example, the unusually large SYN may trigger IP fragmentation and
may confuse firewalls or middleboxes, causing SYN retransmission and may confuse firewalls or middleboxes, causing SYN retransmission and
other side effects. Therefore the client MAY limit the cached MSS to other side effects. Therefore, the client MAY limit the cached MSS
1460 bytes for IPv4 or 1440 for IPv6. to 1460 bytes for IPv4 or 1440 for IPv6.
4.1.3.1 Client Caching Negative Responses 4.1.3.1. Client Caching Negative Responses
The client MUST cache negative responses from the server in order to The client MUST cache negative responses from the server in order to
avoid potential connection failures. Negative responses include avoid potential connection failures. Negative responses include the
server not acknowledging the data in SYN, ICMP error messages, and server not acknowledging the data in the SYN, ICMP error messages,
most importantly no response (SYN/ACK) from the server at all, i.e., and (most importantly) no response (SYN-ACK) from the server at all,
connection timeout. The last case is likely due to incompatible i.e., connection timeout. The last case is likely due to
middle-boxes or firewall blocking the connection completely after it incompatible middleboxes or firewall blocking the connection
sees data in SYN. If the client does not react to these negative completely after processing the SYN packet with data. If the client
responses and continue to retry Fast Open, the client may never be does not react to these negative responses and continues to retry
able to connect to the specific server. Fast Open, the client may never be able to connect to the specific
server.
For any negative responses, the client SHOULD disable Fast Open on For any negative responses, the client SHOULD disable Fast Open on
the specific path (the source and destination IP addresses and ports) the specific path (the source and destination IP addresses and ports)
at least temporarily. Since TFO is enabled on a per-service port at least temporarily. Since TFO is enabled on a per-service-port
basis but cookies are independent of service ports, the client's basis, but cookies are independent of service ports, the client's
cache should include remote port numbers too. cache should include remote port numbers, too.
4.2. Fast Open Protocol 4.2. Fast Open Protocol
One predominant requirement of TFO is to be fully compatible with One predominant requirement of TFO is to be fully compatible with
existing TCP implementations, both on the client and the server existing TCP implementations, on both the client and server sides.
sides.
The server keeps two variables per listening socket (IP address & The server keeps two variables per listening socket (IP address and
port): port):
FastOpenEnabled: default is off. It MUST be turned on explicitly by FastOpenEnabled: default is off. It MUST be turned on explicitly by
the application. When this flag is off, the server does not perform the application. When this flag is off, the server does not
any TFO related operations and MUST ignore all cookie options. perform any TFO-related operations and MUST ignore all cookie
options.
PendingFastOpenRequests: tracks number of TFO connections in SYN-RCVD PendingFastOpenRequests: tracks the number of TFO connections in SYN-
state. If this variable goes over a preset system limit, the server RCVD state. If this variable goes over a preset system limit, the
MUST disable TFO for all new connection requests until server MUST disable TFO for all new connection requests until
PendingFastOpenRequests drops below the system limit. This variable PendingFastOpenRequests drops below the system limit. This
is used for defending some vulnerabilities discussed in the "Security variable is used for defending some vulnerabilities discussed in
Considerations" section. the "Security Considerations" section (Section 5).
The server keeps a FastOpened flag per connection to mark if a The server keeps a FastOpened flag per connection to mark if a
connection has successfully performed a TFO. connection has successfully performed a TFO.
4.2.1. Fast Open Cookie Request 4.2.1. Fast Open Cookie Request
Any client attempting TFO MUST first request a cookie from the server Any client attempting TFO MUST first request a cookie from the server
with the following steps: with the following steps:
1. The client sends a SYN packet with a Fast Open option with a 1. The client sends a SYN packet with a Fast Open option with a
length field of 0 (empty cookie field). Length field of 0 (empty cookie field).
2. The server responds with a SYN-ACK based on the procedures 2. The server responds with a SYN-ACK based on the procedures in the
in the "Server Cookie Handling" section. This SYN-ACK may "Server Cookie Handling" section (Section 4.1.2). This SYN-ACK
contain a Fast Open option if the server currently supports may contain a Fast Open option if the server currently supports
TFO for this listener port. TFO for this listener port.
3. If the SYN-ACK has a Fast Open option with a cookie, the client 3. If the SYN-ACK has a Fast Open option with a cookie, the client
replaces the cookie and other information as described in the replaces the cookie and other information as described in the
"Client Cookie Handling" section. Otherwise, if the SYN-ACK is "Client Cookie Handling" section (Section 4.1.3). Otherwise, if
first seen, i.e., not a (spurious) retransmission, the client MAY the SYN-ACK is first seen and not a (spurious) retransmission, the
remove the server information from the cookie cache. If the client MAY remove the server information from the cookie cache.
SYN-ACK is a spurious retransmission, the client does nothing to If the SYN-ACK is a spurious retransmission, the client does
the cookie cache for the reasons below. nothing to the cookie cache for the reasons below.
The network or servers may drop the SYN or SYN-ACK packets with the The network or servers may drop the SYN or SYN-ACK packets with the
new cookie options, which will cause SYN or SYN-ACK timeouts. We new cookie options, which will cause SYN or SYN-ACK timeouts. We
RECOMMEND both the client and the server to retransmit SYN and SYN- RECOMMEND both the client and the server to retransmit SYN and SYN-
ACK without the cookie options on timeouts. This ensures the ACK packets without the cookie options on timeouts. This ensures the
connections of cookie requests will go through and lowers the latency connections of cookie requests will go through and lowers the latency
penalty (of dropped SYN/SYN-ACK packets). The obvious downside for penalty (of dropped SYN/SYN-ACK packets). The obvious downside for
maximum compatibility is that any regular SYN drop will fail the maximum compatibility is that any regular SYN drop will fail the
cookie (although one can argue the delay in the data transmission cookie (although one can argue the delay in the data transmission
till after 3WHS is justified if the SYN drop is due to network until after the 3WHS is justified if the SYN drop is due to network
congestion). The next section describes a heuristic to detect such congestion). The next section describes a heuristic to detect such
drops when the client receives the SYN-ACK. drops when the client receives the SYN-ACK.
We also RECOMMEND the client to record the set of servers that failed We also RECOMMEND the client to record the set of servers that failed
to respond to cookie requests and only attempt another cookie request to respond to cookie requests and only attempt another cookie request
after certain period. after a certain period.
4.2.2. TCP Fast Open 4.2.2. TCP Fast Open
Once the client obtains the cookie from the target server, it can Once the client obtains the cookie from the target server, it can
perform subsequent TFO connections until the cookie is expired by the perform subsequent TFO connections until the cookie is expired by the
server. server.
Client: Sending SYN Client: Sending SYN
To open a TFO connection, the client MUST have obtained a cookie from To open a TFO connection, the client MUST have obtained a cookie from
the server: the server:
1. Send a SYN packet. 1. Send a SYN packet.
a. If the SYN packet does not have enough option space for the a. If the SYN packet does not have enough option space for the
Fast Open option, abort TFO and fall back to regular 3WHS. Fast Open option, abort TFO and fall back to the regular 3WHS.
b. Otherwise, include the Fast Open option with the cookie b. Otherwise, include the Fast Open option with the cookie of the
of the server. Include any data up to the cached server MSS or server. Include any data up to the cached server MSS or
default 536 bytes. default 536 bytes.
2. Advance to SYN-SENT state and update SND.NXT to include the data 2. Advance to SYN-SENT state and update SND.NXT to include the data
accordingly. accordingly.
To deal with network or servers dropping SYN packets with payload or To deal with network or servers dropping SYN packets with payload or
unknown options, when the SYN timer fires, the client SHOULD unknown options, when the SYN timer fires, the client SHOULD
retransmit a SYN packet without data and Fast Open options. retransmit a SYN packet without data and Fast Open options.
Server: Receiving SYN and responding with SYN-ACK Server: Receiving SYN and responding with SYN-ACK
Upon receiving the SYN packet with Fast Open option: Upon receiving the SYN packet with Fast Open option:
1. Initialize and reset a local FastOpened flag. If FastOpenEnabled 1. Initialize and reset a local FastOpened flag. If FastOpenEnabled
is false, go to step 5. is false, go to step 5.
2. If PendingFastOpenRequests is over the system limit, go to step 5. 2. If PendingFastOpenRequests is over the system limit, go to step 5.
3. If IsCookieValid() in section 4.1.2 returns false, go to step 5. 3. If IsCookieValid() (in Section 4.1.2) returns false, go to step 5.
4. Buffer the data and notify the application. Set FastOpened flag 4. Buffer the data and notify the application. Set the FastOpened
and increment PendingFastOpenRequests. flag and increment PendingFastOpenRequests.
5. Send the SYN-ACK packet. The packet MAY include a Fast Open 5. Send the SYN-ACK packet. The packet MAY include a Fast Open
Option. If FastOpened flag is set, the packet acknowledges the SYN option. If the FastOpened flag is set, the packet acknowledges
and data sequence. Otherwise it acknowledges only the SYN the SYN and data sequence. Otherwise, it acknowledges only the
sequence. The server MAY include data in the SYN-ACK packet if the SYN sequence. The server MAY include data in the SYN-ACK packet
response data is readily available. Some application may favor if the response data is readily available. Some applications may
delaying the SYN-ACK, allowing the application to process the favor delaying the SYN-ACK, allowing the application to process
request in order to produce a response, but this is left up to the the request in order to produce a response, but this is left up to
implementation. the implementation.
6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the 6. Advance to the SYN-RCVD state. If the FastOpened flag is set, the
server MUST follow [RFC5681] (based on [RFC3390]) to set the server MUST follow [RFC5681] (based on [RFC3390]) to set the
initial congestion window for sending more data packets. initial congestion window for sending more data packets.
If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK If the SYN-ACK timer fires, the server SHOULD retransmit a SYN-ACK
segment with neither data nor Fast Open options for compatibility segment with neither data nor Fast Open options for compatibility
reasons. reasons.
A special case is simultaneous open where the SYN receiver is a A special case is simultaneous open where the SYN receiver is a
client in SYN-SENT state. The protocol remains the same because client in SYN-SENT state. The protocol remains the same because
[RFC793] already supports both data in SYN and simultaneous open. But [RFC793] already supports both data in the SYN and simultaneous open.
the client's socket may have data available to read before it's But the client's socket may have data available to read before it's
connected. This document does not cover the corresponding API change. connected. This document does not cover the corresponding API
change.
Client: Receiving SYN-ACK Client: Receiving SYN-ACK
The client SHOULD perform the following steps upon receiving the SYN- The client SHOULD perform the following steps upon receiving the SYN-
ACK: ACK:
1. If the SYN-ACK has a Fast Open option or MSS option or 1. If the SYN-ACK has a Fast Open option, an MSS option, or both,
both, update the corresponding cookie and MSS information in the update the corresponding cookie and MSS information in the cookie
cookie cache. cache.
2. Send an ACK packet. Set acknowledgment number to RCV.NXT and 2. Send an ACK packet. Set acknowledgment number to RCV.NXT and
include the data after SND.UNA if data is available. include the data after SND.UNA if data is available.
3. Advance to the ESTABLISHED state. 3. Advance to the ESTABLISHED state.
Note there is no latency penalty if the server does not acknowledge Note there is no latency penalty if the server does not acknowledge
the data in the original SYN packet. The client SHOULD retransmit any the data in the original SYN packet. The client SHOULD retransmit
unacknowledged data in the first ACK packet in step 2. The data any unacknowledged data in the first ACK packet in step 2. The data
exchange will start after the handshake like a regular TCP exchange will start after the handshake like a regular TCP
connection. connection.
If the client has timed out and retransmitted only regular SYN If the client has timed out and retransmitted only regular SYN
packets, it can heuristically detect paths that intentionally drop packets, it can heuristically detect paths that intentionally drop
SYN with Fast Open option or data. If the SYN-ACK acknowledges only SYNs with the Fast Open option or data. If the SYN-ACK acknowledges
the initial sequence and does not carry a Fast Open cookie option, only the initial sequence and does not carry a Fast Open cookie
presumably it is triggered by a retransmitted (regular) SYN and the option, presumably it is triggered by a retransmitted (regular) SYN
original SYN or the corresponding SYN-ACK was lost. and the original SYN or the corresponding SYN-ACK was lost.
Server: Receiving ACK Server: Receiving ACK
Upon receiving an ACK acknowledging the SYN sequence, the server Upon receiving an ACK acknowledging the SYN sequence, the server
decrements PendingFastOpenRequests and advances to the ESTABLISHED decrements PendingFastOpenRequests and advances to the ESTABLISHED
state. No special handling is required further. state. No special handling is required further.
5. Security Considerations 5. Security Considerations
The Fast Open cookie stops an attacker from trivially flooding The Fast Open Cookie stops an attacker from trivially flooding
spoofed SYN packets with data to burn server resources or to mount an spoofed SYN packets with data to burn server resources or to mount an
amplified reflection attack on random hosts. The server can defend amplified reflection attack on random hosts. The server can defend
against spoofed SYN floods with invalid cookies using existing against spoofed SYN floods with invalid cookies using existing
techniques [RFC4987]. We note that although generating bogus cookies techniques [RFC4987]. We note that although generating bogus cookies
is cost-free, the cost of validating the cookies, inherent to any is cost free, the cost of validating the cookies, inherent to any
authentication scheme, may be substantial compared to processing a authentication scheme, may be substantial compared to processing a
regular SYN packet. We describe these new vulnerabilities of TFO and regular SYN packet. We describe these new vulnerabilities of TFO and
the countermeasures in detail below. the countermeasures in detail below.
5.1. Resource Exhaustion Attack by SYN Flood with Valid Cookies 5.1. Resource Exhaustion Attack by SYN Flood with Valid Cookies
An attacker may still obtain cookies from some compromised hosts, An attacker may still obtain cookies from some compromised hosts,
then flood spoofed SYN with data and "valid" cookies (from these then flood spoofed SYN packets with data and "valid" cookies (from
hosts or other vantage points). Like regular TCP handshakes, TFO is these hosts or other vantage points). Like regular TCP handshakes,
vulnerable to such an attack. But the potential damage can be much TFO is vulnerable to such an attack. But the potential damage can be
more severe. Besides causing temporary disruption to service ports much more severe. Besides causing temporary disruption to service
under attack, it may exhaust server CPU and memory resources. Such an ports under attack, it may exhaust server CPU and memory resources.
attack will show up on application server logs as an application Such an attack will show up on application server logs as an
level DoS from Bot-nets, triggering other defenses and alerts. application-level DoS from botnets, triggering other defenses and
alerts.
To protect the server it is important to limit the maximum number of To protect the server, it is important to limit the maximum number of
total pending TFO connection requests, i.e., PendingFastOpenRequests total pending TFO connection requests, i.e., PendingFastOpenRequests
(Section 4.2). When the limit is exceeded, the server temporarily (Section 4.2). When the limit is exceeded, the server temporarily
disables TFO entirely as described in "Server Cookie Handling". Then disables TFO entirely as described in "Server Cookie Handling"
subsequent TFO requests will be downgraded to regular connection (Section 4.1.2). Then, subsequent TFO requests will be downgraded to
requests, i.e., with the data dropped and only SYN acknowledged. This regular connection requests, i.e., with the data dropped and only
allows regular SYN flood defense techniques [RFC4987] like SYN- SYNs acknowledged. This allows regular SYN flood defense techniques
cookies to kick in and prevent further service disruption. [RFC4987] like SYN cookies to kick in and prevent further service
disruption.
The main impact of SYN floods against the standard TCP stack is not The main impact of SYN floods against the standard TCP stack is not
directly from the floods themselves costing TCP processing overhead directly from the floods themselves costing TCP processing overhead
or host memory, but rather from the spoofed SYN packets filling up or host memory, but rather from the spoofed SYN packets filling up
the often small listener's queue. the often small listener's queue.
On the other hand, TFO SYN floods can cause damage directly if On the other hand, TFO SYN floods can cause damage directly if
admitted without limit into the stack. The RST packets from the admitted without limit into the stack. The reset (RST) packets from
spoofed host will fuel rather than defeat the SYN floods as compared the spoofed host will fuel rather than defeat the SYN floods as
to the non-TFO case, because the attacker can flood more SYNs with compared to the non-TFO case, because the attacker can flood more
data to cost more data processing resources. For this reason, a TFO SYNs with data and incur more cost in terms of data processing
server needs to monitor the connections in SYN-RCVD being reset in resources. For this reason, a TFO server needs to monitor the
addition to imposing a reasonable max queue length. Implementations connections in SYN-RCVD being reset in addition to imposing a
may combine the two, e.g., by continuing to account for those reasonable max queue length. Implementations may combine the two,
connection requests that have just been reset against the listener's e.g., by continuing to account for those connection requests that
PendingFastOpenRequests until a timeout period has passed. have just been reset against the listener's PendingFastOpenRequests
until a timeout period has passed.
Limiting the maximum number of pending TFO connection requests does Limiting the maximum number of pending TFO connection requests does
make it easy for an attacker to overflow the queue, causing TFO to be make it easy for an attacker to overflow the queue, causing TFO to be
disabled. We argue that causing TFO to be disabled is unlikely to be disabled. We argue that causing TFO to be disabled is unlikely to be
of interest to attackers because the service will remain intact of interest to attackers because the service will remain intact
without TFO hence there is hardly any real damage. without TFO; hence, there is hardly any real damage.
5.1.1 Attacks from behind Shared Public IPs (NATs) 5.1.1. Attacks from behind Shared Public IPs (NATs)
An attacker behind a NAT can easily obtain valid cookies to launch An attacker behind a NAT can easily obtain valid cookies to launch
the above attack to hurt other clients that share the path. the above attack to hurt other clients that share the path.
[BRISCOE12] suggested that the server can extend cookie generation to [BRISCOE12] suggested that the server can extend cookie generation to
include the TCP timestamp---GetCookie(IP_Address, Timestamp)---and include the TCP timestamp -- GetCookie(IP_Address, Timestamp) -- and
implement it by encrypting the concatenation of the two values to implement it by encrypting the concatenation of the two values to
generate the cookie. The client stores both the cookie and its generate the cookie. The client stores both the cookie and its
corresponding timestamp, and echoes both in the SYN. The server then corresponding timestamp, and it echoes both in the SYN. The server
implements IsCookieValid(IP_Address, Timestamp, Cookie) by encrypting then implements IsCookieValid(IP_Address, Timestamp, Cookie) by
the IP and timestamp data and comparing it with the cookie value. encrypting the IP and timestamp data and comparing it with the cookie
value.
This enables the server to issue different cookies to clients that This enables the server to issue different cookies to clients that
share the same IP address, hence can selectively discard those share the same IP address; hence, it can selectively discard those
misused cookies from the attacker. However the attacker can simply misused cookies from the attacker. However, the attacker can simply
repeat the attack with new cookies. The server would eventually need repeat the attack with new cookies. The server would eventually need
to throttle all requests from the IP address just like the current to throttle all requests from the IP address just like the current
approach. Moreover this approach requires modifying [RFC1323] to send approach. Moreover, this approach requires modifying [RFC1323]
non-zero Timestamp Echo Reply in SYN, potentially causing firewall (obsoleted by [RFC7323]) to send a non-zero Timestamp Echo Reply in
issues. Therefore we believe the benefit does not outweigh the the SYN, potentially causing firewall issues. Therefore, we believe
drawbacks. the benefit does not outweigh the drawbacks.
5.2. Amplified Reflection Attack to Random Host 5.2. Amplified Reflection Attack to Random Host
Limiting PendingFastOpenRequests with a system limit can be done Limiting PendingFastOpenRequests with a system limit can be done
without Fast Open Cookies and would protect the server from resource without Fast Open cookies and would protect the server from resource
exhaustion. It would also limit how much damage an attacker can cause exhaustion. It would also limit how much damage an attacker can
through an amplified reflection attack from that server. However, it cause through an amplified reflection attack from that server.
would still be vulnerable to an amplified reflection attack from a However, it would still be vulnerable to an amplified reflection
large number of servers. An attacker can easily cause damage by attack from a large number of servers. An attacker can easily cause
tricking many servers to respond with data packets at once to any damage by tricking many servers to respond with data packets at once
spoofed victim IP address of choice. to any spoofed victim IP address of choice.
With the use of Fast Open Cookies, the attacker would first have to With the use of Fast Open cookies, the attacker would first have to
steal a valid cookie from its target victim. This likely requires the steal a valid cookie from its target victim. This likely requires
attacker to compromise the victim host or network first. But in some the attacker to compromise the victim host or network first. But, in
cases it may be relatively easy. some cases, it may be relatively easy.
The attacker here has little interest in mounting an attack on the The attacker here has little interest in mounting an attack on the
victim host that has already been compromised. But it may be victim host that has already been compromised. But it may be
motivated to disrupt the victim's network. Since a stolen cookie is motivated to disrupt the victim's network. Since a stolen cookie is
only valid for a single server, it has to steal valid cookies from a only valid for a single server, it has to steal valid cookies from a
large number of servers and use them before they expire to cause large number of servers and use them before they expire to cause
sufficient damage without triggering the defense. sufficient damage without triggering the defense.
One can argue that if the attacker has compromised the target network One can argue that if the attacker has compromised the target network
or hosts, it could perform a similar but simpler attack by injecting or hosts, it could perform a similar but simpler attack by injecting
bits directly. The degree of damage will be identical, but TFO- bits directly. The degree of damage will be identical, but a TFO-
specific attack allows the attacker to remain anonymous and disguises specific attack allows the attacker to remain anonymous and disguises
the attack as from other servers. the attack as from other servers.
For example with DHCP an attacker can obtain cookies when he (or the For example, with DHCP, an attacker can obtain cookies when he (or
host he has compromised) owns a particular IP address by performing the host he has compromised) owns a particular IP address by
regular Fast Open to servers supporting TFO and collect valid performing regular Fast Open to servers supporting TFO and he can
cookies. The attacker then actively or passively releases his IP collect valid cookies. Then, the attacker actively or passively
address. When the IP address is re-assigned to a victim, the attacker releases his IP address. When the IP address is reassigned to
now owning a different IP address, floods spoofed Fast Open requests another host (victim) via DHCP, the attacker then floods spoofed Fast
to perform an amplified reflection attack on the victim. Open requests with valid cookies to the servers. Since the cookies
are valid, these servers accept the requests and respond with a SYN-
ACK plus data packets to the victim instead of the attacker. Thus,
the attacker is able to launch amplified reflection attacks to other
hosts that share IP addresses.
The best defense is for the server not to respond with data until The best defense is for the server not to respond with data until the
handshake finishes. In this case the risk of amplification reflection handshake finishes. In this case, the risk of an amplification
attack is completely eliminated. But the potential latency saving reflection attack is completely eliminated. But the potential
from TFO may diminish if the server application produces responses latency saving from TFO may diminish if the server application
earlier before the handshake completes. produces responses earlier before the handshake completes.
6. TFO's Applicability 6. TFO Applicability
This section is to help applications considering TFO to evaluate This section is to help applications considering TFO to evaluate
TFO's benefits and drawbacks using the Web client and server TFO's benefits and drawbacks using the Web client and server
applications as an example throughout. Applications here refer applications as an example throughout. Applications here refer
specifically to the process that writes data into the socket. For specifically to the process that writes data into the socket -- for
example a JavaScript process that sends data to the server. A example, a JavaScript process that sends data to the server. A
proposed socket API change is in the Appendix. proposed socket API change is provided in the Appendix.
6.1 Duplicate Data in SYNs 6.1. Duplicate Data in SYNs
It is possible that using TFO results in the first data written to a It is possible that using TFO results in the first data written to a
socket to be delivered more than once to the application on the socket to be delivered more than once to the application on the
remote host (Section 2.1). This replay potential only applies to data remote host (Section 2.1). This replay potential only applies to
in the SYN but not subsequent data exchanges. data in the SYN but not subsequent data exchanges.
Empirically [JIDKT07] showed the packet duplication on a Tier-1 Empirically, [JIDKT07] showed the packet duplication on a Tier-1
network is rare. Since the replay only happens specifically when the network is rare. Since the replay only happens specifically when the
SYN data packet is duplicated and also the duplicate arrives after SYN data packet is duplicated and also the duplicate arrives after
the receiver has cleared the original SYN's connection state, the the receiver has cleared the original SYN's connection state, the
replay is thought to be uncommon in practice. Nevertheless a client replay is thought to be uncommon in practice. Nevertheless, a client
that cannot handle receiving the same SYN data more than once MUST that cannot handle receiving the same SYN data more than once MUST
NOT enable TFO to send data in a SYN. Similarly a server that cannot NOT enable TFO to send data in a SYN. Similarly, a server that
accept receiving the same SYN data more than once MUST NOT enable TFO cannot accept receiving the same SYN data more than once MUST NOT
to receive data in a SYN. Further investigation is needed to judge enable TFO to receive data in a SYN. Further investigation is needed
about the probability of receiving duplicated SYN or SYN-ACK with to judge the probability of receiving duplicated SYN or SYN-ACK
data in non-Tier 1 networks. packets with data in networks that are not Tier 1.
6.2 Potential Performance Improvement 6.2. Potential Performance Improvement
TFO is designed for latency-conscious applications that are sensitive TFO is designed for latency-conscious applications that are sensitive
to TCP's initial connection setup delay. To benefit from TFO, the to TCP's initial connection setup delay. To benefit from TFO, the
first application data unit (e.g., an HTTP request) needs to be no first application data unit (e.g., an HTTP request) needs to be no
more than TCP's maximum segment size (minus options used in SYN). more than TCP's maximum segment size (minus options used in the SYN).
Otherwise the remote server can only process the client's application Otherwise, the remote server can only process the client's
data unit once the rest of it is delivered after the initial application data unit once the rest of it is delivered after the
handshake, diminishing TFO's benefit. initial handshake, diminishing TFO's benefit.
To the extent possible, applications SHOULD reuse the connection to To the extent possible, applications SHOULD reuse the connection to
take advantage of TCP's built-in congestion control and reduce take advantage of TCP's built-in congestion control and reduce
connection setup overhead. An application that employs too many connection setup overhead. An application that employs too many
short-lived connections will negatively impact network stability, as short-lived connections will negatively impact network stability, as
these connections often exit before TCP's congestion control these connections often exit before TCP's congestion control
algorithm takes effect. algorithm takes effect.
6.3. Example: Web Clients and Servers 6.3. Example: Web Clients and Servers
6.3.1. HTTP Request Replay 6.3.1. HTTP Request Replay
While TFO is motivated by Web applications, the browser should not While TFO is motivated by Web applications, the browser should not
use TFO to send requests in SYNs if those requests cannot tolerate use TFO to send requests in SYNs if those requests cannot tolerate
replays. One example is POST requests without application-layer replays. One example is POST requests without application-layer
transaction protection (e.g., a unique identifier in the request transaction protection (e.g., a unique identifier in the request
header). header).
On the other hand, TFO is particularly useful for GET requests. GET On the other hand, TFO is particularly useful for GET requests. GET
requests replay could happen across striped TCP connections: after a request replay could happen across striped TCP connections: after a
server receives an HTTP request but before the ACKs of the requests server receives an HTTP request but before the ACKs of the requests
reach the browser, the browser may timeout and retry the same request reach the browser, the browser may time out and retry the same
on another (possibly new) TCP connection. This differs from a TFO request on another (possibly new) TCP connection. This differs from
replay only in that the replay is initiated by the browser, not by a TFO replay only in that the replay is initiated by the browser, not
the TCP stack. by the TCP stack.
6.3.2. HTTP over TLS (HTTPS) 6.3.2. HTTP over TLS (HTTPS)
For TLS over TCP, it is safe and useful to include TLS CLIENT_HELLO
in the SYN packet to save one RTT in TLS handshake. There is no
concern about violating idem-potency. In particular it can be used
alone with the speculative connection above.
6.3.3. Comparison with HTTP Persistent Connections For Transport Layer Security (TLS) over TCP, it is safe and useful to
include a TLS client_hello in the SYN packet to save one RTT in the
TLS handshake. There is no concern about violating idempotency. In
particular, it can be used alone with the speculative connection
above.
6.3.3. Comparison with HTTP Persistent Connections
Is TFO useful given the wide deployment of HTTP persistent Is TFO useful given the wide deployment of HTTP persistent
connections? The short answer is yes. Studies [RCCJR11][AERG11] show connections? The short answer is yes. Studies ([RCCJR11] [AERG11])
that the average number of transactions per connection is between 2 show that the average number of transactions per connection is
and 4, based on large-scale measurements from both servers and between 2 and 4, based on large-scale measurements from both servers
clients. In these studies, the servers and clients both kept idle and clients. In these studies, the servers and clients both kept
connections up to several minutes, well into "human think" time. idle connections up to several minutes, well into "human think" time.
Keeping connections open and idle even longer risks a greater Keeping connections open and idle even longer risks a greater
performance penalty. [HNESSK10][MQXMZ11] show that the majority of performance penalty. [HNESSK10] and [MQXMZ11] show that the majority
home routers and ISPs fail to meet the 124-minute idle timeout of home routers and ISPs fail to meet the 124-minute idle timeout
mandated in [RFC5382]. In [MQXMZ11], 35% of mobile ISPs silently mandated in [RFC5382]. In [MQXMZ11], 35% of mobile ISPs silently
timeout idle connections within 30 minutes. End hosts, unaware of time out idle connections within 30 minutes. End hosts, unaware of
silent middle-box timeouts, suffer multi-minute TCP timeouts upon silent middlebox timeouts, suffer multi-minute TCP timeouts upon
using those long-idle connections. using those long-idle connections.
To circumvent this problem, some applications send frequent TCP keep- To circumvent this problem, some applications send frequent TCP keep-
alive probes. However, this technique drains power on mobile devices alive probes. However, this technique drains power on mobile devices
[MQXMZ11]. In fact, power has become such a prominent issue in modern [MQXMZ11]. In fact, power has become such a prominent issue in
LTE devices that mobile browsers close HTTP connections within modern Long Term Evolution (LTE) devices that mobile browsers close
seconds or even immediately [SOUDERS11]. HTTP connections within seconds or even immediately [SOUDERS11].
[RCCJR11] studied Chrome browser [Chrome] performance based on 28 [RCCJR11] studied the performance of the Chrome browser [Chrome]
days of global statistics. The Chrome browser keeps idle HTTP based on 28 days of global statistics. The Chrome browser keeps idle
persistent connections for 5 to 10 minutes. However the average HTTP persistent connections for 5 to 10 minutes. However, the
number of the transactions per connection is only 3.3 and TCP 3WHS average number of the transactions per connection is only 3.3, and
accounts for up to 25% of the HTTP transaction network latency. The the TCP 3WHS accounts for up to 25% of the HTTP transaction network
authors estimated that TFO improves page load time by 10% to 40% on latency. The authors estimated that TFO improves page load time by
selected popular Web sites. 10% to 40% on selected popular Web sites.
6.3.4. Load Balancers and Server farms 6.3.4. Load Balancers and Server Farms
Servers behind a load balancers that accept connection requests to Servers behind load balancers that accept connection requests to the
the same server IP address should use the same key such that they same server IP address should use the same key such that they
generate identical Fast Open Cookies for a particular client IP generate identical Fast Open cookies for a particular client IP
address. Otherwise a client may get different cookies across address. Otherwise, a client may get different cookies across
connections; its Fast Open attempts would fall back to regular 3WHS. connections; its Fast Open attempts would fall back to the regular
3WHS.
7. Open Areas for Experimentation 7. Open Areas for Experimentation
We now outline some areas that need experimentation in the Internet We now outline some areas that need experimentation in the Internet
and under different network scenarios. These experiments should help and under different network scenarios. These experiments should help
the community evaluate Fast Open benefits and risks towards further evaluate Fast Open benefits and risks and its related protocols.
standardization and implementation of Fast Open and its related
protocols.
7.1. Performance impact due to middle-boxes and NAT 7.1. Performance Impact Due to Middleboxes and NAT
[MAF04] found that some middle-boxes and end-hosts may drop packets [MAF04] found that some middleboxes and end hosts may drop packets
with unknown TCP options. Studies [LANGLEY06, HNRGHT11] both found with unknown TCP options. Studies ([LANGLEY06] [HNRGHT11]) have
that 6% of the probed paths on the Internet drop SYN packets with found that 6% of the probed paths on the Internet drop SYN packets
data or with unknown TCP options. The TFO protocol deals with this with data or with unknown TCP options. The TFO protocol deals with
problem by falling back to regular TCP handshake and re-transmitting this problem by falling back to the regular TCP handshake and
SYN without data or cookie options after the initial SYN timeout. retransmitting the SYN without data or cookie options after the
Moreover the implementation is recommended to negatively cache such initial SYN timeout. Moreover, the implementation is recommended to
incidents to avoid recurring timeouts. Further study is required to negatively cache such incidents to avoid recurring timeouts. Further
evaluate the performance impact of these drop behaviors. study is required to evaluate the performance impact of these drop
behaviors.
Another interesting study is the loss of TFO performance benefit Another interesting study is the loss of TFO performance benefit
behind certain carrier-grade NAT. Typically hosts behind a NAT behind certain Carrier-Grade NAT (CGN). Typically, hosts behind a
sharing the same IP address will get the same cookie for the same NAT sharing the same IP address will get the same cookie for the same
server. This will not prevent TFO from working. But on some carrier- server. This will not prevent TFO from working. But, on some CGN
grade NAT configurations where every new TCP connection from the same configurations where every new TCP connection from the same physical
physical host uses a different public IP address, TFO does not host uses a different public IP address, TFO does not provide latency
provide latency benefits. However, there is no performance penalty benefits. However, there is no performance penalty either, as
either, as described in Section "Client: Receiving SYN-ACK". described in the "Client: Receiving SYN-ACK" text in Section 4.2.2.
7.2. Impact on congestion control 7.2. Impact on Congestion Control
Although TFO does not directly change the congestion control, there Although TFO does not directly change TCP's congestion control, there
are subtle cases that it may. When SYN-ACK times out, regular TCP are subtle cases where it could do so. When a SYN-ACK times out,
reduces the initial congestion window before sending any data regular TCP reduces the initial congestion window before sending any
[RFC5681]. However in TFO the server may have already sent up to an data [RFC5681]. However, in TFO, the server may have already sent up
initial window of data. to an initial window of data.
If the server serves mostly short connections then the losses of SYN- If the server serves mostly short connections, then the losses of
ACKs are not as effective as regular TCP on reducing the congestion SYN-ACKs are not as effective as regular TCP on reducing the
window. This could result in an unstable network condition. The congestion window. This could result in an unstable network
connections that experience losses may attempt again and add more condition. The connections that experience losses may attempt again
load under congestion. A potential solution is to temporarily disable and add more load under congestion. A potential solution is to
Fast Open if the server observes many SYN-ACK or data losses during temporarily disable Fast Open if the server observes many SYN-ACK or
the handshake across connections. Further experimentation regarding data losses during the handshake across connections. Further
the congestion control impact will be useful. experimentation regarding the congestion control impact will be
useful.
7.3. Cookie-less Fast Open 7.3. Cookie-less Fast Open
The cookie mechanism mitigates resource exhaustion and amplification The cookie mechanism mitigates resource exhaustion and amplification
attacks. However cookies are not necessary if the server has attacks. However, cookies are not necessary if the server has
application-level protection or is immune to these attacks. For application-level protection or is immune to these attacks. For
example a Web server that only replies with a simple HTTP redirect example, a Web server that only replies with a simple HTTP redirect
response that fits in the SYN-ACK packet may not care about resource response that fits in the SYN-ACK packet may not care about resource
exhaustion. exhaustion.
For such applications the server may choose to generate a trivial or For such applications the server may choose to generate a trivial or
even a zero-length cookie to improve performance by avoiding the even a zero-length cookie to improve performance by avoiding the
cookie generation and verification. If the server believes it's under cookie generation and verification. If the server believes it's
a DoS attack through other defense mechanisms, it can switch to under a DoS attack through other defense mechanisms, it can switch to
regular Fast Open for listener sockets. regular Fast Open for listener sockets.
8. Related Work 8. Related Work
8.1. T/TCP 8.1. T/TCP
TCP Extensions for Transactions [RFC1644] attempted to bypass the TCP Extensions for Transactions [RFC1644] attempted to bypass the
three-way handshake, among other things, hence shared the same goal 3WHS, among other things; hence, it shared the same goal but also the
but also the same set of issues as TFO. It focused most of its effort same set of issues as TFO. It focused most of its effort battling
battling old or duplicate SYNs, but paid no attention to security old or duplicate SYNs, but paid no attention to security
vulnerabilities it introduced when bypassing 3WHS [PHRACK98]. vulnerabilities it introduced when bypassing the 3WHS [PHRACK98].
As stated earlier, we take a practical approach to focus TFO on the As stated earlier, we take a practical approach to focus TFO on the
security aspect, while allowing old, duplicate SYN packets with data security aspect, while allowing old, duplicate SYN packets with data
after recognizing that 100% TCP semantics is likely infeasible. We after recognizing that 100% TCP semantics is likely infeasible. We
believe this approach strikes the right tradeoff, and makes TFO much believe this approach strikes the right trade-off and makes TFO much
simpler and more appealing to TCP implementers and users. simpler and more appealing to TCP implementers and users.
8.2. Common Defenses Against SYN Flood Attacks 8.2. Common Defenses against SYN Flood Attacks
[RFC4987] studies on mitigating attacks from regular SYN flood, i.e., [RFC4987] studies the mitigation of attacks from regular SYN floods,
SYN without data. But from the stateless SYN-cookies to the stateful i.e., SYNs without data. But from the stateless SYN cookies to the
SYN Cache, none can preserve data sent with SYN safely while still stateful SYN Cache, none can preserve data sent with SYNs safely
providing an effective defense. while still providing an effective defense.
The best defense may be to simply disable TFO when a host is The best defense may be simply to disable TFO when a host is
suspected to be under a SYN flood attack, e.g., the SYN backlog is suspected to be under a SYN flood attack, e.g., the SYN backlog is
filled. Once TFO is disabled, normal SYN flood defenses can be filled. Once TFO is disabled, normal SYN flood defenses can be
applied. The "Security Consideration" section contains a thorough applied. The "Security Considerations" section (Section 5) contains
discussion on this topic. a thorough discussion on this topic.
8.3. Speculative Connections by the Applications 8.3. Speculative Connections by the Applications
Some Web browsers maintain a history of the domains for frequently Some Web browsers maintain a history of the domains for frequently
visited web pages. The browsers then speculatively pre-open TCP visited Web pages. The browsers then speculatively pre-open TCP
connections to these domains before the user initiates any requests connections to these domains before the user initiates any requests
for them [BELSHE11]. While this technique also saves the handshake for them [BELSHE11]. While this technique also saves the handshake
latency, it wastes server and network resources by initiating and latency, it wastes server and network resources by initiating and
maintaining idle connections. maintaining idle connections.
8.4. Fast Open Cookie in FIN 8.4. Fast Open Cookie-in-FIN
An alternate proposal is to request a TFO cookie in the FIN instead, An alternate proposal is to request a TFO cookie in the FIN instead,
since FIN-drop by incompatible middle-boxes does not affect latency. since FIN-drop by incompatible middleboxes does not affect latency.
However paths that block SYN cookies may be more likely to drop a However, paths that block SYN cookies may be more likely to drop a
later SYN packet with data, and many applications close a connection later SYN packet with data, and many applications close a connection
with RST instead anyway. with RST instead anyway.
Although cookie-in-FIN may not improve robustness, it would give Although cookie-in-FIN may not improve robustness, it would give
clients using a single connection a latency advantage over clients clients using a single connection a latency advantage over clients
opening multiple parallel connections. If experiments with TFO find opening multiple parallel connections. If experiments with TFO find
that it leads to increased connection-sharding, cookie-in-FIN may that it leads to increased connection-sharding, cookie-in-FIN may
prove to be a useful alternative. prove to be a useful alternative.
8.5. TCP Cookie Transaction (TCPCT) 8.5. TCP Cookie Transaction (TCPCT)
TCPCT [RFC6013] eliminates server state during initial handshake and TCPCT [RFC6013] eliminates server state during the initial handshake
defends spoofing DoS attacks. Like TFO, TCPCT allows SYN and SYN-ACK and defends spoofing DoS attacks. Like TFO, TCPCT allows SYN and
packets to carry data. But the server can only send up to MSS bytes SYN-ACK packets to carry data. But the server can only send up to
of data during the handshake instead of the initial congestion window MSS bytes of data during the handshake instead of the initial
unlike TFO. Therefore the latency of applications such as Web may be congestion window, unlike TFO. Therefore, the latency of
worse than with TFO. applications (e.g., Web applications) may be worse than with TFO.
9. IANA Considerations 9. IANA Considerations
IANA is requested to allocate one value from the TCP Option Kind IANA has allocated one value, 34, in the "TCP Option Kind Numbers"
Numbers: The constant-TBD in Section 4.1.1 has to be replaced with registry. See Section 4.1.1. The length of this new TCP option is
the newly assigned value. The length of the new TCP option Kind is variable, and the Meaning as shown in the "TCP Option Kind Numbers"
variable and the Meaning should be set to "TCP Fast Open Cookie". registry is set to "TCP Fast Open Cookie". Current and new
Early implementation before the IANA allocation SHOULD follow implementations SHOULD use option (34). Existing implementations
[RFC6994] and use experimental option 254 and magic number 0xF989 (16 that are using experimental option 254 per [RFC6994] with magic
bits), then migrate to the new option after the allocation number 0xF989 (16 bits) as allocated in the IANA "TCP Experimental
accordingly. Option Experiment Identifiers (TCP ExIDs)" registry by this document,
SHOULD migrate to use this new option (34) by default.
10. Acknowledgement 10. References
We thank Bob Briscoe, Michael Scharf, Gorry Fairhurst, Rick Jones, 10.1. Normative References
Roberto Peon, William Chan, Adam Langley, Neal Cardwell, Eric
Dumazet, and Matt Mathis for their feedbacks. We especially thank
Barath Raghavan for his contribution on the security design of Fast
Open and proofreading this draft numerous times.
11. References [RFC793] Postel, J., "Transmission Control Protocol", STD 7, RFC
793, September 1981,
<http://www.rfc-editor.org/info/rfc793>.
11.1. Normative References [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, October 1989,
<http://www.rfc-editor.org/info/rfc1122>.
[RFC793] Postel, J. "Transmission Control Protocol", RFC 793, [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
September 1981. Requirement Levels", BCP 14, RFC 2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - [RFC3390] Allman, M., Floyd, S., and C. Partridge, "Increasing
Communication Layers", STD 3, RFC 1122, October 1989. TCP's Initial Window", RFC 3390, October 2002,
<http://www.rfc-editor.org/info/rfc3390>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC5382] Guha, S., Ed., Biswas, K., Ford, B., Sivakumar, S., and
Requirement Levels", BCP 14, RFC 2119, March 1997. P. Srisuresh, "NAT Behavioral Requirements for TCP", BCP
142, RFC 5382, October 2008,
<http://www.rfc-editor.org/info/rfc5382>.
[RFC5382] S. Guha, Ed., Biswas, K., Ford B., Sivakumar S., Srisuresh, [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion
P., "NAT Behavioral Requirements for TCP", RFC 5382 Control", RFC 5681, September 2009,
<http://www.rfc-editor.org/info/rfc5681>.
[RFC5681] Allman, M., Paxson, V. and E. Blanton, "TCP Congestion [RFC6994] Touch, J., "Shared Use of Experimental TCP Options", RFC
Control", RFC 5681, September 2009 6994, August 2013,
<http://www.rfc-editor.org/info/rfc6994>.
[RFC6994] Touch, Joe, "Shared Use of Experimental TCP Options", 10.2. Informative References
RFC6994, August 2013.
[RFC3390] Allman, M., Floyd, S., and C. Partridge, "Increasing TCP's [AERG11] Al-Fares, M., Elmeleegy, K., Reed, B., and I. Gashinsky,
Initial Window", RFC 3390, October 2002. "Overclocking the Yahoo! CDN for Faster Web Page Loads",
in Proceedings of Internet Measurement Conference,
November 2011.
11.2. Informative References [BELSHE11] Belshe, M., "The Era of Browser Preconnect", February
2011, <http://www.belshe.com/2011/02/10/
the-era-of-browser-preconnect/>.
[AERG11] Al-Fares, M., Elmeleegy, K., Reed, B., Gashinsky, I., [BRISCOE12] Briscoe, B., "Some ideas building on draft-ietf-tcpm-
"Overclocking the Yahoo! CDN for Faster Web Page Loads". fastopen-01", message to the tcpm mailing list, July
In Proceedings of Internet Measurement Conference, 2012, <http://www.ietf.org/mail-archive/
November 2011. web/tcpm/current/msg07192.html>.
[Chrome] Google Chrome,
<https://www.google.com/intl/en-US/chrome/browser/>.
[HNESSK10] Haetoenen, S., Nyrhinen, A., Eggert, L., Strowes, S., [HNESSK10] Haetoenen, S., Nyrhinen, A., Eggert, L., Strowes, S.,
Sarolahti, P., Kojo., M., "An Experimental Study of Home Sarolahti, P., and M. Kojo, "An Experimental Study of
Gateway Characteristics". In Proceedings of Internet Home Gateway Characteristics", in Proceedings of Internet
Measurement Conference. October 2010 Measurement Conference, October 2010.
[HNRGHT11] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A., [HNRGHT11] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A.,
Handley, M., Tokuda, H., "Is it Still Possible to Handley, M., and H. Tokuda, "Is it Still Possible to
Extend TCP?". In Proceedings of Internet Measurement Extend TCP?", in Proceedings of Internet Measurement
Conference. November 2011. Conference, November 2011.
[LANGLEY06] Langley, A, "Probing the viability of TCP extensions", [JIDKT07] Jaiswal, S., Iannaccone, G., Diot, C., Kurose, J., and D.
URL http://www.imperialviolet.org/binary/ecntest.pdf Towsley, "Measurement and Classification of Out-of-
Sequence Packets in a Tier-1 IP Backbone" IEEE/ACM
Transactions on Networking (TON), Volume 15, Issue 1, pp
54-66.
[LANGLEY06] Langley, A., "Probing the viability of TCP extensions",
<http://www.imperialviolet.org/binary/ecntest.pdf>.
[MAF04] Medina, A., Allman, M., and S. Floyd, "Measuring [MAF04] Medina, A., Allman, M., and S. Floyd, "Measuring
Interactions Between Transport Protocols and Interactions Between Transport Protocols and
Middleboxes". In Proceedings of Internet Measurement Middleboxes", in Proceedings of Internet Measurement
Conference, October 2004. Conference, October 2004.
[MQXMZ11] Wang, Z., Qian, Z., Xu, Q., Mao, Z., Zhang, M., [MQXMZ11] Wang, Z., Qian, Z., Xu, Q., Mao, Z., and M. Zhang, "An
"An Untold Story of Middleboxes in Cellular Networks". Untold Story of Middleboxes in Cellular Networks", in
In Proceedings of SIGCOMM. August 2011. Proceedings of SIGCOMM, August 2011.
[PHRACK98] "T/TCP vulnerabilities", Phrack Magazine, Volume 8, Issue [PHRACK98] "T/TCP vulnerabilities", Phrack Magazine, Volume 8, Issue
53 artical 6. July 8, 1998. URL 53, Article 6, July 8, 1998,
http://www.phrack.com/issues.html?issue=53&id=6 <http://www.phrack.com/issues.html?issue=53&id=6>.
[RCCJR11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A., [RCCJR11] Radhakrishnan, S., Cheng, Y., Chu, J., Jain, A., and B.
Raghavan, B., "TCP Fast Open". In Proceedings of 7th Raghavan, "TCP Fast Open", in Proceedings of the 7th ACM
ACM CoNEXT Conference, December 2011. CoNEXT Conference, December 2011.
[RFC1323] Jacobson, V., Braden, R., Borman, D., "TCP Extensions for [RFC1323] Jacobson, V., Braden, R., and D. Borman, "TCP Extensions
High Performance", RFC 1323, May 1992. for High Performance", RFC 1323, May 1992,
<http://www.rfc-editor.org/info/rfc1323>.
[RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions [RFC1644] Braden, R., "T/TCP -- TCP Extensions for Transactions
Functional Specification", RFC 1644, July 1994. Functional Specification", RFC 1644, July 1994,
<http://www.rfc-editor.org/info/rfc1644>.
[RFC2460] Deering, S., Hinden, R., "Internet Protocol, Version 6 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6
(IPv6) Specification", RFC 2460, December 1998. (IPv6) Specification", RFC 2460, December 1998,
<http://www.rfc-editor.org/info/rfc2460>.
[RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common
Mitigations", RFC 4987, August 2007. Mitigations", RFC 4987, August 2007,
<http://www.rfc-editor.org/info/rfc4987>.
[RFC6013] Simpson, W., "TCP Cookie Transactions (TCPCT)", RFC6013,
January 2011.
[SOUDERS11] Souders, S., "Making A Mobile Connection".
http://www.stevesouders.com/blog/2011/09/21/making-a-
mobile-connection/
[BRISCOE12] Briscoe, B., "Some ideas building on draft-ietf-tcpm- [RFC6013] Simpson, W., "TCP Cookie Transactions (TCPCT)", RFC 6013,
fastopen-01", tcpm list, January 2011, <http://www.rfc-editor.org/info/rfc6013>.
http://www.ietf.org/mail-archive/web/tcpm/
current/msg07192.html
[BELSHE11] Belshe, M., "The era of browser preconnect.", [RFC6247] Eggert, L., "Moving the Undeployed TCP Extensions RFC
http://www.belshe.com/2011/02/10/ 1072, RFC 1106, RFC 1110, RFC 1145, RFC 1146, RFC 1379,
the-era-of-browser-preconnect/ RFC 1644, and RFC 1693 to Historic Status", RFC 6247, May
2011, <http://www.rfc-editor.org/info/rfc6247>.
[JIDKT07] Jaiswal, S., Iannaccone, G., Diot, C., Kurose, J., [RFC7323] Borman, D., Braden, B., Jacobson, V., and R.
Towsley, D., "Measurement and classification of Scheffenegger, Ed., "TCP Extensions for High
out-of-sequence packets in a tier-1 IP backbone.". Performance", RFC 7323, September 2014,
IEEE/ACM Transactions on Networking (TON), 15(1), 54-66. <http://www.rfc-editor.org/info/rfc7323>.
[Chrome] Chrome. https://www.google.com/intl/en-US/chrome/browser/ [SOUDERS11] Souders, S., "Making A Mobile Connection",
<http://www.stevesouders.com/blog/2011/09/21/
making-a-mobile-connection/>.
Appendix A. Example Socket API Changes to support TFO Appendix A. Example Socket API Changes to Support TFO
A.1 Active Open A.1. Active Open
The active open side involves changing or replacing the connect() The active open side involves changing or replacing the connect()
call, which does not take a user data buffer argument. We recommend call, which does not take a user data buffer argument. We recommend
replacing connect() call to minimize API changes and hence replacing the connect() call to minimize API changes, and, hence,
applications to reduce the deployment hurdle. applications to reduce the deployment hurdle.
One solution implemented in Linux 3.7 is introducing a new flag One solution implemented in Linux 3.7 is introducing a new flag,
MSG_FASTOPEN for sendto() or sendmsg(). MSG_FASTOPEN marks the MSG_FASTOPEN, for sendto() or sendmsg(). MSG_FASTOPEN marks the
attempt to send data in SYN like a combination of connect() and attempt to send data in the SYN like a combination of connect() and
sendto(), by performing an implicit connect() operation. It blocks sendto(), by performing an implicit connect() operation. It blocks
until the handshake has completed and the data is buffered. until the handshake has completed and the data is buffered.
For non-blocking socket it returns the number of bytes buffered and For a non-blocking socket, it returns the number of bytes buffered
sent in the SYN packet. If the cookie is not available locally, it and sent in the SYN packet. If the cookie is not available locally,
returns -1 with errno EINPROGRESS, and sends a SYN with TFO cookie it returns -1 with errno EINPROGRESS, and sends a SYN with a TFO
request automatically. The caller needs to write the data again when cookie request automatically. The caller needs to write the data
the socket is connected. On errors, it returns the same errno as again when the socket is connected. On errors, it returns the same
connect() if the handshake fails. errno as connect() if the handshake fails.
An implementation may prefer not to change the sendmsg() because TFO An implementation may prefer not to change the sendmsg() call because
is a TCP specific feature. A solution is to add a new socket option TFO is a TCP-specific feature. A solution is to add a new socket
TCP_FASTOPEN for TCP sockets. When the option is enabled before a option, TCP_FASTOPEN, for TCP sockets. When the option is enabled
connect operation, sendmsg() or sendto() will perform Fast Open before a connect() operation, sendmsg() or sendto() will perform a
operation similar to the MSG_FASTOPEN flag described above. This Fast Open operation similar to the MSG_FASTOPEN flag described above.
approach however requires an extra setsockopt() system call. This approach, however, requires an extra setsockopt() system call.
A.2 Passive Open A.2. Passive Open
The passive open side change is simpler compared to active open side. The passive open side change is simpler compared to the active open
The application only needs to enable the reception of Fast Open side. The application only needs to enable the reception of Fast
requests via a new TCP_FASTOPEN setsockopt() socket option before Open requests via a new TCP_FASTOPEN setsockopt() socket option
listen(). before listen().
The option enables Fast Open on the listener socket. The option value The option enables Fast Open on the listener socket. The option
specifies the PendingFastOpenRequests threshold, i.e., the maximum value specifies the PendingFastOpenRequests threshold, i.e., the
length of pending SYNs with data payload. Once enabled, the TCP maximum length of pending SYNs with data payload. Once enabled, the
implementation will respond with TFO cookies per request. TCP implementation will respond with TFO cookies per request.
Traditionally accept() returns only after a socket is connected. But Traditionally, accept() returns only after a socket is connected.
for a Fast Open connection, accept() returns upon receiving a SYN But, for a Fast Open connection, accept() returns upon receiving a
with a valid Fast Open cookie and data, and the data is available to SYN with a valid Fast Open cookie and data, and the data is available
be read through, e.g., recvmsg(), read(). to be read through, e.g., recvmsg(), read().
Acknowledgments
We thank Bob Briscoe, Michael Scharf, Gorry Fairhurst, Rick Jones,
Roberto Peon, William Chan, Adam Langley, Neal Cardwell, Eric
Dumazet, and Matt Mathis for their feedback. We especially thank
Barath Raghavan for his contribution on the security design of Fast
Open and proofreading this document numerous times.
Authors' Addresses Authors' Addresses
Yuchung Cheng Yuchung Cheng
Google, Inc. Google, Inc.
1600 Amphitheatre Parkway 1600 Amphitheatre Parkway
Mountain View, CA 94043, USA Mountain View, CA 94043
United States
EMail: ycheng@google.com EMail: ycheng@google.com
Jerry Chu Jerry Chu
Google, Inc. Google, Inc.
1600 Amphitheatre Parkway 1600 Amphitheatre Parkway
Mountain View, CA 94043, USA Mountain View, CA 94043
United States
EMail: hkchu@google.com EMail: hkchu@google.com
Sivasankar Radhakrishnan Sivasankar Radhakrishnan
Department of Computer Science and Engineering Department of Computer Science and Engineering
University of California, San Diego University of California, San Diego
9500 Gilman Dr 9500 Gilman Drive
La Jolla, CA 92093-0404 La Jolla, CA 92093-0404
United States
EMail: sivasankar@cs.ucsd.edu EMail: sivasankar@cs.ucsd.edu
Arvind Jain Arvind Jain
Google, Inc. Google, Inc.
1600 Amphitheatre Parkway 1600 Amphitheatre Parkway
Mountain View, CA 94043, USA Mountain View, CA 94043
United States
EMail: arvind@google.com EMail: arvind@google.com
 End of changes. 217 change blocks. 
621 lines changed or deleted 667 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/