draft-ietf-tcpimpl-pmtud-02.txt   draft-ietf-tcpimpl-pmtud-03.txt 
Network Working Group K. Lahey Network Working Group K. Lahey
Expires: January 2000 Expires: October 2000
TCP Problems with Path MTU Discovery TCP Problems with Path MTU Discovery
<draft-ietf-tcpimpl-pmtud-02.txt> <draft-ietf-tcpimpl-pmtud-03.txt>
1. Status of this Memo 1. Status of this Memo
This documnt is an Internet-Draft and is in full onformance with all This document is an Internet-Draft and is in full conformance with
provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
This document is an Internet Draft. Internet Drafts are working This document is an Internet Draft. Internet Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute and its working groups. Note that other groups may also distribute
working documents as Internet Drafts. working documents as Internet Drafts.
Internet Drafts are draft documents valid for a maximum of six Internet Drafts are draft documents valid for a maximum of six
months, and may be updated, replaced, or obsoleted by other documents months, and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet Drafts as reference at any time. It is inappropriate to use Internet Drafts as reference
material or to cite them other than as ``work in progress''. material or to cite them other than as ``work in progress''.
skipping to change at page 2, line 14 skipping to change at page 2, line 28
issues, but not the TCP issues brought up in this document. issues, but not the TCP issues brought up in this document.
Each problem is defined as follows: Each problem is defined as follows:
Name of Problem Name of Problem
The name associated with the problem. In this memo, the name is The name associated with the problem. In this memo, the name is
given as a subsection heading. given as a subsection heading.
Classification Classification
One or more problem categories for which the problem is classified: One or more problem categories for which the problem is classified:
"congestion control", "performance", "reliability", "non-interoper- "congestion control", "performance", "reliability", "non-
ation - connectivity failure". interoperation -- connectivity failure".
Description Description
A definition of the problem, succinct but including necessary back- A definition of the problem, succinct but including necessary
ground material. background material.
Significance Significance
A brief summary of the sorts of environments for which the problem A brief summary of the sorts of environments for which the problem
is significant. is significant.
Implications Implications
Why the problem is viewed as a problem. Why the problem is viewed as a problem.
Relevant RFCs Relevant RFCs
The RFCs defining the TCP specification with which the problem con- The RFCs defining the TCP specification with which the problem
flicts. These RFCs often qualify behavior using terms such as conflicts. These RFCs often qualify behavior using terms such as
MUST, SHOULD, MAY, and others written capitalized. See RFC 2119 MUST, SHOULD, MAY, and others written capitalized. See RFC 2119
for the exact interpretation of these terms. for the exact interpretation of these terms.
Trace file demonstrating the problem Trace file demonstrating the problem
One or more ASCII trace files demonstrating the problem, if appli- One or more ASCII trace files demonstrating the problem, if
cable. applicable.
Trace file demonstrating correct behavior Trace file demonstrating correct behavior
One or more examples of how correct behavior appears in a trace, if One or more examples of how correct behavior appears in a trace, if
applicable. applicable.
References References
References that further discuss the problem. References that further discuss the problem.
How to detect How to detect
How to test an implementation to see if it exhibits the problem. How to test an implementation to see if it exhibits the problem.
skipping to change at page 3, line 28 skipping to change at page 3, line 36
3.1. 3.1.
Name of Problem Name of Problem
Black Hole Detection Black Hole Detection
Classification Classification
Non-interoperation -- connectivity failure Non-interoperation -- connectivity failure
Description Description
Path MTU Discovery (PMTUD) works by sending out as large a packet A host performs Path MTU Discovery by sending out as large a packet
as possible, with the Don't Fragment (DF) bit set in the IP header. as possible, with the Don't Fragment (DF) bit set in the IP header.
If the packet is too large for a router to forward on to a particu- If the packet is too large for a router to forward on to a
lar link, the router must send an ICMP Destination Unreachable -- particular link, the router must send an ICMP Destination
Fragmentation Needed message to the source address. Unreachable -- Fragmentation Needed message to the source address.
The host then adjusts the packet size based on the ICMP message.
As was pointed out in [RFC1435], routers don't always do this cor- As was pointed out in [RFC1435], routers don't always do this
rectly -- many routers fail to send the ICMP messages, for a vari- correctly -- many routers fail to send the ICMP messages, for a
ety of reasons ranging from kernel bugs to configuration problems. variety of reasons ranging from kernel bugs to configuration
Firewalls are often misconfigured to supress all ICMP messages. problems. Firewalls are often misconfigured to suppress all ICMP
IPsec [RFC2401] and IP-in-IP [RFC2003] tunnels shouldn't cause messages. IPsec [RFC2401] and IP-in-IP [RFC2003] tunnels shouldn't
these sorts of problems, if the implementations follow the advice cause these sorts of problems, if the implementations follow the
in the appropriate documents. advice in the appropriate documents.
PMTUD, as documented in [RFC1191], fails when confronted with PMTUD, as documented in [RFC1191], fails when the appropriate ICMP
routers that fail to send the appropriate ICMP message. The upper- messages are not received by the originating host. The upper-layer
layer protocol continues to try to send large packets and, without protocol continues to try to send large packets and, without the
the ICMP messages, never discovers that it needs to reduce the size ICMP messages, never discovers that it needs to reduce the size of
of those packets. Its packets are disappearing into a PMTUD black those packets. Its packets are disappearing into a PMTUD black
hole. hole.
Significance Significance
When PMTUD fails due to the lack of ICMP messages, TCP will also When PMTUD fails due to the lack of ICMP messages, TCP will also
completly fail under some conditions. completely fail under some conditions.
Implications Implications
This failure is especially difficult to debug, as pings and some This failure is especially difficult to debug, as pings and some
interactive TCP connections to the destination host work. Bulk interactive TCP connections to the destination host work. Bulk
transfers fail with the first large packet and the connection even- transfers fail with the first large packet and the connection
tually times out. eventually times out.
These situations can almost always be blamed on a misconfiguration These situations can almost always be blamed on a misconfiguration
within the network, which should be corrected. However it is not within the network, which should be corrected. However it seems
appropriate for some TCP implementations to suffer interoperability inappropriate for some TCP implementations to suffer
failures over paths which do not affect other TCP implementions interoperability failures over paths which do not affect other TCP
(i.e. those without PMTUD). implementations (i.e. those without PMTUD). This creates a market
disincentive for deploying TCP implementation with PMTUD enabled.
This creates a market disincentive for deploying TCP implementation
with PMTUD enabled.
Relevant RFCs Relevant RFCs
RFC1191 describes Path MTU Discovery. RFC 1435 provides an early RFC1191 describes Path MTU Discovery. RFC 1435 provides an early
description of these sorts of problems. description of these sorts of problems.
Trace file demonstrating the problem Trace file demonstrating the problem
Made using tcpdump [Jacobson89] recording at an intermediate host. Made using tcpdump [Jacobson89] recording at an intermediate host.
20:12:11.951321 A > B: S 1748427200:1748427200(0) 20:12:11.951321 A > B: S 1748427200:1748427200(0)
win 49152 <mss 1460> win 49152 <mss 1460>
skipping to change at page 5, line 13 skipping to change at page 5, line 22
20:13:47.703184 A > B: . 1:1461(1460) ack 1 win 49152 (DF) 20:13:47.703184 A > B: . 1:1461(1460) ack 1 win 49152 (DF)
20:14:52.780640 A > B: . 1:1461(1460) ack 1 win 49152 (DF) 20:14:52.780640 A > B: . 1:1461(1460) ack 1 win 49152 (DF)
20:15:57.856037 A > B: . 1:1461(1460) ack 1 win 49152 (DF) 20:15:57.856037 A > B: . 1:1461(1460) ack 1 win 49152 (DF)
20:17:02.932431 A > B: . 1:1461(1460) ack 1 win 49152 (DF) 20:17:02.932431 A > B: . 1:1461(1460) ack 1 win 49152 (DF)
20:18:08.009337 A > B: . 1:1461(1460) ack 1 win 49152 (DF) 20:18:08.009337 A > B: . 1:1461(1460) ack 1 win 49152 (DF)
20:19:13.090521 A > B: . 1:1461(1460) ack 1 win 49152 (DF) 20:19:13.090521 A > B: . 1:1461(1460) ack 1 win 49152 (DF)
20:20:18.168066 A > B: . 1:1461(1460) ack 1 win 49152 (DF) 20:20:18.168066 A > B: . 1:1461(1460) ack 1 win 49152 (DF)
20:21:23.242761 A > B: R 1461:1461(0) ack 1 win 49152 (DF) 20:21:23.242761 A > B: R 1461:1461(0) ack 1 win 49152 (DF)
The short SYN packet has no trouble traversing the network, due to The short SYN packet has no trouble traversing the network, due to
its small size. Similarly, ICMP echo packets used to diagnose con- its small size. Similarly, ICMP echo packets used to diagnose
nectivity problems will succeed. connectivity problems will succeed.
Large data packets fail to traverse the network. Eventually the Large data packets fail to traverse the network. Eventually the
connection times out. This can be especially confusing when the connection times out. This can be especially confusing when the
application starts out with a very small write, which succeeds, application starts out with a very small write, which succeeds,
following up with many large writes, which then fail. following up with many large writes, which then fail.
Trace file demonstrating correct behavior Trace file demonstrating correct behavior
Made using tcpdump recording at an intermediate host. Made using tcpdump recording at an intermediate host.
skipping to change at page 6, line 6 skipping to change at page 6, line 13
In this case, the sender sees four packets fail to traverse the In this case, the sender sees four packets fail to traverse the
network (using a two-packet initial send window) and turns off network (using a two-packet initial send window) and turns off
PMTUD. All subsequent packets have the DF flag turned off, and the PMTUD. All subsequent packets have the DF flag turned off, and the
size set to the default value of 536 [RFC1122]. size set to the default value of 536 [RFC1122].
References References
This problem has been discussed extensively on the tcp-impl mailing This problem has been discussed extensively on the tcp-impl mailing
list; the name "black hole" has been in use for many years. list; the name "black hole" has been in use for many years.
How to detect How to detect
This shows up as a TCP connection which hangs (fails to make This shows up as a TCP connection which hangs (fails to make
progress) until closed by timeout (this often manifests itself as a progress) until closed by timeout (this often manifests itself as a
connection that connects and starts to transfer, then eventually connection that connects and starts to transfer, then eventually
terminates after 15 minutes with zero bytes transfered). This is terminates after 15 minutes with zero bytes transfered). This is
particularly annoying with an application like ftp, which will work particularly annoying with an application like ftp, which will work
perfectly while it uses small packets for control information, and perfectly while it uses small packets for control information, and
then fail on bulk transfers. then fail on bulk transfers.
A series of ICMP echo packets will show that the connection is A series of ICMP echo packets will show that the two end hosts are
still passing packets, a series of MTU-sized ICMP echo packets still capable of passing packets, a series of MTU-sized ICMP echo
will show some fragmentation, and a series of MTU-sized ICMP echo packets will show some fragmentation, and a series of MTU-sized
packets with DF set will fail. This can be confusing for network ICMP echo packets with DF set will fail. This can be confusing for
engineers trying to diagnose the problem. network engineers trying to diagnose the problem.
There are several traceroute implementations that do PMTUD, and can There are several traceroute implementations that do PMTUD, and can
demonstrate the problem. See, for example, demonstrate the problem.
ftp://ftp.psc.edu/pub/networking/tools/traceroute.tar
How to fix How to fix
TCP should notice that the connection is timing out. After several TCP should notice that the connection is timing out. After several
timeouts, TCP should attempt to send smaller packets, perhaps turn- timeouts, TCP should attempt to send smaller packets, perhaps
ing off the DF flag for each packet. If this succeeds, it should turning off the DF flag for each packet. If this succeeds, it
continue to turn off PMTUD for the connection for some reasonable should continue to turn off PMTUD for the connection for some
period of time, after which it should probe again to try to deter- reasonable period of time, after which it should probe again to try
mine if the path has changed. to determine if the path has changed.
Note that, under IPv6, there is no DF bit -- it is implicitly on at Note that, under IPv6, there is no DF bit -- it is implicitly on at
all times. Fragmentation is not allowed in routers, only at the all times. Fragmentation is not allowed in routers, only at the
originating host. Fortunately, the minimum supported MTU for IPv6 originating host. Fortunately, the minimum supported MTU for IPv6
is 1280 octets, which is significantly larger than the 68 octet is 1280 octets, which is significantly larger than the 68 octet
minimum in IPv4. This should make it more reasonable for IPv6 TCP minimum in IPv4. This should make it more reasonable for IPv6 TCP
implementations to fall back to 1280 octet packets, when IPv4 implementations to fall back to 1280 octet packets, when IPv4
implementations will probably have to turn off DF to respond to implementations will probably have to turn off DF to respond to
black hole detection. black hole detection.
While, ideally, the ICMP black holes should be fixed when they are Ideally, the ICMP black holes should be fixed when they are found.
found, the large number of these requires some more aggressive
response on the part of host implementations. Any system that uses If hosts start to implement black hole detection, it may be that
Path MTU Discovery should consider support some for some form of these problems will go unnoticed and unfixed. This is especially
black hole detection. unfortunate, since detection can take several seconds each time,
and these delays could result in a significant, hidden degradation
of performance. Hosts that implement black hole detection should
probably log detected black holes, so that they can be fixed.
3.2. 3.2.
Name of Problem Name of Problem
Stretch ACK due to PMTUD Stretch ACK due to PMTUD
Classification Classification
Congestion Control / Performance Congestion Control / Performance
Description Description
When a naively implemented TCP stack communicates with a PMTUD- When a naively implemented TCP stack communicates with a PMTUD-
equipped stack, it will try to generate an ACK for every second equipped stack, it will try to generate an ACK for every second
full-sized segment. If it determines the full-sized segment based full-sized segment. If it determines the full-sized segment based
on the advertised MSS, this can degrade badly in the face of PMTUD. on the advertised MSS, this can degrade badly in the face of PMTUD.
The PMTU can wind up being a small fraction of the advertised MSS; The PMTU can wind up being a small fraction of the advertised MSS;
in this case, an ACK would be generated only very infrequently. in this case, an ACK would be generated only very infrequently.
Significance Significance
Stretch ACKs have a variety of unfortunate effects, more fully out- Stretch ACKs have a variety of unfortunate effects, more fully
lined in [RFC2525]. Most of these have to do with encouraging a outlined in [RFC2525]. Most of these have to do with encouraging a
more bursty connection, due to the infrequent arrival of ACKs. more bursty connection, due to the infrequent arrival of ACKs.
They can also impede congestion window growth. They can also impede congestion window growth.
Implications Implications
The complete implications of stretch ACKs are outlined in The complete implications of stretch ACKs are outlined in
[RFC2525]. [RFC2525].
Relevant RFCs Relevant RFCs
RFC 1122 outlines the requirements for frequency of ACK generation. RFC 1122 outlines the requirements for frequency of ACK generation.
[RFC2581] expands on this and clarifies that delayed ACK is a [RFC2581] expands on this and clarifies that delayed ACK is a
SHOULD, not a MUST. SHOULD, not a MUST.
Trace file demonstrating it Trace file demonstrating it
Made using tcpdump recording at an intermediate host. The times- Made using tcpdump recording at an intermediate host. The
tamp options from all but the first two packets have been removed timestamp options from all but the first two packets have been
for clarity. removed for clarity.
18:16:52.976657 A > B: S 3183102292:3183102292(0) win 16384 <mss 18:16:52.976657 A > B: S 3183102292:3183102292(0) win 16384
4312,nop,wscale 0,nop,nop,timestamp 12128 0> (DF) () <mss 4312,nop,wscale 0,nop,nop,timestamp 12128 0> (DF) ()
18:16:52.979580 B > A: S 2022212745:2022212745(0) ack 3183102293 win 18:16:52.979580 B > A: S 2022212745:2022212745(0) ack 3183102293 win 49152
49152 <mss 4312,nop,wscale 1,nop,nop,timestamp 1592957 12128> (DF) () <mss 4312,nop,wscale 1,nop,nop,timestamp 1592957 12128> (DF) ()
18:16:52.979738 A > B: . ack 1 win 17248 (DF) () 18:16:52.979738 A > B: . ack 1 win 17248 (DF) ()
18:16:52.982473 A > B: . 1:4301(4300) ack 1 win 17248 (DF) () 18:16:52.982473 A > B: . 1:4301(4300) ack 1 win 17248 (DF) ()
18:16:52.982557 C > A: icmp: B unreachable - need to frag (mtu 1500) 18:16:52.982557 C > A: icmp: B unreachable -
(DF) () need to frag (mtu 1500)! (DF) ()
18:16:52.985839 B > A: . ack 1 win 32768 (DF) () 18:16:52.985839 B > A: . ack 1 win 32768 (DF) ()
18:16:54.129928 A > B: . 1:1449(1448) ack 1 win 17248 (DF) () 18:16:54.129928 A > B: . 1:1449(1448) ack 1 win 17248 (DF) ()
. .
. .
. .
18:16:58.507078 A > B: . 1463941:1465389(1448) ack 1 win 17248 (DF) () 18:16:58.507078 A > B: . 1463941:1465389(1448) ack 1 win 17248 (DF) ()
18:16:58.507200 A > B: . 1465389:1466837(1448) ack 1 win 17248 (DF) () 18:16:58.507200 A > B: . 1465389:1466837(1448) ack 1 win 17248 (DF) ()
18:16:58.507326 A > B: . 1466837:1468285(1448) ack 1 win 17248 (DF) () 18:16:58.507326 A > B: . 1466837:1468285(1448) ack 1 win 17248 (DF) ()
18:16:58.507439 A > B: . 1468285:1469733(1448) ack 1 win 17248 (DF) () 18:16:58.507439 A > B: . 1468285:1469733(1448) ack 1 win 17248 (DF) ()
18:16:58.524763 B > A: . ack 1452357 win 32768 (DF) () 18:16:58.524763 B > A: . ack 1452357 win 32768 (DF) ()
skipping to change at page 8, line 44 skipping to change at page 9, line 7
18:16:58.538328 A > B: . 1487109:1488557(1448) ack 1 win 17248 (DF) () 18:16:58.538328 A > B: . 1487109:1488557(1448) ack 1 win 17248 (DF) ()
Note that the interval between ACKs is significantly larger than Note that the interval between ACKs is significantly larger than
two times the segment size; it works out to be almost exactly two two times the segment size; it works out to be almost exactly two
times the advertised MSS. This transfer was long enough that it times the advertised MSS. This transfer was long enough that it
could be verified that the stretch ACK was not the result of lost could be verified that the stretch ACK was not the result of lost
ACK packets. ACK packets.
Trace file demonstrating correct behavior Trace file demonstrating correct behavior
Made using tcpdump recording at an intermediate host. The times- Made using tcpdump recording at an intermediate host. The
tamp options from all but the first two packets have been removed timestamp options from all but the first two packets have been
for clarity. removed for clarity.
18:13:32.287965 A > B: S 2972697496:2972697496(0) win 16384 18:13:32.287965 A > B: S 2972697496:2972697496(0)
<mss 4312,nop,wscale 0,nop,nop,timestamp 11326 0> (DF) win 16384 <mss 4312,nop,wscale 0,nop,nop,timestamp 11326 0> (DF)
18:13:32.290785 B > A: S 245639054:245639054(0) ack 2972697497 win 18:13:32.290785 B > A: S 245639054:245639054(0)
34496 ack 2972697497 win 34496 <mss 4312> (DF)
<mss 4312> (DF)
18:13:32.290941 A > B: . ack 1 win 17248 (DF) 18:13:32.290941 A > B: . ack 1 win 17248 (DF)
18:13:32.293774 A > B: . 1:4313(4312) ack 1 win 17248 (DF) 18:13:32.293774 A > B: . 1:4313(4312) ack 1 win 17248 (DF)
18:13:32.293856 C > A: icmp: B unreachable - need to frag (mtu 1500) 18:13:32.293856 C > A: icmp: B unreachable -
(DF) need to frag (mtu 1500)! (DF)
18:13:33.637338 A > B: . 1:1461(1460) ack 1 win 17248 (DF) 18:13:33.637338 A > B: . 1:1461(1460) ack 1 win 17248 (DF)
. .
. .
. .
18:13:35.561691 A > B: . 1514021:1515481(1460) ack 1 win 17248 (DF) 18:13:35.561691 A > B: . 1514021:1515481(1460) ack 1 win 17248 (DF)
18:13:35.561814 A > B: . 1515481:1516941(1460) ack 1 win 17248 (DF) 18:13:35.561814 A > B: . 1515481:1516941(1460) ack 1 win 17248 (DF)
18:13:35.561938 A > B: . 1516941:1518401(1460) ack 1 win 17248 (DF) 18:13:35.561938 A > B: . 1516941:1518401(1460) ack 1 win 17248 (DF)
18:13:35.562059 A > B: . 1518401:1519861(1460) ack 1 win 17248 (DF) 18:13:35.562059 A > B: . 1518401:1519861(1460) ack 1 win 17248 (DF)
18:13:35.562174 A > B: . 1519861:1521321(1460) ack 1 win 17248 (DF) 18:13:35.562174 A > B: . 1519861:1521321(1460) ack 1 win 17248 (DF)
18:13:35.564008 B > A: . ack 1481901 win 64680 (DF) 18:13:35.564008 B > A: . ack 1481901 win 64680 (DF)
skipping to change at page 9, line 38 skipping to change at page 9, line 50
18:13:35.616211 A > B: . 1527161:1528621(1460) ack 1 win 17248 (DF) 18:13:35.616211 A > B: . 1527161:1528621(1460) ack 1 win 17248 (DF)
18:13:35.616228 B > A: . ack 1505261 win 64680 (DF) 18:13:35.616228 B > A: . ack 1505261 win 64680 (DF)
18:13:35.616327 A > B: . 1528621:1530081(1460) ack 1 win 17248 (DF) 18:13:35.616327 A > B: . 1528621:1530081(1460) ack 1 win 17248 (DF)
18:13:35.616349 B > A: . ack 1508181 win 64680 (DF) 18:13:35.616349 B > A: . ack 1508181 win 64680 (DF)
18:13:35.616448 A > B: . 1530081:1531541(1460) ack 1 win 17248 (DF) 18:13:35.616448 A > B: . 1530081:1531541(1460) ack 1 win 17248 (DF)
18:13:35.616565 A > B: . 1531541:1533001(1460) ack 1 win 17248 (DF) 18:13:35.616565 A > B: . 1531541:1533001(1460) ack 1 win 17248 (DF)
18:13:35.616891 A > B: . 1533001:1534461(1460) ack 1 win 17248 (DF) 18:13:35.616891 A > B: . 1533001:1534461(1460) ack 1 win 17248 (DF)
In this trace, an ACK is generated for every two segments that In this trace, an ACK is generated for every two segments that
arrive. (The segment size is slightly larger in this trace, even arrive. (The segment size is slightly larger in this trace, even
though the source hosts are the same, because of the lack of times- though the source hosts are the same, because of the lack of
tamp options in this trace.) timestamp options in this trace.)
How to detect How to detect
This condition can be observered in a packet trace when the adver- This condition can be observed in a packet trace when the
tised MSS is significantly larger than the actual PMTU of a connec- advertised MSS is significantly larger than the actual PMTU of a
tion. connection.
How to fix How to fix
Several solutions for this problem have been proposed: Several solutions for this problem have been proposed:
A simple solution is to ACK every other packet, regardless of size. A simple solution is to ACK every other packet, regardless of size.
This has the drawback of generating large numbers of ACKs in the This has the drawback of generating large numbers of ACKs in the
face of lots of very small packets; this shows up with applica- face of lots of very small packets; this shows up with
tions like the X Window System. applications like the X Window System.
A slightly more complex solution would monitor the size of incoming A slightly more complex solution would monitor the size of incoming
segments and try to determine what segment size the sender is segments and try to determine what segment size the sender is
using. This requires slightly more state in the receiver, but has using. This requires slightly more state in the receiver, but has
the advantage of making receiver silly window syndrome avoidance the advantage of making receiver silly window syndrome avoidance
computations more accurate. computations more accurate.
3.3. 3.3.
Name of Problem Name of Problem
skipping to change at page 10, line 36 skipping to change at page 11, line 6
This results in an advertised MSS that is smaller than the largest This results in an advertised MSS that is smaller than the largest
MTU the system can receive. MTU the system can receive.
Significance Significance
The advertised MSS is an indication to the remote system about the The advertised MSS is an indication to the remote system about the
largest TCP segment that can be received [RFC879]. If this value largest TCP segment that can be received [RFC879]. If this value
is too small, the remote system will be forced to use a smaller is too small, the remote system will be forced to use a smaller
segment size when sending, purely because the local system found a segment size when sending, purely because the local system found a
particular PMTU earlier. particular PMTU earlier.
Given the asymmetric nature of many routes on the Internet [Pax- Given the asymmetric nature of many routes on the Internet
son97], it seems entirely possible that the return PMTU is differ- [Paxson97], it seems entirely possible that the return PMTU is
ent from the sending PMTU. Limiting the segment size in this way different from the sending PMTU. Limiting the segment size in this
can reduce performance and frustrate the PMTUD algorithm. way can reduce performance and frustrate the PMTUD algorithm.
Even if the route was symmetric, setting this artificially lowered Even if the route was symmetric, setting this artificially lowered
limit on segment size will make it impossible to probe later to limit on segment size will make it impossible to probe later to
determine if the PMTU has changed. determine if the PMTU has changed.
Implications Implications
The whole point of PMTUD is to send as large a segment as possible. The whole point of PMTUD is to send as large a segment as possible.
If long-running connections cannot successfully probe for larger If long-running connections cannot successfully probe for larger
PMTU, then potential performance gains will be impossible to real- PMTU, then potential performance gains will be impossible to
ize. This destroys the whole point of PMTUD. realize. This destroys the whole point of PMTUD.
Relevant RFCs Relevant RFCs
RFC 1191. [RFC897] provides a complete discussion of MSS calcula- RFC 1191. [RFC897] provides a complete discussion of MSS
tions and appropriate values. Note that this practice does not calculations and appropriate values. Note that this practice does
violate any of the specifications in these RFCs. not violate any of the specifications in these RFCs.
Trace file demonstrating it Trace file demonstrating it
This trace was made using tcpdump running on an intermediate host. This trace was made using tcpdump running on an intermediate host.
Host A initiates two separate consecutive connections, A1 and A2, Host A initiates two separate consecutive connections, A1 and A2,
to host B. Router C is the location of the MTU bottleneck. As to host B. Router C is the location of the MTU bottleneck. As
usual, TCP options are removed from all non-SYN packets. usual, TCP options are removed from all non-SYN packets.
22:33:32.305912 A1 > B: S 1523306220:1523306220(0) 22:33:32.305912 A1 > B: S 1523306220:1523306220(0)
win 8760 <mss 1460> (DF) win 8760 <mss 1460> (DF)
22:33:32.306518 B > A1: S 729966260:729966260(0) 22:33:32.306518 B > A1: S 729966260:729966260(0)
skipping to change at page 12, line 15 skipping to change at page 12, line 26
22:33:34.471144 B > A2: . ack 985 win 16384 22:33:34.471144 B > A2: . ack 985 win 16384
22:33:34.476554 A2 > B: . 985:1969(984) ack 1 win 8856 (DF) 22:33:34.476554 A2 > B: . 985:1969(984) ack 1 win 8856 (DF)
22:33:34.477580 A2 > B: P 1969:2953(984) ack 1 win 8856 (DF) 22:33:34.477580 A2 > B: P 1969:2953(984) ack 1 win 8856 (DF)
[...] [...]
Notice that the SYN packet for session A2 specifies an MSS of 984. Notice that the SYN packet for session A2 specifies an MSS of 984.
Trace file demonstrating correct behavior Trace file demonstrating correct behavior
As before, this trace was made using tcpdump running on an interme- As before, this trace was made using tcpdump running on an
diate host. Host A initiates two separate consecutive connections, intermediate host. Host A initiates two separate consecutive
A1 and A2, to host B. Router C is the location of the MTU bottle- connections, A1 and A2, to host B. Router C is the location of the
neck. As usual, TCP options are removed from all non-SYN packets. MTU bottleneck. As usual, TCP options are removed from all non-SYN
packets.
22:36:58.828602 A1 > B: S 3402991286:3402991286(0) win 32768 22:36:58.828602 A1 > B: S 3402991286:3402991286(0) win 32768
<mss 4312,wscale 0,nop,timestamp 1123370309 0, <mss 4312,wscale 0,nop,timestamp 1123370309 0,
echo 1123370309> (DF) echo 1123370309> (DF)
22:36:58.844040 B > A1: S 946999880:946999880(0) 22:36:58.844040 B > A1: S 946999880:946999880(0)
ack 3402991287 win 16384 ack 3402991287 win 16384
<mss 65240,nop,wscale 0,nop,nop,timestamp 429552 1123370309> <mss 65240,nop,wscale 0,nop,nop,timestamp 429552 1123370309>
22:36:58.848058 A1 > B: . ack 1 win 32768 (DF) 22:36:58.848058 A1 > B: . ack 1 win 32768 (DF)
22:36:58.851514 A1 > B: P 1:1025(1024) ack 1 win 32768 (DF) 22:36:58.851514 A1 > B: P 1:1025(1024) ack 1 win 32768 (DF)
22:36:58.851584 C > A1: icmp: 129.99.238.5 unreachable - 22:36:58.851584 C > A1: icmp: 129.99.238.5 unreachable -
skipping to change at page 13, line 12 skipping to change at page 13, line 24
22:37:01.446519 B > A2: . ack 1025 win 16384 22:37:01.446519 B > A2: . ack 1025 win 16384
22:37:01.448465 A2 > B: F 1025:1025(0) ack 1 win 32768 (DF) 22:37:01.448465 A2 > B: F 1025:1025(0) ack 1 win 32768 (DF)
22:37:01.448837 B > A2: . ack 1026 win 16384 22:37:01.448837 B > A2: . ack 1026 win 16384
22:37:01.449007 B > A2: F 1:1(0) ack 1026 win 16384 22:37:01.449007 B > A2: F 1:1(0) ack 1026 win 16384
22:37:01.452201 A2 > B: . ack 2 win 32768 (DF) 22:37:01.452201 A2 > B: . ack 2 win 32768 (DF)
Note that the same MSS was used for both session A1 and session A2. Note that the same MSS was used for both session A1 and session A2.
How to detect How to detect
This can be detected using a packet trace of two separate connec- This can be detected using a packet trace of two separate
tions; the first should invoke PMTUD; the second should start soon connections; the first should invoke PMTUD; the second should
enough after the first that the PMTU value does not time out. start soon enough after the first that the PMTU value does not time
out.
How to fix How to fix
The MSS should be determined based on the MTUs of the interfaces on The MSS should be determined based on the MTUs of the interfaces on
the system, as outlined in [RFC1122] and [RFC1191]. the system, as outlined in [RFC1122] and [RFC1191].
4. Security Considerations 4. Security Considerations
The one security concern raised by this memo is that ICMP black holes The one security concern raised by this memo is that ICMP black holes
are often caused by over-zealous security administrators who block are often caused by over-zealous security administrators who block
all ICMP messages. It is vitally important that those who design and all ICMP messages. It is vitally important that those who design and
deploy security systems understand the impact of strict filtering on deploy security systems understand the impact of strict filtering on
upper-layer protocols. The safest web site in the world is worthless upper-layer protocols. The safest web site in the world is worthless
if most TCP implementations cannot transfer data from it. It would if most TCP implementations cannot transfer data from it. It would
be far nicer to have all of the black holes fixed rather than fixing be far nicer to have all of the black holes fixed rather than fixing
all of the TCP implementations. all of the TCP implementations.
5. Acknowledgements 5. Acknowledgements
Thanks to Mark Allman and Vern Paxson for generous help reviewing the Thanks to Mark Allman, Vern Paxson, and Jamshid Mahdavi for generous
document, and to Matt Mathis for early suggestions of various mecha- help reviewing the document, and to Matt Mathis for early suggestions
nisms that can cause PMTUD black holes, as well as review. The of various mechanisms that can cause PMTUD black holes, as well as
structure for describing TCP problems, and the early description of review. The structure for describing TCP problems, and the early
that structure is from [RFC2525]. Special thanks to Amy Bock, who description of that structure is from [RFC2525]. Special thanks to
helped perform the PMTUD tests which discovered these bugs. Amy Bock, who helped perform the PMTUD tests which discovered these
bugs.
6. References 6. References
[RFC2581] [RFC2581]
M. Allman, V. Paxson, and W. Stevens, "TCP Congestion Control", M. Allman, V. Paxson, and W. Stevens, "TCP Congestion Control",
April 1999. April 1999.
[RFC1122] [RFC1122]
R. Braden, Editor, "Requirements for Internet Hosts -- Communica- R. Braden, Editor, "Requirements for Internet Hosts --
tion Layers", Oct. 1989. Communication Layers," Oct. 1989.
[Jacobson89] [Jacobson89]
V. Jacobson, C. Leres, and S. McCanne, tcpdump, available via V. Jacobson, C. Leres, and S. McCanne, tcpdump, available via
anonymous ftp to ftp.ee.lbl.gov, Jun. 1989. anonymous ftp to ftp.ee.lbl.gov, Jun. 1989.
[RFC1435] [RFC1435]
S. Knowles, "IESG Advice from Experience with Path MTU Discovery", S. Knowles, "IESG Advice from Experience with Path MTU Discovery,"
March 1993. March 1993.
[RFC1191] [RFC1191]
J. Mogul and S. Deering, "Path MTU discovery", Nov. 1990. J. Mogul and S. Deering, "Path MTU discovery," Nov. 1990.
[RFC1981] [RFC1981]
J. McCann, S. Deering & J. Mogul, "Path MTU Discovery for IP ver- J. McCann, S. Deering & J. Mogul, "Path MTU Discovery for IP
sion 6", August 1996 version 6", August 1996.
[Paxson96] [Paxson96]
V. Paxson, "End-to-End Routing Behavior in the Internet", IEEE/ACM V. Paxson, "End-to-End Routing Behavior in the Internet", IEEE/ACM
Transactions on Networking (5), pp.~601-615, Oct. 1997. Transactions on Networking (5), pp.~601-615, Oct. 1997.
[RFC2525] [RFC2525]
V. Paxon, Editor, M. Allman, S. Dawson, W. Fenner, J. Griner, I. V. Paxon, Editor, M. Allman, S. Dawson, W. Fenner, J. Griner, I.
Heavens, K. Lahey, J. Semke, and B. Volz, "Known TCP Implementation Heavens, K. Lahey, J. Semke, and B. Volz, "Known TCP Implementation
Problems", March 1999. Problems", March 1999.
[RFC879] [RFC879]
J. Postel, "The TCP Maximum Segment Size and Related Topics", J. Postel, "The TCP Maximum Segment Size and Related Topics,"
November, 1983. November, 1983.
[RFC2001] [RFC2001]
W. Stevens, "TCP Slow Start, Congestion Avoidance, Fast Retransmit, W. Stevens, "TCP Slow Start, Congestion Avoidance, Fast Retransmit,
and Fast Recovery Algorithms", Jan. 1997. and Fast Recovery Algorithms," Jan. 1997.
7. Author's Address 6.1. Author's Address
Kevin Lahey <kml@novell.com> Kevin Lahey <kml@logictier.com>
Novell, Inc. LogicTier, Inc.
2211 N. First St. Suite 100
San Jose, CA 95131 2 Waters Park Drive
San Mateo, CA 94403
USA USA
Phone: +1 650/604-4334 Phone: +1 650/678-7033
8. Full Copyright Statement 7. Full Copyright Statement
Copyright (C) The Internet Society (1999). All Rights Reserved. Copyright (C) The Internet Society (1999). All Rights Reserved.
This document and translations of it may be copied and furnished to This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing document itself may not be modified in any way, such as by removing
skipping to change at page 15, line 42 skipping to change at page 16, line 5
The limited permissions granted above are perpetual and will not be The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns. revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
This draft was created in August 1999. This draft was created in May 2000.
It expires in January 2000. It expires in October 2000.
 End of changes. 48 change blocks. 
123 lines changed or deleted 127 lines changed or added

This html diff was produced by rfcdiff 1.33. The latest version is available from http://tools.ietf.org/tools/rfcdiff/