[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: (draft-mathis-pmtud-method) 00 01 02 03 04 05 06 07 08 09 10 11 RFC 4821

Network Working Group                                          M. Mathis
Internet-Draft                                                J. Heffner
Expires: November 30, 2004                                           PSC
                                                                K. Lahey
                                                               Freelance
                                                               June 2004


                           Path MTU Discovery
                       draft-ietf-pmtud-method-02

Status of this Memo

   By submitting this Internet-Draft, I certify that any applicable
   patent or other IPR claims of which I am aware have been disclosed,
   and any of which I become aware will be disclosed, in accordance with
   RFC 3668.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that other
   groups may also distribute working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time. It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at http://
   www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on November 30, 2004.

Copyright Notice

   Copyright (C) The Internet Society (2004). All Rights Reserved.

Abstract

   This document describes a robust new method for Path MTU Discovery
   that relies on TCP or other Packetization Layer to probe an Internet
   path with progressively larger packets. This method is described as
   an extension to RFC 1191 and RFC 1981, which specify ICMP based Path
   MTU Discovery for IP versions 4 and 6, respectively. This document
   does not define a protocol, but rather a method to use features of
   existing protocols to discover the path MTU.




Mathis, et al.         Expires November 30, 2004                [Page 1]

Internet-Draft             Path MTU Discovery                  June 2004


   The general strategy of the new algorithm is to start with a small
   MTU and probe upward, testing successively larger MTUs by probing
   with single packets.  If the probe is successfully delivered, then
   the MTU is raised.  If the probe is lost, it is treated as an MTU
   limitation and not as a congestion signal.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  3
   3.  Overview . . . . . . . . . . . . . . . . . . . . . . . . . . .  6
   4.  Requirements . . . . . . . . . . . . . . . . . . . . . . . . .  9
   5.  Implementation Issues  . . . . . . . . . . . . . . . . . . . . 10
     5.1   Layering . . . . . . . . . . . . . . . . . . . . . . . . . 10
       5.1.1   Accounting for Header Sizes  . . . . . . . . . . . . . 10
       5.1.2   Storing PMTU information . . . . . . . . . . . . . . . 11
     5.2   Lower Layers . . . . . . . . . . . . . . . . . . . . . . . 12
       5.2.1   Generating Probes  . . . . . . . . . . . . . . . . . . 12
       5.2.2   Selecting the initial MTU  . . . . . . . . . . . . . . 14
       5.2.3   Normal sequence of events to raise the MTU . . . . . . 14
       5.2.4   Processing MTU Indications . . . . . . . . . . . . . . 15
       5.2.5   Probing Intervals  . . . . . . . . . . . . . . . . . . 20
       5.2.6   Host fragmentation . . . . . . . . . . . . . . . . . . 21
       5.2.7   Multicast  . . . . . . . . . . . . . . . . . . . . . . 22
     5.3   Search Strategy  . . . . . . . . . . . . . . . . . . . . . 22
       5.3.1   Search . . . . . . . . . . . . . . . . . . . . . . . . 23
       5.3.2   Monitor  . . . . . . . . . . . . . . . . . . . . . . . 24
       5.3.3   Suspend  . . . . . . . . . . . . . . . . . . . . . . . 24
     5.4   Specific Packetization Layers  . . . . . . . . . . . . . . 24
       5.4.1   Probing method using TCP . . . . . . . . . . . . . . . 24
       5.4.2   Probing method using SCTP  . . . . . . . . . . . . . . 25
       5.4.3   Probing Method for IP Fragmentation  . . . . . . . . . 27
       5.4.4   Issues for other transport protocols . . . . . . . . . 27
     5.5   Operational Integration  . . . . . . . . . . . . . . . . . 27
       5.5.1   Interoperation with prior algorithms . . . . . . . . . 27
       5.5.2   Interoperation over subnets with dissimilar MTUs . . . 28
       5.5.3   Interoperation with tunnels  . . . . . . . . . . . . . 28
       5.5.4   Diagnostic tools . . . . . . . . . . . . . . . . . . . 29
       5.5.5   Management interface . . . . . . . . . . . . . . . . . 29
   6.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 30
   6.1   Normative References . . . . . . . . . . . . . . . . . . . . 30
   6.2   Informative References . . . . . . . . . . . . . . . . . . . 31
       Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 32
   A.  Security Considerations  . . . . . . . . . . . . . . . . . . . 32
   B.  IANA considerations  . . . . . . . . . . . . . . . . . . . . . 32
   C.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 33
       Intellectual Property and Copyright Statements . . . . . . . . 34




Mathis, et al.         Expires November 30, 2004                [Page 2]

Internet-Draft             Path MTU Discovery                  June 2004


1.  Introduction

   This document describes a method for Packetization Layer Path MTU
   Discovery (PLPMTUD) which is an extension to existing Path MTU
   discovery methods as described in RFC 1191 [2] and RFC 1981 [3].  The
   proper MTU is determined by starting with small packets and probing
   with successively larger packets.  The bulk of the algorithm is
   implemented above IP, in the transport layer (e.g. TCP) or other
   "Packetization Protocol" that is responsible for determining packet
   boundaries.

   This document draws heavily RFC 1191 [2] and RFC 1981 [3] for
   terminology, ideas and some of the text.

   The methods described in this document apply both IPv4 and IPv6, and
   many transport protocols.   This document does not define a protocol,
   but rather a method to use features of existing protocols to discover
   the path MTU.  It does not require cooperation from the lower layers
   (except that they are consistent about what packet sizes are
   acceptable) or the far node.  Variants in implementations will not
   cause interoperability problems.

   The methods described in this document are carefully designed to
   maximize robustness in the presence of less than ideal
   implementations of other protocols or Internet components.

   For sake of clarity we uniformly prefer TCP and IPv6 terminology.  In
   the terminology section we also present the analogous IPv4 terms and
   concepts for the IPv6 terminology.  In a few situations we describe
   specific details that are different between IPv4 and IPv6.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [4].

   This draft is a product of the Path MTU Discovery (pmtud) working
   group of the IETF.  Please send comments and suggestions to
   pmtud@ietf.org.   Interim drafts and other useful information will be
   posted at http://www.psc.edu/~mathis/MTU/pmtud/index.html .

2.  Terminology
   IP Either IPv4 [1] or IPv6 [7].

   node A device that implements IP.







Mathis, et al.         Expires November 30, 2004                [Page 3]

Internet-Draft             Path MTU Discovery                  June 2004


   router A node that forwards IP packets not explicitly addressed to
      itself.

   host Any node that is not a router.

   upper layer A protocol layer immediately above IP. Examples are
      transport protocols such as TCP and UDP, control protocols such as
      ICMP, routing protocols such as OSPF, and Internet or lower-layer
      protocols being "tunneled" over (i.e., encapsulated in) IP such as
      IPX, AppleTalk, IP itself.

   link A communication facility or medium over which nodes can
      communicate at the link layer, i.e., the layer immediately below
      IP. Examples are Ethernets (simple or bridged); PPP links; X.25,
      Frame Relay, or ATM networks; and Internet (or higher) layer
      "tunnels", such as tunnels over IPv4 or IPv6. Occasionally we use
      the slightly more general term "lower layer" for this concept.

   interface A node's attachment to a link.

   address An IP-layer identifier for an interface or a set of
      interfaces.

   packet An IP header plus payload.

   MTU Maximum Transmission Unit, the size in bytes of the largest IP
      packet, including the IP header and payload, that can be
      transmitted on a link or path. Note that this could more properly
      be called the IP MTU, to be consistent with how other standards
      organizations use the acronym MTU.

   link MTU The Maximum Transmission Unit, i.e., maximum IP packet size
      in bytes, that can be conveyed in one piece over a link. Beware
      that this definition differers from the definition used by other
      standards organizations.

      For IETF documents, link MTU is uniformly defined as the IP MTU
      over the link. This includes the IP header, but excludes link
      layer headers and other framing which is not part of IP or the IP
      payload.

      Beware that other standards organizations generally define link
      MTU to include the link layer headers.

   path The set of links traversed by a packet between a source node and
      a destination node





Mathis, et al.         Expires November 30, 2004                [Page 4]

Internet-Draft             Path MTU Discovery                  June 2004


   PMTU, path MTU The minimum link MTU of all the links in a path
      between a source node and a destination node.

   classical PMTU discovery, Process described in RFC 1191 and RFC 1981,
      in which nodes rely on ICMP "Packet Too Big" messages to learn the
      MTU of a path.

   PL, packetization layer The layer of the network stack which segments
      data into packets.

   PLPMTUD Packetization Layer Path MTU Discovers, the method described
      in this document, which is an extension to classical PMTU
      discovery.

   Packet Too Big message An ICMP message reporting that an IP packet is
      too large to forward. This is the IPv6 term that corresponds to
      the IPv4 "ICMP Can't fragment" message.

   flow A context in which MTU discovery is applied. This is naturally
      an instance of the packetization protocol, e.g. one side of a TCP
      connection.

   MPS The maximum IP payload size available over a specific path. This
      is typically the path MTU minus the IP header. As an example, this
      is the maximum TCP packet size, including TCP payload and headers
      but not including IP headers. This has also been called the "L3
      MTU".

   MSS The TCP Maximum Segment Size, the maximum payload size available
      to the TCP layer. This is typically the path MPS minus the size of
      the TCP header.

   probe packet A packet which is being used to test a path for a larger
      MTU.

   probe size The size of a packet being used to probe for a larger MTU.

   successful probe The probe packet was delivered through the network
      and acknowledged by the Packetization Layer on the far node.

   inconclusive probe The probe packet was not delivered, but there were
      other lost packets close enough to the probe where it can not be
      presumed that the probe was lost because it was larger than the
      path MTU. By implication the probe might have been lost due to
      something other than MTU (such congestion), so the results are
      inconclusive.  Inconclusive probes are generally repeated at the
      same probe size, after a suitable delay.




Mathis, et al.         Expires November 30, 2004                [Page 5]

Internet-Draft             Path MTU Discovery                  June 2004


   failed probe The probe packet was not delivered and there were no
      other lost packets close to the probe. This is taken as an
      indication that the probe was larger than the path MTU, and future
      probes should generally be for at smaller sizes.

   errored probe There were losses or timeouts during the verification
      phase which suggest a potentially disruptive failure or network
      condition. These are generally retried only after substantially
      longer intervals.

   probe gap The payload data that will be lost and need to be
      retransmitted if the probe is not delivered.

   probe phase The interval (time or protocol events) between when a
      probe is sent, and when it is determined that the the probe
      succeeded, failed or was inconclusive

   verification phase An additional interval during which the new path
      MTU is considered provisional. Packet losses or timeouts are
      treated as an indication that there may be a problem with the
      provisional MTU.

   Transition phase The interval between the probe phase and the
      verification phase, during which packets using the new MTU
      propagate to the far node and the acknowledgment propagates back.

   full stop timeout a timeout where none of the packets transmitted
      after some event are acknowledged by the receiver, including any
      retransmissions. This is taken as an indication of some failure
      condition in the network, such as a routing change onto a link
      with a smaller MTU.   For the sake of PLPMTUD we suggest the
      following definition of a full stop timeout:  the loss of one full
      window of data and at least one retransmission or at least 6
      consecutive packets including at least 2 retransmissions (along
      with two retransmission timer expirations).   [@@@ This probably
      needs some experimentation.]

   search strategy the heuristics used to choose successive probe sizes
      to converge to the proper path MTU, as described in section 5.5.


3.  Overview

   This document describes a method for TCP or other packetization
   protocols to dynamically discover the MTU of a path without relying
   on explicit signals from the network. These procedures are applicable
   to TCP and other transport- or application-level packetization
   protocols in which the receiver always reports to the sender complete



Mathis, et al.         Expires November 30, 2004                [Page 6]

Internet-Draft             Path MTU Discovery                  June 2004


   information about which packets were lost in the network.

   The general strategy of the new procedure is for the packetization
   layer to find the proper MTU by probing with progressively larger
   packets, without disrupting its normal protocol operation. If a probe
   packet is successfully delivered, then the path MTU is provisionally
   raised. If there are no additional losses during the subsequent
   verification phase, then the path MTU is confirmed (verified) to be
   at least as large as the provisional MTU. PLPMTUD can then probe
   again with an even larger MTU, according to MTU search strategy
   described in Section 5.3.

   The verification phase is used to detect some situations where
   raising the MTU raises the packet loss rate.  For example if a link
   is striped across multiple physical channels with inconsistent MTUs,
   it is possible that a probe will be delivered even if it is too large
   for some of the physical channels. In such cases raising the path MTU
   to the probe size will cause severe periodic loss and abysmal
   performance.  The verification phase is designed to prevent the path
   MTU from being raised if doing so causes excessive packet losses.

   A conservative implementation of PLPMTUD would use a full round trip
   time for the verification phase.  In this case each time PLPMTUD
   raises the MTU it takes three full round trip times to do so. It
   takes one round trip for the probe phase, during which the probe
   propagates to the far node and an acknowledgment is returned.   The
   second round trip is the transitional phase, during which data
   packets using the provisional MTU propagate to the far node and are
   acknowledged. During he third and final round trip time, it is
   verified that raising the MTU does not cause excessive loss.

   The isolated loss of a probe packet (with or without a Packet Too Big
   message) is treated as an indication of an MTU limit, and not as a
   congestion indicator. In this case alone, the packetization protocol
   is permitted to retransmit the probe gap without adjusting the
   congestion window.

   If there is a timeout or any additional lost packets during any of
   the three phases, the loss is treated as a congestion indication as
   well as an indication of some sort of failure of the PLPMTUD process.
   The congestion indication is treated like any other congestion
   indication: window or rate adjustments are mandatory per the relevant
   congestion control standards [8].   Probing can resume with some new
   probe size after a delay which is determined by the nature of the
   indicated failure.

   The most likely (and least serious) PLPMTUD failure is the link
   experiencing legitimate congestion related losses at about the same



Mathis, et al.         Expires November 30, 2004                [Page 7]

Internet-Draft             Path MTU Discovery                  June 2004


   time as the probe.   In this case, it is appropriate to retry the
   probe (with the same probe size) as soon as the packetization layer
   has fully adapted to the congestion and recovered from the losses.

   In other cases, additional losses or timeouts indicate problems with
   the link or packetization layer, and that probes may be disruptive.
   In these situations it is desirable to use progressively longer
   delays depending on the severity of the failure and if it persists.

   PLPMTUD can optionally process Packet Too Big messages to select the
   provisional MTU for faster convergence in exchange for a slight
   decrease in robustness.  Processing malicious or erroneous Packet Too
   Big messages can cause PLPMTUD to arrive at the incorrect MTU for a
   path, which is likely to reduce protocol performance. There are
   several different options for processing Packet Too Big messages: in
   one extreme they could be completely ignored, in the other extreme,
   accept all of them (fully implementing classic PMTUD within PLPMTUD).
   We advocate a compromise, where Packet Too Big messages are only
   processed in conjunction with probes (described in Section 5.2.4.1),
   and Packetization Layer timeouts (described in Section 5.2.4.3).

   Relatively few details of this procedure affect interoperability with
   other standards or Internet protocols.  These details are specified
   in RFC2119 standards language in Section 4.

   Most of the difficulty in implementing PLPMTUD arises because it
   needs to be implemented in several different places within a single
   node.  In general each packetization protocol needs to have it's own
   implementation of PLPMTUD. Furthermore, the natural mechanism to
   share path MTU information between concurrent or subsequent
   connections over the same path is a path information cache in the IP
   layer.  The various packetization protocols need to have the means to
   access and update the shared cache in the IP layer. This memo
   describes PLPMTUD in terms of its primary subsystems without fully
   describing how they are assembled into a complete implementation.
   Section 5 describes: the separation into layers, the mechanics of
   probing from the point of view other lower layers, Maximum Payload
   Size search heuristics; implementation in specific Packetization
   Layers; and operational integration issues.

   The vast majority of the implementation details are recommendations
   based on experiences with earlier versions of path MTU discovery.
   These are motivated by a desire to maximize robustness of PLPMTUD in
   the presence of less than ideal implementations as they exist in the
   field.






Mathis, et al.         Expires November 30, 2004                [Page 8]

Internet-Draft             Path MTU Discovery                  June 2004


4.  Requirements

   All Internet nodes SHOULD implement PLPMTUD in order to discover and
   take advantage of the largest MTU supported along the Internet path.

   Links MUST NOT deliver packets that are larger than their MTU. Links
   that have parametric limitations (e.g. MTU bounds due to limited
   clock stability) MUST include explicit mechanisms to consistently
   reject packets that might otherwise be nondeterministically
   delivered.

   All hosts SHOULD use IPv4 fragmentation in a mode that mimics IPv6
   functionality.  All fragmentation SHOULD be done on the host, and all
   IPv4 packets, including fragments, SHOULD have the DF bit set such
   that they will not be fragmented (again) in the network.  See Section
   5.2.6

   The requirements below only apply to those implementations that
   include PLPMTUD.

   If the Packetization Layer uses application data to implement PLPMTUD
   it MUST use a loss reporting mechanism mechanism (e.g. TCP SACK)
   which avoids spurious retransmission of other data when a probe
   packet is lost.

   A Packetization Layer using application data for probes MUST NOT send
   a probe unless it has sufficient following data available to send
   such that a lost probe will trigger Fast Retransmit or similar data
   recovery algorithm.

   A Packetization Layer using application data for probes SHOULD NOT
   send a probe packet unless the flow is expected to have at least the
   3 round trips worth of data needed to successfully complete the
   probe, transition and verification phases.

   Normal congestion control algorithms MUST remain in effect under all
   conditions except when only an isolated probe packet is detected to
   be lost. In this case alone the normal congestion (window or data
   rate) reduction can be suppressed.  If any other lost data is
   detected, all normal congestion control MUST take place.

   When a probe is lost and normal congestion control is suppressed as
   permitted above, then the Packetization Layer MUST NOT probe again
   until at least an interval equal to the normal congestion control
   cycle.  For TCP and TCP friendly protocols this generally means one
   round trip of elapsed time for each packet permitted under the
   current congestion window.




Mathis, et al.         Expires November 30, 2004                [Page 9]

Internet-Draft             Path MTU Discovery                  June 2004


   If PLPMTUD updates the MTU for a particular path, all Packetization
   Layer sessions that share the flow (path) must be notified.

   Whenever the MTU is raised, the congestion state variables must be
   rescaled to not to raise the window size in bytes (or date rate in
   bytes per seconds).

   Whenever the MTU is reduced (e.g. when unconditionally processing
   ICMP Packet Too Big messages) the congestion state variable must be
   rescaled not to raise the window size in packets.

   All implementations MUST include a mechanism to implement diagnostic
   tools that do not rely on the operating systems implementation of
   path MTU discovery.   This specifically requires the ability to send
   packets that are larger than the known MTU for the path, and
   collecting any resultant ICMP error message. See Section 5.5.4

5.  Implementation Issues

   This section discusses a number of issues related to the
   implementation of Path MTU Discovery.  This is not a specification,
   but rather a set of notes provided as an aid for implementers.

   The issues include:
   o  The seperation into layers
   o  The Mechanics of Probing, as seen by IP and brlow
   o  Search Strategy.
   o  How to implement PLPMTUD in specific Packetization Layers.
   o  How to improve Operational Integration and deployment.

5.1  Layering

5.1.1  Accounting for Header Sizes

   Packetization Layer Path MTU Discovery is most easily implemented by
   splitting its functions between layers.  The IP layer is in the best
   place to keep shared state, collect the ICMP messages, track IP
   headers sizes and manage MTU information from the link layer
   interfaces.  However the procedures that PLPMTUD uses for probing,
   verifications and scanning for the path MTU are very tightly coupled
   to the data recovery and congestion control state machines in the
   Packetization Layer.   The most difficult part of implementing
   PLPMTUD is properly splitting the implementation between the layers.

   Note that this layering is constant with the advice in the current
   PMTUD specifications [2][3]. Today, many implementations of classical
   PMTU Discovery are already split along these same layers.




Mathis, et al.         Expires November 30, 2004               [Page 10]

Internet-Draft             Path MTU Discovery                  June 2004


   Early implementation of PLPMTUD revealed that it is critically
   important to have a good clean mechanism for accounting header sizes
   at all layers.  This is because each Packetization Layer does its
   calculations in its own natural data unit, which are almost always a
   reflection of the service that the Packetization Layer provides to
   the application or other upper layers.  For example, TCP naturally
   performs all of its calculations in terms of sequence numbers and
   segment sizes.   The size of the Probe gap is the size of the data
   segment that was that was carried by the probe packet. However, the
   MTU size being probed, ICMP MTU, etc are measures of full packets,
   which not only include the TCP data (measured in sequence space) but
   also include fixed TCP and IP headers, and may include IPv6 extension
   headers or IPv4 options, TCP options and even IPsec AH or ESP headers
   as well.

   PLPMTUD requires frequent translation between these two domains: the
   Packetization Layer's natural data unit and full IP packet sizes.
   While there are a number of possible ways to accurately implement
   dual size measures, our experience has been that it is best if the
   boundary between the IP layer and the Packetization layer communicate
   in terms of the IP Maximum Payload Size or MPS.  The MPS is the only
   size measure that is common to both the IP and Packetization Layers,
   because it exactly matches the boundary between the layers.  The IP
   Layer is responsible for adding or deducting it's own headers when
   translating between MTU and MPS.  Likewise the Packetization Layer is
   responsible for adding or deducting its own headers when calculations
   in it's natural data units.

   This document does not take a stance on the placement of IPsec, which
   logically sits between IP and the Packetization Layer. As far as
   PLPMTUD is concerned IPsec can be treated either as part of IP or as
   part of the Packetization Layer, as long as the accounting is
   consistent within any given implementation.  If IPsec is treated as
   part of the IP layer, then each security association to a remote node
   may need to be treated as a separate flow for PLPMTUD, if they have
   different length security headers. If IPsec is treated as part of the
   packetization layer, the IPsec header size has to be included in the
   Packetization Layer's header size calculations.

5.1.2  Storing PMTU information

   This memo uses the concept of a "flow" to define the scope in which
   path MTU information is used.  Each flow locally stores its maximum
   payload size (MPS), which is used for packetizing data.
   Packetization Layers may communicate with the IP layer to store or
   access cached MPS values, providing a means by which similar flows
   may share information. The IP layer also stores PMTU and derived MPS
   information when it receives Packet Too Big messages.



Mathis, et al.         Expires November 30, 2004               [Page 11]

Internet-Draft             Path MTU Discovery                  June 2004


   Ideally, a PMTU value should be associated with a specific path
   traversed by packets exchanged between the source and destination
   nodes.  However, in most cases a node will not have enough
   information to completely and accurately identify such a path.
   Rather, a node must associate a PMTU value with some local
   representation of a path.  It is left to the implementation to select
   the local representation of a path.

   An implementation could use the destination address as the local
   representation of a path.  The PMTU value associated with a
   destination would be the minimum PMTU learned across the set of all
   paths in use to that destination.  The set of paths in use to a
   particular destination is expected to be small, in many cases
   consisting of a single path.  This approach will result in the use of
   optimally sized packets on a per-destination basis.  This approach
   integrates nicely with the conceptual model of a host as described in
   [ND@@@@]: a PMTU value could be stored with the corresponding entry
   in the destination cache.   However, NAT and other forms of middle
   boxes may exhibit differing MTUs at as single IP address.

   If IPv6 flows are in use, an implementation could use the IPv6 flow
   id [7][14] as the local representation of a path.  Packets sent to a
   particular destination but belonging to different flows may use
   different paths, with the choice of path depending on the flow id.
   This approach will result in the use of optimally sized packets on a
   per-flow basis, providing finer granularity than PMTU values
   maintained on a per-destination basis.

   For source routed packets (i.e. packets containing an IPv6 Routing
   header, or IPv4 LSRR or SSRR options), the source route may further
   qualify the local representation of a path.    An implementation
   could use source route information in the local representation of a
   path.

   If IPsec is in use, the security association can also be used to
   represent a path.

5.2  Lower Layers

5.2.1  Generating Probes

   A new candidate MTU is tested by sending one "probe packet", which is
   larger than the current MTU.  In this section we present a couple of
   possible ways to alter packetization layers to generate probe
   packets.   The different techniques incur different overheads in
   three areas: difficulty in generating the probe packet (in terms of
   packetization layer implementation complexity and computational
   overhead) possible additional network capacity consumed by the probes



Mathis, et al.         Expires November 30, 2004               [Page 12]

Internet-Draft             Path MTU Discovery                  June 2004


   and the overhead of recovering from failed probes (both network and
   protocol overhead).

   For example some protocols might be extended to allow padding with
   dummy data within their packets.  This would greatly simplify the
   implementation because the probing can be performed without
   participation from the application and if the probe fails, the
   missing data (the "probe gap") is assured to fit within the current
   MTU when it is retransmitted. However, the padding does consume
   network capacity without carrying any useful payload.

   This technique does not work for TCP, because there is not a separate
   length field or other mechanism to differentiate between padding and
   real payload data. With TCP the natural approach is to send
   additional payload data in an over-sized segment.   There are several
   variants which have different tradeoffs.

   In one method, after a TCP probe segment has been sent the subsequent
   segment(s) may be sent as though the probe segment was not
   over-sized.  Thus if the probe segment is lost, it will leave a gap
   in the sequence space that is exactly the correct size to be filled
   by one segment at the current MTU.   Since this method generates
   overlapping data, it will cause duplicate acknowledgments if the
   probe is successfully delivered.  The sender must be capable of
   ignoring these expected duplicate acknowledgments in a manner which
   will not cause unnecessary retransmission or congestion window
   reduction.

   In the second method, after a TCP probe segment has been sent,
   subsequent TCP segments are sent in a non-overlapping manner.  If the
   probe segment is lost, it will leave a gap which will require
   retransmission of multiple segments to fill. This method has lower
   overhead for successful probes, but it requires more complexity in
   the retransmit logic to correctly retransmit the missing data (the
   "probe gap") with multiple segments that fit into the old MTU, while
   properly suppressing the congestion adjustments for this one
   situation and no others.

   Several Packetization protocols may be best served by using an
   adjunct protocol for MTU probing: a separate protocol (or protocol
   feature) that does not carry and real application data.  This greatly
   simplify s implementation because nothing needs to be retransmitted
   when the probe is lost, but it does consume network capacity without
   delivering any useful payload.

   Two important example of this come to mind:  SCTP [9] which might use
   its existing HEARTBEAT facility padded with dummy data to fill out
   the probe packet; and IP fragmentation which is sometimes used as a



Mathis, et al.         Expires November 30, 2004               [Page 13]

Internet-Draft             Path MTU Discovery                  June 2004


   Packetization layer for carrying oversized datagrams as described in
   Section 5.2.6. In the case of IP fragmentation an entire separate
   protocol in need, that has to use the diagnostic interface described
   in Section 5.5.4

   It should be clear that nearly all packetization layers can be
   adapted to support PLPMTUD, possibly in more than one way.

5.2.2  Selecting the initial MTU

   When the PLPMTUD process is started the initial MTU should normally
   be set such that the Packetization Layer can carry 1 kByte data
   segments.    This initial MTU should be 1 kByte plus space for IP and
   Packetization layer headers. (see Section 5.1 on accounting for
   headers).   With the this MTU, RFC2414 [6] allows TCP and other
   transport protocols to start with an initial window of 4 packets.

   We suspect, but have not confirmed that TCP actually starts faster
   (and completes sooner for small packets) with 1kB packets rather than
   1500 byte packets because the 2nd data ACK occurs one round trip
   earlier

   This initial MTU should also be configurable.    One of the
   configuration options should be to set it to default to the
   interfaces MTU, to mimic classical PMTUD behavior. (See Section 5.5.1

5.2.3  Normal sequence of events to raise the MTU

   If the probe size is smaller than the actual path MTU and there are
   no other losses, the normal sequence of events to probe and raise the
   MTU will be:
   1.  The probe is sent, followed by more packets at the current MTU.
       By definition PLPMTUD enters the probe phase.   The probe
       propagates through the network and the far node acknowledges it
       (or possibly latter data, if acknowledgements are cumulative and
       delayed acknowledgement is in effect).

   2.  The acknowledgement for the probe reaches the data sender.   By
       definition, this ends the probe phase.

   3.  The packetization layer provisionally raises the MTU to the probe
       size. PLPMTUD enters the transitional phase when it starts
       sending data using the provisional MTU.

       Note that implementations that use packet counts for congestion
       accounting (e.g. keep cwnd in units of packets) must re-scale
       their congestion accounting such that raising the MTU does not
       raise the data rate (bytes/second) or the total congestion window



Mathis, et al.         Expires November 30, 2004               [Page 14]

Internet-Draft             Path MTU Discovery                  June 2004


       in bytes.

       If the implementation packetizes the data at the application
       programming interface, it may transmit already queued data at the
       current MTU before raising the MTU. In this case this data is not
       part of either the probing or transition phases, because all of
       the packets in flight fit within the current MTU.

   4.  Once the first packet of the transitional phase is acknowledged,
       PLPMTUD enters the verification phase.   In principle the
       verification phase can be of arbitrary duration, however at this
       time we are recommending one full window of data (i.e one full
       round trip time) for most Packetization Layers.

   5.  Once there has been sufficient data delivered and acknowledged in
       the provisional MTU is considered verified and the path MTU is
       updated.   PLPMTUD can then probe for an even larger MTU, as
       described in the searching strategy in Section 5.3.

   Other events described in the next section are treated as exceptions
   and alter or cancel some of the steps above.

5.2.4  Processing MTU Indications

   The descriptions below assume that the Packetization Layer protocol
   that has a TCP fast retransmit style mechanism to synchronously
   detect the loss of a probe packet and trigger retransmission, without
   loss of the protocols self clock.  If this fails, then some sort of
   retransmission timeout will serve to catch the loss.    It also
   assumes that there is some mechanism to detect full-stop timeouts.

   If any of these events (or the receipt of an ICMP Packet Too Big
   message) occurs during the the above process to raise the MTU, then
   it is processed as indicated in the following sections.

5.2.4.1  Processing Packet Too Big Messages

   Classical PMTU discovery specifies the generation of Packet Too Big
   Messages if an over-sized packet (e.g. a probe) encounters a link
   that has a smaller MTU. Since these messages can not be authenticated
   they introduce a number of well documented attacks against classical
   PMTUD [5].

   With PLPMTUD these messages are not required for correct operation,
   and in principle can be summarily ignored at the expense of slower
   convergence to the proper MTU.   However we believe that a slightly
   better compromise is to process Packet too big messages in two
   specific contexts: in conjunction with a PLPMTUD probe or a full-stop



Mathis, et al.         Expires November 30, 2004               [Page 15]

Internet-Draft             Path MTU Discovery                  June 2004


   timeout.

   Every Packet Too Big Message should be subjected to the following
   checks:
   o  If globally forbidden then discard the message.

   o  If forbidden by the application then discard the message.

   o  If this path has been tagged "bogus ICMP messages" then discard
      the message.

   o  If the reported MTU fails consistency checks then set "bogus ICMP
      messages" flag for this path and discards the message.  These
      consistency checks include:
      *  unrecognized or unparseable enclosed header,
      *  reported MTU is larger than the size indicated by the enclosed
         header or
      *  larger than the current MTU, provisional MTU or probe size as
         appropriate.
      *  or fails a ICMP consistency checks specific to the
         Packetization Layer. (E.g. The SCTP Verification-Tag mechanism
         [9][16])
      To ease migration, it is suggested that implementations may
      include global controls to suppress some or all of the consistency
      checks.


   If the Packet Too Big Message is acceptable under all of these checks
   do one of two things on depending on a global configuration switch:
   Emulate classical path MTU discovery by processing the message
   immediately (I.e. set the path MTU to the size indicated in the
   message) or save the "ICMP MTU", pending another PLPMTUD event.   In
   this case the saved ICMP MTU will only be acted upon under
   appropriate conditions if there are lost probes, verification packets
   or a full stop timeout.   This greatly reduces the impact of
   fraudulent ICMP Packet Too Big messages.

   In either case if the Packetization Layer calls for specific actions
   in response to a Packet Too Big message, that action should be
   invoked only at the point when the path MTU is updated from the ICMP
   MTU.


5.2.4.2  Packetization Layer Detects Lost Packets

   Each packetization protocol has it's own mechanism to detect lost
   packets and request the retransmission of missing data. The primary
   signals used by the packetization layer are these protocol specific



Mathis, et al.         Expires November 30, 2004               [Page 16]

Internet-Draft             Path MTU Discovery                  June 2004


   loss indications. The packetization layer is responsible for
   retransmitting the lost data and notifying PLPMTUD that there was a
   loss.
   o  If the probe itself was lost, and there were no other losses
      during the probe phase (The RTT between when the probe was sent
      and the loss detected) than it is taken as an indication that the
      path MTU is smaller than the probe size. In this situation alone
      the Packetization Layer is permitted to retransmit the missing
      data (the "probe gap") without adjusting its congestion window or
      data transmission rate.

      If an accepted Packet Too Big Message was received after the probe
      was sent, and it passes the additional checks that the ICMP MTU is
      greater than the current MTU and less than the probe SIZE, then
      set the probe side to the ICMP MTU, and restart the probe process
      from step 1 in Section 5.2.3.

      If there was not a accepted Packet Too Big Message, then the
      indicated event is a "probe failure", which can be retried with a
      smaller probe size after a suitable delay for a probe_fail_event.
      See Section 5.2.4.2 for more complete descriptions of failure
      events.

   o  If there are losses during the probe phase and the probe was not
      lost, then the probe was successful.  However, since additional
      losses have the potential to spoil the verification phase, it is
      important that PLPMTUD not progress into the transition phase
      (step 3 above) until after the Packetization Layer has fully
      recovered from the losses and completed the congestion window (or
      rate) adjustment.

   o  If there are losses during the probe phase and the probe was also
      lost the outcome depends on the presence an ICMP MTU set by an
      acceptable Packet Too Big Message.

      If there was an accepted Packet Too Big Message received since the
      probe was sent, and it passes the additional checks that the ICMP
      MTU is greater than the current MTU and less than the probe size,
      then set the probe size to the ICMP MTU, and once the
      Packetization Layer completes the recovery from the losses then
      restart the probe process from step 1 in Section 5.2.3.

      If there was not an accepted Packet Too big Message, then the
      probe is inconclusive because the lost probe might have been
      caused by congestion.   The probe can be retried  after a suitable
      delay for a probe_inconclusive_event.





Mathis, et al.         Expires November 30, 2004               [Page 17]

Internet-Draft             Path MTU Discovery                  June 2004


   o  It is unlikely that losses during the transition phase are caused
      by PLPMTUD, however they do potentially complicate the
      verification phase.  Note that we are referring to losses that are
      followed by acknowledgement of packets that were sent at the old
      MTU, while the transition to the provisional MTU is still
      propagating through the network.   The first acknowledgement of
      the provisional MTU (and the transition to the verification phase)
      is most likely going to occur during the recovery of the losses in
      transition phase.   It is important that the Packetization Layer
      retransmission machinery distinguish between loses at the old MTU
      (transition phase) and the provisional MTU (the verification
      phase, discussed next).

   o  Losses during the verification phase are taken as a indication
      that the path may have a non-uniform MTU or some other problems
      such that raising the MTU substantially raises the loss rate.  If
      so, this is potentially a very serious problem, so the provisional
      MTU is considered to have errored and the path MTU is set back to
      the previously verified MTU (the previously current MTU).

      Packet loss during the verification phase might also be due to
      coincidental congestion on the path, unrelated to the probe, so it
      would seem to be desirable to re-probe the path. The risk is that
      this effectively raises the tolerated loss threshold because even
      though raising the MTU seemed to cause additional loss, there is a
      statistical chance that repeated attempts to verify a new MTU may
      yield as false pass.    The compromise is to re-probe once with
      the same probe size (after delay probe_inconclusive_event), and if
      this also fails, then the probe may not be retried until after a
      suitable delay for a verification_error_event, which exponentially
      increases on each successive failure.

5.2.4.3  Packetization Layer Retransmission Timeout

   Note that the we do not make distinctions between the various methods
   that different Packetization Layers might use for detecting and
   retransmitting lost packets.   It is preferable that the
   Packetization Layer uses a recovery mechanism similar to TCP SACK or
   fast retransmit (or other "synchronous" loss recover mechanism) to
   detect losses and recover as quickly as possible.

   Under some conditions the Packetization Layer may have to rely on
   retransmission timeouts or other fairly disruptive techniques to
   recover from losses.   Since these greatly increase the cost of
   failed probes, it is recommended that PLPMTUD use even longer delays
   before re-probing. In these situations replace probe_fail_event with
   probe_timeout_event.




Mathis, et al.         Expires November 30, 2004               [Page 18]

Internet-Draft             Path MTU Discovery                  June 2004


5.2.4.4  Packetization Layer Full Stop Timeout

   Under all conditions (not just during MTU probing) a full stop
   timeout should be taken as an indication of some significantly
   disruptive event in the network, such as a router failure or a
   routing change to a path with a smaller MTU.

   If the ICMP MTU is set, and it is less that the current MTU (or
   provisional MTU during the transitional phase), then the path MTU can
   be reduced to the ICMP MTU.   This is the only situation (a full stop
   timeout) outside of a probe that we recommended that the path MTU is
   set from the ICMP MTU. (In Section 5.5.1 we relax this recommendation
   to facilitate migration to PLPMTUD in exchange for slightly less
   protection from corrupt Packet Too Big messages)

   Note that whenever a problem with the path that causes a full-stop
   timeout (also known as a "persistent timeout" in other documents),
   several different path restart/recovery algorithms may be invoked at
   different layers in the stack.  Some device drivers may be restarted
   [@@], router discovery [@@], ES-IS [@@] and so forth.  We recommend
   that in most situation the first action should be to set the path MTU
   down.   Note that this recommendation is really beyond the scope of
   this document, and may require substantial additional research.

   Therefore, if there is a full stop timeout and there was not an ICMP
   message indicating a reason (Packet Too Big, Net unreachable, etc, or
   the ICMP messages was ignored for some reason), we suggest that the
   first recovery action should be to set the path MTU down to a safe
   minimum "restart MTU" value, and the PLPMTUD search state reset, so
   PLPMTUD will start over again searching for the proper MTU. The
   default restart_MTU should be the minimum MTU as specified by IPv4
   (576)[1] or IPv6 (1280) [7] as appropriate, unless overridden by some
   global control (See Section 5.5.5).

   If and only if the full stop timeout happens during the probe or
   transition phases (e.g. after the sending data using the provisional
   MTU but before any of it is acknowledged) is it considered likely
   that raising the MTU caused the full stop timeout.  If so this
   situation is is likely to be cyclic, because resetting the PLPMTUD
   search state is likely to eventually cause re-probing the same
   problematic MTU.

   It is tempting to define additional states to detect recurrent full
   stop timeouts. However in today's hostile network environment, there
   is little tolerance for nodes that are so fragile that they can be
   disrupted by something as simple as oversized packets.  Therefor we
   do not feel that it is worth the overhead of specifying a state
   machine that is capable of automaticly detecting these situations and



Mathis, et al.         Expires November 30, 2004               [Page 19]

Internet-Draft             Path MTU Discovery                  June 2004


   disabling PLPMTUD.   However, it is important that there be a manual
   way to disable or limit probing on specific paths.  See Section
   5.5.5.

5.2.5  Probing Intervals

   Section 5.2.4.2 describes a number of probe failure events.   In all
   cases the basic response is the same: to wait some time interval
   (dependent on the specific event and possibly the history) and then
   to probe again.  For events that are "inconclusive", it is generally
   appropriate to re-probe with the same probe size.   For events that
   are identified as "failed probes" it is generally appropriate to
   re-probe with a smaller probe size.   The search strategy described
   in Section 5.3 is used to select probe sizes.

   Many of the intervals below are specified in terms of elapsed round
   trips relative to the current congestion window.   This is because
   TCP and other Packetization Layer protocols tend to exhibit periodic
   loses which cause periodic variations of the congestion window and
   possibly the data rate.  It is preferable that the PLPMTUD probes are
   scheduled near the low point of these cycles to minimize ambiguities
   caused by congestion losses.

   In order from least to most serious:
   probe_inconclusive_event Other lost packets near the lost probe made
      the probe result ambiguous.   Since the loss of non-probe packets
      requires a window (or data rate) reduction, it is desirable to
      schedule the re-probe (at the same probe size) at one round trip
      time after the end of the loss recovery.   This will be almost the
      minimum congestion window size, with a small cushion to minimize
      the chances that correlated losses caused by some other bursty
      connection spoil another probe.

   probe_fail_event A probe fail event is the one situation under which
      the Packetization layer is permitted not to treat loss as a
      congestion signal.  Because there is some small risk that
      suppressing congestion control might have unanticipated
      consequences (even for one isolated loss), we require that probe
      fail events be less frequent than the normal period for losses
      under standard congestion control.  Specifically after a probe
      fail event and suppressed congestion control, PLPMTUD may not
      probe again until an interval which is comparable to the expected
      interval between congestion control events. See Section 4.

      The simplest estimate of the interval to the next congestion event
      is the same number of round trips as the current window in
      packets.




Mathis, et al.         Expires November 30, 2004               [Page 20]

Internet-Draft             Path MTU Discovery                  June 2004


   probe_timeout_event Since this event was detected by a timeout, it is
      relatively disruptive to protocol operation.   Furthermore, since
      the event indirectly includes a window adjustment that may have
      been caused by the MTU probe, it is important that the probe not
      be repeated until congestion has more than recovered from the
      loss.   Therefore we recommend five times the probe_fail_event
      interval.   I.e. five times as many round trips as the current
      congestion window in packets.

   verification_error_event A verification fail event indicates that a
      probe was deliver and the verification phase failed twice
      separated by a congestion adjustment (so the second verification
      phase was at a low point in the congestion control cycle). This is
      an indication that one of the following three things might have
      happened: repeated losses unrelated to PLPMTUD; the path is
      striped across links with dissimilar MTUs, or the link layer has
      some parametric limitation such that raising the MTU greatly
      increases the random error rate.

      The optimal method responding to this situation is an open
      research question. We believe that the correct response is some
      combination of exponentially lengthening backoffs (e.g. Starting
      at 1 minute and quadrupling on each repeat.) and implicitly
      treating the situation as a probe fail (and choosing a smaller
      probe size) after some threshold number of repeated
      verification_error_events.

5.2.6  Host fragmentation

   Packetization layers are encouraged to avoid sending messages that
   will require fragmentation (for the case against fragmentation, see
   [17][18]).  However this is not always possible. Some packetization
   layers, such as a UDP application outside the kernel, may be unable
   to change the size of messages it sends.  This may result in packet
   sizes that exceeds the Path MTU.

   IPv4 permitted such applications to send packets without DF set.
   Oversized packets without DF would be fragmented in the network or
   sending host when they encountered a link with a small MTU.   In some
   case, packets could be fragmented more than once if there were
   cascaded links with progressively smaller MTUs.

   This approach is no longer recommended.  We now recommend that IPv4
   implementation use a strategy that mimics IPv6 functionality.  When
   an application sends datagrams that are larger than the known path
   MTU they should be fragmented to the path MTU in the host IP layer
   even if they are smaller than the link MTU of the first hop networks
   directly attached to the host.  The DF bit should be set on the



Mathis, et al.         Expires November 30, 2004               [Page 21]

Internet-Draft             Path MTU Discovery                  June 2004


   fragments, so they will not be fragmented again in the network.

   This technique will minimize future surprises as the Internet
   migrated to IPv6. Otherwise there is the potential for widely
   deployed applications or services relying on IPv4 fragmentation, in a
   way that can not be implemented in IPv6. At least one major operating
   system already uses this strategy.

   Note that in principle the IP fragmentation layer is an example of a
   Packetization Layers, it could implement full PLPMTUD in the
   fragmentation process.

5.2.7  Multicast

   In the case of a multicast destination address, copies of a packet
   may traverse many different paths to reach many different nodes.  The
   local representation of the "path" to a multicast destination must in
   fact represent a potentially large set of paths.

   Minimally, an implementation could maintain a single MPS value to be
   used for all packets originated from the node.  This MPS value would
   be the minimum MPS learned across the set of all paths in use by the
   node.  This approach is likely to result in the use of smaller
   packets than is necessary for many paths.

   Alternatively, if the application using multicast gets complete
   delivery reports (unlikely because this requirement  has poor scaling
   properties), PLPMTUD could be implemented in multicast protocols.

5.3  Search Strategy

   The search strategy described here is a only guide for implementors.
   A standard algorithm is not specified because the strategy can
   include many heuristics to optimize MPS selection for a given path.
   Particularly, it may be appropriate for different protocols to follow
   different strategies.  There is opportunity for future improvements
   to this algorithm.

   The search strategy uses three variables:
      SEARCH_MAX is the largest MPS that a flow might be able to use.
      It is determined by such considerations as interface MTU, widths
      of protocol length fields, and possibly other protocol-dependent
      values, such as the the TCP MSS option. In many cases it would be
      the same as the classical MTU discovery initial MSS, minus the IP
      layer headers.
      SEARCH_LOW is the largest validated MPS, and should be used as the
      effective MPS by the packetization layer.   It is the same as the
      current validated MTU minus the IP layer headers.  The initial



Mathis, et al.         Expires November 30, 2004               [Page 22]

Internet-Draft             Path MTU Discovery                  June 2004


      value for SEARCH_LOW should be a parameter, but a value of 1024
      may be a reasonable default.
      SEARCH_HIGH is the least invalidated MPS.   In most cases is will
      be the most recent failed probe size minus the IP layer headers.
      When PLPMTUD is initialized SEARCH_HIGH should be set to
      SEARCH_MAX.

   There are three major states: Search, Monitor and Suspend. In the
   Search state, it incrementally searches for the largest MPS that the
   path can support, narrowing the difference between SEARCH_LOW and
   SEARCH_HIGH. Once this gap is sufficiently narrow, the probing
   algorithm enters the Monitor state where it probes infrequently to
   detect if the path MPS has become larger.

   If the MPS probing is determined harmful, perhaps by persistent probe
   failures, the flow may enter the Suspend state, completely disabling
   MPS probing.

5.3.1  Search

   In the Search state, the strategy follows a multi-phase scan.  If
   SEARCH_HIGH >= SEARCH_MAX, a course scan is used.  In this mode, each
   probe's payload size should be MIN(2 * SEARCH_LOW, SEARCH_MAX).  If
   SEARCH_HIGH < SEARCH_MAX, the fine scan mode should be used.

   The fine scan algorithm may pursue a number of different methods for
   choosing probe sizes.  It may be useful to choose probe sizes so that
   the final IP packet will fit common link MTUs, for example 1500,
   4352, 9000, 17914.  Optionally, probes smaller than these values by
   common tunnel header sizes may be used.

   When using some protocols, the cost for a failed probe may be
   significantly higher than the cost of a successful probe due to
   retransmission and consequent delay jitter as seen by the
   application.  For this reason, one possible approach to the fine scan
   could be to use probes of size SEARCH_LOW + d, for some increment d.
   It should enter the Monitor state when SEARCH_LOW + d >= SEARCH_HIGH.
   This will result in at most one additional probe failure.

   Another approach may be to use a simple binary search where each
   probe size is (SEARCH_LOW + SEARCH_HIGH) / 2, entering the Monitor
   state when SEARCH_LOW + s >= SEARCH_HIGH for some threshold s.  This
   will converge quickly, but may have a higher number of probe
   failures.  It is more appropriate for a protocol whose probes consist
   entirely of padding.






Mathis, et al.         Expires November 30, 2004               [Page 23]

Internet-Draft             Path MTU Discovery                  June 2004


5.3.2  Monitor

   In the Monitor state, a probe of size SEARCH_HIGH should be sent at
   most once every MONITOR_INTERVAL seconds.  If the probe succeeds,
   then SEARCH_HIGH should be set to SEARCH_MAX, and the state should be
   set to Search.

   If there is evidence that no flow traffic is receiving its
   destination, such as repeated timeouts with no acknowledgements in
   TCP, it may be that the connection was re-routed to a path with a
   smaller MTU, and the Packet Too Big messages are ignored of filtered.
   In this case, SEARCH_LOW and SEARCH_HIGH should be set to initial
   values, and the Search state should be entered.

5.3.3  Suspend

   In the Suspend state, probing is entirely disabled, and the MPS
   should be set to 512 bytes.  The Suspend state should only be used if
   it is heuristically determined that probing is causing harmful
   failures.

5.4  Specific Packetization Layers

   In this section we discuss specific implementation issues different
   Packetization Layer protocols.

5.4.1  Probing method using TCP

   TCP has no mechanism that could be used to distinguish between real
   application data and some other form of padding that might be used to
   fill out probe packets.  Therefore, TCP must generate probes by
   sending oversized segments that are carrying real data from upper
   layers.  As previously mentioned there are two approaches that TCP
   might use to minimize the overheads associated with the probing
   process.

   A TCP implementation of PLPMTUD can elect to send subsequent segments
   overlapping the probe as though the probe segment was not oversized.
   This has the advantage that TCP only need to retransmit one segment
   at the current MTU to recover from failed probes. However the
   duplicate data in the probe does consume network resources and will
   cause duplicate acknowledgments.   It is important that these extra
   duplicate acknowledgments not trigger Fast Retransmit.  This can be
   guaranteed by limiting the largest probe segment size to twice the
   current segment size (causing at most 1 duplicate acknowledgment) or
   three times the current segment size (causing at most 2 duplicate
   acknowledgments).




Mathis, et al.         Expires November 30, 2004               [Page 24]

Internet-Draft             Path MTU Discovery                  June 2004


   The other approach is to send non-overlapping segments following the
   probe. Although this is cleaner from a protocol architecture
   standpoint it clashes with many of the optimizations used improve the
   efficiency of data motion withing many operating systems.  In
   particular many implementations divide the data into segments and
   pre-compute checksums as the data is copied out of user space.  In
   these implementation it can be very expensive to adjust segment
   boundaries after the data is already queued.

   If TCP is using SACK or any other variable length headers, the
   headers on the probe and verification packets should be padded to the
   maximum possible length. Otherwise, future options may cause delivery
   problems if they cause IP packets that are larger than the MTU.

   Note that the header size and overhead calculations described in
   Section 5.1 apply here.  TCP's natural data accounting units are
   sequence space and Maximum Segment Size.  However the the PLPMTUD
   process is described in terms of total packet size, which is larger
   than the MSS by all fixed and optional headers.

   At the point when TCP is ready to start the verification phase, it is
   permitted transmit already queued data at the old MTU rather than
   re-packetize it.  This postpones the verification process by the time
   required to send the queued data.

   If the verification phase experiences any segment losses, TCP is
   required to pull back to the prior MSS.   Since failing the
   verification phase should be an infrequent error condition it is less
   important  that this be  as efficient as probing.

5.4.1.1  Window management

   Some TCP implementations keep the congestion window in units of
   segments. When segment size is increased during a connection, a
   conservative implementation should scale cwnd so that, in units of
   bytes, it will remain unchanged.

   It is recommended that TCP should not probe a new MPS if that MPS
   will likely result in a cwnd of less than 5 segments.

   If the network becomes too congested, it is recommended that the MPS
   be reduced to a smaller size as determined by a heuristic.  The
   recommended heuristic is to reduce the MPS by half if ssthresh is
   reduced to 5 segments or smaller, with a minimum MPS of 512 bytes.

5.4.2  Probing method using SCTP

   In the SCTP protocol packetization is the responsibility of the



Mathis, et al.         Expires November 30, 2004               [Page 25]

Internet-Draft             Path MTU Discovery                  June 2004


   application or protocol above SCTP.  The application writes a set
   message to SCTP and SCTP will "chunkify" it into appropriate sized
   pieces. Some implementations MAY bundle multiple data chunks
   together, but this is NOT required implementation behavior. By
   implication not all SCTP implementations can easily generate probes
   sending additional application data. In particular any implementation
   that does not implement data chunk bundling would not be able to
   implement a probe.

   For SCTP the recommended method for generating probes is to pad SCTP
   HeartBeat messages to the desired probed size. A successful probe
   will be acknowledged without delay by the peer SCTP implementation
   returning the same Heartbeat as a HEARTBEAT-ACK. This assures that
   both directions will support the probed MTU size. [@@@@@ note that
   both sides of the path are tested]

   The verification phase is entered after a successful probe. For
   implementations that can bundle multiple DATA chunks the verification
   phase completes when a windows worth of bundled DATA chunks are
   exchanged at the new MTU value. An SCTP implementation SHOULD arrange
   its fragmentation point to be a suitable multiple of the new MTU size
   (e.g. if the MTU size is 1500 bytes in IPv4 then a fragmentation
   point of 718 bytes might be selected during the verification phase.
   This would allow the two bundled DATA chunks to be put together to
   exactly equal the proposed new PMTU. After verification is complete
   the fragmentation point can then be set to the actual PMTU assuming
   that this new value is the smallest MTU of all of the SCTP paths).
   An SCTP implementation is allowed to transmit already fragmented DATA
   chunks that cannot be bundled together at the new MTU value that were
   previously queued. For implementation that do not allow DATA chunk
   bundling three subsequent HEARTBEAT messages should be sent over the
   next XX@@ RTT's padded to the new proposed MTU value. If all of HB's
   are successful then the new PMTU should be adopted for the path.

   [@@@@NOTE: it might be simpler to always use multiple HB's to prove
   in a PMTU during verification, I leave this up to you. One thing to
   keep in mind is that SCTP normally fragments its messages to the
   SMALLEST PMTU of all paths... since SCTP is multi-homed this makes it
   so any data chunk can fit on ANY path. Most implementations DO bundle
   data chunks for this very reason... its easy to do and it allows
   larger PMTU's on different paths to be utilized. So using the HB may
   be more efficient... its definitely simpler... I leave it to you to
   choose. We may also want to mention the ICMP issue with SCTP since a
   validated ICMP message with SCTP can always be trusted].

   The SCTP Verification-Tag is designed to increase SCTPs robustness in
   the presence of a number of attacks, including forged ICMP messages.
   It relies on a 32 bit Verification Tag which is initialized to a



Mathis, et al.         Expires November 30, 2004               [Page 26]

Internet-Draft             Path MTU Discovery                  June 2004


   random value during connection establishment and placed in the first
   64 bits of all SCTP messages. All subsequent messages (including ICMP
   messages, which copy at least the first 64 bits of the message) must
   match the original Verification Tag, or they are rejected as being
   likely attacks against the connection. [9][16].

   It is believed that the Verification Tag mechanism is strong enough
   where SCTP could unconditionally process Packet Too Large messages
   that would reduce the path MTU at arbitrary times.   As written, this
   document does not encourage this method.  The PLPMTUD ICMP validity
   checks are cascaded with the SCTP checks, such that the messages are
   processed only if they meet all consistency checks.  In particular,
   PLPMTUD only uses the ICMP MTU value following a probe, during MTU
   verification, or following a hard stop timeout.

   To change this an implementation  would have to suppress some of the
   checks in Section 5.2.4.1 for SCTP.

5.4.3  Probing Method for IP Fragmentation

   As mentioned in Section 5.2.6, datagram protocols (such as UDP) can
   rely on IP fragmentation as a packetization layer.   Since the IP
   layer does not have any way to determine if the fragments were
   delivered, it can not do the probing directly.    The probing has to
   be done with an adjunct protocol that uses the diagnostic API
   (Section 5.5.4) to send oversized probes, and some other API to
   update the MPS stored in the IP layer.

5.4.4  Issues for other transport protocols

   Some transport protocols (such as ISO TP4 [ISOTP]) are not allowed to
   repacketize when doing a retransmission.  That is, once an attempt is
   made to transmit a segment of a certain size, the transport cannot
   split the contents of the segment into smaller segments for
   retransmission.  In such a case, the original segment can be
   fragmented by the IP layer during retransmission.  Subsequent
   segments, when transmitted for the first time, should be no larger
   than allowed by the Path MTU.

5.5  Operational Integration

5.5.1  Interoperation with prior algorithms

   Properly functioning Path MTU discovery is critical to the robust and
   efficient operation of the Internet.   Any major change (as described
   in this document) has the potential to be very disruptive if it
   contains any errors or oversights.   Therefore, we offer a deployment
   strategy in which classical PMTUD operation as described in RFC 1191



Mathis, et al.         Expires November 30, 2004               [Page 27]

Internet-Draft             Path MTU Discovery                  June 2004


   and RFC 1981 is unmodified and PLPMTUD is only invoked following a
   full stop timeout, presumably due to an "ICMP black hole". To do
   this:
   o  Relax the ICMP checks in Section 5.2.4.1 specifically to allow an
      ICMP Packet Too Large message to reduce the MTU at arbitrary
      times.
   o  When there is no cached MTU, use the Interface MTU as specified
      classical PMTU discovery, rather the initial MTU as specified in
      Section 5.2.2
   o  MTU searching as described in Section 5.3 is disabled entirely or
      starts in the monitor state.
   o  A full stop timeout is processed as described in Section 5.2.4.4.
      This becomes the only mechanism to invoke the rest of PLPMTUD.

   When configured in this manner, PLPMTUD will increase the robustness
   of classical PMTU discovery in the presence of ICMP black holes and
   other ICMP problems, with minimal exposure to unanticipated problems
   during deployment.  Since this configuration does not help robustness
   in the presence of malicious or erroneous ICMP messages, it is not
   recommended for the long term.

5.5.2  Interoperation over subnets with dissimilar MTUs

   With classical PMTUD, the ingress router to a subnet is responsible
   for knowing what size packets can be delivered to every node attached
   to that subnets.   For most subnet types, this requires that the
   entire subnet has a single MTU which is common to every attached
   node.   (For a few subnets types (e.g. ATM[12]) the nodes on a subnet
   can be negotiate the MTU on a pairwise basis, and the ingress router
   is responsible for knowing the MTU to each of it peers).

   This requirement has proven to be a major impediment to deploying
   larger MTUs in the operational Internet.  Often one single node which
   does not support a larger MTU effectively vetoes raising the MTU on a
   subnet, because the ingress router does not have a mechanism to
   generate the proper Packet Too Big Message for the one attached node
   with a smaller MTU

   With  PLPMTUD, this requirement is completely relaxed.  As long as
   oversized packets addressed the nodes with the smaller MTU are
   reliably discarded, PLPMTUD will find the proper MTU for these nodes.

5.5.3  Interoperation with tunnels

   PLPMTUD is specifically designed to solve many of the problems that
   people are experiencing today due to poor interactions between
   classical MTU discovery, IPsec, and various sorts of tunnels [5].
   As long as the tunnel reliably discards packets that are too large,



Mathis, et al.         Expires November 30, 2004               [Page 28]

Internet-Draft             Path MTU Discovery                  June 2004


   PLPMTUD will discover an appropriate MTU for the path.

   Unfortunately due to the pervasive problems with classical PMTU
   discovery, many manufacturers of various types of VPN/tunneling
   equipment have resorted to ignoring the DF bit.  This not only
   violates the IP standard and many recommendations to the contrary
   [17][18], it also violates the only requirement that PLPMTUD places
   on the link layer: that oversized packets are reliably discarded.
   It is imperative that people understand the impact of ignoring the DF
   bit both to applications and to PLPMTUD.

   We do understand the reality of the situation.  It is important that
   vendors who are building devices the violate the DF specification
   understand that PLPMTUD requires that probe packets be discarded, and
   that sending ICMP packet too big messages alone is insufficient to
   prevent wholesale fragmentation if the probe packets are delivered.

   Therefore, it is imperative that devices that do not honor DF include
   packet size history caches and other heuristics to robustly detect
   and discard probe packets, if delivering them would require
   fragmentation.

5.5.4  Diagnostic tools

   All implementations MUST include facilities for MTU discovery
   diagnostic tools that implement PLPMTUD or other MTU discovery
   algorithms in user mode without help or interference by the PMTUD
   algorithm present in the operating system.  This requires an
   mechanism where a diagnostic application can send packets that are
   larger than the operating system's notion of the current path MTU and
   collect any resulting Packet Too Big Messages or other ICMP messages.
   For IPv4 the diagnostic application must be able to set the DF bit.

   At this time nearly all operating systems support two modes for
   sending UDP datagrams: one which silently fragments packets that are
   too large, and another that rejects packets that are too large.
   Neither of these modes are suitable for efficiently diagnosing
   problems with the MTU discovery, such as routers that return Packet
   Too Big messages containing incorrect size information.

5.5.5  Management interface

   It is suggested that an implementation provide a way for a system
   utility program to:
   o  Globally disable all ICMP Packet Tool Large message processing
   o  Globally suppress some or all ICMP consistency checks described in
      Section 5.2.4.1.  Setting this option foregoes some possible
      security improvements, in exchange for making PLPMTUD behave more



Mathis, et al.         Expires November 30, 2004               [Page 29]

Internet-Draft             Path MTU Discovery                  June 2004


      like classical PMTU discovery.  (See Section 5.5.1)
   o  Globally permit ICMP Packet Tool Large messages to unconditionally
      reduce the MTU, even if there were not lost lost packets.
      Setting option foregoes some possible security improvements, in
      exchange for making PLPMTUD behave more like classical PMTU
      discovery.  (See Section 5.5.1)
   o  Globally adjust timer intervals for specific classes of probe
      failures

   In addition, it is important that there be a mechanism to permit per
   path controls to override specific parts of the PLPMTUD algorithm.
   All of these per path controls can be preset from similar global
   controls.
   o  Disable MTU searching a given path, such that new MTU values are
      never probed.
   o  Set the initial MTU for a given path.   This could be used to
      speed convergence in relatively static environments.   There
      should be an option to cause PLPMTUD to choose the same initial
      value as would be chosen by classical PMTU discovery.  I.e.
      typically the Interface MTU.   This is used in the mode described
      in Section 5.5.1 where PLPMTUD is used only for black hole
      detection in classical PMTU discovery.
   o  Limit the maximum probed MTU for a given path.   This permits a
      manual configuration to work around a link that spuriously
      delivers packets that are larger than the useful path MTU.
   o  Per path and per application controls to disable ICMP processing,
      to further limit possible damage from malicious Packet Too Big
      messages (in addition to the global controls).

6.  References

6.1  Normative References

   [1]  Postel, J., "Internet Protocol", STD 5, RFC 791, September 1981.

   [2]  Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191,
        November 1990.

   [3]  McCann, J., Deering, S. and J. Mogul, "Path MTU Discovery for IP
        version 6", RFC 1981, August 1996.

   [4]  Bradner, S., "Key words for use in RFCs to Indicate Requirement
        Levels", BCP 14, RFC 2119, March 1997.

   [5]  Kent, S. and R. Atkinson, "Security Architecture for the
        Internet Protocol", RFC 2401, November 1998.

   [6]  Allman, M., Floyd, S. and C. Partridge, "Increasing TCP's



Mathis, et al.         Expires November 30, 2004               [Page 30]

Internet-Draft             Path MTU Discovery                  June 2004


        Initial Window", RFC 2414, September 1998.

   [7]  Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6)
        Specification", RFC 2460, December 1998.

   [8]  Floyd, S., "Congestion Control Principles", BCP 41, RFC 2914,
        September 2000.

   [9]  Stewart, R., Xie, Q., Morneault, K., Sharp, C., Schwarzbauer,
        H., Taylor, T., Rytina, I., Kalla, M., Zhang, L. and V. Paxson,
        "Stream Control Transmission Protocol", RFC 2960, October 2000.

6.2  Informative References

   [10]  Mogul, J., Kent, C., Partridge, C. and K. McCloghrie, "IP MTU
         discovery options", RFC 1063, July 1988.

   [11]  Knowles, S., "IESG Advice from Experience with Path MTU
         Discovery", RFC 1435, March 1993.

   [12]  Atkinson, R., "Default IP MTU for use over ATM AAL5", RFC 1626,
         May 1994.

   [13]  Sung, T., "TCP And UDP Over IPX Networks With Fixed Path MTU",
         RFC 1791, April 1995.

   [14]  Partridge, C., "Using the Flow Label Field in IPv6", RFC 1809,
         June 1995.

   [15]  Lahey, K., "TCP Problems with Path MTU Discovery", RFC 2923,
         September 2000.

   [16]  Stewart, R., "Stream Control Transmission Protocol (SCTP)
         Implementors Guide", draft-ietf-tsvwg-sctpimpguide-10 (work in
         progress), December 2003.

   [17]  Kent, C. and J. Mogul, "Fragmentation considered harmful",
         Proc. SIGCOMM '87 vol. 17, No. 5, October 1987.

   [18]  Mathis, M., Heffner, J. and B. Chandler, "Fragmentation
         Considered Very Harmful", draft-mathis-frag-harmful-00 (work in
         progress), July 2004.









Mathis, et al.         Expires November 30, 2004               [Page 31]

Internet-Draft             Path MTU Discovery                  June 2004


Authors' Addresses

   Matt Mathis
   Pittsburgh Supercomputing Center
   4400 Fifth Avenue
   Pittsburgh, PA  15213
   US

   Phone: 412-268-3319
   EMail: mathis@psc.edu


   John W. Heffner
   Pittsburgh Supercomputing Center
   4400 Fifth Avenue
   Pittsburgh, PA  15213
   US

   Phone: 412-268-2329
   EMail: jheffner@psc.edu


   Kevin Lahey
   Freelance

   EMail: kml@patheticgeek.net

Appendix A.  Security Considerations

   Under all conditions the PLPMTUD procedure described in this document
   is at least as secure as the current standard path MTU discovery
   procedures described in RFC 1191 [2] and RFC 1981 [3].

   It the recommended configuration, PLPMTUD is significantly harder to
   attack than current procedures, because ICMP messages are cached and
   only processed in connection with lost packets.   This effectively
   prevents blind attacks on the path MTU discovery system.

   Furthermore, since this algorithm is designed for robust operation
   without any ICMP (or other messages from the network), it can be
   configured to ignore all ICMP messages (globally or on a per
   application basis).  In this configuration it can not be attacked,
   unless the attacker can identify and selectively cause probe packets
   to be lost.

Appendix B.  IANA considerations

   None.



Mathis, et al.         Expires November 30, 2004               [Page 32]

Internet-Draft             Path MTU Discovery                  June 2004


Appendix C.  Acknowledgements

   Most of the SCTP text was contributed by Randall Stewart.

   Matt Mathis and John Heffner are supported in this work by a grant
   from Cisco Systems, Inc.













































Mathis, et al.         Expires November 30, 2004               [Page 33]

Internet-Draft             Path MTU Discovery                  June 2004


Intellectual Property Statement

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights. Information
   on the IETF's procedures with respect to rights in IETF Documents can
   be found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard. Please address the information to the IETF at
   ietf-ipr@ietf.org.


Disclaimer of Validity

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
   ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
   INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Copyright Statement

   Copyright (C) The Internet Society (2004). This document is subject
   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights.


Acknowledgment

   Funding for the RFC Editor function is currently provided by the
   Internet Society.




Mathis, et al.         Expires November 30, 2004               [Page 34]


Html markup produced by rfcmarkup 1.109, available from https://tools.ietf.org/tools/rfcmarkup/