draft-ietf-mpls-entropy-label-06.txt   rfc6790.txt 
Network Working Group K. Kompella Internet Engineering Task Force (IETF) K. Kompella
Internet-Draft J. Drake Request for Comments: 6790 J. Drake
Updates: 3031, 3107, 3209, 5036 Juniper Networks Updates: 3031, 3107, 3209, 5036 Juniper Networks
(if approved) S. Amante Category: Standards Track S. Amante
Intended status: Standards Track Level 3 Communications, LLC ISSN: 2070-1721 Level 3 Communications, Inc.
Expires: March 10, 2013 W. Henderickx W. Henderickx
Alcatel-Lucent Alcatel-Lucent
L. Yong L. Yong
Huawei USA Huawei USA
September 6, 2012 November 2012
The Use of Entropy Labels in MPLS Forwarding The Use of Entropy Labels in MPLS Forwarding
draft-ietf-mpls-entropy-label-06
Abstract Abstract
Load balancing is a powerful tool for engineering traffic across a Load balancing is a powerful tool for engineering traffic across a
network. This memo suggests ways of improving load balancing across network. This memo suggests ways of improving load balancing across
MPLS networks using the concept of "entropy labels". It defines the MPLS networks using the concept of "entropy labels". It defines the
concept, describes why entropy labels are useful, enumerates concept, describes why entropy labels are useful, enumerates
properties of entropy labels that allow maximal benefit, and shows properties of entropy labels that allow maximal benefit, and shows
how they can be signaled and used for various applications. This how they can be signaled and used for various applications. This
document updates RFCs 3031, 3107, 3209 and 5036. document updates RFCs 3031, 3107, 3209, and 5036.
Status of this Memo
This Internet-Draft is submitted in full conformance with the Status of This Memo
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering This is an Internet Standards Track document.
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Further information on
Internet Standards is available in Section 2 of RFC 5741.
This Internet-Draft will expire on March 10, 2013. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc6790.
Copyright Notice Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction ....................................................3
1.1. Conventions used . . . . . . . . . . . . . . . . . . . . . 4 1.1. Conventions Used ...........................................4
1.2. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2. Motivation .................................................6
2. Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2. Approaches ......................................................7
3. Entropy Labels and Their Structure . . . . . . . . . . . . . . 8 3. Entropy Labels and Their Structure ..............................8
4. Data Plane Processing of Entropy Labels . . . . . . . . . . . 9 4. Data Plane Processing of Entropy Labels .........................9
4.1. Egress LSR . . . . . . . . . . . . . . . . . . . . . . . . 9 4.1. Egress LSR .................................................9
4.2. Ingress LSR . . . . . . . . . . . . . . . . . . . . . . . 10 4.2. Ingress LSR ...............................................10
4.3. Transit LSR . . . . . . . . . . . . . . . . . . . . . . . 11 4.3. Transit LSR ...............................................11
4.4. Penultimate Hop LSR . . . . . . . . . . . . . . . . . . . 12 4.4. Penultimate Hop LSR .......................................12
5. Signaling for Entropy Labels . . . . . . . . . . . . . . . . . 12 5. Signaling for Entropy Labels ...................................12
5.1. LDP Signaling . . . . . . . . . . . . . . . . . . . . . . 12 5.1. LDP Signaling .............................................12
5.1.1. Processing the ELC TLV . . . . . . . . . . . . . . . . 13 5.1.1. Processing the ELC TLV .............................13
5.2. BGP Signaling . . . . . . . . . . . . . . . . . . . . . . 13 5.2. BGP Signaling .............................................13
5.3. RSVP-TE Signaling . . . . . . . . . . . . . . . . . . . . 14 5.3. RSVP-TE Signaling .........................................14
5.4. Multicast LSPs and Entropy Labels . . . . . . . . . . . . 14 5.4. Multicast LSPs and Entropy Labels .........................15
6. Operations, Administration, and Maintenance (OAM) and 6. Operations, Administration, and Maintenance (OAM) and
Entropy Labels . . . . . . . . . . . . . . . . . . . . . . . . 15 Entropy Labels .................................................15
7. MPLS-TP and Entropy Labels . . . . . . . . . . . . . . . . . . 16 7. MPLS-TP and Entropy Labels .....................................16
8. Entropy Labels in Various Scenarios . . . . . . . . . . . . . 16 8. Entropy Labels in Various Scenarios ............................16
8.1. LDP Tunnel . . . . . . . . . . . . . . . . . . . . . . . . 16 8.1. LDP Tunnel ................................................17
8.2. LDP Over RSVP-TE . . . . . . . . . . . . . . . . . . . . . 18 8.2. LDP over RSVP-TE ..........................................20
8.3. MPLS Applications . . . . . . . . . . . . . . . . . . . . 19 8.3. MPLS Applications .........................................20
9. Security Considerations . . . . . . . . . . . . . . . . . . . 19 9. Security Considerations ........................................20
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 10. IANA Considerations ...........................................21
10.1. Reserved Label for ELI . . . . . . . . . . . . . . . . . . 20 10.1. Reserved Label for ELI ...................................21
10.2. LDP Entropy Label Capability TLV . . . . . . . . . . . . . 20 10.2. LDP Entropy Label Capability TLV .........................21
10.3. BGP Entropy Label Capability Attribute . . . . . . . . . . 20 10.3. BGP Entropy Label Capability Attribute ...................21
10.4. RSVP-TE Entropy Label Capability flag . . . . . . . . . . 20 10.4. RSVP-TE Entropy Label Capability Flag ....................22
11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 20 11. Acknowledgments ...............................................22
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21 12. References ....................................................22
12.1. Normative References . . . . . . . . . . . . . . . . . . . 21 12.1. Normative References .....................................22
12.2. Informative References . . . . . . . . . . . . . . . . . . 21 12.2. Informative References ...................................23
Appendix A. Applicability of LDP Entropy Label Capability TLV . . 22 Appendix A. Applicability of LDP Entropy Label Capability TLV .....24
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 22
1. Introduction 1. Introduction
Load balancing, or multi-pathing, is an attempt to balance traffic Load balancing, or multi-pathing, is an attempt to balance traffic
across a network by allowing the traffic to use multiple paths. Load across a network by allowing the traffic to use multiple paths. Load
balancing has several benefits: it eases capacity planning; it can balancing has several benefits: it eases capacity planning, it can
help absorb traffic surges by spreading them across multiple paths; help absorb traffic surges by spreading them across multiple paths,
it allows better resilience by offering alternate paths in the event and it allows better resilience by offering alternate paths in the
of a link or node failure. event of a link or node failure.
As providers scale their networks, they use several techniques to As providers scale their networks, they use several techniques to
achieve greater bandwidth between nodes. Two widely used techniques achieve greater bandwidth between nodes. Two widely used techniques
are: Link Aggregation Group (LAG) and Equal-Cost Multi-Path (ECMP). are: Link Aggregation Group (LAG) and Equal Cost Multi-Path (ECMP).
LAG is used to bond together several physical circuits between two LAG is used to bond together several physical circuits between two
adjacent nodes so they appear to higher-layer protocols as a single, adjacent nodes so they appear to higher-layer protocols as a single,
higher bandwidth 'virtual' pipe. ECMP is used between two nodes higher-bandwidth "virtual" pipe. ECMP is used between two nodes
separated by one or more hops, to allow load balancing over several separated by one or more hops, to allow load balancing over several
shortest paths in the network. This is typically obtained by shortest paths in the network. This is typically obtained by
arranging IGP metrics such that there are several equal cost paths arranging IGP metrics such that there are several equal cost paths
between source-destination pairs. Both of these techniques may, and between source-destination pairs. Both of these techniques may, and
often do, co-exist in various parts of a given provider's network, often do, coexist in differing parts of a given provider's network,
depending on various choices made by the provider. depending on various choices made by the provider.
A very important requirement when load balancing is that packets A very important requirement when load balancing is that packets
belonging to a given 'flow' must be mapped to the same path, i.e., belonging to a given "flow" must be mapped to the same path, i.e.,
the same exact sequence of links across the network. This is to the same exact sequence of links across the network. This is to
avoid jitter, latency and re-ordering issues for the flow. What avoid jitter, latency, and reordering issues for the flow. What
constitutes a flow varies considerably. A common example of a flow constitutes a flow varies considerably. A common example of a flow
is a TCP session. Other examples are an L2TP session corresponding is a TCP session. Other examples are a Layer 2 Tunneling Protocol
to a given broadband user, or traffic within an ATM virtual circuit. (L2TP) session corresponding to a given broadband user or traffic
within an ATM virtual circuit.
To meet this requirement, a node uses certain fields, termed 'keys', To meet this requirement, a node uses certain fields, termed "keys",
within a packet's header as input to a load balancing function within a packet's header as input to a load-balancing function
(typically a hash function) that selects the path for all packets in (typically a hash function) that selects the path for all packets in
a given flow. The keys chosen for the load balancing function depend a given flow. The keys chosen for the load-balancing function depend
on the packet type; a typical set (for IP packets) is the IP source on the packet type; a typical set (for IP packets) is the IP source
and destination addresses, the protocol type, and (for TCP and UDP and destination addresses, the protocol type, and (for TCP and UDP
traffic) the source and destination port numbers. An overly traffic) the source and destination port numbers. An overly
conservative choice of fields may lead to many flows mapping to the conservative choice of fields may lead to many flows mapping to the
same hash value (and consequently poorer load balancing); an overly same hash value (and consequently poorer load balancing); an overly
aggressive choice may map a flow to multiple values, potentially aggressive choice may map a flow to multiple values, potentially
violating the above requirement. violating the above requirement.
For MPLS networks, most of the same principles (and benefits) apply. For MPLS networks, most of the same principles (and benefits) apply.
However, finding useful keys in a packet for the purpose of load However, finding useful keys in a packet for the purpose of load
balancing can be more of a challenge. In many cases, MPLS balancing can be more of a challenge. In many cases, MPLS
encapsulation may require fairly deep inspection of packets to find encapsulation may require fairly deep inspection of packets to find
these keys at transit Label Switching Routers (LSRs). these keys at transit Label Switching Routers (LSRs).
One way to eliminate the need for this deep inspection is to have the One way to eliminate the need for this deep inspection is to have the
ingress LSR of an MPLS Label Switched Path extract the appropriate ingress LSR of an MPLS Label Switched Path extract the appropriate
keys from a given packet, input them to its load balancing function, keys from a given packet, input them to its load-balancing function,
and place the result in an additional label, termed the 'entropy and place the result in an additional label, termed the "entropy
label', as part of the MPLS label stack it pushes onto that packet. label", as part of the MPLS label stack it pushes onto that packet.
The packet's MPLS entire label stack can then be used by transit LSRs The entire label stack of the MPLS packet can then be used by transit
to perform load balancing, as the entropy label introduces the right LSRs to perform load balancing, as the entropy label introduces the
level of "entropy" into the label stack. right level of "entropy" into the label stack.
There are five key reasons why this is beneficial: There are five key reasons why this is beneficial:
1. at the ingress LSR, MPLS encapsulation hasn't yet occurred, so 1. At the ingress LSR, MPLS encapsulation hasn't yet occurred, so
deep inspection is not necessary; deep inspection is not necessary.
2. the ingress LSR has more context and information about incoming 2. The ingress LSR has more context and information about incoming
packets than transit LSRs; packets than transit LSRs.
3. ingress LSRs usually operate at lower bandwidths than transit 3. Ingress LSRs usually operate at lower bandwidths than transit
LSRs, allowing them to do more work per packet; LSRs, allowing them to do more work per packet.
4. transit LSRs do not need to perform deep packet inspection and 4. Transit LSRs do not need to perform deep packet inspection and
can load balance effectively using only a packet's MPLS label can load balance effectively using only a packet's MPLS label
stack; and stack.
5. transit LSRs, not having the full context that an ingress LSR 5. Transit LSRs, not having the full context that an ingress LSR
does, have the hard choice between potentially misinterpreting does, have the hard choice between potentially misinterpreting
fields in a packet as valid keys for load balancing (causing fields in a packet as valid keys for load balancing (causing
packet ordering problems) or adopting a conservative approach packet-ordering problems) or adopting a conservative approach
(giving rise to sub-optimal load balancing). Entropy labels (giving rise to sub-optimal load balancing). Entropy labels
relieves them of making this choice. relieve them of making this choice.
This memo describes why entropy labels are needed and defines the This memo describes why entropy labels are needed and defines the
properties of entropy labels; in particular how they are generated properties of entropy labels, in particular, how they are generated
and received, and the expected behavior of transit LSRs. Finally, it and received and the expected behavior of transit LSRs. Finally, it
describes in general how signaling works and what needs to be describes in general how signaling works and what needs to be
signaled, as well as specifics for the signaling of entropy labels signaled as well as specifics for the signaling of entropy labels for
for LDP ([RFC5036]), BGP ([RFC3107]), and RSVP-TE ([RFC3209]). LDP [RFC5036], BGP [RFC4271], and RSVP - Traffic Engineering
(RSVP-TE) [RFC3209].
1.1. Conventions used 1.1. Conventions Used
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119]. document are to be interpreted as described in [RFC2119].
The following acronyms are used: The following acronyms/initialisms are used:
BoS: Bottom of Stack BoS: Bottom of Stack
CE: Customer Edge device CE: Customer Edge
ECMP: Equal Cost Multi-Path ECMP: Equal Cost Multi-Path
EL: Entropy Label EL: Entropy Label
ELC: Entropy Label Capability ELC: Entropy Label Capability
ELI: Entropy Label Indicator ELI: Entropy Label Indicator
FEC: Forwarding Equivalence Class FEC: Forwarding Equivalence Class
LAG: Link Aggregation Group LAG: Link Aggregation Group
LER: Label Edge Router LER: Label Edge Router
LSP: Label Switched Path LSP: Label Switched Path
LSR: Label Switching Router LSR: Label Switching Router
PE: Provider Edge Router PE: Provider Edge
PW: Pseudowire PW: Pseudowire
PHP: Penultimate Hop Popping PHP: Penultimate Hop Popping
TC: Traffic Class TC: Traffic Class
TTL: Time-to-Live TTL: Time to Live
UHP: Ultimate Hop Popping UHP: Ultimate Hop Popping
VPLS: Virtual Private LAN (Local Area Network) Service VPLS: Virtual Private LAN (Local Area Network) Service
VPN: Virtual Private Network VPN: Virtual Private Network
The term ingress (or egress) LSR is used interchangeably with ingress The term "ingress LSR" (or "egress LSR") is used interchangeably with
(or egress) LER. The term application throughout the text refers to "ingress LER" (or "egress LER"). The term "application" throughout
an MPLS application (such as a VPN or VPLS). the text refers to an MPLS application (such as a VPN or VPLS).
A label stack (say of three labels) is denoted by <L1, L2, L3>, where A label stack (say of three labels) is denoted by <L1, L2, L3>, where
L1 is the "outermost" label and L3 the innermost (closest to the L1 is the "outermost" label and L3 the "innermost" (closest to the
payload). Packet flows are depicted left to right, and signaling is payload). Packet flows are depicted left to right, and signaling is
shown right to left (unless otherwise indicated). shown right to left (unless otherwise indicated).
The term 'label' is used both for the entire 32-bit label stack entry The term "label" is used both for the entire 32-bit label stack entry
and the 20-bit label field within a label stack entry. It should be and the 20-bit label field within a label stack entry. It should be
clear from the context which is meant. clear from the context which is meant.
1.2. Motivation 1.2. Motivation
MPLS is a very successful generic forwarding substrate that MPLS is a very successful generic forwarding substrate that
transports several dozen types of protocols, most notably: IP, PWs, transports several dozen types of protocols, most notably: IP, PWs,
VPLS and IP VPNs. Within each type of protocol, there typically VPLS, and IP VPNs. Within each type of protocol, there typically
exist several variants, each with a different set of load balancing exist several variants, each with a different set of load-balancing
keys, e.g., for IP: IPv4, IPv6, IPv6 in IPv4, etc.; for PWs: keys, e.g., IPv4, IPv6, IPv6 in IPv4, etc. for IP and Ethernet; ATM,
Ethernet, ATM, Frame-Relay, etc. There are also several different Frame-Relay, etc. for PWs. There are also several different types of
types of Ethernet over PW encapsulation, ATM over PW encapsulation, Ethernet over PW encapsulation, ATM over PW encapsulation, etc.
etc. as well. Finally, given the popularity of MPLS, it is likely Finally, given the popularity of MPLS, it is likely that it will
that it will continue to be extended to transport new protocols. continue to be extended to transport new protocols.
Currently, each transit LSR along the path of a given LSP has to try Currently, each transit LSR along the path of a given LSP has to try
to infer the underlying protocol within an MPLS packet in order to to infer the underlying protocol within an MPLS packet in order to
extract appropriate keys for load balancing. Unfortunately, if the extract appropriate keys for load balancing. Unfortunately, if the
transit LSR is unable to infer the MPLS packet's protocol (as is transit LSR is unable to infer the MPLS packet's protocol (as is
often the case), it will typically use the topmost (or all) MPLS often the case), it will typically use the topmost (or all) MPLS
labels in the label stack as keys for the load balancing function. labels in the label stack as keys for the load-balancing function.
The result may be an extremely inequitable distribution of traffic The result may be an extremely inequitable distribution of traffic
across equal-cost paths exiting that LSR. This is because MPLS across equal cost paths exiting that LSR. This is because MPLS
labels are generally fairly coarse-grained forwarding labels that labels are generally fairly coarse-grained forwarding labels that
typically describe a next-hop, or provide some of demultiplexing typically describe a next hop, or provide some demultiplexing and/or
and/or forwarding function, and do not describe the packet's forwarding function, and do not describe the packet's underlying
underlying protocol. protocol.
On the other hand, an ingress LSR (e.g., a PE router) has detailed On the other hand, an ingress LSR (e.g., a PE router) has detailed
knowledge of an packet's contents, typically through a priori knowledge of a packet's contents, typically through a priori
configuration of the encapsulation(s) that are expected at a given configuration of the encapsulations that are expected at a given
PE-CE interface, (e.g., IPv4, IPv6, VPLS, etc.). They also have more PE-CE interface, (e.g., IPv4, IPv6, VPLS, etc.). They may also have
flexible forwarding hardware. PE routers need this information and more flexible forwarding hardware, depending on implementation
these capabilities to: details. PE routers need this information and these capabilities to:
a) apply the required services for the CE; a. apply the required services for the CE;
b) discern the packet's CoS forwarding treatment; b. discern the packet's Class of Service (CoS) forwarding treatment;
c) apply filters to forward or block traffic to/from the CE; c. apply filters to forward or block traffic to/from the CE;
d) to forward routing/control traffic to an onboard management d. forward routing/control traffic to an onboard management
processor; and, processor; and,
e) load-balance the traffic on its uplinks to transit LSRs (e.g., e. load balance the traffic on its uplinks to transit LSRs (e.g., P
P routers). routers).
By knowing the expected encapsulation types, an ingress LSR router By knowing the expected encapsulation types, an ingress LSR router
can apply a more specific set of payload parsing routines to extract can apply a more specific set of payload parsing routines to extract
the keys appropriate for a given protocol. This allows for the keys appropriate for a given protocol. This allows for
significantly improved accuracy in determining the appropriate load significantly improved accuracy in determining the appropriate load-
balancing behavior for each protocol. balancing behavior for each protocol.
If the ingress LSR were to capture the flow information so gathered If the ingress LSR were to capture the flow information so gathered
in a convenient form for downstream transit LSRs, transit LSRs could in a convenient form for downstream transit LSRs, transit LSRs could
remain completely oblivious to the contents of each MPLS packet, and remain completely oblivious to the contents of each MPLS packet and
use only the captured flow information to perform load balancing. In use only the captured flow information to perform load balancing. In
particular, there will be no reason to duplicate an ingress LSR's particular, there will be no reason to duplicate an ingress LSR's
complex packet/payload parsing functionality in a transit LSR. This complex packet/payload parsing functionality in a transit LSR. This
will result in less complex transit LSRs, enabling them to more will result in less complex transit LSRs, enabling them to more
easily scale to higher forwarding rates, larger port density, lower easily scale to higher forwarding rates, larger port density, lower
power consumption, etc. The idea in this memo is to capture this power consumption, etc. The idea in this memo is to capture this
flow information as a label, the so-called entropy label. flow information as a label, the so-called "entropy label".
Ingress LSRs can also adapt more readily to new protocols and extract Ingress LSRs can also adapt more readily to new protocols and extract
the appropriate keys to use for load balancing packets of those the appropriate keys to use for load-balancing packets of those
protocols. This means that deploying new protocols or services in protocols. This means that deploying new protocols or services in
edge devices requires fewer concomitant changes in the core, edge devices requires fewer concomitant changes in the core,
resulting in higher edge service velocity and at the same time more resulting in higher edge-service velocity and, at the same time, more
stable core networks. stable core networks.
2. Approaches 2. Approaches
There are two main approaches to encoding load balancing information There are two main approaches to encoding load-balancing information
in the label stack. The first allocates multiple labels for a in the label stack. The first allocates multiple labels for a
particular Forwarding Equivalence Class (FEC). These labels are particular Forwarding Equivalence Class (FEC). These labels are
equivalent in terms of forwarding semantics, but having multiple equivalent in terms of forwarding semantics, but having multiple
labels allows flexibility in assigning labels to flows belonging to labels allows flexibility in assigning labels to flows belonging to
the same FEC. This approach has the advantage that the label stack the same FEC. This approach has the advantage that the label stack
has the same depth whether or not one uses label-based load has the same depth whether or not one uses label-based load
balancing; and so, consequently, there is no change to forwarding balancing; consequently, there is no change to forwarding operations
operations on transit and egress LSRs. However, it has a major on transit and egress LSRs. However, it has a major drawback in that
drawback in that there is a significant increase in both signaling there is a significant increase in both signaling and forwarding
and forwarding state. state.
The other approach encodes the load balancing information as an The other approach encodes the load-balancing information as an
additional label in the label stack, thus increasing the depth of the additional label in the label stack, thus increasing the depth of the
label stack by one. With this approach, there is minimal change to label stack by one. With this approach, there is minimal change to
signaling state for a FEC; also, there is no change in forwarding signaling state for a FEC; also, there is no change in forwarding
operations in transit LSRs, and no increase of forwarding state in operations in transit LSRs and no increase of forwarding state in any
any LSR. The only purpose of the additional label is to increase the LSR. The only purpose of the additional label is to increase the
entropy in the label stack, so this is called an "entropy label". entropy in the label stack, so this is called an "entropy label".
This memo focuses solely on this approach. This memo focuses solely on this approach.
This latter approach uses upstream generated entropy labels, which This latter approach uses upstream-generated entropy labels, which
may conflict with downstream allocated application labels. There are may conflict with downstream-allocated application labels. There are
a few approaches to deal with this: 1) allocate a pair of labels for a few approaches to deal with this: 1) allocate a pair of labels for
each FEC, one that must have an entropy label below it, and one that each FEC, one that must have an entropy label below it and one that
must not; 2) use a label (the "Entropy Label Indicator") to indicate must not; 2) use a label (the "Entropy Label Indicator") to indicate
that the next label is an entropy label; and 3) allow entropy labels that the next label is an entropy label; and 3) allow entropy labels
only where there is no possible confusion. The first doubles control only where there is no possible confusion. The first doubles control
and data plane state in the network; the last is too restrictive. and data plane state in the network; the last is too restrictive.
The approach taken here is the second. In making both the above The approach taken here is the second. In making both the above
choices, the trade-off is to increase label stack depth rather than choices, the trade-off is to increase label stack depth rather than
control and data plane state in the network. control and data plane state in the network.
Finally, one may choose to associate ELs with MPLS tunnels (LSPs), or Finally, one may choose to associate ELs with MPLS tunnels (LSPs) or
with MPLS applications (e.g., VPNs). (What this entails is described with MPLS applications (e.g., VPNs). (What this entails is described
in later sections.) We take the former approach, for the following in later sections.) We take the former approach, for the following
reasons: reasons:
1. There are a small number of tunneling protocols for MPLS, but a 1. There are a small number of tunneling protocols for MPLS, but a
large and growing number of applications. Defining ELs on a large and growing number of applications. Defining ELs on a
tunnel basis means simpler standards, lower development, tunnel basis means simpler standards, lower development,
interoperability and testing efforts. interoperability, and testing efforts.
2. As a consequence, there will be much less churn in the network as 2. As a consequence, there will be much less churn in the network as
new applications (services) are defined and deployed. new applications (services) are defined and deployed.
3. Processing application labels in the data plane is more complex 3. Processing application labels in the data plane is more complex
than processing tunnel labels. Thus, it is preferable to burden than processing tunnel labels. Thus, it is preferable to burden
the latter rather than the former with EL processing. the latter rather than the former with EL processing.
4. Associating ELs with tunnels makes it simpler to deal with 4. Associating ELs with tunnels makes it simpler to deal with
hierarchy, be it LDP-over-RSVP-TE or Carrier's Carrier VPNs. hierarchy, be it LDP-over-RSVP-TE or Carrier's Carrier VPNs.
Each layer in the hierarchy can choose independently whether or Each layer in the hierarchy can choose independently whether or
not they want ELs. not they want ELs.
The cost of this approach is that ELIs will be mandatory; again, the The cost of this approach is that ELIs will be mandatory; again, the
trade-off is the size of the label stack. To summarize, the net trade-off is the size of the label stack. To summarize, the net
increase in the label stack to use entropy labels is two: one increase in the label stack to use entropy labels is two: one
reserved label for the ELI, and the entropy label itself. reserved label for the ELI and the entropy label itself.
3. Entropy Labels and Their Structure 3. Entropy Labels and Their Structure
An entropy label (as used here) is a label: An entropy label (as used here) is a label:
1. that is not used for forwarding; 1. that is not used for forwarding;
2. that is not signaled; and 2. that is not signaled; and
3. whose only purpose in the label stack is to provide 'entropy' to 3. whose only purpose in the label stack is to provide "entropy" to
improve load balancing. improve load balancing.
Entropy labels are generated by an ingress LSR, based entirely on Entropy labels are generated by an ingress LSR, based entirely on
load balancing information. However, they MUST NOT have values in load-balancing information. However, they MUST NOT have values in
the reserved label space (0-15) [IANA MPLS Label Values]. the reserved label space (0-15) [RFC3032].
Since entropy labels are generated by an ingress LSR, an egress LSR Since entropy labels are generated by an ingress LSR, an egress LSR
MUST be able to distinguish unambiguously between entropy labels and MUST be able to distinguish unambiguously between entropy labels and
application labels. To accomplish this, it is REQUIRED that the application labels. To accomplish this, it is REQUIRED that the
label immediately preceding an entropy label (EL) in the MPLS label label immediately preceding an Entropy Label (EL) in the MPLS label
stack be an 'entropy label indicator' (ELI), where preceding means stack be an Entropy Label Indicator (ELI), where preceding means
closer to the top of the label stack (farther from bottom of stack closer to the top of the label stack (farther from bottom of stack
indication). The ELI is a reserved label with value (TBD by IANA). indication). The ELI is a reserved label with value 7. How to set
How to set values of the TTL, TC and 'Bottom of Stack' (BoS) fields values of the TTL, TC, and "Bottom of Stack" (BoS) fields [RFC3032]
([RFC3032]) for the ELI and for ELs is discussed in Section 4.2. for the ELI and for ELs is discussed in Section 4.2.
Entropy labels are useful for pseudowires ([RFC4447]). [RFC6391] Entropy labels are useful for pseudowires [RFC4447]. [RFC6391]
explains how entropy labels can be used for RFC 4447-style explains how entropy labels can be used for pseudowires that are of
pseudowires, and thus is complementary to this memo, which focuses on the RFC 4447 style and is therefore complementary to this memo, which
how entropy labels can be used for tunnels, and thus for all other focuses on how entropy labels can be used for tunnels and thus for
MPLS applications. all other MPLS applications.
4. Data Plane Processing of Entropy Labels 4. Data Plane Processing of Entropy Labels
4.1. Egress LSR 4.1. Egress LSR
Suppose egress LSR Y is capable of processing entropy labels for a Suppose egress LSR Y is capable of processing entropy labels for a
tunnel. Y indicates this to all ingresses via signaling (see tunnel. Y indicates this to all ingresses via signaling (see
Section 5). Y MUST be prepared to deal both with packets with an Section 5). Y MUST be prepared to deal both with packets with an
imposed EL and those without; the ELI will distinguish these cases. imposed EL and those without; the ELI will distinguish these cases.
If a particular ingress chooses not to impose an EL, Y's processing If a particular ingress chooses not to impose an EL, Y's processing
of the received label stack (which might be empty) is as if Y chose of the received label stack (which might be empty) is as if Y chose
not to accept ELs. not to accept ELs.
If an ingress X chooses to impose an EL, then Y will receive a tunnel If an ingress LSR X chooses to impose an EL, then Y will receive a
termination packet with label stack <TL, ELI, EL> <remaining packet tunnel termination packet with label stack <TL, ELI, EL> <remaining
header>. Y recognizes TL as the label it distributed to its packet header>. Y recognizes TL as the label it distributed to its
upstreams for the tunnel, and pops it. (Note that TL may be the upstreams for the tunnel and pops it. (Note that TL may be the
implicit null label, in which case it doesn't appear in the label implicit null label, in which case it doesn't appear in the label
stack.) Y then recognizes the ELI and pops two labels: the ELI and stack.) Y then recognizes the ELI and pops two labels: the ELI and
the EL. Y then processes the remaining packet header as normal; this the EL. Y then processes the remaining packet header as normal; this
may require further processing of tunnel termination, perhaps with may require further processing of tunnel termination, perhaps with
further ELI+EL pairs. When processing the final tunnel termination, further ELI+EL pairs. When processing the final tunnel termination,
Y MAY enqueue the packet based on that tunnel TL's or ELI's TC value, Y MAY enqueue the packet based on that tunnel TL's or ELI's TC value
and MAY use the tunnel TL's or ELI's TTL to compute the TTL of the and MAY use the tunnel TL's or ELI's TTL to compute the TTL of the
remaining packet header. The EL's TTL MUST be ignored. remaining packet header. The EL's TTL MUST be ignored.
If any ELI processed by Y has BoS bit set, Y MUST discard the packet, If any ELI processed by Y has the BoS bit set, Y MUST discard the
and MAY log an error. The EL's BoS bit will indicate whether or not packet and MAY log an error. The EL's BoS bit will indicate whether
there are more labels in the stack. or not there are more labels in the stack.
4.2. Ingress LSR 4.2. Ingress LSR
If an egress LSR Y indicates via signaling that it can process ELs on If an egress LSR Y indicates via signaling that it can process ELs on
a particular tunnel, an ingress LSR X can choose whether or not to a particular tunnel, an ingress LSR X can choose whether or not to
insert ELs for packets going into that tunnel. Y MUST handle both insert ELs for packets going into that tunnel. Y MUST handle both
cases. cases.
The steps that X performs to insert ELs are as follows: The steps that X performs to insert ELs are as follows:
1. On an incoming packet, identify the application to which the 1. On an incoming packet, identify the application to which the
packet belongs; based on this, pick appropriate fields as input packet belongs; based on this, pick appropriate fields as input
to the load balancing function; apply the load balancing function to the load-balancing function; apply the load-balancing function
to these input fields, and let LB be the output. to these input fields and let LB be the output.
2. Determine the application label AL (if any). Push <AL> onto the 2. Determine the application label AL (if any). Push <AL> onto the
packet. packet.
3. Based on the application, the load balancing output LB and other 3. Based on the application, the load-balancing output LB and other
factors, determine the egress LSR Y, the tunnel to Y, the factors, determine the egress LSR Y, the tunnel to Y, the
specific interface to the next hop, and thus the tunnel label TL. specific interface to the next hop, and thus the tunnel label TL.
Use LB to generate the entropy label EL. Use LB to generate the entropy label EL.
4. If, for the chosen tunnel, Y has not indicated that it can 4. If, for the chosen tunnel, Y has not indicated that it can
process ELs, push <TL> onto the packet. If Y has indicated that process ELs, push <TL> onto the packet. If Y has indicated that
it can process ELs for the tunnel, push <TL, ELI, EL> onto the it can process ELs for the tunnel, push <TL, ELI, EL> onto the
packet. X SHOULD put the same TTL and TC fields for the ELI as packet. X SHOULD put the same TTL and TC fields for the ELI as
it does for TL. X MAY choose different values for the TTL and TC it does for TL. X MAY choose different values for the TTL and TC
fields if it is known that the ELI will not be exposed as the top fields if it is known that the ELI will not be exposed as the top
label at any point along the LSP (as may happen in cases where label at any point along the LSP (as may happen in cases where
PHP is used and the ELI and EL are not stripped at the PHP is used and the ELI and EL are not stripped at the
penultimate hop (see Section 4.4). The BoS bit for the ELI MUST penultimate hop (see Section 4.4). The BoS bit for the ELI MUST
be zero. The TTL for the EL MUST be zero to ensure that it is be zero (i.e., BoS is not set). The TTL for the EL MUST be zero
not used inadvertently for forwarding. The TC for the EL may be to ensure that it is not used inadvertently for forwarding. The
any value. The BoS bit for the EL depends on whether or not TC for the EL may be any value. The BoS bit for the EL depends
there are more labels in the label stack. on whether or not there are more labels in the label stack.
5. X then determines whether further tunnel hierarchy is needed; if 5. X then determines whether further tunnel hierarchy is needed; if
so, X goes back to step 3, possibly with a new egress Y for the so, X goes back to step 3, possibly with a new egress Y for the
new tunnel. Otherwise, X is done, and sends out the packet. new tunnel. Otherwise, X is done and sends out the packet.
Notes: Notes:
a. X computes load balancing information and generates the EL based a. X computes load-balancing information and generates the EL based
on the incoming application packet, even though the signaling of on the incoming application packet, even though the signaling of
EL capability is associated with tunnels. EL capability is associated with tunnels.
b. X MAY insert several entropy labels in the stack (each, of b. X MAY insert several entropy labels in the stack (each, of
course, preceded by an ELI), potentially one for each course, preceded by an ELI), potentially one for each
hierarchical tunnel, provided that the egress for that tunnel has hierarchical tunnel, provided that the egress for that tunnel has
indicated that it can process ELs for that tunnel. indicated that it can process ELs for that tunnel.
c. X MUST NOT include an entropy label for a given tunnel unless the c. X MUST NOT include an entropy label for a given tunnel unless the
egress LSR Y has indicated that it can process entropy labels for egress LSR Y has indicated that it can process entropy labels for
that tunnel. that tunnel.
d. The signaling and use of entropy labels in one direction d. The signaling and use of entropy labels in one direction
(signaling from Y to X, and data path from X to Y) is completely (signaling from Y to X and data path from X to Y) is completely
independent of the signaling and use of entropy labels in the independent of the signaling and use of entropy labels in the
reverse direction (signaling from X to Y, and data path from Y to reverse direction (signaling from X to Y and data path from Y to
X). X).
4.3. Transit LSR 4.3. Transit LSR
Transit LSRs MAY operate with no change in forwarding behavior. The Transit LSRs MAY operate with no change in forwarding behavior. The
following are suggestions for optimizations that improve load following are suggestions for optimizations that improve load
balancing, reduce the amount of packet data processed, and/or enhance balancing, reduce the amount of packet data processed, and/or enhance
backward compatibility. backward compatibility.
If a transit LSR recognizes the ELI, it MAY choose to load balance If a transit LSR recognizes the ELI, it MAY choose to load balance
solely on the following label (the EL); otherwise, it SHOULD use as solely on the following label (the EL); otherwise, it SHOULD use as
much of the whole label stack as feasible as keys for the load much of the whole label stack as feasible as keys for the load-
balancing function. In any case, reserved labels MUST NOT be used as balancing function. In any case, reserved labels MUST NOT be used as
keys for the load balancing function. keys for the load-balancing function.
Some transit LSRs look beyond the label stack for better load Some transit LSRs look beyond the label stack for better load-
balancing information. This is a simple, backward compatible balancing information. This is a simple, backward-compatible
approach in networks where some ingress LSRs impose ELs and others approach in networks where some ingress LSRs impose ELs and others
don't. However, this is of limited incremental value if an EL is don't. However, this is of limited incremental value if an EL is
indeed present, and requires more packet processing from the LSR. A indeed present and requires more packet processing from the LSR. A
transit LSR MAY choose to parse the label stack for the presence of transit LSR MAY choose to parse the label stack for the presence of
the ELI, and look beyond the label stack only if it does not find it, the ELI and look beyond the label stack only if it does not find it,
thus retaining the old behavior when needed, yet avoiding unnecessary thus retaining the old behavior when needed, yet avoiding unnecessary
work if not needed. work if not needed.
As stated in Section 4.1 and Section 5, an egress LSR that signals As stated in Sections 4.1 and 5, an egress LSR that signals both ELC
both ELC and implicit null MUST pop the ELI and the next label if it and implicit null MUST pop the ELI and the next label (which should
encounters a packet with the ELI as the topmost label. Any other LSR be the EL), if it encounters a packet with the ELI as the topmost
(including PHP LSRs) MUST drop such packets, as per section 3.18 of label. Any other LSR (including PHP LSRs) MUST drop such packets, as
[RFC3031]. per Section 3.18 of [RFC3031].
4.4. Penultimate Hop LSR 4.4. Penultimate Hop LSR
No change is needed at penultimate hop LSRs. However, a PHP LSR that No change is needed at penultimate hop LSRs. However, a PHP LSR that
recognizes the ELI MAY choose to pop the ELI and following label recognizes the ELI MAY choose to pop the ELI and following label
(which should be an entropy label) in addition to popping the tunnel (which should be an entropy label) in addition to popping the tunnel
label, provided that doing so doesn't diminish its ability to load label, provided that doing so doesn't diminish its ability to load
balance on the next hop. balance on the next hop.
5. Signaling for Entropy Labels 5. Signaling for Entropy Labels
skipping to change at page 12, line 31 skipping to change at page 12, line 37
Note that Entropy Label Capability may be asymmetric: if LSRs X and Y Note that Entropy Label Capability may be asymmetric: if LSRs X and Y
are at opposite ends of a tunnel, X may be able to process entropy are at opposite ends of a tunnel, X may be able to process entropy
labels, whereas Y may not. The signaling extensions below allow for labels, whereas Y may not. The signaling extensions below allow for
this asymmetry. this asymmetry.
For an illustration of signaling and forwarding with entropy labels, For an illustration of signaling and forwarding with entropy labels,
see Section 8. see Section 8.
5.1. LDP Signaling 5.1. LDP Signaling
A new LDP TLV ([RFC5036]) is defined to signal an egress's ability to A new LDP TLV [RFC5036] is defined to signal an egress's ability to
process entropy labels. This is called the ELC TLV, and may appear process entropy labels. This is called the ELC TLV and may appear as
as an Optional Parameter of the Label Mapping Message TLV. an Optional Parameter of the Label Mapping Message TLV.
The presence of the ELC TLV in a Label Mapping Message indicates to The presence of the ELC TLV in a Label Mapping Message indicates to
ingress LSRs that the egress LSR can process entropy labels for the ingress LSRs that the egress LSR can process entropy labels for the
associated LDP tunnel. The ELC TLV has Type (TBD by IANA) and Length associated LDP tunnel. The ELC TLV has Type 0x0206 and Length 0.
0.
The structure of the ELC TLV is shown below. The structure of the ELC TLV is shown below.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|U|F| Type (TBD) | Length (0) | |U|F| Type 0x0206 | Length (0) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: Entropy Label Capability TLV Figure 1: Entropy Label Capability TLV
where: where:
U: Unknown bit. This bit MUST be set to 1. If the ELC TLV is not U: Unknown bit. This bit MUST be set to 1. If the ELC TLV is not
understood by the receiver, then it MUST be ignored. understood by the receiver, then it MUST be ignored.
F: Forward bit. This bit MUST be set be set to 1. Since the ELC F: Forward bit. This bit MUST be set be set to 1. Since the ELC
TLV is going to be propagated hop-by-hop, it should be forwarded TLV is going to be propagated hop-by-hop, it should be forwarded
even by nodes that may not understand it. even by nodes that may not understand it.
Type: Type field. To be assigned by IANA. Type: Type field (0x0206)
Length: Length field. This field specifies the total length in Length: Length field. This field specifies the total length in
octets of the ELC TLV, and is currently defined to be 0. octets of the ELC TLV and is currently defined to be 0.
5.1.1. Processing the ELC TLV 5.1.1. Processing the ELC TLV
An LSR that receives a Label Mapping with the ELC TLV but does not An LSR that receives a Label Mapping with the ELC TLV but does not
understand it MUST propagate it intact to its neighbors and MUST NOT understand it MUST propagate it intact to its neighbors and MUST NOT
send a notification to the sender (following the meaning of the U- send a notification to the sender (following the meaning of the U-
and F-bits). and F-bits).
An LSR X may receive multiple Label Mappings for a given FEC F from An LSR X may receive multiple Label Mappings for a given FEC F from
its neighbors. In its turn, X may advertise a Label Mapping for F to its neighbors. In its turn, X may advertise a Label Mapping for F to
its neighbors. If X understands the ELC TLV, and if any of the its neighbors. If X understands the ELC TLV, and if any of the
advertisements it received for FEC F does not include the ELC TLV, X advertisements it received for FEC F does not include the ELC TLV, X
MUST NOT include the ELC TLV in its own advertisements of F. If all MUST NOT include the ELC TLV in its own advertisements of F. If all
the advertised Mappings for F include the ELC TLV, then X MUST the advertised Mappings for F include the ELC TLV, then X MUST
advertise its Mapping for F with the ELC TLV. If any of X's advertise its Mapping for F with the ELC TLV. If any of X's
neighbors resends its Mapping, sends a new Mapping or Withdraws a neighbors resends its Mapping, sends a new Mapping or sends a Label
previously advertised Mapping for F, X MUST re-evaluate the status of Withdraw for a previously advertised Mapping for F, X MUST re-
ELC for FEC F, and, if there is a change, X MUST re-advertise its evaluate the status of ELC for FEC F, and, if there is a change, X
Mapping for F with the updated status of ELC. MUST re-advertise its Mapping for F with the updated status of ELC.
5.2. BGP Signaling 5.2. BGP Signaling
When BGP [RFC4271] is used for distributing Network Layer When BGP [RFC4271] is used for distributing Network Layer
Reachability Information (NLRI) as described in, for example, Reachability Information (NLRI) as described in, for example,
[RFC3107], the BGP UPDATE message may include the ELC attribute as [RFC3107], the BGP UPDATE message may include the ELC attribute as
part of the Path Attributes. This is an optional, transitive BGP part of the Path Attributes. This is an optional, transitive BGP
attribute of type (to be assigned by IANA). The inclusion of this attribute of value 28. The inclusion of this attribute with an NLRI
attribute with an NLRI indicates that the advertising BGP router can indicates that the advertising BGP router can process entropy labels
process entropy labels as an egress LSR for all routes in that NLRI. as an egress LSR for all routes in that NLRI.
A BGP speaker S that originates an UPDATE should include the ELC A BGP speaker S that originates an UPDATE should include the ELC
attribute only if both of the following are true: attribute only if both of the following are true:
A1: S sets the BGP NEXT_HOP attribute to itself; AND A1: S sets the BGP NEXT_HOP attribute to itself AND
A2: S can process entropy labels. A2: S can process entropy labels.
Suppose a BGP speaker T receives an UPDATE U with the ELC attribute. Suppose a BGP speaker T receives an UPDATE U with the ELC attribute.
T has two choices. T can simply re-advertise U with the ELC T has two choices. T can simply re-advertise U with the ELC
attribute if either of the following is true: attribute if either of the following is true:
B1: T does not change the NEXT_HOP attribute; OR B1: T does not change the NEXT_HOP attribute OR
B2: T simply swaps labels without popping the entire label stack and B2: T simply swaps labels without popping the entire label stack and
processing the payload below. processing the payload below.
An example of the use of B1 is Route Reflectors. An example of the use of B1 is Route Reflectors.
However, if T changes the NEXT_HOP attribute for U and in the data However, if T changes the NEXT_HOP attribute for U and in the data
plane pops the entire label stack to process the payload, T MAY plane pops the entire label stack to process the payload, T MAY
include an ELC attribute for UPDATE U' if both of the following are include an ELC attribute for UPDATE U' if both of the following are
true: true:
C1: T sets the NEXT_HOP attribute of U' to itself; AND C1: T sets the NEXT_HOP attribute of U' to itself AND
C2: T can process entropy labels. C2: T can process entropy labels.
Otherwise, T MUST remove the ELC attribute. Otherwise, T MUST remove the ELC attribute.
5.3. RSVP-TE Signaling 5.3. RSVP-TE Signaling
Entropy Label support is signaled in RSVP-TE [RFC3209] using the Entropy label support is signaled in RSVP-TE [RFC3209] using the
Entropy Label Capability (ELC) flag in the Attribute Flags TLV of the Entropy Label Capability (ELC) flag in the Attribute Flags TLV of the
LSP_ATTRIBUTES object [RFC5420]. The presence of the ELC flag in a LSP_ATTRIBUTES object [RFC5420]. The presence of the ELC flag in a
Path message indicates that the ingress can process entropy labels in Path message indicates that the ingress can process entropy labels in
the upstream direction; this only makes sense for a bidirectional LSP the upstream direction; this only makes sense for a bidirectional LSP
and MUST be ignored otherwise. The presence of the ELC flag in a and MUST be ignored otherwise. The presence of the ELC flag in a
Resv message indicates that the egress can process entropy labels in Resv message indicates that the egress can process entropy labels in
the downstream direction. the downstream direction.
The bit number for the ELC flag is to be assigned by IANA. The bit number for the ELC flag is 9.
5.4. Multicast LSPs and Entropy Labels 5.4. Multicast LSPs and Entropy Labels
Multicast LSPs [RFC4875], [RFC6388] typically do not use ECMP for Multicast LSPs [RFC4875] [RFC6388] typically do not use ECMP for load
load balancing, as the combination of replication and multipathing balancing, as the combination of replication and multi-pathing can
can lead to duplicate traffic delivery. However, these LSPs can lead to duplicate traffic delivery. However, these LSPs can traverse
traverse bundled links [RFC4201] and LAGs. In both these cases, load bundled links [RFC4201] and LAGs. In both these cases, load
balancing is useful, and hence entropy labels can be of value for balancing is useful, and hence entropy labels can be of value for
multicast LSPs. multicast LSPs.
The methodology defined for entropy labels here will be used for The methodology defined for entropy labels here will be used for
multicast LSPs; however, the details of signaling and processing ELs multicast LSPs; however, the details of signaling and processing ELs
for multicast LSPs will be specified in a companion document. for multicast LSPs will be specified in a future document.
6. Operations, Administration, and Maintenance (OAM) and Entropy Labels 6. Operations, Administration, and Maintenance (OAM) and Entropy Labels
Generally OAM comprises a set of functions operating in the data Generally, OAM comprises a set of functions operating in the data
plane to allow a network operator to monitor its network plane to allow a network operator to monitor its network
infrastructure and to implement mechanisms in order to enhance the infrastructure and to implement mechanisms in order to enhance the
general behavior and the level of performance of its network, e.g., general behavior and the level of performance of its network, e.g.,
the efficient and automatic detection, localization, diagnosis and the efficient and automatic detection, localization, diagnosis, and
handling of defects. handling of defects.
Currently defined OAM mechanisms for MPLS include LSP Ping/Traceroute Currently defined OAM mechanisms for MPLS include LSP ping/traceroute
[RFC4379] and Bidirectional Failure Detection (BFD) for MPLS [RFC4379] and Bidirectional Forwarding Detection (BFD) for MPLS
[RFC5884]. The latter provides connectivity verification between the [RFC5884]. The latter provides connectivity verification between the
endpoints of an LSP, and recommends establishing a separate BFD endpoints of an LSP, and recommends establishing a separate BFD
session for every path between the endpoints. session for every path between the endpoints.
The LSP traceroute procedures of [RFC4379] allow an ingress LSR to The LSP traceroute procedures of [RFC4379] allow an ingress LSR to
obtain label ranges that can be used to send packets on every path to obtain label ranges that can be used to send packets on every path to
the egress LSR. It works by having ingress LSR sequentially ask the the egress LSR. It works by having the ingress LSR sequentially ask
transit LSRs along a particular path to a given egress LSR to return the transit LSRs along a particular path to a given egress LSR to
a label range such that the inclusion of a label in that range in a return a label range such that the inclusion of a label in that range
packet will cause the replying transit LSR to send that packet out in a packet will cause the replying transit LSR to send that packet
the egress interface for that path. The ingress provides the label out the egress interface for that path. The ingress provides the
range returned by transit LSR N to transit LSR N + 1, which returns a label range returned by transit LSR N to transit LSR N + 1, which
label range which is less than or equal in span to the range provided returns a label range that is less than or equal in span to the range
to it. This process iterates until the penultimate transit LSR provided to it. This process iterates until the penultimate transit
replies to the ingress LSR with a label range that is acceptable to LSR replies to the ingress LSR with a label range that is acceptable
it and to all LSRs along path preceding it for forwarding a packet to it and to all LSRs along path preceding it for forwarding a packet
along the path. along the path.
However, the LSP traceroute procedures do not specify where in the However, the LSP traceroute procedures do not specify where in the
label stack the value from the label range is to be placed, whether label stack the value from the label range is to be placed, whether
deep packet inspection is allowed and if so, which keys and key deep packet inspection is allowed, and if so, which keys and key
values are to be used. values are to be used.
This memo updates LSP traceroute by specifying that the value from This memo updates LSP traceroute by specifying that the value from
the label range is to be placed in the entropy label. Deep packet the label range is to be placed in the entropy label. Deep packet
inspection is thus not necessary, although an LSR may use it, inspection is thus not necessary, although an LSR may use it,
provided it do so consistently, i.e., if the label range to go to a provided it does so consistently, i.e., if the label range to go to a
given downstream LSR is computed with deep packet inspection, then given downstream LSR is computed with deep packet inspection, then
the data path should use the same approach and the same keys. the data path should use the same approach and the same keys.
In order to have a BFD session on a given path, a value from the In order to have a BFD session on a given path, a value from the
label range for that path should be used as the EL value for BFD label range for that path should be used as the EL value for BFD
packets sent on that path. packets sent on that path.
7. MPLS-TP and Entropy Labels 7. MPLS-TP and Entropy Labels
Since MPLS-TP does not use ECMP, entropy labels are not applicable to Since the MPLS Transport Profile (MPLS-TP) does not use ECMP, entropy
an MPLS-TP deployment. labels are not applicable to an MPLS-TP deployment.
8. Entropy Labels in Various Scenarios 8. Entropy Labels in Various Scenarios
This section describes the use of entropy labels in various This section describes the use of entropy labels in various
scenarios. scenarios. The material in this section is illustrative and offers
guidance to implementations, but it does not form a normative part of
this specification.
In the figures below, the following conventions used to depict In the figures below, the following conventions are used to depict
processing between X and Y. Note that control plane signaling goes processing between X and Y. Note that control plane signaling goes
right to left, whereas data plane processing goes left to right. right to left, whereas data plane processing goes left to right.
Protocols Protocols
Y: <--- [L, E] Y signals L to X Y: <--- [L, E] Y signals L to X
X ------------- Y X ------------- Y
LS: <L, ELI, EL> Label stack Data Plane:
X-Y: <L, ELI, EL> Label Stack from X -> Y
Label Stack Operations:
X: +<L, ELI, EL> X pushes <L, ELI, EL> X: +<L, ELI, EL> X pushes <L, ELI, EL>
Y: -<L, ELI, EL> Y pops <L, ELI, EL> Y: -<L, ELI, EL> Y pops <L, ELI, EL>
This means that Y signals to X label L for an LDP tunnel. E can be This means that Y signals to X label L for an LDP tunnel. E can be
one of: one of:
0: meaning egress is NOT entropy label capable, or 0: meaning egress is NOT entropy label capable or
1: meaning egress is entropy label capable. 1: meaning egress is entropy label capable
The line with LS: shows the label stack on the wire. Below that is The line with LS: shows the label stack on the wire. Below that is
the operation that each LSR does in the data plane, where + means the operation that each LSR does in the data plane, where + means
push the following label stack, - means pop the following label push the following label stack, - means pop the following label
stack, L~L' means swap L with L', and * means that the operation is stack, L~L' means swap L with L'.
not depicted.
8.1. LDP Tunnel 8.1. LDP Tunnel
The following illustrates several simple intra-AS LDP tunnels. The The following figures illustrate several simple intra-AS LDP tunnels.
first diagram shows ultimate hop popping (UHP) with ingress inserting The first diagram shows ultimate hop popping (UHP) with the ingress
an EL, the second UHP with no ELs, the third PHP with ELs, and inserting an EL, the second UHP with no ELs, the third PHP with ELs,
finally, PHP with no ELs, but also with an application label AL and finally, PHP with no ELs, but also with an application label AL
(which could, for example, be a VPN label). (which could, for example, be a VPN label).
Note that, in all the cases below, the MPLS application does not Note that, in all the cases below, the MPLS application does not
matter; it may be that X pushes some more labels (perhaps for a VPN matter; it may be that X pushes some more labels (perhaps for a VPN
or VPLS) below the ones shown, and Y pops them. or VPLS) below the ones shown, and Y pops them.
A: <--- [TL4, 1] A: <--- [TL4, 1]
B: <-- [TL3, 1] B: <-- [TL3, 1]
... W: <-- [TL2, 1]
W: <-- [TL1, 1]
Y: <-- [TL0, 1] Y: <-- [TL0, 1]
X --------------- A --------- B ... W ---------- Y X --------------- A --------- B --- W ---------- Y
LS: <TL4, ELI, EL> <TL3,ELI,EL> <TL0,ELI,EL> Data Plane:
X-A: <TL4, ELI, EL>
A-B: <TL3,ELI,EL>
B-W: <TL2,ELI,EL>
W-Y: <TL0,ELI,EL>
Label Stack Operations:
X: +<TL4, ELI, EL> X: +<TL4, ELI, EL>
A: TL4~TL3 A: TL4~TL3
B: TL3~TL2 B: TL3~TL2
... W: TL2~TL0
W: TL1~TL0
Y: -<TL0, ELI, EL> Y: -<TL0, ELI, EL>
LDP with UHP; ingress inserts ELs Figure 2: LDP with UHP; Ingress Inserts ELs
A: <--- [TL4, 1] A: <--- [TL4, 1]
B: <-- [TL3, 1] B: <-- [TL3, 1]
... W: <-- [TL2, 1]
W: <-- [TL1, 1]
Y: <-- [TL0, 1] Y: <-- [TL0, 1]
X --------------- A --------- B ... W ---------- Y X --------------- A --------- B --- W ---------- Y
LS: <TL4> <TL3> <TL0> Data Plane:
X-A: <TL4>
A-B: <TL3>
B-W: <TL2>
W-Y: <TL0>
Label Stack Operations:
X: +<TL4> X: +<TL4>
A: TL4~TL3 A: TL4~TL3
B: TL3~TL2 B: TL3~TL2
... W: TL2~TL0
W: TL1~TL0
Y: -<TL0> Y: -<TL0>
LDP with UHP; ingress does not insert ELs Figure 3: LDP with UHP; Ingress Does Not Insert ELs
Note that in Figure 3, above, the Egress Y is signaling it is EL-
capable, but the Ingress X has chosen not to insert ELs.
A: <--- [TL4, 1] A: <--- [TL4, 1]
B: <-- [TL3, 1] B: <-- [TL3, 1]
... W: <-- [TL2, 1]
W: <-- [TL1, 1]
Y: <-- [3, 1] Y: <-- [3, 1]
X --------------- A --------- B ... W ---------- Y X --------------- A --------- B --- W ---------- Y
Data Plane:
X-A: <TL4, ELI, EL>
A-B: <TL3,ELI,EL>
B-W: <TL2,ELI,EL>
W-Y: <ELI,EL>
Label Stack Operations:
X: +<TL4, ELI, EL> X: +<TL4, ELI, EL>
A: TL4~TL3 A: TL4~TL3
B: TL3~TL2 B: TL3~TL2
... W: -TL2
W: -TL1
Y: -<ELI, EL> Y: -<ELI, EL>
LDP with PHP; ingress inserts ELs Figure 4: LDP with PHP; Ingress Inserts ELs
A: <--- [TL4, 1] A: <--- [TL4, 1]
B: <-- [TL3, 1] B: <-- [TL3, 1]
... W: <-- [TL2, 1]
W: <-- [TL1, 1]
Y: <-- [3, 1] Y: <-- [3, 1]
VPN: <------------------------------------------ [AL] VPN: <------------------------------------------ [AL]
X --------------- A --------- B ... W ---------- Y X --------------- A --------- B --- W ---------- Y
LS: <TL4, AL> <TL3, AL> <AL> Data Plane:
X-A: <TL4, AL>
A-B: <TL3, AL>
B-W: <TL2, AL>
W-Y: <AL>
Label Stack Operations:
X: +<TL4, AL> X: +<TL4, AL>
A: TL4~TL3 A: TL4~TL3
B: TL3~TL2 B: TL3~TL2
... W: -TL2
W: -TL1
Y: -<AL> Y: -<AL>
LDP with PHP + VPN; ingress does not insert ELs Figure 5: LDP with PHP + VPN; Ingress Does Not Insert ELs
Note that in Figure 5, above, the Egress Y is signaling it is EL-
capable, but the Ingress X has chosen not to insert ELs.
A: <--- [TL4, 1] A: <--- [TL4, 1]
B: <-- [TL3, 1] B: <-- [TL3, 1]
... W: <-- [TL2, 1]
W: <-- [TL1, 1]
Y: <-- [3, 1] Y: <-- [3, 1]
VPN: <--------------------------------------------- [AL] VPN: <--------------------------------------------- [AL]
X --------------- A ------------ B ... W ---------- Y X --------------- A ------------ B --- W ---------- Y
LS: <TL4,ELI,EL,AL> <TL3,ELI,EL,AL> <ELI,EL,AL> Data Plane:
X-A: <TL4,ELI,EL,AL>
A-B: <TL3,ELI,EL,AL>
B-W: <TL2,ELI,EL,AL>
W-Y: <ELI,EL,AL>
Label Stack Operations:
X: +<TL4,ELI,EL,AL> X: +<TL4,ELI,EL,AL>
A: TL4~TL3 A: TL4~TL3
B: TL3~TL2 B: TL3~TL2
... W: -TL2
W: -TL1
Y: -<ELI,EL,AL> Y: -<ELI,EL,AL>
LDP with PHP + VPN; ingress inserts ELs Figure 6: LDP with PHP + VPN; Ingress Inserts ELs
8.2. LDP Over RSVP-TE 8.2. LDP over RSVP-TE
The following illustrates "LDP over RSVP-TE" tunnels. X and Y are Figure 7 illustrates "LDP over RSVP-TE" tunnels. X and Y are the
the ingress and egress (respectively) of the LDP tunnel; A and W are ingress and egress (respectively) of the LDP tunnel; A and W are the
the ingress and egress of the RSVP-TE tunnel. It is assumed that ingress and egress of the RSVP-TE tunnel. It is assumed that both
both the LDP and RSVP-TE tunnels have PHP. the LDP and RSVP-TE tunnels have PHP.
LDP with ELs, RSVP-TE without ELs
LDP: <--- [L4, 1] <------- [L3, 1] <--- [3, 1] LDP: <--- [L4, 1] <------- [L3, 1] <--- [3, 1]
RSVP-TE: <-- [Rn, 0] RSVP-TE: <-- [Rn, 0]
<-- [3, 0] <-- [3, 0]
X --------------- A --------- B ... W ---------- Y X --------------- A --------- B --- W ---------- Y
LS: <L4, ELI, EL> <Rn,L3,ELI,EL> ... <ELI, EL> Data Plane:
DP: +<L4, ELI, EL> L4~<Rn, L3> * -L1 -<ELI, EL> X-A: <L4, ELI, EL>
A-B: <Rn,L3,ELI,EL>
B-W: <L3,ELI,EL>
W-Y: <ELI,EL>
Label Stack Operations:
X: +<L4, ELI, EL>
A: <L4~L3>+Rn
B: -Rn
W: -L3
Y: -<ELI, EL>
Figure 2: LDP over RSVP-TE Tunnels Figure 7: LDP with ELs over RSVP-TE Tunnels without ELs
8.3. MPLS Applications 8.3. MPLS Applications
An ingress LSR X must keep state per unicast tunnel as to whether the For each unicast tunnel starting at an ingress LSR X, X must remember
egress for that tunnel can process entropy labels. X does not have whether the egress for that tunnel can process entropy labels. X
to keep state per application running over that tunnel. However, an does not have to keep state per application running over that tunnel.
ingress PE can choose on a per-application basis whether or not to However, an ingress PE can choose on a per-application basis whether
insert ELs. For example, X may have an application for which it does or not to insert ELs. For example, X may have an application for
not wish to use ECMP (e.g., circuit emulation), or for which it does which it does not wish to use ECMP (e.g., circuit emulation) or for
not know which keys to use for load balancing (e.g., Appletalk over a which it does not know which keys to use for load balancing (e.g.,
pseudowire). In either of those cases, X may choose not to insert Appletalk over a pseudowire). In either of those cases, X may choose
entropy labels, but may choose to insert entropy labels for an IP VPN not to insert entropy labels but may choose to insert entropy labels
over the same tunnel. for an IP VPN over the same tunnel.
9. Security Considerations 9. Security Considerations
This document describes advertisement of the capability to support This document describes advertisement of the capability to support
receipt of entropy labels which an ingress LSR may insert in MPLS receipt of entropy labels that an ingress LSR may insert in MPLS
packets in order to allow transit LSRs to attain better load packets in order to allow transit LSRs to attain better load
balancing across LAG and/or ECMP paths in the network. balancing across LAG and/or ECMP paths in the network.
This document does not introduce new security vulnerabilities to LDP, This document does not introduce new security vulnerabilities to LDP,
BGP or RSVP-TE. Please refer to the Security Considerations section BGP or RSVP-TE. Please refer to the Security Considerations sections
of these protocols ([RFC5036], [RFC4271] and [RFC3209]) for security of these protocols ([RFC5036], [RFC4271], and [RFC3209]) for security
mechanisms applicable to each. mechanisms applicable to each.
Given that there is no end-user control over the values used for Given that there is no end-user control over the values used for
entropy labels, there is little risk of Entropy Label forgery which entropy labels, there is little risk of entropy label forgery, which
could cause uneven load-balancing in the network. could cause uneven load balancing in the network. Note that if the
EL value is calculated only based on packet headers, then a
relatively efficient wiretapping interface could be added depending
on the function used to generate the EL value. An implementation may
protect against this by adding some other input to the generation of
the EL values that would make it harder to build a table of EL values
to tap given knowledge of the keys from the packet. For example, the
ingress LSR could generate a random input to the EL generation
process. In practice, many ECMP hashing algorithms contain a random
factor in any case so as to avoid polarization issues.
If Entropy Label Capability is not signaled from an egress PE to an If Entropy Label Capability is not signaled from an egress PE to an
ingress PE, due to, for example, malicious configuration activity on ingress PE, due to, for example, malicious configuration activity on
the egress PE, then the PE will fall back to not using entropy labels the egress PE, then the PE will fall back to not using entropy labels
for load-balancing traffic over LAG or ECMP paths which is in general for load balancing traffic over LAG or ECMP paths, which is, in
no worse than the behavior observed in current production networks. general, no worse than the behavior observed in current production
That said, it is recommended that operators monitor changes to PE networks. That said, it is recommended that operators monitor
configurations and, more importantly, the fairness of load changes to PE configurations and, more importantly, the fairness of
distribution over LAG or ECMP paths. If the fairness of load load distribution over LAG or ECMP paths. If the fairness of load
distribution over a set of paths changes that could indicate a distribution over a set of paths changes that could indicate a
misconfiguration, bug or other non-optimal behavior on their PEs and misconfiguration, bug, or other non-optimal behavior on their PEs,
they should take corrective action. and they should take corrective action.
10. IANA Considerations 10. IANA Considerations
10.1. Reserved Label for ELI 10.1. Reserved Label for ELI
IANA is requested to allocate a reserved label for the Entropy Label IANA has allocated a reserved label for the Entropy Label Indicator
Indicator (ELI) from the "Multiprotocol Label Switching Architecture (ELI) from the "Multiprotocol Label Switching Architecture (MPLS)
(MPLS) Label Values" Registry. Label Values" registry.
10.2. LDP Entropy Label Capability TLV 10.2. LDP Entropy Label Capability TLV
IANA is requested to allocate the next available value from the IETF IANA has allocated the value of 0x0206 from the IETF Consensus range
Consensus range (0x0001-0x07FF) in the LDP TLV Type Name Space (0x0001-0x07FF) in the "TLV Type Name Space" registry as the "Entropy
Registry as the "Entropy Label Capability TLV". Label Capability TLV".
10.3. BGP Entropy Label Capability Attribute 10.3. BGP Entropy Label Capability Attribute
IANA is requested to allocate the next available Path Attribute Type IANA has allocated the Path Attribute Type Code 28 from the "BGP Path
Code from the "BGP Path Attributes" registry as the "BGP Entropy Attributes" registry as the "BGP Entropy Label Capability Attribute".
Label Capability Attribute".
10.4. RSVP-TE Entropy Label Capability flag 10.4. RSVP-TE Entropy Label Capability Flag
IANA is requested to allocate a new bit from the "Attribute Flags" IANA has allocated a new bit from the "Attribute Flags" sub-registry
sub-registry of the "RSVP TE Parameters" registry. of the "Resource Reservation Protocol-Traffic Engineering (RSVP-TE)
Parameters" registry.
Bit | Name | Attribute | Attribute | RRO Bit | Name | Attribute | Attribute | RRO
No | | Flags Path | Flags Resv | No | | Flags Path | Flags Resv |
----+--------------------------+------------+------------+----- ----+--------------------------+------------+------------+-----
TBD Entropy Label Capability Yes Yes No 9 Entropy Label Capability Yes Yes No
11. Acknowledgments 11. Acknowledgments
We wish to thank Ulrich Drafz for his contributions, as well as the We wish to thank Ulrich Drafz for his contributions, as well as the
entire 'hash label' team for their valuable comments and discussion. entire "hash label" team for their valuable comments and discussion.
Sincere thanks to Nischal Sheth for his many suggestions and Sincere thanks to Nischal Sheth for his many suggestions and comments
comments, and his careful reading of the document, especially with and for his careful reading of the document, especially with regard
regard to data plane processing of entropy labels. to data plane processing of entropy labels.
Most of the work Kireeti Kompella did on this document was done while
he was at Juniper Networks. He has since moved to Contrail Systems.
12. References 12. References
12.1. Normative References 12.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
Label Switching Architecture", RFC 3031, January 2001. Label Switching Architecture", RFC 3031, January 2001.
[RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., [RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y.,
Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack
skipping to change at page 22, line 21 skipping to change at page 24, line 7
Multipoint and Multipoint-to-Multipoint Label Switched Multipoint and Multipoint-to-Multipoint Label Switched
Paths", RFC 6388, November 2011. Paths", RFC 6388, November 2011.
[RFC6391] Bryant, S., Filsfils, C., Drafz, U., Kompella, V., Regan, [RFC6391] Bryant, S., Filsfils, C., Drafz, U., Kompella, V., Regan,
J., and S. Amante, "Flow-Aware Transport of Pseudowires J., and S. Amante, "Flow-Aware Transport of Pseudowires
over an MPLS Packet Switched Network", RFC 6391, over an MPLS Packet Switched Network", RFC 6391,
November 2011. November 2011.
Appendix A. Applicability of LDP Entropy Label Capability TLV Appendix A. Applicability of LDP Entropy Label Capability TLV
In the case of unlabeled IPv4 (Internet) traffic, the Best Current In the case of unlabeled IPv4 (Internet) traffic, the best practice
Practice is for an egress LSR to propagate eBGP learned routes within is for an egress LSR to propagate eBGP learned routes within a
a SP's Autonomous System after resetting the BGP next-hop attribute Service Provider's Autonomous System after resetting the BGP next-hop
to one of its Loopback IP addresses. That Loopback IP address is attribute to one of its loopback IP addresses. That loopback IP
injected into the Service Provider's IGP and, concurrently, a label address is injected into the Service Provider's IGP and,
assigned to it via LDP. Thus, when an ingress LSR is performing a concurrently, a label assigned to it via LDP. Thus, when an ingress
forwarding lookup for a BGP destination it recursively resolves the LSR is performing a forwarding lookup for a BGP destination, it
associated next-hop to a Loopback IP address and associated LDP label recursively resolves the associated next hop to a loopback IP address
of the egress LSR. and associated LDP label of the egress LSR.
Thus, in the context of unlabeled IPv4 traffic, the LDP Entropy Label Thus, in the context of unlabeled IPv4 traffic, the LDP Entropy Label
Capability TLV will typically be applied only to the FEC for the Capability TLV will typically be applied only to the FEC for the
Loopback IP address of the egress LSR and the egress LSR need not loopback IP address of the egress LSR, and the egress LSR need not
announce an entropy label capability for the eBGP learned route. announce an Entropy Label Capability for the eBGP learned route.
Authors' Addresses Authors' Addresses
Kireeti Kompella Kireeti Kompella
Juniper Networks Contrail Systems
1194 N. Mathilda Ave. 2350 Mission College Blvd.
Sunnyvale, CA 94089 Santa Clara, CA 95054
US US
Email: kireeti.kompella@gmail.com EMail: kireeti.kompella@gmail.com
John Drake John Drake
Juniper Networks Juniper Networks
1194 N. Mathilda Ave. 1194 N. Mathilda Ave.
Sunnyvale, CA 94089 Sunnyvale, CA 94089
US US
Email: jdrake@juniper.net EMail: jdrake@juniper.net
Shane Amante Shane Amante
Level 3 Communications, LLC Level 3 Communications, Inc.
1025 Eldorado Blvd 1025 Eldorado Blvd
Broomfield, CO 80021 Broomfield, CO 80021
US US
Email: shane@level3.net EMail: shane@level3.net
Wim Henderickx Wim Henderickx
Alcatel-Lucent Alcatel-Lucent
Copernicuslaan 50 Copernicuslaan 50
2018 Antwerp 2018 Antwerp
Belgium Belgium
Email: wim.henderickx@alcatel-lucent.com EMail: wim.henderickx@alcatel-lucent.com
Lucy Yong Lucy Yong
Huawei USA Huawei USA
5340 Legacy Dr. 5340 Legacy Dr.
Plano, TX 75024 Plano, TX 75024
US US
Email: lucy.yong@huawei.com EMail: lucy.yong@huawei.com
 End of changes. 162 change blocks. 
345 lines changed or deleted 389 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/