< draft-ietf-bess-evpn-inter-subnet-forwarding-05.txt   draft-ietf-bess-evpn-inter-subnet-forwarding-06.txt >
L2VPN Workgroup A. Sajassi, Ed. L2VPN Workgroup A. Sajassi, Ed.
INTERNET-DRAFT S. Salam INTERNET-DRAFT S. Salam
Intended Status: Standards Track S. Thoria Intended Status: Standards Track S. Thoria
Cisco Cisco
J. Drake J. Drake
Juniper Juniper
J. Rabadan J. Rabadan
Nokia Nokia
Expires: January 18, 2019 July 18, 2018 Expires: August 3, 2019 February 3, 2019
Integrated Routing and Bridging in EVPN Integrated Routing and Bridging in EVPN
draft-ietf-bess-evpn-inter-subnet-forwarding-05 draft-ietf-bess-evpn-inter-subnet-forwarding-06
Abstract Abstract
EVPN provides an extensible and flexible multi-homing VPN solution EVPN provides an extensible and flexible multi-homing VPN solution
over an MPLS/IP network for intra-subnet connectivity among Tenant over an MPLS/IP network for intra-subnet connectivity among Tenant
Systems and End Devices that can be physical or virtual. However, Systems and End Devices that can be physical or virtual. However,
there are scenarios for which there is a need for a dynamic and there are scenarios for which there is a need for a dynamic and
efficient inter-subnet connectivity among these Tenant Systems and efficient inter-subnet connectivity among these Tenant Systems and
End Devices while maintaining the multi-homing capabilities of EVPN. End Devices while maintaining the multi-homing capabilities of EVPN.
This document describes an Integrated Routing and Bridging (IRB) This document describes an Integrated Routing and Bridging (IRB)
skipping to change at page 2, line 28 skipping to change at page 2, line 28
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 6 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 EVPN PE Model for IRB Operation . . . . . . . . . . . . . . . . 7 2 EVPN PE Model for IRB Operation . . . . . . . . . . . . . . . . 7
3 Symmetric and Asymmetric IRB . . . . . . . . . . . . . . . . . 8 3 Symmetric and Asymmetric IRB . . . . . . . . . . . . . . . . . 8
3.1 IRB Interface and its MAC & IP addresses . . . . . . . . . . 11 3.1 IRB Interface and its MAC & IP addresses . . . . . . . . . . 11
3.2 Symmetric IRB Procedures . . . . . . . . . . . . . . . . . . 13 3.2 Symmetric IRB Procedures . . . . . . . . . . . . . . . . . . 13
3.2.1 Control Plane - Ingress PE . . . . . . . . . . . . . . . 13 3.2.1 Control Plane - Ingress PE . . . . . . . . . . . . . . . 13
3.2.2 Control Plane - Egress PE . . . . . . . . . . . . . . . 13 3.2.2 Control Plane - Egress PE . . . . . . . . . . . . . . . 14
3.2.3 Data Plane - Ingress PE . . . . . . . . . . . . . . . . 14 3.2.3 Data Plane - Ingress PE . . . . . . . . . . . . . . . . 15
3.2.4 Data Plane - Egress PE . . . . . . . . . . . . . . . . . 15 3.2.4 Data Plane - Egress PE . . . . . . . . . . . . . . . . . 15
3.3 Asymmetric IRB Procedures . . . . . . . . . . . . . . . . . 15 3.3 Asymmetric IRB Procedures . . . . . . . . . . . . . . . . . 16
3.3.1 Control Plane - Ingress PE . . . . . . . . . . . . . . . 15 3.3.1 Control Plane - Ingress PE . . . . . . . . . . . . . . . 16
3.3.2 Control Plane - Egress PE . . . . . . . . . . . . . . . 16 3.3.2 Control Plane - Egress PE . . . . . . . . . . . . . . . 17
3.3.3 Data Plane - Ingress PE . . . . . . . . . . . . . . . . 17 3.3.3 Data Plane - Ingress PE . . . . . . . . . . . . . . . . 17
3.3.4 Data Plane - Egress PE . . . . . . . . . . . . . . . . . 18 3.3.4 Data Plane - Egress PE . . . . . . . . . . . . . . . . . 18
4 Mobility Procedure . . . . . . . . . . . . . . . . . . . . . . . 18 4 Mobility Procedure . . . . . . . . . . . . . . . . . . . . . . . 18
4.1 Mobility Procedure for Symmetric IRB . . . . . . . . . . . . 19 4.1 Mobility Procedure for Symmetric IRB . . . . . . . . . . . . 19
4.1.1 Initiating an ARP Request upon a Move . . . . . . . . . 19 4.1.1 Initiating an ARP Request upon a Move . . . . . . . . . 19
4.1.2 Sending Data Traffic without an ARP Request . . . . . . 20 4.1.2 Sending Data Traffic without an ARP Request . . . . . . 20
4.1.3 Silent Host . . . . . . . . . . . . . . . . . . . . . . 21 4.1.3 Silent Host . . . . . . . . . . . . . . . . . . . . . . 22
5 BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 22 5 BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.1 Router's MAC Extended Community . . . . . . . . . . . . . . 22 5.1 Router's MAC Extended Community . . . . . . . . . . . . . . 22
6 Operational Models for Symmetric Inter-Subnet Forwarding . . . . 23 6 Operational Models for Symmetric Inter-Subnet Forwarding . . . . 23
6.1 IRB forwarding on NVEs for Tenant Systems . . . . . . . . . 23 6.1 IRB forwarding on NVEs for Tenant Systems . . . . . . . . . 23
6.1.1 Control Plane Operation . . . . . . . . . . . . . . . . 24 6.1.1 Control Plane Operation . . . . . . . . . . . . . . . . 25
6.1.2 Data Plane Operation . . . . . . . . . . . . . . . . . . 26 6.1.2 Data Plane Operation . . . . . . . . . . . . . . . . . . 26
6.2 IRB forwarding on NVEs for Subnets behind Tenant Systems . . 27 6.2 IRB forwarding on NVEs for Subnets behind Tenant Systems . . 27
6.2.1 Control Plane Operation . . . . . . . . . . . . . . . . 28 6.2.1 Control Plane Operation . . . . . . . . . . . . . . . . 28
6.2.2 Data Plane Operation . . . . . . . . . . . . . . . . . . 29 6.2.2 Data Plane Operation . . . . . . . . . . . . . . . . . . 29
7 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 30 7 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 30
8 Security Considerations . . . . . . . . . . . . . . . . . . . . 30 8 Security Considerations . . . . . . . . . . . . . . . . . . . . 30
9 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 31 9 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 31
10 References . . . . . . . . . . . . . . . . . . . . . . . . . . 31 10 References . . . . . . . . . . . . . . . . . . . . . . . . . . 31
10.1 Normative References . . . . . . . . . . . . . . . . . . . 31 10.1 Normative References . . . . . . . . . . . . . . . . . . . 31
10.2 Informative References . . . . . . . . . . . . . . . . . . 31 10.2 Informative References . . . . . . . . . . . . . . . . . . 31
skipping to change at page 4, line 28 skipping to change at page 4, line 28
IRB: Integrated Routing and Bridging interface. It connects an IP-VRF IRB: Integrated Routing and Bridging interface. It connects an IP-VRF
to a BD (or subnet). to a BD (or subnet).
MAC-VRF: A Virtual Routing and Forwarding table for Media Access MAC-VRF: A Virtual Routing and Forwarding table for Media Access
Control (MAC) addresses on an NVE/PE, as per [RFC7432]. A MAC-VRF is Control (MAC) addresses on an NVE/PE, as per [RFC7432]. A MAC-VRF is
also an instantiation of an EVI in an NVE/PE. also an instantiation of an EVI in an NVE/PE.
ML: MAC address length. ML: MAC address length.
ND: Neighbor Discovery Protocol. NDP: Neighbor Discovery Protocol.
NVE: Network Virtualization Edge. NVE: Network Virtualization Edge.
GENEVE: Generic Network Virtualization Encapsulation, [GENEVE]. GENEVE: Generic Network Virtualization Encapsulation, [GENEVE].
NVO: Network Virtualization Overlays. NVO: Network Virtualization Overlays.
RT-2: EVPN route type 2, i.e., MAC/IP advertisement route, as defined RT-2: EVPN route type 2, i.e., MAC/IP advertisement route, as defined
in [RFC7432]. in [RFC7432].
skipping to change at page 6, line 9 skipping to change at page 6, line 9
VXLAN: Virtual Extensible LAN, as in [RFC7348]. VXLAN: Virtual Extensible LAN, as in [RFC7348].
This document also assumes familiarity with the terminology of This document also assumes familiarity with the terminology of
[RFC7432], [RFC8365] and [RFC7365]. [RFC7432], [RFC8365] and [RFC7365].
1 Introduction 1 Introduction
EVPN provides an extensible and flexible multi-homing VPN solution EVPN provides an extensible and flexible multi-homing VPN solution
over an MPLS/IP network for intra-subnet connectivity among Tenant over an MPLS/IP network for intra-subnet connectivity among Tenant
Systems (TS's) and End Devices that can be physical or virtual; where Systems (TSes) and End Devices that can be physical or virtual; where
an IP subnet is represented by an EVI for a VLAN-based service or by an IP subnet is represented by an EVI for a VLAN-based service or by
an <EVI, VLAN> for a VLAN-aware bundle service. However, there are an <EVI, VLAN> for a VLAN-aware bundle service. However, there are
scenarios for which there is a need for a dynamic and efficient scenarios for which there is a need for a dynamic and efficient
inter-subnet connectivity among these Tenant Systems and End Devices inter-subnet connectivity among these Tenant Systems and End Devices
while maintaining the multi-homing capabilities of EVPN. This while maintaining the multi-homing capabilities of EVPN. This
document describes an Integrated Routing and Bridging (IRB) solution document describes an Integrated Routing and Bridging (IRB) solution
based on EVPN to address such requirements. based on EVPN to address such requirements.
The inter-subnet communication is traditionally achieved at The inter-subnet communication is traditionally achieved at
centralized L3 Gateway (L3GW) devices where all the inter-subnet centralized L3 Gateway (L3GW) devices where all the inter-subnet
forwarding are performed and all the inter-subnet communication forwarding are performed and all the inter-subnet communication
policies are enforced. When two Tenant Systems (TS's) belonging to policies are enforced. When two TSes belonging to two different
two different subnets connected to the same PE node, wanted to subnets connected to the same PE node, wanted to communicate with
communicate with each other, their traffic needed to be back hauled each other, their traffic needed to be back hauled from the PE node
from the PE node all the way to the centralized gateway node where all the way to the centralized gateway node where inter-subnet
inter-subnet switching is performed and then back to the PE node. For switching is performed and then back to the PE node. For today's
today's large multi-tenant data center, this scheme is very large multi-tenant data center, this scheme is very inefficient and
inefficient and sometimes impractical. sometimes impractical.
In order to overcome the drawback of centralized L3GW approach, IRB In order to overcome the drawback of centralized L3GW approach, IRB
functionality is needed on the PE nodes (also referred to as EVPN functionality is needed on the PE nodes (also referred to as EVPN
NVEs) attached to TS's in order to avoid inefficient forwarding of NVEs) attached to TSes in order to avoid inefficient forwarding of
tenant traffic (i.e., avoid back-hauling and hair-pinning). When a PE tenant traffic (i.e., avoid back-hauling and hair-pinning). When a PE
with IRB capability receives tenant traffic over a single Attachment with IRB capability receives tenant traffic over a single Attachment
Circuit (AC), it can not only locally bridged the tenant intra-subnet Circuit (AC), it can not only locally bridged the tenant intra-subnet
traffic but also can locally route the tenant inter-subnet traffic on traffic but also can locally route the tenant inter-subnet traffic on
a packet by packet basis thus meeting the requirements for both intra a packet by packet basis thus meeting the requirements for both intra
and inter-subnet forwarding and avoiding non-optimum traffic and inter-subnet forwarding and avoiding non-optimum traffic
forwarding associate with centralized L3GW approach. forwarding associate with centralized L3GW approach.
Some TS's run non-IP protocols in conjunction with their IP traffic. Some TSes run non-IP protocols in conjunction with their IP traffic.
Therefore, it is important to handle both kinds of traffic optimally Therefore, it is important to handle both kinds of traffic optimally
- e.g., to bridge non-IP and intra-subnet traffic and to route inter- - e.g., to bridge non-IP and intra-subnet traffic and to route inter-
subnet IP traffic. Therefore, the solution needs to meet the subnet IP traffic. Therefore, the solution needs to meet the
following requirements: following requirements:
R1: The solution MUST allow for both inter-subnet and intra-subnet R1: The solution MUST allow for both inter-subnet and intra-subnet
traffic belonging to the same tenant to be locally routed and bridged traffic belonging to the same tenant to be locally routed and bridged
respectively. The solution MUST provide IP routing for inter-subnet respectively. The solution MUST provide IP routing for inter-subnet
traffic and Ethernet Bridging for intra-subnet traffic. traffic and Ethernet Bridging for intra-subnet traffic.
skipping to change at page 11, line 46 skipping to change at page 11, line 46
gateway IP address but its own MAC address. These MAC addresses are gateway IP address but its own MAC address. These MAC addresses are
aliased to the same anycast default gateway IP address through the aliased to the same anycast default gateway IP address through the
use of the Default Gateway extended community as specified in use of the Default Gateway extended community as specified in
[RFC7432], which is carried in the EVPN MAC/IP Advertisement routes. [RFC7432], which is carried in the EVPN MAC/IP Advertisement routes.
On each PE, this default gateway IP address along with its associated On each PE, this default gateway IP address along with its associated
MAC addresses correspond to the IRB interface connecting the BT MAC addresses correspond to the IRB interface connecting the BT
associated with the tenant's <EVI, VLAN> to the corresponding associated with the tenant's <EVI, VLAN> to the corresponding
tenant's IP-VRF. tenant's IP-VRF.
It is worth noting that if the applications that are running on the It is worth noting that if the applications that are running on the
TS's are employing or relying on any form of MAC security, then TSes are employing or relying on any form of MAC security, then
either the first model (i.e. using anycast MAC address) should be either the first model (i.e. using anycast MAC address) should be
used to ensure that the applications receive traffic from the same used to ensure that the applications receive traffic from the same
IRB interface MAC address that they are sending to, or if the second IRB interface MAC address that they are sending to, or if the second
model is used, then the IRB interface MAC address MUST be the one model is used, then the IRB interface MAC address MUST be the one
used in the initial ARP reply for that TS. used in the initial ARP reply or ND Neighbor Advertisement (NA) for
that TS.
Although both of these options are equally applicable to both Although both of these options are equally applicable to both
symmetric and asymmetric IRB, the option-1 is recommended because of symmetric and asymmetric IRB, the option-1 is recommended because of
the ease of anycast MAC address provisioning on not only the IRB the ease of anycast MAC address provisioning on not only the IRB
interface associated with a given subnet across all the PEs interface associated with a given subnet across all the PEs
corresponding to that EVI but also on all IRB interfaces associated corresponding to that EVI but also on all IRB interfaces associated
with all the tenant's subnets across all the PEs corresponding to all with all the tenant's subnets across all the PEs corresponding to all
the EVIs for that tenant. Furthermore, it simplifies the operation as the EVIs for that tenant. Furthermore, it simplifies the operation as
there is no need for Default Gateway extended community advertisement there is no need for Default Gateway extended community advertisement
and its associated MAC aliasing procedure. Yet another advantage is and its associated MAC aliasing procedure. Yet another advantage is
that following host mobility, the host does not need to refresh the that following host mobility, the host does not need to refresh the
default GW ARP entry. default GW ARP/ND entry.
If option-1 is used, an implementation MAY choose to auto-derive the If option-1 is used, an implementation MAY choose to auto-derive the
anycast MAC address. If auto-derivation is used, the anycast MAC MUST anycast MAC address. If auto-derivation is used, the anycast MAC MUST
be auto-derived out of the following ranges (which are defined in be auto-derived out of the following ranges (which are defined in
[RFC5798]): [RFC5798]):
- Anycast IPv4 IRB case: 00-00-5E-00-01-{VRID} (in hex, in Internet - Anycast IPv4 IRB case: 00-00-5E-00-01-{VRID} (in hex, in Internet
standard bit-order) standard bit-order)
- Anycast IPv6 IRB case: 00-00-5E-00-02-{VRID} (in hex, in Internet - Anycast IPv6 IRB case: 00-00-5E-00-02-{VRID} (in hex, in Internet
skipping to change at page 12, line 40 skipping to change at page 12, line 41
default value for the VRID octet is '01'. Auto-derivation of the default value for the VRID octet is '01'. Auto-derivation of the
anycast MAC can only be used if there is certainty that the auto- anycast MAC can only be used if there is certainty that the auto-
derived MAC does not collide with any customer MAC address. derived MAC does not collide with any customer MAC address.
In addition to IP anycast addresses, IRB interfaces can be configured In addition to IP anycast addresses, IRB interfaces can be configured
with non-anycast IP addresses for the purpose of OAM (such as with non-anycast IP addresses for the purpose of OAM (such as
traceroute/ping to these interfaces) for both symmetric and traceroute/ping to these interfaces) for both symmetric and
asymmetric IRB. These IP addresses need to be distributed as VPN asymmetric IRB. These IP addresses need to be distributed as VPN
routes when PEs operating in symmetric IRB mode. However, they don't routes when PEs operating in symmetric IRB mode. However, they don't
need to be distributed if the PEs are operating in asymmetric IRB need to be distributed if the PEs are operating in asymmetric IRB
mode and the non-anycast IP addresses are configured with individual mode and the non-anycast IP addresses are configured along with
MACs. individual MACs.
Irrespective of using only the anycast address or both anycast and Irrespective of using only the anycast address or both anycast and
non-anycast addresses on the same IRB, when a TS sends an ARP request non-anycast addresses on the same IRB, when a TS sends an ARP request
to the PE that is attached to, the ARP request is sent for the or ND Neighbor Solicitation (NS) to the PE that is attached to, the
anycast IP address of the IRB interface associated with the TS's request is sent for the anycast IP address of the IRB interface
subnet. For example, in figure 4, TS1 is configured with the anycast associated with the TS's subnet and the reply will use anycast MAC
IPx address as its default gateway IP address and thus when it sends address (in both Source MAC in the Ethernet header and Sender
an ARP request for IPx (anycast IP address of the IRB interface for hardware address in the payload). For example, in figure 4, TS1 is
BT1), the PE1 sends an ARP reply with the MACx which is the anycast configured with the anycast IPx address as its default gateway IP
MAC address of that IRB interface. Traffic routed from IP-VRF1 to TS1 address and thus when it sends an ARP request for IPx (anycast IP
SHOULD use the anycast MAC address as source MAC address. address of the IRB interface for BT1), the PE1 sends an ARP reply
with the MACx which is the anycast MAC address of that IRB interface.
Traffic routed from IP-VRF1 to TS1 SHOULD use the anycast MAC address
as source MAC address.
3.2 Symmetric IRB Procedures 3.2 Symmetric IRB Procedures
3.2.1 Control Plane - Ingress PE 3.2.1 Control Plane - Ingress PE
When a PE (e.g., PE1 in figure 4 above) learns MAC and IP address of When a PE (e.g., PE1 in figure 4 above) learns MAC and IP address of
a TS (via an ARP request), it adds the MAC address to the a TS (via an ARP request), it adds the MAC address to the
corresponding MAC-VRF/BT of that tenant's subnet and adds the IP corresponding MAC-VRF/BT of that tenant's subnet and adds the IP
address to the IP-VRF for that tenant. Furthermore, it adds this TS's address to the IP-VRF for that tenant. Furthermore, it adds this TS's
MAC and IP address association to its ARP table. It then builds an MAC and IP address association to its ARP table. It then builds an
skipping to change at page 17, line 6 skipping to change at page 17, line 14
3.3.2 Control Plane - Egress PE 3.3.2 Control Plane - Egress PE
When a PE (e.g., PE2 in figure 4 above) receives this EVPN MAC/IP When a PE (e.g., PE2 in figure 4 above) receives this EVPN MAC/IP
Advertisement route advertisement, it performs the following: Advertisement route advertisement, it performs the following:
- Using MAC-VRF route target, it identifies the corresponding MAC-VRF - Using MAC-VRF route target, it identifies the corresponding MAC-VRF
and imports the MAC address into it. For asymmetric IRB mode, it is and imports the MAC address into it. For asymmetric IRB mode, it is
assumed that all PEs participating in a tenant's VPN are configured assumed that all PEs participating in a tenant's VPN are configured
with all subnets and corresponding MAC-VRFs/BTs even if there are no with all subnets and corresponding MAC-VRFs/BTs even if there are no
locally attached TS's for some of these subnets. The reason for this locally attached TSes for some of these subnets. The reason for this
is because ingress PE needs to do forwarding based on destination is because ingress PE needs to do forwarding based on destination
TS's MAC address and does proper NVO tunnel encapsulation which are TS's MAC address and does proper NVO tunnel encapsulation which are
property of a lookup in MAC-VRF/BT. An implementation may choose to property of a lookup in MAC-VRF/BT. An implementation may choose to
consolidate the lookup at the ingress PE's IP-VRF with the lookup at consolidate the lookup at the ingress PE's IP-VRF with the lookup at
the ingress PE's destination subnet MAC-VRF. Consideration for such the ingress PE's destination subnet MAC-VRF. Consideration for such
consolidation of lookups is an implementation exercise and thus its consolidation of lookups is an implementation exercise and thus its
specification is outside the scope of this document. specification is outside the scope of this document.
- Using MAC-VRF route target, it identifies the corresponding ARP - Using MAC-VRF route target, it identifies the corresponding ARP
table for the tenant and it adds an entry to the ARP table for the table for the tenant and it adds an entry to the ARP table for the
skipping to change at page 19, line 50 skipping to change at page 20, line 8
The target NVE upon receiving this ARP request, updates its MAC-VRF, The target NVE upon receiving this ARP request, updates its MAC-VRF,
IP-VRF, and ARP table with the host MAC, IP, and local adjacency IP-VRF, and ARP table with the host MAC, IP, and local adjacency
information (e.g., local interface). information (e.g., local interface).
Since this NVE has previously learned the same MAC and IP addresses Since this NVE has previously learned the same MAC and IP addresses
from the source NVE, it recognizes that there has been a MAC move and from the source NVE, it recognizes that there has been a MAC move and
it initiates MAC mobility procedures per [RFC7432] by advertising an it initiates MAC mobility procedures per [RFC7432] by advertising an
EVPN MAC/IP route with both the MAC and IP addresses filled in along EVPN MAC/IP route with both the MAC and IP addresses filled in along
with MAC Mobility Extended Community with the sequence number with MAC Mobility Extended Community with the sequence number
incremented by one. incremented by one. The target NVE also exercises the MAC duplication
detection procedure in section 15.1 of [RFC 7432].
The source NVE upon receiving this MAC/IP advertisement, realizes The source NVE upon receiving this MAC/IP advertisement, realizes
that the MAC has moved to the target NVE. It updates its MAC-VRF and that the MAC has moved to the target NVE. It updates its MAC-VRF and
IP-VRF table accordingly with the adjacency information of the target IP-VRF table accordingly with the adjacency information of the target
NVE and withdraws its EVPN MAC/IP route. Furthermore, it sends an ARP NVE and withdraws its EVPN MAC/IP route. Furthermore, it sends an ARP
probe locally to ensure that the MAC is gone and it deletes its ARP probe locally to ensure that the MAC is gone. After no ARP response
entry corresponding to that <IP, MAC> when there is no ARP response. is received, it then deletes its ARP entry corresponding to that <IP,
MAC>. If an ARP response is received, the source NVE updates its ARP
entry timer for that <IP, MAC> and re-advertises an EVPN MAC/IP route
for that <IP, MAC> along with MAC Mobility Extended Community with
the sequence number incremented by one. The source NVE also exercises
the MAC duplication detection procedure in section 15.1 of [RFC
7432].
All other remote NVE devices upon receiving the MAC/IP advertisement All other remote NVE devices upon receiving the MAC/IP advertisement
route with MAC Mobility extended community compare the sequence route with MAC Mobility extended community compare the sequence
number in this advertisement with the one previously received. If the number in this advertisement with the one previously received. If the
new sequence number is greater than the old one, then they update the new sequence number is greater than the old one, then they update the
MAC/IP addresses of the TS in their corresponding MAC-VRF and IP-VRF MAC/IP addresses of the TS in their corresponding MAC-VRF and IP-VRF
tables to point to the target NVE. Furthermore, upon receiving the tables to point to the target NVE. Furthermore, upon receiving the
MAC/IP withdraw for the TS from the source NVE, these remote PEs MAC/IP withdraw for the TS from the source NVE, these remote PEs
perform the cleanups for their BGP tables. perform the cleanups for their BGP tables.
skipping to change at page 20, line 51 skipping to change at page 21, line 17
only the MAC address filled in along with MAC Mobility Extended only the MAC address filled in along with MAC Mobility Extended
Community with the sequence number incremented by one. Community with the sequence number incremented by one.
- The source NVE upon receiving this MAC/IP advertisement, realizes - The source NVE upon receiving this MAC/IP advertisement, realizes
that the MAC has moved to the new NVE. It updates its MAC-VRF table that the MAC has moved to the new NVE. It updates its MAC-VRF table
accordingly by updating the adjacency information for that MAC accordingly by updating the adjacency information for that MAC
address to point to the target NVE and withdraws its EVPN MAC/IP address to point to the target NVE and withdraws its EVPN MAC/IP
route that has only the MAC address (if it has advertised such route route that has only the MAC address (if it has advertised such route
previously). Furthermore, it searches its ARP table for this MAC and previously). Furthermore, it searches its ARP table for this MAC and
sends an ARP probe for this <MAC,IP> pair. The ARP request message is sends an ARP probe for this <MAC,IP> pair. The ARP request message is
sent both locally to all attached TS's in that subnet as well as it sent both locally to all attached TSes in that subnet as well as it
is sent to other NVEs participating in that subnet including the is sent to other NVEs participating in that subnet including the
target NVE. target NVE.
- The target NVE passes the ARP request to its locally attached TS's - The target NVE passes the ARP request to its locally attached TSes
and when it receives the ARP response, it sends an EVPN MAC/IP and when it receives the ARP response, it sends an EVPN MAC/IP
advertisement route with both the MAC and IP addresses filled in advertisement route with both the MAC and IP addresses filled in
along with MAC Mobility Extended Community with the sequence number along with MAC Mobility Extended Community with the sequence number
set to the same value as the one for MAC-only advertisement route it set to the same value as the one for MAC-only advertisement route it
sent previously. sent previously.
- When the source NVE receives the EVPN MAC/IP advertisement, it - When the source NVE receives the EVPN MAC/IP advertisement, it
updates its IP-VRF table with the new adjacency information updates its IP-VRF table with the new adjacency information
(pointing to the target NVE) and deletes the associated ARP entry (pointing to the target NVE) and deletes the associated ARP entry
from its ARP table. Furthermore, it withdraws its previously from its ARP table. Furthermore, it withdraws its previously
skipping to change at page 21, line 45 skipping to change at page 22, line 12
an unicast ARP request to the host and when receiving an ARP an unicast ARP request to the host and when receiving an ARP
response, it follows the procedure outlined in section 4.1.1. response, it follows the procedure outlined in section 4.1.1.
4.1.3 Silent Host 4.1.3 Silent Host
In this scenario when a TS moves from a source NVE to a target NVE, In this scenario when a TS moves from a source NVE to a target NVE,
the TS is silent and it neither initiates an ARP request nor it sends the TS is silent and it neither initiates an ARP request nor it sends
any data traffic. Therefore, neither the target nor the source NVEs any data traffic. Therefore, neither the target nor the source NVEs
are aware of the MAC move. are aware of the MAC move.
On the source NVE, the MAC age-out timer expires and as the result it On the source NVE, the MAC age-out timer (for the silent host that
triggers an ARP probe on the source NVE. The ARP request gets sent has moved) expires and as the result it triggers an ARP probe on the
both locally to all the attached TS's on that subnet as well as it source NVE. The ARP request gets sent both locally to all the
gets sent to all the remote NVEs (including the target NVE) attached TSes on that subnet as well as it gets sent to all the
participating in that subnet. It also withdraw the EVPN MAC/IP route remote NVEs (including the target NVE) participating in that subnet.
with only the MAC address (if it has previously advertised such a It also withdraw the EVPN MAC/IP route with only the MAC address (if
route). it has previously advertised such a route).
The target NVE passes the ARP request to its locally attached TS's The target NVE passes the ARP request to its locally attached TSes
and when it receives the ARP response, it sends an EVPN MAC/IP and when it receives the ARP response, it sends an EVPN MAC/IP
advertisement route with both the MAC and IP addresses filled in advertisement route with both the MAC and IP addresses filled in
along with MAC Mobility Extended Community with the sequence number along with MAC Mobility Extended Community with the sequence number
incremented by one. incremented by one.
When the source NVE receives the EVPN MAC/IP advertisement, it When the source NVE receives the EVPN MAC/IP advertisement, it
updates its IP-VRF table with the new adjacency information updates its IP-VRF table with the new adjacency information
(pointing to the target NVE) and deletes the associated ARP entry (pointing to the target NVE) and deletes the associated ARP entry
from its ARP table. Furthermore, it withdraws its previously from its ARP table. Furthermore, it withdraws its previously
advertised EVPN MAC/IP route with both the MAC and IP addresses. advertised EVPN MAC/IP route with both the MAC and IP addresses.
skipping to change at page 24, line 12 skipping to change at page 24, line 20
Since VxLAN and GENEVE encapsulations require inner Ethernet header Since VxLAN and GENEVE encapsulations require inner Ethernet header
(inner MAC SA/DA), and since for inter-subnet traffic, TS MAC address (inner MAC SA/DA), and since for inter-subnet traffic, TS MAC address
cannot be used, the ingress NVE's MAC address is used as inner MAC cannot be used, the ingress NVE's MAC address is used as inner MAC
SA. The NVE's MAC address is the device MAC address and it is common SA. The NVE's MAC address is the device MAC address and it is common
across all MAC-VRFs and IP-VRFs. This MAC address is advertised using across all MAC-VRFs and IP-VRFs. This MAC address is advertised using
the new EVPN Router's MAC Extended Community (section 5.1). the new EVPN Router's MAC Extended Community (section 5.1).
Figure 6 below illustrates this scenario where a given tenant (e.g., Figure 6 below illustrates this scenario where a given tenant (e.g.,
an IP-VPN instance) has three subnets represented by MAC-VRF1, MAC- an IP-VPN instance) has three subnets represented by MAC-VRF1, MAC-
VRF2, and MAC-VRF3 across two NVEs. There are five TS's that are VRF2, and MAC-VRF3 across two NVEs. There are five TSes that are
associated with these three MAC-VRFs - i.e., TS1, TS4, and TS5 are associated with these three MAC-VRFs - i.e., TS1, TS4, and TS5 are
sitting on the same subnet (e.g., same MAC-VRF/VLAN);where, TS1 and sitting on the same subnet (e.g., same MAC-VRF/VLAN);where, TS1 and
TS5 are associated with MAC-VRF1 on NVE1, TS4 is associated with MAC- TS5 are associated with MAC-VRF1 on NVE1, TS4 is associated with MAC-
VRF1 on NVE2. TS2 is associated with MAC-VRF2 on NVE1, and TS3 is VRF1 on NVE2. TS2 is associated with MAC-VRF2 on NVE1, and TS3 is
associated with MAC-VRF3 on NVE2. MAC-VRF1 and MAC-VRF2 on NVE1 are associated with MAC-VRF3 on NVE2. MAC-VRF1 and MAC-VRF2 on NVE1 are
in turn associated with IP-VRF1 on NVE1 and MAC-VRF1 and MAC-VRF3 on in turn associated with IP-VRF1 on NVE1 and MAC-VRF1 and MAC-VRF3 on
NVE2 are associated with IP-VRF1 on NVE2. When TS1, TS5, and TS4 NVE2 are associated with IP-VRF1 on NVE2. When TS1, TS5, and TS4
exchange traffic with each other, only L2 forwarding (bridging) part exchange traffic with each other, only L2 forwarding (bridging) part
of the IRB solution is exercised because all these TS's sit on the of the IRB solution is exercised because all these TSes sit on the
same subnet. However, when TS1 wants to exchange traffic with TS2 or same subnet. However, when TS1 wants to exchange traffic with TS2 or
TS3 which belong to different subnets, then both bridging and routing TS3 which belong to different subnets, then both bridging and routing
parts of the IRB solution are exercised. The following subsections parts of the IRB solution are exercised. The following subsections
describe the control and data planes operations for this IRB scenario describe the control and data planes operations for this IRB scenario
in details. in details.
NVE1 +---------+ NVE1 +---------+
+-------------+ | | +-------------+ | |
TS1-----| MACx| | | NVE2 TS1-----| MACx| | | NVE2
(IP1/M1) |(MAC- | | | +-------------+ (IP1/M1) |(MAC- | | | +-------------+
skipping to change at page 24, line 47 skipping to change at page 25, line 8
(IP2/M2) | VRF2) | | | | VRF1) | (IP4/M4) (IP2/M2) | VRF2) | | | | VRF1) | (IP4/M4)
+-------------+ | | +-------------+ +-------------+ | | +-------------+
| | | |
+---------+ +---------+
Figure 6: IRB forwarding on NVEs for Tenant Systems Figure 6: IRB forwarding on NVEs for Tenant Systems
6.1.1 Control Plane Operation 6.1.1 Control Plane Operation
Each NVE advertises a MAC/IP Advertisement route (i.e., Route Type 2) Each NVE advertises a MAC/IP Advertisement route (i.e., Route Type 2)
for each of its TS's with the following field set: for each of its TSes with the following field set:
- RD and ESI per [RFC7432] - RD and ESI per [RFC7432]
- Ethernet Tag = 0; assuming VLAN-based service - Ethernet Tag = 0; assuming VLAN-based service
- MAC Address Length = 48 - MAC Address Length = 48
- MAC Address = Mi ; where i = 1,2,3,4, or 5 in the above example - MAC Address = Mi ; where i = 1,2,3,4, or 5 in the above example
- IP Address Length = 32 or 128 - IP Address Length = 32 or 128
- IP Address = IPi ; where i = 1,2,3,4, or 5 in the above example - IP Address = IPi ; where i = 1,2,3,4, or 5 in the above example
- Label-1 = MPLS Label or VNI corresponding to MAC-VRF - Label-1 = MPLS Label or VNI corresponding to MAC-VRF
- Label-2 = MPLS Label or VNI corresponding to IP-VRF - Label-2 = MPLS Label or VNI corresponding to IP-VRF
skipping to change at page 27, line 13 skipping to change at page 27, line 20
destination TS (TS3) over the corresponding interface. destination TS (TS3) over the corresponding interface.
In this symmetric IRB scenario, inter-subnet traffic between NVEs In this symmetric IRB scenario, inter-subnet traffic between NVEs
will always use the IP-VRF VNI/MPLS label. For instance, traffic from will always use the IP-VRF VNI/MPLS label. For instance, traffic from
TS2 to TS4 will be encapsulated by NVE1 using NVE2's IP-VRF VNI/MPLS TS2 to TS4 will be encapsulated by NVE1 using NVE2's IP-VRF VNI/MPLS
label, as long as TS4's host IP is present in NVE1's IP-VRF. label, as long as TS4's host IP is present in NVE1's IP-VRF.
6.2 IRB forwarding on NVEs for Subnets behind Tenant Systems 6.2 IRB forwarding on NVEs for Subnets behind Tenant Systems
This section covers the symmetric IRB procedures for the scenario This section covers the symmetric IRB procedures for the scenario
where some Tenant Systems (TS's) support one or more subnets and where some Tenant Systems (TSes) support one or more subnets and
these TS's are associated with one or more NVEs. Therefore, besides these TSes are associated with one or more NVEs. Therefore, besides
the advertisement of MAC/IP addresses for each TS which can be multi- the advertisement of MAC/IP addresses for each TS which can be multi-
homed with All-Active redundancy mode, the associated NVE needs to homed with All-Active redundancy mode, the associated NVE needs to
also advertise the subnets statically configured on each TS. also advertise the subnets statically configured on each TS.
The main difference between this solution and the previous one is the The main difference between this solution and the previous one is the
additional advertisement corresponding to each subnet. These subnet additional advertisement corresponding to each subnet. These subnet
advertisements are accomplished using EVPN IP Prefix route defined in advertisements are accomplished using EVPN IP Prefix route defined in
[EVPN-PREFIX]. These subnet prefixes are advertised with the IP [EVPN-PREFIX]. These subnet prefixes are advertised with the IP
address of their associated TS (which is in overlay address space) as address of their associated TS (which is in overlay address space) as
their next hop. The receiving NVEs perform recursive route resolution their next hop. The receiving NVEs perform recursive route resolution
skipping to change at page 27, line 39 skipping to change at page 27, line 46
The advantage of this recursive route resolution is that when a TS The advantage of this recursive route resolution is that when a TS
moves from one NVE to another, there is no need to re-advertise any moves from one NVE to another, there is no need to re-advertise any
of the subnet prefixes for that TS. All it is needed is to advertise of the subnet prefixes for that TS. All it is needed is to advertise
the IP/MAC addresses associated with the TS itself and exercise MAC the IP/MAC addresses associated with the TS itself and exercise MAC
mobility procedures for that TS. The recursive route resolution mobility procedures for that TS. The recursive route resolution
automatically takes care of the updates for the subnet prefixes of automatically takes care of the updates for the subnet prefixes of
that TS. that TS.
Figure below illustrates this scenario where a given tenant (e.g., an Figure below illustrates this scenario where a given tenant (e.g., an
IP-VPN service) has three subnets represented by MAC-VRF1, MAC-VRF2, IP-VPN service) has three subnets represented by MAC-VRF1, MAC-VRF2,
and MAC-VRF3 across two NVEs. There are four TS's associated with and MAC-VRF3 across two NVEs. There are four TSes associated with
these three MAC-VRFs - i.e., TS1, TS5 are connected to MAC-VRF1 on these three MAC-VRFs - i.e., TS1, TS5 are connected to MAC-VRF1 on
NVE1, TS2 is connected to MAC-VRF2 on NVE1, TS3 is connected to MAC- NVE1, TS2 is connected to MAC-VRF2 on NVE1, TS3 is connected to MAC-
VRF3 on NVE2, and TS4 is connected to MAC-VRF1 on NVE2. TS1 has two VRF3 on NVE2, and TS4 is connected to MAC-VRF1 on NVE2. TS1 has two
subnet prefixes (SN1 and SN2) and TS3 has a single subnet prefix, subnet prefixes (SN1 and SN2) and TS3 has a single subnet prefix,
SN3. The MAC-VRFs on each NVE are associated with their corresponding SN3. The MAC-VRFs on each NVE are associated with their corresponding
IP-VRF using their IRB interfaces. When TS4 and TS1 exchange intra- IP-VRF using their IRB interfaces. When TS4 and TS1 exchange intra-
subnet traffic, only L2 forwarding (bridging) part of the IRB subnet traffic, only L2 forwarding (bridging) part of the IRB
solution is used (i.e., the traffic only goes through their MAC- solution is used (i.e., the traffic only goes through their MAC-
VRFs); however, when TS3 wants to forward traffic to SN1 or SN2 VRFs); however, when TS3 wants to forward traffic to SN1 or SN2
sitting behind TS1 (inter-subnet traffic), then both bridging and sitting behind TS1 (inter-subnet traffic), then both bridging and
skipping to change at page 28, line 26 skipping to change at page 28, line 33
+-------------+ | | +-------------+ | |
SN3--+--TS3-----|(MAC-\ | | | SN3--+--TS3-----|(MAC-\ | | |
IP3/M3 | VRF3)\ | | | IP3/M3 | VRF3)\ | | |
| (IP-VRF)|---| | | (IP-VRF)|---| |
| / | | | | / | | |
TS4-----|(MAC- / | | | TS4-----|(MAC- / | | |
IP4/M4 | VRF1) | | | IP4/M4 | VRF1) | | |
+-------------+ +----------+ +-------------+ +----------+
NVE2 NVE2
Figure 7: IRB forwarding on NVEs for subnets behind TS's Figure 7: IRB forwarding on NVEs for subnets behind TSes
6.2.1 Control Plane Operation 6.2.1 Control Plane Operation
Each NVE advertises a Route Type-5 (RT-5, IP Prefix Route defined in Each NVE advertises a Route Type-5 (RT-5, IP Prefix Route defined in
[EVPN-PREFIX]) for each of its subnet prefixes with the IP address of [EVPN-PREFIX]) for each of its subnet prefixes with the IP address of
its TS as the next hop (gateway address field) as follow: its TS as the next hop (gateway address field) as follow:
- RD associated with the IP-VRF - RD associated with the IP-VRF
- ESI = 0 - ESI = 0
- Ethernet Tag = 0; - Ethernet Tag = 0;
skipping to change at page 28, line 41 skipping to change at page 29, line 4
[EVPN-PREFIX]) for each of its subnet prefixes with the IP address of [EVPN-PREFIX]) for each of its subnet prefixes with the IP address of
its TS as the next hop (gateway address field) as follow: its TS as the next hop (gateway address field) as follow:
- RD associated with the IP-VRF - RD associated with the IP-VRF
- ESI = 0 - ESI = 0
- Ethernet Tag = 0; - Ethernet Tag = 0;
- IP Prefix Length = 32 or 128 - IP Prefix Length = 32 or 128
- IP Prefix = SNi - IP Prefix = SNi
- Gateway Address = IPi; IP address of TS - Gateway Address = IPi; IP address of TS
- Label = 0 - Label = 0
This RT-5 is advertised with one or more Route Targets that have been This RT-5 is advertised with one or more Route Targets that have been
configured as "export route targets" of the IP-VRF from which the configured as "export route targets" of the IP-VRF from which the
route is originated. route is originated.
Each NVE also advertises an RT-2 (MAC/IP Advertisement Route) along Each NVE also advertises an RT-2 (MAC/IP Advertisement Route) along
with their associated Route Targets and Extended Communities for each with their associated Route Targets and Extended Communities for each
of its TS's exactly as described in section 6.1.1. of its TSes exactly as described in section 6.1.1.
Upon receiving the RT-5 advertisement, the receiving NVE performs the Upon receiving the RT-5 advertisement, the receiving NVE performs the
following: following:
- It uses the Route Target to identify the corresponding IP-VRF - It uses the Route Target to identify the corresponding IP-VRF
- It imports the IP prefix into its corresponding IP-VRF that is - It imports the IP prefix into its corresponding IP-VRF that is
configured with an import RT that is one of the RTs being carried by configured with an import RT that is one of the RTs being carried by
the RT-5 route along with the IP address of the associated TS as its the RT-5 route along with the IP address of the associated TS as its
next hop. next hop.
skipping to change at page 30, line 24 skipping to change at page 30, line 30
SA set to that of the access-facing IRB interface of the egress NVE SA set to that of the access-facing IRB interface of the egress NVE
(NVE2) and the MAC DA is set to that of destination TS (TS3) MAC (NVE2) and the MAC DA is set to that of destination TS (TS3) MAC
address. The packet is sent to the corresponding MAC-VRF3 and after a address. The packet is sent to the corresponding MAC-VRF3 and after a
lookup of MAC DA, is forwarded to the destination TS (TS3) over the lookup of MAC DA, is forwarded to the destination TS (TS3) over the
corresponding interface. corresponding interface.
7 Acknowledgements 7 Acknowledgements
The authors would like to thank Sami Boutros, Jeffrey Zhang, The authors would like to thank Sami Boutros, Jeffrey Zhang,
Krzysztof Szarkowicz, and Neeraj Malhotra for their valuable Krzysztof Szarkowicz, and Neeraj Malhotra for their valuable
comments. comments. The authors would also like to thank Linda Dunbar for her
engaging discussions.
8 Security Considerations 8 Security Considerations
This document describes a set of procedures for Inter-Subnet This document describes a set of procedures for Inter-Subnet
Forwarding of tenant traffic across PEs (or NVEs). These procedures Forwarding of tenant traffic across PEs (or NVEs). These procedures
include both layer-2 forwarding and layer-3 routing on a packet by include both layer-2 forwarding and layer-3 routing on a packet by
packet basis. The security consideration for layer-2 forwarding in packet basis. The security consideration for layer-2 forwarding in
this document follow that of [RFC7432] for MPLS encapsulation and it this document follow that of [RFC7432] for MPLS encapsulation and it
follows that of [RFC8365] for VxLAN or GENEVE encapsulations. follows that of [RFC8365] for VxLAN or GENEVE encapsulations.
 End of changes. 32 change blocks. 
57 lines changed or deleted 68 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/