[Docs] [txt|pdf] [Tracker] [Email] [Nits]

Versions: 00 01 02

BESS Workgroup                                           A. Sajassi, Ed.
INTERNET-DRAFT                                               A. Banerjee
Intended Status: Standards Track                               S. Thoria
                                                               D. Carrel
                                                                 B. Weis
                                                                   Cisco

Expires: May 20, 2019                                   October 20, 2018


                              Secure EVPN
                   draft-sajassi-bess-secure-evpn-00


Abstract

   The applications of EVPN-based solutions ([RFC7432] and [RFC8365])
   have become pervasive in Data Center, Service Provider, and
   Enterprise segments. It is being used for fabric overlays and inter-
   site connectivity in the Data Center market segment, for Layer-2,
   Layer-3, and IRB VPN services in the Service Provider market segment,
   and for fabric overlay and WAN connectivity in Enterprise networks.
   For Data Center and Enterprise applications, there is a need to
   provide inter-site and WAN connectivity over public Internet in a
   secured manner with same level of privacy, integrity, and
   authentication for tenant's traffic as IPsec tunneling using IKEv2.
   This document presents a solution where BGP point-to-multipoint
   signaling is leveraged for key and policy exchange among PE devices
   to create private pair-wise IPsec Security Associations without IKEv2
   point-to-point signaling or any other direct peer-to-peer session
   establishment messages.



Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."



Sajassi et al.            Expires May 20, 2019                  [Page 1]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html


Copyright and License Notice

   Copyright (c) 2014 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.



Table of Contents

   1  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . .  6
   2 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . .  7
     2.1 Tenant's Layer-2 and Layer-3 data & control traffic  . . . .  7
     2.2 Tenant's Unicast & Multicast Data Protection . . . . . . . .  7
     2.3 P2MP Signaling for SA setup and Maintenance  . . . . . . . .  7
     2.3 Granularity of Security Association Tunnels  . . . . . . . .  7
     2.4 Support for Policy and DH-Group List . . . . . . . . . . . .  8
   3  Solution Description  . . . . . . . . . . . . . . . . . . . . .  8
     3.1  Distribution of Public Keys and Policies  . . . . . . . . .  9
       3.1.1  Minimum Set . . . . . . . . . . . . . . . . . . . . . .  9
       3.1.2  Single Policy . . . . . . . . . . . . . . . . . . . . . 10
       3.1.3  Policy-list & DH-group-list . . . . . . . . . . . . . . 10
     3.2  Initial IPsec SAs Generation  . . . . . . . . . . . . . . . 11
     3.3  Re-Keying . . . . . . . . . . . . . . . . . . . . . . . . . 11
     3.4  IPsec Databases . . . . . . . . . . . . . . . . . . . . . . 11
   4 Encapsulation  . . . . . . . . . . . . . . . . . . . . . . . . . 12
     4.1 Standard ESP Encapsulation . . . . . . . . . . . . . . . . . 12
     4.2 ESP Encapsulation within UDP packet  . . . . . . . . . . . . 13
   5 BGP Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . 14
     5.1 ESP Notify Sub-TLV . . . . . . . . . . . . . . . . . . . . . 14
     5.2 ESP Key Exchange Sub-TLV . . . . . . . . . . . . . . . . . . 15
     5.3 ESP Nonce Sub-TLV  . . . . . . . . . . . . . . . . . . . . . 15



Sajassi et al.            Expires May 20, 2019                  [Page 2]


INTERNET DRAFT                Secure EVPN               October 20, 2018


     5.3 ESP Proposals Sub-TLV  . . . . . . . . . . . . . . . . . . . 16
   6  Applicability to other VPN types  . . . . . . . . . . . . . . . 17
   7  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . 18
   8  Security Considerations . . . . . . . . . . . . . . . . . . . . 18
   9  IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 18
   10  References . . . . . . . . . . . . . . . . . . . . . . . . . . 18
     10.1  Normative References . . . . . . . . . . . . . . . . . . . 18
     10.2  Informative References . . . . . . . . . . . . . . . . . . 19
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 20


Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

   AC: Attachment Circuit.

   ARP: Address Resolution Protocol.

   BD: Broadcast Domain. As per [RFC7432], an EVI consists of a single
   or multiple BDs. In case of VLAN-bundle and VLAN-based service models
   (see [RFC7432]), a BD is equivalent to an EVI. In case of VLAN-aware
   bundle service model, an EVI contains multiple BDs. Also, in this
   document, BD and subnet are equivalent terms.

   BD Route Target: refers to the Broadcast Domain assigned Route Target
   [RFC4364]. In case of VLAN-aware bundle service model, all the BD
   instances in the MAC-VRF share the same Route Target.

   BT: Bridge Table. The instantiation of a BD in a MAC-VRF, as per
   [RFC7432].

   DGW: Data Center Gateway.

   Ethernet A-D route: Ethernet Auto-Discovery (A-D) route, as per
   [RFC7432].

   Ethernet NVO tunnel: refers to Network Virtualization Overlay tunnels
   with Ethernet payload. Examples of this type of tunnels are VXLAN or
   GENEVE.

   EVI: EVPN Instance spanning the NVE/PE devices that are participating
   on that EVPN, as per [RFC7432].




Sajassi et al.            Expires May 20, 2019                  [Page 3]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   EVPN: Ethernet Virtual Private Networks, as per [RFC7432].

   GRE: Generic Routing Encapsulation.

   GW IP: Gateway IP Address.

   IPL: IP Prefix Length.

   IP NVO tunnel: it refers to Network Virtualization Overlay tunnels
   with IP payload (no MAC header in the payload).

   IP-VRF: A VPN Routing and Forwarding table for IP routes on an
   NVE/PE. The IP routes could be populated by EVPN and IP-VPN address
   families. An IP-VRF is also an instantiation of a layer 3 VPN in an
   NVE/PE.

   IRB: Integrated Routing and Bridging interface. It connects an IP-VRF
   to a BD (or subnet).

   MAC-VRF: A Virtual Routing and Forwarding table for Media Access
   Control (MAC) addresses on an NVE/PE, as per [RFC7432]. A MAC-VRF is
   also an instantiation of an EVI in an NVE/PE.

   ML: MAC address length.

   ND: Neighbor Discovery Protocol.

   NVE: Network Virtualization Edge.

   GENEVE: Generic Network Virtualization Encapsulation, [GENEVE].

   NVO: Network Virtualization Overlays.

   RT-2: EVPN route type 2, i.e., MAC/IP advertisement route, as defined
   in [RFC7432].

   RT-5: EVPN route type 5, i.e., IP Prefix route. As defined in Section
   3 of [EVPN-PREFIX].

   SBD: Supplementary Broadcast Domain. A BD that does not have any ACs,
   only IRB interfaces, and it is used to provide connectivity among all
   the IP-VRFs of the tenant. The SBD is only required in IP-VRF- to-IP-
   VRF use-cases (see Section 4.4.).

   SN: Subnet.

   TS: Tenant System.




Sajassi et al.            Expires May 20, 2019                  [Page 4]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   VA: Virtual Appliance.

   VNI: Virtual Network Identifier. As in [RFC8365], the term is used as
   a representation of a 24-bit NVO instance identifier, with the
   understanding that VNI will refer to a VXLAN Network Identifier in
   VXLAN, or Virtual Network Identifier in GENEVE, etc. unless it is
   stated otherwise.

   VTEP: VXLAN Termination End Point, as in [RFC7348].

   VXLAN: Virtual Extensible LAN, as in [RFC7348].

   This document also assumes familiarity with the terminology of
   [RFC7432], [RFC8365] and [RFC7365].





































Sajassi et al.            Expires May 20, 2019                  [Page 5]


INTERNET DRAFT                Secure EVPN               October 20, 2018


1  Introduction

   The applications of EVPN-based solutions have become pervasive in
   Data Center, Service Provider, and Enterprise segments. It is being
   used for fabric overlays and inter-site connectivity in the Data
   Center market segment, for Layer-2, Layer-3, and IRB VPN services in
   the Service Provider market segment, and for fabric overlay and WAN
   connectivity in the Enterprise networks. For Data Center and
   Enterprise applications, there is a need to provide inter-site and
   WAN connectivity over public Internet in a secured manner with the
   same level of privacy, integrity, and authentication for tenant's
   traffic as used in IPsec tunneling using IKEv2. This document
   presents a solution where BGP point-to-multipoint signaling is
   leveraged for key and policy exchange among PE devices to create
   private pair-wise IPsec Security Associations without IKEv2 point-to-
   point signaling or any other direct peer-to-peer session
   establishment messages.

   EVPN uses BGP as control-plane protocol for distribution of
   information needed for discovery of PEs participating in a VPN,
   discovery of PEs participating in a redundancy group, customer MAC
   addresses and IP prefixes/addresses, aliasing information, tunnel
   encapsulation types, multicast tunnel types, multicast group
   memberships, and other info. The advantages of using BGP control
   plane in EVPN are well understood including the following:

   1) A full mesh of BGP sessions among PE devices can be avoided by
   using Route Reflector (RR) where a PE only needs to setup a single
   BGP session between itself and the RR as opposed to setting up N BGP
   sessions to N other remote PEs; therefore, reducing number of BGP
   sessions from O(N^2) to O(N) in the network. Furthermore, RR
   hierarchy can be leveraged to scale the number of BGP routes on the
   RR.

   2) MP-BGP route filtering and constrained route distribution can be
   leveraged to ensure that the control-plane traffic for a given VPN is
   only distributed to the PEs participating in that VPN.

   For setting up point-to-point security association (i.e., IPsec
   tunnel) between a pair of EVPN PEs, it is important to leverage BGP
   point-to-multipoint singling architecture using the RR along with its
   route filtering and constrain mechanisms to achieve the performance
   and the scale needed for large number of security associations (IPsec
   tunnels) along with their frequent re-keying requirements. Using BGP
   signaling along with the RR (instead of peer-to-peer protocol such as
   IKEv2) reduces number of message exchanges needed for SAs
   establishment and maintenance from O(N^2) to O(N) in the network. be
   increased from O(N) to O(N^2).



Sajassi et al.            Expires May 20, 2019                  [Page 6]


INTERNET DRAFT                Secure EVPN               October 20, 2018


2 Requirements

   The requirements for secured EVPN are captured in the following
   subsections.

2.1 Tenant's Layer-2 and Layer-3 data & control traffic

   Tenant's layer-2 and layer-3 data and control traffic SHALL be
   protected by IPsec cryptographic methods. This implies not only
   tenant's data traffic SHALL be protected by IPsec but also tenant's
   control and routing information that are advertised in BGP SHALL also
   be protected by IPsec. This in turn implies that BGP session SHALL be
   protected by IPsec.

2.2 Tenant's Unicast & Multicast Data Protection

   Tenant's layer-2 and layer-3 unicast traffic SHALL be protected by
   IPsec. In addition to that, tenant's layer-2 broadcast, unknown
   unicast, and multicast traffic as well as tenant's layer-3 multicast
   traffic SHALL be protected by IPsec when ingress replication or
   assisted replication are used. The use of BGP P2MP signaling for
   setting up P2MP SAs in P2MP multicast tunnels is for future study.

2.3 P2MP Signaling for SA setup and Maintenance

   BGP P2MP signaling SHALL be used for IPsec SAs setup and maintenance.
   The BGP signaling SHALL follow P2MP signaling framework per
   [CONTROLLER-IKE] for IPsec SAs setup and maintenance in order to
   reduce the number of message exchanges from O(N^2) to O(N) among the
   participant PE devices.

2.3 Granularity of Security Association Tunnels

   The solution SHALL support the setup and maintenance of IPsec SAs at
   the following level of granularities:

   1) Per pair of PEs: A single IPsec tunnel between a pair of PEs to be
   used for all tenants' traffic supported by the pair of PEs.

   2) Per tenant: A single IPsec tunnel per tenant per pair of PEs. For
   example, if there are 1000 tenants supported on a pair of PEs, then
   1000 IPsec tunnels are required between that pair of PEs.

   3) Per subnet: A single IPsec tunnel per subnet (e.g., per VLAN/EVI)
   of a tenant on a pair of PEs.

   4) Per pair of IP addresses: A single IPsec tunnel per pair of IP
   addresses of a tenant on a pair of PEs.



Sajassi et al.            Expires May 20, 2019                  [Page 7]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   5) Per pair of MAC addresses: A single IPsec tunnel per pair of MAC
   addresses of a tenant on a pair of PEs.

2.4 Support for Policy and DH-Group List

   The solution SHALL support a single policy and DH group for all SAs
   as well as supporting multiple policies and DH groups among the SAs.



3  Solution Description

   This solution uses BGP P2MP signaling where an originating PE only
   send a message to Route Reflector (RR) and then the RR reflects that
   message to the interested recipient PEs. The framework for such
   signaling is described in [CONTROLLER-IKE] and it is referred to as
   device-to-controller trust model. This trust model is significantly
   different than the traditional peer-to-peer trust model where a P2P
   signaling protocol such as IKEv2 [RFC7296] is used in which the PE
   devices directly authenticate each other and agree upon security
   policy and keying material to protect communications between
   themselves. The device-to-controller trust model leverages P2MP
   signaling via the controller (e.g., the RR) to achieve much better
   scale and performance for establishment and maintenance of large
   number of pairwise Security Associations (SAs) among the PEs.

   This device-to-controller trust model first secures the control
   channel between each device and the controller using peer-to-peer
   protocol such as IKEv2 [RFC7296] to establish P2P SAs between each PE
   and the RR. It then uses this secured control channel for P2MP
   signaling in establishment of P2P SAs between a pair of PE devices.

   Each PE advertised to other PEs via the RR the information needed in
   establishment of pair-wise SAs between itself an every other remote
   PEs. These pieces of information are sent as Sub-TLVs of IPSec tunnel
   type in BGP Tunnel Encapsulation attribute. These Sub-TLVs are
   detailed in section 5 and they are based on IKEv2 specification
   [RFC7296]. The IPsec tunnel TLVs along with its Sub-TLVs are sent
   along with the BGP route (NLRI) for a given level of granularity.

   If only a single SA is required per pair of PE devices to multiplex
   user traffic for all tenants, then IPsec tunnel TLV is advertised
   along with IPv4 or IPv6 NLRI representing loopback address of the
   originating PE. It should be noted that this is not a VPN route but
   rather an IPv4 or IPv6 route.

   If a SA is required per tenant between a pair of PE devices, then
   IPsec tunnel TLV can be advertised along with EVPN IMET route



Sajassi et al.            Expires May 20, 2019                  [Page 8]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   representing the tenant or can be advertised along with a new EVPN
   route representing the tenant.

   If a SA is required per tenant's subnet (e.g., per VLAN) between a
   pair of PE devices, then IPsec tunnel TLV is advertised along with
   EVPN IMET route.

   If a SA is required between a pair of tenant's devices represented by
   a pair of IP addresses, then IPsec tunnel TLV is advertised along
   with EVPN IP Prefix Advertisement Route or EVPN MAC/IP Advertisement
   route.

   If a SA is required between a pair of tenant's devices represented by
   a pair of MAC addresses, then IPsec tunnel TLV is advertised along
   with EVPN MAC/IP Advertisement route.

   If a SA is required between a pair of tenant's devices represented by
   a VLAN or a port, then IPsec tunnel TLV is advertised along with EVPN
   Ethernet AD route.


3.1  Distribution of Public Keys and Policies

   One of the requirements for this solution is to support a single DH
   group and a single policy for all SAs as well as to support multiple
   DH groups and policies among the SAs. The following subsections
   describe what pieces of information (what Sub-TLVs) are needed to be
   exchanged to support a single DH group and a single policy versus
   multiple DH groups and multiple policies.

3.1.1  Minimum Set

   For SA establishment, at the minimum, a PE needs to advertise to
   other PEs, its ID, a notification to indicate if this is its initial
   contact, key exchange including DH public number and DH group, and
   Nonce. When a single policy is used among all SAs, it is assumed that
   this single policy is configured by the management system in all the
   PE devices and thus there is no need to signal it. The information
   that need to be signaled (using RFC7296 notations) are:

   ID, [N(INITIAL_CONTACT),] KE, Ni; where

        ID payload is defined in section 3.5 of [RFC7296]
        N (Notify) Payload in section 3.10 of [RFC7296]
        KE (Key Exchange) payload in section 3.4 of [RFC7296]
        Ni (Nonce) payload in section 3.9 of [RFC7296]

   KE payload contains the DH public number and also identifies which DH



Sajassi et al.            Expires May 20, 2019                  [Page 9]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   group to use. ID sub-TLV would not be needed in BGP because tunnel
   attribute already carries originator ID. Section 5 details these sub-
   TLVs as part of IPsec tunnel TLV in BGP Tunnel Encapsulation
   Attribute.


3.1.2  Single Policy

   If a single policy needs to be signaled among per tenant or per
   subnet among a set of PEs, then in addition to the information
   described in section 3.1.1, Security Association sub-TLV needs to be
   signaled as well. The payload for this sub-TLV is defined in section
   3.3 of [RFC7296] and detailed in section 5.3.

   ID, [N(INITIAL_CONTACT),SA, KE, Ni

        SA (Security Association) payload in section 3.3 of [RFC7296]

   A single SA payload identifies a single IPsec policy. One important
   restriction on the SA Payload is that an standard IKE SA payload can
   contain multiple transform; however, [CONTROLLER-IKE] restricts the
   SA payload to only a single transform for each transform type as
   described in section A.3.1 of [CONTROLLER-IKE].


3.1.3  Policy-list & DH-group-list

   There can be scenarios for which there is a need to have multiple
   policy options.  This can happen when there is a need for policy
   change and smooth migration among all PE devices to the new policy is
   required. It can also happen if different PE devices have different
   capabilities within the network. In these scenairos, PE devices need
   to be able to choose the correct policy to use for each other. This
   multi-policy scheme is described in section 6 of [CONTROLLER-IKE]. In
   order to support this multi-policy feature, a PE device MUST
   distribute a policy list. This list consists of multiple distinct
   policies in order of preference, where the first policy is the most
   preferred one.  The receiving PE selects the policy by taking the
   received list (starting with the first policy) and comparing that
   against its own list and choosing the first one found in common. If
   there is no match, this indicates a configuration error and the PEs
   MUST NOT establish new SAs until a message is received that does
   produce a match.

   Furthermore, when a device supports more than one DH group, then a
   unique DH public number MUST be specified for each in order of
   preference.  The selection of which DH group to use follows the same
   logic as Policy selection, using the receiver's list order until a



Sajassi et al.            Expires May 20, 2019                 [Page 10]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   match is found in the initiator's list.

   In order to support multi-policy a policy list is signaled in
   addition to the information described in section 3.1.1. Furthermore,
   in order to support multi-DH-groups, a DH group list along with its
   nonce list are signaled instead of a single DH group and a single
   nonce as described in section 3.1.1.

   ID, [N(INITIAL_CONTACT), [SA], [KE], [Ni]

        [SA] list of IPsec policies (i.e., list of SA payloads)
        [KE] list of KE payloads


3.2  Initial IPsec SAs Generation

   The procedure for generation of initial IPsec SAs is described in
   section 3 of [CONTROLLER-IKE]. This section gives a summary of it in
   context of BGP signaling. When a PE device first comes up and wants
   to setup an IPsec SA between itself and each of the interested remote
   PEs, it generates a DH pair along for each of its intended IPsec SA
   using an algorithm defined in the IKEv2 Diffie-Hellman Group
   Transform IDs [IKEv2-IANA]. The originating PE distributes DH public
   value along with a nonce (using IPsec Tunnel TLV in Tunnel
   Encapsulation Attribute) to other remote PEs via the RR. Each
   receiving PE uses this DH public number and the corresponding nonce
   in creation of IPsec SA pair to the originating PE - i.e., an
   outbound SA and an inbound SA. The detail procedures are described in
   section 5.2 of [CONTROLLER-IKE].


3.3  Re-Keying

   A PE can initiate re-keying at any time due to local time or volume
   based policy or due to the result of cipher counter nearing its final
   value. The rekey process is performed individually for each remote
   PE. If rekeying is performed with multiple PEs simultaneously, then
   the decision process and rules described in this rekey are performed
   independently for each PE. Section 4 of [CONTROLLER-IKE] describes
   this rekeying process in details and gives examples for a single
   IPsec device (e.g., a single PE) rekey versus multiple PE devices
   rekey simultaneously.


3.4  IPsec Databases

   The Peer Authorization Database (PAD), the Security Policy Database
   (SPD), and the Security Association Database (SAD) all need to be



Sajassi et al.            Expires May 20, 2019                 [Page 11]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   setup as defined in the IPsec Security Architecture [RFC4301].
   Section 5 of [CONTROLLER-IKE] gives a summary description of how
   these databases are setup for the controller-based model where key is
   exchanged via P2MP signaling via the controller (e.g., the RR) and
   the policy can be either signaled via the RR (in case of multiple
   policies) or configured by the management station (in case of single
   policy).


4 Encapsulation

   Vast majority of Encapsulation for Network Virtualization Overlay
   (NVO) networks in deployment are based on UDP/IP with UDP destination
   port ID indicating the type of NVO encapsulation (e.g., VxLAN, GPE,
   GENEVE, GUE) and UDP source port ID representing flow entropy for
   load-balancing of the traffic within the fabric based on n-tuple that
   includes UDP header. When encrypting NVO encapsulated packets using
   IP Encapsulating Security Payload (ESP), the following two options
   can be used:  a) adding a UDP header before ESP header (e.g., UDP
   header in clear) and b) no UDP header before ESP header (e.g.,
   standard ESP encapsulation). The following subsection describe these
   encapsulation in further details.


4.1 Standard ESP Encapsulation

   When standard IP Encapsulating Security Payload (ESP) is used
   (without outer UDP header) for encryption of NVO packets, it is used
   in transport mode as depicted below. When such encapsulation is used,
   the Tunnel Type of Tunnel Encapsulation TLV is set to ESP-Transport
   and the Tunnel Type of Encapsulation Extended Community is set to NVO
   encapsulation type (e.g., VxLAN, GENEVE, GPE, etc.). This implies
   that the customer packets are first encapsulated using NVO
   encapsulation type and then it is further encapsulated & encrypted
   using ESP-Transport mode.
















Sajassi et al.            Expires May 20, 2019                 [Page 12]


INTERNET DRAFT                Secure EVPN               October 20, 2018


       +-+-+-+-+-+-+-+-+-+-+-+-+          +-+-+-+-+-+-+-+-+-+-+-+-+
       |       MAC Header      |          |      MAC Header       |
       +-+-+-+-+-+-+-+-+-+-+-+-+          +-+-+-+-+-+-+-+-+-+-+-+-+
       | Eth Type = IPv4/IPv6  |          | Eth Type = IPv4/IPv6  |
       +-+-+-+-+-+-+-+-+-+-+-+-+          +-+-+-+-+-+-+-+-+-+-+-+-+
       |    IP Header          |          |    IP Header          |
       |    Protocol = UDP     |          |    Protocol = ESP     |
       +-----------------------+          +-----------------------+
       |      UDP Header       |          |    ESP Header         |
       | Dest Port = VxLAN     |          +-----------------------+
       +-----------------------+          |     UDP Header        |
       |     VxLAN Header      |          | Dest Port = VxLAN     |
       +-----------------------+          +-----------------------+
       |    Inner MAC Header   |          |   VxLAN Header        |
       +-----------------------+          +-----------------------+
       |    Inner Eth Payload  |          |   Inner MAC Header    |
       +-----------------------+          +-----------------------+
       |        CRC            |          |   Inner Eth Payload   |
       +-----------------------+          +-----------------------+
                                          |  ESP Trailer (NP=UDP) |
                                          +-----------------------+
                                          |        CRC            |
                                          +-----------------------+



               Figure 3: VxLAN Encapsulation within ESP


4.2 ESP Encapsulation within UDP packet

   In scenarios where NAT traversal is required ([RFC3948]) or where
   load balancing using UDP header is required, then ESP encapsulation
   within UDP packet as depicted in the following figure is used. The
   ESP for NVO applications is in transport mode. The outer UDP header
   (before the ESP header) has its source port set to flow entropy and
   its destination port set to 4500 (indicating ESP header follows). A
   non-zero SPI value in ESP header implies that this is a data packet
   (i.e., it is not an IKE packet). The Next Protocol field in the ESP
   trailer indicates what follows the ESP header, is a UDP header. This
   inner UDP header has a destination port ID that identifies NVO
   encapsulation type (e.g., VxLAN). Optimization of this packet format
   where only a single UDP header is used (only the outer UDP header) is
   for future study.

   When such encapsulation is used, the Tunnel Type of Tunnel
   Encapsulation TLV is set to ESP-in-UDP-Transport and the Tunnel Type
   of Encapsulation Extended Community is set to NVO encapsulation type



Sajassi et al.            Expires May 20, 2019                 [Page 13]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   (e.g., VxLAN, GENEVE, GPE, etc.). This implies that the customer
   packets are first encapsulated using NVO encapsulation type and then
   it is further encapsulated & encrypted using ESP-in-UDP with
   Transport mode.


   [RFC3948]


       +-+-+-+-+-+-+-+-+-+-+-+-+          +-+-+-+-+-+-+-+-+-+-+-+-+
       |       MAC Header      |          |      MAC Header       |
       +-+-+-+-+-+-+-+-+-+-+-+-+          +-+-+-+-+-+-+-+-+-+-+-+-+
       | Eth Type = IPv4/IPv6  |          | Eth Type = IPv4/IPv6  |
       +-+-+-+-+-+-+-+-+-+-+-+-+          +-+-+-+-+-+-+-+-+-+-+-+-+
       |    IP Header          |          |    IP Header          |
       |    Protocol = UDP     |          |    Protocol = UDP     |
       +-----------------------+          +-----------------------+
       |      UDP Header       |          |    UDP Header         |
       | Dest Port = VxLAN     |          | Dest Port = 4500(ESP) |
       +-----------------------+          +-----------------------+
       |     VxLAN Header      |          |    ESP Header         |
       +-----------------------+          +-----------------------+
       |    Inner MAC Header   |          |     UDP Header        |
       +-----------------------+          | Dest Port = VxLAN     |
       |    Inner Eth Payload  |          +-----------------------+
       +-----------------------+          |   VxLAN Header        |
       |        CRC            |          +-----------------------+
       +-----------------------+          |   Inner MAC Header    |
                                          +-----------------------+
                                          |   Inner Eth Payload   |
                                          +-----------------------+
                                          |  ESP Trailer (NP=UDP) |
                                          +-----------------------+
                                          |        CRC            |
                                          +-----------------------+
          Figure 4: VxLAN Encapsulation within ESP Within UDP


5 BGP Encoding

   This document defines two new Tunnel Types along with its associated
   sub-TLVs for The Tunnel Encapsulation Attribute [TUNNEL-ENCAP]. These
   tunnel types correspond to ESP-Transport and ESP-in-UDP-Transport as
   described in section 4. The following sub-TLVs apply to both tunnel
   types unless stated otherwise.

5.1 ESP Notify Sub-TLV




Sajassi et al.            Expires May 20, 2019                 [Page 14]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   This sub-TLV corresponds to Notify payload of IPsec Encapsulation
   Security Payload protocol as defined in IKEv2 [RFC7296]. This payload
   is defined and described in section 3.10 of [RFC7296].

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Next Payload  |C|   Reserved  |      Payload Length           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Protocol ID   |   SPI Size    |      Notify Message Type      |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       ~                Security Parameter Index (SPI)                 ~
       |                                                               |
       +---------------------------------------------------------------+
       |                                                               |
       ~                    Notification Data                          ~
       |                                                               |
       +---------------------------------------------------------------+

                    Figure 5: Notify Payload Format


5.2 ESP Key Exchange Sub-TLV

   This sub-TLV corresponds to Key Exchange payload of IPsec
   Encapsulation Security Payload protocol as defined in IKEv2
   [RFC7296]. This payload is defined and described in section 3.4 of
   [RFC7296].

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Next Payload  |C|   Reserved  |      Payload Length           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Diffie-Hellman Group Number   |          Reserved             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       ~                      Key Exchange Data                        ~
       |                                                               |
       +---------------------------------------------------------------+

                 Figure 6: Key Exchange Payload Format


5.3 ESP Nonce Sub-TLV

   This sub-TLV corresponds to Nonce payload of IPsec Encapsulation



Sajassi et al.            Expires May 20, 2019                 [Page 15]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   Security Payload protocol as defined in IKEv2 [RFC7296]. This payload
   is defined and described in section 3.9 of [RFC7296].

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Next Payload  |C|   Reserved  |      Payload Length           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       ~                           Nonce Data                          ~
       |                                                               |
       +---------------------------------------------------------------+

                    Figure 7: Nonce Payload Format


5.3 ESP Proposals Sub-TLV

   This sub-TLV corresponds to Proposal payload of IPsec Encapsulation
   Security Payload protocol as defined in IKEv2 [RFC7296]. This payload
   is defined and described in section 3.3 of [RFC7296].

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Next Payload  |C|   Reserved  |      Payload Length           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       ~                           <Proposals>                         ~
       |                                                               |
       +---------------------------------------------------------------+

                Figure 8: Security Association Payload

   Proposals (Variable) - one or more proposal substructures
















Sajassi et al.            Expires May 20, 2019                 [Page 16]


INTERNET DRAFT                Secure EVPN               October 20, 2018


        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Last Substruc |     Reserved  |      Proposal Length          |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Proposal Num  |   Protocol ID | SPI Size      | Num Transforms|
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       ~                        SPI (Variable)                         ~
       |                                                               |
       +---------------------------------------------------------------+
       |                                                               |
       ~                         <Transforms>                          ~
       |                                                               |
       +---------------------------------------------------------------+

                    Figure 9: Proposal Substructure


6  Applicability to other VPN types

   Although P2MP BGP signaling for establishment and maintenance of SAs
   among PE devices is described in this document in context of EVPN,
   there is no reason why it cannot be extended to other VPN
   technologies such as IP-VPN [RFC4364], VPLS [RFC4761] & [RFC4762],
   and MVPN [RFC6513] & [RFC6514] with ingress replication. The reason
   EVPN has been chosen is because of its pervasiveness in DC, SP, and
   Enterprise applications and because of its ability to support SA
   establishment at different granularity levels such as: per PE, Per
   tenant, per subnet, per Ethernet Segment, per IP address, and per
   MAC. For other VPN technology types, a much smaller granularity
   levels can be supported. For example for VPLS, only the granularity
   of per PE and per subnet can be supported. For per-PE granularity
   level, the mechanism is the same among all the VPN technologies as
   IPsec tunnel type (and its associated TLV and sub-TLVs) are sent
   along with the PE's loopback IPv4 (or IPv6) address. For VPLS, if
   per-subnet (per bridge domain) granularity level needs to be
   supported, then the IPsec tunnel type and TLV are sent along with
   VPLS AD route.

   The following table lists what level of granularity can be supported
   by a given VPN technology and with what BGP route.









Sajassi et al.            Expires May 20, 2019                 [Page 17]


INTERNET DRAFT                Secure EVPN               October 20, 2018


    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | Functionality |     EVPN    |   IP-VPN    |    MVPN   |   VPLS  |
    +---------------+-------------+-------------+-----------+---------+
    | per PE        |IPv4/v6 route|IPv4/v6 route|IPv4/v6 rte|IPv4/v6  |
    +---------------+-------------+-------------+-----------+---------+
    | per tenant    |IMET (or new)|lpbk (or new)|  I-PMSI   | N/A     |
    +---------------+-------------+-------------+-----------+---------+
    | per subnet    |   IMET      |     N/A     |    N/A    | VPLS AD |
    +---------------+-------------+-------------+-----------+---------+
    | per IP        |EVPN RT2/RT5 |  VPN IP rt  | *,G or S,G|  N/A    |
    +---------------+-------------+-------------+-----------+---------+
    | per MAC       |  EVPN RT2   |     N/A     |    N/A    |  N/A    |
    +---------------+-------------+-------------+-----------+---------+






7  Acknowledgements


8  Security Considerations


9  IANA Considerations

   A new transitive extended community Type of 0x06 and Sub-Type of TBD
   for EVPN Attachment Circuit Extended Community needs to be allocated
   by IANA.

10  References

10.1  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.


   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC2119
              Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May
              2017.

   [RFC7432] Sajassi et al., "BGP MPLS Based Ethernet VPN", RFC 7432,
              February, 2015.


   [RFC8365] Sajassi et al., "A Network Virtualization Overlay Solution



Sajassi et al.            Expires May 20, 2019                 [Page 18]


INTERNET DRAFT                Secure EVPN               October 20, 2018


              Using Ethernet VPN (EVPN)", RFC 8365, March, 2018.

   [TUNNEL-ENCAP] Rosen et al., "The BGP Tunnel Encapsulation
              Attribute", draft-ietf-idr-tunnel-encaps-03, November
              2016.

   [CONTROLLER-IKE] Carrel et al., "IPsec Key Exchange using a
              Controller", draft-carrel-ipsecme-controller-ike-00, July,
              2018.

   [RFC3948] Huttunen et al., "UDP Encapsulation of IPsec ESP Packets",
              RFC 3948, January 2005.

   [IKEV2-IANA] IANA, "Internet Key Exchange Version 2 (IKEv2)
              Parameters", February 2016,
              www.iana.org/assignments/ikev2-parameters/ikev2-
              parameters.xhtml.

   [RFC4301]  Kent, S. and K. Seo, "Security Architecture for the
              Internet Protocol", RFC 4301, DOI 10.17487/RFC4301,
              December 2005.

10.2  Informative References


   [RFC4364] Rosen, E., et. al., "BGP/MPLS IP Virtual Private Networks
   (VPNs)", RFC 4364, February 2006.

   [RFC4761] Kompella, K., et. al., "Virtual Private LAN Service (VPLS)
   Using BGP for Auto-Discovery and Signaling", RFC 4761, January 2007.

   [RFC4762] Kompella, K., et. al., "Virtual Private LAN Service (VPLS)
   Using Label Distribution Protocol (LDP) Signaling", RFC 4762, January
   2007.

   [RFC6513] Rosen, E., et. al., "Multicast in MPLS/BGP IP VPNs", RFC
   6513, February 2012.

   [RFC6514] Rosen, E., et. al., "BGP Encodings and Procedures for
   Multicast in MPLS/BGP IP VPNs", RFC 6514, February 2012.

   [RFC7606]  Chen, E., Scudder, J., Mohapatra, P., and K. Patel,
   "Revised Error Handling for BGP UPDATE Messages", RFC 7606, August
   2015, <http://www.rfc-editor.org/info/rfc7606>.

   [802.1Q] "IEEE Standard for Local and metropolitan area networks -
   Media Access Control (MAC) Bridges and Virtual Bridged Local Area
   Networks", IEEE Std 802.1Q(tm), 2014 Edition, November 2014.



Sajassi et al.            Expires May 20, 2019                 [Page 19]


INTERNET DRAFT                Secure EVPN               October 20, 2018


   [RFC7348]  Mahalingam, M., et al., "Virtual eXtensible Local Area
   Network (VXLAN): A Framework for Overlaying Virtualized Layer 2
   Networks over Layer 3 Networks", RFC 7348, DOI 10.17487/RFC7348,
   August 2014.

   [GENEVE]  Gross, J., et al., "Geneve: Generic Network Virtualization
   Encapsulation", Work in Progress, draft-ietf-nvo3-geneve-06, March
   2018.


Authors' Addresses


   Ali Sajassi
   Cisco
   Email: sajassi@cisco.com


   Ayan Banerjee
   Cisco
   Email: ayabaner@cisco.com


   Samir Thoria
   Cisco
   Email: sthoria@cisco.com


   David Carrel
   Cisco
   Email: carrel@cisco.com


   Brian Weis
   Cisco
   Email: bew@cisco.com















Sajassi et al.            Expires May 20, 2019                 [Page 20]


Html markup produced by rfcmarkup 1.129c, available from https://tools.ietf.org/tools/rfcmarkup/