draft-ietf-nvo3-geneve-06.txt   draft-ietf-nvo3-geneve-07.txt 
Network Working Group J. Gross, Ed. Network Working Group J. Gross, Ed.
Internet-Draft Internet-Draft
Intended status: Standards Track I. Ganga, Ed. Intended status: Standards Track I. Ganga, Ed.
Expires: September 6, 2018 Intel Expires: January 3, 2019 Intel
T. Sridhar, Ed. T. Sridhar, Ed.
VMware VMware
March 05, 2018 July 02, 2018
Geneve: Generic Network Virtualization Encapsulation Geneve: Generic Network Virtualization Encapsulation
draft-ietf-nvo3-geneve-06 draft-ietf-nvo3-geneve-07
Abstract Abstract
Network virtualization involves the cooperation of devices with a Network virtualization involves the cooperation of devices with a
wide variety of capabilities such as software and hardware tunnel wide variety of capabilities such as software and hardware tunnel
endpoints, transit fabrics, and centralized control clusters. As a endpoints, transit fabrics, and centralized control clusters. As a
result of their role in tying together different elements in the result of their role in tying together different elements in the
system, the requirements on tunnels are influenced by all of these system, the requirements on tunnels are influenced by all of these
components. Flexibility is therefore the most important aspect of a components. Flexibility is therefore the most important aspect of a
tunnel protocol if it is to keep pace with the evolution of the tunnel protocol if it is to keep pace with the evolution of the
skipping to change at page 1, line 41 skipping to change at page 1, line 41
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 6, 2018. This Internet-Draft will expire on January 3, 2019.
Copyright Notice Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 41 skipping to change at page 2, line 41
4.1.1. IP Fragmentation . . . . . . . . . . . . . . . . . . 17 4.1.1. IP Fragmentation . . . . . . . . . . . . . . . . . . 17
4.1.2. DSCP and ECN . . . . . . . . . . . . . . . . . . . . 17 4.1.2. DSCP and ECN . . . . . . . . . . . . . . . . . . . . 17
4.1.3. Broadcast and Multicast . . . . . . . . . . . . . . . 18 4.1.3. Broadcast and Multicast . . . . . . . . . . . . . . . 18
4.1.4. Unidirectional Tunnels . . . . . . . . . . . . . . . 18 4.1.4. Unidirectional Tunnels . . . . . . . . . . . . . . . 18
4.2. Constraints on Protocol Features . . . . . . . . . . . . 19 4.2. Constraints on Protocol Features . . . . . . . . . . . . 19
4.2.1. Constraints on Options . . . . . . . . . . . . . . . 19 4.2.1. Constraints on Options . . . . . . . . . . . . . . . 19
4.3. NIC Offloads . . . . . . . . . . . . . . . . . . . . . . 19 4.3. NIC Offloads . . . . . . . . . . . . . . . . . . . . . . 19
4.4. Inner VLAN Handling . . . . . . . . . . . . . . . . . . . 20 4.4. Inner VLAN Handling . . . . . . . . . . . . . . . . . . . 20
5. Interoperability Issues . . . . . . . . . . . . . . . . . . . 20 5. Interoperability Issues . . . . . . . . . . . . . . . . . . . 20
6. Security Considerations . . . . . . . . . . . . . . . . . . . 21 6. Security Considerations . . . . . . . . . . . . . . . . . . . 21
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 6.1. Data Confidentiality . . . . . . . . . . . . . . . . . . 21
8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 22 6.1.1. Inter-data center traffic . . . . . . . . . . . . . . 22
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 24 6.2. Data Integrity . . . . . . . . . . . . . . . . . . . . . 22
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 24 6.3. Authentication of NVE peers . . . . . . . . . . . . . . . 23
10.1. Normative References . . . . . . . . . . . . . . . . . . 24 6.4. Multicast/Broadcast . . . . . . . . . . . . . . . . . . . 23
10.2. Informative References . . . . . . . . . . . . . . . . . 24 6.5. Control plane communications . . . . . . . . . . . . . . 24
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24
8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 25
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 26
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 26
10.1. Normative References . . . . . . . . . . . . . . . . . . 26
10.2. Informative References . . . . . . . . . . . . . . . . . 27
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 29
1. Introduction 1. Introduction
Networking has long featured a variety of tunneling, tagging, and Networking has long featured a variety of tunneling, tagging, and
other encapsulation mechanisms. However, the advent of network other encapsulation mechanisms. However, the advent of network
virtualization has caused a surge of renewed interest and a virtualization has caused a surge of renewed interest and a
corresponding increase in the introduction of new protocols. The corresponding increase in the introduction of new protocols. The
large number of protocols in this space, ranging all the way from large number of protocols in this space, ranging all the way from
VLANs [IEEE.802.1Q_2014] and MPLS [RFC3031] through the more recent VLANs [IEEE.802.1Q_2014] and MPLS [RFC3031] through the more recent
VXLAN [RFC7348], NVGRE [RFC7637], often leads to questions about the VXLAN [RFC7348], NVGRE [RFC7637], often leads to questions about the
skipping to change at page 8, line 11 skipping to change at page 8, line 11
Endpoints must be able to parse the variable header, including any Endpoints must be able to parse the variable header, including any
options, and take action. Since these devices are actively options, and take action. Since these devices are actively
participating in the protocol, they are the most affected by Geneve. participating in the protocol, they are the most affected by Geneve.
However, as endpoints are the ultimate consumers of the data, However, as endpoints are the ultimate consumers of the data,
transmitters can tailor their output to the capabilities of the transmitters can tailor their output to the capabilities of the
recipient. As new functionality becomes sufficiently well defined to recipient. As new functionality becomes sufficiently well defined to
add to endpoints, supporting options can be designed using ordering add to endpoints, supporting options can be designed using ordering
restrictions and other techniques to ease parsing. restrictions and other techniques to ease parsing.
Transit devices MAY be able to interpret the options and participate Transit devices MAY be able to interpret the options, however, as
in Geneve packet processing. However, as non-terminating devices, non-terminating devices, transit devices do not originate or
they do not originate or terminate the Geneve packet. The terminate the Geneve packet, hence MUST NOT insert or delete options,
participation of transit devices in Geneve packet processing is which is the responsibility of Geneve endpoints. The participation
OPTIONAL. of transit devices in interpreting options is OPTIONAL.
Further, either tunnel endpoints or transit devices MAY use offload Further, either tunnel endpoints or transit devices MAY use offload
capabilities of NICs such as checksum offload to improve the capabilities of NICs such as checksum offload to improve the
performance of Geneve packet processing. The presence of a Geneve performance of Geneve packet processing. The presence of a Geneve
variable length header SHOULD NOT prevent the tunnel endpoints and variable length header SHOULD NOT prevent the tunnel endpoints and
transit devices from using such offload capabilities. transit devices from using such offload capabilities.
2.3. Use of Standard IP Fabrics 2.3. Use of Standard IP Fabrics
IP has clearly cemented its place as the dominant transport mechanism IP has clearly cemented its place as the dominant transport mechanism
skipping to change at page 13, line 32 skipping to change at page 13, line 32
payload. payload.
Opt Len (6 bits): The length of the options fields, expressed in Opt Len (6 bits): The length of the options fields, expressed in
four byte multiples, not including the eight byte fixed tunnel four byte multiples, not including the eight byte fixed tunnel
header. This results in a minimum total Geneve header size of 8 header. This results in a minimum total Geneve header size of 8
bytes and a maximum of 260 bytes. The start of the payload bytes and a maximum of 260 bytes. The start of the payload
headers can be found using this offset from the end of the base headers can be found using this offset from the end of the base
Geneve header. Geneve header.
O (1 bit): OAM packet. This packet contains a control message O (1 bit): OAM packet. This packet contains a control message
instead of a data payload. Endpoints MUST NOT forward the payload instead of a data payload. Control messages are sent between
and transit devices MUST NOT attempt to interpret or process it. Geneve endpoints. Endpoints MUST NOT forward the payload and
transit devices MUST NOT attempt to interpret or process it.
Since these are infrequent control messages, it is RECOMMENDED Since these are infrequent control messages, it is RECOMMENDED
that endpoints direct these packets to a high priority control that endpoints direct these packets to a high priority control
queue (for example, to direct the packet to a general purpose CPU queue (for example, to direct the packet to a general purpose CPU
from a forwarding ASIC or to separate out control traffic on a from a forwarding ASIC or to separate out control traffic on a
NIC). Transit devices MUST NOT alter forwarding behavior on the NIC). Transit devices MUST NOT alter forwarding behavior on the
basis of this bit, such as ECMP link selection. basis of this bit, such as ECMP link selection.
C (1 bit): Critical options present. One or more options has the C (1 bit): Critical options present. One or more options has the
critical bit set (see Section 3.5). If this bit is set then critical bit set (see Section 3.5). If this bit is set then
tunnel endpoints MUST parse the options list to interpret any tunnel endpoints MUST parse the options list to interpret any
critical options. On endpoints where option parsing is not critical options. On endpoints where option parsing is not
supported the packet MUST be dropped on the basis of the 'C' bit supported the packet MUST be dropped on the basis of the 'C' bit
in the base header. If the bit is not set tunnel endpoints MAY in the base header. If the bit is not set tunnel endpoints MAY
strip all options using 'Opt Len' and forward the decapsulated strip all options using 'Opt Len' and forward the decapsulated
packet. Transit devices MUST NOT drop or modify packets on the packet. Transit devices MUST NOT drop packets on the basis of
basis of this bit. this bit.
The critical bit allows hardware implementations the flexibility The critical bit allows hardware implementations the flexibility
to handle options processing in the hardware fastpath or in the to handle options processing in the hardware fastpath or in the
exception (slow) path without the need to process all the options. exception (slow) path without the need to process all the options.
For example, a critical option such as secure hash to provide For example, a critical option such as secure hash to provide
Geneve header integrity check must be processed by tunnel Geneve header integrity check must be processed by tunnel
endpoints and typically processed in the hardware fastpath. endpoints and typically processed in the hardware fastpath.
Rsvd. (6 bits): Reserved field which MUST be zero on transmission Rsvd. (6 bits): Reserved field which MUST be zero on transmission
and ignored on receipt. and ignored on receipt.
skipping to change at page 16, line 9 skipping to change at page 16, line 9
option may be between 4 and 128 bytes. A value of 0 in the Length option may be between 4 and 128 bytes. A value of 0 in the Length
field implies an option with only the option header without the field implies an option with only the option header without the
variable option data. Packets in which the total length of all variable option data. Packets in which the total length of all
options is not equal to the 'Opt Len' in the base header are options is not equal to the 'Opt Len' in the base header are
invalid and MUST be silently dropped if received by an endpoint. invalid and MUST be silently dropped if received by an endpoint.
Variable Option Data: Option data interpreted according to 'Type'. Variable Option Data: Option data interpreted according to 'Type'.
3.5.1. Options Processing 3.5.1. Options Processing
Geneve options are primarily intended to be originated and processed Geneve options are intended to be originated and processed by tunnel
by tunnel endpoints. However, options MAY be processed by transit endpoints. However, options MAY be interpreted by transit devices
devices along the tunnel path as well. Transit devices not along the tunnel path. Transit devices not processing Geneve headers
processing Geneve headers SHOULD process Geneve packets as any other SHOULD process Geneve packets as any other UDP packet and maintain
UDP packet and maintain consistent forwarding behavior. consistent forwarding behavior.
In tunnel endpoints, the generation and interpretation of options is In tunnel endpoints, the generation and interpretation of options is
determined by the control plane, which is out of the scope of this determined by the control plane, which is out of the scope of this
document. However, to ensure interoperability between heterogeneous document. However, to ensure interoperability between heterogeneous
devices some requirements are imposed on options and the devices that devices some requirements are imposed on options and the devices that
process them: process them:
o Receiving endpoints MUST drop packets containing unknown options o Receiving endpoints MUST drop packets containing unknown options
with the 'C' bit set in the option type. Conversely, transit with the 'C' bit set in the option type. Conversely, transit
devices MUST NOT drop packets as a result of encountering unknown devices MUST NOT drop packets as a result of encountering unknown
options, including those with the 'C' bit set. options, including those with the 'C' bit set.
o Some options may be defined in such a way that the position in the o Some options may be defined in such a way that the position in the
option list is significant. Therefore, options MUST NOT be option list is significant. Options or their ordering, MUST NOT
reordered by transit devices. be changed by transit devices.
o An option MUST NOT affect the parsing or interpretation of any o An option MUST NOT affect the parsing or interpretation of any
other option. other option.
When designing a Geneve option, it is important to consider how the When designing a Geneve option, it is important to consider how the
option will evolve in the future. Once an option is defined it is option will evolve in the future. Once an option is defined it is
reasonable to expect that implementations may come to depend on a reasonable to expect that implementations may come to depend on a
specific behavior. As a result, the scope of any future changes must specific behavior. As a result, the scope of any future changes must
be carefully described upfront. be carefully described upfront.
skipping to change at page 21, line 24 skipping to change at page 21, line 24
devices do not introduce additional interoperability concerns. devices do not introduce additional interoperability concerns.
To assist with this transition, it is strongly suggested that To assist with this transition, it is strongly suggested that
implementations support simultaneous operation of both Geneve and implementations support simultaneous operation of both Geneve and
existing tunnel protocols as it is expected to be common for a single existing tunnel protocols as it is expected to be common for a single
node to communicate with a mixture of other nodes. Eventually, older node to communicate with a mixture of other nodes. Eventually, older
protocols may be phased out as they are no longer in use. protocols may be phased out as they are no longer in use.
6. Security Considerations 6. Security Considerations
As UDP/IP packets, Geneve does not have any inherent security As encapsulated within an UDP/IP packet, Geneve does not have any
mechanisms. As a result, an attacker with access to the underlay inherent security mechanisms. As a result, an attacker with access
network transporting the IP packets has the ability to snoop or to the underlay network transporting the IP packets has the ability
inject packets. Legitimate but malicious tunnel endpoints may also to snoop or inject packets. Legitimate but malicious tunnel
spoof identifiers in the tunnel header to gain access to networks endpoints may also spoof identifiers in the tunnel header to gain
owned by other tenants. access to networks owned by other tenants.
Within a particular security domain, such as a data center operated Within a particular security domain, such as a data center operated
by a single provider, the most common and highest performing security by a single service provider, the most common and highest performing
mechanism is isolation of trusted components. Tunnel traffic can be security mechanism is isolation of trusted components. Tunnel
carried over a separate VLAN and filtered at any untrusted traffic can be carried over a separate VLAN and filtered at any
boundaries. In addition, tunnel endpoints should only be operated in untrusted boundaries. In addition, tunnel endpoints should only be
environments controlled by the service provider, such as the operated in environments controlled by the service provider, such as
hypervisor itself rather than within a customer VM. the hypervisor itself rather than within a customer VM.
When crossing an untrusted link, such as the public Internet, IPsec When crossing an untrusted link, such as the public Internet, IPsec
[RFC4301] may be used to provide authentication and/or encryption of [RFC4301] may be used to provide authentication and/or encryption of
the IP packets formed as part of Geneve encapsulation. If the remote the IP packets formed as part of Geneve encapsulation.
tunnel endpoint is not completely trusted, for example it resides on
a customer premises, then it may also be necessary to sanitize any
tunnel metadata to prevent tenant-hopping attacks.
Geneve does not otherwise affect the security of the encapsulated Geneve does not otherwise affect the security of the encapsulated
packets. packets. As per the guidelines of BCP72 [RFC3552], the following
sections describe potential security risks that may be applicable to
Geneve deployments and approaches to mitigate such risks.
6.1. Data Confidentiality
Geneve is a network virtualization overlay encapsulation protocol
designed to establish tunnels between network virtualization end
points (NVE) over an existing IP network. It can be used to deploy
multi-tenant overlay networks over an existing IP underlay network in
a public or private data center. The overlay service is typically
provided by a service provider, for example a cloud services provider
or a private data center operator. Due to the nature of multi-
tenancy in such environments, a tenant system may expect data
confidentiality to ensure its packet data is not tampered with
(active attack) in transit or a target of unauthorized monitoring
(passive attack). A tenant may expect the overlay service provider
to provide data confidentiality as part of the service or a tenant
may bring its own data confidentiality mechanisms like IPsec or TLS
to protect the data end to end between its tenant systems.
An NVE, used in multi-tenant environments, MUST have the capability
to encrypt the tenant data end to end between the NVEs. The NVEs may
use existing well established encryption mechanisms such as IPsec,
DTLs, etc., The NVEs SHOULD have a configurable option to disable the
encryption if, for example, the packet data is already encrypted by
the tenant system.
6.1.1. Inter-data center traffic
A tenant system in a customer premises (private data center) may want
to connect to tenant systems on their tenant overlay network in a
public cloud data center or a tenant may want to have its tenant
systems located in multiple geographically separated data centers for
high availability. Geneve data traffic between tenant systems across
such separated networks should be protected from threats when
traversing public networks. Any Geneve overlay data leaving the data
center network, beyond the operators security domain, for example
over a public Internet SHOULD be secured by encryption mechanisms
such as IPsec or other VPN mechanisms to protect the communications
between the NVEs when they are geographically separated over
untrusted network links. Implementation of specific data protection
mechanisms employed between data centers is beyond the scope of this
document.
6.2. Data Integrity
Geneve encapsulation is used between NVEs to establish overlay
tunnels over an existing IP underlay network. In a multi-tenant data
center, a rogue or compromised tenant system may try to launch a
passive attack such as monitoring the traffic of other tenants, or an
active attack such as spoofing or trying to inject unauthorized
Geneve encapsulated traffic into the network. To prevent such
attacks, an NVE MUST not propagate Geneve packets beyond the NVE to
tenant systems and SHOULD employ packet filtering mechanisms so as
not to forward unauthorized traffic between TSs in different tenant
networks.
A compromised network node or a transit device within a data center
may launch an active attack trying to tamper with the Geneve packet
data between NVEs. Malicious tampering of Geneve header fields may
cause the packet from one tenant to be forwarded to a different
tenant network. If an operator determines the possibility of such
threat in their environment, the operator may choose to employ data
integrity mechanisms between NVEs. In order to prevent such risks, a
Geneve NVE MUST have the capability to protect the integrity of
Geneve packets including packet headers, options and payload on
communications between NVE pairs. A cryptographic data protection
mechanism such as IPsec may be used to provide data integrity
protection. The NVE SHOULD have a configuration option to enable or
disable the data integrity protection, based on the presence of
threats in their environment. A data center operator may choose to
deploy any other data integrity mechanisms as applicable and
supported in their underlay networks.
Geneve supports Geneve Options, so an operator may choose to use a
Geneve option TLV to provide a cryptographic data protection
mechanism, to verify the data integrity of the Geneve header, Geneve
options or the entire Geneve packet including the payload.
Implementation of such a mechanism is beyond the scope of this
document.
6.3. Authentication of NVE peers
A rogue network device or a compromised NVE in a data center
environment might be able to spoof Geneve packets as if it came from
a legitimate NVE. In order to mitigate such a risk, a Geneve NVE
MUST support an Authentication mechanism, such as IPsec AH, to ensure
that the Geneve packet originated from the intended NVE peer, in
environments where spoofing or rogue devices is a potential threat.
Other simpler source checks such as ingress filtering for VLAN/MAC/IP
address, reverse path forwarding checks, etc., may be used in certain
trusted environments to ensure Geneve packets originated from the
intended NVE peer.
6.4. Multicast/Broadcast
In typical data center networks where IP multicasting is not
supported in the underlay network, multicasting can be supported
using multiple unicast tunnels. The same security requirements as
described in the above sections can be used to protect Geneve
communications between NVE peers. If IP multicasting is supported in
the underlay network and the operator chooses to use it for multicast
traffic among Geneve endpoints, then Geneve NVEs used in such
environments SHOULD support data protection mechanisms such as IPsec
with Multicast extensions [RFC5374] to protect multicast traffic
among Geneve NVE groups.
6.5. Control plane communications
A Network Virtualization Authority (NVA) as outlined in [RFC8014] may
be used as a control plane for configuring and managing the Geneve
NVEs. The data center operator is expected to use security
mechanisms to protect the communications between the NVA to NVEs and
use authentication mechanisms to detect any rogue or compromised NVEs
within their administrative domain. Data protection mechanisms for
control plane communication or authentication mechanisms between the
NVA and the NVEs is beyond the scope of this document.
7. IANA Considerations 7. IANA Considerations
IANA has allocated UDP port 6081 as the well-known destination port IANA has allocated UDP port 6081 as the well-known destination port
for Geneve. Upon publication, the registry should be updated to cite for Geneve. Upon publication, the registry should be updated to cite
this document. The original request was: this document. The original request was:
Service Name: geneve Service Name: geneve
Transport Protocol(s): UDP Transport Protocol(s): UDP
Assignee: Jesse Gross <jgross@vmware.com> Assignee: Jesse Gross <jgross@vmware.com>
skipping to change at page 22, line 38 skipping to change at page 25, line 14
+----------------+--------------------------------------+ +----------------+--------------------------------------+
| Option Class | Description | | Option Class | Description |
+----------------+--------------------------------------+ +----------------+--------------------------------------+
| 0x0000..0x00FF | Unassigned - IETF Review | | 0x0000..0x00FF | Unassigned - IETF Review |
| 0x0100 | Linux | | 0x0100 | Linux |
| 0x0101 | Open vSwitch | | 0x0101 | Open vSwitch |
| 0x0102 | Open Virtual Networking (OVN) | | 0x0102 | Open Virtual Networking (OVN) |
| 0x0103 | In-band Network Telemetry (INT) | | 0x0103 | In-band Network Telemetry (INT) |
| 0x0104 | VMware | | 0x0104 | VMware |
| 0x0105..0xFFEF | Unassigned - First Come First Served | | 0x0105 | Amazon |
| 0x0106 | Cisco |
| 0x0107..0xFFEF | Unassigned - First Come First Served |
| 0xFFF0..FFFF | Experimental | | 0xFFF0..FFFF | Experimental |
+----------------+--------------------------------------+ +----------------+--------------------------------------+
8. Contributors 8. Contributors
The following individuals were authors of an earlier version of this The following individuals were authors of an earlier version of this
document and made significant contributions: document and made significant contributions:
Pankaj Garg Pankaj Garg
Microsoft Corporation Microsoft Corporation
skipping to change at page 23, line 44 skipping to change at page 26, line 22
USA USA
Email: ddutt@cumulusnetworks.com Email: ddutt@cumulusnetworks.com
Jon Hudson Jon Hudson
Independent Independent
Email: jon.hudson@gmail.com Email: jon.hudson@gmail.com
Ariel Hendel Ariel Hendel
Broadcom Limited Facebook, Inc.
3151 Zanker Road 1 Hacker Way
San Jose, CA 95134 Menlo Park, CA 94025
USA USA
Email: ariel.hendel@broadcom.com Email: ahendel@fb.com
9. Acknowledgements 9. Acknowledgements
The authors wish to thank Martin Casado, Bruce Davie and Dave Thaler The authors wish to thank Martin Casado, Bruce Davie and Dave Thaler
for their input, feedback, and helpful suggestions. for their input, feedback, and helpful suggestions.
10. References 10. References
10.1. Normative References 10.1. Normative References
skipping to change at page 25, line 22 skipping to change at page 27, line 47
[RFC2983] Black, D., "Differentiated Services and Tunnels", [RFC2983] Black, D., "Differentiated Services and Tunnels",
RFC 2983, DOI 10.17487/RFC2983, October 2000, RFC 2983, DOI 10.17487/RFC2983, October 2000,
<https://www.rfc-editor.org/info/rfc2983>. <https://www.rfc-editor.org/info/rfc2983>.
[RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol [RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
Label Switching Architecture", RFC 3031, Label Switching Architecture", RFC 3031,
DOI 10.17487/RFC3031, January 2001, DOI 10.17487/RFC3031, January 2001,
<https://www.rfc-editor.org/info/rfc3031>. <https://www.rfc-editor.org/info/rfc3031>.
[RFC3552] Rescorla, E. and B. Korver, "Guidelines for Writing RFC
Text on Security Considerations", BCP 72, RFC 3552,
DOI 10.17487/RFC3552, July 2003,
<https://www.rfc-editor.org/info/rfc3552>.
[RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation [RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation
Edge-to-Edge (PWE3) Architecture", RFC 3985, Edge-to-Edge (PWE3) Architecture", RFC 3985,
DOI 10.17487/RFC3985, March 2005, DOI 10.17487/RFC3985, March 2005,
<https://www.rfc-editor.org/info/rfc3985>. <https://www.rfc-editor.org/info/rfc3985>.
[RFC4301] Kent, S. and K. Seo, "Security Architecture for the [RFC4301] Kent, S. and K. Seo, "Security Architecture for the
Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, Internet Protocol", RFC 4301, DOI 10.17487/RFC4301,
December 2005, <https://www.rfc-editor.org/info/rfc4301>. December 2005, <https://www.rfc-editor.org/info/rfc4301>.
[RFC5374] Weis, B., Gross, G., and D. Ignjatic, "Multicast
Extensions to the Security Architecture for the Internet
Protocol", RFC 5374, DOI 10.17487/RFC5374, November 2008,
<https://www.rfc-editor.org/info/rfc5374>.
[RFC6040] Briscoe, B., "Tunnelling of Explicit Congestion [RFC6040] Briscoe, B., "Tunnelling of Explicit Congestion
Notification", RFC 6040, DOI 10.17487/RFC6040, November Notification", RFC 6040, DOI 10.17487/RFC6040, November
2010, <https://www.rfc-editor.org/info/rfc6040>. 2010, <https://www.rfc-editor.org/info/rfc6040>.
[RFC6935] Eubanks, M., Chimento, P., and M. Westerlund, "IPv6 and [RFC6935] Eubanks, M., Chimento, P., and M. Westerlund, "IPv6 and
UDP Checksums for Tunneled Packets", RFC 6935, UDP Checksums for Tunneled Packets", RFC 6935,
DOI 10.17487/RFC6935, April 2013, DOI 10.17487/RFC6935, April 2013,
<https://www.rfc-editor.org/info/rfc6935>. <https://www.rfc-editor.org/info/rfc6935>.
[RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, [RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger,
skipping to change at page 26, line 10 skipping to change at page 28, line 45
[RFC7365] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y. [RFC7365] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y.
Rekhter, "Framework for Data Center (DC) Network Rekhter, "Framework for Data Center (DC) Network
Virtualization", RFC 7365, DOI 10.17487/RFC7365, October Virtualization", RFC 7365, DOI 10.17487/RFC7365, October
2014, <https://www.rfc-editor.org/info/rfc7365>. 2014, <https://www.rfc-editor.org/info/rfc7365>.
[RFC7637] Garg, P., Ed. and Y. Wang, Ed., "NVGRE: Network [RFC7637] Garg, P., Ed. and Y. Wang, Ed., "NVGRE: Network
Virtualization Using Generic Routing Encapsulation", Virtualization Using Generic Routing Encapsulation",
RFC 7637, DOI 10.17487/RFC7637, September 2015, RFC 7637, DOI 10.17487/RFC7637, September 2015,
<https://www.rfc-editor.org/info/rfc7637>. <https://www.rfc-editor.org/info/rfc7637>.
[RFC8014] Black, D., Hudson, J., Kreeger, L., Lasserre, M., and T.
Narten, "An Architecture for Data-Center Network
Virtualization over Layer 3 (NVO3)", RFC 8014,
DOI 10.17487/RFC8014, December 2016,
<https://www.rfc-editor.org/info/rfc8014>.
[VL2] Greenberg, A., et al., "VL2: A Scalable and Flexible Data [VL2] Greenberg, A., et al., "VL2: A Scalable and Flexible Data
Center Network", ACM SIGCOMM Computer Communication Center Network", ACM SIGCOMM Computer Communication
Review, DOI 10.1145/1594977.1592576, 2009, Review, DOI 10.1145/1594977.1592576, 2009,
<http://www.sigcomm.org/sites/default/files/ccr/ <http://www.sigcomm.org/sites/default/files/ccr/
papers/2009/October/1594977-1592576.pdf>. papers/2009/October/1594977-1592576.pdf>.
Authors' Addresses Authors' Addresses
Jesse Gross (editor) Jesse Gross (editor)
 End of changes. 20 change blocks. 
49 lines changed or deleted 188 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/