IPPM T. Mizrahi
Internet-Draft Huawei Network.IO Innovation Lab
Intended status: Standards Track F. Brockners
Expires: January 5, 2020 S. Bhandari
R. Sivakolundu
C. Pignataro
A. Kfir
B. Gafni
Mellanox Technologies, Inc.
M. Spiegel
Barefoot Networks
J. Lemon
July 04, 2019

In-situ OAM Flags


In-situ Operations, Administration, and Maintenance (IOAM) records operational and telemetry information in the packet while the packet traverses a path between two points in the network. This document presents new flags in the IOAM Trace Option headers. Specifically, the document defines the Loopback, Active, and Immediate Export flags.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on January 5, 2020.

Copyright Notice

Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1. Introduction

IOAM [I-D.ietf-ippm-ioam-data] is used for monitoring traffic in the network by incorporating IOAM data fields into in-flight data packets.

IOAM data may be represented in one of four possible IOAM options: Pre-allocated Trace Option, Incremental Trace Option, Proof of Transit (POT) Option, and Edge-to-Edge Option. This document defines three new flags in the Pre-allocated and Incremental Trace options: the Loopback, Active, and Immediate Export flags.

2. Conventions

2.1. Requirement Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

2.2. Terminology

Abbreviations used in this document:

In-situ Operations, Administration, and Maintenance
Operations, Administration, and Maintenance

3. New IOAM Trace Option Flags

This document defines three new flags in the Pre-allocated and Incremental Trace options:

Bit 1
"Loopback" (L-bit). Loopback mode is used to send a copy of a packet back towards the source, as further described in Section 4.
Bit 2
"Active" (A-bit). When set, this indicates that this is an active IOAM packet, where "active" is used in the sense defined in [RFC7799], rather than a data packet. The packet may be an IOAM probe packet, or a replicated data packet (the second and third use cases of Section 5).
Bit 3
"Immediate Export" (I-bit). Immediate export mode is used to export IOAM data fields immediately at every IOAM supported network node, instead of adding the IOAM data fields to the packet traversing the network. Further details are provided in Section 6.

4. Loopback in IOAM

Loopback is used for trigerring each transit device along the path to loop back a copy of the data packet. Loopback mode assumes that a return path from transit nodes and destination nodes towards the source exists. The encapsulating node decides (e.g., using a filter) which packets loopback mode is enabled for by setting the loopback bit. The encapsulating node also needs to ensure that sufficient space is available in the IOAM header for loopback operation, which includes intermediate nodes adding trace data on the original path and then again on the return path. A loopback bit that is set indicates to the transit nodes processing this option that they are to create a copy of the received packet and send the copy back to the source of the packet. The copy has its metadata added after being copied in order to allow any egress-dependent information to be set based on the egress of the copy rather than the original. The original packet continues towards its destination. The source address of the original packet is used as the destination address in the copied packet. The address of the node performing the copy operation is used as the source address. The L-bit MUST be cleared in the copy of the packet that a node sends back towards the source. On its way back towards the source, the copied packet is processed like any other packet with IOAM information, including adding any requested data at each transit node (assuming there is sufficient space). Once the return packet reaches the IOAM domain boundary, IOAM decapsulation occurs as with any other packet containing IOAM information. Because any intermediate node receiving such a packet would not know how to process the original packet, and because there would be a risk of the original packet leaking past the initiator of the IOAM loopback, the initiator of an IOAM loopback MUST be the initiator of the packet. Once a loopback packet is received back at the initiator, it is a local matter how it is recognized as a loopback packet.

5. Active Measurement with IOAM

Active measurement methods [RFC7799] make use of synthetically generated packets in order to facilitate the measurement. This section presents use cases of active measurement using the IOAM Active flag.

The active flag indicates that a packet is used for active measurement. An IOAM decapsulating node that receives a packet with the Active flag set in one of its Trace options must terminate the packet.

An example of an IOAM deployment scenario is illustrated in Figure 1. The figure depicts two endpoints, a source and a destination. The data traffic from the source to the destination is forwarded through a set of network devices, including an IOAM encapsulating node, which incorporates one or more IOAM option, a decapsulating node, which removes the IOAM options, optionally one or more transit nodes. The IOAM options are encapsulated in one of the IOAM encapsulation types, e.g., [I-D.ietf-sfc-ioam-nsh], or [I-D.ioametal-ippm-6man-ioam-ipv6-options].

 +--------+     +--------+     +--------+     +--------+     +--------+
 |        |     |  IOAM  |.....|  IOAM  |.....|  IOAM  |     |        |
 +--------+     +--------+     +--------+     +--------+     +--------+
 | L2/L3  |<===>| L2/L3  |<===>| L2/L3  |<===>| L2/L3  |<===>| L2/L3  |
 +--------+     +--------+     +--------+     +--------+     +--------+
   Source      Encapsulating    Transit      Decapsulating   Destination
                   Node           Node           Node

Figure 1: Network using IOAM.

This draft focuses on three possible use cases of active measurement using IOAM. These use cases are described using the example of Figure 1.

6. Immediate Exporting

Immediate exporting can be used to export IOAM data to a collector instead of incorporating this data into en route data packets. The various types of IOAM nodes MUST process packets with the I-bit set as follows:

  1. An encapsulating IOAM node configured to set the I-bit encapsulates the packet with the IOAM header and sets the I-bit, leaving the IOAM header without locally collected data, and exports the requested IOAM data immediately. The encapsulating IOAM node is the only type of node allowed to set the I-bit.
  2. A transit node that processes a packet with the I-bit set is expected to export the requested IOAM data, and not incorporate it into the IOAM header.
  3. A decapsulating IOAM node that processes a packet with the I-bit set is expected to export the requested IOAM data, and decapsulate the IOAM header.

Note that in case of "Immediate Export" being employed, no IOAM trace data is added to the packets traversing the network. As a means to support correlation of exported IOAM data different nodes in the network, a deployment could consider attaching an IOAM E2E option in addition to the trace option, that includes a sequence number. See Bit 1 in the IOAM-E2E-Types. Please refer to [I-D.ietf-ippm-ioam-data] for a discussion of IOAM data export and associated formats.

7. IANA Considerations

IANA is requested to allocate the following bits in the "IOAM Trace Flags Registry" as follows:

Bit 1
"Loopback" (L-bit)
Bit 2
"Active" (A-bit)
Bit 3
"Immediate Export" (I-bit)

Note that bit 0 is the most significant bit in the Flags Registry.

8. Performance Considerations

Each of the three flags that are defined in this document may have performance implications. When using the loopback mechanism a copy of the data packet is sent back to the sender, thus generating more traffic than originally sent by the endpoints. Using active measurement with the active flag requires the use of synthetic (overhead) traffic. The Immediate Export flag triggers exported packets to be exported to a collector, which in some cases may impact the collector's performance, or the performance along the paths leading to the collector.

Each of the three mechanisms has a cost in terms of the network bandwidth, and may potentially load the node that analyzes the data. Therefore, rate limiting may be enabled so as to ensure that the three mechanisms are used at a rate that does not significantly affect the network bandwidth, and does not overload the collector (or the source node in the case of loopback). It should be possible to use each of the three mechanisms on a subset of the data traffic.

9. Security Considerations

The security considerations of IOAM in general are discussed in [I-D.ietf-ippm-ioam-data]. Specifically, an attacker may try to use the functionality that is defined in this document to attack the network.

An attacker may attempt to overload network devices by injecting synthetic packets that include an IOAM Trace Option with one or more of the flags defined in this document. Similarly, an on-path attacker may maliciously set one or more of the flags of transit packets.

In order to mitigate the attacks described above, it should be possible for IOAM-enabled devices to limit each of the three mechanisms to a configurable rate; Network devices should be able to limit the rate of: (i) looped-back traffic, (ii) replicated active packets, and (iii) exported packets.

IOAM is assumed to be deployed in a restricted administrative domain, thus limiting the scope of the threats above and their affect. This is a fundamental assumtion with respect to the security aspects of IOAM, as further discussed in [I-D.ietf-ippm-ioam-data].

10. References

10.1. Normative References

[I-D.ietf-ippm-ioam-data] Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, P., Chang, R., daniel.bernier@bell.ca, d. and J. Lemon, "Data Fields for In-situ OAM", Internet-Draft draft-ietf-ippm-ioam-data-05, March 2019.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997.

10.2. Informative References

[I-D.ietf-sfc-ioam-nsh] Brockners, F., Bhandari, S., Govindan, V., Pignataro, C., Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, P. and R. Chang, "Network Service Header (NSH) Encapsulation for In-situ OAM (IOAM) Data", Internet-Draft draft-ietf-sfc-ioam-nsh-01, March 2019.
[I-D.ioametal-ippm-6man-ioam-ipv6-options] Bhandari, S., Brockners, F., Pignataro, C., Gredler, H., Leddy, J., Youell, S., Mizrahi, T., Kfir, A., Gafni, B., Lapukhov, P., Spiegel, M., Krishnan, S. and R. Asati, "In-situ OAM IPv6 Options", Internet-Draft draft-ioametal-ippm-6man-ioam-ipv6-options-02, March 2019.
[I-D.spiegel-ippm-ioam-rawexport] Spiegel, M., Brockners, F., Bhandari, S. and R. Sivakolundu, "In-situ OAM raw data export with IPFIX", Internet-Draft draft-spiegel-ippm-ioam-rawexport-01, October 2018.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, May 2016.

Authors' Addresses

Tal Mizrahi Huawei Network.IO Innovation Lab Israel EMail: tal.mizrahi.phd@gmail.com
Frank Brockners Cisco Systems, Inc. Hansaallee 249, 3rd Floor DUESSELDORF, NORDRHEIN-WESTFALEN 40549 Germany EMail: fbrockne@cisco.com
Shwetha Bhandari Cisco Systems, Inc. Cessna Business Park, Sarjapura Marathalli Outer Ring Road Bangalore, KARNATAKA 560 087, India EMail: shwethab@cisco.com
Ramesh Sivakolundu Cisco Systems, Inc. 170 West Tasman Dr. SAN JOSE, CA 95134, U.S.A. EMail: sramesh@cisco.com
Carlos Pignataro Cisco Systems, Inc. 7200-11 Kit Creek Road Research Triangle Park, NC 27709 United States EMail: cpignata@cisco.com
Aviv Kfir Mellanox Technologies, Inc. 350 Oakmead Parkway, Suite 100 Sunnyvale, CA, 94085 U.S.A. EMail: avivk@mellanox.com
Barak Gafni Mellanox Technologies, Inc. 350 Oakmead Parkway, Suite 100 Sunnyvale, CA, 94085 U.S.A. EMail: gbarak@mellanox.com
Mickey Spiegel Barefoot Networks 4750 Patrick Henry Drive Santa Clara, CA, 95054 US EMail: mspiegel@barefootnetworks.com
John Lemon Broadcom 270 Innovation Drive San Jose, CA 95134 US EMail: john.lemon@broadcom.com