DetNet B. Varga, Ed.
Internet-Draft J. Farkas
Intended status: Standards Track Ericsson
Expires: November 7, 2019 A. Malis
S. Bryant
Huawei Technologies
J. Korhonen
May 6, 2019

DetNet Data Plane: MPLS over IEEE 802.1 Time Sensitive Networking (TSN)


This document specifies the Deterministic Networking MPLS data plane when operating over a TSN network.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on November 7, 2019.

Copyright Notice

Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents ( in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1. Introduction

[Editor's note: Introduction to be made specific to DetNet MPLS over TSN scenario. May be similar to intro of DetNet IP over TSN.].

Deterministic Networking (DetNet) is a service that can be offered by a network to DetNet flows. DetNet provides these flows with a low packet loss rates and assured maximum end-to-end delivery latency. General background and concepts of DetNet can be found in [I-D.ietf-detnet-architecture].

The DetNet Architecture decomposes the DetNet related data plane functions into two sub-layers: a service sub-layer and a forwarding sub-layer. The service sub-layer is used to provide DetNet service protection and reordering. The forwarding sub-layer is used to provides congestion protection (low loss, assured latency, and limited reordering) leveraging MPLS Traffic Engineering mechanisms.

This document specifies the DetNet data plane operation and the on-wire encapsulation of DetNet flows over an MPLS-based Packet Switched Network (PSN). The specified encapsulation provides the building blocks to enable the DetNet service and forwarding sub-layer functions and supports flow identification as described in the DetNet Architecture. As part of the service sub-layer functions, this document describes DetNet node data plane operation. It also describes the function and operation of the Packet Replication (PRF) Packet Elimination (PEF) and Packet Ordering (POF) functions with an MPLS data plane. It also describes an MPLS-based DetNet forwarding sub-layer that eliminates (or reduces) contention loss and provides bounded latency for DetNet flows.

MPLS encapsulated DetNet flows can be carried over network technologies that can provide the DetNet required level of service. This document defines examples of such, specifically carrying DetNet MPLS flows over IEEE 802.1 TSN sub-networks, and over DetNet IP PSN.

The intent is for this document to support different traffic types being mapped over DetNet MPLS, but this is out side the scope of this document. An example of such can be found in [I-D.ietf-detnet-dp-sol-ip]. This document also allows for, but does not define, associated controller plane and Operations, Administration, and Maintenance (OAM) functions.

2. Terminology

[Editor's note: Needs clean up.].

2.1. Terms Used in This Document

This document uses the terminology established in the DetNet architecture [I-D.ietf-detnet-architecture], and the reader is assumed to be familiar with that document and its terminology.

The following terminology is introduced in this document:

A Detnet "forwarding" label that identifies the LSP used to forward a DetNet flow across an MPLS PSN, e.g., a hop-by-hop label used between label switching routers (LSR).
A DetNet "service" label that is used between DetNet nodes that implement also the DetNet service sub-layer functions. An S-Label is also used to identify a DetNet flow at DetNet service sub-layer.
A DetNet Control Word (d-CW) is used for sequencing and identifying duplicate packets of a DetNet flow at the DetNet service sub-layer.

2.2. Abbreviations

The following abbreviations are used in this document:

Attachment Circuit.
Customer Edge equipment.
Class of Service.
Control Word.
Deterministic Networking.
DetNet Flow.
DetNet Inter-Working Function.
Layer 2.
Layer 2 Virtual Private Network.
Layer 3.
Label Switching Router.
Multiprotocol Label Switching.
Multiprotocol Label Switching - Traffic Engineering.
Multiprotocol Label Switching - Transport Profile.
Multi-Segment PseudoWire (MS-PW).
Native Service Processing.
Operations, Administration, and Maintenance.
Provider Edge.
Packet Elimination Function.
Packet Replication Function.
Packet Replication, Elimination and Ordering Functions.
Packet Ordering Function.
Packet Switched Network.
Quality of Service.
Switching Provider Edge.
Terminating Provider Edge.
Time-Sensitive Network.

3. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

4. DetNet MPLS Data Plane Overview

[Editor's note: simplify this section and highlight DetNet MPLS over subnets scenario being the focus in the remaining part of the document.].

4.1. Layers of DetNet Data Plane

This document describes how DetNet flows are carried over MPLS networks. The DetNet Architecture, [I-D.ietf-detnet-architecture], decomposes the DetNet data plane into two sub-layers: a service sub-layer and a forwarding sub-layer. The basic approach defined in this document supports the DetNet service sub-layer based on existing pseudowire (PW) encapsulations and mechanisms, and supports the DetNet forwarding sub-layer based on existing MPLS Traffic Engineering encapsulations and mechanisms. Background on PWs can be found in [RFC3985] and [RFC3031]. Background on MPLS Traffic Engineering can be found in [RFC3272] and [RFC3209].

  DetNet        MPLS
|  Service   | d-CW, S-Label
| Forwarding | F-Label(s)

Figure 1: DetNet Adaptation to MPLS Data Plane

The DetNet MPLS data plane approach defined in this document is shown in Figure 1. The service sub-layer is supported by a DetNet control word (d-CW) which conforms to the Generic PW MPLS Control Word (PWMCW) defined in [RFC4385]. A d-CW identifying service label (S-Label) is also used.

A node operating on a DetNet flow in the Detnet service sub-layer, i.e. a node processing a DetNet packet which has the S-Label as top of stack uses the local context associated with that S-Label, for example a received F-Label, to determine what local DetNet operation(s) are applied to that packet. An S-Label may be unique when taken from the platform label space [RFC3031], which would enable correct DetNet flow identification regardless of which input interface or LSP the packet arrives on.

The DetNet MPLS data plane builds on MPLS Traffic Engineering encapsulations and mechanisms to provide a forwarding sub-layer that is responsible for providing resource allocation and explicit routes. The forwarding sub-layer is supported by one or more forwarding labels (F-Labels).

4.2. DetNet MPLS Data Plane Scenarios

[Editor's note: simplify this section and highlight DetNet MPLS over subnets scenario being the focus in the remaining part of the document.].

DetNet MPLS       Relay       Transit         Relay       DetNet MPLS
End System        Node         Node           Node        End System
   (T-PE)        (S-PE)       (LSR)          (S-PE)         (T-PE)
+----------+                                             +----------+
|   Appl.  |<------------ End to End Service ----------->|   Appl.  |
+----------+   +---------+                 +---------+   +----------+
| Service  |<--| Service |-- DetNet flow --| Service |-->| Service  |
+----------+   +---------+  +----------+   +---------+   +----------+
|Forwarding|   |Fwd| |Fwd|  |Forwarding|   |Fwd| |Fwd|   |Forwarding|
+-------.--+   +-.-+ +-.-+  +----.---.-+   +-.-+ +-.-+   +---.------+
        :  Link  :    /  ,-----.  \   : Link :    /  ,-----.  \
        +........+    +-[  Sub  ]-+   +......+    +-[  Sub  ]-+
                        [Network]                   [Network]
                         `-----'                     `-----'
        |<- LSP -->| |<-------- LSP -----------| |<--- LSP -->|

        |<----------------- DetNet MPLS --------------------->|


Figure 2: A DetNet MPLS Network

Figure 2 illustrates a hypothetical DetNet MPLS-only network composed of DetNet aware MPLS enabled end systems, operating over a DetNet aware MPLS network. In this figure, relay nodes sit at MPLS LSP boundaries and transit nodes are LSRs.

DetNet end system and relay nodes are DetNet service sub-layer aware, understand the particular needs of DetNet flows and provide both DetNet service and forwarding sub-layer functions. They add, remove and process d-CWs, S-Labels and F-labels as needed. MPLS enabled end system and relay nodes can enhance the reliability of delivery by enabling the replication of packets where multiple copies, possibly over multiple paths, are forwarded through the DetNet domain. They can also eliminate surplus previously replicated copies of DetNet packets. DetNet MPLS nodes provide functionality similar to T-PEs when they sit at the edge of an MPLS domain, and functionality similar to S-PEs when they are in the middle of an MPLS domain, see [RFC6073]. End system and relay nodes also include DetNet forwarding sub-layer functions, support for notably explicit routes, and resources allocation to eliminate (or reduce) congestion loss and jitter.

DetNet transit nodes reside wholly within a DetNet domain, and also provide DetNet forwarding sub-layer functions in accordance with the performance required by a DetNet flow carried over an LSP. Unlike other DetNet node types, transit nodes provide no service sub-layer processing. In a DetNet MPLS network, transit nodes may be DetNet service aware or may be DetNet unaware MPLS Label Switching Routers (LSRs). In this latter case, such LSRs would be unaware of the special requirements of the DetNet service sub-layer, but would still provide traffic engineering services and the QoS need to ensure that the (TE) LSPs meet the service requirements of the carried DetNet flows.

The LSPs may be provided by any MPLS controller method. For example they may be provisioned via a management plane, RSVP-TE, MPLS-TP, or MPLS Segment Routing (when extended to support resource allocation).

[Editor's note: Figure 3. and surrunding text are candidates to delete from this document.].

Figure 3 illustrates how an end to end MPLS-based DetNet service is provided in a more detail. In this figure, the end systems, CE1 and CE2, are able to send and receive MPLS encapsulated DetNet flows, and R1, R2 and R3 are relay nodes as they sit in the middle of a DetNet network. The 'X' in the end systems, and relay nodes represents potential DetNet compound flow packet replication and elimination points. In this example, service protection is supported over four DetNet member flows and TE LSPs. For a unidirectional flow, R1 supports PRF, R2 supports PREOF and R3 supports PEF and POF. Note that the relay nodes may change the underlying forwarding sub-layer, for example tunneling MPLS over IEEE 802.1 TSN Section 6, or simply over interconnect network links.

      DetNet                                           DetNet
MPLS  Service          Transit          Transit       Service MPLS
DetNet  |             |<-Tnl->|        |<-Tnl->|            | DetNet
End     |             V   1   V        V   2   V            | End
System  |    +--------+       +--------+       +--------+   | System
+---+   |    |   R1   |=======|   R2   |=======|   R3   |   |  +---+
|  X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X |
|CE1|========|    \   |       |   X    |       |   /    |======|CE2|
|   |   |    |     \_.|..DF2..|._/ \__.|..DF4..|._/     |   |  |   |
+---+        |        |=======|        |=======|        |      +---+
    ^        +--------+       +--------+       +--------+      ^
    |        Relay Node       Relay Node       Relay Node      |
    |          (S-PE)           (S-PE)           (S-PE)        |
    |                                                          |
    |<---------------------- DetNet MPLS --------------------->|
    |                                                          |
    |<--------------- End to End DetNet Service -------------->|

   -------------------------- Data Flow ------------------------->

    X   = Optional service protection (none, PRF, PREOF, PEF/POF)
    DFx = DetNet member flow x over a TE LSP

Figure 3: MPLS-Based Native DetNet

As previously mentioned, this document specifies how MPLS is used to support DetNet flows using an MPLS data plane as well as how such can be mapped to IEEE 802.1 TSN and IP DetNet PSNs. An equally import scenario is when IP is supported over DetNet MPLS and this is covered in [I-D.ietf-detnet-dp-sol-ip]. Another important scenario is where an Ethernet Layer 2 service is supported over DetNet MPLS and this is covered in [TBD-TSN-OVER-DETNET].

4.3. Packet Flow Example with Service Protection

[Editor's note: this text might be relevant for the discussion of FRER within the TSN sub-network. Needs revision.].

An example DetNet MPLS network fragment and packet flow is illustrated in Figure 4.

   1      1.1       1.1      1.2.1    1.2.1      1.2.2
         \           1.2.1 /                   /
          \1.2     /-----+                   /

Figure 4: Example Packet Flow in DetNet Enabled MPLS Network

In Figure 4 the numbers are used to identify the instance of a packet. Packet 1 is the original packet, and packets 1.1, and 1.2 are two first generation copies of packet 1. Packet 1.2.1 is a second generation copy of packet 1.2 etc. Note that these numbers never appear in the packet, and are not to be confused with sequence numbers, labels or any other identifier that appears in the packet. They simply indicate the generation number of the original packet so that its passage through the network fragment can be identified to the reader.

Customer Equipment CE1 sends a packet into the DetNet enabled MPLS network. This is packet (1). Edge Node EN1 encapsulates the packet as a DetNet Packet and sends it to Relay node R1 (packet 1.1). EN1 makes a copy of the packet (1.2), encapsulates it and sends this copy to Relay node R4.

Note that along the MPLS path from EN1 to R1 there may be zero or more LSRs which, for clarity, are not shown. The same is true for any other path between two DetNet entities shown in Figure 4.

Relay node R4 has been configured to send one copy of the packet to Relay Node R2 (packet 1.2.1) and one copy to Edge Node EN2 (packet 1.2.2).

R2 receives packet copy 1.2.1 before packet copy 1.1 arrives, and, having been configured to perform packet elimination on this DetNet flow, forwards packet 1.2.1 to Relay Node R3. Packet copy 1.1 is of no further use and so is discarded by R2.

Edge Node EN2 receives packet copy 1.2.2 from R4 before it receives packet copy 1.2.1 from R2 via relay Node R3. EN2 therefore strips any DetNet encapsulation from packet copy 1.2.2 and forwards the packet to CE2. When EN2 receives the later packet copy 1.2.1 this is discarded.

The above is of course illustrative of many network scenarios that can be configured. Between a pair of relay nodes there may be one or more transit nodes that simply forward the DetNet traffic, but these are omitted for clarity.

5. DetNet MPLS Data Plane Considerations

[Editor's note: Sort out what data plane considerations are relevant for sub-net scenarios.].

This section provides informative considerations related to providing DetNet service to flows which are identified based on their header information. At a high level, the following are provided on a per flow basis:

Eliminating contention loss and jitter reduction:

Use of allocated resources (queuing, policing, shaping) to ensure that the congestion-related loss and latency/jitter requirements of a DetNet flow are met.
Explicit routes:

Use of a specific path for a flow. This limits misordering and bounds latency.
Service protection:

Which in the case of this document primarily relates to replication and elimination. Changing the explicit path after a failure is detected in order to restore delivery of the required DetNet service characteristics is also possible. Path changes, even in the case of failure recovery, can lead to the out of order delivery of data.
Load sharing:

Generally, distributing packets of the same DetNet flow over multiple paths is not recommended. Such load sharing, e.g., via ECMP or UCMP, impacts ordering and possibly jitter.

For example, to support identification of misbehaving flows.
Recognize flow(s) for analytics:

For example, increase counters.
Correlate events with flows:

For example, unexpected loss.

The DetNet data plane also allows for the aggregation of DetNet flows, e.g., via MPLS hierarchical LSPs, to improved scaling. When DetNet flows are aggregated, transit nodes provide service to the aggregate and not on a per-DetNet flow basis. In this case, nodes performing aggregation will ensure that per-flow service requirements are achieved.

5.1. Sub-Network Considerations

As shown in Figure 2, MPLS nodes are interconnected by different sub-network technologies, which may include point-to-point links. Each of these need to provide appropriate service to DetNet flows. In some cases, e.g., on dedicated point-to-point links or TDM technologies, all that is required is for a DetNet node to appropriately queue its output traffic. In other cases, DetNet nodes will need to map DetNet flows to the flow semantics (i.e., identifiers) and mechanisms used by an underlying sub-network technology. Figure 5 shows several examples of header formats that can be used to carry DetNet MPLS flows over different sub-network technologies. L2 represent a generic layer-2 encapsulation that might be used on a point-to-point link. TSN represents the encapsulation used on an IEEE 802.1 TSN network, as described in Section 6. UDP/IP represents the encapsulation used on a DetNet IP PSN.

                   +------+  +------+  +------+
App-Flow           |  X   |  |  X   |  |  X   |
DetNet-MPLS        | d-CW |  | d-CW |  | d-CW |
                   +------+  +------+  +------+
                   |Labels|  |Labels|  |Labels|
Sub-Network        |  L2  |  | TSN  |  | UDP  |
                   +------+  +------+  +------+
                                       |  IP  |
                                       |  L2  |

Figure 5: Example DetNet MPLS Sub-Network Formats

6. DetNet MPLS Operation Over IEEE 802.1 TSN Sub-Networks

[Editor's note: this is a place holder section. A standalone section on MPLS over IEEE 802.1 TSN. Includes RFC2119 Language.]

This section covers how DetNet MPLS flows operate over an IEEE 802.1 TSN sub-network. Figure 6 illustrates such a scenario, where two MPLS (DetNet) nodes are interconnected by a TSN sub-network. Node-1 is single homed and Node-2 is dual-homed. MPLS nodes can be (1) DetNet MPLS End System, (2) DetNet MPLS Edge or Relay node or (3) MPLS Transit node.

Note: in case of MPLS Transit node there is no DetNet Service sub-layer processing.

   MPLS (DetNet)                 MPLS (DetNet)
      Node-1                        Node-2

   +----------+                  +----------+
<--| Service* |-- DetNet flow ---| Service* |-->
   +----------+                  +----------+
   |Forwarding|                  |Forwarding|
   +--------.-+    <-TSN Str->   +-.-----.--+
             \      ,-------.     /     /
              +----[ TSN-Sub ]---+     /
                   [ Network ]--------+
<---------------- DetNet MPLS --------------->

Note: * no service sub-layer required for transit nodes

Figure 6: DetNet Enabled MPLS Network Over a TSN Sub-Network

The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 Working Group have defined (and are defining) a number of amendments to IEEE 802.1Q that provide zero congestion loss and bounded latency in bridged networks. Furthermore IEEE 802.1CB defines frame replication and elimination functions for reliability that should prove both compatible with and useful to, DetNet networks. All these functions have to identify flows those require TSN treatment.

As is the case for DetNet, a Layer 2 network node such as a bridge may need to identify the specific DetNet flow to which a packet belongs in order to provide the TSN/DetNet QoS for that packet. It also may need a CoS marking, such as the priority field of an IEEE Std 802.1Q VLAN tag, to give the packet proper service.

The challange for MPLS DeNet flows is that the protocol interworking function defined in IEEE 802.1CB works only for IP flows. The aim of the protocol interworking function is to convert an ingress flow to use a specific multicast destination MAC address and VLAN, for example to direct the packets through a specific path inside the bridged network. A similar interworking pair at the other end of the TSN sub-network would restore the packet to its original destination MAC address and VLAN.

As protocol interworking function defined in [IEEE8021CB] does not work for MPLS labeled flows, the DetNet MPLS nodes MUST ensure proper TSN sub-network specific Ethernet encapsulation of the DetNet MPLS packets. For a given TSN Stream (i.e., DetNet flow) an MPLS (DetNet) node MUST behave as a TSN-aware Talker or a Listener inside the TSN sub-network.

6.1. Mapping of TSN Stream ID and Sequence Number

TSN capable MPLS (DetNet) nodes are TSN-aware Talker/Listener as shown in Figure 7. MPLS (DetNet) node MUST provide the TSN sub-network specific Ethernet encapsulation over the link(s) towards the sub-network. An TSN-aware MPLS (DetNet) node MUST support the following TSN components:

  1. For recognizing flows:

  2. For FRER used inside the TSN domain, additonaly:

  3. For FRER when the node is a TSN replication or elimination point, additionally:

[Editor's note: Should we added here requirements regarding IEEE 802.1Q C-VLAN component?]

The Stream Identification and The Sequencing functions are slightly modified for frames passed down the protocol stack from the upper layers.

Stream Identification MUST pair MPLS flows and TSN Streams and encode that in data plane formats as well. The packet's stream_handle subparameter (see IEEE 802.1CB) inside the Talker/Listener is defined based on the Flow-ID used in the upper DetNet MPLS layer. Stream Identification function MUST encode Ethernet header fields namely (1) the destination MAC-address, (2) the VLAN-ID and (3) priority parameters with TSN sub-network specific values. Encoding is provided for the frame passed down the stack from the upper layers.

The sequence generation function resides in the Sequencing function. It generates a sequence_number subparameter for each packet of a Stream passed down to the lower layers. Sequencing function MUST copy sequence information from the MPLS d-CW of the packet to the sequence_number subparameter for the frame passed down the stack from the upper layers.

   MPLS (DetNet)

<--| Service  |-- DetNet flow ------------------
   +----------+    +---------------+
   | L2 with  |<---| L2 Relay with |---- TSN ----
   |   TSN    |    | TSN function  |    Stream
   +-----.----+    +--.---------.--+
          \__________/           \______

     Talker /          TSN-Bridge
     Listener             Relay

         <--------- TSN sub-network ------------

Figure 7: MPLS (DetNet) Node with TSN Functions

The Sequence encode/decode function MUST support the Redundancy tag (R-TAG) format as per Clause 7.8 of IEEE 802.1CB.

6.2. TSN Usage of FRER

TSN Streams supporting DetNet flows may use Frame Replication and Elimination for Redundancy (FRER) [802.1CB] based on the loss service requirements of the TSN Stream, which is derived from the DetNet service requirements of the DetNet mapped flow. The specific operation of FRER is not modified by the use of DetNet and follows IEEE 802.1CB.

FRER function and the provided service recovery is available only within the TSN sub-network however as the Stream-ID and the TSN sequence number are paired with the MPLS flow parameters they can be combined with PREOF functions.

6.3. Procedures

[Editor's note: This section is TBD - covers required behavior of a TSN-aware DetNet node using a TSN underlay.]

6.4. Layer 2 Addressing and QoS Considerations

[Editor's NOTE: review and simplify this section. May overlap with previous sections.]

The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1 Working Group have defined (and are defining) a number of amendments to IEEE 802.1Q that provide zero congestion loss and bounded latency in bridged networks. IEEE 802.1CB defines packet replication and elimination functions that should prove both compatible with and useful to, DetNet networks.

As is the case for DetNet, a Layer 2 network node such as a bridge may need to identify the specific DetNet flow to which a packet belongs in order to provide the TSN/DetNet QoS for that packet. It also will likely need a CoS marking, such as the priority field of an IEEE Std 802.1Q VLAN tag, to give the packet proper service.

Although the flow identification methods described in IEEE 802.1CB are flexible, and in fact, include IP 5-tuple identification methods, the baseline TSN standards assume that every Ethernet frame belonging to a TSN stream (i.e. DetNet flow) carries a multicast destination MAC address that is unique to that flow within the bridged network over which it is carried. Furthermore, IEEE 802.1CB describes three methods by which a packet sequence number can be encoded in an Ethernet frame.

Ensuring that the proper Ethernet VLAN tag priority and destination MAC address are used on a DetNet/TSN packet may require further clarification of the customary L2/L3 transformations carried out by routers and edge label switches. Edge nodes may also have to move sequence number fields among Layer 2, PW, and IP encapsulations.

7. Management and Control Considerations

[Editor's note: This section is TBD Covers Creation, mapping, removal of TSN Stream IDs, related parameters and,when needed, configuration of FRER. Supported by management/control plane. SEE sections in removed text file.]

While management plane and control planes are traditionally considered separately, from the Data Plane perspective there is no practical difference based on the origin of flow provisioning information, and the DetNet architecture [I-D.ietf-detnet-architecture] refers to these collectively as the 'Controller Plane'. This document therefore does not distinguish between information provided by distributed control plane protocols, e.g., RSVP-TE [RFC3209] and [RFC3473], or by centralized network management mechanisms, e.g., RestConf [RFC8040], YANG [RFC7950], and the Path Computation Element Communication Protocol (PCEP) [I-D.ietf-pce-pcep-extension-for-pce-controller] or any combination thereof. Specific considerations and requirements for the DetNet Controller Plane are discussed below.

8. Security Considerations

The security considerations of DetNet in general are discussed in [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security]. Other security considerations will be added in a future version of this draft.

9. IANA Considerations

This document makes no IANA requests.

10. Acknowledgements

Thanks for Norman Finn and Lou Berger for their comments and contributions.

11. References

11.1. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997.
[RFC2211] Wroclawski, J., "Specification of the Controlled-Load Network Element Service", RFC 2211, DOI 10.17487/RFC2211, September 1997.
[RFC2212] Shenker, S., Partridge, C. and R. Guerin, "Specification of Guaranteed Quality of Service", RFC 2212, DOI 10.17487/RFC2212, September 1997.
[RFC3031] Rosen, E., Viswanathan, A. and R. Callon, "Multiprotocol Label Switching Architecture", RFC 3031, DOI 10.17487/RFC3031, January 2001.
[RFC3032] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., Farinacci, D., Li, T. and A. Conta, "MPLS Label Stack Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001.
[RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V. and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001.
[RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, P., Krishnan, R., Cheval, P. and J. Heinanen, "Multi-Protocol Label Switching (MPLS) Support of Differentiated Services", RFC 3270, DOI 10.17487/RFC3270, May 2002.
[RFC3443] Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing in Multi-Protocol Label Switching (MPLS) Networks", RFC 3443, DOI 10.17487/RFC3443, January 2003.
[RFC3473] Berger, L., "Generalized Multi-Protocol Label Switching (GMPLS) Signaling Resource ReserVation Protocol-Traffic Engineering (RSVP-TE) Extensions", RFC 3473, DOI 10.17487/RFC3473, January 2003.
[RFC4206] Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP) Hierarchy with Generalized Multi-Protocol Label Switching (GMPLS) Traffic Engineering (TE)", RFC 4206, DOI 10.17487/RFC4206, October 2005.
[RFC4385] Bryant, S., Swallow, G., Martini, L. and D. McPherson, "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385, February 2006.
[RFC5085] Nadeau, T. and C. Pignataro, "Pseudowire Virtual Circuit Connectivity Verification (VCCV): A Control Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085, December 2007.
[RFC5129] Davie, B., Briscoe, B. and J. Tay, "Explicit Congestion Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January 2008.
[RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic Class" Field", RFC 5462, DOI 10.17487/RFC5462, February 2009.
[RFC7510] Xu, X., Sheth, N., Yong, L., Callon, R. and D. Black, "Encapsulating MPLS in UDP", RFC 7510, DOI 10.17487/RFC7510, April 2015.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017.

11.2. Informative References

[G.8275.1] International Telecommunication Union, "Precision time protocol telecom profile for phase/time synchronization with full timing support from the network", ITU-T G.8275.1/Y.1369.1 G.8275.1, June 2016.
[G.8275.2] International Telecommunication Union, "Precision time protocol telecom profile for phase/time synchronization with partial timing support from the network", ITU-T G.8275.2/Y.1369.2 G.8275.2, June 2016.
[I-D.ietf-detnet-architecture] Finn, N., Thubert, P., Varga, B. and J. Farkas, "Deterministic Networking Architecture", Internet-Draft draft-ietf-detnet-architecture-12, March 2019.
[I-D.ietf-detnet-dp-sol-ip] Korhonen, J., Varga, B., "DetNet IP Data Plane Encapsulation", 2018.
[I-D.ietf-detnet-flow-information-model] Farkas, J., Varga, B., Cummings, R. and Y. Jiang, "DetNet Flow Information Model", Internet-Draft draft-ietf-detnet-flow-information-model-03, March 2019.
[I-D.ietf-pce-pcep-extension-for-pce-controller] Zhao, Q., Li, Z., Negi, M. and C. Zhou, "PCEP Procedures and Protocol Extensions for Using PCE as a Central Controller (PCECC) of LSPs", Internet-Draft draft-ietf-pce-pcep-extension-for-pce-controller-01, February 2019.
[I-D.ietf-spring-segment-routing-mpls] Bashandy, A., Filsfils, C., Previdi, S., Decraene, B., Litkowski, S. and R. Shakir, "Segment Routing with MPLS data plane", Internet-Draft draft-ietf-spring-segment-routing-mpls-21, April 2019.
[I-D.sdt-detnet-security] Mizrahi, T., Grossman, E., Hacker, A., Das, S., "Deterministic Networking (DetNet) Security Considerations, draft-sdt-detnet-security, work in progress", 2017.
[IEEE1588] IEEE, "IEEE 1588 Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems Version 2", 2008.
[IEEE8021CB] Finn, N., "Draft Standard for Local and metropolitan area networks - Seamless Redundancy", IEEE P802.1CB /D2.1 P802.1CB, December 2015.
[IEEE8021Q] IEEE 802.1, "Standard for Local and metropolitan area networks--Bridges and Bridged Networks (IEEE Std 802.1Q-2014)", 2014.
[RFC2205] Braden, R., Zhang, L., Berson, S., Herzog, S. and S. Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification", RFC 2205, DOI 10.17487/RFC2205, September 1997.
[RFC2474] Nichols, K., Blake, S., Baker, F. and D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", RFC 2474, DOI 10.17487/RFC2474, December 1998.
[RFC3272] Awduche, D., Chiu, A., Elwalid, A., Widjaja, I. and X. Xiao, "Overview and Principles of Internet Traffic Engineering", RFC 3272, DOI 10.17487/RFC3272, May 2002.
[RFC3985] Bryant, S. and P. Pate, "Pseudo Wire Emulation Edge-to-Edge (PWE3) Architecture", RFC 3985, DOI 10.17487/RFC3985, March 2005.
[RFC4448] Martini, L., Rosen, E., El-Aawar, N. and G. Heron, "Encapsulation Methods for Transport of Ethernet over MPLS Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006.
[RFC4872] Lang, J., Rekhter, Y. and D. Papadimitriou, "RSVP-TE Extensions in Support of End-to-End Generalized Multi-Protocol Label Switching (GMPLS) Recovery", RFC 4872, DOI 10.17487/RFC4872, May 2007.
[RFC4873] Berger, L., Bryskin, I., Papadimitriou, D. and A. Farrel, "GMPLS Segment Recovery", RFC 4873, DOI 10.17487/RFC4873, May 2007.
[RFC4875] Aggarwal, R., Papadimitriou, D. and S. Yasukawa, "Extensions to Resource Reservation Protocol - Traffic Engineering (RSVP-TE) for Point-to-Multipoint TE Label Switched Paths (LSPs)", RFC 4875, DOI 10.17487/RFC4875, May 2007.
[RFC5440] Vasseur, JP. and JL. Le Roux, "Path Computation Element (PCE) Communication Protocol (PCEP)", RFC 5440, DOI 10.17487/RFC5440, March 2009.
[RFC5586] Bocci, M., Vigoureux, M. and S. Bryant, "MPLS Generic Associated Channel", RFC 5586, DOI 10.17487/RFC5586, June 2009.
[RFC5654] Niven-Jenkins, B., Brungard, D., Betts, M., Sprecher, N. and S. Ueno, "Requirements of an MPLS Transport Profile", RFC 5654, DOI 10.17487/RFC5654, September 2009.
[RFC5921] Bocci, M., Bryant, S., Frost, D., Levrau, L. and L. Berger, "A Framework for MPLS in Transport Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010.
[RFC6003] Papadimitriou, D., "Ethernet Traffic Parameters", RFC 6003, DOI 10.17487/RFC6003, October 2010.
[RFC6006] Zhao, Q., King, D., Verhaeghe, F., Takeda, T., Ali, Z. and J. Meuric, "Extensions to the Path Computation Element Communication Protocol (PCEP) for Point-to-Multipoint Traffic Engineering Label Switched Paths", RFC 6006, DOI 10.17487/RFC6006, September 2010.
[RFC6073] Martini, L., Metz, C., Nadeau, T., Bocci, M. and M. Aissaoui, "Segmented Pseudowire", RFC 6073, DOI 10.17487/RFC6073, January 2011.
[RFC6387] Takacs, A., Berger, L., Caviglia, D., Fedyk, D. and J. Meuric, "GMPLS Asymmetric Bandwidth Bidirectional Label Switched Paths (LSPs)", RFC 6387, DOI 10.17487/RFC6387, September 2011.
[RFC6790] Kompella, K., Drake, J., Amante, S., Henderickx, W. and L. Yong, "The Use of Entropy Labels in MPLS Forwarding", RFC 6790, DOI 10.17487/RFC6790, November 2012.
[RFC7551] Zhang, F., Jing, R. and R. Gandhi, "RSVP-TE Extensions for Associated Bidirectional Label Switched Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015.
[RFC7950] Bjorklund, M., "The YANG 1.1 Data Modeling Language", RFC 7950, DOI 10.17487/RFC7950, August 2016.
[RFC8040] Bierman, A., Bjorklund, M. and K. Watsen, "RESTCONF Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017.
[RFC8169] Mirsky, G., Ruffini, S., Gray, E., Drake, J., Bryant, S. and A. Vainshtein, "Residence Time Measurement in MPLS Networks", RFC 8169, DOI 10.17487/RFC8169, May 2017.

Appendix A. Example of DetNet Data Plane Operation

[Editor's note: Add a simplified example of DetNet data plane and how labels etc work in the case of MPLS-based PSN and utilizing PREOF. The figure is subject to change depending on the further DT decisions on the label handling..]

Authors' Addresses

Balázs Varga (editor) Ericsson Magyar Tudosok krt. 11. Budapest, 1117 Hungary EMail:
János Farkas Ericsson Magyar Tudosok krt. 11. Budapest, 1117 Hungary EMail:
Andrew G. Malis Huawei Technologies EMail:
Stewart Bryant Huawei Technologies EMail:
Jouni Korhonen EMail: