[Docs] [txt|pdf] [Tracker] [Email] [Nits]



Detnet Working Group                                           S. Bryant
Internet-Draft                                                   M. Chen
Intended status: Standards Track                     Huawei Technologies
Expires: September 6, 2018                                March 05, 2018


             Operation of Deterministic Networks over MPLS
                     draft-bryant-detnet-mpls-dp-00

Abstract

   This document specifies Deterministic Networking data plane operation
   over an MPLS Packet Switched Networks.

   This document is a derivative work from draft-ietf-detnet-dp-sol-01.

   Whether this is published as a stand-alone text, or serves as a focal
   point to refine the MPLS design and the key point are subsequently
   re-merged with draft-ietf-detnet-dp-sol-01 is a matter for the DETNET
   WG, as is the editorship of any WG text.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 6, 2018.

Copyright Notice

   Copyright (c) 2018 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect



Bryant & Chen           Expires September 6, 2018               [Page 1]


Internet-Draft             Deterministic MPLS                 March 2018


   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.1.  Terms used in this document . . . . . . . . . . . . . . .   3
     2.2.  Abbreviations . . . . . . . . . . . . . . . . . . . . . .   4
   3.  Requirements language . . . . . . . . . . . . . . . . . . . .   5
   4.  DetNet Over an MPLS Underlay  . . . . . . . . . . . . . . . .   5
   5.  DetNet over MPLS Encapsulation Components . . . . . . . . . .   8
     5.1.  Basic Data Plane Encapsulation  . . . . . . . . . . . . .   8
     5.2.  DetNet Control Word . . . . . . . . . . . . . . . . . . .   9
     5.3.  Flow identification . . . . . . . . . . . . . . . . . . .  10
     5.4.  OAM Indication  . . . . . . . . . . . . . . . . . . . . .  11
     5.5.  Flow Aggregation  . . . . . . . . . . . . . . . . . . . .  11
     5.6.  Aggregation at the LSP  . . . . . . . . . . . . . . . . .  12
     5.7.  Aggregating DetNet flows as a new DetNet flow . . . . . .  12
   6.  Simple Aggregation at the DetNet layer  . . . . . . . . . . .  13
   7.  Indication of the DetNet Payload Type.  . . . . . . . . . . .  13
   8.  Operation of the PREF Functions . . . . . . . . . . . . . . .  14
     8.1.  Operation of PR . . . . . . . . . . . . . . . . . . . . .  14
     8.2.  Operation of EF . . . . . . . . . . . . . . . . . . . . .  15
     8.3.  Packet reordering considerations  . . . . . . . . . . . .  15
     8.4.  Indication of PREF processing . . . . . . . . . . . . . .  16
     8.5.  Placement of PR and EF in a node  . . . . . . . . . . . .  16
   9.  Operation at DetNet Node types  . . . . . . . . . . . . . . .  17
     9.1.  Operation at an Ingress Edge (PE) Node  . . . . . . . . .  17
     9.2.  Operation at a Relay node (S-PE)  . . . . . . . . . . . .  17
     9.3.  Operation at an Egress Edge (PE) Node . . . . . . . . . .  18
     9.4.  Operation at a Transit(P) Node  . . . . . . . . . . . . .  18
   10. Other DetNet data plane considerations  . . . . . . . . . . .  19
     10.1.  Class of Service . . . . . . . . . . . . . . . . . . . .  19
     10.2.  Quality of Service . . . . . . . . . . . . . . . . . . .  19
     10.3.  Bidirectional traffic  . . . . . . . . . . . . . . . . .  20
     10.4.  Layer 2 addressing and QoS Considerations  . . . . . . .  21
   11. Management and control considerations . . . . . . . . . . . .  22
     11.1.  S-Label assignment and distribution  . . . . . . . . . .  22
     11.2.  Explicit routes  . . . . . . . . . . . . . . . . . . . .  22
   12. Security considerations . . . . . . . . . . . . . . . . . . .  22
   13. IANA considerations . . . . . . . . . . . . . . . . . . . . .  23
     13.1.  Acknowledgments  . . . . . . . . . . . . . . . . . . . .  23
   14. References  . . . . . . . . . . . . . . . . . . . . . . . . .  23
     14.1.  Normative References . . . . . . . . . . . . . . . . . .  23
     14.2.  Informative References . . . . . . . . . . . . . . . . .  23



Bryant & Chen           Expires September 6, 2018               [Page 2]


Internet-Draft             Deterministic MPLS                 March 2018


   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  26

1.  Introduction

   This document is a derivative work from [draft-ietf-detnet-dp-sol-
   01].

   Editor's Note: We need to point to the exact version that this was
   derived from, not a generic version of [I-D.ietf-detnet-dp-sol].

   Whether this is published as a stand-alone text, or serves as a focal
   point to refine the MPLS design and the key point are subsequently
   re-merged with draft-ietf-detnet-dp-sol-01 is a matter for the DETNET
   WG.

   Deterministic Networking (DetNet) is a service that can be offered by
   a network to DetNet flows.  DetNet provides these flows extremely low
   packet loss rates and assured maximum end-to-end delivery latency.
   General background and concepts of DetNet can be found in
   [I-D.ietf-detnet-architecture].

   This document specifies the encapsulation and operation of
   deterministic networking over an MPLS data-plane.  The approach is
   modeled on the operation of Pseudowires (PW) over an MPLS Packet
   Switched Network (PSN) [RFC3985][RFC4385].

   The DetNet transport layer functionality that provides congestion
   protection for DetNet flows is assumed to be in place in a DetNet
   node.

   This document does not define the associated control plane functions,
   or Operations, Administration, and Maintenance (OAM).  It also does
   not specify traffic handling capabilities required to deliver
   congestion protection and latency control for DetNet flows at the
   DetNet transport layer.

2.  Terminology

2.1.  Terms used in this document

   Editor's note: This section needs to be reviewed when the body of the
   text is closer to completion.

   This document uses the terminology established in the DetNet
   architecture [I-D.ietf-detnet-architecture] and the DetNet Data Plane
   Solution Alternatives [I-D.ietf-detnet-dp-alt].





Bryant & Chen           Expires September 6, 2018               [Page 3]


Internet-Draft             Deterministic MPLS                 March 2018


   T-Label A label used to identify the LSP used to transport a DetNet
   flow across an MPLS PSN, e.g., a hop-by-hop label used between label
   switching routers (LSR).

   S-Label A DetNet "service" label that is used between DetNet nodes
   that implement also the DetNet service layer functions.  An S-Label
   is also used to identify a DetNet flow at DetNet service layer.

   Local-ID A DetNet Edge and Relay node internal construct that
   uniquely identifies a DetNet flow within a node and never appear on-
   wire.  It may be used to select proper forwarding and/or DetNet
   specific service function.

   PREF A Packet Replication and Elimination Function (PREF) does the
   replication and elimination processing of DetNet flow packets in edge
   or relay nodes.  The replication function is essentially the existing
   1+1 protection mechanism.  The elimination function reuses and
   extends the existing duplicate detection mechanism to operate over
   multiple (separate) DetNet member flows of a DetNet compound flow.

   DetNet Control Word A control word used for sequencing and
   identifying duplicate packets at the DetNet service layer.

2.2.  Abbreviations

   Editor's note: This section needs to be reviewed when the body of the
   text is closer to completion.

   The following abbreviations used in this document:

   AC Attachment Circuit.

   CE Customer Edge equipment.

   CoS Class of Service.

   CW Control Word.

   d-CW DetNet Control Word.

   DetNet Deterministic Networking.

   DF DetNet Flow.

   L2VPN Layer 2 Virtual Private Network.

   LSR Label Switching Router.




Bryant & Chen           Expires September 6, 2018               [Page 4]


Internet-Draft             Deterministic MPLS                 March 2018


   MPLS Multiprotocol Label Switching.

   MPLS-TP Multiprotocol Label Switching - Transport Profile.

   MS-PW Multi-Segment Pseudowire (MS-PW).

   NSP Native Service Processing.

   OAM Operations, Administration, and Maintenance.

   PE Provider Edge.

   PREF Packet Replication and Elimination Function.

   PSN Packet Switched Network.

   PW Pseudowire.

   QoS Quality of Service.

   TSN Time-Sensitive Network.

3.  Requirements language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

4.  DetNet Over an MPLS Underlay

   Figure 1 Shows the basic components of a DetNet enabled MPLS network
   used to transport TSN traffic using an MPLS transport.



















Bryant & Chen           Expires September 6, 2018               [Page 5]


Internet-Draft             Deterministic MPLS                 March 2018


   DetNet    Edge       Transit   Relay       Edge        DetNet
   End Sys   Node        Node     Node        Node        End Sys

   +-----+             End to End Service                 +-----+
   |Appln|<..............................................>|Appln|
   +-----+  +---------+ DN Flow +---------+  +---------+  +-----+
   | TSN |  | Service |<------->| Service |<>| Service |  | TSN |
   +-----+  +---+ +---+ +-----+ +---+ +---+  +---+ +---+  +-----+
   |DNXpt|  |Xpt| |LSP| | LSP | |LSP| |LSP|  |LSP| |Xpt|  |DNXpt|
   +--.--+  +-.-+ +-.-+ +-.-.-+ +-.-+ +-.-+  +-.-+ +-.-+  +--.--+
      :       :     :     : :Link :     :Link  :     :       :
      +-------+    /-------\+-----+     +------+     /-------\
         TSN       |Sub N/W|                         |TSN N/W|
         Link      \-------/                         \-------/

   LSP   = MPLS Transport
   DNXpt & Xpt = DetNet Transport

              Figure 1: A Simple DetNet Enabled MPLS Network

   TSN End Systems send and receive packets over the DetNet.  The
   supported packet types are IP and Ethernet.

   Edge Nodes are responsible for the imposition and disposition of the
   required DetNet encapsulation.  These are functionally similar to
   pseudowire (PW) Terminating Provide Edge (T-PE) equipment.

   Relay nodes are strategically placed and used enhance the reliability
   of delivery by enabling the replication of packets so that multiple
   copies, possibly over multiple paths.  They also reduce the impact of
   replication by the eliminating surplus copies of DetNet packets.
   Replication and elimination may also be performed at ingress and
   egress edge nodes respectively.

   Edge nodes, and relay nodes are aware of the needs of particular
   DetNet flows and take care to process them in accordance with the
   required performance needs.

   Transit nodes are normal MPLS Label Switching Routers (LSRs).  They
   are unaware of the special requirements of DetNet flows, although
   they may be configured to provide traffic engineering services to
   them to enhance the prospect of them meeting the required service
   level agreement (SLA).

   The MPLS LSP may be provided by any MPLS method (LSP, RSVP-TE, MPLS-
   TP, or MPLS Segment Routing (SR)).





Bryant & Chen           Expires September 6, 2018               [Page 6]


Internet-Draft             Deterministic MPLS                 March 2018


   Figure 2 illustrates how end to end MPLS-based DetNet service is
   provided in a more detail.
   In this case, the end systems are able to send and receive DetNet
   flows.  For example, an end system sends data encapsulated in MPLS.
   The 'X' in the end systems, edge and relay nodes represents potential
   DetNet flow packet replication and elimination points.  Here the
   relay nodes may change the underlying transport, for example
   tunneling MPLS over IP [draft-xu-mpls-sr-over-ip], or simply
   interconnect network segments.

         DetNet                                             DetNet
         Service          Transit          Transit          Service
   DetNet  |             |<-Tnl->|        |<-Tnl->|            | DetNet
   End     |             V   1   V        V   2   V            | End
   System  |    +--------+       +--------+       +--------+   | System
   +---+   |    |   R1   |=======|   R2   |=======|   R3   |   |  +---+
   |  X...DFa...|._X_....|..DF1..|.__ ___.|..DF3..|...._X_.|.DFa..|.X |
   |CE1|========|    \   |       |   X    |       |   /    |======|CE2|
   |   |   |    |     \_.|..DF2..|._/ \__.|..DF4..|._/     |   |  |   |
   +---+        |        |=======|        |=======|        |      +---+
       ^        +--------+       +--------+       +--------+      ^
       |        Relay Node       Relay Node       Relay Node      |
       |                                                          |
       |<--------------- End to End DetNet Service -------------->|

             Figure 2: Flows in a DetNet Enabled MPLS Network

   An example MPLS DetNet network fragment and packet flow is
   illustrated in Figure 3.

      1111   11111111  111111   222222   2222222     333
   CE1----EN1--------R1-------R2-------R3--------EN2----CE2
            \2          22222/                 3 /
             \2222222  /----+                 3 /
              +------R4------------------------+
                       333333333333333333333333

       Figure 3: Example Packet flow in DetNet Enabled MPLS Network

   Customer Equipment CE1 send a packet into the DetNet enabled MPLS
   network.  Edge Node EN1 makes a copy of the packet and encapsulates
   each copy as a DetNet packet (packet 1 and packet 2).  It sends one
   copy (1) to Relay Node R1 and the other copy (2) to Relay Node R4.
   R1 sends packet copy 1 to R2.  R4 sends one copy to R2, and a further
   copy (3) to EN2.  R2 receives copy (2) before copy 1, and so
   eliminated copy (1) sending only (2) to EN2.  EN2 receives copy (3)
   first sending it to CE2 and eliminating copy (2).  Note the number




Bryant & Chen           Expires September 6, 2018               [Page 7]


Internet-Draft             Deterministic MPLS                 March 2018


   illustrates the replication instance and should not be confused with
   the sequence number which remains unchanged in all packet copies.

   The above is of course illustrative of many network scenarios that
   can be configured.  Between a pair of relay nodes there may be one or
   more transport nodes that simply forward the DetNet traffic, but
   these are omitted for clarity.

5.  DetNet over MPLS Encapsulation Components

   To carry DetNet over MPLS the following is required:

   1.  A method of identifying the MPLS payload type.

   2.  A method of identifying the DetNet flow group to the processing
       element.

   3.  A method of distinguishing DetNet OAM packets from DetNet data
       packets.

   4.  A method of carrying the DetNet sequence number.

   5.  A suitable LSP to deliver the packet to the egress PE.

   6.  A method of carrying queuing and forwarding indication.

   In this design an MPLS service label (the S-Label), similar to a
   pseudowire (PW) label [RFC3985], is used to identify both the DetNet
   flow identity and the payload MPLS payload type satisfying (1) and
   (2) in the list above.  OAM traffic discrimination happens through
   the use of the Associated Channel method described in [RFC4385].  The
   sequence number is carried in the DetNet Control word which carries
   the Data/OAM discriminator.  The LSP used to transport the DetNet
   packet may be of any type (MPLS-LDP, MPLS-TE, MPLS-TP[RFC5921], or
   MPLS-SR[I-D.ietf-spring-segment-routing-mpls]).  The LSP (T-Label)
   label and/or the S-Label may be used to indicate the queue processing
   as well as the forwarding parameters.

   Note that when the network consists only of DetNet enabled nodes with
   no aggregation, Penultimate Hop Popping (PHP) means that the only
   label in the label stack is the S-label.

5.1.  Basic Data Plane Encapsulation

   Figure 4 shows a DetNet data plane MPLS encapsulation.  This is
   modeled on the encapsulation of pseudowires over MPLS [RFC3985].

   The encapsulation consists of:



Bryant & Chen           Expires September 6, 2018               [Page 8]


Internet-Draft             Deterministic MPLS                 March 2018


   o  DetNet control word (d-CW) containing sequencing information for
      packet replication and duplicate elimination purposes, and the OAM
      indicator.  There MUST a separate sequence number space for each
      DetNet flow.

   o  DetNet Label (S-label) that identifies a DetNet flow to the peer
      node that is to process it.  The S-Label is allocated from the
      platform label space.

   o  Zero or more MPLS transport LSP label(s) (T-label) used to direct
      the packet along the label switched path (LSP) to the next peer
      node along the path.  When Penultimate Hop Popping is in use there
      will be no label T-label in the protocol stack on the final hop.

   o  The necessary data-link encapsulation is then applied prior to
      transmission over the physical media.

   RFC Editor - if you ever get this text please remove this para (text
   to make compiler work).

     +---------------------------------+
     |                                 |
     |           DetNet Flow           |
     |         Payload  Packet         |
     |                                 |
     +---------------------------------+ <--\
     |       DetNet Control Word       |    |
     +=================================+    +--> DetNet data plane
     |             S-Label             |    |    MPLS encapsulation
     +---------------------------------+ <--/
     |           T-Label(s)            |
     +---------------------------------+
     |           Data-Link             |
     +---------------------------------+
     |           Physical              |
     +---------------------------------+

                  Figure 4: MPLS Encapsulation of DetNet

   Flow aggregation may be necessary to achieve the required scaling.
   The extensions to basic encapsulation needed to support flow
   aggregation are described in Section 5.5.

5.2.  DetNet Control Word

   A DetNet Control Word (d-CW) conforms to the Generic PW MPLS Control
   Word (PWMCW) defined in [RFC4385] and is shown in Figure 5.  In a
   DetNet data packet the upper nibble of the d-CW MUST be set to zero



Bryant & Chen           Expires September 6, 2018               [Page 9]


Internet-Draft             Deterministic MPLS                 March 2018


   (0).  The effective sequence number bit length is between 0 and 28
   bits, and configured either by a control plane or manually for each
   DetNet flow.  The sequence number is aligned to the right (least
   significant bits) and unused bits MUST be set to zero (0).  Each
   DetNet flow MUST have its own sequence number counter.  The sequence
   number is incremented by one for each new packet.

   Editor's note: We need to consider whether it is better to allow a
   multiplicity of sequence number lengths with a length configured for
   each flow, or a uniform sequence number of 28.  On reflection it
   seems better to go for the simplicity of a standard length.  If for
   any reason a different length becomes desirable, then it is
   relatively simple to define another type of DetNet d-CW with a
   different standard sequence number length and there is no ambiguity
   of operation since the sequence number length is a parameter of the
   S-Label.

   The d-CW MUST always be present in a packet.  In cases where the
   sequence number is not used (e.g., for DetNet-t-flows) the control
   plane or the manual configuration has to define zero (0) bit length
   sequence number and the value of the sequence number MUST be set to
   zero (0).

   Editors Note: Do we set length zero or say it is not used?  Also need
   to add text to stop relay nodes and egress edge nodes from processing
   the s/n otherwise only one packet ever gets through!

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |0 0 0 0|                Sequence Number                        |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                       Figure 5: DetNet Control Word

5.3.  Flow identification

   DetNet flow identification at a DetNet service layer is realized by
   an S-label.  It maps a DetNet flow to a specific d-CW in a DetNet
   node.

   For a data flow the S-label used for flow identification MUST be
   bottom label of the label stack for a DetNet-s- or DetNet-st-flow and
   MUST precede the d-CW.

   Editor's note: We have specified the above for _data_ to leave the
   door open for the GAL based OAM method as an alternative to the ACH
   mechanism currently specified.  When using GAL the GAL label would be



Bryant & Chen           Expires September 6, 2018              [Page 10]


Internet-Draft             Deterministic MPLS                 March 2018


   after the S-Label.  Do we leave this option in or shut the door on
   it?

   An S-label for a single DetNet flow MUST be unique at the peer at the
   node that is to process it.  The S-label is stored in the platform
   label table to allow for DetNet packet processing independent of the
   interface on which the packet is received on.

5.4.  OAM Indication

   OAM follows the procedures set out in [RFC5085] with the restriction
   that only Virtual Circuit Connectivity Verification (VCCV) type 1 is
   supported.

   Editor's Note - is this restriction acceptable?  Do we need to
   support GAL based OAM indication?

   As shown in Figure 3 of [RFC5085] when the first nibble of the d-CW
   is 0x0 the payload following the d-CW is normal user data.  However,
   when the first nibble of the d-CW is 0X1, the payload that follows
   the d-DW is an OAM payload with the OAM type indicated by the value
   in the d-CW Channel Type field.

   The reader is referred to [RFC5085] for a more detailed description
   of the Associated Channel mechanism, and to the DetNet work on OAM
   for more information DetNet OAM.

5.5.  Flow Aggregation

   The ability to aggregate individual flows, and their associated
   resource control, into a larger aggregate is an important technique
   for improving scaling of control in the data, management and control
   planes.  The DetNet data plane allows for the aggregation of DetNet
   flows, to improved scaling.  There are three methods of introducing
   flow aggregation:

   1.  Aggregate at the LSP (Transport)

   2.  Aggregating DetNet flows as a new DetNet flow

   3.  Simple Aggregation at the DetNet layer

   A further method of using SR to perform aggregation is for further
   study.

   The resource control and management aspects of aggregation (including
   the queuing/shaping/ policing implications) will be covered in other
   documents.



Bryant & Chen           Expires September 6, 2018              [Page 11]


Internet-Draft             Deterministic MPLS                 March 2018


5.6.  Aggregation at the LSP

   DetNet flows transported via MPLS can leverage MPLS-TE's existing
   support for hierarchical LSPs (H-LSPs), see [RFC4206].  H-LSPs are
   typically used to aggregate control and resources, they may also be
   used to provide OAM or protection for the aggregated LSPs.  Arbitrary
   levels of aggregation naturally falls out of the definition for
   hierarchy and the MPLS label stack [RFC3032].  DetNet nodes which
   support aggregation (LSP hierarchy) map one or more LSPs (labels)
   into and from an H-LSP.  Both carried LSPs and H-LSPs may or may not
   use the TC field, i.e., L-LSPs or E-LSPs.  Such nodes will need to
   ensure that traffic from aggregated LSPs are placed (shaped/policed/
   enqueued) onto the H-LSPs in a fashion that ensures the required
   DetNet service is preserved.

   Additional details of the traffic control capabilities needed at a
   DetNet-aware node may be covered in the new service descriptions
   mentioned above or in separate future documents.  Management and
   control plane mechanisms will also need to ensure that the service
   required on the aggregate flow (H-LSP or DSCP) are provided, which
   may include the discarding or remarking mentioned in the previous
   sections.

5.7.  Aggregating DetNet flows as a new DetNet flow

   An aggregate can be built by layering DetNet flows as shown below:

   +---------------------------------+
   |                                 |
   |           DetNet Flow           |
   |         Payload  Packet         |
   |                                 |
   +---------------------------------+ <--\
   |       DetNet Control Word       |    |
   +=================================+    |
   |            S-Label              |    |
   +---------------------------------+    +----DetNet data plane
   |       DetNet Control Word       |    |    MPLS encapsulation
   +=================================+    |
   |            A-Label              |    |
   +---------------------------------+ <--/
   |           T-Label(s)            |
   +---------------------------------+
   |           Data-Link             |
   +---------------------------------+
   |           Physical              |
   +---------------------------------+




Bryant & Chen           Expires September 6, 2018              [Page 12]


Internet-Draft             Deterministic MPLS                 March 2018


   Both the Aggregation (A) label and the S-label have their MPLS S bit
   set indicating bottom of stack, and the d-CW allows the PREF
   functions to work.

   It is a property of the A-label that what follows is d-CW followed by
   an S-label.  A relay node processing the A-label would not know the
   underlying payload type.  This would only be known to a node that was
   a peer of the node imposing the S-label.  However there is no real
   need for it to know the payload type during aggregation processing.

6.  Simple Aggregation at the DetNet layer

   Another approach would be not to include a d-CW for the aggregated
   flow.  This would be functionally similar to aggregation at the
   transport layer using H-LSPs, but would confine knowledge of the
   aggregation to the DetNet layer.  Such an approach shares the
   disadvantage that PREF operations would not be possible.  OAM
   operation in this mode is for further study.

    +---------------------------------+
    |                                 |
    |           DetNet Flow           |
    |         Payload  Packet         |
    |                                 |
    +---------------------------------+ <--\
    |       DetNet Control Word       |    |
    +=================================+    |
    |            S-Label              |    +----DetNet data plane
    +---------------------------------+    |    MPLS encapsulation
    |            A-Label              |    |
    +---------------------------------+ <--/
    |           T-Label(s)            |
    +---------------------------------+
    |           Data-Link             |
    +---------------------------------+
    |           Physical              |
    +---------------------------------+

7.  Indication of the DetNet Payload Type.

   The only node types that needs to know the payload type is the
   ingress node which has to know how to process the packet it receives
   on the ingress AC, and egress edge node which has to know how to
   prepare the packet for transmission on the egress AC.

   On ingress a DetNet edge node has to classify the packets into those
   that are for transmission as Detnet packets and those that are for
   transmission as "normal" packets at one of more lower priorities.



Bryant & Chen           Expires September 6, 2018              [Page 13]


Internet-Draft             Deterministic MPLS                 March 2018


   The packet type is indicated to the egress edge node through the
   value of the S-label.  Thus when the egress edge node looks up the
   S-label one of the parameters returned is the packet type which in
   turn tells the egress edge node how to prepare the packet for
   transmission over the egress AC.

   Editor's note: Do we only find one type on the ingress and egress or
   do we need to run missed mode, i.e. will we have to send some packets
   across the network as Ethernet packets and some as IP packets?

8.  Operation of the PREF Functions

   The Packet Replication and Elimination Functions (PREF) are designed
   to enhance the reliability of delivery by network layer packet
   replication (PR), whilst at the same time minimizing network
   congestion and the duplicate delivery of packets over an egress AC by
   eliminating duplicate packets (EF).

   PR and EF are independent functions that operate on a DetNet flow at
   strategic places in the network.  The placement of a function is a
   matter for the network operator.

8.1.  Operation of PR

   A PR function creates two or more copies of a packet, and forwards a
   copy to each of the designated peers of the replicating node.  A PR
   function may be placed in an ingress edge node, a relay node or an
   egress edge node.

   We consider first a DetNet relay node.  The packet is received from
   the upstream DetNet peer, and if present the T-label is popped.  The
   S-label is looked up in the platform label table and if the
   forwarding instructions indicate that replication is required, the
   following happens for each next hop in the DetNet layer:

   o  A copy of the payload is made.

   o  An identical copy of the d-CW from the original packet is pushed.

   o  The S-Label required by the next hop in the DetNet layer is
      pushed.

   o  The DetNet packet copy is forwarded on the LSP to the next hop in
      the DetNet layer using normal MPLS forwarding.

   An ingress node operates in the same way as a relay node, except that
   it is responsible for initial encapsulation of the packet.  The
   packet is received from the AC, classified and prepared for



Bryant & Chen           Expires September 6, 2018              [Page 14]


Internet-Draft             Deterministic MPLS                 March 2018


   forwarding over the DetNet network as described in Section 9.1,
   except that for each next-hop in the DetNet Layer (i.e. each node
   that is to receive a copy of the DetNet packet) :

   o  A copy of the payload is made.

   o  The d-CW is constructed using the next sequence number in the
      sequence associated with this service.

   o  An identical copy of the d-CW is pushed.

   o  The S-Label required by the next hop in the DetNet layer to
      recognize this service is pushed.

   o  The DetNet packet copy is forwarded on the LSP to the next hop in
      the DetNet layer using normal MPLS forwarding.

   An egress node operates in the same way as
   as described in Section 9.1, except that where the S-Label indicates
   that the the packet is to be forwarded to a multiply attached system
   in the payload layer a similar copy (modified as needed to conform to
   any MAC requirements) is forwarded out of each egress AC.

   Editor's Note: For normal PW, there will be one to one AC to PW
   binding relationship, for DetNet, no matter the ingress or egress
   side, there may be multiple AC corresponding to a single DetNet PW
   (S-label), should we state explicitly?

8.2.  Operation of EF

   The EF function eliminates duplicate copies of a packet.  The node
   identifies the service from the packet S-Label.  If the S-Label
   indicates that the packet elimination function is required to operate
   on this service at this node, it uses the sequence number in the d-CW
   to determine whether or not the packet is a duplicate that must be
   eliminated, and precedes accordingly.

   The EF may be placed in a Relay node, or an Edge Egress node.

8.3.  Packet reordering considerations

   The DetNet service layer introduces packet reordering functionality
   for use in DetNet edge and relay node and end system packet
   processing.  The reordering functionality MAY be enabled in a DetNet
   node.  The reordering functionality relies on a presence of sequence
   numbers the d-CW.





Bryant & Chen           Expires September 6, 2018              [Page 15]


Internet-Draft             Deterministic MPLS                 March 2018


8.4.  Indication of PREF processing

   The indication that PR or EF processing is needed at a DetNet Relay
   node or a DetNet egress edge node is carried as an implicit
   characteristic of the S-Label.  Thus, when a service is established
   the ingress edge node is configured to use an active rather than a
   static zero sequence number, and DetNet relays and egress edge nodes
   are configured to run PR and/or EF functionality on on services
   identified by specific S-labels.

8.5.  Placement of PR and EF in a node

   This section is not normative.

   The placement of the PR and EF functions within the node is a matter
   for the node designers, and this specification makes no determination
   on this matter.  However the placement may well have implications for
   the management and control of a node, and thus the following is worth
   noting.

   In a bladed system a common processing model is to analyze the packet
   for forwarding close to the ingress interface and to either to fully
   prepare it for forwarding as part of ingress processing, or to
   package it with internal metadata for final preparation close to the
   egress interface.  In such systems the natural place to perform
   replication is as part of the ingress processing since egress
   processors often do not have the capability (for example processing
   power) to further process the packet, often do not normally have the
   required data paths to facilitate replication for transmission to
   other line cards.

   Furthermore, in bladed systems it is to be expected that a packet
   from a peer in the MPLS layer may arrive on any of the blades.
   Whilst in principle some constraints could be applied on which
   interfaces the packets will arrive on, experience shows that such
   constraints are operationally impractical.  To perform elimination
   state must be shared amongst the forwarders performing elimination.
   Sharing the state required for elimination between forwarders on
   different blades without sacrificing performance is technically
   difficult.  Thus, the natural place for elimination in a distributed
   design is close to the egress interface.

   On mono-processor systems these constraints do not apply in quite the
   same way, although even in this case it is necessary for the
   equipment designer to consider the implications of particular
   forwarder design, for example the allocation of Network Processing
   Unit (NPU) cores to particular interfaces.




Bryant & Chen           Expires September 6, 2018              [Page 16]


Internet-Draft             Deterministic MPLS                 March 2018


   A bladed system may place the PR and EF functions on a processing
   function other than the ingress or egress forwarding processor,
   whereupon the mono-processor considerations apply.

   As noted above the placement of the PR and EF functions is a matter
   for the system designer.  However it is important that nothing in the
   control, configuration, or OAM system results in undue difficulties
   for any of the forwarding models.

9.  Operation at DetNet Node types

   This section considers the operations at the various DetNet node
   types in more detail.

9.1.  Operation at an Ingress Edge (PE) Node

   An ingress Edge Node first classifies received traffic into DetNet
   flows and non-Detnet traffic.  The DetNet flows are sent over the
   DetNet network using the procedures defined in the document.  The
   classification process and the handling of non-DetNet traffic is out
   of scope for this document.

   The processing of non-DetNet flows is currently outside the scope of
   this document

   The packet is encapsulated as described in Section 5.1.  First, if
   the flow is to be carried as IP the MAC header and checksum are
   removed.  Otherwise the checksum is removed.  The d-CW is constructed
   using the next sequence number associated with the service, and is
   pushed.  The S-label corresponding to this DetNet flow is then
   pushed.  The packet is then forwarded to the required egress edge-
   node by pushing the required T-Labels and the required data-link
   headers.

   Editor's Note: we need text on how we indicate IPv4 vs IPv6.  In MPLS
   they are carried on different FECs.  Presumable we need to have a
   different S-Label for each IP type.  This implies a different s/n for
   each IP type, but since it seems unlikely that we need to maintain
   them in a common sequence that looks to be OK.

9.2.  Operation at a Relay node (S-PE)

   At a relay node packet forwarding follows the same process used in a
   PW S-PE [RFC5659], save for the additional processing required for
   any required PREF processing.

   The packet is received from the incoming LSP and any T-labels which
   are still present are popped.  The S-label is looked up in the



Bryant & Chen           Expires September 6, 2018              [Page 17]


Internet-Draft             Deterministic MPLS                 March 2018


   platform label table to determine the forwarding parameters.  If PREF
   processing is required the process described in Section 8 is
   followed, and if required the locally held sequence number
   information associated with the S-label is updated to avoid future
   duplicate forwarding.

   For each copy of the packet to be forwarded the S-label is swapped
   for the S-label required by either the next relay node, or by the
   egress edge node.  For each copy of the packet, the T-label(s) needed
   to reach the next relay node or the egress edge node is pushed and
   the packet forwarded towards its next hop in the MPLS layer.

9.3.  Operation at an Egress Edge (PE) Node

   At an egress edge node, the S-Label is looked up in the platform
   label table and is used to determine the egress packet processing
   parameters.  The sequence number in d-CW and the recorded sequence
   number history is compared to perform any required ER processing
   Section 8.2.  If the packet is not eliminated, the d-CW is stripped.

   If the packet is to be treated as an IP packet, this is processed in
   the normal way for an IP packet egressing an MPLS tunnel (for example
   the data-link header is constructed) and the packet forwarded towards
   its destination.

   Editor's note: Do we need to look the IP address up in a forwarding
   table, if so, which one, or is the next hop a forwarding parameter.
   Is the MAC address a forwarding parameter, or do we need to run ARP
   as normal?

   If the packet is to be treated as an Ethernet packet it is forwarded
   unmodified, save for the addition of the CRC as described in
   [RFC4448].

9.4.  Operation at a Transit(P) Node

   Operation at a transit (P) node is normal MPLS forwarding.  The outer
   label is either swapped or popped as required, and the packet is
   forwarded along the LSP.  If an entropy label is present in the label
   stack, this may be used by the Equal Cost Multi-Path (ECMP) selection
   process.  No other label is inspected as part of forwarding.

   The Traffic Class field and/or incoming LSP transport label may be
   used to indicate how the packet is to be processed and queued
   including which forwarding resources are to be applied to the packet
   ((RFC3270}}. Any of the methods of constructing a physical LSP (for
   RSVP-TE signaling or MPLS-TP style controller based configuration) or
   a virtual LSP (Segment Routing



Bryant & Chen           Expires September 6, 2018              [Page 18]


Internet-Draft             Deterministic MPLS                 March 2018


   [I-D.ietf-spring-segment-routing-mpls]) may be used to indicate not
   only the next hop, but the priority of processing and any physical
   resources dedicated to the service [I-D.bryant-rtgwg-enhanced-vpn].

10.  Other DetNet data plane considerations

10.1.  Class of Service

   Class and quality of service, i.e., CoS and QoS, are terms that are
   often used interchangeably and confused.  In the context of DetNet,
   CoS is used to refer to mechanisms that provide traffic forwarding
   treatment based on aggregate group basis and QoS is used to refer to
   mechanisms that provide traffic forwarding treatment based on a
   specific DetNet flow basis.  Examples of existing network level CoS
   mechanisms include DiffServ which is enabled by IP header
   differentiated services code point (DSCP) field [RFC2474] and MPLS
   label traffic class field [RFC5462], and at Layer-2, by IEEE 802.1p
   priority code point (PCP).

   CoS for DetNet flows carried in PWs and MPLS is provided using the
   existing MPLS Differentiated Services (DiffServ) architecture
   [RFC3270].  Both E-LSP and L-LSP MPLS DiffServ modes MAY be used to
   support DetNet flows.  The Traffic Class field (formerly the EXP
   field) of an MPLS label follows the definition of [RFC5462] and
   [RFC3270].  The Uniform, Pipe, and Short Pipe DiffServ tunneling and
   TTL processing models are described in [RFC3270] and [RFC3443] and
   MAY be used for MPLS LSPs supporting DetNet flows.  MPLS ECN MAY also
   be used as defined in ECN [RFC5129] and updated by [RFC5462].

   One additional consideration for DetNet nodes which support CoS
   services is that they MUST ensure that the CoS service classes do not
   impact the congestion protection and latency control mechanisms used
   to provide DetNet QoS.  This requirement is similar to requirement
   for MPLS LSRs to that CoS LSPs do not impact the resources allocated
   to TE LSPs via [RFC3473].

10.2.  Quality of Service

   Quality of Service (QoS) mechanisms for flow specific traffic
   treatment typically includes a guarantee/agreement for the service,
   and allocation of resources to support the service.  Example QoS
   mechanisms include discrete resource allocation, admission control,
   flow identification and isolation, and sometimes path control,
   traffic protection, shaping, policing and remarking.  Example
   protocols that support QoS control include Resource ReSerVation
   Protocol (RSVP) [RFC2205] and RSVP-TE [RFC3209] and [RFC3473].  The
   existing MPLS mechanisms defined to support CoS [RFC3270] can also be
   used to reserve resources for specific traffic classes.



Bryant & Chen           Expires September 6, 2018              [Page 19]


Internet-Draft             Deterministic MPLS                 March 2018


   In addition to explicit routes Section 11.2, and packet replication
   and elimination, described in Section 8 above, DetNet provides zero
   congestion loss and bounded latency and jitter.  As described in
   [I-D.ietf-detnet-architecture], there are different mechanisms that
   maybe used separately or in combination to deliver a zero congestion
   loss service.  These mechanisms are provided by the either the MPLS
   or IP layers, and may be combined with the mechanisms defined by the
   underlying network layer such as 802.1TSN.

   A baseline set of QoS capabilities for DetNet flows carried over MPLS
   can be provided with Traffic Engineering (MPLS-TE) [RFC3209] and
   [RFC3473].  TE LSPs can also support explicit routes (path pinning).
   Current service definitions for packet TE LSPs can be found in
   "Specification of the Controlled Load Quality of Service", [RFC2211],
   "Specification of Guaranteed Quality of Service", [RFC2212], and
   "Ethernet Traffic Parameters", [RFC6003].  Additional service
   definitions are expected in future documents to support the full
   range of DetNet services.  In all cases, the existing label-based
   marking mechanisms defined for TE-LSPs and even E-LSPs are use to
   support the identification of flows requiring DetNet QoS.

   Packets that are marked with a DetNet Class of Service value, but
   that have not been the subject of a completed reservation, can
   disrupt the QoS offered to properly reserved DetNet flows by using
   resources allocated to the reserved flows.  Therefore, the network
   nodes of a DetNet network MUST:

   o  Defend the DetNet QoS by discarding or remarking (to a non-DetNet
      CoS) packets received that are not the subject of a completed
      reservation.

   o  Not use a DetNet reserved resource, e.g. a queue or shaper
      reserved for DetNet flows, for any packet that does not carry a
      DetNet Class of Service marker.

10.3.  Bidirectional traffic

   Some DetNet applications generate bidirectional traffic.  Using MPLS
   definitions [RFC5654] there are associated bidirectional flows, and
   co-routed bidirectional flows.  MPLS defines a point-to-point
   associated bidirectional LSP as consisting of two unidirectional
   point-to-point LSPs, one from A to B and the other from B to A, which
   are regarded as providing a single logical bidirectional transport
   path.  This would be analogous of standard IP routing, or PWs running
   over two reciprocal unidirectional LSPs.  MPLS defines a point-to-
   point co-routed bidirectional LSP as an associated bidirectional LSP
   which satisfies the additional constraint that its two unidirectional
   component LSPs follow the same path (in terms of both nodes and



Bryant & Chen           Expires September 6, 2018              [Page 20]


Internet-Draft             Deterministic MPLS                 March 2018


   links) in both directions.  An important property of co-routed
   bidirectional LSPs is that their unidirectional component LSPs share
   fate.  In both types of bidirectional LSPs, resource allocations may
   differ in each direction.  The concepts of associated bidirectional
   flows and co-routed bidirectional flows can be applied to DetNet
   flows as well.  PWs [RFC3985] are always created as bidirectional
   constructs.

   While the MPLS data planes must support bidirectional DetNet flows,
   there are no special bidirectional features with respect to the data
   plane other than need for the two directions take the same paths.
   Fate sharing and associated vs co-routed bidirectional flows can be
   managed at the control level.  Note, that there is no stated
   requirement for bidirectional DetNet flows to be supported using the
   same MPLS Labels in each direction, and indeed to do so would
   introduce significant implementation issues.  Control mechanisms will
   be need to support such bidirectional flows DetNet over MPLS, but
   such mechanisms are out of scope of this document.  An example
   control plane solution for MPLS can be found in [RFC7551].

10.4.  Layer 2 addressing and QoS Considerations

   Editor's note: Add references needed by this section

   The Time-Sensitive Networking (TSN) Task Group of the IEEE 802.1
   Working Group have defined (and are defining) a number of amendments
   to IEEE 802.1Q [IEEE8021Q] that provide zero congestion loss and
   bounded latency in bridged networks.  IEEE 802.1CB [IEEE8021CB]
   defines packet replication and elimination functions that should
   prove both compatible with and useful to, DetNet networks.

   As is the case for DetNet, a Layer 2 network node such as a bridge
   may need to identify the specific DetNet flow to which a packet
   belongs in order to provide the TSN/DetNet QoS for that packet.  It
   also will likely need a CoS marking, such as the priority field of an
   IEEE Std 802.1Q VLAN tag, to give the packet proper service.

   Although the flow identification methods described in IEEE 802.1CB
   [IEEE8021CB] are flexible, and in fact, include IP 5-tuple
   identification methods, the baseline TSN standards assume that every
   Ethernet frame belonging to a TSN stream (i.e.  DetNet flow) carries
   a multicast destination MAC address that is unique to that flow
   within the bridged network over which it is carried.  Furthermore,
   IEEE 802.1CB [IEEE8021CB] describes three methods by which a packet
   sequence number can be encoded in an Ethernet frame.

   Ensuring that the proper Ethernet VLAN tag priority and destination
   MAC address are used on a DetNet/TSN packet may require further



Bryant & Chen           Expires September 6, 2018              [Page 21]


Internet-Draft             Deterministic MPLS                 March 2018


   clarification of the customary L2/L3 transformations carried out by
   routers and edge label switches.  Edge nodes may also have to move
   sequence number fields among Layer 2, PW, and IPv6 encapsulations.

11.  Management and control considerations

   While management plane and control planes are traditionally
   considered separately, from the Data Plane perspective there is no
   practical difference based on the origin of flow provisioning
   information.  This document therefore does not distinguish between
   information provided by a control plane protocol, e.g., RSVP-TE
   [RFC3209] and [RFC3473], or by a network management mechanisms, e.g.,
   RestConf [RFC8040] and YANG [RFC7950].

   Editor's note: Not sure if RSVP-TE can be used, indeed unsure what
   routing protocols can be use other than to create point to point MPLS
   transport paths.  Normally we construct a a single path through the
   network with RSVP-TE, but here we need to construct an explicit mesh
   at the DetNet layer.  The classical routing protocols are not really
   capable of constructing graphs of the sort needed here either.

11.1.  S-Label assignment and distribution

   A mechanism based on the existing PW label distribution protocol
   [RFC8077] can be used to exchange labels between DetNet nodes.  The
   protocol may however need extending depending on the preferred format
   of the DetNet flow identifiers.

   A mechanism to create the flow graph through the relay nodes will
   need to be specified.  Initially this is likely to be based on a
   network controller of some sort rather than a distributed routing
   protocol.

   The details of the control plane protocol solution required for the
   label distribution and the management of the label number space are
   out of scope of this document.

11.2.  Explicit routes

   Editor's note describe the applicability of explicit routes as a
   method of constructing paths

12.  Security considerations

   The security considerations of DetNet in general are discussed in
   [I-D.ietf-detnet-architecture] and [I-D.sdt-detnet-security].  Other
   security considerations will be added in a future version of this
   draft.



Bryant & Chen           Expires September 6, 2018              [Page 22]


Internet-Draft             Deterministic MPLS                 March 2018


13.  IANA considerations

   This document makes no IANA requests.

13.1.  Acknowledgments

   We acknowledge the authors of draft-ietf-detnet-dp-sol-01.

14.  References

14.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997, <https://www.rfc-
              editor.org/info/rfc2119>.

   [RFC4448]  Martini, L., Ed., Rosen, E., El-Aawar, N., and G. Heron,
              "Encapsulation Methods for Transport of Ethernet over MPLS
              Networks", RFC 4448, DOI 10.17487/RFC4448, April 2006,
              <https://www.rfc-editor.org/info/rfc4448>.

   [RFC5085]  Nadeau, T., Ed. and C. Pignataro, Ed., "Pseudowire Virtual
              Circuit Connectivity Verification (VCCV): A Control
              Channel for Pseudowires", RFC 5085, DOI 10.17487/RFC5085,
              December 2007, <https://www.rfc-editor.org/info/rfc5085>.

14.2.  Informative References

   [I-D.bryant-rtgwg-enhanced-vpn]
              Bryant, S. and J. Dong, "Enhanced Virtual Private Networks
              (VPN+)", draft-bryant-rtgwg-enhanced-vpn-01 (work in
              progress), October 2017.

   [I-D.ietf-detnet-architecture]
              Finn, N., Thubert, P., Varga, B., and J. Farkas,
              "Deterministic Networking Architecture", draft-ietf-
              detnet-architecture-04 (work in progress), October 2017.

   [I-D.ietf-detnet-dp-alt]
              Korhonen, J., Farkas, J., Mirsky, G., Thubert, P.,
              Zhuangyan, Z., and L. Berger, "DetNet Data Plane Protocol
              and Solution Alternatives", draft-ietf-detnet-dp-alt-00
              (work in progress), October 2016.







Bryant & Chen           Expires September 6, 2018              [Page 23]


Internet-Draft             Deterministic MPLS                 March 2018


   [I-D.ietf-detnet-dp-sol]
              Korhonen, J., Andersson, L., Jiang, Y., Finn, N., Varga,
              B., Farkas, J., Bernardos, C., Mizrahi, T., and L. Berger,
              "DetNet Data Plane Encapsulation", draft-ietf-detnet-dp-
              sol-01 (work in progress), January 2018.

   [I-D.ietf-spring-segment-routing-mpls]
              Bashandy, A., Filsfils, C., Previdi, S., Decraene, B.,
              Litkowski, S., and R. Shakir, "Segment Routing with MPLS
              data plane", draft-ietf-spring-segment-routing-mpls-12
              (work in progress), February 2018.

   [I-D.sdt-detnet-security]
              Mizrahi, T., Grossman, E., Hacker, A., Das, S., Dowdell,
              J., Austad, H., Stanton, K., and N. Finn, "Deterministic
              Networking (DetNet) Security Considerations", draft-sdt-
              detnet-security-01 (work in progress), July 2017.

   [RFC2205]  Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
              Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
              Functional Specification", RFC 2205, DOI 10.17487/RFC2205,
              September 1997, <https://www.rfc-editor.org/info/rfc2205>.

   [RFC2211]  Wroclawski, J., "Specification of the Controlled-Load
              Network Element Service", RFC 2211, DOI 10.17487/RFC2211,
              September 1997, <https://www.rfc-editor.org/info/rfc2211>.

   [RFC2212]  Shenker, S., Partridge, C., and R. Guerin, "Specification
              of Guaranteed Quality of Service", RFC 2212,
              DOI 10.17487/RFC2212, September 1997, <https://www.rfc-
              editor.org/info/rfc2212>.

   [RFC2474]  Nichols, K., Blake, S., Baker, F., and D. Black,
              "Definition of the Differentiated Services Field (DS
              Field) in the IPv4 and IPv6 Headers", RFC 2474,
              DOI 10.17487/RFC2474, December 1998, <https://www.rfc-
              editor.org/info/rfc2474>.

   [RFC3032]  Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y.,
              Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack
              Encoding", RFC 3032, DOI 10.17487/RFC3032, January 2001,
              <https://www.rfc-editor.org/info/rfc3032>.

   [RFC3209]  Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
              and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
              Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001,
              <https://www.rfc-editor.org/info/rfc3209>.




Bryant & Chen           Expires September 6, 2018              [Page 24]


Internet-Draft             Deterministic MPLS                 March 2018


   [RFC3270]  Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen,
              P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi-
              Protocol Label Switching (MPLS) Support of Differentiated
              Services", RFC 3270, DOI 10.17487/RFC3270, May 2002,
              <https://www.rfc-editor.org/info/rfc3270>.

   [RFC3443]  Agarwal, P. and B. Akyol, "Time To Live (TTL) Processing
              in Multi-Protocol Label Switching (MPLS) Networks",
              RFC 3443, DOI 10.17487/RFC3443, January 2003,
              <https://www.rfc-editor.org/info/rfc3443>.

   [RFC3473]  Berger, L., Ed., "Generalized Multi-Protocol Label
              Switching (GMPLS) Signaling Resource ReserVation Protocol-
              Traffic Engineering (RSVP-TE) Extensions", RFC 3473,
              DOI 10.17487/RFC3473, January 2003, <https://www.rfc-
              editor.org/info/rfc3473>.

   [RFC3985]  Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation
              Edge-to-Edge (PWE3) Architecture", RFC 3985,
              DOI 10.17487/RFC3985, March 2005, <https://www.rfc-
              editor.org/info/rfc3985>.

   [RFC4206]  Kompella, K. and Y. Rekhter, "Label Switched Paths (LSP)
              Hierarchy with Generalized Multi-Protocol Label Switching
              (GMPLS) Traffic Engineering (TE)", RFC 4206,
              DOI 10.17487/RFC4206, October 2005, <https://www.rfc-
              editor.org/info/rfc4206>.

   [RFC4385]  Bryant, S., Swallow, G., Martini, L., and D. McPherson,
              "Pseudowire Emulation Edge-to-Edge (PWE3) Control Word for
              Use over an MPLS PSN", RFC 4385, DOI 10.17487/RFC4385,
              February 2006, <https://www.rfc-editor.org/info/rfc4385>.

   [RFC5129]  Davie, B., Briscoe, B., and J. Tay, "Explicit Congestion
              Marking in MPLS", RFC 5129, DOI 10.17487/RFC5129, January
              2008, <https://www.rfc-editor.org/info/rfc5129>.

   [RFC5462]  Andersson, L. and R. Asati, "Multiprotocol Label Switching
              (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic
              Class" Field", RFC 5462, DOI 10.17487/RFC5462, February
              2009, <https://www.rfc-editor.org/info/rfc5462>.

   [RFC5654]  Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed.,
              Sprecher, N., and S. Ueno, "Requirements of an MPLS
              Transport Profile", RFC 5654, DOI 10.17487/RFC5654,
              September 2009, <https://www.rfc-editor.org/info/rfc5654>.





Bryant & Chen           Expires September 6, 2018              [Page 25]


Internet-Draft             Deterministic MPLS                 March 2018


   [RFC5659]  Bocci, M. and S. Bryant, "An Architecture for Multi-
              Segment Pseudowire Emulation Edge-to-Edge", RFC 5659,
              DOI 10.17487/RFC5659, October 2009, <https://www.rfc-
              editor.org/info/rfc5659>.

   [RFC5921]  Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau,
              L., and L. Berger, "A Framework for MPLS in Transport
              Networks", RFC 5921, DOI 10.17487/RFC5921, July 2010,
              <https://www.rfc-editor.org/info/rfc5921>.

   [RFC6003]  Papadimitriou, D., "Ethernet Traffic Parameters",
              RFC 6003, DOI 10.17487/RFC6003, October 2010,
              <https://www.rfc-editor.org/info/rfc6003>.

   [RFC7551]  Zhang, F., Ed., Jing, R., and R. Gandhi, Ed., "RSVP-TE
              Extensions for Associated Bidirectional Label Switched
              Paths (LSPs)", RFC 7551, DOI 10.17487/RFC7551, May 2015,
              <https://www.rfc-editor.org/info/rfc7551>.

   [RFC7950]  Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language",
              RFC 7950, DOI 10.17487/RFC7950, August 2016,
              <https://www.rfc-editor.org/info/rfc7950>.

   [RFC8040]  Bierman, A., Bjorklund, M., and K. Watsen, "RESTCONF
              Protocol", RFC 8040, DOI 10.17487/RFC8040, January 2017,
              <https://www.rfc-editor.org/info/rfc8040>.

   [RFC8077]  Martini, L., Ed. and G. Heron, Ed., "Pseudowire Setup and
              Maintenance Using the Label Distribution Protocol (LDP)",
              STD 84, RFC 8077, DOI 10.17487/RFC8077, February 2017,
              <https://www.rfc-editor.org/info/rfc8077>.

Authors' Addresses

   Stewart Bryant
   Huawei Technologies

   Email: stewart.bryant@gmail.com


   Mach Chen
   Huawei Technologies

   Email: mach.chen@huawei.com







Bryant & Chen           Expires September 6, 2018              [Page 26]


Html markup produced by rfcmarkup 1.126, available from https://tools.ietf.org/tools/rfcmarkup/